10 Things You Need to Know About Big Data Now

May 23, 2013

By Joyce Wells

DBTA’s Big Data Boot Camp provided attendees with an immersion experience in the world of big data. Presentations and panel discussions with data experts covered topics such as what Hadoop actually is, the value related solutions provide, how it all fits into the overall enterprise data management picture, real-world use cases in which big data technologies are now being deployed, how to get started, the legal implications of big data management that organizations need to be aware of before they initiate a big data project, and how to successfully engage with customers through social media.

The two-day conference, chaired by Peter Auditore, principal of the consultancy Asterias Research put a sharp focus on what organizations need to know in order to implement a project and how to get business value from big data.

In addition to Auditore, conference speakers included Stephen E. Arnold, managing director, ArnoldIT.com; James Dawson, manager, Forensic Technology, KPMG; George Everitt, founder, Applied Relevance; Fred Gallagher, general manager, Vectorwise, Actian Corporation; Alex Gorbachev, CTO, Pythian; Amir Halfon, MarkLogic Corp., Robert Hodges, CEO, Continuent, Inc.; Debbie Hutchings, social media lead, BMC Software; Alon Israely, co-founder, Business Intelligence Associates, Inc. (BIA); David Jonker, senior director, Big Data Product Marketing, Technology & Innovation Platform, SAP Labs; John O'Brien, founder and principal, Radiant Advisors; Steve Sarsfield, product marketing manager, Talend; Tony Shan, Global Advisory Consultant; John Tredennick, CEO, Catalyst Repository Systems; and Mouli Venkatesan, CEO, MEICS Inc.; and Scott Zoldi, vice president for Transaction Analytics, FICO.

10 Salient Points Emerged from DBTA's Big Data Boot Camp

While big data is a heavily used buzzword, technologies such as Hadoop are now just in the nascent stages of adoption. If you haven’t already dipped your toes in the water, don’t panic; it’s still early. But now is the time to begin exploring the technologies, which are maturing rapidly.
Big data is not new. The difference now is that there are technologies that are widely available to everyone. In terms of volume, many enterprises have had large data stores for some time. Data variety and velocity has been around too. There just have not been affordable and accessible options for deriving value from it. (John O'Brien, Radiant Advisors)
An inevitable trade-off in achieving a best-of-breed approach with regard to big data is that you have to manage complexity. (John O'Brien,Radiant Advisors)
In-memory technologies are looming large due to the need for timely access to data. (David Jonker, SAP Labs)
The advantage of Hadoop and NoSQL – or schema-less – systems is that you don’t have to prepare the data, making it ideal for data of unknown value. However, Hadoop is not just an archive or a staging area. If you are using it as a staging area, you are missing the point. (Amir Halfon, MarkLogic and John O'Brien, Radiant Advisors)
Since Hadoop is an open source technology that can run on commodity hardware, the technology has a fairly low cost-barrier to entry, but the skills required to use it are not readily available. Data scientists are hard to come by and some companies are finding that in order to employ these experts they must open offices in California. Some decide to simply outsource the job. If you don’t have access to staff with the expertise that Hadoop and related technologies require, a pure open source Hadoop implementation is not optimal. You may be better off starting with a third-party distribution or proprietary package that gives you things like bug fixes and training and support. (Alex Gorbachev, Pythian, and David Jonker, SAP Labs)
SQL is not going away. Relational databases are still important and there are more solutions coming on the scene that provide front-end SQL access to big data stores of schema-less data. (David Jonker, SAP Labs)
Ultimately, the value of big data lies in opening it up to more people for analytics, making it more accessible and easier to use. Hadoop doesn't have the open accessibility that it needs yet, but it will soon. (John O'Brien, Radiant Advisors)
Be sure you know how you are protecting data in big data systems – but also have an awareness of how difficult it could be to pull data out of a system if necessary for legal proceedings. Lawyers and judges won’t care how hard it is to do it. Ensure that you are thinking about the legal side, because it will impact you. (Alon Israely, Business Intelligence Associates, Inc.)
Social media is not a threat. But remember that collecting and analyzing social media is more than just racking up “likes.” Involving employees in meaningful one-on-one social media interaction on behalf of your company can provide unique insight into your customers' sentiments and help create a closer relationship with them. (Debbie Hutchings, BMC Software, and Peter Auditore, Asterias Research)