12 Big Data Predictions for 2014

“Big Data” has been one of the most-used technology buzzwords of 2013. The market has seen rapid growth and change in the past year. We’ve seen Hadoop and the ecosystem around it evolve from a technology only very talented programmers and engineers could use to one usable by data scientists. Companies have begun to embrace big data technologies and put applications into production.

What lies ahead for 2014? Here are 12 predictions for what to expect in the big data trenches in the coming year.

In 2014, the allure of “big data” as a buzzword will wear off, but big data itself will be table stakes, according to Gainsight, which uses big data analytics in its SaaS platform to provide what it calls “customer success management.” Gainsight believes every cloud application provider will evolve its backend infrastructure to support big data principles in 2014.

The cloud won’t be the only place where big data technology makes itself known. In 2014, Hadoop will move beyond batch-based data processing and storage to a general purpose compute infrastructure that will be core to the enterprise data fabric, according to strategic analytics platform provider Alteryx. This means analytics will continue to be the number one use case for big data.

Companies are going to get serious about leveraging the digital breadcrumb trail customers leave as they interact with products and services online, according to Gainsight. To get there, data analytics have to drive beyond the BI team to provide business value throughout the company.

Analytics and big data for marketing purposes will have another big year in 2014, influencing advertising, promotions and consumer behavior, Alteryx says. The key moments will be innovations that emerge around the World Cup and Winter Olympics.

Empowering analysts in business departments with big data and analytics will become more important than filling the perceived need for millions of data scientists, says Alteryx. This will drive down the high salaries offered for data science roles in 2014, the company predicts.

The Institute for Electrical and Electronics Engineers (IEEE) professional association believes that 2014 will see the Internet of Things—where identifiable objects are seamlessly integrated into the information network—give way to the Web of Things. The Web of Things, it says, will take advantage of mobile devices’ and sensors’ capability to observe and monitor their environments to increase the coordination between things in the real world and their counterparts on the Web.

“The Web of Things will produce large volumes of data related to the physical world, and intelligent solutions are required to enable connectivity, inter-networking and relevance between the physical world and the corresponding digital world resources,” it says.

The volume, velocity and variety of data will continue to grow exponentially in 2014, simpler analytics tools will be needed to leverage the “data deluge.”

“It’s more than the three Vs—volume, velocity, and variety—that make big data such a difficult tiger to tame,” the IEEE says. “It’s that the technology world hasn’t quite caught up with the need for trained data scientists and the demand for easy-to-use tools that can give industries—from financial and insurance companies to marketing, healthcare, and scientific research organizations—the capability to put the data they gather into meaningful perspective. The current era of extreme data requires new paradigms and practices in data management and analytics, and in 2014 the race will be on to establish leaders in the space.”

Analytics based on the R programming language, geared specifically for data scientists performing statistical analysis, will go mainstream in 2014, according to Alteryx, edging out legacy environments like SAS and SPSS.

“With over 2 million users and another 3 million analysts looking for better solutions, R’s time is now,” Alteryx says.

The coming year will see an explosion of interactive applications—Web apps, mobile apps, social apps—built on the Hadoop platform and interacting with people in real-time, says Monte Zweben, co-founder and CEO of Splice Machine, provider of a real-time SQL-on-Hadoop database.

“2014 is going to bring the real-time big data application platform,” Zweben says. “You’re not just going to be analyzing yesterday’s data today. You’re going to be analyzing data from five minutes ago, one minute ago. You’re going to have interactive applications that make decisions in real-time.”

“While Hadoop is a fantastic platform, it still requires a little more work,” Splice Machine’s Zweben says. “It requires some handholding.”

In 2014, he says, you’ll see Hadoop evolve techniques for deeper security, deeper operational management, deeper resource management and multi-site replication.

“You’ll see all the kinds of things that enterprises require,” he says. “They’ll become more standardized in these platforms. I think that’s going to be a major focus.

To signal the widely accepted commercialization of big data platforms, at least one Hadoop or NoSQL provider will go public in 2014, predicts Larry Warnock, president and CEO of Gazzang, a provider of security solutions for big data and cloud environments.

“I’m not going to predict which one, but I expect at least one Hadoop or NoSQL provider to go public in 2014,” Warnock says. “According to Wikibon, revenue for NoSQL software and services made up $286 million in 2012 and is forecast to reach $1.825 billion in 2017. The market is being driven by the growing enterprise demand for more flexible, scalable and affordable data management solutions (both analytical and transactional) designed for the new age of the cloud and big data.”

A new data and analytics stack will emerge in 2014 with new solutions for databases, analytics and visualization, all disrupting the traditional mega-vendors, Alteryx predicts. The mega-vendors will respond with hastily thrown together new solutions.