Big Data Grows Up at Strata+Hadoop World 2016

I recently spent time at Strata+Hadoop World 2016 in New York. I attended this event and its predecessor, Hadoop World, off and on for the past six years. This one in New York had a different feel from previous events including the most recent event in San Jose at the end of March. Perhaps because of its location in one of the financial and commercial hubs of the world, the event had much more of a business orientation. But it’s not just location. Past events have been held in New York also, and I see the business focus as a sign of the Hadoop market maturing.

Our research shows that big data can have significant business benefits. In our Big Data and Analytics benchmark research, more than three-quarters (78%) of participants indicated that predictive analytics isvr_big_data_analytics_19_important_areas_of_big_data_analytics_updated the most important area of big data  analytics for their organization. In our Predictive Analytics research almost three out of five (57%) organizations said they have achieved a competitive advantage through their application of advanced analytics. Thus we are moving beyond the early adopter phase of the technology adoption life cycle into the early majority. More and more organizations recognize that big data and advanced analytics can provide a competitive advantage. As a result, we see more focus on the business value of it, not just the technology required to pursue this advantage.

At the Strata+Hadoop World keynote presentations many vendors chose to bring their customers on stage or share stories about how their customers are positively impacting their organizations with big data technology. There were also plenty of technical training sessions, including two full days of training prior to the keynotes and expo, but the main stage of the event was focused on what you can do with big data rather than how to do it. The attendees also seemed to bring a business focus to the event. I spoke with multiple vendors in the expo hall who had attended both the Strata+Hadoop event in San Jose earlier this year and the New York event. They all described customer interactions that had more of a business focus than at previous events. People came looking for ways to apply big data technology to real business needs.

This is not say there wasn’t plenty of technology at the event including in particular data science, streaming data and data preparation and governance. Tutorials were offered on a variety of data science topics including how to implement machine learning in programming languages such as Python and Spark. Our research shows that Python is one of the most popular languages for data science analyses, in use by more than one-third (36%) of organizations. As I have written previously, Spark is growing in popularity as a way of providing big data, machine learning and real-time capabilities. At least half a dozen vendors ranging from large to small participated in the expo, touting their data science capabilities, and many other vendors’ marketing materials described how they support data science, for instance with data preparation tools that enable the data science process.

Processing streaming data in real time was also a frequent theme. Part of what makes big data big is that it is being generated constantly. It follows that you can probably get value out of analyzing that data in real time as it is being generated. In our research real-time analytics is the second-most frequently cited (by 54%) area of big data analytics, after predictive analytics. In its original incarnation, Hadoop was designed as a batch-oriented system, but as it has grown in popularity, much attention has been given to adding real-time capabilities to the Hadoop ecosystem, which I have described.

The themes of data preparation and governance come as no surprise. Our Big Data Integration benchmark research shows that reviewing data for quality and consistency issues (52%) and preparing data (46%) are cited as the two most time-consuming aspects of the big data integration process. Similarly our big data analytics research shows that data quality and information management is the second-most common barrier to big data analytics, cited by 39 percent of organizations. Vendors and the big data community are on the right track in addressing these issues.

The big data community continues to evolve, and the Strata+Hadoop World events are helping to foster dialog, education and growth. I’d say that this most recent event is evidence that the big data community is “growing up,” meaning that the focus has shifted to delivering business value. Strata+Hadoop World is a place where you can learn not only about the technology of big data but also how to solve business problems.


David Menninger

SVP & Research Director

Follow Me on Twitter @dmenningerVR and Connect with me on LinkedIn.

Who’s Hot in Analytics and Business Intelligence

Ventana Research recently completed the most comprehensive evaluation of analytics and business intelligence products and vendors available anywhere. As I discussed recently, such research is necessary and timely as analytics and business intelligence is now a fast-changing market. Our Value Index for Analytics and Business Intelligence in 2015 scrutinizes 15 top vendors and their product offerings in seven keyvr_VI_BI_2015_Weighted_Overall categories: Usability, Manageability, Reliability, Capability, Adaptability, Vendor Validation and TCO/ROI. The analysis shows that the top supplier is Information Builders, which qualifies as a Hot vendor and is followed by 10 other Hot vendors: SAP, IBM, MicroStrategy, Oracle, SAS, Qlik, Actuate (now part of OpenText) and Pentaho.

The evaluations drew on our research and analysis of vendors’ and products along with their responses to our detailed RFI or questionnaire, our own hands-on experience and the buyer-related findings from our benchmark research on next-generation business intelligence, information optimization and big data analytics. The benchmark research examines analytics and business intelligence from various perspectives to determine organizations’ current and planned use of these technologies and the capabilities they require for successful deployments.

We find that the processes that comprise business intelligence today have expanded beyond standard query, reporting, analysis and publishing capabilities. They now include sourcing and integration of data and at later stages the use of analytics for planning and forecasting and of capabilities utilizing analytics and metrics for collaborative interaction and performance management. Our research on big data analytics finds that new technologies collectively known as big data vr_Big_Data_Analytics_15_new_technologies_enhance_analyticsare influencing the evolution of business intelligence as well; here in-memory systems (used by 50% of participating organizations), Hadoop (42%) and data warehouse appliances (33%) are the most important innovations. In-memory computing in particular has changed BI because it enables rapid processing of even complex models with very large data sets. In-memory computing also can change how users access data through data visualization and incorporate data mining, simulation and predictive analytics into business intelligence systems. Thus the ability of products to work with big data tools figured in our assessments.

In addition, the 2015 Value Index includes assessments of their self-service tools and cloud deployment options. New self-service approaches can enable business users to reduce their reliance on IT to access and use data and analysis. However, our information optimization research shows that this change is slow to proliferate. In four out of five organizations, IT currently is involved in making information available to end users vr_Info_Optimization_01_whos_responsible_for_information_availabilityand remains entrenched in the operations of business intelligence systems.

Similarly, our research, as well as the lack of maturity of the cloud-based products evaluated, shows that organizations are still in the early stages of cloud adoption for analytics and business intelligence; deployments are mostly departmental in scope. We are exploring these issues further in our benchmark research into data and analytics in the cloud, which will be released in the second quarter of 2015.

The products offered by the five top-rated com­pa­nies in the Value Index provide exceptional functionality and a superior user experi­ence. However, Information Builders stands out, providing an excep­tional user experience and a completely integrated portfolio of data management, predictive analytics, visual discovery and operational intelligence capabilities in a single platform. SAP, in second place, is not far behind, having made significant prog­ress by integrating its Lumira platform into its BusinessObjects Suite; it added pre­dictive analytics capabilities, which led to higher Usability and Capability scores. IBM, MicroStrategy and Oracle, the next three, each provide a ro­bust integrated platform of capabilities. The key differentiator between them and the top two top is that they do not have superior scores in all of the seven categories.

In evaluating products for this Value Index we found some noteworthy innovations in business intelligence. One is Qlik Sense, which has a modern architecture that is cloud-ready and supports responsive design on mobile devices. Another is SAS Visual Analytics, which combines predictive analytics with visual discovery in ways that are a step ahead of others currently in the market. Pentaho’s Automated Data Refinery concept adds its unique Pentaho Data Integration platform to business intelligence for a flexible, well-managed user experience. IBM Watson Analytics uses advanced analytics and VR_AnalyticsandBI_VI_2015natural language processing for an interactive experience beyond the traditional paradigm of business intelligence. Tableau, which led the field in the category of Usability, continues to innovate in the area of user experience and aligning technology with people and process. MicroStrategy’s innovative Usher technology addresses the need for identity management and security, especially in an evolving era in which individuals utilize multiple devices to access information.

The Value Index analysis uncovered notable differences in how well products satisfy the business intelligence needs of employees working in a range of IT and business roles. Our analysis also found substantial variation in how products provide development, security and collaboration capabilities and role-based support for users. Thus, we caution that similar vendor scores should not be taken to imply that the packages evaluated are functionally identical or equally well suited for use by every organization or for a specific process.

To learn more about this research and to download a free executive summary, please visit.


Ventana Research