I have recently written about the organizational and cultural aspects of being data-driven, and the potential advantages data-driven organizations stand to gain by responding faster to worker and customer demands for more innovative, data-rich applications and personalized experiences. I have also explained that data-driven processes require more agile, continuous data processing, with an increased focus on extract, load and transform processes — as well as change data capture and automation and orchestration — as part of a DataOps approach to data management. Safeguarding the health of data pipelines is fundamental to ensuring data is integrated and processed in the sequence required to generate business intelligence. The significance of these data pipelines to delivering data-driven business strategies has led to the emergence of vendors, such as Astronomer, focused on enabling organizations to orchestrate data engineering pipelines and workflows.
In my first perspective on cloud computing realities, I covered some of the cost considerations associated with cloud computing and how the cloud costing model may be different enough from on-premises models that some organizations are taken by surprise. In this perspective. I’d like to focus on realities of hybrid and multi-cloud deployments.
I recently wrote about the need for organizations to take a holistic approach to the management and governance of data in motion alongside data at rest. As adoption of streaming data and event processing increases, it is no longer sufficient for streaming data projects to exist in isolation. Data needs to be managed and governed regardless of whether it is processed in batch or as a stream of events. This requirement has resulted in established data management vendors increasing their focus on streaming data and event processing through product development as well as acquisitions. It has also resulted in streaming and event specialists, such as Confluent, adding centralized management and governance capabilities to their existing offerings as they seek to establish or reinforce the strategic importance of streaming data as part of a modern approach to data management.
The migration to cloud is obvious. Organizations are adopting cloud computing for all variety of applications and use cases. Managed cloud services, commonly referred to as software as a service (SaaS), offer many benefits to organizations including significantly reduced labor costs for system administration and maintenance, as many of these costs are shifted to the software vendor. SaaS also provides organizations with faster time to value as they adopt new technologies by eliminating the need to acquire and configure hardware, and it also eliminates the need to install software. In fact, we assert that by 2025, nine in 10 organizations will be using multiple cloud applications in order to minimize the costs of administration and maintenance. Yet, there are some challenges associated with cloud computing I’d like to address in a series of Analyst Perspectives:
I have written recently about increased demand for data-intensive applications infused with the results of analytic processes, such as personalization and artificial intelligence (AI)-driven recommendations. Almost one-quarter of respondents (22%) to Ventana Research’s Analytics and Data Benchmark Research are currently analyzing data in real time, with an additional 10% analyzing data every hour. There are multiple data platform approaches to delivering real-time data processing and analytics and more agile data pipelines. These include the use of streaming and event data processing, as well as the use of hybrid data processing to enable analytics to be performed on application data within operational data platforms. Another approach, favored by a group of emerging vendors such as Rockset, is to develop these data-intensive applications on a specialist, real-time analytic data platform specifically designed to meet the performance and agility requirements of data-intensive applications.