Ventana Research Analyst Perspectives

As enterprises seek to increase data-driven decision-making, many are investing in strategic data democratization initiatives to provide business users and data analysts with self-service access to data across the enterprise. Such access has long been a goal of many enterprises, but few have achieved it. Only 15% of participants in Ventana Research’s Analytics and Data Benchmark Research say their organization is very comfortable allowing business users to work with data that has not been integrated or prepared for them by IT. Removing barriers that prevent or delay users from gaining access to data enables it to be treated as a product that is generated and consumed, either internally by workers or externally by partners and customers. 

While data democratization facilitates access to data, it is not a free-for-all. Making data available as a product requires an understanding of how, when and why data is produced and consumed across an enterprise and by whom. The capabilities to provide enterprises with information about how data is generated and used across the organization already exist but are distributed across a variety of products, including data integration, data catalog, data quality, data lineage, metadata management and master data management tools. Bringing these capabilities together to provide a holistic view of data production and consumption and streamlining analytics and BI is the realm of data intelligence. 

The term data intelligence has been used by multiple vendors across analytics and data for several years, typically with unique definitions that make self-serving reference to the functional strengths of their products. Over time, it has become clear that there is a common thread to these multiple definitions related to providing enterprises with information about how data is produced and consumed across the organization. It has also become clear that data intelligence is fundamental to providing data analysts and business users with governed self-service access to data across an enterprise.  

When providing self-service access to data, it is not enough for enterprises to know in which data platforms the data is persisted. Enterprises need to ensure that business users and data analysts can find the data they need, understand what it means, and trust that it is valid, current, and can be relied upon in business decision-making. Data intelligence tools and platforms represent a layer in the stack above data platforms that provides information management capabilities including integration, catalog, quality, lineage, metadata and master data. I assert that through 2027, three-quarters of enterprises will be engaged in data intelligence initiatives to increase trust in data by leveraging metadata to understand how, when and where data is used in the organization and by whom. 

Many organizations see data catalogs as the solution to data democratization because they provide a central repository of the data used across an enterprise, along with guided data discovery capabilities and natural language search. Self-service access to data is only truly valuable, however, if users can trust the data they have access to. In addition to core data and data catalog functionality, data democratization requires data lineage and data quality capabilities as well as contextual understanding of the data, such as its criticality and whether it is subject to regulatory requirements.  

To enable data-as-a-product, enterprises also need to understand how data ownership maps to logical business units and organizational structure. This is facilitated by curated semantic data definitions enabled by artificial intelligence-driven semantic data modeling that provide a common understanding of the data as well as knowledge graphs that represent data and metadata usage and reflect the relationships between data elements. 

These capabilities need to be automated, collaborative and agile to adapt as the business evolves. Ventana Research’s recent DataOps Buyers Guide explained how DataOps supports the automated, collaborative and agile orchestration of data integration and processing pipelines, incorporating improved data reliability and integrity via data monitoring and observability. While DataOps processes and practices are employed by data engineering professionals to accelerate and improve the production of data, we have previously argued that those responsible for delivering analytics initiatives also need to adopt AnalyticOps processes and practices to support automated, collaborative and agile approaches to the consumption of data. 

By providing a holistic view of data production and consumption, data intelligence becomes the connective tissue that brings together investments in data fabric and data mesh. I previously explained that – despite often being used interchangeably – data fabric and data mesh relate to independent but intersecting concepts. Data fabric is differentiated by its focus on how data is produced – specifically, the tools and technologies primarily aimed at data management and governance practitioners used to deliver agile data integration.  

Data fabric products are largely agnostic to who owns the data and how it is consumed. In comparison, while data mesh is agnostic to the technology that generates, integrates and manages the data, it is focused on who owns the data and how it is consumed by business users. Domain-oriented data ownership is integral to data mesh, with the business departments or units that generate the data responsible for managing ownership of and access to the data. 

Managing data production and data consumption are separate disciplines with different roles, responsibilities, skills and tools. Although this is likely to remain the case, joining the dots between data production and data consumption with data intelligence is essential to delivering on priorities for the use of data and adoption of AI. To deliver data intelligence, I recommend that enterprises look for data integration, data catalog, data quality, data lineage, metadata management and master data management products that enable collaborative approaches to data management and governance, with semantic data modeling and knowledge graph capabilities as well as analytics products that are adaptable to change. I will be evaluating software providers for their ability to deliver functionality required to enable data intelligence in a forthcoming Ventana Research Buyers Guide that will span application integration, data governance, data integration, data quality and MDM. 

Regards,

Matt Aslett