The Key Value of Database Simplicity

I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is why I am examining them over a series of analyst perspectives. Following a closer look at graph databases and document-oriented databases, I now turn to key-value stores. 

As I previously described, the key-value model represents the most simplistic of the various NoSQL data models, storing data as a list of pairs of keys (such as name, gender, age) and associated values. The key-value approach can also form the basis of other NoSQL database models: wide-column stores, in which multiple sets of values can be associated with each individual key; document-databases, in which a series of nested key-value pairs can be stored as a semi-structured document; and graph databases, in which the relationships between keys and values are also stored as primary attributes. Given that the functionality provided by key-value stores underpins more sophisticated databases with richer data models, one might reasonably ask why anyone would choose to limit themselves with a key-value database.  

To understand the usefulness of the key-value model, one need only look back to the research and development projects that inspired the evolution of the various open-source and proprietary implementations that exist today. Two research projects were particularly important in the emergence of key-value databases: Google’s Bigtable and Amazon’s Dynamo. The research papers associated with these projects were published in 2006 and 2007 respectively and articulate why it was necessary for both companies to develop their own key-value databases despite having large-scale implementations of relational databases at their disposal. As Amazon’s chief technology officer Werner Vogels elaborates, Amazon’s ecommerce business was at the time pushing the limits of its relational databases in terms of availability, scalability and performance.  

Just as significantly, however, the relational model was unsuited to Amazon’s workload. About 70% of Amazon’s database queries were single-row lookups. The breadth of functionality provided by the relational model is not just overkill for these types of queries but may impede performance. Key-value stores are designed to favor availability over consistency. While this means key-value stores are unsuitable for workloads that require atomic, consistent, isolated and durable transactions, it does mean that key-value stores are designed to be highly available. The inherent flexibility of the key-value model is also beneficial: there are no structural requirements on the values associated with each key and no limitations on the number of individual keys that can be stored — at least, in theory. Many key-value stores are deployed in-memory to meet performance requirements of the associated application and are therefore limited to the number of rows that can be stored in the available memory. That is a limitation of being in-memory rather than the data model.  

Less than one-half (43%) of participants in Ventana Research’s Analytics and Data Benchmark Research are using NoSQL databases in production today or plan to do so within two years, and key-value stores only represent a proportion of those. In comparison, more than four in five participants (81%) are using or plan to use relational databases.  

It is a common fallacy that increased adoption of NoSQL databases depends on the direct replacement of relational databases, however. Key-value stores can be, and often are, used in concert with relational databases, with the key-value store handling simple but high-performance lookup queries and a relational database being used for more complex workloads. Any application that requires low-latency processing of a high volume of simple read/write operations is a good use case for a key-value store. Examples include maintaining session state for web applications and gaming environments, application data caching, serving user profiles and preferences, providing leaderboards, supporting shopping carts and serving advertisements.  

These use cases need not be standalone applications but could be discrete microservices delivered as part of a larger application architecture alongside a relational database serving more complex transactional and analytic operations. In an e-commerce use case, for example, a key-value store might be used to support the shopping cart functionality as well as the recording of session-related data, while a relational database might be used to perform the actual purchase transaction and provide information about a customer’s previous purchases and search history.  

In a microservices-based architecture, databases can be adopted to add new and innovative functionality to complement established and mature capabilities that run on an incumbent database. I assert that through 2026, emerging relational and non-relational database providers will primarily be adopted for new applications with incumbent relational database vendors deployed for the majority of existing operational workloads.  

I posed the question above as to why anyone would choose a key-value store when more sophisticated databases with richer data models are available. To use an analogy, while multi-function tools are extremely useful for numerous home maintenance use cases, sometimes what you really need is a hammer. I recommend that all organizations considering options for databases to support new applications evaluate the potential suitability of key-value stores and choose the most appropriate data model to fulfill the task at hand. 

Regards,

Matt Aslett