NoSQL, NewSQL and Beyond: The answer to SPRAINed relational databases

The 451 Group’s new long format report on emerging database alternatives, NoSQL, NewSQL and Beyond, is now available.

The report examines the changing database landscape, investigating how the failure of existing suppliers to meet the performance, scalability and flexibility needs of large-scale data processing has led to the development and adoption of alternative data management technologies.

Specifically, the report covers:

• NoSQL databases designed to meet scalability requirements of distributed architectures and/or schema-less data management requirements, including big tables, key value stores, document database and graph databases

• NewSQL databases designed to meet scalability requirements of distributed architectures or to improve performance such that horizontal scalability is no longer a necessity, including new MySQL storage engines, transparent sharding technologies, software and hardware appliances, and completely new databases

• Data grid/cache products designed to store data in memory to increase application and database performance, covering a spectrum of data management capabilities from non-persistent data caching to persistent caching, replication, and distributed data and compute grid functionality

You can see how these products fit into the wider data management landscape from the chart below. The shaded areas are those specifically covered in this report.

The answer to SPRAINed relational databases

SPRAIN, used in the above graphic, is an acronym that refers to the six key factors driving the adoption of alternative data management technologies to traditional relational databases that are being ‘sprained’ as a result of being stretched beyond their normal capacity by the needs of high-volume, highly distributed or highly complex applications.

Those six key drivers, and their associated sub-drivers, are as follows:

• Scalability – hardware economics
• Performance – MySQL limitations
• Relaxed consistency – CAP theorem
• Agility – polyglot persistence
• Intricacy – big data, total data
• Necessity – open source

The report examines each of these drivers and sub-drivers in turn, investigating how they are driving interest in alternative database approaches in general, and how they prompted the development of specific NoSQL, NewSQL and data grid/cache products and services.

It continues with profiles of the individual database alternatives and their use cases and case studies before concluding with a discussion of the impact of these database alternatives on the wider database market and the likely consolidation, confluence and proliferation of various technologies looking forward.

Here’s a selection of some of our key findings:

• The database market remains dominated by relational databases and the incumbent industry giants, but the emergence of NoSQL and NewSQL alternatives has in part been driven by the inability of these products to address emerging distributed and schema-less data management requirements.

• Polyglot persistence, and the associated trend toward polyglot programming, is driving developers toward making use of multiple database products depending on which might be suitable for a particular task.

• The NoSQL projects were developed in response to the failure of existing suppliers to address the performance, scalability and flexibility requirements of large-scale data processing, particularly for Web and cloud computing applications.

• NewSQL and data-grid products have emerged to meet similar requirements among enterprises, a sector that is now also being targeted by NoSQL vendors.

• While NoSQL is seen as a software innovation prompted by the need to deal with large volumes of data, the software innovation was a direct response to the improved performance of commodity hardware clusters and the ability to spread data storage and processing across that hardware.

• Changing hardware economics mean that distributed server architecture is increasingly being adopted in traditional enterprise environments. The emergence of NewSQL providers is a direct response to the increasing need for scalable data management products to make more efficient use of this architecture.

• Distributed data-grid/cache products are increasingly being positioned as potential alternatives to relational databases as the primary platform for distributed data management, with a relational database relegated to a supporting role.

The report is available now from both the Information Management and Open Source practices (non-clients can apply for trial access). I will also be presenting the findings at the forthcoming Open Source Business Conference.