MySQL NoSQL survey highlights role of polyglot persistence

The MySQL developer website is currently running a poll to gauge the adoption of NoSQL database projects by MySQL developers.

The results are interesting, particularly in relation to our research report on the emergence and adoption of NoSQL and NewSQL databases, which I am completing this week.

Our research has shown that one of the drivers of NoSQL has been performance, and in particular the failure of MySQL to provide predictable performance at scale. We do see NoSQL being deployed for applications that previously ran on MySQL, or for which MySQL would previously have been the natural choice.

For example, while Facebook continues to run its core applications on MySQL running the InnoDB storage engine and memcached it also created what became Apache Cassandra to power its inbox search, and selected Apache HBase for its Messages application, which was updated in late 2010 to combine chat, email, and SMS, having found that MySQL was unable to deliver the performance required for large data sets.

Similarly, content discovery service StumbleUpon adopted HBase following problems with MySQL failover, Digg replaced its MySQL cluster with Apache Cassandra, and Wordnik replaced MySQL with MongoDB.

Clearly, however, not every MySQL application is suitable for a NoSQL database. Just because almost 80% of the MySQL survey respondents are adopting NoSQL database, does not mean they are replacing MySQL with NoSQL.

Like Facebook, many major NoSQL users also continue to use MySQL, including Twitter which back-tracked on a planned migration of its core status table to Apache Cassandra in 2010. It continues to use MySQL, but is adopting Cassandra for newer projects.

The adoption of multiple database products depending on the nature of the application is another of the six major drivers for NoSQL and NewSQL adoption highlighted by our research.

The theory of polyglot persistence has developed based on the fact that different data storage models have their own strengths and the acceptance that while the relational model is suitable for a large proportion of data storage requirements, there are times when a document, graph, or object database might be more suitable, or even a distributed file system.

Facebook and Twitter are prime examples of polyglot persistence in action, and the survey of MySQL developers shows that the practice is widespread. At the time of writing 205 people have responded to the survey, providing 421 responses.

If we exclude the 42 that indicate they are not using a NoSQL database, that means that the remaining 163 people are using 379 NoSQL databases, which equates to 2.33 databases per respondent, not including their existing use of MySQL or other traditional or NewSQL databases.

I’ll provide more details of the research report, including the other four adoption drivers, once the report is published. The report contains analysis of the drivers behind the development and adoption of NoSQL and NewSQL databases, as well as the evolving role of data grid technologies, as well as the associated use cases. It will be available soon for clients of our Information Management and CAOS practices.

Tags: , , , ,

4 comments ↓

#1 Mat Keep on 03.24.11 at 1:17 pm

The research report sounds great – I was wondering if it includes MySQL Cluster in the NewSQL category ? (need to declare an interest as I work on MySQL Cluster)

MySQL Cluster (aka NDB) provides:
– SQL and non-SQL (dare I say NoSQL interfaces (C++, REST, Java, JPA, LDAP – with memcached under development)
– auto-partitions tables across pools of commodity servers, which coupled with multi-master access gives very high write scalabiity
– the ability to modify schema on-line, so apps can rapidly evolve without ALTER TABLE statements that take the database offline for a week
– auto replication, failover and recovery
– geo-distribution

Either way, look forward to reading it – I’m sure it will contain the usual quality insight of CAOS reports

#2 Matthew Aslett on 03.25.11 at 9:10 am

Hi Mat,

Thanks for the note. MySQL Cluster certainly gets a mention.

Matt

#3 Mat Keep on 03.31.11 at 9:18 am

Great Matt (I think !), look forward to reading it

#4 Karan Kumar on 04.08.11 at 3:12 pm

Nice peace of info on MySQL forks..!! πŸ™‚