Entries from July 2011 ↓

NoSQL has never really been about SQL. As we pointed out in our NoSQL, NewSQL and Beyond report, “[one] of the NoSQL idiosyncrasies is that in most cases SQL itself is not the ‘problem’ being avoided. Indeed, a better term might be ‘NoSchema,’ given that a more common quality is a rejection of fixed table schema and join operations”.

Nevertheless the NoSQL term has stuck, and also inspired NewSQL (which, as critics have pointed out, is not really about SQL either), while a number of NoSQL providers started to look at how they could actually add support for SQL queries to their respective databases.

The recently-released version 0.8 of Apache Cassandra features the first implementation of Cassandra Query Language (CQL), an SQL-like query language, for Cassandra.

Meanwhile Couchbase and SQLite have teamed up to create UnQL (Unstructured Query Language), a new data query language for unstructured data. Pronounced ‘uncle’, UnQL is designed to remove the burden of query planning, optimization and execution from NoSQL developers by providing an adaptation of the SQL structured query language for unstructured data models.

As can be seen by an example of the draft syntax, UnQL is designed to be familiar to SQL developers, while also enabling querying over complex and unstructured storage models, such as document models.

UnQL was created by Couchbase CTO and CouchDB creator Damien Katz, alongwith SQLite creator and founder Richard Hipp and both Couchbase and SQLite have committed to implementing UnQL in future versions of their database products.

UnQL is not designed to be specific to select database products, however, and the specification is being released to the public domain at www.unqlspec.org. There is also the potential that open source parsers and query planning implementations will be created to foster adoption.

One of the principle drivers behind UnQL’s development is that a common query language is necessary to drive NoSQL adoption in the same way SQL drove adoption in the relational database market.

It remains to be seen whether UnQL will be picked up by other projects, although the release to the public domain should give confidence that this is not an attempt to force the industry to adopt a ‘standard’ from a single vendor.

When I was messing around with Indeed.com job trends the other day I was struck by an interesting trend relating to the five recent major M&A deals involving analytic database vendors: Netezza, Sybase, Greenplum, Vertica and Aster Data.

netezza, sybase iq, greenplum, vertica, aster data Job Trends

Netezza jobs – Sybase Iq jobs – Greenplum jobs – Vertica jobs – Aster Data jobs

The trends aren’t immediately obvious from that chart, but if we break them out individually and add a black dot to indicate the approximate date of the acquisition announcement it all becomes clear.

(Note: scale varies from chart to chart)

While the acquisitions have accelerated job postings for all acquired analytic databases, Greenplum has clearly been the biggest beneficiary. Indeed.com’s data also explains why this might be: EMC/Greenplum is responsible for over 50% of the current Greenplum-related job postings on the site (excluding recruiter postings).

Greenplum had 140 employees when it was acquired in July 2010. Based on the hiring growth illustrated above, EMC’s Data Computing Products Division is set to reach 650 by the end of the year.

Netezza started with a much larger base, but IBM is expected to increase headcount at Netezza from 500 in September 2010 to a target of 800 by year-end. Thanks, no doubt, to Netezza’s larger installed base, IBM is responsible for just 7.7% of Netezza job postings.

This highlights something we recently noted in a 451 Group M&A Insight report: in order to make a considerable dent in the dominance of the big four, any acquiring company will not only have to buy a data-warehousing player but also invest in its growth.

While Vertica and Aster Data are both heading in the right direction, we believe that HP and Teradata will have to accelerate their investment in the Vertica subsidiary and the new Aster Data ‘center of excellence’ respectively.

HP recently told us headcount has grown about 40% since the acquisition (it wasn’t being specific, but Vertica reported 100 employees in January). HP/Vertica is currently responsible for 13.9% for Vertica-related job postings on Indeed.com

We had speculated that Teradata would need to similarly boost the headcount at Aster Data beyond the estimated 100 employees. Teradata/Aster Data is responsible for 24% of job postings for Aster Data.

But what of Sybase? While Sybase IQ also has a larger installed base, SAP/Sybase are responsible for just 6.4% of the Sybase IQ-related job postings on Indeed.com. The Sybase IQ chart illustrates some common sense investment advice: the value of your investment can go down as well as up.

Recently there have been a spate of postings regarding job trends for distributed data management technologies including Hadoop and the various NoSQL databases.

One thing you rarely see on these job trends charts is comparison with an incumbent technology, for context. There’s a reason for that, as this comparison of database-related jobs from Indeed.com illustrates:

Although there has been a recent increase in job postings related to Hadoop and MongoDB, both are dwarfed, in absolute terms, by the number of job postings involving SQL Server and MySQL.

So why all the fuss about Hadoop and NoSQL, from a corporate perspective? This chart, showing the relative growth for the same data management technologies, says it all:

It has been fascinating to watch how the industry has responded to ‘NewSQL’ since we published our first report using the term.

From day one the term has taken on a life of its own as the vendors such as ScaleBase, VoltDB, NimbusDB and Xeround have picked it up and run with it , while the likes of Marten Mickos and Michael Stonebraker have also adopted the term.

The reaction hasn’t been all positive, of course, although much of the criticism has been of the “are you kidding?” or “this is getting silly” variety rather than constructive debate about either the term or the associated technologies.

Another popular response is along the lines of “does this mean the end of NoSQL?”. I think it is important to address this question because it depends on a common misunderstanding about technology: that in order for the latest technology to succeed it is necessary for the technology that immediately preceded it to fail.

While our report into NoSQL, NewSQL and Beyond identified common drivers for interest in NoSQL and NewSQL databases, as well as data caching/grid technologies, in truth there is a significant difference between the requirements for databases that provide relaxed consistency and/or schema dependency and those that retain the ACID properties of transactional database systems.

Although there will be isolated examples, it is going to be rare, therefore, that any potential adopter would be directly comparing NoSQL and NewSQL technologies unless they are still at the stage trying to figure out the level of consistency required for an individual application.

The other option they would have is to use an existing SQL database, particularly Oracle’s MySQL, which provides the middle ground that overlaps both NoSQL and NewSQL. A significant number of the NoSQL deployments we have identified have migrated from MySQL, while existing MySQL deployments (although probably not the same ones) are also targets for the numerous NewSQL vendors.

VoltDB is a primary example, as last’s week’s GigaOm article covering CTO Michael Stonebraker’s view on Facebook’s MySQL ‘fate worse than death’ illustrated.

Much debate (125 comments at last count) has followed Stonebraker’s assertion that Facebook would be better off migrating to a NewSQL offering like VoltDB, most of which has not supported his view.

There’s a good reason for that. There is a good argument to be made that if you were trying to create Facebook from scratch today you probably wouldn’t choose the shard management overhead involved in MySQL. In that regard, Stonebraker has a point.

However, the fact is that MySQL was pretty much the only logical choice when Facebook began and its commitment to MySQL has grown over the years. The company is now probably one of the world’s experts in scaling and managing MySQL – to the extent that Facebook engineer Domas Mituzas argues that the operational overhead in handling sharding and availability of MySQL has become a constant cost.

Under those circumstances it would take something significant for a company like Facebook to even consider migrating to a MySQL alternative. Database migration projects are costly and complex and extremely rare – even at non-Facebook scale.

And it is not as if the company hasn’t experimented with other database technologies – having created Apache Cassandra and adopted Apache HBase for its Messages update.

This is exactly the polyglot persistence strategy we are seeing from NoSQL and NewSQL adopters: retaining MySQL (or another SQL database) where is makes sense to do so, while adding NoSQL and perhaps NewSQL for new projects and applications for which it is appropriate.

One other point to note, however, is that adopting a NewSQL technology might not require migrating away from MySQL. While the NewSQL category includes new database products such as VoltDB, it also includes alternative MySQL storage engines and database load balancing and clustering products such as ScaleBase and ScalArc, which are specifically designed to improve the scalability of MySQL (with other SQL databases to come) in order to avoid migration to an alternative database.

Adoption of these technologies does not require the complete abandonment of ‘standard MySQL’ any more than the adoption of NoSQL for non-ACID application requirements does, and it certainly doesn’t require the abandonment of NoSQL.