Interesting news from Ingres today that it is teaming up with VectorWise, a database engine spin-off from Amsterdam’s Centrum Wiskunde & Informatica (CWI) scientific research establishment, to collaborate on a new database kernel project.
The Ingres VectorWise project will create a new open source storage engine for the Ingres Database that will better enable it to be positioned as a platform for data warehouse and analytic workloads, although Ingres does not have detailed plans for the productization of the technology at this stage. The starting point for the project is the theory that modern multi-core parallel processors now look like, and behave like, symmetrical multi processing (SMP) servers, and that on-chip memory is taking the place of RAM, but that database software has not been updated to take advantage of process developments.
In order to do so Ingres and VectorWise will be collaborating on vectorized execution, which sees multiple instructions processed simultaneously, and in-cache processing, through which the execution occurs within the CPU cache and main memory is effectively treated like disk. The result, according to Ingres, is to reduce the I/O bottleneck for query processing. Additionally, the VectorWise engine enables on the fly decompression and operation handling in memory and includes a compressed column store.
It is claimed that the Ingres VectorWise project will deliver 10x performance increases over the current Ingres database.
VectorWise span off from CWI in 2008 to commercialize the the X100 system previously created by its database architecture research group. Development of X100, now also known as VectorWise, has been led by respected research scientists Peter Boncz and Marcin Zukowski.
Ingres maintains that by working with the CWI research scientists it has proven that their theories are technically feasible in a commercial product. Bringing such a commercial product to general availability is the next step, and history has proven that can be easier said than done. With that caveat we are impressed with the vision and ambition that Ingres is demonstrating.
Subscribe via RSS
7 comments ↓
[…] launches new database engine project Matthew Aslett, July 29, 2009 @ 5:47 am ET I’ve just posted details of a new database engine project launched by Ingres over at The 451 Group’s Too Much […]
Hi Matthew,
Are you sure the new engine will be open source? I know Ingres is open source, but I think the question of whether or not the VectorWise engine will be open source is still “to be determined”.
Also, I think VectorWise is more than just an in-memory engine. I believe data is stored on disk, but by being a compressed column-store, disk is not the bottleneck, so VectorWise runs at the speed of memory.
I have blogged what I know about VectorWise at:
http://dbmsmusings.blogspot.com/2009/07/watch-out-for-vectorwise.html
Clearly this deal has been in the works for a while. As I “tweeted” on April 2 (@sethgrimes): According to this page, Marcin Zukowski of CWI spoke on MonetDB/X100 at Ingres Engr Summit last April: http://tinyurl.com/clzeu2
Hi Daniel,
Thanks for the comment. I enjoyed reading your perspective and I agree with your views on the potential benefits, limitations and go to market strategy.
I asked Ingres directly if Ingres VectorWise would be open source and they said yes, over time, and that they had academic partners working on the source code.
And you are right I was over-simplifying somewhat on the in-memory aspect. We have more details in our formal report, which will be available for clients shortly at: http://www.the451group.com/report_view/report_view.php?entity_id=59203
Matt
Matt,
Thanks. Also: I can confirm that I was also told that academic partners will receive the source code.
[…] Amsterdam’s Centrum Wiskunde & Informatica (CWI) scientific research establishment, to collaborate on a new database-kernel project designed to better enable it to be positioned as a platform for […]
[…] the Ingres VectorWise project (previously mentioned here) will create a new storage engine for the Ingres Database positioned as a platform for […]