Newsletters




The NoSQL Movement: Hype or Hope?


If you spend any time at all reading IT trade journals and websites, you've no doubt heard about the NoSQL movement.  In a nutshell, NoSQL databases (also called post-relational databases) are a variety of loosely grouped means of storing data without requiring the SQL language.  Of course, we've had non-relational databases far longer than we've had actual relational databases.  Anyone who's used products like IBM's Lotus Notes can point to a popular non-relational database.  However, part and parcel of the NoSQL movement is the idea that the data repositories can horizontally scale with ease, since they're used as the underpinnings of a website.  For that reason, NoSQL is strongly associated with web applications, since websites have a history of starting small and going "viral," exhibiting explosive growth after word gets out. 

In contrast, most relational database platforms require a lot of modifications to successfully grow in scalability from small to medium to global.  For a good review of such a growth pattern, and the frequent re-designs that explosive growth requires, read the story of MySpace's evolution as a Microsoft SQL Server shop at http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000004532.

On the negative side, NoSQL databases circumvent the data quality assurance of relational databases best known as ACID (atomicity, consistency, isolation, durability) property of transactions.  So, while NoSQL databases might be very fast and scale easily, they do not typically guarantee that a transaction will be atomic, consistent, isolated, and durable.  In other words, you could lose data, and there is no guarantee that a transaction will always complete successfully, or completely roll back.

The market for NoSQL is still very immature and crowded with many open source and proprietary products.  Well-known vendors for NoSQL databases include Google's Big Table offering and Amazon's Dynamo, both of which are available as inexpensive cloud services.  Some of the most talked about NoSQL platforms on the open source side include Apache's HBase and CouchDB; Facebook's Cassandra; and LinkedIn's Project Voldemort.

All of this hype begs the question of why the NoSQL movement is gaining traction.  While many analysts declare it's because web developers are loath to learn another language like SQL, I believe that is only a symptom of a deeper issue.  Basically, web developers don't want to learn relational database design!  They don't want to go through the traditional steps of normalization and a waterfall-style development process.  This matches exactly my belief that developers are being  falsely classified as "software engineers," when, in fact, they are much more like artists and other creative types who enjoy starting a project with a quick sketch, then returning many times to fill in more detail with each iteration.

So, what does the future hold for the NoSQL movement?  Is it simply an over-hyped technology - like the object-oriented database - that will soon vanish?  My assertion is that while some specific products in the NoSQL movement will perish in a sort of Darwinian contest, the overall movement will continue to refine itself until a handful of top-tier products remain.  A deeper set of questions should be asked - what will the top relational database vendors (Oracle, Microsoft, and IBM) do to address the needs of this market, and can they react in time to take advantage of the market opportunity?

You can find lots of high-level information about the NoSQL movement at http://en.wikipedia.org/wiki/NoSQL and http://nosql-database.org/. 


Sponsors