The relational database management system (RDBMS) held a stranglehold on enterprise data management from the mid-1980s through to the mid-2000s. For 25 years the question was never “should I use a relational database,” but “which relational database should I use.” That dominance broke down in the first decade of this century when traditional database architectures failed to meet the challenges of emerging “Web 2.0” internet applications.
The new breed of databases that arose around 2008-2010 was generally referred to as “NoSQL” databases—the SQL language being a hallmark of the relational system. But in a way, SQL was a baby thrown out with the bathwater; it was the transactional model of the RDBMS, not the SQL language which powered the NoSQL revolution.
Over the past 10 years we’ve seen a proliferation of non-relational database systems, usually based around a distributed, fault-tolerant architecture with flexible consistency models—databases such as DynamoDB, Cassandra, and Hadoop. However, in recent years, a new set of cloud-native, SQL-enabled databases have established significant traction.
Particularly notable is Snowflake, which recently achieved the biggest software IPO of all time. Snowflake is a cloud-native SQL-based data warehousing platform, which largely replaced Hadoop as the leading contender for next generation data warehousing.
SQL is on the resurgence in the online transaction processing (OLTP) world as well. The RDBMS faced huge problems supporting OLTP at global scale, since the strict ACID consistency model favored by all traditional RDBMS systems could not maintain global availability in the event of a network interruption. The initial wave of NoSQL operational databases adopted “eventual consistency” models rather than strict ACID consistency and generally discarded the SQL language in the process. These databases showed robust adoption, but were functionally deficient when compared to traditional database systems such as Oracle or SQL Server.
Google’s Spanner (www.dbta.com/Columns/Emerging-Technologies/Emerging-Technologies-Spanner-Stretches-the-CAP-Theorem-131092.aspx) attempted to plot a middle course. Spanner took advantage of Google’s highly redundant network to maintain very high availability while still maintaining strict consistency and the SQL language. Spanner also employs an elastic-cloud native architecture which allows it to scale resources to match workload demands.
In 2015, a trio of Google employees founded Cockroach Labs with the ambition of building a Spanner-inspired database system. The result was CockroachDB.
CockroachDB is designed to run on any hardware, so it can’t take advantage of the Google redundant network and atomic clock time synchronization which are important aspects of Google Spanner. But it does preserve the highly redundant and fault-tolerant distributed database architecture of Spanner and has strong support for ANSI SQL language and ACID transactions.
Furthermore, the CockroachDB architecture allows it to easily embrace containerized database deployments—arguably one of the next battlegrounds for database technology. The ability to deploy Cockroach clusters using Kubernetes allows for cloud-native deployments on any cloud or on-premise. And CockroachDB also provides a relatively new fully managed database-as-a-service offering.
In total, CockroachDB offers a highly scalable, transactionally consistent, SQL-compatible, cloud-native database platform. That’s quite a mouthful, but it is an attractive combination of features that provides a lot of what was promised by the NoSQL movement without the compromises often exacted by systems without true transactional or SQL language support.
CockroachDB has an impressive array of adopters, including SpaceX, DoorDash, and eBay.
We left the “one-size-fits-all” era for database management systems a long time ago, and there will continue to be a variety of database architectures to support specific workload demands. NoSQL platforms such as Cassandra and MongoDB can be expected to show continuing growth. But the idea that you can have what you liked from SQL and transactions while still getting what you need from a distributed database architecture is a compelling story. I wouldn’t be surprised to see CockroachDB mirror Snowflake’s success in an eventual IPO.