MongoDB 3.6 is now Generally Available (GA), and ready for production deployment. In this short blog series, I’ll be taking you on a whirlwind tour of what’s new in this latest release:
- Today, we’ll take a look at the new capabilities designed specifically to help developers build apps faster. We’ll take a look at change streams, retryable writes, developer tools, and fully expressive array manipulation
- In part 2, we’ll dive into the world of DevOps and distributed systems management, exploring Ops Manager, schema governance, and compression
- Part 3 will cover what’s new for developers, data scientists, and business analysts with the new SQL-based Connector for BI, richer in-database analytics and aggregations, and the new recommended driver for R
- In our final part 4, we’ll look at all of the new goodness in our MongoDB Atlas fully managed database service available on AWS, Azure, and GCP, including cross-region replication for globally distributed clusters, auto-scaling, and more.
MongoDB Atlas is a cloud-hosted MongoDB service engineered and run by the same team that builds the database. It incorporates operational best practices we've learned from optimizing thousands of deployments across startups and the Fortune 100.
Developer-First
MongoDB has always been a developer-first technology. Its document data model maps naturally to objects in application code, making it simple for developers to learn and use. A document’s schema can be dynamically created and modified without downtime, making it fast to build and evolve applications. Native, idiomatic drivers are provided for 10+ languages – and the community has built dozens more – enabling ad-hoc queries, real-time aggregation and rich indexing to provide powerful programmatic ways to access and analyze data of any structure.
MongoDB 3.6 builds upon these core capabilities to allow developers to create rich apps and customer experiences, all with less code.
Change Streams
Change streams enable developers to build reactive, real-time, web, mobile, and IoT apps that can view, filter, and act on data changes as they occur in the database. Change streams enable seamless data movement across distributed database and application estates, making it simple to stream data changes and trigger actions wherever they are needed, using a fully reactive programming style.
Implemented as an API on top of MongoDB’s operation log (oplog), consumers can open change streams against collections and filter on relevant events using the $match, $project, and $redact aggregation pipeline stages. The application can register for notifications whenever a document or collection is modified, enabling downstream applications and consumers to act on new data in real time, without constantly querying the entire collection to identify changes. Applications can consume change streams directly, via a message queue, or through a backend service such as MongoDB Stitch (coming soon).
Use cases enabled by MongoDB change streams include:
- Powering trading applications that need to be updated in real time as stock prices rise and fall.
- Synchronizing updates across serverless and microservices architectures by triggering an API call when a document is inserted or modified. For example, new customer orders written to the database may automatically trigger functions to generate invoices and delivery schedules.
- Updating dashboards, analytics systems, and search engines as operational data changes.
- Creating powerful IoT data pipelines that can react whenever the state of physical objects change. For example, generating alarms whenever a connected vehicle moves outside of a geo-fenced area.
- Pushing new credit card transactions into machine learning training models to re-score fraud classifications.
- Refreshing scoreboards in multiplayer games.
Some MongoDB users requiring real-time notifications have built their own change data capture processes that “tail” the oplog. By migrating to change streams, these users can reduce development and operational overhead, improve usability, and increase data reliability. When compared to both oplog tailing and change notifications implemented by alternative databases, MongoDB change streams offer a number of advantages:
- Change streams are flexible – users can register to receive just the individual deltas from changes to a document, or receive a copy of the full document.
- Change streams are consistent – by utilizing a global logical clock, change streams ensure a total ordering of event notifications across shards. As a result, MongoDB guarantees the order of changes will be preserved, and can be safely processed by the consuming application in the order received from the stream.
- Change streams are secure – users are able to create change streams only on collections to which they have been granted read access.
- Change streams are reliable – notifications are only sent on majority committed write operations, and are durable when nodes or the network fails.
- Change streams are resumable – when nodes recover after a failure, change streams can be automatically resumed, assuming that the last event received by the application has not rolled off the oplog.
- Change streams are familiar – the API syntax takes advantage of the established MongoDB drivers and query language, and are independent of the underlying oplog format.
- Change streams are highly concurrent – up to 1,000 change streams can be opened against each MongoDB instance with minimal performance degradation.
Review the MongoDB change streams documentation to learn more.
Retryable Writes
The addition of retryable writes to MongoDB moves the complexity of handling temporary system failures from the application to the database. Now, rather than the developer having to implement custom, client-side code, the MongoDB driver can automatically retry writes in the event of transient network failures or a primary replica election, while the MongoDB server enforces exactly-once processing semantics.
By assigning a unique transaction identifier to each write operation, the driver re-sends that ID to enable the server to evaluate success of the previous write attempt, or retry the write operation as needed. This implementation of retryable writes offers a number of benefits over approaches taken by other databases:
- Retryable writes are not limited to idempotent operations only. They can also be applied to operations such as incrementing or decrementing a counter, or processing orders against stock inventory.
- Retryable writes are safe for operations that failed to acknowledge success back to the application due to timeout exceptions, for example due to a transient network failure.
- Retryable writes do not require developers to add any extra code to their applications, such as retry logic or savepoints.
Applications that cannot afford any loss of write availability, such as e-commerce applications, trading exchanges, and IoT sensor data ingestion, immediately benefit from retryable writes. When coupled with self-healing node recovery – typically within 2-seconds or less – MongoDB’s retryable writes enable developers to deliver always-on, global availability of write operations, without the risks of data loss and stale reads imposed by eventually consistent, multi-master systems.
Tunable Consistency
With tunable consistency, MongoDB affords developers precise control over routing queries across a distributed cluster, balancing data consistency guarantees with performance requirements. MongoDB 3.4 added linearizable reads, which were central to MongoDB passing Jepsen – some of the most stringent data safety and correctness tests in the database industry.
Now the MongoDB 3.6 release introduces support for causal consistency – guaranteeing that every read operation within a client session will always see the previous write operation, regardless of which replica is serving the request. By enforcing strict, causal ordering of operations within a session, causal consistency ensures every read is always logically consistent, enabling monotonic reads from a distributed system – guarantees that cannot be met by most multi-node databases.
Causal consistency allows developers to maintain the benefits of strict data consistency enforced by legacy single node relational databases, while modernizing their infrastructure to take advantage of the scalability and availability benefits of modern distributed data platforms.
Developer Tooling: MongoDB Compass
As the GUI for MongoDB, Compass has become an indispensable tool for developers and DBAs, enabling graphical schema discovery and query optimization. Compass now offers several new features:
- Auto-complete: Enables developers to simplify query development with Compass providing suggestions for field names and MongoDB operators, in addition to matching braces and quotes as they code.
- Query History: Allows developers to re-run their most recently executed queries, and save common queries to run on-demand.
- Table View: Now developers can view documents as conventional tables, as well as JSON documents.
MongoDB Compass is not just a single tool – it’s a framework built to allow for the addition of modular components. Compass now exposes this as the Compass Plugin Framework, making Compass extensible by any user with the same methods used by MongoDB’s software engineers. Using the plugin API, users can build plugins to add new features to Compass. Examples include a GridFS viewer, a sample data generator, a hardware stats viewer, a log collector/analyzer, and more.
You can learn more about these new features in the MongoDB Compass documentation.
MongoDB Compass Community
With the MongoDB 3.6 release, the Compass family has expanded to now include the new, no-cost Compass Community edition.
Compass Community provides developers an intuitive visual interface to use alongside the MongoDB shell. It includes the core features of Compass, enabling users to review the hierarchy and size of databases and collections, inspect documents, and insert / update / delete documents. Developers can use the GUI to build queries, examine how they’re executed, and add or drop indexes to improve performance. Compass Community also supports the latest Compass functionality available with MongoDB 3.6, making developers even more productive.=
Fully Expressive Array Updates
Arrays are a powerful construct in MongoDB’s document data model, allowing developers to represent complex objects in a single document that can be efficiently retrieved in one call to the database. Before MongoDB 3.6, however, it was only possible to atomically update the first matching array element in a single update command.
With fully expressive array updates, developers can now perform complex array manipulations against matching elements of an array – including elements embedded in nested arrays – all in a single atomic update operation. MongoDB 3.6 adds a new arrayFilters option, allowing the update to specify which elements to modify in the array field. This enhancement allows even more flexibility in data modeling. It also delivers higher performance than alternative databases supporting JSON data as entire documents do not need to be rewritten when only selective array elements are updated.
Learn more from the array update documentation.
Next Steps
That wraps up the first part of our what’s new blog series. Remember, if you want to get the detail now on everything the new release offers, download the Guide to what’s New in MongoDB 3.6. Alternatively, if you’d had enough of reading about it and want to get started now, then:
- Spin up MongoDB 3.6 on the MongoDB Atlas database service.
- Sign up for our free 3.6 training from the MongoDB University.
- Read part 2 where we’ll dive into the world of DevOps and distributed systems management, exploring Ops Manager, schema governance, and compression