By Colleen Betik
Product Marketing Manager, Amazon RDS Open Source, AWS
In 2009, AWS pioneered fully-managed cloud database services for open-source database management systems (DBMS) with the launch of Amazon Relational Database Service, or Amazon RDS. The ambition was to accelerate the adoption of open-source databases, like MySQL and PostgreSQL, so developers can focus on building new applications while removing the worries associated with managing database instances. In parallel, AWS has fostered a close relationship with multiple open-source communities to accelerate the adoption of open-source databases while providing enhancements that developers want with over 435 accepted contributions from the AWS Databases team back to the open-source communities, year-to-date.
Amazon RDS has grown over the past decade to support seven managed database engines, covering everything from database operational features to security best practices. With Amazon RDS, it is simple to set up, operate, and scale databases in the cloud. Of its seven database engines, five of RDS’s engines are open-source or fully open-source compatible: Amazon Aurora PostgreSQL-Compatible Edition, Amazon Aurora MySQL-Compatible Edition, Amazon RDS for MySQL, Amazon RDS for MariaDB, and Amazon RDS for PostgreSQL. By providing support for the most popular open-source databases along with commercial databases, like Oracle and SQL Server, customers can migrate all their database workloads to the cloud with Amazon RDS, improving productivity and transforming the economics of databases ownership.
Developers love building with open-source databases due to their rich feature set, permissive licensing, and pace of innovation. AWS helps developers deploy their apps with these open-source databases to production by providing the latest capabilities for security, durability, availability, and reliability. This approach gives developers the best of both worlds—access to familiar open-source tools and code and freedom from undifferentiated database management tasks with the benefits from AWS continuous investment in innovations for performance at scale and operational excellence. A great example is when we announced Amazon RDS support for MariaDB 10.6. This release allowed MariaDB users to access recent innovations available to the broader MariaDB community, such as support for MyRocks storage engine, Oracle PL/SQL compatibility, Atomic DDL, and many other features. Simultaneously, developers who love MariaDB can enjoy AWS-led innovations like Amazon RDS Blue/Green Deployments, which automate an advanced DevOps operational technique and will be covered further later in this article.
The drive to provide developers with a resilient, highly-available database platform led to a number of new AWS-specific capabilities this year including Amazon RDS Multi-AZ (Availability Zone) with two readable standbys, which provides up to 2x faster transaction commit latency, automatic failovers in typically under 35 seconds, and additional read capacity. Amazon RDS also added support for up to a maximum of 155 read replicas with cascaded read replicas in RDS for PostgreSQL. This capability, already available on RDS for MySQL and RDS for MariaDB, provides users with three levels of cascaded read replicas. Developers can create Single-AZ or Multi-AZ with one standby cascaded read replica database instances in the same region or any one cross-Region from another read replica instance. Lastly, across MySQL, MariaDB, and PostgreSQL engines, Amazon RDS has enhanced read capacity with support of up to 15 read replicas per instance, including up to 5 cross region read replicas.
Fast forward to re:Invent 2022, where AWS announced four more new key capabilities that raise the bar for performance at scale and security. The first is Trusted Language Extensions for PostgreSQL, a new open-source project that makes it easier to quickly build and safely run PostgreSQL extensions. The second is Amazon RDS Blue/Green Deployments, a new feature that makes it safer, simpler, and faster to update databases in as fast as a minute. Finally, two performance enhancing features for RDS for MySQL including Amazon RDS Optimized Writes that provides up to 2x improvement in write transaction throughput and Amazon RDS Optimized Reads that provides up to 2x faster query processing.
Quickly and Safely Install PostgreSQL Extensions into Production
PostgreSQL has become one of the most popular relational database engines for developers, who love the extensive library of available extensions. It is well-known that building a PostgreSQL extension often requires that builders have a deep understanding of PostgreSQL and the C language. Given the advanced knowledge requirements, it is common that existing PostgreSQL extensions might have software defects that can crash the database and make it unavailable. As a result, in the past, managed services like Amazon Aurora and Amazon RDS, would only offer a selection of certified extensions to run on their databases. Many developers have requested the freedom to use any extension within a managed environment.
AWS launched Trusted Language Extensions (or TLE) for PostgreSQL, a new development kit and open-source project where PostgreSQL developers can now quickly build and deploy high performance extensions to production in managed services, like Amazon Aurora and Amazon RDS, using trusted programming languages, including JavaScript, PL/pgSQL, Perl, and SQL. TLE is designed to protect user databases by preventing access to unsafe resources and preventing extension defects from crashing the entire database. TLE also provides database administrators with fine-grained control over who can install an extension and can create a permissions model for running them. If developers find an existing extension meets their needs, they can now move ahead into production without waiting for AWS to certify the extension. TLE also enables Independent Software Vendors to share their extensions for Amazon Aurora and Amazon RDS with their customer base.
Best of all, TLE is an open-source project so anyone can use and contribute to the project using the official Trusted Language Extensions GitHub repo—all contributions are welcome!
Making Database Updates Faster, Safer, and Simpler with Zero Data Loss
Updating databases is often a complex process that can leave DevOps professionals and DBAs choosing between database availability and new benefits from updates, like security patches. Prior to re:Invent 2022, Amazon Aurora and Amazon RDS DevOps professionals and DBAs could update their databases using one of two methods. First, they could do an “in-place database update.” Using this method, they would simply overwrite older databases with a new version that has their desired change implemented. This method of updating risks the safety of their production database and can lead to long, unpredictable downtime. The second option is to use Amazon RDS database cloning and Amazon RDS Read Replicas to self-manage a staging environment and keep it up-to-date with the production environment. The desired changes would be performed on the self-managed staging environment, which will then be manually promoted to production. This method of making changes is costly to build and manage, requiring considerable orchestration of resources and careful planning.
AWS launched the “easy button” for database updates with Amazon RDS Blue/Green Deployments, a new enterprise-class feature that brings best-in-class database update processes to Amazon Aurora MySQL-Compatible Edition, Amazon RDS for MySQL, and Amazon RDS for MariaDB. With Blue/Green Deployments, DBAs and DevOps professionals can make various database updates, including major or minor version upgrades, schema changes, instance scaling, engine parameter changes, and maintenance updates with zero impact to their production workload. With just a few clicks, Blue/Green Deployments simply create a staging environment or “green environment” that is a copy of the current production environment or “blue environment,” including its primary instance, replicas, and enabled features. Blue/Green Deployments keeps them in sync using logical replication. With a single click, promote the staging environment as the new production in as fast as a minute with no application changes and zero data loss. During the duration of promotion, Blue/Green Deployments protect production workloads using switchover guardrails. These guardrails block writes to blue and green environments during switchover to ensure customers’ green environment is up-to-date with their blue environment before promotion. They also check for replication errors, assess instance health, detect long-running transactions, and times out your promotion if it exceeds the maximum tolerable downtime that you set. Blue/Green Deployments are available for Aurora MySQL-Compatible Edition versions 5.6 and higher, RDS for MySQL versions 5.7 and higher, and RDS for MariaDB versions 10.2 and higher.
Increasing Performance with up to 2x Improved Write Transaction Throughput in Amazon RDS for MySQL
When running a relational database, developers and DBAs expect a certain level of durability to protect their data. MySQL handles database writes by writing 16 kibibyte (KiB) in memory data pages to storage in four KiB chunks. However, if there is a system failure some of these chunks may not get written to storage and become corrupted. To protect from this, MySQL uses a “doublewrite buffer” so data is first written to the buffer and then to storage. This means, even in the event of a failure and the data in storage is corrupted, there will still be an intact copy of the written data in the buffer. While this protects MySQL users from data loss, it usually takes twice the amount of time versus writing once, requires twice as much IOPS bandwidth, and negatively affects database throughput and performance. If a workload has high concurrent transactions, developers may need to provision additional IOPS to meet performance requirements.
Amazon RDS Optimized Writes is a new feature that improves write transaction throughput by up to 2x at no additional cost. Optimized Writes protect customer data by writing 16 KiB pages to storage in a single atomic operation. To write the databases in a single step, Optimized Writes uses the recently announced Torn Write Prevention, a new feature of the next generation of EC2 instances, AWS Nitro System, to ensure the writes are safely written to table storage and protected from failure while writing. Optimized Writes is available for new RDS for MySQL databases using supported instances and versions 8.0.30 and higher.
Accelerating Query Processing by up to 2x Faster in Amazon RDS for MySQL and Amazon RDS for MariaDB
When processing queries, RDS for MySQL and RDS for MariaDB do this by reading from Amazon Elastic Block Store (EBS). While this process is efficient for most workloads, those that deal with complex queries, such as those that require grouping or sorting, require that RDS for MySQL and RDS for MariaDB generate temporary objects. When these objects don't fit into memory, they are moved to the disk storage. In the case of Amazon RDS, this means that temporary objects would be written-to and read-from EBS.
To speed up complex query processing on RDS for MySQL, Amazon RDS Optimized Reads offer up to 2x faster query processing. Optimized Reads support complex queries that use temporary tables, such as queries involving sorts, hash aggregations, high-load joins, and Common Table Expressions (CTEs). Rather than place temporary tables on EBS, Optimized Reads place complex queries’ temporary tables directly on the database instance’s local NVMe storage, yielding up to 2x faster query processing. Optimized Reads is available for RDS for MySQL versions 8.0.28 and higher and RDS for MariaDB versions 10.4.25, 10.5.16, 10.6.7, and higher.
AWS believes open-source is good for everyone. These announcements reflect their commitment to furthering the adoption and growth of open-source databases, including contributions back to the open-source communities. Additional information on all recently launched features can be found within the Amazon Aurora and Amazon RDS documentation pages. If you have questions, go to Contact Us and you'll hear back from AWS in one business day to discuss how AWS can help your organization.