Airbyte, the creators behind a fast-growing open source data integration platform, is announcing the launch of three new features—checkpointing, column selection, and schema propagation. These new capabilities are designed to improve Airbyte’s resource usage, efficiency, and reliability while simultaneously enhancing flexibility when addressing compliance requirements.
The addition of checkpointing, or a data processing method that allows users to recover from interruptions, generates a save-state of data streams continuously, largely improving data reliability within the Airbyte platform. In the event of failure, data loss, or redundant processing, checkpointing enables users to restore from the most recent checkpoint, mitigating the potential damage.
Airbyte’s column section feature brings fine-grained replication to the forefront by enabling column-specific selection from a source for replication. In turn, users are allotted more agility when deciding what data requires replication the most, inherently optimizing resource utilization and overall efficiency. Column selection also introduces an opportunity for enhanced security, where users can opt to exclude sensitive data from replication for better adherence to privacy regulations.
The final capability introduced to Airbyte, schema propagation, alleviates the pains of manual change implementation. Instead of integrating changes from source schema to destination schema by hand, Airbyte propagates these changes automatically, improving usability while mitigating administrative overhead. Users can also maintain control over how schema propagation is done for each connection, according to the vendor.
“We’re getting closer to making Airbyte best-in-class with this significant release while furthering our mission to commoditize data integration,” said Michel Tricot, co-founder and CEO, Airbyte. “At the same time, we’re reducing resource requirements and giving users of both Airbyte Open Source and Airbyte Cloud more flexibility in how they comply with their internal or regulatory requirements. And there are even more updates coming in the weeks ahead.”
Checkpointing, column selection, and schema propagation are now available within Airbyte Cloud and Airbyte Open Source.
To learn more about Airbyte’s latest capabilities, please visit https://airbyte.com/.