Airbyte, the provider of an open-source data integration platform, is unveiling its Free Connector Program which grants accessibility to any data pipeline on Airbyte Cloud that uses alpha or beta connectors. Committed to making data portable and fluid, Airbyte’s latest program continues to make data available and reliable in the face of increased commodification of data movement and integration.
The Free Connector Program provides free-usage of alpha and beta connectors to improve their quality; increased usage of these “long-tail” connectors, or connectors that are not widely implemented, allows Airbyte to continuously enhance them and increase their value.
“The Free Connector Program completes the flywheel we put in place to commoditize data integration,” said John Lafleur, co-founder and chief operating officer at Airbyte. “What we were missing is significant usage across all our long-tail connectors—those less used—in order to help identify the edge cases we don’t fully support yet.”
Most often, long-tail connectors are not often supported by close-source ETL (extract, load, transform) technologies. This is further compounded with the fact that many organizations are forced to engage in custom work around pre-built connectors to make them accommodate their unique data infrastructure. By providing free usage of alpha and beta connectors, Airbyte can regularly improve its reliability, further graduating its quality.
“You need to have as much usage as possible on the connector to detect edge cases, to detect specificity in the source. So, the more users you have, the more reliable your connector becomes,” explained Michel Tricot, co-founder and CEO at Airbyte. “The free alpha-beta connectors allow us to have that usage and to discover all of its problems much faster, fix them, and involve the community on how we fix them.”
While these alpha and beta connectors may not work for everything an enterprise needs—with beta connectors averaging a 93% sync success rate and alpha connectors a 90% sync success rate—its usage pays forward its quality and reliability while existing at no cost to its implementers. Airbyte only charges for usage of its platform with data pipelines leveraging generally available connectors.
“In a way, it's being more honest with our users and setting the right expectations,” said Tricot.
Maintaining connectors is a particularly demanding and resource exhaustive endeavor; for most companies, cost of maintenance is extremely high. Airbyte’s open source community allows the company to scale to a large quantity of connectors while still being able to comprehensively support them. Airbyte has the largest data engineering contributor community with more than 600 contributors, according to the enterprise.
“Any company that starts to use Airbyte will depend on that connector to be reliable. And if one day breaks—and they will always break because they pull data from systems that people don't control—they will have now an incentive to actually go fix the connector, and that will automatically propagate to the rest of the community,” explained Tricot.
While the majority of enterprises plateau at about 150 connectors, Airbyte aims to make 1,000 data connectors available, currently offering around 300 connectors. If any source or destination connector is alpha or beta, the entire data pipeline is free. You can view the complete list of connectors here.
Connectors can also be built by users if not currently offered by Airbyte using the Connector Developer Kit, which generates 75% of the code required, as well as offers templates in Java and Python.
For more information about Airbyte’s Free Connector Program, please visit https://airbyte.com/.