Global provider of modern data integration, Fivetran, is debuting the availability of its Metadata API for advanced data tracking within Fivetran-managed pipelines. Data access control is key to Fivetran’s latest API, tracking in-flight data from source to target to provide deep insights regarding governance and observability tool employment—allowing users to know who is accessing what data.
Fivetran’s enterprise ambition highlights modernizing organizations’ data stacks through their data movement tool. Primarily moving data to data lakes and data warehouses, Fivetran aims to help analytics and operational teams aggregate information to make more data-driven decisions. Streamlining and self-service are key values for the enterprise’s business ventures, enabling customers adopting the cloud to easily access the information they need, when they need it, according to the company. With automated management from the enterprise’s tool, Fivetran enables its customers to “set it and forget it,” according to the vendor; needs for manual transformation set-up or pipeline modification no longer burden users employing Fivetran’s platform.
Customers began to ask the enterprise for a few vital features to be integrated with the Fivetran platform: being able to control who has access to what data within the platform, as well as understanding the exact functions occurring within it to appropriately take action on it.
Fivetran’s Metadata API is a direct response to these user requests, using data lineage and governance automation as the north star for the API’s directive without overhauling an enterprise’s pre-existing governance tools.
“One of the key things that we hear from our customers is that governance is a bad word. It’s complicated, it’s heavy—it has a very high failure rate in most companies. When we did research into our customers, one of the things we were thinking about is, ‘should we expose this information within Fivetran so people can action on it?’ We realized a lot of people already had these tools and processes they have set up already; they wanted Fivetran to integrate into what they have existing,” said Meera Viswanathan, senior product manager specializing in data governance at Fivetran. “And that’s one of the big reasons we went with an API-first approach. The goal is to provide that visibility, so you are not tied to a specific tool; Fivetran is a partner to your governance organization to make sure you are compliant.”
Targeted towards data analysts and stewards, security teams, and data engineering teams of large enterprises with complex data or rigid governance requirements, Fivetran’s Metadata API is critical toward advanced data visibility. By shedding light on in-flight data, the API aids data analysts to gain insights towards data sources and run impact analyses on it. Data stewards, who are responsible for ensuring data security and adherence to governance requirements, will be able to effectively do so with Fivetran’s Metadata API. Security audits experience an efficiency boost with the API, assisting security and legal teams to complete audits and validate data movement compliance with governance regulations. Further, insights towards upstream schema changes for data engineering teams enables accurate and streamlined downstream process updates.
Fivetran’s API is also deeply integrated with their partners, including Atlan, data.world, Alation, and Collibra, via product integrations built for Fivetran’s platform. Users employing those tools from their respective Fivetran partners can directly plug-and-play within those platforms, immediately receiving necessary data.
“Every enterprise knows it must be data-driven, but traditional data governance has been a barrier with manual processes and reactive enforcement of policies. That's not a scalable approach, especially as data infrastructure grows to thousands of pipelines,” said Fraser Harris, VP of product at Fivetran. “With Metadata API, our customers get out-of-the box data governance automations and data quality workflows so they can proactively identify and take action on governance issues before they become a problem. Our automated in-flight approach enables data access at scale without increasing risk to the business.”
Continued improvements to self-service, security, and governance are in store in the future. Standardizing data stacks, adapting to given skill sets, as well as increasing and efficient automation services guide the enterprise’s efforts towards embracing data-driven decision-making through democratizing data environments.
To learn more about Fivetran’s Metadata API, please visit https://www.fivetran.com/.