The push for cloud continues to be a relevant differentiator for enterprises looking to come out on top of their competitors. Whether the environment is multi-cloud, hybrid cloud, or any other configuration, it’s clear that data and analytics benefit from digital transformation’s promise of fast, easy insights.
Cloud experts joined DBTA’s roundtable webinar, Modernizing Your Data and Analytics in the Cloud, to discuss the ways a cloud-first strategy can successfully manifest to meet the data demands of modern business.
Maciej Szpakowski, co-founder of Prophecy.io, kicked off the conversation by explaining that since raw data is rarely suitable for immediate consumption, data transformations are the key to building AI- and analytics-ready data products.
On top of that, existing transformation options have significant shortcomings, including vendor lock-in, SQL limitations, non-native performance, a lack of support for DataOps, and more.
Szpakowski introduced Prophecy, the complete, low-code data transformation platform solution with native-to-cloud execution that spans a variety of data engineering areas—including data pipeline development, deployment, management, and orchestration—for faster data pipelines.
Prophecy’s low-code design, paired with its 100% open, git-commit code, ensures that its platform is both accessible and applicable to any workflow. Additionally, Prophecy empowers data standardization and reuse, as well as quick generative AI (genAI) application builds on unstructured enterprise data.
Paige Roberts, open source relations at OpenText, succinctly explained the benefits—and caveats—of going to cloud. While cloud offers significant elasticity suitable for dynamic analytic workloads, it comes at the risk of cost, where cloud implementations usually cost more than on-prem environments.
Fortunately for viewers, there are strategies to help mitigate this financial risk, according to Roberts. These include:
- Incremental cloud migration, where workloads are migrated one at a time and followed by a cost-benefit analysis after the workload has been moved
- Implement cloud repatriation if cloud costs for a certain workload exceed the budget
- Ask about egress fees—or fees that certain cloud providers or SaaS apps require you to pay in order to pull data out of their cloud—upfront when migrating data to the cloud
- Employ efficient software that implements guardrails for limiting autoscaling
As John de Saint Phalle, senior product manager at Precisely, put it, there are three trends influencing how companies are moving to the cloud:
- Financial responsibility for IT expenditures
- Data-centric cloud computing
- Democratized data
Despite these trends, de Saint Phalle explained that implementing a modern data integration framework isn’t easy. Factors such as real-time CDC, shortage of skills and staff, data accessibility, budget, data quality, legacy systems, data silos, and scalability can often prevent a successful data migration from coming to fruition.
Precisely targets these detrimental factors through its Data Integrity Suite, which breaks down data silos by quickly building modern data pipelines that drive innovation. The solution offers the following differentiators:
- Real-time data streaming for fresh, rapid data access
- Business-friendly UI for greater accessibility
- Build once, deploy anywhere principle for building and deploying data pipelines
- Over 50 years of domain expertise built into the Data Integration module
- Integration with the Data Integrity Suite Foundational for integrating metadata into other modules
Ryan Kearns, founding data scientist at Monte Carlo, narrowed down the perils of cloud data systems to data quality, where the business impact of poor data includes an increase in high severity events and wasted data engineering time often spent on fire drills, as well as significant revenue loss.
Kearns further argued that detection, resolution, and schema management workflows are the issue. Data quality incidents are often due to an inability to see the downstream, predict the ways data will break, or know when data is bad. Compounding this lack of data transparency is the fact that problems with data quality are identified reactively, resulting in significant data downtime.
Monte Carlo’s Data Observability platform drastically improves proprietary data quality, adhering to the five pillars of data observability:
- Freshness
- Volume
- Distribution
- Schema
- Lineage
Ultimately, with good data comes greater trust and adoption, Kearns explained. Monte Carlo’s platform enhances data productivity and accessibility, improves data quality, increases transparency, and enhances data adoption across teams through self-serve data reliability and greater data trust.
For an in-depth discussion of cloud-based strategies for modernizing data and analytics, you can view an archived version of the webinar here.