Modern data warehousing is not only being shaped by the need for businesses to deliver data faster to more users, but the need for a richer picture of their operations afforded by a greater variety of data for analysis.
A growing number of organizations are modifying their data warehouse infrastructures with new technologies, from in-memory databases to Hadoop – and a flourishing market of cloud solutions.
To accommodate the different types of data sources, workloads, applications, and users that big data presents us with, a variety of systems are needed. And to avoid data silos, these technologies need to be integrated under a common architecture.
DBTA recently held a webinar with Kevin Petrie, senior director and technology evangelist at Attunity, and Danil Zburivsky, director of big data and data science at Pythian, who discussed key challenges and critical success factors in data warehousing today.
Decision engines are changing, according to Petrie, and today’s challenges break traditional tools. Hand-coded ETL ties up programmers, processes vary by end point, copies disrupt production, data usage a “black box,” and data unready for analytics.
There’s a new way to create modern, efficient data pipelines, Petrie explained. Attunity Enterprise Manager can offer intelligent management and control.
The platform can deliver more data to the business, enable agile analytics, and reduce labor and cost.
The drive to make better use of data means data warehouses are under pressure by several major forces, Zburivsky explained.
All of this pressure on a system that wasn’t designed to meet all these needs has resulted in lost opportunities for the business, unhappy users, performance problems, and rising costs.
This is where the modern data platform comes in. The modern data platform is a single, unified platform that is capable of carrying out high-performance analytics on both relational and nonrelational data, supporting both traditional and exploratory use cases.
A well-designed data platform:
- Offers both a data lake and a data warehouse
- Separates storage from compute
- Is designed for easy management
- Enables you to optimize ongoing operational costs
- Is modular for future agility
Old BI architectures will not work efficiently if ported as-is - a different approach is needed.
An archived on-demand replay of this webinar is available here.