Limitations in scalability and the business's demand for analytics are causing IT departments to rethink the traditional data warehouse/data mart strategy in favor of a powerful, centralized business analytics information grid.
Few things in the world are changing as dramatically as data. Data has tapped out a powerful rhythm to keep time with technology's bleeding edge, leaving many technologies struggling to keep up. It should come as no surprise, then, that many of the data strategies that IT departments developed, and still widely rely upon, are no longer sufficient for today's needs. You can put data marts near the top of that list.
Data marts were a reaction to the extreme performance limitations of traditional enterprise data warehouses. The data warehouse itself, which came of age in the 1990s, represented a tremendously enticing vision - offering virtually every department across the enterprise an opportunity to see the metrics of its performance and answer questions about what's working and why.
That is, data warehouses would have answered all of those questions, if only users could get to the data. Most organizations quickly discovered that data warehouses-with their centralized, brittle architecture-performed abysmally under unpredictable workload demands. Even the load of just a few users could degrade performance precipitously. It quickly became clear that if they wanted to scale the data warehouse, organizations would need to replicate and distribute the data locally. Thus, data marts were deployed.
The Power of Prediction
While data marts were never a perfect solution, they adequately addressed businesses' urgent need to let stakeholders from across the organization explore the data and uncover the insights they hold. But while data mart deployments have largely continued unabated for the past decade, business has changed dramatically: Global competition, mobility, social media, and the accelerating pace of business are forcing enterprises to re-evaluate how they think about data.
In this fast-paced business climate, it's no longer enough to use the data warehouse to find out what happened in the past; today's businesses need real-time data-data capable of making credible predictions about what will happen in the future. Business leaders are looking for ways to gain deeper insights from data, to enable more business users to search for these deep insights and to directly embed these insights into core business processes.
Such predictive analytics can have a tremendously uplifting effect on business, especially when they can be embedded into the workflows and applications that power key business processes. For example, analytics can be used on the fly to determine the likelihood of fraud for any transaction, identify cross-sell opportunities, or to single out particularly influential customers. Imagine the power of being able to alert call center operators or branch agents to such conditions at the earliest moments in the customer contact. AOK Hessen saved $3.2 million by using predictive analytics to identify fraudulent insurance claims. HMV Japan used it to better predict the interests of its customers and increased per-transaction revenue by 300% as a result.
Intelligence for Everyone
Thus, at many organizations today, IT is under mounting pressure to abandon these wallflowers - traditional data warehouses (and data marts) - in favor of a quick-footed modernized architecture that:
- is able to answer complex questions using massive volumes of data
- can scale massively to support the analytics needs of all enterprise users
- can embed advanced analytical models into end processes to help increase revenue and limit risk.
These new business demands are driving recognition of a number of critical technology challenges:
1. Big data - Today's businesses are deluged with a massive volume of data, created in part by the recognition that all the data available to an enterprise can be analyzed.
2. Data type diversity - There are dozens of structured and unstructured data types that must be included in the data warehousing effort, including numeric, text, audio, video, high-resolution imagery, SMS, RFID, clickstream, and more.
3. Complex questions - The requirement for in-depth knowledge discovery means the solution must be capable of recognizing and adapting to data anomalies, recognizing data clusters and trends, identifying influencing factors and making reliable assumptions.
4. Decision velocity - Enterprises are looking to make decisions in seconds and minutes, and not days or weeks. The solution must be able to answer user questions at the speed of thought, and in some cases remove the user from the equation entirely.
5. Many users - Today's business analytics environment must support decision-making at all levels of the organization-tactical, operational, and strategic. Furthermore, enterprises are increasingly looking to incorporate analytics directly into business operations.
While each is undoubtedly a critical requirement of the new architecture, the fifth challenge, servicing many users, is perhaps the one that will most definitively set apart successful solutions from those that fail to live up to their potential. After all, even the most insightful conclusions are of limited value if the data isn't seen by the right people at the right time. For this reason, the next frontier in analytics is delivering intelligence for everyone.
The secret to delivering business analytics to the whole organization is to harness smart parallelization techniques. Whereas traditional data warehouses use a shared-nothing architecture, forcing users to wait in queues while resources are locked by other queries, a high-performance business analytics information grid will instead employ a shared-everything architecture. This will make it possible to:
- Share resources, making all data accessible to any server or a group of servers, allowing many simultaneous users with diverse workloads
- Scale out independently and heterogeneously across resources with our without private clouds
- Provide a "self-service" methodology that supports data analysis from anywhere, including from specialized applications, through the web or on mobile devices.
The Death of the Data Mart
Building an enterprise data warehouse is generally viewed as a long-term investment. And, yet, traditional solutions have proved to be surprisingly brittle-inadequate for the business needs of tomorrow and unable to learn the steps to data's new groove. Tomorrow's massively scalable grid-style architectures provide an opportunity to create truly flexible and predictive business analytics while solving the very problem data marts were invented to address in the first place: a central place for all business users to access and analyze all enterprise data.
An analytics-optimized information grid is the right dance partner for today's data. It will not only usurp the departmental data mart but it will take inflexible, flat-footed data warehouses of the past along with it.
About the author:
David Jonker is the product marketing director for Sybase IQ.