A new survey of 421 data managers and professionals affiliated with the Independent Oracle Users Group (IOUG) members finds that while most companies have well-established data warehouse systems, adoption is still limited within their organizations. Many respondents report a significant surge of data within their data warehouses in recent times, fueled not only by growing volumes of transaction data but unstructured data as well. Now, the challenge is to find ways to extend data analysis capabilities to additional business areas.
Companies recognize the analytical power that a data warehouse foundation provides. Data warehouses serve as the foundation for business intelligence and analytics applications across 66% of the companies in the survey, underwritten by Oracle Corporation and conducted by Unisphere Research, a division of Information Today, Inc. and publisher of Database Trends and Applications. However, in many cases, the survey finds that these data warehouses remain isolated from mainstream business operations. Most are in-house, customized systems, mainly used by analysts and top decision makers. Only 33% of respondents' companies enable access to their marketing and sales departments-one of the most critical areas where data warehouse value is realized. As one respondent observed, her organization "is not fully leveraging the capabilities of data warehousing. However, there is a high expectation, and integration insights gained are wonderful. We're looking forward to including more business intelligence analytical tools."
About one-third of respondents report that they made significant data warehouse investments in 2010-2011 but are uncertain about investments in 2012. Sixty-two percent of respondents report that their budgets have grown, in many cases, significantly. Thirty percent, in fact, report budget increases exceeding 25%. However, as of mid-2011 - the time when this survey was conducted-plans for data warehouse budgets in the future were uncertain. Only 36% plan budget increases, and only a handful (3%) indicate that growth will exceed 25% over this year.
At the same time, data warehousing is undergoing its most radical transformation since it was first conceived of in the 1970s and brought to market in the late 1980s and 1990s. Not only do companies need to manage and store hundreds of terabytes to petabytes of data, they also need to find ways to extract the maximum value from this torrent of increasingly complex and diverse information - including unstructured or schema-less data-that is being generated at an escalating rate. (See Figure 1.)
Close to 90% of respondents say the volume of data in their warehouses has increased over the past year, and they expect this data growth to continue. Close to 50%, however, are not certain if their data warehouses will be able to scale to meet future big data requirements. (See Figure 2.) And, while companies are conservative in their plans for data warehouse upgrades, there is considerable movement toward implementation of ready-to-run solutions, including appliances and cloud-based offerings. (See Figure 3.)
Respondents expect a range of business and technical benefits resulting from their data warehouse implementations. From a business perspective, close to half expect improved internal operations, lower costs, greater insights for decision makers, and a better ability to meet reporting or compliance mandates. Technically, 66% of respondents anticipate better performance, and a majority of respondents expect enhanced scalability. In an age of efficiency and cost cutting, many organizations are looking at ways to consolidate their data center operations. One way to do this is to consolidate online transaction processing (OLTP) and analysis (OLAP) applications onto a single platform. In fact, many respondents are already doing this, with 40% of respondents claiming that some, if not all, instances of OLTP and OLAP can be found on the same server. In addition, 37% will consider consolidating OLTP and OLAP applications on the same server in the future.
The executive summary of "A New Dimension to Data Warehousing: 2011 IOUG Data Warehousing Survey" is available from the IOUG.