Despite efforts to “democratize” business intelligence, it has remained stubbornly confined to a chosen few within organizations. Although vendors have worked hard to convince enterprises that their BI solutions could be extended to line-of-business managers and employees, high-end analytic tools have remained confined to power users or analysts with statistical skills, while the remainder of the organization relies on spreadsheets to cobble together limited pieces of information.
This disconnect was confirmed in a 2007 survey conducted by Unisphere Research for the Oracle Applications Users Group, which found that most companies are still a long way off from the ideal of BI for all. The OAUG survey found that for the most part, BI reporting remains tied up in IT departments, and is still limited to analysts or certain decision-makers. The majority of survey respondents said that it takes more than three to five days to get a report out of IT. Overall, the survey found, fewer than 10 percent of employees have access to BI and corporate performance management tools.
“People want the ability to ask questions of their data and get answers, and they are disappointed in the status quo,” according to Chris Stolte, vice president, engineering and co-founder for Tableau Software.
Now, a new generation of technologies - driven by easier-to-use tools, and the collaboration and openness of the so-called Web 2.0 solutions - promises to dramatically change the BI landscape, and finally open up analytical capabilities to the enterprise at large. “Advancements in analytical capabilities of some BI solutions mean that deriving intelligence from numbers will not continue to be the exclusive domain of those trained in quantitative and statistical analyses,” Shadan Malik, president and CEO of iDashboards, told DBTA. “Add to this the key role that enhanced data visualization will play in expanded downstream adoption of BI and savvy business users will increasingly be taking advantage of the capabilities of BI.”
Forces of Change
There are several forces at work in the BI market that are making this possible. Traditionally, BI data and analysis was only available through proprietary tools or ERP reporting systems - all siloed applications. Now, companies are finding much of the key data they need may be outside of their firewalls - coming in from various end-user devices, or in online customer conversations, for example. Don Tapscott, CEO of New Paradigm and co-author of Wikinomics: How Mass Collaboration Changes Everything, calls this a transformation from "business intelligence" to "collective intelligence."
The social aspects of Web 2.0, such as blogs, podcasts and wikis, “are shaping the way consumers access information,” said Chris Ferrara, vice president of business intelligence for ISA Consulting. The greatest challenge for the year ahead “will be in creating a seamless process that makes the consumption of BI a more natural part of the end-user’s workflow. Integrating increasingly diverse sets of data into a cohesive BI framework will be a struggle for many companies in 2008.”
This calls for tools that can help decision-makers make sense out of the masses of data that are flooding in from all directions. The growth in data is accelerating at a tremendous rate. As Gaurav Verma, senior BI strategist with SAS Institute, pointed out, digital data is now estimated to be doubling every 11 months - and this will accelerate to doubling every few days by 2010. This is directly due to Web 2.0 approaches, Verma told DBTA, adding that “the increased numbers of knowledge workers throughout organizations means more people are seeking access to more data than ever before. The need to present that data to workers lacking specialized data extraction skills has never been greater.”
“Few people now have the time and/or patience to learn the more complex BI applications; why should they, when all the bright and shiny Web 2.0 applications are out there, waiting to be adopted?” asked Nobby Akiha, senior vice president of marketing for Actuate. “If something can be done more easily and with a lower investment it’s a no-brainer; people will take adopt the tools that are more readily accessed, easier to use, and offer the fastest ROI.”
"Business users understand the evolution in software that is being coined ‘Web 2.0’ - it is about simple and useful applications that are centered on the users and communities,” said Stolte. "This is a stark contrast to traditional BI, which has locked away data with a few specialists and provided only complex tools designed around technology.”
Overcoming Legacy
However, shifting the emphasis in BI away from locked-in proprietary systems to open Web 2.0 networks will not be an easy process. Current systems may hold back such efforts, cautioned Dan Graham, senior marketing manager for Teradata’s Active Data Warehouse Program. “For over a decade, companies have sought to unify their consumer interaction using a single profile, analysis, and communication, ” he told DBTA. “With the consumer shift towards the Web and cell phones as primary channels, older architectures for automated selling are failing to keep up.”
Graham advocated retiring the “operational data store-per-touchpoint architectures that sprouted in the 1990s,” and replacing them with “fresh data and real time access to the enterprise data warehouse. Today, visionary enterprises are also experimenting with Web 2.0 touch points such as online social communities, forums, chat, and wikis.”
Where Web 2.0 Will Make a Difference
What’s the most practical way to leverage Web 2.0 capabilities into BI? Michael Mediterraneo, president of Cambridge Integrated Services, recommended “real-time portals with collaboration features.” Already, he told DBTA, he sees companies “helping to provide the Web 2.0 vision through today's technologies to meet the business needs brought on by the real-time business world.”
Ferrara speculated that in this brave new world of BI, “the word ‘user’ may well be an antiquated term because Web 2.0 technology strives for participation. It is this participant culture that has the potential to shape BI in 2008 and beyond.” Colin Shearer, senior vice president of market strategy for SPSS, agreed that “the emergence of Web 2.0 data sources - such as, blogs, wikis and other social networks - as well as information derived from call center notes and conversations, makes it even more crucial for organizations to better understand their customers.” Shearer noted that “by including these valuable opinions, preferences and attitudes of customers collected through both of these new sources and with traditional data sources, organizations can better understand customers, predict their actions and improve and optimize business processes.”
However, don’t expect to see the current phase of Web 2.0 - the blogs, wikis, and AJAX clients - simply applied directly to BI. Rather, the impact will be felt on the back end, which will be enriched by simpler, more collaborative, and easier-to-user interfaces. “We will begin to see a rapid evolution where Web 2.0 philosophies will start to play a role in BI, but not in the manner they have thus far,” Stolte explained. “Rather than arbitrary deployments of technology such as tagging, wikis and AJAX, we will see tools starting to become more simple and focus on the end user and his or her key task: to understand and work with data.”
For example, Stolte continued, “rolling out simple, easy-to-use analysis tools throughout organizations will create cultures of curiosity and introspection and help organizations get real value from their data. This simplification and enrichment of applications will overlap with the rapidly emerging fast database market, which is making ad hoc query viable on almost all data set sizes.”
Such an impact will be felt with the emerging field of text analytics, which leverages the multiple forms of data feeding in from multiple sources. For example, Dr. David Bean, chief technology officer of Attensity Corp., pointed out that “customers place calls, send emails, complete surveys, and talk among themselves online in blogs and product forums. Customers also tell companies about product failure, request help, and offer opinions that may contain valuable insights for organizations that care to listen.” This is all data that can be captured and analyzed to improve products and services.
However, he cautioned, this is all unstructured or non-traditional data beyond the reach of many current BI systems. “The vast majority of this customer feedback is unstructured data that may be easy for people to read but nearly impossible for databases to understand,” he told DBTA. "Moreover, traditional channels for feedback are being joined - and often superceded by - new channels such as blogs, forums, wikis, and other web-enabled spaces not authorized, organized, controlled, or often even monitored by the company.”
Thus, this year, expect to see greater investments in BI software applications “to explore how best to capture and analyze unstructured information seamlessly with structured data,” Bean said. “Doing so will enable companies to connect the reason why a customer gives a high or low score, with a particular product, customer segment, or even an individual customer identified by structured fields such as customer identifiers, product SKUs, scalar feedback scores, and assigned codes.”
Scott Spangler, a researcher with IBM Almaden Research Center, also sees a growing role for such nontraditional data in BI. This year will see an “increased interest and focus on mining sources of data that exists outside of an organization or data that seemingly does not exist but can quickly be generated internally from ad hoc communities,” he told DBTA. “Companies are finding this data, such as blogs or message boards that talk about a company or its products, to be a rich source of direct customer feedback. This data can be analyzed to point out important trends or issues that an organization should be aware of and act on quickly.” This requires innovative, new approaches to text based analysis, Spangler continued, such as a "discovery-based approach” employing new search technologies.
As a result, when companies connect structured data with the “why,” Bean explained, “they have their first real opportunity to see a complete view of their customers. The power of this untapped reservoir of mission-critical information will drive an organization’s strategy tied to products, services, markets, communications, and employees. Organizations that take on this challenge will successfully gain access to finer details, deeper insights, and additional opportunities about their customers and products.”