Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Information Is Your Business
   Information Is Your Business Advanced Search

Business Intelligence
Corporate Performance Management
Data Governance
Data Integration
Data Quality
Data Warehousing Basics
Enterprise Decision Management
Enterprise Information Integration
Master Data Management
View all Portals

Scheduled Events
Software Demo Series

White Paper Library
Research Papers



DM Review Home
Current Magazine Issue
Magazine Archives
DM Review Extended Edition
Online Columnists
Ask the Experts
Industry News
Search DM Review

Tech Evaluation Center:
Evaluate IT solutions
Buyer's Guide
Industry Events Calendar
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Data Strategy Adviser:
Making Sense Out of the Business Intelligence Market

online columnist Lou Agosta     Column published in DMReview.com
February 15, 2007
  By Lou Agosta

Philip Howard of Bloor Research recently laid down a challenge in an engaging, short article published January 18, 2007 entitled "Second Generation BI." Philip calls for a holistic approach to the next generation of business intelligence (BI).1 The premise of Philip's argument is that we are unable to see the forest of business value for the trees of tools and point products. Philip acknowledges that he does not have an answer to the dilemma. If anything he is a voice crying in the wilderness - make ready the way of next generation BI! At the risk of mixing the metaphor, he invites suggestions as to how to find a way out of the BI jungle, providing only one clue - it might have something to do with architecture. Here is my answer. In one word, the answer is - innovation.2

The messy and dynamic BI market confronting companies in 2007 is a function of dozens of trends. The top three are 1) innovations in the underlying system capabilities such as data warehousing appliances, 2) innovations in integrating the business activity with information technology even though data integration is still incomplete, 3) innovations in surfacing business value through breakthroughs in usability and user-interface design.

In spite of the seemingly random and borderline-chaotic market dynamics, BI tools and technologies are easy to categorize. This can be done by means of an architecture that distinguishes front-end, middle and back-end functions. Process-centric solutions such as business process monitoring (BPM) cut across all these layers and add an essential time dimension to the information supply chain connecting transactional data to BI decision-making. The information supply chain is rendered dynamic by inclusion of functions of transformation such as extract, transform and load (ETL), message brokering or on-the-fly, interactive enterprise information integration. What's really new here? In next generation BI, the latter infrastructure of information transformation traverses architectural layers and will be organized as part of an enterprise services bus using SOA.3

This is a straightforward partitioning of the problem. It is a basic principle of architecture that business value migrates in the direction of the user interface. This gathers traditional query and reporting tools together in a competitive landscape with next generation, metadata-enabled search technologies as well as data visualization and in-line analytics to deliver the business analyst's "Ah ha!" moment.

At the back end, data warehousing appliances have traction and will continue the march up market. Enterprises have seen the future of data management a it requires simplification, high performance and business value add. This is precisely the formula giving open, interoperable data warehousing appliances the market traction they are currently enjoying. Early entrants with proprietary hooks and legacy appliances will give way to standard, general-purpose solutions from the established relational database vendors. Going forward, clients likely will need only one kind of database to perform transactional, content-centric (e.g., XML) and business intelligence workloads, admittedly on different database instances. What gets this architecture to the next generation is remixing it with an enterprise services bus (ESB) by means of service-oriented architecture (SOA). It is not enough simply to connect everything to the ESB, though that is surely useful and necessary. In order to succeed, SOA must be accompanied by implementations of metadata to rationalize the semantics (the meaning and context) of the diverse, heterogeneous data stores that are being connected, compared and managed. This is a work in progress, and the future belongs to those vendors that execute on the roadmap.

One popular megatrend - BI for the masses - will not survive holistic thinking and will not survive the deployment of next generation business intelligence. So be ready for surprises. Simply stated, the masses are not what they used to be. Instead, a diversity of different kinds of users - with different requirements and jobs to do - will make it clear that a next generation interface is required to deliver on the transformation of raw data into enterprise intelligence. This is the result of a profound realization that generating intelligent information integration at the business level is not a user interface function at all. The real work happens elsewhere - upstream.

The information supply chain will be rendered hyper-responsive to client requirements by means of SOA. Of course, this includes the interface where the value is delivered. So a dynamic interface will be able to morph between different classes of users based on profiles, personalization and preferences. Different classes of users will deploy a dynamic, intelligent interface to accommodate their different requirements. Executives will favor scorecards and dashboards, power users will leverage their data cubes and OLAP in spite of their latency, specialists will require advanced analytics for predictive analysis, clerk administrators will value standard predefined reporting and all users will benefit from proactive notification and alerts.

The problem of the proliferation of tools will fix itself as the consolidation of the software market continues. Such proliferation is frequently described by economists as the "outsourcing of innovation" and, as industry analysts have pointed out, the process is not pretty and the result can be disruptive and discomforting to established vendors. This is good and keeps everyone on their toes. Of course, the business risk of orphaned products should be an important part of every product purchase decision. Some large enterprises will decide it is worth the risk and end up increasing their total cost of ownership over the long term. This represents a limit to the holistic approach.

No enterprise that I know of has the luxury of "blowing up" its existing BI systems and leaping to the next generation in one giant step. But even if it did, the next generation BI would be distinguished by features that sound like a wish list in terms of today's technology and business practices. That is how a holistic envisioning of the future shows up. At the front end, personalization will accommodate different classes of users by means of a dynamic interface that is able to switch context based on user profile and metadata that reaches back to authoritative data sources; in the middle layer, semantic consistency and dynamic metadata will enable intelligent information integration; at the back end, autonomic computing will facilitate flexibility, administration, maintenance, a low total cost of ownership of data regardless of where it is "persisted" (data warehoused); and throughout the ESB will deploy high performance networks and grid computing will to deliver low latency and near real time answers and updates for business tasks, operational decision making, and basic business intelligence.

Going beyond next generation BI to "blue sky" BI - that is, BI beyond our wildest imaginings - something like a "semantic chip" will be required - even if that is only a metaphor for profound change in possibilities - to reach the holistic BI invoked by analysts such as Philip Howard.4 Such a chip, even if it only a cipher for change, would be the singularity required to reconcile the inconsistent, heterogeneous data needed for a holistic view of enterprise BI. No number of database administrators (DBAs) will be able to rationalize all the data and keep up with the maintenance to keep it current. The result will drive all the hard work of reconciling the ten different meanings of customer and twenty different meanings of product down to the hardware level where results are nearly instantaneous. This implies solving some "wicked" problems such as generating original and correct utterances in natural language, which will still provide the most powerful BI media (and interface) known to businesses - and humanity. This will require developments that are still several breakthroughs into the future before they can be sketched on the drawing board, but, thanks to innovations in architecture such as SOA, we can be confident they are coming.


1.  Philip Howard, Director of Research - Technology. "Bloor Research - Second Generation BI." January 18, 2007 Philip writes: "To conclude, I don't think I have an answer for this: I am merely pointing out the problem and suggesting that it needs resolving. It needs an architecture if you like. I would welcome reader's comments and suggestions as to what this might look like." These are my comments and suggestions. http://www.it-analysis.com/technology/content.php?cid=9098

2. "IBM Global CEO Survey: Expanding the Innovation Horizon." ibm.com/special/ceo10

3. Holistic thinking invites another one word answer - "SOA." This essay is not intended to be a tutorial on service-oriented architecture. As a working definition, in SOA "all functions, or services, are defined using a description language and where their interfaces are discoverable over a network. The interface is defined in a neutral manner that is independent of the hardware platform, the operating system, and the programming language in which the service is implemented. One of the most important advantages of a SOA is the ability to get away from an isolationist practice in software development, where each department builds its own system without any knowledge of what has already been done by others in the organization. This "silo" approach leads to inefficient and costly situations where the same functionality is developed, deployed and maintained multiple times. A SOA is based on a service portfolio shared across the organization and it provides a way to efficiently reuse and integrate existing assets." For details go to http://www-128.ibm.com/developerworks/webservices/library/ws-soa-eda-esb/ .

4. As well as futurists such as Ray Kurzweil, see "The Law of Accelerating Return." Ray Kurzweil. March 7, 2001. http://www.kurzweilai.net/articles/art0134.html?printable=1


For more information on related topics visit the following related portals...
Business Intelligence (BI).

Lou Agosta is a business intelligence analyst with IBM. A former industry analyst at Giga Information Group, Agosta has published extensively on industry trends in data warehousing, data mining and data quality. He can be reached at LoAgosta@us.ibm.com.

E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2007 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.