Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Beyond the Data Warehouse:
Beyond Rows and Columns - Extreme Information Maturity (a.k.a. Enterprise Data Management)

online columnist John Ladley     Column published in DMReview.com
December 6, 2002
  By John Ladley

Moving beyond rows and columns requires advanced meta data. There is a rich menu of beneficial applications that can only be efficient and sustainable if they are based on extreme information maturity, i.e., a formal enterprise data management process.

Normally, the content of these articles is planned months in advance. Editorial calendars, travel, billable work (now there's a concept!) all influence when material is published. This month we are revising the sequence. This is due to the breath-taking amount of mail after the previous column, "Practical Knowledge Management (or What Comes After the Rows and Columns)". posted to DMReview.com in October, 2002. Many of the e-mails asked us to delve into the extreme information maturity aspect, as many shops are apparently trying to get a handle on information management as a more formal component of business strategy. So we will do that one first and then tie back unstructured information and collaborative intelligence in upcoming articles and wrap up by revisiting all the topics and supplying some real-life examples.

As we stated in the last article, the knowledge management (KM) discipline is providing a host of neat approaches for new information and knowledge challenges. But unlike IT, KM has turned into a group of philosophies rather than a corporate discipline. The KM concepts are shepherding many folks through the evolution from historical and operational rows and columns and predictable latency to predictive, proactive uses of structured and unstructured information. The upshot of all of this is the need to create a layer of meta data that will literally take an active part in managing the business. The KM concepts that are supported are:

  • Human Capital Management
  • Organizational Learning
  • Collaboration
  • Knowledge Identification and Dissemination

The extended BI/DW processes supported (and hinting heavily at KM as well) are:

  • Unstructured Information Usage
  • Actionable Use of Information (structured and unstructured)
  • Identification of Knowledge and Information Assets
  • Closed Loop Agents

Enterprise Data Management

Obviously, these new meta data requirements reflect a more rigid philosophy for the management of information (and knowledge) as an asset. Not an asset in a figurative sense, but a real, accountable component of an organization's value. This functionality and discipline needs to be incorporated into existing DW/BI programs as well as used for the extended DW. New information management processes are required to support moving through enterprise data management (EDM) to an extreme maturity level of knowledge and information asset management.

Note: The term that seems to be arising to cover this area is enterprise data management (EDM). While I am not sure if this term will "stick," we can use it in this article.

From an evolutionary view, EDM starts by deemphasizing business events in terms of collection and processing. The emphasis becomes twofold: 1) process data so the organization keeps useful, high quality information, and 2) view data from a process standpoint (if it isn't used, it has no value). Therefore, EDM places an emphasis on workflow.

The meta data in this scenario is obviously active. There will be multiple layers of meta data. (Yes Virginia, there will be meta meta data - sigh.) EDM will manage various layers of abstraction of meta data, processes and a wide range of other information and knowledge content. The Object Management Group (OMG) calls this a Model Driven Architecture. That is a good phrase. The figure below is a recap of the OMG layers of abstraction, called the Meta Object Facility (MOF) - go to www.omg.org to download some excellent information on this topic.

Figure 1: Object Management Group's Meta Object Facility
Figure 1: Object Management Group's Meta Object Facility

Notice abstraction, notice that the meta layers accommodate all types of content and honors other meta models that may previously existed.

Besides layers of abstraction, the technology for EDM will feature XML. While XML offers little value from an operational performance viewpoint (so far), it is quite excellent at collecting and managing the necessary meta data aspects for EDM.

Well beyond technology for collection, documentation and navigation, the meta data aspect of these concepts offers some new and interesting content. The requirements for extreme information maturity carry a few particular wrinkles, namely some new meta data requirements and new aspects of managing the information asset.

Meta Data Requirements

The new meta data requirements for these types of information and knowledge functions range from advanced to extreme. It is important to note that meta data is not an option, as it has been with pure data warehouses. There must be a meta data layer that contains all of the traditional functionality as well as new components.

  • Mandatory Governance - Given that many components of information collection and usage will be driven by meta data, the meta data layer will be active. Not only active in the sense of launching a query as current, but also all codes and dimensions are maintained and administered in the meta data layer. Business processes and rules are contained in the meta layer. No queries, applications, views, universe, tables, content of any sort can move to active or production status (including ad hoc queries) without going through an approval process operated by a meta data administration function.
  • Workflow and Process Attributes - meta data will contain workflow information - start, end times, predecessors, parameters, responsibilities and business rules for context.
  • Business Rules - Rule-based efforts and theory have been around a while. These rules provide context for information, however. They must be present in the meta data layer.
  • Privacy - Privacy is really a subset or type of business rules. However, the particulars of specific requirements must be stored in meta data even if the DBMS engine physically takes care of encryption, etc.
  • Abstraction Layers - The key to advanced mete data is abstraction. Rather than the simple definitional meta data we are used to, advanced meta data contain the layers that provide the rules and context for information. Context is absolutely critical to understanding the value of information within a certain portion of an enterprise.
  • Inventory Tag - In previous articles, I mentioned how specific data needs to be tagged, counted and tracked. The meta data layer will provide the "inventory" attributes. This inventory data will also provide information that will eventually permit creation of financial valuation of information (more on that in a future article.)
  • Process Attributes - Like workflow , process attributes need to contain rules, times, triggers, etc. The difference is a process may or may not be part of workflow . I suppose there is an argument to be made to workflow being a type of process, but that is for later discussion.
  • Document Attributes - A key application in the KM realm has been document management. Usage, owners, type are all attributes of documents. Additionally, meta data will manage this in the future.

Remember the preceding meta data content is in addition to "traditional" meta data information - definitions, ranges, synonyms, security etc. The Figure 2 demonstrates how the various meta data types interact with advanced information and knowledge functions.

Figure 2: How Meta Data Interacts with Information and Knowledge Functions

Value Proposition

What type of benefits and projects will this future meta data support? After all, the meta data layer isn't going to be done first; rather, an EDM philosophy will evolve the meta layer. Here is a brief example of where the meta layer we have discussed comes in handy.

A specialty manufacturer can design and produce thousands of different items but produces orders in relatively small quantities. Therefore, drawing upon the various setups for a job and product is knowledge-intensive. A portal is implemented that accesses a knowledge base of prior setups. In addition, operators can access best practices and customer specifications to alter setups.

  • Human Capital Management - The meta layer manages the knowledge map so setups and product characteristics can be accessed.
  • Organizational Learning - As new products are designed and set up, the actions of the operators are recorded. The meta layer contains the rules for what is measured, the definition of the measurements and the metrics around the operators actions. This allows future reference to determine best practices.
  • Collaboration - The meta layer supports to what extent the operators can share customer setups and information with other operators.
  • Knowledge Identification and Dissemination - Triggers are set up and defined in the meta layer to enable alerts to management to unique setups or operator actions that require recognition (or remediation).
  • Unstructured Information Usage - The meta data layer manages content of competitors products (gathered from available Web catalogs) so management can compare to their own set ups.
  • Actionable Use of Information (structured and unstructured) - Over time, the meta layer accrues a count of what data is accessed and used more frequently that others.
  • Identification of Knowledge and Information Assets - Data that is used more often and involved in more critical metrics is tracked in the meta layer. Crucial workflows and data elements are identified as critical to business success, and business policy considers the ongoing quality of this data.
  • Closed Loop Agents - Over time, the information usage metrics allow creation of alerts that can tell management of a difficult set up in advance of a costly design process.

The Figure 3 demonstrates a sample evolution that is enabled by maturing meta data content. Note that, excluding the top two rectangles; these steps are obtainable by most shops familiar with managing a data warehouse or sophisticated content management. We will explore how to develop your own evolution path in the fourth article in this series.

Figure 3: Evolution Enabled by Maturing Meta Data Content

Moving beyond rows and columns requires advanced meta data. Plain and simple. Without a sound EDM evolution strategy, organization may be able to answer "How did we do that?" questions but only on a siloed basis. The benefits, like those of older data marts, will soon disappear in the morass of non-integration. There is a rich menu of beneficial applications that can only be efficient and sustainable if they are based on extreme information maturity, i.e., a formal enterprise data management process.


For more information on related topics visit the following related portals...
Business Intelligence (BI), Data Management, DW Design, Methodology, Enterprise Information Management and Unstructured Data.

When John is not writing poetry as a hobby, he is a director for Navigant Consulting, which recently acquired KI Solutions, a management consulting firm specializing in knowledge and information asset management and strategic business intelligence planning and delivery. Ladley is an internationally recognized speaker and, more importantly, hands-on practitioner, of information and knowledge management solutions. He can be reached at jladley@navigantconsulting.com. Comments, ideas, questions and corroborating or contradictory examples are welcomed.

E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.