Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Information Is Your Business
   Information Is Your Business Advanced Search

Business Intelligence
Corporate Performance Management
Data Integration
Data Quality
Data Warehousing Basics
Master Data Management
View all Portals

Scheduled Events

White Paper Library
Research Papers



DM Review Home
Current Magazine Issue
Magazine Archives
DM Review Extended Edition
Online Columnists
Ask the Experts
Industry News
Search DM Review

Tech Evaluation Center:
Evaluate IT solutions
Buyer's Guide
Industry Events Calendar
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Knowledge Integrity:
Philosophy, Data Integration and Meta Data

  Column published in DM Review Magazine
June 2005 Issue
  By David Loshin

In my last few columns, I have focused on technical implementations of object representation; therefore, this month, I'd like to look at how an organization's long-term information strategy affects how data integration projects are performed.

Presented with a collection of legacy data sets for the purpose of enterprise data integration, an information engineer may opt to proceed based on two different philosophies, which for convenience sake I will refer to as "pragmatic" versus "holistic." While either approach should have the side effect of integrating the data, the selected philosophy will have a fundamental effect on the planning, execution and long-term strategic value of the integration project.

The performance target of the pragmatic approach is the same as the side effect: integrated data. Because the goal is to take the legacy data sets and effectively merge them together, the practitioner will probably apply a relatively well-worn process:

  • Understand business needs of integration project,
  • Evolve "to-be" data model,
  • Profile the legacy data,
  • Resolve and consolidate frequently used reference data,
  • Determine relational structure embedded within legacy data,
  • Document source and target meta data,
  • Develop transformation routines,
  • Populate target data model.

Of course, I have simplified this process by boiling it down to a handful of bullet points. Anyone that has worked on these kinds of projects knows that there are significant challenges in this process, and one may encounter a number of failures before getting it right. And even after the data has been integrated, there remain lingering questions about quality, consistency with previous applications using the data and usefulness of any new system depending on the migrated information.

Alternatively, the holistic approach seeks to understand the business use and value of the information asset as embodied by the data, the structural meta data and an additional layer of meta data that consolidates meaning and intent embedded within the structure and use of the data. One might refer to the latter as "meta knowledge" - information about the knowledge embodied in the data. The holistic approach might apply this kind of process:

  • Understand the fundamental entities that interact within the business,
  • Evolve a strategy for representation of those entities that can be applied in both operational and analytical/intelligent applications,
  • Profile the legacy data,
  • Identify key reference data objects and document their application uses and business needs,
  • Determine connectivity model between legacy entities and capture business meaning inherent in application use,
  • Document source and target meta data,
  • Document business rules governing the use of the legacy data and the needs of the "to-be" system,
  • Develop transformation routines,
  • Facilitate new information use through information consolidation.

I have presented two high-level approaches, which basically do the same thing. The difference lies in practitioner intent for exploiting what you have learned during the integration process. In the pragmatic approach, because the goal is to bring the legacy data sets together, when that goal is reached, the process is finished. In the holistic approach, the goal is to understand how using information adds value across the enterprise, and integrating the data sets is achieved as a by-product.

Is one approach better than the other? It depends on the strategy that your organization has adopted around data. In businesses that rely on data as part of the day-to-day operations, failures in operational processing erode the business, and if there is no significant need for business intelligence, then the pragmatic approach provides the optimal way to achieve business objectives.

On the other hand, an organization that expects to derive synergistic value through information integration may prefer the holistic approach in that it presents opportunities for improving the business. However, the key individuals in that organization must be prepared to justify the effort and resources required to effectively capture and internalize the discovered meta knowledge. This means being able to articulate concepts of information value to senior managers and gain their active sponsorship (as opposed to passive consent). In addition, one must also justify the acquisition of the components to support a meta knowledge infrastructure. That includes, but is not restricted to data profiling, meta data repositories, semantic and ontological meta data organization and business rules management systems, to name a few.

Lastly, and probably most important, is the existence of an information strategy that looks at past use, current needs and future utility of information as a fundamental corporate asset. In this context, a forward-thinking senior manager can establish an information value proposition that persists through and thereby facilitates evolution in information support of the business environment.


For more information on related topics visit the following related portals...
Data Integration.

David Loshin is the president of Knowledge Integrity, Inc., a consulting and development company focusing on customized information management solutions including information quality solutions consulting, information quality training and business rules solutions. Loshin is the author of Enterprise Knowledge Management - The Data Quality Approach (Morgan Kaufmann, 2001) and Business Intelligence - The Savvy Manager's Guide and is a frequent speaker on maximizing the value of information. Loshin may be reached at loshin@knowledge-integrity.com.

Solutions Marketplace
Provided by IndustryBrains

Free DB Modeling Trial with ER/Studio
Design and Build More Powerful Databases with ER/Studio.

Database Access Tool
Reverse engineers relational schemas and automatically generates data access code that can be used in Service Oriented Architectures.

DeZign for Databases - Database Design Made Easy
Create, design & reverse engineer databases with DeZign for Databases, a database design tool for developers and DBA's with support for Oracle, MySQL, MS SQL, MS Access, DB2, PostgreSQL, InterBase, Firebird, NexusDB, dBase and Pervasive.

Data Mining Courses: Strategy, Methods & Apps
Learn how experts build and deploy predictive models by attending The Modeling Agency's vendor-neutral courses. Leverage valuable information hidden within your data through predictive analytics. Click through to view upcoming events.

Backup SQL Server or Exchange Continuously
FREE WHITE PAPER. Recover SQL Server, Exchange or NTFS data within minutes with TimeSpring?s continuous data protection (CDP) software. No protection gaps, no scheduling requirements, no backup related slowdowns and no backup windows to manage.

Click here to advertise in this space

View Full Issue View Full Magazine Issue
E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2007 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.