Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Meta Data & Knowledge Management:
Data Assurance Road Map, Part 2

  Column published in DM Review Magazine
January 2005 Issue
  By David Marco

In the first part of this two-part series on data assurance, I walked through the stumbling blocks that plague many data assurance initiatives. In addition, I discussed the first three phases of the data assurance road map:

  • Set/Refine Objectives: What are the business objectives?
  • Data Assessment: Where are we currently?
  • Data Alignment: Where are we going (end state)?

In this second installment, I will address the final three phases of the data assurance road map:

  • Data Harmonization: How do we get there?
  • Solution Implementation: Implement the processes.
  • Continuous Improvement: Feedback to refine and extend.

Figure 1: Data Assurance Road Map

Data Harmonization. Data harmonization is the process of mapping the "current" state of our data (as defined in the data assessment phase) to the target "end" state of our data (as defined in the data alignment phase).

A managed meta data environment (MME) is the key application in the data harmonization phase (see Figure 2). The MME is responsible for the enterprise-wide meta data management and utilization. The IT staff and data stewards will directly interact with the MME to perform the process of mapping the current data state to the end data state. This process is vital in many applications as it allows an organization to transform their data into a single version of the truth.

Figure 2: Managed Meta Data Environment

Solution Implementation. The goal of the solution implementation phase is to physically build the processes and procedures to achieve the data harmonization. During this phase, several items must be considered.

First, as bad/inaccurate data is identified, it must be tagged and placed into a database for reporting purposes. Second, the business users must be actively involved in the process of examining the inaccurate data so that feedback can be provided back to the data's source.

Third, often data will error-out because the business rules for the data are not complete or correct. At this point, the business users should be able to identify these situations, and the process must be modified for future data extractions. Lastly, records that error-out should be updated as new data becomes available or if the source record itself has been updated. In these cases, the records would then recirculate back into the process. In the case of a data warehouse, these previously tagged inaccurate records would then be marked as "clean" and subsequently loaded into the data warehouse.

Continuous Improvement. Data assurance is an evolutionary process. Therefore, organizations need to have an ongoing feedback mechanism and update capability to modify and improve its performance. Meta data and proper meta data management is the key to this initiative as it supports all of the six phases discussed here. For example, all of the information collected during the monitoring of an ETL process is meta data. This meta data needs to be persistently stored and historically tracked within the MME so that the IT staff and data stewards can leverage it to accomplish each of the phases of the road map. For a greater discussion on meta data management, please read Building and Managing the Meta Data RepositoryUniversal Meta Data Models (David Marco, Wiley 2000) and (David Marco & Michael Jennings, Wiley 2004).

When properly implemented, data assurance can greatly aid a company's ability to adapt their systems to change and to deliver strategic and tactical information to their business users in business terms. Keep in mind that "easy" is the one word you will not hear in connection with implementing data assurance. To be successful in your data assurance program takes knowledge, discipline, talented employees and good old-fashioned hard work, just like any other major IT endeavor.

The quality of information that flows through the enterprise applications, databases, networks, business processes and business rules, external processes linking to customers/suppliers/partners, external market knowledge and syndicated data sources is extremely critical in not only the day-to-day operation, but also in making tactical decisions and setting strategic direction for the enterprise. In order to enable the highest level of information integrity, we must first assess the level of quality of information at each layer -- not just the technology layer, but also the process and people layer. While data assurance is not a silver bullet, it provides a framework to avoid costly mistakes and disappointments when using data integration technology.

This column is based on the "How to Design and Deliver an Effective Data Assurance Solution" white paper. Please visit www.EWSolutions.com to download a free copy of the paper.


For more information on related topics visit the following related portals...
Data Management.

David Marco is an internationally recognized expert in the fields of enterprise architecture, data warehousing and business intelligence and is the world's foremost authority on meta data. He is the author of Universal Meta Data Models (Wiley, 2004) and Building and Managing the Meta Data Repository: A Full Life-Cycle Guide (Wiley, 2000). Marco has taught at the University of Chicago and DePaul University, and in 2004 he was selected to the prestigious Crain's Chicago Business "Top 40 Under 40."  He is the founder and president of Enterprise Warehousing Solutions, Inc., a GSA schedule and Chicago-headquartered strategic partner and systems integrator dedicated to providing companies and large government agencies with best-in-class business intelligence solutions using data warehousing and meta data repository technologies. He may be reached at (866) EWS-1100 or via e-mail at DMarco@EWSolutions.com.

Solutions Marketplace
Provided by IndustryBrains

Data Validation Tools: FREE Trial
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Backup SQL Server or Exchange Continuously
FREE WHITE PAPER. Recover SQL Server, Exchange or NTFS data within minutes with TimeSpring?s continuous data protection (CDP) software. No protection gaps, no scheduling requirements, no backup related slowdowns and no backup windows to manage.

Manage Data Center from Virtually Anywhere!
Learn how SecureLinx remote IT management products can quickly and easily give you the ability to securely manage data center equipment (servers, switches, routers, telecom equipment) from anywhere, at any time... even if the network is down.

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Click here to advertise in this space

View Full Issue View Full Magazine Issue
E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.