Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search
advertisement

RESOURCE PORTALS
View all Portals

WEB SEMINARS
Scheduled Events

RESEARCH VAULT
White Paper Library
Research Papers

CAREERZONE
View Job Listings
Post a job

Advertisement

INFORMATION CENTER
DM Review Home
Newsletters
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

GENERAL RESOURCES
Bookstore
Buyer's Guide
Glossary
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

DM REVIEW
About Us
Press Releases
Awards
Advertising/Media Kit
Reprints
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Business Intelligence:
What is This Data? End User Presentation of Meta Data

online columnist Jonathan Wu     Column published in DMReview.com
October 13, 2000
 
  By Jonathan Wu

An increasing number of our clients are asking, "What is this data?" Sure, the business intelligence (BI) applications they are using are providing data, but they want to feel comfortable with the origin of the data, the business rules that were used to manipulate it and want the context in which the description of the data elements to make sense. In these situations, our clients are asking about the end-user presentation of meta data. Simply defined, meta data is the data definition of data. There are two primary levels of meta data: administrative and end-user layers.

The administrative layer of meta data relates to the application and the database configurations that are stored in entities within a database. While this is an important layer, the end-user layer of meta data is where most of the questions and differences of opinion arise. Figure 1 attempts to define and classify the various levels of end user meta data.

Level Level Name Genesis
5 Report data Within a report, data is manipulated into a format that is required.
4 Business intelligence data Data which has been extracted by a BI application. This data may be manipulated using business rules.
3 Transformed data Data that has been transformed using business rules into a format that is meaningful and is stored within a database.
2 Cleansed data Data which has been processed through a cleansing routine which standardizes the format and is stored within a database.
1 Migrated data Data which has been migrated from several source systems into one database and is commingled.
0 Source system data Data which has been entered or processed by an online transaction processing application.

While these levels define the transgression of data from its origin to a report, the complexities of each level can have a significant impact on the definition of the data. Please note that not all levels of end-user meta data are encountered by an individual using a BI application. Levels 1 - 3 relate to the data warehousing of information, in which case an individual lacking access to a data warehouse may encounter only levels 0, 4 and 5 when extracting data directly from a source system.

Source System Data

In each transactional system, a user of the system must enter data. For example, a sales representative enters information about a product that was sold to a customer within an order entry application. The information that is entered by the sales representative is accepted by the application and stored within a database. The data elements of information relating to the sales transaction may include the customer name, shipping and billing addresses, terms of the transaction, product purchased, quantity sold, sales price and discounts.

Migrated Data

Assuming that an organization has multiple transactional systems and the desire to warehouse the data, migrating the data from the source systems to an operational data store or a data warehouse entails the identification of the source system as the data is commingled. For example, customer information is migrated from the order entry and the customer relationship management application into a single database used as either an operational data store or the staging area of a data warehouse. The data elements of information relating to the customer information may include customer name, tax payer identification number, customer type, contact name, address and phone number from each of the source systems.

Cleansed Data

After the data has been migrated to a central database, a cleansing routine can be applied to standardize the format of the data and validate the quality of the data. For example, customer information has been migrated from several source systems. As part of the cleansing process, data may be standardized and redundancy eliminated. With one of our clients, we noted that one customer name was uniquely defined in each of the source systems and that the redundancy was not eliminated until the cleansing process was conducted. In this example, imagine one customer with the following uniquely defined names: base consulting group, Base Consulting Group, BASE Consulting Group and BASE Consulting Group, Inc. The cleansing process would eliminate the redundancy, standardize the format and validate the data so that one customer name would remain: BASE Consulting Group, Inc.

Transformed Data

Business rules and definitions are used to transform the cleansed data into meaningful information. For example, revenue may be defined as the quantity sold multiplied by the sale price less any discounts. This transformation of data derives a new data element from the data created by the source system(s). Defining the business rules as well as achieving acceptance of the definition can be challenging, and it is the level that encounters the most difficulty when trying to achieve consensus.

Business Intelligence data

Most BI applications can access data from a variety of sources including databases, ASCII files or spreadsheets. Additional transformations or masking of data can be directed by the BI application. For example, a data element called segment1 that contains information about purchase order numbers can be masked and renamed to PO Number. From the end user perspective the data element is called PO Number, but the database name for it is segment1, which does not provide a user with a descriptive understanding of the type of data that is associated with it.

Report data

Within a report, data can be derived that further enhances the meaningfulness of the information. However, this is the last meta data level within the end user layer. Data derived at this level typically relates to calculations such as subtotals and grand totals or conditional statements such as, "If gross margin % >= 50% then 'Highly Profitable Product' is listed in the column next to the gross margin %."

Summary

As users embrace BI technology for analysis and reporting, they want to know what the data is so that they can put it into the appropriate context. By understanding the various levels of end user meta data, users can fathom the metamorphosis of data from its origin to the report.

References:
LeBaron, M. and Adelman, S. Meta Data Standards. DM Review magazine. February, 1997.
Inmon, W. Enterprise Meta Data. DM Review magazine. November, 1998.

...............................................................................

For more information on related topics visit the following related portals...
Meta Data and Business Intelligence.

Jonathan Wu is a senior principal with Knightsbridge Solutions. He has extensive experience designing, developing and implementing information solutions for reporting, analysis and decision-making purposes. Serving Fortune 500 organizations, Knightsbridge delivers actionable and measurable business results that inform decision making, optimize IT efficiency and improve business performance.  Focusing exclusively on the information management disciplines of data warehousing, data integration, information quality and business intelligence, Knightsbridge delivers practical solutions that reduce time, reduce cost and reduce risk. Wu may be reached at jwu@knightsbridge.com.

Solutions Marketplace
Provided by IndustryBrains

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

Data Quality Tools, Affordable and Accurate
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

dotDefender protects sites against Web attacks
30-day evaluation period for dotDefender, a high-end cost-effective security solution for web servers that protects against a broad range of attacks, is now available. dotdefender supports Apache, IIS and iPlanet Web servers and all Linux OS's.

Click here to advertise in this space


E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Advertisement
advertisement
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.