Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search
advertisement

RESOURCE PORTALS
View all Portals

WEB SEMINARS
Scheduled Events

RESEARCH VAULT
White Paper Library
Research Papers

CAREERZONE
View Job Listings
Post a job

Advertisement

INFORMATION CENTER
DM Review Home
Newsletters
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

GENERAL RESOURCES
Bookstore
Buyer's Guide
Glossary
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

DM REVIEW
About Us
Press Releases
Awards
Advertising/Media Kit
Reprints
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Meta Data & Knowledge Management:
Managed Meta Data Environment: A Complete Walk-Through, Part 8

  Column published in DM Review Magazine
November 2004 Issue
 
  By David Marco

This article is adapted from the book Universal Meta Data Models by David Marco and Michael Jennings, John Wiley & Sons.

Over the last seven columns, I have been presenting the six major components of a managed meta data environment (MME): meta data sourcing layer, meta data integration layer, meta data repository, meta data management layer, meta data marts, meta data delivery layer. This eighth and final installment will discuss the MME's fifth and sixth components: meta data marts and the meta data delivery layer.

Meta Data Marts

A meta data mart is a database structure, usually sourced from a meta data repository, designed for a homogenous meta data user group (see Figure 1). "Homogenous meta data user group" is a fancy term for a group of users with like needs.


Figure 1: Meta Data Marts

There are two reasons why an MME may need to have meta data marts. First, a particular meta data user community may require meta data organized in a manner other than what is in the meta data repository component. Second, an MME with a larger user base often experiences performance problems because of the number of table joins that are required for the meta data reports. In these situations, it is best to create meta data marts targeted specifically to meet those users' needs. The meta data marts will not experience the performance degradation because they will be modeled multidimensionally. In addition, a separate meta mart provides a buffer layer between the end users from the meta data repository. This allows routine maintenance, upgrades and backup and recovery to the repository without impacting the availability of the meta data mart.

Meta Data Delivery Layer

The meta data delivery layer is the sixth and final component of the MME architecture. It delivers the meta data from the meta data repository to the end users and any applications or tools that require meta data feeds (Figure 2).1


Figure 2: Meta Data Delivery Layer

The most common targets that require meta data from the MME are:

  • Applications
  • Data warehouses and data marts
  • End users (business and technical)
  • Messaging and transactions
  • Meta data marts
  • Software tools
  • Third parties
  • Web sites and e-commerce

Applications

Quite often, applications such as customer relationship management (CRM) and enterprise resource planning (ERP) need to receive meta data from the MME for their own use. In these situations, it is most common to have the meta data repository create an extract file that can be brought into the application. Typically, the repository will generate a flat file and place it in a holding area that, when the application is ready, can read it in.

Data Warehouses and Data Marts

The meta data delivery layer of data warehouses and their associated data marts (usually query and reporting are executed at the data mart level) are separate from applications because of some subtle differences in the use of the meta data. Figure 3 shows the data mart query and report bringing in meta data from the MME. Typically, data marts are accessed via front-end tools (e.g., Business Objects, Cognos, Hyperion, MicroStrategy). These tools generate SQL code. Because the meta data repository component is stored on an open, relational database, it is easy enough to "point" these tools at the repository and bring the meta data directly into the query/report (see Figure 3 for an example).


Figure 3: Meta Data Delivery Layer ­ Data Warehouse and Data Marts

For some data warehousing applications where the data in the data mart(s) is voluminous or the number of end users is high, the overhead involved with going to a separate database may create too great a time delay for end users. These technical implementation issues can be remedied by loading the meta data directly into the data marts (Figure 4) during the marts' load cycle.


Figure 4: Loading Meta Data Directly Into The Data Marts

End Users

The meta data delivery layer typically brings meta data directly to both business and technical end users. Usually this meta data is delivered directly to the user's computer in the form of a document or spreadsheets, or through a thick or thin client front-end meta data access tool.

Messaging and Transactions

As previously discussed, many companies use some form of messaging and transactions - whether EAI or XML - to transfer data from one system to another. Although most companies are not very advanced in their use and application of EAI or XML, these types of applications do utilize meta data. If companies continue to grow these applications, their need for meta data will continue to rise.

Meta Data Marts

As discussed earlier in this column, there are situations where meta data will be extracted from the repository and brought into a meta data mart component. These meta data marts are database structures designed for a homogenous group of meta data users.

Software Tools

Sharing and interchange of meta data among various software tools' internal meta data repositories is particularly desirable for global enterprises with dispersed teams trying to solve similar or related data analysis problems using an integrated computing approach.2 Hopefully, industry meta model standards such as Common Warehouse Metamodel (CWM) and ISO11179 should make this effort easier. Today, most companies have to analyze the software tool's meta models and then build technical processes that will share meta data between these tools. Anyone that has had the opportunity to build and maintain these types of processes will attest to how difficult it is.

Third Parties

Some MMEs need to send meta data to third parties: vendors or suppliers, customers, government agencies or regulatory bodies and business partners (Figure 5). Typically, when meta data is exchanged with these third parties, it is done through the generation of a flat file; however, more and more companies are beginning to use XML as transportation syntax.


Figure 5: Meta Data Delivery Layer ­ Third Parties

Web Sites and E-Commerce

Web sites and e-commerce applications also need meta data. Meta data-related Web sites are a very effective way to present meta data to the end users.3 E-commerce - with its trend toward XML and its need to interact with customers and partners - will continue to trend toward needing meta data in its processes.

This column marks the eighth and final installment of the key architectural components of a MME. Professionals who have built an enterprise meta data repository realize that it is much more than just a database that holds meta data and pointers to meta data. Rather, it is an entire environment. The purpose of the MME is to illustrate the major architecture components of that managed meta data environment.

References:

  1. See Chapter 10 of Building and Managing the Meta Data Repository (David Marco, Wiley 2000) for a detailed discussion on meta data consumers and meta data delivery.
  2. See Chapter 2 of Building and Managing the Meta Data Repository (David Marco, Wiley 2000) for a more detailed discussion on this topic.
  3. See Chapter 3 of Universal Meta Data Models (David Marco & Michael Jennings, Wiley 2004) for several examples.
...............................................................................

For more information on related topics visit the following related portals...
Meta Data.

David Marco is an internationally recognized expert in the fields of enterprise architecture, data warehousing and business intelligence and is the world's foremost authority on meta data. He is the author of Universal Meta Data Models (Wiley, 2004) and Building and Managing the Meta Data Repository: A Full Life-Cycle Guide (Wiley, 2000). Marco has taught at the University of Chicago and DePaul University, and in 2004 he was selected to the prestigious Crain's Chicago Business "Top 40 Under 40."  He is the founder and president of Enterprise Warehousing Solutions, Inc., a GSA schedule and Chicago-headquartered strategic partner and systems integrator dedicated to providing companies and large government agencies with best-in-class business intelligence solutions using data warehousing and meta data repository technologies. He may be reached at (866) EWS-1100 or via e-mail at DMarco@EWSolutions.com.

Solutions Marketplace
Provided by IndustryBrains

Verify Data at the Point of Collection: Free Trial
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Data Mining: Levels I, II & III
Learn how experts build and deploy predictive models by attending The Modeling Agency's vendor-neutral courses. Leverage valuable information hidden within your data through predictive analytics. Click through to view upcoming events.

Click here to advertise in this space


View Full Issue View Full Magazine Issue
E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Advertisement
advertisement
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.