Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Meta Data & Knowledge Management:
Managed Meta Data Environment: A Complete Walk-Through, Part 7

  Column published in DM Review Magazine
October 2004 Issue
  By David Marco

This column is adapted from the book Universal Meta Data Models by David Marco and Michael Jennings, John Wiley & Sons.

In last month's column, I presented the meta data management layer of a managed meta data environment (MME). That column discussed the following meta data management layer functions: archive, backup, database modifications, database tuning, environment management, job scheduling and load statistics.

In this month's column, I will complete my walk-through of the meta data management layer by discussing the remaining functions: purging, query statistics, query and report generation, recovery, security processes, source mapping and movement, user interface management and versioning.


This part of the meta data management layer defines the criteria for MME purging requirements. Your MME's specific purging requirements and criteria will be governed by its business requirements. As a general rule, meta data that is inaccurate or improperly loaded should be purged; all other meta data should be archived.

Query Statistics

Once the various meta data delivery layer reports and queries are generated, it is important to capture the user statistics associated with the access to these reports and queries. At a minimum, the meta data management layer needs to historically capture:

  • Who is accessing the report or query?
  • What report or query are they accessing?
  • When are they accessing the report or query?
  • How often are they accessing the report or query?
  • How long do they access the report or query?

Query and Report Generation

The reports and queries used in the meta data delivery layer are defined and generated in the report generation section of the meta data management layer. How this is accomplished depends on whether a custom meta data delivery application is developed or a meta data access tool is implemented. This component also needs to manage any publish and subscribe capabilities that are required.


There are many situations that can require a company to recover or reload a previous version of their MME: hardware failure, database corruption, power outage or errors in the meta data integration layer. The recovery process needs to be tightly tied to the backup and archiving facilities of the meta data management layer. Recovery processes may be manually built or utilize existing recovery processes within the meta data integration tool. Both approaches need to be integrated into the organization's existing application recovery processes.

Security Processes

The security processes manage:

  • Creation of new MME users.
  • Grouping of MME users.
  • Setting MME user privileges and profiles.
  • Firewalls/infrastructure.
  • Password management.
  • User location verification.
  • Meta data masking (data level or access level).

The extensiveness of security processes is dependent on the business requirements of the MME. For instance, the security requirements around a Department of Defense or Federal Bureau of Investigation MME are far greater than those of a bank MME.

Source Mapping and Movement

Meta data sources need to be mapped to the correct attributes and entities of the meta data repository. This mapping and movement process needs to be captured and managed in the meta data management layer of the MME.

User Interface Management

This involves the processes for the building, managing and maintaining the Web site (recommended user interface) that the end users will visit to navigate through the MME. Typically, the view (version of the Web site) end users see depends on their security privileges and profiles. A business user would not be interested in looking at program code changes; therefore, it makes sense to not have meta data reports or queries related to highly technical meta data available to the traditional business user through control of role access.


As meta data is changed, added and purged, it will be important for the meta data management layer to historically track (or version) it. There are two common techniques for meta data versioning. The first is the use of date stamps. Date stamping - assigning a date to each row of your meta model entities - allows a firm to make whatever changes are necessary to the meta data while maintaining its historical relevance.

The second technique is to assign version numbers to each row of the meta model. Version numbers are merely generated unique values (e.g., 1.1, 1.2, 1.a, etc.). Version numbers are more limiting than date stamping, which offers more meaning to MME users. Version numbers can be associated with specific dates and times; however, this adds additional complexity to the loading of the meta data and an additional join in the access queries and reports.

Another challenge of versioning is capturing meta data that will become current at some point in time in the future. For example, a new physical table may be moved into the production environment at a future date. To handle these versioning situations, an "effective dated rows" process can be useful. An effective dated rows process allows data to be entered into a group (table) for subsequent processing when the effective date becomes current. Effective dated data can be historical, current or future. Following are the key effective dated rows concepts:

Effective date: The date on which the row of data becomes effective; the date it can be acted on.

Effective status: Allows an application to select the appropriate effective dated rows when combined with the effective date (domain value list: "active" or "inactive").

Effective status comments: The current row is the first row of data with an effective date equal to or less than the system date and an effective status of "active". Only one row can be in this state. Future rows are meta data rows that have a date greater than the system date. Historical rows are rows of meta data that have an effective date less than the current row.

Figure 1 illustrates the effective dated rows technique. In this example, the current system date is January 20, 2004. The meta data row dated "2004-01-27" has an effective status of "inactive." However, once the current date reaches January 27, 2004, the meta data row dated "2004-01-18" will become a historical row and the row dated "2004-01-27" will have its "effective status" changed from "inactive" to "active."

Figure 1: Effective Dated Rows Table

Next month, I will conclude our eight-part MME odyssey by reviewing the meta data marts and the meta data presentation layer.


Check out DMReview.com's resource portals for additional related content, white papers, books and other resources.

David Marco is an internationally recognized expert in the fields of enterprise architecture, data warehousing and business intelligence and is the world's foremost authority on meta data. He is the author of Universal Meta Data Models (Wiley, 2004) and Building and Managing the Meta Data Repository: A Full Life-Cycle Guide (Wiley, 2000). Marco has taught at the University of Chicago and DePaul University, and in 2004 he was selected to the prestigious Crain's Chicago Business "Top 40 Under 40."  He is the founder and president of Enterprise Warehousing Solutions, Inc., a GSA schedule and Chicago-headquartered strategic partner and systems integrator dedicated to providing companies and large government agencies with best-in-class business intelligence solutions using data warehousing and meta data repository technologies. He may be reached at (866) EWS-1100 or via e-mail at DMarco@EWSolutions.com.

Solutions Marketplace
Provided by IndustryBrains

Data Validation Tools: FREE Trial
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Backup SQL Server or Exchange Continuously
FREE WHITE PAPER. Recover SQL Server, Exchange or NTFS data within minutes with TimeSpring?s continuous data protection (CDP) software. No protection gaps, no scheduling requirements, no backup related slowdowns and no backup windows to manage.

Manage Data Center from Virtually Anywhere!
Learn how SecureLinx remote IT management products can quickly and easily give you the ability to securely manage data center equipment (servers, switches, routers, telecom equipment) from anywhere, at any time... even if the network is down.

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Click here to advertise in this space

View Full Issue View Full Magazine Issue
E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.