|Sign-Up for Free Exclusive Services:||Portals|||||eNewsletters|||||Web Seminars|||||dataWarehouse.com|||||DM Review Magazine|
|Covering Business Intelligence, Integration & Analytics||Advanced Search|
Meta Data & Knowledge Management:
This column is adapted from the book Universal Meta Data Models by David Marco and Michael Jennings, John Wiley & Sons.
In last month's column, I presented the meta data management layer of a managed meta data environment (MME). That column discussed the following meta data management layer functions: archive, backup, database modifications, database tuning, environment management, job scheduling and load statistics.
In this month's column, I will complete my walk-through of the meta data management layer by discussing the remaining functions: purging, query statistics, query and report generation, recovery, security processes, source mapping and movement, user interface management and versioning.
This part of the meta data management layer defines the criteria for MME purging requirements. Your MME's specific purging requirements and criteria will be governed by its business requirements. As a general rule, meta data that is inaccurate or improperly loaded should be purged; all other meta data should be archived.
Once the various meta data delivery layer reports and queries are generated, it is important to capture the user statistics associated with the access to these reports and queries. At a minimum, the meta data management layer needs to historically capture:
The reports and queries used in the meta data delivery layer are defined and generated in the report generation section of the meta data management layer. How this is accomplished depends on whether a custom meta data delivery application is developed or a meta data access tool is implemented. This component also needs to manage any publish and subscribe capabilities that are required.
There are many situations that can require a company to recover or reload a previous version of their MME: hardware failure, database corruption, power outage or errors in the meta data integration layer. The recovery process needs to be tightly tied to the backup and archiving facilities of the meta data management layer. Recovery processes may be manually built or utilize existing recovery processes within the meta data integration tool. Both approaches need to be integrated into the organization's existing application recovery processes.
The security processes manage:
The extensiveness of security processes is dependent on the business requirements of the MME. For instance, the security requirements around a Department of Defense or Federal Bureau of Investigation MME are far greater than those of a bank MME.
Meta data sources need to be mapped to the correct attributes and entities of the meta data repository. This mapping and movement process needs to be captured and managed in the meta data management layer of the MME.
This involves the processes for the building, managing and maintaining the Web site (recommended user interface) that the end users will visit to navigate through the MME. Typically, the view (version of the Web site) end users see depends on their security privileges and profiles. A business user would not be interested in looking at program code changes; therefore, it makes sense to not have meta data reports or queries related to highly technical meta data available to the traditional business user through control of role access.
As meta data is changed, added and purged, it will be important for the meta data management layer to historically track (or version) it. There are two common techniques for meta data versioning. The first is the use of date stamps. Date stamping - assigning a date to each row of your meta model entities - allows a firm to make whatever changes are necessary to the meta data while maintaining its historical relevance.
The second technique is to assign version numbers to each row of the meta model. Version numbers are merely generated unique values (e.g., 1.1, 1.2, 1.a, etc.). Version numbers are more limiting than date stamping, which offers more meaning to MME users. Version numbers can be associated with specific dates and times; however, this adds additional complexity to the loading of the meta data and an additional join in the access queries and reports.
Another challenge of versioning is capturing meta data that will become current at some point in time in the future. For example, a new physical table may be moved into the production environment at a future date. To handle these versioning situations, an "effective dated rows" process can be useful. An effective dated rows process allows data to be entered into a group (table) for subsequent processing when the effective date becomes current. Effective dated data can be historical, current or future. Following are the key effective dated rows concepts:
Effective date: The date on which the row of data becomes effective; the date it can be acted on.
Effective status: Allows an application to select the appropriate effective dated rows when combined with the effective date (domain value list: "active" or "inactive").
Effective status comments: The current row is the first row of data with an effective date equal to or less than the system date and an effective status of "active". Only one row can be in this state. Future rows are meta data rows that have a date greater than the system date. Historical rows are rows of meta data that have an effective date less than the current row.
Figure 1 illustrates the effective dated rows technique. In this example, the current system date is January 20, 2004. The meta data row dated "2004-01-27" has an effective status of "inactive." However, once the current date reaches January 27, 2004, the meta data row dated "2004-01-18" will become a historical row and the row dated "2004-01-27" will have its "effective status" changed from "inactive" to "active."
Figure 1: Effective Dated Rows Table
Next month, I will conclude our eight-part MME odyssey by reviewing the meta data marts and the meta data presentation layer.
David Marco is an internationally recognized expert in the fields of enterprise architecture, data warehousing and business intelligence and is the world's foremost authority on meta data. He is the author of Universal Meta Data Models (Wiley, 2004) and Building and Managing the Meta Data Repository: A Full Life-Cycle Guide (Wiley, 2000). Marco has taught at the University of Chicago and DePaul University, and in 2004 he was selected to the prestigious Crain's Chicago Business "Top 40 Under 40." He is the founder and president of Enterprise Warehousing Solutions, Inc., a GSA schedule and Chicago-headquartered strategic partner and systems integrator dedicated to providing companies and large government agencies with best-in-class business intelligence solutions using data warehousing and meta data repository technologies. He may be reached at (866) EWS-1100 or via e-mail at DMarco@EWSolutions.com.
|View Full Magazine Issue|
|E-Mail This Column|