Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Meta Data and Data Administration:
Meta Data ROI: The Evolution of Technology, Part 2

  Column published in DM Review Magazine
September 1999 Issue
  By David Marco

In order to fully understand meta data's value, it's imperative to identify the changes impacting businesses today.

This month's column on meta data return on investment (ROI) examines the technical challenges that arise as a result of the evolutionary corporate changes.

The days of building an operational system and using it for 10 to 20 years without major enhancements (typically complete rewrites) are over. In order for companies to adapt to the market's rapidly changing landscape, the enterprise's data processing systems must be more flexible and robust than ever before. Many of our current systems have been built as "stovepipe" applications, meaning that they do not communicate easily with other corporate systems. Moreover, these stovepipe applications form their own system "islands" with their own hardware platforms, databases and development languages. Corporations are demanding new systems changes at an astounding rate; and, unfortunately, these old legacy systems do not adapt well to change.

This is most clearly evident with the Y2K problem that has haunted most every company in the world. If we stop and think about it, Y2K is merely a date field that has been designed to hold a two-digit year, instead of a four-digit year. When the problem is considered in these terms, it does not appear to be very significant. Of course, we know that the Y2K problem is very significant because the code that manipulates the date field is very convoluted, difficult to identify and hard to locate.

For those who believe that the need to make global systems changes end with Y2K, keep in mind that immediately following the Y2K will be the new standardized European currency (EURO) challenge. While the EURO will significantly impact all corporations, financial institutions will be most affected.

Along with the rapid change in business needs, there is an equally rapid change in the technology that fuels systems development. As a result, corporations are working feverishly to keep their systems up to speed with the extreme rate of change in technology. It seems that every week there is a new software application or piece of hardware that would aid a company in meeting their corporate goals. This situation is certainly not going to slow down in the upcoming years. Corporations are demanding faster hardware and more software functionality and integration to gain a corporate advantage.

Another challenge facing companies today is getting their systems to meet the needs of their business users in an understandable manner and language. As an industry, we are still not sufficiently meeting the needs of our business users. One recent survey asked CEOs if their IT systems were meeting the needs of their businesses. Over 84 percent of the CEOs felt that their systems do not. As an IT professional, this is a statistic that cannot go unnoticed. Instead of designing systems that speak in business terms, the systems we are building still communicate in IT terms. For example, I experienced a system that showed the user a report called XC001AB, which is a report that shows product sales by region and by marketing campaign over time. Clearly, a marketing analyst would much rather see this report entitled "Product Sales by Region and by Marketing Campaign Over Time." This small example illustrates why so few senior executives believe that their corporation's IT strategy is well integrated with their business strategy.

The good news is that the vast majority of these senior executives do believe that their systems are critical to the success of their business. Operational systems are typically designed to manage products. These systems have evolved over the years to produce, deliver and invoice products and services. Unfortunately, when remedial questions are posed - Who are our most profitable customers? Which segment of our market offers the greatest future potential profit? Which of our products are complementary (market basket analysis)? Which competitors pose the greatest threat to our existence? Which product/service is providing the greatest value to our customers? - the operational systems cannot easily or quickly provide answers to these fundamental questions that are redefining the world of business.

The emergence of decision support looks to transition legacy (operational) systems to decision support systems (DSSs) that help companies answer these questions which support a one-to-one marketing future. The DSS system is designed to manage customers, not products. DSSs are built to handle the "strategic" questions that a company's key decision-makers need to answer. Moreover, companies now understand that without a DSS, they will not be able to compete effectively enough to survive in tomorrow's marketplace.

It is easy to see the need for decision support systems. Unfortunately, the task of implementing these systems is anything but easy. The challenges for implementing a DSS system (also known as a data warehouse) come from both the business and technical arenas.

The most common cause for DSS project failure is that once the system is built, it does not meet the business objectives of the organization. Data warehouses that do not satisfy the business users' decision support needs are not accessed, and they eventually die.

Business objectives can be clearly defined and are measurable. This activity is critical since once the DSS project is completed, the management team will have to justify the expenditures. Moreover, it's important to understand that a data warehouse is not a project; it is a process. Data warehouses are organic in nature. They grow very fast and in directions that are very difficult to anticipate. Most warehouses double in size and in the number of users in their first year of production. Once a cost justification can be quantified for the initial release, the process for gaining funding for the follow-up releases is greatly simplified.

From a political perspective, an enterprise data warehouse requires consent and commitment from all of the key departments within a corporation. This challenge is difficult since it is rare that separate departments agree. As a result, the political aspects of implementing these systems are challenging.

DSS projects technically stretch an organization in ways unlike that of traditional systems projects. Typically, a data warehouse will source data from most, if not all, of the key operational systems within a company. The task of integrating this data from across the enterprise is considerable. For example, it is probable that a company would want to store customer data in the data warehouse. More than likely, there is customer data in several of the firm's operational systems. All of this dispersed customer data has to be integrated and cleansed before it can be loaded into the data warehouse - a complicated process.

DSSs tend to store large quantities of data. It is even becoming somewhat common to see one or more terabytes of data stored and accessed. By adding massive amounts of data into the equation, the points of failure increase significantly. Moreover, large volumes of data push the envelope of the database management system (DBMS), middleware and hardware. If massively parallel processing (MPP) architecture is needed, parallel development techniques must be used. Keep in mind, the answer to many of these challenges comes in the form of a hefty price tag. As a result, adding the dimension of size can be painful and costly.

Many legacy systems contain redundant or inaccurate data. This lack of data quality in the operational systems has caused more than one decision support effort to fail. As a result, operational data must be cleansed before it is loaded into the data warehouse to ensure its usability. Meta data is critical for monitoring and improving the quality of the data coming from the operational systems. Meta data tracks the number of errors that occur during each data warehouse load run and can report when certain error thresholds are reached. In addition, the DSS data quality metrics should be stored over the history of the DSS. This allows corporations to monitor their data quality over time.

Next month's column will wrap up this series on meta data ROI by examining the solutions meta data provides to these business and technical challenges.


For more information on related topics visit the following related portals...
DW Administration, Mgmt., Performance.

David Marco is an internationally recognized expert in the fields of enterprise architecture, data warehousing and business intelligence and is the world's foremost authority on meta data. He is the author of Universal Meta Data Models (Wiley, 2004) and Building and Managing the Meta Data Repository: A Full Life-Cycle Guide (Wiley, 2000). Marco has taught at the University of Chicago and DePaul University, and in 2004 he was selected to the prestigious Crain's Chicago Business "Top 40 Under 40."  He is the founder and president of Enterprise Warehousing Solutions, Inc., a GSA schedule and Chicago-headquartered strategic partner and systems integrator dedicated to providing companies and large government agencies with best-in-class business intelligence solutions using data warehousing and meta data repository technologies. He may be reached at (866) EWS-1100 or via e-mail at DMarco@EWSolutions.com.

Solutions Marketplace
Provided by IndustryBrains

Data Validation Tools: FREE Trial
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Speed Databases 2500% - World's Fastest Storage
Faster databases support more concurrent users and handle more simultaneous transactions. Register for FREE whitepaper, Increase Application Performance With Solid State Disk. Texas Memory Systems - makers of the World's Fastest Storage

Manage Data Center from Virtually Anywhere!
Learn how SecureLinx remote IT management products can quickly and easily give you the ability to securely manage data center equipment (servers, switches, routers, telecom equipment) from anywhere, at any time... even if the network is down.

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Click here to advertise in this space

View Full Issue View Full Magazine Issue
E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.