Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Why is Data Architecture so Darn Hard to Implement?

  Article published in DM Direct Newsletter
April 11, 2003 Issue
  By R. Todd Stephens, Ph.D.

Let's begin by asking the question many data architecture people in the world fail to address. After 20 years of defining, developing and implementing data architecture, why are so few organizations really practicing true data architecture at the enterprise level? It's time for us to stop practicing the Sgt. Shultz architecture approach of "I see nothing, nothing" - the nothing wrong with skipping or omitting enterprise data architecture from the corporate architecture landscape.

To start, let's remove one basic excuse used within other architecture disciplines: change. Unlike technical and application architecture, data architecture is not a fluid architecture. In other words, the data architecture doesn't change very often. New concepts - data stewardship, meta data management, data modeling? None of these data architecture segments are all that new. In fact, when was the last time a new data modeling approach was introduced? Not to mention and new meta data repository solution that is robust enough to handle an enterprise meta data strategy. Has our business model changed so much in the past decade that we now have new data content descriptors? The answer to these questions is obviously no. But everything in life changes doesn't it? Why not data architecture?

Well, it does in a way. But let's clarify some points: how we use data changes, a lot! How much data we can store now changes, a lot! But how data relates to other data, advocating corporate data stewards, etc., that hasn't really changed that much. When was the last time the components of your address changed? Still have a house or apartment number? How about a street name? City? ZIP code?

If the techniques, tools and functions of data architecture are fairly stable, why is it so darn hard? One mistake many people make is the expectation that when you buy an enterprise package such as customer relationship management (CRM), supply chain management (SCM) or enterprise resource planning (ERP), you automatically purchase a data architecture. However, for that to be true, the organization would expect to receive a solid set of conceptual, logical and physical data models. The internal meta data would be easily exchanged with other applications and the data access API layer would be modeled and readily available for all to see. Process flows, metrics and an extensive array of documentation would only be one click away. And since we're talking nirvana, shouldn't we get updates to our models for every new or maintenance release we receive from our package vendor?

However, as we all know this information or knowledge is rarely shared unless surgically removed from a consultant's head. Who is to blame here? We are! The data professionals that fail to ask or better yet demand these data architecture artifacts are the ones that should accept full responsibility. No one wants to hear you say, "If the product is so good, why won't they share their data model?" That is, until a few years into maintenance mode when people begin ask why we didn't ask for this information before the check was signed.

Does the lack of these artifacts mean the application you just bought is bad? Well, maybe and maybe not. It does provide me, the purchaser/owner of it, some perception that cognitive reasoning went into the creation of the application. Put it this way, every time I put together a gas grill, I know somehow all these parts will come together to form the grill (with one or two extra just for confusion!). That's what existence of these models tell us - some application will be built!

We must realize data architecture is a "grey architecture." That means the principles, policies and relationships are not going to be as clear cut as the decision to deploy J2EE or .NET. The fuzziness that surrounds data architecture is not a fault but a reality. Take data quality for instance. How do you define data quality? Is data quality of 99.9 percent a good data architecture standard? It all depends. For social security numbers 99.9 percent is achievable (as long as all your employees or customers have one), but for street address 99.9 percent simply cannot be attained. Oh sure, you can populate the address fields with data, but really, is it quality data? How many times have you moved in your life? Don't you think that somewhere in the great ether you are still associated with an old address?

The "grey- ness" of the architecture does not take away from its validity. Many people get caught up in the details of creating the architecture. Architecture is not about defining a percentage of accuracy. The Institute of Electrical and Electronic Engineers (IEEE) has defined architecture as "the structure of the components, their relationships and the principles and guidelines governing their design and evolution over time." The best thing about a grey architecture is you can develop a flexible data architecture that can be modified and updated within a short period of time. Hence, the evolution over time defines the level of flexibility.

Is data architecture too complex? Have we as data professionals created an environment where the perception of data architecture is so intricate we cannot even define value to the organization? Do we confuse the principle audience with numerous semantics and the newest buzzwords? Take the statement: That's an operational database. OK, is it operational based on the type of data involved or related to the function of the database. Is the data temporary or transactional? What's the difference? Data architecture does not need to be as complex as we make it. If a data architect cannot sit down with someone and explain the architecture at multiple levels of detail then maybe we need to review our method of presentation. Once again we are the ones to blame for creating the complexity and an inability to present a consistent value add to the organization.

Data is one of the top assets of a corporation. Many decision-makers require only "data and fact" to solve problems a corporation faces. However, these decision-makers cannot easily make decisions when there are inconsistencies in data. Therefore, the goal of data architecture is to provide the framework where data quality is improved by managing the data as an asset. Data architecture is the blueprint for the life cycle of business information. The life cycle includes the structure of the information (as represented by data models and databases) as well as the business events and movement of the information. Data architecture is based on business goals and objectives, technical goals and objectives, and the desired state definitions of the other technical (e.g., platform) and functional architectures (i.e., applications). It is the corporation's expression of strategy for creating and managing the use of data in order to transform data into information. Recognizing that data is a strategic asset that is expensive to handle and easy to waste, the data architecture must assure:

  • Standardization of data structures (logical and physical).
  • Definition and protection of the data resource.
  • Consistency and quality of the data resource.
  • Judicial use of corporate resources (e.g., personnel) in managing the data asset.
  • That credible and timely data is delivered throughout the enterprise at a reasonable cost.
  • That it can be driven by the needs of business and can adapt to changes in the business.
  • That it can be implemented in any technical environment.

Finally, data architecture just isn't cool enough. With so few dramatic changes in the approach, tools and methodology, information technology folks just don't want to hear another conversation over why we should have naming standards or consistent structures. After 10 years of college, I can count on one hand the number of times meta data, data stewardship, data usage and data content was mentioned during a class. However, I have no problem counting the endless hours studying LISP, SNOBOL, Assembler, Pascal, PL/1, ADA; and when was the last time you whipped out a handy LISP utility? Ironically, I have been involved with an enterprise meta data implementation over the past three years, and the experience and knowledge that I have gained has been enormous. Using technologies such as XML, object oriented (OO) frameworks and Web services, this project has provided a challenge unlike any I have faced in my 20-year career.

Are there other reasons we should pursue data architecture - the move toward component-based modeling, new XML technologies or the commercial off-the-shelf implementations? Yes, but it is time for a change. Instead of continuing to focus on the things done wrong in the world of data architecture, we should look for the things done right. Data architecture should be a lynchpin for successful projects. The data warehouse is an excellent example of where the principles of data architecture can deliver value to the corporation. An enterprise database or enterprise meta data implementation can provide the detailed specifics of value that will enable you to implement data architecture at the corporate level. The process takes work and a continuous evolution over time. Thinking about why data architecture will fail will only increase the odds that it will. Successful data architecture requires perseverance. Perseverance does not mean sticking with the same old worn out strategy. Robot architecture will fail to inspire, motivate and drive true change within the world of data. Data architecture is our responsibility and the old approach just isn't working:

Newly Hired Data Architect: Where are our data standards and policies?
Manager: Here and there.
Newly Hired Data Architect:   What are the data architectural standards?
Manager: This and that.
Newly Hired Data Architect: What specific areas are we focusing on?
Manager: Anything and everything.
Newly Hired Data Architect: When do the standards get applied?
Manager:  Now and then.
Newly Hired Data Architect:    We need to create an enterprise data architecture to address these problems.
Manager:  Great, when would I begin to see the value add?
Newly Hired Data Architect: Sooner or later.

As data architects, we realize that data architecture is hard. Implementing these concepts at the enterprise level is, to say the least, a challenge. The one thing we should work on is stop making it harder than it needs to be. Everyone wants higher quality, well-defined, understandable and documented data. Proven best practices are available from just about every corporation. With so much going for it, is it any wonder that enterprise data architecture is so darn easy to implement.


For more information on related topics visit the following related portals...
DW Design, Methodology.

R. Todd Stephens, Ph.D. is the director of Meta Data Services Group for the BellSouth Corporation, located in Atlanta, Georgia. He has more than 20 years of experience in information technology and speaks around the world on meta data, data architecture and information technology. Stephens recently earned his Ph.D. in information systems and has more than 70 publications in the academic, professional and patent arena. You can reach him via e-mail at Todd@rtodd.com or to learn more visit http://www.rtodd.com/.

Solutions Marketplace
Provided by IndustryBrains

Verify Data at the Point of Collection: Free Trial
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Data Mining: Levels I, II & III
Learn how experts build and deploy predictive models by attending The Modeling Agency's vendor-neutral courses. Leverage valuable information hidden within your data through predictive analytics. Click through to view upcoming events.

Click here to advertise in this space

E-mail This Article E-Mail This Article
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.