Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Information Management:
Straight Talk: Data Warehousing and Offshore Development

  Column published in DM Review Magazine
November 2004 Issue
  By Bill Inmon

Two phenomena are occurring simultaneously today - the advent of corporations building data warehouses and the movement of development of IT projects offshore. It is therefore a natural question to ask: Are we going to be developing data warehouses offshore? However, there are many good reasons why offshore development of data warehouses is not in the cards.

Why is it that data warehouses do not lend themselves to offshore development? Why are data warehouses different from other types of IT projects and structures?

From a tactical perspective, data warehouses deal with and operate on data, not processes. It is processes that are economically built offshore. Data is a different story altogether. Whereas you can specify processes down to a fine degree, specifying, defining and developing data for a data warehouse is a different beast altogether.

Another factor making data warehouses poor candidates for offshore development is that there is no such thing as a generic data warehouse. Each data warehouse is different for each company - even companies in the same line of business. You cannot make cookie-cutter data warehouses.

Another critical tactical reason why you should not develop data warehouses offshore is that the very nature of data warehouses is an environment in which requirements are never firm. Data warehouses are built for end users. Additionally, end users operate in a mode of not knowing what they want. End users operate in a mode of, "Give me what I say I want, then I can tell you what I really want." They also operate in a mode of discovery. They say, "Ah - now that I see what the possibilities are, I can tell you what I really want. However, until I can see what the possibilities are, I really can't tell you what I want." This very fluid development environment does not lend itself to offshore development because offshore development is geared for very rigid requirements.

There is also another very different set of motivations for not building a data warehouse offshore, stemming from the fact that a data warehouse is an essential architectural component of the corporation. Once the data warehouse is built, various other processes emanate from it. The data warehouse becomes the hub for many other kinds of processing including data mining and exploration, data marts, operational data stores, decision support system applications and project marts. Once built properly, the data warehouse becomes the central nervous system for the intelligence of the corporation. Therefore, the data warehouse must be built architecturally in a very proper manner. Some of the more important architectural considerations of a data warehouse include:

Size. If my data warehouse starts to grow beyond approximately ten terabytes, the cost of the infrastructure becomes a burning issue. Therefore, data warehouses must be built so that they can support very large amounts of data without the cost of the infrastructure obviating the benefits of building the data warehouse in the first place.

Unstructured data. One of the advantages of a data warehouse is it can support unstructured data as well as structured data. However, the warehouse must be built properly in order for unstructured data to be properly integrated.

Data mining and exploration. One of the long-term benefits of a data warehouse is that it can support data mining and data exploration. Again, the warehouse must be built properly from the outset in order to be able to take advantage of these kinds of technology.

ODS. One of the most important types of architecture is often the operational data store (ODS). The ODS and the data warehouse need to cohabitate in a complementary and constructive manner. The lines between the ODS and the data warehouse must be constructed properly from the outset.

And there are many other architectural considerations for the building a data warehouse. The immediate set of end-user requirements are only one set of considerations - although certainly an important set. In the long run, the architectural considerations for the building of a data warehouse actually outweigh the immediate end-user requirements.

Therefore, there are many important architectural considerations that need to be factored into the building of a data warehouse. It is fair to say that offshore developers have not reached the levels of sophistication to be able to deal with the architectural considerations of the design and development of a data warehouse.

Another important strategic reason for not offshoring the development of a data warehouse is that - in a way - data warehouses are never complete. As long as the business is growing and as long as the world is changing, there will be a need for adding to or altering a data warehouse. Having the ability to alter the data warehouse in house is very important. Relying on an offshore source for the ongoing maintenance and development of a data warehouse is very risky business.

Yet another reason for not offshoring the development of data warehouses is that the security and privacy of the data is subject to compromise. Data warehouses contain corporations' bedrock data. Allowing that data to get out of control is taking a very serious risk, one where the very core business of the corporation may be compromised. Indeed, in some cases it may actually be illegal to build a data warehouse offshore because of the nature and content of the data.

In some ways, building a data warehouse is like having brain surgery. If a person must have brain surgery, he or she surely wants the brain surgery to be done correctly, where there are the best chances of a successful operation and recovery. You want an experienced brain surgeon, one who has seen abnormal conditions before and knows how to handle them. Having a rookie explore the brain as part of a learning experience introduces a level of risk that is unacceptable in most environments.


For more information on related topics visit the following related portals...
DW Administration, Mgmt., Performance, DW Design, Methodology and Outsourcing.

Bill Inmon is universally recognized as the father of the data warehouse. He has more than 35 years of database technology management experience and data warehouse design expertise. His books have been translated into nine languages. He is known globally for his seminars on developing data warehouses and has been a keynote speaker for many major computing associations. For more information, visit www.inmongif.com and www.inmoncif.com. Inmon may be reached at (303) 681-6772.

Solutions Marketplace
Provided by IndustryBrains

Verify Data at the Point of Collection: Free Trial
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Data Mining: Levels I, II & III
Learn how experts build and deploy predictive models by attending The Modeling Agency's vendor-neutral courses. Leverage valuable information hidden within your data through predictive analytics. Click through to view upcoming events.

Click here to advertise in this space

View Full Issue View Full Magazine Issue
E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.