Data Warehousing Lessons Learned:
Data Mart Consolidation Drivers
Data marts rarely grow up to be an enterprise data warehouse, regardless of the volume point reached. Many firms can benefit from data mart consolidation, though it is not an unconditionally positive move for all. Enterprises are driven to consider the advantages of consolidating diverse data marts into an enterprise data warehouse for a variety of reasons. The list of drivers includes business, organizational, operational and technological.
Mergers and acquisitions continue apace. The consolidation of product and customer dimensions enables cross-selling and up- selling in customer relationship management (CRM) as well as substituting information for inventory in product and supply chain applications.
The requirement for intelligent information integration at an enterprise level is exemplified by business intelligence demands such as knowing the lifetime value of a customer. The lifetime value of a customer cannot be known without aggregating a lifetime of transactions across all the customer's touchpoints. That requires the organization to move beyond departmental data mart silos to gain an enterprise perspective.
The need for operational efficiencies in the IT utility is seemingly unending and urgent. Centralization provides for greater efficiencies and reduced coordination costs in managing the decision-support infrastructure. Data center consolidation invites the consolidation of the data mart applications that are supported by the servers. The benefits of data center consolidation are mirrored by data mart consolidation. These two kinds of initiatives go hand in hand and reinforce each other as demonstrated by the work of my colleagues Colin Rankine and Brad Day at Giga.
The benefits of resource sharing and management are more widely available than ever before thanks to the build-out of storage technology infrastructure in the form of storage area networks (SANs) and network attached storage (NAS). As indicated, data marts rarely grow up to become data warehouses, regardless of the volume of data involved. However, after any data mart or warehouse reaches either a terabyte of raw data or 1,000 data structures, organizations face a storage technology problem to manage all the space allocations efficiently. Centralized and policy-based storage resource management (SRM) is an important dimension of any consolidation solution, and it is best undertaken using data center resources and professional practices and staff.
Many enterprises have taken an incremental approach to data warehousing, implementing data marts step by step. One thing in particular that will require work is the positioning of the consolidation effort on a spectrum of options that range from data integration to the physical collocation of information assets on the same server or even within different servers in the same location. Those firms that followed Giga's recommendation - design the data warehouse; implement the data mart - will have an easier time consolidating precisely because a consistent unified design exists across data mart boundaries. Those that implemented silos helter-skelter will find that consolidation also requires significant design rework, potential platform conversions and product migrations in addition to infrastructure centralization. If an enterprise believes it is being moved by the drivers identified in this column, it should undertake a data mart consolidation-readiness assessment. Such an assessment looks at the inventory of information assets likely to be affected by consolidation, assembles a cross- functional team and builds the business case for such an undertaking.
When Giga refers to data mart consolidation, the intended meaning is "data mart integration," not merely physical collocation of servers and data warehousing infrastructure. This is represented toward the right-hand side of Figure 1. As indicated, data mart consolidation extends along a spectrum of alternatives.
Figure 1: Data Mart Consolidation Spectrum of Alternatives
Those enterprises considering data mart consolidation should consider where the organization lies on the continuum of alternatives represented in Figure 1. Giga suspects that the majority of data marts are silos - islands of information, not enterprise (federated) data marts incrementally implemented according to a central plan. Those enterprises with data marts that are physically dispersed and lacking in a consistent, central design of customers, products, etc. have the most to gain from a data mart consolidation initiative. Those enterprises that have implemented data marts with a consistent design will find that benefits are more in the area of operational efficiency than information integration - unless such enterprises have failed to capitalize on the consistent design. In that case, they too will be able to reap information integration as well as operational benefits from consolidation. However, the redesign of the silos to conform to a consistent, unified representation of customer, product, etc. will be an additional cost and effort to be incurred by the project. In short, data mart consolidation seems like an interesting idea. The question is - have any enterprises done it and what were the results? If you have, please let me hear from you about the results at email@example.com. (Replies will be treated confidentially, though the data may be aggregated for purposes of market research.)
For more information on related topics visit the following related portals...
Business Intelligence (BI) and
Lou Agosta, Ph.D., joined IBM WorldWide Business Intelligence Solutions in August 2005 as a BI strategist focusing on competitive dynamics. He is a former industry analyst with Giga Information Group, has served as an enterprise consultant with Greenbrier & Russel and has worked in the trenches as a database administrator in prior careers. His book The Essential Guide to Data Warehousing is published by Prentice Hall. Agosta may be reached at LoAgosta@us.ibm.com.
Provided by IndustryBrains
|Design Databases with ER/Studio: Free Trial|
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.
|SAP & TommorrowNow Support|
SAP & TomorrowNow offer competitor users full support cost savings through 2015 up to 50%. You get the time you need to make informed decisions about future migrations.
|FREE WP - Manage the Data Center from Anywhere!|
Learn how SecureLinx remote IT management products can quickly and easily give you the ability to securely manage data center equipment (servers, switches, routers, telecom equipment) from anywhere, at any time... even if the network is down.
|DeZign for Databases - Database Design Made Easy|
Create, design & reverse engineer databases with DeZign for Databases, a database design tool for developers and DBA's with support for Oracle, MySQL, MS SQL, MS Access, DB2, PostgreSQL, InterBase, Firebird, NexusDB, dBase and Pervasive.
|Data Mining: Levels I, II & III|
Learn how experts build and deploy predictive models by attending The Modeling Agency's vendor-neutral courses. Leverage valuable information hidden within your data through predictive analytics. Click through to view upcoming events.
|Click here to advertise in this space|