FREE DM Review Site Registration!
Sign-up today and access DM Review on the Web!

Your FREE registration entitles you to:
FREE email newsletters

FREE access to all DM Review content

FREE access to web seminars, resource portals, our white paper library and more!

   

Publisher reserves the right to serve qualified requesters only.

Data Integration Will Break Out of the Silo: BI and DW Trends, 8 for ’08

BI Briefs

This is part of a series of columns on business intelligence (BI) and data warehousing (DW) trends for 2008.

 

How’s this for irony? A lot of integration projects create more silos!

 

This happens because companies implement integration technology and products on a project-focused basis. The integration projects concentrate on specific applications and processes, in other words they integrate silos. Each tactical integration task is completed, and eventually the company discovers that they need to integrate the silos created with their integration projects.

 

The bad news is that many companies have already created these integration silos. The business groups that invested in these integration projects are now back to using spreadsheets, or data shadows systems, to reconcile the conflicting numbers coming out of these silos.

 

The good news is that many IT and business groups are educated consumers. They have seen the result of their myopic integration efforts and understand they need to change their ways.

 

The Integration Sea Change

 

What should you do if you find yourself in this situation? Step back from the tunnel vision of tactical data integration projects and design an overall integration architecture. After you have this overall architectural blueprint you can then design your individual projects, fitting them into this overall blueprint. Just like you hand your house blueprint to various contractors so that everything fits together, you need your data integration blueprint to get things to work together too. This blueprint encompasses architectures for data, technology, product and information (business data transformed).

 

Enterprise data management (EDM) is the blueprint (or holistic approach in consultant lingo) that people should be striving for. Integration efforts, be they enabled using enterprise application integration (EAI), service oriented architecture (SOA), enterprise information integration (EII) or extract, transform and load (ETL) technologies, all involve integrating data.

 

Data integration processes involve mapping one or more data sources to a data target and transforming the data along the way. Regardless of whether you are using messaging, services or batch-driven technologies you are performing the same processes, just the transport is different.

 

Rather than taking a common approach, most integration projects start with an entirely different set of technologies or products. However, these products and technologies overlap and, more importantly, the data they integrate overlaps. The result is silos and business people manually reconciling the data they just spent a lot of money and effort integrating.

 

Fortunately, if your company is in this situation you are not alone (misery loves company) and great options have emerged in the market. First, there are best practices to design your data integration framework (DIF) or blueprint. Second, there are proven program and project approaches that incrementally build an EDM and migrate from your silos.

 

Finally, some of the top ETL vendors have recognized the need for a comprehensive approach and have transformed their ETL products to data integration suites. These suites have expanded beyond batch-oriented ETL to include EAI, EII and SOA. In addition, it is becoming increasingly common for these suites to offer data profiling and data quality functions. No longer are you forced to buy separate best-of-breed products supporting niche technologies. You can find products that will support your DIF.

 

The market leaders, according to Gartner, Inc. are Informatica and IBM.1 Both companies have expanded their data integration capabilities through acquisitions; most notably IBM’s Information Server obtained a significant portion of its functionality from its acquisition of Ascential Software.

 

Forrester Research concurs with the leaders but feels that Oracle (via its Sunopsis acquisition) and SAP (via its Business Objects acquisition) are catching up.2 I’d also watch smaller competitors such as Pervasive Software and Sybase that are busy putting together their own suites.

 

Change is in the air! Companies need to shift from their tactical, project-focused approach to integration to an enterprise perspective. Many in the industry are realizing that they need to make their integration efforts provide comprehensive and consistent business information, not create yet another silo to then reconcile with their other silos. The will and ability (products) are in place. This trend will gain more momentum this year providing case studies and knowledge to expand beyond technology’s early adopters.

 

References:

 

  1. Ted Friedman, Mark A. Beyer and Andreas Bitterer. “Magic Quadrant for Data Integration Tools, 2007.” Gartner, 2007.
  2. Rob Karel. “The Forrester Wave: Enterprise ETL, Q2 2007.” Forrester, May 2007.

Rick Sherman has more than 20 years of business intelligence and data warehousing experience, having worked on more than 50 implementations as a director/practice leader at PricewaterhouseCoopers and while managing his own firm. He is the founder of Athena IT Solutions, a Boston-based consulting firm that provides data warehouse and business intelligence consulting, training and vendor services. Sherman is a published author of over 50 articles, an industry speaker, a DM Review World Class Solution Awards judge, a data management expert at searchdatamanagement.com and has been quoted in CFO and Business Week. Sherman can be found blogging on performance management, data warehouse and business intelligence topics at The Data Doghouse.You can reach him at rsherman@athena-solutions.com or (617) 835-0546.

In addition to teaching at industry conferences, Sherman offers on-site data warehouse/business intelligence training, which can be customized and teaches public courses in the Boston area. He also teaches data warehousing at Northeastern University 's graduate school of engineering.

 

For more information on related topics, visit the following channels:



Industry Vendors