DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search
advertisement

Resource Portals
Analytic Applications
Business Intelligence
Business Performance Management
Data Quality
Data Warehousing Basics
EDM
ETL
Storage
More Portals...

Advertisement

Information Center
DM Review Home
Online Conference & Expo
Web Seminars & Archives
Newsletters
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News

General Resources
Bookstore
Industry Events Calendar
Vendor Listings
White Paper Library
Glossary
Software Demo Lab

General Resources
About Us
Press Releases
Awards
Media Kit
Reprints
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

The new trend has been demanding real-time data from the data warehouse.

    Ask The Experts published in DMReview.com
July 8, 2004
 
  By Sid Adelman and Chuck Kelley and Les Barbusinski

Q:  

The new trend has been demanding real-time data from the data warehouse: 1) to generate operational report at high frequency 2) to provide a golden source (real-time reference data) for multiple OLTP Applications. Which is the right ETL tool to extract data from multiple OLTP sources with no latency and importantly no performance impact on OLTP. RDBMSs are Informix, Oracle and DB2 UDB?

A:  

Sid Adelman's Answer: To those of you who believe that the support garment industry can produce pantyhose in a one-size-fits-all package, a real-time data warehouse might be appealing but you will deliver a compromise that is sometimes baggy and sometimes stretched to the breaking point. Analysts and knowledge workers need a stable source of data. When they run a query and run it again 10 minutes later, they want the results to be the same. Go back to the basics and learn the purpose of a data warehouse and don't let the current trend push you into a foolish implementation.

Chuck Kelley's Answer:In a word, "never." There will always be some degradation or latency. But most likely both will occur.

Les Barbusinski's Answer: Both Informatica and Ascential offer real-time/zero latency ETL engines that work with a variety of relational databases and packaged applications (e.g., ERP, SCM, CRM, etc.). How much of an impact these have on the OLTP environment depends on the implementation.

The only other near real-time ETL option I've seen is to use database replication to create a copy of the operational database on a separate server, and have an ETL tool monitor that replicated copy for changes that need to be posted to the data warehouse. Obviously, this method introduces latency factors that can make the data several minutes old before it gets into the data warehouse, but it has the benefit of having near-zero impact on the OLTP environment. Hope this helps.

,


Sid Adelman is a principal in Sid Adelman & Associates, an organization specializing in planning and implementing data warehouses and in establishing effective data architectures and strategies. He jointly developed a methodology, MapXpert for Data Warehouse, that provides a master plan for implementing a data warehouse. Adelman is a regular speaker at The Data Warehouse Institute and IBM's DB2 and Data Warehouse Conference. He chairs the "Ask the Experts" column on DMReview.com and is a founding member of the Business Intelligence Alliance. Adelman is a frequent contributor to journals that focus on the data warehouse. He coauthored Data Warehouse Project Management with Larissa Moss and Impossible Data Warehouse Situations: Solutions from the Experts. He can be reached at (818) 783-9634 or sidadelman@aol.com. Visit his Web site at www.sidadelman.com.

Chuck Kelley is a senior advisory consultant for Navigator Systems (www.navigatorsystems.com), an independent consulting firm fully focused on corporate performance management (CPM) and BI solutions. Kelley is an internationally known expert in database and data warehousing technology. He has more than 25 years of experience in designing and implementing operational/production systems and data warehouses. Kelley has worked in some facet of the design and implementation phase of more than 45 data warehouses and data marts. He also teaches seminars, coauthored three books on data warehousing and has been published in many trade magazines on database technology, data warehousing and enterprise data strategies. He can be contacted at ckelley@navigatorsystems.com.

Les Barbusinski is vice president of technology and co-founder of Digital Symmetry, LLC, a consulting firm that specializes in the design and development of data warehousing and business intelligence solutions. He has more than 20 years of experience in data warehouse and operational systems development and provides hands-on expertise in data warehouse design, development and project management. Les can be reached at dwexpert@dsym.com.

Solutions Marketplace
Provided by IndustryBrains

Test Drive the Standard in Data Protection
Double-Take is more affordable than synchronous mirroring and enables you to recover from an outage more quickly than tape backup. Based upon the Northeast blackout and the west coast wild fires, can you afford to be without it?

MicroStrategy Data Warehousing Tools
Access any size data warehouse with our award-winning software, MicroStrategy 7i. Try our award-winning software for 30 days FREE!

Help Desk Software Co-Winners HelpSTAR and Remedy
Help Desk Technology's HelpSTAR and BMC Remedy have been declared co-winners in Windows IT Pro Readers' Choice Awards for 2004. Discover proven help desk best practices right out of the box.

Data Mining: Strategy, Methods & Practice
Learn how experts build and deploy predictive models by attending The Modeling Agency's vendor-neutral courses. Leverage valuable information hidden within your data through predictive analytics. Click through to view upcoming events.

Click here to advertise in this space


E-mail This Ask The Experts E-Mail This Ask The Experts
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Advertisement
advertisement
Site Map Terms of Use Privacy Policy

Thomson Media

2004 The Thomson Corporation and DMReview.com. All rights reserved.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.