I am the quality assurance (QA) manager for the Data Management Services department of the Bank of Montreal.
||Ask The Experts published in DMReview.com
April 17, 2002
I am the quality assurance (QA) manager for the Data Management Services department of the Bank of Montreal. We build data warehouses and data marts as well as do ETL which feeds data into other applications. We test ETL by writing SQL queries against source tables and comparing results to those of the SQL queries against target tables. SQL queries for source tables are rather complex as they are based on a very complex set of business rules. Are you aware of any tools that can help us in this testing process?
|David Marco's Answer: If you are using Ascential DataStage, they have a data quality product calling Quality Manager. If you are using Informatica, I am less certain what their Quality Manager equivalent is. Regardless of the tool, all of the meta data that you are capturing from the SQL queries against the target tables needs to be stored in a meta data repository. A good repository is historical, so all of the data quality will be tracked over time. By storing this in your repository it will greatly aid in your testing.|
For more information on related topics visit the following related portals...
DW Administration, Mgmt., Performance.
David Marco is an internationally recognized expert in the fields of enterprise architecture, data warehousing and business intelligence and is the world's foremost authority on meta data. He is the author of Universal Meta Data Models (Wiley, 2004) and Building and Managing the Meta Data Repository: A Full Life-Cycle Guide (Wiley, 2000). Marco has taught at the University of Chicago and DePaul University, and in 2004 he was selected to the prestigious Crain's Chicago Business "Top 40 Under 40." He is the founder and president of Enterprise Warehousing Solutions, Inc., a GSA schedule and Chicago-headquartered strategic partner and systems integrator dedicated to providing companies and large government agencies with best-in-class business intelligence solutions using data warehousing and meta data repository technologies. He may be reached at (866) EWS-1100 or via e-mail at DMarco@EWSolutions.com.
Provided by IndustryBrains
|Data Validation Tools: FREE Trial|
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.
|Recover SQL Server or Exchange in minutes|
FREE WHITE PAPER. Recover SQL Server, Exchange or NTFS data within minutes with TimeSpring?s continuous data protection (CDP) software. No protection gaps, no scheduling requirements, no backup related slowdowns and no backup windows to manage.
|Speed Databases 2500% - World's Fastest Storage|
Faster databases support more concurrent users and handle more simultaneous transactions. Register for FREE whitepaper, Increase Application Performance With Solid State Disk. Texas Memory Systems - makers of the World's Fastest Storage
|Design Databases with ER/Studio: Free Trial|
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.
|Free EII Buyer's Guide|
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.
|Click here to advertise in this space|