FREE DM Review Site Registration!
Sign-up today and access DM Review on the Web!

Your FREE registration entitles you to:
FREE email newsletters

FREE access to all DM Review content

FREE access to web seminars, resource portals, our white paper library and more!

   

When we deliver a data warehouse solution, the users tell us it doesn’t match what they need. What do we need to do?

Question: We don’t seem to be able to get it right. When we deliver a data warehouse solution, the users tell us it doesn’t match what they need. We went through what we thought were the right steps to gather the business requirements, but either the requirements have changed or we never really understood what they wanted. Help!

 

Chuck Kelley’s Answer: How long did it take before you delivered the data? My guess is that it was longer than three months. Within three months, the requirements generally change and since you did not ask during the development if the requirements changed, you were not notified.

 

I suggest putting together a team of folks (business and technical) and a development process that take the business requirements (small number of them) and product something just about every month to make sure that you are on track. If they keep saying "yes," then they won’t say no at the end.

 

Larissa Moss’s Answer: I strongly suggest an agile approach. I call it "extreme scoping." The first thing to do is to break the habit of delivering an application with one project. While most organizations realize, by now, that they cannot build an entire data warehouse (DW) all at once, they don’t realize that they cannot and should not build an application all at once either.

 

Why? Some answers are:

 

  • The requirements are most likely unstable;
  • The scope is probably too large or the deadline is too tight;
  • Some technology components might be unproven;
  • Data integration complexity is most likely not fully understood;
  • The quality of the source data probably has not been vigorously profiled because many (if not most) business rules are not documented or even known;
  • The project team may be too small, too large or untrained; and
  • Business users are probably too busy to provide continuous feedback.

With a project like this, following a traditional methodology and staying on paper for weeks or months is the kiss of death. You have to get physical as quickly as possible. That means prototyping, prototyping and more prototyping.

 

How do you prototype an entire application without falling back into the waterfall sequence of requirements, analysis, design, coding and testing? You break the application into multiple software releases. The scope of each software release will contain only a small portion of the application requirements, hence the name extreme scoping. Each software release becomes a project. Each project is developed using an iteractive prototyping approach. Each prototype is time-boxed (anywhere from 10 to 60 days). Most software releases should produce a production-worthy deliverable (although, there are occasions where you want to further refine refactor, a deliverable in the next software release, before putting it into production). After several software releases, you will have a completed and fully-functioning application.

The quality of the application will be higher with this approach than with a traditional waterfall approach, because mistakes can be found and fixed early in the development process. With the extreme scoping approach, you have sliced the development process vertically across the application scope instead of horizontally across the development phases. This allows:

  • The requirements to be refined with each software release until they become stable;
  • The scope of each software release to be small enough to fit into a tight deadline;
  • Technology components to be evaluated early on;
  • Data integration complexity to be handled in small increments;
  • The quality of the source data to be vigorously profiled, because you will incrementally find and document the business rules;
  • The project team to be small or even in training during the early software releases with minimal impact on the entire application;
  • Business users to be more involved in prototyping activities than on traditional projects where they hand over requirements and wait for user acceptance testing; and
  • Many mistakes or misunderstandings, that cannot be found on paper and only found once the entire application is in production, to be found and corrected during the development process, before the entire application is completed.

For more information, read my article "Extreme Scoping" in the Library EIMI Archives or attend one of my seminars on agile project management occasionally offered publicly at TDWI conferences.


Chuck Kelley is an internationally known expert in database and data warehousing technology. He has 30 years of experience in designing and implementing operational/production systems and data warehouses. Kelley has worked in some facet of the design and implementation phase of more than 50 data warehouses and data marts. He also teaches seminars, co-authored four books on data warehousing and has been published in many trade magazines on database technology, data warehousing and enterprise data strategies. He can be contacted at chuckkelley@usa.net.

Larissa Moss is founder and president of Method Focus Inc., a company specializing in improving the quality of business information systems. She has more than 20 years of IT experience with information asset management. Moss is coauthor of three books: Data Warehouse Project Management (Addison-Wesley, 2000), Impossible Data Warehouse Situations (Addison-Wesley, 2002) and Business Intelligence Roadmap: The Complete Project Lifecycle for Decision- Support Applications (Addison-Wesley, 2003). Moss can be reached at methodfocus@earthlink.net.

For more information on related topics, visit the following channels:



Industry Vendors