Ask the Experts
Editor's note: This is a sample of the questions asked of and answered by our illustrious panel of experts. Find questions archived by topic at http://www.dmreview.com/editorial/online/ate_archive.cfm.
Q: What would you recommend for a cookbook/method to set up a central business intelligence competence center in an organization that used to allow decentralized development of reporting and BI solutions?
Sid Adelman's Answer:
Look at Corporate Information Factory by Bill Inmon, Claudia Imhoff and Ryan Sousa and the Business Intelligence Roadmap by Larissa Moss and Shaku Atre.
Tom Haughey's Answer:
For the following answer, I will have to make assumptions about the meaning of the term, "central business intelligence competence center." I will assume it means not just a set of tools and a team of people that can be used centrally, but also a more centralized data warehouse itself.
First, you need to formulate a strategy and have a plan. The steps to do this are well-established, as follows:
- Objectives: Define the objectives of the central business intelligence competence center
- Future state: Define what it should look like in the future and what the future vision is.
- Assess the current environment: Evaluate the current state against objectives, current needs and industry standards.
- Determine the gaps between the future state and the current state: There will be shortcomings and the strategy and plan must address these.
- Create a migration plan: How will you get there from here. This plan needs to be pragmatic and progressive, achieving results in short increments.
While all these steps are important, it is critical to have a future vision of what you want your warehouse and your organization to look like.
I recommend that you do steps two through four over four main areas, often called BIAT, standing for business process, information, application and technology. Business process covers large business functions; information includes data subject areas; applications are the computer and manual systems that support the processes; and the technology covers the hardware and software resource types.
The next critical thing is a sensible migration plan to get you from your current state to your intended future state. Keep in mind several broad principles.
- Nobody gets there all at once.
- Create a three-year plan.
- Divide the work into short increments that are doable. This will reduce risk by ensuring you have realistic, achievable projects.
- Deliver the work at short intervals. This will improve ROI by giving you early payback, which will also win you user support because they will be getting short-term value out of the BI solution. The delivery of the first release can be three to six months, with all subsequent releases being delivered quarterly. No longer than that.
- Early delivery of a prototype. It is important to ensure that the solutions work right. Early delivery of a prototype provides that confidence because the users get to test the database before it is delivered to ensure it meets requirements.
Centralized Data Warehouse Environment
It seems to me that a more centralized data warehouse solution will be a big part of your future state. Because by definition you have a decentralized environment, two significant tasks will almost certainly be necessary: some redesign and some replatforming. You want your new environment to be a success, but you want it to survive. It is likely that your current environment will not have sufficient performance capabilities, storage and scalability to support a centralized environment. An investment in new robust technology may be essential. If you currently have a scalable platform, you are fortunate and can grow it to meet needs. In my opinion, parallel technology is the most scalable and the most performance oriented, so I would go with the latest version of tha t from your preferred vendor. If you do not have or know a vendor that could deliver this, then a vendor evaluation and even a shoot-out among vendors will be necessary. For a centralized environment to work well, common masterfiles are essential. Master files will provide many of your dimensions. This means that dimension tables should be centrally maintained. This is also important to a decentralized environment but is often ignored there and individualized dimensions populated to each data mart. Of course, feeding consistent copies of dimensions and facts across data marts is a better answer even in a decentralized environment and this is often called "conformed dimensions." Surprisingly, in a centralized data warehouse environment, there is less need for conformed dimensions because of this: the central database in a centralized data warehouse (DW) environment will ideally have one copy of dimensions and thereby there is nothing to conform. Conformity requires at least two copies, which is why conformity is so essential in the typical decentralized business intelligence (BI) environment. Of course, even in a centralized DW environment, BI applications are often distributed, and the dimensions within them must conform. They should be drawn from the centralized DW.
How to Start
Many possibilities exist for developing the centralized environment. This is where the redesign concept I alluded to earlier comes in. Which solution is best significantly depends on your existing situation, so you will have to decide. Here are some alternatives:
Adapt an existing data mart. Upgrade an existing data mart to become the new centralized data warehouse database. If you have any existing robust data mart database, this can be a good starting point. I think that this would have to be one that is currently storing data at the transaction level. The main database needs to be flexible and general purpose, and it can only be that if it stores relevant detail. I'm suggesting transaction detail is the best choice. This data mart can progressively be enhanced with new types of data, new sources, new dimensions, etc. This solution could work even if you have to adjust the granularity of the data in certain ways. An example of adjusting the granularity could be as follows: you currently store data at the transaction level; you expect to add other sources to the database; increase the key to include source system (or company or country, whatever you need).
Create a new data model. Design a new structure capable of supporting your long-term needs. This would be required if no data mart (or set of data mart) exists that can be enhances, as described above. I am not saying do all of this all at once. It can be done incrementally. Here's one way. First, create a high level model of the data. Then, divide the model into subject areas. Next, prioritize the subject areas. Then, drive the prioritized areas into full detail and implement them one by one - in quarterly releases.
Merge two or several existing data marts into one. This is a synchronization, not just a consolidation, process. Take what is in several related data marts, adjust the data model to accommodate the synchronized requirements, and merge the marts into one. This usually requires a new data model, not just a consolidation of existing structures. Whenever you adjust an existing database (and even if you design a new one), make sure you consider data requirements that are wider and deeper than the original databases and than requirements indicate. This holds true regardless of the approach you take. People always think later about things they need that go beyond original requirements.
Joe Oates' Answer:
I don't know of a cookbook that can tell you how to set up a central business intelligence competency center but I can share some guidelines.
First of all, you need to determine the scope of the competency center. Is it mainly to standardize presentation tools? Does it also include the goal of being the "go to" group that provides training, best practices, project management, and guidance in getting BI projects started and implemented?
A BI competency center requires the staff to be experienced individuals who collectively have been successful in all aspects of BI. Additionally, the staff of the BI competency center should be in one location. This way, there is a focused group with the organizational backing required to manage implement and support complex cross-business unit BI projects.
An effective BI competency center requires planning, management commitment demonstrated by an adequate budget, the willingness to obtain the right people with the appropriate skills and experience. Most organizations cannot fully staff the BI competency center internally. Management must be willing to also go outside the organization to find the right people when necessary to make a BI competency center successful.(Posted February 7, 2006)
For more information on related topics visit the following related portals...
Business Intelligence (BI) and
DW Administration, Mgmt., Performance.
Sid Adelman is a principal in Sid Adelman & Associates, an organization specializing in planning and implementing data warehouses, in data warehouse and BI assessments, and in establishing effective data architectures and strategies. He is a regular speaker at DW conferences. Adelman chairs the "Ask the Experts" column on www.dmreview.com. He is a frequent contributor to journals that focus on data warehousing. He co-authored Data Warehouse Project Management and is the principal author on Impossible Data Warehouse Situations with Solutions from the Experts. His new book, Data Strategy, is scheduled for publication this year. He can be reached at 818-783-9634 or email@example.com. Visit his Web site at www.sidadelman.com.
Tom Haughey is the president of InfoModel LLC, a training and consulting company specializing in data warehousing and data management. He has worked on dozens of database and data warehouse projects for more than two decades. Haughey was former CTO for Pepsi Bottling Group and director of enterprise data warehousing for PepsiCo. He may be reached at (201) 337-9094 or via e-mail at tom.haughey@InfoModelUSA.com.
Joe Oates is an internationally known speaker, author and consultant on data warehousing. Oates has more than 30 years of experience in the successful management and technical development of business, real-time and data warehouse applications for industry and government clients. He has designed or helped design and implement more than 30 successful data warehouse projects. He can be reached at firstname.lastname@example.org.