Set Your Boundaries

Last month, I urged you to pause briefly before charging forward on your ambitious data warehousing/business intelligence (DW/BI) project. You were supposed to use this pause to answer a checklist of major environmental questions regarding business requirements, quality data and whether your organization is ready to attack the hard issues of integration, compliance and security.

• Tie the rolling operational results to the general ledger (GL).
• Implement effective compliance.
• Identify and implement all the key performance indicators (KPIs) needed by marketing, sales and finance and make them available in the executive dashboard.
• Encourage the business community to add new cost drivers to our system requirements so that they can calculate activity-based costing and accurate profit across the enterprise. And while we are adding these cost drivers, we’ll work out all the necessary allocation factors to assign these costs against various categories of revenue.
• Identify and implement all the customer satisfaction indicators needed by marketing.
• Seamlessly integrate all the customer-facing operational processes into a single coherent system.
• Promise to use exclusively the front-end, middleware and back-end tools provided by the enterprise resource planning (ERP) vendor whose worldwide license was just signed by our CEO.
• Be the first showcase application for the new service-oriented architecture (SOA) initiative, and we’ll implement, manage and validate the new infrastructure.
• Implement and manage server virtualization for the DW/BI system. And this new system will be “green.”
• Implement and manage the storage area network (SAN) for the DW/BI system.
• Implement and manage security and privacy for all data in the DW/BI system, including responsibility for the LDAP directory server and its associated authentication and authorization functions. We’ll also make sure that all data accesses by the sales force in the field are secure.
• Define the requirements for long-term archiving and recovery of data looking forward 20 years.

Looking at this list of promises all at once, you might wonder who in their right mind would agree to them. Actually, I am much more sympathetic than it may seem. You must address these topics because they are all key facets of the DW/BI challenge. But if you gave the answers as literally stated, you have lost control of your boundaries. You have taken on far too much, you have made promises you can’t deliver and your business clients and enterprise bosses have abrogated or avoided key responsibilities that they must own. More seriously, even if you think you can deliver all these promises, you are not in a powerful enough position in your enterprise to make all these results happen.

You don’t have to be a grumpy curmudgeon to be a good DW/BI system manager. This isn’t about saying no to every possible responsibility. You will be doing your enterprise a favor by alerting and educating your business users and enterprise bosses to the appropriate boundaries of responsibilities. You can still be an enthusiastic advocate, as long as your boundaries are clear. Let’s describe the key boundaries.

Boundaries with the business users. Your job is to find the business users, interview them and interpret what they tell you into specific DW/BI deliverables. You must assemble a findings document that describes the results of the interviews and how you interpreted what the business users told you. Their responsibility is to be available for the interviews and to put energy into describing how they make decisions. Later in the process, the business users have a responsibility to provide feedback on your findings. You cannot attempt to define business requirements without a full 50 percent participation from the business user community.

Your job is not over after the first round of interviews. You must encourage ongoing business user feedback and suggestions, and also educate the business users as to the realities of system development. View this as a mutual learning process. In the latter stages of development of a DW/BI system, you simply cannot add new KPIs and especially new data sources to the project without slipping the delivery date. You cannot suddenly change a batch-oriented system into a real-time pipeline. Your business users must be understanding and trusting partners of the DW/BI system development, and they have to understand the costs of sudden new requirements. Bottom line: business users must become sophisticated observers of the DW/BI development process and know when it is inappropriate to change the scope by adding new KPIs, new data sources or new real-time requirements.

Boundaries with finance. Of the promises you made, several should be the responsibility of finance. You should never agree to implement cost allocations, even if the “profit system” is your main responsibility. Not only are cost allocations very complex, but the assignment of costs to various revenue-producing departments is bad news politically. In this case, finance should work out the logical and political implications of the cost allocations, and you can quietly implement them.

You also should never agree to tie rolling operational results to the GL. In dimensional modeling parlance, you can’t make this happen because the GL dimensions, such as organization and account, can’t be conformed to the operational dimensions, such as customer and product. Also, special GL transactions, such as journal adjustments done at the end of the month, often cannot be put into an operational context. Again, you need to hand this issue back to finance and wait for a solution from them.

Boundaries across organizations. These days it is hard to find anyone who argues against integration of all your data assets under the DW/BI umbrella. But this challenge is 70 percent political and only 30 percent technical. Your executives must establish a corporate culture that sends a very clear message to all the separate departments that they must come together to agree on common dimensional attributes, key performance metrics and calendars. Your executives must lead the way before you can do your job.

Boundaries with legal. In the early 90s, we often lamented that the data warehouse wasn’t seeing widespread use. Well, now we have the opposite problem. A big piece, shall I say headache, of being taken very seriously is providing adequate security, privacy, archiving and compliance across the DW/BI system. But you can’t do anything until you understand your enterprise’s policies. You must not define these policies yourself. You can lose your job and go to jail if you get these wrong. Go to your legal department with a list of areas where you need firm guidance.

Boundaries with IT. Strangely, one of the most important boundaries you must maintain is IT. You should be able to rely on other groups within IT for storage (either SAN or NAS), server virtualization, LDAP server maintenance, authentication technologies and providing new infrastructure such as SOA.

Most of us in the DW/BI business are natural salespeople. We are evangelists for the use of our systems because we really believe they will benefit the business. But we need to be conscious of trying to please the client too much. Ultimately, the DW/BI system will be much more successful if all the other parties described in this column are equal, responsible partners.

Ralph Kimball is the founder of the Kimball Group and Kimball University where he has taught data warehouse design to more than 10,000 students. He is known for the best selling series of data warehouse "Toolkit" books. He started with a Ph.D. in man-machine systems from Stanford in 1973 and has spent the last 34 years designing systems for end users that are simple and fast. You can reach him at ralph@kimballgroup.com.

For more information on related topics, visit the following channels:

• Best Practices/Benchmarking
• DW Basics
• DW Design, Methodology
• Data Management