One of our customers outsourced their data warehouse. Now they want to bring it back in house. What all factors should be considered and what are the risks?

Q:  

One of our customers outsourced their data warehouse. Now they want to bring it back in house. What all factors should be considered and what are the risks?

A:  

Sid Adelman's Answer: The biggest risk seems to be the lack of intelligent decision making capability of your customer's management to have misunderstood the issues and underestimated the problems of outsourcing before they signed the contract. My guess is they will be making comparably inappropriate decisions as they insource the system. Having said that some other risks are:

• Your customer probably lost most of their subject matter experts and lost their intellectual capital when they outsourced. The good folks are gone and ask yourself, why would they be ready to return to such a diminished capacity organization.
• You as the outsourcer are probably not much incented to help your customer reverse the outsourcing process and so the outsource unwinding will be ugly.
• The morale and reputation of your customer would be at an all-time low making recruitment very difficult.

Outsourcing is either the result of the bean counters being powerful enough to force through the decision or the result of an ineffectual IT organization which had been unable to support the business. The question is what has changed?

Danette McGilvray's Answer: I would first find why they want to bring the data warehouse back in house. What are all the reasons for this decision? I suspect one of the reasons is that the business has concerns with the quality of the information coming out of the warehouse. I will focus my comments on information quality since it is often overlooked completely or postponed "until the transition is complete."

The quality questions were probably not adequately addressed during the initial development and implementation of the warehouse. A common - yet ineffective approach - is to say "Let's just get the warehouse built. We will take care of the information quality after we are in production." What is left is a warehouse that was loaded with poor quality data. To further add to the problem, ongoing processes that load, maintain, and use the data were not developed with data quality in mind. The result is knowledge workers who will not use the warehouse because they do not trust the data.

Along with the common attitude just expressed of addressing quality after implementation is another perspective that includes the statement, "Yes, we have a data quality team. It is working in parallel with the project." This statement indicates those responsible for data quality are actually separate from the warehouse project. Don't repeat that mistake. Any project team formed to bring the data warehouse back in house should include those who bring the data quality point of view to the project. The data quality issues and tasks must be integrated into the overall project tasks and timeline.

What do I mean by the data quality point of view? Look at the life cycle of the information coming into the warehouse - planning for the data (this is where the project team comes in); obtaining the data (from source systems); applying the data (by knowledge workers, for example); maintaining or updating the data; and disposing (archiving or deleting) the data. Use the acronym POAMD to help you remember the information life cycle. This is similar to the four essential database operations: CRUD (Create-Read-Update-Delete). However, the information life cycle approach also takes the business needs, processes and people viewpoints into account along with the technology aspect.

For example, in the obtain phase data comes into the warehouse from a source system. Do you have processes for validating the data? What happens if the data does not pass the validation tests? Do you have processes and responsibilities in place to handle content problems? Are there people on both the warehouse and source system sides who are responsible for communicating with each other and resolving the issues? For the maintain phase, what if the source system finds out several weeks after a load that an incorrect file was sent? Who takes care of the issue? Can you catch the problem at load time? Do you have metrics in place to monitor key information? These questions provide only an introduction into what needs to be considered.

To touch on the question of risk - if data quality is one of the concerns, I strongly recommend completing some assessments to determine the extent of the quality problem. The results will help you estimate the time and cost needed to make the warehouse acceptable and useful to the business. The assessments could include such activities as data profiling, user interviews, process analysis, etc. The assessments you choose depend on the nature of the data quality concerns. The risk if you do not do this? You will not factor in some of the most important activities needed to ensure the data warehouse is a trusted source for the business. Yes, there is a cost to these activities. But a bigger cost is the business not using the warehouse because they have no confidence in the information.

Clay Rehm's Answer: Since the data warehouse has already been developed, it will most likely be stored in a specific RDBMS. Is that RDBMS supported in your organization? If not, how easy will it be to perform a data conversion to your database?

The warehouse is being populated through ETL processes and the business rules should be described there. Will you be able to access and modify those rules? Does your organization use the same ETL tools that the outsourcer used?

The warehouse should have had data stewards or data owners. Who will those owners now be? Do they understand their roles in data quality, security, access, etc.?