Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

The Next Big Thing for Data Warehouses

  Article published in DM Review Magazine
July 2000 Issue
  By Richard Tanler

One report is common to virtually every enterprise and is eagerly anticipated at the end of each month. The report not only reaches the highest levels of the enterprise's management, but is often the catalyst for business analysis across departments and through-out the enterprise. The report compares actual performance with plan.

Surprisingly, this report is rarely produced from data stored in the data warehouse. The simple reason is that plan data is seldom included in the data warehouse design.

Why do so few data warehouses contain plan data? Or more importantly, what role should the data warehouse play in the business planning process? Should business intelligence applications play a role in the business planning process?

Even current, state-of-the-art business intelligence applications backed by the best data warehouses lack important capabilities required to support the process of decision making and decision implementation. Certainly, business intelligence systems excel in providing users with valuable insight into what happened. Decision-making value comes from the analysis and simulation steps required to evaluate alternative decisions and influence what will happen.

A more significant shortcoming of the data warehouse and business intelligence applications is the support provided for decision implementation. Decision implementation is a collaborative process involving many people throughout the enterprise and often external to the enterprise. Decisions must be approved and communicated to everyone who must take action. In order to support decision implementation, information must flow back into the operational and financial systems that serve as the sources for the data warehouse.

Today, most organizations maintain multiple, unrelated forecasting systems.

The linking of decision making and decision implementation establishes a closed-loop business management process. The data warehouse's role becomes more vital to the entire process of business management.

The data warehouse is the single, integrated source of data for decision making. The next big thing for data warehouses is the support of the collaborative planning functions essential for decision implementation. The most actionable data, uniting marketing, finance and operations, is the demand forecast. A significant change in the forecast affects virtually every business management discipline. The forecast is the language of interdepartmental collaboration.

The Internet makes it easier for suppliers and vendors to collaborate to achieve a new level of supply chain efficiency. An essential goal in this business-to-business collaboration is reaching consensus on the forecast. Again, the demand forecast represents the common language for collaboration among members of the trading community.

Forecasting vs. Budgeting

Business planning has two interrelated activities. Forecasting is the projection of future demand, while budgeting is the consolidation of expense line items. The forecast is used to estimate revenue for budgeting; however, forecasts are also critical to operational systems used by purchasing, manufacturing, inventory management, logistics, etc.

The next big thing is for the data warehouse to be come a read/write tool at the center of your universe. All entries work from this single source, creating a collaborative demand forcast.

Budgeting is an iterative process of entering and consolidating data, generally with predefined financial goals. Executive management establishes the annual revenue and income goals. Departmental managers iterate through revenue and expense estimates, making the appropriate adjustments, to operate within the defined goals.

Virtually every organization has a highly disciplined process - not always an efficient process - for creating and consolidating budgets. Consolidated, the budgets become the enterprise's financial plan. Once the financial plan is approved, it becomes the yardstick against which the enterprise's performance is measured. The details of the budgets are some of the most closely guarded secrets of the enterprise.

The demand forecast is a key input into the budgeting process, since revenue is a function of unit sales, price and discounts. Forecasts are also required to support a wide range of business operations. The forecasting process considers historical trends, market factors and subjective judgment in projecting future sales. Forecasts are frequently updated to reflect current business conditions, since operational systems must react quickly to market conditions. While an organization generally consolidates a single enterprise budget (the financial plan), forecasting systems proliferate.

Another key difference between forecasting and budgeting is the frequency of updates. Ideally, the budget is created annually and remains unchanged throughout the year. Practically, the budget is updated to reflect major changes in the business; however, often the annual financial goals are not altered. Forecasts are created regularly throughout the year. One area for improving forecast accuracy is increasing the frequency of forecasting. For example, a major supplier of perishable ingredients for the manufacture of food products forecasts the demand from 2,000 customers twice monthly. This company is considering increasing the frequency to weekly.

As a decision support tool, the budget establishes a framework for expense controls and identifies areas for expense reduction if revenue goals are not met. Organizations have financial analysts that continuously analyze performance and identify opportunities for improving financial performance.

The value of the forecast is in supporting the operational decisions that an organization makes each day. In other words, forecasting is less a reflection of what management wants to happen and more of a reflection of what is most likely to happen. A critical requirement in forecasting is to provide an early warning of significant revenue variances.

Forecast error is computed by comparing actual performance to a prior forecast. Forecast error is generally acceptable at a summary level, the level that it is translated into the enterprise's annual revenue projection. Forecast error increases at lower levels, the level at which operational decisions are made. In most companies, forecast error is unacceptably high at this level. The economic impact is felt both in terms of high operational costs and poor customer service.

Why make this distinction? Many organizations are satisfied with their budgeting systems. However, increasingly the brutally efficient nature of business today requires companies to dramatically improve operational efficiencies. Not only can the data warehouse play a key role, but the Internet will increasingly offer the opportunity for inter-enterprise collaboration in consensus forecast development. The economic impact is large and measurable.

The Changing Role of the Data Warehouse

Arguably, the best source of data to support the demand forecasting process is the data warehouse. The data has been subjected to considerable cleansing in order to accurately reflect consistent historical trends. The data is also extracted from a number of sources, providing a rich analytic base for cause-and-effect analysis. The one thing that statisticians will agree on is that the weakness of a sophisticated statistical forecasting model is the lack of reliable data.

For many reasons, the budgeting process may remain external to the warehouse. However, the data warehouse and business intelligence should play a role in sales forecasting. Much as the data warehouse provides a single source of management information about the past, it should also provide information about the future.

The data warehouse was created because there was a need to separate user-accessible data for business intelligence from the transactional data required by operational systems. Bill Inmon's original definition of a data warehouse was ". . . a subject-oriented, integrated, time- variant, nonvolatile collection of data in support of managements' decision- making process" (Building the Data Warehouse, John Wiley & Sons, 1992- 1993). This definition does not specify that the data warehouse is intended to be read only. Bill chose his words carefully in using nonvolatile.

Maintaining a forecast in the data warehouse requires periodic update of the tables or, more appropriately, creating new tables reflecting a new forecast version. This translates into providing users with limited read/write access to portions of the data warehouse. In order to enforce the nonvolatile requirement, a method of committing each forecast version is required. As each new forecast version is developed, users are granted limited read/write access. Once the forecast is committed, it becomes a nonvolatile version that can be used for reporting and analysis.

The Changing Role of Business Intelligence

The goal of business intelligence is to answer business managers' endless streams of questions. The answer to one question almost always generates a new question. Each new question requires more comprehensive analysis. Many business intelligence tools focus on query generation and reporting data with "light" analysis capabilities.

As organizations become successful in answering managements' easy questions, they must anticipate more difficult questions. The hardest questions to answer are: Why? What if? Answering these questions is essential, but requires sophisticated data modeling and projection tools. This is not to suggest that all business managers require statistical analysis and data mining tools. A few might, but most simply want the application of the technology delivered as a solution to their business issues. In other words, as business intelligence matures, it will take on the form of solutions to business issues.

The application of the technology of business intelligence is already addressing issues associated with customer relationship management (CRM). Advanced business intelligence technology is required for analysis of customer behavior and market segmentation. A second application direction for business intelligence is demand forecasting. Improving operational efficiencies within the enterprise and across the extended supply chain is one of the most important business issues confronting organizations today. As noted earlier, markets are becoming brutally efficient, and organizations must become equally efficient in order to survive.

Demand forecasting addresses the what-if questions which are essential to business planning. Forecasting requires statistical modeling and projection technology as well as the capability to override the statistical forecast. A wide range of statistical techniques from simple projection of trends to advanced causal modeling is available. The selection of the appropriate statistical model requires careful analysis. Statistical forecasting is useful, but must be supplemented with a management review process. The reason is that most statistical modeling looks at history as the primary indicator of what will happen in the future. An organization's marketing and sales managers are working hard to substantially improve upon history.

Business managers must be able to simulate various scenarios in projecting the impact of their decisions and change the forecast when appropriate. Ideally, agent processes that issue alerts to managers only when there is a need for review should drive the forecast review process.

Forecasting is a business process that should involve many people throughout the enterprise and between businesses. Business intelligence needs to support each step of the process: analysis, modeling, review and publication of forecast versions to operational systems. To do this, business intelligence must encourage greater user collaboration associated with business management processes.

Extending the Boundaries

Internet technologies are moving rapidly from the business to consumer (B2C) form of e- commerce to the creation of applications for business to business (B2B). Ultimately, the challenge will be to go beyond brokering collaboration between two companies, extending to everyone within a trading community (BXB). Information is at the core of inter-enterprise collaboration. More specifically, the demand forecast is the language of collaboration between businesses. The one thing on which both supplier and vendor must reach consensus is the forecast; otherwise, neither can achieve optimal levels of efficiency.

The implication is that the data warehouse and business intelligence must operate outside of the boundaries of the single enterprise. Business intelligence must be Web optimized for ease of delivery, and issues such as security and scalability become extremely important.

Certainly, extending the boundaries of data warehousing and business intelligence has the potential to change the rules for how a company operates and competes in what can only be termed a fast world. The non-trivial challenge is positioning the data ware-house and business intelligence in the center of what a business requires to compete in the fast world.

The Language of Collaboration

The next big thing is the integration of the data warehouse and business intelligence with business management processes. Internet technologies promote a new level of global inter-departmental and, increasingly, inter-enterprise collaboration. The language of business collaboration both inter-departmental and inter-enterprise - is demand forecasting.


For more information on related topics visit the following related portals...
Data Marts, DW Design, Methodology, DW Basics, Business Intelligence and Forecasting.

Richard Tanler is the founder of eiVia Inc., the leader in providing technology that the company refers to as an ON-Demand Spreadsheet. Tanler was the founder of Information Advantage, Inc. a leader in the business intelligence market (acquired by Computer Associates) and is the author of the Intranet Data Warehouse (John Wiley & Sons) and numerous articles on business intelligence and data warehousing. You can contact him at rtanler@eivia.com.

Solutions Marketplace
Provided by IndustryBrains

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

dotDefender protects sites against Web attacks
30-day evaluation period for dotDefender, a high-end cost-effective security solution for web servers that protects against a broad range of attacks, is now available. dotdefender supports Apache, IIS and iPlanet Web servers and all Linux OS's.

Verify Data at the Point of Collection: Free Trial
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Data Mining: Levels I, II & III
Learn how experts build and deploy predictive models by attending The Modeling Agency's vendor-neutral courses. Leverage valuable information hidden within your data through predictive analytics. Click through to view upcoming events.

Click here to advertise in this space

View Full Issue View Full Magazine Issue
E-mail This Article E-Mail This Article
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.