Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

The Data Strategy Advisor:
Basic Business Imperatives Drive Predictive Analytics

  Column published in DM Review Magazine
January 2005 Issue
  By Lou Agosta

Predictive analytics is a source of fresh insight based on finding patterns in the tidal wave of transactional data that is stored and aggregated in existing transactional and data warehousing systems. In its predictive form, an inference about future behavior, performance or events is an essential part of the analytic model - the prediction is inside the model, not merely posed by the inquirer in a database query. Major drivers include:

The need to get to know the customer as a system of dispositions and behaviors. Firms need to relate to the customer as more than a series of clicks on a Web page or an aggregation of transactions. The view of the customer is transformed by predictive analytics. The customer is a bundle of dispositions and propensities - to buy, to jump to the competition or to be loyal. As the economy turns from cost cutting to generating incremental revenue, predictive analytics is an important method of producing results because it is able to determine a future outcome of customer, product and market behavior with a determinate, scientific probability.

The need to work smarter and reduce costs. During the economic slump of the past two and half years, firms have cut costs mercilessly, in some cases cutting capacities that are essential to recovery. However, firms recognize the need for a better way to reduce costs than across-the-board cuts that take out both waste and essential infrastructure together. Predictive analytic applications are one way to reduce the constraints of inventory, operations and risk management. These predictive applications are applied to cut costs through smarter forecasting, demand planning, supply chain optimization and credit worthiness.

Exploratory data warehousing. Forrester's Business Technographics April 2004 North American Benchmark study reports that fully 54 percent of respondents stated they had a centralized data warehouse of customer information either in production or they were in the midst of a rollout.1 While data warehousing is a mature technology, the advanced data warehouse is not. Next-generation data warehouses are now making inroads into complex problems in merchandising, optimizing product mixes at a store level where previously individual SKUs had to suffice. Retailers such as Ann Taylor and Marshall Field's are using the detailed POS data to analyze and track buying behavior as it lines up with customer loyalty across time, share of pocketbook and lifetime value. The availability of exploratory data warehousing - a clean, consistent body of information stored in a data warehouse - opens the possibility of predictive inferences about key markets, brands and customers.

Innovations in predictive analytic algorithms. Fast algorithms based on the research of Vladimir Vapnik and implemented by KXEN are posing new challenges to established market leaders such as Computer Associates (CA), SAS, SPSS and IBM. Genetic algorithms are inherently parallel, enabling large numbers of suboptimal predictive models to be invalidated simultaneously. These algorithms are surprisingly rapid in effectively searching complex, highly nonlinear, multidimensional search spaces. Upstart applications are largely limited to marketing, but it will not be long before the results are generalized to predictive analytics at large. Similar considerations apply to the inherent parallelism of genetic algorithms as commercially implemented by Genalytics, now also implemented in CA's Predictive Analytic Server. As these companies continue to have success in implementing genetic algorithms in commercial contexts, they will redefine and extend the limits of what is possible with the technology. This will drive the following trends regardless of the near-term success (or not) of one or two particular software companies:

Intensified competition between workbenches and wrappered applications. Innovations in fast algorithms are causing intensified competition by creating possibilities for benchmark shootouts where the upstart wins. Meanwhile, the competition between predictive analytic workbenches and predictive marketing applications will intensify with the pendulum swinging back in the direction of the workbenches. While algorithms have been incorporated into application suites - Siebel and PeopleSoft OEM predictive functions from Angoss, and Business Objects uses KXEN - the general-purpose workbench is and remains a viable platform for solving analytics problems in horizontal business domains.

Predictive analytics will assimilate predictive marketing. Consolidation is an ongoing trend, with Data Distilleries being acquired by SPSS in November 2003 but only recently brought back to market. Predictive marketing upstarts such as Sigma Dynamics are reportedly having trouble making their numbers. The one exception is Ingeneo, which won new accounts in Israel where it also has a development lab. Companies with only a single algorithm to their name such as Trajecta, Salford Systems or NeoVista - no matter how high performance - will struggle for market traction and either be acquired or fail outright. One-algorithm wonders are out of favor. The one hope: an OEM strategy such as that which has buoyed the fortunes of KXEN.

An architecture for every budget - desktop, in-database, server-based. The popularity of desktop tools for predictive analysis is holding strong. Tools such as S-Plus from Insightful, See5 from RuleQuest and SPSS Answer Tree are less expensive and less powerful than server-based options. After having stalled during the recession, in-database predictive analytics will get new impetus from the need to ship the function to the data rather than move the data to the function. For those enterprises that have large volumes of data in data warehousing systems, the line of least resistance is to analyze the data for predictive insights where it is - in the database.

However, diverse data stores will sustain the role of analytic servers. In-database predictive model building represents a game-changing architecture that can cannibalize the market of standalone analytic engines. However, given the diversity of data stores and databases in the multidivisional enterprise, most companies will continue to leverage the power, scalability and database independence of the analytic server dedicated to implementing the predictive model.

1. Forrester's Business Technographics April 2004 North American Benchmark study indicates that the adoption of customer data warehousing is reaching late maturity in the information technology adoption curve. In addition, data warehouses are meeting expectations at a high satisfaction level.


For more information on related topics visit the following related portals...
DW Administration, Mgmt., Performance, DW Basics and Predictive Analytics.

Lou Agosta, Ph.D., is a business intelligence strategist with IBM WorldWide Business Intelligence Solutions focusing on competitive dynamics. He is a former industry analyst with Giga Information Group and has served many years in the trenches as a database administrator. His book The Essential Guide to Data Warehousing is published by Prentice Hall. Please send comments and questions to Lou in care of LAgosta@acm.org.

Solutions Marketplace
Provided by IndustryBrains

Design Databases with ER/Studio: Free Trial
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.

Data Quality Tools, Affordable and Accurate
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Data Mining: Levels I, II & III
Learn how experts build and deploy predictive models by attending The Modeling Agency's vendor-neutral courses. Leverage valuable information hidden within your data through predictive analytics. Click through to view upcoming events.

Click here to advertise in this space

View Full Issue View Full Magazine Issue
E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.