Data Warehousing Lessons Learned:
Data Mining Market Consolidation Lessons Learned
The consolidation in the data mining (predictive analytics) market during the past two years has been continuous and merciless. One of the key lessons for both users and providers of data mining technology and applications is that it is in the process of transitioning to predictive analytics.
Silicon Graphics' MineSet, which once boasted a substantial installed base, is not currently shipping a product. Trajecta was a data mining startup circa 1999 emerging from the incubation laboratory at Microelectronics and Computer Technology Corporation. Fair Isaac acquired the core Trajecta technology in 2002. Fair Isaac and HNC merged in August 2002. The sun has set on Business Miner from Business Objects, although Business Miner continues to have significant recognition in the desktop data mining market. Meanwhile, Business Objects has been offering KXEN predictive analytics as an encapsulated part of its Application Foundation and as a separately licensable product since January 2003 (initially bundled as part of the entire suite in June 2002). Instead of Darwin, Oracle now offers a migration path to Oracle9i Data Mining analytics.
In the face of a perfect storm of bad economic news and externalities, even top-notch technology can be culled from the flock as if by a Darwinian process, including tools with that name. The lessons of consolidation are described in the following paragraphs.
The promise of giving all different classes of end users access to data mining (predictive analytics) is oversold -- data mining will require a cross-functional, collaborative effort and predictive modeling tools that support collaboration will have an advantage.
Data preparation can consume a substantial effort, and data mining tools require further productivity enhancements to address those requirements.
The data is to be found in the database and "in database data mining" is the technology of the future and always will be, unless ways of rendering the technology administratively usable and manageable can be found.
Templates and recommendation engines wrapped in applications are adequate for well-defined problems, but the advantage will otherwise go to those tools for predictive analysts that are able to support collaboration between business analysts, statisticians and information technology (IT) data preparation staff. Attempts to dispense with even one of these over the life cycle of the data mining application are rarely successful. Even a deployed, debugged application must have the underlying predictive model revalidated from time to time in the face of different data, different business circumstances or different results from the model itself. This will inevitably require the statistician and IT support staff.
There are a number of predictive analytics ("data mining") problems around churn, attrition, cross-selling and upselling. On the product side, demand planning and just-in-time inventory are defined well enough to be addressable by predefined applications (solutions) within their respective vertical markets in telecommunications, finance and logistics. Companies such as Unica, KXEN and Quadstone, and offerings wrapped as recommendation engines in applications from E.piphany and Siebel Analytics (even SAP, Oracle and PeopleSoft) have demonstrated success. However, mapping less well-defined problems to the available technology and predictive algorithms remains a challenge for statisticians in collaboration with business analysts and IT data preparation staff. Here, the business is up against the problem of "not knowing what they don't know." Unsupervised knowledge discovery holds great promise and always will, unless a framework for collaboration between statisticians, IT technology experts and business analysts is available to support the design, deployment and maintenance of the applications.
Prospective purchasers of predictive analytics software should take advantage of the buyers' market in qualifying alternative predictive products and approaches to data mining and predictive applications. Once a purchase has been made, manage and reduce the risk of technology lock-in by conforming to open standards such as predictive modeling markup language (PMML) and commodity hardware where feasible. When the business needs to design and implement a wide variety of predictive applications that cannot necessarily be defined in advance or readily encapsulated in off-the-shelf packages, end-user enterprises should choose a general purpose workbench such as SAS Enterprise Miner, SPSS Clementine, IBM DB2 Intelligent Miner for Data or Angoss KnowledgeSTUDIO. When they have well-defined problems requiring predictive analysis, enterprises should choose an analytic application from the marketing automation, customer relationship management and enterprise application vendors containing predictive analytics. In particular, enterprises should plan on obtaining the predictive technology wrapped in the application to address specifically delimited problems in customer buying behavior, upselling and cross-selling, campaign management, customer profiling, personalization and promotional applications.
For more information on related topics visit the following related portals...
Lou Agosta, Ph.D., is a business intelligence strategist with IBM WorldWide Business Intelligence Solutions. He is a former industry analyst with Giga Information Group and has served many years in the trenches as a database administrator. His book The Essential Guide to Data Warehousing is published by Prentice Hall. Please send comments and questions to Lou in care of LAgosta@acm.org.
Provided by IndustryBrains
|Autotask: The IT Business Solution|
Run your tech support, IT projects and more with our web-based business management. Optimizes resources and tracks billable project and service work. Get a demo via the web, then try it free with sample data. Click here for your FREE WHITE PAPER!
|Data Quality Tools, Affordable and Accurate|
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.
|Design Databases with ER/Studio: Free Trial|
ER/Studio delivers next-generation data modeling. Multiple, distinct physical models based on a single logical model give you the tools you need to manage complex database environments and critical metadata in an intuitive user interface.
|Free EII Buyer's Guide|
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.
|File Replication and Web Publishing - RepliWeb|
Cross-platform peer-to-peer file replication, content synchronization and one-to-many file distribution solutions enabling content delivery. Replace site server publishing.
|Click here to advertise in this space|