FREE DM Review Site Registration!
Sign-up today and access DM Review on the Web!

Your FREE registration entitles you to:
FREE email newsletters

FREE access to all DM Review content

FREE access to web seminars, resource portals, our white paper library and more!

   

Text, Too

Many organizations view data mining - extracting trends, meaning and insights from corporate databases - as increasingly critical, but mining data is only a part of the performance management process. Leading companies today are enhancing business performance and exploiting competitive advantages by turning text into intelligence, too.

Data and text mining are not mutually exclusive alternatives. There are valuable insights to be gleaned from "structured" data sources such as customer relationship management (CRM), enterprise resource planning (ERP), data warehouses, metadata, data marts and operational data stores. Successful companies in every industry use this information to make decisions about where to devote or pull back resources, which customers to pursue and how to price, package and sell product offerings.

The business benefits are magnified, however, when text is analyzed alongside structured data. "Unstructured" content offers valuable, qualitative insight - customer needs, wants, product feedback, emotional attachments, even competitive threats - that is lacking in structured data, and there is simply a great deal more of it available to the company that is capable of cost-effectively collecting it. Online product reviews, call-center notes, survey verbatims, CRM text fields and other Web content can help companies ascertain customer sentiment and enhance product planning. Quality problems can be illuminated and eliminated - and warranty claims reduced - by culling CRM data, call-center notes, email and instant messages. Internet-based, consumer-generated content can prove especially valuable for spotting competitive threats and boosting customer experiences and retention.

Text mining, simply stated, is the practice of gathering textual information, structuring it for databases and deriving useful business intelligence (BI) from it. E arly adopters across industries including consumer goods, retail, life sciences and financial services already achieve measurable benefits through text mining, in part due to the convergence of three business and technological trends:

  • Simultaneous development in technology areas such as BI, data warehousing, CRM and natural language processing (NLP) have fueled the rollout of mature, useful, "commercialized" text-mining solutions.
  • With the Internet growing more mainstream, companies have available to them more untapped sources of unstructured textual information containing the insights of an increasingly populous and influential customer base.
  • Companies are more focused than ever on measuring return on investment (ROI) in every area of the business - even marketing, perhaps the last vestige of purely subjective spending.

Better Questions, Better Answers

Text mining is enabling companies to use unstructured customer and market feedback data in ways not possible just a few years ago:

  • Aggregating customer contacts from blog posts, user feedback and comments at online communities to build a picture of customer sentiment for informing product offerings and eliminating problems;
  • Analyzing the call-center notes, emails and instant messages from support cases to enhance customer service and loyalty;
  • Harvesting online conversations to better understand customer attitudes, motivations and opinions;
  • Merging "de-identified" patient-care notes with demographic information and treatment histories for research purposes, and
  • Honing in on the particular problems encountered by their most profitable customers, reducing loyal-customer churn.

Several "point" products that perform specific functionality relevant to text mining have been available for years, but these required complex configuration, setup and linguistic programming skills, preventing them from achieving any mainstream use by most organizations more accustomed to reporting and BI solutions. Only recently have integrated, end-to-end solutions emerged, spurring text mining's widespread adoption across industries. These commercialized solutions - the first to be purpose-built precisely for commercial application of text mining - are easy to use by general business users (as opposed to IT power users), yield business insights quickly and adapt flexibly to a given company's operations and markets.

Coming of Age

The commercialized text-mining solutions exploit developments in an array of underpinning technologies:

  • BI-enabled reporting tools. The industry's leading BI tools have undergone, in some cases, more than a decade of development. They are in high version numbers and widely deployed, and general business users are well trained in leveraging them to analyze structured data.
  • Enterprise data warehouses and operational data marts. Companies have assembled large, federated data warehouses for storing structured sales, financial and operational data that not long ago was stranded in stovepiped systems. Scalable to multiterabyte sizes, the data warehouses are capable of accommodating unstructured text, too.
  •  CRM data sources. CRM is well established among companies, but its potential business benefits have been limited to date by the fact that CRM has typically leveraged only structured data. CRM informed with unstructured text delivers a true, 360-degree picture of a customer's needs and sentiments.
  • NLP and machine learning categorization algorithms. NLP technologies help computers discern meaning from human language. NLP capabilities such as entity extraction, fact extraction, categorization and sentiment extraction have been improved after early implementations revealed ambiguities and complexities. Recently, NLP has become increasingly optimized for the unique requirements of commercial use.

Unstructured, Untapped and Uncommonly Valuable

Mining structured transactional information (such as sales or product-return data) offers a company a valuable record of historical events. Mining unstructured text (from email, Web sites, audio, video or graphical files) promises an even more valuable look forward at customer intentions, needs and plans.

The vast majority of information available to a company is, in fact, unstructured - as much as 80 to 85 percent by some estimates. And most of that information sits outside the enterprise. It resides within social networking sites, wikis, folksonomies, blogs, external user groups and other Web 2.0 sites where a company's existing and potential customers are most honestly and clearly revealing their loyalties, moods, interests, needs and expectations.

Leading companies are tapping these sources of intelligence to get a sense of coming trends in the markets that they serve - before transactional histories exist and before their competition is able to react. Mining and analyzing unstructured text could enable a company to, for example, dynamically adjust a marketing campaign during a product launch to ensure that key audiences are reached.

The Internet becomes more mainstream every day, and, so, every day the content on the Internet holds more insights from more of a company's customer base. The ability to rapidly and simply convert that content into useful intelligence will soon be regarded as a competitive necessity.

The Pressure to Prove ROI

The emergence of text mining in business comes at a time when companies are under unprecedented pressure to prove positive ROI in everything they do. Text mining is already proving a valuable tool for this task among departments as diverse as customer support, research and development, quality and marketing. Text mining's potential impact on marketing is especially compelling.

Historically, marketing departments have found measurement difficult. Because so many factors might lead to a customer's buy/no-buy decision, the success of a particular marketing campaign is difficult to measure. With the advent of Web-based marketing and quantifiable statistics (clicks, hits, etc.), companies today require a reasonable expectation of return on their marketing campaigns. An August 2006 Coremetrics Inc. study of marketers in the U.S. and UK showed that 86 percent said their decision-making was more reliant on marketing analytics than it had been two years before.

Without text-mining capability, a company might simply measure sales before and after a campaign to measure its effectiveness. With text mining, a marketing department can benchmark buzz for the product before, throughout and post launch. Sentiment of particular groups of customers could be mapped, enabling the marketing department to redirect efforts as necessary and on the fly.

Text Mining's Moment

Most companies do at least some data mining and use BI reporting tools. But it is still relatively few companies that take the next step of mining all of the structured data and unstructured text available to them - whether residing inside the enterprise in, say, call-center notes or beyond it at an Internet message board - and availing general business users to all of that intelligence and analysis.

With text mining's evolution into a fully commercialized capability, however, this scenario stands to change. Soon, it will not be the few users of text mining in an industry separating from the mass of competition; it will be the mass of an industry leaving behind its most sluggish players. A line is forming.

Is our current ad campaign reaching the customer demographic we have to reach this quarter and actually producing sales? ... Why are we seeing so many returns of this particular product from one group of users? ... What are the most common switching behaviors among our biggest customers? ...

Mining unstructured text and analyzing it alongside structured data makes a company better informed than ever previously possible.


Sid Banerjee is chairman and CEO of Clarabridge. He is a notable entrepreneur in the industry, a co-founder of Clarabridge and Claraview, with over 15 years of business intelligence consulting experience. He currently resides in Washington, DC.

For more information on related topics, visit the following channels:



Industry Vendors