Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Building Intelligent Agents Using Business Activity Monitoring

  Article published in DM Review Magazine
December 2005 Issue
  By Mike Ferguson

Most companies today have a significant investment in business intelligence (BI) to measure business performance. To date, most of these BI systems have been built to support decision making in specific business functional areas such as marketing, finance, HR and so on as opposed to being built around core operational processes. Most businesses are probably not using BI to continually and automatically monitor events in their operational business processes as their businesses operate to rapidly respond to detected problems or to predict if problems lie ahead. In general, therefore, companies have no active real-time element to their BI systems. The consequences are that nothing is helping the business to automatically respond immediately when problems occur or opportunities arise. Also, there is no automatic notification or flagging of alerts to take action that may avoid unnecessary costs, business disruption, operational mistakes and unhappy customers in the future.

Yet, the business processes that span all the functional areas, from front-office order entry to back-office shipping, billing and payment collection, govern how businesses operate. Making sure that core processes perform optimally has a major impact on overall business performance.

This lack of BI systems involvement in everyday operations often means that business executives see BI as passive, with no active role in managing operational performance to optimize real-time operations. To implement operational performance management, companies need to extend their BI systems, adding new technology components to help integrate BI with operational business processes. These components can be used to automatically guide people and optimize operations. Examples of such automation include:

  • Automatic monitoring of events in operational business process activities as processes execute to see if any immediate action needs to be taken either inside or outside the enterprise.
  • Automatic, rule-based action taking, whereby technology can continually test for specific business conditions and changes in operations. If necessary, alert people, make recommendations, send messages to other applications and/or invoke whole business processes when such conditions or changes do or do not occur.
  • Automatic alerting upon detection of an actual business problem or opportunity, e.g., a canceled cargo order will leave a ship half empty; immediately alert sales to resell the new capacity.
  • Automatic alerting to draw attention to a specific predicted business problem or predicted opportunity, e.g., it is predicted that an aircraft engine part may fail the next 20 flights, therefore just-in-time maintenance needs to be scheduled on the next landing while the aircraft is in turnaround. Alternatively, an order change requires an increase in product manufactured and additional distribution capacity to manage the additional product(s) ordered.
  • Automatic action messages sent to other applications and processes to cause action in response to detected problems and opportunities.

One of the key types of event-driven BI processing that makes these kinds of automation possible is business activity monitoring (BAM).

What is Business Activity Monitoring?

Business activity monitoring is the ability to automatically monitor events associated with specific activities in an executing business process. Some would argue that BAM can monitor events whether or not they are process related. That may well be the case, but it is difficult to put events into a business context if they are not associated with a business process. Given that process execution is managed by business process management software, BAM technology can be very effective when it is integrated with process management software that is governing the execution of processes. BAM software integrated with process management software allows us to extend BI systems to monitor process activities.

Figure 1 shows an example of an order entry, fulfillment and tracking process, consisting of several process activities. BAM allows companies to select specific events to monitor that are associated with specific activities in a business process. Note that BAM can be used to monitor executing processes in multiple places. Using the example process in Figure 1, a company may choose to monitor events associated with orders, package assembly, shipping and payments. This means that multiple BAM jobs or intelligent agents can be created to monitor different events in different parts of a business process as it executes. Multiple intelligent agents can be built and deployed across business operations.

Figure 1: Monitoring Events Associated with Specific Process Activities

Possible business reasons might be:

  • To detect any changes in orders (canceled orders, large orders, orders from valuable customers) and, if need be, to alert the sales force and other people in operations when specific events have a business impact, e.g., to respond when canceled orders occur.
  • To check orders or predicted demand against inventory to optimize a supply chain by reordering/delaying/canceling supply of inventory.
  • To detect or predict bottlenecks in the package assembly activity.
  • To detect or predict delays in shipments for valuable customers.
  • To detect or predict cash flow problems because of late payments.

In order to implement BAM and create multiple intelligent agents, companies need technology to allow them to:

  • Listen for and capture specific event(s) as a business process executes.
  • Integrate a necessary set of data using event-driven data integration technology as soon as each specific event being monitored occurs and to make this data available for automatic analysis. This data may come from data warehouses, BI reports, cubes, operational systems, external data sources or some combination thereof. Note that the target is not a database. This data is handed directly to a model for automatic analysis, i.e., the target is a service.
  • Automatically analyze the selected integrated set of data to produce intelligence.
  • Use rules to automatically determine whether to take action on the intelligence and if necessary to actually take those actions.

Note that actions include firing alerts to draw attention to a problem/opportunity; alert with recommendations on what to do next; alert with guided analytics, automatically send messages to one or more internal and/or external applications to cause something to happen; invoke a whole business process (i.e., a whole chain reaction) and so forth.

Technology Components Needed for BAM

Figure 2 shows how a BAM agent works to monitor events associated with activities in executing business processes. As can be seen from Figure 2, three main software components need to be included as extensions to a BI system to make operational performance management using BAM possible. These are:

  1. Event-driven data integration,
  2. Automated analysis (scoring) via deployed data mining models, and
  3. Automated action taking via a rules engine.

In addition, these components need to be linked together via workflow.

Figure 2: Event-Driven Automated Right-Time Processing

Event-driven data integration can be provided by extract, transform and load (ETL) tools. Many ETL tools are already event driven because they can plug into any Java messaging service (JMS)-based message queuing software (a.k.a. enterprise application integration or EAI) to subscribe to messages. Messages are of course applications events.

Automatic analysis can be achieved via mining models that have been deployed as Web services. These models can be trained to specifically analyze data and look for particular conditions in the data that would indicate a problem or opportunity.

Automatic decision making can be done via a rules engine. Here, the rules engine looks at intelligence produced and makes a decision based on rules and values of data. Typically, these rules are self-tuning in that they can detect when rules become stale and when they need to be replaced. Several rules engines exist in the marketplace.

The key question, however, is the workflow, i.e., how to link these three technology components together to create a BAM intelligent agent. Workflow can be implemented in at least three ways:

  1. Invocation of data integration, mining models and rules engine from within an event-driven ETL tool workflow. The mining models and rules engines that respectively score integrated data and take automatic action just appear as "special transforms" in the extended ETL workflow. Data in this case is not placed in a data store but simply flowed between the components.
  2. Model a business process to invoke data integration, automated analysis mining models and the rules engine as Web services. This would effectively create a monitoring process that can be attached to a specific activity.
  3. A specialized BAM server product with its own workflow capability.

In the first option, all three of the above components can be executed in a sequence as part of an event-driven ETL workflow. Figure 3 shows an example. Many different data integration products are capable of doing this today.

Figure 3: BAM Agent Workflow - Event-Driven ETL with Automated Analysis and Rules Engine Services

Alternatively, as long as all three components (ETL data integration job, mining model and rules in a rules engine) can be published as Web services, they can be linked together in a BAM process, using any business process-modeling tool and attached to a process activity that you want to monitor. This is shown in Figure 4.

Figure 4: Using Business Process Management Software to Create an Intelligent Agent Process

Another alternative is to use one of the powerful, new event-driven BAM server products available from any one of several young but very powerful start-up vendors.

Business Benefits

The business benefits of BAM are potentially massive. This kind of technology can prevent business disruption problems that may cause significant unexpected operational costs, delays and penalty costs as well as upsetting customers. If such events do occur, BAM makes a business much more responsive so that it can seize opportunity, maintain/improve operational performance and minimize impact on costs of problems that arise. As an example, if a last minute order cancelation (event) on Monday causes half the cargo capacity of a ship scheduled for departure on Friday to become available, then BAM helps the business to detect this and alert the necessary sales and cargo operations people to respond rapidly so that they can respectively resell the available cargo capacity before the ship sails and optimize labor resources needed to manage the cargo handling on that voyage.

Without BAM, sales and operations people may not even know about the cancelation and, therefore, not realize the impact. In turn, this may result in excessive labor costs if too many people have been hired for unnecessary cargo handling, a missed sales opportunity and a lower yield by needing to sail on Friday with only half the cargo capacity occupied.

Figure 5: BAM in a Process Workflow

In order to maximize business value, companies need to identify where in their business processes (i.e., what specific activities) they should deploy BAM and also identify the activities that would benefit from on-demand recommendations and alerts. An obvious example would be to use on-demand recommendations in the customer contact center or, indeed, any front-office customer touchpoint where individual customer intelligence could contribute to increasing sales, stopping churn and improving customer satisfaction. It would also be beneficial to monitor events in supply and distribution chains to facilitate just-in-time delivery at minimum operational cost.

To take advantage of BAM, companies need to appoint business process owners, define their core operational business processes and identify which process activity events to monitor that will yield maximum return on investment. These processes would execute under the control of business process management and application integration software that is integrated with event-driven data integration and predictive analytics to support BAM.

The trend here is toward agents, whereby many BAM jobs can be built with each one becoming a monitoring agent. Layers of these intelligent BAM agents can then be deployed to help monitor and manage business operations at different levels in the enterprise (see Figure 6).

Figure 6: BAM Trends - Layers of Intelligent Agents will Emerge to Manage Business Operations at Different Levels

Ultimately, this is about self-tuning business, whereby BAM agents keep their finger on the pulse of business as it operates to keep everything running smoothly and optimally. If any events indicate change or problems, the idea is that the business automatically adjusts itself to re-optimize operations (see Figure 7).

Figure 7: Event-Driven Self-Tuning Business Automation

To use an analogy, business should be like the human body. If I run up stairs, my heart beats faster, I breathe more heavily and I sweat a little. Nothing tells my body to do this; it just happens. The same thing needs to happen for business. This is what we are trying to achieve with BAM - "always on" event-driven automated business performance management as the business operates.


For more information on related topics visit the following related portals...
Business Activity Monitoring (BAM), Business Intelligence (BI) and ETL.

Mike Ferguson is managing director of Intelligent Business Strategies, a leading information technology analyst and consulting company. As an analyst and consultant, he specializes in enterprise business intelligence and enterprise business integration. With more than 23 years of IT experience, Ferguson has consulted for dozens of companies, spoken at events all over the world and written numerous articles. He can be contacted at +44 1625 520700 or via e-mail at mferguson@intelligentbusiness.biz.

View Full Issue View Full Magazine Issue
E-mail This Article E-Mail This Article
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.