FREE DM Review Site Registration!
Sign-up today and access DM Review on the Web!

Your FREE registration entitles you to:
FREE email newsletters

FREE access to all DM Review content

FREE access to web seminars, resource portals, our white paper library and more!

   

A New Approach to Automating Real-World Business Processes

The shortest distance between two points may be a straight line, but real-world business processes rarely follow a straight line. Automating business processes has been the mantra of a type of technology called business process management (BPM) and has also been embraced by many vendors of integration software. Unfortunately, BPM's evolution and implementation has been built on a false premise, namely that complex business processes can be reduced to linear sequences of tasks with one step myopically handing off to another. The result is yet another technology requiring companies to change the way they do business in order to benefit from the technology, the proverbial cart leading the horse, reminiscent of enterprise resource planning's salad days.

A new approach to automating business processes promises to overcome these shortcomings by providing context: the bigger picture comprehension needed to intelligently evaluate and deal with complex processes. This approach, called complex event processing (CEP), is not a technology, per se, but rather an approach to modeling, detecting and responding to business activity, either in a global sense using rules or through structured processes using event flows, a souped-up workflow.

The History and Basic Principles of CEP

CEP was developed at Stanford University in the mid-1990s under the auspices of a DARPA grant. According to its creator, Professor David Luckham:

The goal of CEP is to enable the information contained in the events flowing through all of the layers of the enterprise IT infrastructure to be discovered, understood in terms of its impact on high-level management goals and business processes, and acted upon in real time.

Today's CEP implementations are built around a number of core principles.

An event is the occurrence of something of interest and is technology neutral, such as a new purchase, a change of address or an attempt to break into a network. Events can come from people, devices, applications, networks or databases.

Events have context, that is, an instance of an event implies timing (when it happened, both in absolute terms and relative to other events), sequence (again relative to other events) and linking relationships to other events (patterns of events, either expected or implied).

Events can also carry information about themselves. For example, a purchase event might contain the product purchased and the purchaser.

Events can be evaluated, either based on their context, their data or additional data that may not accompany the event but is relevant (is the purchaser a "gold" customer, for example).

Events form patterns in time and/or sequence that may be interesting. For example, a change of address followed by the reporting of a lost ATM card may indicate an attempt to profit from identity theft.

Events comprise ad hoc processes, a sequence of activities that results in the execution of a process. For example, a revenue recognition process may be composed of the events "contract signing," "purchase order received," and "product shipped." An explicitly defined process automated using events is called an EventFlow.

Events may be abstracted into other events. For example, a change of address event followed by the reporting of a lost ATM card event may be represented by the compound event of "attempted fraud."

Events can optionally generate responses, or actions. For example, an attempted fraud event may trigger, in some cases, a "put account on referral" action to make sure downstream account activity is legitimate.

Beyond BPM: A New Approach

Taken together, these basic principles represent a paradigm shift in the approach to understanding and responding to business activity through IT infrastructure.

This shift is best illustrated by comparing CEP to traditional process automation approaches:

Figure 1: CEP Compared to Traditional Process Automation

Applying this framework to some examples in order to compare how each technology works allows us to quickly see why CEP provides tremendous value in today's real-world business environment.

The first example is expense report approval:

  1. Report is submitted;
  2. Based on who submitted it, for what, and for how much, someone must approve it;
  3. In some cases, a secondary approval is necessary;
  4. If all approvals are received, the report is submitted to accounts payable, if not, the report is returned and the process begins again.

Traditional BPM is excellent at this kind of automation because BPM technology has three primary functions:

  1. Execute a step, providing any data necessary;
  2. Receive the result of that step and any data returned, and
  3. Based on that data, evaluate where to go next.

However, these functions always happen in this order, and each one has visibility only to the previous step and any data collected along the way. Because of this, BPM is useful only for processes like expense report approval that are predictable, linear and controlled based on data that is collected or derived during the process. The process is distillable to a defined number of ordered steps, each of which can be predicted based on the results of the step before it. There may be branches and loops, but fundamentally the process can be understood in all its permutations and enforced. The process begins and ends, with no need to evaluate outcomes for future iterations of the process.

In addition, technology implementations of BPM have tended to rely on structured languages to define business rules, resulting in brittle implementations that work well only with relatively static business rules. They also typically require that all information required to evaluate steps be stored as fielded data. This includes any information related to timing, sequence and relationships between steps within the process (or, indeed, other processes), resulting in costly custom data integration.

Contrast the expense report approval process with the following, which involves an attempt to convert a stolen identity into cash:

  1. Identity thief logs onto online banking and changes password;
  2. Identity thief logs onto online banking and changes email address;
  3. Identity thief calls customer service and changes address;
  4. Identity thief calls customer service and reports a lost/stolen ATM card;
  5. If this combination of activity has increased more than 10 percent above the 30-day moving average in the last 48 hours, the bank put a "phishing attack" warning out to customers;
  6. Bank places account activity on review;
  7. Identity thief receives replacement card and begins to withdraw funds;
  8. Bank freezes account, notifies authorities, begins fraud investigation.

Traditional BPM technology would require quite a bit of custom application development to handle this real-world complex process, while CEP is uniquely suited to it for a number of reasons:

  • The process is unpredictable. Steps 1 through 4 could happen in any timeframe, and not all of them may occur at all.
  • The process is nonlinear, as steps 1 through 4 could happen in any order and step 6 depends not just on the outcome of step 4, but on the outcomes of steps 1 through 4.
  • The process is not controllable, and it is impossible to enforce the behavior of the identity thief.
  • The process is dynamic, as the identity thief will continue to get smarter and vary his or her tactics.
  • Timing, sequence and affinity between steps are critical in determining what to do. Clearly, steps 3 and 4 must happen in order but not necessarily on the same day (or even in the same week), but the timing and sequence of 1 and 2 are less important. Therefore, an understanding of timing and sequence among these related events is important. It is quite costly and constraining to store and evaluate all permutations of timing and sequence for steps 1 through 4 in a database, as BPM technology would require.
  • The various permutations of this process cannot all be defined. The identity thief may in fact perform other transactions between steps 1 and 4, creating a large number of actual pathways through the process. Because of this, it is better to define this process in terms of "exceptions" (like some combination of steps 1 through 4 in some timeframe) than to outline all the possible sequences and timeframes. This also means it is better to utilize rules to define some of these steps and workflow for steps, regardless of how the process is executed.
  • It is impossible to control the prospects process for research and purchase, or to predict it. Each step in this process cannot be predicted completely from the prior step but rather, from a number of different prior steps and even from prior iterations of the process. Because CEP focuses on events and the correlation of events, it is much easier to reduce the cross-sell process to a set of complex rules sequenced into a larger process.
  • Finally, step 5 requires insight into the outcomes of prior instances of the process. Specifically, anytime some combination of steps 1 through 4 has happened, that information needs to be counted and made available to any current processes that have reached step 5. This closed-loop processing is very difficult to achieve without much custom development in BPM technology designed for straight-through processing.

Key Features of CEP Technology

To address complex, real-world processes, technology which utilizes CEP requires some unique innovations. These innovations fall into two categories.

The first category is functionality designed to implement the core principles of CEP.

  • Event Generation: The technology must be able to generate events in response to events.
  • Aggregate Events: The technology must be able to link multiple events into higher level events. For example, a combination of "change of address" and "report of lost or stolen ATM card" within two days might aggregate to a "suspected identity theft" event.
  • Ad Hoc Streams: Groups of events must be linked and evaluated as a group.
  • Event Flows: Explicit workflows of events, or event flows, must be defined to govern aspects of a process, which must be sequential.
  • Context Correlation : Timing, sequence and affinity between events must be maintained at a level of detail sufficient for complex logic.
  • Active Monitoring : Event history must be stored, monitored and displayed, and thresholds must trigger additional events.
  • Built-in Nonevent Detection: The absence of an event must be easily accessible to the business logic.
  • Time-Based Evaluation: Business logic must use both absolute and relative time for specifying complex event patterns.

The second category of functionality involves features that, while not unique to CEP technology, are required for the practical implementation of the approach.

  • Codeless UIs: Complex processes must be defined without structured languages to enable rapid development and modification.
  • Automatic Data Retrieval: Information from outside the process (perhaps stored in external databases) must be easily accessible to each step without requiring additional steps to acquire it.
  • Intermediate Objects: Information must be separated from each process step to minimize data integration and allow nontechnical users to develop processes without worrying about the flow of data.
  • Object Orientation (Decoupling): Each component of a rule or workflow must be independent and reusable in order to maximize flexibility and responsiveness of the technology to rapidly changing, complex processes.
  • Unified Metadata Repository: Processes must be stored centrally in order to allow "hot-swapping" of process updates and to enable collaboration among nontechnical users.
  • Standards-Based Connectors: Connectivity to human interfaces, systems, databases and devices must be easily achieved without significant custom code.
  • Result Events: When an action is triggered as part of a process, a corresponding result event must be returned to enable compensation and rollback.
  • Advanced Architecture: The technology must have an architecture to support high performance, high availability and open standards.

New Requirements, A New Approach

Taken together, this functionality creates a truly powerful new approach to automating complex, real-world processes. While BPM provided a necessary first-generation technology, its limitations are clear and must be overcome for process automation to reach its potential. With today's businesses focused on responding to both threats and opportunities as they operate in a hyper-competitive economy, operational efficiency becomes a tremendous advantage.

The new generation of capabilities provided by CEP will help advance this objective and give businesses an effective tool for dealing with the real-world complexities of business processes.


David Cameron is vice president of marketing and product integration at AptSoft Corporation. He can be reached at david.cameron@aptsoft.com.

For more information on related topics, visit the following channels:



Industry Vendors