FREE DM Review Site Registration!
Sign-up today and access DM Review on the Web!

Your FREE registration entitles you to:
FREE email newsletters

FREE access to all DM Review content

FREE access to web seminars, resource portals, our white paper library and more!

   

Integration: Then and Now, Part 1 - SOA Architecture

Thoughts from the Integration Consortium

This month's column was contributed by Yogish Pai, Integration Consortium member and SOA practitioner.

Integration of business systems is one of the key capabilities that should be provided by IT to enable business agility. This has been the primary objective of all integration effort to date; however, due to various factors most of the integration efforts have not achieved the expected business benefit. Of course, projects with the right executive backing have been successful but have never reached their full potential. The objective of this column is to demonstrate how adopting a service-oriented architecture (SOA) enables a company to achieve business agility and flexibility. This is a two-part column; this first installment deals with high-level integration architecture based on SOA and the second will focus on SOA governance.

Integration Then

Following is a brief overview of the traditional integration best practices adopted by the industry. Typically majority of the integration efforts follow the hub-and-spoke pattern - with the hub bridging the gap between two or more business applications. In addition, this approach relies either on an enterprise application integration (EAI) or an extract, transform and load (ETL) tool to move data from the transactional systems to the operational data store (ODS) and/or data warehouse. The executive dashboard is developed on top of these databases using a business intelligence (BI) tool. Due to the processing time required for moving and aggregating the data, it would be next to impossible for the decision-makers to get a near real-time analysis of the current state of their business. As the accuracy of all major business decisions is directly proportional to quality of data and with most of the business systems siloed (the enterprise data such as customers, products and orders is typically entered in multiple systems), it is very unlikely that the decision-makers are getting an accurate view. One could potentially resolve this architecturally by custom developing all of the business systems - which is not practical for most of the enterprises.

Business does also plays in important role in driving IT in this direction. As time-to-market is key for each of the business silos, IT organization are directed to invest heavily on implementing packed applications and expected to stitch them together later. An additional drawback to the traditional approach is due to the lack of available infrastructure, IT organizations were forced to procure each of the technology components such as EAI, ETL, BI, data quality tools separately and custom develop the infrastructure.

In short, IT organizations were spending most of their effort in developing the infrastructure, instead of working on enabling business agility.

Integration Now

One of the reasons why SOA is gaining such momentum is because it is clearly not a technology solution; rather, it is a way of aligning IT closer to the business. SOA practitioners (also members of the Integration Consortium) first published the SOA definition in the SOA Blueprint white paper during the 2006 Global Integration Summit at Boston which states:

SOA is the business operations strategy for leveraging information to meet their objectives, such as increasing overall revenue, increasing customer satisfaction, improving product quality, etc.

The SOA Blueprint document was later updated and published as a three part SOA Practitioners Guide.

Figure 1: SOA Reference Architecture

In order to ensure that there is a common vocabulary, the SOA Practitioners developed the SOA Reference Architecture as shown in Figure 1, which is well understood by the contributors and documented in the SOA Practitioners Guide, Part 2. The primary principle of SOA is that all services developed should map back to business requirements, i.e., IT should not be developing any infrastructure or services that are not linked to the business requirement. The right way to capture these requirements is to first map the business process and later drill down to understand the business services required to fulfill the business process.

Figure 2: Service Layer Aligned with Business-Supported Activities

Figure 2 illustrates how business alignment and flexibility is achieved by adopting SOA for integration. On the business side most business processes have just grown and evolved over time, and it is a fact that many enterprises do not precisely understand their processes and bottlenecks. This makes it very hard to make improvements, gain operational efficiency and react quickly to the marketplace. So businesses are looking to achieve flexibility by being able to describe their business processes as a flow of discrete business activities, and this flow or process can be optimized and changed.

On the IT side, analogous to the business side, most of the technology is a set of applications, interfaces and data that have evolved and grown over time. Early SOA efforts have even exposed some of these applications and data as services. But the issue on the IT side is that these applications, interfaces and services do not quite match what the business process needs, and a lot of time and effort is spent in coding and development to try and achieve this alignment. What is needed is a services layer that is aligned with business activities which support business processes. It is the provisioning of this layer that enterprise integration and SOA addresses.

One way to create a business service is using a data integration pattern, where one aggregates disparate data from multiple applications. For example, a customer's account information may be fragmented across different accounts that the customer holds with the enterprise. One might need an account service that aggregates fragments of account information from multiple data sources and applications. Another way to create a business service is the service orchestration pattern. Here the system is orchestrating across a set of finer grained services or application interfaces. In the illustrated example in Figure 2, there is an order fulfilment business service which does an inventory check. If there is inventory - it initiates shipment; otherwise, it notifies the customer when delivery is possible. This is implemented by a process service within an integration tool such as WebLogic Integration (WLI). A process service in WLI can be implemented with process technology and can be augmented by business logic in Java. There are other ways, too. For example, a customer may write a custom application in Java on an application server.

In the illustrated example the overall order management process is implemented in a business process management (BPM) tool. Now the notion of a process is present in both AquaLogic Business Process Management (ALBPM) and WLI; but as you can see, the use of this technology is in different contexts and the two technologies are optimized for different use cases.

The final component for achieving flexibility is the service integration layer. Service integration mediates across different types of services, whether business services or applications or other types of services that an enterprise may have in its system landscape and exposes them via proxy services in a format that's consumable by any other service in the enterprise. This layer is typically implemented by an enterprise service bus (ESB) such as AquaLogic Service Bus (ALSB).

Figure 3: Service Integration Layer

Figure 3 shows the SOA Integration pattern, independent of the products or the business problem being solved.

Integration Case Study

This is a real-life case study where one of the large financial institutions spanning the Nordics, Baltics and Poland with 10+ million customers has successfully adopted such as approach across the enterprise. This institution grew by mergers and acquisitions of multiple companies over last 10 years. It was interested in creating new opportunities by launching e-services to provide 24x7 service, improve cross-sell rates, streamline front-end processes, etc. and mask complexity of core systems in different countries.

The financial institution adopted SOA to integrate their entire business based on an open, service-centric infrastructure which enables them to build and launch new services and processes rapidly.

Figure 4 shows the high-level integration approach which enabled this large financial institution to reduce production cost immediately by 55 percent in the first country they implemented it in as well as drive new annual net revenue counted in millions of euros by effective channel utilization.

Figure 4: High-Level Integration Approach

In conclusion, the following SOA-based integration pattern has truly enabled business to be both agile and flexible.

Figure 5: Example for the Business User

Figure 5 shows a pattern that is independent of the products or the tools used to integrate the enterprise and can easily be explained to the business. In addition, this also enables the decision-makers (by leveraging any reporting or BI tool) to review their business in real time. One final comment, integration now no longer means IT infrastructure but means business integration at all levels.


The Integration Consortium is a non-profit, leading industry body responsible for influencing the direction of the integration industry. Its members champion Integration Acumen by establishing standards, guidelines, best practices, research and the articulation of strategic and measurable business benefits. The Integration Consortium's motto is "Forging Integration Value." The mission of the member-driven Integration Consortium is to establish universal seamless integration which engages industry stakeholders from the business and technology community. Among the sectors represented in the Integration Consortium membership are end-user corporations, independent software vendors (ISVs), hardware vendors, system integrators, academic institutions, non-profit institutions and individual members as well as various industry leaders. Information on the Integration Consortium is available at www.integrationconsortium.org.

For more information on related topics, visit the following channels:



Industry Vendors