Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Covering Business Intelligence, Integration & Analytics
   Covering Business Intelligence, Integration & Analytics Advanced Search

View all Portals

Scheduled Events
Archived Events

White Paper Library
Research Papers

View Job Listings
Post a job


DM Review Home
Current Magazine Issue
Magazine Archives
Online Columnists
Ask the Experts
Industry News
Search DM Review

Buyer's Guide
Industry Events Calendar
Monthly Product Guides
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Enterprise Architecture: The Holistic View:
Managing Data in an SOA

online columnist JP Morgenthal     Column published in DMReview.com
November 23, 2005
  By JP Morgenthal

In my September column, "Reuse vs. Performance in SOA," I discussed the benefits of loose-coupling to reuse. One of the concepts that were discussed in that piece was to loosely couple your data, which may require you to break the relationships formed in your relational database and foster those relationships across services. Certainly, this architecture is not appropriate for a high-transaction volume environment, but for modest and small transaction environments, this architecture can yield significant reuse. The one downside is that the facilities provided by the relational database, such as relational integrity, concurrency and transactions, need to now be supported in the composite application development tools.

To reiterate the point I made in my aforementioned column, the value of breaking these relationships is reuse. When services are based on tables that have tightly bound relationships to other tables, but which have no relevance to your particular service, you limit the ability of that service to support other applications without having to recreate the entire database to support this one service. By allowing each service to manage its own data model, you can you make the service more lightweight and increase the ability of that service to support other functions.

For example, many legacy applications are integrated using a common data model. Often, these integrations were made to support an existing process and embed the assumption of the process into the integration. Thus, the integration of the accounting data model and the procurement data model is being managed by the underlying database. This means that certain database operations on procurement tables may invoke a trigger that updates certain accounting tables. Without the existence of those tables, those triggers would fail and possibly cause the entire transaction to fail.

Hence, building out the procurement web service for the purposes of automating the procurement approval process and exposing functionality directly to suppliers would require duplication of the accounting tables to ensure proper operation. In this example, not decoupling these models means that something your suppliers do could directly impact your financials.

Decoupling your data model and supporting data access through a services layer has additional complexities because web services should be developed in a stateless manner. This means that how you manage relational integrity, concurrency and transactions must also change. The rest of this column deals with direct experiences implementing an SOA with loosely coupled data sources and ways that the issues were handled.


One of the first problems that needs to be solved, whether you are refactoring an existing application or developing a new services-oriented application, is establishing relationships between the data. The reason relational databases are so popular is because they allow you to create and manage the relationships between data in a simple and common manner.

In a loosely coupled application, where the services are atomic and each manager their own data model, fostering relationships between data is a much more complex initiative. Indeed, these relationships are really being fostered across the services' view of the data as opposed to the data itself.

The other major change that is brought about by this approach to application development is where the relationships are being maintained. With this approach, the relationships move into the composite application layer. However, one needs to be careful not to build the relationships directly into the presentation layer, because it will be extremely costly to continually rebuild these relationships for each application. This means you will want to develop a composite service that aggregates data across services and exposes these aggregations as new business structures delivered through business services.


Databases provide us with concurrency with ensures that two individuals are not attempting to update the same record simultaneously. They do this through locking mechanisms. However, two things change when you move to a service-oriented architecture. First, services are designed to be stateless, this means that the services will not be maintaining connections on your application's behalf. Secondly, you will not be able to maintain a lock on a record in a database safely, which means that you need to implement a means to ensure that requests that come in for data that's currently being edited are satisfied in a read-only manner.

There are many probably many ways to solve this problem effectively. However, the one that we most recently implemented allows users to request a record locked and we implemented a locking mechanism with an expiration date. In this scenario, all records that are requested without lock are deemed read-only. When the requesting application attempts to update, the service checks to see if this is the user that requested the lock and that the lock has not expired.


Before we look at the issues with transactions in a loosely coupled environment, let's first revisit the concept of statelessness. In an SOA, the services should be stateless by design because doing so increases scalability since affinity for a particular service instance is not required. It does not mean that the service cannot persist its state on behalf of a process as long as the service has a way to associate the persisted state with a particular consumer. It is important to understand this distinction, since we will rely on the ability of a service to maintain the state of a transaction.

There is a specification emerging in the web services world called WS-AtomicTransactions. Many people are confusing the need for this service with the belief that SOA doesn't have transaction support today; this is a misconception . WS-AtomicTransactions provide a web services interface for coordination and participation in a distributed transaction. This protocol exists today in another form called the X/Open XA protocol. Indeed, today's transaction managers are already services that provide the coordination and management of transactions.

What's missing from this picture is a way that a web service can participate in this type of activity, which is where the WS-AtomicTransactions standard comes in. When a service implements this interface, it will be able to ensure ACID-ity over the data that it is charged with persisting.


SOA changes the way your applications are designed. It may force you to decouple and refactor existing applications in order to achieve the required level of reuse. This decoupling and refactoring will require you break the tightly bound relationships between data formed inside the database and establish those relationships by aggregating data together across a set of services. Additionally, you need implement mechanisms for concurrency to ensure that data is updated in a consistent manner. Finally, you need to manage commit and rollback semantics when updating data across multiple services.

Again, I reiterate, you will need to evaluate this approach relative to the volume of transactions that your system needs to support, now and in the future. With today's infrastructure, we can easily handle low to moderate transaction volumes; however, high-volume transaction traffic should be developed directly against the database, which may limit your overall level of reusability.


For more information on related topics visit the following related portals...
Enterprise Achitecture and Service-Oriented Architecture (SOA).

JP Morgenthal is managing partner for Avorcor, an IT consultancy that focuses on integration and legacy modernization. He is also author of Enterprise Information Integration: A Pragmatic Approach. Questions or comments regarding this article can be directed to JP via e-mail at morgenthaljp@avorcor.com. Do you have and idea for a future Enterprise Architecture column? Send it to JP; and if it is used, you will win a free copy of his book.

E-mail This Column E-Mail This Column
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2006 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.