FREE DM Review Site Registration!
Sign-up today and access DM Review on the Web!

Your FREE registration entitles you to:
FREE email newsletters

FREE access to all DM Review content

FREE access to web seminars, resource portals, our white paper library and more!

   

No Time Like the (Real-Time) Present

  • Chris McAllister
  • DM Direct, August 2007

The term real time would likely place in the top 10 list of business buzzwords that is used frequently today but also among the least well-defined and understood. Not surprisingly, the meaning of the term as it pertains to data integration and movement within the enterprise has become somewhat vague over the past few years, to the point where real time can now mean anything from the delivery of information instantly, to seconds-old latency, to a delay of several minutes or even hours, in some cases. Although the definition of the term has eroded, the relevancy of the concept itself has grown in the past few years.

Leading industry analysts look at the real-time enterprise as eliminating delays and using up-to-date information and speed of data for competitive advantage. Regarding business data used for operations, analytics, reporting and intelligence, real time typically refers to delivery with subsecond latency or seconds. For business executives struggling to make sense of the hype and the reality of real time, perhaps the best way to define it is the delivery of information as it happens, which if you think about it is really when it matters most.

Call it what you will, but there's no doubt that immediacy is a driving competitive force and a key factor for success in many aspects of business. As consumers, we live in a culture of instant gratification, around-the-clock transactions and consumption, and because of this "always open" global economy, real time translates to now, immediate, on demand. Take a look at any business, regardless of industry, and there is a need to offer real-time services and attention to meet consumer demand. Online retailers, banking, financial services, airline and travel providers all need to provide answers to customers in a fast and efficient manner. Today's consumer of information has come to expect an almost instantaneous response - whether in sales, marketing, operations or customer service.

That's the human side of it, and we see this permeate our culture in products or offerings such as "real-time four-wheel drive" and "movies on-demand." On the business side, real-time business intelligence (BI) is no longer the realm of pie-in-the-sky boardroom discussions; rather, it has instead taken on a sense of urgency from the executive team all the way to the data center, as everyone in the organization tries to figure out how to access information about the business faster in order to drive better decision-making, and all without breaking the bank.

Real Time for Real-World Industries

Here is where many organizations are trying to catch up. Across a wide range of sectors, businesses recognize that the "freshness" of their data is a critical factor toward generating revenue and maintaining competitiveness. The term "right time" versus "real time" refers to how the business consumes the information. It is true that not every aspect of the business needs data with seconds after the event occurs, but a company can commit to providing access to the freshest data and then let the business decide when they need to use it. This is why in recent years real time has become a cross-industry discussion. Where previously it had greater applicability to companies conducting e-commerce and pure-play Web businesses, now health care institutions, banks, telecommunications providers and traditional retailers are deploying real-time capabilities.

There are a wide range of business drivers for real-time BI and data across the enterprise. The business need for a bank is very different from that of a telecommunications or major health care provider. Also, it's important to point out that executives and stakeholders articulate their need for access to real-time data using language that talks about the direct benefits to the business. For example, the CEO of a health care provider would say, "We need to provide up-to-the-minute electronic access for our doctors, nurses and practitioners who diagnose our sick patients, so that we can administer the best treatment and improve the health of the patient quickly." On the other hand, an executive for a telecommunications provider might state, "Field customer service and support teams need information at their fingertips to view any account and understand what new products and services will best meet a customer's needs and increase monthly and quarterly revenue." The reality is that underlying these statements is the same desire to integrate data across enterprise applications so that the business has access to fresh data, at any point in time.

The basic problem for many of these organizations that exist within "conventional" business sectors is that there is a good deal of confusion in the market, particularly for those companies trying to determine how to achieve real-time business operations, intelligence and analytics. Of course, most large, global organizations have legacy applications and systems already in place, in addition to some open systems and maybe even open source environments. The bottom line is that these heterogeneous environments often don't communicate with each other. Integration may be the single most significant challenge that IT groups face today. The upshot is that IT executives tasked with transitioning the organization to real-time operations frequently have little idea whether the systems they have in place are capable of supporting real-time information delivery. Add to these misgivings the typical reservations about cost, complexity and need that accompany any new IT initiative, and it's no wonder that the very concept of real time is often surrounded with unproven myths and a certain amount of unwillingness on the part of the IT department to take the real-time "plunge."

Real Time: Separating Myth from Reality

However, any trepidation around real time can be set aside; here's why. The conventional wisdom that has built up around real time over the past few years is predicated on several myths, the most prevalent being:

  • Real time is costly;
  • Real time is complex and difficult to maintain; and
  • Real time isn't for everyone.

    It's Always about the Cost

    Myth #1: Real-time is costly. There's a good reason why cost is the number-one concern. When it comes to IT expenditures, business executives have been conditioned over many years to understand that speed and processing power have a direct correlation with cost. Simply put, the faster and the more powerful the system in question, the more expensive it is going to be. It goes without saying that when a CEO or other executive with spending authority hears the words "real time," they immediately think "increased spending." Yes, most new technology comes with a price tag, but the widely held myth that real time requires a rip and replace endeavor - abandoning legacy infrastructure (and the investments made in this infrastructure over many years) for costly new hardware, software and training - is simply not so. In fact, businesses making a transition to enable real-time BI may be able to leverage much of what they already have in place.

    To cite one example, airline ticketing giant Sabre implemented a real-time airfare search system several years ago using inexpensive, off-the-shelf Linux servers running a low-cost open source database and real-time data integration software. In this way, the company was able to continuously stream data from its mainframe transactional systems to the low-cost open-source environment. In effect, Sabre leveraged its legacy IT infrastructure and repackaged existing enterprise data to deliver a valuable new real-time customer service, without imposing strain on existing resources and while keeping total cost of ownership as low as possible.

    With the rapid advances in data integration technology, new real-time capabilities are more affordable and can usually be integrated with existing technologies. Counterintuitive perhaps, but the methods that provide the fastest data delivery speeds and highest volume performance are not always the most expensive. In fact, in many situations, an architecture that supports continuous stream updates with near real-time data feeds has become the low-cost alternative. In effect, it is now possible to get the high-volume data acquisition found with daily batch techniques, and with the speed of enterprise application integration techniques, at a fraction of the cost of extraction, transformation and loading.

    Supporting this view is a recent analyst study that looked at data acquisition for analytics and reporting, and found that the infrastructure costs for supporting continuous stream updates (i.e., low-latency data) are highly cost-effective when compared to the more traditional approaches. In fact, the study found that continuous stream data updates can be considerably less expensive than intraday batch updates or continuous mini-batch updates. This leads to the question: if fresh data can convey a business benefit, and you have the choice between real-time continuous data acquisition or updates once or twice a day - with comparable costs for both methods - wouldn't it make far more sense to choose the continuous stream?

    But it's Too Complex

    Myth #2: Real time is complex and difficult to maintain. Real-time capabilities can be a business-changing enabler, and so many assume that complexity comes with it. While many organizations immediately understand the business value, some delay implementation because they feel that once some systems operate in real time, then all the systems will demand the same level of service for the business. The IT team fears additional burden and worries about keeping the system highly available because the business comes to rely upon it. However, the same technology that integrates the data can also be leveraged to keep the system highly available.

    Still, while decision-makers within the enterprise may immediately understand the business value of implementing real-time processes, the specter of inherent complexity leads many organizations to delay or drag their feet. Or more accurately, it becomes easier to brand real time a future objective but not an immediate requirement for doing business. These companies might feel it isn't urgent and put off any real-time initiatives for another year or two, especially given the perceived added complexity and maintenance. Why not wait?

    The argument to act now is strong. With recent technology advances, real-time data can be used across the organization without the need for batch outage windows. Because business applications increasingly serve users on a worldwide basis, new transactions are continuously being made, and, therefore, it becomes impractical to impose any outage or interruption in the overall process. For critical and revenue-generating data, batch loading that data nightly or even several times a day is no longer efficient - data transactions need to be integrated seconds or minutes later to any target system that "needs to know." And once a business moves to real-time operations, it will rarely or never go back to batch processing.

    The last and probably most important point on this note of complexity is that it's possible to do all of this with very low overhead and burden on the source system. Many large organizations have stringent rules around what touches their most critical applications, and so it is critical that there is no intrusiveness on the production system. Also, real-time continuous data streams eliminate a middle tier, offering a simpler architecture and fewer points of failure - not to mention the cost factor again. Surprisingly, real time is actually less complex than batch processing approaches and is also easier to implement, customize and maintain.

    It's Not Applicable to our Business

    Myth #3: Real time isn't for everyone. It is a somewhat dangerous myth that real-time data is only critical for companies doing business online. Real-time transactions are everywhere in our daily lives, even though we might take them for granted, and most businesses today do have an online presence, commerce-enabled or not. There is no business that won't benefit from the value that real time delivers in data speed, integrity and availability. Take for example, a customer-support hotline for a cable satellite provider. The customer-service representative at the other end of the phone will need to know what happened, maybe just minutes before the customer's call. By having accurate, up-to-date information, the customer is not only impressed but also more efficiently serviced. Today's consumer is extremely fickle, and the competition is just a phone call or a click away.

    Real-time data not only speeds up business processes but can change their value altogether. By providing a real-time view and understanding of events, customers and information, businesses can make adjustments and decisions in an instant. With fresh data available for analysis, any business can operate to its full potential and impact revenue and profitability.

    Real time is not just for e-commerce. Telecommunications providers require real-time data integration into call center systems to meet ever more stringent SLAs. Banks increasingly need to support customer-facing Web applications, and to integrate data in real time across global ATM networks. Modern consumer practices mean that credit information is often shared over the phone or online, making real-time card authorization critical to fraud prevention in all areas of retail. It's common now to hear stories about cardholders getting calls from their credit card company seconds after making a big purchase just to make sure that it's a legitimate transaction. In the same way, when we make a deposit into our checking account at the ATM, we expect those funds to be available for buying groceries minutes later at the supermarket. More and more, the conviction among consumers is that real time is the speed at which business is conducted. And business has to keep up.

    No Time Like the (Real-Time) Present

    For all of these reasons, real time is quickly becoming a standard in many industries. There are quite a few organizations still contending that real time isn't right for them, or the time isn't right for real time, but it's a stance that they will not be able to maintain for long. The competitive pressures are simply too great. Especially as the myths cited above continue to be debunked in the technical and business press, and in global businesses finding that real time is not costly, complex or limited by industry. The comfort of conventional wisdom will be no refuge when your primary competition realizes that real time is not as costly or complicated as imagined, and that it provides better insight and decision-making across the organization with a direct impact to the bottom line. The smarter thinking is that the right time to design for real time is now.

    [1] Knowing Sooner Rather Than Later: Cost Analysis of Low-Latency Data in Enterprise Data Warehousing. Richard Hackathorn and Jack Garzella, Bolder Technologies. February, 2007


    • For more information on related topics, visit the following channels:



    Industry Vendors