FREE DM Review Site Registration!
Sign-up today and access DM Review on the Web!

Your FREE registration entitles you to:
FREE email newsletters

FREE access to all DM Review content

FREE access to web seminars, resource portals, our white paper library and more!

   

Keeping Score

With business spread around the world and changing constantly, those who plan and execute performance strategies need the latest information as soon as it becomes available. Advances in technology helped create these conditions, and new developments can assist businesspeople in dealing with them. In particular, three related technologies - scorecards, dashboards and performance alerts - establish meaningful contexts that enable users to analyze, measure, share and act on information quickly.

Scorecards, dashboards and alerts all support performance management but differ significantly in how they do so. A dashboard is an application that helps you monitor an organization's performance, whereas a scorecard helps you manage performance. Performance alerts are notifications of key trends or business events that tie to either scorecard or dashboard goals.

With business spread around the world and changing constantly, those who plan and execute performance strategies need the latest information as soon as it becomes available. Advances in technology helped create these conditions, and new developments can assist businesspeople in dealing with them. In particular, three related technologies - scorecards, dashboards and performance alerts - establish meaningful contexts that enable users to analyze, measure, share and act on information quickly.

Scorecards, dashboards and alerts all support performance management but differ significantly in how they do so. A dashboard is an application that helps you monitor an organization's performance, whereas a scorecard helps you manage performance. Performance alerts are notifications of key trends or business events that tie to either scorecard or dashboard goals.

For example, a typical manufacturing business would use a corporate scorecard to manage its overall progress toward a yearly strategic goal, such as expanding market share in Asia, by mapping its operational, financial, customer and organizational expansion initiatives to specific scorecard measures. The scorecard then would show not only progress toward the goal but also where imbalances exist and where action needs to be taken.

Dashboards, on the other hand, are a way to display and monitor progress on organizational business process and individuals' measures, such as sales quotas and product quality. Users can then set performance alerts to monitor dashboard or scorecard measures that notify them of actual or potential goal shortfalls, such as a poor regional revenue outlook or rising complaint calls at the contact center.

Survey Parameters

As part of Ventana Research's ongoing primary research, we set out to learn how extensively companies are employing these tools and to determine trends and priorities in their use. In January and February 2006, we conducted a survey of 590 managers and analysts in companies that use scorecards, dashboards and performance alerts. Nearly half (45 percent) of the respondents came from the lines of business, followed by 41 percent from the IT function. Finance accounted for 11 percent of respondents, and the remaining three percent work in HR. The survey participants are from a variety of industries, none of which contributed more than 14 percent of the respondents.

Our analysis of the survey results yielded a wealth of data about how these organizations are using the tools. From encouraging trends to inhibiting limitations, from expressions of satisfaction to complaints about functionality, they showed that scorecards, dashboards and performance alerts carry great potential as business tools, although most companies have yet to apply their full capabilities. We hope that the following highlights of the study will add to general awareness of what these tools are and are not being used for and how they may help other businesses in meeting the incessant challenges of change.

Reasons for Adoption

Ventana sought to learn why companies are deploying scorecards, dashboards and performance alerts. The top business driver that respondents cited for adopting them is to align operations with corporate strategy and goals. Approximately one-third of companies identified this as the most significant reason for their initiative. It was most prevalent for scorecards (36 percent), not quite as much for dashboards (29 percent) and least so for alerts (24 percent) (see Figure 1). We consider this significant because alignment is a step toward performance management, which links strategy with corporate objectives in ways that make the best use of a company's resources by coordinating the efforts of every member of the organization.


Figure 1: Why do companies adopt scorecards, dashboards and performance alerts?

The similarity of responses regarding scorecards and dashboards is one indication that there is not a clear distinction between the two in the marketplace and that companies use the terms interchangeably. Other responses confirmed this conclusion. In theory, scorecards follow a management methodology; there is no such requirement for dashboards. But when scorecards are deployed without a clear underlying methodology, the line between the two gets blurred.

Following closely behind alignment as a business driver for adoption were improvement in decision-making, business process performance and use of company resources. These suggest that many companies do not currently use these tools to drive strategy developed at the executive level throughout the organization.

Respondents ranked supporting better decision-making as the second most important deployment reason for scorecards and dashboards, but for performance alerts it came fourth, at 10 percent, behind both improved business process performance (19 percent) and better use of company resources (18 percent). We were not surprised by this finding because alerts are often used to monitor business processes and events in order to reallocate resources in response to changing needs.

Deployment Size and Frequency

Today, deployments of scorecards, dashboards and performance alerts are small: the majority of companies (60 percent) have fewer than 100 users. Nevertheless, the results show that IT departments plan to considerably increase the number of users these applications serve. Over the next 12 months or at full deployment, the number of projects having more than 100 users will increase in more than 20 percent of the companies surveyed.

As well as being small, many current deployments are narrowly focused. Generally, scorecards, dashboards and alerts are being deployed, in order of most to least often, to finance, sales, IT and marketing. The frequency of deployments into the sales function confirms that companies value revenue goals tracking over tracking balanced measures of executing operational strategy. We also found that the lowest frequency of deployment occurs in supply chain and customer operations.


This narrow focus is confirmed by a look at who uses the tools. The largest group is mid-level managers rather than executives. Simply put, scorecards and dashboards are most often being deployed tactically as supervisory tools for middle managers rather than strategically as a tool for top executives to manage the health, direction and performance of the business.

Kudos and Complaints

Following up on the number-one business driver, our study found that scorecards and dashboards are effective in helping companies achieve alignment to goals and strategies. Only about half of the respondents reported that before using them they were effective in achieving performance alignment, but after adopting them more than 80 percent said this (see Figure 2). Backing up this finding, less than one-fifth of respondents said they have difficulty quantifying a clear ROI or finding room in their budgets for these initiatives. This seems to be a positive endorsement because these two challenges are often the biggest hurdles to deployment of new software applications.


Figure 2: Do scorecards help to achieve performance alignment?

In other areas of effectiveness, however, responses were mixed. Strong majorities of scorecard users reported that they receive the right amounts of information on achieving company goals and objectives (78 percent) and concerning business unit performance (79 percent) (see Figure 3). A smaller but still substantial percentage of respondents (61 percent) said that they receive approximately the right amount of information on leading indicators about company health.


Figure 3: Do scorecards provide the right amounts of information?

On the downside, only approximately half (53 percent) said they get the right amount of information on achieving individual workers' objectives. And 42 percent declared that their scorecard provides too little information in this area.

More than half of respondents (53 percent) said that dashboards track progress toward goals and objectives, but 41 percent indicated that their dashboards do not do it well enough. This is a concern because the purpose of a dashboard is to give executives a single-screen information center that focuses exclusively on their enterprise.

Similarly, only half of the users believe their data is sufficiently timely; the other half says it is often out of date. This finding undercuts the core value of dashboards and alerts, which is to quickly notify users of performance issues and guide them to take immediate action.

While 38 percent of respondents said their alerts were always timely, 57 percent said they were sometimes out of date. Respondents also felt that performance alerts are not in sync with goals and measures. While users reported getting both detection alerts (about what has happened) and prediction alerts (about what will happen), their biggest complaint was that alerts are not indexed to specific performance goals (36 percent). They also complained that alerts are not tied to their scorecards and dashboards. We found this lack of connection alarming because context is critical if alerts are to aid in managing performance.

Business users' number-one complaint about their scorecards and dashboards - each cited by 42 percent of respondents - was an inability to drill down to details. Revealing something of a disconnect, IT said business users complain most about a lack of data integration from multiple sources. We suggest that data integration is not actually a business user complaint, but rather is the root cause of many complaints that business users have, including the inability to adequately drill down. Users also complained about problems in defining key performance indicators and in changing perspectives on the fly.

Homegrown Solutions

In looking for the sources of these and related inadequacies, our survey found that, despite the availability of mature offerings from software vendors, 64 percent of scorecard, dashboard and performance alert applications are developed internally by IT departments. One result of this tendency is that users don't get capabilities such as drill-down tools that support detailed analysis and thus better decision-making. IT departments could address many of these complaints by choosing to seek out commercial products that integrate data and provide detailed analysis.

The reliance on homegrown solutions extends to the methodologies on which companies base their scorecards: 37 percent use their own internally developed methodology. Only one-fourth use the balanced scorecard method. The remaining 38 percent are dispersed among 10 different methods. We conclude that many companies are not following well-defined methodologies in this area. When so many scorecards are internally developed, it is likely that measures are not the same from company to company.

To fully benefit from deploying scorecards, dashboards and alerts, companies will need to accept that achieving success takes time. Our research shows that it can take up to two years to optimize the content and context of these tools. Even though the technology can be implemented quickly, its use requires individual and cultural change, which takes longer. We also find that executive sponsorship matters. When the CEO or CFO is a sponsor, there is a greater likelihood of adoption and less likelihood of project abandonment.

We recommend that companies employ a step-by-step approach to developing solutions, one in which IT and users consistently re-evaluate requirements through a formal process, whether they build the applications in-house or purchase them from vendors.


Colin Snow is VP and research director - Operational and Supply Chain Performance Management. Snow heads up the Ventana Research Operational Performance Management (OPM) practice focusing on the alignment of business and information technology in the areas of supply chain, operations and business process management.  His research investigates what organizations need to manage their operational processes and supply chain for performance improvement.  

For more information on related topics, visit the following channels:



Industry Vendors