Optimizing Entity Data Quality, Part 2

In Part 1 of this column, I noted that rationalizing, cleaning and maintaining all entity data across your entire enterprise is a daunting prospect. Successful data quality management is a process of aligning your areas of focus for data quality enhancements with the strategic and tactical goals of your business end users. Last month, I focused on the attributes of entity data. This month, I will complete the high-level framework by describing how you can optimize your data quality enhancement efforts by focusing on different quality characteristics.

As a quick reminder, last month I provided an example of a company with a strategic goal to sell more products to existing customers. The entity data attributes most important for achieving the objective were entity identity, individual identity and transactional data for sales, customer service and product development. These three types of data have quite different chronic data quality issues for most companies.

The two most significant problems you will probably encounter with entity identity data are the rate of change (timeliness) and multiple instances of the same entity (consistency). Companies regularly move, change their names or merge, and they frequently have many names (some similar and some not) and addresses that are often not discovered with basic matching routines. These are significant issues that generally dwarf the other types of quality issues associated with entity identity data, such as data entry errors (accuracy), missing fields (completeness) and format standardization for addresses, phone numbers, titles and so on (consistency).

Individual identity data, which in our example is linked directly to an entity, has different characteristics and, thus, quality issues in most cases. The most significant issue I have observed with individuals at companies is completeness (lack of phone numbers, email addresses, etc.). This is difficult data to source, but once sourced, individuals generally retain the same identity characteristics as long as they are at the entity level. Accuracy is a common secondary problem because there are few consistent rules capturing errors in individual identity information.

Figure 1: Percentage of Answers Provided by Each User Type

Transactional (or role) data will suffer most from consistency problems because the data will generally be collected and maintained in many places within your enterprise. As an example, your customer service team may identify an entity with a different name or address than the salesperson calling on the customer. Similarly, bill-to names are frequently different from ship-to names. Aggregating transactional data across your company will require developing processes to identify and consolidate these multiple instances of the same entity. Accuracy is the second most common problem with transactional data because the people entering the data are often line-of-business staff such as sales or customer service personnel.

The aim of this example was to demonstrate that enhancing data quality is not a “one size fits all” activity. To ensure that data enhancement efforts meet the needs of the business end users in this example, the grid in Figure 1 can be used to gain alignment on the goals of the quality program. Once the goals have been agreed to, metrics can be created for monitoring the timeliness and consistency of entity identity data, the completeness of individual identity data, and the accuracy and consistency of transaction data.

You have neither the resources nor the need to rationalize, clean and maintain all the entity data across your entire enterprise. If you focus on the entity data attributes that are most critical for your business end users to achieve their strategic and tactical goals, you can create data quality enhancement programs and metrics that will meet their needs.