DM Review Published in DM Review in February 2004.
Printed from

Meta Data & Knowledge Management: The Importance of Technical Architecture in Software Selection

by David Marco

Author's note: David wishes to thank Mike Reed, principal at EWSolutions, for his invaluable contribution to this month's column.

During the process of product selection, a great deal of attention is given to the functional capabilities of the software being evaluated. While this aspect is obviously important, ignoring the technical mechanisms by which the software actually operates can be fatal to a project.

In order to evaluate the functions/features of products being considered, most corporations and government entities understand that detailed scripted scenarios should be performed with software vendors on the short list of candidates. Unfortunately, many companies neglect to investigate the technological underpinnings of these products during the evaluation phase. This can lead to the purchase of a functionally excellent product, but one that is difficult (or impossible) to support in the customer's environment.

It must be stressed to the vendor that this phase of the software selection process is designed to evaluate technical issues (i.e., Does the server run on Windows?), not functional ones (i.e., How is payroll processed?). The vendor should be provided with a list of questions in advance and given adequate time to prepare a response. Someone in the organization should be designated to obtain clarifications for the vendor where required, before the on-site meeting.

Major Evaluation Criteria

Product architecture: Examples include processing through application tiers, middleware and messaging components, and scalability.

Data protection and restoration: Backup and restore methodologies, and archiving.

Security: User authentication and authorization, transaction and database security, and directory services support.

Tools: Diagnostics, session control and print spooling.

Performance: Performance degradation when multiple application windows are opened on the same workstation, thread control between tiers of the application, and network bandwidth and latency, etc.

Reporting: Support for ad hoc reporting, third-party integration and custom report design.

Support for client-developed applications: Tools and control mechanisms, configuration management and methodologies.

Vendor demonstration of selected features: The vendor should demonstrate the most important technical features (i.e., assignment of security to a user, monitoring application performance, etc.).

The most important criteria for the success of a technical architecture software evaluation are the presence and participation of individuals qualified to hold the discussion. Once the proper subject areas have been determined, it is imperative that the customer provide a "jury" of individuals in the proper disciplines to ensure that subjects are properly covered. These individuals must be tasked with the background work to understand what the vendor is proposing and fit that proposal into the "template" of their unique environment.

It may not be necessary to designate a single individual for each of the following areas. Some employees will be able to speak to multiple subjects. If both the customer and the vendor do not send qualified individuals to the meetings, this effort will be severely hampered.

Overall process administrator: An individual responsible for the overall coordination of the "jury." This individual tallies ratings from the other participants and collates a collective score for each function point. This individual is usually also responsible for reporting progress to, and getting clarifications from, higher levels of management within the company.

Network administrator: To address issues such as bandwidth requirements and network protocols.

Database administrator: To discuss needs for specific databases (e.g., product only runs on Microsoft SQL Server).

System administrator: For discussion of platforms, disk space usage, CPU requirements.

Security officer: To examine implications of LDAP (lightweight directory access protocol) support, single sign-on and role-based access.

Application programmer: To discuss availability of vendor-published API (application programming interface) to enable custom enhancements/extensions.

Web programmer: To investigate Web server and browser requirements, the need for active server pages or other software dependencies, etc.

Functional representative: To aid the group in understanding the objective.

With the proper preparation and engagement, technical architecture discussions can be extremely fruitful. Various architectural features of the products can be compared and contrasted. Some vendors have been surprised to see that customers are interested in how things work, but soon realize that this is to everyone's advantage. A full understanding by all parties will help ensure a smooth rollout, implementation and ongoing support of the product. A good "out of box" experience is best for all concerned.

David Marco is an internationally recognized expert in the fields of enterprise architecture, data warehousing and business intelligence and is the world's foremost authority on meta data. He is the author of Universal Meta Data Models (Wiley, 2004) and Building and Managing the Meta Data Repository: A Full Life-Cycle Guide (Wiley, 2000). Marco has taught at the University of Chicago and DePaul University, and in 2004 he was selected to the prestigious Crain's Chicago Business "Top 40 Under 40."  He is the founder and president of Enterprise Warehousing Solutions, Inc., a GSA schedule and Chicago-headquartered strategic partner and systems integrator dedicated to providing companies and large government agencies with best-in-class business intelligence solutions using data warehousing and meta data repository technologies. He may be reached at (866) EWS-1100 or via e-mail at

Copyright 2005, SourceMedia and DM Review.