Successful EDM Implementation:

Vendor Selection

by Rainer Hoff, Ph.D., P.Eng.

Introduction

This is part 6 in a series about successful EDM implementation. Part 1 and the methodology diagram are given in the Jan/Feb 1998 edition of Document Management. An information systems project ("the Project’), either in the realm of electronic document management ("EDM") or product data management ("PDM"), usually requires that a Commercial-Off-The-Shelf ("COTS") software product be purchased from a vendor of such software. This activity is the "Vendor Selection" phase.

The Vendor Selection Process

I shall describe a vendor selection process that has worked successfully at our clients. This is a comprehensive vendor selection process targeted at making a strategic decision for your organization. I am assuming a project where the initial software purchase is about $100,000, and the total (software + hardware + services) initial project cost is about $1 million. Smaller projects can benefit from the logic presented herein.

.

Advance Preparation

All of the phases described in the previous articles of this series, including project scoping, the business case, the I.T. strategy and (written) requirements will have positioned you, your budget and your project for success. I would discourage you from beginning Vendor Selection without the proper budget. Why consume resources to select a vendor if you can’t buy the system at the end of it?

While the advance work is being conducted, you are likely building your knowledge of EDM/PDM. This can be done by reading Document Management and other trade publications and by attending vendor-sponsored demos and sales events. Trade shows and conferences are also helpful; the most popular ones in the EDM/PDM area are sponsored by AIIM, Cimdata, Daratech, Kalthoff International and Management Roundtable (URL’s at the end of this paper).

Software Vendor Selection Kickoff

One Step or Two Step?

The following can represent a typical deployment:

Core Product + Deployment = System.

The Core Product is either the EDM or PDM software, as appropriate. The Deployment activities include the integration with related products like scanners and viewers, which puts the System into productive use. Years ago, it was only practical to procure an entire System, where the Core Product vendor and Deployment vendor were the same. You can still engage system integrators in the vendor selection process, and they will bid products that they are familiar with, subject to your specifications.

Today, most companies are first making a strategic choice for the Core Product, since it’s intended that the Core Product will be used for the initial and subsequent installations, as well as additional sites around the world. Once the Core Product decision is made, the Deployment is pursued for each site. This results in a two-stage procurement, since you must first select the Core Product before integrators can effectively bid on the Deployment.

Vendor Categorization

There are many vendors in the EDM/PDM market space. It’s worthwhile to divide them into the following categories:

Point product providers: e.g. viewers, scanners, content based retrieval

Core product vendors: Document management software suppliers, PDM software suppliers

Subsystem suppliers: e.g. scanning subsystems,

System suppliers: those that provide complete systems, including implementation

System integrators: those that do not sell their own products but implement a combination of products from other vendors.

This will help you focus on and shortlist the vendors that are relevant to your procurement strategy. This should allow you to select 5 vendors from the category of interest—3 would be better.

Issue Procurement document

In order to communicate your needs to the candidate vendors, you should provide them with a procurement document. This will be derived from the Requirements document, completed in a previous phase. For a Core Product procurement, you will probably send a Request For Information ("RFI"); for a System procurement, you will probably send a Request For Proposal ("RFP"). The RFI response describes if and how the standard product will satisfy the stated needs. The RFP response describes in greater detail how your specific system will be deployed.

While you are working on the procurement document, you should consider how the responses are to be evaluated. This clarifies your thinking, and will help you "tighten" the content and wording of the procurement document. The vendor will need at least 30 calendar days to provide a written response—45 days would be typical.

Review The Software

It’s very difficult to evaluate software from bid documents alone. The main reason is that if one hasn’t seen the software operate, it’s difficult to get a mental picture of what the bid document says. It’s a good idea to see the vendor software before you are asked to evaluate it. This is often done during vendor demos. Vendor demos are best held about 2 weeks after the procurement document has been forwarded to the vendors—by that time the vendors have a better idea of your needs. A debatable question is whether to do the vendor demo at the vendor’s site or at your site. Traditionally, it was only possible to conduct a realistic demo at the vendor’s site, due to the requirement for large format peripherals, CAD, and other systems. Today, it is much easier to make a demo portable, and large format peripherals are integrated through Windows device drivers and applications; so, there is less reason to insist on a demo at the vendor’s site. Many vendors have no demo facility at their headquarters. I still like to go to the headquarters because this is the only place that you’ll get a sense for the vendor’s software quality assurance program, software production and distribution system, and 24-hour support infrastructure.

Evaluate Bid Documents

The evaluation stage is when you will be thankful for having completed written, detailed requirements. Your bids will be evaluated against the procurement document, using the evaluation criteria developed previously. Don’t have detailed requirements? Didn’t create a detailed procurement document? You’ll find that the evaluation process becomes a frustrating, politically charged event, rather than the enlightening, technically oriented event it should be.

The RV Evaluation Method

I’ve named this after my colleague, Roger Volk, who first introduced me to this set of evaluation rules. Every point must be evaluated and scored. Compare the answers from all vendors. The vendor who best satisfies a requirement scores 10. All others receive lesser scores, compared to the best. This technique avoids having to create an evaluation system for each point.

The evaluation team must reach unanimity on each point before moving to the next. This leads to some points requiring an hour to debate, while other points require less than a minute. This unanimity avoids later statements by evaluation team members that they really didn’t agree with how the bids were evaluated.

Evaluating Vaporware

We’ve all heard bold claims from vendors (even Microsoft!) about features that would be available in the next release. But, when considering claims of a future capability, one is evaluating both the software and the quality assurance system. If a vendor has a stringent quality assurance system (such as a CMM Level 3 rating from the Software Engineering Institute (www.sei.org), then scheduled features could be evaluated as if they existed in the present product. Feature development that is unscheduled or cannot be substantiated by a QA program should be ignored in your evaluation.

Decision Matrices

The scoring from each point evaluation can be represented in a decision matrix. Often the top vendors have close scores at the end of the evaluation—simply picking the one with the higher numerical score may not be convincing. I suggest using the decision matrix to clarify thinking, and not to make the final decision. Let’s assume that each chapter of your procurement document deals with a certain category of features. One technique is to take the decision matrix results for each feature category and plot them as shown in

Figure 1.

The debate about the impact of selecting a particular vendor and "what are we missing?" should help to add confidence to your evaluation.

Conduct A Live Test Demonstration

An "Apparent Winning Vendor" is declared after the evaluation of the written bid documents. It is now necessary to validate the vendor’s written claims, and your understanding thereof. This can be done via a Live Test Demonstration, "LTD".

A script is prepared which contains a sampling of the kinds of functions that would be performed once the system is put into productive use at your site. You would provide the script and representative data/documents to the vendor about two weeks before the LTD. Some finesse is required in scripting an LTD since it must be sufficiently complete to demonstrate representative functionality, yet it can’t be so complex that the vendor is unable to prepare for it in a few days.

After 3 to 5 days on-site, executing the LTD, you will have a much better understanding of the software on which you will be staking your company’s future.

Send a Check!

If the vendor passes the LTD, you’re ready to complete contract negotiations and purchase the year 1 software. Congratulations! You’re now a member of an elite club: owners of EDM or PDM software.

Other Issues

Being Fair To The Vendors

You should be candid with the vendors about the status of project funding. If the funding is in doubt, then the vendors should be informed of this BEFORE you ask them to spend their time and resources on your project. They may still participate, they may not, but either way you have acted ethically and appropriately. Another issue is the amount of effort that a vendor is asked to expend on your project.

Table 1 shows what we’re asking from the vendors. Most high tech companies expect revenues of $250,000/person/year, so 45 days represents about $48,000 in opportunity cost. So if your software budget is much less than $100,000, then don’t expect the vendors to commit to this level of effort. When software cost thousands of dollars per seat, it made economic sense for the vendors to provide you with substantial sales support. Now that the market has set expectations at hundreds of dollars per seat, vendors have become very particular about how much effort they will invest in a particular opportunity.

Table - Sales effort for an EDM/PDM initial system.

Activity Vendor Effort

Preliminary discussions 3

Receive RFI 1

Respond to RFI 2 persons * 4 days

Pricing proposal 2

Terms & conditions review 1

Follow up on your questions 2 persons * 2 days

Sales opportunity status reporting in vendor’s company 2

Live Test Demo prep 2 persons * 3 days

Live Test Demo 2 persons * 5 days

LTD follow-up 2

Contract negotiations 2 persons * 3 days

TOTAL 45

Role of Consultants

Many companies use experienced consultants to assist in the Vendor Selection process. A consultant should facilitate the process, NOT dictate the results. A consultant is present to build confidence in the client, that the client has thought of every angle and is making a correct and informed decision. After all, management will hold the client team directly responsible for the outcome. The consultant should educate the client by asking lots of questions, both of the vendors and of the clients.

A consultant should NOT tell a client what to buy, or even what the consultant would buy if the consultant were in the client’s shoes. The client team must decide and buy into the decision.

Consultant neutrality

The consultant, of course, needs to be vendor neutral. A consultant that might be beholden to certain vendors, may impact his or her actions. How can you tell? Ask them:

Has the consultant performed consulting work for vendors?

Has the consultant conducted market surveys, strategy studies, or other studies for vendors?

Does the consultant make any profit from fees or payments from vendors?

If the answer to any of these questions is YES, then you need to convince yourself, that there is no conflict. A YES answer doesn’t prove that there is a conflict, but it does give detractors from your project, one more arrow in their proverbial quivers: the "tainted consultant" arrow. It is best to avoid this issue entirely and use a vendor-neutral consultant.

Additional References

Information about trade shows and conferences can be found at the following sites:

www.aiim.org

www.cimdata.com

www.datatech.com

www.kalthoff.com

www.managementroundtable.com.

Feel free to e-mail me at rhoff@gatewaygrp.com if you’d like more information or elaboration.

Gateway Consulting Group, Inc.

8610 Transit Road, East Amherst, New York, 14051

Telephone: (800) 668-2334 Fax: (800) 668-2336 Email: rhoff@gatewaygrp.com