Results
Which vendor leads?
| Vendor | Weighted score | Gates | Rank among eligible |
|---|
Rate the vendors to see the results.
Calculator Library / AI Operations
AI Vendor Scorecard
Compare AI vendors on weighted criteria with must-pass gates, and see whether the leader holds up when the weights move. Nothing is sent anywhere.
Tool
Score the vendors
Score = Σ weight × rating ÷ 5
Name up to four vendors, rate each criterion from 1 (poor) to 5 (excellent), and edit the weights to suit your use. Tick the must-pass gates each vendor meets. A vendor that fails a gate is out, whatever its score. The tool also tests whether the best vendor changes when each weight moves by five points. Load the example to see three vendors.
Everything runs in your browser. Nothing is sent to a server, and entries are not saved between visits. The example is made-up data. Weights and ratings are judgments, not measurements.
Picture
Weighted scores
What This AI Vendor Scorecard Does
The scorecard combines your ratings of up to four vendors on eight weighted criteria into a score out of 100, applies three must-pass gates, ranks the eligible vendors, and tests how stable the ranking is. The criteria are fit on your data, security and data handling, data rights and exit, transparency and change notice, integration, support and service levels, three-year cost, and vendor viability. You can edit the weights.
It runs in your browser and does not send your data anywhere. It supports the process in the Buying and Evaluating AI Tools and Vendors Guide. The AI Vendor Evaluation Scorecard workbook adds the questionnaire and evidence log.
How the Scoring Works
| Step | What it does |
|---|---|
| Weighted score | Sum of weight times rating, divided by five times the total weight, times 100 |
| Gates | Each vendor must meet every must-pass gate, such as no training on your data without opt-in; one that does not is out whatever its score |
| Rank | Eligible vendors are ranked by weighted score |
| Sensitivity | Each weight is moved up and down by five points (rescaled to the same total) and the leader is recomputed; the tool reports how often the leader stays the same |
Worked Example
The pre-loaded example scores three vendors for complaint classification. Vendor C scores 78.6, Vendor A 71.6, and Vendor B 66.4. Vendor B is the best fit on the test data but trains on customer data by default with no opt-out, so it fails a gate. The ratings are made up.
Limits and Good Practice
- Ratings and weights are judgments. Agree them with the users and the people who own the risk before you see the scores.
- Rate fit on your own golden test set, not on the vendor’s demo.
- Change the weights and see whether the ranking holds. If it flips, the vendors are close.
- The gates are not negotiable inside the scoring; if you want to negotiate a term, do it with the vendor and re-score.
- This is a decision aid, not legal advice. Have legal counsel review the contract terms.
AI Vendor Scorecard Frequently Asked Questions
How does the AI Vendor Scorecard calculate the score?
For each vendor it multiplies each rating (1 to 5) by the criterion’s weight, adds them, divides by five times the total weight, and multiplies by 100. A vendor with every rating at 5 scores 100.
What are must-pass gates?
Conditions a vendor must meet to be considered at all, such as a written answer to your security questionnaire, no training on your data without your opt-in, and rights to export and delete your data. A vendor that fails one is out whatever its score.
What does the sensitivity test show?
It moves each weight up and down by five points, rescales the others, and counts how often the leading eligible vendor stays the same. If it changes often, the vendors are close and the terms and a pilot should decide.
Can I change the criteria?
You can change the weights. The criteria names are fixed in this version; the workbook lets you edit them.
Does the scorecard store my ratings?
No. It runs in your browser and does not send or save anything.