A short document describing an AI model: what it is for, how it was tested, where it works and fails, and what it should not be used for.
Definition
A model card is a short document that describes an AI model for the people who use or assess it. It says what the model is intended for and not intended for, what data it was built and evaluated on, how it performed, including for different groups and conditions, its known limits and risks, and who to contact. For a purchased tool, a model card or a similar system description helps you judge fit and risk. For your own use of a tool, a short card describing the intended use, the test results, and the limits is a good record.
History
Mitchell and colleagues proposed model cards in 2019 as a way to document trained models. The idea builds on datasheets for datasets and on documentation practices in other fields. Many vendors now publish model or system cards of varying depth.
When to Use
Ask for a model card or equivalent when you evaluate a vendor. Write a short card for each AI use you run, kept with the validation record and updated after changes.
Step-by-Step
- State the intended use and what is out of scope.
- Describe the data the tool uses and the data used for testing.
- Report results with intervals, by case type, and against the baseline.
- List known limits and risks, and the controls.
- Record the version, the owner, and the date.
- Update after any change.
Examples
- A vendor card that lists the training data categories, the evaluation results, and the limits.
- A one-page card for a complaint classifier: intended use, validation results, known weak categories, and the owner.
Common Pitfalls
- Marketing in place of evidence.
- Results with no intervals or no data description.
- A card never updated after the model changed.