A condition, written in advance, that requires an AI use to be re-tested, retrained, or stepped back, such as a control signal, a model change, or a set time.
Definition
A retraining or re-validation trigger is a condition, agreed before launch, that forces a review of an AI use. Examples are a point above the control limit or a long run on one side of the center line, a pooled error rate whose interval reaches above the tolerance, a vendor model change, a change in the process, products, or forms, a new class of input, a serious incident, or the passing of a set time. Each trigger has a named response: investigate, raise sampling, step autonomy down, re-test on the golden test set, retrain, or retire.
History
The idea comes from calibration and revalidation in quality systems and from statistical process control, where rules decide when to react to a signal. Writing the triggers in advance avoids debate under pressure.
When to Use
Set the triggers when you write the monitoring plan, before launch. Review them after each incident and at each portfolio review.
Step-by-Step
- List the triggers across signals, changes, and time.
- Set thresholds from the validation and the tolerance.
- Name the response and the owner for each.
- Make the changes visible, such as vendor notices and process changes, so they can trigger.
- Record each trigger event and the decision.
- Review the triggers themselves.
Examples
- A run of eight weeks above the center line triggers an investigation and a re-test.
- A vendor release note triggers a golden-set re-test before the change reaches live work.
- Six months since the last test triggers scheduled re-validation.
Common Pitfalls
- Triggers written after the problem.
- No owner for the response.
- Changes nobody sees, so triggers never fire.
- Re-testing on the same cases every time.