A shared way for clinicians and vendors to say how serious an AI mistake is
How to grade:
Ask one question: “What could happen if a clinician acted on this answer?” Grade the possible harm, not how wrong the answer looks.
Tier 1: Unsafe
What it means: Acting on it could cause patient harm.
Example: The summary says “no known drug allergies” when a penicillin allergy is documented.
What to do: Pause that use, review the same day, and find the root cause before restarting.
Tier 2: Misleading
What it means: The answer is wrong or confusing in a way a clinician might not catch.
Example: A discontinued medication is listed as active.
What to do: Fix quickly, add the case to the test set, and check for similar cases.
Tier 3: Incomplete
What it means: The answer is correct but leaves out something that matters.
Example: A summary mentions a lab result but not that it has been rising for three visits.
What to do: Log it, review monthly, and look for patterns.
Tier 4: Cosmetic
What it means: Wording or formatting problems that don’t change the meaning.
Example: Awkward phrasing, or a heading in the wrong place.
What to do: Batch these for routine updates.
Tips for using it:
The same error can be a different tier in a different workflow. A wrong date in a draft letter is minor, but in a medication history it isn’t.
Have clinicians assign the tier, not the vendor.
Agree on response times for each tier before go-live, not after the first incident.