We built this methodology by diagnosing real failures in production healthcare AI systems and designing the infrastructure to catch them before they reach patients.
How we work
Three phases. Each phase moves the needle on eval capability and maturity.
Our engagements follow this methodology with actual work scope tailored to your existing maturity level, eval tools and processes.
Phase 1
01
Instrumentation and observability
2-6 weeks
Phase 2
02
Structured
Evaluation
4-8 weeks
Phase 3
03
Continuous
Governance
4-12 weeks, then ongoing
Instrumentation and observability
Phase 1: Complete visibility into every AI decision your systems make
Phase 1 wires your AI systems for structured observability so that every LLM call, every retrieval event, every tool invocation, and every agent decision is captured as a trace with the metadata that makes it queryable later.
That means prompt text and completion text, model version and generation parameters, token counts and latency at each span, and the identifiers that tie everything together: session ID, user ID, patient ID, experiment ID. The trace hierarchy mirrors your actual system architecture, so a multi-step generation pipeline reads as a single traceable event rather than a collection of disconnected API calls.
Traces go into structured storage with retention policies, access controls, and query interfaces that both your engineering team and your clinical team can work with. This is the observation layer that your entire evaluation infrastructure will depend on, and getting the architecture right at this stage is what makes rigorous evaluation possible in Phase 2.
Timeline
Phase 1 typically runs 2 to 6 weeks. By the end, your AI systems are fully observable, every output is traceable, and every failure is reproducible.
Structured Evaluation
Phase 2: The evaluation layer that tells you exactly what is working and what is not
Phase 2 builds the evaluation infrastructure that tells you whether your AI is right, how often it is wrong, and what kind of wrong it is when it fails. This phase works directly with clinical subject matter experts, and there is no shortcut around that requirement.
Golden dataset construction
We work with your SMEs to build a labeled dataset of known-correct outputs for your specific workflows. These are real clinical scenarios reviewed and annotated by your clinicians, covering the distribution of cases your AI actually encounters in production, with edge cases weighted deliberately. The golden dataset becomes the ground truth that your entire evaluation system scores against.
Eval metrics
What gets measured depends on the workflow under evaluation. For medical coding, that means exact match accuracy, item-level accuracy, fabrication rate, abstention accuracy, and support level distinction. For clinical documentation, it means factual consistency, completeness against source records, and hallucination rate. Metrics are defined per workflow rather than per model, because the same model can perform very differently across different clinical tasks.
Failure taxonomy
Treating all errors the same way wastes engineering time, so we categorize them into four distinct buckets based on root cause. A fabrication, where the model invented clinical evidence, requires a completely different fix than a documentation gap, where the evidence existed but never reached the model. A wrong abstention, where the model refused to answer despite having sufficient evidence, requires a different fix than a generation coding error, where the model attempted the item and applied the wrong clinical logic. Each bucket routes to a different upstream layer of the system so the fix goes to the right place.
Domain-specific rubrics
A rubric is what bridges the gap between a raw metric and an actual clinical judgment. Asking "is this OASIS item coded correctly" requires more than a yes or no. The rubric needs to capture the actual CMS coding logic for that specific item, spelling out what clinical evidence supports each code value, what documentation would justify it, and what the most common misapplication patterns look like in practice. We build rubrics at that level of specificity for every item type under evaluation, because generic "correct or incorrect" scoring misses the clinical nuance that determines whether an output would hold up under audit.
Evaluators
Three types of evaluators run together, each handling a different layer of the scoring. Rule-based scorers cover the deterministic checks like whether required fields are present, whether code values fall within valid sets, and whether dates land inside the assessment window. LLM-judge scorers take on the more ambiguous cases, applying rubrics to evaluate whether the clinical reasoning in an output actually matches the logic that a given item type requires. Human-in-the-loop annotation validates what both automated layers produce and continuously feeds new labeled examples back into the golden dataset, keeping the ground truth current as your workflows and patient population evolve.
Experiments
Every evaluation runs as a structured experiment with a clearly defined scope, covering which items are included, which patient cohort is used, and which prompt version is being tested, along with fully traced execution and scored results stored with complete provenance. That structure means you can compare prompt version A against version B across the same golden dataset and see exactly where performance shifted, by how much, and in which direction, giving your team the evidence they need to make confident decisions about what to ship.
Timeline
Phase 2 typically runs 4 to 8 weeks. By the end, you have a working evaluation system producing scored results against clinical ground truth.
Continuous Governance
Phase 3: Where evaluation becomes a self-improving system that is audit-ready at every moment
Phase 3 is where the evaluation infrastructure you built in Phase 2 evolves into something more sophisticated: a continuous governance layer that improves itself over time, produces the documentation regulators and procurement teams are starting to require, and makes every AI output in your organization traceable back to the evidence that supports it.
This is the phase that separates organizations running AI from organizations that can prove their AI works.
CI/CD integration
Evaluation becomes part of the deployment pipeline itself. Every prompt change, retrieval configuration update, or model version swap triggers a traced evaluation run against the golden dataset, with results scored automatically and regressions flagged before they reach production. A prompt change that drops accuracy on a specific item type from 88% to 72% simply does not ship. Your release process and your evaluation process become one and the same.
Model cards
Every AI system in production receives a model card: a structured, living artifact documenting what the model does, what evaluation has been performed, what the known limitations are, and how it performs across defined subgroups. This is the artifact format CHAI (Coalition for Health AI) has standardized. It is what procurement teams and regulators are starting to ask for. We produce it as a deployment artifact, not a retrospective document.
Evidence linkage
Every claim in a generated output is traced back to the specific source evidence that supports it. If the AI generates a functional status code, the evidence linkage shows exactly which clinical documentation supports that code, how strong the support is, and whether any contradictory evidence exists elsewhere in the record. This is the layer that transforms AI outputs from "the model said so" into independently verifiable clinical decisions.
HITL override analysis
When a clinician overrides an AI recommendation, that override is captured, categorized, and fed back into the golden dataset as a new labeled example. Over time, override patterns reveal systematic insights about the model, the rubric, or the data that would remain invisible from evaluation metrics alone. This is where the judgment of your clinical staff becomes a direct input to the system's continuous improvement, creating a feedback loop between the people who use the AI and the infrastructure that evaluates it.
Calibration
LLM-as-judge scorers are calibrated on a scheduled cadence against human annotator judgment. We run calibration sets where the same cases are scored by both the LLM judge and human reviewers, measure agreement, and refine the judge prompt and rubric until alignment reaches the threshold your clinical team requires. Because both the model behavior and the clinical context evolve over time, calibration is a recurring discipline rather than a one-time setup.
Self-improvement loops
This is where all the Phase 3 components converge. HITL overrides feed new labeled examples into the golden dataset, making it richer and more representative over time. Calibration helps refine AI scorer/evaluator accuracy. Failure taxonomy analysis reveals which error categories are declining and which ones persist. Experiment results across prompt versions surface which changes produced genuine improvement. The evaluation system measures itself measuring the AI, and both layers improve in concert.
Timeline
Phase 3 is ongoing and is typically built in about 4 to 12 weeks, with the governance loops running continuously from that point forward.