← Back to the workflow

Evidence & evaluation

What the guarantee covers.
What your notes must establish.

Separate the error definition, the prepared demonstration and the evidence needed for your own workflow.

The statistical statement

Errors in original drafts
eligible to skip the extra source check.

Numerator: eligible original drafts containing at least one agreed medication or allergy error relative to the supplied source. Denominator: all drafts eligible to skip the additional structured source check, before audits.

Agree which errors count, including medication, dose, allergy and omission cases. A later audit, correction or signature does not change the original draft’s classification. Conformal risk control must support this selected-draft statement, rather than a different average over all notes.

Scope, assumptions and clinical meaning

Specify source availability and quality, language, specialty, note type, model/template/scorer versions and the statistical confidence or risk statement. Address selection bias, repeated patients and related encounters. Keep scorer development, policy selection and independent evaluation appropriately separated.

Source fidelity does not establish clinical truth: an incomplete or incorrect source can be faithfully reproduced. Every note retains clinician review and sign-off; this statement does not validate treatment decisions.

What is demonstrated today

Synthetic encounters.
Prepared policy outcomes.

The note walkthrough illustrates an additional source check and a separate draft proceeding to routine clinician review. It uses synthetic encounters and prepared results, not a calculated clinical bound, live integration or measured customer saving.

In the prepared 1,000-note scenario, 600 original drafts are eligible to skip the additional check. Six contain a target error: an observed 1% in that eligible group. The illustrated target is 2%; the scenario does not calculate statistical support for that target.

Open the note example

What your evaluation establishes

A comparison with your existing checking process.

  1. 01 / Define

    Agree the task and reference

    Choose one workflow, the error protocol, risk target and comparison process with clinical and product owners.

  2. 02 / Prepare

    Assess the source and reviews

    Retain uncorrected drafts, source snapshots, versions, sampling and patient groupings. Resolve reference disagreements.

  3. 03 / Assess

    Report risk and operating value

    Establish statistical support where possible, then compare independent results, checks, audits, corrections and clinical time.

The evaluation delivers a scoped risk statement, evidence limitations, operating comparison and integration requirements. It may support a monitored pilot, require further preparation or retain full additional checking.

Evidence you already have

Keep the original draft
and the source available at drafting.

Existing reviews may be reusable when source snapshots, original drafts, compatible versions, reviewer decisions and selection history remain available. Corrected or signed notes alone cannot reveal the original error.

Check missing cases, source quality, selection bias, repeated encounters and any use in scorer development. Prior labels are not automatically fresh calibration evidence. A new language, specialty, template or encounter mix requires a scope review.

There is no universal label count or deployment time. Requirements depend on error prevalence, the target limit, scorer quality, dependence, reference quality and intended use.

Measuring the value

Extra checks avoided.
Audits and clinical work retained.

The prepared 1,000-note example retains 400 additional structured checks and 60 audits: 460 extra checks, or 540 fewer than checking every draft. Every note still requires clinician review and sign-off.

That is an illustrative task comparison against a review-all baseline. Your evaluation must use your actual current process and measure checking time, corrections, investigation, monitoring and operating cost. Reduced task counts do not by themselves establish a time saving.

Integration and operation

Agree who supplies and maintains each part.

EyeTrustAI

Statistical design and assessment

Assess evidence and scorer suitability, establish supported limits and report checking effort and reassessment requirements.

Your product team

Integration and operation

Own model/template versions, activation, monitoring, the review interface and the agreed data handling and source access.

Clinical partners

References and clinical review

Supply qualified reviews, resolve disagreements, carry out audits and retain corrections, sign-off and care decisions.

What happens when the workflow changes?

Monitor versions, source quality, language and encounter mix. Unchanged models can face changed populations. Audit results arrive after the work; score shifts are investigation signals rather than proof of an error-rate breach.

Agree when reduced checking stops, who investigates affected notes and what evidence is needed to resume. New work returns to additional checking on suspension. A clinical owner decides whether earlier records require review or amendment; the demo does not recall or overwrite signed records.

Discuss a note evaluation Discuss a clinical collaboration