EyeTrustAIRadEvalX · Completed method investigation
The denominator changes the guarantee.
A completed retrospective method comparison separates harmful outputs per processed report from error within selected reports.
Calibration / evaluation
30 / 30Reports in each role; retrospective reuse.
Selective method
0 selectedAt both 5% and 10% settings.
Marginal 10% method
1 error / 3Among its three selected reports.
Controlling harmful releases per input does not control the error fraction among released outputs. Selecting nothing is not an automation success.
What we tested
The corrected binary SCoRE-SDR method was compared with classical marginal CRC on the same previously used 30 calibration and 30 locked radiology reports. The RadCliQ score is reference-assisted. Selections were frozen before the evaluation labels were joined.
What we found
| Method | Setting | Selected | Errors among selected |
|---|---|---|---|
| Marginal CRC | 5% | 0 / 30 | Undefined: none selected |
| Selective-risk method | 5% | 0 / 30 | Undefined: none selected |
| Marginal CRC | 10% | 3 / 30 | 1 / 3 · 33.33% |
| Selective-risk method | 10% | 0 / 30 | Undefined: none selected |
The result identifies a feasibility limitation instead of claiming a successful deployment policy.
What the evidence supports
Classical CRC targeted harmful accepts per processed report. Binary SCoRE-SDR targeted expected selective deployment risk. Both small-sample constraints and the score’s ordering prevented nonzero selection by the selective method. Zero selected-set loss uses the theorem’s convention; conditional error is undefined when nothing is selected.
The statistical statement, in more detail
This is an expectation guarantee under the declared method-specific benchmark assumptions. The two methods do not certify the same quantity and their percentages are not interchangeable.
Observed evaluation error, calibration support and practical output size measure different things. The study’s declared population and reference determine how its result should be read.
Scope of the result
A reference-assisted score is not a demonstrated reference-free runtime scorer. Cohort reuse makes this a retrospective method comparison. No real-time, individual-report, clinical or operational benefit is established.
Sources & data
This summary draws on the completed experiment records. The downloadable table contains the aggregate results shown here. A standalone public reproduction package for this investigation is not linked.
Download the result table (CSV)Numbers are rounded for reading. These are research results, not measured customer outcomes. Dataset examples and individual records are not redistributed on this page.