EyeTrustAIChákṣu → RIGA · Completed benchmark study
A model can transfer while its calibration does not.
A new retinal-image population tests source-policy transfer and local recalibration while keeping the segmentation model fixed.
Transferred policy
19.44%Mean expert-region omission on RIGA.
Locally recalibrated
3.45%Mean omission on the same 182 images.
Spatial cost
4.88×The mean share of the image included in the region.
Local recalibration reduced omission without retraining the model. It did not recover precision for free: the regions became much larger.
What we tested
The frozen Chákṣu model was applied to RIGA. Six expert contours were reconstructed per eligible image. Of 750 originals, 679 met the reference-extraction rule. Final local calibration used 181 images, with 182 separate evaluation images for both transferred and local policies.
What we found
| Policy at the 5% target | Observed omission | Mean image area | RIGA statistical support |
|---|---|---|---|
| Transferred source policy | 19.4384% | 0.3626% | No transferred guarantee |
| RIGA-local recalibration | 3.4541% | 1.7687% | Local expected-risk calibration |
At local targets of 1%, 5%, 10% and 20%, observed omission was 0.21%, 3.45%, 6.88% and 15.48%. Mean image area was 8.91%, 1.77%, 0.59% and 0.40%, respectively. The risk budget changed the spatial output substantially.
What the evidence supports
The RIGA-local adjusted conservative calibration quantity was 4.9964%, meeting the 5% expected omission target. The source certificate did not formally transfer to RIGA.
The statistical statement, in more detail
The method controls expected loss across calibration samples and a new example under its exchangeability assumptions, with a fixed model and a bounded, monotone loss family. It is not a 95% confidence bound for each fitted rule. A finite evaluation can exceed the chosen target.
Observed evaluation error, calibration support and practical output size measure different things. The study’s declared population and reference determine how its result should be read.
Scope of the result
The guarantee averages over images and six expert references. It is not patient-level, source-specific or worst-expert control. The 23-image Magrabi slice had about 10.35% observed omission. This retrospective research comparison establishes neither clinical safety nor that recalibration replaces model adaptation. No source retinal images are redistributed here.
Sources & data
The public collection documents methods and reproduces calibration calculations and aggregate reports. It does not reconstruct every historical model prediction from raw data.
Code & reproducible researchDownload the result table (CSV)Numbers are rounded for reading. These are research results, not measured customer outcomes. Dataset examples and individual records are not redistributed on this page.