EyeTrustAIPolypGen · Completed benchmark study
Can adaptive regions stay precise under risk control?
Endoscopic segmentation tests adaptive region construction, then examines controlled and real changes of data distribution.
Expected omission target
5%Each spatial method calibrated independently.
Held-out masks
200Across the C1–C5 source-centre population.
Adaptive region area
33.22%Compared with 48.91% for the global policy.
Adaptive scoring reduced region area, but the tail-risk findings were mixed. Better average geometry did not mean improvement on every risk measure.
What we tested
One fixed segmentation scorer supported four spatial policies: a global threshold, conformal risk allocation (CRA), calibrated risk allocation (CCRA), and CCRA within three predeclared predicted-mass strata. Each was independently calibrated before the 200-mask evaluation.
What we found
| Spatial policy | Observed mean omission | Mean image area | 95th-percentile omission |
|---|---|---|---|
| Global | 4.4608% | 48.9065% | 27.5265% |
| CRA | 4.9888% | 33.2170% | 34.4673% |
| CCRA | 5.3985% | 30.7946% | 36.4676% |
| CCRA-S | 4.6164% | 36.0718% | 31.4281% |
CRA reduced cases with more than 50% omission from seven to two, while its 95th-percentile omission increased. CCRA did not improve incrementally over CRA on the locked risk gap or 95th percentile. CCRA-S improved both relative to CCRA at the cost of a larger region.
Known distribution change: a controlled method experiment
A separate experiment constructed source and target distributions with exactly known weights. At 200 calibration examples, exact weighted CRC had 1.36% mean target omission over 5,000 calibration replicates, with 86.50% mean image area. This validates the known-weight construction, not an estimated-weight guarantee in a new hospital.
Real centre transfer: the hard case
On 83 eligible C6 images, naive source transfer produced 8.59% mean omission at 54.84% image area. Classifier-estimated weighting reduced omission to 2.51% while expanding area to 78.54%; it had no exact finite-sample C6 certificate. Kernel-mean-matching variants returned the entire image and did not certify.
Local evidence and its cost
A labelled target-local comparator calibrated on 41 C6 cases and evaluated on 42 separate cases. It observed 1.94% omission at 91.16% image area. These results expose the limits of transfer, not universal superiority of weighting or adaptation.
What the evidence supports
All four policies calibrated at 5%, 10% and 20%. Global, CRA and CCRA targeted marginal expected image-level omission. CCRA-S also controlled expected loss within its three fixed predicted-mass strata; that does not imply centre-specific or individual-image control.
The statistical statement, in more detail
The method controls expected loss across calibration samples and a new example under its exchangeability assumptions, with a fixed model and a bounded, monotone loss family. It is not a 95% confidence bound for each fitted rule. A finite evaluation can exceed the chosen target.
Observed evaluation error, calibration support and practical output size measure different things. The study’s declared population and reference determine how its result should be read.
Scope of the result
Centre effects remained heterogeneous, with only 15 C4 images in the source evaluation. The real C6 study used estimated weights, so the exact known-weight theorem cannot be applied to those estimates as if they were true ratios. The public segmentation repository currently covers Chákṣu and RIGA, not this PolypGen program. This page presents aggregate research results only.
Sources & data
This summary draws on the completed experiment records. The downloadable table contains the aggregate results shown here. A standalone public reproduction package for this investigation is not linked.
Download the result table (CSV)Numbers are rounded for reading. These are research results, not measured customer outcomes. Dataset examples and individual records are not redistributed on this page.