Diabetic retinopathy model's high score prompts test for hidden shortcuts
A developer trained an EfficientNet-B0 ensemble to grade diabetic retinopathy on the APTOS 2019 dataset, reaching a quadratic weighted kappa of 0.909 on its held-out test split. A metadata-only classifier using file properties such as resolution and brightness scored 0.652, which suggested that camera-linked acquisition differences correlate with disease labels. Testing on IDRiD, a single-camera dataset, the model's specificity held up, indicating it did not rely solely on that resolution proxy. On the Messidor-2 dataset, graded by adjudicating panels, a predicted referable AUC of at least 0.93 failed, with referable sensitivity at 0.26, mainly because Moderate cases were graded lower. The author attributes this to differences in label boundaries between single graders and panels rather than to image differences, and reports that fine-tuning on adjudicated labels improved referable AUC to 0.9435 compared with a control arm.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in