Robust Disease Classification under Spurious Shortcut Shift Using Causal Shortcut Suppression

Abstract

Shortcut learning remains a major obstacle in medical image classification because deep neural networks may rely on acquisition artifacts, scanner-specific patterns, patient-position markers, or background structures instead of disease-related morphology. This study investigates shortcut identification and removal for robust chest radiograph classification under distribution shift. We propose a Causal Shortcut Suppression Network (CSS-Net), which first estimates candidate shortcut variables through environment-wise risk variation and representation sensitivity analysis, and then reduces their influence by applying a causal invariance penalty and counterfactual feature masking during training. The model was evaluated on a multi-source chest X-ray dataset containing 112,480 images from 30,805 patients, including 14 disease labels and metadata related to scanner type, projection view, patient age, sex, and hospital source. To test robustness, we constructed three external validation settings: cross-hospital testing, projection-shift testing, and shortcut-amplified testing, where shortcut-related correlations were artificially increased from 0.35 to 0.80. Compared with a standard ResNet-50 classifier, CSS-Net improved mean AUC from 0.842 to 0.879 in internal validation and from 0.731 to 0.812 in cross-hospital validation. Under shortcut-amplified distribution shift, the false positive rate decreased by 18.6%, while sensitivity for disease-positive cases increased from 72.4% to 79.8%. Feature attribution analysis further showed that CSS-Net reduced attention on image borders and device markers by 41.3% and increased lesion-region activation by 27.6%. These results indicate that causally guided shortcut suppression can improve both predictive stability and clinical interpretability when image models are deployed across heterogeneous medical environments.