Title: A Unified Framework for Reliable Deep Learning: Conformal Prediction Meets Out-of-Distribution Detection

Authors: Long Xue; Tao Wang; Yiu-ming Cheung; Xingyu Zhao; Sai-Ho Chung; Xiaoge Zhang

Abstract: Ensuring the safety and reliability of deep learning models is critical for their dependable deployment in real-world settings. In this paper, we introduce a unified framework that integrates conformal prediction (CP) with out-of-distribution (OOD) detection to robustly handle diverse inputs and deliver reliable predictions for downstream decision-making. CP provides distribution-free, finite-sample coverage guarantees by constructing prediction sets that contain the true label with a user-specified probability. However, in an open-world environment, the presence of OOD inputs violates the exchangeability assumption that underpins CP’s statistical validity, potentially undermining the coverage guarantee. To address this challenge, we propose a distance-aware OOD detection mechanism that combines spectral normalization with a sparse variational Gaussian process, enabling effective separation of in-distribution (InD) and OOD inputs. While no OOD detector is perfect, we provide a rigorous theoretical analysis of how the undetected OOD contamination affects CP’s coverage, and derives corresponding worst-case coverage bounds. Additionally, we introduce a differentiable loss function that penalizes large prediction sets, encouraging the model to produce compact and informative sets. Together, these components yield an end-to-end trainable framework that delivers strong OOD detection performance, reliable prediction sets, and computational efficiency in a single forward pass. Extensive experiments demonstrate that our method achieves competitive performance with state-of-the-art baselines while substantially reducing the average prediction set size and providing reliable uncertainty quantification in open-world scenarios.

Learn More