Comparing Conformal and Quantile Regression for Uncertainty Quantification: An Empirical Investigation
DOI:
https://doi.org/10.47941/ijce.1925Keywords:
Uncertainty Quantification, Machine Learning, Quantile Regression, Conformal Regression, Prediction Intervals, Error Rate, CatboostAbstract
Purpose: This research assesses the efficacy of conformal regression and standard quantile regression in uncertainty quantification for predictive modeling. Quantile regression estimates various quantiles within the conditional distribution, while conformal regression constructs prediction intervals with guaranteed coverage.
Methodology: By training models on multiple quantile pairs and varying error rates, the analysis evaluates each method's performance.
Findings: Results indicate consistent trends in coverage and prediction interval lengths, with no significant differences in performance. Quantile regression intervals lengthen toward the distribution tails, while conformal regression intervals lengthen with higher coverage.
Unique contribution to theory, policy and practice: On the tested dataset, both methods perform similarly, but further testing is necessary to validate these findings across diverse datasets and conditions, considering computational efficiency and implementation ease to determine the best method for specific applications.
Downloads
References
Y. Romano, E. Patterson, and E. Candes, “Conformalized quantile regression,” Advances in neural information processing systems, vol. 32, 2019.
R. Koenker and G. Bassett, “Regression Quantiles,” Econometrica, vol. 46, no. 1, p. 33, Jan. 1978, doi: https://doi.org/10.2307/1913643.
R. Koenker and K. F. Hallock, “Quantile regression,” Journal of economic perspectives, vol. 15, Art. no. 4, 2001.
N. Meinshausen and G. Ridgeway, “Quantile regression forests.,” Journal of machine learning research, vol. 7, Art. no. 6, 2006.
I. Steinwart and A. Christmann, “Estimating conditional quantiles with the help of the pinball loss,” 2011.
I. Takeuchi, Q. Le, T. Sears, and A. Smola, “Nonparametric quantile estimation,” MIT Press, 2006.
V. Vovk, A. Gammerman, and G. Shafer, Algorithmic learning in a random world, vol. 29. Springer, 2005.
V. Vovk, I. Nouretdinov, and A. Gammerman, “On-line predictive linear regression,” JSTOR, 2009.
J. Lei, J. Robins, and L. Wasserman, “Distribution-free prediction sets,” Journal of the American Statistical Association, vol. 108, Art. no. 501, 2013.
A. N. Angelopoulos and S. Bates, “A gentle introduction to conformal prediction and Distribution-Free uncertainty quantification,” arXiv.org, Jul. 15, 2021. https://arxiv.org/abs/2107.07511
J. C. Cresswell, Y. Sui, B. Kumar, and N. Vouitsis, “Conformal prediction sets improve human decision making,” arXiv.org, Jan. 24, 2024. https://arxiv.org/abs/2401.13744
I.-C. Yeh, “Concrete Compressive strength.” UCI Machine Learning Repository, 2007. doi: 10.24432/C5PK67.
L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, “CatBoost: unbiased boosting with categorical features,” Advances in neural information processing systems, vol. 31, 2018.
Donlnz, “GitHub - donlnz/nonconformist: Python implementation of the conformal prediction framework.,” GitHub. https://github.com/donlnz/nonconformist
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Bhargava Kumar, Tejaswini Kumar, Swapna Nadakuditi, Hitesh Patel, Karan Gupta
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution (CC-BY) 4.0 License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
