01339nas a2200121   4500008004100000245006100041210005800102260001500160520095300175100002001128700002401148856004501172       2023                        eng d00aA Model-Agnostic Heuristics for Selective Classification0 aModelAgnostic Heuristics for Selective Classification  bAAAI Press3 aSelective classification (also known as classification with reject option) conservatively extends a classifier with a selection function to determine whether or not a prediction should be accepted (i.e., trusted, used, deployed). This is a highly relevant issue in socially sensitive tasks, such as credit scoring. State-of-the-art approaches rely on Deep Neural Networks (DNNs) that train at the same time both the classifier and the selection function. These approaches are model-specific and computationally expensive. We propose a model-agnostic approach, as it can work with any base probabilistic binary classification algorithm, and it can be scalable to large tabular datasets if the base classifier is so. The proposed algorithm, called SCROSS, exploits a cross-fitting strategy and theoretical results for quantile estimation to build the selection function. Experiments on real-world data show that SCROSS improves over existing methods.1 aPugnana, Andrea1 aRuggieri, Salvatore  uhttps://doi.org/10.1609/aaai.v37i8.26133