Jackknife(+) predictive intervals — predictive_interval.conformal • conformalbayes

Construct finite-sample calibrated predictive intervals for Bayesian models, following the approach in Barber et al. (2021). By default, the intervals will also reflect the relative uncertainty in the Bayesian model, using the locally-weighted conformal methods of Lei et al. (2018).

Usage

# S3 method for conformal
predictive_interval(object, probs = 0.9, plus = NULL, local = TRUE, ...)

Arguments

object: A fitted model which has been passed through loo_conformal()
probs: The coverage probabilities to calculate intervals for. Empirically, the coverage rate of the constructed intervals will generally match these probabilities, but the theoretical guarantee for a probability of \(1-\alpha\) is only for coverage of at least \(1-2\alpha\), and only if plus=TRUE (below).
plus: If TRUE, construct jackknife+ intervals, which have a theoretical guarantee. These require higher computational costs, which scale with both the number of training and prediction points. Defaults to TRUE when both of these numbers are less than 500.
local: If TRUE (the default), perform locally-weighted conformal inference. This will inflate the width of the predictive intervals by a constant amount across all predictions, preserving the relative amount of uncertainty captured by the model. If FALSE, all predictive intervals will have (nearly) the same width.
...: Further arguments to the posterior_predict() method for object.

Value

A matrix with the number of rows matching the number of predictions. Columns will be labeled with a percentile corresponding to probs; e.g. if probs=0.9 the columns will be 5% and 95%.

References

Barber, R. F., Candes, E. J., Ramdas, A., & Tibshirani, R. J. (2021). Predictive inference with the jackknife+. The Annals of Statistics, 49(1), 486-507.

Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018). Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523), 1094-1111.

Examples

if (requireNamespace("rstanarm", quietly=TRUE)) suppressWarnings({
    library(rstanarm)
    # fit a simple linear regression
    m = stan_glm(mpg ~ disp + cyl, data=mtcars,
        chains=1, iter=1000,
        control=list(adapt_delta=0.999), refresh=0)

    m = loo_conformal(m)
    # make predictive intervals
    predictive_interval(m)
})
#>                            5%      95%
#> Mazda RX4           17.110736 26.90264
#> Mazda RX4 Wag       17.268230 26.73333
#> Datsun 710          21.941012 31.39870
#> Hornet 4 Drive      15.243863 24.92812
#> Hornet Sportabout    9.726555 19.60827
#> Valiant             15.596218 25.40610
#> Duster 360           9.504357 19.22656
#> Merc 240D           20.180579 30.86500
#> Merc 230            20.807250 30.73146
#> Merc 280            17.035988 26.40159
#> Merc 280C           16.709179 26.46263
#> Merc 450SE          11.328152 21.52446
#> Merc 450SL          11.335674 21.03168
#> Merc 450SLC         11.263284 21.38011
#> Cadillac Fleetwood   6.616124 18.02776
#> Lincoln Continental  6.867457 18.06346
#> Chrysler Imperial    7.736221 17.83694
#> Fiat 128            22.164125 32.14907
#> Honda Civic         21.960209 32.07576
#> Toyota Corolla      22.135744 31.95427
#> Toyota Corona       21.133056 31.30478
#> Dodge Challenger    10.040998 20.24118
#> AMC Javelin         10.870161 20.61888
#> Camaro Z28           9.814642 19.47436
#> Pontiac Firebird     8.900908 18.89298
#> Fiat X1-9           22.341701 32.20451
#> Porsche 914-2       21.215152 31.51973
#> Lotus Europa        21.553038 31.24433
#> Ford Pantera L       9.450562 19.28230
#> Ferrari Dino        17.126817 27.39371
#> Maserati Bora       10.782410 20.76264
#> Volvo 142E          20.796653 31.32089