Enable leave-one-out conformal predictive intervals for a fit model

Prepares for jackknife(+) conformal prediction by performing Pareto-smoothed importance sampling to yield leave-one-out residuals.

Usage

loo_conformal(fit, ...)

# S3 method for default
loo_conformal(fit, truth, chain = NULL, est_fun = c("mean", "median"), ...)

# S3 method for stanreg
loo_conformal(fit, est_fun = c("mean", "median"), ...)

# S3 method for brmsfit
loo_conformal(fit, est_fun = c("mean", "median"), ...)

Arguments

fit: Model fit; an object with posterior_predict() and log_lik() methods. Can also be an array of posterior predictions.
...: Ignored.
truth: True values to predict. Not required for rstanarm or brms models.
chain: An integer vector identifying the chain numbers for the posterior draws. Should be provided if multiple chains are used.
est_fun: Whether to use the posterior mean (the default) or median as a point estimate.

Value

A modified fit object with an additional class conformal. Calling predictive_interval() on this new object will yield conformal intervals.

References

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., & Gabry, J. (2015). Pareto smoothed importance sampling. arXiv preprint arXiv:1507.02646.

Examples

if (requireNamespace("rstanarm", quietly=TRUE)) suppressWarnings({
    library(rstanarm)
    # fit a simple linear regression
    m = stan_glm(mpg ~ disp + cyl, data=mtcars,
        chains=1, iter=1000,
        control=list(adapt_delta=0.999), refresh=0)

    loo_conformal(m)
})
#> Loading required package: Rcpp
#> This is rstanarm version 2.21.3
#> - See https://mc-stan.org/rstanarm/articles/priors for changes to default priors!
#> - Default priors may change, so it's safest to specify priors, even if equivalent to the defaults.
#> - For execution on a local, multicore CPU with excess RAM we recommend calling
#>   options(mc.cores = parallel::detectCores())
#> stan_glm
#>  family:       gaussian [identity]
#>  formula:      mpg ~ disp + cyl
#>  observations: 32
#>  predictors:   3
#> ------
#>             Median MAD_SD
#> (Intercept) 34.7    2.7  
#> disp         0.0    0.0  
#> cyl         -1.6    0.8  
#> 
#> Auxiliary parameter(s):
#>       Median MAD_SD
#> sigma 3.1    0.5   
#> 
#> ------
#> * For help interpreting the printed output see ?print.stanreg
#> * For info on the priors used see ?prior_summary.stanreg
#> (conformalbayes enabled, with estimated CI inflation factor 0.94)