Skip to contents

Enable leave-one-out conformal predictive intervals for a fit model

Source: R/loo_conformal.R
loo_conformal.Rd

Prepares for jackknife(+) conformal prediction by performing Pareto-smoothed importance sampling to yield leave-one-out residuals.

Usage

loo_conformal(fit, ...)

# Default S3 method
loo_conformal(
  fit,
  truth,
  chain = NULL,
  trans = function(x) x,
  inv_trans = function(x) x,
  est_fun = c("mean", "median"),
  ...
)

# S3 method for class 'stanreg'
loo_conformal(
  fit,
  trans = function(x) x,
  inv_trans = function(x) x,
  est_fun = c("mean", "median"),
  ...
)

# S3 method for class 'brmsfit'
loo_conformal(
  fit,
  trans = function(x) x,
  inv_trans = function(x) x,
  est_fun = c("mean", "median"),
  ...
)

Arguments

fit

Model fit; an object with posterior_predict() and log_lik() methods. Can also be an array of posterior predictions.

...

Ignored.

truth

True values to predict. Not required for rstanarm or brms models.

chain

An integer vector identifying the chain numbers for the posterior draws. Should be provided if multiple chains are used.

trans, inv_trans

A pair of functions to transform the predictions before performing conformal inference.

est_fun

Whether to use the posterior mean (the default) or median as a point estimate.

Value

A modified fit object with an additional class conformal. Calling predictive_interval() on this new object will yield conformal intervals.

References

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., & Gabry, J. (2015). Pareto smoothed importance sampling. arXiv preprint arXiv:1507.02646.

Examples

if (requireNamespace("rstanarm", quietly=TRUE)) suppressWarnings({
    library(rstanarm)
    # fit a simple linear regression
    m = stan_glm(mpg ~ disp + cyl, data=mtcars,
        chains=1, iter=500,
        control=list(adapt_delta=0.999), refresh=0)

    loo_conformal(m)
})
#> Loading required package: Rcpp
#> This is rstanarm version 2.32.1
#> - See https://mc-stan.org/rstanarm/articles/priors for changes to default priors!
#> - Default priors may change, so it's safest to specify priors, even if equivalent to the defaults.
#> - For execution on a local, multicore CPU with excess RAM we recommend calling
#>   options(mc.cores = parallel::detectCores())
#> stan_glm
#>  family:       gaussian [identity]
#>  formula:      mpg ~ disp + cyl
#>  observations: 32
#>  predictors:   3
#> ------
#>             Median MAD_SD
#> (Intercept) 34.9    2.5  
#> disp         0.0    0.0  
#> cyl         -1.7    0.7  
#> 
#> Auxiliary parameter(s):
#>       Median MAD_SD
#> sigma 3.1    0.4   
#> 
#> ------
#> * For help interpreting the printed output see ?print.stanreg
#> * For info on the priors used see ?prior_summary.stanreg
#> (conformalbayes enabled, with estimated CI inflation factor 0.99 and
#> transformation pair function (x) x and function (x) x)