Meta-analytic leave-p-out cross validation

Use leave-p-out cross-validation (LpO CV) to compare an external control analysis approach that uses the ecmeta package to adjust for bias to a standard approach that makes no adjustments. LpO CV uses p data points in the data for testing and all other data points for training.

ecmeta_lpo_cv(
  data,
  p = 1,
  method = "ml",
  ecmeta_args = NULL,
  predict_args = NULL
)

Arguments

data	A data frame with columns param, analysis_num, estimate, and se. param should have three values representing the log hazard ratio of a comparison between (i) the treatment and the external control (trt_ec), (ii) the internal control and the external control (ic_ec), and (iii) the treatment and the internal control (trt_ic). analysis_num is the analysis (i.e., study) number, estimate is the point estimate, and se is the standard error.
p	Number of data points to use for testing. Default is 1 which is leave-one-out cross validation.
method	The method/software used to estimate the meta-analytic model as in ecmeta.
ecmeta_args	A named list containing additional arguments to pass to ecmeta::ecmeta().
predict_args	A named list containing additional arguments to pass to ecmeta::predict.ecmeta().

Value

A data frame where there are np rows (n = each time the model is trained and tested; p is the number of testing observations for each of the n iterations). Each row contains performance metrics.

Details

LpO CV requires training and validating the dataset \(C_p^n\) where \(n\) is the number of observations in the original dataset. For each of the \(C_p^n\) iterations, a meta-analytic model is fit to the training data using ecmeta::ecmeta() and a prediction on the training set is made with ecmeta::predict.ecmeta(). Predictions that are adjusted with the meta-analytic method are compared to those without using any adjustment.