Fundamental statistical methods for IPD metaanalysis

Statistical methods for IPD metaanalysis are needed for a quantitative synthesis of the IPD.

These use either a twostage or a onestage approach to produce summary results (e.g. about treatment effect).

Recommendations for choosing between the approaches are available.

Below we provide key guidance, and for further details see Chapters 5 to 8 of our book and various references.
Twostage approach

The first stage typically involves a standard regression analysis in each trial separately to produce aggregate data, such as treatment effect estimates and their variances.

The second stage uses wellknown (e.g. inverse variance weighted) metaanalysis methods to combine this aggregate data and produce summary results and forest plots. Either a commoneffect or a randomeffects model is assumed.

A commoneffect model assumes that the true treatment effect is the same in every trial.

A randomeffects model allows for betweentrial heterogeneity in the true treatment effect, and is more plausible because included trials often differ in their characteristics.

A frequentist or Bayesian estimation framework can be used.

Bayesian estimation is appealing, to produce direct probabilistic statements that account for all parameter uncertainty, and to include prior distributions for the betweentrial variance.

In a frequentist framework, restricted maximum likelihood (REML) estimation is recommended for fitting the randomeffects model in the second stage, with confidence intervals derived using the approach of HartungKnappSidikJonkman.

Heterogeneity can be summarised by the estimate of betweentrial variance of true treatment effects, and a 95% prediction interval for the potential true treatment effect in a new trial.

A metaregression extends the randomeffects model by including triallevel covariates (that define subgroups of trials) that may explain betweentrial heterogeneity. However, metaregression usually has low power and should be interpreted cautiously.

If appropriate, aggregate data from nonIPD trials can be included in the second stage of the twostage metaanalysis, alongside the aggregate data derived directly from IPD trials.

This helps explore whether the main IPD metaanalysis results (based on IPD trials only) are robust to the inclusion of nonIPD trials. However, suitable aggregate data from nonIPD trials may be limited.

A twostage IPD metaanalysis can be implemented using the ipdmetan package in Stata, for a wide variety of outcome types.
Onestage approach

A onestage approach analyses the IPD from all trials altogether in a single statistical analysis.

This typically requires a generalised linear mixed model (GLMM) or a hierarchical survival (frailty) model, which extend standard models (such as linear, logistic, Poisson and Cox) used in a single trial setting.

A onestage IPD metaanalysis utilises a more exact statistical likelihood than a twostage metaanalysis approach, which is advantageous when included trials have few participants or outcome events.

Onestage IPD metaanalysis models usually include multiple parameters and these are estimated simultaneously. For each parameter (such as the intercept, treatment effect, residual variances).

The analyst must specify whether they are common (the same in each trial), stratified (different in each trial) or random (different in each trial and assumed drawn from a particular distribution).

Clustering of participants within trials must be accounted for (e.g. using a stratified trial intercept or random trial intercepts for GLMMs), as otherwise summary metaanalysis results may be biased or overly precise.

A stratified trial intercept is generally preferred, unless there are computational concerns.

The use of random trial intercepts allows information about baseline risk to be shared across trials, which may compromise randomisation within each trial (e.g. when the treatment:control allocation ratio differs across trials).

Restricted maximum likelihood (REML) estimation is recommended for onestage models with continuous outcomes, with confidence intervals derived using the approach of KenwardRoger or Satterthwaite.

For binary outcomes, unless most included trials have sparse numbers of events, REML estimation of a pseudolikelihood is recommended.

Where REML or a pseudolikelihood is not available or not appropriate, maximum likelihood estimation of a onestage model with a stratified intercept can be improved by trialspecific centering of the treatment variable, and by deriving confidence intervals using the tdistribution.

Bayesian estimation of onestage models is an appealing alternative to frequentist estimation methods, especially to enable direct probabilistic statements that account for all parameter uncertainty.
Twostage versus onestage: differences and recommendations
Twostage, or not twostage, that is the question

Many articles have compared the onestage and twostage approaches via theory, simulation and empirical examples.

Crucially, differences in onestage and twostage summary results will usually be small when the same assumptions and estimation methods are used.

When notable differences do arise, generally they are caused by a change in modelling assumptions and/or estimation methods, and not due to using a onestage or twostage process per se.

Thus, the choice of model assumptions and estimation methods is usually more important than selection of a onestage or twostage approach.

An important exception is when most trials in the metaanalysis are small (in terms of numbers of participants or outcome events). In this case a onestage approach is recommended as it uses a more exact statistical likelihood than that assumed in the second stage of the twostage approach.

In other situations, researchers can feel free to choose either a onestage or a twostage approach, with due care given to their choice of modelling assumptions, parameter specification, and estimation methods.

Unless most trials in the IPD metaanalysis are small, generally the twostage approach will suffice. It is also more accessible (especially for those familiar with conventional aggregate data metaanalysis approaches), and more easily enables visual summaries (e.g. forest plots). It is also more convenient for including trials that only provide remote access to their IPD (so cannot be merged with other IPD), and trials that only provide aggregate data.

It may be helpful to do both a onestage and a twostage analysis (and report both), to check whether conclusions are robust to the choice of approach.