Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: a review, reanalysis, and simulation studyReport as inadecuate




Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: a review, reanalysis, and simulation study - Download this document for free, or read online. Document in PDF available to download.

Trials

, 17:438

First Online: 06 September 2016Received: 20 April 2016Accepted: 24 August 2016DOI: 10.1186-s13063-016-1571-2

Cite this article as: Kahan, B.C., Forbes, G., Ali, Y. et al. Trials 2016 17: 438. doi:10.1186-s13063-016-1571-2

Abstract

BackgroundCluster randomised trials CRTs are commonly analysed using mixed-effects models or generalised estimating equations GEEs. However, these analyses do not always perform well with the small number of clusters typical of most CRTs. They can lead to increased risk of a type I error finding a statistically significant treatment effect when it does not exist if appropriate corrections are not used.

MethodsWe conducted a small simulation study to evaluate the impact of using small-sample corrections for mixed-effects models or GEEs in CRTs with a small number of clusters. We then reanalysed data from TRIGGER, a CRT with six clusters, to determine the effect of using an inappropriate analysis method in practice. Finally, we reviewed 100 CRTs previously identified by a search on PubMed in order to assess whether trials were using appropriate methods of analysis. Trials were classified as at risk of an increased type I error rate if they did not report using an analysis method which accounted for clustering, or if they had fewer than 40 clusters and performed an individual-level analysis without reporting the use of an appropriate small-sample correction.

ResultsOur simulation study found that using mixed-effects models or GEEs without an appropriate correction led to inflated type I error rates, even for as many as 70 clusters. Conversely, using small-sample corrections provided correct type I error rates across all scenarios. Reanalysis of the TRIGGER trial found that inappropriate methods of analysis gave much smaller P values P ≤ 0.01 than appropriate methods P = 0.04–0.15. In our review, of the 99 trials that reported the number of clusters, 64 65 % were at risk of an increased type I error rate; 14 trials did not report using an analysis method which accounted for clustering, and 50 trials with fewer than 40 clusters performed an individual-level analysis without reporting the use of an appropriate correction.

ConclusionsCRTs with a small or medium number of clusters are at risk of an inflated type I error rate unless appropriate analysis methods are used. Investigators should consider using small-sample corrections with mixed-effects models or GEEs to ensure valid results.

KeywordsCluster randomised trials Small-sample corrections Degree-of-freedom corrections Mixed-effects models Generalised estimating equations AbbreviationsCRTCluster randomised trial

CIConfidence interval

GEEGeneralised estimating equations

TRIGGERTrial in Gastrointestinal Transfusion

Download fulltext PDF



Author: Brennan C. Kahan - Gordon Forbes - Yunus Ali - Vipul Jairath - Stephen Bremner - Michael O. Harhay - Richard Hooper - Nei

Source: https://link.springer.com/







Related documents