Intraclass Correlations for Three-Level Multi-Site Cluster-Randomized Trials of Science AchievementReport as inadecuate

Intraclass Correlations for Three-Level Multi-Site Cluster-Randomized Trials of Science Achievement - Download this document for free, or read online. Document in PDF available to download.

Society for Research on Educational Effectiveness

A cluster-randomized trial (CRT) relies on random assignment of intact clusters to treatment conditions, such as classrooms or schools (Raudenbush & Bryk, 2002). One specific type of CRT, a multi-site CRT (MSCRT), is commonly employed in educational research and evaluation studies (Spybrook & Raudenbush, 2009; Spybrook, 2014; Bloom, Richburg-Hayes, & Black, 2007). The three-level MSCRT is a nested design with the level-three units (or sites) treated as a blocking variable, and the level-two units randomly assigned to treatment and control within each site. As in all experimental studies, evaluators must design CRTs with appropriate power to detect an expected effect. A common challenge for evaluators planning CRTs is selecting an appropriate intraclass correlation (ICC), an estimate of the percentage of total variance that exists at the group level, to accurately power the study. For studies with more than one level of nesting, multiple ICCs must be estimated. This study aims to improve the design of MSCRTs for science achievement studies by producing estimates of within-district ICCs across all districts in an entire state. The following research questions guide this investigation: (1) What is the distribution of within-district ICCs for science education; and (2) Does the number of districts in an MSCRT affect the mean within-district ICC? The results of this study are from one large, and representative (in terms of the grades in which testing occurs) state. Educational systems in other states may be structured quite differently, and further work in this area is needed to empirically estimate design parameters for planning MSCRTs. It will also be useful to explore whether significant differences exist between mean within-district ICC values in higher grades. Tables and figures are appended.

Descriptors: Correlation, Randomized Controlled Trials, Science Achievement, Cluster Grouping, Statistical Analysis, Science Education, School Districts, Research Design, Computation, Hierarchical Linear Modeling, Grade 5, Grade 8, Grade 10, Grade 11, Science Tests

Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries[at]; Web site:

Author: Westine, Carl D.


Related documents