An Investigation of the Impact of Misrouting under Two-Stage Multistage Testing: A Simulation Study. Research Report. ETS RR-14-01Report as inadecuate




An Investigation of the Impact of Misrouting under Two-Stage Multistage Testing: A Simulation Study. Research Report. ETS RR-14-01 - Download this document for free, or read online. Document in PDF available to download.



ETS Research Report Series, Jun 2014

The purpose of this study was to investigate the potential impact of misrouting under a 2-stage multistage test (MST) design, which includes 1 routing and 3 second-stage modules. Simulations were used to create a situation in which a large group of examinees took each of the 3 possible MST paths (high, middle, and low). We compared differences in examinees' scores associated with different paths under 2 MST assembly conditions: a small-difference condition in which 3 modules at Stage 2 overlap in difficulty, and a large-difference condition in which the 3 modules at Stage 2 are distinct in difficulty. We also compared examinees' score differences associated with the target MST path with the ones obtained when the second-stage module was 1 level of difficulty above or below the target module. When the second-stage modules overlapped in difficulty, the score differences (i.e., bias) associated with different MST paths were negligible for practical purposes. Similar trends appeared even when the second-stage modules were significantly distinct in difficulty. The impact of misrouting was generally minimal under the MST design used in this study.

Descriptors: Comparative Analysis, Difficulty Level, Scores, Test Wiseness, Cognitive Style, Simulation, Test Format, Learning Modules, Equated Scores, Raw Scores, Group Testing, Sequential Approach, Computer Assisted Testing

Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb[at]ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets





Author: Kim, Sooyeon; Moses, Tim

Source: https://eric.ed.gov/?q=a&ft=on&ff1=dtySince_1992&pg=816&id=EJ1109329







Related documents