Challenges of a cross-national computer-based test adaptationReport as inadecuate




Challenges of a cross-national computer-based test adaptation - Download this document for free, or read online. Document in PDF available to download.

Empirical Research in Vocational Education and Training

, 8:18

Technology-based assessment of domain-specific work-related competences

Abstract

BackgroundIn an increasingly globalized world, the call for internationally comparable competence measurements has emerged. After several international studies on pre-college education, the focus has shifted to international assessments of vocational education and training VET. VET researchers in Germany developed a computer-based test ALUSIM that measures the competence of German apprentices in internationally defined core commercial areas. Our own study deals with the adaptation of this test to Switzerland and a discussion of the challenges involved, with the aim of providing guidance for future adaptations. More specifically, despite commonalities between the German and Swiss VET systems, it is necessary to examine whether the contents and technical aspects of ALUSIM are appropriate for Swiss conditions in order to ensure validity and evidence based on test content.

MethodsSeveral methods were used to examine the criteria context, construct, and IT components of the German computer-based test ALUSIM in order to successfully adapt it for Swiss commercial VET. To this end we first analyzed and compared the German and Swiss commercial VET systems and commercial curricula context to assess whether the tasks of the test were also embedded in the Swiss curriculum and whether any specific Swiss commercial contents were not represented by the test. Second, we interviewed experts in the commercial area to learn more about representative commercial job requirements construct. Finally, we interviewed apprentices and tested our initial adaptation of ALUSIM to the Swiss context in order to assess the test’s IT requirements.

ResultsThe analysis revealed similarities between the German and Swiss VET and the construct ‘commercial competence’. However, commercial work conditions and cultural characteristics differ between these countries and lead to different job requirements. Therefore, only a subset of ALUSIM tasks is valid for the construct ‘commercial competence’ in Switzerland. Thus, the addition of further tasks for Switzerland would more validly represent the construct ‘commercial competence’. Moreover, IT components need to be adjusted because the technical implementation represents the measured construct.

ConclusionsThe proceedings and findings of our adaptation study imply that context, construct, and IT components need to be analyzed before doing adaptations. Even when dealing with countries with a similar context such as comparable educational system and language, it is necessary to carefully examine and test an adaptation in advance. Therefore, creating a successful and internationally comparable adaptation is admittedly possible but challenging, costly and time-consuming.

Download fulltext PDF



Author: Doreen Holtsch - Silja Rohr-Mentele - Eva Wenger - Franz Eberle - Richard J. Shavelson

Source: https://link.springer.com/



DOWNLOAD PDF




Related documents