The sensitivity of TIMSS country rankings in science achievement to differences in opportunity to learn at classroom level

Periodical
Large-scale Assessments in Education
Volume
6
Year
2018
Issue number
1
Page range
1-31
Relates to study/studies
TIMSS 2015

The sensitivity of TIMSS country rankings in science achievement to differences in opportunity to learn at classroom level

Abstract

Background

Fair comparisons of educational systems in large-scale assessments can be made only if the differences in curricula have little impact on the outcomes. This study investigated the sensitivity of science achievement rankings to varying degrees of curriculum implementation in the Trends in International Mathematics and Science Study (TIMSS).

Methods

Country-specific teacher-reported curriculum implementation profiles across the TIMSS science domains were charted including their within-country variability across the classrooms for 33 participating countries of TIMSS 2015. A sensitivity test compared the original ranking to TIMSS curriculum implementation scenarios (a least-possible, a most-possible, and more realistic country-specific median implementation).

Results

In contrast to expectations, no support was found for a positive relationship between opportunity to learn and science achievement at the between-country level or the within-country level, with only minor exceptions. The sensitivity analysis under different curriculum implementation scenarios also suggests little impact on the rank order of the countries.

Conclusions

Plausible explanations for this null finding are addressed; attention and research efforts should focus on improving the quality of curriculum implementation indicators in large-scale assessments.