How representative are student convenience samples?

Periodical
PLOS ONE
Volume
17
Year
2022
Issue number
7
Access date
May 31, 2023
Relates to study/studies
PIAAC Cycle 1

How representative are student convenience samples?

A study of literacy and numeracy skills in 32 countries

Abstract

Psychological research, including research into adult reading, isfrequently based on convenience samples ofundergraduat estudents. This practice raises concerns about the external validity ofmany accepted findings. The present study seeks todetermine how strong this student sampling bias isinliteracy and numeracy research. We use the nationally representative cross-national data from the Programme for the International Assessment ofAdult Competencies toquantify skill differences between (i)students and the general population aged 16–65, and (ii) students and age-matched non-students aged 16–25. The median effect size for the comparison (i)ofliteracy scores across 32 countries was d=.56, and for comparison (ii) d=.55, which exceeds the average effect size inpsychological experiments (d=.40). Numeracy comparisons (i)and (ii) showed similarly strong differences. The observed differences indicate that undergraduate students are not representative ofthe general population nor age-matched non-students.