PIAAC Cycle 1 Design
- Overall approach to data collection: computer-based or paper-based assessment and personal interview survey
- Specification: trend study
- Intended: non-institutionalized individuals ages 16 to 65 who are residents in the participating country/economy
- Countries/economies were permitted to extend the target population.
- The target population covered all residents regardless of citizenship, nationality, and language.
- Adults in institutional collective dwelling units, such as prisons, hospitals, nursing homes, military barracks, and military bases, were not included.
- A probability-based sample, which should be representative of the target population of the country/economy
- The following PIAAC standards were communicated to participating countries/economies. A typical sample design consists of:
- A stratified multistage clustered area sample for geographically large countries/economies
- Less clustering and fewer stages of sampling for geographically small countries/economies
- Each country/economy was permitted to select a sample design in line with the PIAAC standards.
- National sample designs as decided by countries/economies
One-stage sample designs
- With or without explicit stratification
Two-stage stratified probability-proportionate-to-size (PPS) designs
- First stage: primary sampling units (PSUs)
- Second stage: persons to be sampled in each unit
Three-stage stratified PPS designs
- First stage: PSUs
- Second stage: dwelling units (DUs)
- Third stage: persons from the housing unit
Four-stage stratified PPS designs
- First stage: PSUs
- Second stage: secondary selection units (SSUs)
- Third stage: DUs
- Fourth stage: persons from the housing unit
- Round 1 (determined by the optional components of the psychometric assessments)
- Min. of 5,000 individuals for the component problem-solving (only); for the main language of the country
- Otherwise 4,500; for the main language of the country
- Round 2 (standard minimum sample size)
- Min 5,000 individuals in each participating country/economy; for the main language
Approximately 250,000 adults in 33 countries/economies
Data collection techniques and instruments
Technique 1: Individual interview (BQ)
- Administered in a computer-assisted personal interview (CAPI) format by the interviewer
- 258 variables
Technique 2: Core tests (ICT core and domain-based core part)
- Computer self-assessment under supervision of the interviewer
- Sets of core items used in selecting the paper or computer path for the respondents
Technique 3: Literacy, numeracy, and problem-solving achievement tests
- CBA (computer-based assessment) format under supervision of the interviewer; multistage adaptive design
- Respondents with no (or extremely limited) experience with the use of computers were given a pencil-and-paper version of the literacy and numeracy components.
- 341 booklets: 4 paper-based booklets and 337 computer-based booklets/paths
- 204 items: literacy (95 items), numeracy (95items), problem solving (14 items)
- Short, open-ended response formats on paper or computer-based open-response formats (e.g., highlighting the correct phrase or word)
Technique 4: Reading components achievement test
- Respondents with poor literacy and numeracy skills were directed to the reading components test, which was taken in pencil-and-paper format only.
- Interviewers timed the completion of the reading components tasks using the above computer application.
- 100 items
- achievement test
- individual interview
- Administration of the background questionnaire in 34 languages
- Administration of assessment instruments in 31 languages
- Development of an international version of all assessment instruments in English
- Translation into applicable languages of instruction by participating entities (national centers); guidance and assistance provided to countries throughout process by PIAAC Consortium, by developing and conducting linguistic quality assurance (LQA) and linguistic quality control (LQC) processes.
- Translation verification by linguistic and assessment experts in order to ensure equivalence to the international version
Quality control of operations
Measures during data collection
- General note: Participants were responsible for data collection within their own territories.
- Provision of a set of technical standards and guidelines (TSG) for survey/operations/data collection.
- Provision of software tools for supporting activities
- Quality control monitoring with continuous support to countries in accordance with the TSG (standardized forms and reports to be completed at each step of the data collection, regular communication, etc.)
- Training program produced by the Consortium (data management training, etc.)
Measures during data processing and cleaning, using the Statistical Analysis Software (SAS)
- Standardized cleaning process at the national level (using IEA DME software)
- Recoding, cleaning, and editing at the national level as discussed with and approved by the Consortium
- Data processing at the international level by the Consortium
- Repetition of data cleaning and comparison of new data sets with preceding version
- Iterative process of data review and correction within the Consortium, the participating countries, and the OECD Secretariat.
Sources - Technical documentation