ICILS 2013 Results
Scale Creation
Prior to scaling, an extensive analysis of scaling properties was carried out that included reviews of missing values, test coverage, assessment of item fit, differential item functioning by gender, and cross-national measurement equivalence.
The ICILS test items were scaled using item response modeling with the (one-parameter) Rasch model.
- The CIL scale was derived from student responses to the 62 test questions and large tasks (which corresponded to a total of 81 score points).
- Most questions and tasks corresponded to a single item each; however, each ICILS large task was scored against a set of criteria (each criterion with its own unique set of scores) relating to the properties of the task. Each large-task assessment criterion was therefore also an item in ICILS.
Additionally, plausible values as ability estimates were generated with full conditioning in order to take all student-level and between-school differences into account.
The final reporting scale was set to a metric with a mean of 500 (the ICILS average score) and a standard deviation of 100 for the equally weighted national samples.
Four proficiency levels were established, thereby providing test item locations on the CIL achievement scale and allowing a description of these levels complete with example test items.
List of Achievement Scales
The computer and information literacy scale
Scale Creation
Two general types of indices could be distinguished, both of which derived from the ICILS questionnaires:
Simple indices
- They were constructed through arithmetical transformation or simple recoding.
- For example: ratios between ICT and students or an index of immigration background based on information about the country of birth of students and their parents.
Scale indices
- They were derived from the scaling of items, a process typically achieved by using item response modeling of dichotomous or Likert-type items.
- Item response modeling (applying the Rasch partial credit model) provided an adequate tool for deriving 10 international student questionnaire scales, nine teacher questionnaire scales, and seven school questionnaire scales.
- A composite index reflecting socioeconomic background was derived using principal component analysis of three home background indicators, namely, parental occupation, parental education, and home literacy resources.
- Generally, the scales used in ICILS had sound psychometric properties, such as high reliability.
- Confirmatory factor analyses showed satisfactory model fit for the measurement models underpinning the scaling of the questionnaire data.
Only scale indices are reported below.
List of Background Scales
Student questionnaire
Students’ use of ICT applications
- Students’ use of specific ICT applications
- Students’ use of ICT for social communication
- Students’ use of ICT for exchanging information
- Students’ use of ICT for recreation
Students’ school-related ICT use
- Students’ use of ICT for (school-related) study purposes
- Students’ use of ICT during lessons at school
- Students’ reports on learning ICT tasks at school
Students’ ICT self-efficacy, interest, and enjoyment
- Students’ confidence (ICT self-efficacy) in solving basic computer-related tasks
- Students’ confidence (ICT self-efficacy) in solving advanced computer-related tasks
- Students’ interest and enjoyment in using computers and computing
Teacher questionnaire
Teachers’ confidence in computer tasks (self-efficacy)
Teachers’ use of ICT applications for teaching
Teachers’ use of ICT for activities and practices in class
- Teachers’ use of ICT for learning at school
- Teachers’ use of ICT for teaching at school
Teachers’ emphasis on ICT in teaching
Teachers’ views on using ICT for teaching and learning
- Positive views on using ICT in teaching and learning
- Negative views on using ICT in teaching and learning
Teachers’ views on the context for ICT use at their school
- Teachers’ perspectives on the lack of computer resources at school
- Teachers’ perspectives on collaboration between teachers in using ICT
School questionnaires
ICT coordinators’ reports on ICT resources at school
ICT coordinators’ perceptions of hindrances to ICT use at school
- ICT use hindered in teaching and learning: lack of hardware
- ICT use hindered in teaching and learning: other obstacles
School principals’ perceptions of the importance of ICT at school
- Principals’ perceptions of using ICT for educational outcomes
- Principals’ perceptions of the ICT use expected of teachers: learning
School principals’ views of ICT priorities at school
- Principals’ views of priorities for facilitating use of ICT: hardware
- Principals’ views of priorities for facilitating use of ICT: support
Variations in student achievement on the CIL scale
- Variations across countries
- Student CIL varied considerably across ICILS countries.
- Factors associated with variations in CIL
- Higher socioeconomic status was associated with higher CIL proficiency both within and across countries.
- Female students had higher CIL scale scores in all but two countries.
- Students who spoke the language of the CIL assessment (which was also the language of instruction) also performed better on it.
- Multiple regression techniques showed that the following variables had statistically significant positive associations with CIL in most countries: students’ gender (female compared to male), students’ expected educational attainment, parental educational attainment, parental occupational status, number of books in the home, and ICT home resources.
- Student experience of computer use and their frequency of computer use at home were positively associated with CIL scores in most countries.
- CIL achievement was also positively associated with basic ICT self-efficacy but not with advanced ICT self-efficacy.
- Student use of ICT
- Almost all ICILS students reported that they were experienced users of computers and had access to them at home and at school. On average across the ICILS countries, more than one third of the Grade 8 students said they had been using computers for 7 or more years, with a further 29% reporting that they had been using computers for between 5 and 7 years.
- Students across the ICILS countries reported using computers more frequently at home than elsewhere.
- Computer use outside school
- ICILS indicated that students were making widespread and frequent use of digital technologies when outside school.
- Students tended to use the Internet for social communication and exchanging information, computers for recreation, and computer utilities for school work and other purposes.
- Students also reported using computer utilities (applications) outside school. Generally across the ICILS countries, the most extensive weekly use of computer utilities involved “creating or editing documents” (28% of students).
- Use of ICT for school work
- Cross-nationally, just under half (45%) of the ICILS students, on average, were using computers to “prepare reports or essays” at least once a week.
- A similar extent of use for “preparing presentations” (44%) was recorded.
- The subject area in which computers were most frequently being used was information technology or computer studies (56%).
Teacher and school use of ICT
- Teacher use of ICT
- Teachers were making extensive use of ICT in their schools (three out of every five teachers said they used computers at least once a week when teaching).
- In general, teachers were confident about their ability to use a variety of computer applications; two thirds of them expressed confidence in their ability to use these for assessing and monitoring student progress.
- Teachers recognized the positive aspects of using ICT in teaching and learning at school, especially with respect to accessing and managing information; a substantial majority of the ICILS teachers were using ICT in their teaching.
- According to the ICILS teachers, the utilities most frequently used in their respective reference classes were those concerned with word processing, presentations, and computer-based information resources, such as websites, wikis, and encyclopedias.
- Teachers’ emphasis on developing CIL in students was positively associated with teachers’ ICT-self efficacy, their positive views of ICT, and the amount of collaboration regarding ICT use teachers reported in their schools. The combination of these three factors together accounted for 23% of the variance among countries that met the ICILS teacher sampling requirements.
- School-based ICT provision and use
- There were substantial differences across countries in the number of students per available computer in a school.
- Computers in schools were most often located in computer laboratories and libraries; however, there were differences among countries as to whether schools had portable class-sets of computers on offer or whether students brought their own computers to class.
- ICT coordinators reported a range of impediments to teaching and learning ICT. In general, the coordinators rated personnel and teaching support issues as more problematic than resource issues.