The Education & Skills Online cognitive assessment includes background questions and modules measuring core cognitive skills (literacy and numeracy) as well as optional modules to assess reading components and problem solving in technology-rich environments.
Education & Skills Online test items
The cognitive test items in Education & Skills Online include existing items from the OECD Survey of Adult Skills (PIAAC) as well as newly developed items for the domains of literacy and numeracy. The existing PIAAC items are included to ensure that Education & Skills Online and PIAAC results could be linked and to establish comparable scales across the two measures. The newly developed items were field tested to determine their reliability for the Education & Skills Online language versions.
Literacy
The Education & Skills Online literacy items were developed and selected to represent three major aspects of processing continuous and non-continuous texts and documents: accessing and identifying, integrating and interpreting, and reflecting on and evaluating information.
The Literacy questions measure how well a person understands and uses information found in materials such as newspapers, brochures, manuals or websites. They also measure how well one finds and uses information in forms, schedules, charts or tables of information.
Reading components are the skills that work together to help individuals understand what they read. There are many different reading components. In this test, there are three sections: vocabulary, sentence comprehension, and passage comprehension.
Numeracy
The Numeracy questions measure how well a person is able to interpret, communicate, or use mathematical information to solve a problem or understand a situation. One may find mathematical information in materials such as tables, graphs, maps, product labels or advertisements.
Numeracy tasks were developed to cover a range of difficulty as a result of combining variables that include:
- the kind and degree of interpretation and reflection required by the problem
- representation skills
- mathematical skills
- mathematical argumentation
- degree of familiarity with the context
- extent to which tasks require test takers to perform known procedures and steps or present novel situations requiring more creative responses
Problem Solving in Technology-Rich Environments
The Problem Solving in Technology-Rich Environments assessment is organised around three core dimensions: the cognitive strategies and processes a person uses to solve a problem, the tasks or problem statements that trigger and condition problem solving, and the technologies through which the problem solving is conducted. The three dimensions are described as follows:
- technology features – types of applications, amount of navigation required, use of tools required
- task features – number of steps involved, number of individuals dealt with in the task
- cognitive processes – whether a goal is defined, use of criteria, demands for monitoring progress, evaluation of relevance, level of reasoning
Variations within and across all of the dimensions contribute to the overall difficulty of the problems presented in the assessment. For example, a problem is likely to be more complex if it is ill-defined as opposed to explicitly stated, if it requires complex problem solving strategies such as defining goals and resolving impasses, and/or if it requires the use of multiple technology environments (e.g. respondents must utilise both emails and spreadsheets).
The problem solving in technology-rich environments questions measure how well a person uses different types of technology to solve everyday problems and complete tasks to successfully meet goals. They also measure how well one understands and uses information in different environments, such as email, web pages, or spreadsheets. In this test, a problem is any situation where one doesn’t already have a good idea about how to achieve a goal. This may be because the strategy to use does not appear obvious or because one has never tried such a task in the past.