The New International Adult Literacy Survey (IALS):
Does it meet the Challenges of Validity to the Old IALS?

Thomas G. Sticht
International Consultant in Adult Education

Abstract

In the mid-1990s nations in the Organization for Economic Co-operation and Development [OECD] conducted the first International Adult Literacy Survey (IALS). Soon a new version of the IALS will soon be released. This paper reviews evidence that challenged the validity of the earlier IALS standardized performance scales including (1) the construct validity of the measurement scales, that is, the question of just what it is that the IALS scales measure, (2) the standards validity, that is, the question of “how good is good enough” to be considered competent at whatever the scales measure, and (3) the use validity , that is, the extent to which the findings are useful for various purposes and do not produce social harm. It will be important for those interested in the new IALS to see how it has met the challenges of validity raised here.

Introduction

Early in the 1990s, the Organization for Economic Co-operation and Development [OECD] noted that, in regard to concerns for economic competitiveness among member nations, "... one area that is receiving growing attention from educational policymakers and analysts in a number of OECD countries is the direct measurement of literacy levels in the labor force of industrialized countries" (Benton & Noyelle, 1992, p. 11). Just three years later, a series of reports was initiated presenting the results of the first International Adult Literacy Survey (IALS), which eventually involved 22 nations (OECD, 1995, 1997; Tuijnman, 2000).

One of the methodological innovations of the IALS was the adaptation of a measurement system developed in the United States for use in other nations (OECD, 1995, 1997). The measurement system consisted of three literacy scales: prose, document and quantitative literacy. The measurement range of scores for each scale was 0 to 500, though in practice scores fell primarily in the 180-390 range of the three scales. For each scale, five levels of literacy were defined, increasing from the lowest level, Level 1 (scores from 0 to 225), through Levels 2 (226-275) , 3 (276-325), 4 (326-375) and 5 (376-500), the highest level of literacy. The three scales and five levels were formed on the basis of door-to-door surveys in which, among other things, adults in the age range from 16 through 65 were asked to perform a number of “real world” tasks involving printed materials and oral instructions for each of the three scales.

In addition to the performance task scales used to assess literacy skills, the IALS also created another set of scales which asked adults to provide self-assessments of how well their reading, writing, and numeracy skills met the demands for such skills in their daily lives and at work. The measurement scale for each of these literacy and numeracy skills consisted of five categories: No Opinion, Poor, Moderate, Good, and Excellent.