Research Note #1 March 5, 1999

How Many Low Literate Adults Are There in Canada, the United States, and United Kingdom? Should the IALS Estimates be Revised?

Thomas G. Sticht
Applied Behavioral & Cognitive Sciences, Inc.

The free Adobe(R) Acrobat(R) Reader allows you to view, navigate, and print PDF files across all major computing platforms. To download Acrobat(R) Reader click on the button, and follow the instructions to install it.

In January of this year I prepared a working draft of a paper on assessing adult literacy for the Education Statistics Services Institute (ESSI) sponsored by the U. S. National Center for Education Statistics in Washington, DC. I spent the second week of February in London listening to British government officials and others describe the United Kingdom's National Literacy Strategy, which included an overview of new adult literacy initiatives in the United Kingdom. Then I spent the third week of February in Canada and discussed with government officials and others issues concerning adult literacy provision.

In this writing and travel I noted that in each of these three nations there has been the need to determine the "scale of need" for literacy provision for adults. To do this, the governments of these nations have drawn on the results of the first International Adult Literacy Survey (IALS) (see Sticht, 1998 for citations to IALS reports). That survey, completed in 1995, used essentially the same procedures for measuring and scaling adult literacy as had been used in the earlier National Adult Literacy Survey (NALS) of the United States. This included the use of three scales, Prose, Document and Quantitative literacy, with scores on each scale ranging, at least in principle, from 0 to 500.

In reporting scores, the IALS, like the earlier NALS, divided the range of scale scores into five literacy levels, numbered 1, 2, 3, 4, and 5. The lowest level of literacy, Level 1, is what has been used in these three nations to estimate the percentage of adults most in need of adult literacy provision. In Canada, this lead to the estimate that 16.6 percent of adults were of low literacy (using just the Prose scale for the present discussion, the other scales give similar results within one or two percentage points), in the United States the estimate of lowest literacy was 20.7 percent and in the United Kingdom, the estimate of those adults of least literacy was 21.8 percent.

Issues in Setting Standards For Adult Literacy. When the National Adult Literacy Survey (NALS) was released in 1993 I raised the question as to why it was that 66 to 75 percent of adults in Literacy Level 1, the lowest level, thought they could read and write "well" or "very well," figures that were similar to those found in the later IALS for the three nations under discussion. I noted that this might result from the fact that on the Prose scale of the NALS, people with a score of 200 were assigned to Level 1 because they had an 80 percent probability of being able to do the average task at level 1. However, these same people would be expected to be able to respond correctly to over 45 percent of the average tasks at Level 2, 25 percent of the average tasks at Level 3, and even 15 percent of the average tasks at Level 5, the highest level of literacy. Yet, because they were assigned to Level 1, all competence above that level was denied to them. Similar findings held for Document and Quantitative scales.

This earlier note raised concern with the problem of how to determine on a continuum a point or points that divide the continuum into various "amounts" of literacy. This is an issue that is being vigorously pursued in measurement and standards setting arenas today in the United States (Kolstad, et al., 1998). It is critical because it underlies the practice of determining how many adults are "at risk" for poorly developed literacy skills and might therefore benefit from some level of government support for adult education.

Figure 1 illustrates the different sorts of conclusions that one can reach depending upon how the standards for literacy are defined.

Both the International Adult Literacy Survey (IALS) and the National Adult Literacy Survey (NALS) set a criterion of having an 80 percent probability of getting the average item at a given Literacy Level correct to be assigned to that level of skill. This resulted in some 20.7 percent of adults in the United States being classified as in the lowest level of literacy.

However, Andrew Kolstad, the leader of the NALS project at the National Center for Education statistics has argued that the 80 percent probability level is arbitrary (Kolstad, 1996). He calculated the percentage of adults who would be in Level 1 if a criterion of 65 percent, which is used by the National Assessment for Educational Progress (NAEP) for the K-12 system, was used. In this case the percentage of adults scoring below Level 3 dropped to 32, a reduction of some 15 percent.

Kolstad also determined the consequences of using the 50 percent probability criterion used by the widely employed Comprehensive Adult Student Assessment System (CASAS) which is included in the federal government's dissemination network. In this case, only 9 percent were in Level 1, the lowest level of literacy.

Figure 1 shows the percentages of adults within Canada, the United States and United Kingdom that fall within the lowest level of literacy for the Prose scale using the original estimates from the IALS, and the estimates that result when Kolstad's findings using a 50 percent probability criterion are applied to the data. This results in each country in about a 50 percent reduction of adults considered most "at risk" for literacy using the 80 percent criterion of the IALS for assigning adults to the various levels of literacy.

For the three nations discussed here (and others that have used the IALS, too), it is important to realize that the decision to set the criterion standard of performance at the 80 percent probability level to categorize adults into one of the five levels of literacy on the IALS was an essentially arbitrary decision about what constitutes "mastery" of literacy at different levels. As noted above, it denies to adults the skills they have for actually performing tasks above the level to which they have been arbitrarily assigned. This may explain why in the minds of most of the adults in Literacy Level 1 they thought their literacy skills were just fine for meeting the requirements of everyday life. They could, in fact, perform quite a few (one in six) of the most difficult literacy tasks.

In more recent analyses, Kolstad (personal communication, January 26, 1999) has demonstrated that the use of the 50 percent probability criterion produces the least errors in predicting whether adults can or cannot perform literacy tasks across the full range of tasks included in the NALS. At the present time in the United States, stimulated by Kolstad’s work, there is considerable debate going on at the National Center for Education Statistics about just which standards should be used for all national assessments, those for school children and adults as well (Kolstad, et al., 1998).

This debate is likely to extend beyond the United States into Canada and the United Kingdom as these three nations, and others in the industrialized world, try to formulate national policies for adult literacy provision based upon methods for representing the "true" scale of need for adult literacy provision in the new millennium.

References

Kolstad, A. (1996, April) The Response Probability Convention Embedded in Reporting Prose Literacy Levels from the 1992 National Adult Literacy Survey. Washington, DC: National Center for Education Statistics.

Kolstad, A. et. al. (1998, May) The Response Probability Convention Used in Reporting Data From IRT Assessment Scales: Should NCES Adopt a Standard? Washington, DC: National Center for Education Statistics.

Sticht, T. (1998, September). Beyond 2000: Future Directions for Adult Education. Washington, DC: U. S. Office of Education, Division of Adult Education and Literacy.

| NALD Research Materials |