The disadvantages of using only standardized tests in workplace literacy programs were introduced in Chapter 1. These tests measure general reading abilities and not the special sorts of literacy skills required in the workplace. A learner in a general basic skills class may improve in general reading abilities. For example, a learner could move from a fifth grade level (i.e., understanding the comics and very simple stories) to an eighth grade level (i.e., understanding the sports page and USA Today news stories). Though the improved reading ability may be of some use, the learner is not likely to be able to transfer those skills easily to reading an SPC chart, a technical manual, specialized work-orders, and industry-specific textbooks. The most efficient way to ensure improvement in these areas is to teach using these materials. Unfortunately, gains made in reading job-related materials may be only partially reflected in a standardized test which evaluates general reading skills.

More subtle criticisms have been leveled against the use of standardized test to evaluate workplace literacy programs. First, the information revealed by these tests presents an incomplete picture of adult learning (as described above, an adult may read familiar materials somewhat better than general test scores indicate). Second, the effects of such test scores on instruction are considered to be adverse by some (i.e., when teachers teach to the test and ignore materials that learners need for the job). Third, the way in which standardized test scores are reported can be humiliating to adults and counterproductive to learning. Some educators argue that when adults are informed that their performance is equivalent to a low grade level (i.e., sixth grade or lower), it becomes a reminder of their failure rather than an objective description of current abilities.

Some workplace literacy programs find that the disadvantages of standardized tests outweigh the advantages and rely instead on interviews, questionnaires, and other indicators to assess program effectiveness. Such assessments are custom-designed for the program being evaluated. Other programs use standardized tests as part of a mix of assessments. If standardized tests are used, they should never be the sole measure of learner gain in a workplace literacy program.

Custom-designed instruments which are based on workplace materials and activities can supplement or provide an alternative to standardized tests. To design such instruments and, indeed, to custom-design training programs, one first needs to determine how workers use literacy in a particular workplace. The first step is to perform a literacy task analysis.

Literacy task analysis is a way of identifying those aspects of job tasks which require reading and problem solving. It is done through a combination of observing workers, interviewing top performers, and gathering samples of printed materials used in the workplace and training classes. The goal is to determine the mental processes used by top performers as they solve problems and complete tasks which involve literacy. This information can be used to construct both test scenarios and instructional materials. It is important that these two be developed together so that tests and instructional materials can be linked directly to the workplace, and tests can assess what learners are really taught.

Observations of and interviews with supervisors and workers are used to identify the areas in which performance needs to be improved. Prime targets for analysis are tasks where basic skills deficiencies cost money or threaten health and safety. Other tasks can be identified by noting changes in the workplace (e.g., new technology, changed jobs, or promotions) which confront some workers with new and sometimes troublesome literacy tasks.

A good deal has been written about the techniques of literacy task analyses (see Mikulecky, 1985; U.S. Departments of Education and Labor, 1988; Drew & Mikulecky, 1988; Philippi, 1988, 1991). Most involve determining the elements of a task and the strategies (both visible and mental) employed to accomplish the task. For example, filling in forms in some quality assurance procedures involves elements such as reading two-column charts, computing using decimals, knowing special vocabulary and abbreviations, and summarizing sequences of events. Within each of these elements, top performers employ a variety of strategies (e.g., skimming, estimating, interpolating, etc.)

BACK / NEXT PAGE