Chapter 2

Q & A on the Evaluation of Workplace Literacy Programs

This chapter is based on a paper prepared for the National Workplace Literacy Program (NWLP) of the U. S. Department of Education (Sticht, 1991), which has since been replaced by the Workforce Investment Act, Title 2: Adult Literacy and Family Literacy Act of 1998, and the Work in America Institute's Job-Linked Literacy Network (Sticht, 1992). The purpose of the latter paper was to stimulate discussion of issues related to the evaluation of workplace (job-linked) literacy programs. In preparing the paper, a Question & Answer (Q & A) format was followed and responses were prepared to four questions asked by the Work in America Institute. Those same questions are presented here in bold type, followed by their responses, which have been enlarged to include part of the NWLP paper, too. No claim is made that responses are complete and nor that they fully explicate all the nuances and chains of thought that the various questions stimulate. And they certainly do not form the last word on what could be said about these complex issues.

By what criteria should a company judge the value of its program?

Clearly, a company should use criteria for evaluating its program that reflect the goals of the program. That is, the company needs to know whether the program is accomplishing the goals that it has set for the program. This, then, produces the problem of how to establish goals for the program. If one knows what one wants to achieve with the program then criteria that reflect those achievements can be developed.

For example, if one goal of a workplace literacy program is to improve people's ability to read job-related materials, then it is a reasonable question to ask, "Can people who have completed the workplace reading program read their job-related materials better after completing the program than they could before they took the program?"

In turn, this raises the important questions of how many program participants should improve their abilities to read job-related materials to what extent? If 100 employees take a job-related reading course that teaches 100 applications of reading as used on the job, should all who take the course master all job-related reading tasks? And should they all be expected to do this within the same period of time? For instance, in the period of one 45 hour program.

A Case Study With Evaluation Data. In one hospital-based workplace literacy program (Nurss, 1990), pre-and post-program tests were constructed based on information in the employee handbook and job memos that were applicable to all departments. The reading tests were 20 cloze tests (cloze tests are constructed by deleting every fifth word in a passage. The examinee then guesses what the missing word is. This type of test is highly correlated with other types of reading tests).