While the formative evaluation provides early information about the effectiveness of program operation, the summative evaluation provides information about whether the program achieved its goals.

Evidence of Goal Attainment. Well-evaluated workplace literacy programs gather baseline data before instruction begins. Typically data is gathered on the reading abilities, practices, and beliefs of learners. In addition, pre-program data is gathered on worker productivity or any other goal espoused by the program. Data-gathering is accomplished using formal tests, informally constructed tests related to workplace expectations, questionnaires, and interviews with learners and sometimes supervisors. In addition, company records on productivity, safety, attendance, and enrollment in subsequent classes can expand the evidence available for assessing program impact.

Such company information establishes a base for later comparisons to end-of-program performance. At the end of the program, all learners are once again assessed using the same instruments. In some cases, it is possible to compare the performances of learners in a workplace literacy program to those of a control group of comparable employees who haven't yet been able to receive workplace literacy training. To do this, the control group takes pre and post assessments which parallel those taken by the instructional group.

Program goals determine the types of information gathered to assess program impact. For example, if the program is to improve the ability of learners to perform more effectively in quality assurance groups, evidence needs to be gathered on such performance before and after training. If training is supposed to have a positive impact on learner reading habits at home and at work, these, too, need to be assessed before and after the program.

Chapters 3, 4 and 5 will provide samples of methods and instruments for assessing the impact of workplace literacy programs on learner literacy abilities, practices, plans, and beliefs. In addition, methods for assessing the impact of workplace literacy programs upon productivity and upon the families of learners will be discussed and sample measures will be provided.

Only a few workplace literacy programs have been evaluated well, even though millions of dollars have been invested in their development and operation. To evaluate workplace literacy programs effectively it is desirable to perform both formative and summative evaluations. Formative evaluation takes place during beginning and middle stages of program operation and is designed to identify problem areas which can be addressed and modified while change is still possible and productive. The process usually involves the use of interviews, document analysis, and observations. Summative evaluation of workplace literacy programs usually takes place at the end of program operation and is designed to assess how well the program has succeeded. It requires gathering pre- and post-program data and then analyzing that data. This implies using and developing measures directly related to program goals. Typical goals for workplace literacy programs include improved learner literacy abilities, improved literacy practices at work and elsewhere, changed learner beliefs about literacy, self, and education, and improved learner productivity on the job.

BACK / NEXT PAGE