Section One

Sharing What We Do

All too often evaluation consists of a kit or set of guidelines handed out to practitioners, for their consideration in individual practice. When program staff meet, evaluation issues are frequently at the heart of many concerns about program. Yet it is probably the case that practitioner questions about outreach, program participation, or staff training are not understood as evaluation questions. As well, little time is set aside for staff to bring questions, ideas, and concerns about how to evaluate, why evaluate this way or that, or to review the different ways other colleagues are using assessment.

There is a tendency to see evaluation as an external process which is done to or for a program in order to provide a picture of its effectiveness. This perspective is akin to seeing evaluation as research which uses precise measuring instruments developed by experts in order to provide longitudinal information about changes in literacy. Seen from this big picture view of evaluation, accomplished with sophisticated techniques, day-to-day observations about learners and programs may not qualify as real evaluation.

This notion of evaluation seems to contribute to a devaluing of practitioner/learner judgment and observation, making these judgments appear as an unreliable basis for assessing family literacy. Indeed, many practitioners are reluctant to offer their observations as valid information about family literacy development. While some practitioner hesitation is related to the desire to be objective in making judgments based on learner behaviour rather than informal opinion, some practitioner reluctance is also tied to a sense of inadequacy about evaluation expertise.

Nevertheless, because effective program evaluation must rely on practitioner/learner observation of literacy development, practitioners must develop confidence in their observations, along with willingness to construct ways to record such observations in a systematic fashion. It is by building on routine observations that a picture of program effectiveness emerges. Over time, an accumulation of observations adds up to provide answers to more complex questions about program effects on family literacy use, school achievement of children, vocational and occupational development.