The literature also suggests that evaluations to date have not allowed enough time for gains and
outcomes to become fully apparent (Gray 2006: 76-7). There is therefore a call for “longer
timeframes to allow for more robust data collection”
(Benseman and Sutton 2007: 9-10). In this
connection, a 2002 study on training and ROI in the UK (Keep, Mayhew and Corney) reviewed
by Gray pointed out that, “Lasting gains are those most to be desired, but are the most
problematic to evaluate because the modern political process is extremely impatient. It wants
long-term results, but it wants them to be demonstrated fast”
(Gray 2006: 62).
The literature on workplace literacy and essential skills training often notes a shortage of
information on evaluation. For years, the extent to which programs incorporate evaluation and
the methods they use have been very difficult to determine with any precision, and this remains
a problem. Within studies on training evaluation broadly speaking, very few deal specifically
with literacy and essential skills education among workers. A 2008 report published as part of a
larger (European) project to develop a workplace LLN evaluation toolkit, found, after a
comprehensive review of international sources, that there is “little literature available
specifically on . . . evaluation of training interventions on basic skills, literacy or numeracy in
the workplace”. The report cautioned that, given how “sparse and diverse”
the literature, it
would be difficult to “draw generalised conclusions on the most appropriate techniques”
(Pye
and Hattam, 2008:10, 38).
French-language sources are also lacking. A recently published review of French-language literature on evaluation of basic skills training in the workplace found not a single source dealing specifically with the subject. The authors were able to identify a body of literature on evaluation in the literacy field generally, but these were “rare” and not directly concerned with workplace interventions. The authors concluded that relevant literature on the evaluation of workplace literacy and essential skills training is “practically non-existent” (Daniau and Bélanger, 2008: 5, 14-15).
Recent literature discussing evaluation of workplace literacy and essential skills training raises
the important issue of barriers to evaluation. On a general level, literacy and essential skills
training “is too easily assumed to produce positive results in and of itself”
(Bélanger and
Robitaille 2008: 64). But, there are other reasons as well. According to a 2008 report, American
employers who offer such training have tended to neglect evaluation of their programs because
monitoring and reporting “do not always suit business interests or needs . . .”
Also, some
employers are uncomfortable publicizing the basic skills problems of their employees (Rosen
2008: 12). In other cases there is a lack of interest or initiative (Bélanger and Robitaille 2008:
65). For instance, a 2007 review of New Zealand’s national Workplace Literacy Fund (WLF)
that supports workplace programs noted not only an “absence of standardised reporting of
learning outcomes… and a limited number of final reports”
, but also the fact that most managers
seemed to be “quite disinterested”
in the final reports (Gray and Sutton, 2007: 52).
Return to note 1 This section follows closely the discussion that appears in “Evaluation in Workplace Literacy/Essential Skills Programs: Public Sector - A Review of the Literature.” Unpublished Literature Review prepared by The Centre for Literacy of Quebec for the Canadian Association of Municipal Administrators (CAMA) and the Canadian Union of Public Employees (CUPE) [cited with permission].