Accountability Perspectives

by Katrina Grieve

Over the last several months I have been following an online discussion in the United States regarding literacy policy. I have been struck by the similarities in discussions about accountability, and the pressures that result when government funders attempt to measure the impact of the money they spend in adult literacy programs. In the US, things are taken a step further in that there has been increasing pressure to use standardized tests to measure literacy gains. What is obvious from these discussions is that there is a huge gap in how learners, practitioners, administrators, and government decisionmakers think about these issues and in the language they use.

While many practitioners are reluctant to use standardized systems of levels to report "progress", there is a wide range of perspectives about the need to use such systems to show accountability. I am going to quote from some of the debate from the AAACE-NLA discussion to give you a taste of these different perspectives.

— "Assessment systems have greatly improved, with more consistent and widespread use of standardized, psychometrically sound assessments and abandonment of subjective assessment or teacher judgments that do not accurately measure student learning… Programs have replaced measures such as self-esteem and student appreciation of the classes with objective measures of student literacy gains." (Department of Education: annual report on adult education to Congress). ie – standardized tests are objective while learners' and teachers' judgments of progress are not valid.

— "The data collected through the National Reporting System (NRS) is almost completely useless. It is based on totally arbitrary ‘benchmarks' that have been set at different levels, using different methods, with different indicators at the state and federal levels." (Thomas Sticht)

— "I buy the taxpayer argument. I am a taxpayer too, and I want to know that my money is actually helping students make adult literacy gains. The crux of the problem is how literacy gains are to be measured. Do higher levels translate into jobs? More income? Better housing? Health care? ... As a taxpayer I want to know that my money makes a measurable difference in increasing adult literacy."

— "We need to stop looking at the people with literacy problems as facts and figures and look at them for what they are—real people." (Archie Willard, learner)

— "I feel that ‘hard scientific evidence' is really an illusion. Adult literacy needs discovery which may come at different speeds to most people, but it does happen. Can this be tested? I don't know. Discovery is the fun and amazement of learning. Hard science, illusive." (practitioner)

— "Programs like ours had their voices and their learners' voices stripped from them because they would not agree to the NRS, nor timed testing of adults with low reading skills. We became non-members of the Adult Education & Literacy System (losing funding) and are far better for it." (practitioner)

— "Even when there is some direct measurable outcome, it is often problematic to attribute it to the literacy factor. More typically, it is one of a variety of variables interacting as adults engage literacy programs as part of a developmental process of change wherein socio-emotional factors may well be as important as progress gained in reading...When the concept of accountability (itself a metaphor) gets linked to another metaphor (return on investment), then we have a very restricted environment that allows for the allocation of funds and a very restricted construction of reality." (George Demetrion)

— So how do we measure success in learning? "One way of doing it is to expose people to different kinds of environments and try to document the extent to which they can participate in those contexts. Another way is by collecting anecdotal evidence. I as a taxpayer would be happier if a teacher reported that she believes her students can better use the health care system, than if she reports on her students' increase in level on a particular assessment."

Similar discussions on accountability can be heard across Ontario programs. In the US, many small community-based programs have dropped out of the official Adult Education System because they were unable or unwilling to meet the accountability expectations of government funders, which placed a huge burden on their programs without accurately reflecting the kind of learning that was taking place. And so I pose the question: "Could the same thing happen here?" 

You can find the discussion at: http://lists.literacytent.org/mailman/listinfo/aaace-nla

Reflections of a practitioner

by Susan Lefebvre

I have just finished reading an article titled Education for the Soul by Jack Miller (1996), an educator at OISE. He claims that the twentieth century has not been good for the soul and that a mechanized approach to living has contributed to the loss of soul. The article resonated with me, as he described a society that values a "continuous improvement, performance-based" approach to life and an educational system that is concerned, it seems, with only "efficiency and effectiveness" and outcomes. LBS reform policy is laden with language that reflects disrespect for the human spirit/soul. Is trying to survive in a system where my success (not to mention learners' success) is measured against welldefined rigid indicators partly responsible for my struggle to characterize spirit?

Robert Sardello (1992) contends that education:
… has become an institution whose purpose in the modern world is not to make culture, not to serve the living cosmos, but to harness humankind to the dead forces of materialism. Education as we know it, from preschool through graduate school, damages the soul. (p. 50)

My musings may give the impression that I am pessimistic, but for the most part when I focus on the learners and not on policy demands, I am most hopeful. I see many moments of spontaneous insights and unexpected learnings that are more valuable than any Level indicators. I witness a vitality and excitement in learners which inspires, motivates and feeds their souls.

I like what Miller had to say about soul: As a source of energy we can sometimes feel the soul expand. A beautiful piece of music can make our souls feel expansive; likewise, in a threatening or fearful situation, we can feel our souls contract or shrink. A soulful curriculum would provide a nourishing environment for the soul's expansion and animation.

Miller also suggests we, as teachers, should bring our souls to the classroom. Two qualities that the soulful teacher can bring to the classroom are presence and caring. Presence arises from mindfulness where the teacher is capable of listening deeply. Caring can encourage the development of community in the classroom. I have observed and been part of this sense of community in groups at Literacy for East Toronto. I believe this community helps to keep learners "present" and to make the all important connections and meanings necessary to the belief in the possibility of goals.  

Miller, J.P. Education for the Soul Ontario Institute for Studies in Education/University of Toronto Paper presented at AME, November 15, 1996