It is also likely that the lower the resource costs of accountability are, the more likely it is to be done well at the program level. This is simply due to the limited resources literacy programs work with, and the need to stretch them a long way. Accountability data may not be the priority for programs. The lightest touch that can get the necessary data is most likely the best approach.
These comments are more concrete, and tend to be relevant to the development of a system.
Over the last 15 years the approach to measuring literacy originally developed for the International Adult Literacy Survey of the mid-1990s has grown into a system. It has been refined and developed in a responsible and interesting way by those involved in the various projects over the intervening years. However, IALS approaches literacy in a very specific way, and it is important to be aware of what this approach can be used for, and where it is more problematic.
In general terms, the understanding of literacy behind IALS(S) is designed to give a snapshot of a population’s literacy abilities at a specific point. It is not a way to talk about an individual’s literacy abilities, which would require far more extensive exploration than the IALS(S) permits. It is also not diagnostic, and cannot be used to identify what an individual needs to learn. This is because IALS(S) tests deliberately jumble up several different skills in each question.
The levels associated with IALS(S) also need to be approached with some care. The claim that the Level 2 to 3 transition is an important indicator of functional literacy is a statistically sound claim, but it does not indicate what will happen to an individual. Many other factors will the influence the outcomes of literacy learning.
A key principle of assessment is not holding people accountable for things they have no power to change. In the case of adult literacy programs, this could include employment of participants, or whether they continue their education. Things that programs can be responsible for include a welcoming and effective induction process, and demonstrated achievement of learning objectives. The first examples are program outcomes, the second group of examples are program outputs.