Ruth Sutton on Rethinking Accountability

This was written by Ruth Sutton as the Forward for the Alberta Assessment Consortium's research update A New Look at Public Assurance: Imagining the Possibilities for Alberta Students. I had the pleasure of speaking with Ruth Sutton years ago when I was initially going through my own personal teaching formation. She won't likely remember me, but my short time with her left a lasting impression on me.

by Ruth Sutton

In 1988, a quarter of a century ago, the first meta-analysis of global research on educational assessment was published. 

This study of the links between assessment, learning and achievement revealed for the first time what many people interested in this field – myself included – had always suspected. To use an agricultural analogy, ‘Weighing the pig doesn’t make it grow.’ 

Since then the evidence about what kinds of educational assessment have the most positive impact on educational outcomes has kept coming, inexorably. Whole cohort testing, once the norm in many jurisdictions, has been questioned, adapted, reviewed, and questioned again. The issue is not whether we need information about the learning and achievement of our children and young people, but what kind of information we need, and how best to gather it. The first requirement of world-class public assurance and accountability is world-class information about student achievement.

On the large scale of a province, or the small scale of a school, we must pursue valid information about student learning. How accurate is the information relative to the kind of learning we aspire to achieve? Secondly, and most importantly when assessment information may be used to compare outcomes from one place to another, one group to another, or a cohort of students compared with the same cohort previously, we need to reduce the variables that infuse educational assessment. Thirdly, we must pay attention to the cost and logistical implications of information gathering. Is the benefit to be derived from the assessment process worth the cost, in terms of both dollars and time? To use another North American phrase, ‘How do we get more bang for our buck’?

This very timely paper reviews the decisions facing Alberta, and how other states and jurisdictions are currently finding their way through the maze. It identifies the skills and capacities our students need to be successful learners and earners in the twenty-first century, different in many respects from the skills of previous decades. Many of these skills do not lend themselves to ‘data capture’ through traditional forms of testing: they are manifest not in answers to a pencil and paper test but in student responses to on-going classroom challenges, observed by their teachers. Time and money spent on using inappropriate methods of data capture could be time and money wasted.

To satisfy public assurance without relying on outdated testing procedures, we will need to look carefully at the variables involved and find new ways of reducing their impact, using proven strategies from elsewhere around the world. The two greatest variables are the design of the assessment task itself, and the interpretation of assessment outcomes. The design of tasks can be supported by central professional expertise, and through specific training for teachers in this aspect of their work. The problem of variable interpretation of results is tackled by requiring teachers to share their expectations and evidence of learning, and then to ‘moderate’ their judgements, reaching a collective rather than an individual decision.

Information about student achievement provides the building blocks for the structure of public assurance. If the information is of poor quality, the structure will always be weak. Once the quality of information is assured, then desirable and effective structures can be built. 

These aspirations are worthy in themselves, but the proposals in this paper go further. Learning from others’ experience, Alberta can develop a system of public assurance that not only measures the impact of schooling but actually strengthens both classroom learning and the schools and local organizations that provide it. 

Involving teachers collectively in assessment and evaluation provides highly effective professional development, as teachers learn from each other and share successful practice. Involving schools and their communities in the process of school and district review and evaluation serves the same purpose, sharing the most successful and effective strategies for improving educational outcomes for all our students.

Research and experience over the last twenty-five years has shown the way forward. Now we need to help our communities and our tax-payers understand these irresistible messages. Involving teachers, parents, students and the community in the process of assessment and evaluation is the key to a form of public assurance that serves to improve and develop our students and our schools, not just to measure them.