Test Score Ambiguity

The problem with using test scores to tell us something about a student, teacher, school or community is that there are far too many variables.

Here's a question from the book The Myths of Standardized Tests:

If 85 percent or more the students in your child's classroom or school meet or exceed the proficiency standards, that means: 

a) your child has an exemplary teacher.
b) your school has an exemplary principal.
c) both a and b.
d) your school community is wealthier than average.
e) all, any combination, or none of the above.

While it may be true that inside the classroom the quality of the teacher has the greatest influence on student learning, the rest of the world outside of the classroom is much larger -- which is why testing experts like Harvard's Daniel Koretz warn:

A great many things other than the quality of schools influence educational achievement, and the impact of these noneducational factors can be huge... 

People routinely misinterpret differences in test scores, commonly attributing more to quality of education than they ought... 

Trends in scores over time, whether down or up, are often influenced by social factors and, in the case of seeming improvements, by inappropriate teaching to the test. Not all low scoring schools offer as weak an educational program as their scores might suggest. By the same token, if your neighborhood schools have high scores, that may mean less about the quality of their programs than you'd like. 

 The point to be taken here is that when we ask tests to be a window into the quality and quantity of student learning AND an educator's teaching, we are asking test scores to do something they can never do.

Speaking (and owning) your mind

Have you ever noticed how often people feel the need to explain that the views they express are their own and not necessarily a reflection of their employer?

I've seen some people change jobs and then announce to the world that they can now speak their mind. Have you noticed that it's at this point that the things they say are sometimes for the first time worth listening to?

If you can't speak your mind because of your employer, are you sure your mind is still yours?

I've seen it all

If you catch yourself saying "I've seen it all", you are either lying or wilfully blind -- either way, you are a part of the problem.

You haven't seen it all.

No one has.

If a 6 year old said this, an adult might chastise them for being an ignorant know-it-all, and yet that same adult might say it and consider themselves experienced.

Is there a case for summative assessment?

This was written by David Didau who is a Head of English at Priory Community School. Davidblogs here and tweets here.

by David Didau

I’ve written a lot on the importance of formative assessment recently and feel pretty cear in my own mind of its efficacy. In contrast I see summative assessment as existing only as an external measure of success or failure. I know it exists and I know it’s fairly important to my students’ life chances. It’s also one of the primary means by which my professional practice is judged, so I’d better take notice of it. This seems like a necessary evil, not something to be celebrated.

Cristina Milos and Jennifer Borgioli have challenged this view and asserted that actually, summative assessment has its place in the classroom and fulfils a useful purpose. Cristina says that summative assessment “need not be constructed in terms of pass/fail” and Jennifer suggests that “quality assessment is about balance” and using the right tool and the right time. She also says that the problem with using formative assessment to evaluate whether learning has occurred is that it is “unethical to evaluate a learner while they are still learning”.

Now, I consider myself to be an open minded chap, so it seemed reasonable to check out what’s been written on the subject to see if there is merit in their views.

Firstly, let’s define our terms. Summative Assessment is the formal testing of what has been learned in order to produce marks or grades which may be used for reports of various types. This is different from Formative Assessment, in which the emphasis is on on-going assessments of different types used to judge how best to help pupils learn further. Have I got this right? I’d be very grateful if anyone would like to (politely) set me straight.

Research suggests that while summative assessment can be motivating for some students, it has a negative effect on others:

• After summative assessment, low-achieving pupils had lower self-esteem than higher-achievers, whereas there had been no correlation between self-esteem and achievement before
• Repeated practice tests reinforce low self-esteem of low achievers
• When results of summative assessment are presented as primarily relating to individual pupils the negative effect on low-achievers is more pronounced than when the results are for evaluation of school or authority standards.
• Secondary age low-achievers may deliberately underperform in summative assessments because they are failing anyway
• Summative assessments can be limiting for the most able
• “Big bang” tests cause anxiety in pupils, especially girls and widen the gap between high and low achievers’ motivation
• Summative assessment promotes “extrinsic” motivation, in which pupils respond to the promise of some kind of reward rather than “intrinsic” motivation in which they perform because they are interested and want to do the work

Summative assessment can also produces problems for the curriculum and with teaching:

• The curriculum can be narrowed by “teaching to the test” which can take away from curriculum content
• It can produce distortion in terms of teaching techniques
• Summative test questions may not be framed in the same way as those preferred for formative assessment
• Teachers can spend a lot of time on summative assessment which does not directly improve pupils’ learning
• Teachers sometimes adopt a more didactic “transmission” style of teaching which disadvantages those who don’t respond well to it

Now, I accept that many of these problems can be planned for so that their effects are minimised if not avoided all together. Here are some of the ways to avoid some of the pitfalls:

• Intrinsic interest in tasks can be encouraged
• Students’ awareness of learning goals rather than test performance goals can be developed
• A wide range of types of understanding can be included in summative assessment
• Some formative assessment evidence may be included in summative reports
• Peer- and self-assessment could be included in summative records
• Tests don’t need to be formal written assessments
• The comparison of individual pupils on the basis of scores can be avoided
• Summative tests can be placed before the end of a teaching block so that there is some opportunity for follow-up based on the results, and even reassessment
• Summative judgements can be made on the basis of a variety of tests (varied both in form and content)
• Students could carry forward lessons from assessments even into the next school session (eg in the form of a copy of their school report)
• Feedback can be given to pupils in terms of the learning goals rather than just a test mark
• Tests might be devised to assess separate elements of the course separately
• In practising for summative assessment, pupils can make up and answer their own questions. (Research has shown this to be an effective strategy)
• Tests can be timed according to pupil readiness rather than leaving them to the end of the block of work
• Summative assessment can be presented realistically, as being limited
• Tests can provide evidence for evaluating courses and teaching approaches
• Whole-school discussion of such assessment principles can be helpful

The problem is though that these measures, whilst all very worthy, only provide a way to cope with something that is essentially damaging. Look how tentative the language is: the potential harm of summative assessment can, could, might be ameliorated by conducting it in a more formative way. If that’s the case, why bother? Why not simply ensure that all assessment provides opportunities for formative feedback which enable the teacher to redirect teaching and the learner to make progress?

Maybe I’m being dense but I really struggle with the idea the summative assessment is useful for anything other than providing data on students’ (and teachers’) success and failure. I’m not being bombastic and I like to think that I don’t deal in “vague rhetoric” as Cristina accuses me. In fact I’m desperately keen to be shown the error of my ways and will respond positively and graciously to any comments. 

Over to you.

Grading Currency

Grades are the currency students sell their love for learning to obtain.



For more on this topic, I suggest reading Alfie Kohn's answer to the question: should we pay kids to learn?

What's the point of assessment?

This was written by David Didau who is a Head of English at Priory Community School. David blogs here and tweets here.

by David Didau

Came across an interesting challenge by @purposeducation –#500words campaign, This week the topic is #purposedassess, so here goes…

Everyone knows that there’s two different types of assessment, right? There’s summative assessment which is all about finding out whether students have learnt everything they’ve been taught. This is the kind of assessment that the media reports on and which schools are judged on. GCSEs, SATs, A levels etc.

Then there’s formative assessment, or Assessment for Learning as its been rebranded. This is all about finding out what kind of progress students are making. This is (hopefully) what goes on in classrooms day in day out. If this isn’t what’s happening then frankly, I despair. Research (check on Dylan William’s Inside the Black Box) clearly shows that formatively assessing students’ work is the single most important thing you can do as a teacher.

The point of summative assessment is to classify and make judgements on people and institutions. Obviously this is important, but what happens is that we fall into the trap of ignoring anything we can’t measure. As Einstein said, not everything that counts can be counted. Most education systems seem to concentrate on a very narrow range of academic achievement and don’t value anyone who doesn’t succeed within this range. Sir Ken Robinson has a lot of very interesting things to say on this subject in his wonderful book The Element. He says that most education systems are designed to meet the needs of the Industrial Revolution and are failing the needs of the post-digital age where many of the children we are currently teaching will be doing jobs that have not yet been thought of using technologies that haven’t been invented. Summative assessment is all about trying to provided a fixed determination on what is good and whether we’ve achieved it. If you don’t meet the grade, you’re a failure.

Formative assessment, on the other hand, is all about finding out how individuals learn and tailoring future learning to ensure that their needs are met. It’s about giving students a toolkit to be able to make their own judgements on their progress and what they need to do to further improve. Assessment for learning allows students to take control of their education and encourages them to adopt a growth mindset: failure is a learning opportunity and lets teachers and students know what to concentrate on next.

Basically, the benefits of formative assessment can be summarised as:

• Students learn more effectively
• Students feel more involved in the schooling process and become less disaffected
• Teaching is focused more effectively on the individual student
• Positive effects may be particularly evident in the less able
• Learning in the wider (not subject-specific) sense can be enhanced as students become more confident with taking risks.

So, what’s the point of assessment? Is to decide who has failed and who has succeed with a narrow and outdated educational paradigm? Or is it to hand young people the reins of their own learning and allow them to become the risk-taking, free-thinking, creative individuals we’ll need in the wide uncertain future?

Troublemakers

Being a troublemaker is one of the best ways to get fired -- yet paradoxically, it's also one of the best ways to get ahead.

Not being a troublemaker helps to get through the day -- being a troublemaker helps to sleep at night.

Not being a troublemaker helps to get along with others but hard to live with yourself.

Being a troublemaker might be one of the best ways to ensure that you act on the differences between doing things right and doing the right things.

Troublemakers are outcasts -- until of couse everything changes -- then they're revered.

All revolutions are impossible... until they happen.

The most successful people in this world are troublemakers... it's just that when they become popular, we call them leaders.