Provincial Achievement Tests are gone! Now what?

On May 9, 2013, the Alberta Government announced that they are dropping the grade 3, 6 and 9 Provincial Achievement Tests in favor of what they are calling "more student-friendly assessments".

Some history:

• In 2010, Alberta Education made changes to their organization structure by eliminating the Accountability and Reporting Division.
• In 2011, Alison Redford became Premier of Alberta and promised to bring an end to grade 3 and 6 Provincial Achievement Tests.
• Former Education Minister Thomas Lukaszuk announced that the government wished to make changes to Provincial Achievement tests.
• Alberta teachers have long opposed Provincial Achievement Tests as the scores were either at best unhelpful for teaching and learning and at worst harmful.

What we know about the new tests:

• The new tests will be computer-based tests known as "Student Learning Assessment" and will be developed by "experts" and administered at the start of grades 3, 6 and 9. 
• They will focus on assessing numeracy and literacy while also assessing competencies such as creativity, critical thinking and problem-solving.

Skeptically Optimistic

The proposed changes to Provincial Achievement Tests brings both crisis and opportunity. As a critic of standardized testing, I am happy that Alberta is removing the current Provincial Achievement Tests. 

In fact, I'm ecstatic!

However, while we are writing epitaphs for the old tests, we shouldn't stand over the grave too long. After all, someone who the government is calling a group of experts is already planning the next foray of tests.

I'm also concerned that many Albertans have come to define their successes and failures with Provincial Achievement Test results, but now that these tests are on the way out, many Albertans will experience a kind of existential vertigo - how else can they define themselves if not by fancy bar-graphs and shiny pie charts depicting their test results? Many Albertans will need to be coached through this shift from test-based accountability to public assurance.

There are a lot of ways the Government could get these new tests right and there are a lot of ways they could get them wrong.

Getting the new tests wrong:

• Change for the sake of change is no better than tradition for the sake of tradition. If the only difference between the new tests and the old tests is their scheduling (spring vs fall) and their format (computer vs paper), they will only be "new" in name.

• Too many teachers and schools misused Provincial Achievement Test scores by making them count as a part of the student's report card or final exam. It would be educational malpractice to make these new diagnostic tests count on a report card.

• Putting these new tests online might be an improvement but not if they are still multiple choice. These new tests cannot be multiple choice. 

• If the government maintains their monopoly over accountability and public assurance and develop these tests in secret, we will have wasted the opportunity to involve the public in public assurance.

Getting the new tests right:

• The old tests were problematic for many reasons but here are at least three things the new tests should not do: (1) place undue stress on students or teachers; (2) unfairly rank and sort schools without regard for their unique circumstances in which they operate; (3) narrow educational opportunities for students.

• The old tests were terrible at helping teachers improve their teaching so that students could learn more. The new tests need to be diagnostic in nature which means the information would be used to improve -- not to prove.

• Ruth Sutton tells us, "Involving teachers, parents, students and the community in the process of assessment and evaluation is the key to a form of public assurance that serves to improve and develop our students and our schools, not just to measure them." The Alberta Teachers Association and the public must play an active and authentic role in designing and implementing these new tests.

• Public assurance needs to be more than a one-time computer test . We need to design authentic assessments that support learning while simultaneously assuring the public that students are learning. This includes the use of project-based learning, performance assessments, and learning portfolios. For more on this consider reading A New Look at Public Assurance: Imagining the Possibilities for Alberta Students.

What do you think? Will a shift away from Provincial Achievement Tests lead to a crisis or opportunity?

Valuing the score or the child?

Take a minute to watch this 1 minute clip.

The Problem with Percentages

This was written by Ron Eberts is a teacher and an administrator in Red Deer, Alberta. Ron blogs here and tweets here. This post was originally found here.

by Ron Eberts

Over the past several weeks I have been following a controversy about assessment in a neighbouring school jurisdiction. The controversy has to do with how Battle River School Division will be reporting student progress at the high school level. Traditionally percentages have been used to report to parents how their children are doing in their various courses. In Battle River the jurisdiction is moving to a system in which student assessment will be based on comparing a student’s competency in respect to specific outcomes in the Programs of Study, as opposed to simply an arithmetic compilation of a number of marks collected from assignments, quizzes and tests. They are proposing a process in which students progress, in relation to the outcomes, will be reported as falling into one of four ranges:

• Beginning
• Developing
• Achieving
• Excelling

These ranges would describe how the individual student is progressing in relation to the specific outcomes being assessed at the time. For instance, one of the outcomes in the Grade 10 Social Studies curriculum asks students to explore the impacts of globalization on their lives; specifically, students must evaluate efforts to promote languages and cultures in a globalizing world, and within this evaluation students must include an analysis of language laws, linguistic rights, cultural content legislation, cultural revitalization, and linguistic revitalization.

Proponents of percentages would say that the number “83.6%” would provide more accuracy in respect to a students competency in relation to this outcomes than the description “developing” or “excelling” would… and herein lies the problem… what exactly does 83.6% tell the parent (or the student, for that matter)? Does it mean that the student in question understands how to evaluate efforts to promote languages and cultures in a globalizing world 83.6% of the time? Or, does it mean that the student grasps 83.6% of the various ways that languages and cultures can be influential in a globalizing world? Or does it mean that the student only included 83.6% of the necessary analysis of the language laws, linguistic rights, cultural content legislation, cultural revitalization, and linguistic revitalization that was required in the demonstration of this outcome?

When viewed this way, the number, 83.6% provides almost no insight whatsoever in respect to an individual student’s understanding or application of the outcome being assessed. Yet, a large number of parents and students (and apparently, if the news articles are to be believed, teachers too) believe being told the student is excelling at this outcome is less descriptive than the student is 83.6% of this outcome.

Now, notwithstanding the lack of accuracy a percentage provides when reporting progress against an outcome, there is the mathematical magic that goes on behind the electronic gradebook scenes that makes a single number assessment even more suspect. Let me illustrate this by borrowing from assessment guru Thomas Gusky, in his article entitled, “Computerized Gradebooks And the Myth Of Objectivity”; in the image below a teacher has collected percentages from a number of students. Depending on the methodology used to calculate the “final” percentage, the students can range from all having the identical number, to a range of over 19 points difference:

Advocates of percentages who are arguing about the specificity and accuracy of using percentages cannot dispute that depending on the teacher and his/her method of calculating a “final” mark, an almost 20 point swing is anything but specific and accurate! What is worse is when the art of WEIGHTING is brought into the discussion. Weighting is when categories of “learning” are designated a certain percentage weight out of the total course. For instance, some traditional categories for weighting include such labels as tests, quizzes, worksheets, projects, and homework (and, unfortunately, sometimes labels with no academic value at all, such as participation, effort, or even attendance are weighted into a final mark). Let me illustrate…

Below are a set of numbers collected by a teacher for “Johnny”:

• Quiz 1 = 39/45 –> 87%

• Quiz 2 = 46/50 –> 92%

• Quiz 3 = 32/40 –> 80%

• Unit Test = 58/70 –>83%

• Assignment 1 = 12/15 –> 80%

• Assignment 2 = 15/20 –> 75%

• Assignment 3 = 4/8 –> 50%

• Assignment 4 = 6/10 –> 60%

• Group Project = 59/100 –> 59%

Method for grade calculation #1: Raw total divided by number of items (every item is weighted only by the raw point value of the task) = 267.5/343 –> 76%

Method for grade calculation #2: Tests 50%; Quizzes 25%; Assignments 10%; Group Project 15% = 80%

Method for grade calculation #3: Tests and Quizzes 10%; Assignments 50%; Group Project 40% = 67%

So, as you can see, even “hard data”, strictly based on percentages, yielded a 13% swing in results. This leads me to the question, who again believes percentages are specific and accurate?

In the end I understand that most high schools will continue to use percentages to report student progress. I just wish that the proponents of percentages would at least be honest about why they prefer them – not because they are specific or accurate, but because they are easy! If they really wanted accurate and descriptive, knowing how their children are doing in respect to the outcomes they are being taught would result in a descriptor-based system, not percentages.

Below you can find some news articles on this issue:

http://www.edmontonjournal.com/Camrose+protest+student+grading+system+draws+people+video/8230250/story.html

http://www.edmontonjournal.com/Editorial+Getting+testy+over+grades/8246281/story.html

http://www.edmontonjournal.com/grading+system+sparks+controversy+Battle+River+schools/8241655/story.html

Who will pack your parachute

Here is a guest post by Cherra-Lynne Olthof. She teaches middle school math, language arts and social studies. She tweets here and blogs at Teaching on Purpose. This post was originally seen here.

by Cherra-Lynne Olthof

Battle River School Division I applaud your new grading system. I have a few slight issues with how you really are still using percentages, but on the whole you are breaking new ground and I applaud you. You’re trying to change the system and that needs to be acknowledged.

When we changed our grading system in my middle school from percentage based reporting to skill level based reporting there was huge backlash from our parents. They said that percentages were much more accurate and this new system was “wishy-washy”. As a teacher I knew the real truth….the percentage system is actually the easy way out, while the skill based reporting is far more work and requires much more effort on the part of the teacher.

Concern #1: This promotes teacher laziness.

Here’s me teaching using percentages. Assignment, Assignment, Quiz, Quiz, Test….aaaaaaand marks plunked into a computer based reporting system that spits out an average. Little to no effort on my part really, other than the time to physically mark it. And I can get around that by making it mostly multiple choice so a machine can do it for me. And so simple to justify because when a parent says….how did my kid get this mark? I have 5 marks to show them. And our entire conversation revolves around that – the mark. I don’t even need to show you the assignments because when I put a mark sheet in front of you that a computer program processed for me, you won’t even question it.

The skills based system requires that I have much more knowledge of my learner because if I am called to justify the mark I have assigned I must be able to provide evidence in the form of the child’s work. And I must be able to clearly demonstrate what your child CAN and CANNOT do. It’s the reason my students put together portfolios of their best moments. Because then our conversation is around the skills of the child and not the mark. Each time your child hands something in I must sit there very carefully and really look at what your child has proven himself capable of. This isn’t so easy. Throwing your kid’s multiple choice test through a scan tron machine takes me 10 seconds. The funny thing is….parents will accept this as a more “truthful” assessment.

Concern #2: Why change the system that works? Can’t teaches stick with percentages but just add comments?

Dylan Wiliam has 5 strategies to improve student learning. None of them involve percentage based grades.

Wiliam has shown in previous research that when students are given a grade along with feedback that students only look at the grade. They asked students with this type of assessment two questions, “What was your grade?” (Pretty much all of the students reported back immediately and accurately.) “What did your teacher say about why you got that grade?” (More than half the students were clueless because they hadn’t read it.)

Other researchers have shown the same findings (as evidenced in this blog post by Joe Bower.)

Concern #3: Although not said outright like this (because it would be mean), basically the concern is….but how will I know how my child compares to others?

There’s a danger in comparing kids using percentages. When we first switched our system we used this example to say to parents…..who is the better student?

Student A: Unit Test Mark – 75%

Student B: Unit Test Mark – 75%

The parents had no response. Was this a trick question? And so I said to them, what if I could tell you that one of these students is clearly ahead of the other? They were confused.

So I provided slightly more information. Both these students wrote a test on adding, subtracting, multiplying and dividing fractions. There were 5 questions in each section. Each student got 5 questions wrong. Who has a better grasp of them material? No one knew. So I went deeper.

It was revealed that Student A’s mistakes were quite minimal. A few calculation errors here and there. The mathematical reasoning behind their skills with fractions was sound, he was making simple mistakes like multiplying 4 and 6 and accidentally writing 20 instead of 24. But on the whole, clearly understood how to add, subtract, multiply, and divide fraction. Got 1 wrong in each section, 2 wrong in multiplying fractions. All errors that the kid recognized immediately and had nothing to do with process. 5 minutes of corrections and it was done.

Student B got everything correct……until it came to dividing questions where he got every single question wrong. He didn’t have a clue that the second fraction needed to be inverted BEFORE multiplying to find the answer. This is a processing error. And since the outcome clearly states that students need to be able to add, subtract, multiply, AND divide fractions…..? This student would not receive a passing grade in my class. He clearly cannot do all four and we need some extra work here.

Same mark…..but 2 completely different students with 2 completely different grasps on the concepts. The problem is that when a kid goes home and mom says, what did you get on your test? Kid responds with 75% and mom says, “That’s great! Go get washed up for supper.” Conversation over. Kids usually don’t flip open the test to see why they got the 75%. The scary thing is that many of their teachers don’t either……

In the new grading system when mom says, “What did you teacher say about your test?” My student would respond with, “I need to practice dividing fractions. I did everything else right but I got all those questions wrong.”

See how the conversation changes from what the mark is to what the skills are?

THE ANALOGY

Everyone is always so concerned about how we can possibly know who is on top, who is the best, and who is clearly on the bottom. The problem is that percentage grades don’t give us the whole story, especially when we bring averaging into the mix.

I heard this story once and it really hit home with me.

Two students take a course in parachute packing. At the end of each week they are asked to pack 5 parachutes and then they are tested for effectiveness. If the parachute was packed in such a way that it would fail to open when the string is pulled, the student is give 0. If it would open, the student is given a 1. Marks are reported like this…

Student A has learned nothing in 5 weeks. His accuracy is the same at the end as it was in the beginning. 1 out of every 5 parachutes he packs still won’t open. Student B finished with a lower mark and started off really poorly. But consistently packed every parachute accurately in the last two weeks of the course.

But if all you were given was their grade in the course to choose by……who would you have chosen?

Student A has the better mark? I’ll pick him. (Good luck with that, hope the odds are in your favour!)

Neither! I’m not taking anyone unless they got 100%. (Fair enough, you’re missing out on the experience then.)

And this would be my answer….I’ll take Student B. He’s proven he can learn and improve and now has a 100% accuracy rating. Let’s go.

Obviously analogies are meant to prove a point and this wouldn’t really happen but think a little deeper about what the question is asking of you. It asks…..which student is truly the stronger of the two? The one with the better mark? Or the one who has proven he can learn and improve even though he started off as a failure?

As a teacher, my dream class is the one filled with Student Bs.

Anyways…..just something to think about…….

All I know is this. When I began skill based reporting I found that as a teacher I knew my students far better then I ever did when reporting percentages. My time spent assessing increased under this system as did my time spent preparing quality learning assessments.

Who’s being lazy? It sure as heck isn’t me.

Confession: It’s those days when I’m tired and exhausted that part of me wishes we had a percentage based system. Because that was SO much easier than what I’m doing now. And I rarely had parents question my marks. Then I remember that the way I assess now is far more valuable and I suck myself back up to return to what I know is good teaching.

Cute or morally bankrupt?

I suppose at first glance someone might think this video is cute.

I don't.

And if you know anything about how our obsession with testing is squeezing the intellectual life out of our schools, then I bet you don't find this cute, either.

As a parent of two young children, and a teacher of 13 years, I find this video intellectually indefensible and morally bankrupt.

Alfie Kohn summarizes my distaste for this video:

It drives me nuts when I see schools offer stress coping strategies - we'll give you a hand parents and kids at putting up with the stuff we are forcing your kids to do that has no justification. In other words, it's like a factory that's pumping pollutants into the air but is happy to distribute respirators to the neighbours. We need to shut off the pollution at it's source.

For those that think this is just kids having fun, and that I just need to lighten up, keep in mind that this is what they look like when they get older.

Should union dues be optional?

Honestly, I used to think that union dues should be optional. I've since changed my tune. I think I'm a lot more informed about the importance of collective bargaining and how and why labour needs to have a say.