Focus on opportunity, not cost

How some simple tweaks to alternative grading systems can significantly improve student perceptions

Photo by Gabriele Romano on Unsplash

Today, we bring you a guest post by Dave Musicant. Dave is a professor of computer science at Carleton College in Northfield, MN, where he teaches courses across the computer science curriculum. Dave has co-led discussion groups about alternative grading at Carleton, and has been experimenting with applying the ideas to his own classes.

---

Alternative grading offers students significant opportunities for learning and flexibility that traditional grading approaches don’t. I’ve been excited about using specifications grading in my courses over the last five years. And yet there are significant challenges that I’ve struggled to work through. Despite years of tweaking my techniques to accommodate thoughtful feedback from my students, too many of my students resented the frameworks I was using. Some of them ended up even more hyper-focused on their grades and on the principles of the grading system than they did on my old “bag of points” approach, which is exactly the opposite of what I intended.

After a fantastic discussion with the great folks on the Alternative Grading Slack¹, I realized that the biggest problems were fixable with some fairly minimal changes. It’s now obvious in hindsight that a critical issue for success is how the system is presented regarding student opportunities. The language and structure I had used unintentionally highlighted ways for students to fail, rather than their opportunities to succeed. Based on ideas generated in that conversation, I was able to make updates to my system that barely changed the substance of my approach, but dramatically improved my students’ views of it. I’ll explain those approaches here.

Problem: Negative grading information can disrupt learning

Many of my students are intensely focused on their final course grade. A strong subset of those students are deeply invested in getting an A in particular. But whatever their individual goals regarding their course grade may be, a large number of my students experience considerable stress about their grade. This comes to a head if they learn that the grade that they have received, or will receive, is not the grade that they had hoped to get.

Back in the bad old days when I was using my antiquated “grade on a curve” approach, my homework and exams were designed so that students often got low numeric scores on them. Because students knew the class would be curved during the final course grading, they didn’t stress too much about getting imperfect scores. Though they might be a little grumpy if they didn’t achieve as well as they hoped, they could still hold some faith that the curve at the end of the course might work in their favor, and to some degree they would put those concerns aside and continue studying. They might be upset if the final course grade didn’t work out as they hoped, but this happened after the class was over when it didn’t interfere much with their efforts during the term.

In the last few years, I’ve worked to make my grading system more transparent in a manner consistent with specifications grading. At least for the homework assignments, I use a “bundle” system, which indicates what a student’s final course grade will be based on how many submissions have achieved a certain level of success. The specific problem I ran into occurred when a student ended up at a point where the course grade they wanted was no longer achievable. This would typically occur if they had failed to submit successful work on too many homework assignments, even after resubmission opportunities were offered.

With my old system, students never really knew what their final course grade might be, so they never really knew that they were in this situation. However, the transparent system I switched to meant that everyone was precisely aware of where the lines were for particular grades. My students therefore put an enormous amount of attention on the grading mechanisms. I had regular conversations with severely distraught students about the fact that they knew they were on the edge of not getting the grade they had been hoping for, and were terrified about crossing that line. This had a profoundly negative effect on their experience in the course, and their learning. Students in this situation paradoxically lost much of their focus on learning the course material, and instead used most of their energy categorizing the ways in which things might go wrong, and trying to negotiate with me administrative workarounds.

I did offer opportunities for resubmissions and extensions which were supposed to help, but these opportunities were not infinite. And so either they ran out for some students, or for others the problem was simply the fear that they might run out in the future. These situations brought us right back to the original problem of stressed students working to negotiate with me exceptions and administrative solutions, rather than engaging in positive action to learn the course content.

This isn’t what I expected to happen! So much of what I’ve read about transparency in grading emphasizes its positives, of which there are many. But the reality I faced was that many students saw these transparent grading thresholds as an extreme source of stress when their performance didn’t line up with where they hoped. This resulted in challenging conversations, attempted negotiations, and excessive focus on administrative details that I had believed specifications grading would help to diminish.

Fix: Focus on the positives

I posted about my struggles on the Alternative Grading Slack, and received a lot of great suggestions. Based on those, I implemented two changes that made an enormous difference. The key detail I was missing was focusing appropriately on how students perceived their opportunities to improve their grade from where they were.

The first change I made was with regard to what I thought was a minor detail of my specifications-grading “bundles.”

The concept of bundles is that there is a set of criteria for each possible letter grade for the course. For example:

• If you succeed on 11 or more of the 12 homework assignments [and satisfy other requirements, such as on exams], you will receive an A.

• If you succeed on 10 or more of the 12 homework assignments [...], you will receive a B.

• etc.

A challenge I face is that I often don’t know at the start of a course precisely how many homework assignments I’ll have. Sometimes I’ll run out of time for something, or I’ll toss in one or two extras depending on how things are going. That makes it tricky for me to define bundles with the typical language described above. After playing around with some options, I initially settled on what I thought was a good solution: I’ll just use subtraction instead of addition. I used what I thought was cumbersome-but-effective language that looked something broadly like this:

N is the number of homework assignments in the course. This will be approximately 11-12 or so, but this may shrink or grow depending on how things go.

• If you succeed on N-1 or more of the homework assignments, you will receive an A.

• If you succeed on N-2 of the homework assignments, you will receive a B.

• etc.

Problem solved... or so I thought. It turns out that it focused students on negative outcomes. The original example I provided above, with precise numbers, directs students’ attention to opportunity. “If I want an A, I just need to succeed at 11 of these things, and every homework assignment that I succeed at helps me get there.” The second approach focuses on the costs. “If I want an A, I can only miss one homework assignment.” Even though these two sets of bundles are functionally identical, the subtraction signs in the second one call students’ attention earlier to costs rather than opportunity, and primes them for stress. (There were many contributors to this portion of the Slack conversation to whom I am grateful, though David Clark was the one who most directly pointed out this issue.)

After these conversations, I realized that I could in fact implement the first approach without knowing for certain how many homework assignments I would have; I just needed to pick an upper bound. For example, suppose I decided that the number might vary, but I was confident that it would not go over 15. Then I would define my bundles as follows:

• If you succeed on 14 or more of the 15 homework assignments, you will receive an A.

• If you succeed on 13 of the 15 homework assignments, you will receive a B.

• etc.

This seems well and good, but then what happens when the course is nearing the end, time ran short, and I only assigned 12 homeworks, instead of the originally promised 15? I simply create three fictional stub homeworks to represent the missing assignments, and announce to my students that they will all receive scores of “succeed” on all three of them. Huzzah! The students are thrilled that they got positive scores for free, and they perceive a sudden drop in the amount of work. It’s a win for them. But look carefully at what I’ve done. My original goal for an A, for example, was to specify that “N-1 successful homeworks are sufficient for an A,” based on whatever N turned out to be for the course. That negative language worked poorly for the reasons described earlier, but with a little sleight of hand, I’ve pulled off precisely that goal. without having to know in advance how many homeworks I would have time to assign. This system of adding fictional assignments at the end results in exactly the same grades for students that they would have earned under the previous negative grading system I was using. But students’ perception of this new approach is dramatically better.

Fix: There’s always hope

Despite the above being helpful, the fundamental problem remains. Some students undergo significant stress and/or detachment from the course if they end up at a point when the grade they want becomes unachievable. It was pointed out to me in the Slack conversation that some opacity in grading might not be a bad thing in this environment, especially when framed as an opportunity. To that end, my syllabi now typically include language that looks something like this:

“During final course grading, if a student shows significantly stronger grades on exams than on homework assignments, the final course grade may potentially be nudged up. Remember that exam scores can be improved via the retakes offered during the final exam period.”

Adding this language has a number of benefits. It means that if students can demonstrate via in-class assessments that they have mastered the content that they missed on the homeworks (even after maximizing resubmission opportunities), there’s an opportunity to improve their grade as a result. But regarding the specific problem described above, it allows students to maintain hope for the grade they wish through the end of the course. If they don’t get the grade they want, they’ll be grumpy whenever that happens, but this changes the timing of when certainty occurs so it is less likely to get in the way of their learning during the course.² This loophole reintroduces some of the lack of clarity that my old grading system had, but only to a small degree. It has applied thus far to no more than one or two students — and sometimes none at all – in a typical class of around 30 students. But it has considerably reduced the temperature and frequency of conversations I’ve had with students that were struggling to hit the grade they hoped for.

Problem: Late tokens must be hoarded

It is enormously helpful to give students flexibility to sometimes turn in homework assignments late. Granting unlimited opportunities for submitting late work, however, can do more harm than good. Many resources on alternative grading suggest using some sort of token system to manage this. Students receive a fixed number of tokens, and every time they wish to turn in a homework assignment late, they use a token. I used an approach like this a number of times, and had mixed success with it. While it generally worked for some students, there were others who were particularly anxious about the possibility that they might need late tokens in the future after they had exhausted them all. For these students, every time that an occasion arose when they wanted to turn in something late, they met with me and tried to negotiate an exception to the policy so that they could turn in their work late without using a token. If they failed to do so, they sometimes submitted substandard work rather than taking the extra time that they needed... purely because they were worried they might need that late token for something more severe in the future. This level of complexity is never what I intended or hoped for.

Fix: A renewable economy

It turns out that there was an easy fix for this as well that largely achieves the same goals: I switched over to a renewable token system. A common approach that multiple people took was to give students one late token a week. They could use that token once during that week, or not, as they wished; regardless, at the end of the week, the token would expire and they would receive a new one the following week. I liked the idea, but for many of my courses, the homework assignments come in a slightly less-regular rhythm, so I instead chose an approach where every set of three consecutive homeworks has a late token associated with that set. For example, homeworks 1-3 have a single token associated with them, homeworks 4-6 have a single token associated with them, and so on. If students wish, they can use a particular late token for any one (but only one) of the three homeworks it applies to. This has been a big improvement over the “X late tokens for the course” approach that I was using. Students are happier to use the tokens if they need to because there is no gain to be had by hoarding them. I also don’t have to handle the challenging situation that arises when students use all of their late tokens early in the course and then attempt to negotiate for more. That said, there are downsides. With renewable tokens, some students see them as “use it or lose it” resources that they should spend, and sometimes use one every week just because they can, not because they actually need the extra time. While the costs of using a late token are not dramatic, it does contribute to students struggling more to understand in-class content because their out-of-class work is more behind the rest of the students.

Focus on opportunities for students, not costs

A common thread throughout this experience is that reframing costs as opportunities makes a big difference in reducing stress for some of our students, and in helping them to focus on learning instead of on gaming the system. I’m still experimenting with variations on the above to try to improve things further, but the approaches that I’ve laid out here have been a notable improvement.

1. If you wish to join the Alternative Grading Slack, post a comment on this blog. The organizers will then send you an invitation by email.

2. The core of this idea, implemented very differently, was suggested to me by Kevin Lin.