Transparency in grading
Transparency in grading
Four ways to think about whether grading is transparent, or not.
One of the early chapters of the forthcoming Grading For Growth book
goes deep on the origins of "traditional" grading systems (more on what that means here), piecing together the story how the initial conditions on grading have morphed into serious issues affecting student learning and growth today. As I wrote this chapter, I started wondering: Why would any instructor choose a system like this? I knew there were instructors out there who were aware of the shortcomings of traditional grading, who know about alternative approaches and have the position and freedom to do something different -- but don't. Why?I was genuinely curious, so I put that question out on Twitter. It got some serious responses from instructors, and one of the most common reasons for sticking with traditional grading even when other options are available, was transparency. As in, Traditional grading is transparent for all those involved.
Originally this article was going to be an extension of David's "mythbusters" posts where I would take this notion that traditional grading is transparent and once and for all extinguish it with a series of devastating counter-arguments. But it turns out that it's more complicated than that. In this post, I want to unpack what I think “transparency” means, what I think other people think it means, and how this entire concept works in grading of any variety.
Seeing through transparency
Transparency means you can see through something, like a clean window. The meaning when translated into more abstract settings is basically the same. Generally, it means openness and honesty. There aren’t any hidden or invisible moving parts to the system, process, or policy. The entirety of it is visible and comprehensible down to the verbal substrate used to write it up.
You can feel it when a policy or process is not transparent. With grading, I felt this with my kids. Their schools use standards-based grading up until the fifth grade. Instead of a report card, they got a two-page printout of all the standards, along with a simple 0-4 numerical indicator on how far along they were on each. This was an incredibly useful resource as a parent, since I knew exactly what each kid needed to work on. But after the fourth grade, everything switches to traditional grading, and transparency vanishes. My oldest kid once came home with an "F" in math, and after tracking down her teacher, it turns out she had an “F” because there had been exactly one assessment, a single 5-point quiz on which my daughter had made a 3. SBG was real transparency; the traditional grading almost felt deliberately opaque.
In practice, transparency is complicated, and people can mean different things when they talk about it. To clarify the meaning of transparency with respect to grading, I asked on Twitter again:
Four basic ideas emerged about what "transparency" means, and why traditional grading is transparent:
Familiarity;
Objectivity;
Predictability; and
Auditability.
Let's take a look at these one at a time.
Familiarity
The argument: Traditional grading is what we know. It doesn't take a lot of extra work for students to comprehend what is going on in a traditional system. Put differently, traditional grading lowers the extraneous cognitive load on students in an otherwise difficult course. As a student, you can walk into a traditionally graded course and instantly have one less thing to worry about learning.
I think this argument has merit. There's no question that traditional systems are what students are used to seeing, and those of us who use alternative systems like specs grading or SBG know all too well how much work it requires for students, and for us, to come to terms with how those systems work. Otherwise we wouldn't constantly be asking or fielding questions like How do you get students to buy in to your system? Nobody ever asks this about traditional systems because the buying-in has already happened (or students just accept it).
So I'm willing to stipulate this argument in favor of traditional grading. It's clearly an advantage over alternative methods -- for now at least, until alternatives become more common. The real question for me is whether familiarity is enough to justify the use of traditional grading in light of the other issues traditional grading poses.
Objectivity
The argument: Traditional grading uses quantitative measures of student learning that express student achievement on scales that are more accurate, more realistic, and less susceptible to personal bias than alternative approaches, where student work is evaluated either in oversimplified binary learned it/didn't learn it categories or through the instructor's opinion. And once the individual items of work are graded, the result of the grade is mathematically determined rather than being left up to individuals to decide in the moment.
There are some significant holes in the argument for objectivity. David wrote an entire post about this. Rather than remake his arguments, I’ll just summarize:
Traditional grades don’t represent student learning. You can put a number on student work if you want. But the meaning of that number does not convey the full range of information about the work to which it’s being attached. David’s post gives an example where two students both end up with a 60% average in a course, but the same number maps to two completely different but equally statistically justifiable meanings.
Traditional grades are determined by humans. Just because a grade is represented by a number, it doesn’t mean it is more objective than a mark that is not a number. In fact, those numbers are determined by humans, not by hooking students up to some kind of learning measurement machine. Human beings decide how many points an item or assignment is worth, how those points are granted or withheld, and how the work translates to points in a rubric. And those decisions are not scientific. They are professional judgment calls, meaning that faculty make these decisions about point allocations based on their professional expertise and their understanding of what constitutes different levels of quality in the work. To be clear: Professional judgment calls are not bad! But it’s not being transparent if we hide the fact behind a number.
Traditional grades are “objectivity theater”. They give a false sense of objectivity without being actually objective. It’s like averaging ZIP codes.
But to be completely fair, I appreciated this article titled "The Benefits of Traditional Assessment". The author states it's tongue-in-cheek but I think this point is serious:
Traditional assessment doesn’t try to stamp a particular skill or topic as “learned/didn’t learn” but instead gives a number that is more or less a percentage of accuracy, averaged in some way between quizzes, tests, a final, and maybe some other stuff. Trying to say someone has “mastered” a topic (a common theme in nontraditional grading) misunderstands the role of forgetting, false positives, and accuracy. Traditional grading provides more information about the student’s learning and doesn’t mislead students into stamping something “mastered” when they actually haven’t, or when they’re likely to forget it the next day.
I have issues (maybe the author does too) about how accurate this is, but I do think the point about "forgetting, false positives, and accuracy" is important. We preach keeping it simple around this blog, but an oversimplified alternative system that says a student has mastered a topic without sufficient evidence is of no help to the student. (It's one of the several reasons we moved away from calling it "mastery grading" a while back.)
Predictability
The argument: Traditional grading gives students the ability to know where they stand and where they are headed at any point in the course. And it makes it easier for the instructor to be consistent in grading, since point values can be determined by rubrics and not by judgment calls.
There are issues with the argument from predictability too, and they are rooted in the issues with objectivity:
If the number grade itself hides information about how that number was assigned or what it means in terms of learning, then a student can crunch numbers all they want to "find out where they stand" in a course, but in the end all they will get is a feeling about where they stand. And this is very often a dangerous false certainty. That's not transparency, and we owe more to students than this.
If feedback is not part of the grading process, then it's difficult or impossible to truly know where you stand or where you're going with a grade in the course if you're a student. The numbers might work out so that you need a 75% on the final exam to get a B in the course. But how will you know what a 75% on the final means, or what you need to improve on in order to get that 75%?
About consistency: I'm willing to stipulate the idea that a well-designed rubric improves the consistency of grading; and it's an obvious good, and it promotes transparency, to have grading that is consistent. Anybody who, like me, has graded the free-response portions of an Advanced Placement exam knows how important consistency is, and the value of a rubric in getting there.
But as any former AP grader also knows, having a good rubric is necessary but not anywhere near sufficient for consistency. Back in the day when I was part of a small army of math teachers grading the AP Calculus exam, we spent precious hours during the one week (!) we had to grade tens of thousands of AP exams, just training on the rubrics for each question. And yet, there were always significant differences in the scores given to the same work, based on whether it was graded early versus late in the day, or before versus after lunch, or on Tuesday versus Friday. We had grading overlords ("table leaders") checking our work for consistency and inter-rater reliability, and yet still had to stop at times to retrain or even change the rubric.
Better than points given by a well-designed rubric, I think, are simple and honest judgment calls, given to students in the form of helpful feedback, then ironed out through participation in a feedback loop. For more on consistency and the issues that come with it, check out Sharona Krinsky’s recent guest post on normed grading.
Auditability
The argument: Finally, traditional grading makes it possible for students to tell if they have been graded unfairly, and possible for parents and institutions to make sense of our grades. I think of this as the ability to audit the grading process, hence the (entirely made-up) name.
Two things are definitely true about the argument from auditability. First, we know perhaps all too well that traditional numerical grades let students check to see if an instructor's grading has been consistent. Try giving one student an 8/12 on an assignment and another student 7/12 for what appears to be the same work, and see what happens. Second, it is historically accurate to say that grades arose partly out of an emerging desire to involve parents in their kids' education (this is why Horace Mann invented the report card) and an emerging need for different school systems to communicate with each other about individual students' academic records as families in the mid-19th century became more socially and geographically mobile.
But there are issues here as well:
Student "audits" of their grades do indicate transparency. But what we gain in transparency, we lose -- with interest -- in the kind of learning environment and relationships we get. Many of us shifted to alternative grading precisely to put an end to the endless point-grubbing, competition, and adversarial student-professor relationships that these audits engender.
Anybody who's ever been involved with evaluating a course for credit transfer knows that the fact that a student got a "C" in a course at one institution does not mean that the student has done "C" work in the course with the same or a similar name at your institution. All the questions we at this blog ask about grades, come up in this discussion: What was involved in the course? What were the assessments like? What did the student learn? What does the student know now? The letter "C" hides all this information, and even if you could trace the "C" back the original assessments, all you'd get are numbers -- marks that do not indicate progress.
Let's be fair
I've highlighted some of the issues with the notion of transparency applied to traditional grading. But in all fairness, each of these four arguments has a grain of truth in it -- and a warning for alternative grading.
Traditional grading is more familiar to students, and using an alternative approach does increase cognitive load on students. There's no doubt about this. If you're using alternative grading, you need to be mindful of it.
Just because traditional grading isn't free from personal bias, it doesn't mean that alternative grading is bias-free. We have to be constantly on guard to make sure our feedback is truly helpful and given equitably.
While the "knowing where you stand" benefit of traditional grading is largely illusory, it is important that students be able to know how they are doing in the course at any given time. If we're doing alternative grading, then it ought to be easy for students to get this knowledge.
And although points don't guarantee consistency, consistency is nonetheless important and something that ought to be a goal of our system, whatever it is.
So, is traditional grading transparent? In the end, I think it could be. You would have to have points that are assigned to student work through well-designed rubrics that have been checked for inter-rater reliability, and which are made available to students. You would need to give an explanation for each point allocation (i.e. why the work mapped to a certain level of the rubric which then mapped to a point value). The course grade formula would have to be comprehensible to the students so they could use it to find where they stand.
By that point, though, I think you're so close to using alternative grading, something predicated on engagement with a feedback loop, that it would be less work to just go the whole way: Drop points altogether, let students do reattempts without penalty, and spend your time giving helpful feedback instead of allocating points (and adjudicating the complaints that arise from them).
Still, just as traditional could be transparent, alternative grading can very easily be incredibly opaque. I can show you some of my early attempts at specs grading if you don't believe me. So transparency isn't something that is inherently present or absent in a given grading system -- it's there as a result of the intent and work of the instructor who makes the system work.
Still on track for an April/May publication date, thanks for asking.
Excellent post - but I'd offer one pushback on "familiarity": just because it's familiar and default, that doesn't mean that students have necessarily bought into the system. There are lots of students who haven't fully bought in, even if they accept it as their default mode of assessment. I think many struggles that some students face are precisely because the assessment systems they are working under don't really make sense to them - but because they are default, they don't feel like they can question them. One advantage of non-traditional grading systems is their very defamiliarity - because the systems need to be learned and understood, they promote more overt and conscious buy-in, rather than default acceptance.
Loved this post as a place for thinking. Note that you've predicated it on hitting how traditional grading is or isn't transparent based on four components (familiarity, objectivity, predicatablity, and auditability). Primarily, you've used traditional grading examples and both given credit for the way traditional-points-based grading can be those things and also a few drawbacks or "myths" as you will, but not given quite the same attention to how alternative or ungrading methods can still meet these four components.
I found myself thinking about how ungrading, specifically, can be familiar - if we make it so, can be reasonably objective - as much as any other assessment (I am thinking of writing assessment here), and auditable - if we define the way progress will be monitored and then follow through. I left predictable for last because it's the one my students struggle the most to wrap their thinking around. They often voice concerns that without points to continually check, they don't know "how they are doing" or can't predict how they will do. However, once the buy-in happens, they realize ungrading actually provides THE MOST predictable outcomes. If students are working toward learning, hitting due dates and benchmarks/goals, and reflecting in ways that move them forward, they can reasonably predict success (in whatever way they define it). Because I use a grad pitch method, their predictions should hold true most all of the time. . . and in a few situations where that doesn't seem to be the case, it can usually be sorted out with a candid conversation.
I guess I'm saying there are ways to be transparent in all four of these sub-categories even without the use of traditional points, scores, and averages. Thanks for driving the conversations on better assessment.