Limitations

What is beyond the power of alternative assessments?

Photo by Sigmund on Unsplash

Robert and I write a lot about the benefits of using assessments that follow the four pillars. Improving your assessment system can decrease test anxiety in students. Students tend to focus more on learning, especially by revisiting previous topics that they still need to work on. Office hour conversations improve, students who might have disappeared or dropped can succeed—there are tons of benefits.

Today, we’re not going to talk about any of that. I want to give an honest picture of the ups and downs of alternative assessments. So, let’s talk about what alternative assessment systems don’t do. These systems aren’t magic, and they can’t cure all ills. So what are their limitations? How could they make things worse? What shouldn’t you expect to happen if you completely change your assessment system?

It’s still grading

In this section, we’ll focus on standards-based grading, specifications grading, mastery-based testing, and similar methods whose common features I previously called “all-or-nothing grading¹”. In these systems, students earn “marks” for their work, but the marks are essentially feedback on whether the relevant standards have been met or not. Either the work is acceptable (meets the specifications, demonstrates proficiency, etc.), or it isn’t. I’m avoiding ungrading in this section since there is a key difference: The systems we’ll focus on do involve assigning marks during the semester.

Let’s cut right to the chase: There’s no question that SBG, Specs, and similar systems are clearly grading. That means they invoke the incentives and baggage that comes with grades. As I wrote a few weeks ago, they are still a game that students have to play.

As Butler and Nisan showed, when students get grades along with feedback, they focus on the grades, not the feedback.

This isn’t to say that assigning grades is inherently “bad”, or that feedback is wasted. Indeed, one of the strengths of these systems is that they have built-in incentives for students to use and act on feedback, through revisions and reassessments. But the act of assigning grades, even ones that are clear indicators of progress, pulls against this strength. It’s a mixed message.

Playing a game isn’t necessarily bad either. Clear structure and visible expectations are good things that help students understand what to focus on. They break down barriers created by implicit assumptions about what students should already “know”. This is especially true for younger students and those who are not familiar with navigating traditional academic systems. But be aware that, when designing an assessment system, you’re creating the rules for a game that students will play. These rules may be clear and well-thought-out, and they should encourage beneficial behavior—but they are ultimately still rules. Don’t be surprised if you see students focusing on the rules as they play the game.

Trust and trauma

We’ve often written about trust, and how building a trusting relationship with students is critical to success in every class, much less in a class with some weird new grading system.

Robert and I are both mathematicians. Our students—especially younger ones—tend to come to us with a great deal of mistrust. They are used to the “gotcha,” the clever trick, the tiny detail that makes an entire solution invalid, the error that propagates throughout an exam and turns a passing grade into a failing one.

This is especially prevalent in math, but it is by no means unique to us. All students bring their past experiences with them into class—they are never a blank slate. This often includes trauma, anxiety, and inequity rooted in their past experiences with grading. It can also involve anxiety from the expectation to succeed, imposed by themselves, parents, or others. This is especially true for students who have “always done well” and have come to tie their identity to performance in school.

Changing your assessments can be an excellent first step towards undoing these experiences. But it does not, and can not, singlehandedly reverse a lifetime’s worth of trauma.

Implementing a new assessment system doesn’t create trust by itself. However, that trust is necessary for the new assessment system to work. Building trust is a separate process, and it can be a difficult one. Don’t underestimate: Trust-building is an uphill struggle, and students are likely getting conflicting messages from other classes.

In particular, we reap what we sow. Almost nothing can help if an instructor has a distrustful, antagonistic, or coercive attitude towards students. It will show through in the design and implementation of every part of class, even in alternative assessments. The best way to build trust, is to start by trusting.

The good news is that the way in which you implement and introduce a new assessment system can help build that trust. You can tell and show students that you’re not going to pull the rug out from under them. It’s worth addressing this head on: “I want you to succeed, and this class is designed to help you succeed. I believe that everyone in this class can earn an A. You will always know what you need to do to earn any grade you want. It may not be easy, but there won’t be surprises. My goal is to help you make it happen.”

Don’t be surprised if students resist your messaging at first—they probably have a lifetime of experience with grading that works against your message. Keep at it, more frequently than you may expect². Show—by your actions as well as your words—that things are different in your class.

To support that message, improving your assessments should be part of a suite of interventions. Other interventions can include using universal design for learning (UDL), implementing active learning techniques, not using remote proctoring or other invasive technologies, being flexible with deadlines, and generally recognizing students as humans and treating them with empathy and kindness rather than suspicion and mistrust.

The rest of the class

There are many moving parts in every class. Assessments, class structure, policies, communication, organization… these each contribute to the culture and atmosphere of a class. Your choices send messages. They can reinforce and support each other, or they can send mixed messages and cause confusion.

A beautifully designed standards-based grading system is great, but it won’t help as much if it’s paired with a traditional droning-on lecture. Specifications grading paired with poor organization and inconsistent communication? Might do more harm than good.

This doesn’t mean that you need to completely blow up all aspects of a class and redesign them. But, an assessment structure can’t overcome bad decisions or poor execution in other areas of course design and instruction.

Before you jump feet-first into (say) ungrading, take a careful look at the other parts of your class that you have the power to change. You don’t need to use the most cutting-edge innovative methods everywhere, but be aware of what is possible. Are you using baseline good ideas like active learning? (This can be done even in huge lecture sections, with techniques like peer instruction.) Is your course well organized on your LMS, or whatever else you use? Is your communication clear and consistent? Organization and communication in particular are places where student comments (such as on end-of-semester surveys) can tell you a lot.

If you find that students are already confused or concerned about non-assessment portions of your class, then adding in a brand new assessment system could actually make things worse. It can add another layer of confusion, and reinforce a negative narrative about the class. An innovative assessment system isn’t magic, and you need to view it in the whole context of the class.

Am I ready?

Teaching is a multifaceted thing. It is a collection of many interlocking skills that can grow with training, practice, and reflection over many years. Assessing student learning is just one part.

The purpose of this post isn’t to discourage anyone interested in improving their assessments. Rather, I want to set realistic expectations for what can be done by changing your assessments—and perhaps point you towards more critical issues. Assessments must be viewed as an integrated part of your teaching practice.

I recently came across an article that will be very helpful for anyone who’s feeling a bit uncertain: “Is Specifications Grading Right For Me? A Readiness Assessment to Help Instructors Decide” by Adriana C. Streifer and Michael S. Palmer (College Teaching, 2021). Come to think of it, this is probably a good idea for everyone to read, uncertain or not! It steps the reader through a structured set of questions, ranging from institutional and class context through instructor attitudes, that will help you decide if you are ready to try something new with your assessments. Basically none of it is specific to Specifications Grading—the same readiness assessment will work for anyone thinking of making any big change to their assessments.

One item I would add to this self-assessment: Is your purpose to increase student grades? If so, that’s not a particularly good reason to change to SBG, Specs, or even ungrading. I know what you’re saying: Didn’t this guy just write that average grades tend to increase with an improved assessment system? It’s true, but as the article describes, a shifted grade distribution is a consequence of having more learning-focused grading policies. Your focus should be on how your system can improve learning, and make grades more meaningful, not simply how to increase grades. When making a big change in assessments, final grades can change significantly, and for any individual student that might be a rise or fall compared to a “traditional” class. If you’re focused on increasing grades directly, that can lead to poor choices when designing assessments, and missing out on other benefits.

As always, I monitor the comments here as well as on Twitter (where I am @dccmath). Feel free to ask questions, challenge my assumptions, and continue the conversation!

---

1. I somewhat regret the term “all-or-nothing,” since it comes too close to “one-and-done,” which describes the incentives traditional grading systems give: Looking at your grade, sighing, and tossing it in the waste basket since there’s nothing to be done about it.

2. I usually advise having a plan to help students understand how assessments work when they are most ready and primed to think about it: Before classes start, on the first day of class, just before the first big assignment, after the first big assignment is graded and returned, around midterm time, and before final exams.