Entering the feedback loop: Robert's origin story

Sometimes snap decisions work out pretty well

My story with alternative grading started on a December afternoon in 2014, entering course grades. Normally this is a happy moment, the ritual act of closing the books on another semester and entering into the break. But this time, I was deeply unhappy — in fact, I was angry, and ready to do something about it. And it was all because of one student.

I had taught Calculus during Fall 2014 semester, like many Fall semesters before. On the first day, one student really stood out. She was visibly excited to be in class, talkative, energetic. A biology major, she loved math and was ready to learn everything she could. In the first few weeks, she was fully engaged in the class and helping others be engaged. She was like a power source for the class — the kind of student professors dream of having.

Then came the first test.

It happened at the usual time for tests, around week 4. As the hour unfolded, a palpable change came over this student. That electric enthusiasm became shrouded and muted. Something wasn’t going well on that test for her. By the end of the hour, she was visibly upset. She turned in her test and left without a word. Later, while grading, I could see why. I forget the exact score she earned, but it was low, shockingly low for someone so engaged and intelligent. In fact, it was so low that it took her out of contention for an “A” in the class, in one stroke.

She returned to class changed. She attended, took notes dutifully, and never missed an assignment — but the energy was gone. She was just an ordinary student doing what she needed to do to get a B in the class. But I began to notice something about this student. Her work on assessments would initially be poor, but as newer concepts that built upon earlier ones came up, her work would rapidly ramp up and not only not be poor but demonstrate outright mastery of those older concepts. She was learning… on a consistent two-week delay from my course schedule. And unfortunately, the assessments and therefore the grades happened on my schedule.

At that point in my career, I had been teaching math for 20 years. I liked to think that my assessment and grading system for Calculus had evolved into something fair, functional, and accurate. It wasn’t just a three-tests-and-a-final setup; it was nuanced and had measures built in to give students choice over how much items would count toward the grade. Students never complained about it. It seemed to be simple, reliable, and pretty darned progressive.

And for this student, it was also totally broken. Pretty soon, she was broken. The remaining tests, assignments, and final exam went about as well as the first test. After Test 2, she was mathematically eliminated from a “B” in the course. Her classroom behavior went from muted and ordinary to downright disruptive. One time she interrupted class by loudly saying “I don’t know why we’re doing this, I’m not going to learn it anyway, so what’s the point?” By the last month of the term, there was no way she could earn a C, and she spiraled into anxiety and disruptive behavior and eventually dropped off the map. I never heard from her again.

---

Sometimes I'll enter a low course grade for a student at the end of the semester, and while I wonder what the student or I could have done better or differently, the grade itself feels right. The student did poorly on assignments, and got a bad grade, and that's how it's supposed to work. This time, though, when I entered in that "F" for my student, everything felt wrong. She was slam-dunk for an "A" based on all the signals a professor normally relies on, and in fact her work eventually showed mastery. But because she operated on a 10-day delay from everyone else, she ended up being broken on the wheel of my grading system. She was more than just a false negative --- she was a person, dehumanized by my system.

I typed in the "F" to our records website and clicked "Submit", and then declared that this can never happen again. So I made the snap decision to take all three of my next semester's classes --- which had already been built and the syllabi written --- throw all of it out the window, and start over with anything but the grading systems I'd used throughout my career. I had no idea what I would replace them with. I had three weeks to figure it out.

---

Sometimes, by simply being open to change, the means for making change happen will appear.

The day after submitting grades, I came across an article titled “Advocating a new way of grading”. It grabbed my attention on two levels. First, I just so happened to be in a frantic search for a new way of grading. Second, it involved Linda Nilson, who had been a friend and mentor for over 20 years since working for her at the Vanderbilt Center for Teaching as a Master Teaching Fellow. So imagine my pleasant shock to learn that she had just published an entire book on exactly what I was grappling with.

This article gives a good overview of the concept of specifications grading if you’re unfamiliar with it. The important things to know for this story are (1) it was not traditional points-based grading, which I had sworn not to use; and (2) it was predicated on feedback loops. In specs grading — and in most forms of what we now call mastery-based grading, of which specs grading is a flavor — students do work; they are given feedback on its quality relative to appropriate and well-understood standards; then they try again, incorporating the feedback in to the next iteration.

I spent an entire week devouring this book and making pages of notes about what it all meant, and how it might work. By the end, I was converted — I’d be installing specifications grading into all of my Winter 2015 courses. Did I fully know what I was doing? No. Has this ever stopped me before? Also no. Sometimes you have to run before you can walk.

---

That semester was… very interesting.

My first attempt to implement specifications grading, on a short time scale and basically sight-unseen, was not perfect. My system was far too complicated, using a system of “concept checks” and “learning modules” that barely make sense to me looking back on them in 2021, and 68 learning objectives! I was in “grading jail” often throughout the semester. Students (and the professor) struggled at times to understand how it all worked. You can read for yourself what I did, and what students did, through these documents:

• The syllabus

• A sample Concept Check

• A sample Learning Module

And yet, it was also really good. Students understood and appreciated the concept that there were no one-and-done assessments in the class; that anything that wasn’t “good enough” could be redone until it was; that they could on some level relax and stop obsessing over their grades and focus on learning.

Although I was definitely not “saving time” as advertised in the subtitle of Linda’s book, the time I did spend on grading felt much more meaningful. I was no longer wasting time agonizing over whether a student’s work got 9/12 points versus 8/12 points. If the work didn’t meet the specs in my opinion, I’d just mark it No Pass, give some detail about what was good and what needed improvement, and have them redo it. The focus of conversation with students moved from What do I need to make on the next test to keep a B in the class? to things like What is a graph isomorphism and what can I do to understand them better?

I got into this profession to make a positive difference in the lives of college students. I hadn’t realized up until then just how much a traditional grading system was working against that vision. By repealing and replacing it, even with a flawed system, the class became a safe space to make mistakes — an invitation to engage in a feedback loop that makes you a better thinker and problem solver, not a Skinnerian dystopia where your every move is audited and the humanity of learning is outlawed. This what what I signed up for.

And most importantly, I felt like if my student from Calculus had been in a class with this system installed, she wouldn’t have been left behind.

---

Each semester since then — for almost seven years now — I have tinkered with that early system and tried to improve it, to simplify it, to make it more accurate with fewer moving parts. My attempts haven’t always worked; in particular the second attempt in Fall 2015 was mind-blowingly bad. (I’ve since apologized publicly to all my students from that semester.) But I think I’m slowly getting there, in the same way I ask my students to get there — by engaging in a feedback loop where we try things, examine the results, get feedback, and try again.

One thing is clear: I am never going back to points-based traditional grading. If I really care about students, and I do, then I am going to do whatever it takes to ensure the best possible learning environment for them. Traditional grading, quite simply, pulls in the opposite direction of that goal, and it’s obvious now that I’ve gotten away from it.

So that’s my story. And it’s the reason I’m contributing to this newsletter with David and writing a book about mastery grading with him, and the reason I feel so strongly about creating strong communities of practice around this idea. I’m glad you are along for the ride.

---

Subscribe to get Grading For Growth posts delivered directly to your inbox, once a week.