How I am updating my alternative grading approach for Fall
Embracing my inner McNulty
Good morning, it’s mid-August. That means it’s time to get those Fall classes ready! I’ve been heads-down getting mine prepared for the last few weeks, but all summer I have been thinking about how to improve my experience from last year. I never wrote a postmortem here about that year, but in a nutshell: While my courses were not terrible, I struggled with teaching last year1 and my grading setup did not always make things better.
Last month, David shared how he is updating his alternative grading setup for Calculus 2, after a six-year hiatus from teaching that course. He referenced some of my methods in his post (like the use of engagement credits) and I got a lot of ideas from him in return. Today, I wanted to share some changes I’ve made and how things are shaping up for the start of the new year.
Background
The class2 I am teaching this fall is Discrete Structures for Computer Science 2. This is the second class in a two-semester sequence; I ordinarily teach the first course, and I haven’t taught the second course since 2022. The entire sequence covers the essential math content needed for further study in Computer Science3. The class size is usually around 30, with typically 90%+ of the students being second- or third-year Computer Science majors4.
The course has an interesting, and challenging, dual focus. On the one hand, this is where computer science majors learn how to read and write mathematical proofs; the first semester of the course dabbles in this but then it’s a major course goal in the second semester. On the other hand, the content of the course (other than proof techniques) is mostly applied graph theory – of which there is a tremendous amount in a formal study of computer science. So there are a lot of basic skills in the course in the form of learning the language of graph theory and applying the theory to real life problems; and an equal amount of material centering on crafting clear, logical proofs about graphs, trees, relations, and so on.
That dual focus has implications for how to assess, and then grade, student learning in the course. In our book, David and I suggest that when choosing assessments and grading setups, you should decide whether the course is primarily about content and skills (in which case something like standards based grading would be a good fit) or about concepts and processes (in which case, consider specifications grading or ungrading)5. If it’s an almost 50/50 split, then the grading system will need to reflect this somehow.
How it started
Last academic year, I taught just one course, the first semester of this course sequence (two sections each semester). I wrote a little about what I was doing in this post which described some changes I was making between Fall and Winter semesters. The upshot was that the Fall version was too complicated and I was making some changes for Winter to simplify. Here is a quick summary of how that semester was structured:
There were three main contributors to a course grade: Mastery of basic skills, evidence that students can apply those basics to challenging problems, and engagement.
Mastery of basic skills involved having 16 “Skills” (for example, “I can determine elements of a recursively-defined sequence using a recurrence relation.”), six of which are designated as Core skills and the other ten Supplemental skills. Students’ ability with these Skills was assessed through timed assessments called Checkpoints (here is a sample) that were intended to be bi-weekly and cumulative. There were also mini-Checkpoints focusing on 1-3 single Skills. Student work on these was graded Success or Retry. If the work was a Retry, they’d try again at a later Checkpoint. Otherwise, students had to earn Success on two different Checkpoints in order to complete that Skill.
Applying basics to challenging problems was assessed through Challenge Problems (here are the problems I gave). Students picked problems off the list and wrote up solutions, which were turned in and graded Success or Retry, then a feedback loop ensued with revisions done and submitted until there was Success.
Engagement was assessed using engagement credits which I mentioned above. A specific assignment, “Class Prep” (where students would watch videos, do readings, and answer simple questions in preparation for the next class) was part of the engagement credit total.
Here is the full syllabus for the course. At this spot in the syllabus you can see how the final course grade was determined, basically by counting up how many accomplishments in each of the main areas you’ve made.
This version of the course was the result of simplifying the previous semester’s version. And I do believe it was simpler. But I also ran into several significant problems during that semester, chief among which were:
A culture of constant quizzing. The intention, as I stated, was that Checkpoints would be bi-weekly. But as we stepped into the semester, we ended up having to change the frequency to a more or less weekly pace6. Since Checkpoints are cumulative, they got bigger and bigger in size, and took up more and more of students’ headspace. The focus of the class became testing not learning and many students had a hard time thinking about anything other than the next Checkpoint.
Challenge Problems being… challenging. I did not put a deadline on Challenge Problems, other than no further submissions were allowed during the last week of classes or afterwards. This gave students flexibility to take their time, but also led to some problems with procrastination. While three submissions a week were allowed, the median number of weekly submissions through the first half of the semester was either zero or close to it. There was also no layer of assignments in between Checkpoints and Challenge Problems to bridge student learning between the bottom two levels of Bloom’s Taxonomy and the top two7. It led to a situation for some where they put off working on Challenge Problems until well past mid-semester, then didn’t have the practice in jumping from basic tasks to high-level applications.
Class Prep issues. Completion rates for Class Prep hovered around 70% all semester. I think this is reasonably OK – it’s certainly not the epochal levels of disengagement I’d heard about from others – but since Class Prep was needed for the next class (I didn’t lecture on the material in the Class Prep) many times we faced the issue of ⅓ or more of the class not having the necessary background to do the work. Part of the problem was how Class Prep counted for a grade: It was just another source of engagement credits, albeit the main source, so if a student felt they could reach a critical mass of engagement credits through other means, they would be tempted to skip doing Class Prep (and many did just that).
How it’s going
My experience in Winter with the first semester of this course wasn’t bad by any means. I’ve always liked teaching it, and I connect with the students in this course sequence (who tend to be a glorious bunch of nerds, i.e. my people). But I wasn’t satisfied with the outcomes in Winter, and all summer I have been trying to figure out what to add, subtract, or modify to make my Fall course one that I look forward to going to every day. Here is what I have come up with.
Go full McNulty re: assessment. I decided early on this summer that Job #1 was to reduce the frequency of Checkpoints so the class doesn’t become a single 15-week long quiz and review session. I immediately turned to the work of Prof. George McNulty, professor of mathematics at the University of South Carolina until his passing in 2023.
George McNulty is in some ways the Godfather of alternative grading. He devised, seemingly from thin air, what amounts to the world’s first standards-based grading system for university calculus. You can learn more about him and see some of his syllabi in this guest post from Kate Owens, who was one of Prof. McNulty’s Ph.D. students. His syllabi for Calculus 2 and 3 were on my mind constantly when I was first experimenting with alt-grading and I have always admired the simplicity of his course design.
McNulty’s syllabi for Calculus are a master class in simplicity8, especially with assessment. In Calculus 2, there are 16 skills, each assessed by a particular problem type. These problems appear on just four exams – Skills 1-4 on Exam 1, Skills 1-8 on Exam 2, etc. – with new variants of each problem appearing on subsequent exams. Each problem is graded on three levels: Master, Journeyman, or Apprentice. Earning “Master” on a problem/skill means you’ve mastered it, no reassessment required. The course grade was based on how many skills you’ve mastered. There’s a little more to it, but not much more, and you can read all about it and see a sample final exam in Kate’s post.
I decided to adopt McNulty’s approach wholesale for my Discrete Structures 2 course:
There are 14 skills, 5 of which are Core.
These are assessed through problem types, appearing on a series of four exams, each of which is cumulative (like McNulty’s) and with new variants of those problems each time.
The work on those skills/problems will be graded Master, Proficient, or Beginner according to standards that I have set up here. (I have some issues with the word “Master” but I think students understand this term better than its synonyms. “Journeyman” on the other hand has some gender-specificity problems, and both “Journeyman” and “Apprentice” are a little outdated in my view.)
The course grade will be determined in part by how many Skills you Master.
Under this approach, there are only four Checkpoints instead of 10+. I have set aside other days in the semester calendar for makeups, and the final exam is just another round of variants of all 14 skills. Reducing the number of checkpoints like this creates scarcity, which as David wrote about here can actually be beneficial to students as long as there is flexibility for students, which there is (it’s in the syllabus). And not constantly having a checkpoint bearing down on you will hopefully allow students to breathe a little.
Separating Applications and Proof. I’m taking a page from David’s playbook and creating something called Advanced Homework that sits in between basic skills and high-level creative work. These will be roughly bi-weekly (but not more often!) and will have some ungraded9 practice exercises along with 1-3 mid-level application problems involving the basic skills. These will be graded Success or Retry and students can revise each one once. Then, students will work on Proof Problems (here is the list of these so far) to build a portfolio of well-written proofs that’s required for a grade of A or B. This creates two types of assessment where there used to be only one. But it makes sense because I need that intermediate layer between basics and high-level work; and because proof is a particular focus of this course, unlike in the first semester.
Goodbye Class Prep, hello Homework. I have been rethinking my approach to flipped learning ever since Covid hit, and this semester I’m trying something different. Instead of Class Prep being focused solely on watching video, doing reading, and working basic exercises that move the initial contact with new ideas out of the class meeting (the usual approach to flipping), I’m turning that assignment into two things: Review of recent class meetings, and a lighter preview of the next class which would involve things like refamiliarizing yourself with prerequisite material that’s needed, experimenting with a concept and recording the results, making guesses about the outcome of a thought experiment, and so on. In other words, getting familiar with the upcoming new ideas not through sitting through a video lecture but through playing with the ideas and bringing the results to class. Then I’ll give a brief lecture (there, I said it) at the beginning of class to formally introduce things, followed by active learning activities.
I’m rebranding this assignment using the old-fashioned term Homework. It’s not my favorite, but it’s a term that students understand, and it connotes an assignment that they need to do. Homework, like Class Prep before it, will be graded on completeness, effort and timeliness only – not on correctness. So it’s easy to do; all I really want is the data from the results to know how to adjust my in-class plans. And, it will be its own separate column in the grade table in the syllabus – not just another way to earn engagement credits, so there’s no getting around doing it.
Here is that grade table, as of today:
Full details are in the syllabus.
What’s next
Classes start on August 26 for us, and I am out of town for about half of the time between now and then, doing speaking and workshop gigs. So I’ve been double-timing it to get the course built as much as possible before today.
I consider the basic structure of the class to be final, and I’m feeling pretty good about it. The assessment scheme makes sense and doesn’t seem to be heading me straight to grading jail. The grading system feels pretty straightforward. Even if the assessments themselves feel like throwbacks (exams! homework!), the way they work and the feedback loops attached to them give students the iterate-until-successful mindset I want them to have.
One thing that definitely has not changed, is the 12-Week Plan, whereby I cover no content at all in the first week or last two weeks of classes. Week 1 for us is solely about onboarding, which includes an entire day of practice with this grading system. By the second week of classes, the plan is that students will be familiar enough with the ideas that they won’t really think about grades at all until the end of the semester. It helps that about a third of my students are former students from the first semester of the course sequence, so they are totally familiar with how this works – and I think that’s a vote of approval from them about it.
Part of my struggles had to do with larger-scale issues, which I’m currently writing about at Intentional Academia.
Two sections of this one course.
Except linear algebra, which ought to be in there somewhere, but that’s another post.
The first semester of the sequence is often first- and second-year students. It’s quite common for students to put a year or more in between the two courses in the sequence, and the difference in experience levels is significant.
This happened because on two different occasions, I was scheduled to be off campus for a speaking engagement on a Checkpoint day., But I could not find anybody willing or able to fill in for me, and we couldn’t shift the day of the Checkpoint during that week. So they had to be scheduled in consecutive weeks, which created a domino effect where Checkpoints happened in back to back weeks elsewhere.
I tried a type of assessment, called “Application/Analysis” to fill this gap in a different course, but it was a flop and I killed it off mid-semester.
Although: When you read his syllabi, you might be struck, as I was, by what isn’t in the them: No policies on late work or makeups (what if a student is absent for an exam?), no policy on academic dishonesty, no description of how plus/minus grades are handled, etc. So they’re not as simple as they look.
Literally, as in, there is no grade on this part, only feedback.