Mixing and matching SBG and Specifications (part 1)

How I use multiple grading approaches in an intro-to-proofs class

Photo by Mike Petrucci on Unsplash

This coming fall, I’ll be teaching MTH 210: Communicating in Mathematics for the umpteenth time.¹ This 4-credit sophomore-level course, capped at 20 students, is a required course for all math majors in our program. It forms a bridge between our introductory classes (like Calculus) and higher-level major classes, many of which have MTH 210 as a prerequisite.

The course has many different goals. On the one hand, it introduces fundamental mathematical skills and tools that students use throughout the rest of their years as a math major. On the other (very big) hand, it’s an introduction to writing proofs: logical, written justifications of mathematical statements. Proofs are the core of much of advanced math, and as such, this class is an introduction to the conventions and culture of professional mathematics. This really is writing: A typical MTH 210 proof ranges from 2 paragraphs to 2 pages, more or less in the format of an essay with a heavy focus on logic. The course has a heavy enough emphasis on writing that it satisfies GVSU’s discipline-based writing requirement.²

The course also uses active learning, in class and out. I use Ted Sundstrom’s excellent (and free!) Mathematical Reasoning: Writing and Proof which is designed for a flipped classroom. In class, students spend most of their time actively engaged with worksheets, writing proofs at boards, discussions, and more.

Because of these multiple goals and design choices, I use an unusual mix of alternative grading systems in this class. I’ve refined this approach over the semesters, and I think it works well for this class’s many competing requirements.

Over the next two weeks, I’ll describe this grading system. This week, I’ll focus on the details of the multiple approaches I use and why I used them. Next week, we’ll take a look at things I don’t grade, how final grades work, and some final advice and reflections.

The design process: What do I care about?

When I first designed my alternative grading system for MTH 210, I asked a key question: What things do I care about most in this class, and how can I assess them? My answer had three parts:

• Discrete skills: MTH 210 introduces students to key skills that they’ll need in many future math classes. These are similar to what you might expect in a more introductory math class: discrete methods, definitions, and techniques that students can learn and practice more or less separately from other skills.

• Communication and synthesis: Students need to learn more than individual skills. They also need to put them together to solve problems, to write clear and convincing proofs that incorporate them, and generally to synthesize and apply skills in new contexts. In particular, this means being able to construct abstract arguments that bring together multiple techniques and skills in an impossible-to-separate way. I want to assess this ability to synthesize and communicate a holistic understanding that goes beyond individual skills.

• Learning how to learn: Coming into the class, students often have a very static view of math: It’s about facts, it’s about numbers, there’s always one right answer, the trick is to find the right formula and apply it. Professional mathematicians see math very differently, in terms of patterns, connections, and especially sharing and communicating precise logical ideas. Learning and studying look very different with this higher-level viewpoint, and I want to help students make this transition.

These three goals are intertwined. But because they look so different, I realized that I needed to assess each of them differently too. In the next few sections, I’ll outline how I do that.

Discrete skills: Standards-Based Grading on weekly quizzes

To assess the discrete skills that students learn in MTH 210, I use a fairly standard model: Standards-Based Grading (SBG) on weekly quizzes.

In class, we study individual skills that students can learn and practice more or less separately from other skills. These involve using and applying definitions, specific formulas, methods, etc. in relatively concrete situations.

To assess these, I created a list of 18 standards (which I call “learning targets”) that describe the specific skills. I always indicate relevant standards on any class activity, slides, videos, reference sheets, etc. Here are a few examples, along with the short codes that identify them:

• L.3: Negate a conditional statement.

• N.4: (core) Congruences, modular arithmetic, and modular arithmetic shortcuts.³

• S.1: (core) Create sets using set builder notation and translate sets into this notation.

Each week we have a “skill quiz” that covers learning targets that we’ve recently learned about in class, typically two targets per quiz. Each quiz has one page of questions per target. There are usually multiple questions that address each target from several viewpoints. The questions are fairly standard and involve stating definitions, applying the skills in concrete situations, and analyzing or critiquing examples. The target being assessed is clearly stated at the top of the page. Here’s a typical example, featuring yet another learning target:

Students earn a mark on each learning target. That mark indicates their overall progress towards demonstrating their understanding of that skill, taking all of their work on that page into consideration. The marks I use are:

• Successful: Demonstrates a thorough and consistent understanding of the target. Any errors are irrelevant to the target.

• Revisable: There’s an error that brings into question the student’s understanding, but I think it might just be a minor misunderstanding or typo. Students can come to an office hour, tell me what was wrong, and how they would fix it. If they do, I upgrade their mark to Successful.⁴

• Needs new attempt: There is an error central to the target, something serious enough that the student needs to start from scratch: reflect, study, and make a new attempt on a future quiz. I also use this mark for frequent small errors or significant lack of attention to detail.

I give detailed written feedback as necessary. If a student earns Revisable, I usually just give a hint about the general location or type of error, to help them focus.

In a typical week, the quiz is done asynchronously during a 6 hour window, with students choosing any 30 minutes during that window to take and submit the quiz via our LMS. I don’t use surveillance tools, but I do ask students to write and sign an academic honesty statement.

But because these are fairly basic questions, I’ve recently moved some of the assessments in-class. About every 3-4 weeks, I dedicate one hour of class time to a “big quiz”. This in-class quiz contains only new attempts on targets that students have previously attempted on an out-of-class quiz. This gives students built-in reattempts while providing a little more security.

In order to consider a target “complete” (which contributes to their final grade), students need to earn two Successful marks on each learning target. I generally aim to offer 3-5 attempts on quizzes spread throughout the semester. I even share a planned schedule of learning targets with students so that they can see when future attempts will be available.⁵ I do reserve the right to rearrange the schedule based on which targets are most needed, and I’ll sometimes ask students for advice about which targets they think are most urgent to reassess.

This regular schedule means that reassessments without penalty are built right into the quiz setup. One big advantage of this approach is that, other than the occasional Revisable mark, all reassessments happen on a predictable weekly schedule. That helps both students and me manage the workload.

The final exam is one last “big quiz” with a new attempt on every learning target. On the final exam, students also need to “recertify” six special targets that are marked “(core)”, even if they’ve already earned two Successful marks. This is to demonstrate retention of the most critical skills in the class. I’ll say more about how this, including how targets count towards final grades, in next week’s post.

Communication and synthesis: Specifications Grading on a portfolio

As part of the communication focus of the class, I especially care about students being able to put individual skills together to write mathematical proofs and to practice with the conventions of written mathematics. This isn’t about individual skills, but rather about how they all fit together in a logical argument, which means that SBG isn’t a good fit to assess this part of the class.

Instead, I use Specifications Grading with a portfolio of written proofs. The key idea here is that Specifications Grading lets me assess these written proofs holistically, based on a clear list of specifications that describe these holistic properties. This makes more sense than checking individual skills that contribute to the written work.

I begin with fairly standard assignments for a proof-based math class: Throughout the semester, every 1-2 weeks, I assign a new set of proofs for students to solve and write up, effectively in the form of a short mathematical essay. There are 9 of these assignments total throughout the semester, and collectively they ask students to put together the individual skills, methods, and techniques we’ve learned about in class and write well-communicated proofs that use them.

Students engage in a fairly standard revision cycle: They first submit a draft and, one week later, a revision. I grade these using a consistent set of specifications that spell out the qualities of a successful proof. There are two lists of specifications: First is a required list, which students must fully meet without significant errors. Second is a list of what I consider “excellence” specifications: more advanced requirements for which there is correspondingly more wiggle room. I’ll describe these both in detail a little later.

Students earn one of these marks on each submission, which you’ll note are a bit longer and more descriptive than my quiz marks:

• Math successful: This applies to a draft that follows all instructions and is mathematically correct, with enough detail and justification to make it understandable, but there is little or no attempt made at professional communication.

• Math & writing successful: This describes a submission that meets all of the required specifications, and may have consistent errors in at most one of the “excellence” specifications, or occasional errors in up to three of them.

• Math & writing excellent: This is reserved for a submission that meets all specifications with no significant departures.

If none of these apply, students earn a mark of Not Yet, and I ask them to revise and resubmit their work. It’s worth noticing that it’s impossible to earn a mark only for good writing: The underlying math has to be correct in all cases, which is a key aspect of good mathematical writing. I explain this to students explicitly: “Writing a proof that sounds really good but is logically wrong is kind of what ChatGPT does.”

I encourage students to intentionally submit rough-draft work that will earn Math successful for their first attempt. This lets them focus on the underlying logic without worrying as much about the communication-related details of the specifications (they sometimes find this encouragement surprising, but quickly see the advantages). This helps students be more confident before continuing to write a full proof. There’s plenty of time to write that proof, since the schedule for every proof includes a revision – which should follow all specifications – due one week after the draft.

Here’s an example of what some of the required specifications look like. Students must thoroughly address all of these in their work in order to earn Math & writing successful or higher:

1. Include a true and correctly worded theorem statement just before the proof.

2. At the very beginning of the proof, state all assumptions and explain what will be proved (even if this repeats what was in the theorem statement).

3. State the proof method being used. If using proof by contrapositive, contradiction, cases, or induction, clearly state the new assumptions or statement being used.

4. Give clear, correct, and organized justification for each statement. Directly explain why each statement is true.

There are more, but you get the idea. The last required specification is “Follow all instructions on the portfolio sheet, including additions or changes to these specifications.” This most often includes additional instructions about generating examples or counterexamples.

There is also a somewhat longer list of “excellence” specifications that students must address to earn Math & writing excellent. Here’s an example of some of these “excellence” specifications:

1. Use “we” instead of “I” or “you”.

2. Include every equation or formula in a sentence, including punctuation where appropriate.

3. Use appropriate symbols in formulas and equations; avoid excessive symbols (or writing out formulas in words) otherwise.

Some of these specifications are detail-oriented, others are quite broad. Some focus more on the mechanics of the math and logic, others on the written quality of the work. Most of them are introduced in our textbook as “writing guidelines”, and we spend significant time in class understanding the motivation for these guidelines, practicing with them, and critiquing sample proofs.

As an aside, the “excellence” specifications might not strike you as the most important qualities of “excellent” writing. That’s actually the result of some serious reflection on my part about what “excellence” should look like in this class, and indeed, I used to define excellence in more general terms: “complete, correct, clear, and concise.” But this is an introductory class on mathematical writing. The goal is not to write beautiful proofs, or even for the students to really be fully-formed experts at proof-writing (there are many follow-up classes where they continue to improve and refine their proof writing skills, not to mention grad school if they choose that career path). Rather, I care that they can understand and implement logic and disciplinary conventions. Thus I’ve decided that “excellent” means “careful and thorough attention to detail” as represented in these specifications, rather than some more ambiguous sense of elegance or beauty. This approach also helps me avoid (some of) the issues I wrote about in The Enigma of Exemplary.

After the draft and revision, students can continue to work on these proof assignments throughout the semester. I answer questions about them during office hours, but don’t accept additional submissions for grading until the very end of the semester. Students resubmit all proofs one more time at the final exam in one big “proof portfolio”. I reread and regrade each of them, and this final mark is the only one that counts in their final grade (although to be honest, if a student already reached Math & writing excellent earlier then I only skim to ensure the logic remains sound). Thus the final proof portfolio is effectively a second revision that’s available to all students. Like with quizzes, this regular schedule helps students know what to expect and plan their schedule accordingly.

A student’s final grade depends on the number of portfolio problems that have reached each level of this rubric – more details next week.

Next week: Learning how to learn, final grades, and a reflection

I hope you’ve enjoyed this tour of two major parts of my class. By using Standards-Based Grading to assess discrete skills on quizzes, while separately using Specifications Grading to assess synthesis and communication in a portfolio, I play to the strengths of each assessment system and help support students in different types of learning.

Next week, we’ll start by taking a look at my third goal for the class: Learning to learn. What does that even mean, and how do I grade it?

Then we’ll dig into how I put together these different approaches in a final grade. Last but not least, I’ll describe how I’ve made changes over the years to reach this assessment system, and reflect on how well it works. See you then!

Update: Here’s the link to the second part!

1. Many sections across 11 semesters, it turns out.

2. Our students must take two writing-intensive “Supplemental Writing Skills” classes, which have our university-wide intro-to-writing course as a prerequisite. In these courses students get at least 4 hours of explicit writing instruction, submit at least 3000 words of writing during the semester, and work with the instructor to revise their writing. A bit mysteriously for an alternative graded class, the university requires that “at least one-third of the final grade is based upon the writing assignments” – I’ll say more about that next week.

3. This learning target doesn’t exactly follow our advice about writing standards, especially the fact that each standard should have a clear action verb. While trying to make my standards short and snappy, I decided that the full description – something like “State the definitions of <list of topics> and use them in concrete calculations” was too long and didn’t actually communicate more than just the list of topics themselves.

4. You can read much more about the benefits of Revisable marks here: When students get stuck at Not Yet.

5. I deliberately spread out the attempts to help ensure students have retained some of the knowledge, and to reduce the amount of work on any one quiz. In particular, I generally don’t offer a new attempt in the week immediately after the first attempt.