Mastery Exams
A timed test of proof and logic without (as much) anxiety
A few years ago, I taught a brand new (to me) two-course sequence: Real Analysis 1 & 2. These are upper-level courses, required for some of our majors and taken by electives as others. Each class is capped at 24 students and is often much smaller.
These courses require students to write and understand proofs, which are written mathematical arguments. If you’re not familiar with them, you might be surprised at what an upper-level math proof looks like: It’s mostly words, sentences, and paragraphs with some carefully selected symbols. A proof can range from one paragraph through multiple pages. The essence of a proof is to communicate precise technical ideas in carefully selected language, and small variations in wording can lead to big differences in meaning. Writing proofs is a skill that students – and all mathematicians – build throughout their lives, beginning as undergraduates.
For these reasons and more, students view Real Analysis 1 & 2 as some of the hardest courses we offer, and a test of their mathematical skills at the end of their major.
While designing the classes, I wanted a way to assess students’ proof-writing abilities, as a way to let them show deep understanding of class topics. One way I did this was through take-home homework, which gives them lots of time to think, collaborate, and write. But I also wanted something proctored, to give me a bit more confidence about students’ individual knowledge.
That’s the pain point that I encountered: Writing mathematical proofs in a timed, high-stress exam environment is really hard to do. Proofs that students encounter in Real Analysis are often long, detailed, and abstract. Coming up with the central logic of such a proof, much less working out the details and ensuring its correctness, can take days. It’s a process that benefits from lots of time sitting in a comfy chair, staring at a wall, and just thinking. Timed proof-writing is also inauthentic: Mathematicians just plain don’t do math in that way. Math takes time and requires a lot of thought, false starts, and wrong turns, on the way to eventual success. How could I assess that in a shorter, proctored setting?
To solve this problem, I turned to an experience that I’d had many years ago as an undergraduate: “mastery exams”.1 In today’s post, I’ll walk you through how I used them, how well they worked, and things you might want to consider when using them yourself.
How mastery exams work
Mastery exams are not typical in-class exams. These are brief, focused, timed exams in which students are asked to recreate proofs that they’ve already seen. Generally these proofs are too complex to memorize, and are selected so that a student who understands their fundamental ideas can reconstruct the details on the fly. This gives students a chance to show in-depth knowledge in a proctored setting, with the benefits of being able to prepare, struggle, and learn outside of that proctored setting. It also removes the surprise and difficulty of having to invent a proof from scratch.2
I assign three mastery exams per semester, each covering a few main topics from the last month or so. Each exam contains a short list of proofs or definitions that students will be expected to be able to recreate in a clear, complete, and correct form. Importantly, these are all results that we’ve already seen and studied before, either during class or on a homework assignment – so students always have a chance to practice the exam content before taking it.
I post the full requirements for each mastery exam shortly after we’ve finished studying those topics in class. That’s right – I tell students exactly what will be on each exam (here’s an example of the first mastery exam from last year). That linked file is exactly what is posted on our LMS. In class, I announce when a new mastery exam is open (typically during weeks 6, 9, and 12 of our 15-week semester).
Mastery exams happen outside of class, during an office hour or a time of students’ choosing. After the exam has been posted, students can choose when they attempt each exam. This gives them time to study and practice the proofs.
To take the exam, a student comes to any office hour — or sets up another time individually — and asks for the exam. I give the student a paper copy of the exam — the exact thing they saw on our LMS — and they sit down at a table and write out complete proofs of those results, from scratch, based only on their memory and understanding. I allow up to 30 minutes for this, although typically students use less. Students can’t use any resources when taking the exam, but I encourage them to prepare for the exams by working through proofs with classmates and using any regular class materials they’d like.
I hold office hours in a common study space, and I often have multiple students taking mastery exams at various tables around the room. When they are ready, they bring their written work to me and I read it for completeness, correctness, and clarity. This is usually a quick process, since I’ve also reviewed the proofs ahead of time and know what to expect.
If all of the proofs are complete, correct, and clear, then the student earns Successful on the entire exam, otherwise Not Yet. I grade the exam “live” in front of the student, talking through my thought process as I do. If I run into a minor issue that is worth fixing, but not so serious that it requires a new attempt, I’ll ask the student to explain or correct their work on the fly. Occasionally this turns into the student reworking bits of the proof on a whiteboard.
In the case of Not Yet, a student can reattempt the same exam up to three times total. Attempts must be done on different days so that they have time to review and improve their understanding. The questions are the same on each reattempt.3 All that I record is a student’s ultimate grade: Successful or Not Yet, regardless of the number of attempts.
There are three different mastery exams during the semester, and they must be taken in order: students must pass Mastery Exam #1 before they can attempt #2, and must pass that before they can take #3. I require this because the topics in Real Analysis build heavily on each other, so each exam requires a solid foundation from the earlier exam topics. If you’re thinking about using a similar setup in your class, this strict sequencing might or might not be necessary.
Mastery exams are a key element of the final grade. Along with other requirements,4 in order to earn an A, a student must earn Successful on all three mastery exams. For a B, two exams, and for a C, just one exam. Not earning Successful on any exams means earning a D or lower.
Results
I’ve loved using mastery exams. They satisfy their main purpose well: to show me a student’s understanding of some key proof ideas in a proctored setting. This is supported by having a carefully curated collection of proofs that are laser-focused on the big ideas of each class. When a student completes a mastery exam, I’m quickly able to tell if they truly understood the key ideas in each proof. Better yet, if they have some confusion, the exams reveal exactly where that confusion is, and I can give targeted feedback to help them improve.
I worried that adding a timed test element to these classes might cause too much anxiety for students. I didn’t need to worry. In anonymous surveys (and also through informal comments during office hours), students said that they liked the challenge that the exams posed. Having multiple attempts without penalty was a “stress-reliever”, and students liked the flexibility in choosing when to take each exam.
Not all students were thrilled by the exams, but their main complaints were actually about logistics: For students who had a hard time making it to my office hours regularly, there was an additional step of scheduling a special time to take the exam. I was very flexible about this, but some students found that extra step annoying.5
As is usually true in alternative grading, this structure gives students the power to set a grade goal. If a student passed exams 1 and 2 and were happy to earn a B, they could decide not to worry about the last exam. Several students did that quite intentionally and told me so.
On the flip side, even though students are limited to three attempts on each exam, nobody has ever run out of attempts in the three semesters I’ve taught these courses. Some have certainly needed all three attempts, especially on the first exam (it takes some time for students to get used to the format and expectations). The median number of attempts required on the very first mastery exam is 2 (out of 3 possible). For the second and third mastery exam in each semester, students needed a median of 1.5 attempts each.
Q&A
You likely have some questions about this approach to exams, so I’ve put together some of the most common concerns I hear:
Won’t students just memorize the proofs? What are you really testing them on, then? I choose proofs that are too big and too complex to memorize in full. I deliberately choose proofs that require certain key methods or ideas at their heart, and are structured in a way that a student who understands those key methods can reconstruct the rest of the proof. The result is that students must work to understand the overall structure and logic of the proof, but then recreate the details on the fly. If they don’t come into the exam truly understanding the underlying ideas, it immediately shows in their work. Students notice this as well – many have told me that they had to truly understand the big ideas of each proof before they could pass the exam.
Why not just give everyone the exam all at once in class? I could, but my more flexible approach lets students take the exams when they feel confident and have had time to study. That’s an important part of reducing the stress and anxiety of a timed exam. Plus, this approach saves class time and gets students to come to office hours! I did notice that some students delayed taking the first exam until I started bugging them about it. When I encountered mastery exams as a college student, my professor had a way of dealing with this: If a student hadn’t passed Mastery Exam #1 by the Friday before spring break, they were required to come to class that day to attempt it again (everyone else could leave for break early). This was very effective at motivating students to prepare.
Aren’t these still high-stakes, high-stress exams? I was worried about this, and students certainly take the exams seriously, but they don’t report a lot of stress or anxiety. Built-in reassessments help, as does the ability to choose when they take the exam. Being able to collaborate and even have practice sessions with classmates also improves students’ confidence. The first attempt on the first exam is often a bit rough, as it is any time students encounter a new type of assessment – but it gets considerably better after that as students learn and adjust, with no grade penalty.
Do students need to produce polished proofs? Isn’t that a lot to ask in a timed setting? Polish isn’t required, but students do need to communicate well enough to unambiguously show correct understanding of mathematical and logical ideas. Part of the test is understanding how and when to show that understanding most clearly. You can see my advice about writing at the top of the first exam. I do find that students are sometimes surprised at how “silly writing mistakes” can actually lead to really serious logical errors.6 If a student earns Not Yet on their first attempt for this reason, I immediately pivot to helping them see the underlying logical issues.
What other classes might mastery exams work in? I’ve only used this approach in upper-level proof-based math classes. Proofs require multiple steps of logic and an understanding of how all of those steps fit together. Even a student who has seen a complete proof still needs to do some serious logical work to fully understand the proof’s logic, which is what the mastery exams best show. Other situations that involve longer-form logical arguments (perhaps philosophy), or the ability to reproduce a detailed chain of logic that’s based on important core principles (like diagnosing and designing treatments for patients) might benefit from this same approach. Likewise, any situation where a student would benefit from taking time to more deeply process an argument that they’ve already seen would be appropriate for a mastery exam.
As I mentioned above, the goal isn’t to test a student’s creativity, since that’s not something that fits well in a timed environment – so I would avoid mastery exams when creativity or originality is the standard that you care most about. In addition, I wouldn’t use mastery exams with content that is more procedural or rote, since that could lend itself to memorization.
In the end
Mastery exams fit a very specific niche, but if you’re in a similar situation, you might want to give them a try. If you do, please let me know how they work for you!
Mastery exams are yet another idea inspired by one of my undergraduate mentors, who I wrote about in Drawing on past experiences.
When I decided to use mastery exams, one of the key things I realized is that I don’t want to try to proctor creativity. That is, I’m not asking students to perform creative and original math in a timed setting – I just want to know if they have thorough understanding of existing ideas. This is one reason why I’m happy to test their knowledge of previously-seen proofs.
In the linked example I say that I might change the numbers in one of the questions, but I rarely do unless the student is specifically struggling with something numerical. The numbers aren’t the essence of that proof.
You can find some details of how final grades work in My go-to alternative grading templates.
Robert suggested that I could proctor the exam over Zoom, which is totally doable – and also not something that any student has asked about (nor that I thought about offering). I’ll keep that in my back pocket for future semesters! Generally I have a high degree of trust in my students, and I’m more than happy to be flexible about how they take the exams.
For those who are familiar, getting quantifiers in the correct order is the most common example of a serious logic error that students might think is just a minor writing error. Is “For all epsilon, there exists an N such that…” the same as “There exists an N such that for all epsilon, …”? That’s a discussion I have quite often!