Case Study: Dustin Locke's "Levels System" in a writing-intensive Philosophy class
This week, we’re back with another case study from our upcoming book, Grading for Growth. Today, we feature a writing-intensive Philosophy class that uses a unique variation on specifications grading.
Specifications work in many settings. This is part of its beauty: The idea of providing clear specifications for “success” is beneficial on any kind of assessment. Specs differ from standards mainly in that specifications describe a cohesive set of requirements for the entire assignment, while standards describe discrete skills that might be met — or not! — separately.
As always, please leave us feedback in the comments. Do you find it helpful? Is there something more you’d like to know? What could make it better?
Dustin Locke is an associate professor in the Philosophy department at Claremont McKenna college, a small private liberal arts college in California. Locke uses a variation on Specifications Grading that he calls the “Levels system” in his introductory philosophy course1. Locke notes that this system has been used by colleagues across many types of institutions.2
The assignments in Locke’s introductory philosophy courses are primarily essays. These are focused on identifying, analyzing, comparing, and critiquing arguments made by others. Locke defines four “levels” for essays (called “level 1” through “level 4”), corresponding roughly to four levels of Bloom’s taxonomy. The description of what is required in an essay constitutes the specifications for that level.
Throughout the semester, each student will write up to seven short essays on a regular schedule, beginning with a level 1 essay and “graduating” to level 2 only when they have successfully met the specifications for level 1. Then they write level 2 essays until successfully meeting the level 2 specifications, and then graduate to level 3, and so on.
Every student begins by writing a “level 1” essay, which is a fairly direct assignment focused on identifying logical arguments and placing them into a form studied in class. The specifications include several items, but the critical ones are:
Find a medium-sized argument in a passage from a reading assigned for one of your non-philosophy courses and put it into ‘standard form’ [a skill which has been studied and practiced in class].
By ‘medium-sized’, I mean that once in standard form, the argument contains between 5 and 10 numbered steps.
Your standard form should be clear, concise, accurate, and easy to follow.
You might notice that these blur the line between “instructions” and “specifications.” In general, instructions are directions, and specifications describe what successful completion of the directions looks like. These can often contain the same information, phrased slightly differently.
This level 1 essay is graded “Complete” or “Not Yet” based on the specifications. As always, the standard for “Complete” is not perfection, but whether the student’s work demonstrates sufficient proficiency with the goals of the level. Locke describes “sufficient proficiency” as meaning that any remaining issues are “unlikely to seriously hinder their efforts to develop the additional skills required” to succeed at the next level.
If a student earns “Complete,” then their next regularly scheduled essay will be done at level 2, which has a more advanced set of specifications. If they instead earn “Not Yet,” then their next essay will be done again at level 1 but must be based on a different reading. That is, the next attempt at a level 1 essay is indeed a wholly new essay on a new topic, but with the same instructions and specifications for successful completion.
This continues, with students writing new essays at the same level until they earn “Complete,” and then writing the next essay at the next higher level. There are four levels, with level 4 representing an advanced explanation and criticism of an argument. There are seven scheduled essays, and so students essentially have seven attempts to work their way up to the highest level they can reach.
It may sound like a grading nightmare: Every student, doing something different? Practically, each time an essay is due, there are at most four different types of essays to grade (corresponding to the four levels). Almost all students are focused in one or two levels.
A key feature of this “levels system” is that the levels build on each other: By successfully completing an earlier level, students have practiced a skill that is important for success at the next level.
Here’s a sample of Locke’s specifications for a level 4 assignment. The specifications begin with some clear instructions:
Choose one particular argument from the readings assigned for this class since the last attempt was due and clearly and accurately explain and criticize that argument in a short paper.
Next follow “mechanical” specifications with a word count, style, formatting, and citation formats. For example:
Your paper must be no longer than 1000 words.
(We leave out the other specifications of this type since they are quite familiar in most essay assignments.)
Next we reach the central specifications, describing the key qualities of a successful level 4 essay:
Your criticism must be focused: you are making exactly one criticism of exactly one premise or inference.
Your criticism must be original: you may draw on ideas others have put forwards, but ultimately your criticism must contain some central idea that you came up with.
Your criticism must be plausible: your criticism does not need to persuade me, but your criticism has to be plausible enough to be taken seriously. (For example, if your criticism simply assumes that ‘The Earth is flat’, or some other outlandish idea, it is not plausible enough to be taken seriously.)
At the end of your paper, discuss at least one possible objection to your criticism and respond to it. This will give you the opportunity to clarify and defend your argument.
At the beginning of your paper, include an introductory paragraph (or two) that briefly introduces the topic, explains what your paper is going to be about (including what criticism you will make), and explains how your paper will proceed.
Again, these mix “instructions” with specifications. The line between instructions and specifications is fine. Some instructors, such as Kay C Dee, write instructions in assignments and then include “All instructions have been followed” as a specification. Others, such as Locke, include the instructions as part of the specifications. The key, however, is that the instructor must be able to judge if a student’s work successfully meets the specifications provided. So, like standards, these specifications must be written in a way that is clear enough to assess.
Specifications are written so that students have a choice of topics for each essay: generally, anything discussed in class since the previous essay was due. Thus each essay has a new topic, but if a student has not yet succeeded at a particular level, the essay must still be written using the same specifications as a student’s previous essay. This is Locke’s main form of reassessment: Students receive detailed feedback on their previous essay, written in terms of the specifications. They use the feedback to improve their work on the next one. However, this new attempt is not a revision of a previous essay. Locke says, to explain this choice:
A student who submits an excellent paper after revising it in light of instructor feedback may have simply mastered the skill of revising in light of instructor feedback.
Phrased differently,
...while revisions primarily improve the paper, retries [new attempts] primarily improve the writer.
This view of revisions may or may not work for other instructors, depending on their assignments and class contexts.
Locke also provides a type of pre-assessment feedback: Peer feedback on drafts of papers. This happens during a “workshop day,” which is a class held shortly before each essay is due. Students trade drafts of their essays and give each other feedback. They are also able to ask Locke questions during the workshop. Locke provides suggestions, comments, and clarifications as needed, in particular reminders about the specifications for the relevant levels. At this point Locke checks only that the drafts are completed, which counts in a separate part of the final grade.
Locke does also require revisions of previous essays, which are counted in a completion-only portion of the final grade that we will describe below.
A student’s final grade has two parts. The “essay” portion of the grade is based on the highest level that the student has successfully completed, with modifications based on the quality of their last attempt. The basic requirements for each grade are straightforward:
A: A successful level 4 essay.
B: A successful level 3 essay.
C: A successful level 2 essay.
D: A successful level 1 essay.
However, on the seventh and final attempt, Locke makes slightly more detailed notes about a student’s progress. Essentially, the final attempt is graded with marks of “complete”, “almost”, “some progress”, or “not yet”, and these are used to determine +/- grades. For example, if a student successfully completes a level 2 essay, their final grade is at least a C. But this would become a C+ if the student made a solid but incomplete attempt at a level 3 essay (“some progress”), or a B- if the student nearly passed the level 3 essay (“almost”).
The grading system could end here, focused only on the essays. To encourage good habits of studying and revision, Locke includes a separate “quizzes and homework” portion of the final grade. This contains items that are worth one “point” each when completed successfully. For example, revision attempts of graded essays are included in this part of the grade. These are not fully regraded at their corresponding level, rather, they are marked as a “reasonable revision” or not. Another example: single-question quizzes covering the assigned reading. These are graded based on correctness, but again, they are worth exactly one point: essentially “satisfactory” or “not satisfactory.” Locke allows 3 “free” misses, that is, the quizzes and homework grade is calculated as if 3 of the points were not required.
“Levels” and “quizzes and homeworks” count as 50% each, and the letter grades earned in each part are averaged together. For example, the “average” of an A and C is a B. Locke provides rules for how this “averaging” works in his syllabus. This could also be done with a grade table, as we have seen in other case studies.
Locke has seen many benefits from the Levels system. For example, he points out that most of the assignments are essentially “ungraded” and focus entirely on feedback, which in turn removes the extrinsic motivation provided by grades and instead helps students focus on learning and growth. He gives each essay only one brief mark (Complete or Not Yet):
Since this mark makes no further assessment of the quality of the student’s work, it very likely has a different psychological impact on the student than does a traditional grade. To put it roughly, where a traditional paper grade looks backwards, and any accompanying comments tend to be seen as justification for that grade, a mark of ‘complete’/‘not yet’ on a level attempt looks forwards, and any accompanying comments tend to be seen as guidance for future attempts.
Locke also finds the system easy to manage: “I have found it easier to administer the Levels System than it was to administer … more traditional writing assignments.” Student writing consistently improves, and the total amount of writing (and hence grading) is relatively small compared to a more traditional philosophy course with a few longer essays. By having seven regularly scheduled essays, the workload both for Locke and for his students is predictable and moderate.
This Levels approach works best when a course naturally includes assessments in a “hierarchy,” each a prerequisite for the next one in increasing levels of complexity. Locke compares this to two things: First, the traditional system of prerequisite courses in higher education, in which it is assumed that students have learned prerequisite content before continuing to a more advanced course. The Levels system attempts to create a similar system within a single course. Second, video games: Players can only advance to the next level of a video game when they have reached a sufficient level of proficiency with the skills required at the previous level. If they aren’t there yet, they can try again, having learned useful lessons along the way.
Locke’s “Levels system” demonstrates a way to use alternative grading in a writing-intensive course. It’s also a reminder that alternative grading systems can be used across all disciplines — not only the STEM examples we’ve seen so far.
If you’d like to see our other case studies (so far!), here are some quick links:
Thanks for reading this case study! Do you find it helpful? Was there something you especially liked? What would you add or remove? Let us know in the comments!
Locke is actually ambivalent about whether the Levels System meets Linda Nilson’s definition of Specifications Grading or not. We’re not interested in being definition police (see our argument in Finding Common Ground). Locke writes clear descriptions of what a successful essay must do, grades Pass/Fail based on those specifications, and uses the results to determine final grades. We think that gets at the essence of Specifications Grading.
Locke was kind enough to share a submitted journal article in which he describes the Levels System; much of this case study is based on that article: Locke, D. (2022) “The Levels System: An Application of Mastery Learning to the Teaching of Philosophical Writing.” Submitted.