What does it mean to meet a standard?

Photo by Jungwoo Hong on Unsplash

I’m taking a short break from case studies to follow up on an important idea. Watch for more case studies in a few weeks!

Last week, Robert wrote about a key pillar for alternative assessments: marks should indicate progress towards meeting standards or specifications¹ (if you use marks at all—which isn’t necessary, but can be convenient).

This is fundamentally a binary decision. Even if you use a multi-level rubric like EMRN, there are still two main options: The work meets the standard, or it doesn’t (yet!). Other levels of the rubric simply communicate details of this progress.

This kind of decision can feel quite different from adding or deducting points, and it’s one of the places where new alternative graders can get hung up. It’s also something that will become quicker and easier with experience—so much so, that many instructors we talk with cite this kind of “binary grading” as a major time saver!

So, what does “meeting a standard” mean? That’s our topic for this week.

---

Before we begin, I want you—yes you!—to get some skin in this game. Here are a few common situations that you might encounter when grading student work. What do you think should happen in each of them?

These are general: Imagine a corresponding situation that you might encounter in your classes.

1. You’re assessing a standard that requires using a specific method (e.g. “I can use factoring to find the zeroes of a quadratic formula”). A student solves the problem correctly, using a different method. Do they meet the standard?

2. You’re assessing a standard about using a process. A student’s technical work is probably correct, but their writing is so disorganized that you have a hard time following it. Do they meet the standard?

3. You’re assessing a standard covering a computational technique. A student mis-copies one number in the problem. They solve this new problem correctly. Do they meet the standard?

4. You’re assessing specifications that asks students to explain connections among a set of ideas. Everything the student writes is clear and correct, but they completely leave out one key idea. Do they meet the specs?

5. You’ve asked students three questions that all address the same standard. One student does excellent work on the first two problems, but shows a serious misconception partway through the third. Do they meet the standard?

Take some time to think through each of these. We’ll return to these at the end of this post.

---

Here’s how I think about “meeting a standard” on an assessment:

A student’s work meets a standard if it consistently demonstrates relevant evidence of learning without important errors.

Let’s pull that sentence apart, in reverse order:

Important errors matter. An important error is one that’s central to the standard you’re assessing. Only important errors, not irrelevant ones, should lead to a “not yet” mark. For example, if you’re assessing “I can identify and explain the immediate causes of the American Civil War” then the causes and explanation are what’s important. If a student cites incorrect causes, or shows a misunderstanding of the reasons behind them, that’s an important error. However, spelling and grammar are not important for this standard—as long as the student’s ideas are clear. On the other hand, if you’re assessing specifications for writing a professional-quality essay about the causes of the American Civil War, then spelling, grammar, and attention to detail are absolutely important issues. It matters what your standards (or specs) do or don’t include.

Important is not the same as “big” or “small”: In a calculation, a small typo like leaving off a key negative sign may show an important misunderstanding of the underlying ideas. This cuts both ways: A student who makes a copy error might still show great understanding of the underlying ideas.²

This brings up an important misconception: Never require perfection. Perfection is dangerous: It treats everything like a key issue, down to the finest detail. Perfection makes students less clear on what matters, and what they need to do in order to learn. Minor imperfections that would result in a ½ or 1 point deduction in traditional grading (such as typos or minor copy errors) are usually irrelevant for meeting a standard.

Identifying what matters—that is, which issues are central to a standard, and which aren’t—isn’t always easy. It is an excellent exercise to write down examples of what’s important in a standard, what isn’t, and which common types of errors matter for meeting that standard. That process is valuable for you, but it’s also valuable for students—share it!

Specifications can help make “what matters” clearer. Specs are essentially a longer, more detailed form of standards that spell out in detail what is important and what isn’t. You can see some examples in Kay C. Dee’s Specifications Grading case study.

Work must be relevant to the standard. This means that the student’s work is not just correct, but is related to that specific standard. The standard you’re assessing matters. Students sometimes answer a question in a way that avoids the key issue—for example, applying a different method than you asked for, or using technology to do the main work for them. When these things happen, the work isn’t relevant, even if the result is correct. If you’re assessing “I can use factoring to find the zeros of a quadratic equation,” then a student who correctly finds the zeros using a different method (such as the quadratic formula, or a calculator) has not met that standard even if their results and intermediate work are correct. They didn’t demonstrate that they have met that standard.

The same work might be relevant for a different standard, such as “I can identify and use an appropriate method to find the zeros of a quadratic equation” (notice that the method isn’t specified). In feedback, I often point out that students would earn credit for the other standard, and explain why they didn’t earn credit for the original standard. Others use a special mark for correct work that misses the standard, e.g. “Insufficient evidence,” to differentiate it from something with important errors.

This also illustrates why it is important for standards to be written at the right level of specificity—you probably do want to assess whether students can choose an appropriate method to solve a problem, in addition to practicing with specific methods. So be sure to write a standard for each situation.

Students must demonstrate consistent evidence of learning. Standards describe actions, and so a student must positively demonstrate that they can do that action. If a student doesn’t provide evidence, then they haven’t met the standard. For example, if you are assessing a process, and a student leaves out a critical step, then the work does not demonstrate enough to meet the standard (you could call this “thoroughness”). Similarly, if work is so unclear or filled with small errors that you can’t be sure what a student actually did, then they haven’t positively demonstrated appropriate evidence.

That demonstration should also be consistent throughout the assessment. It’s common to ask multiple questions in a single assessment that all address the same standard. If a student’s work meets the standard on one question, but not on another, then they haven’t shown consistent evidence of learning at that time. They’re clearly getting there, but not quite enough.³

Sometimes instructors get caught up in the idea that they have to give students credit for meeting a standard “by default” unless the student’s work clearly shows a major error. Inconsistent or missing evidence of learning are good reasons why a student’s work may not meet a standard. If your holistic reading of all of the student’s work indicates that they have missed something, or haven’t shown consistent evidence of learning, they haven’t met the standard. This is a major difference from traditional grading—there is no “partial credit.”

---

If it looks like I’m saying we should hold students to a high standard: I am! Not perfection, of course, but we want to hold students to a high standard that is informed by a genuine inquiry into what they have (or haven’t) demonstrated.

Because students have chances to reassess without penalty, you can be more willing to hold students to such a high standard. Unlike with points and partial credit, you’re not permanently hurting a student by assigning a low grade. Rather, you can say “you’re not there yet, and here’s some feedback on why—you can show me when you get there.” In the same spirit, be open to multiple forms of evidence: Working through a problem at a whiteboard during office hours might be a great demonstration of student understanding, even if it’s not what you expected.

You may not like my description of what it means to meet a standard—that’s fine! Many others have invented different descriptions that you might find helpful. For example, here is a question that you might ask yourself:

Will this student benefit from studying this topic again?⁴

If the answer is “yes,” then the student hasn’t met the standard (yet!). Other descriptions that might help include:

• Correct in all fundamentals.

• Thorough, relevant, and with at most minor errors unrelated to the standard.

• Any remaining difficulties the student has with this topic won’t seriously hinder them in future work.

• Demonstrates a solid enough understanding to re-teach the idea to themselves when needed in the future.

---

Finally, let’s return to the scenarios I posed at the top of this post. Here are my thoughts about each—yours may differ!

1. You’re assessing a standard that requires using a specific method (e.g. “I can use factoring to find the zeroes of a quadratic formula”). A student solves the problem correctly, using a different method. Do they meet the standard?
   David thinks: No, they didn’t provide relevant evidence for this standard, but they probably met a different standard. I would leave feedback explaining this and possibly give credit for the other standard.

2. You’re assessing a standard about using a process. A student’s technical work is probably correct, but their writing is so disorganized that you have a hard time following it. Do they meet the standard?
   David thinks: Yes, unless the writing is so hard to understand that you can’t tell if they actually demonstrated relevant work. Assess the standard, not something else!

3. You’re assessing a standard covering a computational technique. A student mis-copies one number in the problem. They solve this new problem correctly. Do they meet the standard?
   David thinks: Yes, as long as the error doesn’t significantly change the meaning of, or trivialize, the problem. If the new problem is essentially equivalent to the original, the student has still demonstrated relevant work.

4. You’re assessing specifications that asks students to explain connections among a set of ideas. Everything the student writes is clear and correct, but they completely leave out one key idea. Do they meet the specs?
   David thinks: Probably not. The work lacks a demonstration of something the specs require. This might be a good time for a low-stakes reassessment: “Needs revision; You can tell me how you’d fix it in an email or office hour.”

5. You’ve asked students three questions that all address the same standard. One students does excellent work on the first two problems, but shows a serious misconception partway through the third. Do they meet the standard?
   David thinks: No, this doesn’t consistently demonstrate the standard. For more explanation, see the note about asking multiple questions on a standard from Hubert Muchalski’s case study.

1. Everything I say here applies to specifications just as well as standards. To avoid writing “standards or specifications” over and over and over, for the rest of the post I’ll use the word “standards” to describe both.

2. I would say that a small error of this type meets the standard and earns full credit (e.g. “M” or “Satisfactory”). But others would argue that this is a good time to assign a mark like “Needs revision” and invite the student to a free office hour revision to clear up the details.

3. By “consistent,” I mean “within a single assessment.” Whether students should meet a standard on multiple occasions throughout the semester is a different question.

4. This comes from this wonderful article: J. B. Collins, A. H. Ramsay, J. Hart, K. A. Haymaker, A. M. Hoofnagle, M. Janssen, J. S. Kelly, A. T. Mohr & J. OShaughnessy (2019): Mastery-Based Testing in Undergraduate Mathematics Courses, PRIMUS, 29:5, 441-460, https://doi.org/10.1080/10511970.2018.1488317