When reading about "lazy graders," I immediately thought about how much I dislike the use of online homework paired with auto-grading software (e.g. ALEKS). And I congratulated myself for not being one of "those teachers."
Then I read your last paragraph about not being quick to jump to conclusions.
So now I'm wondering if there is a place for auto-grading software. It certainly does provide quick feedback. The big drawbacks I see are how it can encourage mimicking procedures (as opposed to learning), and how classes can slip into using the software as the only form of feedback.
Do you know of any instructors in the alternative grading community that use auto-grading software? Do you have any principles regarding its use?
I use autograded online homework as well, WeBWorK not ALEKS. I wrote about the role it plays in my current class here: https://rtalbert.org/building-a-specifications-grading-course-part-2/ Basically, it's practice for students -- it's for them, not for me, and they are expected to keep current with practice as part of the general expectation of "being part of the learning community" . I answer questions as students ask them. Otherwise I don't really touch it.
We have enough other assessments in the class that hit the top 2/3 of Bloom's Taxonomy that I think it's safe to say students don't get the wrong impression in the class from the practice sets.
I do! I use it as a way to provide practice with basic skills, and to unlock reassessments. So, say a student attempts Standard X on a quiz and earns "Not Yet". They can reassess during an office hour, but to unlock that opportunity they must first try some autograded practice problems also covering Standard X. This also tends to bring up useful questions during the office hour! So not exactly what you mean -- its focus isn't feedback -- but very useful!
I've definitely heard of others who use autograding more vigorously than I do, perhaps they'll chime in!
(This is the kind of reflection about grading that I love! There's not just one way, and it's great to think about how we can use ALL of the tools at hand to support students.)
Feb 13, 2023·edited Feb 13, 2023Liked by David Clark
This comment is not from a myth believer as I am using semi-mastery-grading this semester. I want to have a healthy discussion in a polarized world.
"Although in this context, the same problem arises in traditional grading, when instructors give a cumulative final exam or offer a "practice test" as part of a review day."
Practice tests do not need to have the same problems as those given in the exam. Cumulative final exams should not either.
Final exams have questions that mostly synergize various topics, and there is a high interleaving need on the part of students as the question is not identified as a particular standard.
SBG misses the component of a student having to identify which topic the question belongs to, especially when final exams or cumulative projects are not given.
Regarding the issue with giving the same problems, it is an issue if say, just limits of integration are changed for an integral. Some SBG advocates let the question be taken home if they still fail.
There is an equity issue as well - who can take it more times and afford to meet out of class - the person taking a light load of courses and not working outside of school.
SBG success depends on some guardrails and flexibility like anything else in life! As an example, many of us have witnessed how the flipped learning purity has changed over the years. Most include some minilectures, acknowledge that some topics need more addressing than others by the instructor, and that flipped learning resources can be made available for non-flipped sections of the class to witness the narrowing of the gap between the cognitive performance of students.
Hi Autar, and thank you for these thoughts! I agree with you broadly and thought some responses might help clarify:
- My point about practice tests or review days is that the issues are the same between alternative grading and traditional grading -- in each, instructors might, or might not, ask "too similar" questions. That's important, and worth discussion, but I think it's a red herring to claim that it's a particular issue in alternative grading.
- Having students identify what topic a question belongs to: I would say that this is actually a strength of SBG! That's because, when aligning questions with standards, the instructor must think carefully about their purpose: Do you want students to demonstrate *use* of a particular strategy, or to identify the strategy itself (and then perhaps use it)? Those are different skills, and in an SBG class I would say that you should have separate standards for each of them. For example, "I can use integration by parts to evaluate an integral" tests students on a specific method and tells them what to use. That's valuable and worth assessing. "I can identify an appropriate integration method and justify my choice" asks students to identify the strategy. That's also valuable (and you could also assign students credit for the first standard if they successfully use the method after identifying it). I'd say that the second one rises higher in conceptual understanding, and so I'd make it a part of a higher grade bundle. I see SBG as making these sorts of things clearer to the instructor and to students -- but like with practice tests and reassessments, it's something that instructors can do well or poorly.
- I absolutely agree that the amount of time required, especially for reassessments, is an equity issue. Adding reassessments on top of an already full assessment load only benefits students who have the ability to spare extra time. My usual advice is to deliberately account for reassessments as part of the class's normal workload, possibly cutting out or reducing other assessments to make room. (I say much more about it here: https://gradingforgrowth.com/p/grading-for-equity-with-grading-for-e1f ). Again, something that can be done well or poorly, and an instructor needs to watch out!
- Finally, I absolutely agree about the benefits of guardrails and flexibility. :) There's not one right way to do it -- where "it" can be alternative grading, flipped learning, or just about anything else in education. :)
When reading about "lazy graders," I immediately thought about how much I dislike the use of online homework paired with auto-grading software (e.g. ALEKS). And I congratulated myself for not being one of "those teachers."
Then I read your last paragraph about not being quick to jump to conclusions.
So now I'm wondering if there is a place for auto-grading software. It certainly does provide quick feedback. The big drawbacks I see are how it can encourage mimicking procedures (as opposed to learning), and how classes can slip into using the software as the only form of feedback.
Do you know of any instructors in the alternative grading community that use auto-grading software? Do you have any principles regarding its use?
Thanks for the great post!
I use autograded online homework as well, WeBWorK not ALEKS. I wrote about the role it plays in my current class here: https://rtalbert.org/building-a-specifications-grading-course-part-2/ Basically, it's practice for students -- it's for them, not for me, and they are expected to keep current with practice as part of the general expectation of "being part of the learning community" . I answer questions as students ask them. Otherwise I don't really touch it.
We have enough other assessments in the class that hit the top 2/3 of Bloom's Taxonomy that I think it's safe to say students don't get the wrong impression in the class from the practice sets.
I do! I use it as a way to provide practice with basic skills, and to unlock reassessments. So, say a student attempts Standard X on a quiz and earns "Not Yet". They can reassess during an office hour, but to unlock that opportunity they must first try some autograded practice problems also covering Standard X. This also tends to bring up useful questions during the office hour! So not exactly what you mean -- its focus isn't feedback -- but very useful!
I've definitely heard of others who use autograding more vigorously than I do, perhaps they'll chime in!
(This is the kind of reflection about grading that I love! There's not just one way, and it's great to think about how we can use ALL of the tools at hand to support students.)
This comment is not from a myth believer as I am using semi-mastery-grading this semester. I want to have a healthy discussion in a polarized world.
"Although in this context, the same problem arises in traditional grading, when instructors give a cumulative final exam or offer a "practice test" as part of a review day."
Practice tests do not need to have the same problems as those given in the exam. Cumulative final exams should not either.
Final exams have questions that mostly synergize various topics, and there is a high interleaving need on the part of students as the question is not identified as a particular standard.
SBG misses the component of a student having to identify which topic the question belongs to, especially when final exams or cumulative projects are not given.
Regarding the issue with giving the same problems, it is an issue if say, just limits of integration are changed for an integral. Some SBG advocates let the question be taken home if they still fail.
There is an equity issue as well - who can take it more times and afford to meet out of class - the person taking a light load of courses and not working outside of school.
SBG success depends on some guardrails and flexibility like anything else in life! As an example, many of us have witnessed how the flipped learning purity has changed over the years. Most include some minilectures, acknowledge that some topics need more addressing than others by the instructor, and that flipped learning resources can be made available for non-flipped sections of the class to witness the narrowing of the gap between the cognitive performance of students.
Hi Autar, and thank you for these thoughts! I agree with you broadly and thought some responses might help clarify:
- My point about practice tests or review days is that the issues are the same between alternative grading and traditional grading -- in each, instructors might, or might not, ask "too similar" questions. That's important, and worth discussion, but I think it's a red herring to claim that it's a particular issue in alternative grading.
- Having students identify what topic a question belongs to: I would say that this is actually a strength of SBG! That's because, when aligning questions with standards, the instructor must think carefully about their purpose: Do you want students to demonstrate *use* of a particular strategy, or to identify the strategy itself (and then perhaps use it)? Those are different skills, and in an SBG class I would say that you should have separate standards for each of them. For example, "I can use integration by parts to evaluate an integral" tests students on a specific method and tells them what to use. That's valuable and worth assessing. "I can identify an appropriate integration method and justify my choice" asks students to identify the strategy. That's also valuable (and you could also assign students credit for the first standard if they successfully use the method after identifying it). I'd say that the second one rises higher in conceptual understanding, and so I'd make it a part of a higher grade bundle. I see SBG as making these sorts of things clearer to the instructor and to students -- but like with practice tests and reassessments, it's something that instructors can do well or poorly.
Oddly, I don't think I've written about the idea of "strategy standards" on this blog, but it gets some time in our book. I've also written in some detail here: https://docs.google.com/document/d/1oWBOxRhU3kqizhJcbSYFc-33p_HyftA4FYh4zI6-ZUA/edit?usp=sharing
- I absolutely agree that the amount of time required, especially for reassessments, is an equity issue. Adding reassessments on top of an already full assessment load only benefits students who have the ability to spare extra time. My usual advice is to deliberately account for reassessments as part of the class's normal workload, possibly cutting out or reducing other assessments to make room. (I say much more about it here: https://gradingforgrowth.com/p/grading-for-equity-with-grading-for-e1f ). Again, something that can be done well or poorly, and an instructor needs to watch out!
- Finally, I absolutely agree about the benefits of guardrails and flexibility. :) There's not one right way to do it -- where "it" can be alternative grading, flipped learning, or just about anything else in education. :)