Alternative grading in a six-week asynchronous course

Lots of constraints and lots of questions

Photo by Andy Beales on Unsplash

The last time I taught a summer course, I swore to myself that I would never do it again. Not because I didn’t enjoy it – I did! – but because I enjoyed not teaching more, and I decided I value the personal time in the summer more. But then my middle child started college last fall, and, well, the idea of summer teaching (or to be honest, the income) started sounding pretty good again.

That course begins today. It is the Discrete Structures course I have written about many times before, but with a big difference, as I mentioned in my last post about the course: This one is an asynchronous online version of the course, compressed into a six-week schedule. These constraints introduce several challenges for implementing an alternative grading system. As the course begins, I wanted to share how I’m thinking about how to “grade for growth” in a learning environment like this.

The parameters

This course is the same one I teach frequently: the first of a two-semester sequence on the foundational mathematics for computer science. Here is the syllabus from my Winter semester course, done in the usual in-person, three-day-per-week, 15-week semester format¹.

The three major constraining factors on this summer’s version of the course each pose distinct challenges.

• The online setting: Online courses are challenging from an assessment and grading standpoint. The last time I wrote about this course, I explained how I was mitigating the risk of academic misconduct posed by generative AI tools by moving all the major assessments to in-class, on-paper exams. I also noted there that while reasonably successful in a face-to-face setting, this strategy was out the window for an online asynchronous course². Any time the student engages with the class, generative AI tools will be right there as well.

• The asynchronous modality: There are many potential issues with asynchronous courses, in addition to the risks of academic misconduct and AI; others have pointed these out in detail and there is a lot of truth to those criticisms. One of the biggest is that asynchronous courses require students to be highly self-motivated and skilled at managing time, attention, and information — all necessary components to engaging with a feedback loop. While many students possess those skills and put them to good use, for others this is their weakest link, and it’s the place where the modality puts the greatest pressure.

• The compressed time frame: This course runs in our “Spring” term, the first half of the summer from May 5 through June 16. That’s six weeks, which is 2.5 times shorter than a regular semester. A lot of scary math can be done with that scaling factor. For example, whereas a three-credit course in a regular semester would be a roughly 9-hour-per-week time commitment for students (three hours for the class meetings plus six hours for outside work), that figure scales up to about 20-22 hours per week on this schedule. It doesn’t give students a lot of time to iterate through feedback loops, nor does it give me much time to give that feedback once work is turned in.

My overall strategy

When I’ve mentioned all these aspects of my Spring class to others, the response has been Good luck with that. To be honest, I have been baffled by the design of this course ever since I signed up to teach it, and I did not come to a final decision about how it would work until late last week. Time will tell if what I have planned actually works. But here is my overall strategy.

1. First and foremost keep it simple. I am not attempting to innovate with this course. I had a lot of big ideas at first (make the whole thing ungraded; convert it all to project-based learning; etc.) but with everything else being so different, lots of headline-making changes to the structure of the course did not seem to be in the best interests of anybody. My goal is, insofar as possible, to try to make the assessment and grading and other aspects of the course look and feel like something familiar, and don’t overcomplicate it.

2. In terms of assessment, adopt an approach of numerous small items due frequently. In a six-week asynchronous course, any hesitation to engage and any instance of procrastination is almost immediately deadly. Stuff is happening at 2.5 the pace of a regular semester class and there is no “class” to make you engage with it. So I really want students to be turning in something, often something easy and small, every day.

3. Use a lot of small assessments that are easy to grade to surface misconceptions and generate feedback. Then use a smaller number of larger assessments, which use the formative assessments, as the primary basis for the course grade.

4. Trust students; but also verify. If I had chosen, I could have designed the whole course around mitigating threats to academic integrity from AI and other sources. I also could have chosen to ignore all those threats and “just trust students”. In the end I chose a middle path. I am not going to create a police state where AI’s are lurking in every assignment, and my job is to root them out. But I am going to take reasonable measures to ensure that students' grades have reliable construct validity, that is, they are speaking truthfully about student learning in the end. More on that below.

Weekly cadence and assessments

The assessment/grading scheme in this course is best understood in the context of the weekly cadence I’ll be using:

• Sunday: There’s a course announcement with an overview, assignments and due dates, and actual announcements for the week. This is also when I open up the folder of content for the week. Students are free to start, even finish assignments on Sunday, but nothing’s due.

• Monday: Students complete a series of videos and readings to introduce the concepts for the week. Then they complete two assignments: A Concept Check which is a short objective quiz over main ideas from the new material, and a Video Reflection where they are asked to explain one of the new concepts in their own words on a video. Both are due by 11:59pm that day, and are graded on completeness and effort only. The Concept Check provides some immediate feedback on initial understanding, and the video gives students a chance to process the new stuff further.

• Tuesday: Part of the material released on Sunday is a list of practice exercises split into “Level 1” (basic) and “Level 2” advanced. Here is the first one. Each student is assigned one of the Level 1 practice problems, works it out, and puts their work on a FigJam board that everyone can see and comment on. Their work is due on the board by 11:59pm and is graded on completeness and effort. Also on Tuesdays, I will be posting video walkthroughs of the Concept Check from Monday, as well as leaving feedback on student work on the FigJam.

• Wednesday: I’ll post more video of worked practice exercises (including some of the Level 1 practice plus a few others) and responses to any student questions that have arisen by this point. Nothing is due though.

• Thursday: Students complete a second Concept Check for the week, to get a second read on their understanding of the material, and graded on completeness and effort. Also, students will have been working on the “Level 2” practice — 2-3 problems a little further up the Bloom’s Taxonomy pyramid — and they will write up their work and submit it on Blackboard on Thursdays, and this is graded Success or Retry based on standards that are spelled out in this document.

• Friday: I’ll post a video summarizing the main points from the week and any observations and coaching notes from student work through the week as well as a walkthrough of the second Concept Check. Students will post a second video reflection to talk about what they found interesting, what they found difficult, and how their practice is going. This is just a weekly check-in and is graded on completeness and effort.

• Saturday: This is set aside to complete two other larger items that are assigned at the beginning of the week and are worked on throughout the week. First there is a Problem Set with 2-3 problems that require more careful analysis and communication. Second there are Checkpoints which are quizzes that gauge mastery of the 15 Learning Targets in the course. The Problem Sets are written up and turned in on Blackboard. The Checkpoints are done directly on Blackboard, with Respondus Lockdown Browser use to proctor. Both are due by 11:59pm but can be completed at any time during the week. The Problem Sets and Checkpoints are graded Success or Retry based on standards that are spelled out in this document.

You can see there’s quite a bit of feedback built into this cadence. The Success/Retry assessments, in addition, can be revised and resubmitted: Students can submit revisions of up to two Problem Sets per week and revisions of each Level 2 Practice item once, and Checkpoints are refreshed every Sunday with new versions of all the old ones available.

Three questions you (and I) probably have at this point

Is this appropriate? As I write this up for an article, it seems like an insane amount of work. But you have to keep in mind that the course is compressed from 15 weeks into 6. If you zoom out on the above cadence, putting the same workflow into a 15 week class, it sounds more normal: Problem sets due every other week, one video and one concept check per week, and so on. Yes, it is a very busy schedule. But it comes with the territory if you are taking a 6-week asynchronous course — these are incredibly dense and fast paced, and there’s just not much you can do to make it otherwise.

Is this sustainable? For me, I think so. There are currently 10 students in the class. Each week I will be responsible for giving Success/Retry grades 10 sets of problems, 10 sets of practice questions, and 10 sets of Checkpoints. The rest is either auto-graded by the LMS or graded on completeness and effort. On average this is not much more work than I normally put into a 3-credit course although the time factor will mean I’ll be prepping and grading quite frequently. (I’m installing a time box of 1-4pm Monday through Friday starting today and ending on June 16 for this.)

For students? It depends. I stand by the expectation of a 20-22 hour per week time commitment. Although there are items due almost every day, they are often low-stakes formative items graded on completeness and effort, or items that can be spread out over 2-3 days. But make no mistake, it’s a very brisk schedule and will be a hard 6-week sprint for even the most capable learners. If there are a lot of competing claims on time and attention, or if a student struggles to manage time and attention, or struggles to learn from feedback, things could get ugly. But, again, this is the nature of the beast in a 6-week asynchronous learning environment.

How flexible are you willing to be on deadlines? The answer here is not very. That’s not my normal approach but, once again, due to the compressed time scale, student work has to happen on a tight schedule with hard deadlines or else the feedback they get has already passed its sell-by date by the time students receive it. For example, I really insist that students watch an hour or so of video and complete the first Concept Check and video reflection on Monday, and not later. In past asynchronous courses I moved this Monday deadline to Tuesday based on student requests, but it did not help them — all that happened was the students who needed the feedback the most, procrastinated on the assignments until Tuesday, didn’t get feedback on their work until mid-week, and by that point it was too late.

What could go wrong?

Lots:

• There could be technology fails. For example, I mentioned I am using Respondus Lockdown Browser to administer Checkpoints. I swore to myself in the past that I would never use such a tool, but I was convinced otherwise by a colleague in my department who teaches Calculus asynchronously and finds it useful. In my personal tests, it seems less terrible than I expected. But I can imagine a lot of ways it could fail at the worst possible times. And this is just one item of technology in a heavily tech-infused course.

• Students will have significant competing claims on attention. I already know that some of my students are taking multiple classes or have 20+ hour per week job requirements. I am sincerely hoping, and trying to make very clear at the outset, that while it’s none of my business what you do with your time, 20-22 hours per week of it belongs to this course, and the more a person has to divide their attention the harder it will be to succeed³.

• Procrastination. I’ve taught this course in a 6-week format in the past and my students have said they actually like the pacing, because it is almost literally impossible to procrastinate when things are happening that fast. They have a point. The flip side of that point, is that if you procrastinate or decide to take a few days off, you will likely find yourself in an irreversibly bad situation almost immediately. In alternatively graded courses we believe in “eventually” — as long as students eventually demonstrate mastery, it’s OK if they need extra time or attempts to get there. But in a six-week asynchronous course though, “eventually” is measured in terms of days, not weeks.

• The professor falls behind. I’m a point of failure too. I pride myself on staying 100% on top of things in the grading department most of the time, but there are a lot of unknowns here, not to mention a lot of work, and I can’t take days off either. These feedback loops will need constant feeding over the next six weeks.

• AI and academic misconduct. Here’s the big elephant in the room. The doomsday scenario is that most of my students end up cheating with AI on assignments most of the time. The reality is that there is not much I can do to definitively prevent this from happening apart from creating a police state where everything is monitored. Instead, I am trusting in the strategy of taking reasonable steps to mitigate the risks (for example by using Respondus), put an emphasis on frequent low-stakes formative assessments, have a robust reattempt/revision policy on the weightier assessments, and most importantly just try to build trust and a friendly environment from the beginning, as much as possible given the constraints of asynchronous learning.

The end of the beginning

This course officially starts today. There have already been a few technical glitches I’ve needed to iron out but otherwise, the ship did not sink the moment it entered the water, so that’s good. Will the course be a successful learning experience for all students? Will there be something like parity in the learning experience for this course versus the in-person version? Will I learn some useful lessons about online learning and alternative grading? Will I go insane with this workload? For the answers to all these questions, check back in late June or early July. And wish me and my students luck in the meantime.

1. At my institution, the regular semesters are “Fall” and “Winter” — not “Spring” because that doesn’t really accurately describe the climate in Michigan at that time of the year.

2. It would be difficult to manage even in a synchronous online course, because of the need for proctoring.

3. Here is my #1 criticism of asynchronous courses: They are often marketed as being for “busy working adults” who have families, hobbies, and 40+ hour per week jobs. Many times, these folks are well-served by these courses. But other times, we in higher ed give the impression, often deliberately, that asynchronous courses can be done as an overlay on top of all the rest and will fit right into whatever schedule a person may have. This is simply not true. The only way it might be true, is if the academic quality of the course is dialed down to a level where it’s no more challenging than a binge-watch TV series or a casual video game. There are some serious ethical questions about the product we are selling to these busy working adults and the way we are marketing it that demands closer inspection.