Who's tried Ungrading in STEM courses?

[symbolipoint]

@vela, post #54 very well said (written)

 

[ZapperZ]

I'm curious if anyone has tried "ungrading" in their STEM courses and if so, how it worked out. The idea is to get away from the using points to determine a student's grade and use different types of assessment that better motivate students to learn.

https://www.jessestommel.com/why-i-dont-grade/

https://www.chemedx.org/blog/ungrading-what-it-and-why-should-we-use-it

My interest arises from my experiences since classes went remote because of the pandemic. Like many other instructors, I saw the mysterious increase in performance by many students on exams (as well as obvious signs of cheating in some cases). To reduce the incentive to cheat, I replaced most of these high-stakes assignments with low-stakes weekly problems, where students had to write up a solution where they had to identify the relevant physical concepts, explain their problem-solving strategy, and finally solve the problem. It wasn't enough to just write down a bunch of math, which they could easily find on Chegg or somewhere else on the internet; they actually had to articulate the reasoning involved. I have to say I was pleasantly surprised by the quality of the write-ups from some of my students.

There were some problems, however. The main thing was assessment. I developed a rubric, but then it would sometimes end up resulting in a grade I didn't feel accurately reflected the quality of the work. Over time, I've modified the rubric, but I've never been happy with the results. This semester, I'm considering just giving them scores of "satisfactory," "needs revision," and "not submitted," and record audio feedback on what I thought they did well, what could use improvement, etc. I'm still thinking about how to translate these results into a letter grade that I have to assign at the end of the semester.

Anyway, I would love to hear any comments or idea, tips, and about anyone's (student or faculty) experiences with these types of assessments.

vela,

Like you, I too had to adjust quite a bit of my assessment while conducting online or remote classes. The problem of "cheating" is certainly an issue with online exams. As with you, I had to de-emphasize exams, making them less than 50% of the total grade.

My assessment comes in different forms. Since most of the classes that I have taught during the past couple of years had been the General Physics courses, they had lab components. I upped the percentage of the grade from labs (up to 25%) after I was introduced to Pivot Interactives. I find them to be the best solution to "virtual experiments" because of two things: (i) other than actually performing the experiment itself, the students had do their own measurements of a real observation (data had all the errors of a typical experimental data), and (ii) I could modify the instruction to include a large portion of the material. It made it easier to test their knowledge of the topic by asking specific questions that I would have done in an exam. And they certainly could not look those up on Chegg or any other online websites since this often depended upon what they were observing.

Another form of assessment came from weekly discussion forum. This, I will admit, took a lot of effort and a lot of planning ahead, but sometime, there were happy coincidences. For example, it was the week that we were about to start a lesson on magnetism when the news reported on the testimony of that nurse in Florida claiming that the vaccine caused her to become magnetized. I immediately scrapped my planned topic for that week and instead, used the news report to ask the students to analyze scientifically every single claim made. They were to write their discussion post based only on the physics and not to make any kind of moral, social, or political judgement. And oh, the kicker was, this was a physics class for bio, pre-med, and life-science majors, so they knew a bit about physiology. It was as if they stars aligned and dropped this topic right on our laps at the right time.

Unlike the instructor in the video, I did not "ungrade". Rather, I emphasized some things that I did not normally do in f2f classes, and de-emphasized other things that normally would be a huge part of f2f classes. I had extensive training and got certified by the school as an online instructor, meaning that I could teach the courses that are exclusively online, so a lot of the ideas, techniques, and philosophy of teaching remotely came from what I acquired from that training. I learned that there are many different ways to student engagement in remote classes, and a variety of methodologies to assess their achievements (Bloom Taxonomy galore!). What I had mentioned above are only some of the things that I have done. Since most of my classes during the pandemic were synchronous, we also did a lot of graded "activities" as part of the online lessons.

BTW, my technique in combating rampant cheating during the exams for online classes is this: (i) all my questions are original and came out of my head (ii) my exams have strict time limits and only open over a specific window of time, usually during the published class time for synchronous classes. Let's say that I designed an exam that should take them 90 minutes. I will open the exam for a specific 2 hour window, say from 1 pm to 3 pm, and they can take the exam at any time in that time period, but they have only 90 minutes from the moment they start, with a hard stop at 3:00 pm no matter when they start.

Now, I have told them that they should treat the exams as if they were closed book exams, even though I specified that they were open book (restricted to only using the text, class notes, and everything from our LMS page). I also warned them that if they continuously needed to keep referring to their notes and text, they would run out of time. In other words, if they had to run to Chegg each time, there was no chance that they'd finish the exam. Texting each other for help to solve the questions also won't work all the time because I had a bank of questions that randomly assigned different questions to different students.

The result so far seems optimistic. Other than Spring 2020 semester when all hell broke loose, the percentage of A's, B's, C's, and D's in my classes haven't showed significant differences than my previous f2f classes (I got trained as an online instructor over Summer 2020 and started implementing what I learned in Fall 2020). I'm still refining and modifying my courses each semester, and just when I thought I was getting the hang of it, we've moved back to mostly f2f classes.

Zz.

 

[swampwiz]

I'm curious if anyone has tried "ungrading" in their STEM courses and if so, how it worked out. The idea is to get away from the using points to determine a student's grade and use different types of assessment that better motivate students to learn.

https://www.jessestommel.com/why-i-dont-grade/

https://www.chemedx.org/blog/ungrading-what-it-and-why-should-we-use-it

My interest arises from my experiences since classes went remote because of the pandemic. Like many other instructors, I saw the mysterious increase in performance by many students on exams (as well as obvious signs of cheating in some cases). To reduce the incentive to cheat, I replaced most of these high-stakes assignments with low-stakes weekly problems, where students had to write up a solution where they had to identify the relevant physical concepts, explain their problem-solving strategy, and finally solve the problem. It wasn't enough to just write down a bunch of math, which they could easily find on Chegg or somewhere else on the internet; they actually had to articulate the reasoning involved. I have to say I was pleasantly surprised by the quality of the write-ups from some of my students.

There were some problems, however. The main thing was assessment. I developed a rubric, but then it would sometimes end up resulting in a grade I didn't feel accurately reflected the quality of the work. Over time, I've modified the rubric, but I've never been happy with the results. This semester, I'm considering just giving them scores of "satisfactory," "needs revision," and "not submitted," and record audio feedback on what I thought they did well, what could use improvement, etc. I'm still thinking about how to translate these results into a letter grade that I have to assign at the end of the semester.

Anyway, I would love to hear any comments or idea, tips, and about anyone's (student or faculty) experiences with these types of assessments.

After I had set the curve on a Mechanism Design test, the instructor undergraded by letting everyone else take the exam again.

 

[symbolipoint]

to all of the above. I also think that the main reason teachers don't read essays written in cursive is that cursive is no longer how students communicate there, they're, their thoughts and desires. Instagram has replaced the billet-doux. I also point out that people, who put their trust in spellcheckers and unquestioningly accept their suggestions, are inviting trouble. My wife, a Russian historian, once showed me a student essay in which the author maintained that " $\dots~$ the surfs and pheasants rose in rebellion." Perplexed? Click on the spoiler.

Spoiler

$\dots~$ the serfs and peasants rose in rebellion.

I had to actually read the Spoiler to become aware that I did not even notice the spelling mistakes in the passage; I still made the correct reading and understood, not even realizing I read through those spelling mistakes. On the other hand, if I had been the author of that passage, I WOULD HAVE spelled 'serf' and 'peasant' correctly without any struggle.

 

[DaveE]

the surfs and pheasants

Like this? Doesn't seem too revolting to me.

 

[Dr Transport]

vela,

Like you, I too had to adjust quite a bit of my assessment while conducting online or remote classes. The problem of "cheating" is certainly an issue with online exams. As with you, I had to de-emphasize exams, making them less than 50% of the total grade.

...

BTW, my technique in combating rampant cheating during the exams for online classes is this: (i) all my questions are original and came out of my head (ii) my exams have strict time limits and only open over a specific window of time, usually during the published class time for synchronous classes. Let's say that I designed an exam that should take them 90 minutes. I will open the exam for a specific 2 hour window, say from 1 pm to 3 pm, and they can take the exam at any time in that time period, but they have only 90 minutes from the moment they start, with a hard stop at 3:00 pm no matter when they start.

Two interesting points here, I'm sorry I did not see them before.

The university where I am an adjunct does not allow for homework to be more than 25% of the total grade, so they are wed to midterms/quizzes/final exams. So this semester, in my graduate class, I assigned homework, but will not grade it nor collect it, unless they want a bump in their grades at the end of the semester. They can hand in ALL the homework assigned for a partial letter score upgrade, i.e. B- to B. I told them it must be neat and not just what is online on chegg or what they find in a solutions manual. I won't be giving any grade adjustments, I assigned over 500 problems and I know they are not doing the homework from the test scores.

In addition, I make two versions of the exam and they are alternating when I hand them out. It is blatant copying, I can say and even my grader has noticed that they copy on the quizzes. I did the paperwork to turn them in and that was a nightmare, but you have to do what you have to do. I will say that about 1/3 of the class will fail because they are simply not prepared for this level of class. (I'm basically teaching a sophomore level differential equations class to engineering graduate students, 90% of which are international students who, from our experience, have doctored transcripts. It takes a semester to figure out the ones who do not have valid credentials and throw them out. My department eliminated 50% of their 1st year grad students last year because of faked transcripts.)

As for making up exams, the one time I did use book problems on an online exam via zoom call, the grades went up from a 50% average to a 85%, so cheating is rampant. I am changing texts next semester, the first thing I did was looked for the solutions manual online (not chegg) and when I was satisfied I didn't see it after 10 minutes of searching, I decided on the new text.

Because I have huge classes, (next semester I have 2 sections of a grauate class with 25 students limits and they will be full), I've decided to forgo the weekly quizzes and just give 4 midterms. They will be difficult to say the least and not from the text. I could get away with a single exam but there is no testing center to send them to where I can ensure that they can't cheat.

 

[gmax137]

... international students who, from our experience, have doctored transcripts. It takes a semester to figure out the ones who do not have valid credentials and throw them out. My department eliminated 50% of their 1st year grad students last year because of faked transcripts.)

[TANGENT]
Wow.
I'm trying to square this with the dozens and dozens of threads here on PF from aspiring grad students who ask what they can do to prop up their "measly" 3.2 GPA. Seems like a lost cause if there are others faking transcripts.
[/TANGENT]

 

[gleem]

I will say that about 1/3 of the class will fail because they are simply not prepared for this level of class.

It's hard to imagine 1/3 of grad students failing a course.

 

[symbolipoint]

Dr. Transport, gmax137, gleem, others,
about all the cheating and the remedies you have been trying, what more can you do? What else are you ALLOWED to do?

 

[DaveE]

It's hard to imagine 1/3 of grad students failing a course.

I saw the result many years ago in interviewing EE candidates. It was pretty common for people to have taken classes and have no idea how to do the basic stuff they said they could, that should have been easy. I had zero interest in their grades, that meant nothing compared to asking for solutions to problems. I didn't care how or where they learned the material. Grad school is also a place where knowledge has to actually be applied, not just collected on a transcript.

 

[vela]

I could get away with a single exam but there is no testing center to send them to where I can ensure that they can't cheat.

I take it your courses will be taught remotely. My colleagues often complained about obvious signs of cheating on exams, like using weird ass notation. I had some students who copied stray marks that were on the solution they found online because they clearly didn't understand the solution. One thing you can do is make students write explanations on exams, so even if they find a solution online, it's useless unless they understand what's going on.

I'm happy to be back in the classroom this semester, and so are the students for the most part. The remote classes definitely took a toll on some students. I'm teaching the second semester of intro physics, but I noticed some students really don't know what it takes to succeed in a physics course. Some don't attend regularly (and miss the labs as a result); some don't do much of the homework; etc.

 

[vela]

I saw the result many years ago in interviewing EE candidates. It was pretty common for people to have taken classes and have no idea how to do the basic stuff they said they could, that should have been easy. I had zero interest in their grades, that meant nothing compared to asking for solutions to problems.

This gets to the idea underlying ungrading.

The traditional idea behind grades is that students who learn the material well will do well on assignments and exams, which will be reflected in their grade for a course. But there are a lot of students who suck at learning, so the emphasis on earning points/grades incentivizes cheating and other bad behaviors over learning. If a student happens to learn something along the way, great. But getting 100% on the exam turns out to be the main goal, not understanding the material.

I imagine the way the system is supposed to work is in fact how it worked out for many of us here. So we have the mindset that since the system worked for us, we're just going to stick with it, despite obvious signs it doesn't work for many students. The goal of ungrading is to set up a course so that learning, rather than earning points, becomes the main focus for students.

 

[symbolipoint]

I saw the result many years ago in interviewing EE candidates. It was pretty common for people to have taken classes and have no idea how to do the basic stuff they said they could, that should have been easy. I had zero interest in their grades, that meant nothing compared to asking for solutions to problems. I didn't care how or where they learned the material. Grad school is also a place where knowledge has to actually be applied, not just collected on a transcript.

That reminds us of why good interviewers for scientific or engineering position candidates ask basic applied questions, such as mostly involving basic arithmetic and basic algebra, including for some simple applications directly related to the job to be filled. The competent candidates will logically pull their way through the problem; and the incompetent candidates will not due to not having the concepts or to not having the academic skills. But OF COURSE, this does not account for social inter-network connections.

 

[anorlunda]

Allow me to reminisce about the old days. When I went to college, we had an enforced policy that C must be average. Actually the mean, with half above and half below. That means the average GPA was 2.0 by definition. Flunk out was 1.85.

We lost half the students each semester. We started with 340 freshmen EEs and graduated 30. Nobody ever asked about rank in class.

On the other hand, tuition my freshman year was only $300, and by senior year $600. Admission was lax by today's standard. It was an egalitarian system that granted admission to a large number of candidates, but graduated only the worthy. Many flunk outs were able to get degrees in less demanding fields in less demanding colleges. They could transfer credits for the courses they did pass. In that way, those who flunked out did not lose a fortune. The phrase for that is, "Reach for the brass ring."

p.s. It was the Vietnam War that changed everything. Suddenly, a grade less than B was seen as a death sentence. The flunk rate dropped drastically. After the war, it never reverted to the previous state.

 

[DaveE]

an enforced policy that C must be average

Oh, so you didn't go to Stanford then...

 

[swampwiz]

It's hard to imagine 1/3 of grad students failing a course.

It's not so hard when the course is Theory of Elastic Stability.

 

[gleem]

It's not so hard when the course is Theory of Elastic Stability.

My point was that most of those people should not have been admitted to the graduate program.

 

[swampwiz]

Allow me to reminisce about the old days. When I went to college, we had an enforced policy that C must be average. Actually the mean, with half above and half below. That means the average GPA was 2.0 by definition. Flunk out was 1.85.

We lost half the students each semester. We started with 340 freshmen EEs and graduated 30. Nobody ever asked about rank in class.

p.s. It was the Vietnam War that changed everything. Suddenly, a grade less than B was seen as a death sentence. The flunk rate dropped drastically. After the war, it never reverted to the previous state.

Perhaps it was like this?

 

[Dr Transport]

Dr. Transport, gmax137, gleem, others,
about all the cheating and the remedies you have been trying, what more can you do? What else are you ALLOWED to do?

I've asked for the largest lecture halls on campus, that way I can get away with one version of the exam and spread them out so that they can't cheat. Right now I am in a classroom that has about 5 chairs empty, so spreading them out isn't an option.

We are not a big university, nor a major research university, so we don't have the amenities that others have. I'm really only on campus for about 8 hours a week, so taking time to do other approaches isn't viable.

Part of the cheating is cultural, they feel honored to have someone view their work as good and want to copy it. The other side of this, is that they feel the obligation to help each other out, again it is a cultural thing. In the US, we are brought up to help, but it is every man for himself, which is not how they were raised.

I had three of my students who got a zero for copying beg me for a makeup exam, I gave in, but it was extremely difficult and I'm going to be a bastard grading it, they won't get better than an 50%. When they came to take it, I locked up their bags and phones in my lab store room and put them in three different rooms and told them explicitly that if I saw any evidence of cheating this time, zeros would be given again and they'd flunk my course, period. Come back next semester, buy the new text and try again.

As for what else I can do, I don't know.

 

[anorlunda]

Part of the cheating is cultural

It is also cultural to not "rat" on friends or family. I think it is strange how some social instincts are actually antisocial in effect.

 

[Dr Transport]

I take it your courses will be taught remotely.

Nope, all in person. That is how rampant and blatant it is. These people are pros at it, they've copied off of others for their entire careers.

 

[gleem]

Will this be the great imposter generation?

 

[Cave Diver]

I'm curious if anyone has tried "ungrading" in their STEM courses and if so, how it worked out. The idea is to get away from the using points to determine a student's grade and use different types of assessment that better motivate students to learn.

https://www.jessestommel.com/why-i-dont-grade/

https://www.chemedx.org/blog/ungrading-what-it-and-why-should-we-use-it

My interest arises from my experiences since classes went remote because of the pandemic. Like many other instructors, I saw the mysterious increase in performance by many students on exams (as well as obvious signs of cheating in some cases). To reduce the incentive to cheat, I replaced most of these high-stakes assignments with low-stakes weekly problems, where students had to write up a solution where they had to identify the relevant physical concepts, explain their problem-solving strategy, and finally solve the problem. It wasn't enough to just write down a bunch of math, which they could easily find on Chegg or somewhere else on the internet; they actually had to articulate the reasoning involved. I have to say I was pleasantly surprised by the quality of the write-ups from some of my students.

There were some problems, however. The main thing was assessment. I developed a rubric, but then it would sometimes end up resulting in a grade I didn't feel accurately reflected the quality of the work. Over time, I've modified the rubric, but I've never been happy with the results. This semester, I'm considering just giving them scores of "satisfactory," "needs revision," and "not submitted," and record audio feedback on what I thought they did well, what could use improvement, etc. I'm still thinking about how to translate these results into a letter grade that I have to assign at the end of the semester.

Anyway, I would love to hear any comments or idea, tips, and about anyone's (student or faculty) experiences with these types of assessments.

Mr. Kohn talks nicely but lives in lalaland. If his theories mattered then grades would be a thing of the past. We need proctored tests and grades to know how well students know the material. We also need them so that students get stressed out, do their best, and become better people through developing mechanisms of coping with life, pressure, competition, and having to find ways to do things they hate. You take a wrong turn and you are dead. This is how life is. It's OK for students to take a test and a grade. They will become far more traumatized if they go out there like a bunch of entitled weaklings. That's hard to hear but it shouldn't be. This is how the world has been spinning since prehistory. It won't change for you or me the last 20 years just because we've been pressing buttons on a screen. Why is it so hard to understand the obvious? It's interesting that people with PhDs are capable of confusing reality with wishful thinking to such an extent, or worse, knowingly lie with a straight face. It is equally interesting that other people take them seriously and even more interesting that it is nearly impossible to find any articles criticizing 'ungrading' for what it is: a dishonest and hypocritical device.

 

[symbolipoint]

Some training systems use a chart of objectives. The person who trains the learner makes a mark to affirm that a specific objective is satisfied. The objectives might or might not be met in any decided sequence. Only important is at some point in the training or during an assessing period, every objective is met. The training administrators could choose to apply a scoring system, or not, depending on how important a scoring system is for the training/learning to be done.

 

[anorlunda]

We need proctored tests and grades to know how well students know the material. We also need them so that students get stressed out, do their best, and become better people through developing mechanisms of coping with life, pressure, competition, and having to find ways to do things they hate.

Bravo. Well said.

Future employers also need confidence that graduates have the knowledge needed to do the jobs and that they have performed well under pressure. A grade transcript provides useful clues.

 

[PeroK]

Mr. Kohn talks nicely but lives in lalaland. If his theories mattered then grades would be a thing of the past. We need proctored tests and grades to know how well students know the material. We also need them so that students get stressed out, do their best, and become better people through developing mechanisms of coping with life, pressure, competition, and having to find ways to do things they hate. You take a wrong turn and you are dead. This is how life is. It's OK for students to take a test and a grade. They will become far more traumatized if they go out there like a bunch of entitled weaklings. That's hard to hear but it shouldn't be. This is how the world has been spinning since prehistory. It won't change for you or me the last 20 years just because we've been pressing buttons on a screen. Why is it so hard to understand the obvious? It's interesting that people with PhDs are capable of confusing reality with wishful thinking to such an extent, or worse, knowingly lie with a straight face. It is equally interesting that other people take them seriously and even more interesting that it is nearly impossible to find any articles criticizing 'ungrading' for what it is: a dishonest and hypocritical device.

I don't agree with this. Knowledge of a subject can be separated from working under pressure. A good tennis coach, for example would assess a player's ability and their mental strength separately.

Also, I had someone working for me who was technically very sound (IT) but flaky under pressure. So, I got him to do what he was good at: writing and testing scripts. But, I wouldn't have asked him to work a weekend on his own.

Your attitude is too all or nothing for me. We ought to make the working environment more inclusive and try to exploit the strengths people have. Not throw people on the scrapheap because they have a weakness.

 

[DaveE]

A grade transcript provides useful clues.

When I was hiring EEs, I never asked for transcripts. I just didn't care what grade they got in physics. I asked questions; my own impromptu oral exam. No one ever asked for my transcripts either. But solving problems for people is a constant activity. Honestly, I don't know how to compare an A- from Texas Tech to a B- from MIT. Grading just isn't consistent enough to mean much outside of a single institution (maybe inside too).

 

[anorlunda]

When I was hiring EEs, I never asked for transcripts.

It varied by decade. When grade inflation was rampant, grades became useless. Before the Vietnam War, successful graduation was sufficient evidence. In many cases, the recommendation from a known professor was better than grades. In others, yes you're right interview questions.

Our biggest complaint was that engineering students were deficient in writing and language skills. A consulting company's major products were reports and presentations.

 

[gleem]

Do I/we really need to see the rest of the video?

Let's be open-minded now.

I read the article by Kohn reference by @vela "The Case Against Grades" which elaborate on his ideas and gives examples of their success(?), Have any scientific studies (think double-blind) been conducted comparing ungrading programs to grading programs, and the effect on students' future success? Kohn is down on grading in elementary and high school but doesn't say much about college. For schools that ungrade, he states

Moreover, these schools point out that their students are often more motivated and proficient learners, thus better prepared for college, than their counterparts at traditional schools who have been preoccupied with grades.

"Often more motivated and proficient learners" By not quantifying this statement what do we take it to mean in support of his thesis? He also does not give much advice as to how assessments are to be made except you dare not use numbers.

Something happened during the '60s and '70s. I met one of my ug physics profs several years after graduating, and he commented that students were less hard-working.

Regarding grades reducing the incentive to learn was not my experience. As far as driving students to cheat can we blame increased research misconduct on grading? Some of these students became parents who in recent years tried to bribe university officials to assist in the admission of their children to university. Did their grades make them do it?

Maybe we parents must shoulder some blame for promising too much and not being realistic.

 

[Cave Diver]

I don't agree with this. Knowledge of a subject can be separated from working under pressure. A good tennis coach, for example would assess a player's ability and their mental strength separately.

Also, I had someone working for me who was technically very sound (IT) but flaky under pressure. So, I got him to do what he was good at: writing and testing scripts. But, I wouldn't have asked him to work a weekend on his own.

Your attitude is too all or nothing for me. We ought to make the working environment more inclusive and try to exploit the strengths people have. Not throw people on the scrapheap because they have a weakness.

I don't think that you disagree with anything I actually said. I am not talking about stringent time limits or cut-throat oral examinations, or even interviews for jobs ... (right?) I am talking about fully-proctored, in-class tests with very liberal time limits ... Is this attitude all or nothing for you?

My recommendation letters, and I have written hundreds of them, mean a ton and my grades mean a ton and I make sure I let graduate schools and employers know with great success as you may imagine. Unfortunately, the grades of many of my colleagues mean nothing. That is the truth. They needed to get tenured, promoted, or accepted by students, colleagues, and the community.