TA Refuses to Give Out 100%s on Lab Reports: Analyzing Pros & Cons

[kurros]

Many people are attracted to science classes because of the concrete nature of the investigations as compared to the subjective nature of other disciplines.

HAHAHAHA! Well if that's the case, then best we crush that misconception as quickly as possible, and lab class is usually the place to introduce them to the real world.

As for the OP, I wouldn't stress about this issue, in the end it makes little difference. I give out 100's when marking not because I think the report it perfect, but because I don't often have the time to properly write down a justification of whatever I am still nitpicking at that point.

As for your policy of being harder on the good students, I do this too, but the motivation from my perspective is this:

1. Docking marks requires giving the students written feedback to justify it.
2. This takes time.
3. I have already burned enough time describing the big issues to be fixed to the weaker students. I am not going to waste more time telling them the minutia that need to be fixed; indeed this will distract attention from the bigger issues they should focus on and they probably won't read it anyway.
4. Strong students are more likely to actually pay attention to feedback about finer details, thus I in turn am more willing to give them this portion of my time.

In the end the bias is small so it is pretty unimportant. It may be perfectly "fair", but everybody gets the feedback most relevant to them. It is not worth the effort required to ensure that the marking meets some magic standard of consistency across every detail. So long as the overall final marks are a good reflection of the overall standard I think that is fine.

 

[ChiralWaltz]

<snip>

The people in that house need to take the trash out. It's a mess in there.

If he got under it, he could probably touch the goal. I doubt he would be able to get it down though. It is wrong to tape that just out of his reach. I'm glad a friend understood what was happening and handled the situation appropriately.

I assume some students are trying to score 100s on their labs. Have you been able to explain to them how to do so?

 

[MalachiK]

HAHAHAHA! Well if that's the case, then best we crush that misconception as quickly as possible, and lab class is usually the place to introduce them to the real world.

Err, I'm pretty sure that if a bunch of students are carrying out an experiment to find the Rydberg constant or something then we should probably expect them to be able to come up with a number that matches the one in the textbook (give or take the uncertainty in their experiment) or suggest a pretty good reason why their result is different. Sure, lab results are messy. But one of the first things we teach is that in science we should try to quantify the uncertainties in our results. It's not all just a subjective free for all!

 

[kurros]

Err, I'm pretty sure that if a bunch of students are carrying out an experiment to find the Rydberg constant or something then we should probably expect them to be able to come up with a number that matches the one in the textbook (give or take the uncertainty in their experiment) or suggest a pretty good reason why their result is different. Sure, lab results are messy. But one of the first things we teach is that in science we should try to quantify the uncertainties in our results. It's not all just a subjective free for all!

If you accept in advance that you understand perfectly well all the theory behind the experiment you are trying to do (and if indeed this is correct...) then sure you can make people go through this mechanical exercise of checking they can get the right numbers out, and yes this is what we make junior undergraduates to do get a feeling for how to operate in the lab.

But this is not what science is about or how it works. One does experiments that are supposed to push the boundaries of what accepted theory describes, and when discrepancies arise and they need to be understood then things can become incredibly subjective. We usually expose senior undergraduates to this by giving them longer, more complicated/realistic experiments to do, and giving them only a simplified theoretical model of what is expected to happen (partially because the full theory is a bit beyond them still). All kinds of weird discrepancies can show up and it is good for them to have to really think about the various explanations for what is going on. Things are rarely black and white.

But no, I didn't mean to imply it was a "free-for-all". But neither are the humanities total free-for-alls. In so much as there are various approaches to the philosophy of science there are also constraints on (say, for example) theories of meta-ethics or political thought.

 

[mathwonk]

you sound like a crooked gymnastics judge from a few decades ago. you don't determine your grade standards as you grade, but before giving the test. be honest; define your standards and give grades according to how students meet them. otherwise you are like a home base referee who brags about "my strike zone".

 

[kurros]

you sound like a crooked gymnastics judge from a few decades ago. you don't determine your grade standards as you grade, but before giving the test. be honest; define your standards and give grades according to how students meet them. otherwise you are like a home base referee who brags about "my strike zone".

The standards are of course determined before grading, but when there is a stack of 30 lab reports that are 20+ pages each to grade then there is a significant element of pragmatism that has to enter in terms of what aspects of the criteria one focuses on. Otherwise it is impossible to get through them before the universe fades into heat-death. There is also the matter of it being almost impossible to define what the standards are. These are complicated things to mark and the variation between markers is more significant than the variation due to focusing on different criteria.

 

[MalachiK]

But this is not what science is about or how it works. One does experiments that are supposed to push the boundaries of what accepted theory describes, and when discrepancies arise and they need to be understood then things can become incredibly subjective. We usually expose senior undergraduates to this by giving them longer, more complicated/realistic experiments to do, and giving them only a simplified theoretical model of what is expected to happen (partially because the full theory is a bit beyond them still). All kinds of weird discrepancies can show up and it is good for them to have to really think about the various explanations for what is going on. Things are rarely black and white.

Maybe you're right - I'm not a university level educator. But surely on a taught course where you're not looking for new science the assessment criteria can be pinned down pretty exactly? I understand that there are some very high level taught courses out there, but even so the aims of the course must surely be written down somewhere. If the student has demonstrated that they've achieved all of the objectives of the course (which may very well include outstanding organisation, written English and presentation to a standard that could be published) then why not give them 100%?

I've always wondered about what goes on in the humanities, where the 100% mark essay is as rare as hen's teeth. If the nobody can adequately define the standard required to gain all of the marks, can anyone in the faculty claim that they understand 100% of the course that they're teaching? Quite often you can ask what you could have done to improve on a >95% essay and you just get a shrug of the shoulders or some guff about the correct use of the semi colon.

 

[kurros]

...then why not give them 100%?

I am happy to do this, it was the OP who was concerned. I just sympathise with their issue.

I've always wondered about what goes on in the humanities, where the 100% mark essay is as rare as hen's teeth. If the nobody can adequately define the standard required to gain all of the marks, can anyone in the faculty claim that they understand 100% of the course that they're teaching?

Well, let's consider labs first. In the lab I TA in there are about 30 experiments (i.e. equipment for doing experiments) scattered about 3 rooms, and the students spend 12 or so hours over a couple of weeks trying to measure something interesting with the stuff. They are supposed to rigorously record what they are doing as they go in a log book. There are quite detailed guidlelines they are given about the way they are supposed to write this log book, which has a somewhat more vague marking scheme associated with it, but in the end every experiment is different and every student is going to have different experiment with their machine. Each experiment is assigned to a TA (who "supervises" several experiments) and who marks work done for that experiment. It is up to them to try and figure out the best mapping from what the student has done into the generic marking scheme. So it is not easy.

As for humanities, well I guess I don't know what they do, but I assume it is similar. Lecturer asks for an essay on some topic, gives the students some university-approved guidelines and generic marking scheme for how essays are supposed to be written/marked, and then either marks the essays themself, deciding what they think the essay is worth and then trying to map it to the marking criteria, or handball it to a grad student to do the same thing.

Quite often you can ask what you could have done to improve on a >95% essay and you just get a shrug of the shoulders or some guff about the correct use of the semi colon.

Probably they have a better reason but they can't quite articulate it. Often one has an overall "feeling" that a piece of work could have been done better, or even that specific areas were not quite up to scratch, but it is difficult to point to a sentence and say exactly what is wrong with it.

What I mean is that constructive criticism is quite an art form. It is easy to know that something isn't right, but much hard to clearly explain what that something is and how to fix it. I guess the former is an "intuitive" or system 1 inference, while the latter takes formal logic and reasoning, or system 2 (if you are into dual-process theory). Yet a lot of the actual marking is done by system 1 I expect, and merely rationalised into the given marking scheme afterwards. I'm not saying markers do this on purpose, but that is how the process looks to me.

 

[Redbelly98]

What this policy really does is change the scale range to 0-99 instead of the usual 0-100. I think that's fine IF the students are aware of this. If they are led to believe it is still a 0-100 scale, then there is some level of dishonesty at play. Striving for that final point is a futile effort. At the very least the students should be informed about the actual scale range.

Get rid of the notion that a maximum score must indicate a perfect paper. It does NOT have to do that. It's just an indication that a student is well above the ability of the others in the class.

This isn't true if there are other TA's for the course in which case there is a 0-99 scale in his section but a 0-100 in another section

It took me a while to get what you are saying here, or at least I think I get it. While technically it's unfair that students in other lab sections can earn a higher grade in the same course for the same quality of work, for all practical purposes I can't imagine there's any significant difference from this policy. It only affects a small fraction of students (those getting 99's and 100's), and only changes the grade by 1 percentage point on something that likely counts for a quarter or so of the total grade -- so around 0.25% in the overall grade.

 

[Andy Resnick]

If you accept in advance that you understand perfectly well all the theory behind the experiment you are trying to do (and if indeed this is correct...) then sure you can make people go through this mechanical exercise of checking they can get the right numbers out, and yes this is what we make junior undergraduates to do get a feeling for how to operate in the lab.

But this is not what science is about or how it works. One does experiments that are supposed to push the boundaries of what accepted theory describes, and when discrepancies arise and they need to be understood then things can become incredibly subjective. We usually expose senior undergraduates to this by giving them longer, more complicated/realistic experiments to do, and giving them only a simplified theoretical model of what is expected to happen (partially because the full theory is a bit beyond them still). All kinds of weird discrepancies can show up and it is good for them to have to really think about the various explanations for what is going on. Things are rarely black and white.

<snip>

I totally agree with this. The problem is that all labs, from intro 200-level courses to senior 400-level courses, by their very nature, are standard 'cookbook' procedures.

A better option is the authentic research experience, for example a senior thesis or 'capstone' project. The lab courses are then oriented towards learning specific measurement techniques instead of vague historical recreations of 'famous experiments'.

 

[atyy]

If you accept in advance that you understand perfectly well all the theory behind the experiment you are trying to do (and if indeed this is correct...) then sure you can make people go through this mechanical exercise of checking they can get the right numbers out, and yes this is what we make junior undergraduates to do get a feeling for how to operate in the lab.

But this is not what science is about or how it works. One does experiments that are supposed to push the boundaries of what accepted theory describes, and when discrepancies arise and they need to be understood then things can become incredibly subjective. We usually expose senior undergraduates to this by giving them longer, more complicated/realistic experiments to do, and giving them only a simplified theoretical model of what is expected to happen (partially because the full theory is a bit beyond them still). All kinds of weird discrepancies can show up and it is good for them to have to really think about the various explanations for what is going on. Things are rarely black and white.

I totally agree with this. The problem is that all labs, from intro 200-level courses to senior 400-level courses, by their very nature, are standard 'cookbook' procedures.

A better option is the authentic research experience, for example a senior thesis or 'capstone' project. The lab courses are then oriented towards learning specific measurement techniques instead of vague historical recreations of 'famous experiments'.

How can science be subjective? Hasn't one made an error if science is subjective?

 

[kurros]

How can science be subjective? Hasn't one made an error if science is subjective?

Science is subjective when you don't have sufficient data to easily discriminate between what competing explanations there are for a phenomenon, which is basically all the time for practicing scientists. In the end the summary of what you do and don't know should be pretty objective (although if you start reading into the philosophy of statistics/probability you quickly find that even that is not so straightforward as it seems...)* and that is what ends up being published, but the way you get there involves a lot of subjective guesswork and intuition. Not to mention that people have to decide what to spend time and money investigating, and there is no objective way to make this decision. You might argue that those concerns are not part of science, and ok maybe that is valid. But still the core practice of science is a game of probability and uncertainty. Eventually you hope the probabilities all collapse to approximately zero or one, but that is the only the best case scenario.


* or even aside from that consider say the current searches for dark matter: DAMA, Pamela (and now AMS it seems), CoGeNT, and I think at least one other experiment I am forgetting all see signals that could be interpreted as originating from dark matter. Yet other experiments (Xenon, CDMS etc) claim to be sensitive enough that they can exclude the usual dark matter models that could explain those signals. What is going on? Are all those experiments so flawed? Are there some strange other background sources that have not be considered? Do we really need to cook up some "ugly" (from an Occams razor perspective, itself a subjective criterion) convoluted dark matter model to explain all this stuff simultaneously? No one really knows. In the end we hope that some really excellent experiments will be done that clearly demonstrate what is happening, but this is not the ordinary state of affairs. If one starts looking at fields like medicine then the problems only get worse...

 

[kurros]

If you want to take it even further you can start thinking about the philosophy of science. Kuhn says we are all just working in a convenient theoretical framework that we as a community have collectively come to agree upon, and that we will cling to it for dear life until some sufficiently drastic experimental evidence force us to abandon it and begin the tower anew. We already know that our models are "wrong" in an absolute sense, so when we talk about gravity being due to the curvature of space and electrons being excitations of quantum fields, how literally do we mean it? People still argue about interpretations of quantum mechanics a century after its discovery. One can argue that as long as we are getting the right answers from the math then that is enough, that is the part that is science, but I have a hard time accepting that. Generally we grant quite different ontological status to the postulates of a theory motivated by physical principles versus one which is "merely" an empirical description (such as a neural network that has learned the patterns of some data, or even say the difference between Kepler's and Newton's explanations of planetary motion). So getting the right answers doesn't seem to be enough...

But there is still plenty of subjectivity in the daily practice of science even before you start to trek down such rabbit holes :p.

 

[MalachiK]

So the argument is that because human knowledge of the universe is less than absolute - you won't grade a paper to 100%. Why does the scale go up to 100 then?

Even where there are competing conclusions and it's not possible to objectively choose between them then it is still possible to give an account of these different interpretations and evaluate their merits. Even where there are differing statistical interpretations that would yield differing conclusions it's still possible to talk about them. If this is the best that can be done then that that's a 100% paper. Not giving full marks because your student isn't omniscient seems an odd way to go. But hey, it's not up to me.

 

[atyy]

Science is subjective when you don't have sufficient data to easily discriminate between what competing explanations there are for a phenomenon, which is basically all the time for practicing scientists. In the end the summary of what you do and don't know should be pretty objective (although if you start reading into the philosophy of statistics/probability you quickly find that even that is not so straightforward as it seems...)* and that is what ends up being published, but the way you get there involves a lot of subjective guesswork and intuition. Not to mention that people have to decide what to spend time and money investigating, and there is no objective way to make this decision. You might argue that those concerns are not part of science, and ok maybe that is valid. But still the core practice of science is a game of probability and uncertainty. Eventually you hope the probabilities all collapse to approximately zero or one, but that is the only the best case scenario.* or even aside from that consider say the current searches for dark matter: DAMA, Pamela (and now AMS it seems), CoGeNT, and I think at least one other experiment I am forgetting all see signals that could be interpreted as originating from dark matter. Yet other experiments (Xenon, CDMS etc) claim to be sensitive enough that they can exclude the usual dark matter models that could explain those signals. What is going on? Are all those experiments so flawed? Are there some strange other background sources that have not be considered? Do we really need to cook up some "ugly" (from an Occams razor perspective, itself a subjective criterion) convoluted dark matter model to explain all this stuff simultaneously? No one really knows. In the end we hope that some really excellent experiments will be done that clearly demonstrate what is happening, but this is not the ordinary state of affairs. If one starts looking at fields like medicine then the problems only get worse...

But if "no one really knows", that's pretty objective. It's only subjective if everyone really knows and disagrees. Or if no one can really know, and the question must be answered subjectively. In the cases you mentioned, more experiments should give the answer, not much different from a lab report.

For example, the OP mentioned "neatness and organization". That is indeed (objectively) subjective. Therefore I think it is fair that full marks are never given (which is quite different from saying one cannot get an "A" in the class).

I've always wondered about what goes on in the humanities, where the 100% mark essay is as rare as hen's teeth. If the nobody can adequately define the standard required to gain all of the marks, can anyone in the faculty claim that they understand 100% of the course that they're teaching? Quite often you can ask what you could have done to improve on a >95% essay and you just get a shrug of the shoulders or some guff about the correct use of the semi colon.

No one really understands what goes on in music. If Beethoven handed in Op 106 for his composition homework, would he deserve 100%? I think it is quite correct to say we don't know what a "perfect" piece of music is in general. Even then, while much of Mozart's work is perfect, not all the perfect works are masterpieces. On the other hand, Op 106 may or may not be perfect, but it is undoubtedly a masterpiece. And yes, I do believe that these subjects with subjective components have their place in school, alongside the objective subjects like physics.

 

[reenmachine]

Personally , in high school at least , I hated having a professor who added his little touch to my final result.When I ace the test , I ace the test.That means 100%.Scoring 100% was a great motivation to try and achieve 100% on every single math tests back in high school.

I disagree with this philosophy.

It might be different in university , I will reserve my judgement on this.

 

[kurros]

So the argument is that because human knowledge of the universe is less than absolute - you won't grade a paper to 100%. Why does the scale go up to 100 then?

This was just following on from an argument that I accidentally started regarding the objectivity of science. I actually said I was in principle happy to give out 100% on anything I mark. I guess I let that get a bit off topic, considering that the subjectivity of marking criteria is really what this thread is about, not the subjectivity of the scientific method.

 

[MalachiK]

If Beethoven handed in Op 106 for his composition homework, would he deserve 100%?

I don't know. Maybe. I guess it would depend on the criteria of the taught course that he was taking. I mean, if the assignment was to compose the most perfect piece of music then I can see how you could argue that he hadn't met the standard. On the other hand, if he was being assessed against a pre agreed set of compositional techniques then it would rather depend on if he'd demonstrated the required competences. I've never seen course titles like 'Writing the ultimate lab report; Perfect knowledge will be yours! Science will be complete!' or 'Write the one perfect poem so that we can clear out all the other stuff from the library!" Usually they're called things like - "Experimental Techniques" or "English 206".

Just because perfection is unattainable, I don't see why we can't specify the standard that would completely satisfy the requirements of the courses we teach. Or are we saying that nothing less than a complete ToE is needed to demonstrate that a student really gets that post grad course quantum field theory?

But whatever. I'll agree with any policy that you like if it means that I get to work with a bunch of students for whom the 100% is of immediate practical concern! It'd make a change from arguing the toss over rounding half marks over the pass / fail boundary.

 

[Andy Resnick]

How can science be subjective? Hasn't one made an error if science is subjective?

I think you are taking my comments out of context. The correct context for my point is that 'standardized' labs generally do not correlate with the actual practice of laboratory science.

 

[jesse73]

The standards are of course determined before grading, but when there is a stack of 30 lab reports that are 20+ pages each to grade then there is a significant element of pragmatism that has to enter in terms of what aspects of the criteria one focuses on. Otherwise it is impossible to get through them before the universe fades into heat-death. There is also the matter of it being almost impossible to define what the standards are. These are complicated things to mark and the variation between markers is more significant than the variation due to focusing on different criteria.

This is the longest way possible of saying you think its too much work to stick to your grading standards.

 

[jesse73]

It took me a while to get what you are saying here, or at least I think I get it. While technically it's unfair that students in other lab sections can earn a higher grade in the same course for the same quality of work, for all practical purposes I can't imagine there's any significant difference from this policy. It only affects a small fraction of students (those getting 99's and 100's), and only changes the grade by 1 percentage point on something that likely counts for a quarter or so of the total grade -- so around 0.25% in the overall grade.

The point is not the amount of bias but that there is a systematic bias which isn't fair. Random biases and mistakes are fine but not a systematic bias.

 

[kurros]

This is the longest way possible of saying you think its too much work to stick to your grading standards.

Sure, if you want to put it that way. But extra work for me also means extra money spent to pay me to do it; and the university administration isn't going to be happy if I double the time it takes to mark each log book for the sake of removing a small bias. It is just not worth it.