Physics Challenge Exams: Placement & Evaluation Research

[Vanadium 50]

. Maybe making multiple versions of the test is how to do it.

Even scramble tests (same questions, different order) work to a degree.

 

[Andy Resnick]

What a disappointment. What can you do to make that kind of cheating become unworkable for the student?

... that was before I read the posts which followed. Maybe making multiple versions of the test is how to do it.

Cheating, like any other kind of crime, cannot be completely and permanently eliminated. My approach is to disincentivize it as much as possible.

 

[Andy Resnick]

<snip>the fact that most students in most courses get 50% or lower is something that we can't avoid taking very, very seriously.

I agree, definitely. I'm struggling with how to best remediate the students.

 

[symbolipoint]

Hello fellow physicists,
I teach first and second year physics at a community college. In particular, I teach two algebra based- and three calculus-based physics courses. I wanted to know if any of you have used any type of challenge exams to place out of the first year physics course. I am also researching to see if there are any standard exams that we can use to evaluate our students at the end of the semesters.

PS. I hope I am clear enough. My boss asked me to research, but I have not found any. I know chemistry has standard exams for intro classes.

- Thank you!

Not really the type of thing you're looking/asking for, but
One of the university Physics departments had (maybe still do has) a mathematics skills assessment administered at the beginning of the class term in Phyics 1 - Fundamental Mechanics, part of the engineering Physics course sequence. If student performs too low, then student is REQUIRED to enroll in a short - few weeks remedial course to bring up students basic intermediate algebra and basic trigonometry skills; and if student does not enroll in this course and pass it, then the physics department will administratively drop the student from the Physics 1 - Fundamental Mechanics course.

 

[bcrowell]

Not really the type of thing you're looking/asking for, but
One of the university Physics departments had (maybe still do has) a mathematics skills assessment administered at the beginning of the class term in Phyics 1 - Fundamental Mechanics, part of the engineering Physics course sequence. If student performs too low, then student is REQUIRED to enroll in a short - few weeks remedial course to bring up students basic intermediate algebra and basic trigonometry skills; and if student does not enroll in this course and pass it, then the physics department will administratively drop the student from the Physics 1 - Fundamental Mechanics course.

We tried to do something like this. There are a lot of barriers.

1) We would want to convince ourselves that our math pretest was very strongly correlated with success in phyiscs. We constructed a test and didn't find such a strong correlation.

2) We're a public school (California community college), and there are court decisions that make it very difficult to do this sort of thing.

3) We would want to have some evidence that the required remediation was effective. Actually the best evidence I've seen is that math remediation in general simply doesn't work.

4) Many students with inadequate math skills have passed the relevant math courses at our school. Neither they nor the math department want to hear that a C in math equals total incompetence.

 

[Dr. Courtney]

We tried to do something like this. There are a lot of barriers.

1) We would want to convince ourselves that our math pretest was very strongly correlated with success in phyiscs. We constructed a test and didn't find such a strong correlation.

2) We're a public school (California community college), and there are court decisions that make it very difficult to do this sort of thing.

3) We would want to have some evidence that the required remediation was effective. Actually the best evidence I've seen is that math remediation in general simply doesn't work.

4) Many students with inadequate math skills have passed the relevant math courses at our school. Neither they nor the math department want to hear that a C in math equals total incompetence.

Then perhaps your school should replace diplomas with participation trophies.

USAFA developed a math assessment using ALEKS that was an excellent predictor of success in both Calculus and Physics. Many math departments now are using some kind of ALEKS assessment/remediation for placement in Calculus, and there are strong correlations with success in Calculus. Since I was the person in the USAFA math dept who knew the most about what it takes to succeed in Physics, it was pretty simple to get their ALEKS assessment focused enough on right triangle trig and the parts of Algebra 1 used in Physics to ensure relevance and correlation.

The ALEKS pre-calc course is excellent preparation for both Calculus and Physics. Tuning it for Physics is more a matter of removing all the bits that are less important, nothing really needs to be added.

 

[symbolipoint]

We tried to do something like this. There are a lot of barriers.

1) We would want to convince ourselves that our math pretest was very strongly correlated with success in phyiscs. We constructed a test and didn't find such a strong correlation.

2) We're a public school (California community college), and there are court decisions that make it very difficult to do this sort of thing.

3) We would want to have some evidence that the required remediation was effective. Actually the best evidence I've seen is that math remediation in general simply doesn't work.

4) Many students with inadequate math skills have passed the relevant math courses at our school. Neither they nor the math department want to hear that a C in math equals total incompetence.

Maybe I should have replaced "remediation" with "review". Either way, the students in the Physics 1 Fundmtl Mechanics course DID have the official prerequisite Mathematics courses of credit, but that was seen as irrelevant (or should this be, 'not adequate by itself' ?). The department tested to see if each student had current competency in the needed Mathematics for the course. Some of us did not have as much competence as were necessary, so were required to take the review course. Did it really help those who scored low in math skills assessment? Maybe. Would the review course had helped? Maybe. Did the review course hurt any of us? NO.
Not sure really how to judge this. You mentioned "evidence" for predicting success.

My opinion is that the department was (still is) justified in giving this skills assessment test. Students should not be allowed to fool themselves into thinking that if they have the official prerequisite Math courses that they have the needed competence too. Assessing and then if necessary requiring the review course makes sure that these students review the skills needed for the Physics 1- Mechanics course.

 

[Andy Resnick]

<snip> a mathematics skills assessment administered at the beginning of the class term in Phyics 1 - Fundamental Mechanics, part of the engineering Physics course sequence. If student performs too low, then student is REQUIRED to enroll in a short - few weeks remedial course to bring up students basic intermediate algebra and basic trigonometry skills; <snip>

We've discussed adding a 'placement exam' to our College Physics (algebra-based) sequence as a remediation tool: currently, the prerequisite is (IIRC) 2 units of high school math: trig and geometry. This sounds reasonable, except that many of the students took these courses > 6 years prior to College Physics and don't recall any of that material. The idea is that a placement exam will identify the students who are at risk of under-performing in Physics due to the lack of required mathematical knowledge.

Unfortunately, adding a placement exam means changing the existing prerequisites, and so the proposed change must travel through the University curriculum modification system: this is not an insurmountable problem, but it will take a concerted effort over at least 1 year to steer the proposal through the various committees. None of us are willing (at this time) to put in that kind of effort.