Increasing the number of Questions on the exams and the effects of it

[Arman777]

One of our classes in the Uni, we have only 3 questions in the exam and we have a curve-grade system. I wonder increasing the question number would have any affects on the curve or etc. Is there any study on this topic ?

 

[Vanadium 50]

If you have more questions, don't you need more time?

 

[Arman777]

If you have more questions, don't you need more time?

Well yes

 

[Vanadium 50]

And where is that time going to come from?

 

[Dr_Nate]

Hi Arman, this is a good question. Professors in research intensive universities are not hired due to their abilities to educate others. In my experience they generally don't consider what skills they want their students to demonstrate beyond "doing the problem". You can see how V-50 doesn't see beyond this either.

In high school curricula and for standardized tests, there are lists of standards that breakdown the skills. It's not so difficult to test many complex skills with multiple choice question and free response questions. For example, just check out the AP Physics exams.

A good teacher could take your 3 questions and make a test of many questions by turning the steps into skill checks. You could also keep one question whole to differentiate between students that have the skills and those students that can use those skills together unprompted.

 

[Vanadium 50]

You can see how V-50 doesn't see beyond this either.

I think that's unfair.

I didn't say anything about who R1s should be hiring or that R1s don't consider educational outcomes. I did say that if you want to have (e.g.) a three-hour test in a class that meets for one hour, you need to figure out how this would work.

 

[Dr_Nate]

I think that's unfair.

I didn't say anything about who R1s should be hiring or that R1s don't consider educational outcomes. I did say that if you want to have (e.g.) a three-hour test in a class that meets for one hour, you need to figure out how this would work.

I meant that sentence to only refer to your questions about time. My perception from your questions was that you didn't see how they could keep the same exam time and get in more questions. My apologies in any case if I misrepresented what you were saying.

 

[Vanadium 50]

My perception from your questions was that you didn't see how they could keep the same exam time and get in more questions.

Yes, that's correct. But it has nothing to do with R1's. If it takes an hour for a student to do three problems, it takes two for the student to do six. Where does this hour come from? (And the answer I am pointing to is "it will have to come out of instruction")

 

[vela]

A good teacher could take your 3 questions and make a test of many questions by turning the steps into skill checks.

But that also changes the nature of the test. It might be acceptable at the high school level to expect students to only have to identify the right formula to use and plug the numbers into see which multiple-choice answer is correct, but at the college level, most instructors likely expect students to achieve a higher level of understanding. Being able to execute the steps is less important than being able to determine the steps needed to solve the problem.

 

[Dr_Nate]

But that also changes the nature of the test. It might be acceptable at the high school level to expect students to only have to identify the right formula to use and plug the numbers into see which multiple-choice answer is correct, but at the college level, most instructors likely expect students to achieve a higher level of understanding.

Multiple choice question can be more powerful than testing plug-and-chug skills. But the utility of MC questions is that they can rapidly test many skills quickly. One should augment that with free-response questions to make sure they can bring it all together.

Check out the AP Physics test on pages 18 - 39 in this pdf to see how much material it is possible to test: https://apcentral.collegeboard.org/pdf/ap-physics-c-mechanics-practice-exam-2012.pdf

I emphasize the word possible because I don't think it's practical for a university professor to achieve this level. The college board has an army of people creating these tests. I suggest these tests could serve to inspire us to make better tests.

Being able to execute the steps is less important than being able to determine the steps needed to solve the problem.

I would agree that sometimes the steps are much less important. For example, why make a student chug through manipulating a series of trig identities or vector identities if that isn't the focus of the class? They could make a simple mistake and not make progress on the part on the problem. Moreover, that time could be better spent testing on the actual focus of the class.

If being able to determine the steps needed is more important, then we could ask students to simply outline a solution. Much time would be saved on the exam that now can be used to test other skills.

 

[Vanadium 50]

Writing multiple choice tests is hard. The hard part is writing good distractors:

Choose the best answer:
How tall am I?
A. 2 feet
B. 4 feet
C. 6 feet
D. 8 feet
E. 10 feet

Even without having met me, most people will get the right answer. (98% of the US male population is between 5 and 7 feet tall.)

The more information required in answering a multiple-choice test, the more likely it will be regarded as a trick question by the students, and the more likely the more legalistically-minded of them will be pounding on the department head's door.

I had a high school chemistry teacher write some excellent tests. You walked out knowing more than you went in with. But they were hard, and this made her unpopular. I doubt she would be able to teach this way today. One kind of question is she would give two statements, I and II, and the choices were:

A. I and II are true and I causes II
B. I and II are true and II causes I
C. I and II are true and neither causes the other
D. I is true and II is false
E. II is true and I is false
F. Both are false