Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

What makes a good question in your opinion?

  1. Apr 11, 2017 #21
    A question which provokes thoughts.
  2. Apr 11, 2017 #22
    I think it depends on the learning objectives of the course. I tend to design physics courses as vehicles to improve quantitative problem solving skills which include sequences of deliberate problem solving steps, unit analysis, and careful assessments.

    Cutesy problems where two competing effects exactly cancel out don't really give students the opportunities I like to practice and grow in those things, but rather depend on them seeing a trick of clever physical insight. Great question for courses designed more to build qualitative physical insight. But for my purposes, I guess it depends how the quantitative problem solving is balanced in the rest of the course.
  3. Apr 12, 2017 #23
    I don't quite follow. Because the explanation to the question requires an equation, i would argue that it is a balance of quantitative and qualitative reasoning (you are reading an equation, and using words to apply it to your scenario). I think you are arguing that quantitative reasoning requires the use of actual numbers, which are absent in my question. Then, would you argue that undergraduate classes become qualitative only at junior and senior year, because they are almost entirely derivation based?

    And, i don't quite understand why it is a cutesy problem. AP Physics 1+2 are now heavily reliant on these types of questions, and students are doing awfully on these exams. My experience with teachers is that they haven't changed their practices much after the AP change. They may teach slightly different material, but they are generally using the same quantitative-only methods. If these problems require clever physical insight (which i think is a great thing to have), then they clearly aren't getting that insight using the methods currently in use.
  4. Apr 12, 2017 #24


    User Avatar
    Homework Helper
    Education Advisor
    Gold Member

    All this "cutesy" and qualitative stuff has not been explained very clearly. There was once a professor who gave instructions on a test once, " Assign variables to ALL numbers, and solve each problem purely in variables - NO substitution of any actual numeric values". That person was a Physics professor.
  5. Apr 12, 2017 #25
    If you think an equation is required, perhaps you are reading

    "A person in a boat floating in a small pond throws an anchor overboard. What happens to the water level of the pond? Defend your response."

    differently from me. There is no explicit requirement for an equation in the statement of the problem, so my expectation from years of teaching is that most students I've had would be unlikely to give one.

    Quantitative thinking does not always require the use of actual numbers, but if you are thinking that the above problem requires the use of an equation, it is a BAD question, because the above question is guilty of ambiguity. A better question would specify how the response should be defended: "Defend your answer by quantifying any change in the pond's water level in terms applicable quantities: the density of water, the density of the anchor, etc."
  6. Apr 12, 2017 #26
    I would never make a determination regarding whether the teacher or the students are more at fault in poor exam performances without knowing how much of the reading and homework assignments the students are doing. Teachers cannot and should not shoulder the blame for poor exam performance when students who are not doing the assigned reading and homework exercises score poorly on the exams.

    I am also not a big fan of AP courses. I've known too many cases where students perform very poorly in downstream courses having managed to get through the AP courses without learning (or perhaps not retaining) very much. I'm not sure what you mean by "quantitative only" methods. Most quantitative homework and exam problems require a lot of qualitative thinking to draw the right picture, identify the right physical principles involved, and assess the reasonableness of the numerical answer and the units. Sure, students may be tempted to approach problems with formula roulette, but it's part of a teacher's job to steer them away from that and to assign questions that do not reward that approach.
  7. Apr 12, 2017 #27

    Andy Resnick

    User Avatar
    Science Advisor
    Education Advisor

    Anyone can nit-pick any question ad infinitum: most of my students don't know how to drive a car and/or have never been on a roller coaster; I suppose that puts them at a disadvantage for those types of questions.

    Since my central point is that 'good' questions are thought-provoking and lead to open-ended discussion, I'm glad that you are able to further explore this question!
  8. Apr 12, 2017 #28
    OK, that's a fine approach for some courses.

    Knowing that many downstream courses require lots of practice with numbers and units, this is not an approach I would take in introductory courses I've taught. Many students in intro courses are still coming up the learning curve with things like orders of magnitude, reasonable accelerations and velocities, significant digits, and things that are better practiced by substituting in actual numeric values. Taking that approach would leave them under-prepared for downstream science and engineering courses.

    In these courses, my approach has tended to be to emphasize solving problems in terms of variables as far as possible, and then substituting in numbers and units at the end - performing the required math operations on the units as well as the numbers to use whether the units obtained agree with the expectations as an additional assessment of the reasonableness of the final expression before substituting in numbers.
  9. Apr 12, 2017 #29


    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor

    In all of this, there appears to be a lack of appreciation of the type of students one is teaching to.

    I'm teaching two very different groups of students: (i) science/engineering majors and (ii) non-STEM majors.

    It will be very silly to teach the same thing and the same way to this very different groups of students, not just because they have different preparations, but also because they have different set of interests. That is why I stated that the original question stated by the OP may be inadequate for group (i) but highly appropriate for group (ii).

    But even "bad" questions can easily be a learning point. I consider it a bonus if a student brings up an ambiguity in a question in which he/she could come up with a different scenario, interpretation, or even answer. Only someone who has either an understanding of the concept involved, or someone who has made an effort to tackle the problem, will be able to discover this, and either one of those is very encouraging. How many times have we ourselves in this forum found someone asking a question which the member thought was "clear" or "simple", and yet, we found to be vague, confusing, or can have varying interpretation? Our background and expertise allow us to see the bigger picture and all the possible alternatives.

    There is no such thing as a perfect question. As Andy has stated, we can always nitpick something about any question being asked. There will always be some unintended inherent bias in many physics questions, because we often have to appeal or refer to something "familiar", which may not be familiar to others. I once tried to find something to replace "projectile motion" as popular example of basic 2D motion in intro physics, because many female students told me that they don't usually play with "cannons" or throw balls up in the air. However, a female professor cautioned me that if I bring in a more "feminine" examples, I may be stereotyping the female students. That essentially put a stop to that.

    So again, I see no issue with the original question of this thread. I'd love to see how a student would tackle it and see his/her line of thought.

  10. Apr 12, 2017 #30
    I think it depends on how you model answering the question. It could be argued that there is no explicit requirement for an equation for the question posed in the OP. I could expect some students trying to pull a fast one with "Really tall." Both questions relate answers to quantifiable things (height of the tower or water level). If i model both types of questions (one with defined values, and the other without) using equations, i would definitely expect of students to answer these types of questions using an equation.

    I would be hesitant to use your suggestion, unless i snip it off at 'applicable quantities.'
  11. Apr 12, 2017 #31
    I apologize. Upon moving to a new country, i have had a lot of students truly struggle with some of those types of questions, and it has made me more cautious in question writing.
  12. Apr 12, 2017 #32
    Well, i wouldn't want to put the entire blame on the teacher as well, but i don't think the students would suddenly jump in blame over the course of one year. There was a clear drop in performance after the change, and still has a lower achievement rate (even considering the change to scoring) now, compared to AP B years. A huge part of this drop is the quick shift in expectations.

    Nonetheless, a lot of teachers i have discussed these changes with do not care for and struggle implementing qualitative responses. The textbook may use words to explain the physics. The teacher may use words to explain the physics. Why is the student restricted from using words to explain physics? You may argue this is not the case, but student work suggests otherwise. You may see a picture; it would be rare to see a written explanation of the physical insight to solving the problem; either way it becomes all math from there. You rarely see a written explanation after a numerical solution.

    What i am asking for is what qualities of a question would better generate these types of responses. Ones where emphasis is placed on physical insight. The question in the OP has, in my opinion, low physical insight, and hence why i don't consider it to be a good question.

    And, by no means am i suggesting the abolishment of math in physics (i say this because a lot of teachers and professors believe that conceptual physics is physics without math). I am arguing for the inclusion of more qualitative explanations in addition to that math.
  13. Apr 12, 2017 #33
    It seems like you are wanting to solve with questions issues that can be addressed with a grading rubric.

    Both me and several departments I know awarded most points on physics problems based on factors other than the mathematical solution.

    20% was for drawing and labeling a picture or diagram
    20% was for identifying the important physical principle (Conservation of Energy, for example, or Newton's 2nd law)
    20% was for writing down an orderly sequence of steps to solve the problem
    20% was for the numerical solution
    20% was for a written assessment for whether and why the numerical solution was correct

    This grading rubric awards 80% of the points for stuff on the paper other than math.
  14. Apr 12, 2017 #34


    User Avatar
    Homework Helper
    Education Advisor
    Gold Member

    Most of those "20%" items ARE the Mathematics; but this is a matter of interpretation.
  15. Apr 12, 2017 #35
    I have used a similar rubric for grading in the past. However, i did not include the last step, as you have it. How would you model the written assessment of why the numerical solution is correct?

    I have abandoned grading this way because i am trying to expand on the student's giving me sufficient physical insight. I think using Newton's 2nd Law does not satisfy that.
  16. Apr 12, 2017 #36
    I think having students take limiting cases could be one way for a student to rationalize why there solution is correct/reasonable. As for numerical answers: order of magnitude, comparison to real world scenarios, etc. are other ways of rationalizing that the numerical answer makes sense. I took a class in undergrad where the instructor had us make an explanation for whether our answer was reasonable or not. From personal experience it is very difficult to guide students into developing this intuition. However, it's not an easy thing to develop in the first place either.
  17. Apr 13, 2017 #37

    Andy Resnick

    User Avatar
    Science Advisor
    Education Advisor

    Being cautious is good- it means you are being thoughtful! Slightly off-topic, but when my students ask for study tips, I often suggest they invent 'test-like' questions because of the mental effort involved. Those that do often remark how effective that strategy is.
  18. Apr 13, 2017 #38
    That depends on the topic at hand. I usually make significant efforts to teach topic-appropriate assessment methods throughout the course, so that students have ample instruction and lots of practice. But in general, I emphasize that good assessments have three components: a double check on magnitude of the number, the direction (or sign), and the units. Units tend to be similar across topics. A student should do the math on the units when they substitute numbers and units of quantities into their final symbolic expression. An assessment on units can be as simple as "the units of the answer are as expected for an acceleration, m/s/s."

    If the answer is a vector, it might be a good assessment of direction to say, "the direction of the acceleration is the same as that of the net force." Or if it is a scalar, "It makes sense that the final velocity is negative, because the ball is falling at the end, and the positive direction was defined to be upward."

    Assessing the numerical magnitude tends to be more specific to the topic. But when working with Atwood machines and objects sliding or rolling down inclined planes with gravity as the only external force, I point out that the magnitude of right answers is always between 0 and 9.8 m/s/s. Numbers above 9.8 m/s/s in these kinds of problems need a lot of extra scrutiny and are probably wrong.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted