Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Seeking advice on creating ratio scales from ordinal scale instruments

  1. Jun 21, 2014 #1
    I have a psychological testing instrument that produces an ordinal measure (0, 1, 2, 3, 4, 5). I want to change this to a ratio scale with range 0 to 5.

    The instrument is designed so the first 5 questions are very easy (Level 1), the next 5 questions are harder (Level 2), the next set is even harder (Level 3)... etc. A participant scores 0 if they cannot answer any questions. They reach the "Level" if they answer 4 out of 5 of any numbers at that level.

    To make this a ratio scale, I was thinking about scoring with the following points per problem at each level:

    Level 1: Each problem worth 1 point
    Level 2: Each problem worth 6 points
    Level 3: Each problem worth 36 points
    Level 4: Each problem worth 216 points
    Level 5: Each problem worth 1296 points

    I was going to take the sum, ##x##, and then calculate ##\log_6{x}##.

    I'm wondering if there are any other suggestions about how to do this.
     
  2. jcsd
  3. Jun 21, 2014 #2

    Simon Bridge

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    2016 Award

    There are infinite ways to turn thisinto a ratio scale.
    i.e. just score each question as shown ignoring the levels, add the scores and multiply by 5/<max possible score>.
    So you need to spell out the constraints on the problem - what is the proportional score supposed to represent? How will it be used?
     
  4. Jun 21, 2014 #3
    The constraint is that I want to keep this feature built into the instrument: "The instrument is designed so the first 5 questions are very easy (Level 1), the next 5 questions are harder (Level 2), the next set is even harder (Level 3)... etc. A participant scores 0 if they cannot answer any questions. They reach the "Level" if they answer 4 out of 5 of any numbers at that level."

    A level 1 question is more like a drawing of 3 circles and asking how many circles are there. A level 5 question is like creating a proof by contradiction with awareness of non-Euclidean mathematics. So there has to be a weighting on the problems.

    The scale needs to reflect the differences in the difficulty of the problem sets.
     
  5. Jun 21, 2014 #4

    Simon Bridge

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    2016 Award

    The scheme in post #2 reflects the difficulty of each level by giving a bigger score to the higher level questions.
    All I have basically said is there is no need to take a logarithm.

    The details of the weighting applied dependson what you want the result to tell you - what are you trying to acheive?

    After all, if all you want to do is preserve the "level" feature then you'd just be using the old system.
     
  6. Jun 22, 2014 #5
    I don't understand what you mean by "just score each question as shown ignoring the levels" (#2). If you added an image in that post, it is not showing.
     
  7. Jun 22, 2014 #6

    Simon Bridge

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    2016 Award

    Score as follows:
    Level 1: Each problem worth 1 point
    Level 2: Each problem worth 6 points
    Level 3: Each problem worth 36 points
    Level 4: Each problem worth 216 points
    Level 5: Each problem worth 1296 points

    add up the points for all correct answers, divide by 1555.
    This will give you a score out of 5, which is weighted according to the difficulty of the questions, as requested.
     
  8. Jun 23, 2014 #7
    Ok, that makes more sense. It might give me some normality issues, but it makes sense. thanks
     
    Last edited: Jun 23, 2014
  9. Jun 25, 2014 #8

    Simon Bridge

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    2016 Award

    If you made each question at level n worth 10n-1 points, then the raw score will also preserve the achievement level. Divide by 11111 to get an overall score out of 5.

    Overall, what you use depends on what you want to do with it.
    You will need to check the scoring approach against some data to see if the scores follow a normal distribution.
    Considering the way the questions are ranked, I would not expect it to anyway.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Seeking advice on creating ratio scales from ordinal scale instruments
  1. 4 balls and the scales (Replies: 25)

Loading...