Calculating Signal to Noise Ratio for Multiple Choice Exams

In summary: What exactly are you looking to measure?In summary, you can find the signal to noise ratio by measuring the output when there is no signal and comparing it to the output when there is a signal.
  • #1
curiousasker
9
0
I've been trying to figure out how to get the signal to noise ratio of a completely multiple choice type exam. I can't do it and it's driving me nuts, so finally I found this website and decided I'd ask. :D

Say you let your students take a multiple choice test and you are able to get all their test answer sheets. How would you go about finding the signal to noise ratio, where the signal is the correct answer?
 
Physics news on Phys.org
  • #2
curiousasker said:
I've been trying to figure out how to get the signal to noise ratio of a completely multiple choice type exam. I can't do it and it's driving me nuts, so finally I found this website and decided I'd ask. :D

Say you let your students take a multiple choice test and you are able to get all their test answer sheets. How would you go about finding the signal to noise ratio, where the signal is the correct answer?

There are a number of ways to express the signal to noise ratio in science and engineering depending on the specifics. In your case it would seem straightforward. If a correct answer is a signal and a wrong answer is noise and there are no other possibilities then the raw signal to noise ratio is [itex]SNR = \frac{signal}{noise} = \frac{n (correct.answers)}{n(incorrect.answers)}[/itex]. Sometimes the ratio is expressed logarithmically. It depends on what you want to do.
 
Last edited:
  • #3
So it's simply the average of the test scores divided by the average of the part they got wrong? Isn't noise the variance of the signal, rather than the incorrect answers?

The way I understand signal to noise ratio is through the gauge r&r, but that requires students to retake the same test, and that both test takings be independent events. It's not possible to have that since their second take will definitely be affected by their first take. So I'm not quite catching how to get SNR without repeat and independent trials.

It puzzles me because if a 40 item test is perfected by everybody, the mean of the correct answers is 40 and the variance is zero, so the SNR is infinite. In the same way, if it was the incorrect answers, the noise would still be zero and the SNR would be infinite.

Also I know that SNR is expressed in decibels most of the time, but what would the SNR in this particular case be expressed as?
 
  • #4
curiousasker said:
So it's simply the average of the test scores divided by the average of the part they got wrong? Isn't noise the variance of the signal, rather than the incorrect answers?

The way I understand signal to noise ratio is through the gauge r&r, but that requires students to retake the same test, and that both test takings be independent events. It's not possible to have that since their second take will definitely be affected by their first take. So I'm not quite catching how to get SNR without repeat and independent trials.

It puzzles me because if a 40 item test is perfected by everybody, the mean of the correct answers is 40 and the variance is zero, so the SNR is infinite. In the same way, if it was the incorrect answers, the noise would still be zero and the SNR would be infinite.

Also I know that SNR is expressed in decibels most of the time, but what would the SNR in this particular case be expressed as?

Frankly I've never heard of the SNR being applied this way. For a voltage dependent signal a standard formula would be [itex]SNR = 20_{log 10} (V_s/V_n)[/itex]. Note it is still a simple ratio with a logarithmic conversion to suit the particular situation. You could also express the SNR in terms of the mean and standard deviation of a signal. In this case that would be [itex] SNR = \mu/ \sigma[/itex].

I can't tell you how to best express it for your purposes other than what I've already done because I don't know your objectives. And yes, the signal to noise ratio can be infinite but almost never is in situations where it is usually applied. Perhaps someone else will assist you.
 
Last edited:
  • #5
Hi. Thanks for the quick response :)

What I'm trying to do is something like in this article here (under Discrete Stimuli). It's basically how to get how strong is the signal over the noise in an experimental set up? I don't quite understand the article so I'm looking for help (and I've only just stumbled upon it too).

Unfortunately, the test method is a discrete case, so finding a way to measure by the usual decibels or voltage is a little difficult for me.
 
  • #6
As far as I understand it, to determine the SNR you need to be able to compare the output when there's no signal with the output when there is a signal.
Moreover, I would have thought that in the context of multiple choice questions the signal was any answer the student actually believed and the noise was the random guesses; can't see how you'd distinguish those in general, but you might be able to put a limit on it. E.g. If it's a five-way choice each time and the student scores 20% it could well be all noise, whereas if the student scores 90% then it's unlikely to be much more than 13% noise.
 
  • #7
Hi haruspex. I figured it out. I'll say it here since it might be interesting to you guys =)

It's signal detection theory that should be used in this case, because it can identify hits, false alarms, misses, and correct rejections (basically signal present-response present; signal absent-response present; signal present-response absent; and signal absent-response absent).

But since it's a multiple choice, they always have to answer (so it's always signal present). What I used here was a k-alternative forced choice analysis instead. What that does is basically determine how well people can distinguish the signal from k-1 noises (ie, wrong choices). It calculates the distance d' from the mean of the correct choice and the mean of the incorrect choices, which says something about their discrimination ability.

Fortunately, (d')^2 = SNR for this kind of test (when there are only two response categories = {correct,incorrect}).

So that's how I got the SNR for a multiple choice test. It even gives the answer in terms of a linear power ratio, not decibels, which is what I need for my purpose =)
 

1. What is "Signal to Noise ratio"?

The "Signal to Noise ratio" (SNR) is a measure of the strength of a signal compared to the level of background noise. It is used in various fields, including telecommunications, electronics, and science, to determine the quality and reliability of a signal.

2. Why is Signal to Noise ratio important?

SNR is important because it helps determine the accuracy and reliability of a signal. A high SNR indicates a strong signal, which means that the information being transmitted is less likely to be corrupted by noise. A low SNR, on the other hand, can result in errors and decrease the quality of the signal.

3. How is Signal to Noise ratio calculated?

SNR is calculated by dividing the power of the signal by the power of the noise. In other words, it is the ratio of the signal amplitude to the noise amplitude. The higher the resulting value, the better the quality of the signal.

4. What factors affect Signal to Noise ratio?

Several factors can affect SNR, including the distance between the transmitter and receiver, the type of transmission medium, the amount of interference, and the quality of the equipment used to capture and process the signal. Additionally, increasing the signal's power and reducing the noise level can improve the SNR.

5. How can Signal to Noise ratio be improved?

To improve SNR, you can increase the signal's power, use better quality equipment, or reduce the amount of noise in the environment. Another way is to use signal processing techniques, such as filtering and modulation, to enhance the signal and reduce the impact of noise. Additionally, using error correction codes can help improve SNR by detecting and correcting errors in the signal.

Similar threads

  • General Engineering
Replies
4
Views
2K
Replies
5
Views
387
  • Engineering and Comp Sci Homework Help
Replies
9
Views
2K
  • Introductory Physics Homework Help
Replies
7
Views
983
  • Engineering and Comp Sci Homework Help
Replies
4
Views
1K
Replies
9
Views
2K
Replies
9
Views
1K
  • Electrical Engineering
Replies
3
Views
761
Replies
37
Views
3K
Replies
4
Views
2K
Back
Top