Significant figures when taking measurements.

[opticaltempest]

I am taking measurements off the screen of an analog oscilloscope in order to calculate voltage and frequency of a sinusoidal wave form. The volts per division is set to 0.50V, and the time per division is set to 0.50 ms.

I measure the vertical length of the trace to be 2.67 divisions. I measure the horizontal length of the trace to be 0.87 divisions. I will assume that I can estimate my measurement to within 1/100 of a division.

Now I must do the following calculations to get the voltage and period of the wave form:

2.67 divisions * 0.50 V = 1.335 V
0.87 divisions * 0.50 ms = 0.435 ms

According to rules of significant figures, my answers should be rounded to 1.34 V and 0.44 ms. Why is the zero in 0.87 ignored as a significant figure? I know for a fact that my estimated measurement of 0.87 is not 1.87, not 2.87, not 3.87, etc. It doesn't make sense to me that I keep 3 significant figures for the measured voltage and only 2 significant figures for the measured period when both measurements are being measured using the same device-the division tick marks on the screen of the oscilloscope.

 

[Hootenanny]

You only acutally measured a your time period to two sig. figs. and hence you can only quote your final answer to such an accuracy. However, I would say that if these figures are to be used in further computations I would keep at least 5sf to ensure that any rounding errors are not significant.

 

[vanesch]

I am taking measurements off the screen of an analog oscilloscope in order to calculate voltage and frequency of a sinusoidal wave form. The volts per division is set to 0.50V, and the time per division is set to 0.50 ms.

I measure the vertical length of the trace to be 2.67 divisions. I measure the horizontal length of the trace to be 0.87 divisions. I will assume that I can estimate my measurement to within 1/100 of a division.

How many meters large was your oscilloscope screen ?

Honestly, to 1/100 of a division on an oscilloscope sounds strongly overestimated. 1/10 would already be a very good guess, I'd say.

 

[opticaltempest]

I know 1/100 of a division is unrealistic. I would also use 1/10. However, our lab instructions told us to assume that we could measure to within 1/100 of a division.

 

[AlephZero]

I am taking measurements off the screen of an analog oscilloscope in order to calculate voltage and frequency of a sinusoidal wave form. The volts per division is set to 0.50V, and the time per division is set to 0.50 ms.

I measure the vertical length of the trace to be 2.67 divisions. I measure the horizontal length of the trace to be 0.87 divisions. I will assume that I can estimate my measurement to within 1/100 of a division.

Now I must do the following calculations to get the voltage and period of the wave form:

2.67 divisions * 0.50 V = 1.335 V
0.87 divisions * 0.50 ms = 0.435 ms

According to rules of significant figures, my answers should be rounded to 1.34 V and 0.44 ms. Why is the zero in 0.87 ignored as a significant figure? I know for a fact that my estimated measurement of 0.87 is not 1.87, not 2.87, not 3.87, etc. It doesn't make sense to me that I keep 3 significant figures for the measured voltage and only 2 significant figures for the measured period when both measurements are being measured using the same device-the division tick marks on the screen of the oscilloscope.

0.87 means "between 0.865 and 0.875" so the time is between .4325 ms and .4375 ms. The "5" in your ".435" doesn't have any meaning.

2.67 means "between 2.665 and 2.675" so the voltage is between 1.3325 and 1.3375. The choice of whether to call that 1.33 or 1.34 can get into the realm of philosophy rather than mathematics or common sense.

A better approach is to keep track of the actual errors - so the reading of 0.87 +/- 0.005 would become 0.435 +/- 0.0025. Well, actually it would not, because your "0.5 ms/div" also has an error - the scope's manual will tell you what the error is supposed to be, and calibrating the scope against a frequency standard will tell you what the error really is.

 

[vanesch]

I know 1/100 of a division is unrealistic. I would also use 1/10. However, our lab instructions told us to assume that we could measure to within 1/100 of a division.

Ah. No discussion in that case (you might write in your report however that you find that irrealistic!).

Usually, for analog instruments, one takes 1 or half the finest indicated division.