Inconsistency in Significant Digits

[RobbieM.]

A question posed as an example: converting 1.55 and 0.55 inches to cm.

1.55 * 2.5400000000... = 3.937 which rounds to 3.94 cm per the rules

and

0.55 * 2.5400000000... = 1.397 which rounds to 1.4 cm per the rulesIf both inputs are known to the same precision (hundredth of an inch), why are the outputs not known to the same precision (tenth or hundredth of a cm)?

 

[phinds]

A question posed as an example: converting 1.55 and 0.55 inches to cm.

1.55 * 2.5400000000... = 3.937 which rounds to 3.94 cm per the rules

and

0.55 * 2.5400000000... = 1.397 which rounds to 1.4 cm per the rulesIf both inputs are known to the same precision (hundredth of an inch), why are the outputs not known to the same precision (tenth or hundredth of a cm)?

Who says they are NOT known to the same precision? I'm not talking about what you choose to round them to, I'm talking about precision (which can only be accurately expressed as a percent of the measurement regardless of units)

 

[RobbieM.]

Who says they are NOT known to the same precision? I'm not talking about what you choose to round them to, I'm talking about precision (which can only be accurately expressed as a percent of the measurement regardless of units)

Thanks for the response.

Why does precision need to be expressed as a percentage of a measurement? And, if that is the case, shouldn't the larger measurement have a larger absolute uncertainty? That doesn't seem to be consistent with the result above.

My question is just about the rules for significant figures. I am converting some drawing dimensions from inch to cm and it was curious to me that the converted values (shown above) ended up being reported to different absolute numerical precision (e.g. they were both known to the hundredth of an inch and after conversion were reported to the hundredth and tenth of a cm, respectively).

 

[RobbieM.]

Another example of weird behavior using sig. fig. rules:
Converting the length range 0.471-in -- 0.474-in to cm gives the result 1.20 cm -- 1.20 cm. This is because 0.001-in is 0.003 cm, a level of precision truncated away if you use standard rules.

 

[Drakkith]

Another example of weird behavior using sig. fig. rules:
Converting the length range 0.471-in -- 0.474-in to cm gives the result 1.20 cm -- 1.20 cm. This is because 0.001-in is 0.003 cm, a level of precision truncated away if you use standard rules.

The conversion factor for cm per inch is exact, meaning that you don't use the standard rules for sig figs if you are merely converting a known number (not a measured value). So 0.474 inches (exact) converts to 1.20396 cm (exact). If you have a measurement of 0.474 in, accurate to a thousandth of an inch, then yes, you would need to worry about sig figs when converting.

 

[RobbieM.]

The conversion factor for cm per inch is exact, meaning that you don't use the standard rules for sig figs if you are merely converting a known number (not a measured value). So 0.474 inches (exact) converts to 1.20396 cm (exact). If you have a measurement of 0.474 in, accurate to a thousandth of an inch, then yes, you would need to worry about sig figs when converting.

0.471 and 0.474 are not exact. They should be read as accurate to the thousandths place. So, I think you are saying the sig. fig. rules should be applied. In that case, the result given above is given... which I find confusing.

 

[phinds]

Thanks for the response.

Why does precision need to be expressed as a percentage of a measurement? And, if that is the case, shouldn't the larger measurement have a larger absolute uncertainty? That doesn't seem to be consistent with the result above .

Actually, I was carelessly mixing accuracy with precision. Very imprecise of me and not very accurate either.

Precision is not measured as a percentage, it is absolute. If I measure an 8 foot long piece of lumber to a precision of 1/32 of an inch precision and 1/16 of an in accuracy, and then I measure a 1" long piece of wood to a precision of 1/32 of an inch and an accuracy of 1/16 of an inch the both measurements are precise to 1/32 and while you can certainly say that they are both accurate to 1/16th, that doesn't really tell you much. 1/16th on a 8 foot board is pretty good and 1/16th on a 1" piece is really awful. That is, the 8 foot measurement is accurate to within ((1/16) / (8 x 12) x 100 % = .065% The one-inch measurement on the other had is accurate to (( 1/16) / 1) x 100 % = 6.25% which is worse by a factor of about 100

 

[jbriggs444]

A question posed as an example: converting 1.55 and 0.55 inches to cm.

1.55 * 2.5400000000... = 3.937 which rounds to 3.94 cm per the rules

and

0.55 * 2.5400000000... = 1.397 which rounds to 1.4 cm per the rules

If both inputs are known to the same precision (hundredth of an inch), why are the outputs not known to the same precision (tenth or hundredth of a cm)?

Significant figures do not offer many options for the level of precision asserted in a reported quantity. You can assert a error bound of 0.5 units, 0.05 units, 5 units, 50 units or any other bound that is some power of ten times those. You cannot assert an error bound of 0.3 units. You can assert an error bound of 1 unit.

That means that when it comes time to report a measurement using significant figures you are either going to over-report your accuracy (going to the next higher error bound) or under-report it (going to the next lower error bound). That's quantization error.

You should know that a 3 sig fig measurement where the first digit is large has a smaller relative error bound than a 3 sig fig measurement where the first digit is small. In the case at hand we have a 3 sig fig measurement where the first digit is 1. That's less precision than "3 sig figs" would suggest. And we have a 2 sig fig measurement where the first digit is 5. That's slightly more precision than "2 sig figs" would suggest. Now you multiply those numbers by 2.54.

The multiplication pushes the 1.55 up to 3.94. That's in the high 3 sig fig range. You are now over-reporting the precision of that result.

The multiplication pushes the 0.55 up to 1.4. That's in the low 2 sig fig range. You are now under-reporting the precision of the result.

That's quantization error for you.

If you want to get rid of the quantization error in the error bounds, don't use significant figures. If you're willing to tolerate being out by a factor of $\sqrt{10}$ high or low in your error bounds, then significant figures may be adequate for your purposes.

 

[RobbieM.]

Thanks for the reply briggs. I'm still trying to digest that.

 

[insightful]

It's interesting that the mid-point between 0.471 and 0.474 is 0.4725 inches = 1.200 cm. Makes me wonder if the original spec was 1.200 +or- 0.004 cm. Some judgement is required in many of these conversion exercises.