# Inconsistency in Significant Digits

• RobbieM.
In summary, the significant figures do not allow for precise reporting of measurements, resulting in rounding and potential inaccuracy. This is due to the limitations of significant figures and quantization error.
RobbieM.
A question posed as an example: converting 1.55 and 0.55 inches to cm.

1.55 * 2.5400000000... = 3.937 which rounds to 3.94 cm per the rules

and

0.55 * 2.5400000000... = 1.397 which rounds to 1.4 cm per the rulesIf both inputs are known to the same precision (hundredth of an inch), why are the outputs not known to the same precision (tenth or hundredth of a cm)?

RobbieM. said:
A question posed as an example: converting 1.55 and 0.55 inches to cm.

1.55 * 2.5400000000... = 3.937 which rounds to 3.94 cm per the rules

and

0.55 * 2.5400000000... = 1.397 which rounds to 1.4 cm per the rulesIf both inputs are known to the same precision (hundredth of an inch), why are the outputs not known to the same precision (tenth or hundredth of a cm)?
Who says they are NOT known to the same precision? I'm not talking about what you choose to round them to, I'm talking about precision (which can only be accurately expressed as a percent of the measurement regardless of units)

sophiecentaur
phinds said:
Who says they are NOT known to the same precision? I'm not talking about what you choose to round them to, I'm talking about precision (which can only be accurately expressed as a percent of the measurement regardless of units)
Thanks for the response.

Why does precision need to be expressed as a percentage of a measurement? And, if that is the case, shouldn't the larger measurement have a larger absolute uncertainty? That doesn't seem to be consistent with the result above.

My question is just about the rules for significant figures. I am converting some drawing dimensions from inch to cm and it was curious to me that the converted values (shown above) ended up being reported to different absolute numerical precision (e.g. they were both known to the hundredth of an inch and after conversion were reported to the hundredth and tenth of a cm, respectively).

Another example of weird behavior using sig. fig. rules:
Converting the length range 0.471-in -- 0.474-in to cm gives the result 1.20 cm -- 1.20 cm. This is because 0.001-in is 0.003 cm, a level of precision truncated away if you use standard rules.

RobbieM. said:
Another example of weird behavior using sig. fig. rules:
Converting the length range 0.471-in -- 0.474-in to cm gives the result 1.20 cm -- 1.20 cm. This is because 0.001-in is 0.003 cm, a level of precision truncated away if you use standard rules.

The conversion factor for cm per inch is exact, meaning that you don't use the standard rules for sig figs if you are merely converting a known number (not a measured value). So 0.474 inches (exact) converts to 1.20396 cm (exact). If you have a measurement of 0.474 in, accurate to a thousandth of an inch, then yes, you would need to worry about sig figs when converting.

Drakkith said:
The conversion factor for cm per inch is exact, meaning that you don't use the standard rules for sig figs if you are merely converting a known number (not a measured value). So 0.474 inches (exact) converts to 1.20396 cm (exact). If you have a measurement of 0.474 in, accurate to a thousandth of an inch, then yes, you would need to worry about sig figs when converting.
0.471 and 0.474 are not exact. They should be read as accurate to the thousandths place. So, I think you are saying the sig. fig. rules should be applied. In that case, the result given above is given... which I find confusing.

RobbieM. said:
Thanks for the response.

Why does precision need to be expressed as a percentage of a measurement? And, if that is the case, shouldn't the larger measurement have a larger absolute uncertainty? That doesn't seem to be consistent with the result above .
Actually, I was carelessly mixing accuracy with precision. Very imprecise of me and not very accurate either.

Precision is not measured as a percentage, it is absolute. If I measure an 8 foot long piece of lumber to a precision of 1/32 of an inch precision and 1/16 of an in accuracy, and then I measure a 1" long piece of wood to a precision of 1/32 of an inch and an accuracy of 1/16 of an inch the both measurements are precise to 1/32 and while you can certainly say that they are both accurate to 1/16th, that doesn't really tell you much. 1/16th on a 8 foot board is pretty good and 1/16th on a 1" piece is really awful. That is, the 8 foot measurement is accurate to within ((1/16) / (8 x 12) x 100 % = .065% The one-inch measurement on the other had is accurate to (( 1/16) / 1) x 100 % = 6.25% which is worse by a factor of about 100

RobbieM. said:
A question posed as an example: converting 1.55 and 0.55 inches to cm.

1.55 * 2.5400000000... = 3.937 which rounds to 3.94 cm per the rules

and

0.55 * 2.5400000000... = 1.397 which rounds to 1.4 cm per the rules

If both inputs are known to the same precision (hundredth of an inch), why are the outputs not known to the same precision (tenth or hundredth of a cm)?
Significant figures do not offer many options for the level of precision asserted in a reported quantity. You can assert a error bound of 0.5 units, 0.05 units, 5 units, 50 units or any other bound that is some power of ten times those. You cannot assert an error bound of 0.3 units. You can assert an error bound of 1 unit.

That means that when it comes time to report a measurement using significant figures you are either going to over-report your accuracy (going to the next higher error bound) or under-report it (going to the next lower error bound). That's quantization error.

You should know that a 3 sig fig measurement where the first digit is large has a smaller relative error bound than a 3 sig fig measurement where the first digit is small. In the case at hand we have a 3 sig fig measurement where the first digit is 1. That's less precision than "3 sig figs" would suggest. And we have a 2 sig fig measurement where the first digit is 5. That's slightly more precision than "2 sig figs" would suggest. Now you multiply those numbers by 2.54.

The multiplication pushes the 1.55 up to 3.94. That's in the high 3 sig fig range. You are now over-reporting the precision of that result.

The multiplication pushes the 0.55 up to 1.4. That's in the low 2 sig fig range. You are now under-reporting the precision of the result.

That's quantization error for you.

If you want to get rid of the quantization error in the error bounds, don't use significant figures. If you're willing to tolerate being out by a factor of ##\sqrt{10}## high or low in your error bounds, then significant figures may be adequate for your purposes.

Thanks for the reply briggs. I'm still trying to digest that.

It's interesting that the mid-point between 0.471 and 0.474 is 0.4725 inches = 1.200 cm. Makes me wonder if the original spec was 1.200 +or- 0.004 cm. Some judgement is required in many of these conversion exercises.

## What is inconsistency in significant digits?

Inconsistency in significant digits refers to the situation where the number of significant digits used to represent a value or measurement varies within the same calculation or set of data.

## Why is inconsistency in significant digits a concern for scientists?

Inconsistency in significant digits can lead to errors in calculations and can affect the accuracy and precision of scientific data. It can also make it difficult to compare and analyze data from different sources.

## How can inconsistency in significant digits be avoided?

To avoid inconsistency in significant digits, scientists must follow the rules of significant figures and ensure that all values and calculations are reported with the appropriate number of significant digits. This involves rounding off numbers and maintaining consistency throughout all calculations.

## What are the potential consequences of ignoring inconsistency in significant digits?

Ignoring inconsistency in significant digits can lead to incorrect results and misinterpretation of data. It can also undermine the reliability and validity of scientific findings.

## Can inconsistency in significant digits ever be acceptable?

In some cases, small variations in significant digits may be acceptable, particularly when dealing with very large or very small numbers. However, it is important to maintain consistency and follow established rules when reporting scientific data.

• General Math
Replies
12
Views
2K
• Precalculus Mathematics Homework Help
Replies
5
Views
2K
• Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
• Classical Physics
Replies
18
Views
2K
• Computing and Technology
Replies
9
Views
2K
• Astronomy and Astrophysics
Replies
2
Views
1K
• Electrical Engineering
Replies
7
Views
3K
• Introductory Physics Homework Help
Replies
4
Views
3K
• Classical Physics
Replies
18
Views
2K
• Precalculus Mathematics Homework Help
Replies
5
Views
4K