0.471 and 0.474 are not exact. They should be read as accurate to the thousandths place. So, I think you are saying the sig. fig. rules should be applied. In that case, the result given above is given... which I find confusing.
Another example of weird behavior using sig. fig. rules:
Converting the length range 0.471-in -- 0.474-in to cm gives the result 1.20 cm -- 1.20 cm. This is because 0.001-in is 0.003 cm, a level of precision truncated away if you use standard rules.
Thanks for the response.
Why does precision need to be expressed as a percentage of a measurement? And, if that is the case, shouldn't the larger measurement have a larger absolute uncertainty? That doesn't seem to be consistent with the result above.
My question is just about the rules for...
A question posed as an example: converting 1.55 and 0.55 inches to cm.
1.55 * 2.5400000000... = 3.937 which rounds to 3.94 cm per the rules
and
0.55 * 2.5400000000... = 1.397 which rounds to 1.4 cm per the rulesIf both inputs are known to the same precision (hundredth of an inch), why are the...
I need some help finding an appropriate statistics model for some experimental data. Thanks in advance for any suggestions.
I am trying to compare simulated results from a code that models nuclide concentrations in spent nuclear fuel to experimental data. These concentrations have...
I have two histograms that I would like to compare quantitatively. The values of the first histogram have respective relative errors for each bin. The second histogram has no statistical uncertainty.
I could compute probabilities for each bin that the exact values would fall into a given...