Why do 0.1 and 1/10 work but 0.3-0.2 doesn't?

[SamRoss]

I understand that some numbers cannot be represented in floating point but I don't understand why the way we obtain those numbers matters. In python, if I type print(0.1) or print(1/10) I get 0.1 but if I type print(0.3-0.2) I get 0.09999999... Why the difference?

 

[Drakkith]

You can't write 0.3 in binary without an infinite number of digits, much like how you can't write 1/3 in decimal without also using an infinite number of digits. So you won't get an exact answer when you do the math in a computer.

0.3 in binary: 0.01001100110011...
Using floating point, this is truncated at some point and we wind up with a number very slightly less than 0.3. So you're actually doing something more like 0.29999999 - 0.2 instead of 0.3 - 0.2.

Edit: Also, turns out that 0.2 itself is also not representable in base 2 with a finite number of digits. 0.2 in binary: 0.00110011001100110011001100110011...

 

[SamRoss]

I get that, but I know that 0.1 also cannot be represented in binary so why does the computer NOT have a problem with doing 1/10?

 

[Drakkith]

Good question. I should have checked all three numbers before answering. Perhaps it's because you aren't doing math with 0.1 in your examples above? I'm not sure to be honest.

 

[SamRoss]

Sounds reasonable.

 

[willem2]

Good question. I should have checked all three numbers before answering. Perhaps it's because you aren't doing math with 0.1 in your examples above? I'm not sure to be honest.

The main reason is loss of precision when subtracting big numbers to get a small number.

0.1 is stored as: 1.1001100110011001100110011001100110011001100110011010 * 2^(-4)
0.2 is stored as: 1.1001100110011001100110011001100110011001100110011010 * 2^(-3)
0.3 is stored as: 1.0011001100110011001100110011001100110011001100110011 * 2^(-2)

these are stored as binary fractions, with 52 digits after the point. This called the mantissa
the mantissa is always between 1 and 2. (except if the number is 0)

when you subtract 0.2 from 0.3, the exponents get adjusted so you subtract

(0.11001100...) * 2^(-2) from ( 1.001100110...) * 2^(-2)
this will produce 0.0110011001100110011001100110011001100110011001100110 * 2^(-2)
the mantissa will then have to be shifted 2 bits the the left to get it between 1 and 2.
1.1001100110011001100110011001100110011001100110011000 * 2^(-4)
the result is that the last two bits will be 0, whereas they were 10.

Extra question:
write a python program that produces exactly the same floating point numbers as this one, without using a list:

 fractions  = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0]
 for f in fractions:
     print (f)

 

[Mark44]

I get that, but I know that 0.1 also cannot be represented in binary so why does the computer NOT have a problem with doing 1/10?

Because the Python print() function rounds the numbers it displays.
As already noted, many fractions that have terminating representations in base-10, such as 1/10, 1/5, 3/10 and so on, don't have terminating representations in base-2. However, any fraction that is a negative power of 2 or a sum of negative powers of 2, does have a terminating representation as a binary (i.e., base-2) fraction. Examples include 1/2 (=$2^{-1}$), 1/4 (=$2^{-2}$), 3/8 (=$2^{-2} + 2^{-3}$), and so on.

 

[vela]

Perhaps it's because you aren't doing math with 0.1 in your examples above? I'm not sure to be honest.

Yeah, that's pretty much it.

It's similar to how some students plugging numbers in right from the start when solving a problem and then make it worse by truncating the numbers they write down at each step. Each time a number is truncated, there's a loss of precision, which can accumulate to cause the final result to be much different than if numbers were plugged into an expression at the end.

It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999... With double precision floating point numbers, I found the difference between 0.3-0.2 and 0.1 is $2^{-55} \approx 3\times 10^{-17}$.

 

[PeterDonis]

It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999...

The Python documentation discusses these issues with floating point numbers, arithmetic, and display here:

https://docs.python.org/3/tutorial/floatingpoint.html

 

[Vanadium 50]

One can avoid the gory details of floating point arithmetic so long as "close enough is good enough". If you need to go beyond that, you need to understand what is being done.

 

[DrClaude]

It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999... With double precision floating point numbers, I found the difference between 0.3-0.2 and 0.1 is $2^{-55} \approx 3\times 10^{-17}$.

It is worse than I thought:

>>> print(0.3-0.2)
0.09999999999999998

What is this awful precision?

 

[pbuk]

>>> print(0.3-0.2)
0.09999999999999998

What is this awful precision?

What do you expect $0.1 - 2^{-55}$ to look like?

 

[pbuk]

It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999...

Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.

 

[DrClaude]

What do you expect $0.1 - 2^{-55}$ to look like?

Looking at the binary, I thought that there was an error on the last 2 bits, but checking again I see I was wrong, this is indeed a rounding error on the last bit only. I stand corrected.

 

[PeroK]

Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.

It's still the wrong answer, whatever the excuse!

 

[DrClaude]

It's still the wrong answer, whatever the excuse!

But it is not the wrong answer. 0.3 is an approximation for the 64-bit floating point binary number

0​

01111111101​

0011001100110011001100110011001100110011001100110011​

while 0.2 is an approximation for the 64-bit floating point binary number

0​

01111111100​

1001100110011001100110011001100110011001100110011010​

which are the two numbers being subtracted. The result of the subtraction is

0​

01111111011​

1001100110011001100110011001100110011001100110011000​

which has an approximate decimal representation of 9.99999999999999777955395074969E-2.

"But I wanted decimal 0.3 minus decimal 0.2," you reply. But this is not what the computer is asked to do when it encounters 0.3-0.2. There is of course a way for the computer to calculate what you want (working with decimal numbers), but that requires a different syntax, such as

>>> Decimal('0.3')-Decimal('0.2')
Decimal('0.1')

If you are counting money, of course you should use an exact decimal representation. But if you are doing physics, you are going to make rounding errors somewhere (you cannot have an infinite number of digits), so there is no benefit of having perfect decimal representation of floating point numbers, but there is strong performance advantage of using a binary representation of floating point numbers.

 

[PeroK]

But it is not the wrong answer. 0.3 is an approximation for the 64-bit floating point binary number

​

0​

01111111101​

0011001100110011001100110011001100110011001100110011​

while 0.2 is an approximation for the 64-bit floating point binary number

​

0​

01111111100​

1001100110011001100110011001100110011001100110011010​

which are the two numbers being subtracted. The result of the subtraction is

​

0​

01111111011​

1001100110011001100110011001100110011001100110011000​

which has an approximate decimal representation of 9.99999999999999777955395074969E-2.

"But I wanted decimal 0.3 minus decimal 0.2," you reply. But this is not what the computer is asked to do when it encounters 0.3-0.2.

$0.3-0.2 = 0.1$

I don't care what the computer thinks it's doing instead. It gives the wrong answer.

When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.

 

[anorlunda]

Who else is old enough to remember the early IBM computers, 1401, 1620, 650?

They used BCD (binary coded digits) and arithmetic was done in decimal using table look-up.

Tesla's new Dogo CPU for AI training uses the BF16 numerical format which also would give different numerical results than 64 bit IEEE floating point.

In theory, one could implement a language like Python on any of those machines. If so, the calculated results would be different.

My point is that precision is a function of the underlying hardware, not the programming language.

 

[DrClaude]

$0.3-0.2 = 0.1$

I don't care what the computer thinks it's doing instead. It gives the wrong answer.

When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.

We'll have to disagree then, because here I think the fault lies in the user. Computers are primarily binary machines.

 

[vela]

Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.

Because according to the Python documentation, "That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value instead."

I guess it depends on whether you think the user should cater to the computer or the other way around. Most people don't need or want floating-point results to 15~17 decimal places, so Python should display the useful result, not the pedantically correct one, as the default.

 

[FactChecker]

Because according to the Python documentation, "That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value instead."

This sounds like Python might interpret "print(0.1)" or "print(1/10)" as something where a small number of digits is desired in the printout, but it interprets "print(0.3-0.2)" as something where a large number of digits is desired in the printout. That introduces Python's interpretation of the user preference into the discussion. I'm not sure how I feel about that.

 

[vela]

That introduces Python's interpretation of the user preference into the discussion. I'm not sure how I feel about that.

It does boil down to what's considered useful, so it depends on the users' preferences. APL has a system variable to specify printing precision, which defaults to 10 digits, to address this issue. Mathematica seems to default to rounding to six digits, so both 0.1 and 0.3-0.2 are presented as 0.1 unless one specifically asks for all the digits. I just think it's weird that the Python developers decided that most people want to see 16 or so digits of precision.

 

[PeroK]

I just think it's weird that the Python developers decided that most people want to see 16 or so digits of precision.

Especially if it introduces nuisance imprecision.

 

[PeroK]

We'll have to disagree then, because here I think the fault lies in the user. Computers are primarily binary machines.

A computer could still get the right answer if it did the calculations with enough precision relative to its default output format. Python is simply returning a number of digits beyond the precision maintained by its calculating algorithm.

 

[Mark44]

Computers are primarily binary machines.

This...

 

[PeterDonis]

$0.3-0.2 = 0.1$

I don't care what the computer thinks it's doing instead. It gives the wrong answer.

No, it isn't. You are telling the computer to do the wrong thing. The computer is doing what you told it to do. It's up to you to understand what you are telling it to do.

If you write $0.3 - 0.2$ in Python, without using the Decimal method described earlier, you are telling the computer to do floating point arithmetic as @DrClaude described in post #16. If that's not what you want to do, then you need to tell the computer to do what you want to do, namely, the exact decimal subtraction 0.3 - 0.2, as @DrClaude also described in post #16.

You could complain that you think Python should default to decimal subtraction instead of floating point subtraction when you write $0.3 - 0.2$, but the proper venue for that complaint is the Python mailing lists. And unless Python makes that change to its language specification, Python does what it does, and it's up to you to understand the tools you are using. It's not up to the tools to read your mind and change their behavior according to what you were thinking.

When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.

If you think you can design a computer system that doesn't have these limitations and faults, go to it. Python is open source. Nothing is stopping you from producing your own hacked copy of the interpreter in which writing $0.3 - 0.2$ does a decimal subtraction instead of a floating point subtraction. The result won't meet the Python language specification, of course, but whether that's a problem for you depends on what you intend to use your custom interpreter for.

 

[PeterDonis]

A computer could still get the right answer if it did the calculations with enough precision relative to its default output format.

The computer can get the right right answer by you telling it to do the right thing. If you tell it to do the wrong thing, pointing out that it could still get what you consider to be the right answer by adding a special-case kludge on top of your wrong instructions is not, IMO, a valid argument.

As for "default output format", $0.3$ and $0.2$ are inputs to your calculation, not outputs. Python does not interpret these inputs as specifying a number of significant figures. @DrClaude has already explained in post #16 what exact floating point numbers correspond to the input notations $0.1$, $0.2$, and $0.3$. You could also denote those same exact floating point numbers in Python with different input notations. As for output of floating point numbers, how Python chooses that (as well as other relevant issues, including the ones mentioned above) is described in the Python documentation page I linked to in post #9.

 

[PeterDonis]

Python should display the useful result, not the pedantically correct one, as the default.

Python's floating point display conventions have evolved over several decades now, in response to a lot of user input through their mailing lists. I strongly suspect that the issue you raised was discussed.

Also, Python already has rounding functions, so users who want results rounded to a "useful" number of digits already have a way of getting them. Since different users will want different numbers of "useful" digits, saying that "Python should display the useful result" doesn't help since there is no single "useful result" that will work for all users. So it seems better to just leave the "pedantically correct" result as the default to display, and let users who want rounding specify the particular "useful" rounding they need for their use case.

 

[PeterDonis]

This sounds like Python might interpret "print(0.1)" or "print(1/10)" as something where a small number of digits is desired in the printout, but it interprets "print(0.3-0.2)" as something where a large number of digits is desired in the printout. That introduces Python's interpretation of the user preference into the discussion.

Bear in mind that the default output you get from print() if you don't use any format strings or specifiers is not intended for actual display in production. It's intended for "quick and dirty" uses like doing a check in the interactive interpreter or debugging, where you probably don't want results of computations to be truncated or rounded, you want to see what the interpreter is actually dealing with at a lower level. If you want nicely formatted output, you're supposed to use the extensive formatting functions that Python provides.

 

[FactChecker]

Bear in mind that the default output you get from print() if you don't use any format strings or specifiers is not intended for actual display in production. It's intended for "quick and dirty" uses like doing a check in the interactive interpreter or debugging, where you probably don't want results of computations to be truncated or rounded, you want to see what the interpreter is actually dealing with at a lower level. If you want nicely formatted output, you're supposed to use the extensive formatting functions that Python provides.

That makes me wonder what it would print for 0.1 or 1/10 if we force a format with the number of digits of the 0.09999999999999998 printout.