Why do 0.1 and 1/10 work but 0.3-0.2 doesn't?

  • Python
  • Thread starter SamRoss
  • Start date
  • #1
SamRoss
Gold Member
254
36
I understand that some numbers cannot be represented in floating point but I don't understand why the way we obtain those numbers matters. In python, if I type print(0.1) or print(1/10) I get 0.1 but if I type print(0.3-0.2) I get 0.09999999... Why the difference?
 

Answers and Replies

  • #2
Drakkith
Staff Emeritus
Science Advisor
22,046
6,128
You can't write 0.3 in binary without an infinite number of digits, much like how you can't write 1/3 in decimal without also using an infinite number of digits. So you won't get an exact answer when you do the math in a computer.

0.3 in binary: 0.01001100110011...
Using floating point, this is truncated at some point and we wind up with a number very slightly less than 0.3. So you're actually doing something more like 0.29999999 - 0.2 instead of 0.3 - 0.2.

Edit: Also, turns out that 0.2 itself is also not representable in base 2 with a finite number of digits. 0.2 in binary: 0.00110011001100110011001100110011...
 
Last edited:
  • Like
Likes Dale, russ_watters, Vanadium 50 and 3 others
  • #3
SamRoss
Gold Member
254
36
I get that, but I know that 0.1 also cannot be represented in binary so why does the computer NOT have a problem with doing 1/10?
 
  • #4
Drakkith
Staff Emeritus
Science Advisor
22,046
6,128
Good question. I should have checked all three numbers before answering. Perhaps it's because you aren't doing math with 0.1 in your examples above? I'm not sure to be honest.
 
  • #5
SamRoss
Gold Member
254
36
Sounds reasonable.
 
  • #6
willem2
2,097
359
Good question. I should have checked all three numbers before answering. Perhaps it's because you aren't doing math with 0.1 in your examples above? I'm not sure to be honest.
The main reason is loss of precision when subtracting big numbers to get a small number.

0.1 is stored as: 1.1001100110011001100110011001100110011001100110011010 * 2^(-4)
0.2 is stored as: 1.1001100110011001100110011001100110011001100110011010 * 2^(-3)
0.3 is stored as: 1.0011001100110011001100110011001100110011001100110011 * 2^(-2)

these are stored as binary fractions, with 52 digits after the point. This called the mantissa
the mantissa is always between 1 and 2. (except if the number is 0)

when you subtract 0.2 from 0.3, the exponents get adjusted so you subtract

(0.11001100...) * 2^(-2) from ( 1.001100110...) * 2^(-2)
this will produce 0.0110011001100110011001100110011001100110011001100110 * 2^(-2)
the mantissa will then have to be shifted 2 bits the the left to get it between 1 and 2.
1.1001100110011001100110011001100110011001100110011000 * 2^(-4)
the result is that the last two bits will be 0, whereas they were 10.

Extra question:
write a python program that produces exactly the same floating point numbers as this one, without using a list:
Python:
 fractions  = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0]
 for f in fractions:
     print (f)
 
  • Like
Likes .Scott, Dale, Drakkith and 2 others
  • #7
36,437
8,412
I get that, but I know that 0.1 also cannot be represented in binary so why does the computer NOT have a problem with doing 1/10?
Because the Python print() function rounds the numbers it displays.
As already noted, many fractions that have terminating representations in base-10, such as 1/10, 1/5, 3/10 and so on, don't have terminating representations in base-2. However, any fraction that is a negative power of 2 or a sum of negative powers of 2, does have a terminating representation as a binary (i.e., base-2) fraction. Examples include 1/2 (=##2^{-1}##), 1/4 (=##2^{-2}##), 3/8 (=##2^{-2} + 2^{-3}##), and so on.
 
  • Informative
  • Like
Likes Drakkith, berkeman and FactChecker
  • #8
vela
Staff Emeritus
Science Advisor
Homework Helper
Education Advisor
15,675
2,312
Perhaps it's because you aren't doing math with 0.1 in your examples above? I'm not sure to be honest.
Yeah, that's pretty much it.

It's similar to how some students plugging numbers in right from the start when solving a problem and then make it worse by truncating the numbers they write down at each step. Each time a number is truncated, there's a loss of precision, which can accumulate to cause the final result to be much different than if numbers were plugged into an expression at the end.

It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999... With double precision floating point numbers, I found the difference between 0.3-0.2 and 0.1 is ##2^{-55} \approx 3\times 10^{-17}##.
 
  • #10
Vanadium 50
Staff Emeritus
Science Advisor
Education Advisor
2021 Award
29,181
14,425
One can avoid the gory details of floating point arithmetic so long as "close enough is good enough". If you need to go beyond that, you need to understand what is being done.
 
  • #11
DrClaude
Mentor
7,952
4,615
It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999... With double precision floating point numbers, I found the difference between 0.3-0.2 and 0.1 is ##2^{-55} \approx 3\times 10^{-17}##.
It is worse than I thought:
Python:
>>> print(0.3-0.2)
0.09999999999999998
What is this awful precision?
 
  • Like
Likes FactChecker and PeroK
  • #12
pbuk
Science Advisor
Gold Member
3,870
2,249
Python:
>>> print(0.3-0.2)
0.09999999999999998
What is this awful precision?
What do you expect ##0.1 - 2^{-55}## to look like?
 
  • #13
pbuk
Science Advisor
Gold Member
3,870
2,249
It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999...
Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.
 
  • #14
DrClaude
Mentor
7,952
4,615
What do you expect ##0.1 - 2^{-55}## to look like?
Looking at the binary, I thought that there was an error on the last 2 bits, but checking again I see I was wrong, this is indeed a rounding error on the last bit only. I stand corrected.
 
  • #15
PeroK
Science Advisor
Homework Helper
Insights Author
Gold Member
2021 Award
23,231
14,739
Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.
It's still the wrong answer, whatever the excuse!
 
  • #16
DrClaude
Mentor
7,952
4,615
It's still the wrong answer, whatever the excuse!
But it is not the wrong answer. 0.3 is an approximation for the 64-bit floating point binary number
0
01111111101
0011001100110011001100110011001100110011001100110011
while 0.2 is an approximation for the 64-bit floating point binary number
0
01111111100
1001100110011001100110011001100110011001100110011010
which are the two numbers being subtracted. The result of the subtraction is
0
01111111011
1001100110011001100110011001100110011001100110011000
which has an approximate decimal representation of 9.99999999999999777955395074969E-2.

"But I wanted decimal 0.3 minus decimal 0.2," you reply. But this is not what the computer is asked to do when it encounters 0.3-0.2. There is of course a way for the computer to calculate what you want (working with decimal numbers), but that requires a different syntax, such as
Python:
>>> Decimal('0.3')-Decimal('0.2')
Decimal('0.1')

If you are counting money, of course you should use an exact decimal representation. But if you are doing physics, you are going to make rounding errors somewhere (you cannot have an infinite number of digits), so there is no benefit of having perfect decimal representation of floating point numbers, but there is strong performance advantage of using a binary representation of floating point numbers.
 
  • #17
PeroK
Science Advisor
Homework Helper
Insights Author
Gold Member
2021 Award
23,231
14,739
But it is not the wrong answer. 0.3 is an approximation for the 64-bit floating point binary number
0
01111111101
0011001100110011001100110011001100110011001100110011

while 0.2 is an approximation for the 64-bit floating point binary number
0
01111111100
1001100110011001100110011001100110011001100110011010

which are the two numbers being subtracted. The result of the subtraction is
0
01111111011
1001100110011001100110011001100110011001100110011000

which has an approximate decimal representation of 9.99999999999999777955395074969E-2.

"But I wanted decimal 0.3 minus decimal 0.2," you reply. But this is not what the computer is asked to do when it encounters 0.3-0.2.
##0.3-0.2 = 0.1##

I don't care what the computer thinks it's doing instead. It gives the wrong answer.

When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.
 
  • #18
anorlunda
Staff Emeritus
Insights Author
11,009
8,359
Who else is old enough to remember the early IBM computers, 1401, 1620, 650?

They used BCD (binary coded digits) and arithmetic was done in decimal using table look-up.

Tesla's new Dogo CPU for AI training uses the BF16 numerical format which also would give different numerical results than 64 bit IEEE floating point.

In theory, one could implement a language like Python on any of those machines. If so, the calculated results would be different.

My point is that precision is a function of the underlying hardware, not the programming language.
 
  • Like
Likes FactChecker
  • #19
DrClaude
Mentor
7,952
4,615
##0.3-0.2 = 0.1##

I don't care what the computer thinks it's doing instead. It gives the wrong answer.

When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.
We'll have to disagree then, because here I think the fault lies in the user. Computers are primarily binary machines.
 
  • #20
vela
Staff Emeritus
Science Advisor
Homework Helper
Education Advisor
15,675
2,312
Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.
Because according to the Python documentation, "That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value instead."

I guess it depends on whether you think the user should cater to the computer or the other way around. Most people don't need or want floating-point results to 15~17 decimal places, so Python should display the useful result, not the pedantically correct one, as the default.
 
  • Like
Likes FactChecker and PeroK
  • #21
FactChecker
Science Advisor
Gold Member
7,439
3,219
Because according to the Python documentation, "That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value instead."
This sounds like Python might interpret "print(0.1)" or "print(1/10)" as something where a small number of digits is desired in the printout, but it interprets "print(0.3-0.2)" as something where a large number of digits is desired in the printout. That introduces Python's interpretation of the user preference into the discussion. I'm not sure how I feel about that.
 
  • #22
vela
Staff Emeritus
Science Advisor
Homework Helper
Education Advisor
15,675
2,312
That introduces Python's interpretation of the user preference into the discussion. I'm not sure how I feel about that.
It does boil down to what's considered useful, so it depends on the users' preferences. APL has a system variable to specify printing precision, which defaults to 10 digits, to address this issue. Mathematica seems to default to rounding to six digits, so both 0.1 and 0.3-0.2 are presented as 0.1 unless one specifically asks for all the digits. I just think it's weird that the Python developers decided that most people want to see 16 or so digits of precision.
 
  • #23
PeroK
Science Advisor
Homework Helper
Insights Author
Gold Member
2021 Award
23,231
14,739
I just think it's weird that the Python developers decided that most people want to see 16 or so digits of precision.
Especially if it introduces nuisance imprecision.
 
  • #24
PeroK
Science Advisor
Homework Helper
Insights Author
Gold Member
2021 Award
23,231
14,739
We'll have to disagree then, because here I think the fault lies in the user. Computers are primarily binary machines.
A computer could still get the right answer if it did the calculations with enough precision relative to its default output format. Python is simply returning a number of digits beyond the precision maintained by its calculating algorithm.
 
  • #26
39,712
17,406
##0.3-0.2 = 0.1##

I don't care what the computer thinks it's doing instead. It gives the wrong answer.
No, it isn't. You are telling the computer to do the wrong thing. The computer is doing what you told it to do. It's up to you to understand what you are telling it to do.

If you write ##0.3 - 0.2## in Python, without using the Decimal method described earlier, you are telling the computer to do floating point arithmetic as @DrClaude described in post #16. If that's not what you want to do, then you need to tell the computer to do what you want to do, namely, the exact decimal subtraction 0.3 - 0.2, as @DrClaude also described in post #16.

You could complain that you think Python should default to decimal subtraction instead of floating point subtraction when you write ##0.3 - 0.2##, but the proper venue for that complaint is the Python mailing lists. And unless Python makes that change to its language specification, Python does what it does, and it's up to you to understand the tools you are using. It's not up to the tools to read your mind and change their behavior according to what you were thinking.

When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.
If you think you can design a computer system that doesn't have these limitations and faults, go to it. Python is open source. Nothing is stopping you from producing your own hacked copy of the interpreter in which writing ##0.3 - 0.2## does a decimal subtraction instead of a floating point subtraction. The result won't meet the Python language specification, of course, but whether that's a problem for you depends on what you intend to use your custom interpreter for.
 
Last edited:
  • Like
Likes rbelli1, DrClaude and phinds
  • #27
39,712
17,406
A computer could still get the right answer if it did the calculations with enough precision relative to its default output format.
The computer can get the right right answer by you telling it to do the right thing. If you tell it to do the wrong thing, pointing out that it could still get what you consider to be the right answer by adding a special-case kludge on top of your wrong instructions is not, IMO, a valid argument.

As for "default output format", ##0.3## and ##0.2## are inputs to your calculation, not outputs. Python does not interpret these inputs as specifying a number of significant figures. @DrClaude has already explained in post #16 what exact floating point numbers correspond to the input notations ##0.1##, ##0.2##, and ##0.3##. You could also denote those same exact floating point numbers in Python with different input notations. As for output of floating point numbers, how Python chooses that (as well as other relevant issues, including the ones mentioned above) is described in the Python documentation page I linked to in post #9.
 
  • #28
39,712
17,406
Python should display the useful result, not the pedantically correct one, as the default.
Python's floating point display conventions have evolved over several decades now, in response to a lot of user input through their mailing lists. I strongly suspect that the issue you raised was discussed.

Also, Python already has rounding functions, so users who want results rounded to a "useful" number of digits already have a way of getting them. Since different users will want different numbers of "useful" digits, saying that "Python should display the useful result" doesn't help since there is no single "useful result" that will work for all users. So it seems better to just leave the "pedantically correct" result as the default to display, and let users who want rounding specify the particular "useful" rounding they need for their use case.
 
  • Like
Likes DrClaude and phinds
  • #29
39,712
17,406
This sounds like Python might interpret "print(0.1)" or "print(1/10)" as something where a small number of digits is desired in the printout, but it interprets "print(0.3-0.2)" as something where a large number of digits is desired in the printout. That introduces Python's interpretation of the user preference into the discussion.
Bear in mind that the default output you get from print() if you don't use any format strings or specifiers is not intended for actual display in production. It's intended for "quick and dirty" uses like doing a check in the interactive interpreter or debugging, where you probably don't want results of computations to be truncated or rounded, you want to see what the interpreter is actually dealing with at a lower level. If you want nicely formatted output, you're supposed to use the extensive formatting functions that Python provides.
 
  • Like
Likes FactChecker
  • #30
FactChecker
Science Advisor
Gold Member
7,439
3,219
Bear in mind that the default output you get from print() if you don't use any format strings or specifiers is not intended for actual display in production. It's intended for "quick and dirty" uses like doing a check in the interactive interpreter or debugging, where you probably don't want results of computations to be truncated or rounded, you want to see what the interpreter is actually dealing with at a lower level. If you want nicely formatted output, you're supposed to use the extensive formatting functions that Python provides.
That makes me wonder what it would print for 0.1 or 1/10 if we force a format with the number of digits of the 0.09999999999999998 printout.
 
  • #31
PeroK
Science Advisor
Homework Helper
Insights Author
Gold Member
2021 Award
23,231
14,739
##0.3 - 0.2 = 0.1## whatever any computer says. The incompetence of computers does not alter mathematical facts.
 
  • #32
39,712
17,406
##0.3 - 0.2 = 0.1## whatever any computer says.
I'm sorry, but you are not even responding to the counter arguments that have been made. You are simply declaring by fiat that the notation ##0.3 - 0.2## means what you say it means, not what the actual language specification of the language we are discussing, Python, says it means for program code in that language. Sorry, but the Humpty Dumpty principle doesn't work for program code. The language is going to interpret your code the way the language specification says, not the way you say.
 
  • #33
39,712
17,406
That makes me wonder what it would print for 0.1 or 1/10 if we force a format with the number of digits of the 0.09999999999999998 printout.
Testing this in the interactive interpreter is simple:

Python:
>>> print("{:.17f}".format(0.1))
0.10000000000000001
>>> print("{:.17f}".format(1/10))
0.10000000000000001
>>> print("{:.17f}".format(0.3 - 0.2))
0.09999999999999998
>>> print("{:.17f}".format(0.3))
0.29999999999999999
>>> print("{:.17f}".format(0.2))
0.20000000000000001

The last two lines make it easier to see what is going on with the subtraction.
 
  • #34
39,712
17,406
The Incompletence of computers does not alter mathematical facts.
This is the programming forum, not the math forum. Nobody is arguing about the mathematical facts. We are simply pointing out, and you are apparently refusing to acknowledge, that which mathematical facts a particular notation in a particular programming language refers to is determined by the language specification, not by your personal desires.
 
  • #35
PeroK
Science Advisor
Homework Helper
Insights Author
Gold Member
2021 Award
23,231
14,739
I'm sorry, but you are not even responding to the counter arguments that have been made. You are simply declaring by fiat that the notation ##0.3 - 0.2## means what you say it means, not what the actual language specification of the language we are discussing, Python, says it means for program code in that language. Sorry, but the Humpty Dumpty principle doesn't work for program code. The language is going to interpret your code the way the language specification says, not the way you say.
Then the language specification is wrong. Or, at least, is flawed. If you write a computer programme that specifies that the capital of the USA is New York, then that is wrong. It's no defence to say that as far as the computer is concerned it's the right answer. It's wrong.
 

Suggested for: Why do 0.1 and 1/10 work but 0.3-0.2 doesn't?

  • Last Post
Replies
5
Views
583
  • Last Post
Replies
1
Views
218
Replies
10
Views
375
  • Last Post
Replies
10
Views
614
Replies
1
Views
573
Replies
0
Views
747
  • Last Post
Replies
2
Views
395
  • Last Post
Replies
14
Views
563
Top