- #1

SamRoss

Gold Member

- 254

- 36

You are using an out of date browser. It may not display this or other websites correctly.

You should upgrade or use an alternative browser.

You should upgrade or use an alternative browser.

- Python
- Thread starter SamRoss
- Start date

- #1

SamRoss

Gold Member

- 254

- 36

- #2

Drakkith

Staff Emeritus

Science Advisor

- 22,047

- 6,129

You can't write 0.3 in binary without an infinite number of digits, much like how you can't write 1/3 in decimal without also using an infinite number of digits. So you won't get an exact answer when you do the math in a computer.

0.3 in binary: 0.01001100110011...

Using floating point, this is truncated at some point and we wind up with a number very slightly less than 0.3. So you're actually doing something more like 0.29999999 - 0.2 instead of 0.3 - 0.2.

Edit: Also, turns out that 0.2 itself is also not representable in base 2 with a finite number of digits. 0.2 in binary: 0.00110011001100110011001100110011...

0.3 in binary: 0.01001100110011...

Using floating point, this is truncated at some point and we wind up with a number very slightly less than 0.3. So you're actually doing something more like 0.29999999 - 0.2 instead of 0.3 - 0.2.

Edit: Also, turns out that 0.2 itself is also not representable in base 2 with a finite number of digits. 0.2 in binary: 0.00110011001100110011001100110011...

Last edited:

- #3

SamRoss

Gold Member

- 254

- 36

- #4

Drakkith

Staff Emeritus

Science Advisor

- 22,047

- 6,129

- #5

SamRoss

Gold Member

- 254

- 36

Sounds reasonable.

- #6

willem2

- 2,098

- 359

The main reason is loss of precision when subtracting big numbers to get a small number.

0.1 is stored as: 1.1001100110011001100110011001100110011001100110011010 * 2^(-4)

0.2 is stored as: 1.1001100110011001100110011001100110011001100110011010 * 2^(-3)

0.3 is stored as: 1.0011001100110011001100110011001100110011001100110011 * 2^(-2)

these are stored as binary fractions, with 52 digits after the point. This called the mantissa

the mantissa is always between 1 and 2. (except if the number is 0)

when you subtract 0.2 from 0.3, the exponents get adjusted so you subtract

(0.11001100...) * 2^(-2) from ( 1.001100110...) * 2^(-2)

this will produce 0.0110011001100110011001100110011001100110011001100110 * 2^(-2)

the mantissa will then have to be shifted 2 bits the the left to get it between 1 and 2.

1.1001100110011001100110011001100110011001100110011000 * 2^(-4)

the result is that the last two bits will be 0, whereas they were 10.

Extra question:

write a python program that produces exactly the same floating point numbers as this one, without using a list:

Python:

```
fractions = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0]
for f in fractions:
print (f)
```

- #7

Mark44

Mentor

- 36,437

- 8,412

Because the Python print() function rounds the numbers it displays.

As already noted, many fractions that have terminating representations in base-10, such as 1/10, 1/5, 3/10 and so on, don't have terminating representations in base-2. However, any fraction that is a negative power of 2 or a sum of negative powers of 2, does have a terminating representation as a binary (i.e., base-2) fraction. Examples include 1/2 (=##2^{-1}##), 1/4 (=##2^{-2}##), 3/8 (=##2^{-2} + 2^{-3}##), and so on.

- #8

vela

Staff Emeritus

Science Advisor

Homework Helper

Education Advisor

- 15,675

- 2,312

Yeah, that's pretty much it.Perhaps it's because you aren't doing math with 0.1 in your examples above? I'm not sure to be honest.

It's similar to how some students plugging numbers in right from the start when solving a problem and then make it worse by truncating the numbers they write down at each step. Each time a number is truncated, there's a loss of precision, which can accumulate to cause the final result to be much different than if numbers were plugged into an expression at the end.

It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999... With double precision floating point numbers, I found the difference between 0.3-0.2 and 0.1 is ##2^{-55} \approx 3\times 10^{-17}##.

- #9

PeterDonis

Mentor

- 39,712

- 17,406

The Python documentation discusses these issues with floating point numbers, arithmetic, and display here:It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999...

https://docs.python.org/3/tutorial/floatingpoint.html

- #10

Vanadium 50

Staff Emeritus

Science Advisor

Education Advisor

2021 Award

- 29,181

- 14,426

- #11

DrClaude

Mentor

- 7,954

- 4,619

It is worse than I thought:It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999... With double precision floating point numbers, I found the difference between 0.3-0.2 and 0.1 is ##2^{-55} \approx 3\times 10^{-17}##.

Python:

```
>>> print(0.3-0.2)
0.09999999999999998
```

- #12

pbuk

Science Advisor

Gold Member

- 3,870

- 2,249

What do you expect ##0.1 - 2^{-55}## to look like?What is this awful precision?Python:`>>> print(0.3-0.2) 0.09999999999999998`

- #13

pbuk

Science Advisor

Gold Member

- 3,870

- 2,249

Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.It does seem a little weird that Python doesn't round the result of 0.3-0.2 at fewer digits and print 0.1 instead of 0.999999...

- #14

DrClaude

Mentor

- 7,954

- 4,619

Looking at the binary, I thought that there was an error on the last 2 bits, but checking again I see I was wrong, this is indeed a rounding error on the last bit only. I stand corrected.What do you expect ##0.1 - 2^{-55}## to look like?

- #15

- 23,231

- 14,739

It's still the wrong answer, whatever the excuse!Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.

- #16

DrClaude

Mentor

- 7,954

- 4,619

But it isIt's still the wrong answer, whatever the excuse!

0 | 01111111101 | 0011001100110011001100110011001100110011001100110011 |

0 | 01111111100 | 1001100110011001100110011001100110011001100110011010 |

0 | 01111111011 | 1001100110011001100110011001100110011001100110011000 |

"But I wanted decimal 0.3 minus decimal 0.2," you reply. But this is not what the computer is asked to do when it encounters 0.3-0.2. There is of course a way for the computer to calculate what you want (working with decimal numbers), but that requires a different syntax, such as

Python:

```
>>> Decimal('0.3')-Decimal('0.2')
Decimal('0.1')
```

If you are counting money, of course you should use an exact decimal representation. But if you are doing physics, you are going to make rounding errors somewhere (you cannot have an infinite number of digits), so there is no benefit of having perfect decimal representation of floating point numbers, but there is strong performance advantage of using a binary representation of floating point numbers.

- #17

- 23,231

- 14,739

##0.3-0.2 = 0.1##But it isnotthe wrong answer. 0.3 is an approximation for the 64-bit floating point binary number

0011111111010011001100110011001100110011001100110011001100110011

while 0.2 is an approximation for the 64-bit floating point binary number

0011111111001001100110011001100110011001100110011001100110011010

which are the two numbers being subtracted. The result of the subtraction is

0011111110111001100110011001100110011001100110011001100110011000

which has an approximate decimal representation of 9.99999999999999777955395074969E-2.

"But I wanted decimal 0.3 minus decimal 0.2," you reply. But this is not what the computer is asked to do when it encounters 0.3-0.2.

I don't care what the computer thinks it's doing instead. It gives the wrong answer.

When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.

- #18

anorlunda

Staff Emeritus

- 11,009

- 8,359

They used BCD (binary coded digits) and arithmetic was done in decimal using table look-up.

Tesla's new Dogo CPU for AI training uses the BF16 numerical format which also would give different numerical results than 64 bit IEEE floating point.

In theory, one could implement a language like Python on any of those machines. If so, the calculated results would be different.

My point is that precision is a function of the underlying hardware, not the programming language.

- #19

DrClaude

Mentor

- 7,954

- 4,619

We'll have to disagree then, because here I think the fault lies in the user. Computers are primarily binary machines.##0.3-0.2 = 0.1##

I don't care what the computer thinks it's doing instead. It gives the wrong answer.

When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.

- #20

vela

Staff Emeritus

Science Advisor

Homework Helper

Education Advisor

- 15,675

- 2,312

Because according to the Python documentation, "That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value instead."Why? If you want to reduce precision you can do that yourself using either Math.round or sigfig.round as appropriate. Python is a scientific tool, not a pocket calculator for people to add up their shopping.

I guess it depends on whether you think the user should cater to the computer or the other way around. Most people don't need or want floating-point results to 15~17 decimal places, so Python should display the useful result, not the pedantically correct one, as the default.

- #21

FactChecker

Science Advisor

Gold Member

- 7,439

- 3,219

This sounds like Python might interpret "print(0.1)" or "print(1/10)" as something where a small number of digits is desired in the printout, but it interprets "print(0.3-0.2)" as something where a large number of digits is desired in the printout. That introduces Python's interpretation of the user preference into the discussion. I'm not sure how I feel about that.Because according to the Python documentation, "That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value instead."

- #22

vela

Staff Emeritus

Science Advisor

Homework Helper

Education Advisor

- 15,675

- 2,312

It does boil down to what's considered useful, so it depends on the users' preferences. APL has a system variable to specify printing precision, which defaults to 10 digits, to address this issue. Mathematica seems to default to rounding to six digits, so both 0.1 and 0.3-0.2 are presented as 0.1 unless one specifically asks for all the digits. I just think it's weird that the Python developers decided that most people want to see 16 or so digits of precision.That introduces Python's interpretation of the user preference into the discussion. I'm not sure how I feel about that.

- #23

- 23,231

- 14,739

Especially if it introduces nuisance imprecision.I just think it's weird that the Python developers decided that most people want to see 16 or so digits of precision.

- #24

- 23,231

- 14,739

A computer could still get the right answer if it did the calculations with enough precision relative to its default output format. Python is simply returning a number of digits beyond the precision maintained by its calculating algorithm.We'll have to disagree then, because here I think the fault lies in the user. Computers are primarily binary machines.

- #25

Mark44

Mentor

- 36,437

- 8,412

This...Computers are primarily binary machines.

- #26

PeterDonis

Mentor

- 39,712

- 17,406

No, it isn't. You are telling the computer to do the wrong thing. The computer is doing what you told it to do. It's up to you to understand what you are telling it to do.##0.3-0.2 = 0.1##

I don't care what the computer thinks it's doing instead. It gives the wrong answer.

If you write ##0.3 - 0.2## in Python, without using the Decimal method described earlier, you are telling the computer to do floating point arithmetic as @DrClaude described in post #16. If that's not what you

You could complain that you think Python should

If you think you can design a computer system that doesn't have these limitations and faults, go to it. Python is open source. Nothing is stopping you from producing your own hacked copy of the interpreter in which writing ##0.3 - 0.2## does a decimal subtraction instead of a floating point subtraction. The result won't meet the Python language specification, of course, but whether that's a problem for you depends on what you intend to use your custom interpreter for.When using a language like Python it pays to know its limitations and faults, but they are limitations and faults of the computer system.

Last edited:

- #27

PeterDonis

Mentor

- 39,712

- 17,406

The computer can get the right right answer by you telling it to do the right thing. If you tell it to do the wrong thing, pointing out that it could still get what you consider to be the right answer by adding a special-case kludge on top of your wrong instructions is not, IMO, a valid argument.A computer could still get the right answer if it did the calculations with enough precision relative to its default output format.

As for "default output format", ##0.3## and ##0.2## are

- #28

PeterDonis

Mentor

- 39,712

- 17,406

Python's floating point display conventions have evolved over several decades now, in response to a lot of user input through their mailing lists. I strongly suspect that the issue you raised was discussed.Python should display the useful result, not the pedantically correct one, as the default.

Also, Python already has rounding functions, so users who want results rounded to a "useful" number of digits already have a way of getting them. Since different users will want different numbers of "useful" digits, saying that "Python should display the useful result" doesn't help since there is no single "useful result" that will work for all users. So it seems better to just leave the "pedantically correct" result as the default to display, and let users who want rounding specify the particular "useful" rounding they need for their use case.

- #29

PeterDonis

Mentor

- 39,712

- 17,406

Bear in mind that the default output you get from print() if you don't use any format strings or specifiers isThis sounds like Python might interpret "print(0.1)" or "print(1/10)" as something where a small number of digits is desired in the printout, but it interprets "print(0.3-0.2)" as something where a large number of digits is desired in the printout. That introduces Python's interpretation of the user preference into the discussion.

- #30

FactChecker

Science Advisor

Gold Member

- 7,439

- 3,219

That makes me wonder what it would print for 0.1 or 1/10 if we force a format with the number of digits of the 0.09999999999999998 printout.Bear in mind that the default output you get from print() if you don't use any format strings or specifiers isnotintended for actual display in production. It's intended for "quick and dirty" uses like doing a check in the interactive interpreter or debugging, where you probably don't want results of computations to be truncated or rounded, you want to see what the interpreter is actually dealing with at a lower level. If you want nicely formatted output, you're supposed to use the extensive formatting functions that Python provides.

- #31

- 23,231

- 14,739

- #32

PeterDonis

Mentor

- 39,712

- 17,406

I'm sorry, but you are not even responding to the counter arguments that have been made. You are simply declaring by fiat that the notation ##0.3 - 0.2## means what you say it means, not what the actual language specification of the language we are discussing, Python, says it means for program code in that language. Sorry, but the Humpty Dumpty principle doesn't work for program code. The language is going to interpret your code the way the language specification says, not the way you say.##0.3 - 0.2 = 0.1## whatever any computer says.

- #33

PeterDonis

Mentor

- 39,712

- 17,406

Testing this in the interactive interpreter is simple:That makes me wonder what it would print for 0.1 or 1/10 if we force a format with the number of digits of the 0.09999999999999998 printout.

Python:

```
>>> print("{:.17f}".format(0.1))
0.10000000000000001
>>> print("{:.17f}".format(1/10))
0.10000000000000001
>>> print("{:.17f}".format(0.3 - 0.2))
0.09999999999999998
>>> print("{:.17f}".format(0.3))
0.29999999999999999
>>> print("{:.17f}".format(0.2))
0.20000000000000001
```

The last two lines make it easier to see what is going on with the subtraction.

- #34

PeterDonis

Mentor

- 39,712

- 17,406

This is the programming forum, not the math forum. Nobody is arguing about the mathematical facts. We are simply pointing out, and you are apparently refusing to acknowledge, thatThe Incompletence of computers does not alter mathematical facts.

- #35

- 23,231

- 14,739

Then the language specification is wrong. Or, at least, is flawed. If you write a computer programme that specifies that the capital of the USA is New York, then that is wrong. It's no defence to say that as far as the computer is concerned it's the right answer. It's wrong.I'm sorry, but you are not even responding to the counter arguments that have been made. You are simply declaring by fiat that the notation ##0.3 - 0.2## means what you say it means, not what the actual language specification of the language we are discussing, Python, says it means for program code in that language. Sorry, but the Humpty Dumpty principle doesn't work for program code. The language is going to interpret your code the way the language specification says, not the way you say.

Share:

- Last Post

- Replies
- 1

- Views
- 301

- Last Post

- Replies
- 5

- Views
- 583

- Last Post

- Replies
- 1

- Views
- 218

- Replies
- 10

- Views
- 375

- Replies
- 5

- Views
- 336

- Last Post

- Replies
- 10

- Views
- 614

- Last Post

- Replies
- 1

- Views
- 573

- Last Post

- Replies
- 0

- Views
- 747

- Last Post

- Replies
- 2

- Views
- 395

- Last Post

- Replies
- 14

- Views
- 563