Python Why do 0.1 and 1/10 work but 0.3-0.2 doesn't?

[FactChecker]

Here is that statement in context:

Python only prints a decimal approximation to the true decimal value of the binary approximation stored by the machine. On most machines, if Python were to print the true decimal value of the binary approximation stored for 0.1, it would have to display

>>> 0.1
0.1000000000000000055511151231257827021181583404541015625

That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value instead

@pbuk, This confuses me. What is the point of storing and working with that many decimal places when the last 37 are garbage? Are you sure that it is not printing a lot more than its calculations support? Is the Python language assuming that the computer supports so much more than 64-bit accuracy or does it try to use software to achieve more significant digits?

 

[pbuk]

Hm, yes, COBOL is an interesting example because it was, AFAIK, mainly intended for financial applications, where you want decimal arithmetic, not floating point.

Yes indeed. One of my first jobs was fixing a payroll application which had been translated from COBOL (fixed place BCD) to Pascal using floating point arithmetic instead of integers.

(Also, IIRC, floating point hadn't even been well standardized when COBOL was originally developed.)

COBOL was around for 25 years before IEEE 754!

 

[PeterDonis]

What is the point of storing and working with that many decimal places when the last 37 are garbage?

It's not storing decimal digits, it's storing binary digits. The article is simply giving the mathematically exact decimal equivalent of the binary number that is stored as the closest floating point number to 0.1.

 

[FactChecker]

It's not storing decimal digits, it's storing binary digits. The article is simply giving the mathematically exact decimal equivalent of the binary number that is stored as the closest floating point number to 0.1.

I see. Thanks. So it is only exact to the extent that the print wants to print the decimal equivalent of the binary. They could have gone on as far as they wanted.

 

[pbuk]

What is the point of storing and working with that many decimal places when the last 37 are garbage?

It's not storing decimal digits, it's storing binary digits. The article is simply giving the mathematically exact decimal equivalent of the binary number that is stored as the closest floating point number to 0.1.

^^^ This. And it is nothing special about Python, the basic operations of https://en.wikipedia.org/wiki/IEEE_754 are implemented in hardware in almost all general purpose floating point processors produced in the last 30 years - including the one in whatever device you are using now.

 

[PeterDonis]

it is only exact to the extent that the print wants to print the decimal equivalent of the binary. They could have gone on as far as they wanted.

The decimal number given is the exact equivalent of the binary number that is the closest floating point number to 0.1. Adding more decimal places would just add zeros to the right. See the "Representation Error" section at the end of the Python doc article I referenced earlier, which gives the details, including how to verify this using the interactive interpreter.

 

[pbuk]

I see. Thanks. So it is only exact to the extent that the print wants to print the decimal equivalent of the binary. They could have gone on as far as they wanted.

No, it is exact. Every finite length binary string representing a number with a fractional part is represented exactly by a finite length decimal string (with a 5 as the last digit).

The converse is not true (with the simplest example being $0.1_{10}$).

 

[DrClaude]

$0.3 - 0.2 = 0.1$ whatever any computer says. The incompetence of computers does not alter mathematical facts.

And $x = x + 2$ is false, but perfectly valid in many programming languages.

 

[PeroK]

And $x = x + 2$ is false, but perfectly valid in many programming languages.

That's purely syntactical.

 

[DrClaude]

That's purely syntactical.

So is $0.3-0.2$. That's the entire point I am making.

 

[PeroK]

So is $0.3-0.2$. That's the entire point I am making.

We are never going to agree on this. You say that whatever the computer does is correct. And I say that the computer may give the wrong answer.

If I know the computer will calculate something wrongly, then I have to work round that. But, how you conclude that makes the computer right is beyond me. It's the equivalent of a known bug, IMO.

 

[DrClaude]

We are never going to agree on this. You say that whatever the computer does is correct. And I say that the computer may give the wrong answer.

If I know the computer will calculate something wrongly, then I have to work round that. But, how you conclude that makes the computer right is beyond me. It's the equivalent of a known bug, IMO.

Writing 0.3 indicates the 64-bit floating-point binary closest to the decimal number 0.3. It is not the exact decimal 0.3, just like "=" is not the equality sign, but the assignment operator.

 

[PeroK]

Writing 0.3 indicates the 64-bit floating-point binary closest to the decimal number 0.3. It is not the exact decimal 0.3, just like "=" is not the equality sign, but the assignment operator.

The worst thing you can do when designing software is to have the software produce unexpected results that only a detailed knowledge of the inner workings of that particular release of that particular code can explain.

As others have pointed out, the problem is not with binary calculations per se, but with Python outputting more decimal places that are supported by its internal calcutations.

 

[PeroK]

PS even if you say that $0.3 - 0.2 = 0.0999999999999998$ is right, you must admit it's a pretty dumb answer!

It's not a great advert for Python.

 

[Mark44]

The worst thing you can do when designing software is to have the software produce unexpected results that only a detailed knowledge of the inner workings of that particular release of that particular code can explain.

Which is why many or most introductory programming classes devote time on exactly this problem of the limitations of real-number arithmetic.

A good programmer must have some knowledge of the inner workings of the machine he or she is working with.

PS even if you say that $0.3 - 0.2 = 0.0999999999999998$ is right, you must admit it's a pretty dumb answer!

It's not a great advert for Python.

This isn't limited to just Python. Just about any programming language that doesn't support (more) exact Decimal representation of numbers will produce results like this.

 

[PeterDonis]

You say that whatever the computer does is correct.

No, that's not what we've been saying. What we've been saying is that if the computer does something wrong, it's not the computer's fault, it's the human's fault. The computer can only do what the human tells it to do. When a Python program outputs $0.0999999999999998$ in response to you inputting $0.3 - 0.2$, it's not arguing with you about the answer to a mathematical question. It's giving the mathematically correct answer to the question you actually told it to answer, even though that's a different mathematical question than the one you wanted it to answer.

We've been over all this already, but I'm repeating it because you still apparently don't get it.

As others have pointed out, the problem is not with binary calculations per se, but with Python outputting more decimal places that are supported by its internal calcutations.

And as other others have pointed out, this is missing the point. The actual number that the computer is working with is not $0.1$ in the case given; it is $0.0999999999999998$. Even if it outputs $0.1$ because of rounding, that doesn't mean "oh, great, it's giving me the right answer". It just means you now are telling it to do two wrong things instead of one: first you're telling it to subtract the wrong numbers (the actual representable floats closest to $0.3$ and $0.2$ instead of those exact decimal numbers), and then you're telling it to output the wrong number (a rounded $0.1$ instead of the actual number it got from the subtraction).

Do you have any actual programming experience? Because this sort of thing leads to disaster in actual programming. If you want exact Decimal math, you tell the computer to do exact Decimal math. Python supports that. The right thing to do is to tell the computer to use that support. Not to complain that it didn't use it when you didn't tell it to.

It's not a great advert for Python.

Tell that to the people who developed the IEEE 754 standard for floating point math, since that's where this originally comes from. It is not limited to Python, as has already been pointed out.

 

[PeterDonis]

Just about any programming language that doesn't support (more) exact Decimal representation of numbers will produce results like this.

Even languages that do have support for exact Decimal representation of numbers will produce results like this if their default interpretation of notations like $0.3 - 0.2$ is as floats, not decimals, most likely because Decimal support was added well after the original introduction of the language, when decisions on how to interpret such notations had to be made and couldn't be changed later because it would break too much existing code. Which, as I have already pointed out, is the case for Python.

 

[Vanadium 50]

he worst thing you can do when designing software

I'n not so sure I agree. The plausible but wrong answer seems to me to be worse.

When I was young, I wrote programs that used sensible defaults. Now I write code so that if something important is missing, the program doesn't just guess. It throws some sort of exception.

 

[PeterDonis]

Python outputting more decimal places that are supported by its internal calcutations.

This is wrong. All representable floating point numbers have exact decimal equivalents. If anything, Python's print() generally outputs fewer decimal places than the exact decimal equivalents of the floating point numbers would require.

 

[PeterDonis]

the problem is not with binary calculations per se, but with

...you wanting Python to interpret the notation $0.3 - 0.2$ as telling it to do the exact decimal subtraction of three-tenths minus two-tenths, instead of as telling it to subtract the closest representable floating point numbers to those decimal numbers. But Python doesn't, and, as I have already said, I think the default interpretation of such notations in Python (not to mention in all the other languages that interpret such notations the same way) is highly unlikely to change. But that doesn't mean you can't do the exact decimal subtraction in Python; it just means you have to tell Python to do it the way Python's API is set up for you to tell it that.

 

[Dale]

the problem is not with binary calculations per se

The problem is with the input. The issue is that 0.3 is not a possible binary number. So we should make the computer program throw an exception if a user inputs 0.3 and force the user to input 1.0011001100110011001100110011001100110011001100110011 * 2^(-2) instead. I am sure that users will be thrilled that their programs are no longer wrong as they were before.

 

[PeroK]

Tell that to the people who developed the IEEE 754 standard for floating point math, since that's where this originally comes from. It is not limited to Python, as has already been pointed out.

There's nothing wrong with the floating point math, given that it must produce an approximation. It's how you present that approximation to the programmer that's the point. If there was a syntax to say "show me everything", then that's fine, you get all the decimal places possible. But, to default to an invalid arithmetic answer in simple cases seems wrong to me. And, it does seem ironic that the average person can do basic arithmetic better than a computer in these simple cases. That I find is wrong in the sense that IT should be better than that. It looks lazy and sub standard to me.

 

[PeroK]

This is wrong.

It's not wrong because it's what I intended to say! Ergo by the logic of this thread it's correct. That's a biological computer functioning according to its specfication.

 

[PeterDonis]

an invalid arithmetic answer

The answer is not invalid. It's the correct mathematical answer to the mathematical problem you told the computer to solve. This point has been made repeatedly and you continue to ignore it.

It's not wrong because it's what I intended to say! Ergo by the logic of this thread it's correct.

This invalid claim has been rebutted repeatedly, yet you continue to make it.

I strongly suggest that you take a step back. You are not actually engaging with any of the points that have been made in response to you. That is not up to your usual quality of discussion.

 

[PeroK]

Okay, but I would say that there may be those who agree with me but are inhibited from taking my side of the argument by the aggressive and browbeating tone of the discussion.

I am surprised that challenging the supposed perfection of the silicon chip has elicited so strong a response.

There are places I am sure where the belief that a computer can never be wrong would be met by the same condemnation with which my belief that they can be wrong has been met here.

 

[PeterDonis]

I would say that there may be those who agree with me

Agree with you about what? That's what you've never even bothered to specify. Or, to put it another way, what, exactly, do you think should be done by programming languages like Python in cases like the one under discussion?

Do you think Python should by default interpret $0.3 - 0.2$ as telling it to do decimal arithmetic instead of floating point? If so, then why can't you just say so? And then respond to the reasons that have already been given why that's unlikely to happen?

Do you think Python's interpreting $0.3 - 0.2$ as telling it to do floating point arithmetic is fine, but it should always round print() output of floats differently than it currently does? If so, how? And what justifies your preferred choice as compared to what Python currently does?

Or do you think Python's default parsing of $0.3 - 0.2$ and its defaults for print() output of floats are both fine, but there's some other problem lurking here that hasn't been identified? If so, what?

Having a substantive discussion with you about your responses to questions like the above would be great. But it can only happen if you stop getting hung up on irrelevancies that nobody else is even claiming and focus on the actual substantive issues.

the supposed perfection of the silicon chip

the belief that a computer can never be wrong

Nobody has made either of these claims. You are attacking straw men and failing to engage with the actual substantive issues.

 

[Mark44]

I am surprised that challenging the supposed perfection of the silicon chip has elicited so strong a response.

What you're seeming to ignore is that the usual floating-point math that is implemented in languages such as Python, C, C++, Java, etc., is incapable of dealing precisely with certain fractional amounts. Most of these languages are capable of working with such expressions as 0.3 - 0.2, provided that you use the appropriate types and library routines; i.e., Decimal and associated routines.

There are places I am sure where the belief that a computer can never be wrong would be met by the same condemnation with which my belief that they can be wrong has been met here.

I don't think anyone in this thread has asserted that computers can never be wrong. If computers are wrong, it is usually because programmers have written incorrect programs, including firmware such as the aforementioned Pentium 5 division bug.

 

[pbuk]

I am going to try showing @PeroK, because telling doesn't seem to be working...

from fractions import Fraction
# Prints 1/10: the Fraction class is designed for working with fractions!
print(Fraction(3, 10) - Fraction(2, 10))

from decimal import Decimal
# Prints 0.1: the Decimal class is designed for working with decimals!
print(Decimal('0.3') - Decimal('0.2'))
# Be careful: why do you think this prints 0.09999999999999997779553950750?
print(Decimal(0.3) - Decimal(0.2))

# Prints 0.1 (this is one way to implement accounting software avoiding FP problems).
print((30 - 20) / 100)

# Prints 0.09999999999999998: the float class is designed to work with floats!
print(float(0.3) - float(0.2))

# Prints <class 'float'> because this is the default type for a numeric literal containing
# a decimal point.
print(type(0.3 - 0.2))

# Prints 0.09999999999999998: float (i.e. IEEE 754 double precision) is the default.
print(0.3 - 0.2)

 

[vela]

I don't think anyone in this thread has asserted that computers can never be wrong.

Actually, @PeterDonis has. But besides bugs in the design, like the Pentium bug, there's always a chance that a circuit fails randomly. Engineers design chips so that the mean time between failure is so long you should virtually never run into an error of that type under normal conditions. But that's not a guarantee it couldn't happen.

 

[PeterDonis]

Actually, @PeterDonis has.

I did in an earlier post, but that was intended in a particular context, namely, the computer is correctly executing its program code, but the program code is not doing what the user intended. My point (which others in this thread have agreed with) was that in such a case, the problem is not that the computer is wrong; the computer is doing what the user told it to do. The issue is that the user is telling the computer to do the wrong thing--something other than what the user actually intended.

I did not intend, and more generally most of the discussion in this thread has not intended, to cover the cases you describe, in which, for a variety of reasons, the computer does not correctly execute its program code. In those cases, yes, the computer is wrong, in the sense that it is not doing what the user told it to do. Fortunately, as you note, such cases are extremely rare.