Why is Excel telling me the GCD of 1.13*100 and 100 is 4?

  • Thread starter SamRoss
  • Start date
  • Tags
    Excel Gcd
In summary: IEEE754In summary, when using the GCD function in Microsoft Excel, floating point numbers are truncated to their integer values. This can cause discrepancies when using numbers that are close in value, such as 1.13*100 and 113. To fix this, one can use the int() function on the floating point number to convert it to an integer before using it in the GCD function.
  • #1
SamRoss
Gold Member
254
36
In Microsoft Excel, if I type in the formula =GCD(113,100) then it gives me the correct answer of 1. However, if I type in =GCD(1.13*100,100), which means the same thing, it tells me 4. What's going on and how can I fix it? Thanks
 
Technology news on Phys.org
  • #2
I think it’s due to the 1.13*100 being treated as a float ie it’s not the same as the integer value 113.

You could test this by using the int() function on 1.13*100 to see if that fixes it.
 
  • Like
Likes WWGD and sysprog
  • #3
jedishrfu said:
I think it’s due to the 1.13*100 being treated as a float ie it’s not the same as the integer value 113.

You could test this by using the int() function on 1.13*100 to see if that fixes it.
That fixed it. Thanks!
 
  • #4
SamRoss said:
In Microsoft Excel, if I type in the formula =GCD(113,100) then it gives me the correct answer of 1. However, if I type in =GCD(1.13*100,100), which means the same thing, it tells me 4. What's going on and how can I fix it? Thanks
Here's the documentation for the gcd function in Excel:
GCD(number1, [number2], ...)
The GCD function syntax has the following arguments:
  • Number1, number2, ... Number1 is required, subsequent numbers are optional. 1 to 255 values. If any value is not an integer, it is truncated.
Due to the way that floating point numbers are stored, the value 1.13*100 turns out to be a little less than 113. This is true of computers in general, and isn't some bug in Excel. You can verify this by the formula
Code:
-(1.13*100 - 113)*10^6
The actual difference between 1.13*100 and 113 is so small that it is displayed as 0, but if you multiply this difference by, say, 10^6, you will get -1.42109*10^(-8).
Since 1.13*100 < 113, according to the docs above, it is truncated, leaving the integer value 112. Certainly gcd(112, 100) = 4
 
Last edited:
  • Like
Likes WWGD, SamRoss, Ibix and 1 other person

Related to Why is Excel telling me the GCD of 1.13*100 and 100 is 4?

1. Why is the GCD of 1.13*100 and 100 different from what I expected?

The GCD (Greatest Common Divisor) is a mathematical concept that represents the largest number that can divide both given numbers without leaving a remainder. In this case, the GCD of 1.13*100 and 100 is 4 because it is the largest number that can evenly divide both 1.13*100 and 100.

2. How does Excel calculate the GCD of two numbers?

Excel uses the built-in GCD function to calculate the GCD of two numbers. This function uses the Euclidean algorithm, which is a commonly used method for finding the GCD of two numbers.

3. Can Excel calculate the GCD of more than two numbers?

Yes, Excel's GCD function can calculate the GCD of multiple numbers. You can enter the numbers as arguments in the function, separated by commas.

4. Why is the GCD of two numbers important in mathematics?

The GCD is an important concept in mathematics because it can help simplify fractions and find common factors between numbers. It is also used in various algorithms and problem-solving techniques.

5. Is there a way to change the GCD result in Excel?

No, the GCD function in Excel always returns the correct GCD of the given numbers. If you want to change the result, you will need to change the input values or use a different function or method for calculating the GCD.

Similar threads

Replies
27
Views
3K
  • Engineering and Comp Sci Homework Help
Replies
27
Views
4K
  • Computing and Technology
Replies
3
Views
2K
  • Programming and Computer Science
Replies
7
Views
998
  • Engineering and Comp Sci Homework Help
Replies
7
Views
2K
  • Other Physics Topics
Replies
1
Views
3K
  • Engineering and Comp Sci Homework Help
Replies
1
Views
1K
  • General Engineering
Replies
5
Views
4K
  • Computing and Technology
Replies
20
Views
1K
  • Programming and Computer Science
Replies
7
Views
561
Back
Top