Calculators TI-84 Rounding Error in Fraction Mode

[DrewD]

I had a student calculate $\tan\left(\frac{5\pi}{6}\right)$ on a TI-84 calculator and he had a rounding error in the 6th or 7th decimal place. This isn't really a big deal, but he asked and we quickly found out it was because he was in fraction display mode.

I don't know too much about computational methods so I have no idea what algorithms might be by TI that would make this error occur in this situation. I am never going to ask the students for more than 4 decimal places and they know enough about rounding errors to realize that it can happens, but I am personally interested in figuring out what error makes this happen.

Thanks.

 

[gsal]

Will you please post the numbers that you are getting?... The correct one and the one with the rounding error?

 

[CalcNerd]

One explanation could be the same as the Hp 35s. It can only resolve fractions down to 2^10 ie 1/1024. Any decimals finer than that resolution are either rounded up or down to the closest fraction to this value which means that the resolution is only 5 or 6 places.

The Ti-84 probably has fraction resolution to 10^13 which would = 1/8192, perhaps 10^15 = 1/32768.

 

[DrewD]

Thanks for the responses. Sorry it took me so long to get back.
The rounding error I get is $\tan\left(\frac{5\pi}{6}\right)=-.5773502688$ and the other answer (I think correct) is $-.5773502692$. When I just compute $\frac{5\pi}{6}$, I get the same answer whether the calculator is in fraction or decimal mode. I assume that it must truncate or round earlier when in fraction mode, but it is off screen and just magnified by the tangent function. It just seems odd to me that one mode would have more precision than the other but both would print decimals in the end.

 

[mfb]

That's 4 in the very last digit. A rounding error in the last digit is not that surprising.

 

[DrewD]

I'm not surprised that there was a rounding error, I was just interested to know why there was a rounding error when the calculator was in fraction mode which otherwise didn't change anything. I was just wondering what about fraction mode would cause additional errors in a calculatiin that didn't involve any rational numbers.

 

[mfb]

The calculation will differ in some way that is probably hard to understand without the source code (and maybe even with it).