Converting Decimal to Hexadecimal for Compass Corrections

[Dbugger2020]

I am re-writing a program that calculates compass corrections and I seemingly cannot get the conversions from Decimal to Hexidecimal correct.

The original program is written in quick basic (but i of course cannot locate the bas (source) file.
For the first correction, let's call it "A". if I enter 1.78 as my decimal, the hex return is 00 05 10. but, if i enter 1.78 as the decimal for "B", i get, 10 00 FE. For "C", i get, 20 00 FE

So obviously, the leading bits of the return (00,10,20) are the place holders for each correction 00 being A, 10 being B, and 20 being C.

Any conversion method I've tried after that place holder yields incorrect results or at least results that do not correspond.

Any help is most definitely appreciated

 

[HallsofIvy]

I would convert to binary first, then to base 16. Since 16 is 2 to the fourth power that second conversion is easy. First 10^0, 16^0, and 2^0 are all one so the integer part of 1.78 is still 1. 0.78= 0.5+ 0.28 so the first bit after the "decimal" point is 1. 0.28= 0.25+ 0.03 so the next bit is also 1.

So far we have 1.11 (base 2) which is 1.75 (base 10). 1/8= 0.125, 1/16= 0.0625, and 1/32= 0.03125 are all larger than 0.03 so the next three bits are 0. 1/64= 0.015625 is less than 0.03 so the next bit is 1. So far we have 1.110001 (base 2) which is 1.765625 (base 10). 0.03- 0.015625= 0.014375. 1/128= 0.00390625 which is smaller so the next bit is 1. 0.014375- 0.00390625= 0.00246875. 1/256= 0.001953125 which is smaller so the next bit is 1. 0.00246875- 0.001953125= 0.000515625. 1/512= 0.001953125 which is larger so the next bit is 0. I think that is sufficient to recognize that the "repetition" (which has to happen since 1.78 is a rational number) is 1.1100011000...= 1.1100 0110 0011 0001 1000 1100...

I wrote it in 4 bit groups because 16= 2^4 so we can directly convert each group into a hexadecimal "digit". 1100= 2^3+ 2^2= 8+ 4= 12= C (hexadecimal), 0110= 4+ 2= 6. 0011= 2+ 1= 3. 0001= 1, and 1000= 8. In 1.78 (decimal) is 1.C6318C6318C6318... (hexadecimal).