-
-
Save sentientmonkey/352480 to your computer and use it in GitHub Desktop.
#include <math.h> | |
#include <stdio.h> | |
int main() { | |
double a = 16.99; | |
double b = 100.0; | |
printf("a is %f\n", a); | |
printf("b is %f\n", b); | |
printf("a*b is %f\n", a*b); | |
printf("floor(a*b) is %f\n", floor(a*b)); | |
return 0; | |
} | |
a is 16.990000 | |
b is 100.000000 | |
a*b is 1699.000000 | |
floor(a*b) is 1698.000000 |
Oh yeah, this is NOT an april fools joke. Unless it turns out that I'm doing something extremely dumb. In which case, it is.
double vs. float... floorf() is your friend :)
floorf(a*b) is 1699.000000
printf('%.16f',17.99 * (10**2))
1798.9999999999997726
That would be your problem :)
So, the real reason I am mucking about with floor() is because that's how ruby's Fixnum.to_i function works...
http://www.ruby-doc.org/core/classes/Float.src/M000240.html
floorf sort of fixes it... for a certain threshold. Here's the output for with more precision (and some more debugging)
a is 16.989999999999998436805981327779591083526611328125000000000000000000000000000000000000000000000000000
b is 100.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
a_b is 1698.999999999999772626324556767940521240234375000000000000000000000000000000000000000000000000000000000
a_b < c
floorf(a*b) is 1699.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Okay, I have no idea what's going on here. floor(1699.0) should be 1699.0, right? What's up with this?
Note, this works fine so far with all other numbers not ending in 99 less than 16. Kinda weird. I know this must be some issue with floating point math, but it's really weird.