Skip to content

Instantly share code, notes, and snippets.

@sentientmonkey
Created April 1, 2010 22:51
Show Gist options
  • Save sentientmonkey/352480 to your computer and use it in GitHub Desktop.
Save sentientmonkey/352480 to your computer and use it in GitHub Desktop.
#include <math.h>
#include <stdio.h>
int main() {
double a = 16.99;
double b = 100.0;
printf("a is %f\n", a);
printf("b is %f\n", b);
printf("a*b is %f\n", a*b);
printf("floor(a*b) is %f\n", floor(a*b));
return 0;
}
a is 16.990000
b is 100.000000
a*b is 1699.000000
floor(a*b) is 1698.000000
@sentientmonkey
Copy link
Author

Okay, I have no idea what's going on here. floor(1699.0) should be 1699.0, right? What's up with this?

Note, this works fine so far with all other numbers not ending in 99 less than 16. Kinda weird. I know this must be some issue with floating point math, but it's really weird.

@sentientmonkey
Copy link
Author

Oh yeah, this is NOT an april fools joke. Unless it turns out that I'm doing something extremely dumb. In which case, it is.

@cyrusd
Copy link

cyrusd commented Apr 1, 2010

double vs. float... floorf() is your friend :)

floorf(a*b) is 1699.000000

@darynnakhuda
Copy link

printf('%.16f',17.99 * (10**2))

1798.9999999999997726

That would be your problem :)

@sentientmonkey
Copy link
Author

So, the real reason I am mucking about with floor() is because that's how ruby's Fixnum.to_i function works...
http://www.ruby-doc.org/core/classes/Float.src/M000240.html

floorf sort of fixes it... for a certain threshold. Here's the output for with more precision (and some more debugging)

a is 16.989999999999998436805981327779591083526611328125000000000000000000000000000000000000000000000000000
b is 100.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
a_b is 1698.999999999999772626324556767940521240234375000000000000000000000000000000000000000000000000000000000
a_b < c
floorf(a*b) is 1699.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment