Skip to content

Instantly share code, notes, and snippets.

@milesrout
Last active January 24, 2017 03:16
Show Gist options
  • Save milesrout/35e1348fbdf2d8cfc8c56d5f412beda7 to your computer and use it in GitHub Desktop.
Save milesrout/35e1348fbdf2d8cfc8c56d5f412beda7 to your computer and use it in GitHub Desktop.
Investigating the fixed-point types in the Chalice programming language

Just as Chalice has a fairly broad selection of integral types, it also has a broad selection of fixed-point types. For the uninitiated, fixed-point arithmetic is like a cross between floating-point and integral arithmetic. Essentially, fixed-point arithmetic involves interpreting integers as being multiples of some value much smaller than 1.

For example:

  • uwrap16 can represent any number between 0x0000 and 0xffff (inclusive) with a precision of 0x0001. 0x1d9f can be represented, but 0x1d.9f cannot.
  • ufix8_8 can represent any number between 0x00.00 and 0xff.ff (inclusive) with a precision of 0x00.01. 0x1d.9f can be represented, but 0x1d.9fb2 cannot.

At each size there is every combination of frac and fix; signed and unsigned; wrapping, saturating and undefined-on-overflow. They are specified by a particular combination of (w|s|)frac_M, u(w|s|)frac_N, u(w|s|)fixN_N or (w|s|)fixM_N, where s is the size of the type in bits, n = s/2 and m = n - 1.

For example:

  • uwfrac_32 is a 32-bit unsigned wrap-on-overflow fixed-point type that can represent any number between 0x0.00000000 (0) and 0x.ffffffff (approximately 2.3283064e-10).
  • sfix15_16 is a 32-bit signed saturate-on-overflow fixed-point type that can represent any number between -0x8000.0000 (-32768) and 0x7fff.ffff (32767.9999847).
  • frac_7 is an 8-bit signed undefined-on-overflow fixed-point type that can represent any number between -0x.80 (-0.5 = -128/256) and 0x.7f (0.49609375 = 127/256).
  • ufix4_4 is an 8-bit unsigned undefined-on-overflow fixed-point type that can represent any number between 0x0.0 (0) and 0xf.f (15.9375 = 15 + 15/16 = 255/16).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment