IEEE-754 is what happens when you take “close enough,” encode it in silicon, make it the default representation for “number,” and then act surprised when civilization starts comparing 0.30000000000000004 to 0.3 with the haunted stare of a man who has just found a raccoon in his type system.
Its first crime is the famous one:
0.1 + 0.2 => 0.30000000000000004
0.1 + 0.2 == 0.3 => false
This is not a Python bug, JavaScript bug, C bug, Java bug, compiler bug, or cosmic clerical error. It is the nature of binary floating point: most finite decimal fractions cannot be represented exactly as finite binary fractions. Python’s own documentation gives the humiliating detail that the stored value for 0.1 is actually the nearest binary fraction, not one tenth itself, and that summing decimal-looking values can therefore miss the decimal-looking result. In other words, the machine smiles, prints 0.1, and secretly means something like 3602879701896397 / 2**55. Adorable. Fraudulent, but adorable. (docs.python.org)
The real sin is not approximation. Approximation is inevitable. The sin is impersonation. IEEE-754 values dress up as ordinary numbers while obeying a foreign algebra. They look like school arithmetic; they behave like a committee minutes document that fell into a vat of rounding modes.
Every operation is a little trial where the exact mathematical result appears before the court, and IEEE-754 says: “Lovely. Now choose the nearest prisoner from this finite zoo.” Oracle’s numerical-computation guide summarizes the basic deal: IEEE formats have finite precision, conversions between decimal and binary are generally inexact, and ordinary single/double formats provide bounded ranges and limited significant digits—about 6–9 decimal digits for single and 15–17 for double. (docs.oracle.com)
That means the number line is not a line. It is a picket fence. Near zero, the slats are close together. Far away, they are barn doors. Add a tiny number to a huge number and the tiny number may simply vanish, not metaphorically but literally, like a principled objection in a sprint-planning meeting.
1e16 + 1 == 1e16
In real arithmetic, adding one changes the value. In IEEE-754, the one may be too small to survive contact with the exponent. This is the arithmetic of aristocracy: large magnitudes get all the representation, small increments are told to know their place.
Then comes non-associativity, the point where IEEE-754 stops merely being “imprecise” and starts being a practical menace:
(a + b) + c != a + (b + c)
This is not some exotic pathology reserved for people solving fluid dynamics equations inside a neutron star. It matters in reductions, sums, averages, dot products, graphics, simulations, ML training, financial-ish calculations written by reckless optimists, and anything parallel. Change the order of summation and you may change the answer. NVIDIA’s own technical blog notes that floating-point reduction results can depend on the order in which elements are combined; when runs combine values in different orders, results can differ slightly. (developer.nvidia.com)
So now arithmetic has scheduling semantics. Congratulations, Peter, you wanted numbers and received distributed-systems nondeterminism with a mantissa.
And the nondeterminism is not only about parallelism. Compilers may fuse multiply-adds. CPUs may use different intermediate precision. GPUs may reduce in different trees. “Fast math” flags may decide that algebraic identities are back on the menu, even though IEEE-754 arithmetic has already replaced algebra with a haunted approximation game. You write x + y + z; the compiler sees a performance opportunity; the answer changes; everyone pretends this is fine because the benchmark improved.
Then there is cancellation: subtract two nearly equal numbers and you can destroy the very information you cared about.
large_precise_thing - almost_the_same_large_precise_thing
The leading digits cancel, and the low-order garbage is suddenly promoted to management. IEEE-754 does not merely lose precision; it sometimes loses precision and then gives the remaining error a corner office.
Now, let us discuss NaN, because no diatribe is complete without the goblin king.
NaN is “Not a Number,” except it lives inside the number type. It is a value that announces failure by infecting later computations. Fine, maybe that is useful. But IEEE-754 then makes it unordered, which means comparisons behave like they were designed during a hostage negotiation. NaN is not less than anything, not greater than anything, and not equal to anything, including itself. The GNU C Library manual states the lunacy plainly: x == x is false if x is NaN. (ftp.gnu.org)
NaN == NaN => false
NaN != NaN => true
NaN < 5 => false
NaN >= 5 => false
This detonates ordinary equality. Reflexivity, one of the least controversial ideas in mathematics, is dragged behind the shed because some invalid operation happened earlier and nobody checked. Hash maps, sorting, deduplication, database-ish semantics, testing frameworks, serialization, and formal reasoning all get a little more cursed. The standard does define ways to classify and order these things, but the common operators—the ones programmers actually use when tired, caffeinated, and underpaid—are booby-trapped.
And there is not just one NaN. There are quiet NaNs, signaling NaNs, payload bits, implementation conventions, and language-specific behavior. A “not a number” can carry extra diagnostic junk, except most of the ecosystem treats that payload like a note left in a bottle and thrown into a volcano. Oracle’s guide notes that NaN can be represented by many bit patterns, with quiet/signaling distinctions. (docs.oracle.com)
IEEE-754 also gives us signed zero, because apparently zero needed a dark twin.
+0 == -0 => true
1 / +0 => +Infinity
1 / -0 => -Infinity
There are defensible numerical reasons for signed zero, especially around limits, branch cuts, and preserving directional information. Fine. Mathematically interesting. Also: for the average programmer, it is another trapdoor under equality. Two values compare equal but can produce different downstream results. It is like saying two doors are the same door, except one opens into the kitchen and the other opens into the Mariana Trench.
Subnormals are next. Gradual underflow is one of IEEE-754’s more defensible choices: rather than abruptly flushing tiny values to zero, it lets representation degrade gradually. That is genuinely useful; Oracle’s guide describes how subnormal numbers preserve useful error properties near underflow. (docs.oracle.com)
But in practice? Subnormals are also a performance cliff on some hardware and often get disabled with “flush-to-zero” modes. So the standard gives you a mathematically careful feature, and then real systems quietly turn it off because performance. Beautiful. A principled compromise, immediately compromised.
Now, to seem fair—and because even a righteous rant should avoid becoming Reddit in a lab coat—IEEE-754 has two real virtues.
First: symbolic infinity is good.
Infinity and -Infinity are not stupid. They are one of the few genuinely elegant features here. Overflowing to a symbolic infinity is often better than wrapping, saturating silently to a max finite value, or detonating the process. Division by a signed zero producing signed infinity can preserve limiting behavior. Algorithms can sometimes proceed through asymptotes without special-case branch confetti. Infinity is a clean admission: “This result is beyond finite representation, but it has a direction.” That is useful. The standard’s exception model includes division by zero, overflow, underflow, invalid operation, and inexact; default results like infinities and NaNs are part of how computation continues without traps. (docs.oracle.com)
Second: it is fast.
But let us not mistake incumbency for virtue, little mantissa gremlin. IEEE-754 is fast in everyday computing because hardware vendors have spent decades building FPUs, SIMD units, GPUs, compiler pipelines, and libraries around it. It is not fast because it is a morally superior notion of number. It is fast because the empire paved every road to its palace.
That speed matters. In scientific computing, graphics, games, signal processing, ML, simulation, and statistics, fixed-size binary floats are often the right engineering compromise. They are compact. They vectorize. They are predictable enough when handled by numerical adults. The problem is that we handed them to everyone as the default “real number” and then acted as if epsilon comparisons were a personality trait.
Compared with bignum rationals, IEEE-754 is the worst kind of liar. Rationals store values as numerator and denominator. 1/10 + 2/10 is exactly 3/10. 1/3 is exactly 1/3. Equality means equality, not “equality except when the ghost of rounding mode past says otherwise.” Python’s docs explicitly point to rational arithmetic as exact arithmetic for values like 1/3. (docs.python.org)
Yes, rationals can get huge. Denominators grow. Arithmetic slows down. You cannot represent irrational results exactly unless you build symbolic algebra, and now you are no longer doing ordinary numeric computation—you are building a temple to algebraic types, as one does. But rationals are honest. When they are exact, they are exact. When they become expensive, they become visibly expensive. IEEE-754, by contrast, is cheap until it sends you an invoice in debugging time.
Compared with fixed-point decimal bignums, IEEE-754 is especially ridiculous for human-scale decimal quantities. Money, tax, accounting, measurements entered in decimal, protocol values, UI quantities—these are often decimal by social contract. Decimal bignums represent values using an integer plus a decimal scale. Java’s BigDecimal, for example, is documented as an immutable arbitrary-precision signed decimal number consisting of an arbitrary-precision integer unscaled value and a scale, with explicit rounding control. (docs.oracle.com)
That is what grown-up arithmetic looks like: scale is explicit, rounding is explicit, precision is explicit, and if an exact result cannot be represented under the requested rules, the system can complain instead of smiling and slipping 0.30000000000000004 into your ledger like a raccoon with a fountain pen.
Decimal bignums are not magic. Division can produce non-terminating decimals. You still need rounding policies. They are slower and bulkier than hardware floats. But they fail in the domain’s native language. IEEE-754 binary float fails in base two and then asks humans to pretend they count that way. We do not. We are apes with ten fingers and invoices.
Compared with posits and unums, IEEE-754 looks like a Victorian plumbing system with good marketing. Posits use tapered precision: they spend more precision near values where many computations tend to live and less at extremes. The posit literature presents them as an alternative to IEEE-754, with regime/exponent/fraction fields and a “quire” mechanism for exact accumulation of products in dot products. (johngustafson.net)
Posits also clean up some of IEEE-754’s grotesque menagerie: standard posits have a single zero and a single NaR/“Not a Real” exceptional value rather than signed zero plus a whole swamp of NaNs. (hpcas.inesc-id.pt)
Are posits a universal replacement? No. They lack IEEE-754’s ecosystem saturation. Hardware support is sparse. Some claims are workload-dependent. Tapered precision is a bet about where accuracy matters, and sometimes that bet is wrong. Posits are not exact real arithmetic; they are still finite encodings. The quire is powerful, but wide accumulators cost hardware area and complexity. Still, they at least ask the right embarrassing question: “Why are we wasting representational budget so stupidly, and why does our default number format need a bestiary of NaN payloads, signed zeros, subnormals, infinities, sticky flags, and rounding folklore?”
IEEE-754’s defenders will say: “But numerical analysts know how to use it.”
Yes. And bomb-disposal experts know how to use a shaped charge. That does not mean the rest of us should store one in the cutlery drawer.
The real indictment is this:
IEEE-754 is not bad because it approximates. It is bad because it makes approximation look like arithmetic, makes failure look like a value, makes equality non-reflexive, makes order of evaluation semantically relevant, makes reproducibility fragile, makes decimal literals deceptive, and then gets away with all of it because the silicon is fast.
It is a brilliant low-level engineering compromise that escaped containment and became a high-level semantic default.
For physics simulations? Fine.
For GPU shaders? Fine.
For signal processing? Fine.
For ML tensor sludge where three correct digits and a prayer are apparently civilization now? Fine.
For casual “number” in a programming language? For money? For deterministic game simulation? For tests that compare results exactly? For distributed systems? For business logic? For serialization round-trips? For anything where a human sees 0.1 and reasonably expects one tenth?
Absolutely not.
IEEE-754 is the worst of these worlds: less exact than rationals, less human than decimal bignums, less conceptually clean than posits/unums, and only faster because the hardware-industrial complex has been lovingly polishing this particular footgun for forty years.
It is not a number system.
It is a performance hack with a standards committee, a priesthood, and excellent vectorization.