Why Floating Point Is Broken (and Why We Use It Anyway)

Open a Python shell and type 0.1 + 0.2. You get 0.30000000000000004. This is not a Python bug, not a CPU bug, and not something that better software engineering would fix. It is a direct consequence of what real numbers are, and no amount of additional hardware could ever fully resolve it.

The rest of this article explains why, how we work around it, and whether anything better is on the horizon.

The mathematical problem is permanent

The real numbers are uncountably infinite. That’s a specific technical claim, not just “very large”. Georg Cantor proved in 1874 that there is no bijection (no one-to-one correspondence) between the real numbers and the natural numbers. You cannot pair them up, even with infinitely many naturals to work with.

The proof is the diagonal argument. Suppose you claimed to have a complete list of all real numbers between 0 and 1, written as infinite binary expansions. Cantor constructs a number not on your list: take the first digit of the first number and flip it, take the second digit of the second number and flip it, and so on. The resulting number differs from every entry on your list in at least one position. So your list was never complete. And this holds regardless of what list you produce.

Now: a computer with $n$ bits can represent exactly $2^n$ distinct values. This is a finite set. No matter how large $n$ is (32, 64, a million), it is finite, and therefore the representable values form a set that is not only much smaller than the reals, but qualitatively different. The reals are uncountable; any finite set of bit patterns is, trivially, countable.

What if you used infinitely many bits? With a countably infinite sequence of bits, you actually could represent every real number, since the binary expansion of any real is precisely such a sequence. So in a strict cardinality sense, infinite precision would suffice: $|\{0,1\}^\omega| = 2^{\aleph_0} = |\mathbb{R}|$. But this doesn’t help us in practice, because processing a number that requires infinitely many bits to write down means you never finish writing it down. Any real computer is a finite-state machine operating on finite strings, and the problem is therefore inescapable. Some approximation is always necessary.

The question floating point answers is: given that we must approximate, where should we be precise?

How IEEE 754 answers that question

The standard governing nearly all floating point arithmetic is IEEE 754, published in 1985. A 32-bit single-precision float looks like this:

The value represented is (ignoring special cases):

$(-1)^s \times 1.\text{mantissa} \times 2^{e - 127}$

The mantissa stores 23 bits of fractional part; the leading 1 is implicit, giving 24 bits of significand in total. The exponent (biased by 127) controls the scale of the number.

This structure has a direct consequence for how representable values are distributed across the real line. Between $2^k$ and $2^{k+1}$, there are exactly $2^{23}$ representable values, evenly spaced within that interval. But between $2^{k+1}$ and $2^{k+2}$, there are also exactly $2^{23}$ values, now spread over twice the range, so spaced twice as far apart.

The result: the smaller the number, the denser the representation. There are as many floats between 0.5 and 1.0 as there are between 1.0 and 2.0, and as many again between 2.0 and 4.0. Roughly half of all representable floats lie between −1 and 1. The precision is logarithmically distributed.

This is intentional, and for most numerical work it’s exactly right. The quantity that matters in practice is relative error (how large is the mistake as a fraction of the value itself), rather than absolute error. A temperature reading of 300.001 K is fine; a reading of 300.001 km when you wanted metres is not. Floating point delivers roughly constant relative precision (about 7 significant decimal digits for a 32-bit float, 15–16 for a 64-bit double) across many orders of magnitude.

The depth buffer: where this distribution is actually useful

The non-uniform spacing of floats isn’t just a theoretical nicety; it turns out to be genuinely useful in graphics rendering.

When rasterising a 3D scene, the depth buffer stores, per pixel, some measure of how far away the nearest geometry is. The obvious choice is to store $z$ directly. But $z$ in view space is linear, which means the depth buffer allocates equal precision to a gap from 1m to 2m and a gap from 1000m to 1001m. For most scenes, this wastes precision on the far distance where misorderings don’t matter visually, and starves precision near the camera where they do.

Perspective projection maps $z$ to something proportional to $1/z$ in clip space. Storing $1/z$ in the depth buffer (as OpenGL does by default) means that close-up geometry gets dense, fine-grained depth values, while distant geometry gets coarser ones. This maps naturally onto the float distribution: small values of $1/z$ (far away) sit in the region where floats are spread out; large values (close to the camera) sit where floats are densest. The two non-uniformities cancel out and conspire to put precision where it’s needed.

Special cases and edge behaviour

IEEE 754 reserves certain bit patterns for values that aren’t ordinary numbers:

• ±Infinity: produced by overflow or division by zero (e.g. 1.0 / 0.0), rather than raising an exception
• NaN (Not a Number): for genuinely undefined results, like 0.0 / 0.0 or sqrt(-1.0). NaN propagates: any arithmetic involving NaN returns NaN
• ±0: signed zero, which compares equal to the other zero but behaves differently in some edge cases (e.g. 1.0 / -0.0 gives -Infinity)
• Subnormal numbers: very small values near zero that sacrifice some precision bits to extend the range below the smallest normalised float, rather than abruptly underflowing to zero

The practical upshot is that floating point arithmetic is closed: any operation on two floats returns a float (or one of the above). This is convenient for hardware but does mean that certain errors are silent: NaN propagates quietly through a long computation, and subnormals can drastically reduce performance on some architectures without any obvious signal.

Is there anything better?

Depending on the problem, yes.

Fixed point stores numbers with a fixed number of digits before and after the binary point. It’s simple, fast, and free of floating point’s quirks, at the cost of a fixed dynamic range you have to know in advance. Still common in digital signal processing, embedded systems, and anywhere the input range is well-constrained.

Posits (also known as Universal Numbers, or unums type III) were proposed by John Gustafson in 2017. They encode numbers so that precision is highest near ±1 (where most numerical computation tends to cluster) and deliberately coarser at very large and very small magnitudes. Early results suggest posits can match double-precision float accuracy with fewer bits on certain workloads. They’re not yet in mainstream hardware, but they’re under active research in high-performance computing.

Interval arithmetic stores a lower and upper bound rather than a single approximate value. Every operation returns an interval guaranteed to contain the true result, so you always know your worst-case error. The catch is that intervals widen with each operation, and overestimation accumulates; a long computation can produce a bound too wide to be useful.

Arbitrary precision (Python’s Decimal, GNU MPFR, Java’s BigDecimal) lets you choose precision at runtime and pays for it in speed. Useful for cryptography, financial calculations, and symbolic mathematics. CPython’s integers are already arbitrary precision as standard.

None of these have displaced IEEE 754 for general numerical work. Floats are fast, hardware-supported on every modern processor, and sufficiently accurate for the overwhelming majority of applications.

Working with it sensibly

A few practical rules that follow from the above:

• Never compare floats for exact equality. Use abs(a - b) < epsilon where epsilon suits your problem.
• Watch out for catastrophic cancellation. Subtracting two nearly-equal floats loses most of your significant digits. Reformulate the calculation if possible.
• For long accumulated sums, Kahan summation (compensated summation) keeps the error bounded at the cost of a few extra operations per term.
• For money or anything requiring exact decimal arithmetic, use a decimal type or integer arithmetic. Floats cannot represent 0.1 exactly (it has no finite binary expansion), so financial rounding errors are not a float bug but a float guarantee.

The error in floating point is not random noise. It is structured. The numbers are dense where they tend to matter and sparse where imprecision is tolerable. IEEE 754 is a well-considered response to a problem that has no perfect solution, and understanding why there’s no perfect solution makes the standard considerably easier to reason about.