floating point representation
Posted by jpluimers on 2026/07/15
Like many programmers, in my early days I was totally unaware how floating point values were stored. Even seemingly simple data structures are worth explaining, especially when debugging.
So I was glad getting referenced to [Wayback/Archive] Float Exposed and to [Wayback/Archive] Floating Point Visually Explained which I will quote a few bits below.
About bits: did you notice you can flick on the float.exposed bits to flip them? Try it!
The references came from replies to [Wayback/Archive] Julia Evans on Twitter: “floating point representation”
- [Wayback/Archive] Julia Evans on Twitter: “Also
float.exposedby @bciechanowski is a really nice website for poking at floating point numbers” - [Wayback/Archive] Daniel on Twitter: “@b0rk
fabiensanglard.net/floating_point_visually_explained“ - [Wayback/Archive] Julia Evans on Twitter: “
float.exposedis really indispensable for explaining floating point — it’s SO fun to open it up and change the bits to show people how floating point works”
The problem is that from the early days until not that long ago, explanation of the binary representation of floating point numbers was hard to understand.
History and historic ways of explaining floating point storage
Contemporary floating point storage formats all started out with the x87 FPU (floating-point unit) instructions introduced by the 8087 FPU which sort of was the reference implementation of IEEE_754-1985 floating point numbers (the standard used by virtually all processors or co-processors introduced from roughly 1980 onwards).
The 8087 FPU used a different approach than the 8088 or 8086 CPU (central-processing unit) it accompanied (nowadays FPU and CPU are integrated) to access processing data: a non-strict operational stack based FPU (hello RPN!) versus a processor register plus memory-address based CPU.
For a long time the we had to do with the books and articles that accompanied it all more or less explaining the storage formats in the same way. Some of them now can be found on-line at sources like these:
- [Wayback/Archive] Assembly language and systems programming for the IBM PC and compatibles (The Little, Brown microcomputer bookshelf): Lemone, Karen A: 9780316520690: Amazon.com: Books
- Publisher : Little, Brown (January 1, 1985)
- Language : English
- Paperback : 407 pages
- ISBN-10 : 0316520691
- ISBN-13 : 978-0316520690
- Item Weight : 1.45 pounds
which you can read parts of at [Wayback/Archive] Assembly Language and Systems Programming for the IBM PC and Compatibles – Karen A. Lemone – Google Books
and both read and borrow the electronic version from [Archive] Assembly language and systems programming for the IBM PC and compatibles : Lemone, Karen A : Free Download, Borrow, and Streaming : Internet Archive
where the most important pages on the used storage formats are:
- [Archive] Assembly language and systems programming for the IBM PC and compatibles : Lemone, Karen A : Free Download, Borrow, and Streaming : Internet Archive pages 310-311 (on the 80-bits internally used temporary real format)
- [Archive] Assembly language and systems programming for the IBM PC and compatibles : Lemone, Karen A : Free Download, Borrow, and Streaming : Internet Archive pages 312-313 (on the short real format, long real format and special datatypes (actually values) like denormal and unnormal numbers, NaN (not-a-number), and a real indefinite)
- [Wayback/Archive] The INTEL® 8087 numeric data proc~sor by JOHN F. PALMER Intel Corporation Santa Clara, California (Proceedings, National Computer Conference, 1980) a dense article which back in the days was hard to get (when you were lucky it was available in a university library) and hard to read.
- [Wayback/Archive] Intel 8087 Math CoProcessor datasheet (document 205835-007) which was impossible to get where I studied, so I am glad it has been preserved. It has these diagrams:
Table 2 lists the seven data types the 8087 supports and presents the format for each type. Internally, the 8087 holds all numbers in the temporary real format. Load and store instructions automatically convert operands represented in memory as 16-, 32-, or 64-bit integers, 32- or 64-bit floating point numbers or 18-digit packed BCD numbers into temporary real format and vice versa. The 8087 also provides the capability to control round off, underflow, and overflow errors in each calculation.
Computations in the 8087 use the processor’s register stack. These eight 80-bit registers provide the equivalent capacity of 20 32-bit registers. The 8087 register set can be accessed as a stack, with instructions operating on the top one or two stack elements, or as a fixed register set, with instructions operating on explicitly designated registers.
The status word shown in Figure 6 reflects the over- all state of the 8087; it may be stored in memory and then inspected by CPU code. The status word is a 16-bit register divided into fields as shown in Figure 6. The busy bit (bit 15) indicates whether the NEU is either executing an instruction or has an interrupt request pending (B=1), or is idle (B=0). Several instructions which store and manipulate the status word are executed exclusively by the CU, and these do not set the busy bit themselves.
The four numeric condition code bits (Co-C3) are similar to flags in a CPU: various instructions update these bits to reflect the outcome of the 8087 operations. The effect of these instructions on the condition code bits is summarized in Table 4.
Bits 14-12 of the status word point to the 8087 register that is the current top-of-stack (TOP) as described above.
Bit 7 is the interrupt request bit. This bit is set if any unmasked exception bit is set and cleared otherwise.
Bits 5-0 are set to indicate that the NEU has detect- ed an exception while executing an instruction.
The tag word marks the content of each register as shown in Figure 7. The principal function of the tag word is to optimize the 8087’s performance. The tag word can be used, however, to interpret the contents of 8087 registers.
The 8087 provides several processing options which are selected by loading a word from memory into the control word. Figure 9 shows the format and encod- ing of the fields in the control word.
The low order byte of this control word configures 8087 interrupts, and exception masking. Bits 5 – 0 of the control word contain individual masks for each of the six exceptions that the 8087 recognizes and bit 7 contains a general mask bit for all 8087 interrupts. The high order byte of the control word configures the 8087 operating mode including precision, rounding, and infinity controls. The precision control bits (bits 9-8) can be used to set the 8087 internal operating precision at less than the default of temporary real precision. This can be useful in providing compatibility with earlier generation arithmetic processors of smaller precision than the 8087. The rounding control bits (bits 11-10) provide for directed rounding and true chop as well as the unbiased round to nearest mode specified in the proposed IEEE standard. Control over closure of the number space at infinity is also provided (either affine closure, ± ∞, or projective closure, ∞, is treated as unsigned, may be specified).
Exception Handling
The 8087 detects six different exception conditions that can occur during instruction execution. Any or all exceptions will cause an interrupt if unmasked and interrupts are enabled.
If interrupts are disabled the 8087 will simply contin- ue execution regardless of whether the host clears the exception. If a specific exception class is masked and that exception occurs, however, the 8087 will post the exception in the status register and perform an on-chip default exception handling procedure, thereby allowing processing to continue. The exceptions that the 8087 detects are the following:
- INVALID OPERATION: Stack overflow, stack underflow, indeterminate form (0/0, ∞ -∞, etc.) or the use of a Non-Number (NAN) as an operand. An exponent value is reserved and any bit pattern with this value in the exponent field is termed a Non-Number and causes this exception. If this exception is masked, the 8087’s default response is to generate a specific NAN called INDEFINITE, or to propagate already existing NANs as the calculation result.
- OVERFLOW: The result is too large in magnitude to fit the specified format. The 8087 will generate an encoding for infinity if this exception is masked.
- ZERO DIVISOR: The divisor is zero while the dividend is a non-infinite, non-zero number. Again, the 8087 will generate an encoding for infinity if this exception is masked.
- UNDERFLOW: The result is non-zero but too small in magnitude to fit in the specified format. If this exception is masked the 8087 will denormalize (shift right) the fraction until the exponent is in range. This process is called gradual underflow.
- DENORMALIZED OPERAND: At least one of the operands or the result is denormalized; it has the smallest exponent but a non-zero significand. Normal processing continues if this exception is masked off.
- INEXACT RESULT: If the true result is not exactly representable in the specified format, the result is rounded according to the rounding mode, and this flag is set. If this exception is masked, processing will simply continue.
BTW: The above document did not have OCR, so I uploaded one that has: [Wayback/Archive] 205835-007.OCR
Note I have listed all the possible exceptions above as well as there is a key difference between floating point numbers and integer numbers for them which are important to other implementations of floating point handling as well.
- [Wayback/Archive] Intel 8087 FPU Information | PCjs Machines
From the iAPX 86 [8086], 88 [8088] USER’S MANUAL, AUGUST 1981, p.S-1, “THE 8087 NUMERIC DATA PROCESSOR”:
…
Table S-2 lists the seven 8087 data types. Internally, the 8087 holds all numbers in the temporary real format; the extended range and precision of this format are key contributors to the NDP’s ability to consistently deliver stable, expected results. The 8087’s load and store instructions convert operands between the other formats and temporary real. The fact that these conversions are made, and that calculations may be performed on converted numbers, is transparent to the programmer. Integer operands, whether binary or decimal, yield correct integer results, just as real operands yield correct real results. Moreover, a rounding error does not occur when a number in an external format is converted to temporary real.Table S-2. Data Types Data Type Bits Significant Digits Approximate Range (Decimal) (Decimal) Word Integer 16 4 -32,768 <= X <= +32,767 Short Integer 32 9 -2x10^9 <= X <= +2x10^9 Long Integer 64 18 -9x10^18 <= X <= +9x10^18 Packed Decimal 80 18 -99...99 <= X <= +99...99 Short Real* 32 6-7 8.43x10^-37 <= |X| <= +3.37x10^38 Long Real* 64 15-16 4.19x10^-307 <= |X| <= +1.67x10^308 Temporary Real 80 19 3.4x10^-4932 <= |X| <= +1.2x10^4932 *The short and long real data types correspond to the single and double precision data types defined in other Intel numerics products.…
- [Wayback/Archive] The Dark Corners of x86 | Floating Point-and-Laugh (printable version of [Wayback/Archive] The Dark Corners of x86 | Floating Point-and-Laugh | InformIT):
…
The 8087 had eight registers ST0–ST7, the ST, of course, standing for “stack.” In fact, 8087 isn’t a pure stack-based implementation. Values are loaded by pushing them on to the stack, and stored by popping them off, but other operations can manipulate them. For example, the FADD instruction takes two operands, a source and a destination, adds them and stores the result in the destination. The source can be either memory or a floating point register, and the destination can be any floating point register. In a pure stack-based approach, FADD would take no operands and add ST0 and ST1, storing the result in ST1 and marking ST1 as the top of the stack. To make matters even more confusing, there is also an instruction that does exactly this (FADDP, for anyone who hasn’t run away screaming yet).One rarely used feature of the 8087 was the ability to load and store values in Binary Coded Decimal (BCD) form. BCD is popular in the financial sector, because it allows decimal values to be represented accurately. A floating point value gives you a value with a certain number of digits of precision in binary; however, some values that can be represented in a short decimal string are recurring values in binary. This leads to some variable (and difficult to spot) imprecision, which is bad when you are dealing with money. BCD gives you a fixed number of decimal digits, which is more likely to be what the financial regulators require. The trade-off is that BCD values take up more space than binary floating point values, because each byte stores two decimal digits, giving a range of 0–99, rather than 0–255. The x87 BCD operations allow developers to combine the precision of binary floating point arithmetic with the storage efficiency of BCD.Could it be any more confusing? Actually, yes. The x86 instruction set seems to have taken a cue from quantum mechanics: The act of observing it changes it. These ST0–7 registers look exactly like general-purpose floating point registers from some positions and like a stack from others. With the last of the Pentium series, Intel added something new: MMX. Now, ST0–ST8 can also look like 64-bit MMX registers, used for integer-vector operations. This decision, like the rest of x86, looked sensible at the time. Because MMX and x87 registers were the same, existing operating systems could context-switch between applications without needing to store any additional state.…
None of the TASM( Turbo Assembler) manuals did not include specifications of the IEEE floating point storage formats and only later versions of the MASM (Microsoft Macro Assembler) documentation did, for instance in the [Archive] 1992 Microsoft MASM Programmer’s Guide, (which you can also download from [Wayback/Archive] www.mikrocontroller.net/attachment/450367/MASM61PROGUIDE.pdf and text from [Wayback/Archive] Full text of “Microsoft Macro Assembler 6.1 Programmers Reference”)
Especially the figures from these pages were helpful back in the days:
- [Wayback/Archive] MASM 61 PROGUIDE : Free Download, Borrow, and Streaming : Internet Archive
[Wayback/Archive] user-images.githubusercontent.com/2033367/223100846-39ae1499-4b53-4c3e-8ff7-d0be27eef418.png
Figure 6.1 Encoding for Real Numbers in IEEE Format
The following list explains how the parts of a real number are stored in the IEEE format. Each item in the list refers to an item in Figure 6.1.
- Sign bit (0 for positive or 1 for negative) in the upper bit of the first byte.
- Exponent in the next bits in sequence (8 bits for a short real number, 11 bits for a long real number, and 15 bits for a 10-byte real number).
- The integer part of the significand in bit 63 for the 10-byte real format. By absorbing carry values, this bit allows 10-byte real operations to preserve precision to 19 digits. The integer part is always 1 in short and long real numbers; consequently, these formats do not provide a bit for the integer, since there is no point in storing it.
- Decimal part of the significand in the remaining bits. The length is 23 bits for short real numbers, 52 bits for long real numbers, and 63 bits for 10-byte real numbers.
The exponent field represents a multiplier 2n. To accommodate negative exponents (such as 2-6), the value in the exponent field is biased; that is, the actual exponent is determined by subtracting the appropriate bias value from the value in the exponent field. For example, the bias for short real numbers is 127. If the value in the exponent field is 130, the exponent represents a value of 2130-127, or 23. The bias for long real numbers is 1,023. The bias for 10-byte real numbers is 16,383.
Once you have declared floating-point data for your program, you can use coprocessor or emulator instructions to access the data. The next section focuses on the coprocessor architecture, instructions, and operands required for floating-point operations
- [Wayback/Archive] MASM 61 PROGUIDE : Free Download, Borrow, and Streaming : Internet Archive page 114:
[Wayback/Archive] user-images.githubusercontent.com/2033367/223100477-ecac07eb-747c-4299-b2e4-d982423e1fcb.png
Figure 6.2 Coprocessor Data Registers
Internally, all calculations are done on numbers of the same type. Since 10-byte real numbers have the greatest precision, lower-precision numbers are guaranteed not to lose precision as a result of calculations. The instructions that transfer values between the main memory and the coprocessor automatically convert numbers to and from the 10-byte real format.
There is a lot of confusion coming from using different terms for the same or similar things, which means you have:
- single precision sometimes called short real format or 32-bit
- double precision sometimes called long real format or 64-bit
- extended precision sometimes called temporary real format or 80-bit
- fraction often called mantissa
- denormal numbers now called subnormal numbers
What you see above is that most of the 8087 is structured into data structures sized a power of 2 bytes large. The exception is the internal 10-byte extended precision.
Are bytes always 8-bits?
01.02.09
byte
A string that consists of a number of bits, treated as a unit,
and usually representing a character or a part of a character.NOTES
1 The number of bits in a byte is fixed for a given data processing
system.2 The number of bits in a byte is usually 8.
01.02.10
octet
8-bit byte
A byte that consists of eight bits.
You can see this in many RFC standards where they explicitly name 8-bit bytes as octets as historically, byte length had varied. The 1974 [Wayback/Archive] RFC 635 – Assessment of ARPANET protocols was the first and many have followed which you can find via for instance
- [Wayback/Archive] octets site:datatracker.ietf.org/doc/html – Google Search
- [Wayback/Archive] octet site:datatracker.ietf.org/doc/html – Google Search
- [Wayback/Archive] octet or octets site:datatracker.ietf.org/doc/html – Google Search (regrettably Google Search has made the OR operator very unreliable nowadays)
I got reminded about the history leading to 8-bit bytes after reading [Wayback/Archive] Julia Evans on Twitter: “some possible reasons for 8-bit bytes” / [Wayback/Archive] Julia Evans: “some possible reasons for 8-bi…” – Mastodon:
[Wayback/Archive] Some possible reasons for 8-bit bytes which summarises
It seems to me like the main reasons for the 8-bit byte are:
- a lot of early computer companies were American, the most commonly used language in the US is English
- those people wanted computers to be good at text processing
- smaller byte sizes are in general better
- 7 bits is the smallest size you can fit all English characters + punctuation in
- 8 is a better number than 7 (because it’s a power of 2, and powers of 2 have some advantages because computers work in base 2)
- once you have popular 8-bit computers that are working well, you want to keep the same design for compatibility
My personal opinion on this is that in the early CPU days everything needed way more balancing out than today even in the early Microprocessor history. The standard set back then stuck because of compatibility.
In the timeline of events and 1970s: microprocessor revolution, you can see that some 55 years ago the first processor appeared settling on 8-bit architecture with octet bytes: the 8008 and later 8080, the direct precursors of the 8086, 8088 and derived x86 families that basically formed 5 decades worth of CPU dominance and only the last decade getting a real challenger on from the ARM architecture side of things.
In those days, every extra bit in a byte required not only external address lines added and bigger memory, but also more space on the CPU die to handle the larger bytes: processor registers, routing of signals, space for operations (memory, program instructions, arithmetic operations, etc) often scaled far more than linear. Scaling in the early 1970s was a problem as back then 4-layer designs were cutting edge so the next power of 2 (16-bytes) was far out of reach.
Since back then 8-bit bytes were at the edge of what was possible and also sufficed for the main data structures back then (a byte could hold a single ASCII character and two BCD digits), the standard settled also outside the x86 realm. That resulted in so much momentum that byte size stayed the same and word and processor register sizes increased.
For more background on the Gene Amdahl comment, you might want to to dig in the 8087 which sparked the IEEE 754-1985 standard (and maybe Phil Kaufman too).
I think it is to balance scale and precision.
I finished a few blog posts around your tweets like
Julia Evans (@b0rk) floating point representation
If you want I can get you early access. DM me if you want that.
I think the 1987 message by Phil Kaufman in the middle of the below Info-IBMPC Digest Friday, 28 August 1987 Volume 6 : Issue 59 gives a very good historic perspective.
Back in ISO/IEC 2382-1: 1993 the byte definition still was:
01.02.09
byte
A string that consists of a number of bits, treated as a unit, and usually representing a character or a part of a character.
…
2 The number of bits in a byte is usually 8.
ISO/IEC 2382-1: 1993 01.02…Nibbles were a thing back in those days as well.
I’m probably showing my age when telling you I used to buy Nibble Magazine in the bookstore (:
en.wikipedia.org/wiki/Nibble_(magazine)
At age 16, NLG 15,90 it was about 3 hours of work getting such a magazine.
USD/NLG exchange rate was 3.19
Archive.org has a black & white copy for viewing/borrowing: [Archive] Nibble 1985-09: Vol 6 Iss 9 : Free Download, Borrow, and Streaming : Internet Archive
Nibble magazine itself sells scanned PDF copies as either DVD or electronic media per year for USD ~10, so volume 6 is at [Wayback/Archive] Nibble 1985 – Nibble Magazine.
Julia’s picture mapped to history
Julia (b0rk) explains in her picture above the structure of a double precision 64-bit floating point number which is using the 64-bit Double Precision IEEE_754-1985 format which uses 52 bits for the fraction and 11 bits for exponent:
The example of the 32-bit Single Precision IEEE_754-1985 format shows it uses 23 bits for fraction, 8 bits for exponent and at least shows an example on how a number is stored:
The picture Julia posted makes it a lot easier to understand the actual representation.
Which brings me back to another post which did a very similar thing. So let’s revisit [Wayback/Archive] Floating Point Visually Explained from where I quoted two bits:
…
HOW FLOATING POINT ARE USUALLY EXPLAINED
In the C language, floats are 32-bit container following the IEEE 754 standard. Their purpose is to store and allow operations on approximation of real numbers. The way I have seen them explained so far is as follow. The 32 bits are divided in three sections:
- 1 bit S for the sign
- 8 bits E for the exponent
- 23 bits for the mantissa
Floating Point internals.
The three sections of a floating point number.So far, so good. Now, how numbers are interpreted is usually explained with the formula:
(−1)S∗1.M∗2(E−127)How everybody hates floating point to be explained to them.
…
A DIFFERENT WAY TO EXPLAIN…
Although correct, this way of explaining floating point will leaves some of us completely clueless. Fortunately, there is a different way to explain it. Instead of Exponent, think of a Window between two consecutive power of two integers. Instead of a Mantissa, think of an Offset within that window.
The three sections of a floating Point number.The window tells within which two consecutive power-of-two the number will be: [0.5,1], [1,2], [2,4], [4,8] and so on (up to [21272127,21282128]). The offset divides the window in 223=8388608223=8388608 buckets. With the window and the offset you can approximate a number. The window is an excellent mechanism to protect from overflowing. Once you have reached the maximum in a window (e.g [2,4]), you can “float” it right and represent the number within the next window (e.g [4,8]). It only costs a little bit of precision since the window becomes twice as large.The next figure illustrates how the number 6.1 would be encoded. The window must start at 4 and span to next power of two, 8. The offset is about half way down the window.
Value 6.1 approximated with floating point.
This made it so much easier for me to understand how floating points are represented and, maybe more importantly, how to get back from that representation to the actual number.
Some of my related blog posts
- A small table that shows differences between decimal, double and float (Single)
- Examples by b0rk of problems with integers and floating pointing point numbers
- b0rk (Julia Evans) on Twitter: “integer overflow”
- Online x86 and x64 Intel Instruction Assembler
- Some lesser known achievements of Phil (Philip A.) Kaufman
The Wiert Corner - irregular stream of stuff 
Leave a comment