IEEE 754
View on Wikipedia
| Floating-point formats |
|---|
| IEEE 754 |
|
| Other |
| Alternatives |
| Tapered floating point |
The IEEE Standard for Floating-Point Arithmetic (IEEE 754) is a technical standard for floating-point arithmetic originally established in 1985 by the Institute of Electrical and Electronics Engineers (IEEE). The standard addressed many problems found in the diverse floating-point implementations that made them difficult to use reliably and portably. Many hardware floating-point units use the IEEE 754 standard.
The standard defines:
- arithmetic formats: sets of binary and decimal floating-point data, which consist of finite numbers (including signed zeros and subnormal numbers), infinities, and special "not a number" values (NaNs)
- interchange formats: encodings (bit strings) that may be used to exchange floating-point data in an efficient and compact form
- rounding rules: properties to be satisfied when rounding numbers during arithmetic and conversions
- operations: arithmetic and other operations (such as trigonometric functions) on arithmetic formats
- exception handling: indications of exceptional conditions (such as division by zero, overflow, etc.)
IEEE 754-2008, published in August 2008, includes nearly all of the original IEEE 754-1985 standard, plus the IEEE 854-1987 (Radix-Independent Floating-Point Arithmetic) standard. The current version, IEEE 754-2019, was published in July 2019.[1] It is a minor revision of the previous version, incorporating mainly clarifications, defect fixes and new recommended operations.
History
[edit]| Year | Official Standard |
|---|---|
| 1982 | IEC 559:1982 |
| 1985 | IEEE 754-1985 |
| 1987 | IEEE 854-1987 |
| 1989 | IEC 559:1989 |
| 2008 | IEEE 754-2008 |
| 2011 | ISO/IEC/IEEE 60559:2011 |
| 2019 | IEEE 754-2019 |
| 2020 | ISO/IEC 60559:2020 |
| 2029 | TBA |
The need for a floating-point standard arose from chaos in the business and scientific computing industry in the 1960s and 1970s. IBM used a hexadecimal floating-point format with a longer significand and a shorter exponent[clarification needed]. CDC and Cray computers used ones' complement representation, which admits a value of +0 and −0. CDC 60-bit computers did not have full 60-bit adders, so integer arithmetic was limited to 48 bits of precision from the floating-point unit. Exception processing from divide-by-zero was different on different computers. Moving data between systems and even repeating the same calculations on different systems was often difficult.
The first IEEE standard for floating-point arithmetic, IEEE 754-1985, was published in 1985. It covered only binary floating-point arithmetic.
A new version, IEEE 754-2008, was published in August 2008, following a seven-year revision process, chaired by Dan Zuras and edited by Mike Cowlishaw. It replaced both IEEE 754-1985 (Binary Floating-Point Arithmetic) and IEEE 854-1987 (Radix-Independent Floating-Point Arithmetic) standards. The binary formats in the original standard are included in this new standard along with three new basic formats, one binary and two decimal. To conform to the current standard, an implementation must implement at least one of the basic formats as both an arithmetic format and an interchange format.
The international standard ISO/IEC/IEEE 60559:2011 (with content identical to IEEE 754-2008) has been approved for adoption through ISO/IEC JTC 1/SC 25 under the ISO/IEEE PSDO Agreement[2][3] and published.[4]
The current version, IEEE 754-2019 published in July 2019, is derived from and replaces IEEE 754-2008, following a revision process started in September 2015, chaired by David G. Hough and edited by Mike Cowlishaw. It incorporates mainly clarifications (e.g. totalOrder) and defect fixes (e.g. minNum), but also includes some new recommended operations (e.g. augmentedAddition).[5][6]
The international standard ISO/IEC 60559:2020 (with content identical to IEEE 754-2019) has been approved for adoption through ISO/IEC JTC 1/SC 25 and published.[7]
The next projected revision of the standard is in 2029.[8]
Formats
[edit]An IEEE 754 format is a "set of representations of numerical values and symbols". A format may also include how the set is encoded.[9]
A floating-point format is specified by
- a base (also called radix) b, which is either 2 (binary) or 10 (decimal) in IEEE 754;
- a precision p;
- an exponent range from emin to emax, with emin = 1 − emax, or equivalently emin = − (emax − 1), for all IEEE 754 formats.
A format comprises
- Finite numbers, which can be described by three integers: s = a sign (zero or one), c = a significand (also called a coefficient or mantissa) having no more than p digits when written in base b (i.e., an integer in the range through 0 to bp − 1), and q = an exponent such that emin ≤ q + p − 1 ≤ emax. The numerical value of such a finite number is (−1)s × c × bq.[a] Moreover, there are two zero values, called signed zeros: the sign bit specifies whether a zero is +0 (positive zero) or −0 (negative zero).
- Two infinities: +∞ and −∞.
- Two kinds of NaN (not-a-number): a quiet NaN (qNaN) and a signaling NaN (sNaN).
For example, if b = 10, p = 7, and emax = 96, then emin = −95, the significand satisfies 0 ≤ c ≤ 9999999, and the exponent satisfies −101 ≤ q ≤ 90. Consequently, the smallest non-zero positive number that can be represented is 1×10−101, and the largest is 9999999×1090 (9.999999×1096), so the full range of numbers is −9.999999×1096 through 9.999999×1096. The numbers −b1−emax and b1−emax (here, −1×10−95 and 1×10−95) are the smallest (in magnitude) normal numbers; non-zero numbers between these smallest numbers are called subnormal numbers.
Representation and encoding in memory
[edit]Some numbers may have several possible floating-point representations. For instance, if b = 10, and p = 7, then −12.345 can be represented by −12345×10−3, −123450×10−4, and −1234500×10−5. However, for most operations, such as arithmetic operations, the result (value) does not depend on the representation of the inputs.
For the decimal formats, any representation is valid, and the set of these representations is called a cohort. When a result can have several representations, the standard specifies which member of the cohort is chosen.
For the binary formats, the representation is made unique by choosing the smallest representable exponent allowing the value to be represented exactly. Further, the exponent is not represented directly, but a bias is added so that the smallest representable exponent is represented as 1, with 0 used for subnormal numbers. For numbers with an exponent in the normal range (the exponent field being neither all ones nor all zeros), the leading bit of the significand will always be 1. Consequently, a leading 1 can be implied rather than explicitly present in the memory encoding, and under the standard the explicitly represented part of the significand will lie between 0 and 1. This rule is called leading bit convention, implicit bit convention, or hidden bit convention. This rule allows the binary format to have an extra bit of precision. The leading bit convention cannot be used for the subnormal numbers as they have an exponent outside the normal exponent range and scale by the smallest represented exponent as used for the smallest normal numbers.
Due to the possibility of multiple encodings (at least in formats called interchange formats), a NaN may carry other information: a sign bit (which has no meaning, but may be used by some operations) and a payload, which is intended for diagnostic information indicating the source of the NaN (but the payload may have other uses, such as NaN-boxing[10][11][12]).
Basic and interchange formats
[edit]The standard defines five basic formats that are named for their numeric base and the number of bits used in their interchange encoding. There are three binary floating-point basic formats (encoded with 32, 64 or 128 bits) and two decimal floating-point basic formats (encoded with 64 or 128 bits). The binary32 and binary64 formats are the single and double formats of IEEE 754-1985 respectively. A conforming implementation must fully implement at least one of the basic formats.
The standard also defines interchange formats, which generalize these basic formats.[13] For the binary formats, the leading bit convention is required. The following table summarizes some of the possible interchange formats (including the basic formats).
| Significand | Exponent | Properties[b] | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Name | Common name | Radix
|
Digits[c]
|
Decimal
digits[d] |
Min | Max | MAXVAL | log10 MAXVAL | MINVAL>0 (normal) |
MINVAL>0 (subnormal) |
Notes |
| binary16 | Half precision | 2 | 11 | 3.31 | −14 | 15 | 65504 | 4.816 | 6.10×10−5 | 5.96×10−8 | Interchange |
| binary32 | Single precision | 2 | 24 | 7.22 | −126 | 127 | 3.40×1038 | 38.532 | 1.18×10−38 | 1.40×10−45 | Basic |
| binary64 | Double precision | 2 | 53 | 15.95 | −1022 | 1023 | 1.80×10308 | 308.255 | 2.23×10−308 | 4.94×10−324 | Basic |
| binary128 | Quadruple precision | 2 | 113 | 34.02 | −16382 | 16383 | 1.19×104932 | 4932.075 | 3.36×10−4932 | 6.48×10−4966 | Basic |
| binary256 | Octuple precision | 2 | 237 | 71.34 | −262142 | 262143 | 1.61×1078913 | 78913.207 | 2.48×10−78913 | 2.25×10−78984 | Interchange |
| decimal32 | 10 | 7 | 7 | −95 | 96 | 1.0×1097 | 97 − 4.34×10−8 | 1×10−95 | 1×10−101 | Interchange | |
| decimal64 | 10 | 16 | 16 | −383 | 384 | 1.0×10385 | 385 − 4.34×10−17 | 1×10−383 | 1×10−398 | Basic | |
| decimal128 | 10 | 34 | 34 | −6143 | 6144 | 1.0×106145 | 6145 − 4.34×10−35 | 1×10−6143 | 1×10−6176 | Basic | |
In the table above, integer values are exact, whereas values in decimal notation (e.g. 1.0) are rounded values. The minimum exponents listed are for normal numbers; the special subnormal number representation allows even smaller (in magnitude) numbers to be represented with some loss of precision. For example, the smallest positive number that can be represented in binary64 is 2−1074; contributions to the −1074 figure include the emin value −1022 and all but one of the 53 significand bits (2−1022 − (53 − 1) = 2−1074).
Decimal digits is the precision of the format expressed in terms of an equivalent number of decimal digits. It is computed as digits × log10 base. E.g. binary128 has approximately the same precision as a 34 digit decimal number.
log10 MAXVAL is a measure of the range of the encoding. Its integer part is the largest exponent shown on the output of a value in scientific notation with one leading digit in the significand before the decimal point (e.g. 1.698×1038 is near the largest value in binary32, 9.999999×1096 is the largest value in decimal32).
The binary32 (single) and binary64 (double) formats are two of the most common formats used today. The figure below shows the absolute precision for both formats over a range of values. This figure can be used to select an appropriate format given the expected value of a number and the required precision.

An example of a layout for 32-bit floating point is

and the 64 bit layout is similar.
Extended and extendable precision formats
[edit]The standard specifies optional extended and extendable precision formats, which provide greater precision than the basic formats.[14] An extended precision format extends a basic format by using more precision and more exponent range. An extendable precision format allows the user to specify the precision and exponent range. An implementation may use whatever internal representation it chooses for such formats; all that needs to be defined are its parameters (b, p, and emax). These parameters uniquely describe the set of finite numbers (combinations of sign, significand, and exponent for the given radix) that it can represent.
The standard recommends that language standards provide a method of specifying p and emax for each supported base b.[15] The standard recommends that language standards and implementations support an extended format which has a greater precision than the largest basic format supported for each radix b.[16] For an extended format with a precision between two basic formats the exponent range must be as great as that of the next wider basic format. So for instance a 64-bit extended precision binary number must have an 'emax' of at least 16383. The x87 80-bit extended format meets this requirement.
The original IEEE 754-1985 standard also had the concept of extended formats, but without any mandatory relation between emin and emax. For example, the Motorola 68881 80-bit format,[17] where emin = − emax, was a conforming extended format, but it became non-conforming in the 2008 revision.
Interchange formats
[edit]Interchange formats are intended for the exchange of floating-point data using a bit string of fixed length for a given format.
Binary
[edit]For the exchange of binary floating-point numbers, interchange formats of length 16 bits, 32 bits, 64 bits, and any multiple of 32 bits ≥ 128[e] are defined. The 16-bit format is intended for the exchange or storage of small numbers (e.g., for graphics).
The encoding scheme for these binary interchange formats is the same as that of IEEE 754-1985: a sign bit, followed by w exponent bits that describe the exponent offset by a bias, and p − 1 bits that describe the significand. The width of the exponent field for a k-bit format is computed as w = round(4 log2(k)) − 13. The existing 64- and 128-bit formats follow this rule, but the 16- and 32-bit formats have more exponent bits (5 and 8 respectively) than this formula would provide (3 and 7 respectively).
As with IEEE 754-1985, the biased-exponent field is filled with all 1 bits to indicate either infinity (trailing significand field = 0) or a NaN (trailing significand field ≠ 0). For NaNs, quiet NaNs and signaling NaNs are distinguished by using the most significant bit of the trailing significand field exclusively,[f] and the payload is carried in the remaining bits.
Decimal
[edit]For the exchange of decimal floating-point numbers, interchange formats of any multiple of 32 bits are defined. As with binary interchange, the encoding scheme for the decimal interchange formats encodes the sign, exponent, and significand. Two different bit-level encodings are defined, and interchange is complicated by the fact that some external indicator of the encoding in use may be required.
The two options allow the significand to be encoded as a compressed sequence of decimal digits using densely packed decimal or, alternatively, as a binary integer. The former is more convenient for direct hardware implementation of the standard, while the latter is more suited to software emulation on a binary computer. In either case, the set of numbers (combinations of sign, significand, and exponent) that may be encoded is identical, and special values (±zero with the minimum exponent, ±infinity, quiet NaNs, and signaling NaNs) have identical encodings.
Rounding rules
[edit]The standard defines five rounding rules. The first two rules round to a nearest value; the others are called directed roundings:
Roundings to nearest
[edit]- Round to nearest, ties to even – rounds to the nearest value; if the number falls midway, it is rounded to the nearest value with an even least significant digit.
- Round to nearest, ties away from zero (or ties to away) – rounds to the nearest value; if the number falls midway, it is rounded to the nearest value above (for positive numbers) or below (for negative numbers).
At the extremes, a value with a magnitude strictly less than will be rounded to the minimum or maximum finite number (depending on the value's sign). Any numbers with exactly this magnitude are considered ties; this choice of tie may be conceptualized as the midpoint between and , which, were the exponent not limited, would be the next representable floating-point numbers larger in magnitude. Numbers with a magnitude strictly larger than k are rounded to the corresponding infinity.[18]
"Round to nearest, ties to even" is the default for binary floating point and the recommended default for decimal. "Round to nearest, ties to away" is only required for decimal implementations.[19]
Directed roundings
[edit]- Round toward 0 – directed rounding towards zero (also known as truncation).
- Round toward +∞ – directed rounding towards positive infinity (also known as rounding up or ceiling).
- Round toward −∞ – directed rounding towards negative infinity (also known as rounding down or floor).
| Mode | Example value | |||
|---|---|---|---|---|
| +11.5 | +12.5 | −11.5 | −12.5 | |
| to nearest, ties to even | +12.0 | +12.0 | −12.0 | −12.0 |
| to nearest, ties away from zero | +12.0 | +13.0 | −12.0 | −13.0 |
| toward 0 | +11.0 | +12.0 | −11.0 | −12.0 |
| toward +∞ | +12.0 | +13.0 | −11.0 | −12.0 |
| toward −∞ | +11.0 | +12.0 | −12.0 | −13.0 |
Unless specified otherwise, the floating-point result of an operation is determined by applying the rounding function on the infinitely precise (mathematical) result. Such an operation is said to be correctly rounded. This requirement is called correct rounding.[20]
Required operations
[edit]Required operations for a supported arithmetic format (including the basic formats) include:
- Conversions to and from integer[21][22]
- Previous and next consecutive values[21]
- Arithmetic operations (add, subtract, multiply, divide, square root, fused multiply–add, remainder, minimum, maximum)[21][22]
- Conversions (between formats, to and from strings, etc.)[23][24]
- Scaling and (for decimal) quantizing[25][26]
- Copying and manipulating the sign (abs, negate, etc.)[27]
- Comparisons and total ordering[28][29]
- Classification of numbers (subnormal, finite, etc.) and testing for NaNs[30]
- Testing and setting status flags[31]
Comparison predicates
[edit]The standard provides comparison predicates to compare one floating-point datum to another in the supported arithmetic format.[32] Any comparison with a NaN is treated as unordered. −0 and +0 compare as equal.
Total-ordering predicate
[edit]The standard provides a predicate totalOrder, which defines a total ordering on canonical members of the supported arithmetic format.[33] The predicate agrees with the comparison predicates (see section § Comparison predicates) when one floating-point number is less than the other. The main differences are:[34]
- NaN is sortable.
- NaN is treated as if it had a larger absolute value than Infinity (or any other floating-point numbers). (−NaN < −Infinity; +Infinity < +NaN.)
- qNaN and sNaN are treated as if qNaN had a larger absolute value than sNaN. (−qNaN < −sNaN; +sNaN < +qNaN.)
- NaN is then sorted according to the payload. In IEEE 754-2008, a NaN with a lesser payload is treated as having a lesser absolute value. In IEEE 754-2019, any implementation-defined ordering is acceptable.
- Negative zero is treated as smaller than positive zero.
- If both sides of the comparison refer to the same floating-point datum, the one with the lesser exponent is treated as having a lesser absolute value.[33]
The totalOrder predicate does not impose a total ordering on all encodings in a format. In particular, it does not distinguish among different encodings of the same floating-point representation, as when one or both encodings are non-canonical.[33] IEEE 754-2019 incorporates clarifications of totalOrder.
For the binary interchange formats whose encoding follows the IEEE 754-2008 recommendation on placement of the NaN signaling bit, the comparison is identical to one that type puns the floating-point numbers to a sign–magnitude integer (assuming a payload ordering consistent with this comparison), an old trick for FP comparison without an FPU.[35]
Exception handling
[edit]The standard defines five exceptions, each of which returns a default value and has a corresponding status flag that is raised when the exception occurs.[g] No other exception handling is required, but additional non-default alternatives are recommended (see § Alternate exception handling).
The five possible exceptions are
- Invalid operation: mathematically undefined, e.g., the square root of a negative number. By default, returns qNaN.
- Division by zero: an operation on finite operands gives an exact infinite result, e.g., 1/0 or log(0). By default, returns ±infinity.
- Overflow: a finite result is too large to be represented accurately (i.e., its exponent with an unbounded exponent range would be larger than emax). By default, returns ±infinity for the round-to-nearest modes (and follows the rounding rules for the directed rounding modes).
- Underflow: a result is very small (outside the normal range). By default, returns a number less than or equal to the minimum positive normal number in magnitude (following the rounding rules); a subnormal number always implies an underflow exception, but by default, if it is exact, no flag is raised.
- Inexact: the exact (i.e., unrounded) result is not representable exactly. By default, returns the correctly rounded result.
These are the same five exceptions as were defined in IEEE 754-1985, but the division by zero exception has been extended to operations other than the division.
Some decimal floating-point implementations define additional exceptions,[36][37] which are not part of IEEE 754:
- Clamped: a result's exponent is too large for the destination format. By default, trailing zeros will be added to the coefficient to reduce the exponent to the largest usable value. If this is not possible (because this would cause the number of digits needed to be more than the destination format) then an overflow exception occurs.
- Rounded: a result's coefficient requires more digits than the destination format provides. An inexact exception is signaled if any non-zero digits are discarded.
Additionally, operations like quantize when either operand is infinite, or when the result does not fit the destination format, will also signal invalid operation exception.[38]
Special values
[edit]Signed zero
[edit]In the IEEE 754 standard, zero is signed, meaning that there exist both a "positive zero" (+0) and a "negative zero" (−0). In most run-time environments, positive zero is usually printed as "0" and the negative zero as "-0". The two values behave as equal in numerical comparisons, but some operations return different results for +0 and −0. For instance, 1/(−0) returns negative infinity, while 1/(+0) returns positive infinity (so that the identity 1/(1/±∞) = ±∞ is maintained). Other common functions with a discontinuity at x = 0 which might treat +0 and −0 differently include Γ(x) and the principal square root of y + xi for any negative number y. As with any approximation scheme, operations involving "negative zero" can occasionally cause confusion. For example, in IEEE 754, x = y does not always imply 1/x = 1/y, as 0 = −0 but 1/0 ≠ 1/(−0).[39] Moreover, the reciprocal square root[h] of ±0 is ±∞ while the mathematical function over the real numbers does not have any negative value.
Subnormal numbers
[edit]Subnormal values fill the underflow gap with values where the absolute distance between them is the same as for adjacent values just outside the underflow gap. This is an improvement over the older practice to just have zero in the underflow gap, and where underflowing results were replaced by zero (flush to zero).[40]
Modern floating-point hardware usually handles subnormal values (as well as normal values), and does not require software emulation for subnormals.
Infinities
[edit]The infinities of the extended real number line can be represented in IEEE floating-point datatypes, just like ordinary floating-point values like 1, 1.5, etc. They are not error values in any way, though they are often (depends on the rounding) used as replacement values when there is an overflow. Upon a divide-by-zero exception, a positive or negative infinity is returned as an exact result. An infinity can also be introduced as a numeral (like C's "INFINITY" macro, or "∞" if the programming language allows that syntax).
IEEE 754 requires infinities to be handled in a reasonable way, such as
- (+∞) + (+7) = (+∞)
- (+∞) × (−2) = (−∞)
- (+∞) × 0 = NaN – there is no meaningful thing to do
NaNs
[edit]IEEE 754 specifies a special value called "Not a Number" (NaN) to be returned as the result of certain "invalid" operations, such as 0/0, ∞×0, or sqrt(−1). In general, NaNs will be propagated, i.e. most operations involving a NaN will result in a NaN, although functions that would give some defined result for any given floating-point value will do so for NaNs as well, e.g. NaN ^ 0 = 1. There are two kinds of NaNs: the default quiet NaNs and, optionally, signaling NaNs. A signaling NaN in any arithmetic operation (including numerical comparisons) will cause an "invalid operation" exception to be signaled.
The representation of NaNs specified by the standard has some unspecified bits that could be used to encode the type or source of error; but there is no standard for that encoding. In theory, signaling NaNs could be used by a runtime system to flag uninitialized variables, or extend the floating-point numbers with other special values without slowing down the computations with ordinary values, although such extensions are not common. A variant of this approach (sometimes called "NaN-boxing") is used by some JavaScript runtimes[41] and LuaJIT[42] to store 64-bit pointer values and IEEE 754 double-precision floating-point values in the same data type, allowing runtimes to eliminate the overhead of extra memory allocations and indirections for floating-point values.
Design rationale
[edit]
It is a common misconception that the more esoteric features of the IEEE 754 standard discussed here, such as extended formats, NaN, infinities, subnormals etc., are only of interest to numerical analysts, or for advanced numerical applications. In fact the opposite is true: these features are designed to give safe robust defaults for numerically unsophisticated programmers, in addition to supporting sophisticated numerical libraries by experts. The key designer of IEEE 754, William Kahan, notes that it is incorrect to "... [deem] features of IEEE Standard 754 for Binary Floating-Point Arithmetic that ...[are] not appreciated to be features usable by none but numerical experts. The facts are quite the opposite. In 1977 those features were designed into the Intel 8087 to serve the widest possible market... Error-analysis tells us how to design floating-point arithmetic, like IEEE Standard 754, moderately tolerant of well-meaning ignorance among programmers".[43]
- The special values such as infinity and NaN ensure that the floating-point arithmetic is algebraically complete: every floating-point operation produces a well-defined result and will not—by default—throw a machine interrupt or trap. Moreover, the choices of special values returned in exceptional cases were designed to give the correct answer in many cases. For instance, under IEEE 754 arithmetic, continued fractions such as
R(z) := 7 − 3/[z − 2 − 1/(z − 7 + 10/[z − 2 − 2/(z − 3)])]will give the correct answer on all inputs, as the potential divide by zero, e.g. for z = 3, is correctly handled by giving +infinity, and so such exceptions can be safely ignored.[44] As noted by Kahan, the unhandled trap consecutive to a floating-point to 16-bit integer conversion overflow that caused the loss of an Ariane 5 rocket would not have happened under the default IEEE 754 floating-point policy.[43] - Subnormal numbers ensure that for finite floating-point numbers x and y, x − y = 0 if and only if x = y, as expected, but which did not hold under earlier floating-point representations.[45]
- On the design rationale of the x87 80-bit format, Kahan notes: "This Extended format is designed to be used, with negligible loss of speed, for all but the simplest arithmetic with float and double operands. For example, it should be used for scratch variables in loops that implement recurrences like polynomial evaluation, scalar products, partial and continued fractions. It often averts premature Over/Underflow or severe local cancellation that can spoil simple algorithms".[46] Computing intermediate results in an extended format with high precision and extended exponent has precedents in the historical practice of scientific calculation and in the design of scientific calculators e.g. Hewlett-Packard's financial calculators performed arithmetic and financial functions to three more significant decimals than they stored or displayed.[46] The implementation of extended precision enabled standard elementary function libraries to be readily developed that normally gave double precision results within one unit in the last place (ULP) at high speed.
- Correct rounding of values to the nearest representable value avoids systematic biases in calculations and slows the growth of errors. Rounding ties to even removes the statistical bias that can occur in adding similar figures.
- Directed rounding was intended as an aid with checking error bounds, for instance in interval arithmetic. It is also used in the implementation of some functions.
- The mathematical basis of the operations, in particular correct rounding, allows one to prove mathematical properties and design floating-point algorithms such as 2Sum, Fast2Sum and Kahan summation algorithm, e.g. to improve accuracy or implement multiple-precision arithmetic subroutines relatively easily.
A property of the single- and double-precision formats is that their encoding allows one to easily sort them without using floating-point hardware, as if the bits represented sign-magnitude integers, although it is unclear whether this was a design consideration (it seems noteworthy that the earlier IBM hexadecimal floating-point representation also had this property for normalized numbers). With the prevalent two's-complement representation, interpreting the bits as signed integers sorts the positives correctly, but with the negatives reversed; as one possible correction for that, with an xor to flip the sign bit for positive values and all bits for negative values, all the values become sortable as unsigned integers (with −0 < +0).[35]
Recommendations
[edit]Alternate exception handling
[edit]The standard recommends optional exception handling in various forms, including presubstitution of user-defined default values, and traps (exceptions that change the flow of control in some way) and other exception handling models that interrupt the flow, such as try/catch. The traps and other exception mechanisms remain optional, as they were in IEEE 754-1985.
Recommended operations
[edit]Clause 9 in the standard recommends additional mathematical operations[47] that language standards should define.[48] None are required in order to conform to the standard.
The following are recommended arithmetic operations, which must round correctly:[49]
- , ,
- , ,
- , ,
- , ,
- for (named compound and used to compute an exponential growth, whose rate cannot be less than −1)[50]
- ,
- , ,
- , , ,
- , , (see also: Multiples of π)
- , , , (see also: Multiples of π)
- , ,
- , ,
The , and functions were not part of the IEEE 754-2008 standard because they were deemed less necessary.[51] and were mentioned, but this was regarded as an error.[5] All three were added in the 2019 revision.
The recommended operations also include setting and accessing dynamic mode rounding direction,[52] and implementation-defined vector reduction operations such as sum, scaled product, and dot product, whose accuracy is unspecified by the standard.[53]
As of 2019[update], augmented arithmetic operations[54] for the binary formats are also recommended. These operations, specified for addition, subtraction and multiplication, produce a pair of values consisting of a result correctly rounded to nearest in the format and the error term, which is representable exactly in the format. At the time of publication of the standard, no hardware implementations are known, but very similar operations were already implemented in software using well-known algorithms. The history and motivation for their standardization are explained in a background document.[55][56]
As of 2019, the formerly required minNum, maxNum, minNumMag, and maxNumMag in IEEE 754-2008 are now deprecated due to their non-associativity. Instead, two sets of new minimum and maximum operations are recommended.[57] The first set contains minimum, minimumNumber, maximum and maximumNumber. The second set contains minimumMagnitude, minimumMagnitudeNumber, maximumMagnitude and maximumMagnitudeNumber. The history and motivation for this change are explained in a background document.[58]
Expression evaluation
[edit]The standard recommends how language standards should specify the semantics of sequences of operations, and points out the subtleties of literal meanings and optimizations that change the value of a result. By contrast, the previous 1985 version of the standard left aspects of the language interface unspecified, which led to inconsistent behavior between compilers, or different optimization levels in an optimizing compiler.
Programming languages should allow a user to specify a minimum precision for intermediate calculations of expressions for each radix. This is referred to as preferredWidth in the standard, and it should be possible to set this on a per-block basis. Intermediate calculations within expressions should be calculated, and any temporaries saved, using the maximum of the width of the operands and the preferred width if set. Thus, for instance, a compiler targeting x87 floating-point hardware should have a means of specifying that intermediate calculations must use the double-extended format. The stored value of a variable must always be used when evaluating subsequent expressions, rather than any precursor from before rounding and assigning to the variable.
Reproducibility
[edit]The IEEE 754-1985 version of the standard allowed many variations in implementations (such as the encoding of some values and the detection of certain exceptions). IEEE 754-2008 has reduced these allowances, but a few variations still remain (especially for binary formats). The reproducibility clause recommends that language standards should provide a means to write reproducible programs (i.e., programs that will produce the same result in all implementations of a language) and describes what needs to be done to achieve reproducible results.
Concrete examples of potentially non-reproducible behavior can be found in C and C++, which allow the use of higher precision for results of floating-point operations and contraction of floating-point expressions, such as regular multiply-and-add into FMA and 1.0/sqrt(x) into a reciprocal square root as a single instruction.[59] C/C++ Compilers such as GCC and cl.exe generally default to allowing both unless specifically asked not to, as these changes can generate faster code without obvious loss of accuracy. Compilers also offer more overtly non-compliant "fast" optimizations.[60][61] C mathematical functions are usually not implemented to be "correctly rounded" and add to the problem.[62] The floating-point environment may also be unexpectedly changed by third-party code.
Character representation
[edit]The standard requires operations to convert between basic formats and external character sequence formats.[63] Conversions to and from a decimal character format are required for all formats. Conversion to an external character sequence must be such that conversion back using round to nearest, ties to even will recover the original number. There is no requirement to preserve the payload of a quiet NaN or signaling NaN, and conversion from the external character sequence may turn a signaling NaN into a quiet NaN.
The original binary value will be preserved by converting to decimal and back again using:[64]
- 5 decimal digits for binary16,
- 9 decimal digits for binary32,
- 17 decimal digits for binary64,
- 36 decimal digits for binary128.
For other binary formats, the required number of decimal digits is[i]
where p is the number of significant bits in the binary format, e.g. 237 bits for binary256.
When using a decimal floating-point format, the decimal representation will be preserved using:
- 7 decimal digits for decimal32,
- 16 decimal digits for decimal64,
- 34 decimal digits for decimal128.
Algorithms, with code, for correctly rounded conversion from binary to decimal and decimal to binary are discussed by Gay,[65] and for testing – by Paxson and Kahan.[66]
Hexadecimal literals
[edit]The standard recommends providing conversions to and from external hexadecimal-significand character sequences, based on C99's hexadecimal floating point literals. Such a literal consists of an optional sign (+ or -), the indicator "0x", a hexadecimal number with or without a period, an exponent indicator "p", and a decimal exponent with optional sign. The syntax is not case-sensitive.[67] The decimal exponent scales by powers of 2. For example, 0x0.1p0 is 1/16 and 0x0.1p-4 is 1/256.[68]
See also
[edit]- bfloat16 floating-point format
- Binade
- Coprocessor
- C99 for code examples demonstrating access and use of IEEE 754 features
- Floating-point arithmetic, for history, design rationale and example usage of IEEE 754 features
- Fixed-point arithmetic, for an alternative approach at computation with rational numbers (especially beneficial when the exponent range is known, fixed, or bound at compile time)
- IBM System z9, the first CPU to implement IEEE 754-2008 decimal arithmetic (using hardware microcode)
- IBM z10, IBM z196, IBM zEC12, and IBM z13, CPUs that implement IEEE 754-2008 decimal arithmetic fully in hardware
- ISO/IEC 10967, language-independent arithmetic (LIA)
- Minifloat, low-precision binary floating-point formats following IEEE 754 principles
- POWER6, POWER7, and POWER8 CPUs that implement IEEE 754-2008 decimal arithmetic fully in hardware
- strictfp, an obsolete keyword in the Java programming language that previously restricted arithmetic to IEEE 754 single and double precision to ensure reproducibility across common hardware platforms (as of Java 17, this behavior is required)
- Table-maker's dilemma for more about the correct rounding of functions
- Standard Apple Numerics Environment
- Tapered floating point
- Posit, an alternative number format
Notes
[edit]- ^ For example, if the base is 10, the sign is 1 (indicating negative), the significand is 12345, and the exponent is −3, then the value of the number is (−1)1 × 12345 × 10−3 = −1 × 12345 × 0.001 = −12.345.
- ^ Approximative values. For exact values see each format's individual Wikipedia entry
- ^ Number of digits in the radix used, including any implicit digit, but not counting the sign bit.
- ^ Corresponding number of decimal digits, see text for more details.
- ^ Contrary to decimal, there is no binary interchange format of 96-bit length. Such a format is still allowed as a non-interchange format, though.
- ^ The standard recommends 0 for signaling NaNs, 1 for quiet NaNs, so that a signaling NaNs can be quieted by changing only this bit to 1, while the reverse could yield the encoding of an infinity.
- ^ No flag is raised in certain cases of underflow.
- ^ See Fast inverse square root and Methods of computing square roots#Iterative methods for reciprocal square roots
- ^ As an implementation limit, correct rounding is only guaranteed for the number of decimal digits required plus 3 for the largest supported binary format. For instance, if binary32 is the largest supported binary format, then a conversion from a decimal external sequence with 12 decimal digits is guaranteed to be correctly rounded when converted to binary32; but conversion of a sequence of 13 decimal digits is not; however, the standard recommends that implementations impose no such limit.
References
[edit]- ^ IEEE 754 2019
- ^ Haasz, Jodi. "FW: ISO/IEC/IEEE 60559 (IEEE Std 754-2008)". IEEE. Archived from the original on 2017-10-27. Retrieved 2018-04-04.
- ^ "ISO/IEEE Partner Standards Development Organization (PSDO) Cooperation Agreement" (PDF). ISO. 2007-12-19. Retrieved 2021-12-27.
- ^ ISO/IEC JTC 1/SC 25 2011.
- ^ a b Cowlishaw, Mike (2013-11-13). "IEEE 754-2008 errata". speleotrove.com. Retrieved 2020-01-24.
- ^ "ANSI/IEEE Std 754-2019". ucbtest.org. Retrieved 2024-01-16.
- ^ ISO/IEC JTC 1/SC 25 2020.
- ^ "Issues for the next revision of 754". IEEE. Retrieved 2024-08-12.
- ^ IEEE 754 2008, §2.1.27.
- ^ "SpiderMonkey Internals". udn.realityripple.com. Retrieved 2018-03-11.
- ^ Klemens, Ben (September 2014). 21st Century C: C Tips from the New School. O'Reilly Media, Incorporated. p. 160. ISBN 9781491904442. Retrieved 2018-03-11.
- ^ "zuiderkwast/nanbox: NaN-boxing in C". GitHub. Retrieved 2018-03-11.
- ^ IEEE 754 2008, §3.6.
- ^ IEEE 754 2008, §3.7.
- ^ IEEE 754 2008, §3.7 states: "Language standards should define mechanisms supporting extendable precision for each supported radix."
- ^ IEEE 754 2008, §3.7 states: "Language standards or implementations should support an extended precision format that extends the widest basic format that is supported in that radix."
- ^ Motorola MC68000 Family (PDF). Programmer's Reference Manual. NXP Semiconductors. 1992. pp. 1–16, 1–18, 1–23.
- ^ IEEE 754 2008, §4.3.1. "In the following two rounding-direction attributes, an infinitely precise result with magnitude at least shall round to with no change in sign."
- ^ IEEE 754 2008, §4.3.3
- ^ IEEE 754 2019, §2.1
- ^ a b c IEEE 754 2008, §5.3.1
- ^ a b IEEE 754 2008, §5.4.1
- ^ IEEE 754 2008, §5.4.2
- ^ IEEE 754 2008, §5.4.3
- ^ IEEE 754 2008, §5.3.2
- ^ IEEE 754 2008, §5.3.3
- ^ IEEE 754 2008, §5.5.1
- ^ IEEE 754 2008, §5.10
- ^ IEEE 754 2008, §5.11
- ^ IEEE 754 2008, §5.7.2
- ^ IEEE 754 2008, §5.7.4
- ^ IEEE 754 2019, §5.11
- ^ a b c IEEE 754 2019, §5.10
- ^ "Implement total_cmp for f32, f64 by golddranks · Pull Request #72568 · rust-lang/rust". GitHub. – contains relevant quotations from IEEE 754-2008 and -2019. Contains a type-pun implementation and explanation.
- ^ a b Herf, Michael (December 2001). "radix tricks". stereopsis: graphics.
- ^ "9.4. decimal — Decimal fixed point and floating point arithmetic — Python 3.6.5 documentation". docs.python.org. Retrieved 2018-04-04.
- ^ "Decimal Arithmetic - Exceptional conditions". speleotrove.com. Retrieved 2018-04-04.
- ^ IEEE 754 2008, §7.2(h)
- ^ Goldberg 1991.
- ^ Muller, Jean-Michel; Brisebarre, Nicolas; de Dinechin, Florent; Jeannerod, Claude-Pierre; Lefèvre, Vincent; Melquiond, Guillaume; Revol, Nathalie; Stehlé, Damien; Torres, Serge (2010). Handbook of Floating-Point Arithmetic (1 ed.). Birkhäuser. doi:10.1007/978-0-8176-4705-6. ISBN 978-0-8176-4704-9.
- ^ Wingo, Andy (2011-05-18). "value representation in javascript implementations". wingolog. Archived from the original on 2025-08-21. Retrieved 2025-09-09.
- ^ Pall, Mike (2009-11-02). "LuaJIT 2.0 intellectual property disclosure and research opportunities". gmane.comp.lang.lua.general (Usenet). Archived from the original (email) on 2009-11-07. Retrieved 2025-09-09.
Design aspects of the VM: [...] NaN-tagging: 64 bit tagged values are used for stack slots and table slots.
- ^ a b Kahan, William Morton; Darcy, Joseph (2001) [1998-03-01]. "How Java's floating-point hurts everyone everywhere" (PDF). Archived (PDF) from the original on 2000-08-16. Retrieved 2003-09-05.
- ^ Kahan, William Morton (1981-02-12). "Why do we need a floating-point arithmetic standard?" (PDF). p. 26. Archived (PDF) from the original on 2004-12-04.
- ^ Severance, Charles (1998-02-20). "An Interview with the Old Man of Floating-Point".
- ^ a b Kahan, William Morton (1996-06-11). "The Baleful Effect of Computer Benchmarks upon Applied Mathematics, Physics and Chemistry" (PDF). Archived (PDF) from the original on 2013-10-13.
- ^ IEEE 754 2019, §9.2
- ^ IEEE 754 2008, Clause 9
- ^ IEEE 754 2019, §9.2.
- ^ "Too much power - pow vs powr, powd, pown, rootn, compound". IEEE. Retrieved 2024-01-16.
Since growth rates can't be less than -1, such rates signal invalid exceptions.
- ^ "Re: Missing functions tanPi, asinPi and acosPi". IEEE. Archived from the original on 2017-07-06. Retrieved 2018-04-04.
- ^ IEEE 754 2008, §9.3.
- ^ IEEE 754 2008, §9.4.
- ^ IEEE 754 2019, §9.5
- ^ Riedy, Jason; Demmel, James. "Augmented Arithmetic Operations Proposed for IEEE-754 2018" (PDF). 25th IEEE Symbosium on Computer Arithmetic (ARITH 2018). pp. 49–56. Archived (PDF) from the original on 2019-07-23. Retrieved 2019-07-23.
- ^ "ANSI/IEEE Std 754-2019 – Background Documents". IEEE. Retrieved 2024-01-16.
- ^ IEEE 754 2019, §9.6.
- ^ Chen, David. "The Removal/Demotion of MinNum and MaxNum Operations from IEEE 754-2018" (PDF). IEEE. Retrieved 2024-01-16.
- ^ Beeraka, Gautham (2021-12-14). "The /fp:contract flag and changes to FP modes in VS2022". devblogs.microsoft.com. Microsoft. Retrieved 2025-06-09.
- ^ "Optimize Options (Using the GNU Compiler Collection (GCC))". gcc.gnu.org.
- ^ "/fp (Specify floating-point behavior)". learn.microsoft.com.
- ^ "Does any floating point-intensive code produce bit-exact results in any x86-based architecture?". Stack Overflow.
- ^ IEEE 754 2008, §5.12.
- ^ IEEE 754 2008, §5.12.2.
- ^ Gay, David M. (1990-11-30), Correctly rounded binary-decimal and decimal-binary conversions, Numerical Analysis Manuscript, Murry Hill, NJ, US: AT&T Laboratories, 90-10
- ^ Paxson, Vern; Kahan, William (1991-05-22), A Program for Testing IEEE Decimal–Binary Conversion, Manuscript, CiteSeerX 10.1.1.144.5889
- ^ IEEE 754 2008, §5.12.3
- ^ "6.9.3. Hexadecimal floating point literals — Glasgow Haskell Compiler 9.3.20220129 User's Guide". ghc.gitlab.haskell.org. Retrieved 2022-01-29.
Standards
[edit]- IEEE Standard for Binary Floating-Point Arithmetic. ANSI/IEEE STD 754-1985. IEEE. 1985-10-12. pp. 1–20. doi:10.1109/IEEESTD.1985.82928. ISBN 0-7381-1165-1.
- IEEE Computer Society (2008-08-29). IEEE Standard for Floating-Point Arithmetic. IEEE STD 754-2008. IEEE. pp. 1–70. doi:10.1109/IEEESTD.2008.4610935. ISBN 978-0-7381-5753-5. IEEE Std 754-2008.
- IEEE Computer Society (2019-07-22). IEEE Standard for Floating-Point Arithmetic. IEEE STD 754-2019. IEEE. pp. 1–84. doi:10.1109/IEEESTD.2019.8766229. ISBN 978-1-5044-5924-2. IEEE Std 754-2019.
- ISO/IEC JTC 1/SC 25 (June 2011). ISO/IEC/IEEE 60559:2011 — Information technology — Microprocessor Systems — Floating-Point arithmetic. ISO. pp. 1–58.
{{cite book}}: CS1 maint: numeric names: authors list (link) - ISO/IEC JTC 1/SC 25 (May 2020). ISO/IEC 60559:2020 — Information technology — Microprocessor Systems — Floating-Point arithmetic. ISO. pp. 1–74.
{{cite book}}: CS1 maint: numeric names: authors list (link)
Secondary references
[edit]- Decimal floating-point arithmetic, FAQs, bibliography, and links
- Comparing binary floats
- IEEE 754 Reference Material
- IEEE 854-1987 – History and minutes
- Supplementary readings for IEEE 754. Includes historical perspectives.
Further reading
[edit]- Goldberg, David (March 1991). "What Every Computer Scientist Should Know About Floating-Point Arithmetic". ACM Computing Surveys. 23 (1): 5–48. doi:10.1145/103162.103163. (With the addendum "Differences Among IEEE 754 Implementations": [1], [2])
- Hecker, Chris (February 1996). "Let's Get To The (Floating) Point" (PDF). Game Developer: 19–24.
- Severance, Charles (March 1998). "IEEE 754: An Interview with William Kahan" (PDF). IEEE Computer. 31 (3): 114–115. doi:10.1109/MC.1998.660194. Retrieved 2019-03-08.
- Cowlishaw, Mike (June 2003). "Decimal floating-point: Algorism for computers" (PDF). 16th IEEE Symposium on Computer Arithmetic, 2003. Proceedings. Los Alamitos, Calif.: IEEE Computer Society. pp. 104–111. doi:10.1109/ARITH.2003.1207666. ISBN 978-0-7695-1894-7. Retrieved 2014-11-14..
- Monniaux, David (May 2008). "The pitfalls of verifying floating-point computations". ACM Transactions on Programming Languages and Systems. 30 (3): 1–41. arXiv:cs/0701192. doi:10.1145/1353445.1353446.: A compendium of non-intuitive behaviours of floating-point on popular architectures, with implications for program verification and testing.
- Muller, Jean-Michel; Brunie, Nicolas; de Dinechin, Florent; Jeannerod, Claude-Pierre; Joldes, Mioara; Lefèvre, Vincent; Melquiond, Guillaume; Revol, Nathalie; Torres, Serge (2018) [2010]. Handbook of Floating-Point Arithmetic (2 ed.). Birkhäuser. doi:10.1007/978-3-319-76526-6. ISBN 978-3-319-76525-9.
- Overton, Michael L. (2001). Written at Courant Institute of Mathematical Sciences, New York University, New York, US. Numerical Computing with IEEE Floating Point Arithmetic (1 ed.). Philadelphia, US: SIAM. doi:10.1137/1.9780898718072. ISBN 978-0-89871-482-1. 978-0-89871-571-2, 0-89871-571-7. 2nd edition, 2025. SIAM. ISBN 978-1-61197-840-7.
- Cleve Moler on Floating Point numbers
- Beebe, Nelson H. F. (2017-08-22). The Mathematical-Function Computation Handbook - Programming Using the MathCW Portable Software Library (1 ed.). Salt Lake City, UT, US: Springer International Publishing AG. doi:10.1007/978-3-319-64110-2. ISBN 978-3-319-64109-6.
- Hough, David G. (December 2019). "The IEEE Standard 754: One for the History Books". Computer. 52 (12). IEEE: 109–112. doi:10.1109/MC.2019.2926614.
External links
[edit]- Kahan on creating IEEE Standard Floating Point. Turing Awardee Clips. 2020-11-16. Archived from the original on 2021-11-08.
IEEE 754
View on GrokipediaDevelopment
History
The development of the IEEE 754 standard began in the mid-1970s amid growing concerns over inconsistencies in floating-point arithmetic implementations across different computer systems, which hindered portability and reliability in numerical computations.[2] In 1976, Intel recruited University of California, Berkeley professor William Kahan as a consultant to design the floating-point unit for its 8087 coprocessor, where he identified key issues and advocated for standardization.[5] The IEEE Floating-Point Working Group (p754) was formed in 1977, with initial meetings addressing these challenges; Kahan, along with graduate student Jerome Coonen and visiting professor Harold Stone, drafted the influential K-C-S proposal that year, outlining binary formats and arithmetic operations.[2] Debates over features like gradual underflow persisted through the late 1970s and early 1980s, but demonstrations of feasible implementations, such as George Taylor's work at Berkeley, helped build consensus.[5] A pivotal contribution came in 1981 with Kahan's report, "Why Do We Need a Floating-Point Arithmetic Standard?", which articulated the need for uniform behavior to ensure reproducible results and simplify error analysis in scientific computing.[6] This document, along with ongoing committee efforts, led to the finalization of the standard. The IEEE 754-1985 standard for binary floating-point arithmetic was published in 1985, specifying interchange formats, basic operations, and exception handling to promote consistency across hardware and software. It quickly gained traction, with major microprocessor manufacturers like Intel and AMD implementing it by 1984, even before official ratification.[5] In 1989, the standard was adopted internationally by the ISO as IEC 60559, extending its influence globally.[6] The standard underwent significant revision in 2008 with IEEE 754-2008, which expanded to include decimal floating-point formats for applications requiring exact decimal representation, such as financial computing, alongside binary formats.[7] This update also introduced the fused multiply-add operation for improved accuracy and efficiency in chained computations, enhanced exception handling for better diagnostics, and clarified rules for interformat operations.[8] A minor revision followed in 2019 as IEEE 754-2019, focusing on bug fixes, clarifications to ambiguous language, and consistent exception handling across formats without introducing major new features.[9] It added recommended optional operations, including augmented addition, subtraction, and multiplication to support higher-precision accumulations and reproducible results in parallel computing, while maintaining backward compatibility with prior implementations.[9] Approved on June 13, 2019, and published in July, this version was subsequently harmonized with ISO/IEC 60559:2020.Design Rationale
The development of IEEE 754 was driven by the need for portability and reproducibility in floating-point computations across diverse hardware platforms, as varying implementations in earlier systems led to inconsistent results that hindered software development and debugging.[5] William Kahan, a primary architect, emphasized that reliable portable numerical software was becoming prohibitively expensive amid the proliferation of microprocessors, necessitating a standard to ensure consistent behavior for both novice and expert programmers.[5] This focus addressed the fragmentation caused by proprietary formats, enabling software to produce the same outputs regardless of the underlying hardware.[10] A key design choice was the adoption of a binary radix over decimal, prioritizing hardware efficiency and computational speed while balancing precision and dynamic range.[4] Although Kahan initially advocated for decimal to better align with human-readable numbers, industry pressures, particularly from Intel, favored binary for its simpler implementation in digital circuits and faster arithmetic operations.[10] Binary formats provided denser representations and easier alignment with integer operations, though they introduced challenges like non-terminating decimal fractions; this trade-off was deemed essential for widespread adoption in computing hardware.[5] To mitigate abrupt precision loss during underflow, the standard introduced gradual underflow via subnormal numbers, which fill the gap between zero and the smallest normal numbers, preserving partial accuracy for small values.[4] Kahan and David Goldberg championed this feature after heated debates spanning six years, arguing it reduced the risk of catastrophic errors for unaware programmers without significantly impacting performance.[10] Subnormals ensure a smooth transition in magnitude, avoiding the "chasm" of sudden precision drop-off seen in prior systems.[5] The inclusion of infinities and Not-a-Number (NaN) values was motivated by the need to handle overflow and invalid operations gracefully, preventing program crashes and facilitating robust numerical algorithms.[4] Infinities represent unbounded results, such as from division by zero, allowing computations to continue while signaling extremes, whereas NaNs propagate through operations to indicate undefined states like square roots of negatives.[5] Kahan specifically advocated for NaN payloads—unused bits to encode diagnostic information, such as error origins or program addresses—to aid debugging, though adoption was limited beyond brief implementations like on HP processors.[10] Trade-offs between fixed-point and floating-point formats favored the latter for its dynamic range, with support for multiple precisions (single, double, and extended) to accommodate varying application needs without mandating a one-size-fits-all approach.[4] Floating-point enabled scalable exponentiation for scientific computing, while multiple precisions balanced storage costs and accuracy; for instance, double precision's 11-bit exponent drew from DEC VAX designs but expanded range.[5] This flexibility addressed limitations in earlier systems like the Cray-1, which offered wide ranges but violated associativity, and the DEC VAX, with its narrower exponents prone to overflow—IEEE 754 improved reproducibility and exception handling to overcome these flaws.[10]Formats and Encoding
Representation and Encoding in Memory
The IEEE 754 standard defines the encoding of floating-point numbers in memory as a fixed-length bit string comprising three fields: a sign bit, a biased exponent field of e bits, and a significand field of f bits.[11] The sign bit s (0 for positive, 1 for negative) indicates the sign of the number.[12] For binary formats, the biased exponent E represents the true exponent plus a bias of to allow unsigned storage while preserving order for comparisons.[12] In normalized binary representations, the exponent field is neither all zeros nor all ones, and the significand is normalized with an implicit leading 1 bit not stored in the f-bit fraction field M.[13] The value of such a number is given byBasic and Interchange Formats
The IEEE 754 standard defines a set of basic and interchange formats for binary and decimal floating-point numbers, establishing minimum requirements for consistent representation and portable computation across diverse systems.[1] These formats specify fixed bit lengths, exponent fields, and significand fields, enabling reliable data interchange without loss of precision or range when adhering to the standard's encoding rules.[11] The basic formats serve as the foundation for arithmetic operations, while the interchange formats ensure compatibility in serialized data transmission or storage.[1]Binary Basic and Interchange Formats
The binary formats use a sign bit, an exponent field in biased form, and a significand field with an implicit leading bit for normalized numbers, following a sign-exponent-significand layout.[11] The standard mandates four binary interchange formats, each with increasing precision and range suitable for applications from embedded systems to high-performance computing.[1]| Format | Total Bits | Exponent Bits | Significand Bits | Precision (p bits) |
|---|---|---|---|---|
| binary16 | 16 | 5 | 10 | 11 |
| binary32 | 32 | 8 | 23 | 24 |
| binary64 | 64 | 11 | 52 | 53 |
| binary128 | 128 | 15 | 112 | 113 |
Decimal Basic and Interchange Formats
Decimal formats in IEEE 754 are designed for applications requiring exact representation of decimal fractions, such as financial computations, using a combination field that encodes both exponent and trailing significand digits in a densely packed decimal (DPD) scheme.[1] The standard requires three decimal interchange formats, encoded in 32, 64, and 128 bits, to support varying levels of decimal precision.[11]| Format | Total Bits | Significand Digits | Precision (p digits) |
|---|---|---|---|
| decimal32 | 32 | 7 | 7 |
| decimal64 | 64 | 16 | 16 |
| decimal128 | 128 | 34 | 34 |
Extended and Extendable Precision Formats
Extended precision formats in IEEE 754 provide optional higher precision and range beyond the basic binary and decimal formats, primarily for use in intermediate computations to minimize rounding errors. A well-known example is the 80-bit binary extended precision format, commonly implemented in Intel's x87 floating-point unit (FPU), which allocates 1 bit for the sign, 15 bits for the biased exponent (with a bias of 16383), and 64 bits for the significand, including an explicit leading integer bit unlike the implicit bit in basic formats. This structure allows for a precision of approximately 19 decimal digits and an exponent range from -16382 to 16383, enabling more accurate accumulation of results before rounding to basic formats.[16][17] Extendable precision formats extend this flexibility further by supporting user-defined significand lengths and exponent ranges, allowing implementations to go beyond fixed basic formats like binary64 or binary128 through software or custom hardware. For instance, while binary128 (quadruple precision) serves as a basic 128-bit interchange format with 1 sign bit, 15 exponent bits, and 112 significand bits (113 bits of precision including the implicit bit), the standard recommends extendable variants up to binary256 (octuple precision), which features 1 sign bit, 19 exponent bits (bias 262143), and 236 significand bits for even greater accuracy in demanding applications. These formats are not required for interchange but facilitate higher numerical stability in computations requiring extensive dynamic range or precision.[1][17] In contrast to basic formats, extended and extendable precisions emphasize larger significands and exponents to achieve superior resolution and magnitude coverage, often at the cost of increased storage and computational overhead, and they remain optional to encourage portability. The Intel x87 FPU exemplifies their practical use, retaining 80-bit precision internally during operations on 32-bit or 64-bit inputs to preserve accuracy until final storage.[16][1] The IEEE 754-2019 revision clarified guidelines for these formats without imposing new requirements, refining definitions for extendable precisions to better support simulation of higher accuracies via recommended operations like augmented addition, while ensuring consistency in exceptional handling across implementations.[1][18]Binary Interchange Formats
The IEEE 754 standard defines binary interchange formats as fixed-length bit strings designed for the efficient and portable exchange of floating-point data between different systems, utilizing a radix of 2 for computational efficiency.[11] These formats follow a general structure consisting of a sign bit, a biased exponent field, and a significand field, with normalized numbers featuring an implicit leading bit of 1 in the significand.[11] Implementations must support the binary32 (single-precision, 32 bits) and binary64 (double-precision, 64 bits) interchange formats, while binary16 (half-precision, 16 bits) and binary128 (quadruple-precision, 128 bits) are optional.[11] For binary32, the format allocates 1 bit for the sign, 8 bits for the biased exponent (with bias 127), and 23 bits for the trailing significand field; the exponent range uses values 1 through 254 for normalized numbers (true exponent -126 to +127), 0 for subnormals and zero, and 255 for infinities and NaNs.[11] Similarly, binary64 employs a bias of 1023 across 11 exponent bits (range 1 to 2046 for normalized, true exponent -1022 to +1023), with 52 trailing significand bits.[11] Conversion between these precisions for interchange purposes follows the standard's rounding rules, typically to nearest (with ties to even) to preserve accuracy while fitting the destination format; exact conversions are possible when the source value is representable in the target without loss.[11] The IEEE 754-2019 revision introduced clarifications ensuring consistent handling of subnormal numbers during such interchanges, mandating that subnormals be preserved or rounded appropriately without flushing to zero unless specified.[11] For file-based interchange, the standard recommends big-endian byte order to facilitate readability across heterogeneous systems, though the actual serialization remains implementation-defined.[11]Decimal Interchange Formats
The IEEE 754 standard defines three decimal interchange formats—decimal32, decimal64, and decimal128—for the precise representation and exchange of decimal floating-point numbers, particularly suited to applications requiring exact decimal arithmetic, such as financial computations.[19] These formats use a base-10 radix to avoid the rounding errors inherent in binary representations for common decimal fractions like 0.1, ensuring that values like are stored exactly without approximation.[20] This exactness is critical in domains like banking, where discrepancies from binary floating-point can accumulate and lead to significant errors in summations or interest calculations. Each format encodes a signed value as , where is the sign bit (0 for positive, 1 for negative), is the integer significand with a fixed number of decimal digits ( for decimal32, for decimal64, for decimal128), and is the unbiased exponent.[21] The significand is constructed from leading digits encoded in the combination field and trailing digits in the significand field, allowing multiple representations (cohorts) for the same value to optimize for exact decimal powers of 10. The exponent is biased to permit both positive and negative values, with biases of 101 for decimal32 (exponent range -95 to 96), 398 for decimal64 (-383 to 384), and 6176 for decimal128 (-6143 to 6144).[20] The bit layout begins with a 1-bit sign, followed by a 5-bit combination field that encodes 2 bits of the exponent and the leading 3 or 4 decimal digits of the significand, depending on whether the leading digit is small (0-7) or large (8-9).[22] This is followed by exponent continuation bits (8 for decimal32, 12 for decimal64, 20 for decimal128) and trailing significand bits (18, 46, and 102 bits, respectively). The trailing significand uses densely packed decimal (DPD) encoding, which represents groups of 3 decimal digits in 10 bits, achieving 3.32 bits per digit efficiency compared to 3.32 bits for unpacked binary-coded decimal but with denser packing for interchange.[21] DPD was designed by the IEEE 754r committee to balance compactness and decodability, using a canonical set of 1000 valid 10-bit patterns (declets) out of 1024 possible, with invalid patterns reserved for future extensions or error detection.[20] These formats were introduced in the 2008 revision of IEEE 754 to standardize decimal floating-point interchange, addressing the need for portable, exact decimal representations across systems.[20] The 2019 revision provided clarifications and fixes for edge cases in DPD encoding, such as handling certain invalid declet patterns and refining cohort adjustments to ensure consistent decoding of special values like infinities and NaNs in decimal contexts.[23]| Format | Bits | Precision ( digits) | Exponent Bias | Combination Field | Exponent Continuation Bits | Trailing Significand Bits (DPD) | Max Value (approx.) |
|---|---|---|---|---|---|---|---|
| decimal32 | 32 | 7 | 101 | 5 | 8 | 18 | |
| decimal64 | 64 | 16 | 398 | 5 | 12 | 46 | |
| decimal128 | 128 | 34 | 6176 | 5 | 20 | 102 |
Arithmetic Operations
Rounding Rules
IEEE 754 specifies that arithmetic operations and format conversions produce results at infinite precision before rounding to the destination format's precision, ensuring the rounded result is as faithful as possible to the exact value.[17] The standard defines five rounding modes to control this process, allowing users to select behaviors suitable for general computation, error bounding, or specific numerical analyses.[17] These modes are dynamically selectable via a status register, with round to nearest, ties to even, as the default for binary formats.[17] The primary rounding mode, round to nearest, ties to even (also known as roundTiesToEven), selects the representable floating-point number closest to the exact result.[17] In cases where the exact result is precisely midway between two representable values, it rounds to the one whose least significant bit is zero (even), minimizing cumulative rounding errors over multiple operations.[24] This mode provides unbiased rounding on average and is the recommended default for most applications.[17] IEEE 754-2008 introduced a fifth rounding mode, round to nearest, ties to away (roundTiesToAway), which rounds midway cases away from zero to the nearest representable value.[25] The three directed rounding modes—round toward positive infinity (roundTowardPositive), round toward negative infinity (roundTowardNegative), and round toward zero (roundTowardZero)—always select the representable value in the specified direction from the exact result.[17] These modes are essential for applications requiring strict bounds, such as interval arithmetic, where roundTowardPositive provides an upper bound and roundTowardNegative a lower bound on the result.[24] In implementations, precise rounding is often achieved using extra bits beyond the destination precision: the guard bit (immediately after the least significant bit), the round bit (next), and the sticky bit (OR of all remaining bits).[26] These bits detect whether truncation would discard significant information, enabling correct decisions for all modes without excessive precision loss during intermediate computations.[26] For the round to nearest modes, the maximum rounding error is bounded by half a unit in the last place (ulp) of the result:Required Operations
IEEE 754 mandates support for a core set of arithmetic operations and conversions to ensure portability and consistency across implementations. These operations apply to all supported formats, including binary and decimal floating-point representations, and are defined in Clause 5 of the standard.[11][27] The basic arithmetic operations required are addition, subtraction, multiplication, division, and square root. For finite non-zero inputs of the same format, these operations compute the result as if performed with infinite precision and then rounded to the destination format according to the specified rounding mode.[11][28] Subtraction and division handle signed zeros and infinities appropriately, while square root is defined only for non-negative operands, signaling invalid operation otherwise.[11] Conversions are also required, including between floating-point formats (e.g., binary16 to binary64), from floating-point to integer (with specified rounding toward zero and overflow handling), from integer to floating-point, and bidirectional conversions between floating-point numbers and external decimal character sequences. These ensure accurate interchange of data, with decimal-binary conversions preserving exact representability where possible.[11][18] Since the 2008 revision, fused multiply-add (FMA) has been required for binary formats, computing the result of as if in a single operation with extended precision, followed by one final rounding to the destination format: . This reduces rounding error compared to separate multiplication and addition, which would involve two roundings.[11][28] The operation signals underflow or overflow only if the exact result does so.[11] The remainder operation is required, defined as , where is the integer closest to (ties to even), yielding a result with the same sign as and magnitude less than . This supports exact divisions without full quotient computation, distinct from modulo which may differ in sign handling.[11][29] The 2019 revision introduced augmented operations, such as augmented addition, which are recommended but enhance required capabilities by returning both a rounded result and an exact error term (the difference to the infinite-precision sum), enabling error tracking in accumulations. For example, augmented addition outputs a pair where is exact.[18][30]Comparison Predicates
IEEE 754 defines six fundamental comparison predicates for floating-point numbers: equality (==), inequality (!=), less than (<), less than or equal (<=), greater than (>), and greater than or equal (>=). These predicates operate on operands of the same format and produce results based on the numerical values they represent, yielding one of four possible relations: less than, equal, greater than, or unordered (the latter occurring only when at least one operand is a NaN).[11] The predicates are required to be supported in all IEEE 754-compliant implementations and form the basis for ordering in arithmetic and algorithmic contexts.[11] For finite non-zero numbers, the ordering follows a lexicographical comparison aligned with their numerical magnitude: first by sign (negative values precede positive ones), then by exponent (larger exponents indicate greater magnitude for the same sign), and finally by significand (larger significands indicate greater values when sign and exponent match). This ensures that the predicates reflect the real-number ordering for representable finites, such as -2 < -1 < 0 < 1 < 2.[11] Infinities are positioned at the extremes of this order, with negative infinity less than all finite values and positive infinity greater than all finites; infinities of the same sign compare equal, while those of opposite signs follow the sign-based ordering (-∞ < +∞).[11] Special values introduce nuanced behaviors to maintain consistency with the standard's arithmetic model. Signed zeros compare equal under all predicates (+0 == -0, and neither is less nor greater than the other), despite their distinct representations, to preserve results from operations like subtraction of equals.[11] NaNs, however, are incomparable to all values, including themselves: any predicate involving a NaN returns false for <, <=, >, >=, and == (thus true for !=), establishing an "unordered" relation without implying order.[11] Quiet NaNs propagate unchanged in these comparisons, while signaling NaNs may trigger an invalid operation exception in signaling variants of the predicates (e.g., compareSignalingLess), though quiet versions do not signal.[11] The absence of a total order arises primarily from NaNs, as they cannot be reliably placed relative to other values, preventing strict sorting without additional mechanisms.[11] Implementations must avoid pitfalls like direct bitwise comparisons, which fail due to the biased exponent encoding (where larger bit patterns do not always mean larger values) and special encodings for zeros, subnormals, and infinities; instead, they decode to numerical values or use standard-compliant logic to evaluate the predicates correctly.[11]Total-Ordering Predicate
The totalOrder predicate establishes a total ordering among all canonical floating-point values in a supported format, ensuring every pair of values from the same format is comparable via a strict weak ordering that agrees with the standard comparison predicates wherever those are defined. Introduced as a required operation in IEEE Std 754-2008 and refined for clarity in the 2019 revision, it addresses limitations in standard comparisons by incorporating NaNs into the order and distinguishing signed zeros, thereby enabling applications like sorting or using floating-point values as keys in ordered data structures without exception handling for special cases.[11] The predicate totalOrder(x, y) returns true if x precedes y in the total order (i.e., x is "less than" y under this relation). For non-NaN operands, it follows the numerical order: negative values precede positive values, with subnormal numbers ordered before normal numbers of the same sign based on their magnitudes, -∞ as the smallest value, and +∞ immediately before NaNs. Specifically, it treats -0 as less than +0, even though they are numerically equal under standard equality. When one or both operands are NaNs, the ordering places all negative NaNs before all non-NaN values and all positive NaNs after, with -0 < +0 preserved in context. Among NaNs, the order is determined first by the sign bit (negative before positive), then by NaN type (signaling NaNs before quiet NaNs for positive sign, reversed for negative sign), and finally by the payload bits in bit-pattern order, allowing consistent placement without signaling exceptions.[11] This bit-pattern-based ordering for NaNs ensures payloads with smaller bit representations precede those with larger ones within their category, promoting reproducibility in sorting while avoiding arbitrary implementation choices beyond the standard's rules. The predicate is computed without signaling any exceptions, making it suitable for quiet operations in numerical software. For example, totalOrder(-∞, +∞) returns true, totalOrder(+0, -0) returns false, and totalOrder(any_number, any_qNaN) returns true if the NaN is positive.[11] Implementation of totalOrder often leverages the format's bit representation for efficiency: the entire encoding is compared as an unsigned integer after inverting the sign bit (via XOR with the format's sign mask), which groups negative values (now starting with 0) before positive ones (starting with 1) and handles positive infinity as the largest non-NaN by its all-ones exponent field post-inversion; additional logic corrects the reversed magnitude order within negative values to align with numerical expectations. This approach works for binary interchange formats and extends analogously to decimal formats using their cohort and significand bits. The 2019 clarification relaxed NaN payload ordering to implementation-defined where not constrained by the type and sign, while preserving the overall structure from 2008.[11]Recommended Operations
The IEEE 754-2019 standard recommends several optional operations beyond the required arithmetic to support enhanced numerical computations, particularly for improving precision and controlling overflow in algorithms. These operations are not mandatory for conformance but are encouraged for implementations where hardware or software support is available, enabling better portability and accuracy in applications such as scientific simulations and financial modeling. Among these, the nextUp and nextDown functions provide access to adjacent representable floating-point numbers, facilitating tasks like interval arithmetic and error analysis. Specifically, nextUp(x) returns the smallest floating-point number greater than x (or +∞ if x is the maximum finite value), while nextDown(x) returns the largest floating-point number less than x (or -∞ if x is the minimum finite value); both are quiet operations except when input is a signaling NaN, and they preserve the preferred exponent range. These functions integrate with required operations like fused multiply-add (fma) for fine-grained control in iterative methods. The minNum and maxNum operations compute the minimum or maximum of two inputs while handling NaNs gracefully: if one operand is a NaN and the other is numeric, the numeric value is returned; if both are NaNs or both numeric, the standard min/max rules apply, ignoring signed zeros in comparisons. In the 2019 revision, these are supplemented by refined variants like minimumNumber and maximumNumber (which propagate NaNs) to address limitations in prior versions, promoting associativity in reductions.[31] For overflow-prone computations, the scaled product operation computes the product of a sequence of floating-point numbers scaled by a power of the radix (2 for binary formats), returning both the scaled result and the scaling exponent as an integer; this allows dot products or summations to avoid intermediate overflows by adjusting the scale post-computation. The logb function extracts the unbiased exponent of a finite non-zero input as an integer, with the preferred status exponent set to zero, aiding in normalization and range checks without full logarithm computation. Division-related recommendations include remainder(x, y), which returns x - y * n where n is the integer closest to x/y (ties to even), ensuring |remainder| ≤ |y|/2 and the sign matches x; this differs from fmod(x, y), which uses truncation toward zero for n, producing a remainder with the same sign as x and |fmod| < |y|. Both support precise modular arithmetic in loops and simulations. A key addition in IEEE 754-2019 is augmented arithmetic, which provides higher effective precision by returning pairs of results: for example, augmentedAdd(x, y) yields a rounded sum s and an error term e such that s + e equals the exact sum, using roundTiesToZero for consistency. Similar operations exist for subtraction and multiplication, enabling exact representation of accumulated errors in summations or products, which benefits reproducible parallel reductions and emulated higher-precision arithmetic like double-double. These enhance accuracy in numerical algorithms without requiring extended formats, particularly in fields like climate modeling where bit-for-bit reproducibility is critical.[32]Exception Handling
Types of Exceptions
IEEE 754-2019 defines five standard floating-point exceptions that occur during arithmetic operations when specific conditions are met, signaling potential issues in computation without necessarily halting execution.[11] These exceptions are invalid operation, division by zero, overflow, underflow, and inexact, each associated with a dedicated status flag that is set (as a "sticky bit") upon occurrence to indicate that the condition has been detected at least once since the flag was last cleared.[11] The flags enable asynchronous detection, allowing programs to check and respond after the fact, in contrast to optional synchronous traps that interrupt execution immediately.[11] The invalid operation exception is signaled for operations lacking a well-defined mathematical result, such as the square root of a negative number, indeterminate forms like 0/0 or ∞ - ∞, or computations involving signaling NaNs.[11] In IEEE 754-2019, handling of signaling NaNs has been unified such that they consistently propagate or trigger this exception across operations, including conversions, to ensure predictable behavior.[11] The default result is a quiet NaN, and the invalid operation flag is raised.[11] The division by zero exception occurs when a finite non-zero operand is divided by zero, or in related operations like computing the logarithm base b of zero, producing an exact infinite result of appropriate sign.[11] This signals the divideByZero flag, with the default outcome being ±∞ matching the sign of the exact result.[11] The overflow exception is triggered when the magnitude of a rounded floating-point result exceeds the largest finite number representable in the destination format, after rounding to the nearest representable value.[11] The default result is infinity with the sign of the intermediate value, and both the overflow and inexact flags are set.[11] For instance, this can produce infinities as special values during arithmetic.[11] The underflow exception arises when a non-zero result is so small that it cannot be represented as a normalized number without underflowing the tiniest normalized value, leading to either a subnormal number or zero after rounding.[11] It is signaled if the result is inexact and tiny (between zero and the smallest normalized magnitude), with the underflow flag raised; implementations support gradual underflow by default using subnormals for smoother transitions.[11] Finally, the inexact exception is signaled whenever the rounded result of an operation differs from the infinitely precise result of the abstract operation, due to information loss during rounding to the destination format.[11] This commonly occurs in most non-exact computations and sets the inexact flag, delivering the appropriately rounded value as the result.[11]Standard Handling Mechanisms
The IEEE 754 standard specifies a default exception handling mechanism that allows computations to continue uninterrupted by producing a best possible result while signaling the occurrence of exceptional conditions through status flags. Under this non-trapping model, when an exception arises during an operation, the system delivers a default result—such as infinity for overflow, zero for underflow to zero, or a quiet NaN for invalid operations—and proceeds with execution without halting. This approach prioritizes robustness in numerical programs, enabling them to handle edge cases gracefully while deferring detailed error management to the programmer. Central to the standard handling are five status flags, corresponding to the exceptions of invalid operation, division by zero, overflow, underflow, and inexact result. These flags are raised as side effects of operations that encounter exceptional conditions, providing a record of what occurred without altering the computational flow. Flags can be queried, cleared, or altered individually or collectively using dedicated operations, allowing programmers to inspect and manage exception states post-computation. By default, traps—mechanisms that would interrupt execution on exceptions—are disabled, ensuring the non-trapping behavior unless explicitly enabled through alternate handling attributes. For invalid operations involving NaNs, the standard mandates propagation of quiet NaNs without raising the invalid flag, while signaling NaNs trigger the invalid operation exception and convert to quiet NaNs. This distinction supports debugging by allowing signaling NaNs to alert developers to issues, while quiet NaNs permit silent propagation in routine calculations. The 2019 revision of IEEE 754 introduced greater consistency in default handling across all operations, including refined rules for NaN payloads and cohort selection in decimal formats, ensuring uniform behavior in diverse implementations. Query operations such as testFlags and clearFlags enable precise control over the status flags, with saveAllFlags and restoreAllFlags supporting preservation of exception states across computational phases. These mechanisms facilitate reproducible results and error detection in complex algorithms, aligning with the standard's emphasis on reliable floating-point arithmetic.Alternate Exception Handling
IEEE 754-2008 and its revision in 2019 introduce alternate exception handling as optional mechanisms that extend beyond the default flag-based model, enabling programmers to define custom responses to floating-point exceptions for improved control and debugging. These attributes allow for resuming execution after an exception or abandoning it, with recommendations for languages to provide syntax and semantics that support such customizations without relying on hardware-specific traps.[17] Alternate modes include trap handlers, which provide synchronous, immediate control by interrupting execution upon an exception, and logging mechanisms that record exception details for later analysis without halting the program. Trap handlers operate in an immediate mode, invoking a user-defined routine directly when an exception occurs, while delayed modes queue exceptions for asynchronous processing at block boundaries, ensuring deterministic behavior in multi-threaded environments. Logging can capture specifics such as the operation type, operands, and results, often using attributes likerecordException to store this information in accessible structures.[17]
The standard recommends implementing trap handlers through operating system signals, such as SIGFPE on Unix-like systems, or via language-specific features that abstract hardware dependencies for portability. For debugging, dynamic mode changes are advised, allowing temporary activation of traps within scoped code blocks to minimize overhead in production environments. Languages are encouraged to define precedence rules for these attributes, ensuring they integrate seamlessly with existing exception models.[17][33]
The 2008 and 2019 revisions enhance signaling with support for non-default NaN payloads, permitting payloads to carry diagnostic information about exceptions, such as error codes or operand histories, which aids in precise exception tracking during alternate handling. Operations like setPayloadSignaling enable the creation of signaling NaNs with customized payloads, facilitating logging without additional storage overhead.[17]
A key trade-off in alternate handling is the balance between enhanced precision in exception responses and performance overhead; trap handlers introduce latency from interruptions and context switches, potentially reducing throughput in high-performance computing, whereas delayed logging maintains speed but may obscure real-time diagnostics. Implementations must weigh these costs, often favoring non-trapping modes for numerical stability in simulations.[17]
In C, integration occurs through the <fenv.h> header, where functions like feenableexcept can enable traps for specific exceptions by masking floating-point control registers, invoking a signal handler upon detection. For example, to trap division by zero, a program might use feenableexcept(FE_DIVBYZERO) before a computation, with a custom SIGFPE handler logging details and resuming via modified operands, aligning with IEEE 754 recommendations for portable alternate control.[33]
Special Values
Signed Zero
In IEEE 754 floating-point formats, signed zero consists of two distinct representations for the value zero: positive zero (+0) and negative zero (-0). These are encoded with the exponent field and significand (or trailing significand in decimal formats) set to all zeros, while the sign bit is 0 for +0 and 1 for -0. In binary formats, this provides a unique encoding for each; decimal formats allow multiple encodings for signed zero due to varying exponent values, though all represent the same mathematical value.[11] Although +0 and -0 are mathematically equivalent and compare as equal in standard floating-point comparisons (i.e., +0 == -0), the sign bit is preserved through most operations to maintain useful information about the origin of the zero result. For instance, the result of subtracting equal-magnitude numbers retains the sign of the zero based on the operation's context, such as (+a - +a) yielding +0 for positive a, while (-a - (-a)) yields -0. Similarly, division by signed zero produces infinities with matching signs: 1 / +0 = +∞ and 1 / -0 = -∞. Other operations, like squareRoot(-0) = -0, also preserve the sign to align with limiting behaviors in real arithmetic.[11] The inclusion of signed zero allows tracking the direction from which a result underflowed to zero or arose from cancellation, providing extra diagnostic information without altering the numerical value. This is particularly valuable in numerical algorithms where the sign of underflowed results can indicate error sources or guide further computations, such as in logarithmic or trigonometric functions where the sign helps preserve directional information from small negative inputs.[34] The 2019 revision of IEEE 754 clarified handling of signed zero in comparisons and the total-ordering predicate. Standard equality and relational operations treat +0 and -0 as equal, ignoring the sign. However, the totalOrder predicate distinguishes them, with totalOrder(-0, +0) evaluating to true, effectively placing -0 before +0 in a total ordering. Additionally, minimum and maximum operations respect the sign: minimum(-0, +0) = -0 and maximum(-0, +0) = +0. The revision also formally defined "signed zero" to address prior ambiguities in terminology.[11][35]Subnormal Numbers
Subnormal numbers, also referred to as denormalized numbers in earlier terminology, represent non-zero floating-point values whose magnitude is smaller than that of the smallest normalized number in a given format. They are defined in the IEEE 754 standard as having fewer than the full precision $ p $ significant digits available to normalized numbers.[17] In binary interchange formats, subnormal numbers are encoded by setting the biased exponent field to zero while the trailing significand field $ T $ is non-zero. Unlike normalized numbers, the significand lacks an implicit leading 1 bit and is instead interpreted with a leading 0, yielding a significand magnitude $ m < 1 $. The value is calculated asInfinities
In IEEE 754 binary floating-point formats, positive and negative infinities are represented by setting the biased exponent field to all ones (E = 2^w - 1, where w is the exponent field width) and the trailing significand field to all zeros (T = 0), with the sign bit S distinguishing between +∞ (S=0) and -∞ (S=1).[17] Arithmetic operations involving infinities follow defined rules to ensure consistent behavior. For addition or subtraction, an infinity plus or minus a finite number yields an infinity of the same sign as the infinity operand, with no exception signaled. Division of a nonzero finite number by zero produces an infinity with the sign matching the dividend, signaling a divideByZero exception. Multiplication of an infinity by zero signals an invalid operation exception. In the 2019 revision, these behaviors extend consistently to augmented arithmetic operations, which return a pair of results (the rounded result and an exact product or sum) while adhering to the same infinity rules.[17] Comparisons treat infinities as the extremes of the number line. Positive infinity is greater than all finite numbers and negative infinity, while negative infinity is less than all finite numbers and positive infinity; infinities of the same sign compare equal.[17] Overflow occurs when the magnitude of an intermediate result exceeds the largest finite representable value, resulting in an infinity with the sign of the intermediate result and signaling an overflow exception; the exact outcome depends on the rounding mode, such as roundTiesToEven or roundTiesToAway.[17]NaNs
In the IEEE 754 standard, Not-a-Number (NaN) values represent the results of undefined or unrepresentable operations in floating-point arithmetic. NaNs are encoded in binary formats with the exponent field set to all 1s (the maximum biased exponent value) and a non-zero significand field, distinguishing them from infinities which have a zero significand. This encoding ensures that NaNs can carry additional diagnostic information while signaling indeterminate computations.[11] There are two categories of NaNs: quiet NaNs (qNaNs) and signaling NaNs (sNaNs). The most significant bit (MSB) of the significand field differentiates them: it is set to 1 for qNaNs and 0 for sNaNs, with at least one other bit in the significand being 1 to avoid confusion with infinities. The remaining bits of the significand form the NaN payload, a non-negative integer that implementations may use for diagnostic purposes, such as identifying the source of an error; by default, the payload is implementation-defined but often set to zero for a canonical default NaN. When a sNaN is quieted—such as during propagation—it has its MSB set to 1 while preserving the rest of the payload.[11] In operations, any arithmetic or computational function involving a NaN produces a NaN as its result. Specifically, qNaNs propagate quietly without signaling exceptions, typically copying the payload from one of the input NaNs (preferring the first if multiple are present) to the result if it fits the destination format. In contrast, operations encountering a sNaN signal an invalid operation exception and deliver a quieted version of that sNaN as the result. The invalid operation exception is also triggered by certain indeterminate forms that produce NaNs, including the square root of a negative number (sqrt(-1)), subtraction of infinities (∞ - ∞), and multiplication of zero by infinity (0 × ∞).[11] The 2019 revision of IEEE 754 standardized operations for manipulating NaN payloads, introducing functions such as getPayload, setPayload, and setPayloadSignaling to access and modify the diagnostic bits in a controlled manner, with admissible payloads being implementation-defined. These enhancements promote consistent handling of NaN diagnostics across systems. Additionally, NaNs are treated as unordered in comparisons: a NaN is neither equal to nor ordered relative to any floating-point value, including itself, so predicates like <, >, ≤, ≥, and == return false when a NaN is involved, while ≠ returns true. This ensures that NaNs do not satisfy equality or ordering relations, reflecting their indeterminate nature.[11]Implementation Guidelines
Expression Evaluation
In IEEE 754, expression evaluation refers to the process of computing compound expressions involving multiple floating-point operations while minimizing accumulated rounding errors. Language standards define the mapping of expressions to basic operations, including the order of evaluation, formats of intermediate results, and rounding behaviors, to ensure predictable outcomes based on operand and destination formats.[36] Individual operations within expressions follow the standard's rounding rules, but compound evaluation requires additional strategies to control overall accuracy.[36] To reduce rounding errors in multi-operation expressions, the standard recommends using extended precision for intermediate results whenever available, such as formats with at least 32 bits of precision for binary32 operands, which allows exact representation of results before final rounding.[36] The preferredWidth attribute facilitates this by specifying wider destination formats for generic operations like addition and multiplication, evaluating expressions in the widest format supported by the implementation to bound error accumulation.[36] For expressions of the form (a × b) + c, the fused multiply-add (FMA) operation is advised, as it performs the multiplication and addition with a single rounding step, potentially halving the error compared to separate operations.[36] Floating-point arithmetic in IEEE 754 is not associative due to finite precision, meaning (a + b) + c may differ from a + (b + c), though using the same operand formats and evaluation rules yields consistent results across implementations.[36] Error analysis for sums and products provides bounds on total rounding error; for example, the computed sum of n positive terms satisfies |computed - exact| ≤ (n-1) × ε × ∑|x_i|, where ε is the unit roundoff (2^{-p} for p-bit precision), ensuring the relative error remains controlled for well-conditioned inputs.[37] In the 2008 revision, evaluation in the widest available format is explicitly recommended to minimize such errors in compound expressions.[36] A representative example is polynomial evaluation using the Horner scheme, which rewrites p(x) = a_n x^n + ... + a_0 as a_0 + x (a_1 + x (a_2 + ... )), reducing the number of multiplications and additions to minimize rounding errors, often combined with FMA for further accuracy.[38] The 2019 revision extends guidance to augmented operations, such as augmented addition and multiplication, which compute both the rounded result and an exact rounding error term, enabling precise error tracking in expressions without additional precision.[17]Reproducibility
The IEEE 754 standard aims to ensure strict reproducibility, meaning that identical inputs, operational modes, and formats produce the same numerical results and exception flags across all conforming implementations and programming languages. This is achieved by mandating that basic arithmetic operations, such as addition, subtraction, multiplication, division, and square root, yield uniquely determined outcomes based solely on the input values, operation sequence, and destination format under user-specified attributes like rounding direction.[8] Key factors enabling this reproducibility include the consistent application of a fixed rounding mode—defaulting to roundTiesToEven for binary formats—and the restriction on using extended precision for intermediate results unless explicitly specified, as wider formats can introduce additional rounding errors that vary by hardware. However, challenges persist due to compiler optimizations, which may reorder operations for performance or fuse multiply-add instructions (e.g., computing with a single rounding step instead of two separate operations), potentially leading to bit-level differences across platforms despite overall compliance.[36][39] The IEEE 754-2008 revision first formalized mandates for reproducibility in basic operations (Clause 5), while the 2019 revision (Clause 11) expanded recommendations for language standards to include a "reproducible-results" attribute, explicit evaluation rules, and prohibitions on value-altering optimizations to enhance portability. Testing reproducibility typically relies on standardized test vectors outlined in the standard's annexes, which provide small, verifiable cases to check consistent behavior for operations, exceptions, and format conversions across implementations.[8][17] Programming language support for these guidelines varies, but Java'sstrictfp modifier exemplifies enforcement by requiring all floating-point expressions within a class or method to be FP-strict, limiting intermediate computations to the precision of the source and destination types (e.g., binary32 or binary64) and adhering strictly to IEEE 754 rounding semantics for cross-platform consistency.
Character Representation
IEEE 754 specifies standardized conversions between its binary and decimal floating-point formats and external character sequences to facilitate input and output operations, ensuring portability across systems. These conversions support both decimal and hexadecimal representations, with rules designed to preserve the exact value, sign, and special attributes of the floating-point numbers. Clause 5.12 of the standard outlines these operations, emphasizing correct rounding and handling of edge cases to avoid loss of information during textual interchange. For decimal output, IEEE 754 requires the generation of character sequences that allow round-trip exactness, meaning a floating-point value converted to decimal and then parsed back must recover the original representation (under the roundTiesToEven mode, except for signaling NaNs which may become quiet). The shortest such string is preferred, using a minimum number of significant digits sufficient for precision: 5 for binary16, 9 for binary32, 17 for binary64, and 36 for binary128, with similar requirements for decimal formats (7 for decimal32, 16 for decimal64, and 34 for decimal128). This ensures unambiguous representation without unnecessary digits, supporting correct rounding in conversions. Hexadecimal representation, introduced in the 2008 revision for binary floating-point formats, uses a notation of the formsignificandp exponent, prefixed by 0x (e.g., 0x1.23p+4 for a normalized value where the significand is in hexadecimal and the exponent is decimal). This format provides an exact, compact textual encoding of the binary significand and exponent, with the significand expressed to the necessary precision (e.g., 6 hexadecimal digits for binary32). Conversions to and from this form are exact for supported precisions, preserving the floating-point value without rounding errors in ideal cases.
Special values are represented consistently to maintain their semantics: infinities as Inf or Infinity (case-insensitive, with optional sign, e.g., -Inf); NaNs as NaN or NaN(payload) for quiet NaNs, or sNaN(payload) for signaling NaNs, where the payload is an optional hexadecimal diagnostic field; and signed zeros as +0 or -0, particularly preserved in operations like division where sign matters (e.g., 1.0 / -0.0 yields -Inf). These rules ensure that special values are not misinterpreted during input or output.
Parsing of character sequences follows strict grammars to prevent ambiguity: decimal inputs use parameterized precision and quantum preservation options, while hexadecimal inputs adhere to the defined syntax with binary rounding attributes. The 2019 revision provided clarifications for decimal character sequences, refining equivalence rules for cohort representations to improve consistency in decimal formats.
In programming languages, these representations are often supported natively; for example, the C standard library's printf function with the %a specifier outputs hexadecimal floating-point values in the IEEE 754 format (e.g., %a for binary64 yields 0x1.999999999999ap0 for π approximation).[40]