Hubbry Logo
search
logo

Fixed-precision arithmetic

logo
Community Hub0 Subscribers
Write something...
Be the first to start a discussion here.
Be the first to start a discussion here.
See all
Fixed-precision arithmetic

Fixed-precision arithmetic, also referred to as finite-precision arithmetic, is arithmetic on numbers that are represented in a fixed number of digits. Examples are integers, fixed-point numbers, and floating-point numbers, but not rational numbers and arbitrary-precision numbers. The number of digits being fixed means that the maximal number of significant digits is also fixed, which complicates the significance arithmetic that has to be used with them.

The values a fixed-precision number format can represent is determined by the radix of its digits, the number of digits (its precision), and the factor the integer value formed by the digits in the radix is multiplied by (i.e. its scale). Integer number formats in this definition have scale 1, fixed-point a fixed integer power of the radix, and with floating-point a variable integer power.

See all
User Avatar
No comments yet.