Community hub
Recent from talks
Contribute something
Nothing was collected or created yet.
Fixed-precision arithmetic
View on WikipediaFixed-precision arithmetic, also referred to as finite-precision arithmetic, is arithmetic on numbers that are represented in a fixed number of digits.[1] Examples are integers, fixed-point numbers, and floating-point numbers, but not rational numbers and arbitrary-precision numbers. The number of digits being fixed means that the maximal number of significant digits is also fixed, which complicates the significance arithmetic that has to be used with them.[2]
The values a fixed-precision number format can represent is determined by the radix of its digits, the number of digits (its precision), and the factor the integer value formed by the digits in the radix is multiplied by (i.e. its scale). Integer number formats in this definition have scale 1, fixed-point a fixed integer power of the radix, and with floating-point a variable integer power.
References
[edit]- ^ Karna, Satish K. Digital Electronics GATE, PSUS AND ES Examination. Vikas Publishing House. p. 7. ISBN 978-93-259-9033-3.
- ^ Sofroniou, Mark; Spaletta, Giulia (2005-07-01). "Precise numerical computation". The Journal of Logic and Algebraic Programming. Practical development of exact real number computation. 64 (1): 113–134. doi:10.1016/j.jlap.2004.07.007. ISSN 1567-8326.
Sources
[edit]- Matula, David W. (2003-01-01). "Precision". Encyclopedia of Computer Science. GBR: John Wiley and Sons Ltd. p. 1420. ISBN 978-0-470-86412-8. Retrieved 2025-06-17.
- "Fixed-precision numbers: DECIMAL and MONEY (Informix Servers)". IBM. 2024-11-19. Archived from the original on 2025-06-17. Retrieved 2025-06-17.
- "Decimal and numeric (Transact-SQL)". Microsoft Learn. 2025-03-27. Archived from the original on 2025-06-17. Retrieved 2025-06-17.
 | This computing article is a stub. You can help Wikipedia by expanding it. |