Minichan

Topic: Here's a number that double precision floating point WILL NOT STORE: 0.1

boof started this discussion 3 years ago #109,809

The nearest lesser number that the standard can take is 0.09999999999999999167332731531132594682276248931884765625.

You see, numbers in the interval 2-4 to 2-3, that is, 0.0625 to 0.125, can only be stored as multiples of 2-56. 0.1 is not such a multiple.
:

Please familiarise yourself with the rules and markup syntax before posting.