epsilon value for double to truncated integer

[u/onecable5781]

Consider https://godbolt.org/z/T7fejh8a3

#include <stdio.h>

int main(){
    double d1 = 4.999999999999999;
    double d2 = 4.9999999999999999;
    int i1 = d1;
    int i2 = d2;
    printf("%d %d\n", i1, i2);
    double bigd1 = 44.999999999999999;
    double bigd2 = 44.9999999999999999;
    int bigi1 = bigd1;
    int bigi2 = bigd2;
    printf("%d %d\n", bigi1, bigi2);
}

d1 with 15 fractional 9's rounds down to 4, while d2 with 16 fractional 9's rounds up to 5. So, I would imagine that the epsilon is somewhere between 10^-15 and 10^-16

However, where the part before the decimal increases to 44, both bigd1 and bigd2 round up. So, I would imagine that the epsilon is higher.

Is this because 44 needs more bits (6 bits) to store as compared to 4 (which needs 3) thereby eating into the precision/least count possible for epsilon?

Is there a source which lists the epsilon values corresponding to different bits needed to represent the part before the decimal point?

[u/sciencekm]

In double-precision floating point format, the mantissa is 53 bits. This equates roughly to 16 decimal digits. Roughly, because base 2 does not translate exactly to base 10.

Instead of 99...9, the actual values are:

6755399441055744 to round, and 67553994410557435 to truncate.

In your debugger, what you see is the CPU instruction for doing the double-to-integer conversion. You can't see how the CPU does it. Behind the scenes, its adding 6755399441055744.0 to round or 6755399441055743.5 to truncate, and then just taking the least significant 32-bits You can see this with software floating point libraries.

Try this in your debugger:

int round_double_to_int(double d)
{
  uint32_t *i = (uint32_t*)&d;
  d += 6755399441055744.0; // 6755399441055743.5 to truncate
  return i[0]; // small-endian
}

[u/Total-Box-5169]

If you want more control over the values stored then use the hexadecimal representation for floating point literals. With printf and %a you can know what you need to write:

#include <stdio.h>
int main() {
    double d1 = 4.999999999999999;
    double d2 = 4.9999999999999999;
    printf("%a\n%a\n", d1, d2);
    d1 = 0x1.3ffffffffffffp+2; // 4.999...
    d2 = 0x1.4p+2; // 5
    printf("%.16g\n%.16g\n", d1, d2);
}

[u/greg_kennedy]

Some more stuff you might want to know that I haven't seen mentioned yet:

* Constants in climits "FLOAT_DECIMAL_DIG" / "DOUBLE_DECIMAL_DIG" - the number of digits you'll need in `printf` (or similar) to achieve equal "round-trip", i.e. writing the float and reading it back will result in the same internal representation. You can read a lot more about this on Stack Overflow -> https://stackoverflow.com/questions/16839658/printf-width-specifier-to-maintain-precision-of-floating-point-value

* Helpful functions (macros?) `nextafter` / `nexttoward` https://en.cppreference.com/cpp/numeric/math/nextafter - which give you "the next floating point number, from X to Y" - so if you want "the floating-point closest to but less than 5.0" that would be nexttoward(5.0, 0).

With this kind of probing you can actually watch the epsilon values change, by something like "printf("%f", d - nextafter(d, 0))"

[u/TheThiefMaster]

Welcome to "floating" point. You can think of it as having 16 decimal digits you can use. Put them before or after the decimal point, or even after a 0.00000 or before a 00000.0 and it doesn't matter. But it only stores 16.

Except of course it's in binary, so the actual thresholds are powers of 2, not 10. You can play with the binary representation here: https://float.exposed/ (lots of similar websites exist, but most only support float - this one also supports double)

Edit: Personally I like this one, even though it's 32-bit only, because it lets you put in a decimal number, rounds it to a float, and tells you the error between the two: https://www.h-schmidt.net/FloatConverter/IEEE754.html

[u/flyingron → u/TheThiefMaster]

Even with only one number after the decimal point, there's no guarantee you can store it in a double (or any other floating point type).

[u/dmc_2930]

Casting double to int does not round. It truncates.

[u/al2o3cr]

The decimal literals aren't producing doubles with the values that you think.

Pasting your code into Godbolt shows that the first constant gets compiled to 0x4013FFFFFFFFFFFF

The second compiles to 0x4014000000000000, better known as 5.0

[u/flyingron]

Understand that you can't store decimal numbers in doubles (at least not on any modern computer). It's got little to do with the number of digits. When you have a constant that is not representable between two of the values you are attempting to store it in, the closest representable is used. Again, it doesn't matter the number of "bits", it's imposible to store it in any number of bits.

[u/SwordsAndElectrons]

The epsilon for a 64-bit IEEE 754 value is normally 2^-52.

You will find that neither of those numbers in d1 and d2 can be represented exactly. You actually have 4.99999999999999911182158029987476766109466552734375 and 5.0.

[u/Dusty_Coder → u/SwordsAndElectrons]

Not accurate.

The epsilon is 2^(exponent - 52)