Errata for IEEE 754-2008 -- precision 1

[Michel Hack <hack@xxxxxxxxxxxxxx>]

Vincent was right when he considered taking the parity of the
exponent into account -- for Binary, but not for Decimal or Hex.

My rule of picking the larger magnitude when both bracketing
results are odd (namely 9 and 1, or F and 1) is the correct
one in this case, as it balances round-up with round-down:
half of the cases (5 resp. 8) each way.

For Binary p=1 however it would always round up, so the correct
rule might indeed be to round to an even exponent!

I'm not sure if it is worth doing this; it could be avoided by
requiring the 754-2008 "extendable" formats to avoid binary
precision 1, as MPFR has done -- and I believe that's the only
precedent, so perhaps we should let Vincent LefÛvre decide what
is best here!

Btw, the nearest-even rule is not really properly balanced anyway:
as FP values are essentially scale-free, the distribution should
be assumed to be logarithmic, which means that, for binary p=1,
41% of the cases should round down, and 59% should round up (the
true half-way point between 1 and 2 is 1.41 (sqrt(2)) and not 1.5).
For decimal p=1 the effect diminishes as the digit increases; the
cutoff between 9 and 10 is 9.486 for example.  For higher precisions
the effect effectively vanishes, but its sign does not:  the cutoff
is always slightly below 50%, which means that bankers got it right
after all!

The net of this is that having binary p=1 round-half-even degenerate
to round-half-up (what IBM calls "biased round to nearest") may not
be so bad.

Michel.

---Sent: 2009-03-31 21:10:27 UTC