Re: roundTiesToEven in precision 1

[Michel Hack <hack@xxxxxxxxxxxxxx>]

Mike pointed out that
>  for x = 1,20 do <anything>
> loops forever in one digit decimal because 10 + 1 = 10.

Yes, Rexx has had this curiosity for a quarter century.

The problem arises with other precisions too (if the step
size is too small relative to the iteration variable).
So I don't buy this argument.

Also, if somebody wants to define a logarithmic number type
by using a 754-2008 extendable format with precision 1 and
a suitably large exponent range, why not?

Dan Zuras wrote:
> Oh, I/O is no problem.

Yes it is, because rounding has to be defined for output
precision 1.  As 754-2008 stands, the definition of correct
rounding is stated precisely in a way that does not cover the
case of precision 1, since it was given in the context of the
basic formats (clause 4.3.1), even though 4.3 mentions that
it also applies to conversions to character strings.  So we
were negligent, and I'm glad Vincent LefÛvre discovered this.

We could change the last sentence of the "roudTiesToEven"
paragraph of 4.3.1 to read as follows:

   ...; if the two nearest floating-point numbers bracketing
   an unrepresentable infinitely precise result are equally
   near, the one with an even least significant digit (before
   taking rounding overflow into account) shall be delivered.

I leave it to the editors to decide whether this merits a footnote
to point out the rare situations where that parenthetic remark
makes a difference, namely precision 1, possible for toCharacter
conversions of clause 5.12, and perhaps for extendable formats of
clause 3.7.

On second thought, the notion of "rounding overflow" has not been
defined.  So perhaps it would be better to write:

   ...; if the two nearest floating-point numbers bracketing
   an unrepresentable infinitely precise result are equally
   near, the one with an even least significant digit shall
   be delivered; if that is not possible, the one larger in
   magnitude shall be delivered.

(Now the guessing is as to when it is impossible to deliver
the one with an even least significant digit!)

Vincent actually mentioned taking the exponent into account:
>    ... I suggest that one should consider the exponent of the
>    value that is closer to 0, then consider the significands
>    of both bracketing values, and choose the even one.  With
>    the above example, the exponent of b-1 would be chosen,
>    i.e. 0; then the significands are respectively b-1 and b,
>    and the result b should be returned since b is even.

I'm not sure I understand this.  First of all, b cannot be a value
of a 1-digit significand.  And, in any case, would 9.5e-2 be rounded
in a different direction than 9.5e1, because the bracketing values
are in the opposite order relative to exponent zero?  (Note that
this case is only possible with DFP.)

Michel.
Sent: 2009-03-26 01:51:38 UTC