[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Reproducibility of minNum() and maxNum() for decimal



Michel Hack wrote:
Here is something that we should have noticed months ago... but I
only realised this when David Hough wrote "except for minNum() ..."
in a draft of a possible revision of Clause 10.

The definition allows either operand to be returned when both are
numeric and compare equal, i.e. have the same value.

In the case of decimal, this implies that the quantum of the result
is implementation-dependent in the case of equal values but different
quanta.

This effect is observable with the sameQuantum() predicate, and this
is a problem for the "reproducible results" section, and a few other
places that mention "equal results".

There may also be an effect on signaling Invalid -- a flag that is
supposedly reproducible -- because quantize(x,y) may signal Invalid
depending on the quantum of y.

The easiest fix would be to require that minNum() and maxNum() return
the first operand in case of numerical equality, at least in decimal,
so as to avoid this unpredictability.
I feel it would be preferable to return the operand with the maximum quantum. In this way, these functions are commutative. Returning the operand with the maximum quantum avoids attaching undeserved precision to the result.

Regards,
Peter Henderson


754 | revision | FAQ | references | list archive