[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: round32 ( round64 ( X ) ) ?= round32 ( X )

I believe that two successive roundings are not equivalent to a single
rounding, just using conventional grade school math.  For example,
round 0.146 to two places, getting 0.15, now round again getting 0.2.
But if you round directly to one place, 0.146 rounds to 0.1.
Therefore it is a known problem in statistics (for example) that you
cannot just take data from one study (presumably already rounded)
and then round it again.


On 3/31/2011 12:54 PM, Peter Lawrence wrote:
     my apologies in advance if this is trivial and/or non-sense, but I did not find the answer in a quick scan of David Goldberg's "What every computer scientist should know about floating point arithmetic", nor in other more specifically IEEE-754 documents that I have.

consider the effect of first rounding (round-to-nearest-even) to some number of bits, followed by another rounding to a smaller number of bits, the question is is that always the same as directly rounding to the smaller number of bits.

is the following observation mathematically (round-to-nearest-even) correct:

    the commas are for readability, the semicolons indicate where rounding is to take place:

        1.aaaa0,10000;0xxx ==> 1.aaaa0,10000    1.aaaa0;10000 ==> 1.aaaa0    round to 10 bits, followed by round to 5
        1.aaaa0;10000,0xxx ==>                                ==> 1.aaaa1    directly round to 5 bits

(the "0xxx", and "0000,0xxx" are some of the bits of some mathematically exact result which are not all zeros, which would be represented by a non-zero "sticky bit" in an actual hardware implementation.  In the first case the sticky bit gets truncated, in the second case the sticky bit causes a round upwards.)

if the above is a correct observation, then

    round32 ( round64 ( X ) )   is not always equal to   round32 ( X )

which seems sort of counter-intuitive, at least I started out thinking it would always be, but thought I had better prove it first, and then came up with this counter example. If it is true, I wonder if it is well known or not.

Peter Lawrence.

Fred Zemke | SQL standards representative
Phone: +1 6505062051

          Oracle Oracle is committed to developing practices and products that help protect the environment

754 | revision | FAQ | references | list archive