I believe that two successive roundings are not equivalent to a
rounding, just using conventional grade school math. For example,
round 0.146 to two places, getting 0.15, now round again getting
But if you round directly to one place, 0.146 rounds to 0.1.
Therefore it is a known problem in statistics (for example) that you
cannot just take data from one study (presumably already rounded)
and then round it again.
On 3/31/2011 12:54 PM, Peter Lawrence wrote:
my apologies in advance if this is trivial and/or non-sense,
but I did not find the answer in a quick scan of David Goldberg's
"What every computer scientist should know about floating point
arithmetic", nor in other more specifically IEEE-754 documents
that I have.
consider the effect of first rounding (round-to-nearest-even) to
some number of bits, followed by another rounding to a smaller
number of bits, the question is is that always the same as
directly rounding to the smaller number of bits.
is the following observation mathematically
the commas are for readability, the semicolons indicate where
rounding is to take place:
1.aaaa0,10000;0xxx ==> 1.aaaa0,10000 1.aaaa0;10000
==> 1.aaaa0 round to 10 bits, followed by round to 5
==> 1.aaaa1 directly round to 5 bits
(the "0xxx", and "0000,0xxx" are some of the bits of some
mathematically exact result which are not all zeros, which would
be represented by a non-zero "sticky bit" in an actual hardware
implementation. In the first case the sticky bit gets truncated,
in the second case the sticky bit causes a round upwards.)
if the above is a correct observation, then
round32 ( round64 ( X ) ) is not always equal to round32 (
which seems sort of counter-intuitive, at least I started out
thinking it would always be, but thought I had better prove it
first, and then came up with this counter example. If it is true,
I wonder if it is well known or not.
Fred Zemke | SQL standards representative
Phone: +1 6505062051
Oracle is committed to developing practices and
products that help protect the environment