More on pi/2
My analysis of how to handle a range of (\-pi/2, pi/2) for atan()
was incomplete. I wrote:
| Let PI be the closest floater not exceeding mathematical \pi.
| For binary64 and a value near 1.5, one ulp is 2^-52.
|
| Take atan(x), whose given range is (-\pi/2, +\pi/2), for an argument
| x that exceeds tan(PI/2) -- i.e. an interval with values greater than
| about 2^52, a quite moderate floater. Should the result then include
| nextup(PI/2), or should it stay within the specified bounds and return
| the equally valid -PI/2 which *is* within the specified bounds? After
| all, the reflected just-out-of-bounds angle is just within bounds:
| let PI/2 = \pi/2 - eps, with eps < ulp
| the next floater after PI/2 is PI/2+ulp = \pi/2-eps+ulp
| this reflects to PI/2+ulp-\pi = -(\pi/2 -(ulp-eps))
| which is inside (-\pi/2, +\pi/2)
Yes, that exact value -(\pi/2 - (ulp-eps)) is inside the range,
but the corresponding outward-rounded floater may not be!
If eps<ulp/2 it will be -PI/2, which is ok, but if eps>ulp/2
there is no solution, as rounding would flip again to the
other side, and ALWAYS stay outside the strict range.
It would work for binary64 and binary128, but not binary32.
It would work for decimal32 and decimal128, but not decimal64.
For reference:
* ....:.|..1....:|...2....:....3...|:....4
PI/2 = 1.570796326794896619231321691639751442098584...
The -L below indicates that an extra 32 bits of fraction were
computed (it gives the total length in bytes of the result,
so we can see how it would be rounded):
Binary32-L8 \pi/2 = 3FC90FDA A22168C2
Binary64-L12 \pi/2 = 3FF921FB 54442D18 469898CC
Binary128-L20 \pi/2 = 3FFF921F B54442D1 8469898C C51701B8 39A25205
I conclude that the written spec cannot in general be honoured,
and that endpoints just outside the mathematical range should
be picked instead of "flipping to the other side".
Michel
---Sent: 2009-12-01 01:32:40 UTC