Re: (Basenote drift) Determinism
I have changed the subject line again, as I was using complex
numbers purely as a trivial example.
On Aug 4 2011, Dan Zuras Intervals wrote:
Inter alia, it has led to an almost complete lack of communication
between the "IEEE floating-point always gives precisely the same
answers" community and many language and library standards, compiler
vendors and numeric programmers and users of their applications.
Ah, you refer to the Intel-goes-its-own-way-&-damn-the-
rest-of-the-world attitude.
On this we agree completely.
I did NOT say that, and I did not JUST say vendors.
As an aside, it includes almost every vendor of Fortran, C, C++ and
similar languages, also including GNU, IBM, Sun, Pathscale, Cray,
Hitachi, Fujitsu, NAG and almost all of the many dozens of others I have
worked with in my time (and not just for those languages, either).
The IEEE 754 model is flatly incompatible with the arithmetic models
that have always been used for almost every compiled language, and NONE
of C, C++ or Fortran have adopted it or have any plans to do so. Well,
perhaps I should say that C99 has adopted it, but ALSO kept its existing
incompatible description AND added some wording to allow a new form of
variation. I should point out that I have been or am involved with all
of WG5, WG14 and WG21.
And the same remark applies to all of the dominant library interfaces
and language extensions, whether numerical or parallel, including the
BLAS, LAPACK, MPI and OpenMP.
And, lastly, when numeric programmers and their users are given a
choice between deterministic results and much faster execution,
they choose the latter every time.
And you are correct that there was a limited amount that
we (754) could do about it after the fact. Clause 10 in
the new standard was about the best that was possible &
even THAT was a poor repair.
Yes, it was a defect of the original design. The general arithmetic
model used by 'third generation' languages was well established by the
mid-1960s, and it was a pity that IEEE 754 was a purely hardware-based
design, and for 'RISC' hardware only, at that.
It forbids any non-trivial expression rearrangement, including many
cases of common expression elimination, for a start.
It does not.
Read it again.
It says "all operations we require" shall round correctly.
I address the issue of expressions further down.
That was the "blind expression" thing you had trouble with,
remember?
That is the assembler-based design again. In all mainstream languages,
expressions are the primary concept and low-level operations scarcely
figure. As a trivial example of where the models conflict, consider:
A = P*Q*R*S
B = R*P*S*T
> And there is no question of the precision available.
> It is decided in the declaration of the interval
> objects. WhereEVER the CPUs lie.
I am sorry, but that makes little sense in either a language context
or a hardware one. A declaration is more often associated with
the storage representation than the arithmetic operations. In
particular, that is true for many or most coprocessors. Consider
the common and simple example of whether they use hard or soft
underflow; that is NOT associated with the type, but whether the
operation is passed to the coprocessor or not.
I just don't get you here.
As far as I know, ALL languages have some convention
for the type of their expressions. It is either
declared or implied by the expression/language. One
way or another, it is known.
Actually, no, that is not true, not even in C or Fortran, but
let that pass. My main point was that the 'type of an expression'
(insofar as it is defined) is a separate concept from the actual
sematics of its evaluation.
Everything you mention is predictable if we write our
standard carefully.
In assembler. Nobody writes large numeric programs in that any more.
If the entire thrust of your argument is that it is
difficult (even under the new 754) to obtain bit-for-bit
compatible floating-point results across platforms,
I wholeheartedly agree.
It's not.
If it is further your opinion that we, 1788, should
desire such a thing, I agree with this as well.
It isn't. Not merely is it infeasible, it is actively undesirable.
If you further believe that it will be difficult for us
to specify such behavior, another agreement.
If you believe that it will be impossible &, therefore,
we shouldn't try, THAT is where you leave me behind.
It is well-known to be mathematically impossible without serialising
parallel execution, and parallelism is where the future is. I have
several times given the simple example of a reduction:
There are N separate processes, each delivering a value, and
the requirement is to form a<op>b<op>c<op>... over all of them.
Several people have used the finiteness of IEEE 754 to claim that it can
be done in fixed point, but that really only applies to summation.
Consider 10^9 processes and 128-bit numbers; then summation can be done
exactly using only 4K byte intermediate values. Now try multiplication.
Or, for a seriously nasty example, consider the reduction that returns
the count and the first K moments.
Note that even 754 leaves these as implementation-defined
approximations.
The new 754 took a stand & said (in clause 9) that
either a transcendental function rounds correctly all
the time or it is not sanctioned by the standard.
It was a hard line to take & difficult to get past the
committee but we all know it is possible & 2 generations
of researchers have spent their careers making it both
possible & efficient.
Actually, we know that it is impossible for a feasible amount of
computation, except for the simplest functions. It's a standard
result in complexity theory.
When I last checked, there wasn't any useful 'exact' code for even the
recommended function list over their whole ranges, once you want 128-bit
numbers, but things may have improved. Furthermore, that list contains
only the very simplest functions, and doesn't even include the error
function and its inverse. Once you get onto the more complicated
functions, it's completely infeasible and will remain so.
Regards,
Nick Maclaren.