GRE: rogue comment: IEEE 754R and Reproducibility

[Nick Maclaren <nmm1@xxxxxxxxxxxxx>]

[ Please do not copy me on replies. ]

I am sorry, but the objective and paper remain, as they always were,
so unrealistic as to justify some impoliteness.  It is possible to
argue that reproducibility is a desirable objective, yes, but it is
fallacious to imply that it is entirely beneficial.  But let's skip
that aspect.

It is theoretically possible to support reproducibility in combination
with most forms of (serial) optimisation, though IEEE 754R does not
support the necessary approach and it would have to be done as another
layer on top.  The key is that a datum must be a value+flags, so that
global flags (with their implied ordering) are entirely abolished.
It still sacrifices SOME optimisations, but let that pass.

And, despite what you say, reproducible reductions are not hard to
specify for any simple, side-effect-free elemental operation.  For
example, just require them to be done in extra precision and to
return an error bound; if that leads to ambiguity, repeat in serial
mode or iterate up precisions.  Whether it's worth the effort is
less clear.

But it is PROVABLY IMPOSSIBLE to achieve in a general parallel code,
and attempting a target that has a mathematical proof of absolute
impossibility is Not Clever.  If you want to do that (and it is a
perfectly reasonable objective), you MUST face up to the issue and
specify some way out.

As I pointed out, consider a program with a fixed number of threads
(thus evading your dynamic exclusion), all loads and stores atomic
(to avoid race conditions and subsequent undefined behaviour).
Requiring a reproducible result needs more than Lamport sequential
consistency (in itself a problem), because every implementation
must meet the SAME sequential ordering.  You are therefore demanding
a global, absolute, precise clock model - and nobody in the parallel
field believes that such a thing is even theoretically possible, in
combination with even tolerable efficiency OR scalability.

[ In this context, "global" means that the same clock is accessible
to all threads, "absolute" means that all threads get the same value
and "precise" means that any two events have different timestamps. ]


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1@xxxxxxxxx
Tel.:  +44 1223 334761    Fax:  +44 1223 334679