Re: Languages, languages ... (was: 754R and Reproducibility)

[Nick Maclaren <nmm1@xxxxxxxxxxxxx>]

David Hough 754R work <754r@xxxxxxxxxxx> wrote:
> > As far as parallelism is concerned, the same should be done for
> > at least OpenMP Fortran (probably the most widespread shared-memory
> > paradigm, and certainly one of the best defined).  While I believe
> > that it might be feasible, I believe that the effect would be to
> > eliminate the usefulness of many OpenMP facilities.
>
> In terms of number of floating-point programs in the world today, I suspect
> Excel vastly exceeds OpenMP.    Most Excel programmers would be far better
> off with decimal floating point arithmetic and reproducible results than
> what they have today...

Most Excel programmers would be far better off with a better ternary
implementation than what they have today!  It isn't a relevant
example of a parallel language, for many reasons:

    It isn't a specification, let alone a standard, as it does what it
does and users have to find out what that is.

    It has a very poor reputation for correctness among numerical
experts.

    Its parallelism is rudimentary - a fairly simple form of SPMD,
almost just vector - and that doesn't show the real problems.

> In terms of number of IMPORTANT floating-point programs, though, measured
> by number of fp ops executed per day or economic value at stake, I would
> still expect OpenMP to win for a while.    
> I don't expect OpenMP programmers to ask for
> reproducible results except perhaps during preliminary debugging stages.
> But the presumption is that they will be experienced enough to be
> able to debug the programs they finally produce.

What I am trying to get you to address is the problems of trying to
SPECIFY reproducibility in those languages.  I am not talking about
actual implementations, but whether it is possible to specify it at
all.  I don't think that it is, and I know as much about the semantic
problems of language standards as most people involved in them.

If you think that it is possible for those languages to specify
reproducibility, then please show us how, and be prepared for me to
point out why your proposals are non-starters.

If it ISN'T possible, then including it (even as a recommendation)
will discourage the languages from including IEEE 754R support.

> More generally, the challenge in computer science is to find a way to debug
> programs that exploit massive concurrency - possibly on heterogeneous
> computing elements - and the most effective way to solve such problems is
> to engineer them out of the way as much as possible by effective higher-level
> language design.     (Hence 754R's emphasis on language definitions.)

That is true, except that it is a lesser challenge than finding a
way of getting them to scale and tuning their parallelism.

> When it's not possible to engineer them away, 
> the first step at least is to get the program debugged in a 
> reproducible mode.

That is not.  It is a matter of opinion, and many of us with fairly
extensive experience in the area think that the approach of starting
with reproducibility actually causes more trouble than it resolves.

> Floating-point programs have the additional challenge
> of gratuitous platform-specific variations in results even with no
> parallelism.     Expecting most programmers to be able to debug programs
> with variable behavior due both to parallelism and floating-point variations
> seems to be asking a lot.

That is true, except that most of the variations are not gratuitous.

> It should be possible to disable both of
> these sources of confusion first, then enable them one at a time when and
> where needed.

That is wishful thinking, at best, and probably a delusion.  I have
very good reason to believe that it is theoretically impossible in
any language flexible enough to be useful.

In particular, it is an extension of the myth that you should debug
without optimisation.  For reasons very familiar to compiler and
application experts, all that means is that you have to debug the
program twice, as most of the 'hard' problems arise only when you
turn optimisation on.  Exactly the same is true when you debug with
synchronous, serial scheduling and then enable asynchronous, parallel
scheduling.



Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1@xxxxxxxxx
Tel.:  +44 1223 334761    Fax:  +44 1223 334679