Re: Mixing reproducible and non-reproducible code
On Dec 5 2012, Vincent Lefevre wrote:
Grrk. I did agree that it helps a great deal with that, but the proposal
was for universal reproducibility, which isn't critical for that and has
two VERY serious downsides:
Demanding cross-system reproducibility is an intolerable burden for
any non-simple function.
Demanding reproducibility across versions prevents any algorithmic
improvements.
Not necessarily, if the reproducibility is based on the behavior,
not how it is obtained. For instance, in tightest accuracy mode,
results will be reproducible at the individual function level, but
how it is computed will differ from one implementation to another
one.
Perhaps an additional requirement would be that the behavior be
sufficiently documented so that anyone could build an implementation
on his system, where the same results will be obtained.
We are clearly at cross-purposes. There are two aspects, of which the
first is:
If we specify that the standard is defined only for programs with
known, straightforward ways of achieving reproducible results
(i.e. those that use only trivial functions in trivial ways, the
standard becomes useless.
If we demand absolutely reproducibility for some operations and
not others, we are (a) conflicting with many language's numeric
models and (b) imposing a burden on the implementation difficulty
that delivers nothing useful for the vast majority of programs.
The second builds on that, and is based on the well-known fact that
there are lots of commonly used procedures that have no known
accuracy properties and no known, feasible way of achieving a
pre-required accuracy. In some cases, it is known that the time
(and even space!) complexity is super-exponential in the inverse
of the accuracy.
If we specify the accuracy of the best known and published feasible
method, we are preventing any future upgrade or other system from
using a better algorithm (whether 'better' means faster, more
accurate or whatever).
We are also requiring languages to locate such a method, and
specify it as part of their standard. No way, Jose! I can tell
you what several ISO working groups would say, and it would be
best not put in Email.
We are also forbidding some classes of algorithm entirely, such
as parallel Monte-Carlo integration, which is actually the best
way of calculating some functions!
If you want reproducibility as an option that consenting languages
can specify, fine. Only Java and a few others will adopt it.
Making it mandatory as an option is a sure way of getting 1788 rejected
or grossly abused by most of the important ones. That is the mistake
that 754 has made, twice, and the results are clear.
Regards,
Nick Maclaren.