Re: reproducible format

[Nick Maclaren <nmm1@xxxxxxxxxxxxx>]

stds-754@xxxxxxxxxxxxxxxxx wrote:
> Ivan Godard wrote:
>
> > This has reached pin-dancing. Apparently academia needs a reproducible 
> > standard for algorithm proofs.

Yer, whaa?  I know of no such requirement, and suspect that the real
desire is to be able to publish calculations that give the right
answer and claim that they prove the algorithm does.  Any anyone
which Half A Clue about the underlying mathematics should know why
such practices should not be encouraged.

> Industries that work on critical systems (read here : airplanes 
> fly-by-wire, train anti-collision systems, etc.) prefer to be able to 
> reproduce behaviors. On some current systems such as Intel's IA32, this 
> is very difficult because seemingly irrelevant changes (such as 
> inserting logging or printing instructions) may change the result.
>
> That's nothing academic, that's a precise industrial need.

You started off fine, and then went off the rails.  Yes, there is
a desire to do that, but it is usually (not always) not backed up
by a need.

It is often the same as the academics' desire to be allowed more
sloppiness.  While extra consistency does make it easier to track
down oddities, it does nothing to demonstrate that the properties
proved for the test data are guaranteed for all almost-equivalent
data.

Now, where they DO have a requirement is in the case where all the
input data is being logged, with timestamps et al.  And, of course,
the system is running in purely synchronous mode, so the order of
separate processes is deterministic.  You can then rerun the data
for the aircraft that has just gone nose-down into the nearest hill
and ask why.

There is also the case where you run everything on three systems
and see if they match.  But, in that case, you need merely
reproducible behaviour for the same compiled code.

However, such circumstances are damn rare.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1@xxxxxxxxx
Tel.:  +44 1223 334761    Fax:  +44 1223 334679