Reproducibility; with appropriate limitations...

["David James" <dvj@xxxxxxxxxxxx>]

All,

On the topic of reproducibility. A good goal, although
a bit late to consider right now. A few thoughts:

  1) Modes. At best we can recommend that compilers
     provide a reproducibility mode, where the code
     ensures reproducibility at the expense of performance.

  2) Scope. Our scope must be limited to code whose
     floating-point operations are defined in the
     specification (including a few more, as needed).
     We cannot handle the full scope of all possible
     scientific routines/applications (matrix multiply,
     FFT, etc.).

  3) Options. We would need to select from among the
     options allowed by the standard (more later...).
 
  4) Purpose. As stated in our PAR, reproducibility is
     a key purpose of the standard. Not really debatable,
     unless we choose to change the PAR. Should limit,
     however, to things currently within the standard.

  5) Details. Detailed tables are required for most operations.

     In their absence, different interpretations will arise.
     If we are not confident enough to do this ourselves,
     we can not be confident others will do it correctly.

     Page count is not limited by the scope or purpose.
 
     Unintended options are just as deadly as intentionally
     obfuscated options.

  6) Limits. We do _not_ need to choose between binary and
     decimal encodings; these are transparent to applications.

As per the options, a few thoughts:

  a) qNaN/sNaN encodings.
     We should define a few primitives that isolate the
     application from the details of the NaN encodings,
     as was done with the decimal encodings. Perhaps like:
       x = QnanEncode(n);
       x = SnanEncode(n);
       n = QnanDecode(x);
       n = SnanEncode(x);
     where n is an unsigned integer, representing the payload,
     and x is a floating-point format.
     This could insolate the application from the specific
     qNaN/sNaN encoding differences, as well as the encoding
     of the payload (bit reversed left justified vs normal
     right justified).

  b) Rounding. Tiny values detected after rounding.
     Easier to specify. Might be difficult to emulate,
     but should probably be specified also.

  c) logB and scaleB. I prefer n to be a 64-bit integer,
     to remove all practical limitations for scaledProd, etc.
     I could be convinced to compromise on 32-bit.

  d) fusedMultiplyAdd. An exception from a qNaN/sNaN input
     has precedence over a computational exception. This
     is most consistent with general definitions, and
     is reasonably emulated by HW that does otherwise.

  e) Decimal characters. The option to preserve the
     distinction between qNaN and sNaN values is the best.

As per completeness:

  f) We can reasonably ensure that _our_ defined operations
     can yield reproducible results (though some additional
     work is required).

  g) This will be insufficient for many applications that
     utilize other tuned applications (such as FFTW). (Such
     application optimizations are based on run-time
     mini-benchmark results, not only compile-time options.)
     However, the framework we adopt may encourage such
     applications to support a future reproducibility option.

  h) Considerable language-dependent support will be required.
     We cannot do this work nor can we practically legislate
     its existence or implementation details. However, our
     framework can at least be useful for properly motivated
     language designers.
     
Respectfully,
DVJ