Richard,
I like the idea of Michel Hack about preprocessor from my format to your format.
I shall try to write it in Java. I'm Sorry :). You can write it as separate phase before Lisp.
So I answer your questions and I do not argue about which format is better.
> > 1) I would like to use in test suite symbolic names like "add" instead of numbers.
> I would personally prefer this too.
> I was trying to make it
perfectly unambiguous and not have
> to worry about character sets, capitalization, Add vs ADD vs unicode
vs ASCII.
I suggest to use exacty the names that are in Draft of the standard.
All of them are ASCII. The names must match exactly. "add" can't be replaced by "Add" or "ADD".
> > 2) Is it necessary to put name on every line ?
> > There may be a header line
with operation name before a few lines with operation instances.
>
> I think each line should stand alone without reference to any
other line, so that the
> line can be run through a program all by itself.
Ok. Preprocessor will put operation name on every line.
> > 3) The number can of inputs in operation instance is fixed for most operations.
>
> Yes
>
> > It is variable for such operations like "min", "max".
> > There are also overloaded reverse operations invRev(c) and invRev(c,x).
> > Nevertheless, reader can count number of inputs in a line for such operations.
>
> I am trying to make the access entirely uniform and not make the
reader program
> any more complicated than necessary.
>
> > It is not necessary to write the counter explicitly
>
> For the same reason as above, I think each line should have a
sequence number.
> There might be gaps in the sequence, but I think it is good for each
line
> to be referred to by a single identifier
> so you can say "my program gets a
different result for line xxxxx.
> Is my program wrong or is the test suite
wrong?"
Ok. Preprocessor will put number of inputs and number of outputs on every line.
> > 4) The outputs may be optional for some tools.
> > For example Reference Checker reads input, computes tightest interval internally
> > and checks
that it implementation under test returns result that contains the tightest.
>
> I don't have "reference checker" in my list..
> But if it computes
something like "Yes this result is checked"
> then I suppose it could be added to
the test suite.
> If a program has no output then it computes nothing and can be
discarded :)
Checker permits to reuse test suite line for different datatypes.
So it treats inputs as exact rational intervals.
Intervals like [0.1,1/3] are allowed in inputs.
The interval datatype T is the parameter of test run.
Checker reads operation name and rational inputs.
read(f);
read(x).
It convert them to specified datatype by T-hull operation if necessary.
xt = hull_T(x).
Then it found the tigtest result
zt = hull_T (f_exact (xt)).
The it runs operation from library under test
zi = f_impl(xt).
Thet it checks that (zt \subseteq zi) and reports result.
This is the reason why outputs are optional in my format.
> > I suggest to separate inputs and outputs (if present) by some mark, for example "->".
>
> why? A program to read and format the test suite,
> or to parse
some other input and produce the
> test suite format, could be written .
> I wanted to make the format
so simple
> that any programming language could easily read it,
> and not have to check to see if the
next field is a number or a mark.
Ok. Preprocessor will not put any mark between inputs and outputs.
> > 5) I suggest to use "hexadecimal-significand" form as described in section 13.4 of Draft 7.1
instead of tripples.
>
> I disagree.
> Why should each person have to write a parser for such
elaborate stuff.
> Furthermore, it doesn't even cover everything the triples cover,
> because it deals
only with finite numbers not infinities etc etc.
> Furthermore, the representations are subject
to rounding to get to the actual numbers.
> So, unless I am misunderstanding something here, it looks
unsuitable.
How do you want to represent infinitives by triples ?
How do you want to represent empty interval ?
> > 6) Is test sequence number necessary ?
Shouldn't it be optional
>
> No, but if you wish to provide a test, I will be happy to provide a
test sequence number for it!
Can it be test name that is not necessary a number ?
>> 7) Are comments allowed
>
> I'm agreeable to anything that is convenient.
> What would you like?
> For Lisp, the convention is that any line beginning with
semicolon ";" is a comment.
> If you want to use some convention for comments at the end of a
line, that's OK too.
> If you wish to use // or .... # that's OK with me.
I think that most people will prefer C++ style comments starting with // .
> alternatively, since each line has a sequence number, a separate
file with sequence numbers and comments
> may be useful; this may keep the test file relatively compact.
Ok. Preprocessor can strip out all comments.
> Frankly, I don't know what you mean by [1,2].
> Is the input widened? is the output? how do you express oo or -oo?
+oo and -oo are expressed by +infinity and -infinity .
Intervals bounds are widened only if they do not fit in format.
Input [1,2] remains as [0x1.0000000000000p1,0x1.0000000000000p2]
Input [0.1,1] is widend to [0x1.9999999999999p-4,0x1.0000000000000ap1]
Preprocessor will write widend tripple.
> I have not put notation in for decorations.
> I would prefer that
stuff not be alphabetic.
> I suppose that each interval could be (triple)(triple) DECORATION
> where DECORATION is 0,1,2,3,4
> Though illegal and empty can be detected by special triples, I
think.
> com, dac, def may be all that is necessary.
Please give me example how you want to represent empty intervals.
-Dima