From: owner-stds-1450-4@majordomo.ieee.org on behalf of Ernst_Wahl [wahl@aloft.agere.com] Sent: Thursday, April 04, 2002 7:10 AM To: stds-1450-4@majordomo.ieee.org Subject: stds-1450.4: Re: Jim O'Reilly's assessment Jim and all, This is the fun part where 20 blind people touch the elephant and compare notes in the hope of comprehending the beast as a whole. Since you were kind enough to share your concerns about my 'vision of the elephant', i offer a preview of my response interspersed in your email below. If i can carry this analogy just a bit further, please note my optimism in being convinced, maybe deliriously so, that we're all touching the same animal. Anyway ... ----------------------------------------------------------------------- Ernie Wahl Agere Systems Tel: 610.712.6720 Computer-Aided Test 55F-124D, 6755 Snowdrift Rd Fax: 610.712.4235 Allentown, PA, 18103 Email: ejwahl@agere.com ----------------------------------------------------------------------- > Hi, all > > Ernie, thanks for the good start on definitions. I thought I'd add my $0.02 > worth, to give everyone something to think about before Friday's meeting. > > 1. You mention that a "Testflow-node" can have multiple exit ports > (post-actions?), but earlier, you mention that the "Test" itself > has only two exit ports (post-actions?) - a pass and a fail port. > The test block itself is described as "consisting of a flow of > tests/testblocks" - and the flow is executed by the test-method of > the test block. By all this, are we suggesting that the lowest > level (the "Test" block) can have only two output ports (pass and > fail)? Right or wrong, i hadn't thought of a test as having ports since it doesn't control flow directly, rather i thought of it as a repository for the method, pre and post actions, and results (pass/fail and parametric). Testflow-nodes control flow directly and so i thought of them being connected, out-port to in-port, each port with 0 or more associated actions. > If so, I'm not sure I would agree. I can think of at least two > scenarios where it might be desirable to have more than just a pass > and a fail port from the test block. > > The first is a test block whose test method is "search". The possible > outcomes from a search that I can think of offhand are: Not real pertinent to your point but in the scheme i'm describing, a test block whose test method is "search" isn't possible: a test may employ test method "search" but a block is a test employing dedicated method "execute_flow_within" as you alluded in your first paragraph. > - a transition is found, and it's greater than a programmed lower limit, > and less than a programmed upper limit. Depending on circumstances, > either the lower limit only, the upper limit only, or both, may be > present. This would represent a passing search. > - a transition is found, but its either lower than the lower limit, or > greater than the upper limit (or perhaps both, depending on the values > assigned to the upper and lower limits - but in that case, the results > would probably NOT be what the user intended!). This would represent > a failing search. > - No transition is found in the range searched. In most cases, I as a > user would want to distinguish this condition from a search in which > a transition was found, but was outside the pass/fail limits. > The test, via the testmethod, has a pass/fail status and potentially, parametric results. Would it be sufficient to, in the flow, distinguish the "no transition is found in the search range" situation by a fail with no parametric result ? Naturally, this kind of concern falls within the debate over whether we want STIL to provide a testflow prescription or a complete testflow capable of running on any ATE (i favor the former but don't want to create a structure that impedes the latter). > The second situation is if the test "method" consisted of a block of > custom, user-written code. In that case, in the typical scenario I've > seen, the return value from the top-level function invoked by such a > test block is an integer that dictates which exit port is selected. > It might be useful to consider supporting such a construct. (Which, > of course, would constrain the return value type of functions invoked > as a test method to be integers - meaning that any other type of data > returned by such a function would ave to be specified in the argument > list, as a pointer type. But such considerations would seem to be > getting too close to having STIL support constructs which are probably > best left to a general-purpose programming language - and we've already > had a bit of THAT discussion when talking about the flow ranging from > a simple linear flow to being able to implement a state machine in > the flow. I suspect that discussion will continue . . .) > > In summary, I'm not sure how either of the above scenarios (which > consist of just a single test) could be accomadated if the "test" has > just a pass and a fail post-action. It seems that we might need at > least an "else" or "otherwise" type post-action. > A test return value to suggest a testflow-node exit port seems like a useful notion but we must be prepared for the same test executed at different testflow-nodes to require different exit ports. An arbiter at the testflow-node level that looks at test results, and variables and softbins set by test actions to determine the exit port. My experience to date with very different testflow implementations by various ATE vendors, tempts me to want to see whether we can implement what we need while restricting ourselves to pass/fail, parametric, and variable information. Even with that restriction, some ATE programming environments make it very difficult to generate efficient and user-friendly test programs. > 2. It's not explicitly stated (though is somewhat implied by the order in > which Ernie listed the items) what the hierarchy is - here's my > understanding of the hierarchy, top to bottom. > Comments/corrections welcome. > > Test program > Not mentioned by Ernie, but described in the original 1999 draft of the > .4 document > > TestFlow > (the top level flow node, not pointed to by any other flow node). I > don't think we were necessarily limiting ourselves to a single > top-level flow node. In fact, in the original draft .4 document > from 1999, there appear to be (at least) 5 top-level flows: > Start (the normal flow) > Reset > Load > Init > Powerdown. > And there might be others, including various flows for hardware > alarms (or software exceptions). > I don't know what the advantage of having multiple TestFlows is over having a single TestFlow whose top-level TestBlocks are: Start (the normal flow) Reset Load Init Powerdown Either way, there must be a selection mechanism (can i run Powerdown from Start ?). To me it seems an unnecessary complication. Also, the debate over test flow prescription vs executable code applies here: do we need Load ? Reset ? We also need to explore the interaction between TestFlow, Init, Powerdown, and the STIL dc levels extension. With regard to hierarchy, some vendors define the test program as including the testflow. > > TestBlock > > Test > > Test-method > > The binmap is a separate construct, associated perhaps with a particular > testflow, or perhaps even one level above that, associated with a test > program. > > > That's my take so far - talk to you all on Friday, > > Regards, > > Jim O'Reilly >> -----Original Message----- >> From: dave_dowding@agilent.com [mailto:dave_dowding@agilent.com] >> Sent: Wednesday, April 03, 2002 3:45 PM >> To: stds-1450-4@majordomo.ieee.org >> Subject: stds-1450.4: Terms/Concepts/Classifications/Definitions from >> Ernie Wahl >> >> >> >> Greetings, >> The following is a set of information Ernie >> offered last week. >> Dave Dowding >> >> >From Ernie Wahl: ====================================== >> >> We were discussing a glossary of terms. In the hope >> of arriving at a small and powerful system, I offer >> some debatable concepts, terms, classifications, and >> definitions: >> >> Types: >> >> Testmethod >> >> parameters and a defined or implied sequence >> of events yielding a pass/fail judgment and >> potentially parametric data. >> >> Objects: >> >> Test >> >> an object with a pre and two post actions (pass >> and fail) and a body (the test method instantiation). >> >> Testblock >> >> A Test whose associated test method consists of >> executing the flow of Tests/Testblocks inside >> the block. >> >> Testflow >> >> The top-level Testblock. The only Testblock not >> pointed to by a Testflow-node. >> >> Testflow-node >> >> an object with a pre and multiple post actions >> and associated ports (port of entry, pass, fail, >> etc ports) and a pointer to a Test or Testblock. >> >> BinMap >> >> minimally stores software to hardware bin >> mappings, i.e., might additionally store >> alphanumeric alias to bin number mappings. >> >> Variables >> Mathematical Expressions >> Logical Expressions >> >> These form the basis for the following Actions. >> >> Actions (Test/Testflow-node related): >> >> Assign: conditional/unconditional, pre/post test >> execution >> >> assign a value, potentially a mathematical >> expression, to a variable potentially based >> on the evaluation of a logical expression. >> >> Bin: conditional/unconditional, pre/post test >> execution >> >> set a soft bin potentially based on the >> evaluation of a logical expression. >> >> Execute: conditional, pre test execution >> >> execute the test if the associated logical >> expression evaluates as true (unconditional >> test execution is the default). >> >> =========================================================