Day 1: Tuesday April 27, 1999: Convened at 9:00 AM

1 Attendees are shown in Enclosure 5.

1.0 Co-Chair John Sheppard called the meeting to order.

1.1 The agenda was reviewed and approved with modifications.

1.2 Chair's Report

1.2.1  John Sheppard talked about 1232.2.  It has been published and sent to each balloter.

1.2.2 John had a meeting with Lee Shombert about maintaining a strong formal relationship between Test Requirements Model (TeRM) and AI-ESTATE.  TeRM is currently attempting to become part of TTSC.
 
1.3 Secretary's Report

1.3.1 Minutes from the 99-A meeting were approved with a modification to identify application of AI-ESTATE, as reflected in AI-ESTATE Log #285.

1.3.2 Action items were reviewed and updated, as shown in Enclosure 1.

1.4  Liaison Reports:

1.4.1  Computer Society Liaison:
John reported that there is a new VP of standards, Steve Diamond.  He reaffirmed John as Computer Society liaison.
 
1.4.2  IEC TC93/WG7:
John reported that the Technical Advisor for the IEC has not been able to be contacted to move the AI-ESTATE standards through fast track.  John contacted the Secretary of US National Committee for ANSI, who has the 1232.2 in PDF format and is also attempting to contact the IEC Technical Advisor.

1.4.3  NATO CALS:
Greg Bowman reported that Anne Wrightson of Huddersfield University, of the STEP/SGML Harmonization effort, has begun using the AI-ESTATE 1232.1 fault tree model to demonstrate an SGML implementation.  The in-process web site, describing the progress of this work can be found at http://helios.hud.ac.uk/staff/scomaw/interop.htm.

1.4.4 Test and Diagnostics Consortium (TDC):
Mark Kaufman sent his trip report to the AI-ESTATE reflector after attending the recent TDC meeting.  It was entered as Log Item #286.

1.5 The subcommittee began discussion of the draft Unified Common Element Model (See Log #287).  Tim Wilmering presented a model, based on changes made to the model discussed at the 99-A AI-ESTATE meeting.

1.5.1 A discussion was entered about making the 'cost' representation more understandable, by linking it directly to test as well as the existing relationship to 'resource'.  John offered a model construct, which was accepted, in which 'cost' and 'resource' are linked directly to 'action', but are constrained in their relationship to each other.

1.5.2 Next, the subcommittee discussed the implementation of "part_of" in the model.  It was agreed to replace the two way lines with an inverse relationship, as follows:

        ENTITY d_m;
        ...
            model_tests: SET [1:?] of test;
        ...
        END ENTITY;

        ENTITY test;
        ...
            INVERSE
                part_of : SET [1:?] of d_m for model_tests;
        ...
        END ENTITY;

1.6 The subcommittee attended a presentation by  Greg Saunders, from the Defense Standardization Program Office, at a nearby Society of Automotive Engineers (SAE) Reliability, Maintainability, Supportability, Logistics, and probabilistic Engineering (G-11) meeting.  The meeting provided information regarding the status of DoD standardization efforts, and that they are headed towards performance based specifications and standards.

1.7 Amanda Giarla presented a model for resolution of counting and existence services.  Graphical Kernel System (GKS) was used as an existing, successful approach for providing counting and service existence (Log Item #288).

1.8 Dave Kleinman entered Log Items #289 and #290, providing suggestions from QSI to solve the issues of existence and counting services.  The committee further discussed counting, status codes and existence.  For counting, it was agreed to:
    a. provide additional arguments for any service which include a list or array,
    b. change Functions involving arrays into Procedures.  They will return requested data item and its size.
For existence, it was agreed to proceed as is currently modeled.

1.9 The subcommittee briefly discussed the use of EXPRESS for modeling, versus Unified Modeling Language (UML) and decided to stay the course, since EXPRESS has proven to be robust enough to handle the constructs AI-ESTATE requires.

1.10 The STRIPS model was discussed next. John Sheppard presented the model (Log Item #291).  STRIPS Operators can be tied to 'tests' and/or 'actions' in the CEM model.  It appears that this model operates at a very elemental level, possibly raising the STRIPS model as an alternative (in combination with the new CEM model) to utilizing the Fault Tree or EDIM.
A point was raised by Dave Kleinman, that a 'setup' entity should be included in the CEM model.  It was agreed to think about the issue.

1.11 John described trials using NetMeeting.  Firewalls cause problems, inhibiting both audio and video commnication.
There may be additional tools that we can use, to better communicate in the interim between meetings, but email seems to be the best tool currently available.
.

2.0 Attendees are shown in Enclosure 5.

2.1 The subcommittee continued discussion of the STRIPS model to solve ordering issues in the 1232 standard.

2.2 The PAR and Requirements Document for P1522 were reviewed.

2.2.1 In discussion of the Requirements Document, an issue was raised by David Kleinman about implementation independence, in that EXPRESS modeling may be interpreted as a dependence.  An example metric, Fraction of Faults Detected - FFD (Log Item # 292) was introduced by John Sheppard to assist in resolving the issue.

2.2.1.1 It was agreed to change requirement #2 to add the word "software" between "specific" and "implementation".
For requirememnt #3, it was agreed to move the word "definitions" after "English and mathematical" and remove the parentheses.

2.2.2 Requirement #4 was changed to read "To avoid ambiguity, the mathematical definition will always take precedence over the English definition".

2.2.3 An issue was then raisied by Tim Wilmering, whether there will be a mathematical definition for each characteristic. The issue for a requirement that all metrics and characteristics be measurabe was also raised.  The subcommittee decided to add a new requirement (#7) worded as follows:
"Selected qualitative characteristics that are not mathematically definable shall be included in an informative annex or separate recommended practice."

2.2.4 In discussion of requirement 5.2 the issue of tying predictive metrics to actual ones was discussed.  Wording was agreed to be changed to:
"Actual metrics defined in the standard shall be determined by calculation from measurable data."

2.2.4.1 Further discussion of requirement 5.2 led to a discussion of whether 'actual' measures require ties to the information models.  Advantages identified include that the models maintain consistency between the predictive and actual measures.  Additionally, although the testability end user may not be concerned with models, giving the tool implementers models as part of the standard that the tool developers can advertize as "AI-ESTATE" conformant, and tool output that end users can trust. 

2.2.4.2 A combined requirement #5 was agreed upon, as follows:

"5. Metric Definitions - Predictive and empirical metrics defined in this standard shall be derivable from standard information models (at least those in the AI-ESTATE standards) and verifiable or determined from measurement data."

2.3.1 Several issues related to the FFD metric were identified and are documented in Log #292.  One issue, regarding unconstrained sets, is captured as Open Issue #99B-1.  Actions were identified, to explore multi-outcome metrics (Action Items 99B-7, 99B-8)

2.4 The subcommittee discussed developing a Recommended Practice for testability.  There may be an opportunity to combine the best of what currently exists (DEF Stan 0013, Mil Hbk 2165, etc.) to assist in programmatics of implementing a Testability program (See Action Items #99B-4 and #99B-5). 

Day 3: Thursday, April 29, 1999 convened at  9 AM

3.0 Attendees for this day are shown in Enclosure 5.

3.1.1 The group discussed, at length, possible changes to the name of the family of standards from AI-ESTATE to better represent our work in Testability and Maintenance as well as Diagnostics.  The consensus was that, since all our standards are based on a set of information models and we distinguish ourselves from the Test Information Integration subcommittee by focusing on diagnostics, there should be a common name tie between the 1232 and other standards in the group that contains some form of the word 'diagnostic'.

3.1.2.  Reviewing the Scope statement, the following was hammered out, in order to identify what is the extent of the standard.  There was  :
"This standard defines formal specifications for supporting system diagnosis. These specifications support the exchange and processing of diagnostic information, and the control of diagnostic processes.  Diagnostic processes include, but are not limited to, testability analysis, diagnosibility assessment, diagnostic reasoning, maintenance support and diagnostic maturation."

3.1.3. Reviewing the Purpose statement, the group nailed down why the standard is being developedandto keep it simple. The following resulted:
"This standard provides formal models of diagnostic information to ensure unambiguous access to and understanding of the information supporting system test and diagnosis.  The standard unifies and expands upon the specifications published in IEEE Std 1232-1995, IEEE Std. 1232.1-1997, and IEEE Std. 1232.2-1998."

3.2 The subcommittee reviewed the 1232 Requirements document, dated 1990.

3.2.1 Each of the requirements was reviewed and reworded as follows:

    "2.1.1 Technology Independence: The information base specifications defined in this standard shall be independent of any target technology."

    "2.1.2 Implementation Independence: The information base specifications in this standard shall not require any specific software implementation."

    "2.1.3 Unambiguous Definitions: To avoid ambiguity, the information base specifications shall be defined using formal modeling."

    "2.1.4 Extensibility: The standard shall provide an extensibility mechanism."

    "2.1.5 Portability: The information base specifications in this standard shall promote portability of implementations between conformant implementations."

    "2.1.6 Interoperability: The information base specifications in this standard shall promote interoperability between conformant implementations."
 
3.2.2 The group discussed subjects for interim electronic discussion.  It was decided that John or Mark will initiate discussions for open issues #98C-5 and #99B-1 (Action Item 99B-11). 

3.3 The meeting was adjourned at 3:10PM.
 

 
Action Item 98C-8: Mark K. to gather Testability information from Eric Gould.  Get copies of RADC report from John Sheppard (when he gets them back from Tony Bartolini) to Arnie (Meeting Co.) for copying for the 98-2 SCC20 meeting.
Status: CLOSED

Action Item 98C-10: Greg Bowman to arrange for AI-ESTATE members to be added to the STEP/SGML NIST exploder.
Status: OPEN.  Check with Joed to make sure list has been added to NIST exploder...

Action Item 98C-13: Greg Bowman will contact Lisa Fowbles (DoD Config Mgmt) about exploring opportunities to marry AI-ESTATE to NATO CALS model.
Status: ONGOING.  Greg Bowman exploring resource/funding opportunities to establish formal relationship.

Action Item 98C-15: Tim Wilmering will contact Charles Amaral about obtaining a copy of AP-208 (Life Cycle Mgmt for Process Plants).
Status: OPEN. Tim will send electronic copies of TC-184 Product Life Cycle Support - Statement of technical requirements.

Action Item 98D-3: Greg Bowman to contact Luigi Napoli to attempt to raise the size limit on the AI-ESTATE reflector.
Status: CLOSED Limit now 1 MB.

Action Item 99A-1: A. Giarla has an action to explore the status and future of IEC liaison by determining the pros and cons of possible options.
Status: OPEN
 
Action Item 99A-2: Dave Kleinman and Amanda Giarla to provide a proposal for solving the counting/existence services issue based on a specific 'cheese' implementation.
Status: CLOSED

Action Item 99A-3: John Sheppard to talk TeRM with Lee Shombert.
STatus: CLOSED

Action Item 99A-4: Tim W. accepted an action to make modifications to the model in Visio, as discussed in the meeting.Greg Bowman will provide one portion (the diagnostic_model level) to Tim via email in Visio format.  Coordination to be accomplished with Mark Kaufman.
Status: CLOSED

Action Item 99A-5: John Sheppard took an action to identify and define the primitives and calculation for Fraction of Faults Detectable, Fraction of Faults Isolatable, and Operational Isolation/Fault Isolation Resolution.
Status: CLOSED.  FFD was provided to exploder via email.

Action Item 99A-6: Randy Simpson to send the AUTOTESTCON reference (Keiner 1980) for the testability definition to Mark Kaufman.
Status: OPEN

Action Item 99A-7: John accepted an action to develop a proposal to create a planning model (strips) to address the issue.
Status: CLOSED

Action Item 99A-8: Dave Kleinman accepted an action to develop a proposal to integrate ordering elements into the existing models (CEM, EDIM, and/or FTM).
Status: CLOSED.

Action Item 99A-10: Randy to create a proposal for false alarm metric(s) for the Testability standard.
Status: OPEN

Action Item 99A-11:  Greg accepted an action to explore the possibility of creating a demonstration of AI-ESTATE as part of V-22 RTCASS transportability, to test capability to provide partial/reconfigurable TPS test functionality.
Status: ONGOING.  Proposal for implementation is the next step.

Action Item 99A-12: Bill Rodriguez will explore the USMC position on AI-ESTATE capability in NxTest.
Status: OPEN.  Tim Bearse to follow up with Bill.

Action Item 99A-13: Tim Bearse will explore role of AI-ESTATE in TDC initiatives.
Status: ONGOING  Mark Kaufman attended the meeting and distributed his trip report to the AI-ESTATE reflector.

New Action Items

Action Item 99B-1: Amanda Giarla will provide the AI-ESTATE Secretary with an electronic copy of Log Item #288.

Action Item 99B-2:  John will flesh out the STRIPS proposal, integrating it into the draft Unified CEM.

Action Item 99B-3: Mark Kaufman has an action to send the latest draft of 1522 to the AI-ESTATE members only reflector (keep it under 1MB).

Action Item 99B-4: Tim Bearse determine who has custody of 1232 Mil Handbook and explore interest in modifying and releasing its programmatic content as a Recommended Practice.

Action Item 99B-5: Jack Taylor to determine who has custody of DEF Standard 0013 and explore interest in modifying and releasing its programmatic content as an IEEE DMC Recommended Practice.

Action Item 99B-6: John Sheppard accepted an action to incorporate redlines from the 99-B minutes into the Requirements document for 1522.

Action Item 99B-7: Dave Kleinman to provide a proposal based on Qualtech's approach to computing metrics under multi-outcome assumptions, for the purpose of assisting in defining model-based metrics for P1522.

Action Item 99B-8: Eric Gould to provide a proposal based on DSI's approach to computing metrics under multi-outcome assumptions, for the purpose of assisting in defining model-based metrics for P1522.
 
Action Item 99B-9: Greg Bowman will ensure members only reflector list is current.

Action Item 99B-10: John Sheppard accepted action to revise the PAR and Requirements document for the Diagnostic and Maintenance Control Information Base (D&MCIB) Requirements document.  This will be presented at the 99C meeting for subcommittee approval.

Action Item 99B-11: John or Mark will initiate discussions for open issues #98C-5 and #99B-1.

#286  TDC Trip Report (4/27/99)

#287  Draft Unified Common Element Model  (4/27/99)

#288  GKS Model for Counting/Existence Services (4/27/99)

#289  'Setup' Entity and Array Passing: Counting/Existence Services (4/27/99) 

#290  Rev'ing Up AI-ESTATE: Counting/Existence/Test Availability Services (4/27/99)

#291  STRIPS Model Proposal (4/27/99)

#292  FFD Testability Metric (4/28/99)
 

98B-8: Should a standard or recommended practice be generated to represent a baseline expected behavior?
Status: OPEN -Low Priority
 
99A-1: Organization and name of the AI-ESTATE standards.  Do we change the name of the organization to better represent our work?  Do we incorporate the Testability standard into the unified document?
Status: CLOSED
 
New Issues
 
99B-1: The current definition of test_outcome, and the new definition of diagnostic_outcome have weak semantics. This means it is difficult to determine whether a fault is actually detected by examining the outcomes since we can no longer tell which correspond to "pass" or "good" in all cases. We may need to address this in the update to the model.
Status: High Priority

II. Approval of Agenda

III.Reports
        A. Chair
        B. Secretary
        C. Liaison

IV. 15 Minute Presentations

V. P1232 Unification Issues
        A. Model Revisions
        B. PAR/Requirements Document
        C. Open Issues
        D. Proposals
            i. Counting/Existence Services
            ii. STRIPS Model
            iii. Integrated Ordering Model

VI. P1522 Testability/Diagnosibility
        A. PAR/Requirements Document Review
        B. Proposals
            i. FFD/FFI/FIR
            ii. False Alarm
        C. Draft Review
        D. Model Issues
        E. Author Assignment and Schedule
        F. Recommended Practice

VII. New Action Items Review

VIII. Set Time and Location for Next Meeting

IX. Set Agenda for 99-C

X. Adjourn
 

II. Approval of Agenda

III.Reports
        A. Chair
        B. Secretary
        C. Liaison

IV. 15 Minute Presentations
        A. Web Based Diagnostics & Demo (Qualtech)
        B. TeRM (Lee Shombert)

V. P1232 Unification Issues
        A. Approval of PAR/Requirements Document
        B. Proposals
            i. STRIPS Model
            ii. Semantics for test and diagnostic outcome
        C. Model Review
        D. Open Issues

VI. P1522 Testability/Diagnosibility
        A. Approval of Requirements Document
        B. Results of Draft Review
        C. Proposals
            i. False Alarm Metric
            ii. Multi-Outcome Metrics
            iii. FFI/FIR Metrics
        D. Model Issues
        E. Author Assignment and Schedule
        F. Recommended Practice
            i. DEF Std 0013
            ii. Mil Hdbk 2165

VII. DMC Subcommittee Operations Manual Review

VIII. New Action Items Review

IX. Set Time and Location for 99-D

X. Set Agenda for 99-D

XI. Adjourn