IEEE D&MC SCC20 Meeting 03-C Minutes: Disneyland Hotel, Anaheim, CA

Contents:
Daily Minutes
Meeting Agenda
Enclosure 1: Action Item Summary
Enclosure 2: Recent Log Items
Enclosure 3 Open Issues
Enclosure 4: Meeting Attendees List

Daily Minutes

Day 1: Thursday, 25 September 2003

Attendees are shown in enclosure 4.

1.0 Call to Order

Co-Chair Tim Wilmering called the meeting to order at 2:45 pm.

1.1 Agenda Review

The 03C agenda was reviewed and accepted as modified.

1.2 Chair's Report

Forming balloting pools for the upcoming ballots (P1522, P1598 and IEEE Std 1445), the existing ballot pool held by the IEEE was found to be horribly out of date. Tim attempted to put together a list of names with the help of John Sheppard and Dave Droste to correct the ballot pool, but was told by the IEEE that the change was too extensive and the ballot pool would have to be reformed through the IEEE website. A message was sent to all SCC20 members requesting them to do so if interested. Note that this will constitute the ballot pool for all of SCC20. The URL is http://standards.ieee.org/db/balloting/ballotform.html.

Tim was asked to attend the Automatic Test Markup Language (ATML) “consortium” to observe their activities and provide the DMC perspective as appropriate. The ATML group is not applying information modeling for their approach. Their focus is on identifying the types of information they need to do their jobs. They are creating a “higher-level” schema to hold the common elements across the set of schemas. The diagnostic schema has not been started. We are in a position where we as a committee might be able to influence the way DML comes about. It is Tim's perspective that the ATML DML may not be fully aligned with the DMC view of what is required to do diagnostic reasoning.

The subject of standards bodies also came up within ATML. They felt that standards committees are slow, laborious ways to get work done. Tim observed that the ATML meetings seem to work the same way as ours. In fact, they seem to be at least as slow. Tim pointed out that the only source of additional overhead in terms of working with a standards committee is dealing with the procedural elements with PARs, ballots, etc. But this also gives credibility and protection to the resulting product. The feeling of the ATML group remains ambivalent w.r.t. standards groups.

1.3 Secretary's Report

The minutes were reviewed and accepted.

1.4 Action Items and Open Issues

Action Items(enclosure 1) and Open Issues (enclosure 3) were reviewed and updated.

1.5 Liaison Reports

1.5.1 Computer Society Report

John had nothing to report.

1.5.2 PLCS Report

PLCS has balloted ISO 10303, AP 239 (the initial information model coming out of PLCS), and it passed at some level as an international standard. Boeing may be trying to influence the development of PLCS is a rather large way. Joed Dougherty has been funded to create a Boeing proposal that would include a diagnostic/prognostic data model. An additional issue is how AI-ESTATE will harmonize with PLCS.

1.5.3 OSA-CBM Report

Tim wants to give a presentation to the ATML group on OSA-CBM.

1.6 Project Reports

1.6.1 DTIF Report

Mukund has assembled all of the material for reballot except for the ballot group. Tim submitted the reaffirmation form for DTIF based on the input provided by Mukund. The ballot pool is being assembled as we speak. See chair's report.

1.6.2 Testability Report

Tim submitted the ballot form. The ballot pool is being assembled as we speak. See chair's report.

1.6.3 TeRM Report

Tim submitted the ballot form. The ballot pool is being assembled as we speak. See chair's report.

1.6.4 SIMICA Report

Tim reassembled all of the information models into the form last reviewed. The dynamic context model from 1232 was added to the Visual Express “database” and all references to the common element model were changed to references to the dynamic context model (as appropriate). One exception was repair_item, which only resides in the common element model.

1.7 ATML Discussion

Tim and John had an offline meeting with Bob Fox (NAVAIR) to discuss the relationship between SCC20 (DMC) and ATML. Mutual liaisons were established between the two bodies: Tim Wilmering will serve as liaison to ATML and Tim Davis (NAVAIR) will be liaison from ATML to DMC. In addition, Les Orlidge and Brit Frank will be attending ATML meetings. The hope is for ATML to establish a more formal process for management and approval of their specifications, at which time it is likely that they would use SCC20 as an umbrella organization. There is also a concern relative to legal liability associated with specifying the ATML specifications. Using the IEEE (and SCC20 specifically) would also provide indemnification for the participants.

1.8 Testability/Diagnosability Discussion

The ballot resolution committee for P1522 was formed. It consists of Mark Kaufman, Sherif Abdelwahed, and Eric Gould. Brit Frank and Bill Gerstein will serve as QA.

1.9 TeRM Discussions

The ballot resolution committee for P1598 was formed. It consists of Tim Wilmering, John Sheppard, and Eric Bukata. Steve Cmiel (realligned from 1522) and Les Orlidge will serve as QA.

1.10 DTIF Discussions

The Ballot Resolution Committee was formed to support the reaffirmation ballot of IEEE Std 1445-1998. The BRC (if needed) will consist of Brit Frank and Mukund Modi. At this point, it is assumed that the QA role is not required.

Day 2: Friday, 26 September 2003

2.1 Presentation (W. Gerstein)

Bill presented the results of exploring a roadmap on diagnostics and where the community has gone in the area. Discussion commenced on what this exploration was to cover. Bill started by looking at the old Mil-Specs on how the diagnostic process runs as well as the specifications for performance of diagnostics. Intially, we want to examine existing and proposed standards as well as the information flow through and between those standards relative to the test and diagnostic process. The hope is for this to serve as a way to re-validate the work of the committee as well as providing the start of a committee Guide or Recommended Practice. Further, it would be good to have a basis for specifying performance-based requirements, including recommendations on how industry might use the standards to satisfy those requirements.

2.2 SIMICA Discussion

Discussion commenced on the development of a schedule for completion of the SIMICA standard. The PAR was approved March 2003. Typically, PARs have a four-year life (with up to two extensions permitted). Milestones to be completed include the following:

• Develop (high-level) usage scenarios. (Due 04-A meeting)

• Develop information model. (Initial model due 04-A meeting. Final model due 06-A meeting.)

• Develop textual content for model. (Initial text due 04-B meeting. Final text due 06-A meeting.)

• Develop background textual material. (Initial text due 04-B meeting. Final text due 06-A meeting.)

• Develop XML-based exchange format. (Base this format on 1232 supplement work. Begin development 05-B. Due 06-B meeting.)

• Develop services with textual content. (Begin development 05-B meeting. Due 06-B meeting.)

• Develop conformance section. (Develop during 06-B meeting.)

• Ballot standard. (Commence 06-C meeting.)

Discussion on the issue of part identification proceeded. Tim examined part 41 and part 44 within the ISO 10303 family and determined that the STEP folks are “abstract dudes.” Tim opined that STEP part 44 product definition representation was slanted more towards product design and manufacturing. John opined that both part 44 and part 41 are, in fact, so abstract as to have lost focus sufficient for SIMICA needs.

2.3 AI-ESTATE Supplement Discussion

As a way of dealing with issues relative to the detail required in expressing EXPRESS schemata in XML, Tim proposed a two-level approach. At one level, an XML schema would be defined at the structural level (i.e., the “EXPRESS-G” level) to facilitate harmonization with ATML (perhaps at the DML level). At the second level, a detailed XML schema incorporating the detail resident in lexical EXPRESS (as indicated in ISO 10303-28) would be developed to facilitate migration of the ATML work to the more semantically rigorous level expected to be required.

Offline discussions were held between Tim Wilmering, Ian Neag, Chris Gorringe, John Sheppard, and Mike Malesich over how to introduce the AI-ESTATE approach into ATML. It was agreed that Ian and Chris will provide feedback on the generated example prior to presentation at the next ATML meeting. The purpose of this feedback is to ensure support from multiple sources during the presentation and to ensure that the approach addresses the ATML issues. Ian and Chris also agreed that an AI-ESTATE based approach for DML is desirable.

Day 3: Saturday, 27 September 2003

3.1 Remaining Discussions

Discussion continued on the 1232 supplement with a focus on whether to include an additional model for Bayesian diagnosis. It was determined that additional expertise needs to be brought into the discussion, so invitations will be extended to Impact Technologies and Williams Pyro.

3.2 Next Meeting

It is anticipated that a DMC meeting will be held at ISIS, Vanderbilt University (Nashville, TN) either late January or early February.

3.3 Adjournment

The meeting adjourned at 11:45 am.

Committee Roadmap

DMC 03-C Meeting Agenda: 25 – 27 September 2003, Anaheim, CA

Day 1: Thursday

• Call to Order

• Approval of Agenda

• Reports

  • Chair's Report

  • Secretary's Report

  • Liaison Reports

  • Project Reports

    • P1445 DTIF

    • P1522 Testability

    • P1598 TeRM

    • P1636 SIMICA

• ATML Discussions

• Testability/Diagnosability

  • Form Ballot Resolution Committee

• TeRM Discussions

  • Form Ballot Resolution Committee

• DTIF Discussions

  • Form Ballot Resolution Committee

Day 2: Friday

• Brief Presentations

  • Approach to Information Flow through the DMC Standards (W. Gerstein)

• SIMICA Discussions

• Work on Initial Draft

• Work on Information Model

• AI-ESTATE Supplement Discussions

Day 3: Saturday

• Final Discussions on All Topics

• Review Old Action Items

• Review New Action Items

• Set Time and Location of 04-A Meeting

• Set Agenda for 04-A Meeting

• Adjourn

Enclosure 1 Action Item Summary

Old Action Items

Action Item 01C-1: John S. to look into publishing a “plea for participation” in various IEEE publications
CLOSED OBE

Action Item 03A-2: Mark will move the False Alarm annex into the 1522 draft.

CLOSED

Action Item 03A-3: Mark will prepare a writeup on CND based on the ARINC report to RADC and will circulate via email to the committee prior to the June meeting.

CLOSED Can't find report.

Action Item 03A-7: Mark to check the “complete” draft of 1522 for consistency.

CLOSED

Action Item 03B-1: Tim will examine STEP standards to determine if/how part identification is modeled relative to system indenture identification.

CLOSED

Action Item 03B-2: Mukund to contact Narayanan Ramachandran about assigning DTIF to DMC for purposes of running a reaffirmation ballot.

CLOSED

Action Item 03B-3: Revisions required for TeRM to be addressed by Eric B. n.l.t. June 13, 2003.

1. Reference to signal will be limited to the signal_instance portion of the model and grammar.

2. Explanatory text will be added to bridge the gap between the model and execution trace.

3. Prepare a short writeup explaining the formal tie between the information model and the TeRM grammar. This part of the Action Item will not hold up the decision on whether to ballot TeRM (i.e., not tied to due date).

4. Prepare a conformance section that does not allow for extension or subsetting. Applications either conform or they don't.

5. A description on the relationships between start times of activities will be included somewhere.

6. The width attribute of activity_definition will be dropped from the model and grammar. Either that, or they will be made consistent.

7. Examine where attributes were rolled back from the grammar into the model.

8. Booleans vs. Logicals will be examined to determine appropriate types and supporting commentary.

9. Entity signal_instance will be renamed to reflect the fact that it is simply an activity instance in which the start/stop times correspond to the start/stop times of the parent activity definition (rather than just relying on this being pointed out in the commentary). We may be able to eliminate all references to “signal.” Check this out.

10. Text on 5.3.3. will be reworded to clarify roles of occurrence trees.

11. Language on constructors will be clarified.

12. The discussion on functions will be clarified.

13. Nets will be removed from the product schema.

14. Examples will be incorporated.

15. Investigate the necessity of using the design as a bridge between the product definition and the set of behaviors.

16. Consider placing an inverse attribute between product definition and design (or behaviors) in order to accommodate HW that is reconfigurable based on test outcomes.

17. Write an overview section that appears at the beginning (i.e., prior to Clause 1) that extolls the benefits of the approach.

18. Finish addressing Tim's comments.

CLOSED

Action Item 03B-4: John will investigate ISO STEP work in the area of using XML as an exchange format based on EXPRESS information models.

CLOSED

Action Item 03B-5: Mark will make sure the following actions need to be taken to prepare 1522 for ballot:

1. Insert a “using the fundamentals” section similar to clause 4.3 in 1232 as additional text in 1522 Clause 4.2.

2. Incorporate redlines.

3. Make sure all fundamental measures reference a model (i.e., have mdl as a parameter).

CLOSED

Action Item 03B-6: One of the co-chairs will call an electronic meeting by the end of June with two agenda items: approval of P1522 to go to ballot and approval of P1598 to go to ballot.

CLOSED

Action Item 03B-7: Mark will provide the false alarm annex to Tim for inclusion in P1636.

CLOSED

New Action Items

Action Item 03C-1: Mark Kaufman will contact Don Messina at the IEEE to ask for the IEEE position on using a current, published standard (or significant portion) as supporting information in a ballot. Assuming the IEEE allows making this information available, Mark will also arrange for the information to be provided with the ballot.

OPEN

Action Item 03C-2: Bill Gerstein will update his presentation on the flow of information through the diagnostic process and the role of diagnostic/diagnosability standards in that process (see 2.1 from the 03C minutes).

OPEN

Action Item 03C-3: Tim Wilmering will explore the role of services with the ATML group. Specifically, what is the role of ATML in actual information exchange within either a .NET or servlet/JSP environment.

OPEN

Action Item 03C-4: Each active DMC member interested in SIMICA development will develop a high-level usage scenario for the way they envision diagnostic maturation occurring. They will prepare a presentation to be given at the next DMC meeting for purposes of committee discussion and consensus building. The goal is to develop a single consensus-based usage model.

OPEN

Action Item 03C-5: Tim Wilmering to develop an XML “snippet” at the structural level to illustrate the importance of capturing constraints. John Sheppard will identify an entity (or entities) within the AI-ESTATE model that contains constraints appropriate for this snippet. John and Tim will develop the example together that illustrates the issues related to defining semantically loose XML schemata.

OPEN

Action Item 03C-6: In support of coordination with the ATML group, Tim Wilmering will develop a complete “structural” XML schemata corresponding to the AI-ESTATE schemata by the 04-A meeting.

OPEN

Action Item 03C-7: John Sheppard to continue researching the issue of representing EXPRESS schemata (and instantiations) based on XML Schema (rather than DTDs).

OPEN

Action Item 03C-8: Mike Malesich to talk to Impact Technologies and Williams Pyro about getting involved in the DMC work. Tim Wilmering to send emails to both inviting their participation.

OPEN

Action Item 03C-9: John Sheppard to prepare draft PAR and requirements document for the 1232 supplement.

OPEN

Enclosure 2 Recent Log Items

Items logged at 02A

#333 Draft Proposal “Requirements Document for IEEE Standard for Diagnostic and Maintenance Information Retrieval”

#334 MOA on Operations Suitability Terminology and Definitions

Items logged at 02B

#335 False Alarm Appendix

#336 Eric Gould excerpt from STAT user’s guide

#337 Eric Gould excerpt from minutes of 98-B AI-ESTATE meeting

#338 Eric Gould excerpt from his Autotestcon 1999 paper

#339 DMC Briefing to the ATS Management Board

#340 False Alarm prediction presentation (Greg Bowman’s)

#341 False Removal Metrics Under a Single Fault Assumption document provided by Sudipto Ghoshal

Items Logged at 02C

#342 SIMICA online PAR form

Items Logged at 03A

#343 BIT and False Alarm Excerpt

#344 BIT and False Alarm Slides

Items Logged at 03B

#345 Tim Wilmering's comments on TeRM.

#346 IEEE Patent Slides

#347 ATML Presentation

#348 John's Comments on TeRM

Items Logged at 03C

#349 ISO 10303-28 n190, XML Representation of EXPRESS Schemas

#350 ISO 10303-41, Fundamentals of Product Description and Support

#351 ISO 10303-44, Product Structure Configuration

Enclosure 3 Open Issues

Current Issues

Issue 00B-1: There are no higher order services currently defined with respect to the static models. To aid in manipulation, analysis etc., of these models higher order services may be beneficial. For example the following service has been proposed by Qualtech. get_test_outcome_from_diagnosis(diagnosis, set of test) Qualtech: Allows user to get the test outcomes predicted from the seeding of a particular diagnosis (fault). This corresponds to a lookup of the fault in the test-diagnosis (D-matrix) matrix. Additional Notes from 00C: The question has to be addressed as to the general applicability of the services from a standardization perspective. It would be of value to consider these services in the context of the 1522. In particular an annex to 1522 of AI-ESTATE services providing analyses and calculation of metrics might be of value. Moved to SIMICA,
Status: OPEN Medium Priority

Issue 01C-1: Testability metrics based on maintenance philosophy, such as Fault Resolution, can provide a means of validating predictive measures. At this point, the information models used to support definition of metrics in P1522 are insufficient to address maintenance philosophy – it is hoped that this deficiency can be addressed in the future through the creation of the SIMICA information model.
Moved to SIMICA,

Status: OPEN Medium Priority.

Issue 02A-1: Log # 335 – a draft version of the False Alarm Appendix (Annex) submittal contains several section enumerating metrics for False Alarm and Assurance Tolerance. These metrics are to be considered for later inclusion in the standard – after we have formalized the definitions of False Alarm metrics.
Moved to SIMICA,

Status: OPEN Medium Priority

Issue 03A-1: Several issues have been identified that need to be addressed, possibly in a supplement to 1232. These issues will all be collected in this section.

• The failure rate entity in the CEM lacks any way to record a value for failure rate. Three options have been suggested: 1) restore the construct from 1232.1-1997, 2) create an attribute of failure rate tied to non-time-cost, or 3) create an attribute of failure rate tied to frequency.

• Change the action_count attribute of frequency to event_count.

• Although failure rate is an attribute of diagnosis (and therefore fault and failure), and we can get to repair item through fault and failure (via func), there is no explicit relationship between a repair item's failure rate (which is not specifiable in the model) and a diagnosis's failure rate (which is). A repair item's failure rate should be an aggregate of the failure rates of its constituent faults XOR failures.

• Examine viability of either replacing or supplementing exchange format with an XML DTD/Schema.

• Revisit all definitions within the information models.

Status: OPEN High Priority

New Issues

Issue 03C-1: Committee will investigate ISO STEP work in the area of using XML as an exchange format based on EXPRESS information models.

Status: OPEN High Priority

Issue 03C-2: Committee will examine STEP standards to determine if/how part identification is modeled relative to system indenture identification.

Status: OPEN Medium Priority

Enclosure 4: Meeting Attendees List

DMC 04-A Draft Meeting Agenda: Winter 2004, Vanderbilt, University

Day 1: Monday

• Call to Order

• Approval of Agenda

• Reports

  • Chair's Report

  • Secretary's Report

  • Liaison Reports

• Brief Presentations

  • Approach to Information Flow through the DMC Standards (W. Gerstein)

• AI-ESTATE Discussions

  • Supplement PAR/Requirements Document

  • Bayesian Model Discussions

  • XML Schema Development

Day 2: Tuesday

• Testability/Diagnosability

  • Ballot Resolution

• TeRM

  • Ballot Resolution

• DTIF

  • Ballot Resolution

Day 3: Wednesday

• Testability/Diagnosability

  • Ballot Resolution

• TeRM

  • Ballot Resolution

• DTIF

  • Ballot Resolution

Day 4: Thursday

• SIMICA Discussions

  • Discuss Usage Scenarios

  • Work on Initial Draft

  • Work on Information Model

• Final Discussions on All Topics

• Review Old Action Items

• Review New Action Items

• Set Time and Location of 04-B Meeting

• Set Agenda for 04-B Meeting

• Adjourn