IEEE D&MC SCC20 Meeting 04-A Minutes: ARINC, Annapolis, MD

Contents:
Daily Minutes
Meeting Agenda
Enclosure 1: Action Item Summary
Enclosure 2: Recent Log Items
Enclosure 3 Open Issues
Enclosure 4: Meeting Attendees List

Daily Minutes

Day 1: Tuesday, 27 January 2004

Attendees are shown in enclosure 4.

1.0 Call to Order

Co-Chair Mark Kaufman called the meeting to order at 8:40 am.

1.1 Agenda Review

The 04A agenda was reviewed and accepted as modified.

1.2 Chair's Report

The chair provided a summary of the three ballots. All passed. DTIF passed with no comments. TeRM had two negative ballots and only three ballots (including the negatives) with comments. 1522 had only one negative ballot and only four ballots (including the negative) with comments.

Tim Wilmering met with the ATML group and became the Diagnostics Working Group chair. Discussions are under way to bring ATML standardization under SCC20 with the specific “components” parceled out to the appropriate, existing subcommittees.

1.3 Secretary's Report

The minutes were reviewed and accepted.

1.4 Action Items and Open Issues

Action Items(enclosure 1) and Open Issues (enclosure 3) were reviewed and updated.

1.5 Computer Society Liaison Report

John reported that he is encountering issues with the Computer Society SAB in that liaison appointments now require full voting positions on the SAB. John accepted the liaison appointment for 2004 but declined the appointment to the board. Jim Moore, VP Standards, is now investigating the “legal” issues associated with having a non-voting liaison on the board.

John also reported that the TTTC Board Test Working Group is planning a panel at the next ITC on Board Test Coverage. In discussing this panel, John informed the group of the 1522 work, which generated considerable interest from BTWG members. It is possible that as many as four members of the BTWG will get involved with the DMC.

1.6 Project Reports

1.6.1 DTIF Report

The standard was reaffirmed. There were 25 balloters in the ballot group with 20 affirmative votes (no comments), no negative votes, and no abstentions. The standard is reaffirmed. See action item 04A-1.

1.6.2 Testability Report

The standard passed its ballot. There were 27 balloters in the ballot group with 20 affirmative votes (three with comments), 1 negative vote, and 1 abstention. A total of 126 comments were recorded. Ballot resolution will be conducted at this meeting.

1.6.3 TeRM Report

The standard passed its ballot. There were 27 balloters in the ballot group with 17 affirmative votes (three with comments), 2 negative votes, and 3 abstentions. A total of 129 comments were recorded. Ballot resolution will be conducted at this meeting.

1.7 AI-ESTATE Amendment Discussion

Discussion focused (initially) on what items need to be covered by the amendment project. It was agreed that a prioritized list of issues be compiled. That list follows.

1. Correct the failure rate portion of the model and associated relationships (e.g., to fault, failure, or repair_item) (erratum).

2. Change “action_count” attribute of “frequency” entity to “event_count” (erratum).

3. Tighten the definition of “level” by making the “at_indenture_level” attribute of “hierarchical_element” a list (erratum).

4. Change the “no_fault” diagnosis to a separate entity, subtyped off of entity “diagnosis.”

5. Incorporate XML as an exchange medium.

6. Add a Bayesian diagnostic model.

John Sheppard moved to accept the 1232 Amendment PAR and submit it to Steering for approval. The motion was seconded by Eric Gould. The motion was passed unanimously.

John Sheppard moved to accept the 1232 Amendment Requirements Document and submit it to PCS for review and subsequent approval. The motion was seconded by Bill Gerstein. It was passed unanimously.

Discussion proceeded on the Bayesian model. It was affirmed that adding such a model would be useful and within the scope of the amendment. The discussion then focused on some of the specifics related to modeling the conditional probability tables. An approach was shown as an add-on to the current extend_bayes model, and this will be used as the basis for further exploration. It was also suggested that an annex be included in the amendment to discuss the reason for adding Bayesian diagnosis to the standard. This might also suggest a modification to clause 4 that provides the overview to the AI-ESTATE approach and underlying models.

Discussion then commenced on XML. The issue of constraints were discussed. The AI-ESTATE constraints were reviewed to determine how to represent them in XML Schema. The discussion also raised the issue of whether constraints should be included explicitly within the schema. It was pointed out that constraints need not be included in the exchange file since those constraints must be enforced by the end system reading the file. However, since an XML schema can include certain constraints, it may be beneficial to include those constraints that are possible. The issue is still unresolved at this point.

The question was raised about the appropriateness of OWL as an alternative representation language. This led to a discussion of the specification language (e.g., EXPRESS) vs. the exchange language (e.g., XML) and where OWL would best fit. The point was made that switching to OWL as a specification language would be an uphill climb, but it might be viable for exchange, as long as no issues of proprietary rights or openness in the standard are raised.

Day 2: Wednesday, 28 January 2004

2.1 Ballot Resolution

The committee broke into two groups to perform ballot resolution on P1522 and P1598.

Day 3: Thursday, 29 January 2004

3.1 Award Presentation

The day commenced with a special award presentation. John Sheppard presented the IEEE Computer Society “Outstanding Contribution Award” to Mark Kaufman “for outstanding technical and leadership contributions as Co-Chair fo the Working Group developing IEEE Standard 1232-2002.”

3.2 ATML Discussion

Tim Wilmering presented the talk he gave at the ATML meeting that was held the week of January 19 in Orlando (Log 352). Discussion followed the presentation focusing on issues of the role of the DCM and the specifics of the XML interface. The issue of handling constraints was revisited. Tim suggested looking at XPath and XQuery as ways of pulling out specific details within complex models. The discussion ended with Tim Davis presented a preliminary (Draft 0) XML schema for the Common Element Model (Log 353).

3.3 Ballot Resolution

Ballot resolution continued on P1522. Ballot resolution continued on P1598 and was completed.

3.4 SIMICA Presentations

Three presentations were delivered on use cases for SIMICA. Tim Wilmering presented a view of why diagnosis is hard and, as a result, why maturation is hard (Log 354). John Sheppard presented a view of the process as learning and put the learning process in the context of maturation (Log 355). Mark Kaufman presented the view of maturation from a reliability and availability perspective. He also discussed this in the context of missile diagnostics, which has a number of unique characteristics over other forms of weapon system diagnostics.

Day 4: Friday, 30 January 2004

4.1 Remaining Discussions

It was reiterated that the focus on the initial efforts on ATML for diagnostics it the development of the schemata. At this point, we will not worry about how information would be transferred to/from the diagnostic reasoner via the ATML interface. Tim Davis also pointed out that he had draft schemata for the DIM and EDIM. These are logged as Log 356 and Log 357 respectively.

A final review of the action items was completed.

4.2 Next Meeting

The next meeting will be held at ISIS (Vanderbilt University) in Nashville, TN in the April-May timeframe.

4.3 Adjournment

The meeting adjourned at 9:50 am, and ballot resolution resumed on P1522 after the meeting.

Committee Roadmap

DMC 04-A Meeting Agenda: January 27-30, 2004, ARINC, Annapolis, MD

Day 1: Tuesday

• Call to Order

• Approval of Agenda

• Reports

  • Chair's Report

  • Secretary's Report

  • Liaison Reports

• Brief Presentations

  • Approach to Information Flow through the DMC Standards (W. Gerstein)

• AI-ESTATE Discussions

  • Amendment PAR/Requirements Document

  • Bayesian Model Discussions

  • XML Schema Development

Day 2: Wednesday

• Testability/Diagnosability

  • Ballot Resolution

• TeRM

  • Ballot Resolution

• DTIF

  • Ballot Resolution

Day 3: Thursday

Testability/Diagnosability

• Ballot Resolution

• TeRM

  • Ballot Resolution

Day 4: Friday

• SIMICA Discussions

  • Discuss Usage Scenarios

  • Work on Initial Draft

  • Work on Information Model

• Final Discussions on All Topics

• Review Old Action Items

• Review New Action Items

• Set Time and Location of 04-B Meeting

• Set Agenda for 04-B Meeting

• Adjourn

Enclosure 1 Action Item Summary

Old Action Items

Action Item 03C-1: Mark Kaufman will contact Don Messina at the IEEE to ask for the IEEE position on using a current, published standard (or significant portion) as supporting information in a ballot. Assuming the IEEE allows making this information available, Mark will also arrange for the information to be provided with the ballot.

CLOSED

Action Item 03C-2: Bill Gerstein will update his presentation on the flow of information through the diagnostic process and the role of diagnostic/diagnosability standards in that process (see 2.1 from the 03C minutes).

OPEN, still revising

Action Item 03C-3: Tim Wilmering will explore the role of services with the ATML group. Specifically, what is the role of ATML in actual information exchange within either a .NET or servlet/JSP environment.

CLOSED

Action Item 03C-4: Each active DMC member interested in SIMICA development will develop a high-level usage scenario for the way they envision diagnostic maturation occurring. They will prepare a presentation to be given at the next DMC meeting for purposes of committee discussion and consensus building. The goal is to develop a single consensus-based usage model.

CLOSED

Action Item 03C-5: Tim Wilmering to develop an XML “snippet” at the structural level to illustrate the importance of capturing constraints. John Sheppard will identify an entity (or entities) within the AI-ESTATE model that contains constraints appropriate for this snippet. John and Tim will develop the example together that illustrates the issues related to defining semantically loose XML schemata.

CLOSED

Action Item 03C-6: In support of coordination with the ATML group, Tim Wilmering will develop a complete “structural” XML schema corresponding to the AI-ESTATE schema by the 04-A meeting.

CLOSED: New action item assigned.

Action Item 03C-7: John Sheppard to continue researching the issue of representing EXPRESS schemata (and instantiations) based on XML Schema (rather than DTDs).

CLOSED

Action Item 03C-8: Mike Malesich to talk to Impact Technologies and Williams Pyro about getting involved in the DMC work. Tim Wilmering to send emails to both inviting their participation.

CLOSED

Action Item 03C-9: John Sheppard to prepare draft PAR and requirements document for the 1232 supplement.

CLOSED

New Action Items

Action Item 04A-1: Les Orlidge will submit a letter to the Standards Board on reaffirmation of IEEE Std 1445-1998, DTIF requesting approval of the standard for another five years.

OPEN

Action Item 04A-2: John Ralph, Tim Davis, Ion Neag, and Eric Gould will generate initial XML schemata based on the Common Element Model, the Fault Tree Model, and the Dynamic Context Model to be reviewed by the committee at the 04-B meeting. Tim Wilmering will provide EXPRESS guidance and insight as necessary to coordinate the efforts.

OPEN

Action Item 04A-3: Eric Gould will explore issues of representing constraints in XML Schema and will provide a presentation on findings with a recommendation at the 04-B meeting. This review will include a closer look at the advantages and disadvantages of Part 28 for this task.

OPEN

Action Item 04A-4: Mark Kaufman will put together a formal presentation on the missile diagnostic maturation use case that he presented informally at the 04-A meeting.

OPEN

Action Item 04A-5: John Sheppard will update the proposed Bayesian diagnostic model and distributed to selected members of the committee prior to the 04-B meeting. The updated model will then be presented (as modified following feedback) at the 04-B meeting.

OPEN

Action Item 04A-6: Mark Kaufman will incorporate the changes from the P1522 ballot resolution and prepare the redline standard, clean revised standard, and comment resolution matrix for submission to the committee for approval. The package will then undergo a QA review by Brit Frank. Following QA, the package will be submitted to the IEEE for recirculation.

OPEN

Action Item 04A-7: Eric Bukata will incorporate the changes from the P1598 ballot resolution and prepare the redline standard, clean revised standard, and comment resolution matrix for submission to the committee for approval. The package will then undergo a QA review by Bill Gerstein. Following QA, the package will be submitted to the IEEE for recirculation.

OPEN

Action Item 04A-8: Les Orlidge will submit a revised PAR on P1598 to NESCOM, prior to final submission of the standard to REVCOM, reflecting the downgrade from full use to trial use.

OPEN

Action Item 04A-9: John Sheppard will coordinate with Les Orlidge on the scheduling of SCC20 meetings relative to AUTOTESTCON and provide information back to the DMC to facilitate DMC scheduling of their meetings.

OPEN

Enclosure 2 Recent Log Items

Items Logged at 03A

#343 BIT and False Alarm Excerpt

#344 BIT and False Alarm Slides

Items Logged at 03B

#345 Tim Wilmering's comments on TeRM.

#346 IEEE Patent Slides

#347 ATML Presentation

#348 John's Comments on TeRM

Items Logged at 03C

#349 ISO 10303-28 n190, XML Representation of EXPRESS Schemas

#350 ISO 10303-41, Fundamentals of Product Description and Support

#351 ISO 10303-44, Product Structure Configuration

Items Logged at 04A

#352 AI-ESTATE Semantics, XML Representation Issues, T. Wilmering

#353 Draft 0 Common Element Model XML Schema, T. Davis

#354 Why Diagnostic Maturation is Hard, T. Wilmering

#355 Diagnostic Maturation Usage Scenario, J. Sheppard

#356 Draft 0 Diagnostic Inference Model XML Schema, T. Davis

#357 Draft 0 Enhanced Diagnostic Inference Model XML Schema, T. Davis

Enclosure 3 Open Issues

Current Issues

Issue 00B-1: There are no higher order services currently defined with respect to the static models. To aid in manipulation, analysis etc., of these models higher order services may be beneficial. For example the following service has been proposed by Qualtech. get_test_outcome_from_diagnosis(diagnosis, set of test) Qualtech: Allows user to get the test outcomes predicted from the seeding of a particular diagnosis (fault). This corresponds to a lookup of the fault in the test-diagnosis (D-matrix) matrix. Additional Notes from 00C: The question has to be addressed as to the general applicability of the services from a standardization perspective. It would be of value to consider these services in the context of the 1522. In particular an annex to 1522 of AI-ESTATE services providing analyses and calculation of metrics might be of value.

The issue of where to address this needs to be decided.

Status: OPEN Medium Priority

Issue 01C-1: Testability metrics based on maintenance philosophy, such as Fault Resolution, can provide a means of validating predictive measures. At this point, the information models used to support definition of metrics in P1522 are insufficient to address maintenance philosophy – it is hoped that this deficiency can be addressed in the future through the creation of the SIMICA information model.
Moved to SIMICA,

Status: OPEN Medium Priority.

Issue 02A-1: Log # 335 – a draft version of the False Alarm Appendix (Annex) submittal contains several section enumerating metrics for False Alarm and Assurance Tolerance. These metrics are to be considered for later inclusion in the standard – after we have formalized the definitions of False Alarm metrics.
Moved to SIMICA,

Status: OPEN Medium Priority

Issue 03A-1: Several issues have been identified that need to be addressed, possibly in an amendment to 1232. These issues will all be collected in this section.

• The failure rate entity in the CEM lacks any way to record a value for failure rate. Three options have been suggested: 1) restore the construct from 1232.1-1997, 2) create an attribute of failure rate tied to non-time-cost, or 3) create an attribute of failure rate tied to frequency.

• Change the action_count attribute of frequency to event_count.

• Although failure rate is an attribute of diagnosis (and therefore fault and failure), and we can get to repair item through fault and failure (via func), there is no explicit relationship between a repair item's failure rate (which is not specifiable in the model) and a diagnosis's failure rate (which is). A repair item's failure rate should be an aggregate of the failure rates of its constituent faults XOR failures.

• Examine viability of either replacing or supplementing exchange format with an XML DTD/Schema.

• Revisit all definitions within the information models.

Status: OPEN High Priority

Issue 03C-1: Committee will investigate ISO STEP work in the area of using XML as an exchange format based on EXPRESS information models.

Status: OPEN High Priority

Issue 03C-2: Committee will examine STEP standards to determine if/how part identification is modeled relative to system indenture identification.

Status: OPEN Medium Priority

New Issues

Issue 04A-1: Create a publicity document for dissemination on the DMC website and possible circulation that discusses the committee approach to standards for diagnostic maturation.

Status: OPEN High Priority

Enclosure 4: Meeting Attendees List

DMC 04-B Draft Meeting Agenda: Spring 2004, Vanderbilt, University

Day 1: Wednesday

• Call to Order

• Approval of Agenda

• Reports

  • Chair's Report

  • Secretary's Report

  • Liaison Reports

• Brief Presentations

  • Approach to Information Flow through the DMC Standards (W. Gerstein)

  • Handling EXPRESS-Based Constraints in XML (E. Gould)

  • The Role of ISO 10303-28 in AI-ESTATE/ATML (E. Gould)

• SIMICA Discussions

  • Discuss Usage Scenarios

  • Work on Initial Draft

  • Work on Information Model

Day 2: Thursday

• AI-ESTATE Amendment

  • Model Corrections

  • Bayesian Model Discussions

  • Initial Draft Review

  • XML Schema Review/Development

  • XML Service-Related Discussion

Day 3: Friday

• AI-ESTATE Amendment

  • Revised Draft Development

• Final Discussions on All Topics

• Review Old Action Items

• Review New Action Items

• Set Time and Location of 04-C Meeting

• Set Agenda for 04-C Meeting

• Adjourn