IEEE D&MC SCC20 Meeting 04-C Minutes: IEEE Headquarters, Piscataway, NJ

Contents:
Daily Minutes
Meeting Agenda
Enclosure 1: Action Item Summary
Enclosure 2: Recent Log Items
Enclosure 3 Open Issues
Enclosure 4: Meeting Attendees List

Daily Minutes

Day 1: Tuesday, 24 August 2004

Attendees are shown in enclosure 4.

1.0 Call to Order

Co-Chair Tim Wilmering called the meeting to order at 10:30 am. John Ralph served as interim secretary in John Sheppard’s absence.

1.1 Agenda Review

The 04C agenda was reviewed and accepted as modified.

1.2 Chair's Report

Tim Wilmering reported on the status of DMC activities at the SCC20 Plenary session. As all DMC attendees were at the Plenary, the report was not repeated in this meeting.

1.3 Secretary's Report

The minutes from the 04-B meeting were reviewed and accepted.

1.4 Action Items and Open Issues

Action Items(enclosure 1) and Open Issues (enclosure 3) were reviewed and updated.

1.5 Liaison Reports

Nothing to report.

1.6 Project Reports

1.6.1 AI-ESTATE XML Presentation

John Ralph presented the 1232 XML Schema work that has been done so far. Recent changes included eliminating hierarchical element from the XML, flattening out that portion of the model and providing the attributes of hierarchical element to each of that entity’s children. Joe Stanco pointed out that we should document lessons learned and “business rules” that we acquire as we learn how to model test and diagnostic information in XML Schemas.

John further cleaned up some of the syntactical issues to more closely resemble the AI-ESTATE intent.

T. Wilmering suggested that the next step is to create XML instantiations of DIM and EDIM. J. Stanco pointed out that there will be overlap between elements “common” within AI-ESTATE’s XML and ATML’s XML. The FTM as presented may be sufficient to represent Test Sequence as initially represented in the Test Description schema (see below).

1.6.2 ATML Test Description Presentation

T. Davis presented current version of the ATML Test Description schema. In the course of the presentation, it became apparent that “Test” is a semantically overloaded term with distinct meanings in Diagnostics, Test Description and Test Results. Hence, it may be necessary to use a different name in each context.

Test Outcome, on the other hand, has no meaning UNTIL it is considered in a specific context.

The main thought throughout this discussion has been that additional work is needed to determine the commonality between Test Description (ATML) and the 1232 CEM/FTM. It will be of benefit to factor the schemas as much as possible and reuse definitions. It will probably be necessary to define distinct “Tests”, each of which will have appropriate child elements & attributes for their application.

1.6.3 IEC/IEEE Joint Publication Discussion

Bill Ash, IEEE liaison to SCC20, gave a short presentation of the agreement between IEEE and IEC to publish “dual-logo” standards. He would like SCC20 to submit 1232 for IEC consideration (discussed separately at Steering). Overlap between the IEEE standards process/output and IEC. IEC and IEEE have an agreement to develop dual-level documents whereby after review by IEC, IEEE standards may be published as an international standard with the IEC imprimatur. The benefit of this is that IEEE standards may have a broader audience (i.e., international).

1.6.4 SIMICA Discussion

Tim Wilmering presented input from B. Frank and W. Gerstein. This was two Excel spreadsheets that depicted a data dictionary in two use cases. The data appears to be Test Results related. Tim also presented FRACAS and FMECA information elements that are of interest. S. Cmiel has submitted info from a NAVAIR form that needs further discussion. T. Wilmering has begun an effort to consolidate these into a single glossary/data dictionary. Suggestion made to begin a SIMICA sub-working group to perform the data winnowing process.

Tim Wilmering suggested that SIMICA should be flat and broad when transferred to XML. The SIMICA structure is too deep for practical expression in XML. Flattening/broadening will better support creation of efficient XML structures (this principle may also be applicable to future revisions of 1232 models).

Day 2: Wednesday, 25 August 2004

2.1 TeRM Discussion

John Sheppard joined via telecon. Due to exodus of participants, John Sheppard suggested that it might be advisable to shut down the TeRM project.

Agreement reached to postpone discussion on future of this effort until AutoTestCon, when it will be possible to get input from all participants. Erik Bukata has tentatively promised a completed version of the document, with all comments incorporated, including model revisions, by 9/17, new job status permitting.

2.2 Bayes Model Discussion

John Sheppard presented his submitted AI-ESTATE EXPRESS Bayes modeling schema. Pat Kalgren of Impact Technologies joined the meeting at this point. Pat expressed his opinion that the Bayes model should satisfy Impact Technologies’ requirements. Pat & John agreed to initiate a dialog to explore model structure, including development of a “toy” model and determine if further model changes necessary. Results will be posted to the DMC mailing list.

2.3 Diagnostic Services Discussion

Tim Wilmering initiated discussion of services by presenting Tony Alwardt’s (Boeing) briefing on the current thoughts being pursued at Boeing. There was a suggestion that it would be possible to define an XML schema to define services. This evolved into a suggestion that WSDL would be a workable method of defining the interface for diagnostic reasoners. See action 04C-5. This method will be compared with Tony’s approach (which uses an insurance industry standard- ACORD - for XML transactions) at the October ATML meeting.

2.4 Next Meeting

It was agreed that the next meeting will be at AutoTestCon, Thursday p.m. (on a non-conflicting basis with Steering) and Friday a.m. (Steering Committee Thursday p.m.).

2.5 Adjournment

The meeting adjourned at 13:45.

Committee Roadmap

DMC 04-C Meeting Agenda: August 24-25, 2004, IEEE Headquarters, Piscataway, NJ

Day 1: Tuesday

• Call to Order

• Approval of Agenda

• Reports

  • Chair's Report

• AI-ESTATE Amendment

  • Bayesian Model Discussion

  • Work on Initial Draft (postponed)

Day 2: Wednesday

• AI-ESTATE Amendment

  • Model Corrections (postponed)

  • XML Schema Review/Development

  • XML Service-Related Discussion

• SIMICA Discussions

  • Initial Draft Development

  • Data Element/Data Source Review

  • Model Revision/Correction

• AI-ESTATE Amendment

  • Entity/Attribute Definitions (are definitions appropriate for the scope of an amendment?)

  • Conformance (postponed)

  • Draft Revision (postponed)

• P1598 Status Discussion

• Final Discussion on All Topics

• Review Old Action Items

• Review New Action Items

• Set Time and Location of 04-D Meeting

• Adjourn

Enclosure 1 Action Item Summary

Old Action Items

Action Item 03C-2: Bill Gerstein will update his presentation on the flow of information through the diagnostic process and the role of diagnostic/diagnosability standards in that process (see 2.1 from the 03C minutes).

OPEN, still revising

Action Item 04A-3: Eric Gould will explore issues of representing constraints in XML Schema and will provide a presentation on findings with a recommendation at the 04-B meeting. This review will include a closer look at the advantages and disadvantages of Part 28 for this task.

OPEN

Action Item 04A-4: Mark Kaufman will put together a formal presentation on the missile diagnostic maturation use case that he presented informally at the 04-A meeting.

OPEN

Action Item 04A-5: John Sheppard will update the proposed Bayesian diagnostic model and distributed to selected members of the committee prior to the 04-B meeting. The updated model will then be presented (as modified following feedback) at the 04-B meeting.

CLOSED

Action Item 04A-6: Mark Kaufman will incorporate the changes from the P1522 ballot resolution and prepare the redline standard, clean revised standard, and comment resolution matrix for submission to the committee for approval. The package will then undergo a QA review by Brit Frank. Following QA, the package will be submitted to the IEEE for recirculation.

CLOSED

Action Item 04A-7: Eric Bukata will incorporate the changes from the P1598 ballot resolution and prepare the redline standard, clean revised standard, and comment resolution matrix for submission to the committee for approval. The package will then undergo a QA review by Bill Gerstein. Following QA, the package will be submitted to the IEEE for recirculation.

OPEN

Action Item 04B-1: Steve Cmiel to enumerate data elements to be collected relative to part/system info and spares/support equipment, and manpower statistics. See 1.8 of 04-B minutes. Due to Tim Wilmering: 5/17/04

CLOSED

Action Item 04B-2: Tim Wilmering to enumerate data elements to be collected relative to design analysis/design documents and mission recorder data. See 1.8 of 04-B minutes. Due to Tim Wilmering: 5/17/04

CLOSED

Action Item 04B-3: Bill Gerstein to enumerate data elements to be collected relative to maintenance actions and operational failures. Mark Kaufman will assist with operational failures. See 1.8 of 04-B minutes. Due to Tim Wilmering: 5/17/04

CLOSED

Action Item 04B-4: Mike Bodkin to enumerate data elements to be collected relative to observations/symptoms and maintenance metrics. See 1.8 of 04-B minutes. Due to Tim Wilmering: 5/17/04

OPEN

Action Item 04B-5: Brit Frank to enumerate data elements to be collected relative to test results. See 1.8 of 04-B minutes. Due to Tim Wilmering: 5/17/04

CLOSED

New Action Items

Action 04C-1: Tim Davis assigned action to extract data elements from NALCOMIS to determine overlap with the SIMICA data dictionary. (Provide to Tim Wilmering)

Status: OPEN

Action 04C-2: T. Wilmering to get MOQS/NALDA data dictionary or glossary.

Status: OPEN

Action 04C-3: Joe Stanco assigned to work with Tim Wilmering in compiling SIMICA data dictionary.

Status: OPEN

Action 04C-4: Mike Seavey to obtain G-ARMY data dictionary. (Provide to Tim Wilmering)

Status: OPEN

Action 04C-5: John Ralph to develop a presentation on a possible WSDL implementation of the AI-ESTATE Services definition.

Status: OPEN

Action 04C-6: Les Orlidge, Tim Wilmering and John Sheppard to develop contingency plan for disposition of TeRM should document not be complete by September meeting.

Status: OPEN

Action 04C-7: Refer to Issue 04A-1. Assigned to Tim Wilmering.

Status: OPEN

Enclosure 2 Recent Log Items

Items Logged at 04A

#352 AI-ESTATE Semantics, XML Representation Issues, T. Wilmering

#353 Draft 0 Common Element Model XML Schema, T. Davis

#354 Why Diagnostic Maturation is Hard, T. Wilmering

#355 Diagnostic Maturation Usage Scenario, J. Sheppard

#356 Draft 0 Diagnostic Inference Model XML Schema, T. Davis

#357 Draft 0 Enhanced Diagnostic Inference Model XML Schema, T. Davis

Items Logged at 04B

#358 ATML Use Cases Presentation, M. Bodkin

#359 Use Case Representation of the Environment where ATML will be utilized, M. Bodkin

#360 Diagnostics and Prognostics Report, Alabama A&M University, T. Nunn and C. Pendleton

#361 DMC Standards Roadmap, W. Gerstein

#362 AI-ESTATE XML Schema Presentation, J. Ralph

#363 Notes on Developing CEM XML Schema, J. Ralph

#364 Draft 1 of Common Element Model XML Schema, J. Ralph

#365 Draft 0.1 P1232a Amendment, DMC

Items Logged at 04C

#366 Tony Alwardt (Boeing) Services presentation.

#367 Draft 2 of Common Element Model XML Schema, J. Ralph

#368 Draft 1 of Fault Tree Model XML Schema, I. Neag

#369 Steve Cmiel SIMICA Data Element Submittal

#370 Brit Frank SIMICA Data Element Submittal

#371 Bill Gerstein SIMICA Data Element Submittal

#372 Tim Wilmering SIMICA Data Element Submittal (FRACAS)

#373 SIMICA Master Data Element List

#374 SIMICA Combined Data Element List

#375 1232 Bayesian Model Proposal

Enclosure 3 Open Issues

Current Issues

Issue 00B-1: There are no higher order services currently defined with respect to the static models. To aid in manipulation, analysis etc., of these models higher order services may be beneficial. For example the following service has been proposed by Qualtech. get_test_outcome_from_diagnosis(diagnosis, set of test) Qualtech: Allows user to get the test outcomes predicted from the seeding of a particular diagnosis (fault). This corresponds to a lookup of the fault in the test-diagnosis (D-matrix) matrix. Additional Notes from 00C: The question has to be addressed as to the general applicability of the services from a standardization perspective. It would be of value to consider these services in the context of the 1522. In particular an annex to 1522 of AI-ESTATE services providing analyses and calculation of metrics might be of value.

The issue of where to address this needs to be decided.

Status: OPEN Medium Priority

Issue 01C-1: Testability metrics based on maintenance philosophy, such as Fault Resolution, can provide a means of validating predictive measures. At this point, the information models used to support definition of metrics in P1522 are insufficient to address maintenance philosophy – it is hoped that this deficiency can be addressed in the future through the creation of the SIMICA information model.
Moved to SIMICA,

Status: OPEN Medium Priority.

Issue 02A-1: Log # 335 – a draft version of the False Alarm Appendix (Annex) submittal contains several section enumerating metrics for False Alarm and Assurance Tolerance. These metrics are to be considered for later inclusion in the standard – after we have formalized the definitions of False Alarm metrics.
Moved to SIMICA,

Status: OPEN Medium Priority

Issue 03A-1: Several issues have been identified that need to be addressed, possibly in an amendment to 1232. These issues will all be collected in this section.

• The failure rate entity in the CEM lacks any way to record a value for failure rate. Three options have been suggested: 1) restore the construct from 1232.1-1997, 2) create an attribute of failure rate tied to non-time-cost, or 3) create an attribute of failure rate tied to frequency.

• Change the action_count attribute of frequency to event_count.

• Although failure rate is an attribute of diagnosis (and therefore fault and failure), and we can get to repair item through fault and failure (via func), there is no explicit relationship between a repair item's failure rate (which is not specifiable in the model) and a diagnosis's failure rate (which is). A repair item's failure rate should be an aggregate of the failure rates of its constituent faults XOR failures.

• Examine viability of either replacing or supplementing exchange format with an XML DTD/Schema.

• Revisit all definitions within the information models.

Status: OPEN High Priority

Issue 03C-1: Committee will investigate ISO STEP work in the area of using XML as an exchange format based on EXPRESS information models.

Status: OPEN High Priority

Issue 03C-2: Committee will examine STEP standards to determine if/how part identification is modeled relative to system indenture identification.

Status: OPEN Medium Priority

Issue 04A-1: Create a publicity document for dissemination on the DMC website and possible circulation that discusses the committee approach to standards for diagnostic maturation.

Status: OPEN High Priority

New Issues

Issue 04C-1: TBD

Status: OPEN XXX Priority

Enclosure 4: Meeting Attendees List

DMC 04-D Draft Meeting Agenda: September 23-24, 2004, IEEE SA AUTOTESTCON, San Antonio, TX

Day 1: Thursday

• Call to Order

• Approval of Agenda

• Reports

  • Chair's Report

  • Secretary's Report

  • Liaison Reports

• AI-ESTATE Amendment

  • Bayesian Model Discussion

  • Work on Initial Draft

  • Model Corrections

• XML Schema Review/Development

• XML Service-Related Discussion

• SIMICA Discussions

  • Data Element/Data Source Review

Day 2: Friday

• P1598 Status Discussion

• Final Discussions on All Topics

• Review Old Action Items

• Review New Action Items

• Set Time and Location of 05-A Meeting

• Set Agenda for 05-A Meeting

• Adjourn