IEEE D&MC SCC20 Meeting 03-B Minutes: ARINC, Annapolis, MD

Contents:
Daily Minutes
Meeting Agenda
Enclosure 1: Action Item Summary
Enclosure 2: Recent Log Items
Enclosure 3 Open Issues
Enclosure 4: Meeting Attendees List

Daily Minutes

Day 1: Tuesday, 3 June 2003

Attendees are shown in enclosure 4.

1.0 Call to Order

Co-Chair Mark Kaufman called the meeting to order at 9:10 am. The meeting began with introductions.

1.1 Agenda Review

The 03B agenda was reviewed and accepted as modified.

1.2 Chair's Report

SIMICA PAR was approved with issues resolved among railway folks in the IEEE Standards Board. Members of the DMC were recognized for the contributions to publishing the 1232 standads, namely Tim Wilmering, Eric Gould, Sudipto Ghoshal, and Jack Taylor. Congratulations to the awardees.

1.3 Secretary's Report

The minutes were reviewed and accepted.

1.4 Action Items and Open Issues

Action Items(enclosure 1) and Open Issues (enclosure 3) were reviewed and updated.

1.5 Liaison Reports

1.5.1 Computer Society Report

John Sheppard reported that there was concern raised about the role of abstention in balloting. The IEEE rule is that abstentions are basically the same as a negative vote. John sent a message to the CS SAB reflector indicating that the SCC20 approach is to emphasize consensus and balance in the standards process. The SAB appeared to be satisfied.

1.5.2 PLCS Report

No activity to report, except there was a one-way communication with Joed Dougherty. Informed him of desire to share closed websites to PLCS members and shared the SIMICA PAR with him to update PLCS on those activities.

1.5.3 OSA-CBM Report

Work is continuing at a low level of funding

1.6 Presentations

The AMB presentation was delivered as a way of introducing new members of DMC to the work of the committee.

1.7 SIMICA Discussions

Discussion began with a review of the PAR and the history of its approval. The history of TMIMS was also covered.

Discussion continued using the LOGTMIMS information model developed at ARINC. Tim used this as the basis for introducing a new SIMICA model for committee discussion.

One of the issues explored focused on how to represent part identification. Specifically, there was an issue with whether and how one represents, for example, WUCs (work unit codes). John suggested that the committee look at the STEP work (e.g., Part 41 or Part 44) to determine if this issue has already been addressed. In addition, under the assumption that STEP does not have this covered, it was felt that modeling part identification should be covered as a separate schema rather than incorporated directly into the main model.

Tim described his approach to modeling as striving to define all of the necessary information elements to bridge the data elements available in maintenance and other pertinent product data systems with those elements required to support diagnostic maturation analysis.

In the introduction, it will be mentioned that the type of information being collected for diagnostic maturation is applicable to prognostics as well. Care will be taken to minimize attention being drawn to prognostics.

While type of test and test location are considered to be important for maturation, the elaborate data structures of the LOGTMIMS model are unnecessary.

1.8 DTIF Discussion

Mukund Modi brought up the issue of DTIF (IEEE Std 1445-1998) reaffirmation. The standard was approved by the Standards Board in 12/98. There is no DTIF committee anymore. Of the three subcommittees of SCC20, it was thought that DTIF fit best within DMC. Mukund is filling out the reaffirmation request form and is listing DMC/SCC20 as the sponsor. It was also pointed out that SCC20 has a new staff liaison—Don Messina.

The issue of ballot constituency was also discussed. The initial ballot constituency will be the current SCC20 membership. Additional members, based on previous ballot, will be identified.

The consensus of the committee was that the DMC would be willing to provide a home to DTIF for purposes of reaffirmation. This will be communicated to the SCC20 Chair.

Day 2: Wednesday, 4 June 2003

At the start of the meeting, the IEEE patent slides were shown. Opportunity was also provided for committee members to identify or disclose patents essential to the use of the standards. No members advised that patents were an issue.

2.1 TeRM Discussions

A review of the TeRM action items started the discussion. After it was determined that most of the action items were satisfied, discussion proceeded with comments from Tim and John.

It was decided that the model and grammar will be limited w.r.t. signals to “signal instance” and the explanatory text will avoid all reference to signals. Instead, the focus will be on values of variables.

The conformance discussion ended with consensus that you either conform to TeRM or you don't. At this point, no subsets and no extensions will be permitted.

Tim's comments were addressed as a whole by logging the associated version of the TeRM document that contains the comments. Eric will address the entire group as a single action item.

The subcommittee decided to set a deadline of June 13 for all revisions to TeRM to be completed. This will be firm. When the deadline arrives, whatever has been completed will be used as the basis for holding an electronic meeting to approve the document for ballot.

2.2 ATML Presentation

Mukund gave a presentation on current work in the ATML effort. Several schemata exist and will be provided to the committee. The schemata have been done in XML SPY. More information can be found at http://www.atml.org.

2.3 Testability Discussion

An issue was raised about whether to repeat definitions in 1522. Each will be handled on a case-by-case basis.

Clause 4.3 of 1232 provides a “binding strategy” for AI-ESTATE services. Something like this needs to be included in 1522 to describe how to use the fundamentals defined within clause 5. The new section should be placed in Clause 4 (or possibly Clause 6) of 1522.

Day 3: Thursday, 5 June 2003

3.1 P1522 Review (continued)

Review of the P1522 document continued with examination of the fundamentals. The document was decimated. (Not really.) But it was greatly improved.

Brit and Steve agreed to take on the QA role for 1522.

3.2 Next Meeting

It is anticipated that an SCC20 meeting will be held in conjunction with AUTOTESTCON in Anaheim. If so, the next meeting will be held then. Otherwise, a meeting will be scheduled in the October timeframe in St. Louis.

3.3 Adjournment

The meeting adjourned at 1:07 pm.

Committee Roadmap

DMC 03-B Meeting Agenda: 3 – 5 June 2003, Annapolis, MD

Day 1: Tuesday, June 3, 2003

• Call to Order

• Approval of Agenda

• Reports

  • Chair's Report

  • Secretary's Report

  • Liaison Reports

  • Project Reports

    • P1522 Testability

    • P1598 TeRM

    • P1636 SIMICA

• Brief Presentations

  • AMB Brief Reprise

  • ATML Presentation

  • DTIF Status

  • IEEE IP Issues

• SIMICA Discussions

• Work on Initial Draft

• Work on Information Model

Day 2: Wednesday, June 4, 2003

• TeRM

  • Complete or Terminate

• Testability/Diagnosability

  • Review Draft from Working Group

• Complete and Incorporate Conformance Clause

• Review Status for Ballot

Day 3: Thursday, June 5, 2003

• Final Discussions on All Topics

• Review Old Action Items

• Review New Action Items

• Set Time and Location of 03-C Meeting

• Set Agenda for 03-C Meeting

Adjourn

Enclosure 1 Action Item Summary

Old Action Items

Action Item 01C-1: John S. to look into publishing a “plea for participation” in various IEEE publications
OPEN

Action Item 02B-1:Sherif to write a proposal for a model theory definition of a false alarm.
CLOSED: OBE

Action Item 02B-2:John will provide text for the information model in the false alarm annex
CLOSED

Action Item 02B-6:Eric Bukata perform the following modifications to the TeRM spec:

1. Structure the document according to the following outline:

   • Front Matter

     • Overview

     • Definitions

     • Technical Overview

     • Conformance

   • Model

     • Add comments

     • Make corrections as noted

   • Grammar

   • Informative/explanatory Annex

     • Include examples

2. In the overview, include a discussion on Definition vs. Instance vs. Occurrence vs. Execution

3. Consider adding an annex with an XML DTD to be used as an alternative to the grammar

4. Consider adding elements enforced in the grammar (i.e., attributes) into the model.

5. For the activity instance model, either remove signal instances or differentiate them from activity instances

6. Investigate whether or not a a function definition should be abstract.

7. Define more clearly what a constructor is.

8. Investigate the necessity of using the design as a bridge between the product definition and the set of behaviors.

9. Consider placing an inverse attribute between product definition and design (or behaviors) in order to accommodate HW that is reconfigurable based on test outcomes/

10. Demonstrate that the term type is not abstract and that it has an optional supertype.

11. Check the data types to guarantee that all intended types are included and named appropriately.

12. Consider the readability issues of using the term_type name, as opposed to using just type (may be prohibited in EXPRESS), etc.

13. Include discussion of declarative nature of TeRM.

14. Include discussion of mapping from grammar to information model.

CLOSED

Action Item 02B-7:Mark K generate “front matter” for an informative annex on “other potentially useful specification metrics” metrics to include are Isolation Effectiveness and Projected Percentage of False Removals, and possibly Block/Serial Replacement and Repair Item Involvement Ratio metrics. This includes pulling in the informative sections of Clause 6
CLOSED

Action Item 02B-8:Eric Gould generate the technical writeups (to go in the informative annex on “other potentially useful specification metrics”) for Isolation Effectiveness and Projected Percentage of False Removals. CLOSED

Action Item 02C-1:Tim to contact Joed Dougherty (PLCS) to attempt an exchange of private web site access info
CLOSED

Action Item 02C-2:Tim needs to ask if the Boeing As Supported Data Collection (ASDC) model can be released for committee use so that we can use it for a strawman.
CLOSED Tim asked. Answer was “NO!!”

Action Item 02C-4:John and Greg to correct the false alarm rate equations on the FA slides to be :
FA Rate (2165): (FF/FF+TF)
False Indication Rate (include FAs and nondetects): (FF+FP)/(FF+FP+TF+TP)
John will document the mathematical derivation of the False Indication Rate to its simplified form, i.e., (FF+FP)/2
Greg will also explain why FA rate is an unacceptable metric (due to its inherent variability for correcting errors in diagnosis – and why FI rate is more acceptable for measuring improvements (Design of Experiments explanation relevant?).
CLOSED

Action Item 02C-5:Greg to add historical FA information to the FA Annex – he will provide the aerospace perspective (and identify it as such) in some detail, then summarize the medical perspective for contrast purposes.
CLOSED: OBE

Action Item 02C-6:John provide Tim with the EDM models for the CEM, DCM, Upper Layers, TMIMS (from Log Item 322 ) models .
CLOSED

Action Item 02C-7:Tim create an initial SIMICA information model using the models provided by John (in action item 02C-6) and insight from previous maintenance information integration model experience.
CLOSED

Action Item 03A-1: Louis Ungar to provide material for supporting the development of the false alarm annex.
CLOSED

Action Item 03A-2: Mark will move the False Alarm annex into the 1522 draft.
OPEN

Action Item 03A-3: Mark will prepare a writeup on CND based on the ARINC report to RADC and will circulate via email to the committee prior to the June meeting.
OPEN

Action Item 03A-4: Mark, John, and Eric G. will meet as a working group in March (hopefully) to complete the 1522 draft. This will involve carrying out the restructuring plan described in the minutes of the 03A meeting.
CLOSED

Action Item 03A-5: Eric G. will complete the informative annex on additional metrics using the restructured specification.
CLOSED: OBE

Action Item 03A-6: Tim and John will review the results of Eric B.'s TeRM draft and provide comments back to Eric. A draft will then be provided to Louis U. for a “fresh” review.
CLOSED

Action Item 03A-7: Mark to check the “complete” draft of 1522 for consistency.
OPEN

New Action Items

Action Item 03B-1: Tim will examine STEP standards to determine if/how part identification is modeled relative to system indenture identification.
OPEN

Action Item 03B-2: Mukund to contact Narayanan Ramachandran about assigning DTIF to DMC for purposes of running a reaffirmation ballot.
OPEN

Action Item 03B-3: Revisions required for TeRM to be addressed by Eric B. n.l.t. June 13, 2003.

1. Reference to signal will be limited to the signal_instance portion of the model and grammar.

2. Explanatory text will be added to bridge the gap between the model and execution trace.

3. Prepare a short writeup explaining the formal tie between the information model and the TeRM grammar. This part of the Action Item will not hold up the decision on whether to ballot TeRM (i.e., not tied to due date).

4. Prepare a conformance section that does not allow for extension or subsetting. Applications either conform or they don't.

5. A description on the relationships between start times of activities will be included somewhere.

6. The width attribute of activity_definition will be dropped from the model and grammar. Either that, or they will be made consistent.

7. Examine where attributes were rolled back from the grammar into the model.

8. Booleans vs. Logicals will be examined to determine appropriate types and supporting commentary.

9. Entity signal_instance will be renamed to reflect the fact that it is simply an activity instance in which the start/stop times correspond to the start/stop times of the parent activity definition (rather than just relying on this being pointed out in the commentary). We may be able to eliminate all references to “signal.” Check this out.

10. Text on 5.3.3. will be reworded to clarify roles of occurrence trees.

11. Language on constructors will be clarified.

12. The discussion on functions will be clarified.

13. Nets will be removed from the product schema.

14. Examples will be incorporated.

15. Investigate the necessity of using the design as a bridge between the product definition and the set of behaviors.

16. Consider placing an inverse attribute between product definition and design (or behaviors) in order to accommodate HW that is reconfigurable based on test outcomes.

17. Write an overview section that appears at the beginning (i.e., prior to Clause 1) that extolls the benefits of the approach.

18. Finish addressing Tim's comments.

OPEN

Action Item 03B-4: John will investigate ISO STEP work in the area of using XML as an exchange format based on EXPRESS information models.
OPEN

Action Item 03B-5: Mark will make sure the following actions need to be taken to prepare 1522 for ballot:

1. Insert a “using the fundamentals” section similar to clause 4.3 in 1232 as additional text in 1522 Clause 4.2.

2. Incorporate redlines.

3. Make sure all fundamental measures reference a model (i.e., have mdl as a parameter).

OPEN

Action Item 03B-6: One of the co-chairs will call an electronic meeting by the end of June with two agenda items: approval of P1522 to go to ballot and approval of P1598 to go to ballot.
OPEN

Action Item 03B-7: Mark will provide the false alarm annex to Tim for inclusion in P1636.
OPEN

Enclosure 2 Recent Log Items

Items logged at 02A

#333 Draft Proposal “Requirements Document for IEEE Standard for Diagnostic and Maintenance Information Retrieval”

#334 MOA on Operations Suitability Terminology and Definitions

Items logged at 02B

#335 False Alarm Appendix

#336 Eric Gould excerpt from STAT user’s guide

#337 Eric Gould excerpt from minutes of 98-B AI-ESTATE meeting

#338 Eric Gould excerpt from his Autotestcon 1999 paper

#339 DMC Briefing to the ATS Management Board

#340 False Alarm prediction presentation (Greg Bowman’s)

#341 False Removal Metrics Under a Single Fault Assumption document provided by Sudipto Ghoshal

Items Logged at 02C

#342 SIMICA online PAR form

Items Logged at 03A

#343 BIT and False Alarm Excerpt

#344 BIT and False Alarm Slides

Items Logged at 03B

#345 Tim Wilmering's comments on TeRM.

#346 IEEE Patent Slides

#347 ATML Presentation

Enclosure 3 Open Issues

Current Issues

Issue 00B-1: There are no higher order services currently defined with respect to the static models. To aid in manipulation, analysis etc., of these models higher order services may be beneficial. For example the following service has been proposed by Qualtech. get_test_outcome_from_diagnosis(diagnosis, set of test) Qualtech: Allows user to get the test outcomes predicted from the seeding of a particular diagnosis (fault). This corresponds to a lookup of the fault in the test-diagnosis (D-matrix) matrix. Additional Notes from 00C: The question has to be addressed as to the general applicability of the services from a standardization perspective. It would be of value to consider these services in the context of the 1522. In particular an annex to 1522 of AI-ESTATE services providing analyses and calculation of metrics might be of value. Moved to SIMICA,
Status: OPEN Medium Priority

Issue 01C-1: Testability metrics based on maintenance philosophy, such as Fault Resolution, can provide a means of validating predictive measures. At this point, the information models used to support definition of metrics in P1522 are insufficient to address maintenance philosophy – it is hoped that this deficiency can be addressed in the future through the creation of the SIMICA information model.
Moved to SIMICA, Status: OPEN Medium Priority.

Issue 02A-1: Log # 335 – a draft version of the False Alarm Appendix (Annex) submittal contains several section enumerating metrics for False Alarm and Assurance Tolerance. These metrics are to be considered for later inclusion in the standard – after we have formalized the definitions of False Alarm metrics.
Status: OPEN High Priority

Issue 03A-1: Several issues have been identified that need to be addressed, possibly in a supplement to 1232. These issues will all be collected in this section.

• The failure rate entity in the CEM lacks any way to record a value for failure rate. Three options have been suggested: 1) restore the construct from 1232.1-1997, 2) create an attribute of failure rate tied to non-time-cost, or 3) create an attribute of failure rate tied to frequency.

• Change the action_count attribute of frequency to event_count.

• Although failure rate is an attribute of diagnosis (and therefore fault and failure), and we can get to repair item through fault and failure (via func), there is no explicit relationship between a repair item's failure rate (which is not specifiable in the model) and a diagnosis's failure rate (which is). A repair item's failure rate should be an aggregate of the failure rates of its constituent faults XOR failures.

• Examine viability of either replacing or supplementing exchange format with an XML DTD/Schema.

• Revisit all definitions within the information models.

Status: OPEN Medium Priority

New Issues

Enclosure 4: Meeting Attendees List

DMC 03-C Draft Meeting Agenda: [Month] 2003, Location TBD

Day 1: Tuesday

• Call to Order

• Approval of Agenda

• Reports

  • Chair's Report

  • Secretary's Report

  • Liaison Reports

  • Project Reports

    • P1522 Testability

    • P1598 TeRM

    • P1636 SIMICA

• Brief Presentations

  • Approach to Information Flow through the DMC Standards (W. Gerstein)

• Testability/Diagnosability

  • Review Draft from Working Group

  • Complete and Incorporate Conformance Clause

• Review Status for Ballot

Day 2: Wednesday

• TeRM

  • Complete or Terminate

• SIMICA Discussions

  • Work on Initial Draft

  • Work on Information Model

Day 3: Thursday

• Final Discussions on All Topics

• Review Old Action Items

• Review New Action Items

• Set Time and Location of 03-C Meeting

• Set Agenda for 03-C Meeting

• Adjourn