DIAGNOSTIC AND MAINTENANCE CONTROL SUBCOMMITTEE
IEEE STANDARDS COORDINATING COMMITTEE 20
07-B MEETING
Madrid, Spain
April 16-19, 2007


07-B Agenda

Monday, April 16, 2007

The meeting was called to order at 9:32 am by Tim Wilmering.

It was moved and seconded that the agenda be approved. It was approved without opposition.

Chair's Report: There was nothing to add beyond the report given in the SCC20 plenary.

Secretary's Report: The minutes were reviewed. The minutes were approved without modification.

Action Items and Open Issues were then reviewed.

CS Liaison had nothing to report.

I&M Liaison has been conducting telecons with TC10 to coordinate signal-related ATML issues.

AES Liaison has nothing to report.

OSA-CBM Liaison was absent; however, OSA-CBM was absorbed into MIMOSA and is being standardized through ISO. There is also a new CBM standards group meeting in Prague that may be somehow related to OSA-CBM.

Discussion proceeded on the order constraints for P1232. The working group reviewed the three proposals from Darryl Busch. There are some elements of a separate schema that are quite appealing; however, we quickly moved into a discussion of whether ordering belongs in the purview of Test Description or Diagnostics. We decided that it was not a simple answer since there are places where either or both should have responsibility. The general consensus, however, was that we should not declare order constraints to be “out of scope.” Additional consensus was that, at a minimum, the preconditions defined at the moment are required.

There was no consensus on the extent to which postconditions or other logical constructs should be included. For example one of the elements of Proposal 2 from Darryl (see Log #451) included a general “if-then” rule construct that incorporated full propositional logic. While different in form, this is equivalent to the original “STRIPS” operator from the current 1232-2002 standard. The working group decided at the 06-A-1 1232 working group meeting to remove this from the model for two reasons: 1) the complexity of the model, and 2) the fact that this permitted a compliant diagnostic system to be built without using one of the diagnostic models (other than the fact you MUST use one of the diagnostic models).

An additional point made on the order constraint issue was that, the extent that constraint-based or temporal diagnostic reasoners exist should be what drives what goes into the standard. It is not clear at this time that there are a lot of these types of reasoners in the industry, and there does not seem to be specific experience with these types of reasoners in the working group. Therefore, it seems more reasonable to keep the ordering constructs “minimalist” until such time as the industry indicates the need for these types of standardized constructs. Note that, while not ideal, more sophisticated order can be managed through an application-specific application executive (i.e., if ordering beyond the “minimalist” constructs should be required).

Resolution: Consensus was to keep the construct as is.

Tim found an email from Darryl indicating his recommendations for the Bayesian model were more stylistic than anything else. Therefore, there were no Bayesian proposals to consider at this time.

Michelle presented a revised state diagram (Log #452). She pointed out that Darryl was hoping for more detail, perhaps along the lines of a UML sequence diagram. We are querying Darryl for more information on what, exactly, he is looking for. The concern is that such a sequence diagrams may go too far in specifying implementation.

In reviewing the state diagram, the only change noted was that 1232-2002 includes a transition from “Step Created” back to itself upon a transition with applyOutcome. This transition has been removed. In addition, the transition from “Step Created” to “Step Performed” has been relabeled with applyOutcome instead of updateState. Originally, these were separate services to support batch processing of test results. There is a question as to whether or not updateState is now embedded within the applyOutcome service. Action on the state diagram is deferred until we hear back from Darryl and Alicia.

We discussed the DIM and the EDIM. The DIM was intended to provide a way of reasoning with the so-called D-matrix. The EDIM was an enhancement of the DIM intended to permit any arbitrary logical combination of inferences. Unfortunately, the DIM ended up being more general than the D-matrix, and the EDIM ended up not being general enough. It was proposed that the two should be merged and modified to satisfy the original intent of the EDIM.

After considerable discussion, the working group decided to keep two separate models. The DIM was renamed the “DmatrixInferenceModel” and restricted to represent only what can be handled in a “canonical” D-matrix. The EDIM was also modified to represent more general logical constructs. It was also renamed the “DiagnosticLogicModel.” The EDIM has been removed.

The changes to the DmatrixInferenceModel can be summarized with the following comment on the entity by the same name:

This entity represents the constituents of the D-Matrix Inference Model. The expectation is that this model represents a "canonical" D (i.e., dependency) matrix from the perspective of what can be inferred from test outcomes. Specifically, in a canonical D-matrix, the columns correspond to tests. Rows of the matrix can be either tests or diagnoses. It is required that, if a test passes, then all of the "column" inferences drawn in the matrix be logically ANDed together. Conversely, if a test fails, it is required that all of the "column" inferences drawn in the matrix be "ORed" together. It is also required that inferences be of like type (i.e., Pass => Pass/Good or Fail => Fail/Candidate).

Note that this prohibits the use of “Bad” as a diagnosisOutcome in the D-matrix model. It was suggested that John's JETTA paper be included in the references to illustrate/explain the limitations of what can and what cannot be represented in a D-matrix.

Similarly, the changes resulting in the DiagnosticLogicModel can be summarized with the following comment on the entity by the same name:

This entity represents the constituents of a generic Diagnostic Logic Model. The model facilitates representing expressions of the form Outcome => AND(OR(outcomes)) or Outcome => OR(AND(outcomes)).

Note that the DLM is capable of representing any arbitrary logical expression as well as handle “asymmetric” reasoning. The asymmetry can arise simply by a) not including equivalent expressions for complementary outcomes or b) providing different (i.e., non-converse) inferences for the complementary outcomes. For example, if we have (Tx = Pass) => (Ty = Pass & Tz = Pass), symmetric inference would require (Tx = Fail) => (Ty = Fail | Tz = Fail). For part a, the Tx = Fail rule would not exist. An example for part b is (Tx = Fail) => (Ty = Fail | Tw = Fail). Finally, the DLM associates inferences from outcomes of all types and values; therefore, for example, inferences from diagnostic outcomes are permitted as are inferences of diagnoses being “Bad.”

Tuesday, April 17, 2007

The meeting convened by considering a service specification proposal from Darryl Busch based on manipulating “whole entities.” This proposal is included in Log #453. It was accepted in principle by consensus among the services ad hoc working group.

The issue of how to deprecate previous versions of the standard needs to be addressed. Specifically, for those that have implemented 1232-2002, to what extent will that standard remain available for conformance once the radically revised P1232 is published? What deprecation/migration strategy will be supported by the most recently published standard? This needs to be addressed in the conformance section.

The question was raised about whether the standard should continue to support primitive accessors or the standard should be limited to “whole entity” accessors. This issues will be taken up by the 1232 ad hoc committee.

Another question was raised about how to specify the serialization format. Is an object model required? For example, ISO 10303 Part 22 (SDAI) is based on the underlying EXPRESS schema, so entities would be passed according to that schema. A corresponding C++ or Java implementation does not have the schema, so how does one specify, in a standard way, how to serialize the objects? This needs to be addressed in 1232.

The proposal was reviewed and comments embedded in the logged version.

Discussion continued with Joe discussing the status of MAI. The updated schema is Log 454. We also discussed a document summarizing some work examining maintenance data related to the flight data recorder (FDR), which has been logged as Log 455.. This was in response to action item 07A-2. During the discussion, it was reaffirmed that the FDR would be useful. Tim also raised the question about the applicability of FOQA/MFOQA (Flight Operations Quality Assurance/Military Flight Operations Quality Assurance). This also seems to be something of interest.

Tim suggested that once MAI is completed, we should start considering a PAR for performance data. Such a component would then open the can of worms of how to handle domain/technology specific data since much performance data will be tied to the specific type of platform. The group discussed several possible approaches for dealing with these different domain types but came to no consensus on how to address them. The committee agreed that it is important to get people involved with experience in capturing performance data as well as people with vested interests in the various domains.

The working group had a prolonged discussion on the relationship between MAI and ATML. The initial issue was the role of Common. The consensus of the group was to continue to use Common whenever it made the most sense. A related issue discussed was the role of services in SIMICA and their relationship to ATML. It was proposed that, following publication of P1636.1 (Test Results), the DMC put in a PAR to amend Test Results to include a set of services. In fact, all of the SIMICA PARs identify the need to create services. In the interim, a generic “getDocument” service was proposed to be added to P1232 (actually, this may already exist under a different name in 6.2). While this service can be provided by an AI-ESTATE reasoner, it can also be provided by other applications, such as Test Results, as well. (P1232 also needs some services to support submitting Test Result and MAI-related data to the reasoner as well.)

The complete MAI schema was reviewed and modified based on several comments. The primary work items outstanding include completing the definitions, completing the information model, and developing the draft standard document.

Wednesday, April 18, 2007

Discussion began with what to do with 1522. The issues before us are:

The intent is for the standard by program managers/procurement folks to levy testability requirements. Bill pointed out that most customers are either using 2165 or ad hoc requirements. Joe's opinion is that the primary issue is publicity.

John made the point that the problem lies in the history of how the standard came about. In 1994, the Perry Memo cancelled all Mil Stds (except those receiving waivers), and this led to MIL STD 2165 becoming MIL HDBK 2165. This also led to the DoD (under Marty Meth's office) wanting to create an equivalent IEEE standard, and DMC received the ball to run with. They created 1522 as an attempt to do that in a way that would be enforceable because of the tie to formal models. Unfortunately, at that time, the DoD stepped out of the process, and 1522 likely diverged from what the DoD thought they need. He also expressed the opinion that no revision will be successful unless we get those with a vested interest (i.e., the DoD and MoDs) directly involved in developing and championing the revised standard. Therefore, John recommends that DMC go to Steering and do one of two things:

  1. Submit a PAR for revising 1522 and recommend that the PAR be approved only if the defense (DoD and MoD) liaisons actively solicit and engage support from the relevant PGMs or PMAs to ensure the revision directly addresses defense procurement needs.

  2. Defer submitting the PAR for revising 1522 until such time as the DoD or MoD programs commit to assisting in revising the standard in such a ways as their requirements are satisfied and then commit to carry the banner to specify the standard in future procurement.

It was moved by John S. that the DMC proceed with option 2. Joe S. seconded. There was no discussion and no opposition. The motion carried.

We received a response from Darryl B. on the order constraint issue. Darryl attempted to argue his case. The subcommittee re-examined his proposal at length in light of his rebuttal and decided to reaffirm its decision from Monday. In addition, an open issue on order constraints was created to be taken up on the next revision of the standard (07B-1). The complete email thread between Darryl and the DMC (via Tim) is Log #456.

Discussion moved to SIMICA with a review of the conceptual model. During the discussion, the role and location of Discrepancy came up. The resolution of the discussion resulted in a new action item for Joe S. and Mukund M. to revise MAI (07B-5). In addition, John S. suggested that the Cause (which was renamed Discrepancy) entity in the 1232 DCM be replaced with an interface (REFERENCE FROM) to the MAI information model. This led to a discussion of ballot schedule with the working group setting a goal for MAI to have a relatively complete draft by the June meeting. AI-ESTATE already has that goal. The end goal is for both 1232 and 1671.2 to go to ballot at the same time (hopefully, by the end of the year).

Thursday, April 19, 2007

Work returned to the SIMICA model. Several major design decisions were made along the way. Specifically, placeholders were placed in the model to permit ties to external standards; however, the consensus was to tie only to information models in those standards or information models created based on the information specified in the standards. It is anticipated that, should SIMICA require ATML standards, the DMC will have to create the corresponding information model for the required standard.

Several modifications to the information model were made, including a refinement of the DesignLibrary and TestResource structures. Clearer identification of ties to existing or emerging DMC standards (namely, MAI, TRSI, and AI-ESTATE) was also made. Consensus was reached that the SIMICA model represents a top-level ontology and should have no instantiated data. Instead, it should (and currently does) identify top level concepts and point to other standards/models from which instantiations would be derived.

The working group reviewed Darryl's email on the state diagram and agreed with all of his comments. That email is logged as Log #459.

Work then proceeded to the definitions in the 1232 models. The results are in Log 460. One point that needs to be included in the standard is that the focus of the models is to define semantics and to facilitate information exchange. As a result, it is possible, if not likely, that an instantiation of the model will not be optimized for computational purposes.

The meeting ended with administrivia. The action items were reviewed. The meeting location was selected as Boston, MA on June 12-14, 2007 in conjunction with ATML. Note, however, that the room rate at the hotel is only good until May 11. Hotel details are:

Courtyard Marriott, at Natick: 800 321-2211. Ask for the code: MATMATA, The ATML/SIWG rate of $109.

The meeting concluded by reviewing the roadmap.

The meeting adjourned at 5:50 pm.

Attendees:

Name

Affiliation

Email

Day 1

Day 2

Day 3

Day 4

Scott Gearhart

US Army (Picatinny Arsenal)

sgearh@pica.army.mil


X



Bill Gerstein

HamiltonSundstrand

we.gerstein@hs.utc.com

X

X

X

X

Michelle Harris

Lockheed-Marton

michelle.l.harris@lmco.com

X

X

X

X

Hugh Pritchett

Analysis Integration & Design Inc.

hpritchett@aidinc-usa.com



X


John Sheppard

Johns Hopkins University

jsheppa2@jhu.edu

X

X

X

X

Joe Stanco

SSAI

jstanco@ieee.org

X

X

X

X

Tim Wilmering

Boeing

timothy.j.wilmering@boeing.com

X

X

X

X

Action Item Summary

Old Action Items


New Action Items


Open Issues

Recent Log Items

Items Logged at 06C

Items Logged at 07A

Items Logged at 07B

Proposed 07-C Agenda