Day 1: Tuesday February 9, 1999: Convened
at 9:15 AM
1 Attendees are shown in Enclosure 5.
1.0 Co-Chair John Sheppard called the meeting to order.
1.1 The agenda was reviewed and approved with modifications.
1.2 Chair's Report
1.2.1 John Sheppard talked about 1232.2. It will be published 2/19 and is now at the printer.
1.2.2 The new .2 was forwarded to an IEC representative, who will forward it to ANSI. There are now 3 documents in fast track, the longest having been there for three years.
1.2.3 Two electronic meetings were held since the last meeting. The first was resolved, with T. Bearse leading discussion at this meeting.
1.2.4 The subject of
distribution of the standard was discussed. The IEEE has not yet
determined if models can be removed from the document and then published
only via web or diskette.
1.3 Secretary's Report
1.3.1 Minutes from the 98-B meeting were approved without modifications as reflected in AI-ESTATE Log #279. Two electronic meetings were held between AI-ESTATE meetings and Log items were added, documenting the discussion traffic.
1.3.2 Action items were reviewed and updated, as shown in Enclosure 1.
1.3.3 Open Issues were reviewed and updated, as shown in Enclosure 3.
1.3.4 The subcommittee discussed the failure distribution issue, to address Open Issue 98B-4. A motion was made to accept the result of the e-meeting (Log #280), that no additional distributions be identified in the model. The motion passed with no objection.
1.4 Working Group Reports
This will be the last meeting
in which working group reports are given, due to the small size of the
subcommittee. From the next meeting on, "focals" will report on activity.
1.4.1 Testability Standard
Working Group:
M. Kaufman reported that,
since there may be changes as a result of the changes to the unified standard,
and the priority of that document, no activity occurred with the Testability
standard.
1.4.2 1232 Unification
Standard Working Group:
M. Kaufman reported that
he has prepared the latest revision of the unified standard. The
CEM was cleaned up
1.4.3 1232.2 Services Working
Group:
No report
1.5 Liaison Reports:
1.5.1 Computer Society
Liaison:
In the Computer Society
Test Technology technical committee is now the Test Technology Technical
Council (TTTC).
As a result of discussion
at 98D, John Sheppard talked to Test Technology Standards Committee (TTSC)
officers about moving AI-ESTATE to the Computer Society. Most were
amenable to the idea. John recommended that there is no need to take
action immediately to move AI-ESTATE to Computer Society, since even if
SCC20 goes away, the IEEE will work to find homes for all continuing subcommittees.
John requested that all subcommittee members look at keeping costs low
for our meetings.
1.5.2 IEC TC93/WG7:
1232.2 electronic format
(PDF) has been submitted to IEC, but no other activity has occurred.
John suggested that we should
pull the three documents we have submitted to IEC fast track. Randy
cautioned that there are political repercussions of such an action.
The current fast track actions are expected not to result in IEC approval
of our standards, since there will be no direct support in WG7 from AI-ESTATE
members. Action will be taken to
determine recommended next steps.
1.5.3 NATO CALS:
Greg Bowman reported that
initial contact has been made with the NATO CALS folks. There is
definite interest in working together to possibly harmonize the two efforts.
Additional discussions will occur, with Tim Wilmering becoming involved
to explore TMIMS possibilities.
1.5.4 SCC20:
Randy Simpson reported
that a new chair was elected and the SPWG report was approved. Appropriate
operating procedures for the SCC20 to implement the SPWG report were not
in place for the 99-1 meeting, so an electronic discussion was held to
resolve operating procedures before the meeting. Four liaisons (NATO,
French, German, and ARINC) will no longer be part of SCC20.
1.5.5 Publicity:
Data sheets have not been
updated. The AI-ESTATE web site is continuously updated.
Avionics magazine published
an article in the January 1999 edition.
1.6 The subcommittee next discussed the unified standard, looking at the changes to the CEM.
1.6.1 Tim Bearse gave a presentation of using inherent EXPRESS-I representations to obviate the need for lower level services, such as counting.
1.6.1.1 The consensus was that we need some method of determining counts, etc. It was agreed that the best method for count was not low level services, EXPRESS-I, or including it in the bindings. Dave Kleinman and Amanda Giarla accepted an action to provide a proposal for solving the counting/existence services issue based on a specific 'cheese' implementation.
1.6.2 Model changes were made to rename 'object_of_repair' to 'repair_item'.
1.6.3 The group discussed
software versus hardware failure modes and faults. We expanded the definition
of failure mode to include software 'design errors', since software does
not fail and to disambiguate.
Day 2: Wednesday February 10, 1999: Convened at 08:30AM
2.0 Attendees are shown in Enclosure 5.
2.1 A 15 minute presentation on GRADME was given by Tim Bearse.
2.1.1 An issue was raised and discussed about tying TeRM to 'actions' and 'resources' in our model. John Sheppard accepted an action to discuss the subject further with Lee Shombert.
2.2 A 15 minute presentation on SBIR Topic AF98-252 Phase-1 was given by Amanda Giarla. She described the latest progress in the SBIR, including their focus on demonstration of feasibility of a larger picture than just AI-ESTATE, to include the implementation of a communications methodology (TCP/IP, COM, CORBA, etc.) within an open systems approach.
2.3 The subcommittee returned to discussion of the unified standard and the associated model.
2.3.1 The capability to represent "functions" as well as "physical faults" and provide a method for mapping between the two was discussed. A model representation was identified, but the mapping was not determined.
2.3.2 The capability to "enable" or "disable" entities (initially for the purpose of testability analysis), and an attempt to determine which entity or entities to give an "enable" attribute did not lead to a recommended model structure.
2.3.3 Tim Wilmering accepted an action to make modifications to the model in Visio, as discussed in the meeting. Greg Bowman will provide one portion (the diagnostic_model level) to Tim via email in Visio format.
2.4 The subcommittee next discussed the metrics of the testability standard, defining primitive elements from which the metrics can be calculated.
2.4.1 John Sheppard took an action to put together a proposal and identify and define the primitives and calculation for Fraction of Faults Detectable, Fraction of Faults Isolatable, and Operational Isolation/Fault Isolation Resolution. Others were encouraged to identify and define additional metrics.
2.4.2 The definition of 'characteristic' was changed from the Toulouse meeting, removing "observable", as below. Characteristic: A quality or property of a system. Characteristics may or may not be measurable.
2.4.3 Randy Simpson accepted an action to send the AUTOTESTCON reference (Keiner 1980) for the testability definition to Mark Kaufman.
2.4.4 Mark Kaufman will make modifications to the IEEE P1522 straw man, as identified during meeting discussions and start a revision history for the testability document, tracking reasoning behind each substantive change to the document. Mark will also provide the AI-ESTATE Secretary with an electronic copy of the most recent 1522 straw man, for logging.
2.4.5 The group agonized over the overall approach to AI-ESTATE, in that we now have IEEE 1232 standards AND IEEE 1522, for heavily related topics. The eventual development of TMIMS-related standards, and the currently developing TeRM represent other related standards work that may better fit under one standards umbrella.
2.5 The subcommittee
decided to wait for the SCC20 operating procedures changes to become official
before we change the AI-ESTATE operating procedures.
Day 3: Thursday, February 11, 1999 convened at 8:15AM
3.0 Attendees for this day are shown in Enclosure 5.
3.2 Next, the subcommittee
addressed issue 98A-2, to make sure we were
allowing constraints to be built from the current common element model
entities (as currently exists in the unified document draft). It
was agreed to add 'priority' as a descriptor attribute of the 'context'
entity. Additionally, each entity below 'context' (e.g. 'repair_context')
will be defined as Sets.
3.3 The group discussed
approaches to including a constraint to specify ordering of tests, not
in the simple Fault Tree method of "hard" ordering, but in a "soft" ordering,
that allows the model developer to introduce targeted 'never perform 'test
a' before 'test b'. The group focused on two options, integrated
model elements and a planning model.
John accepted an action
to develop a proposal to create a planning model (strips) to address the
issue.
Dave Kleinman accepted an
action to develop a proposal to integrate
ordering elements into the existing models (CEM, EDIM, and/or FTM).
3.4 The subject of electronic meetings was discussed. The group decided that 0-3 scheduled meetings (SCC20 or otherwise), plus ad hoc face-to-face meetings plus e-meetings would be the model that we would use for future meetings. We also decided to attempt to use Net Meeting to conduct a trial meeting. This meeting, will be organized by John Sheppard, will decide if the tool is useful.
3.5 The AI-ESTATE road map review was postponed until next meeting, since we have many changes anticipated to our path.
3.6 Review of action items and open issues were accomplished.
3.7 Agenda for the 99B meeting was established, as shown in Enclosure 6, and it was approved unanimously.
3.8 The meeting adjourned at 12:40PM.
Action Item 98A-5. John Sheppard
will provide knowledge and data specification revisions to establish an
ABSTRACT Supertype/Subtype relationship between "diagnostic_model" and
FTM/EDIM.
Status: CLOSED OBE, Items
1,2 and 4 below were completed in P1232 revision. Item 3 to be completed
by John Sheppard and provided to Mark Kaufman.
1. drop subtype relationship
between CEM and FTM/EDIM
2. Add "corresponds_to"
link from CEM/FTM back to CEM
3. Add name/description
to FTM/EDIM
4. Retain conformance restriction
that AI-ESTATE require FTM or EDIM.
Action Item 98B-2: Randy
Simpson will bring up the issue of electronic representation of standards
to the IEEE Staff and, if appropriate, ProCom, for the purpose of efficient
and informative linking of standards data.
Status: CLOSED
Action Item 98C-1: John Sheppard/Les
Orlidge/Greg Bowman to consult electronically, to clean up the log items
on the web site.
Status: CLOSED Posted
on the web site.
Action Item 98C-7: Tim Bearse
to send electronic copy of 1232.1 comments, based on GRADME implementation,
to John Sheppard for posting on the web.
Status: CLOSED
Action Item 98C-8: Mark
K. to gather Testability information from Eric Gould. Get copies
of RADC report from John Sheppard (when he gets them back from Tony Bartolini)
to Arnie (Meeting Co.) for copying for the 98-2 SCC20 meeting.
Status: OPEN.
Mark Kaufman will distribute to AI-ESTATE Testability Working Group members
(attendance list of last meeting).
Action Item 98C-10: Greg
Bowman to arrange for AI-ESTATE members to be added to the STEP/SGML NIST
exploder.
Status: OPEN, list
of members to be provided to Joed Dougherty. Members list provided
to Joed. Check with Joed. Some members receiving, others not.
Action Item 98C-12: Greg
Bowman to get approval (through Joed Dougherty) to post NCDM_202 (NATO
CALS model) on the AI-ESTATE password protected web site.
Status: CLOSED.
Posted in private area.
Action Item 98C-13: Greg
Bowman will contact Lisa Fowbles (DoD Config Mgmt) about exploring opportunities
to marry AI-ESTATE to NATO CALS model.
Status: OPEN.
Greg Bowman sent email to John Dunford, who is responsible for the NATO
CALS model. Funding opportunities to establish formal relationship.
Action Item 98C-15: Tim Wilmering
will contact Charles Amaral about obtaining a copy of AP-208 (Life Cycle
Mgmt for Process Plants).
Status: OPEN.
Action Item 98D-1: John Sheppard
to investigate copyright issue with Bill Hagen at IEEE whether the EXPRESS
code could be posted on the AI-ESTATE web site for the purpose of separating
the repetitive volume of code from residing in the standard. It would
be balloted, but not included in the paper copy.
Status: CLOSED, with
ongoing discussions.
Action Item 98D-2: Mark Kaufman
to make changes to the unified standard document, per the subcommittee
discussion, to close actions 98A-5,6,7.
Status: CLOSED
Action Item 98D-3: Greg Bowman
to contact Luigi Napoli to attempt to raise the size limit on the AI-ESTATE
reflector.
Status: OPEN. John
electronically updated the limit using the 'MajorCool' IEEE web interface.
Randy will test the limit with meeting pictures 10K<x<1M.
Action Item 98D-4: Amanda
Giarla to contact Tim Bearse to explore requirements for a Guide to the
use of AI-ESTATE.
Status: CLOSED
New Action Items
Action Item 99A-1: A. Giarla has an action to explore the status and future of IEC liaison by determining the pros and cons of possible options.
Action Item 99A-2: Dave Kleinman and Amanda Giarla to provide a proposal for solving the counting/existence services issue based on a specific 'cheese' implementation.
Action Item 99A-3: John Sheppard to talk TeRM with Lee Shombert.
Action Item 99A-4: Tim W. accepted an action to make modifications to the model in Visio, as discussed in the meeting.Greg Bowman will provide one portion (the diagnostic_model level) to Tim via email in Visio format. Coordination to be accomplished with Mark Kaufman.
Action Item 99A-5: John Sheppard took an action to identify and define the primitives and calculation for Fraction of Faults Detectable, Fraction of Faults Isolatable, and Operational Isolation/Fault Isolation Resolution.
Action Item 99A-6: Randy Simpson to send the AUTOTESTCON reference (Keiner 1980) for the testability definition to Mark Kaufman.
Action Item 99A-7: John accepted an action to develop a proposal to create a planning model (strips) to address the issue.
Action Item 99A-8: Dave Kleinman accepted an action to develop a proposal to integrate ordering elements into the existing models (CEM, EDIM, and/or FTM).
Action Item 99A-10: Randy to create a proposal for false alarm metric(s) for the Testability standard.
Action Item 99A-11: Greg accepted an action to explore the possibility of creating a demonstration of AI-ESTATE as part of V-22 RTCASS transportability, to test capability to provide partial/reconfigurable TPS test functionality.
Action Item 99A-12: Bill will explore the USMC position on AI-ESTATE capability in NxTest.
Action Item 99A-13: Tim Bearse will explore role of AI-ESTATE in TDC initiatives.
#275 98C AI-ESTATE Meeting Minutes, August 11-13 1998, The Boeing Company, Philadelphia, PA (11/16/98)
#276 Unified 1232 Draft (Big Mammu) (E) (11/16/98)
#277 IEEE Standard Testability and Diagnosibility Characteristics and Metrics (E) (11/17/98)
#278 Operations Manual for 1232 (11/18/98)
#279 98D AI-ESTATE Meeting Minutes, November 1998, Toulouse France (2/9/99)
#280 Traffic for electronic discussion on including failure rate distributions in the standard
#282 Strawman Draft of 1522, Testability Standard
#283 Traffic for electronic discussion on low level services
#284
Unified 1232 Standard Draft
98B-2:
Should we specify higher order services corresponding to functions satisfying
constraints in the 1232.1 and 1232.2 information models?
Status: CLOSED.
No changes will be made now.
98B-3: Should we specify
additional low level services (e.g. counting and existence services)?
Status: CLOSED, see
Action Item 99A-2.
97D-3:1232.1 should be amended
to address the appropriate conceptual information model entities covering
functional (quasi FMEA) definitions, as shown in Log 262. Those not appropriate
for 1232.1 should be addressed during context model discussions. Those
overlapping entities (Strategies, Test Nodes, Test Methods) should be addressed
by the ABBET liaison Lee Shombert. Need to clear up definition of
"functional" and "context" for Unified document.
Status: CLOSED
98A-2.
There is a need to represent constraints (safe-to-power-on, test ordering,
modes, etc.) in existing models.
Status: CLOSED.
See action items 99A-7,8.
98C-5 (From the 1232.2 Recirculation
comments) Section 7 is also loosely worded and ambiguous. It allows
implementations claiming to be standards compliant to have stubs that return
the “Service_not_available” status code, while insisting subset compliance
is not permitted.
As stated in the original
response, “We appreciate the concern as regards a baseline expected behavior.
We hope to address this concern, perhaps in the development of one or more
AI-ESTATE Recommended Practices in the future.” Concerning the suggestion
for a state diagram, an early version of 1232.2 had such a diagram and
found all services allowable in all cases (at least for now). Concerning
clause 7, we felt that requiring implementation of all services beyond
stubs was potentially too demanding; however, we did not want to open the
door to subsetting until we knew which subsets made sense. This is
an issue we expect to resolve when moving to full use.
Status: OPEN -Medium
Priority
98D-1: Is an interchange
format needed for the Dynamic Context Model, perhaps to deal with TMIMS-related
issues and/or case based reasoning?
Status: OPEN -Low
Priority
97B-3: (Unification) High
order reasoning services, what are they and where do they belong? Should
they be part of Application Executive or part of the diagnostic reasoner
or should they stand alone. Additional higher order functions, including
'revert' and 'choose test' are examples. An earlier draft of 1232.2 should
be reviewed to identify many of the higher order services.
Status: OPEN - Low
Priority .
98B-8: Should a standard
or recommended practice be generated to represent a baseline expected behavior?
Status: OPEN -Low
Priority
98B-4:
Should we represent distributions of failure rates in 1232.1 other than
as exponential distributions (e.g Weibull, log normal)?
Status: CLOSED
A motion was made and passed
to accept the result of the e-discussion, that no additional distributions
be identified in the model.
98B-7: Need an explicit
link between anomaly and level of repair, in order to constrain diagnosis
to piece parts, functions, procedures, and adjustments, etc.
Status: CLOSED, addressed
by changes identified in Toulouse.
New Issues
99A-1: Organization and name of the AI-ESTATE standards. Do we change the name of the organization to better represent our work? Do we incorporate the Testability standard into the unified document?
Morning
I. Opening the Meeting
1. Changes to Agenda
2. Chair's Report
3. Secretary's Report
Minutes of 98-D, Toulouse, France
Action Items
Issues
II. Reports and Presentations
1. Working Group Reports
Unification Working Group
Services Working Group
Testability Working Group
2. Liaison Reports
Computer Society
IEC/TC93/WG7
Review of AI-ESTATE/IEC Relationship
NATO CALS and SGML/STEP Contacts
SCC20
3. AI-ESTATE Publicity Report
Update Data Sheets
Update Web Pages
4. 15 minute presentations
AI-ESTATE - IN GRADME Presentation - T. Bearse
SBIR - A. Giarla
TeRM - L. Shombert
III. Operating Procedures
Afternoon
IV. P1232 Unification Review
1. Organization of document
2. Model revisions
4. PAR and Requirements Documents
5. Issues for Completing P1232 Document
a. Low Level Services
b. Failure frequency
c. 'where' clause functions as high order services
d. Functional/Context Issue
e. Constraints Issue
f. Conduct of Electronic Meetings
Wednesday, February 10, 1999
Morning
V. Testability Standard
1. Review Strawman
2. Maintenance Activity Model
- Review Applicability of RADC Model
- Recommendation for Modifications
- Establish Consensus Activity Model
Afternoon
V. Testability Standard (cont.)
3. Testability Information Model
- Revise Information Model
- Identify Constraints
Thursday, February 11,
1999
Morning
V. Testability Standard (cont.)
4. Identify
Strawman Holes
5. Assign
Authors
6. Testability
Completion Schedule
VI. Re-examine/Update
P1522 Requirements Document
VII. Review Issue of Testability
Recommended Practice
Afternoon
VIII. Administrative Reviews
1. AI-ESTATE Standards Development Roadmap
2. Review Action Items and Open Issues
3. Subcommittee Organization
IX. Set and Approve Agenda for 99-B Meeting
X. Adjourn
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Afternoon
I. Opening the Meeting
1. Presentation of Agenda (changes to be made as needed)
2. Chair's Report
3. Secretary's Report
Minutes of 99-A Monterey, CA
Action Items
Issues
II. Reports and Presentations
1. Focal Point Reports
Unification
Testability
TMIMS
2. Liaison Reports
Computer Society
NATO CALS and SGML/STEP Contacts
3. AI-ESTATE Publicity Report
Update Data Sheets
Update Web Pages
4. 15 minute presentations
TeRM - L. Shombert
Web-based Diagnostics - Qualtech
III. Operating Procedures
1. Conduct of Electronic Meetings
2. Evaluate relationship to SCC20 OP.
Tuesday, April 27, 1999
Morning
IV. P1232 Unification Review
1. Model revisions
Afternoon
IV. P1232 Unification Review
(continued)
2. PAR and Requirements Documents
3. Issues for Completing P1232 Document
4. Prioritize and Address open issues
Wednesday, April 28, 1999
Morning
IV. P1232 Unification Review
(continued)
4. Proposals
a. Counting and Existence
Afternoon
IV. P1232 Unification Review
(continued)
4. Proposals
b. Planning Model
c. Integrated ordering model constructs
Thursday, April 29, 1999
Morning
V. Testability Standard
1. Review of PAR
2. Proposals
a. FFD, FFI, OI/FIR
b. False Alarm
c. Others...
Afternoon
V. Testability Standard (continued)
3. Identify Draft Holes
4. Assign Authors
5. Testability Completion Schedule
VI. Re-examine/Update P1522 Requirements Document
VII. Review Issue of Testability
Recommended Practice
Friday, April 30, 1999
Morning
VIII. Administrative Reviews
1. AI-ESTATE Standards Development Roadmap
2. Review Action Items and Open Issues
3. Subcommittee Organization
IX. Set and Approve Agenda for 99-C Meeting
X. Adjourn
98-D Meeting (Toulouse,
FR)
- Review Stawman of Testability
Standard
- Identify key elements
of maintenance activities
- Review Issue of Testability
Recommended Practice
- Review Resources Available
for TMIMS Work
- Issues for Completing
P1232 Document
- Review PAR and Requirements
Document for 1232
- Review Draft P1232
Between 98-D and 99-A
Meeting
- Incorporate Changes To
Testability Standard
- Revise/Distribute P1232
to Reflect AI-ESTATE and Testability Issues
- Conduct Electronic Discussion
on the following issues
98B-3:
Should we specify additional low level services (e.g. counting and existence
services)?
98B-4:
Should we represent distributions of failure rates in 1232.1 other than
as exponential distributions (e.g Weibull, log
normal)?
99-A Meeting (Monterey,
CA)
- Discuss Maintenance Activity
Model
- Review Information Models
- Review Testability Standard
Draft
- Review Available TMIMS
Resources/Form Task Group on TMIMS
- Review Recommended Practices
Issues
- Review Draft P1232
- Resolve P1232 PAR Issue
- Report on NATO CALS and
SGML/STEP Contacts
- Re-examine/Update P1522
Requirements Document
- Discuss P1232 Open Issues
Between 99-A and 99-B
Meeting
- Revise Testability Information
Model
- Publication of P1232.2
- Information Gathering
for TMIMS
- Revise Testability Standard
- Revise P1232 Draft
- Address P1232 Open Issues
- Informal circulation for
comment of P1232
99-B Meeting: IDA,
Alexandria, VA
- Review AI-ESTATE and Testability
Information Models
- Review P1232 Draft
- Review P1522 Comments
- TMIMS Task Group report
- Revise and approve (AI-ESTATE/Steering)
P1232 PAR and Requirements Document
Between 99-B and 99-C
Meeting
- Complete AI-ESTATE and
Testability Information Models
- Incorporate P1232 Open
Issues into P1232
- Incorporate Testability
Definition, based on models into P1522
- Informal circulation for
comment of P1522
- Complete P1232 Document
99-C Meeting
- Review Draft P1522
- Review Comments on P1522
- Approve P1232 for Ballot
Between 99-C and 99-D
Meeting
- Prepare Final Version
of P1522
- Generate P1232 Ballot
Constituency
- Ballot P1232
- Begin P1232 Ballot Resolution
99-D Meeting
- Approval of Ballot for
P1522
- Complete P1232 Negative
Ballot Resolution
- Approve P1232 for Ballot
Recirculation
Between 99-D and 2000-A
Meeting
- Generate Testability Ballot
Constituency
- Ballot P1522
- Recirculate P1232
2000-A Meeting
- Resolve P1522 Ballot Comments
- Approve P1232 for Ballot
Recirculation
Between 00-A and 00-B
Meeting
- Recirculate P1232