G-2.1.6/25

Item 1 - Welcome and Introduction by Interim Chairman, of IEEE G-2.1.6

The third meeting of G-2.1.6, Audio Video Techniques Committee, the Compression and Processing Subcommittee, G-2.1.6 was brought to order by Mr. Alan Godber, Chair of G-2.1 at 8:10 AM at the Sheraton Orlando North Hotel in Maitland, Florida. Attendees were introduced.

Item 2 - Approval of Draft Agenda.

Alan Godber presented the Draft Agenda this meeting. No suggestions or changes were offered. It was approved as proposed by those present.

Item 3 - Review and Approval of Minutes of the Previous Meeting #2, September 20^th, 1996.

The minutes of the last meeting were reviewed. The following corrections were offered:

• Item 8 (continued), Paragraph 6, the third sentence, referring to the issue of audio-video timing, should be corrected to read "Al Morton commented that T1A1.5 has an advanced draft standard covering this." (Page 14)
• In Item 13, paragraph 3, fifth sentence, referring to volunteers for the task force to develop the first straw man measurement method/standard, the name Al Morton should be James O'Neal.
• In Appendix B, Attendance Record, telephone numbers for Peter Dare at Sony were changed to (408) 955-5331 (voice) and (408) 955-5301 (fax). The e-mail address for David Fibush at Tektronix was changed to davef@tv.tv.tek.com.

Alan Godber assigned the document number "20" to the minutes of the September 20^th, 1996 meeting. Those present approved the minutes, as amended, as a satisfactory record of the meeting.

Item 4 - Matters Arising from the Minutes.

David Fibush asked if we had a revised Scope for Committee work. Alan Godber replied that it was included under Item 7 on the agenda.

Item 5 - Report of Task Force on Compression Measurements Information Gathering Chair, Bill Zou.

Alan Godber acknowledged that we were short some people, noting that Task Force Chair Bill Zou was not here. He was working on adding to his extensive list of compression measurement information, but nothing has been submitted since the last meeting. The item was tabled. Alan Godber noted that copies of the list are available for those that need them. Arthur Webster, Aidan Moore, and Eric Hauch requested copies.

5.1 Further Discussion and Action.

Arthur Webster distributed copies of document T1A1.5/96-121, Standards Project: Analog Interface Performance Specifications for Digital Video Teleconferencing/Video Telephony Service (T1Q1-12), Subject: Objective and Subjective Measures of MPEG Video Quality, Source: GTE Laboratories, NTIA/ITS, Dated October 28, 1996 to those present. The document was given G-2.1.6 document number "21".

The document is also available on the T1 web page, http://www.t1.orghttp://www.t1.org, under committees/T1A1.5. A list of documents including a short summary is available at http://www.t1.org/index/0612.htm.

Arthur Webster described the report. GTE did tests on MPEG1+ and MPEG2 compression systems at various bit rates and resolutions. The report compares the various systems, covering subjective and objective evaluation of the test material. Full details are in the report. A copy of the ANSI T1.801.03-1996 (Document G-2.1.6/05) standard is needed as it is referenced in the document.

The subcommittee decided to discuss this report further under Agenda Item 9.

Item 6 - Report of Task Force on Test Imaging Materials - Chair, Peter Symes.

Chairman Alan Godber said Peter Symes was going to look into sources of imagining material. Peter Symes was not present at this meeting.

Alan Godber has attempted to meet with people from ATTC. He noted that ATTC test materials for NTSC are on D-2, composite format while the HDTV materials are in component HDD-1000 format. He said these materials are available, but the question is how to make use of them. A recorder is needed. Acceptable transfers must be done. It will come down to money at some point, from some place. Alan Godber said we can approach ATTC to see if member companies are willing to contribute, but he wasn't sure they would be willing to fund all costs. He said ATTC would like to work with this subcommittee, but questioned what would happen when the actual work is needed. Alan Godber hopes to meet with them soon.

Chairman Godber asked for guidance on how to proceed, noting that at some point we will need to test correlation.

6.1 Further Discussion and Action.

David Fibush said there were many test materials around and we would not be able to use them all. There has to be a way to narrow it down to a relatively small number of scenes. He stated that at some point we need to get to a point where we have approximately six scenes and they can be used for all the tests. He added that the materials have to be available to everyone and suggested it is possible that the industry can get permission to use some of the scenes available if it goes through a committee like SMPTE. At Tektronix, he observed, there is some urgency to this. They expect to have a product to do measurements on compression later this year and he hopes we will propose some traditional scenes (although it is unknown if we can get permission from the source to use them) for testing.

Alan Godber agreed that work has to proceed rapidly, saying that we need to discuss scenes, on paper at least. Work has to go on in parallel on what type of tests we are going to do.

David Fibush suggested sports material as an example - it could include scenes with action moving through the picture and scenes with the picture following the action. He said the Europeans are looking at how long the scene needs to be and noted that we need to restrict what subjective tests should be. We should try to come up with something we can make measurements with. He observed that the Europeans are not done - they keep expanding the ideas of what needs to be done but haven't done any tests yet. In reference to the European studies, he noted that in some cases 4:2:2 video didn't even pass!

Al Morton remarked he saw the same problems with test materials and methods. There were too many. He warned an aggressive beta testing program may not allow enough time to look at methods other people used to see which are effective.

David Fibush answered that if people want an instrument to make measurements, we have to get it out. Alan Godber added that we can always improve it later by developing new tests with more precision.

Leon Stanger recommended forming a task force to make actual selection of test material. He offered that perhaps we should start with HDTV test material - 5 to 10 segments - and call that the standard material. From that, we can derive a 4:2:2 subset, but this will give us a consistent set of tools for HDTV when we are done. Automated tests of this type won't depend on what the materials are in order to develop the K factor. He said that it is not as critical as it might seem, provided we have a good reference and the computer is looking at the material both before and after compression.

David Fibush commented that it depends on what the compression system thinks of the material. Different material stresses the systems in different ways. Leon Stanger answered that it is not going to be a huge problem that needs to involve looking at a lot of scenes.

Alan Godber said the problem is finding what stresses the compression system. He noted that one vendor recommended a "compression buster" - a sports scene. This result could not be predicted by inspecting the image.

Leon Stanger replied that we need input on that, but suggested we define a starting point now. As an example, he observed that MPEG hates water in any form, even snow. He suggested we pick a handful of representative samples and not take this to extremes.

John Grigg added that the test materials should reflect real life. We should use real life scenes so that companies don't adjust their algorithm to match the test material. Manufacturers, if the know what the input will be, can try to do this.

David Fibush commented that test scenes will change as we do more work, but said we need to reject scenes that break the system. For the measurement, we need to use things that people can't get around by tweaking their algorithms.

John Grigg used a football game as an example. Alan Godber suggested a basketball game with camera motion and players moving around with the audience in fine details, adding that is isn't unusual. Lorence Brown suggested scenes shot with a camera on a track following action.

Arthur Webster remarked there needs to be a range of scenes, saying that for digital TV there will be a wide range of bit rates available for different channels. For example, a movie producer may require a higher level of quality than a football game. He said there has to be a test scene that will be difficult enough to code that will measure this by subjective and objective means. There has to be one scene difficult, if not a breaker, to stress the system.

Alan Godber noted that motion pictures can drop to 24 frames per second, which makes it easier to compress them.

David Fibush answered that the result still has to be measured in a high quality manner, which is a function of the measuring equipment's resolution. Alan Godber said this should be able to be measured in different modes. David Fibush suggested picking six or ten types of test scenes and looking at what we can put into those slots. One possible problem, he offered, was that material down-converted from HDTV to SDTV (Standard Definition TV) could be more difficult to compress than material that originated on SDTV. Alan Godber agreed with this.

Leon Stanger suggested a range of materials from junky video to the best of HDTV. He asked if picture quality can be defined with one overall K factor, one that a motion picture producer, for example, could contract for by saying "using these standardized test materials a result of at least a '48' must be delivered." Arthur Webster replied that is what we are working on. Leon Stanger commented that its becoming a continuum type world - dial a bandwidth / dial a resolution.

David Fibush answered that the problem is that while feature extraction methods might be able to be used to compare these various methods, difference methods can't be used. He noted that the only measurements people know how to make is if the format going in is similar to the format coming out. When the resolutions of the two differ, there is a problem. Arthur Webster observed we are a long ways off from this. David Fibush added it was like measuring absolute picture quality. Leon Stanger said we can't cross changing scanning formats, but we should be able to do whatever resolution is chosen within the format.

Rick Redford agreed there was a lot to be said for one common set of scenes, but we have to decide if we are going to draw from what exists or create them. Alan Godber commented it would be expensive to create them and suggested that material could be down converted from existing ATTC HDTV test materials. Rick Redford answered we could do it in different formats. David Fibush agreed, only it must be down converted in a manner that wouldn't stress compression. Alan Godber said we wouldn't want to use some of the ATTC scenes, since they were made with tube cameras. David Fibush agreed that there are some scenes we don't want to include. Alan Godber said there are a number of scenes, particularly graphic scenes, that are of high quality. David Fibush asked who owns them. Alan Godber suggested ATTC, or perhaps the Advisory committee. This needs to be confirmed. While ATTC wants to see the material used, the problem remains who pays for what.

David Fibush agreed this is important. We need to know what is important from a legal standpoint. This is a good place to start. Alan Godber said he will follow up on this and would like some help. David Fibush volunteered. Peter Symes should also be able to help get this going. Alan Godber asked what we should call this effort. Leon Stanger suggested making it an extension of Peter Symes' committee on test materials.

Alan Godber said he was collecting and indexing images. At some point we will need to view them and narrow the selection. The task remains the same - test materials. David Fibush will take over chairing the task force. Peter Symes will remain as a member.

David Fibush suggested the task force go along the lines of locating the material available and the business aspects of using it. He asked what people think about using down converted material. Will people be happy with this? He noted that if the source material is HDTV, its lines can be down-converted, but frame rate could be a problem. One trick might be to ignore the fact that it is running at the wrong speed - run it at the frame rate you want to run it at, 50 or 60. This might be acceptable, although there would be some effect or temporal effects.

Alan Godber announced that David Fibush will chair the task force, John Grigg joins it and Arthur Webster is back in. Other members include Al Morton, Leon Stanger and Alan Godber.

David Fibush asked what formats were available. Alan Godber answered that the early material was in multiple formats, electronically converted. The latest material is down to three formats. [The three are 1080I 60 field component; 720P 60 frames component; 483I 59.94 field composite.]

David Fibush recommended starting with original material in component HDTV format. Arthur Webster said he has the source of the ANSI test scenes if those are of interest. Al Morton added that a lot of Boulder scenes, artsy stuff, are what Leon thought would be tough for MPEG and it might be the compliment of scenes we are considering. Arthur Webster said we don't have rights to the scenery scenes, only the teleconferencing scenes they shot.

David Fibush recommended starting with ATTC, then looking elsewhere when needed. He commented that you can have busy scenes with a lot of errors and humans won't see it. Alan Godber added that with fireworks you couldn't see what was going on. Leon Stanger countered with other examples, such as raindrops on a pond. Humans know what to expect. If MPEG put blocks on the waves, it is obvious. He added that bricks are another problem. The results are visually very significant - you can quickly see the problem. David Fibush said there were some things we will have to live with, such as images with striped shirts on football fields.

Eric Hauch asked if the purpose of the test scenes was for subjective testing of broadcast material. If they are to be used with lower bit rates, we should discuss the applicability of these scenes. Rick Redford suggested that these should include a range of difficulty. Alan Godber said this was a question of scale. If it fails, it is not useful. David Fibush asked if we could drop the resolution and make the images useful. Rick Redford answered that the scenes have to be typical. Eric Hauch said we need to be careful what is judged as typical for teleconferencing. One example was noted. Using one test scene of a slow pan of a kitchen, when a gust of air suddenly moved a curtain some compression systems would cause the curtain to disappear. There were other weird effects as well. With some systems this material would be very stressful.

David Fibush said we need to get a hold of some scenes and see what is applicable. Then we can look for other material. Eric Hauch said the T1A1 work has a huge volume of subjective work tied to their scenes. David Fibush commented that a lot of the scenes were poor to start with, too poor for 20 megabit per second codecs. Ideally we want one scene that will work through the range of applications. Arthur Webster suggested that it would be good to use even one of the T1A1 scenes, if only because of the subjective data available. For example, he noted that one test with a high level of noise actually showed better subjectively. These scenes need to be a component in this application.

Rick Redford asked if there should be one or two electronically generated patterns to go along with these scenes. Patterns that weren't pictures, not even program material. Something like a zone plate with a precise amount of noise. The idea would be to help determine what the problem is.

David Fibush said this would be useful. He commented on the tape from Rohde and Schwarz for testing DCT systems -- while it is useful for designers, there is no correlation between subjective and objective measurements. Rick Redford offered that although this was outside of the purpose of the real scenes, it would be useful as an adjunct for equipment calibration. David Fibush said he was looking at this for equipment calibration.

Alan Godber questioned the use of test material if we aren't sure what it means with relationship to real pictures. The purpose of the standard is to tie the objective to the subjective. David Fibush said these test patterns will be developed and will probably come from manufacturers, but not from this committee. Rick Redford asked why not make them part of the standard. Alan Godber mentioned the problem of correlating the test patterns to subjective quality. Rick Redford replied they would be like color bars at the start of a tape. David Fibush recommended discussing it when there is equipment that can measure it.

Leon Stanger recommended that material should provide a valid score if equipment is used back to back. This should be a computer generated scene, due to the sharper lines available and ability to provide more difficult objects. David Fibush commented that if equipment is calibrated using a natural or computer scene and subjected to a known algorithm everyone should get the same number. He said that in the actual measurement equipment, with filters, there will be synthetic scenes to test equipment.

Arthur Webster cautioned that for objective measurements some of these objective scenes may require subjective measurements.

Alan Godber said objective tests of digital systems are difficult to interpret. The old method of objective tests doesn't work. New methods are needed and people need to trust them.

David Fibush observed that he doesn't see artifacts on Digital TV, specifically DirecTV. Alan Godber answered that he has noticed it on PBS, like noise. He added that people don't seem to care about it. He also saw periods where 10 to 15 seconds of video were lost.

Alan Godber asked for other ideas and reminded the subcommittee about his chart of images and asked if they should be put on a database. David Fibush replied that we should now be in the "lets make a small list" phase.

The subcommittee took a break from 10:00 AM to 10:25 AM

Item 7 - Report of Task Force on preparation of Scope for Committee Work - Chair, Leon Stanger.

Draft 4, Detailed Scope of Activities, was distributed and given document number "22". Leon Stanger said that references to audio in Draft 4 should be removed, per the previous decision.

Alan Godber asked for a Task Force update on changes.

Leon Stanger said he had collected comments from committee members. He had submitted the draft to committee members and had received one comment back.

7.1 Further Discussion and Action.

The committee agreed to review the document a paragraph at a time.

Purpose:

David Fibush said he understands why the work is needed, but is not sure it is clear to all. We need to say why the current practices don't work and why the indirect method isn't appropriate for compressed systems.

Alan Godber asked for suggestions. David Fibush said the word "indirect" is good.

After a short discussion, Alan Godber suggested it would be easier to expedite the work if we don't worry about the exact wording at this meeting, but rather allow Leon Stanger to work on it. Leon Stanger said he would add a sentence to cover it.

Goal:

Arthur Webster suggested changing document to documents. The subcommittee agreed.

Leon Stanger said we may need to add that the subcommittee will define the test segments.

Activities and Scope of Work:

Item 1: Arthur Webster said NTIA should be changed to NTIA/ITS. David Fibush asked if ANSI should be used in place of ATIS. Anthony Schiano agreed we could say ANSI. Richard Cochran thought ANSI was too broad. Eric Hauch said is was appropriate to say committee T1. He said ATIS is the most recent incarnation of ECSA, made up of exchange carriers and government organizations. It is a sponsoring or secretarial organization, not an accredited standards organization.

Item 2: David Fibush suggested changing the reference from ATIS to ANSI T1.

There was a discussion about ANSI standards. Alan Godber said IEEE standards can be given ANSI numbers as well. Rick Redford said ANSI takes standards of other organizations and adopts them but doesn't create standards of its own. Alan Godber replied that this may not be the case now. Anthony Schiano said standards adopted by ANSI have to meet a set of ANSI legal standards. Eric Hauch said that committee T1 produces documents which are published but which are not ANSI documents. Whenever a draft standard is produced, it is voted on by T1 using procedures based on an ANSI standards book. T1 produces standards for recommendation to ANSI.

David Fibush repeated his suggestion to change the reference to ANSI T1. All agreed.

Item 3: David Fibush mentioned that SMPTE works with 625 as well as 525. He asked if we should include 625 as well. Leon Stanger said this was assumed. Alan Godber said people may make the wrong assumption and asked for agreement on specifically including 625.

Rick Redford cautioned that we don't want to create conflicting standards with EBU work on 625, noting this was a consideration for G-2.1.4. David Fibush said that since 525 and 625 have no differences, they should be considered the same. Alan Godber warned we needed close liaison with EBU. David Fibush responded that EBU doesn't seem to be writing any documents on serial digital, noting they've backed off a lot of things. He added that often if EBU didn't want to do the work, SMPTE would and EBU would agree to it. As long as the signal is component, there isn't a problem.

Leon Stanger agreed to add wording to "standard definition" to include 625.

Rick Redford questioned the use of "standard definition", saying he has used "conventional TV". David Fibush said "standard definition" is what is commonly used now. Alan Godber agreed that "standard definition" is the accepted term.

Item 4: David Fibush remarked that while component is preferred, we need to say something about composite. Set top boxes, for example, will be composite. The measurement system must take into account measurements through composite systems.

Leon Stanger said this was discussed. Measurements will be digital to digital and done in component. David Fibush replied that sometimes we will have to measure composite. A decoder will be required and it must be a certain quality so we aren't measuring the decoder. Leon Stanger suggested adding the words "if component systems are to be measured, we will measure digital to digital" and said that for the sake of this committee, measurements will be done component to component.

David Fibush commented that T1A1 work was done composite in and out. Arthur Webster agreed, noting all of it was done on D-2. David Fibush emphasized that if someone is using a composite output with a component input they will see the composite decoder in the results. It is better to have a composite reference too, so we need to include some wording for composite. Arthur Webster added that there is a lot of composite equipment out there.

Leon Stanger agreed to add a sentence stating composite as well as component test methods will be investigated.

Item 5: No comments.

Item 6: No comments.

Item 7: David Fibush noticed that we left out the T1A1 group. Leon Stanger asked for comments on what words to use. John Grigg said that "common carrier" was traditionally used. The group agreed that "common carriers" would be added to the list of intended users.

Activities Beyond The Scope of This Subcommittee:

Item 1: Alan Godber said this seems to exclude all that we are doing! Leon Stanger said that it means that we won't write performance specifications.

Item 2: Leon Stanger said he would exclude audio.

Baseline Assumptions:

This replaces "Red Herrings" in the previous draft.

Item 1: Arthur Webster questioned the meaning of this, noting that signal to noise ratio is important. Leon Stanger agreed that on source material it can have a big effect on how a compression system compresses it and noise can make a good picture bad. However, by using predefined source material, noise is no longer a variable in its own right. Alan Godber said this depends on whether we are talking about verification of a test method versus application of a test method and asked if we will get usable results with a noisy system.

Leon Stanger said the difference is in-service versus out-of-service testing. Out-of-service we can set a predefined test sequence run through a computer. It will be straightforward, predictable and repeatable. In-service testing may have source material with variations in noise and complexity. While in-service testing could be valuable, it is beyond the scope of what we want to define here. Our goal is to define a standardized and repeatable method, implying out-of-service testing.

David Fibush agreed that is one aspect, but what could have happened elsewhere is an issue. In terms of equipment we are designing, we will provide some traditional measurements as well. There is a traditional measurement part to be done, but this group should not put that into its measurements.

Arthur Webster said the sentence as written sounds strange. David Fibush recommended crossing out the first sentence. Leon Stanger questioned if that was enough - do we want to entertain a wider series of testing to include in-service testing. David Fibush said we do want to measure what happened to the signal through the compression system. Leon Stanger said it was difficult to get quantifiable results through the industry. Perhaps it could be done as a lead in to a movie. Arthur Webster agreed, but if we say input signal characteristics are not part of the measurement, what happens if there are multiple dubs? Leon Stanger agreed this was important, but asked what was the next step.

David Fibush suggested that we say that for predefined inputs, here are the measurements, here is the correlation. Rick Redford offered "Conventional defects in the analog representation of the input signal will not be part of the measurement criteria." David Fibush asked analog or digital? Leon Stanger said the doable part of our committee work is to run the test materials through a system. This doesn't say that in-service measurements aren't important.

Richard Cochran offered that the assumption is we are going to predefine the test signal material. David Fibush suggested putting the last sentence first. Cochran agreed this puts them in the order we are dealing with. David Fibush agreed and added that it also takes care of the digital part.

Item 2: Alan Godber said this item seems to contradict the first item. Leon Stanger pointed that what we are saying is that here is a way to test the system end-to-end, including multiple compression systems. The system can be defined as end-to-end or by individual items. As a committee, we won't exclude or include concatenation.

David Fibush said what we want is a measurement that gives us a number of each part of the system that when added give a system performance number. Rick Redford said this would be nice, but it isn't feasible. Alan Godber said this is the real world. David Fibush said the issue wasn't will the numbers add, but how will they add. Rick Redford said this is done in analog measurements, but the test material used to test the second system is pristine.

Leon Stanger said there were a lot of "red herrings". David Fibush agreed, saying this may not work. We are not going to try to insure that by this method that we can concatenate measurements. Rick Redford warned we don't want to get bogged down in this. Leon Stanger said we can apply this tool in a general way, but don't ask us to explain concatenation. Eric Hauch pointed out the in the analog world we are talking about laws of impairment summation.

Arthur Webster suggested eliminating the first sentence. David Fibush agreed, saying then it would work. Alan Godber wasn't sure, saying we can't bury our heads in the sand - concatenation is the real world. Al Morton said the words "end to end" would help us. Rick Redford suggested the wording "Any concatenation of the same or different compression systems will be addressed as the end to end system under test." Leon Stanger stated we will not address impairment summation. Arthur Webster said we may address this in the future. Alan Godber said it happens now. David Fibush said that while we are going to want to do it, we first need to get something that works end to end.

Alan Godber worried that if we limit it to one pass through an algorithm we will get calls from broadcasters. Leon Stanger noted we are saying end to end is a valid measurement. Anthony Schiano offered that off air becomes multiple end to end cases once the signal goes to analog. David Fibush pointed out that what we are saying is that if we make individual measurements, they won't add. Eric Hauch said we should say this implicitly. David Fibush said that's why we are using the 4:2:2 profile, main level. The station may not decompress and re-compress the signal. Alan Godber was concerned we were making an assumption about how the studio is going to operate. David Fibush remarked that with regards to news, almost anything is acceptable.

Leon Stanger observed that given the large number of possibilities of compression and decompression, the limits explode. Alan Godber said we provide another tool. Leon Stanger answered we provide a tool to make any individual end-to-end measurement. Joe Zebarth recommended using these words: "The process for synthesizing concatenated system performance from individual system test results will not be addressed." David Fibush added "at this time". Leon Stanger agreed to add it under "Activities Beyond the Scope of The Subcommittee". Alan Godber agreed.

Item 4: The first sentence in item 4 was deleted. Al Morton suggested adding the second sentence of item 4 to item 1. All agreed. Item 4 disappeared.

Item 5: Leon Stanger felt item 5 is too broad. Rick Redford asked if any we were going to define any process to calibrate the test method. David Fibush questioned how we are going to put a number down. Arthur Webster said we don't have to define it now. Alan Godber said the subjective must agree with the objective to a degree acceptable to the committee. When its done we have a standard.

Anthony Schiano observed that it is hard to get subjective correlation. David Fibush said that we hoped to develop a standard with correlation good enough for comparison of systems. Alan Godber suggested using another committee's work as an example. David Fibush noted there is a fundamental limitation, even when comparing subjective tests to subjective tests. Alan Godber noted that with High Definition, they found you can learn and you can get a group that gives repeatable results. If equipment is calibrated, he said, you can get close results. Al Morton concurred, noting that the experience of T1A1, in three different laboratories, was a happy one. Correlation between labs was high.

Alan Godber said it was not easy and careful set up and documentation was necessary. Problems occurred when setups were not documented. David Fibush said this will be even more of an issue with objective measurements when items like positioning, gain, etc. are considered, even more so than with subjective testing.

Eric Hauch said "test method" had to be defined further and recommended use the words "subjective viewing results" instead of "subjective viewing standards". Leon Stanger agreed, saying the goal is to define results. David Fibush said we must correlate it with the subjective. Eric Hauch said there are subjective results in the public domain that could be used. David Fibush added "if appropriate for what we are trying to do." Eric Hauch mentioned other sources, such as T1Y1. David Fibush said that is why we are looking at the higher bit rate ATTC and GTE results. He asked if ATTC has output recordings. Alan Godber replied "some". Alan Godber said the main question is how well it is documented.

This concluded the detailed discussion of the scope document.

Leon Stanger asked if it captured the essence of what we set out to do.

David Fibush suggested putting the statement about correlation into the goal.

There were no further comments. Alan Godber asked Leon Stanger to go back, word smith the document and present another document by e-mail before the next meeting if possible.

Item 8 - Report of Task Force on Compression Measurement Methodologies - Temporary Chair, Alan Godber.

Alan Godber said the task force was a long way from anything definitive. There are five people on the task force. Neither a list sent to members or a second e-mail elicited any response.

Alan Godber distributed copies of a skeleton "IEEE Standard for the Measurement of Television Video Signals in the 525 Line Interlaced Format which Have Been Compressed". It was assigned document number "23". Godber described the document. The References section refers to standards already approved. The Standard Conditions section outlines preconditions prior to doing the measurement. He noted that while the question is sometimes asked if this is needed, he preferred to err on the side of including it. He asked if the Research and Development section was useful or needed. The "Validity" portion of each method is important. The outline is based on analog measurements and may or may not be applicable.

8.1 Further Discussion and Action.

David Fibush said work on this document is important, but it is like trying to develop a product without a technology. He indicated it would be more appropriate perhaps three or four meetings down the road. Alan Godber agreed that we don't need to proceed very far with it at this moment, but he wanted to know what we do now. David Fibush suggested that the NTIA GTE document distributed earlier is one of the best .The comparison of methods, tests and the technology is what we need to get into now.

Alan Godber recommended that everyone in the committee read T1A1.5/96-121, G-2.1.6 Document 21.

Item 9 - Discussion of Compression Measurement Methodologies, including compression work by ATIS.

David Fibush asked about Bill Zou's summary of key documents on measurement methodologies. Alan Godber said it was going to address the key issues in the documents.

David Fibush said he was working on an item for Tektronix. He offered that while it may not be suitable for this committee and may have a point of view not agreeable to the committee, it would be a start. Alan Godber agreed it would be valuable and that it and Bill Zou's input would be a good start on a standard. Godber asked if there was any other input.

David Fibush offered that the T1 document (T1A1.5/96-121, G-2.1.6 Document 21) has a lot of methods and is an important addition to this group. Leon Stanger suggested a working group to dig into existing documents and recommend methods, to outline, for example, specific impairments and the tools and methods to find them. Arthur Webster said there will be contributions coming into ITU-T from Japan and elsewhere. He offered to bring them to the meeting in May. Al Morton noted that ITU-T Working Group 2 was meeting April 14-17. Arthur Webster said the study group from Japan is coming in a week later and that ITU-R2 is also meeting at the same time. Al Morton and Arthur Webster will be attending study group 12. David Fibush reminded everyone that the important thing about going to these meetings is to get the documents.

Alan Godber asked about inputs from David Fibush. He replied that he might have something by NAB and that it should still be useful even with the Tektronix slant.

The timing of submissions by Bill Zou was less certain, according to Alan Godber. If they come in soon enough a task force could examine them. He asked for other suggestions for the task force and for a volunteer to chair a task force on compression measurement. No one offered. David Fibush said the important thing is to gather together what has been done to get a technical feel for the possibilities. Alan Godber said Bill Zou would be the ideal person, since he's read all the documents.

David Fibush saw the effort more like a literature review. We need a good, concise review of what are the possibilities. Al Morton said many will either look at the methods and be champions of a method or will propose improvements. It is a natural process. You need a champion for each method. That is one way to proceed with it. It will happen anyway. Alan Godber said he wanted to make more progress before the next meeting. With good information coming in from ITU, David Fibush and Bill Zou before the next meeting there won't be much time to analyze it.

David Fibush suggested that we might need some more in depth discussions and half a day may not be enough. Alan Godber said this had to be worked out with G-2.1.4. David Fibush commented that a lot of work in G-2.1.4 may interest T1. Eric Hauch said there was consideration on adding digital to 502, but it was abandoned. Alan Godber suggested we consider a day and half of meetings. David Fibush wanted to see how it fit in with T1A1.5. Eric Hauch said the schedule was driven by more than broadcasting. He couldn't guess how long it will be until after the current meeting cycle is completed.

Alan Godber offered the possibility of having a smaller group get together to discuss this input before the meeting. David Fibush said T1 can't be pushed later in the week. Alan Godber suggested a Sunday afternoon meeting. David Fibush said that if there is enough input for the Task Force, he was agreeable to extra meetings. Alan Godber said they will definitely have David Fibush's input and might have some from Bill Zou. Eric Hauch said he will have the Group 5 agenda by Thursday.

David Fibush said that there are other fundamental documents created prior to the ANSI standard that should be studied. He offered the IQR study as an example. In terms of discussing the ways these things can get done, there are two or three internal ANSI documents that may be useful. Arthur Webster will talk to Steve Wolf and try to get some of these together. David Fibush added that the existing standard itself has a lot of the theory built into it. Arthur Webster said ITS is working on a web page containing all these documents.

Alan Godber asked if people would show up if we have a Sunday afternoon meeting.

David Fibush, Bill Zou, Arthur Webster and Leon Stanger were suggested as attendees, with R.J. Green and Bill Meeker possibly attending. Alan Godber also wanted to attend.

Al Morton said Arthur Webster was the best person to bring in the ITU documents. Arthur Webster said he will be there or will ask Steve Wolf to attend.

Alan Godber said that, based on an IEEE audit, the subcommittee requires a Vice Chairman position. He wants to talk to Bill Zou about doing this. Arthur Webster indicated that Steve Wolf might be interested. Alan Godber said the work includes general coordination work. Arthur Webster said he doesn't have the time. Alan Godber said he will try to press Bill Zou for the Chair or Vice Chair position, nothing that he has the support of his organization.

Alan Godber asked for other suggestions. David Fibush recommended reading the Sarnoff document (G-2.1.6 Document 07). Alan Godber said that makes it about six documents to read. David Fibush replied "maybe more."

Item 10 - Discussion of Future Work, Additional Assignments, etc.

This topic was covered in other items.

Item 11 - Any Other Business.

Al Morton distributed a contribution to the T1 Standards Project on Multimedia Communications Delay, Synchronization and Frame Rate Measurement. The Draft document is ready for voting. It might be useful for other G-2.1 groups to look at this.

Rick Redford suggested this belonged in Randy's group. Alan Godber said that hadn't been decided. Alan Godber offered to document it here and put it on file. It was given document number "24".

Al Morton commented it was more important to some systems than others. Eric Hauch outlined the Letter Ballot Process. Unless there are surprises, this document will go forward to ANSI as a standard.

Item 12 - Date(s) of Future Meeting(s).

A joint meeting with T1A1 in Boulder Colorado on May 5^th, 1997, plus either May 4^th or May 6^th was suggested. Eric Hauch thought it would generally work. Anthony Schiano commented that there is no schedule T1A1.15 or T1A1.12 meeting until next year. Eric Hauch thought they might even be able to squeeze the half day out of the T1A1 schedule. Alan Godber said this needs to be known soon and added that the beginning of the session would seem to work. Alan Godber asked if the extra meeting could be at NTIA if it was on Sunday. Arthur Webster said that could be possible, but he will have to check.

The subcommittee thanked AT&T and ECI Telecom for providing meeting facilities and refreshments.

A motion was made to adjourn the meeting. It was approved and the meeting adjourned at 12:30 PM.

Submitted by:

H. Douglas Lung

Secretary

Draft Agenda - IEEE Compression and Processing Subcommittee G-2.1.6 Third Meeting Monday January 20th, 1997 , Alan Godber, Chairman, 20 January 1997.

Draft Meeting Record, G-2.1.6, Compression and Processing Subcommittee, Meeting #2, September 30, 1996, NIST, Gaithersburg, Maryland, Doc. G-2.1.6/20, 20 January 1997.

TIA1.5/96-121, Committee T1 Performance Standards Contribution: Objective and Subjective Measures of MPEG Video Quality, October 28,1996, Doc.G-2.1.6/21, 20 January1997.

DRAFT 4, Detailed Scope of Activities, Leon Stanger, IEEE Compression and Processing Subcommittee G-2.1.6 Task Force on Preparation of Scope for Subcommittee Work, Leon Stanger, Doc. G-2.1.6/22, 20 January 1997.

Document Framework: IEEE Standard for the Measurement of Television Video Signals in the 525 Line Interlaced Format Which Have Been Compressed, January 19 1997, Alan Godber, G-2.1.6/23, 20 January 1997.

T1A1.5/96-101, Draft ANSI T1 Standard on Multimedia Communications Delay, Synchronization, and Frame Rate, Technical Subcommittee T1A1, G-2.1.6/24, 20 January 1997.