Discussion Document to Expand on the ideas in the IEEE G-2.1.6 Scope

The purpose of this document is to provide a more detailed input to the ITU-T joint rapporteurs group (JRG) representing the broadcasters' view on the development of an objective picture quality assessment method. All participants in the IEEE G-2.1.6 are invited to comment and recommend changes. "Picture" is used as a shorthand for "motion video picture sequence".

General application areas

Operations

1. Complete post production, many edits and processing layers, intra-plant transmission. Also used for remote site to plant transmission (backhaul).
   
2. Simple modifications, few edits, character/logo overlays, inter-facility transmission. An example would be network to affiliate transmission.
   
3. No changes, delivery to the home/consumer.
   

Notes:

1. These do correspond with the SG9 classes 1 through 3; contribution, primary and secondary distribution, however we don't want to cause term definition confusion.
   
2. It is realized that home delivery video could go to a studio for post production, however measurements relating to intermingling of these applications are not being considered.
   

Equipment

Design, product selection/acceptance, installation, maintenance, operational monitoring.

Classes

Arbitrary class divisions should not be made until investigation of the picture assessment technologies and their usefulness for various applications makes clear what classifications must be defined. We would like to maintain one underlying technology if possible. Use of completely different technologies for a few percent gain in effectiveness in a different class of service would not be good for the relatively small television test equipment industry. Once the assessment technology is determined, separate assessment methods for applying that technology to different classes may be appropriate. Performance requirements for a more extensive classifications of services can be specified by the appropriate organizations. Having said that, there are some class divisions which may turn out to be necessary:

High compression, high artifact, drop/repeat frame quality verses broadcast quality.

Lossless verses lossy compression. (Lossless has been called Class 0. It is not expected to require a picture quality assessment method as traditional measurement methods apply.)

Assessment method validation

We are going to investigate various assessment technologies and select one or more assessment methods. (Not necessarily winner-take-all, a combination of proposals may be developed if required.) In the process of validating the proposals there are a number constraints planned and a number of potential parameters not included. The ITU-T SG12 and SG9 report (T1A1.5/97-108) contains a list of assumptions:

• Video is aligned is space and time
• Laboratory measurement: both source and degraded images are available
• A single figure of merit is a goal
• Both dropped-frames and non-dropped-frames test scenes will be used
• Temporal distortions due to frame rate variations will not be considered

Other limitations that would be appropriate are:

• Rec 601/656 I/O
• No analog paths in the system
• No change through the system or pre-corrected for the validation tests
• Picture cropping
• Chroma and luma gain
• Black level
• Chroma/luma differential timing
• Sync/burst jitter (picture jitter)
  

Note: It is known that some compression systems may vary these parameters based on scene content. There may not be a valid reason for such operation, that is, it may be bad design. For the purpose of the validation tests, such systems will not be included due to the extensive resources that would be required to correct these defects.

An aspect that should be included is testing with picture defects due to transmission errors occurring at a rate appropriate for the operational application.

System considerations

For today's practical applications and for future applications, there are a number of system considerations which are not to be directly included in selection of a basic assessment technology. To the extent appropriate, investigation of the application of proposed technologies to these types of systems will be undertaken.

• Composite encoded (NTSC/PAL) signals primarily used in the analog domain
• HDTV in one or more ATSC scanning format
• Progressive scan as opposed to interlace
• Concatenated codecs of the same or different compression method
  

There are a number of parameters which can be measured by means other than direct picture quality assessment. These are important for a real measurement instrument and are listed here to make sure they are not forgotten in development of the recommendation. To some extent they do not affect picture quality, however they are expected to be reported by a measurement system to allow user adjustment of the system or automatic correction prior to applying the measurement algorithm. Gross misadjustment of these parameters, particularly if they can't be corrected, may be considered system quality degradation at the discretion of the user.

• Spatial alignment *
• Picture cropping *
• Chroma and luma gain *
• Black level *
• Chroma/luma differential timing *
• Sync/burst jitter (picture jitter) *
• Bandwidth/rise-time
• Transient response
• Other traditional test signal characteristics

It is known that some compression systems may vary these parameters based on scene content. There may not be a valid reason for such operation, that is, it may be bad design. In that case traditional television measurement methods will not do the job. If such degradations are expected, the measurement system may need to include a picture related method to measure those parameters indicated by an * either as an operational process or to help locate the design implementation creating the problem.