Objective Video Quality Measurement

Overview Objective quality measurement is the use of algorithmic procedures that compute the quality of a video signal. There are a number of different methods that have been developed to measure video quality objectively. These methods can be classified into three categoires: picture difference methods; parametric methds; and perceptual methods. Traditional objective video quality metrics such as peak signal-to-noise ratio (PSNR), provide a definitive measure of the difference between a source signal and its processed counterpart. PSNR and other picture difference methods (such as MSE) indicate image fidelity - how closely the processed signal resembles the original source signal. This reliance on access to both source and processed signals is referred to as a full reference method. No reference methods have been developed that estimate PSNR (ITU-T Rec. J.240). In no reference methods, quality is measured by analysing the processed signal alone. Parametric measurement methods are used to indicate the quality of a transmitted signal. Parametric methods typically use a model of how network performance impacts on a media signal to predict quality (ITU-T G.107, ITU-T G.1070, IETF RFC 4445).

For picture difference measures, an absolute fidelity (often incorrectly termed as a quality) measure is obtained. Parametric methods are designed for transmission planning or network performance monitoring tasks where an accurate prediction of subjective quality is not essential. Parametric models typically provide a transmission rating or quality index value. Although simple to compute, picture difference and parametric methods have proved to be indequate indicators of subjective quality and this predictive limitation has led to the development of perceptually relevant quality metrics. Objective perceptual quality measurement methods attempt to predict the quality of a signal as perceived by end users. Perceptual quality metrics have been developed and standardized for voice (ITU-T Rec. P.563, ITU-T Rec. P.862), audio (ITU-R Rec. BS.1387) and video (ITU-T Rec. J.144, ITU-R Rec. BT.1683, ITU-T Rec. J.246, ITU-T Rec. J.247). Perceptual models evaluate the subjective quality of a service and output a predicted mean opinion score (MOSp).

The ability to objectively measure the quality of video services has significant commercial benefits (e.g. test and measurement, quality profiling, quality assurance, QoS^? monitoring, fault detection, setting/policing Service Level and Vendor Agreements). A number of video quality measurement tools are being developed in the Perceptual Engineering Group. These tools model human perception to provide accurate predictions of video quality. The power of these tools is that the quality predictions are representative of human quality opinions.

The group is actively involved in various standards activities, most notably the Video Quality Experts Group, ITU-T SG9 and ITU-R WP6C.

The following models have been developed or are under development:

Full-reference model (FR) The FR model has complete access to both the original or reference video as well as the processed video. Note that the content of the reference and processed video is identical, but the quality is likely to be different. The FR model is a general purpose objective quality measurement method. A fully tested version of this model, capable of measuring the perceived quality of broadcast, internet and mobile video material has been developed.

BT's FR model is part of two ITU standards

Our FR was awarded a technology medal by the British Computer Society in 2005 and was short-listed by the IEE Innovation awards and BT Innovation Awards.

Reduced-reference model (RR) The RR model is a variant of the FR model. The reduced-reference video model operates in a similar fashion to the full-Reference model but does not have full access to the reference video itself. Instead, the RR model has access to limited information obtained from the reference. This information is extracted from the unprocessed video at source and this reference video data is transmitted alongside the processed video signal. Using specific but limited information extracted from the original video sequence enables the reduced reference model to provide reliable quality predictions. The predictive performance of RR models is inferior to FR methods, but RR methods tend to be less computationally demanding than FR models.

No-reference model (NR) The NR model operates on the processed signal alone and has no information at all from the reference.

Bit-stream based no-reference model (NR-BS) The bit-stream version of the NR model extracts information directly from the video bit-stream. The bit-stream may be considered the purest level of video representation. The bitstream model has access to information that is lost once the picture has been fully decoded.

Hybrid no reference model(NR-H) A mix of parameter information extracted from both the bitstream and the decoded picture has many advantages of other forms of NR method. Most importantly, the hybrid model has quality-critical information available from the bitstream, but the decoded picture provides supplementary information (e.g. image contrast) that improves the predictive performance compared to alternative NR approaches. Perhaps most important of all, it is by examining the decoded pixel-domain information that effects of error concealment can be captured. Accommodating the perceptual influence of concealment is essential for remote quality monitoring tasks. We have developed a Quality Assurance tool based on our hybrid model that is used operationally by BT.