- 28 -
FG IPTV– DOC– 0086
INTERNATIONAL TELECOMMUNICATION UNION / Focus Group On IPTVTELECOMMUNICATION
STANDARDIZATION SECTOR
STUDY PERIOD 2005-2008 / FG IPTV-DOC-0086
English only
WG(s): 2 / 4th FG IPTV meeting:
Bled, Slovenia, 7-11 May 2007
OUTPUT DOCUMENT
Source: / Acting Editor
Title: / Working document: Quality of Experience Requirements for IPTV
Summary
TBD
Keywords
TBD
Introduction
TBD
Table of Contents
1 Scope 4
2 References 4
3 Definitions 4
4 Abbreviations and acronyms 4
5 Conventions 5
6 Introduction to QoE 5
7 QoE for video and audio 6
7.1 Requirements for media compression and synchronization 6
7.1.1 Standard definition (SD) TV: General minimum objectives 9
7.1.2 Standard definition (SD) TV: VoD and Premium Content Objectives 11
7.1.3 High definition (HD) TV: Objectives 12
7.2 Requirements for network transmission 14
7.2.1 Standard Definition Video: Broadcast TV Transport Layer Performance Objectives 17
7.2.2 Standard Definition Video: VoD and Premium Content Transport Layer Performance Objectives 18
7.2.3 High Definition TV: Transport Layer Performance Objectives 18
8 QoE for text and graphics 22
9 QoE for control functions 22
9.1 QoE requirements for channel zapping time 22
9.1.1 Classification of channel zapping time 23
9.1.2 Requirements for channel zapping time 23
9.2 QoE requirements VOD trick mode 23
10 QoE for other IPTV services 23
10.1 QoE requirements for EPG 23
10.2 QoE requirements for Metadata 23
10.3 QoE requirements for Browser 24
11 QoE requirements with service billing 25
11.1 Service billing and its impact on end-users’ expectation for a service quality 25
11.2 Proposed QoE requirements with service billing for IPTV service: End-users’ Utility 25
12 Service support 25
13 Accessibility requirements 25
Appendix I Network QoS parameters affecting QoE 26
I.1 Transport Impairments 26
I.1.1 IP Packet Transfer Delay 26
I.1.2 IP Packet Loss Ratio (PLR) 26
Appendix II Bibliography 28
1 Scope
This document defines user requirements (Quality of Experience, or QoE) for IPTV services. The QoE requirements are defined from an end user perspective and are agnostic to network deployment architectures and transport protocols. The QoE requirements are specified as end-to-end and information is provided on how they influence network transport and application layer factors.
2 References
The following ITU-T working text and other references contain provisions, which, through reference in this text, constitute provisions of this working text. At the time of publication, the editions indicated were valid. All Recommendations and other references are subject to revision; users of this working text are therefore encouraged to investigate the possibility of applying the most recent edition of the Recommendations and other references listed below. A list of the currently valid ITU-T Recommendations is regularly published.
The reference to a document within this working text does not give it, as a stand-alone document, the status of a Recommendation
[ITU-R BT.500] ITU-R Recommendation ITU-R BT.500-11, Methodology for the subjective assessment of the quality of television pictures. (Question ITU-R 211/11)
[ITU-R BT.601-5]
[ITU-T E.800] ITU-T Recommendation E.800 (1994), “Terms and Definitions Related to Quality of Service and Network Performance Including Dependability”
[ITU-T Y.1541] ITU-T Recommendation Y.1541 (2006), “Network Performance Objectives for IP-based Services”
[IETF RFC 3393] IETF RFC 3393 (2002), “IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)”
[IETF RFC 3357] IETF RFC 3357 (2002), “One-Way Loss Pattern Sample Metric”
[DSL TR-126] DSL Forum TR-126 (2006), “Triple-play Services Quality of Experience (QoE) Requirements”
[ITU-T P.10/G.100] ITU-T Recommendation P.10/G.100 Appendix I, “Definition of Quality of Experience (QoE),” Jan. 2007.[MPEG x]
[ITU-T H.264]
[VC-1]
[ISO/IEC 14496-3]
3 Definitions
TBD
4 Abbreviations and acronyms
This working document uses the following abbreviations and acronyms:
FEC Forward Error Correction
MOS Mean Opinion Score
MP Measured Point
PDV Packet Delay Variation
PHB Per-Hop Behaviour
PLR Packet Loss Ratio
PTD Packet Transfer Delay
QoE Quality of Experience
QoS Quality of Service
STB Set-Top Box
VoD Video on Demand
5 Conventions
TBD
6 Introduction to QoE
QoE is defined in [ITU-T P.10/G.100] as the overall acceptability of an application or service, as perceived subjectively by the end-user. It includes the complete end-to-end system effects (client, terminal, network, services infrastructure, etc) and may be influenced by user expectations and context. Hence the QoE is measured subjectively by the end user and may differ from one user to the other. However it is often estimated using objective measurements.
Contributing to the QoE are objective service performance measures such as information loss and delay. Those objective measures together with human components that may include emotions, linguistic background, attitude, motivation, etc [1] determine the overall acceptability of the service by the end user. Figure 1 shows factors contributing to QoE. These factors are organised as those related to quality of service and those that can be classified as human components.
QoE is often measured via carefully controlled subjective tests [BT.500] where video samples are played to viewers, who are asked to rate them on a scale. The rating assigned to each case are averaged together to yield the mean opinion score (MOS).
Quality of service (QoS) is defined in [ITU-T E.800] as the collective effect of performance which determines the degree of satisfaction of a user of the service. In telecom QoS is usually a measure of performance of the network itself. QoS mechanisms include any mechanism that contributes to improvement of the overall performance of the system and hence to improving end user experience. QoS mechanisms can be implemented at different levels. For example at the network level it includes traffic management mechanisms such as buffering and scheduling employed to differentiate between traffics belong to different applications. Other QoS mechanisms at levels other than the transport include loss concealment, application forward error correction (FEC), etc.
Related to QoS are the QoS performance parameters. Similar to the QoS mechanisms QoS parameters can be defined for different layers. At the network layer those parameters usually include information loss rate and information delay and delay variations.
Figure 6-1-QoE Dimensions
The following text is taken from [DSL TR-126]:
In general there is correlation between the subjective QoE as measured by the MOS and various objective parameters of service performance (e.g. encoding bit rate, packet loss, delay, availability, etc.).
Typically there will be multiple service level performance (QoS) metrics that impact overall QoE. The relation between QoE and service performance (QoS) metrics is typically derived empirically. Having identified the QoE/QoS relationship, it can be used in two ways:
a. Given a QoS measurement, one could predict the expected QoE for a user
b. Given a target QoE for a user, one could deduce the net required service layer performance.
To ensure that the appropriate service quality is delivered, QoE targets should be established for each service and be included early on in system design and engineering processes where they are translated into objective service level performance metrics. Quality of experience will be an important factor in the marketplace success of triple-play services and is expected to be a key differentiator with respect to competing service offerings. Subscribers to network services do not care how service quality is achieved. What matters to them is how well a service meets their expectations for effectiveness operability, availability, and ease of use.
7 QoE for video and audio
7.1 Requirements for media compression and synchronization
One of the main components of QoE for video and audio is digitization and compression of video and audio source materials and the various settings and parameters selected. Since video compression schemes such as MPEG are lossy and an identical copy of the original cannot be recovered, there are potentially negative impacts on video picture quality and therefore on viewer QoE. The main factors influencing video QoE at the application layer due to compression are:
• Quality of source material
– “garbage in = garbage out”
• The baseline quality (no network impairments) of the codec standard used
– there are a range of video codecs available, but typically television applications will use one of the following: MPEG-2, MPEG-4 AVC (also known as MPEG-4 Part 10 or H.264) and SMPTE VC-1 (previously known as VC-9, the standardized version of Windows Media™ 9)
• Resolution
– Some systems reduce the horizontal resolution to achieve the target bit rates for example in SD the resolution maybe reduced to ‘Half’ or “Three Quarters” which produces a less sharp picture than ‘Full’ resolution
• Bit rate
– During periods of high complexity (entropy) compression may leave visible artifacts if the bit rate is not sufficient
• Application layer video encoding - Constant bit rate (CBR) vs. Variable Bit Rate (VBR) at the encoder output
– Video encoding is naturally variable bit rate but to simplify network engineering for Telco delivery systems, the video encoders are set to provide a constant bit rate (as averaged over some specified time period on the order of seconds).
– VBR streams such as those used in DVD encoding have constant quality since the bit rate is allowed to vary to accommodate varying complexity of the source material
– CBR streams have variable quality since there may be times when the bit rate is insufficient to accommodate the video complexity but CBR steams enable more straightforward traffic engineering and system design
• Encoder quality and settings
– Group of Pictures (GOP) structure]
• Shorter GOPs improve quality but reduce the improvement to bit rate from compression.
• Longer GOPs improve maximum compression ratio, but increase channel change time and the amount of damage a lost packet will cause.
• Dynamic GOPs can be used to better handle scene changes and other effects but are not always implemented on STBs. In addition, dynamic GOPs can impact the variability of zapping latency and may complicate mechanisms to increase zapping speed considerably.
– Motion Vector Search Range
• Wider searches provide improved quality but at increased complexity and encoder delay
• Large search ranges are required for high motion content such as sports
– Rate Control
• Mode decisions greatly affect the bit rate
• Proprietary schemes are commonly used to gain competitive advantage
• Preprocessing (such as noise reduction)
– usually proprietary and non-standard but can improve bit rate / quality tradeoff
• Tandem encoding and rate shaping (e.g. digital turnaround)
Video Compression Artifact Examples
Figure 7-１ illustrates several kinds of compression artifacts that are largely due to insufficient bits allocated resulting in too coarse quantization of DCT coefficients or motion vectors and/or otherwise poor motion estimation. Addition details of compression artifacts may be found in Wolf (1990) [].
Figure 7-1-Compression Artifacts [ITS Video Quality Research (2003)]
Similarly, there are multiple choices of audio codecs and similar parameter implications on the audio side. Most video service offerings (e.g. those using MPEG Transport Streams or similar) are capable of supporting more than one audio codec along with a single or sometimes multiple video encoding schemes depending on the headend equipment and settop box. Commonly used audio formats for television applications include MPEG Layer II (also known as Musicam used in DVB systems, and MPEG-1, Audio Layer 2), Dolby Digital used in ATSC systems (formerly known as AC-3), NICAM 728 (European digital format for PAL), Advance Audio Coding – AAC (either MPEG-2 AAC or MPEG-4 AAC ([ISO/IEC 14496-3], Subpart 4)), and sometimes other formats such as MP-3 (MPEG-1 Audio Layer 3) will be used, particularly for music content [*].
In addition to the separate audio and video application layer impairments, the synchronization between audio and video components must be maintained to ensure satisfactory QoE. There has been a great deal of research on A/V synchronization requirements in video conferencing and analog broadcast systems and specifications in such bodies as ITU-R [*][*][*]. Because audio that appears before video is very unnatural (sound takes longer to propagate than light so sound lagging visual is normal) some bodies specifying television specific A/V synchronization have recommended tighter tolerances than typically used for video conferencing applications [*].
Recommended minimum engineering objectives for application layer, data plane parameters are presented in the following sections for various video services. In general these parameters are guided by industry best practices (e.g. CableLabs® specifications, encoder vendor guidelines), performance of competitive systems (ex. cable, satellite benchmarks), telco deployment experiences (e.g. FastWeb), and the state encoding technologies (e.g. MPEG-2, MPEG-4 AVC, VC-1, etc. commercial offerings) at the time of publication of this document.
7.1.1 Standard definition (SD) TV: General minimum objectives
Table 1 lists the recommended minimum video application layer performance objectives at the MPEG level, prior to IP encapsulation for broadcast SD (480i / 576i). The audio stream bit rates are additional and specified separately below. Assumptions include:
Source material:
• 4:3 aspect ratio
• Source could enter the head end in analog or digital form
Maximum Viewable Resolution:
• Horizontal x Vertical: 720 pixels x 480 lines (North America) ITU-R BT.601-5 or 720 pixels x 576 lines (Europe)
• Lower resolutions (ex. ¾ Horizontal or ½ Horizontal – so called ½ D1) could be used to ensure encoding quality is maintained for complex materials
Frame rate:
• 29.97 fps (North America) or 25 fps (Europe)
• 23.97 / 24 fps may also be used for film-based materials (with 3:2 pulldown in North America for conversion to 30 fps)
• Two interlaced fields per frame
Table 1 Recommended Minimum Application Layer Performance for Standard Definition Broadcast Program Sources
Video Codec standard / Minimum Bit Rate (video only) / Preprocessing EnabledMPEG-2 - Main profile at Main level (MP@ML) / 2.5 Mbps CBR / Yes (if available)
MPEG-4 AVC (Main profile at Level 3.0) / 1.75 Mbps CBR / Yes (if available)
SMPTE VC-1 / 1.75 Mbps CBR / Yes (if available)
Notes on SDTV Video Bit Rate
The bit rates achieved by a particular video encoder undergo continuous improvement over time particularly when first introduced. As is the case with MPEG-2 since its commercialization in the mid 1990s, improvements have typically followed McCann’s law that states encoder bit rate improves approximately 15% per year with the same quality [*]. In most cases the encoder improvements are done within the scope of the existing standards and therefore do not require upgrades to the decoders.