24
WTSA16/11-E
/ World Telecommunication Standardization Assembly (WTSA-16)Hammamet, 25 October - 3 November 2016 /
PLENARY MEETING / Document 11-E
July 2016
Original: English
ITU-T Study Group 12
Performance, QoS and QoE
REPORT OF ITU-T SG12 TO THE WORLD TELECOMMUNICATION STANDARDIZATION ASSEMBLY (WTSA-16), PART I: GENERAL
Abstract: / This contribution contains the report of ITU-T Study Group 12 to WTSA-16 concerning its activities during the 2013-2016 study period.
Note by the TSB:
The report of Study Group 12 to the WTSA-16 is presented in the following documents:
Part I: Document 11 – General
Part II: Document 12 – Questions proposed for study during the study period 2017-2020
CONTENTS
Page /1 Introduction 2
2 Organization of work 5
3 Results of the work accomplished during the 2013-2016 study period 9
4 Observations concerning future work 19
5 Updates to the WTSA Resolution 2 for the 2017-2020 study period 20
ANNEX 1 List of Recommendations, Supplements and other materials produced or deleted during the study period 21
ANNEX 2 Proposed updates to the Study Group 12 mandate and Lead Study Group roles 27
1 Introduction
1.1 Responsibilities of Study Group 12
Study Group 12 was entrusted by the World Telecommunications Standardization Assembly (Dubai, 2012) with the study of 17 Questions in the area of performance, quality of service (QoS) and quality of experience (QoE), covering the full spectrum of terminals, networks and services, from speech over fixed circuit-based networks to multimedia applications over networks that are mobile and packet based.
1.2 Management team and meetings held by Study Group 12
Study Group 12 met six times in Plenary and two times in Working Parties in the course of the study period[1] (see Table 1) under the chairmanship of Mr Kwame BAAH-ACHEAMFUOR (Ghana), assisted by Vice-Chairmen Mr Paul BARRETT (United Kingdom), Mr Vincent BARRIAC (France), Mr Gamal Amin ELSAYED (Sudan), Mr Hyung-Soo (Hans) KIM (Korea (Rep. of)), Mr Al MORTON (United States), Mr Feng QI (China), Mr Akira TAKAHASHI (Japan), and Mr Hassan TALIB (Morocco). Mr José Guadalupe Rojas RAMÍREZ (Mexico) did not participate in any SG12 meetings.
In addition many Rapporteurs' meetings took place during the study period in different locations, see Table 1-bis.
TABLE 1
Meetings of Study Group 12 and its Working Parties
SG/WP 12 / Geneva, 19-28 March 2013 / COM 12 – R 1 to R 4
SG12RG-AFR / Ouagadougou, 19 July 2013 / COM 12 RGAFR – R 1
SG/WP 12 / Geneva, 3-12 December 2013 / COM 12 – R 5 to R 8
WP 2/12 / Solothurn, 25 March 2014 / COM 12 – R 9
SG12RG-AFR / Kampala, 26 June 2014 / COM 12 RGAFR – R 2
SG/WP 12 / Geneva, 2-11 September 2014 / COM 12 – R 10 to R 13
WP 1/12 / Herzogenrath, 12 December 2014 / COM 12 – R 14
SG12RG-AFR / Dakar, 23 March 2015 / COM 12 RGAFR – R 3
SG/WP 12 / Geneva, 5-14 May 2015 / COM 12 – R 15 to R 18
SG/WP 12 / Geneva, 12-21 January 2016 / COM 12 – R 19 to R 22
SG12RG-AFR / Livingstone, 18 March 2016 / COM 12 RGAFR – R 4
SG/WP 12 / Geneva, 7-16 June 2016 / COM 12 – R 23 to R 26
TABLE 1-bis
Rapporteur meetings organized under Study Group 12 during the study period
2013-01-15
to
2013-01-16 / France [Lannion]/Orange / 5/12 / Rapporteur group meeting for Q5/12
2013-07-01
to
2013-07-02 / China [Shenzhen]/Huawei Technologies / 9/12 / Rapporteur group meeting for Q9/12
2013-07-15
to
2013-07-17 / Switzerland [Geneva] / 4/12 / Rapporteur group meeting for Q4/12
2013-09-04
to
2013-09-06 / Austria [Vienna]/FTW / 13/12
14/12
17/12 / Rapporteur group meeting for Q13, 14 and 17/12
2013-10-07
to
2013-10-08 / Switzerland [Geneva] / 5/12 / Rapporteur group meeting for Q5/12
2014-02-26
to
2014-02-28 / Sweden [Stockholm]/Ericsson / 13/12
14/12
17/12 / Rapporteur group meeting for Q13, 14 and 17/12
2014-03-24
to
2014-03-25 / Switzerland [Solothurn]/SwissQual/Rohde & Schwarz / 9/12 / Rapporteur group meeting for Q9/12
2014-05-20
to
2014-05-22 / United States [Mountain View, CA]/Audience / 9/12 / Rapporteur group meeting for Q9/12
2014-06-03
to
2014-06-05 / Germany [Chemnitz]/Chemnitz University of Technology / 14/12
17/12 / Rapporteur group meeting for Q14 and 17/12
2014-06-11
to
2014-06-13 / United States [Detroit, Michigan]/QNX Software Systems Inc. / 4/12 / Rapporteur group meeting for Q4/12
2014-06-17
to
2014-06-18 / Sweden [Stockholm]/Ericsson / 10/12 / Rapporteur group meeting for Q10/12
2014-06-25
to
2014-06-26 / Sweden [Lund]/Sony / 5/12 / Rapporteur group meeting for Q5/12
2014-12-10
to
2014-12-12 / Germany [Herzogenrath]/HEAD acoustics / 4/12 / Rapporteur group meeting for Q4/12
2015-02-02
to
2015-02-03 / Switzerland [Geneva] / 5/12 / Rapporteur group meeting for Q5/12
2015-03-10
to
2015-03-12 / United States [Brighton, MI] / 4/12 / Rapporteur group meeting for Q4/12
2015-10-07
to
2015-10-08 / United States [Detroit, Michigan]/General Motors / 4/12 / Rapporteur group meeting for Q4/12
2015-11-11
to
2015-11-13 / Germany [Berlin]/T-Labs / 13/12
14/12 / Rapporteur group meeting for Q13, 14/12
2016-04-27
to
2016-04-29 / Germany [Berlin]/T-Labs / 13/12
14/12
17/12 / Rapporteur group meeting for Q13, 14 and 17/12
2 Organization of work
2.1 Organization of studies and allocation of work
2.1.1 At its first meeting of the study period, Study Group 12 decided to establish three Working Parties. At the same meeting in March 2013, Study Group 12 received the Deliverables of Focus Group on Driver Distraction (FG Distraction) which had successfully concluded its work.
2.1.2 Table 2 shows the number and title of each Working Party, together with the number of Questions assigned to it and the name of its Chairman.
2.1.3 Table 3 lists other groups created by Study Group 12 during the study period. It also lists groups under the responsibility of Study Group 12 or with active Study Group 12 participation.
Quality of Service Development Group (QSDG)
Launched in the 1980s, under responsibility of Study Group 12 since WTSA08, the primary aim of the Quality of Service Development Group (QSDG) is to improve the quality of the international service, to the benefit both of the subscribers and administrations. The QSDG held four meetings during the study period. It was instrumental in initiating new work items in Study Group 12 and bringing the work of the SG12 to a wider audience through workshops, symposia and forums on QoS and QoE.
Intersector Rapporteur Group on Audiovisual Quality Assessment (IRG-AVQA)
The Intersector Rapporteur Group on Audiovisual Quality Assessment (IRG-AVQA) was established in accordance with Annex C of WTSA-12 Resolution 18 and the corresponding provisions in Resolution ITU-R 6 in order to study topics related to video and audiovisual quality assessment among ITU-R Study Group 6, ITU-T Study Group 9 and ITU-T Study Group 12.
IRG-AVQA held six meetings during the study period–all of them in colocation with either ITU-R WP6C, ITU-T SG9, ITU-T SG12, or the Video Quality Experts Group (VQEG).
SG12 has requested the other participating groups for approval for an extension of the IRG-AVQA mandate in the next study period.
2.1.4 In line with WTSA-12 Resolution 54, the Regional Group on QoS for the Africa Region (SG12 RG-AFR), launched by Study Group 12 in May 2008, continued to operate in study period 2013-2016. It held meetings during the SG12 plenary meetings in Geneva, as well as four meetings in Africa. Thanks to an increased footprint in Africa, the number of contributions to SG12 from African countries increased in the course of the study period: from one contribution in March 2013 to six contributions in June 2016.
TABLE 2
Organization of Study Group 12
and Vice-Chairmen /
PLEN / Q1/12; Q2/12; QSDG; / — / —
WP1/12 / Q3/12; Q4/12; Q5/12; Q6/12; Q7/12; Q10/12; / Terminals and multimedia subjective assessment / Mr Nielsen Lars Birger (Chairman)
Mrs Berndtsson Gunilla (Vice-chairman)
WP2/12 / Q8/12; Q9/12; Q14/12; Q15/12; Q16/12; / Objective models and tools for multimedia quality / Mr Barrett Paul (Chairman)
Mr Barriac Vincent (Vice-chairman)
WP3/12 / Q11/12; Q12/12; Q13/12; Q17/12; / Multimedia QoS and QoE / Mr Coverdale Paul (Chairman)
Mr Takahashi Akira (Vice-chairman)
TABLE 3
Other Groups
Quality of Service Development Group (QSDG) / Mrs Yvonne UMUTONI (Rwanda) / Ms Stavroula BOUZOUKI (Greece), Mr Wenyan JIN (China), Mrs Louisa SOSU (Ghana), Mr Jacob MUNODAWAFA (Mozambique), Mr Mohammad Qasim NASIMEE (Afghanistan), Mr Arvind CHAWLA (India), Mr Tiago SOUSA PRADO (Brazil), Mr Mehmet ÖZDEM (Turkey) /
Regional Group on QoS for the Africa Region (SG12 RG-AFR) / Gamal Amin ELSAYED (Sudan) / Mr Robert ECHEDA (Uganda), Mr Seyni Malan FATY (Senegal), Mr Hassan TALIB (Morocco) /
Intersector Rapporteur Group Audiovisual Quality Assessment (IRG-AVQA) / Chulhee LEE (Korea, Rep of),
QUAN Huynh-Thu, (Australia),
Jens BERGER (Germany) / N/A /
2.2 Questions and Rapporteurs
2.2.1 WTSA-12 assigned to Study Group 12 the 17 Questions listed in Table 4.
2.2.2 The Questions listed in Table 5 have been adopted during this period.
2.2.3 The Questions listed in Table 6 have been deleted during this period.
TABLE 4
Study Group 12 – Questions assigned by WTSA-12 and Rapporteurs
1/12 / SG 12 work programme and QoS/QoE coordination in the ITU-T / PLEN / Mr Baah-Acheamfuor Kwame (Rapporteur)
Mr Echeda Robert (Rapporteur)
Mr Kim Hyung-Soo (Hans) (Rapporteur)
Mr Pomy Joachim (Rapporteur)
2/12 / Definitions, guides and frameworks related to QoS/QoE / PLEN / Mr Pomy Joachim (Rapporteur)
3/12 / Speech transmission characteristics of communication terminals for fixed circuit-switched, mobile and packet-switched (IP) networks / 1/12 / Mr Yi Gaoxiong (Rapporteur)
4/12 / Hands-free communication and user interfaces in vehicles / 1/12 / Mr Gierlich Hans Wilhelm (Rapporteur)
5/12 / Telephonometric methodologies for handset and headset terminals / 1/12 / Mr Nielsen Lars Birger (Rapporteur)
6/12 / Analysis methods using complex measurement signals including their application for speech enhancement techniques and hands-free telephony / 1/12 / Mr Gierlich Hans Wilhelm (Rapporteur)
7/12 / Methods, tools and test plans for the subjective assessment of speech, audio and audiovisual quality interactions / 1/12 / Mr Sharpley Alan (Rapporteur)
Mr Usai Paolo (Rapporteur)
8/12 / E-Model extension in wideband transmission and future telecommunication and application scenarios / 2/12 / Mr Möller Sebastian (Rapporteur)
9/12 / Perceptual-based objective methods for voice, audio and visual quality measurements in telecommunication services / 2/12 / Mr Berger Jens (Rapporteur)
10/12 / Conferencing and telemeeting assessment / 1/12 / Mrs Berndtsson Gunilla (Rapporteur)
Mr Skowronek Janto (Rapporteur)
11/12 / Performance interworking and traffic management for Next Generation Networks / 3/12 / Mr Pomy Joachim (Rapporteur)
12/12 / Operational aspects of telecommunication network service quality / 3/12 / Mr Talib Hassan (Rapporteur)
13/12 / QoE, QoS and performance requirements and assessment methods for multimedia / 3/12 / Ms García Marie-Neige (Rapporteur)
Mr Takahashi Akira (Rapporteur)
14/12 / Development of parametric models and tools for multimedia quality assessment / 2/12 / Mr Gustafsson Jörgen (Rapporteur)
Mr Raake Alexander (Rapporteur)
15/12 / Objective assessment of speech and sound transmission performance quality in networks / 2/12 / Mr Barriac Vincent (Rapporteur)
Mr Pomy Joachim (Rapporteur)
16/12 / Framework for diagnostic functions and their interaction with external objective models predicting media quality / 2/12 / Mr Malfait Ludovic (Rapporteur)
17/12 / Performance of packet-based networks and other networking technologies / 3/12 / Mr Morton Al (Rapporteur)
TABLE 5
Study Group 12 – New Questions adopted and Rapporteurs
None
TABLE 6
Study Group 12 – Questions deleted
None
3 Results of the work accomplished during the 2013-2016 study period
3.1 General
During the study period[2], Study Group 12 examined 395 contributions and generated a large number of TDs and liaison statements. It also:
– drew up 23 new Recommendations;
– revised 25 existing Recommendations;
– issued 17 Amendments and 2 Corrigenda;
– developed 2 Supplements and 8 Implementer's Guides.
3.2 Highlights of achievements
The main results achieved on the various Questions assigned to Study Group 12 are briefly summarized below. Formal replies to the Questions are given in a synoptic table in Annex 1 of this report.
a) Achievements of Questions reporting to the SG12 plenary
Q1/12 - SG12 work programme and QoS/QoE coordination in the ITU-T (Rapporteurs Mr Kwame Baah-Acheamfuor, Mr Robert Echeda, Mr Hyung-Soo (Hans) Kim, Mr Joachim Pomy)
As in previous study periods, Q1/12 fulfilled the role to co-ordinate QoS/QoE-related activities in the ITU-T and collaboration on QoS/QoE with other standards bodies. Q1/12 provided a temporary home to work item proposals, when they were not directly related to existing Questions. As the lead study group on QoS/QoE, SG12 used Q1 to foster consistency on QoS/QoE matters within the ITU, and with related external organizations (e.g. 3GPP, IETF). In addition, any business concerning the Regional Group on QoS for the Africa Region (SG12 RG-AFR) was discussed under the stewardship of Q1.
Q2/12 - Definitions, guides and frameworks related to QoS/QoE (Rapporteur Mr Joachim Pomy)
Q2/12 is in charge of the development and maintenance of ITU-T Recommendations providing the definitions needed to support new or revised Recommendations developed by the other Questions under study in SG12. The responsibility of Q2 includes the study of new or revised definitions to be included in Recommendation ITU-T P.10/G.100 "Vocabulary for performance and quality of service".
A major milestone was achieved towards the end of the study period when SG12, under the leadership of Q2, withdrew the old definition of QoE from P.10/G.100, and inserted three new terms and definitions, and one additional bibliographical reference as Draft Amendment5 "New definitions for inclusion in Recommendation ITU-T P.10/G.100":
Quality of Experience (QoE) is the degree of delight or annoyance of the user of an application or service.