INTERNATIONAL ORGANISATION FOR STANDARDISATION
ORGANISATION INTERNATIONALE DE NORMALISATION
ISO/IEC JTC1/SC29/WG11
CODING OF MOVING PICTURES AND AUDIO
ISO/IEC JTC1/SC29/WG11
MPEG2008/N10353
April 2009, Lausanne, Swiss
Source: Video Subgroup
Status: Approved
Title: Text of ISO/IEC 23002-4/PDAM1 Video Tool Library Conformance and Reference Software
Editor: Gwo Giun (Chris) Lee, Christophe Lucarz, and He-Yuan Lin
ISO/IECJTC1/SC29N
Date:2009-03-30
ISO/IEC23002-4:200X/PDAM1
ISO/IECJTC1/SC29/WG11
Secretariat:
Information technology— MPEG video technologies— Part4: Video tool librar, AMENDMENT 1: Video tool library conformance and reference software
Élément introductif— Élément central— Partie4: Titre de la partie
Warning
This document is not an ISO International Standard. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an International Standard.
Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation.
ISO/IEC23002-4:200X/PDAM1
Copyright notice
This ISO document is a working draft or committee draft and is copyright-protected by ISO. While the reproduction of working drafts or committee drafts in any form for use by participants in the ISO standards development process is permitted without prior permission from ISO, neither this document nor any extract from it may be reproduced, stored or transmitted in any form for any other purpose without prior written permission from ISO.
Requests for permission to reproduce this document for the purpose of selling it should be addressed as shown below or to ISO's member body in the country of the requester:
[Indicate the full address, telephone number, fax number, telex number, and electronic mail address, as appropriate, of the Copyright Manger of the ISO member body responsible for the secretariat of the TC or SC within the framework of which the working document has been prepared.]
Reproduction for sales purposes may be subject to royalty payments or a licensing agreement.
Violators may be prosecuted.
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IECJTC1.
International Standards are drafted in accordance with the rules given in the ISO/IECDirectives, Part2.
The main task of the joint technical committee is to prepare International Standards. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75% of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
Amendment1 to ISO/IEC230024:200X was prepared by Joint Technical Committee ISO/IECJTC1, Information technology, Subcommittee SC29, Coding of audio, picture, multimedia and hypermedia information.
©ISO/IEC2009– All rights reserved / iiiISO/IEC23002-4:200X/PDAM1
Information technology— MPEG video technologies— Part4: Video tool librar, AMENDMENT 1: Video tool library conformance and reference software
1 Scope
MPEG has produced many important and innovative standards, such as MPEG-1, MPEG-2, MPEG-4, MPEG-7, MPEG-21 and MPEG-A. MPEG continues to propose innovations in the media coding field that are capable of satisfying the changing landscape and needs of media coding applications. With this objective in mind, MPEG intends to standardize a broader framework named reconfigurable media coding (RMC). The current standard addresses primarily reconfigurable video aspects and will only focus on the description of representation of video codec configurations in the RVC framework. The RVC framework is designed to be also capable of including and supporting other non-MPEG codec configurations.
The objective of RVC is to offer a framework that is capable of configuring and specifying video codecs as collection of “higher level” modules by using video coding tools. The video coding tools are defined in video tool libraries. ISO/IEC 23002-4 defines the MPEG video tool library, which contains tools drawn from existing MPEG coding standards and will contain possibly new coding tools approved by MPEG. Moreover, non-MPEG tool libraries, that are conformant to the normative specifications of RVC, are also supported by the RVC framework.
In addition to the addition of new beneficial coding tools to the tool library, the corresponding conformal tests should also be specified before being accepted in the RVC specification. In using these conformal tests with their corresponding coding tools, the RMC framework is able to configure audiovisual codec with high level of confidence in the conformance in behavior to the normative textual description. The current specification defines the conformance testing for corresponding video tool and provides sufficient information for the environmental setups. Furthermore, the RVC Simulation Model is also described in this document.
2 Normative references
ISO/IEC 13818-2:2000, Information technology — Generic coding of moving pictures and associated audio information: Video
ISO/IEC 14496-2:2004, Information technology — Coding of audio-visual objects: Visual
ISO/IEC 14496-10:2005, Information technology — Coding of audio-visual objects: Advanced video coding
3 Terms and definitions
For the purposes of this document, the following terms, abbreviations and definitions apply.
3.1 Reconfigurable video coding (RVC)
A framework defined by MPEG to promote coding standards at tool-level while maintaining interoperability between solutions from different implementers.
3.2 Video tool library (VTL)
A collection of FUs.
3.3 MPEG video tool library (MPEG VTL)
MPEG VTL is a video tool library that is defined in ISO/IEC 23001-4 Video Tool Library. It contains FUs from existing MPEG standards as well as new FUs that are considered beneficial by MPEG.
3.4 Functional unit (FU)
A modular tool, which consists of a processing unit characterized by an input/output behaviour.
3.5 Decoder configuration
A decoder configuration is a conceptual configuration of a decoding solution. Using the MPEG VTL, decoder configuration can be designed as one of the following cases:
· A decoding solution of an existing MPEG standard at a specific profile and level.
· A new decoding solution built from tools of an existing MPEG standard.
· A new decoding solution built from tools of an existing MPEG standard and some new MPEG tools included in the MPEG video tool library.
· A new decoding solution that is composed of new MPEG tools included in the MPEG video tool library.
In summary a RVC decoder description essentially consists of a list of FU and of the specification of the FU connections (FND expressed in FNL) plus the implicit specification of the parser in terms of bistream syntax description (BSPD expressed in RVC-BSDL). So as to be complete behavioural model (the ADM) an RVC decoder description needs to make reference to the behaviour of each FU that is provided in terms of I/O behaviour by the RVC video toolbox library specified in RVC-CAL.
3.6 Model instantiation
The model instantiation consists in building the Abstract Decoder Model from the Decoder Description (made of the BSD and the FND) and from FUs from VTL. During the model instantiation, the parser FU is reconfigured according to the Bitstream Syntax Description (BSD) or loaded from VTL.
3.7 Decoder description
Decoder Description (DD) represents a particular decoder configuration which consists of two parts; FU Network Description (FND) and Bitstream Syntax Description (BSD).
3.8 Abstract decoder model
The Abstract Decoder Model (ADM) is the model resulting of the instantiation of the FUs coming for the Video Tool Libraries, according to the FND.
3.9 Decoding solution
The decoding solution is the implementation of the Abstract Decoder Model.
3.10 Token
Defined in the literature as data entities exchanged among FU’s
3.11 Connections
Links from output ports to input ports of FUs that enable token exchange between these corresponding FUs.
3.12 FU network description
FU network description (FND) depicts FU connections used in forming a decoder. The FND is modeled using FNL.
3.13 Bitstream syntax description (BSD)
Bitstream syntax description (BSD) contains the bitstream syntax, its implicit parsing rules, and possibly tables (e.g. VLDs tables if not already existing in the RVC toolbox library) to define parser FU. The BSD is expressed by using RVC-BSDL.
3.14 FU network language
A language that describes the FND, known also as “network of FUs”.
3.15 BSDL
Bitstream Syntax Description Language (BSDL) was standardized in MPEG-B Part 5 and it describes the bitstream syntax and the parsing rules.
3.16 RVC-BSDL
A pertinent subset of BSDL which is defined within the scope of the current RVC framework.
3.17 Black box testing
In a black box test, only the input, output, of the object under verification are specified. The internals of the object being tested is irrelevant and are therefore hidden.
3.18 White box testing
In a white box test, in addition to the input, output, the internal of the box or the object under test can also be seen.
4 Conformance testing for MPEG-C Part 4 (ISO/IEC FDIS 23002-4)
In order to warrant the reusability, maintainability, and flexibility of the functional units, their corresponding testbeds should also be created for conformance validation prior to inclusion in the RVC specification. These testbeds also serve as executable “application notes” which provide the users with concise procedures on the utilization of theRVC video tool library. Furthermore, specification of the I/O behaviors of the RVC video tool library is also achieved by executing the tool with its conformance testbed. Hence, the testbed should not only verify the functionality but also validate the conformance of the FU’s.
.
4.1 Internal description of FU’s and Conformance Testing Environment
As mentioned in previous sections, the RVC toolbox is composed of functional units and tools to test the conformance of functional units as shown in Figure 1 [2].
Figure1— Basic Functional Unit of Video Tool Library
FU’s in RVC are modeled as actors in the CAL language, where the behavior of the FU’s depend on the input tokens and the internal state of the actors. Benefiting from the CAL-based FU’s, the verification processes of the FU’s should be performed by feeding tokens to the FU’s and checking the state and corresponding output tokens, since the functional units are encapsulated independently. The conformance testing tools should also be implemented in CAL, where the testing tools act as the testbed for functional units in Moses as shown in Figure 2.
The testbed is composed of the test pattern database, the golden response database, and the automatic comparison and analysis unit. The test pattern database contains the test patterns or tokens to feed the functional unit under verification or testing for different purposes. The tokens are categorized (tbd) into different classes and the tokens in both the test pattern database and the expected golden response database should be conformal to the textual definitions in the RVC specification. Scheduling of the tokens will also be described in the textual description for conformal testing purposes (tbd).
Figure2— Conformance testbed of functional units
4.1.1 Generation of the Test Pattern and Golden Response
Test patterns or predefined input tokens and the corresponding golden responses or expected output tokens to and from the FU’s will be generated from the corresponding MPEG reference software by using the associated development kit from CAL (tbd).
4.1.2 Conformance
The verification tests should be conformal to the normative textual description for the corresponding functional units in the RVC Video Tool Library specification. For conformance purposes, all normative portions of the FU’s should be verified using black box strategy where the informative parts are implemented in white box verification tests.
4.1.3 Different categories of verification Tests
The verification tests in RVC conformance tools are categorized into real code testing, corner case testing, random testing, and regression testing.
4.1.3.1 Real code testing: The test patterns or predefined and classified input tokens to FU’s are obtained or genertaed from the reference software of the corrsponding MPEG standard to avoid misunderstanding of the MPEG specifications.
4.1.3.2 Corner case testing: Input tokens to FU’s are created to test the scenarios which are most probable of crashing the functional units.
4.1.3.3 Random testing: Constrained random pattern or random tokens generation scheme is also adopted to create the cases that might not be considered.
4.1.3.4 Regression testing: It requires that all conformal test cases be repeated after modification of even one test
4.1.4 Verification test coverage
Coverage analysis as described below is used in the informative white box verification of RVC to measure the quality of the testbed quantitatively. This analysis will indicate both the parts of the functional units that have been executed and also specify places where further verification are required. The major types of coverage measures include action coverage, condition coverage, branch coverage, and path coverage.
4.1.4.1 Action coverage: The percentage of all the statements of actions which have been exercised in the actors of a FU for CAL codes.
4.1.4.2 Branch coverage: The percentage of the branches which has been tested, where the branch is the outcome of a decision.
4.1.4.3 Condition coverage: The percentage of the outcome (true or false) of each conditional sub-expression that has been executed.
4.1.4.4 Path coverage: The percentage of the paths in the FU that has been executed. A path is the independent sequence of all the possible combinations of the branches in the FU.
4.2 Functional Unit Conformance
The functional units of RVC are classified into two major categories, namely algorithmic content FU’s and data management FU’s [7].
4.2.1.1 Algorithmic content FU’s: This type of FU comprises a coding tool of a specific MPEG standard with particular profile-and-level. The behaviors of an algorithmic content FU should be completely conformal to the correspding specifaction described in MPEG standard document.
4.2.1.2 Data management FU’s: This category of FU packs the required decoding information and data into tokens in CAL and feeds the tokens into the other functional units in the correct order.
The conformance test requirements for these two types of functional units are slightly different, since the functionalities of algorithmic content FU’s have been specified in the existing MPEG standard but the data management FU’s, however, are brand-new and under development. Therefore, for the data management FU’s, it is strongly recommended that sufficient test patterns and the corresponding golden response be provided to achieve 100% test coverage to guarantee the functionality of such FU’s. In order to claim that an algorithmic content FU is compliant to a specific MPEG standard with particular profile and level, the FU is strictly required to successfully pass all the conformance test cases extracted from decoding the existing conformance bitstreams of the normative test suite by using the corresponding reference software. Note that normative test suites of each standard can be found in the electronic annex of the MPEG standards. In addition, the testbed for the algorithmic content FU’s must accomplish 100% test coverage as well. After RVC is standardized, some of the test patterns and golden response generated for verification of functional units will serve as the conformance test suites of FU’s directly.