AMHS CONFORMANCE TEST TOOLACP WGN05 (Montréal, 11th- 20th May 2005)
ACP – WG N/SG N3-WP/3-5 (IP)
24/Apr/2005
AERONAUTICAL COMMUNICATIONS PANEL (ACP)
Working Group N – Networking
SUBGROUP N3 – Ground-Ground Applications
Montreal, 12th– 13th May2005 (third meeting)
Agenda Item 3 : ATS Message Handling Services
AMHS CONFORMANCE TEST TOOL
Prepared by Klauspeter Hauf
Summary
DFS started the project "AMHS conformance test tool" in August 2004 together with industry partners based on experiences from the FIRST interoperability tests. The now available test tool interfaces an AMHS implementation with a simulated AMHS/AFTN environment and can run test scripts created through a graphical user interface.
Such conformance testing prepares AMHS implementation for subsequent interoperability testing and reduces costs for co-ordination effort and communication services which are needed for the test performance between distant sites.
The paper provides an outline of the concept and features of the AMHS conformance test tool. Its functions have been successfully verified by conformance test exercises with AMHS implementations supplied by two manufacturers.
WG N/SG N3-IPxxpage 126/04/05
AMHS CONFORMANCE TEST TOOLACP WGN05 (Montréal, 11th- 20th May 2005)
1.Introduction
Generally, Conformance testing can be defined as the exhaustive testing of an implementation under test (IUT) against functions and procedures defined in a Standard specification. A test tool provides the IUT with defined inputs and observes the resulting behaviour of the IUT. Conformance testing should also allow to "provoke" the IUT with an incorrect behaviour for study of the IUT´s stability (negative testing). Typically, conformance testing is performed locally between the IUT and the test tool.
Conformance testing will not supersede, but prepare IUTs for subsequent interoperability testing, thereby reducing the total costs that in particular interoperability tests impose concerning co-ordination effort and needed communication services. - In this view, an AMHS conformance test tool is the equivalent facility to the CIDIN test tool CITEST which was developed in support of the CIDIN implementation.
Initiated by the experience made during the FIRST interoperability tests, the DFS started in August 2004, together with partners from the industry, the development of an AMHS conformance test tool. In particular, the test tool should allow to perform such tests which had to be suspended from the FIRST interoperability tests due to constraints that are inherent to interoperability tests with real systems.
The AMHS conformance test tool is now available and runs an initial set of conformance tests including such test cases which had been defined for the FIRST interoperability tests. This set of tests addresses the communications between AMHS users as well between AFTN and AMHS users by means of the AFTN/AMHS Gateway. The technical concept behind the test tool provides a grade of flexibility which may qualify the tool to run also conformance test which may be developed in the future by standardisation bodies.
2.Conceptual approach
The conformance test tool interfaces the IUT with a simulated AMHS/AFTN environment consisting of a number of adjacent ATS Message Servers (MTAs) and one adjacent AFTN/CIDIN Centre (see Figure 1). Each of the ATS Message Servers in turn represents a number of AMHS users (UAs). Analogously, behind the AFTN/CIDIN Centre there is a number of AFTN users.
For the simulated AMHS/AFTN environment the intended scope of test scenarios has been specified by definition of test groups and test classes. Seven test groups were defined to provide a logical high-level ordering in conformance tests: submission operations from AMHS users, AMHS transfer operations, delivery operations to AMHS users, AFTN/AMHS Gateway operations, naming and addressing, AMHS parameters (as the ATS Message Header), and traffic logging.
For each of the above test groups test classes were defined relating to normal communication and some failure situations in terms of generated erroneous Messages, Report and Probes, and procedural errors.
The defined test groups and classes set up the requirements for the software design and operation of the test tool. The concept was successfully verified by conformance test exercises with AMHS implementations from two manufacturers.
Figure 1: AFTN/AMHS environment simulated for the IUT
1.3.Realisation of the test tool
The AMHS test tool (see Figure 2) simulates an operational environment for the IUT with three MTA instances (representing three adjacent ATS Message Servers or three neighbour PRMDs) and one AFTN/CIDIN source/sink (representing an adjacent AFTN/CIDIN environment).
The AMHS test tool was built on the basis of commercial off-the-shelf software products providing the X.400 protocols and the MTA. Also, existing components supporting the CIDIN protocol and AFTN functions were integrated. The TTMS (Test Tool for Messaging Systems) is the test control and evaluation unit that implements test scripts, supports test step verification and maintains test result logs including the records of messages exchanged with the IUT.
The test tool generates AMHS data at the X.400/P1 level. It sends and receives AMHS messages, reports and probes via one or several AMHS transfer ports (see Figure 2). Whenever the IUT's gateway is involved in the test, the test tool makes use of its AFTN/CIDIN interface in either direction.
The IUT is assumed to have a local attached AMHS user agent that can be used for AMHS submission/delivery tests. Optionally, a local AFTN user terminal and a monitor and control position can be used in the tests.
Figure 2: Abstract test configuration
2.4.Features of the test tool
The test tool supports test data and script preparation, script-based execution and evaluation.
Test preparation
An AMHS editor is available to create messages, reports and probes. The user enters AMHS parameter values in a graphical user interface, thereby making use of templates and a number of data samples.
Test scripts are created in a graphical user interface. There is no need to know any scripting language.
Test scripts can be combined to support complex scenarios. Messages can be sent to the IUT in a controlled sequence.
The test tool can be configured to return a report (DR or NDR) or a notification (receipt or non-receipt) response to any messages received from the IUT.
Test execution
Test scripts are presented in table-like format. Each line contains a test step with commands and parameters for actions and queries.
Test scripts can run in full automatic mode or manually be controlled step by step. On-line modification is possible to optimise a test script in the course of a test dry run.
Test evaluation
The test tool analyses data captured from the IUT's interfaces and compares the received with the expected AFTN or AMHS parameter values. Deeper analysis of captured AMHS data is possible with the AMHS viewer.
Each executed test step indicates its success or failure with a green or red result field. All test results are automatically stored.
3.5.Supported test scenarios
The test tool supports conformance tests for native AMHS communication and AMHS/AFTN gateway functions in either direction. An initial set of test scripts has been implemented and already been verified with two candidate implementations under test (IUTs) from different manufacturers.
The test tool is able to send
- Messages with ia5-text or general-text body containing correctly formatted ATS-message-header and ATS-message-text,
- Messages (IPMs) with ATS-message-header syntax errors or with (intentional) erroneous IPM heading,
- Notifications, reports and probes with user-provided values.
With the test tool the user (test personal) is able to view and evaluate
- AMHS parameter values of the envelope and content for every incoming/outgoing message,
- external/internal trace information conveyed in the envelope of messages received from the IUT,
- contents of delivery and non-delivery reports showing reason and diagnostic code in plain text.
With the test tool the user (test personal) is able to verify
- conversion of the ATS-message-header to correct AFTN format,
- translation from AFTN addresses to XF- or CAAS-addresses.
WG N/SG N3-IPxxpage 126/04/05