OTT 2005 Interoperability test template
Use this template to create your interoperability test report.
1st page: should contain the header and all information (group number etc.) necessary
Table of Content
Table of Content 2
Introduction 3
Test case descriptions 3
TEST CASE 1. Playing local wav file 3
TEST CASE 2. Fetching a remote m3u 3
TEST CASE 3. Fetching a missing file 4
TEST CASE 4. Sending 404 response 4
TEST CASE 5. Offering an m3u-file for download 4
TEST CASE 6. Playing a remote wav file 5
TEST CASE 7. Offering a wav file for download 5
TEST CASE 8. Sending files while playing. 6
TEST CASE 9. Stress test: multiple requests 6
SUMMARY OF RESULTS 6
Test case results 7
CONCLUSION 7
Introduction
Write us a nice introduction about the interoperability testing.
Test case descriptions
This chapter describes the test cases used in the interoperability testing. You do not need to include this chapter in your interoperability test report.
TEST CASE 1. Playing local wav file
Interoperability
This test case is performed individually.
Description
The group tests whether their implementation is capable of playing a local wav- file stored in their own board.
PASS criteria
The test case is passed if the program plays the song successfully.
FAIL criteria
The test case is failed if the song is not played, or the program cannot play the song completely to the end.
TEST CASE 2. Fetching a remote m3u
Interoperability
This test case is performed with two different groups, resulting in sub-tests a.) and b.).
Description
The program needs to be able to fetch an index.m3u file from a remote server and display it to the user.
PASS criteria
Each sub-test is passed if an m3u file is successfully fetched and displayed.
FAIL criteria
Each sub-test is failed if the m3u file fails to download or is not displayed properly.
THEIR FAULT criteria
If the action fails due to a reason that can be clearly attributed as a failure of the other group. Very strong arguments need to be presented for why this is the Case. An example would be that the other group simply cannot respond to proper GET requests.
TEST CASE 3. Fetching a missing file
Interoperability
This test case is performed with two different groups, resulting in sub-tests a.) and b).
Description
The program needs to be able to react in an orderly fashion to a 404 response sent by the other groups. The group asks for a file that is known not to exist.
PASS criteria
Each sub-test is passed if the 404 response is properly handled.
FAIL criteria
Each sub-test is failed if the program fails to respond in a correct fashion to the 404 response received.
THEIR FAULT criteria
If the other group fails to send a 404 response to a well-formed request for a non-existing file, the test case result is “their fault”.
TEST CASE 4. Sending 404 response
Interoperability
This test case is performed with two different groups, resulting in sub-tests a.) and b).
Description
The program needs to be able to react in an orderly fashion to a GET requests that requests a file that does not exist locally and send a 404 response back.
PASS criteria
Each sub-test is passed if the 404 response is properly sent.
FAIL criteria
Each sub-test is failed if the program fails to send the 404 response.
THEIR FAULT criteria
If the 404 response is not sent due to a clearly erroneous GET request from the other group, the result can be marked as “their fault”.
TEST CASE 5. Offering an m3u-file for download
Interoperability
This test case is performed with three different groups, resulting in sub-tests a.) and b.).
Description
The program needs to be able to offer an index.m3u file for download
PASS criteria
Each sub-test is passed if the other group can succesfully fetch and display an m3u file from the implementation.
FAIL criteria
Each sub-test is failed if the m3u file fails to download or is not displayed properly.
THEIR FAULT criteria
If the file is not sent due to a clearly erroneous GET request by the other group, the test result can be marked as “their fault”.
TEST CASE 6. Playing a remote wav file
Interoperability
This test case is performed with three different groups, resulting in sub-tests a) and b).
Description
The program needs to be able to fetch and play a remote wav file.
PASS criteria
Each sub-test is passed if the implementation can fetch and play wav-files from the other groups implementation.
FAIL criteria
Each sub-test is failed if the wav file fails to download or is not played properly.
THEIR FAULT criteria
If the playing fails because the other group fails to respond to a perfectly correct GET request, or for some other reason that can be clearly attributed to the other group, the result can be marked as “their fault”.
TEST CASE 7. Offering a wav file for download
Interoperability
This test case is performed with three different groups, resulting in sub-tests a.) and b.).
Description
The program needs to be able to offer a local wav-file for download.
PASS criteria
Each sub-test is passed if the other groups implementation can fetch and play wav-files from the groups implementation.
FAIL criteria
Each sub-test is failed if the wav file fails to download or is not played properly.
THEIR FAULT criteria
If the action fails due to a reason that can be clearly attributed as a failure of the other group. Very strong arguments need to be presented for why this is the case.
TEST CASE 8. Sending files while playing.
Interoperability
This test case is performed with one other group. The other test group should be one that has been successfully able to fetch files from the implementation.
Description
The program attempts to offer local files (m3u and wav) for download while playing a song.
PASS criteria
If the program is capable of responding to http requests also while playing a wav file.
FAIL criteria
The test is failed if the implementation does not respond to the requests while playing a song.
THEIR FAULT criteria
A very strong argument is needed if this test case is labeled as “their fault”.
TEST CASE 9. Stress test: multiple requests
Interoperability
This test case is performed with all the other groups in the interoperability testing.
Description
All the other groups in the testing session attempt to simultaneously download the same file from the implementation.
PASS criteria
If the program is capable of responding to all of the requests in a timely fashion.
FAIL criteria
If some of the groups do not receive an answer to their requests. The report on this test should clearly state what was the overall performance: how many groups were served successfully?
SUMMARY OF RESULTS
Test case / With group / Description / RESULT(pass, fail or “their fault”) /
1 / self / Playing local wav file / Pass - fail
2a / Fetching remote m3u / Pass - fail - their fault
2b / Fetching remote m3u / Pass - fail - their fault
3a / Fetching a missing file / Pass - fail - their fault
3b / Fetching a missing file / Pass - fail - their fault
4a / Sending 404 response / Pass - fail - their fault
4b / Sending 404 response / Pass - fail - their fault
5a / Offering an m3u file for download / Pass - fail - their fault
5b / Offering an m3u file for download / Pass - fail - their fault
6a / Playing a remote wav file / Pass - fail - their fault
6b / Playing a remote wav file / Pass - fail - their fault
7a / Offering a wav file for download / Pass - fail - their fault
7b / Offering a wav file for download / Pass - fail - their fault
8 / Sending files while playing / Pass - fail - their fault
9 / ___ groups / Stress test: multiple requests / Pass - fail
CONCLUSION
The conclusion summarizes the results of the tests in a concise way.