6.Test and Acceptance

6.1Introduction

Sprint will conduct or support all testing activities related to its services and systems as defined in this section. Sprint will be responsible for assuring that the services, capabilities, and features provided at Service Delivery Points conform with the interface specifications and all other requirements defined in Section C of the RFP. Sprint will also be responsible for ensuring that all support systems (e.g., Service Ordering, Billing, etc.) essential to the provisioning, installing, maintaining and reporting performance on these services are in place, tested, and approved by the Government prior to installing the services.

6.2Generic Verification Test Plan

This section provides a description of the commercial processes proposed to verify that services are performing satisfactorily after transitions are completed. Sprint will provide Transition, Migration and Implementation Verifications Test Plans to the CO as described in RFP Section E.2.2.2.1. The Verification Test Plans will include:

A description of the types of tests to be performed at locations and performed on the services Sprint may transition from FTS2000 or a legacy network or implement as a project or new service on the network.
Remote or on-site locations at which testing will be initiated and performed.
Type and description of test equipment to be used.
Identification (ID) of personnel from organizations other than Sprint's organization who participate in the tests.
Checklists of all activities required of each participant for a particular type of cutover technique.
Examples of reports to be provided by Sprint after the successful completion of Verification tests. These reports will include:

Final Verification test results

Names of Sprint personnel who conducted the tests

Time and date that the tests were successfully completed.

For each location being transitioned to Sprint, the site-specific cutover plans will include the specific testing activities and procedures applicable to each location being cutover to Sprint. The plans will also include copies of the applicable test forms for each of the services at the location. Site-specific cutover plans submitted to the responsible LGCs, at least 45 calendar days prior to the physical cutover of sites, will include detailed information on the testing proposed for each site being cut over. Sprint will use existing commercial test procedures. Test plans for other services will be provided as applicable.

Sprint will perform the appropriate cutover tests during the timeframe established for the actual physical cutover of service at a location (i.e., outside of normal business hours unless otherwise arranged for with, and approved by the LGC). The OTM will verify that the services have been cut over to Sprint, along with all the features and functionality required under the FTS2001 contract.

The LGC will oversee the cutover tests and system tests. Once all cutover tests are verified as satisfactory, and the LGC is satisfied that the services have been successfully cut over, Sprint will officially notify the LGC and the TCC that the cutover has been completed and provide the test results.

Following award, and for transition, Sprint will update the Transition Verification Test Plan (TVTP) to reflect agencies identified by the Government that will transition to Sprint’s services. The schedule will follow the sequence of events shown in Table 6-1. Sprint will submit its updated TVTP to the CO within 30 calendar days after notice to proceed with transition activities. It is expected the Government will issue transition service orders within 30 calendar days of contract award, and that the Government will provide a review of the updated TVTP to Sprint within 30 calendar days of its receipt. Sprint will submit a revised plan within 21 calendar days after receiving the review from the Government. The Government is expected to approve Sprint’s TVTP within 21 calendar days after receipt of the revised plan. Sprint will provide the appropriate sections of the TVTP to the Local Government Contact (LGC) responsible for the location(s) where verification testing will be conducted within 7 calendar days after approval of the revised plan or within 7 calendar days after the LGC is selected, whichever comes later.

Table 6-1: Verification Test Plan(s) Schedule
Government issues transition order(s) / A
Sprint submits Verification Test Plan(s) / A+30 (B)
Government provides review of Verification Test Plan(s) / B+30 (C)
Sprint submits revised plan / C=21 (D)
Government approves plan / D+21 (E)
Appropriate sections of plan delivered to LGC* / E+7
*assuming LGC has been selected

Generic test plans for each service and feature are described herein. Where test reports are generated, they will include:

Type of Test(s) performed;
Final verification results;
Names of contractor personnel who conducted the tests; and
Time(s) and date(s) that tests were successfully conducted.

A sample test report is included as Figure 6-1.

Identification of "types" of personnel, other than Sprints', who are expected to participate in these tests are as follows:

Only two "types" of non-Sprint controlled personnel are expected to participate in conduct of these tests, Local Exchange Carrier (LEC) technicians, and agency Information Systems (IS) personnel.
In cases where access circuits are provisioned by Sprint, LEC technicians may be involved in the delivery and/or testing of those circuits. Tests are conducted under the supervision of the Sprint Private Line Service Center (PLSC), with LEC technician involvement limited to assistance in establishing test loops, and correction of any detected circuit problems.
Agency IS personnel are expected to be involved only in those delivery tests aimed at confirming proper interaction between the Sprint network and agency's end systems. Depending on the requested Service Delivery Point (SDP), this could involve system interaction with agency switches or routers. Tasks agency IS personnel would be expected to perform would be limited to operation and configuration of GFE CPE.

Some of the tests, such as for video teleconference, do not involve separate test equipment. In those cases where separate test equipment is used, the specific equipment used will vary. Equipment utilized will always comply with best commercial practice for conduct of each test. However, specific make and manufacture of the equipment used to support a specific delivery will depend not only on delivery location, but also time within the contract, since Sprint is continuing to upgrade its systems and delivery methods.

6.2.1Testing of Telecommunications Services

Two types of testing will apply to Sprint’s telecommunications services:

Verification Testing. Verification tests are conducted by Sprint to verify the correct operational performance of all services and appropriate support system(s) prior to the transition, migration, or implementation of live traffic that uses these services or systems. Sprint will perform verification tests using X. The Government has the option to observe, or have a representative observe, all tests. Verification testing will be completed before acceptance testing begins. Verification Testing and the Verification Test Plans are discussed in Volume 2, Section B.2.6 of this proposal.
Acceptance Testing. Acceptance testing is a joint effort of the Government and Sprint. The purpose of acceptance tests is to demonstrate proper operation of service after Sprint’s transition, migration, or implementation activities are complete at a location. The acceptance tests will be conducted for 72 consecutive hours over three consecutive business days (Monday through Friday, excluding Federal holidays) and are intended to test and verify that performance is satisfactory and that all ordered features and functions operate properly and are accessible as required. Sprint will answer questions raised by Government observers and users and correct any deficiencies identified during the acceptance tests.

Many carriers and vendors, including the RBOC/LECs, interexchange carriers and specialized carriers such as fiber-optic service companies, provide local connections and may furnish facilities for SDP connection to the Sprint FTS2001 network. These facilities are provided according to Sprint’s agreements with X to meet FTS2001 requirements.

6.2.2Analog Facilities

Sprint will conduct the following tests on analog facilities to ensure that performance parameters for the service are met:

On dedicated analog facility connections (SDP to Sprint POP), standard tests will be conducted by the carrier/vendor, appropriate to the conditioning specifications ordered to ensure that the requested parameters are correct.

6.2.3Dedicated Digital Facilities

Dedicated digital facilities tests include X performance measurements as outlined in Sprint Network Practices and Procedures.

6.2.4Network Compatibility Testing

During the user location inventory, Sprint, the Government representative, and the GFE vendor will identify SDP interface parameters. Sprint will program this information into its switches during the service provisioning process to ensure switch software compatibility. Prior to access facilities being accepted and connected to the network and the SDP, Sprint will conduct a compatibility and verification test between GFE-vendor-provided equipment and the FTS2001 network. The test consists of:

accessing the FTS2001 network from a user station to verify signaling and address protocols and answer supervision, and
accessing the GFE (i.e., PBXs and Key Systems).
X These standard compatibility tests include the following:
X

Sprint will conduct compatibility testing while Sprint or its subcontractor personnel are at the SDP. If there are any incompatibilities, they are corrected by switch software changes within minutes. Transmission measurements will be performed by Sprint Network Operations using the Automatic Trunk Test function in the Network switches, a function that provides for trunk testing on demand by accessing a responder, attached temporarily or permanently to user GFE, in order to make appropriate transmission measurements, Sprint will provide responders.

6.2.5DS-1 Interface Testing

X Normal compatibility and call-through testing will verify that the interface works.

Switched Digital Service, Switched Digital Integrated Service, and ISDN all use similar testing, however verification tests for video, Frame Relay and ATM are in an evolving state. Sprint will develop specific plans for these services and incorporate them in the Transition/ Migration/ Implementation Test Plans.

Sprint will perform verification tests to validate the operational performance of all services at SDPs prior to the activation of live traffic using simulated traffic, as appropriate. Examples of test results to be provided are referenced in Section 2.6.

6.2.6ATM Testing

ATM testing, separate from any associated service and/or product delivery testing, consists of two components: Physical Layer and Logical layer.

Physical Layer. X
Logical Layer. X
X

6.2.7Video Acceptance Testing

Video Teleconference (VTC) system configuration an incorporate a wide range of options and features because customer-specific requirements and the proprietary nature of the related systems already in use.

All hardware and software required by the specific VTC system must be completed. X must be installed and tested prior to initiation of the acceptance test. X

No separate test equipment is generally required for conduct of this testing.

6.2.8Frame Relay Testing

6.3User Training

Sprint is committed to keeping customers informed and educated regarding products and services offered, product enhancements, and corporate direction. This commitment extends from the Account Manager to our corporate officers.

Sprint Network Training is usually divided into two phases—pre-service and post-service turn-up. Instructor-led training will be conducted by assigned Technical Applications Consultant (TAC) at a mutually agreed upon time prior to service cutover. During and after service turn-up, the TAC is available on an ongoing basis to answer questions, conduct periodic refresher training, provide additional manuals as needed, etc. All Sprint training is provided X.

Prior to service turn-up, the TAC will conduct a customized training class for designated personnel to discuss exactly how the Sprint service will operate. A thorough discussion of features and modifications required to update an agency’s corporate directory is presented.

Our experience has shown that large, multi-location clients derive the most benefit from a “train the trainer” concept where Sprint trains an in-house person or persons. Any staff members can be designated to serve as the in-house experts on the procedures. There is no limit set as to the number that may receive this training and its accompanying documentation, nor is there a limit to the number of training classes and training locations.

It is rarely necessary to do any face-to-face training of station users because of the ease-of-use of Sprint’s network. If a change in the use of the network affects station users, a memo describing the change and new procedures is usually all that is required. Sprint will gladly assist in the writing of any necessary training materials.

Because Sprint is responsible for all technical assistance and maintenance, specific training of your maintenance technicians is not required. However, since in-house technicians often become involved in any and all types of service problems, they are welcome to attend the training classes.

Training for non-dedicated customer locations, such as Dial 1 WATS locations, will be conducted as follows:

A proactive telephone call will be made by a Sprint Technical Applications Consultant three days prior to the install to discuss cutover procedures and answer questions.
After the cutover, follow-up proactive telephone calls will be made on a monthly basis to ensure customer satisfaction.

Training for customer VPN locations will be conducted as follows:

A proactive telephone call will be made by a Sprint Technical Applications Consultant three weeks prior to installation to discuss the implementation and operation.
On the date of installation, a Sprint Technical Applications Consultant will be available to train designated customer agency’s personnel.
After cutover, follow-up proactive telephone calls will be made on a regular basis to ensure customer satisfaction.

In both training scenarios, length of training will be determined by the customer agency’s location.

Our goal is to provide training in the manner and depth desired by our customers. Sprint will gladly work with staff members to ensure Agency/GSA’ssatisfaction.

6.4Acceptance Procedures

After being notified by Sprint that the service order is complete (Section C.3.5.4.5), the customer agency will begin acceptance testing. The Government will conduct acceptance tests using live traffic for up to 72 consecutive hours over three consecutive business days. The acceptance tests are intended to test and verify that service performance is satisfactory and that all ordered features and functions operate properly and are available and accessible as required. Sprint will answer questions raised by Government observers and users and correct any deficiencies identified during the acceptance testing period. Performance shall be considered satisfactory when service(s), and their associated features and functions, perform as specified in the contract. If problems are encountered relative to performance during the conduct of the acceptance tests, Sprint will work cooperatively with the LGC, the agency, other vendors, and other contractors (to the extent allowed by law) to isolate and eliminate problems between its network(s), system(s), and their connecting devices or facilities.

Acceptance testing is required of provisioned services only. Within three business days from the time the acceptance testing is successfully completed, the Government will notify Sprint via facsimile (FAX) and/or electronic mail (E-Mail) that the Government has successfully completed the acceptance testing. The Government’s notification to Sprint will be done by a Government employee cognizant of the results of the acceptance testing.

Acceptance of the service(s) will be effective immediately following successful completion of the acceptance testing. If during the 72hour test period, the acceptance test results are considered unacceptable by the Government, the Government may, at its option:

Withdraw the services from acceptance testing pending resolution of all unacceptable elements. For services being transitioned or migrated, the contractor shall immediately return the service(s) to their original network;
Schedule a restart of the acceptance test period;
Request a replacement of the service(s) (in whole or in part);
Terminate the service order; or
Initiate default proceedings.

Should the Government elect any of these options as a consequence of unacceptable acceptance test results, all expenses incurred by the Government in connection therewith, including recurring charges, will be borne by Sprint. In cases (a) and (b) above, Sprint will initiate corrective actions to remedy the problems.

In cases where the Government cannot successfully complete acceptance testing, Sprint will promptly notify the PMO and LGC (or Government representative) of the details and the steps that the contractor will take or has taken to correct the problem. Sprint will perform this notification within three business days after being advised by the Government that the acceptance testing was unsuccessful. In this case, the Government retains the five options (a) through (e), above. Sprint will work with the PMO and LGC to overcome the problem. If Sprint still cannot resolve the problem as a result of its effort, Sprint will immediately notify the CO and furnish all supporting documentation associated with the actions it has taken to remedy the problem. Sprint will provide this supporting documentation, in accordance with Section 3.2.7, within one business day after completion of its attempts to resolve the acceptance testing failure.

6.5Emergency Procedures

During meetings of the local cutover coordinating committees and during site surveys, Sprint will document any special agency-specific or site-specific requirements necessary to ensure service restoration in the event that a cutover is not successful. These special requirements will be documented in the site-specific cutover plans. The implementation of any contingency plans, whether normal or special, will be calculated to take no longer than four hours to implement during the cutover and acceptance period. These contingency plans will be fully maintained until service at the site has been accepted by the local agency, and the TCC/MCC is notified of cutover acceptance. Contingency plans will be immediately implemented by Sprint for any of the following reasons: