Department of Information Technology IV&V Guidelines
Independent Verification and Validation Guidelines
June 14, 2010
Quality Assurance Bureau
Project Oversight and Compliance Division
Department of Information Technology
About This Document
This document is intended for New Mexico state agencies to prepare IV&V information technology (IT) professional services contracts with appropriate IV&V deliverables and IV&V vendors to prepare IV&V reports.
Independent verification and validation (IV&V) is required for all certified IT projects except those for which IV&V is waived by the Secretary of the Department of Information Technology (DoIT).
This document:
1. Covers IV&V as part of the DoIT mandate to ensure adequate risk management for information technology projects.
2. Clarifies the IV&V procurement process.
3. Reviews templates that serve as the foundation for the IV&V review of certified projects.
4. Covers agency and IV&V vendor requirements.
Revision History
Revision Number / Date / Comment12 / June 14, 2010 / Quality Assurance Bureau initial publication
Table of Contents
About This Document 2
Revision History 2
background and DoIT authority 4
Project Certification and IV&V 4
IV&V Definitions 4
Verification 5
Validation 5
Independent 5
Independent Verification and Validation 5
IT Projects Must Follow Product Development Life Cycle 5
Certification Process 6
Project certification phases and implementation phase reporting 7
Lead Agency and IV&V Vendor responsibilities 7
Lead Agency IV&V Responsibilities by Project Certification Phase 7
IV&V Procurement and Contract Process 8
IV&V Scope of Work 9
Report Submission 12
Appendix A 13
Project Deliverables Task Items / Reporting Topics 13
Appendix B 27
IV&V Reporting Template 27
background and DoIT authority
The DoIT Secretary’s July 5, 2007 IT Project Oversight Process Memorandum (Project Oversight Memo) is a restatement of 1.12.5.2 NMAC which was repealed September 30, 2005.
The authority for this memorandum is contained in the Department of Information Technology Act , Chapter 9, Article 27 NMSA 1978, which states that the secretary shall:
“Provide oversight of information technology projects, including ensuring adequate risk management, disaster recovery and business continuity practices and monitoring compliance with strategies recommended by the information technology commission for information technology projects that impact multiple agencies.”
The Project Oversight Memo delineates agency responsibilities for following state project management methodology, including IV&V, and expectations for IV&V vendor reporting.
Project Certification and IV&V
The project certification and IV&V processes are DoIT risk management tools which rely heavily on strong project management.
• Project management is risk mitigation. It is about how we create and manage a temporary organization to deliver a unique product, service or result.
• Project management reduces the risk of counterproductive chaos by providing structure to the temporary organization and the solution development/deployment process.
• Acknowledging and communicating risks to stakeholders keeps them in the loop, and often involves them in risk mitigation or reducing impact.
Project Certification provides risk management through controlled release of project funds. Agencies are required to provide evidence of adequate planning through identification of business requirements and project organizational structure to ensure project success and responsible expenditure of state federal funds.
The IV&V process reduces project failure through identification and mitigation of risks which are possible roadblocks to success. Projects may have unique risks and issues; however, common risks are undocumented processes and procedures, or not adhering to them when documented.
Verification and validation by independent entities is fundamental to DoIT’s responsibility to “provide oversight of information technology projects, including ensuring adequate risk management.”
IV&V Definitions
Consistent with oversight responsibilities of the DoIT Project Oversight and Compliance Division the following define independent review of a project, its process and its deliverables.
Verification
“Verification” means the project is adhering to project management disciplines, planned and performed according to its project plans and that such adherence can be verified by an independent examination of project documents and other evidence.
Validation
“Validation” means that the project deliverables and project results meet the business and technical objectives established by the project sponsors, ensuring that the end product meets the documented performance outcomes and requirements of the project.
Independent
“Independent” means autonomous and impartial verification and validation assessment of a project’s adherence to project management plans and compliance with business requirements. These independent assessments are performed by an entity that is not responsible for developing the product or performing the activity being evaluated.
Independent Verification and Validation
“Independent verification and validation (IV&V)” is the means of obtaining an independent and objective view of an IT project with the intent of protecting the state of New Mexico’s interests, and is focused on the management of the project and its compliance with specified requirements through its development stages.
IT Projects Must Follow Product Development Life Cycle
The Project Oversight Memo establishes the requirement for a product or solutions development life cycle approach to state IT projects and mandates its use as the basis of agency and IV&V reporting:
· “Product development life cycle” is a series of phases comprised of iterative disciplines such as requirements, analysis and design, implementation, test and deployment implemented to build a product or develop a service.
· “During the project management lifecycle, agencies shall select and implement a phased product development lifecycle methodology approved by the Department.”
· “Lead Agency shall: Prepare a written risk assessment report at the inception of a project and at the end of each product development lifecycle phase or more frequently for large and high-risk projects. “
· “IV&V Reporting: Prepare interim reports based on the phases as indicated within the project schedule. Included in the report will be an evaluation on whether product development requirements are being met, project management is effective, continuing risk analysis, and how the project is implementing previous recommended risk mitigation strategies.
Product Development life cycle approval
Agencies must present the project’s product development life cycle in Section 4.2 of the Department of Information Technology project management plan template.
When the version of the Project Management Plan containing the product development life cycle is accepted by the Project Oversight and Compliance Division, that life cycle is approved.
It is expected that the IV&V vendor will follow the phases presented by the agency as the basis for monthly and development phase assessments.
Certification Process
The IT certification process establishes basic requirements for each phase in order to recommend release of funds commensurate with project status. The Initiation, Planning, and Closeout phases within the product development life cycle are synonymous with their corresponding phases in the certification process. The remaining phases of any approved product development life cycle are contained in the Implementation Phase of the certification process.
The Project Initiation and Planning phases of the certification process address the organization of the project around business and technical objectives. The focus is on the governance structures, the budget and schedule, as well as the potential impact on the state’s information technology infrastructure – in particular any impact on the DoIT. It may also include development of business requirements.
The Implementation phase, using an approved product development life cycle, may include the elaboration of business and technical requirements; a system design that evolves from these requirements; building, testing and accepting the solution, deployment into production and transfer to operations.
Project certification phases and implementation phase reporting
The illustration below shows the product development life cycle using a traditional implementation methodology as the structure for the implementation phase of the certification process. The implementation methodology is defined in the agency’s Project Management Plan and approved by the Department of Information Technology.
Lead Agency and IV&V Vendor responsibilities
The lead agency on a multi-agency project is the agency with fiscal accountability for the project or the agency designated lead by legislation, usually the General Appropriation Act. The majority of IT projects are deployed by a single agency, which is the lead by default.
Lead Agency IV&V Responsibilities by Project Certification Phase
The lead agency is accountable for quality strategies, including IV&V, in compliance with DoIT best practices and standards.
Initiation Phase – Project Charter included with certification request indicates IV&V plans
Planning Phase – Project Management Plan indicates IV&V planning and procurement approach
Implementation Phase – Executed IV&V contract included with certification request and periodic IV&V reports received for the duration per that contract
Closeout Phase – Final IV&V report, preferably a post-implementation report, must be submitted, specifically addressing any open items
IV&V Procurement and Contract Process
It is important to consider the likelihood of an IV&V engagement requiring more than one year or ultimately costing over $200,000 when considering the appropriate procurement method. Consultation with the agency’s DoIT IT Business Consultant prior to embarking on a specific approach is highly recommended.
The General Services Department State Purchasing Division maintains price agreements with responsive vendors for IV&V services under $200,000 and lasting no more than 365 days. If there is any possibility that either of these limitations will be exceeded during project execution – by taking into consideration the not uncommon possiblity of project setbacks, delays, changes in scope and schedule, etc., an RFP procurement provides the most flexibility for changes to scope and total cost of IV&V services warranted by anticipated, and unanticipated, implementation delays and changes.
IV&V services are required to be maintained continuously throughout project implementation regardless of setbacks, delays and price agreement limitations, underscoring the need to include adequate time for IV&V procurement in the project schedule and vigilance by the project director, manager(s), steering committee(s) and executive sponsor(s) to ensure successful procurement planning and navigation of procurement intricacies and processes.
All IV&V contracts must include:
· language that requires submission of all IV&V deliverables to DoIT to a location designated by DoIT; and
· a report format consistent with the current DoIT IV&V reporting template.
When an IV&V contract is fully executed, i.e. all signatories have signed, an electronic copy of the executed contract must be emailed to a location designated by DoIT.
The scope of work contained in IV&V contracts executed under a price agreement may be extended beyond one year if:
· sole source documentation submitted with a new contract with the same IV&V vendor is approved; and
· any additional activities and costs do not incur total project lifetime costs for IV&V services to exceed the $200,000 limit; and
· the new contract scope of work does not duplicate the original contract scope of work.
Price agreement IV&V services may be modified by amendment as long as the amendment does not extend the term beyond 365 days nor increase the total amount of the contract above $200,000. IV&V services procured by RFP may be amended at any time without these limitations, depending on term status of the contract under the RFP, i.e. the contract hasn’t expired.
IV&V Scope of Work
Deliverables should be tailored to the project in its entirety, which means tailored to the project risk, project scope and project budget. The deliverables listed below are high level suggestions. Project deliverables and task items may also include reporting topics in Appendix A.
Small Projects – Scope of work may include COTS Applications: Installation, Configuration and Migration. IV&V Project Management Plan, Conduct Initial Review, Conduct Periodic Review(s), Planning Oversight, Project Management, Quality Management, Training, Requirements Management and System and Acceptance Testing.
Medium Projects – Scope of work may include COTS Applications: Installation, Modification, Conversion, Implementation, Testing and Multi-Stakeholders. IV&V Project Management Plan, Conduct Initial Review, Conduct Periodic Review(s), Planning Oversight, Project Management, Quality Management, Training, Requirements Management, System and Acceptance Testing, and Data Management.
Large Projects – Scope of work may include New Development, Requirements Gathering, Architecture Design, Implementation, Conversion, Testing, Deployment, Multi-Stakeholders, and Multiple Agencies. IV&V Project Management Plan, Conduct Initial Review, Conduct Periodic Review(s), Planning Oversight, Project Management, Quality Management, Training, Requirements Management, Operating Environment, Development Environment, Software Development, System and Acceptance Testing, Data Management and Operations Oversight.
All projects subject to IV&V activities must have at a minimum the following IV&V activities:
· IV&V project management plan
· initial assessment
· initial status reports
· interim project progress
· ongoing risk analysis
· deliverable forecast
DoIT may require other specific deliverables for a project depending on the risk of the project.
Upon acceptance of the IV&V project management plan deliverables, the next report is usually an initial assessment of the project.
During the course of the project at prescribed intervals, the IV&V vendor shall prepare interim project progress reports as follows: It is highly recommended that the IV&V vendor conduct a post-implementation assessment after the project has been in production for several months to report whether business and technical objectives were achieved based on project scope and acceptance criteria. This report, or the final IV&V report, is submitted with the request to certify project close-out. If the report contains any outstanding issues, the agency must address them in writing.
Report Submission
The IV&V vendor shall submit all reports to the agency contractual recipients, agency executive steering committee and the Department of Information Technology Project Oversight and Compliance Division () within five (5) business days of each deliverable due date as indicated in the IV&V contract.