Helena NF – NEPA Contracting After Action Review

NEPA Contracting After Action Review Report
Helena National Forest
Warm Springs, Telegraph, Stone Dry, and Grassy Mountain Projects
First Notice to Proceed: November 7, 2009
Termination for Convenience: September 15, 2010

INTRODUCTION

NEPA contracting in general is often described by the Forest Service as necessary, but difficult at best. The experience that the Helena National Forest had was unique in some ways, but too familiar in others. This after action review was conducted in an effort to gain information for future projects where contracting is used to complete NEPA analysis.

This report was compiled based on interviews with staff from the Helena National Forest (referred to as “the Forest” from here out), as well as with staff from ERG (referred to as “the Contractor” from here out), the contractor who was awarded the contracts for the analysis of three EISs and one EA. The interviews with the Forest took place in late March 2011. The interview with the Contractor took place in early May 2011. The Contractor was interviewed while engaged in termination settlement negotiations with the Forest Service and has also submitted a FOIA for documents related to the analyses. This factor may have had an impact on the frankness and openness of the Contractor’s responses during the interview.

The after action review asked four questions of each party:

  1. What did we set out to do?
  2. What actually happened?
  3. What did we do well?
  4. What could we do better next time?

The Forest and the Contractor disagreed to some degree on the answers to each of the after action review questions that were asked in each interview. In an effort to remain objective, each party’s answers are described independently below.

FOREST INTERVIEW

What did the Forest set out to do?
The Forest entered into a series of four contracts with ERG, an environmental consulting contractor based in Missoula, MT. The contracts required the Contractor to conduct the environmental analysis for three EISs and one EA for a number of timberand salvage sales on the Forest. The sales were proposed in response to a mountain pine beetle outbreak that had led to high mortality rates on the Forest. Two of the projects were already on the Forest’s program of work; the other two were developed after it was decided to contract the analysis to an outside vendor.

The Forest reviewed the lessons that the Beaverhead-Deerlodge NF had learned when they contracted similar studies a few years ago. These lessons included some of the features the Forest carried into their Request for Proposals (RFP).

The Forest prepared a RFP with a detailed statement of work (SOW) for each contract/sale. Since GIS is a crucial element of an environmental analysis today, the Forest provided the GIS layers, files, and models to be used. Supporting information, checklists, monitoring reports, report templates and examples of the required reports were also included in the RFP – as well as a list of required resource/discipline skills and experiencefrom contractor personnel. The Forest was very deliberate about including the information they felt would give the Contractor the tools and direction they needed to successfully complete the EISs and EA.

The idea was to stagger the start of each contract until the Contractor had shown acceptable progress on the previous projects. The contractor was tasked with finalizing the Proposed Action based on the Purpose andNeed, writing specialist reports for affected resources, developing alternatives based on scoping commentsand writing a camera-ready DEIS.

The Forest also was deliberate in formalizing the communication paths between the contractor personnel and forest resource specialists. The Forest did this in order to control and document the communication, ensuring that no changes were made to timeframes, deliverables, or contract without the COR and CO’s awareness and approval. The Forest was specific about laying out the review and inspection of deliverables.The written reports had a review process that required the Contractor to submit a polished draft to the Forest for review; the Forest then had a specific amount of time to get its suggested/required edits back to the Contractor and a specific amount of time for the Contractor to make the required edits.

What actually happened?
The RFP package was sent to 3 vendors for proposals. The contracts were awarded to ERG based on their proposal and a series of interviews and presentations, all of which showed their firm to have the best technical approach and experience, combined with best value to the government. ERG showed a list of highly-qualified individuals assigned and sub-contracted to work on the project, including a former Forest Service wildlife biologist. They had also successfully completed a number of projects for the Forest Service.

Work began on the project and almost from the start the Contractor’s writing required significant editing and revision from the Forest. The Forest spent a lot of time reviewing the written deliverables, documenting the required edits and conducting subsequent reviews of the revised reports to ensure the required changes were made. In many cases the initial reports were missing large sections of information,meaning the subsequent report had material that had not been previously reviewed. At one point the wildlife specialist report was submitted by the Contractor for the Forest’s review, as laid out in the contract. The Forest reviewed and provided the Contractor with over 700 comments and suggestions for improving the report. When the Contractor sent the revised report back to the Forest, many of the changes the Forest had suggested were not made. The Forest asserts that part of the reason the wildlife report was insufficient was the fact that there were multiple biologists working on the report and they were located in different states As a result, the biologists did not communicate well-enough to provide a cohesive analysis that could stand up to professional and legal scrutiny.There was disagreement between the Contractor and the Forest about how many reviews and revisions took place. Budget and timeframe overruns began to creep into the process.

Adding to these problems, some of the Contractor’s GIS analysis had to be re-run and corrected by the Forest. Admittedly, the Forest discovered there were some mistakes in some of the data they gave the Contractor. When this happened, the Forest worked with the Contractor to revise timeframes for deliverables. There is some disagreement between the Contractor and the Forest on whether these renegotiated timeframes were sufficient to account for the “trickle-down” of impacts that errors in the written reports and GIS analysis had on subsequent deliverables.

Despite these initial setbacks, the Forest felt like the Contractor could succeed and worked with the Contractor to adjust timeframes for deliverables. The Contractor agreed to most of these adjustments without requiring modifications to the contract and budget. Based on the experience they had on the first project, the Contractorfelt they would make up the difference on future efficiencies they would gain on the subsequent 3 projects. The Forest visited the Contractor’s office in a final effort to salvage the efforts put into the contract to that point. The Forest gave the Contractor information about some resources that could help salvage the contract, including former Forest Service NEPA practitioners. The Contractor did not contact them, for reasons unknown.

After receiving numerous insufficient “camera-ready” DEISs, the Forest finally reached a point where they had spent too much time working on the project and revising the Contractor’s work; the Forest came to the conclusion that the contract had to be terminated.

During this after action review, drafts of the wildlife report and of the camera-ready DEIS were reviewed by a Regional Office (RO) NEPA specialist with expertise in NEPA litigation. The RO reviewer found that the wildlife report was incomplete and contained inconsistencies,errors and comments that are inappropriate for a NEPA analysis. For example, many important species that were mentioned in the proposal were not discussed in the environmental consequences section of the report, there were cases where totals in figures and in the text were different, and MIS, sensitive and threatened species were consistently misidentified. The report would not have been able to withstand review under appeal. There was a vast difference in the wildlife expertise demonstrated by the report compared to the wildlife expertise demonstrated by the Contractor’s proposal (Study Proposal). It is the opinion of the RO reviewer that the Contractor’s wildlife report was far from the quality that would be required for a contract where only one review is expected prior to finalization.

Additionally, the camera ready DEISs were not ready for publication. For example, in numerous places tables and table headers were split across pages, which is not a normal procedure for printed public documents. It did not appear to the RO reviewer that there had been any quality control reviews of these deliverable products that were submitted to the Forest.

What did we do well?
The Forest felt they did a number of things well and they would do some of the same things again. On the contract side:

  • The Forest felt they prepared a very specific RFP that gave the awarded contractor the tools they would need to successfully complete the analyses;
  • The scope of work was laid out specifically and in detailed fashion;
  • They selected a contractor after doing a lot of research and interviews. The Forest felt they had a good pool of contractors to solicit proposals from in the RFP;
  • They did not choose the cheapest vendor. Instead, they looked for best value to the government. Two tools added value to the contracting process: 1) the Forest negotiated a discount based on the award of all 4 contracts to one vendor and 2) the Forest promised the vendor payment incentives for work completed on time.
  • In the RFP, the Forest laid out exact points where reviews would be conducted on each deliverable; furthermore, the time the Forest had to conduct its review and submit the deliverable back to the contractor for corrections (if needed) were specifically laid out in a detailed table in the RFP;
  • The Forest felt they developed a clear line of communication where misinterpretation was limited, unintended contract modifications were avoided and communication was documented by the COR.

During the analysis phase, the Forest felt they also did a number of things well and would do them again in the future:

  • They followed their timeframes for review and return of documents that needed reworking;
  • They were respectful of the efforts the Contractor made, recognizing the process was difficult and complicated.
  • They worked with the Contractor to extend timeframes for delivery where the Forest had caused delay, such as when errors in Forest-supplied GIS layers became evident.
  • They provided the Contractor numerous examples of deliverables, templates for reports and other resources to ensure the best chances of success.
  • When it became clear that the Contractor was not in compliance with the contract, the Forest issued noncompliance notices;furthermore, instead of enabling the Contractor through the contract and three subsequent contracts to follow, the Forest cut their losses and terminated the contract.

What could we do better next time?
A number of different points came up in response to this question. Some of the ideas that were discussed reflected on how the Forest could work the same process better in the future.The main ideasthat are feasible from a government contracting perspective include:

  • The Forest felt they should require assurances from any contractor that the personnel and professional experience described in their proposal—which is perhaps some of the most crucial information that gets a contractor the award—are the people who work on the project through completion. The Forest felt they were promised the “A-Team” and then got the “B-Team”. This requirement was included in the contract but the contractor did not observe the requirement. The Forest said that such points in an RFP should be reinforcedand that aforest should immediately issue a notice of non-compliance in such cases in order to correct contractor behavior or performance. If that fails, then the Forest should take advantage of the financial or performance penalties which are contained in the RFP. This point could apply to any number of points/clauses in the contract, whether for quality of written documents or personnel commitments.
  • The Forest felt they could have conducted more intense and lengthy pre-work meetings. Additionally, the Forest recognized that perhaps the contract personnel should have been required to work with their resource counterpart on the Forest for one or two weeks at the beginning of the contract. This would have helped to ensure the contractor knew the information that was supplied to them by the Forest, understood the expectations and was familiar with the local conditions and context in which the projects and analysis occurred.
  • The Forest felt as if the roles of each person involved in the projects could have been clearer. From line officers to specialists, there could have been better lines of authority for who made which decisions. (The number of reviews and the people involved in the deliverables - as well as the roles of line, CO, COR and specialists - were some examples discussed).
  • The Forest said they should have been more direct in their feedback on deliverables. Suggestions for improvements in the documents should have been requirements and this terminology and specification should have been laid out in the RFP.
  • The Forest said they should have chosen a point in the process and “locked” the GIS information.The Forest made changes to GIS layers after delivering them to the Contractor and this had effects on other deliverables. At the same time, the Forest argued that Forest Service inventory and monitoring information—the ingredients of GIS data—are not static and the information always changes from the beginning of a project to the end.
  • The Forest felt they should sequence the specialist reports in the future, requiring the vegetation condition report to be complete before the wildlife report begins. This would reduce the trickle-down impact of data changes for subsequent specialist reports.
  • All contract requirements in the RFP need to be well defined and measurable in order to allow contractors to provide a realistic level of effort and cost to each task/deliverable in their proposal.

A number of points were discussed that centered around using different approaches and tools to complete contract work. Here are some of those main ideas:

  • Keep the GIS analysis in-house so that changes to data, models and interpretation of data could be better tracked and communicated to the Forest and the contract resource specialists.
  • Require periodic in-person meetings between the Contractor and the Forest staff to head off problems while they are still small.
  • Consider contracting only portions of the analysis, such as facilitating public involvement. The most complex pieces, such as the GIS and wildlife specialist reports, would be kept in-house.
  • Similarly, consider housing the contracted personnel on-location with the Forest resource counterpart. The contract personnel would still be working for the Contractor but progress would be monitored by both the Contractor and the Forest.

CONTRACTOR INTERVIEW

What did the contractor set out to do?
The Contractor set out to complete the analysesfor three EISs and one EA for vegetation projects on the Forest. They were contracted touse the best available science to write the specialist reports, prepare a final draft EIS and write the Record of Decision/Finding of No Significant Impact for each analysis. They were also responsible for facilitating the public involvement required for the process. All of this was to be done in a legally defensible manner.

What actually happened?
The Contractor said that almost from the beginning there were problems with the information they got from the Forest. Some of the templates and direction contained in the RFP conflicted with other resources they got from the Forest. The Contractor also said that some of the items the Forest said they would provide in the RFP did not appear in the Appendix of supporting materials.