APPENDIX ______

DATA ASSESSMENT AND REPORTING TEAM

Draft: November 25, 1998

I.INTRODUCTION

The CMARP Stage I report called for the development of a data analysis and reporting process that provides

“… technically sound, understandable reports released in a timely manner [to] provide the all-important feedback about monitoring results to managers, regulators, and stakeholders. Appropriate interpretation and display must accompany monitoring data. Annual monitoring reports are envisioned, which include both data analyses and interpretive graphs and text.”

[CMARP Stage I Report, April 24, 1998, p.20]

A vast array of data collection and analysis is occurring in the San Francisco Bay-Delta area and its associated watershed by federal and state agencies, universities, private institutions, scientists and technicians. The Comprehensive Monitoring, Assessment, and Research Program will build upon these existing efforts to provide CALFED with information needed to function in an adaptive management context and to provide assurances to the public and legislators about the success of CALFED actions. CMARP will facilitate making this information available to managers and to all interested parties in a meaningful and understandable format and will resolve those monitoring, analysis and reporting gaps which exist between the needs of CALFED and the information that is currently available.

Following is a discussion of the audiences for CMARP reports and their various information needs and the objectives and operating principles for a CMARP data assessment and reporting process. The remainder of this report is organized under the following headings: Information Gathering, CMARP Quality Assurance, Analysis & Integration, Reporting, and Examples. This report focuses on the various tasks that need to be accomplished and leaves the discussion of who will accomplish these tasks to Chapter VI – Institutional Structure of the CMARP Report. A further discussion on early implementation is found in Chapter VIII.

Audience for CMARP Reports

CMARP must meet the information needs of a wide and diverse set of people including CALFED Program Managers, the CALFED Policy Group, the CALFED Ops Group, CALFED Agencies, Scientists, Stakeholders, Legislative Staff, and the public. In general the level of detail desired by each group is expected to be different as shown in Figure 1. The process, therefore, must be both robust and flexible to address these diverse needs.

Figure 1. Level of Detail Desired by Different Audiences of CMARP Information and Reports (Note: While some stakeholders are expected to be interested mainly in basic summarized information about the system, other stakeholders are involved either in the actual collection of data or are very interested in information at all levels of the system. Consequently they are included at both levels of the diagram)

Public,

Stakeholders,public indicators

Legislators,

IncreasingCALFED Policy Group

Integration &

Summarization CALFED Program Managers,program indicators

Of Data IntoCALFED Ops Group,CALFED Agencies

Information

Scientists, CALFED Agency Staff, monitoring

Stakeholders, Regulatory agencieselements

Level of Detail Desireddata

Information needs of the three groups

The anticipated needs of each level of the triangle are summarized below.

The Public, Stakeholders, Legislators and the CALFED Policy Group (top of the triangle) are expected to be interested in questions about the “big picture” and less concerned with the details of monitoring and research. Primarily this group’s information needs are anticipated to be:

  • what actions has CALFED taken
  • are CALFED program goals and objectives being met
  • how are indicators of ecosystem health, water quality, water supply reliability, and levee system integrity doing
  • what new issues have arisen
  • what information has been learned that impacts Stage II implementation decisions
  • is the money being spent effectively
  • how does it affect the individual person
  • where can more detailed information be gotten,
  • how can concerns be made known.

Some of the needs of this group will have to be addressed through a joint effort between CALFED and CMARP – for example, in a joint annual report.

In addition to the above list, CALFED Program Managers, CALFED Ops Group and CALFED agencies (middle of the triangle) need much more information on which to make their decisions. Their additional information needs are anticipated to be:

  • specific information to base decisions upon
  • whether individual CALFED project/action goals and objectives being met
  • the status of those factors that influence indicators of valued system components
  • what adaptive management actions could be used to improve knowledge of the system
  • what uncertainties for managers have been removed through research
  • what level of confidence is attached to information and results
  • whether compliance and mitigation regulations are being met
  • computer models and geographic information system (GIS) as tools for decision-making, and
  • a forum to communicate with scientists.

Interviews with four of the CALFED Program Managers are given in sub-appendix ____.

Scientists, agency staff, and some stakeholders are at the base of the triangle and work with very detailed information. This group’s needs are anticipated to be:

  • to have research and monitoring results published in peer review journals rather than only in “grey” literature, i.e. technical reports.
  • general access to data, metadata and reports,
  • increased communication and collaboration with other researchers, stakeholders, and agency staff, and
  • a forum to communicate with managers.

Data Analysis and Reporting Objectives

The future tasks involved in meeting the information needs of these three groups include:

1) using selected indicators to assess the state of valued components of the system and determine if CALFED program goals and objectives are being met,

2) coordinating among existing programs to gather information to meet CALFED needs,

3) maintaining quality of data collection and analysis sufficient to meet CALFED needs,

4) coordinating additional data analyses needed which are currently not being performed by other programs,

5) providing a forum for exchange of information among groups leading to a regional monitoring focus

6) integrating information to improve understanding of the system and provide assistance for decision-makers, and

7) providing feedback on monitoring and research results in technically sound, understandable reports released in a timely manner to managers, regulators, and the public.

The steps for meeting the information needs of these groups include gathering of information, quality assurance, analysis and integration, and reporting. Figure 2 shows the role of CMARP in facilitating getting information to decision-makers. Each vertical step further compiles and integrates the information received from the step below.

Figure 2. Providing Information to Managers and Decision-Makers.

Management Decisions

Decision AnalysisCALFED w/ CMARP

Condensed Information

Regional AnalysisCMARP coordinates

And Integration

Information

Project Analysis,Individual Project

CMARP coordinates if necessary

Sample Data

Data CollectionIndividual Project

Real World

Figure 3 provides a more detailed conceptual model illustrating 1) the steps involved in collecting the different information involved and integrating them for decision-makers,

2) the feedback loop between CALFED and CMARP, and 3) the feedback loop within CMARP as new research and monitoring needs are identified and acted upon.

Data Assessment and Reporting Guiding Principles

To fulfill the objectives described above as effectively and efficiently as possible, the following guiding principles are recommended. CMARP should:

1) coordinate closely with CALFED program managers and agencies in order to be responsive to their scientific information needs.

2) use existing monitoring programs to meet CALFED needs whenever possible.

3) focus on having any new analyses that are needed for CALFED be conducted by the researchers or agencies actually collecting the data, to the extent feasible. This may require additional funding by CALFED. If the original researchers are not able to do the additional analyses needed, then they may be conducted under the direction of CALFED science staff, in collaboration with the original researchers.

4) strongly encourage publication of research, monitoring, and project results in peer-reviewed literature.

…Figure 3. Conceptual Model of Information Flow and Feedback Loops between CMARP and CALFED.

Information flow

Feedback LoopSupplemental Efforts of CMARP

5) make every effort to be an unencumbered channel of information flow between scientists and managers with strong effort made to avoid changes in purpose or content of reports and figures as they travel from scientists to managers. This will require close collaboration and feedback between CMARP and the researchers involved.

6) act as a communication bridge between scientists and managers -- working to get the information produced by scientists into the hands of managers in an understandable form, and working to help scientists better understand the needs of managers.

II. INFORMATION GATHERING

An important function of data management, assessment and reporting is facilitating the process of getting the overwhelming amount of information currently being generated about the CALFED Bay-Delta system into the hands of decision-makers. This involves compiling the results from monitoring of indicators, research programs, regional monitoring analyses, real-time monitoring data, permitting and regulation requirements, GIS efforts, and computer modeling efforts and delivering it to decision-makers in a manner that is accessible, timely and understandable.

Following is a discussion of the purposes of monitoring, the types of information CMARP will be gathering, and how coordination will proceed with existing monitoring programs.

Purposes of Monitoring

There are many ways of describing the various types of monitoring being conducted in the CALFED Bay-Delta system. For the purposes of this appendix, the three principal management purposes of monitoring in the CALFED system are described as (1) monitoring of management actions/projects, (2) monitoring for rapid decision-making, and (3) monitoring of the state of the system. This differs from the simpler definitions in the Scope section of the CMARP report, which focused on the more immediate purposes of monitoring (baseline, trend, effectiveness, compliance/mitigation, and operations monitoring).

Management Actions/ Project Monitoring

CALFED will be implementing specific projects/activities on both a small and large scale, which must be monitored for implementation status, performance/effectiveness and compliance and mitigation requirements. The frequency of such monitoring depends on the individual project. Typically projects which must meet compliance regulations (such as operations of the Central Valley Project (CVP) and the State Water Project (SWP)) will have more frequent measurements than other projects (such as a habitat restoration project).

- Implementation Status - How far has the project proceeded or what percent of the project has been implemented (e.g. % farms participating in using improved irrigation practices)

- Performance/Effectiveness –how effective the project is in meetings its stated goals and objectives. (e.g., improved farm water management practices resulted in a savings of X amount of water). Such evaluations require before/after project implementation monitoring which involves developing baselines and then monitoring for changes and trends. (See Example A & B at end of report)

- Compliance/Mitigation – reports to regulatory agencies to demonstrate the project/action is complying with regulations (e.g. water quality constraints or species take limits (see Example C)) or mitigating against actions (e.g. developing replacement habitat for habitat lost due to the project)

Rapid Decision-Making Monitoring

Some decision-making requires near-immediate information about the system. This typically involves frequent measurements and rapid turn around of data into information for decision-makers so that they can make their decisions. This monitoring information is used for the following purposes:

- Information to make decisions (e.g. flows, flood stage, position of x2) and/or determine when compliance regulations may be exceeded (e.g. position of salmon in system & current salmon “salvage” information (See Example C)).

- Early warning detection (e.g. changes in turbidity and increase in outflow signaling start of salmon migration (See Example C), or changes in benthic invertebrates communities signal water quality problems)

- Model forecasting & validation (e.g. highest flood stage river will reach at various points in system based upon current water coming down the river and water anticipated from rainfall)

- Develop short-term correlations & possible cause-effect relationships about system functioning (e.g. correlations between river flow and bromide concentrations in the delta within a single water year (See Example F)

State of the System Monitoring

Monitoring to assess the long-term trends and status of valued components of the system typically involves less-frequent measurements over a long period of time. However information gathered for rapid-decision making and for project monitoring can also be applied to system monitoring. This monitoring is used to:

- Assess status & trends of valued components of system and the factors that influence them (e.g. change in total salmon population over time) (See Examples B, D, E, G)

- Performance Monitoring – Large-scale effects of CALFED program in meeting stated goals & objectives (e.g. has channel meander been restored?, restoration of riparian habitat achieved?, does evidence support a peripheral canal or not?). This will often involve before/after CALFED implementation monitoring which requires developing baselines and assessing conditions before CALFED so that the changes attributable to CALFED can be quantified. (See Example B)

- Develop long-term correlations & possible cause-effect relationships about system functioning. (e.g. correlations between mysid population abundance, X2 position, and clam density over two decades (See Examples B, D, E, and G))

Types of Information Gathered

The types of information that will be gathered from monitoring programs and research projects fall into four general categories: reports, regulatory information, metadata, and data.

Reports from current monitoring programs

As stated under the guiding principles, CMARP will be using information from existing monitoring programs whenever possible. This will involve coordinating with existing monitoring program managers to get copies of their reports and facilitate getting those reports into the hands of CALFED decision-makers as quickly as possible. A systematic process for coordination of collection of reports will be developed as well as a tracking system for all reports and information moving through the CMARP process. This process will become more clearly developed as CMARP moves closer to implementation.

Reports from research programs

The data assessment and reporting process will also be facilitating getting information from research projects into the hands of decision-makers. A systematic method of keeping track of the large amount of research projects being funded by the CALFED process will be developed together with their associated reports.

Regulations and Permitting Requirements

The data assessment and reporting process will gather together information on the permitting requirements and regulations in order to facilitate reporting to program managers about compliance and mitigation monitoring, to increase communication and understanding among the program managers and to make the process of getting necessary permits easier and more straightforward.

Create metadatabases of monitoring programs, research efforts and computer models

The amount of monitoring, research and computer modeling efforts being conducted in the CALFED Bay-Delta system is so large it is difficult for any one person to know even a fraction of the information available. Over 600 monitoring programs have been identified. In order to avoid duplication of effort, reduce the costs involved in providing information to CALFED, and improve coordination among agencies and researchers, CMARP is building a metadatabase of monitoring programs in the CALFED Bay-Delta system and associated watersheds (see Chapter __). This database forms the basis for determining what data are available and how they could contribute to broader CALFED goals.

Three additional metadatabases are also recommended for development involving: 1) larger research efforts related to CALFED, 2) computer modeling efforts related to CALFED, and 3) GIS efforts related to CALFED. These metadatabases will be accessible to the public via the CALFED/CMARP web page.

CMARP Database

Not all information required by CMARP will be available in current reports. A CMARP database management system is being constructed into which data of interest to CALFED program managers and stakeholders and especially data that will be used in regional monitoring analyses and GIS development will be compiled. This is developed in greater detail in Appendix ______.

Coordination with Monitoring Programs and Research Efforts

Coordination with Individual Programs

The CMARP work teams have made significant progress towards identifying the important ongoing monitoring efforts or new monitoring efforts needed for a comprehensive monitoring program. They have identified over 500 monitoring elements, which are listed in Table 2 attached as a sub-appendix to this report. In order to accomplish the monitoring required to meet CALFED program objectives, CMARP will need 1) to coordinate with existing monitoring programs and 2) arrange for additional supplementary monitoring.

For existing monitoring recommended by the CMARP work teams, CMARP will coordinate with the programs involved to get copies of reports needed by CALFED program managers and copies of the data where it is needed for further regional analyses. Further discussion of quality assurance continues in the next section.

There are gaps between what existing monitoring programs cover and the monitoring needs of CALFED. Some recommended monitoring elements are not being monitored at all. Some monitoring efforts require expansion. CMARP will determine which of these gaps in monitoring are of highest priority and fund new monitoring as needed.

Regional Coordination

An important function of CMARP is to provide a regional focus to monitoring and research in the San Francisco Bay-Delta area and its associated watersheds. Once the CMARP work team recommendations are finalized, CMARP will go through the monitoring recommendations and identify where regional coordination of monitoring efforts are required to provide the information needed by CALFED. The problems of spatial and timing gaps and consistency of protocols associated with regional monitoring are discussed more fully in the next section under quality assurance.