Evaluation of the
National Environmental Research Program (NERP)
On behalf of the Department of the Environment
Vista Advisory Project Team
Caroline Spencer (lead evaluator)
Peter McVay
Sue Sheridan
10 January 2014
Table of Contents
Executive Summary 3
1. Background 8
1.1 Introduction 8
1.2 Program Aim and Objective 8
1.3 Evaluation methodology 11
2. Analysis of NERP’s impact 14
2.1 To what extent is NERP meeting its objectives? 14
2.2 Areas of portfolio impact – case studies 17
2.2.1 Operational Management (National Parks) 17
2.2.2 Regulation 26
2.2.3 Program design and delivery 30
2.3 What has NERP added to the environmental knowledge base?...... 36
2.4 Final note on program impact...... 37
3 – Analysis of the NERP delivery model 39
3.1 Hub Model 39
3.2 Co-investment 41
3.3 Program Establishment 42
3.4 Collaboration 47
3.5 Capacity- building 48
3.6 Knowledge broking and communications 49
3.7 Program Administration 55
3.8 Emerging Priorities 60
Appendix A – What has NERP added to the environmental knowledge base? 63
Appendix B – NERP Timeline 66
Appendix C – List of People consulted 67
Appendix D - Survey Approach 72
Appendix E – Visual Explanation of NERP enabling and delivery strategies 74
Appendix F – Hub Administration Costs 75
Appendix G - Number of interactions between hubs and end-users 76
Appendix H – Administrative gaps addressed since CERF. 78
Executive Summary
Background
The National Environmental Research Program (NERP) has been designed to inform decision makers on how best to respond to environmental challenges in Australia. The objective is to ‘improve our capacity to understand, manage and conserve Australia’s unique biodiversity and ecosystems through the generation of world-class research and its delivery to Australian environmental decision makers and other stakeholders’.
NERP is intended to provide a direct pathway between policy makers and researchers, to inform the management and sustainable use of the environment. NERP has a particular focus on supporting biodiversity conservation by delivering research to the Australian Government, other end-users and stakeholders that will improve the understanding of how ecosystems function, their health, resilience, sustainable use and how market mechanisms can promote biodiversity conservation.
At the time of the evaluation (December 2013) NERP had provided some $86.2 million[1] in funding over three years to five research Hubs and for eighteen emerging priorities projects. The five Hubs are the:
· Environmental Decisions Hub;
· Landscapes and Policy Hub;
· Marine Biodiversity Hub;
· Northern Australia Hub; and
· Tropical Ecosystems Hub.
Some 134 projects have been funded under NERP (including Emerging Priority projects). While only 18 projects have been fully completed[2], substantial progress has been made on many of the incomplete projects. On this basis, the evaluation has been able to draw reasonable conclusions as to the effectiveness and efficiency of the program.
Evaluation Approach
This evaluation focuses predominantly on whether the program is achieving its objectives in relation to informing Environment portfolio end-users. The evaluation team acknowledges the impact the program is having with stakeholders outside the portfolio, and identifies areas that it considers could be improved to maximise impact and benefit to other stakeholder groups in the future. The approach adopted by the evaluation team involved:
· a desktop analysis of departmental records and reports;
· evaluation surveys sent to Hub participants (509), Environment portfolio end-users (113) and other end-users and stakeholders (122), with an overall response rate of 38 per cent[3];
· interviews or roundtable discussions with 72 Hub participants and other stakeholders; and
· interviews or roundtable discussions with 52 Environment portfolio staff within the department, Great Barrier Reef Marine Park Authority and Parks Australia.
Conclusion
This evaluation found that the program has been effective in meeting its objective in several key areas, most notably in informing national park planning and operations, and also across several high profile Environmental Protection and Biodiversity Conservation Act 1999 (EPBC Act) responsibilities. The survey undertaken as part of the evaluation identified that a significant proportion of end-users, both in the Environment portfolio and other stakeholders, consider that the following have been the three major achievements from the NERP:
· improved capacity of decision-makers, policy developers and environmental managers to connect with researchers and make the most of research outputs;
· improved capacity of researchers to meet environmental decision-makers’ needs; and
· improved knowledge of biodiversity or the functioning of ecosystems.
The survey results from Environment portfolio end-users indicated that 97 per cent strongly agreed that NERP is improving Australia’s knowledge base (understanding) of biodiversity and ecosystems. Eighty-six per cent of Environment portfolio end-user respondents strongly agreed that NERP is undertaking research that is relevant and useful to the department. Numerous examples were provided to the evaluation team that illustrated the success of the program. NERP has been particularly successful in assisting national park managers to improve their management practices and address strategic threats to biodiversity in both terrestrial and marine environments. Parks Australia commented that NERP has ‘changed our thinking and the way we manage our parks’ by ‘giving managers confidence in the implementation of their strategies knowing that they have drawn on the best available science and management practices and used robust decision making tools’.
Similarly, the Great Barrier Reef Marine Park Authority (GBRMPA, also within the Environment portfolio) commented on the pivotal role that NERP has played in addressing research gaps and providing evidence to more effectively address water quality challenges in the in-shore waters of the reef. In a resource constrained business environment, this has been particularly important for GBRMPA to enhance its capacity to more effectively respond to international pressures to address the status of the World Heritage Listing of the Great Barrier Reef, and for Parks Australia to address threats in Kakadu and elsewhere and to manage the significant expansion of marine reserves in Commonwealth waters.
At a landscape scale outside of national parks, NERP has provided advice and integrated management approaches to more effectively manage biodiversity conservation. This has been particularly evident through the work of the Hubs in supporting emerging policies and practices under the EPBC Act, most notably in Strategic Assessments and in facilitating Regional Sustainability Planning. The partnerships developed between researchers and the department have been very productive in these areas. Facilitating credible and more efficient protection of matters of national environmental significance is of increasing importance to the department with the move to a ‘one-stop-shop’ approach and reducing green-tape and improving certainty for proponents seeking approvals under state and Commonwealth environmental legislation. Hub researchers also refined the department’s offsets policy and calculator to improve the tools available to the department in the administration of the EPBC Act. In the Australian Alps, NERP research has also assisted both Commonwealth and state and territory governments in the management of threats to biodiversity values at a landscape scale. For example, research has demonstrated that cattle grazing in Alpine areas does not reduce fire risk and that there are cost effective tools for prioritising and managing threats from invasive species.
The administration of the program was generally commented on favourably by researchers and Hub participants. Seventy per cent of researchers considered that the department had managed the program reasonably or well. Through the administration of five main hub contracts[4], the department is receiving research outputs from over 600 researchers for the delivery of 134 projects of direct relevance to the Environment portfolio. Co-investment by partner organisations has almost doubled the total available funding for research under the program to $154.6 million.
In addition, while much of the benefit of collaboration and capacity-building in the program is intangible, evidence from this evaluation shows that collaboration between policy and science is occurring in an active and robust way, albeit with opportunities to improve[5]. The on-call access to world-leading researchers for ad-hoc advice as well as research delivery provides unquestionable capacity enhancement to the department and its portfolio agencies. There is also evidence that significant value is derived from the improved capacity of researchers to understand and meaningfully contribute to portfolio priorities and processes as well as the wider policy agenda of governments. These aspects provide evidence of the effectiveness and efficiency of the program delivery model, including the basis of the hub model when compared with other approaches such as procuring ad-hoc research projects as needs arise.
The information and communication role of the program is very important for the dissemination of research results. It requires close contact and establishing relationships between researchers and departmental officers. There have been substantial efforts within the department and the Hubs to establish connections between researchers and Environment portfolio line areas to promote understanding and uptake of NERP research, and to ensure it meets environmental decision-makers’ needs. Many of these initiatives are not widely used in other research programs[6] and in many cases cited by survey respondents have been effective in NERP. Strong evidence of the Hubs’ initiatives and genuine desire to deliver research to the department and its portfolio agencies that is relevant, useful and timely was identified through this evaluation. The department’s NERP team facilitates regular seminars, workshops, summer scholarships and promotes formal and informal linkages between researchers and Environment portfolio officers.
Nevertheless, survey results and interviews have also highlighted some missed opportunities and areas for improvement in the program’s implementation. Over half of portfolio end-users considered that there were opportunities where NERP research could have been used in Portfolio decision making and policy but was not. Timing was the major challenge although a lack of awareness of NERP within the Portfolio was a significant issue contributing to missed opportunities. Staff turnover within the department and the absence of dedicated communication/brokerage staff within the administration team for NERP has detracted from the dissemination of research results. The absence of business support systems to manage and disseminate research results means that there have been examples where the results from quality projects have not been disseminated to the department’s executive or the Minister at all or in a timely way. This highlights the importance of strengthening science communication as a dedicated core function within program implementation in the future.
Furthermore, the challenge for the department, in balancing administrative requirements with the task of effectively brokering and communicating science research outputs for maximum impact across the Portfolio will increase as the current round of program funding nears completion in 2015. NERP’s 18 completed projects and 116 projects in-progress have produced over 350 peer-reviewed publications to date. The expected ‘tidal wave’ of publications[7] that will reach the department over the next 1-2 years will need to be managed efficiently, and captured and disseminated properly, to maximise the benefits to the department from the NERP investment.
The gap between the commencement of NERP and the predecessor CERF program was regarded as a ‘disaster’ by research organisations because of the loss of quality research staff. The focus on Commonwealth end-users has led to some disenfranchisement of industry and NRM groups that benefitted from the earlier CERF program. The balance between administration and science communication by the department was highlighted as an issue with too much focus on administration or changing financial requirements often cited as detracting from efforts to deliver quality and timely research. In addition, while recognising the challenges of a third party administrator operating on behalf of the Commonwealth, the heavy commitment of departmental staff time and resources to administration in the Tropical Ecosystems Hub was also disproportional to the actual risks involved and a more streamlined arrangement should be considered in any future program.
The strengths of the emerging priority funding stream within the program are significant as they provide the Minister and department with the flexibility to meet priorities not originally foreshadowed in the Multi-year Research Plans within the Hubs. The projects on research into the declared commercial fishing activity (‘super trawler’) and the early intervention options for the crown-of-thorns starfish on the Great Barrier Reef are examples where the department was able to respond in a timely manner to identified issues as they emerged. However, the emerging priority project selection and approval process has tended to occur late in each financial year. A more structured and timely process, including one which draws on the expertise of trusted hub leaders to help identify and shape emerging research priorities, and build on best available knowledge and research approaches, would further enhance the value derived from the emerging priority funding stream.
Nevertheless, the outcomes emerging from NERP to date are very positive and have materially contributed to the understanding, management and conservation of Australia’s unique biodiversity and ecosystems. Australian environmental decision-makers and other stakeholders are now better positioned to improve the delivery of their core business operations because of the direct pathways created between policy makers and researchers through NERP.
Acknowledgements
Vista Advisory gratefully acknowledges the open engagement and generous assistance of all program stakeholders in undertaking this evaluation, including:
· Hub researchers, leaders, administrators and knowledge brokers for their frank engagement and responsive input into the evaluation, including extensive and considered survey responses and interview comments;
· Environment portfolio, including departmental, Great Barrier Reef Marine Park Authority (GBRMPA) and Parks Australia, officers and senior executives for their frank engagement and responsive input into the evaluation, including extensive and considered survey responses and interview comments;
· End-users and potential end-users outside the Environment portfolio (including industry groups, traditional owners, state government officials, amongst others) for their frank engagement and responsive input into the evaluation, including extensive and considered survey responses and interview comments including invaluable perspectives on the use and potential of the program; and
· the Sustainability Research and Science Policy Section of the Department of the Environment (‘the NERP team’) for their frank engagement and responsive input into the evaluation, including identifying opportunities for improvement and assistance with compilation and analysis of information.