Adjusting for Optimism Bias in Decent Homes Standard Investment Programmes

Guidance Note for LSVTs and ALMOs

housing

Adjusting for Optimism Bias in Decent Homes Standard Investment Programmes Guidance Note for Large Scale Voluntary Transfers and Arms Length Management Organisations

March 2007

Department for Communities and Local Government: London

On 5th May 2006 the responsibilities of the Office of the Deputy Prime Minister (ODPM) transferred to the Department for

Communities and Local Government.

Department for Communities and Local Government

Eland House Bressenden Place London SW1E 5DU

Telephone: 020 7944 4400

Website: www.communities.gov.uk

Copyright in the typographical arrangement rests with the Crown.

This publication, excluding logos, may be reproduced free of charge in any format or medium for research, private study or for internal circulation within an organisation. This is subject to it being reproduced accurately and not used in a misleading context. The material must be acknowledged as Crown Copyright and the title of the publication specified.

Any other use of the contents of this publication would require a copyright licence. Please apply for a Click-Use Licence for core material at www.opsi.gov.uk/click-use/system/online/pLogin.asp or by writing to the Office of Public Sector Information, Information Policy Team, St Clements House, 2-16 Colegate, Norwich NR3 1BQ.

Fax: 01603 723000 or email: .

If you require this publication in an alternative format please email

Communities and Local Government Publications

PO Box 236

Wetherby

West Yorkshire

LS23 7NB

Tel: 0870 1226 236

Fax: 0870 1226 237

Textphone: 0870 1207 405

Email:

or online via the Communities and Local Government website: www.communities.gov.uk

Printed in the UK on material containing no less than 75% post-consumer waste.

March 2007

Product Code 06HC04380

CONTENTS

Chapter 1 Purpose 4

Chapter 2 Introduction 5

Chapter 3 Why does optimism bias arise? 6

Chapter 4 Optimism bias evidence base 7

Chapter 5 When to consider optimism bias 8

Chapter 6 Using numerical indicators in project decision-making 10

Chapter 7 Sensitivity analysis 11

Chapter 8 Separation of optimism bias and financial cost development 12

Appendix 1 Glossary of abbreviations 13

Appendix 2 Calculating optimism bias on works duration and capital costs 14

Appendix 3 Mitigation of optimism bias 20

Appendix 5 References and sources of further guidance 22

Chapter 1. Purpose

1.1 The purpose of this Guidance Note is to help Large Scale Voluntary Transfers (LSVT) and Arms Length Management Organisation (ALMO) senior managers and project staff understand:

(i) what optimism bias is;

(ii) how to assess it in Decent Homes projects; and

(iii) to provide information about the techniques that can be used to assess the sensitivity of numerical indicators to optimism bias.

1.2 The Guidance Note considers:

· optimism bias;

· optimism bias evidence base;

· when to consider optimism bias;

· proportionality;

· Decent Homes Standards indicators; and

· financial approvals.

Chapter 2. Introduction

2.1 HM Treasury’s Green Book (2003) introduced the concept of optimism bias and required that it should be taken into account in project appraisal and evaluation. This requirement is picked up in Communities and Local Government’s 3Rs1 but has yet to be applied consistently across the social housing sector and specifically to projects supporting investment in housing stock to the Decent Homes Standard2 (DHS).

2.2 It is recognised that risk and optimism bias are closely linked and that good risk management can reduce optimism bias. However, as guidance on project risk identification, assessment and management is provided elsewhere3 and is not covered in this Guidance Note.

2.3 Optimism bias is the term used to describe the demonstrated, systematic tendency for project appraisers to be overly optimistic about project costs, duration and benefits

(outputs and receipts/income). In other words it is the systematic tendency to view things in an overly positive light. It can arise in relation to any aspect of a project but it particularly applies in Decent Homes Standard programmes for:

· costs (capital and revenue); and

· works duration.

2.4 To redress this tendency HM Treasury’s Supplementary Guidance on Optimism Bias4 recommends that project appraisers should make explicit adjustments to the estimates of project costs, benefits and duration based on empirical data to inform project decisions. Adjusting for optimism bias should help organisations to answer key questions such as:

· How much information do we have on each individual housing unit?

· How reliable and up to date is this information?

· Have we accurately estimated the project cost?

· What are the impacts on cost and time overruns?

2.5 The Guidance Note has been developed to specifically support ALMO and LSVT organisations. Each has to support investment in its housing stock to reach the minimum DHS by 2010/11. It is recognised that some organisations will be investing to a level above the DHS, however the optimism bias drivers will remain consistent.

1 Assessing the Impacts of Spatial Interventions – Regeneration, Renewal & Regional Development – The 3Rs Guidance: ODPM (2004)

2 A Decent Home: The definition and guidance for implementation: Communities and Local Government (2006)

3 HM Treasury Green Book Ch5 and Annex 4; Orange Book and Risk website http://www.hm-treasury.gov.uk/documents/

public_spending_and_services/risk/pss_risk_index.cfm; Office Government Commerce Successful Delivery Toolkit – Management of Risk

4 http://www.hm-treasury.gov.uk/media/885/68/GreenBook_optimism_bias.pdf

Chapter 3. Why does optimism bias arise?

3.1 Projects are inherently risky and the risks increase with the size and complexity of the project. Studies have found that there are deep-seated causes of optimism bias and that they can be broadly split into two categories: technical and institutional.

Technical

· risks and uncertainties associated with forecasting costs, incomes etc;

· changes in project scope; and

· poor project management.

Institutional

· the desire to see projects happen;

· institutional pressures; and

· the decision-making process.

3.2 In one influential study, the authors argued that “political-institutional factors have in the past created a climate where only a few actors have a direct interest in avoiding optimism bias.”5 However, it is the norm in projects, rather than the exception, that unplanned events do occur and experienced project managers should consider the effect of these when appraising projects.

3.3 The challenge for social housing providers is therefore to:

· make explicit allowance for optimism bias in appraisal in a proportional and cost effective way;

· consider whether a project represents value for money for the organisation once an allowance for optimism bias is included; and

· be aware of and work to reduce the causes of both technical and institutional optimism bias.

5 DfT (2004) Procedures for Dealing with Optimism Bias in Transport Planning, Bent Flyvbjerg in association with COWI.

Chapter 4. Optimism bias evidence base

4.1 This Guidance Note is based on research commissioned by ODPM6. The work directly examined 47 Decent Homes Standard (DHS) packages of work undertaken by LSVTs

and ALMOs. It also considered the emerging findings of the BCIS study which examined

1,250 residential facility projects. It further considered wider-scale evaluation studies and involved a series of practitioner consultations and workshops.

4.2 The Guidance Note provides tailored estimates of optimism bias uplifts for project costs and duration for all LSVT and ALMO DHS projects.

4.3 The Guidance Note also provides advice on accounting for optimism bias when assessing DHS projects. The Guidance Note follows the standard approaches outlined by the 3Rs.

6 The responsibilities of the Office of the Deputy Prime Minister (ODPM) were taken over by the newly created Department for

Communities and Local Government on 5 May 2006.

Chapter 5. When to consider optimism bias

Project appraisal

5.1 The development of the investment programme to meet the Decent Home Standard

(DHS) will start with the existing information base, most typically developed through a stock condition survey. This will inform the first planning stage of developing DHS projects. The second phase is delivery. Each stage will experience both time and cost optimism bias drivers.

5.2 In a two-stage appraisal process, it is important to be aware that optimism bias may be greatest at the initial appraisal stage before the project costs, timescales and benefits have been fully worked out.

5.3 At the initial project assessment stage the organisation’s focus is primarily with the project’s ability to meet investment standards (including the DHS), impact on the overall investment/DHS programme, strategic fit and funding availability.

5.4 Few projects are rejected simply on cost/benefit grounds at this stage. Consequently it may be useful to apply the Upper Bound allowances for optimism bias (see Appendix 2

Table 1) to understand whether a project still appears acceptable when these are applied. The appraiser should be clear what steps are to be taken during the full appraisal to mitigate the risk of optimism bias, for example, market research, reviews of business plans, independent assessment of capital costs, etc.

5.5 At the full project appraisal stage, optimism bias should be formally considered in respect of costs, duration and benefits. Appraisals should consider whether:

· a project represents value for money for the organisation once allowance for optimism bias is included; and

· if the inclusion of an allowance for optimism bias affects the relative performance of the project or different DHS investment options.

5.6 A specific driver for DHS projects is that, unlike say regeneration, rejecting projects due to unacceptable optimism bias upper bound allowances may not be feasible. All homes must meet the standard by 2010/11. Rather, the Guidance Note will support the consideration of optimism bias at both the planning and delivery stages and inform risk mitigation strategies.

5.7 If after the application of optimism bias, the project appears to highlight an unacceptable level of uncertainty, then approval should be withheld, or given on a qualified basis, eg requiring further research, costing and risk management.

Monitoring

5.8 Once the organisation has committed funds to the DHS project it should monitor its delivery using the optimism bias factors to guide data gathering and risk management.

Adjusting for Optimism Bias in Decent Homes Standard Investment Programmes

Thus if the project had to come back to the organisation’s decision makers for a further approval they would expect to see evidence that the most likely cost estimate had been adjusted, to reflect this developing experience.

Evaluation

5.9 Post completion, when the project is evaluated, the evaluators should compare the actual and forecast outturn costs, timings, outputs and outcomes to enhance the empirical evidence base of optimism bias factors to inform future appraisals.

Proportionality

5.10 The application of the optimism bias approach via comparators, risk analysis and specific uplifts can become rather complex. It should therefore be applied proportionately eg the DTI’s Single Programme Appraisal Guidance says that:

‘For large complex and high risk projects including those outwith their financial delegations [organisations] should show that they have given full consideration to all the key requirements … [but that] … for smaller simpler less risky projects

[organisations] still need to have addressed all the appraisal requirements but in less detail’.

Chapter 6. Using numerical indicators in project decision-making

6.1 The tools outlined in the rest of the Guidance Note relate to the numerical and financial elements of project proposals/business plans and appraisals. It is therefore important to understand which numerical indicators are important when making a decision about a Decent Homes Standard investment project.

6.2 Having established which financial indicators are most important to the decision to support the project, analysis can be undertaken on how sensitive these are to change.

Chapter 7. Sensitivity analysis

7.1 The assessment of optimism bias provides an input to sensitivity analysis. Once we understand the potential level of optimism bias in the forecasts on a project, we can test a project to see whether it still represents a reasonable investment if the optimism factors were to arise.

7.2 Sensitivity analysis tests appraisal conclusions in the face of uncertainty and optimism bias. It will enable the organisation to:

· identify the key influencing factors in a Decent Homes Standard project;

· provide likely outcomes for certain scenarios;

· inform project and option selection; and

· feed into the project risk management process.

Chapter 8. Separation of optimism bias and financial cost development

8.1 The project’s financial costs should include a sum for reasonable contingencies but should not include the optimism bias mark-up. The purpose of considering optimism bias costs is to assess the value for money of the project. Adding them onto the project financial costs would risk a disincentive effect in effectively managing the project delivery.

8.2 Optimism bias and the sensitivity analysis information should be provided for decision makers to inform them of the consequences should these risk factors occur and the implications for the project’s value for money. Decisions as to whether the project is value for money and should be financially supported by the organisation can then be made in the knowledge of these factors. Organisations should approve Decent Homes Standard investment project financial costs excluding the optimism bias costs.

8.3 If changes are required to the contracted costs, duration or output of the project during its delivery these should be managed using the organisation’s project management procedures. Organisations should review requests for further funding or other contract changes against the optimism bias and switch values to test whether the project would still have offered value for money when it was originally approved. When these projects complete then the organisation can review the actual optimism bias that transpired, incorporate it into its analysis and use it to update its local optimism bias look up tables.

Adjusting for Optimism Bias in Decent Homes Standard Investment Programmes

Appendix 1. Glossary of abbreviations

3Rs – Assessing the Impacts of Spatial Interventions: Regeneration, Renewal and

Regional Development

ALMO – Arms Length Management Organisation

BCIS – Building Cost Information Service

DHS – Decent Homes Standard

DTI – Department of Trade and Industry

LSVT – Large Scale Voluntary Transfer

Adjusting for Optimism Bias in Decent Homes Standard Investment Programmes