Pubwatch Evaluation Toolkit: Collaboration between National Pubwatch and Leeds Beckett University

Page 2

Contents

1.0 What is a Pubwatch Scheme? 1

2.0 Why should we evaluate our Pubwatch Scheme? 1

Figure 1: The Evaluation Cycle 1

2.1 What does evaluation help us do? 2

2.2 What are the aims of a Local Pubwatch? 2

Table 1: Pubwatch Scheme: Objectives to Evaluate 2

3.0 When should Pubwatch evaluations take place? 2

Figure 2: Step-by-Step Guide - Baseline Evaluation Study: for New Pubwatch Schemes 2

Table 2: Evaluation Toolkits Provided 2

4.0 What type of data should I collect? 2

Table 3: Examples of the types of data you can collect 2

5.0 Further Help? 2

The Portman Group, on behalf of the Social Responsibility Alliance, provided the funding for Leeds Beckett University to develop the evaluation toolkit in partnership with the schemes

Page 2

1.0  What is a Pubwatch Scheme?

NationalPubwatchbelieves that a well-run and effectivePubwatchScheme promotes the licensing objectives and furthers the interests of its members, by helping to create a safe and secure drinking environment and reducing alcohol-related crime.

There is increasing evidence that a well-runLocal PubwatchScheme can have a positive impact on the area in which it functions, helping to create a safer and more secure drinking environment and reducing crime and disorder. This is plainly of benefit not only to members of the drinking public, but also to the licensees themselves, who have an interest in maintaining orderly premises which are pleasant for customers to visit. It perhaps goes without saying that areas or premises with a bad reputation tend to be shunned. A person who visits a pub and ends up being the victim of crime will six months later remember only that negative experience, no matter how many other safe and enjoyable evenings he/she has had in the same location. It stands to reason that removing troublemakers can be of enormous benefit to licensed premises in that area, by fostering a general atmosphere of safety and encouraging others to come to that area for their night out.

Local Pubwatch Schemes are set up and run by licensees to help reduce crime and disorder in pubs, clubs and bars. It is well known that Pubwatch Schemes help to reduce violence and other criminal acts such as drug dealing and vandalism in and around licensed premises. Indeed many local Pubwatch co-ordinators put their hands on their hearts and say “My Pubwatch Scheme works”. But not all co-ordinators have hard evidence to show that their Pubwatch Scheme definitely works.

This Pubwatch Evaluation Toolkit provides you with information, processes and ways to provide evidence that your Local Pubwatch Scheme has helped to reduce alcohol-related crime and disorder in your town “after dark”* and to increase the feeling of safety for employees and consumers.

2.0  Why should we evaluate our Pubwatch Scheme?

Evaluation enables you to measure how well your Pubwatch Scheme is going, if it is successful, if its objectives have been met and of course what improvements can be made to the Scheme.

Evaluation, therefore, is a process that involves planning, collecting and analysing information to provide insight and knowledge. See Figure 1 for details of the Evaluation Cycle.

During the Report and Share stage of the Evaluation Cycle it is important to share findings with licensees, key stakeholders and the general public so they know if your Pubwatch Scheme is contributing to improving the safety, security and well-being of staff and customers in your town.

*“After-dark” is defined as between 6pm and 6am.

2.1 What does evaluation help us do?

·  find out if your Local Pubwatch Scheme’s aims and objectives have been met

·  understand what went well and what could be improved

·  justify continuation of your Pubwatch Scheme for years to come

·  show how and why your “Pubwatch Scheme works”

2.2 What are the aims of a Local Pubwatch?

The overall aim of a Local Pubwatch Scheme is “to help create a safe and secure drinking environment and reduce alcohol-related crime”. To help you collect the right data that demonstrates your Pubwatch Scheme “is working” a range of objectives to evaluate are provided. See Table 1 below. Instructions and Evaluation Tools to help you collect the relevant data are shown in Appendix 1 – 3.

Table 1: Pubwatch Scheme: Objectives to Evaluate

Objectives to Evaluate:
Pubwatch will...
Reduce alcohol-related crime, disorder and anti-social behaviour in [include details of town/area]
Reduce alcohol-related health harm
Build positive perceptions of the Local Pubwatch Scheme
Build positive relationships between licensees and other invited guests present at Pubwatch meetings (invited guests include trade, police, local authorities and/or other agencies)
Improve data sharing information about the night-time economy*
Improve business performance during the night-time economy*

*Night-time economy is “after-dark”, defined as between 6pm and 6am.

If your Pubwatch Scheme achieves the objectives shown in Table 1, you will be able to say... “My Pubwatch Scheme works and here is the evidence to prove it.”

3.0  When should Pubwatch evaluations take place?

Evaluations are not one-off activities. Evaluations should be an integral part of your Pubwatch Scheme. The first Evaluation Study is entitled the Baseline Evaluation Study. This Study helps you understand licensees and key stakeholder expectations of Pubwatch and the level of alcohol related crime and disorder in your town. It can also help you understand if your consumers feel safe and secure in your town. Figure 2 below shows the basic steps for the Baseline Evaluation Study.

Figure 2: Step-by-Step Guide - Baseline Evaluation Study: for New Pubwatch Schemes

As stated, Evaluation Studies are not one-off activities. Therefore, after the Baseline Study, further Evaluation Studies are needed to help understand your Pubwatch Scheme’s progress and impact. See Table 2 below.

Table 2: Evaluation Toolkits Provided

Toolkit
/
When?
/
Why?
Essential Baseline Evaluation Study Tools 1
Appendix 1
/
After the first Pubwatch meeting
/
To provide an information base against which to monitor and measure your Pubwatch Scheme’s progress

Essential Follow-up Evaluation Study

Tools 2

Appendix 2

/

One year after Baseline Study

/

To compare Follow-up findings with Baseline Findings. The comparison of findings will indicate if your Pubwatch has achieved its objectives.

Essential Year-On-Year Evaluation Study

Tools 3

Appendix 3

/

Continuously each year

/

To evaluate Pubwatch continuously Year-On-Year. Compare Year-On-Year Findings with Baseline findings to find out is your Pubwatch objectives continue to be achieved.

Additional Suggestions 4

Appendix 4

/

Continuously each year

/

To support Essential findings.

All Evaluation Tools are provided for you in the Appendices.

4.0 What type of data should I collect?

Data for evaluation and reporting can be collected from primary sources and secondary data sources.

Primary data means data collected directly by you specifically to measure the success of your Local Pubwatch Scheme. Possible ways to collect this data include interviews and surveys. Primary data will help you understand the general opinions that licensees and key stakeholders have of your Pubwatch. Primary data will also aid your understanding of how consumers of the night-time economy perceive your town. The Evaluation Tools are provided for you in the Appendices; including discussion guides and questionnaires.

Secondary data means data collected by a third party. The third party collects data and provides you with information relevant to your Pubwatch Scheme. Possible secondary data sources include your Community Safety Partnership/Crime and Disorder Reduction Partnership regarding alcohol related Public Order Offences, A&E admissions, or town centre footfall data; there may be a fee for some data.

The Tools in the Appendices provide you with instructions and templates to use to collect both primary and secondary data - see Table 3 for examples.

Table 3: Examples of the types of data you can collect

Type of Data / Examples
Essential Primary Data -collected by you / Interviews with key stakeholders

Question: “Do you think Pubwatch will help reduce crime and disorder, such as underage drinking, thefts, violence and/or drug crime in your town?”

Licensee Expectation Survey

Q.12 In the last 6 months, to what extent have the following been a problem in YOUR LOCAL AREA?

(please tick one box on each line) /

Major

Problem

/

Moderate

Problem

/

Slight

Problem

/

No problem

at all

/

Don’t

Know

A) Underage drinking

/ o / o / o / o / o
Essential Secondary Data – collected by a third party / Template requesting Public Order Act Offences

E.g. Public Order Act Offences- Section 5, Section 4 and section 3

Thefts (e.g. Purse, handbag, mobile phones etc.)

Criminal damage (e.g. Buildings, cars etc.)

Drunk and disorderly

Directions to leave licensed premises (ASB Act, formally section 27s)

Bonus Secondary Data – collected by third parties / Footfall data
National Crime Data

Note: You may wish to get additional help from a your local college/university, local authority, primary care trust, voluntary groups, local alcohol reduction teams and so on to help with these evaluations. Such support can add to a shared vision of an evaluation and the possible benefits of a project.

5.0 Further Help?

Should you have any questions about collecting, evaluating and reporting your findings, or indeed any other issues affecting your Pubwatch Scheme, contact us at

See Appendices for Evaluation Tools

Appendix 1 Essential Baseline Evaluation Study Tools 1

Appendix 2 Essential Follow-up Evaluation Study Tools 2

Appendix 3 Essential Year-On-Year Evaluation Study Tools 3

Appendix 4 Additional Suggestions 4

Page 5