Contact details

Clear Horizon Contact / Client contact
Dr Bridget Roberts
Senior Consultant
Clear Horizon Consulting
129 Chestnut St, Cremorne VIC 3121
P: (03) 9425 7777
E: / Yvette Clarke
Acting Manager, Monitoring and Evaluation, Koori Justice Unit
Department of Justice and Regulation
Level 18, 121 Exhibition Street, Melbourne VIC 3000, Australia
P: (03) 8684 1744
E:

Document review and authorisation

Version / Date distributed / Issued to: / Comments
V1 / 6 April / Kathryn Vardy
Ada Peterson
V2 / 2 May / Yvette Clarke
Mary Morison
Evaluator and author / Bridget Roberts
Clear Horizon Reference Number / CH14_160

Disclaimer

This document has been produced with information supplied to Clear Horizon by the Department of Justice and Regulation, including performance reports. While we make every effort to ensure the accuracy of the information contained in this report, any judgements as to suitability of the information for the client’s purposes are the client’s responsibility. Clear Horizon extends no warranties and assumes no responsibility as to the suitability of this information or for the consequences of its use.

CH14_160 MDAS Report V2Clear Horizon Consulting1

Contents

Executive Summary

1.Introduction

1.1.Project context, activities and rationale

1.2.Evaluation consultancy scope

1.3.Purpose of report

1.4.Report structure

2.Methodology

2.1.Overview

2.2.Key evaluation questions

2.3.Data sources, methods and tools

2.4.Reporting

3.Findings

3.1.Brief description of the project

3.2.The activities: delivery, effectiveness and outcomes

3.3.Overall effectiveness

3.4.Outcomes overall

3.5.Evaluation support

3.6.Lessons learned

4.Appendices

4.1.Timeline of FCVPP activities

4.2.Draft FVPP program logic, March 2014

4.3.Clear Horizon data collection tools

List of figures

Figure 1: Family and community violence prevention project - Draft program logic

List of tables

Table 1: FVPP timeline of activities 2013-2016

Acronyms

ACCO / Aboriginal community controlled organisation
DOJR / Department of Justice and Regulation (Victoria)
FVPLS / Family Violence Prevention and Legal Service
FCVPP / Family and Community Violence Prevention Project
LAJAC / Local Aboriginal Justice Advisory Committee
MDAS / Mallee District Aboriginal Services
MOU / Memorandum of Understanding

CH14_160 MDAS Report V2Clear Horizon Consulting1

Executive Summary

Background

The Koori Community Safety Grants Program is an initiative of the Department of Justice and Regulation (DOJR) Crime Prevention Unit and managed by the Koori Justice Unit (KJU). It recognised that strengthening Aboriginal culture was important in addressing some of the drivers of family and community violence.Aboriginal Community Controlled Organisations(ACCOs) were invited to use their unique expertise and knowledge to tender for work that met local needs for prevention of family and community violence. This report relates to The Family and Community Violence Prevention Project was delivered by Mallee District Aboriginal Services (MDAS).

The evaluation consultancy and support sought to assist the ACCOs and government to identify project outcomes, identify ‘promising practice’ project elements that could be applied elsewhere, document lessons learned and provide recommendations for future project sustainability.

The evaluation consultancy adopted a collaborative, approach. Evaluation data were drawn from documents (project performance reports, publicity material, facilitators’ reports of specific activities), two training workshops and interviews with stakeholders.

Key messages

Delivery and outcomes

We find that project effectiveness, efficiency and value were remarkable given the barriers of distance in large regions, the challenges of building and sustaining partnerships over a three-year period and the sensitivity of the topic of family violence. The project delivered a substantial proportion of the funded family violence prevention activities. There is evidence that the project have reached the right people in culturally competent ways that were an adequate fit with existing community resources and services. The cultural competence of the project design is evident in the use of a community-wide (and individual community-specific) approach to awareness raising that recognised that family violence is everyone’s business, across families, kinship networks and communities.

The evidence of short and medium-term project outcomes, although sparse, aligns with expectations. The overarching program logic model for the KCSG Program identified intended outcomes as follows:

  • Short term intended outcomes were changes in awareness, knowledge and understanding of violence, individual behaviour and the services available.
  • These would then lead in the medium term to individuals and communities taking actions such as resolving interpersonal conflict non-violently, making use of support services and informal support, or (in the case of local organisations) providing new or more appropriate services for the community.
  • In the longer term, it was expected that individuals would have stronger self esteem and resilience, have more respectful, positive and equitable relationships, be less socially isolated and be better able to refrain from and respond to violent behaviour. Communities would see greater use of supports and services. Organisations would offer more coordinated responses to community conflict and violenceand more appropriate services.
Good practice elements applicable elsewhere
  1. The community media campaign

All MDAS informants highly valued the community media campaign. Key elements worth considering elsewhere were co-development of the messages and stories with community members, the use of local actors and a professional production team, and opportunities for community members to develop skills in design, production and performance. Finally, MDAS engaged a local marketing consultant to support a community attitude survey, with responses collected by the project coordinator and a small team of helpers (for the pre-survey) in three communities face to face before and after the commercials were broadcast. MDAS found additional funds for repeat broadcasts: a budgeting point worth noting if the idea is taken up by other Aboriginal organisations.

  1. Sisters Day Out and Dilly Bag: The Journey

Sister Day Out is a package for women that combines pampering and wellbeing activities with information sharing about and referral to relevant supports and services. Dilly Bag: The Journey is a small-group two or three day experience for vulnerable women, focusing on pride in culture, exploration of identity and resilience. These two events have become known and accepted in the communities where they have been delivered. Not only are they potentially applicable in other communities, but in the Mallee they hold their relevance when repeated, attracting new as well as second-time participants.

  1. Use of consultants

MDAS partnered with the Family Violence Prevention and Legal Service and Kellawan Consultants to deliver Sisters Day Out and Dilly Bag: The Journey. They also sub-contracted DardiMunwurro to run youth camps. These arrangements meant that the consultants’ specialist skills and knowledge could complement local knowledge and skills. As noted in the following section, MDAS there are indications of further developments in the way consultants are used.

Lessons learned

  • Sisters Day Out, Dilly Bag and the community media campaign stand out as models that may be applicable to other organisations.
  • Resourcing: Although the program was explicitly time limited and focused on learning what worked in local community family violence prevention within the three years, many informants thought that the program should have included some support for continuing activities after the funded period.
  • The Project Coordinator learned that delivery of the multi-faceted program of activities needed the help of a project officer. The achievements relied on many hours of voluntary effort.
  • The project steering group is keen to continue to build local capacity in each Mallee community and rely less on external input, so that action is more attuned to local needs and interests and better able to build cultural strength and pride, as well as being more cost-effective. For example, MDAS sees potential in supporting their two-man team to continue the preventive work with young men, drawing on DardiMunwurro mentorship. After reflecting on the partnership with DardiMunwurro, MDAS is negotiating an MOU that relates to delivering work in Swan Hill and Kerang that includes a train-the trainer component with follow-up mentoring. It is hoped that this approach will foster local self determination.
  • In view of the ongoing challenge of mobilising community members, the project steering group has reconfirmed that the IFVRAG is a key mechanism for building community respect and confidence and driving further prevention and early intervention activity.
  • The Project Coordinator has learned, through working on this project, of particular gaps in the family violence response in Swan Hill and Kerang. Plans include annual local Sisters Day Out events, with targeted follow-up based on participant feedback. While pampering activities attract women in a non-threatening way, she is keen to make sure that the day provides real and safe opportunities to connect with needed services.
  • The project steering group has learned that there is an ongoing need for MDAS to become family violence capable. As well as building a Family Violence team, the Project Coordinator (now manager of the team) will continue to deliver education and training across all MDAS services, and ensure the staff are well qualified and policies are updated.
  • Project governance and partnerships needed renewal after the establishment phase: the Steering Committee support lost momentum after the first year of the three-year grant period.

CH14_160 MDAS Report V2Clear Horizon Consulting1

1.Introduction

1.1.Project context, activities and rationale

Context

The Family and Community Violence Prevention Project was a multi-faceted approach to addressing community safety in the areas of Mildura, Robinvale and Swan Hill.

Project funding under the Koori Community Safety Grants Program (an initiative of the DOJR Crime Prevention Unit) was allocated to contribute to the societal goals of reducing family violence, breaking the cycle and reducing intergenerational impacts of family violence, increasing community safety and reducing the need for child protection and family violence interventions. The Program recognised that strengthening Aboriginal culture was important in addressing some of the drivers of family and community violence. ACCOs were invited to use their unique expertise and knowledge to tender for work that met local needs.

Family violence in the Mallee is above the State average and a high proportion of all crime committed is family violence related.[1] Community perceptions, surveyed in late 2013 and early 2016, were that alcohol and other drug availability was a key contributor to the problem (51% in 2016). Abuse (verbal, physical and sexual) were considered[2] to be problems across all communities involved. Unemployment and boredom were believed to be other contributors to the problem.

Activities

The project comprised a range of activities, delivered for the Mallee district in Mildura, Swan Hill and Robinvale:

  • A community wide media campaign
  • Sisters Day Out Workshops for all Aboriginal women
  • Dilly Bag Women’s Behavioural Change Programs for small groups of Aboriginal women;
  • DardiMunwurro Male Youth Behaviour Change Programs
  • Early Years Cultural Safety Workshops for young parents
  • Cultural safety training program.

.

CH14_160 MDAS Report V2Clear Horizon Consulting1

1.2.Evaluation consultancy scope

The evaluation consultancy sought to identify the extent to which intended and unintended project outcomes had been achieved, identify ‘promising practice’ project elements that could be applied elsewhere, document lessons learned and provide recommendations for future project sustainability.In commissioning the evaluators, DOJR representatives understood that there would be greater focus on process than on project outcome.

1.3.Purpose of report

The report is intended to assist the project team, community and government tobuild evidence of what works to prevent family and community violence in VictorianAboriginal communities and to strengthen monitoring and evaluation.

1.4.Report structure

This introduction has briefly described the project and its rationale and the scope and purpose of the evaluation. The next section (Section 2) is a summary of the evaluation method, including the Key Evaluation Questions and data sources, methods and tools. Section 3 outlines the findings overall and by activity, against the Key Evaluation Questions.

Technical details, including data collection tools, are attached as Appendices.

2.Methodology

2.1.Overview

The evaluation adopted a collaborative, mixed-method approach to exploring the effectiveness, sustainability and value of the FCVPP project and the success of the evaluation support. Key questions and sub-questions structured the inquiry, which built on previous work by the Koorie Justice Unit to develop, with MDAS, a program logic and a monitoring and evaluation framework, and to complete an interim process evaluation.

Evaluation data was drawn from documents (project performance reports, facilitators’ reports of specific activities), a training workshop (seven participants) and interviews with seven stakeholders.

2.2.Key evaluation questions

Key evaluation questions (KEQS) outlined in the project-specific monitoring and evaluation framework (developed in 2014by KJU and MDAS) are as follows:

  1. How effective was the project in delivering what was needed to the right people at the right time in the right ways?
  1. How well does the project fit with, complement, make use of and build upon existing community strengths, resources and services?
  2. How much did the project incorporate best practice elements relevant to similar projects and contexts?
  1. What outcomes were achieved, for whom, and how worthwhile were they?
  1. How well did the project and its activities work to address the underlying causes of Koori violence?
  2. How well did the project build individual, community and organisational capability and capacity?
  3. How cost effective, efficient and worthwhile was the project overall?
  1. How successful was the evaluation support?
  2. How did the evaluation support relationship develop?
  3. What worked well and less well in the evaluation training?
  4. What were the lessons learned?
  1. What were the lessons learned (by the project and organisational staff and the evaluators)?
  2. What lessons may be applicable to other organisations?

2.3.Data sources, methods and tools

The project methodology built on the program logic, monitoring and evaluation framework and data collection toolsalready completed by MDAS and the KJU. Clear Horizon reviewed program documentation in order to understand the project background and progress. Senior consultants visited MDAS on three occasions to plan Clear Horizon’s involvement, deliver training, support internal evaluation activities and conduct independent interviews with project staff, organisational staff and other key stakeholders.Data from project documents and staff and stakeholder interviews was triangulated to develop a comprehensive account of project performance.

2.4.Reporting

Clear Horizon drafted a report for each project to answer the key evaluation questionsand capture stories of change and lessons learned. We sought feedback from MDAS and KJU before finalising this report.

3.Findings

3.1.Brief description of the project

Funding and scope

MDAS received $560,000 (excluding GST) to deliver activities between March 2013 and November 2015in Mildura, Robinvale, Ouyen, Kerang, Red Cliffs, Irymple and Swan Hill. The intent was to deliver a suite of family violence prevention activities, working with the community as a whole, women of all ages, parents expecting or with young children, young men aged 14-17 years, and the staff of MDAS.

Program logic

The project was seen to work through three pathways[3]: changes at the community level, at the individual level and in relevant organisations. In the long term, the expectation was that the project activities would contribute to

  • the community
  • increasingly using supports and services
  • Aboriginal people having
  • a stronger connection to, and respect for, culture
  • increased social support networks
  • more respectful relationships (children, parents, Elders)
  • improved attitudes towards gender roles, equity, gender and family violence
  • Aboriginal women
  • feeling supported
  • making positive choices about their lives
  • Services
  • practising better organisational responses to family/ community violence
  • meeting community need better and more appropriately.

Project activities[4]

Project activities comprised:

  • A community wide media campaign
  • Sisters Day Out Workshops
  • Dilly Bag Women’s Behavioural Change Programs
  • Young men’s camps
  • Early Years Cultural Safety Workshops for young parents
  • Cultural safety training for MDAS staff

Governance and operations

MDAS convened a project Steering Committee with 25 representatives of a range of relevant organisations. A Project Coordinator was appointed in April 2013. She saw the project through to completion in April 2016, apart from a period of sick leave that was covered by a temporary coordinator.

The Project Coordinator reported that coordinating the project steering group and communicating with consultants who were working state-wide was a constant challenge that led her well beyond her funded part-time role.

Project delivery compared with plans

MDAS delivered the activities substantially as planned. The main variations of which there is evidence were (a) in the Early Years activity, where a less formal method replaced a formal one day cultural training program and (b) in the youth behavioural change program, where MDAS conducted the third and fourth camp programs without DardiMunwurro. In terms of the number of events, all but one activity reached its target (five of six intended Dilly Bag programs were delivered.)