Strategic Project on VET Provision: Continuing Professional Development of VET Teachers

Strategic Project on VET Provision: Continuing Professional Development of VET Teachers and Trainer:

Supporting the Agency for Professional Development of Teachers in Serbia (ZAVOD)

Project Number: PROVQ WP 16-40-18-PROV

Deliverable 5-2.1

EVALUATION of WORKSHOP 2

MODULE 2:

Educational Leadership for Training of Trainers

Prepared by

Univ. Prof Dr Viktor Jakupec

December 2016 / January 2017

Contents

Evaluation of responses 3

Introduction 3

Background Information 3

Demographic Data 3

Response Rate 3

Questionnaire Data and Design 3

Assessment of Responses 3

Findings 4

Quantitative Data 4

Responses to Relevance for Participants’ Daily Duties and Responsibilities 5

Responses to Topics to be covered in more Detail 6

Qualitative Data 7

Appendix 1: Evaluation Questionnaire 8

Appendix 2: Combined gender disaggregated raw date and weighted average scores 10

Responses to General Module Delivery and Structure 11

Responses to Relevance of Topics to the Daily Business Tasks of Participants 12

Responses to Topics perceived to need more detailed coverage 13

Evaluation of responses

Introduction

This is the evaluations of responses from participants taking part in the second 2nd workshop for ZAVOD staff and other associated institutions and is based on the delivery of:

MODULE 2: Educational Leadership for Training of Trainers

MODULE 2: was designed on the basis of the needs analyses responses obtained from ZAVOD’s National Professional Development Coordinators (NPDC).

The Feedback questionnaire was deigned on basis of the needs/demands and frequency/importance analyses conducted in aid of module content.

Background Information

The MODULE 2: Educational Leadership for Training of Trainers Workshop was conducted on 29th and 30th November in Belgrade

The feedback questionnaire was administered at the completion of the Workshop

Demographic Data

Number of participants (N) / Female
(nf) / Male
(nm)
15 / 8 / 7

Response Rate

Number of maximum possible returns (questionnaires) / Female (F)
total questionnaires
received / Male (M)
total questionnaires
received / Total
questionnaires
received
(F+M)
15
(100%) / 8
(53.33%) / 7
(46.66%) / 15
(100%)

Questionnaire Data and Design

The questionnaire (see Appendix 1)

Generated gender disaggregated data
Is based on the Likert Scaling Research Method to gauge the respondents’ additional needs, attitudes, values and opinions
Consisted of three sets of Likert Scaling generating attitudinal responses to:
General module delivery and structure
Relevance to participants’ duties and responsibilities
Topics to be covered in more detail
Provided opportunities for respondents to provide additional qualitative data and/or comments

Assessment of Responses

The raw data was

computed showing gender disaggregated responses.
weighted using 5-scale Likert Scale values in descending order from 5 to 1, the former being the highest (i.e. strongly agree, or highly relevant, or very desirable)
weighted averages were used to determined responses for
General module delivery and structure
Relevance to participants’ duties and responsibilities
Topics to be covered in more detail

A further analysis was undertaken to determine the MEDIAN the MEAN and the STANDARD DIVIATION (STDIV) of the weighted responses of each set of above stated attitudinal responses.

Findings

Quantitative Data

Responses from female and male participants did not show a significant statistical differences within either of the three above cited sets of Likert Scaling generating attitudinal responses.

Responses to General Module Delivery and Structure

TABLE 1: The gender disaggregated raw data

The weighted averages for all 10 response statements were within the 3rd quartile of values.

The highest possible weighted averages score is 15 and the lowest possible weighted averages score is 5.

TABLE 2: The gender disaggregated raw data

In descending order, the weighted average responses were as follows:

STATEMENTS / Weighted Average Score Obtained / Maximum Weighted Average Score Obtainable
Overall, the program has been beneficial to my area of work / 12.80 / 15.00
The module was delivered consistent with its stated objectives / 12.60
I would recommend this course to others / 12.60
The program material (handouts) was relevant and appropriate / 12.40
I gained more confidence in the topic content area / 12.00
I acquired new knowledge & competencies through participation in the workshop / 11.80
I am confident to use the content of the for training of others / 11.40
Such types of programs require more in-depth & specialized focus / 11.40
A further follow-up program / workshop is necessary / 10.80
I would have preferred a longer duration for the program / 10.80

The above weighted average scores show:

MEAN / 11.86 / MEDIAN / 11.90 / STDIV / 0.743

Main Findings

1. The above finding suggests a high level of participants’ satisfaction with the general Module delivery and structure.

2. The module was delivered consistent with its stated objectives

3. The respondents suggest that they would recommend this course to others.

4. 11 of the 15 participants declared that they were confident that they would be able to communicate the contents of the training to others – however, only 10 of these respondents agreed strongly suggesting that the staff may need external or peer support to cascade this training

Note: For combined raw gender disaggregated data and weighted scores of responses refer to Appendix 2

Responses to Relevance for Participants’ Daily Duties and Responsibilities

TABLE 3: The gender disaggregated raw data

The weighted averages for all 10 response statements were within the 3rd quartile of values.

The highest possible weighted averages score is 15 and the lowest possible weighted averages score is 5.

TABLE 4:

In descending order, the weighted average responses were as follows:

STATEMENTS / Weighted Average Score Obtained / Maximum Weighted Average Score Obtainable
Manage projects –Task management / 14.50 / 15.00
Improving ZAVOD service quality / 13.80
Monitoring and evaluating the effectiveness and efficiency of CPD / 13.80
Supporting the development of schools as organisations / 13.40
Planning effective training interventions / 13.00
Principles and characteristics of educational leadership for training of trainers in a context of CPD / 12.20
Activity: Enacting a Monitoring and Evaluation Framework / 10.40

The above weighted average scores show:

MEAN / 13.01 / MEDIAN / 13.40 / STDIV / 1.36

Main Findings

The findings are:

1. The workshop training topics and associated activities show a high relevance score as they relate to the participants’ daily duties and tasks.

2. The issues of leadership and task management were not perceived as highly relevant. A number of speculative conclusions could be drawn. One potential scenario is that this relevance ranking may be due to the fact that currently the roles of Zavod staff are not clearly articulated as to focus upon these issues.

3. The weighted average for the Activity ‘Enacting a Monitoring and Evaluation Framework’ was ranked significantly lower than other topics and activities. This potentially indicates that it remains unclear to Zavod staff how such a Framework might be relevant to their work. It may therefore be of value if, in the future, Zavod undertakes further work to explore the relevance of such a Framework to its own work and, if appropriate, develop and enact a Monitoring and Evaluation Framework of its own.

Note: For combined raw gender disaggregated data and weighted scores of responses refer to Appendix 2

Responses to Topics to be covered in more Detail

TABLE 5: The gender disaggregated raw data

The weighted averages for all 7 response statements were within the 3rd quartile of values.

The highest possible weighted averages score is 15 and the lowest possible weighted averages score is 5.

TABLE 6:

In descending order, the weighted average responses were as follows:

STATEMENTS / Weighted Average Score Obtained / Maximum Weighted Average Score Obtainable
Improving ZAVOD service quality / 13.40 / 15
Supporting the development of schools as organisations / 13.00
Monitoring and evaluating the effectiveness and efficiency of CPD / 11.20
Planning effective training interventions / 11.20
Manage projects –Task management / 10.80
Principles and characteristics of educational leadership for training of trainers in a context of CPD / 10.00
Activity: Enacting a Monitoring and Evaluation Framework / 9.00

The above weighted average scores show:

MEAN / 11.23 / MEDIAN / 11.20 / STDIV / 1.55

Main Findings

The findings are:

1. The workshop training topics and associated activities related to Manage projects –Task management and Principles and characteristics of educational leadership for training of trainers in a context of CPD show a high relevance

2. The weighted average for the Activity ‘Enacting a Monitoring and Evaluation Framework’ was ranked significantly lower than other topics and activities, which indicates that a possibly more practice focussed should have been adopted during the workshop.

Note: For combined raw gender disaggregated data and weighted scores of responses refer to Appendix 2

Qualitative Data

Participants were asked to:

1. Describe where and how the knowledge and competencies required are used in their daily business activities.

2. Describe how and when they would use the acquired knowledge and competencies in their future work

3. Provide any additional response they see as relevant

Responses to ‘where and how the knowledge and competencies required are used in their daily business activities’.

1. The knowledge and competencies gained in this workshop should be used:

a. as example for structuring delivery of and providing leadership in CPD programmes

b. in enhancing the quality of ZAVOD’s services

c. for CPD in schools

d. Training of trainers in schools

e. Evaluation of and feedback to trainers of trainers

f. Dissemination of leadership functions

g. Evaluation of training offerings in the catalogue

Responses to how and when they would use the acquired knowledge and competencies in their future work

1. The knowledge and competencies gained in this workshop should be used for

a. Collaboration with cooperating experts in the various organisations

b. Representing ZAVOD in various forums

c. Collaboration with schools.

d. Structuring of CPD delivery

e. Collaboration with centres of vocational education

f. Realisation of future training

Additional response

2. There were no additional comments/response

Appendix 1: Evaluation Questionnaire

Evaluation Questionnaire

Workshop Conducted on 29th and 30th November 2016

Name of Respondent
Organisation
Gender / Female / Male
Please provide responses to the following
Statements below
Please √ in the appropriate box / Strongly Agree / Agree / Not sure / Disagree / Strongly Disagree
The module was delivered consistent with its stated objectives
Overall, the program has been beneficial to my area of work
The program material (handouts) was relevant and appropriate
I acquired new knowledge and competencies through participation in the workshop
I would have preferred a longer duration for the program
Such types of programs require more in-depth & specialized focus
I gained more confidence in the program topic area
I am confident to use the content of the for training of others
A further follow-up program / workshop is necessary
I would recommend this course to others
Please rank the topics covered according to relevance for your daily business tasks
Please √ in the appropriate box / Highly Relevant / Relevant / Not sure / Marginally Relevant / Not relevant
Principles and characteristics of educational leadership for training of trainers in a context of CPD
Planning effective training interventions
Improving ZAVOD service quality
Manage projects –Task management
Supporting the development of schools as organisations
Monitoring and evaluating the effectiveness and efficiency of CPD
Activity: Enacting a Monitoring and Evaluation Framework
Please identify topics you would like to cover in more detail
Please √ in the appropriate box / Very Desirable / Desirable / Not sure / Undesirable / Very Undesirable
Topics
Principles and characteristics of educational leadership for training of trainers in a context of CPD
Planning effective training interventions
Improving ZAVOD service quality
Manage projects –Task management
Supporting the development of schools as organisations
Monitoring and evaluating the effectiveness and efficiency of CPD
Activity: Enacting a Monitoring and Evaluation Framework
Please describe where and how the knowledge and competencies you acquired are used in your daily business activities
Please describe how and where you could use the acquired knowledge and competencies in your future work