Learning lab six month review
This report was written in November 2015, and reviews the first six months of the programme. It provides an example of how you might compile a review to demonstrate the impact your programme has had so far. Links to internal documents with further information have been removed.Introduction
The Learning Lab - NCVO’s peer to peer digital skills development programme - has been running for six months. During this time, the digital and communications team have delivered 64 sessions, across 12* different topics ranging from digital publishing to Agile project management. The programme has seen a total of 293 attendees, and of current staff (as correct on 17 November 2015), 76 individuals across NCVO have attended at least one session.
A mid year review has been carried out to gather feedback from NCVO staff, both on the programme as a whole, and individual sessions, to identify areas that are working well and areas that could be improved; review the process and logistics; and consider suggestions and ideas for the next six months.
The review takes into account both qualitative and quantitative feedback from sessions delivered between 18 June and 17 November.
The primary aim of the Learning Lab is to help build a workforce that embodies digital as a core skill, to enable individuals to work together efficiently and effectively and with confidence; to share knowledge, ideas and resources and try new ways of working. This has certainly not yet been achieved however findings from the review indicate that we are making progress and a proportion of NCVO staff are working more effectively and confidently with digital tools and approaches as a result of the Learning Lab.
*13 sessions are currently available however one session – Understanding the Web – has not run yet due to low demand.
Session feedback
The short evaluation forms completed by individuals after they’ve attended a Learning Lab session have provided some useful data on how individuals rated the session in terms of usefulness, content and delivery, and whether they now feel more confident about the subject.
· 97% of attendees said they had learned something new
· 96% said the sessions were good or excellent in terms of overall usefulness
· 95% said they feel more confident about the subject
Individual comments and feedback tell us that people value the practical and interactive elements – staff want to put their learning into practice during a session rather than listen to a presentation for an hour. It is important that we consider this preference when creating new sessions.
Attendance
In this six month period the 64 sessions delivered by the digital and communications team have seen a total of 293 attendees. Including all past and present staff, 82 individuals (out of 136) have attended one or more sessions. Of members of staff currently employed at NCVO, 76 individuals (65%) have attended one or more sessions:
· 35% have attended no sessions
· 28.2% have attended 1–2 sessions
· 30% have attended 3–6 sessions
· 6.8% have attended 7 or more sessions
Individual attendance has been fairly evenly distributed across teams. In most teams, individuals have attended 1-2 sessions or 3-6 sessions. Teams with the lowest attendance levels are Central services, IT, HR, Finance, Research and Infrastructure development. Reasons for non attendance from individuals in these teams largely relate to a lack of time, deadline pressures, or a feeling that digital is not relevant to their role.
Survey results
Survey one
This survey was sent to individuals who have attended one or more sessions. The aim was to gather general feedback; invite suggestions for improving the programme and identify how people have used their new skills and confidence.
46 people (out of a possible 72) completed the survey, providing a 64% response rate.
Results indicate that staff are enjoying the Learning Lab sessions and consider it a valuable resource for NCVO.
The main barrier to attending more sessions is time. When asked if their skills have improved since attending a Learning Lab session, 54% said they’re getting there. The majority of respondents added that this was because they didn’t have time to attend more sessions, or they didn’t have time to implement their learning.
Despite this 74% said they are keen to continue learning and 22% said they are excited to be trying new things. Reassuringly no one said they still want to panic/ cry/ hide in the loo.
Survey two
The second survey was sent to individuals who have not attended any sessions, to identify barriers or reasons for non attendance, and gather suggestions for improving the programme. The response rate for this survey was predictably lower, at just 39%, with only 16 out of a possible 41 possible respondents.
When asked why they hadn’t attended any sessions, only two out of eight possible options were chosen; 65% said they don’t have time and 35% the sessions always seem to clash with other meetings. Interestingly no one said they would prefer to go on external training which indicates this non attendance is genuinely due to a lack of time rather than the quality of the sessions. Further comments reinforced this with people talking about tight deadlines, other priorities or managing workload.
87% agree that the Learning Lab is a good thing for NCVO to be doing. The remaining two people said they weren’t sure, adding the following comments:
“I think so but im not really clear whether the aim is to upskill staff so that we can take on more of the comms stuff around our work or just to improve skills generally.”
“In theory yes – as I haven’t been to any sessions yet I am not really qualified to answer.”
Successes
Overall, feedback received about the Learning Lab has been positive.
According to individual session feedback, 97% of attendees learned something new in their session, and 85% of survey one respondents could articulate at least one thing they’ve done differently since attending one or more Learning Lab session.
Feedback indicates that staff particularly like the length, delivery, content and tone of the sessions. This reinforces the value of the peer to peer approach and we will continue this peer to peer approach, ensuring new sessions are informal, and as practical/ interactive as possible.
People also commented that they like the way sessions allowed them to interact with other staff members they may not usually interact with. While we may consider delivering some sessions in teams, I think there is value in keeping the mixed format for some sessions.
Using Eventbrite has allowed us to streamline online booking, making the process easier for individual staff to manage, as well as decreasing time spent managing individual booking requests.
Postive feedback:
I always find these sessions so useful, but it was especially helpful to have this so tailored to our work.
This is so helpful, thanks. I think this learning lab stuff is such a great thing you’re all doing
Each Learning Lab session I have attended has been hugely informative, really interactive and great fun. Whether its building sturdy structures with spaghetti and marshmallows or building on my knowledge-base, I come out of each session feeling invigorated, upbeat and thrilled albeit sorry that the session wasn’t longer! I wouldn’t miss a session for anything.
It was a useful session and I went away and applied the knowledge on a document straight away.
This is a brilliant resource thanks.
Great thank you and by the way I think the style of the learning lab what it looks like and the language is all really good.
Can I just say (again!) I think this is excellent.
I’ve signed up for sessions and have gone with some of your recommendations. The website is really easy to use and navigate. Everything is clear and well explained, making it easy to identify what would be of interest.
For me the best part of the Learning Lab is that the sessions are short and accessible so you can still learn and develop yourself without having to take a whole day out of a busy schedule. It’s really highlighted to me how NCVO is full of bright and intelligent colleagues who have plenty of knowledge to share – a precious resource that shouldn’t be taken for granted.
Challenges
During the programme planning phase, a number of risks and challenges were identified in a risks register. The below provides an update on how these challenges are being managed:
Maintaining engagement levels
65% of current staff have attended one or more sessions. In order to meet the 80% target, more needs to be done to demonstrate the value of the programme, and meet the needs of staff. Introducing new and more advanced sessions may also increase, or at least, maintain engagement levels.
Meeting expectations
While 97% of attendees stated they had learned something new, feedback from both surveys indicate that more advanced sessions would be welcomed. This will be taken into consideration when creating new sessions.
Managing logistics
Logistics are currently managed by Helen and Debbie. Refining the online booking system has saved time spent on logistics however this remains a fairly time intensive element of the programme and a more efficient process will need to be in place beyond September 2016 when Helen will no longer be in post.
Team resources
The digital and communications team is back up to full capacity, having filled a number of vacant positions. Once these new team members have settled into their roles, they may be willing to deliver new sessions or support the delivery of existing sessions.
Engaging remote staff
With more people people working from home and away from the office, it is important that Learning Lab sessions are made available to all staff, regardless of their location. Sessions can be recorded using ScreenCastify or GoToMeeting and hosted in the Learning Lab library, however this presents two challenges; firstly, including the practical elements of some sessions (ie group exercises etc), and secondly recording ‘attendance’(ie we can record the number of views through Youtube, however we cannot identify who has viewed each video). By watching these videos, individuals will also miss out on group discussion and the opportunity to ask questions. A solution to this would be to offer sessions remotely, but in a live environment. This requires further consideration. See remote attendance plan for more information.
Staff turnover
Staff turnover has been fairly high during the past six months. Adding the Learning Lab to the staff induction means this is now mentioned to all new starters. The challenge is keeping up with the volume of new people at times, however this is an excellent opportunity to engage more members of staff.
Evidence of success
Feedback from individual sessions and the surveys give evidence that the programme has been successful to a point. A more robust monitoring and evaluation process will be put in place for the 12 month review to identify the extent to which improved skills and confidence
Sustaining programme beyond initial scope
This will be a challenge to consider more fully on completion of the 12 month review.
Other challenges
People are busy and their diaries fill up fairly far in advance, therefore people need to know about upcoming sessions further in advance.
A number of sessions have been cancelled due to low or no registrations. Sessions have been designed to be delivered to groups of 6-8 however in reality, most sessions are attended by 3-5 people. When asked how we could improve attendance, the majority of respondents chose the option of delivering sessions in teams. This however would require additional logistical support from team assistants, and it would detract the opportunity for staff to engage with people outside of their team - a feature they specifically identified as being one they liked about the Learning Lab.
Another challenge is how we can promote the programme more effectively. There are a number of things we want people to know about the Learning Lab – new sessions, upcoming sessions, sessions happening over next few months, resources in the library, news, the value and how it can help people in their roles – however getting all of these different messages across and cutting through other messages to staff, is a challenge. A more structured comms plan that works alongside the organisation wide internal comms plan may support this.
Suggestions
Staff were asked to add their ideas, suggestions or requests for other sessions or for improving the Learning Lab in general in both surveys:
General
· Opportunities to attend sessions remotely/ e-learning/ record sessions so people can fit these in around their schedules
· More advanced sessions
· Deliver sessions tailored to teams
New sessions
· Driving business using social media (and evaluating what works well)
· Different online packages for doing surveys
· Using cloud based storage
· Analytics – how to assess the impact of your own online content
· New/ emerging online resources and websites with useful functions (Trello, Slack, Google)
· User stories
Other suggestions
There were a number of suggestions for sessions that may not necessarily be covered by the Learning Lab but might be worth considering. These include: click dimensions, branding, getting intelligence out of CRM, advanced Office tools and techniques, advanced CRM and managing data and building lists.