Teaching Fellowship Award Scheme: Final Report
Name of key contact (project leader): Ricky Lowes
Department: International Centre Telephone: 585711
E-mail:
Names of other staff involved: Emma Purnell, Helen Bowstead
Title of project: ‘Feed-forward: Exploring the staff and student experience of technology facilitated feedback - can technology support and engage staff and students in dialogic feedback?’
Type of project Action Research
Keywords dialogic feedback; language learning; DLE
Aims of project: to explore the use of new feedback processes within four module groups in the Plymouth Business School; to evaluate how two of the institutionally provided learning technologies, Moodle and PebblePad, could support and enhance the online feedback experience for staff/students.
Background/context to project:
ELC101, ELC201, ELC305 and ELC307 are four English language modules where we seek to develop students’ capacity to self-regulate and develop strategies for autonomous learning, in order to further the development of their English language skills. We have noted over the years that students do not seem to engage with feedback as much as we would wish and wanted to see if use of the DLE / Pebblepad would encourage them to adopt a more proactive role in responding to feedback. Both platforms allow students to respond online.
Methods used:
· We set up activities, assignments and feedback channels in Pebblepad and Moodle for the 4 modules.
· A typology of activities, assignments and feedback types and student responses was drawn up.
· We gathered feedback from students on their perceptions of the feedback they received both via module feedback surveys and in focus groups (2).
· The Research Assistant analysed the data on Pebblepad and Moodle.
Results:
In terms of engagement with feedback online, the results were disappointing because they showed minimal student activity. The opportunities were via eSubmission Feedback (Moodle); Forum (Moodle); Journal (Moodle) and the ‘Reply to’ functionality (Pebble).
eSubmission Feedback (Moodle)
ELC305 The feedback channel via eSubmission only got 3 replies on line to 68 feedback comments on a formative piece of work, which were expressions of thanks to the teacher rather than requests for clarification or any other form of discussion. When a sample of another piece of formative work was analysed, only 1 student out of 10 had substantially revised their work in the light of formative feedback, which suggests that students are not responding to feedback and using it for their development. 65 students created a total of 138 forum posts which included an introductory profile, a description of their home town and a comparison of education between the UK and their home. 48 students received a response from the tutor and of these 18 included grammatical pointers. All responses were worded in a way that showed interest in what the students had written. There was no participation mark associated with student feedback and students responded to posts from their peers in seven cases. In all but one of these cases the student making the initial post responded to their fellow student. In three (out of 48) cases, students responded to the tutor. However, overall module marks were slightly higher than last year, (mean of 58.01% v 56.61%) despite a lower entry tariff, so the focus on feedback may have had beneficial results .
Forum (Moodle)
In ELC307 there was evidence of very positive interaction in Moodle forums, which can be regarded as a response to a different kind of ‘feedback’ as students respond to comments made by peers. Analysis of this interaction provided a useful indication of student sociability with regard to later group-work, which has developed into a separate project.
Journal (Moodle)
In ELC307 there was an attempt to use the Journal function to provide a private space for students to receive and respond to feedback. Students were invited to post language work for detailed feedback and correction. Technically the tool proved unfit for purpose as students found that they could only post once and subsequent work then corrupted or deleted the previous work. The teacher could not access each student’s work separately; all work was presented as a continuous whole and the layout was confusing. The experience was so negative for both lecturer and students that the Journal was soon abandoned. However, students showed that they were motivated to reply to feedback but hindered from doing so by technical impediments.
‘Reply to’ functionality (PebblePad)
An assessed language learning portfolio using PebblePad was introduced into the ELC101 module to address a number of key concerns that had been identified in previous years
PebblePad was included in the project as its functionality, like Moodle, allows students to comment on tutor feedback. It was hoped that using an e-portfolio would encourage students to engage in ongoing language learning and consolidation tasks and foster a more proactive response to tutor feedback. One of the key advantages of using PebblePad is that tutors can track an individual student’s level of engagement with the tasks thus enabling the tutor to intervene at an earlier stage in the portfolio process if no activity is recorded. For the students, the main advantage is that the portfolio tasks are all located in one place (the PebblePad Workbook) thereby enabling them to work on tasks and receive feedback throughout the module. Tutor feedback was given in the form of a short, personalised comment followed by a question, which was designed specifically to encourage a response. Feedback was posted on the various portfolio tasks as they were completed in terms one and two. Despite a good overall engagement with the portfolio tasks, only one student response to the tutor feedback (out of a cohort of 17) was received. Although students can reply to feedback, it doesn’t mean they will. The lack of responses might also indicate the existence of certain ‘risk factors’ that may inhibit a student’s motivation to respond, for example: poor language skills, concerns about privacy, fears relating to the ‘permanence’ of responses, impact on marks, etc. However, this requires further exploration.
Positive spinoffs
There were some successes related to the project:
· Some unexpected positive effects on student engagement in general (evidenced in amount of work produced, even when not for assessment)
· Students expressed a high level of satisfaction with feedback on ELC305 (the largest module involved)
· Positive ‘Hawthorne effect’ on tutors’ provision of and engagement with feedback
· Module lead for ELC305, and lead researcher, became increasingly aware of the need for timely and deft opportunities for feedback to feed-forward into upcoming tasks, and further streamlined the design of tasks to facilitate that.
There were also a number of challenges:
· Loss of data on student activity due to failure of Moodle server in autumn term
· Limitations of Moodle functionality (e.g. no alerts on responses to feedback; poor functionality of Journal)
· Inexperience of both tutors and students with Moodle functionality
· Lack of engagement of students with Moodle functionality
Conclusions and Lessons learned
It seems that the inhibitors on students responding to feedback are not simply practical ones, but that they may be deeply embedded in a certain culture of learning and it will take more than a change of technology to alter them, challenges such as:
• Students are used to being ‘passive’ receivers of feedback.
• Students need to understand the value of engaging in dialogic feedback – risk vs reward, marks or no marks?
• Students need to be given ‘permission’ to feed back. They may see the tutor as the expert, not to be questioned. Opportunities to respond to feedback need to be made more explicit to students, to ensure that their failure to respond is not due to them simply not realizing they can. (To be done 2015-16 and results monitored.)
• Concerns about permanency of words online
• Lack of confidence in language skills making students reluctant to be more visible online
• Email is seen as the ‘norm’ in communication for tutors and is difficult to move communication to alternative online methods
As well as cultural issues with learning highlighted above there were certain technological challenges that arose that are being addressed for this year:
• There was no notification of a response to feedback in either of the technologies used
• Students can’t edit their forum post, only tutors can change them in our configuration
• The always connected culture can sometimes unrealistic expectations for online feedback
• Familiarity and comfort with social media doesn’t automatically translate to the ability to interact with learning technology
However, having said that, there are some practical aspects that can be addressed, to see if that allows a shift in practices and attitudes.
Feed-forward into tasks needs to be extremely timely. (CEP structure may facilitate this.) The use of structured feedback/feedforward templates for completion by both tutor and student may be of additional help in this area, too.
Strategies for raising awareness of feedback that emerged as a result of reflecting on the project were:
• Ask explicitly what students had amended in response to feedback on action plans as part of submission (to raise their awareness and make monitoring easier)
• Set a peer review of a suitable piece of work (language learning background for Portfolio brief clarification or report introduction and context for WBL project ELC312 – formerly 307,) and award marks for participation, then intervene at the end of the time period with lecturer feedback.
Associated publications:
A paper: ‘Laying the foundations for teamwork in an on-line forum.’ by Ricky Lowes & Michelle Virgo, has been submitted to Language Learning and Technology and is pending review.
Conference presentations connected with the project
· CRA Researching and Evaluating Recording Achievement, Personal Development Planning and e-Portfolio Plymouth, UK16th - 17th April 2015 (Emma Purnell and Helen Bowstead)
· AAEEBL conference: Moving Beyond “One-Size-Fits-All” by Targeting Three Strategic and Transformative Approaches: Evidence-Based Learning; Personalized Learning and Holistic Outcomes Assessment Hynes Convention CenterBoston, MA. July 27-30, 2015 Should Students Talk Back? Exploring Dialogic Feedback Opportunities in ePortfolio Based Learning.(Emma Purnell)
Innovative Language Teaching and Learning at University: Enhancing Participation and Collaboration. University of Nottingham, 19 June 2015 Feed-forward to scaffold guided autonomous learning (Ricky Lowes)
· SOLSTICE eLearning Conference Edgehill University 4th and 5th June 2015 Feed-forward: using DLE resources to scaffold guided autonomous learning. (Ricky Lowes) Why can’t I just email you instead? The challenges of engaging students in online feedback within a Digital Learning Environment (Emma Purnell)
· Languages in the Globalised World, Leeds Beckett university, May 2015. Engaging multicultural learners using an on-line environment (Ricky Lowes)
· VC’s Teaching and Learning Conference (Plymouth University) June 2015. Exploring dialogic feedback opportunities in the digital learning environment (Ricky Lowes, Emma Purnell and Helen Bowstead).