/ Name of Item
Third Party Learning Resources (TPLRs): an investigation. / Agenda Item
(office use only)
Paper for CAUL Meeting 2014/1
Stamford Plaza Brisbane, 20-21 March 2014 / From: Queensland University of Technology (QUT)
Name of Sponsor: Judy Stokker, Sue Hutley
Authors:
Lynn McAlister, eLearning Services
Regina Obexer, eLearning Services
Martin Borchert, QUT Library
Recommendation
CAUL is requested to
1.  Note the report on the use of third party learning resources (TPLRs) at QUT
2.  Consider how the report could be used to assist CAUL institutions.
Issue
The current investigation seeks to inform the recent recommendation that QUT:
(i) endorse a coordinated approach to managing and implementing third party assessment and learning materials as described in this submission;
(ii) consider broader issues such as brand value, academic leadership, quality and appropriate use of both commercial and open access educational resources.
Background
QUT – Third Party Learning Resources (TPLRs) – an investigation.
Introduction
An increasing number of traditional publishers are developing commercial online learning environments and learning resources and marketing these to universities and other education providers. This report will refer to them as third party learning resources (TPLRs).
The purpose of this report is to identify issues relating the provision of third party learning resources and environments to QUT units, and to also obtain feedback from QUT academics which are using these services. Information will be used to inform QUT and CAUL stakeholders of the issues, and also to inform the development of support services at QUT.
Section One of the current report gives the background to the 2011/2012 investigation, taken from the submission which was submitted to the QUT Learning Environments Working Party (LEWP) in May 2012. This section identifies issues under which responses from the current investigation will be presented.
Section Two details the method of semi-structured interview used to carry out the investigation. The stimulus questions are presented as Appendix A. The summary of responses is discussed, firstly from the perspective of student learning and academic teaching, which is always the core interest when exploring learning and teaching strategies, and further, under the issues identified in Section One. The detailed response list is provided in Appendix B.
Section One
Drivers for the 2012 Learning Environments Working Party (LEWP) submission at QUT
The increasing availability of both commercial and open access educational resources poses many opportunities and also a number of issues, including questions and risks, for universities.
Publishers such as Pearson, Harvard Business School Publishing (HBSP) and McGraw-Hill are pushing into (what is for them) a relatively new market segment and are offering various business models. In the education sector, they offer a range of product categories, including online quizzes and assessment / question banks, online learning resources such as electronic texts, supplementary text materials, and interactive learning objects. Resources currently used at QUT are briefly detailed in Appendix C.
The online learning environments in which resources are presented can add considerable value for academic staff and students. For academics, they can provide readily available content as well as customisable practice and exam questions with the ability to sequence release of content and assessment items, to provide sophisticated feedback, and also manage grades. For students, these environments provide an interactive learning experience with a range of features such as immediate feedback, supporting materials, and references.
Background
In 2011-2012, the then recent QUT experience, negotiating the use of Pearson MyLabsPlus
and Mastering solutions in four units in the Faculty of Science and Engineering,
demonstrated the benefits of centralised coordination of the necessary activities. The experience suggested an immediate need for the University to implement a coordinated approach to adopting commercial assessment and learning resources. In particular, it was noted that benefits could be gained in terms of the following aspects:
Contract negotiation
The initial negotiation with a publisher or other provider can be quite complex and requires liaison with several parts of the University, including the legal office. Issues to be examined include privacy (particularly in regards to transferring student information to external providers), security (particularly with regards to assessable items being hosted by a third party), indemnity, intellectual property, accessibility, and others.
As with other hosted services, there are also aspects regarding service availability, maintenance, problem resolution times, and other factors usually contained in a service level agreement which require consideration. Legal obligations relating to the retention of assessment records also require consideration.
Negotiating pricing
Depending on the business model used by the provider, significant savings can be made by negotiating an institutional price rather than individual arrangements in faculties or schools. Pearson MyLabsPlus and Mastering are priced per student using a stepped scale based on total students at that institution using the system. In addition, the University needs to be aware of what it is paying for and not make assumptions. Students who buy the Pearson textbook, for example, get access to a generic version (not the QUT configured version) of MyLabsPlus or Mastering. Institutions which buy MyLabsPlus or Mastering access for students do not get access to the book or ebook for students at this price. Pearson sells the ebook to libraries, faculties or students on a one copy per student basis. Students may buy the ebook if they want their own copy, or may use a Library copy. McGraw-Hill provides etext pricing for students to purchase their own copy and different pricing for the faculty or library to purchase etexts in bulk for all students. McGraw-Hill pricing for faculty or library bulk purchase of etexts can be lower than 25% of the cost of individual student purchases online. HBSP provides a site-wide licence and prices this based on usage in the previous contract term.
Financial management
It is important for faculties to note that the provisions of the Higher Education Support Act (HESA) apply when using third party assessment and learning resources. The HESA sets out the conditions under which students may be charged incidental fees. The act states that students cannot be charged for learning materials which support assessment such as study guides or other resources if students are not provided with alternative access free of charge, and software required to complete the unit where it is not available free of charge (e.g. through site licences in QUT computer labs). This applies equally to third party assessment and learning resources. For example where simulation software is used to complete an assessment item, the University (faculty or library for example) needs to ensure that free access to the software is also provided as an option for students who choose not to purchase their own copy.
Therefore the costs involved must be supported by the University. There is no central provision for this at QUT and faculties will need to consider the costs of licensing when selecting a product. However, in terms of managing the financial transactions, central invoicing and payment are more efficient and allow QUT to understand the size of the commitment across the University. It was proposed for the QUT Library to manage payment to providers in the first instance, and recover costs as incurred. This arrangement has been in place since 2012.
Integration with University systems and provisioning of accounts
Integrating third party environments with the QUT learning environments, including student account provisioning and integration with QUT Blackboard, is highly desirable in order to provide QUT students with a seamless learning environment and academics with less administrative work. Provisions should be made by the vendor to facilitate this integration, however the Division (eLearning Services, Library) will have to be involved in assessing integration needs and support this process.
Use of the Pearson MyLabsPlus and Mastering systems at QUT currently requires academics to upload spreadsheets of student information into, and download student grades from, the MyLabsPlus or Mastering system. This means considerable workload upon academics, especially prior to the student enrolment census date when enrolment changes are common. Provisioning using spreadsheets is not a scalable solution. Pearson has not provided a fully integrated Blackboard building block to QUT. Other providers (e.g. McGraw-Hill) have worked with Blackboard Inc. to develop such a building block. HBSP requires an institutional account manager to download materials from the HBSP system and to provide them to the academic for uploading in to QUT Blackboard, and to also keep records of each transaction and details of the unit, student numbers etc for reporting to HBSP.
Support
Students and staff using third party learning and assessment resources may expect some central support from eLearning Services or the Library. Whilst providers generally offer students support services (e.g. Pearson provides a 24x7 online chat support), it is important for the IT Helpdesk to be aware of these services so they can relay them to students when enquiries come in. Training and support for staff is also often available from the provider.
Based on these arguments, it is proposed that the QUT Library coordinate the acquisition and implementation of third party assessment and learning resources, and liaise closely with eLearning Services, faculties and vendors in this process .
Issues regarding quality, brand value and academic ownership
In addition to the practical issues discussed above, there are a number of broader questions the University needs to consider regarding the use of both third party and open education resources. These include:
Quality of resources
The quality assurance of resources and environments becomes more critical if the content and assessment items purchased or adopted from one provider make up all or the major part of a unit. Many of the products available bundle learning content and assessment items together. QUT needs to consider the issues relating to the “outsourcing” of content and assessment, e.g. under what circumstances it is appropriate to use a large extent of these resources in a unit, what factors would influence this decision (e.g. type / level of degree).
QUT brand value and academic ownership
Academic staff have always used a range of learning materials created by others (textbooks, journal articles, websites, videos, etc.) and have integrated them in the curriculum to support student learning, and to enable achievement of the learning outcomes required. The shift to the online environments under discussion, changes this playing field. Third party curriculum, learning materials and assessment can be used not only to supplement but to replace QUT environments and materials. Consideration must be given to the question of value of and need to protect the university brand. This includes consideration of how a resource is embedded in the curriculum, and how it is contextualised in a unique university context. Student feedback indicates that students like their resources to be branded QUT and integrated in QUT Blackboard. This question relates not only to commercial learning materials but equally to open education resources.
Based on the experience detailed above, an investigation of current use of TPLRs at QUT was undertaken during November 2013- February 2014.
Interview participants
One of the unit coordinators involved in the 2011-2012 Science and Engineering Faculty activity was invited to participate in the current investigation. Practitioners known to be using or managing the use of TPLRs at QUT were invited to give feedback on their experiences. From the invitation round to this pool of known users, seven people agreed to provide feedback about their experiences. Subsequently, two users identified by the interviewees also agreed to give feedback. A broader invitation was delivered through Yammer to elicit unknown TPLR users but there were no responses to this broader invitation. In total, nine TPLR users were identified and interviewed.
Method
A semi-structured interview method was chosen to best facilitate consultation. Stimulus questions were sent to each interviewee prior to the interview to stimulate subsequent discussion. The stimulus questions are available in Appendix A. Through the semi-structured interviews it was possible to gather information based on the realities of those implementing TPLRs at course and unit level. Semi-structured interviews allow flexibility around a topic as respondents speak from experience and are free to elaborate on significant points or issues (Denscombe, 2007). There was no time limit on the interviews and each interview ran from 40-60 minutes. Response notes were taken at each interview.
To date, nine people have been interviewed. One interview was conducted by telephone at the request of the interviewee; all other interviews were face to face in the workplace. The interviewees include Head of School, course and unit coordinators and faculty administration and represent undergraduate, post graduate and corporate style delivery. Teaching staff for large cohorts of 900-1600 students and smaller cohorts were represented. Disciplines represented include Science and Engineering Faculty – physiology, structural engineering, mechanical engineering, and the QUT Business School – marketing, economics, and MBA.
Summary of responses
While each participant was given the stimulus questions prior to the interview, most chose to speak informally and ‘tell the stories’ about their experiences. The questions served to prompt these stories. The responses have been grouped to address firstly the impact on student learning and academic teaching and then under the headings previously identified in the QUT LEWP submission. Appendix C provides a detailed response list aggregated to illustrate both the large cohort and the small cohort learning and teaching perspectives.
Effect on student learning and academic teaching
Respondents were very positive overall about the capacity of online quizzes, tutorial problems, case studies and simulation to engage students and enhance learning outcomes. Many of the quizzes and problem-based resources used provide feedback to students, giving them the opportunity to redo problems and maximise learning. One respondent has evidence that use of problem sets has lifted the learning attainment in the unit. The literature clearly shows the power of regular feedback to enhance student learning. It is perhaps to be expected that these TPLRs can enhance the student experience by providing 24/7 feedback and support which is not otherwise possible. Student feedback indicates a high level of satisfaction with online quizzes and problems and that they feel supported by regular feedback.