June2007/MHW
E-portfolio report
Mary-Helen Ward, Acting Project Manager, USyd ELearning
Dr Rob Ellis, Director, USyd eLearning
This report is meant as a preliminary investigation into dimensions of e-portfolios as they are represented in practice and some literature internationally. It is not meant to be an exhaustive review, rather a way of structuring the space of e-portfolios relevant to campus-based universities such as Sydney.
Contents
IntroductionBackgroundQuestionsValue and costs of e-portfolio systemsSummaryReferences Appendix 1 – examples of e-portfolio systems Appendix 2 - tools in e-portfolio systems Appendix 3 – e-portfolio conference October 2007
Introduction
The use of portfolios to enhance teaching and learning is a world-wide trend. In the UK, universities are now mandated to provide their students with a tool to measure and record their personal development (a Personal Development Profile/Plan or PDP), and e-portfolio tools are generally being found to be a convenient way to do this (Ward, 2006). In the US many institutions have begun to provide e-portfolio tools for their students; the US undergraduate system of generic degrees (presently being adopted at some institutions in Australia) lends itself well to the formulaic style of e-portfolio, in which a student can measure progress through an entire degree course according to pre-determined parameters. Australian universities are slowly adopting e-portfolios, and some of the pressure to do so is generated by the need for professional courses to demonstrate the acquisition of competencies to registration and professional bodies.
One of the foremost theorists in the uses of e-portfolios in education, Helen Barrett (2004), suggests three general classifications for the use of e-portfolio systems:
- As assessment tools to document progress and the gaining of competencies (thus utilising a positivist paradigm)
- As reflective tools to encourage critical thinking and deeper learning (a constructivist paradigm)
- As showcases of abilities and achievements (a product for marketing of the student’s suitability for employment)
The philosophical conflicts in these positions are clear (see also Acker, 2005). Students may need strong encouragement to use their e-portfolio as a useful reflective tool if they see it as only a repository for marks or formative assessment. If e-portfolios are carefully constructed so that students can measure themselves and reflect on their progress against specified learning outcomes, they are more likely to feel strong ownership in both the process and the product.
Background
The common understanding of what an e-portfolio consists of is summarised by Lorenzo and Ittelson (2005):
“…a digitized collection of artifacts, including demonstrations, resources, and accomplishments that represent an individual, group, community, organization, or institution. This collection can be comprised of text-based, graphic, or multimedia elements archived on a Web site or on other electronic media such as a CD-ROM or DVD”(p2).
However, this discussion has been taken further by Gibson & Barrett (2003) in breaking the approach to planning into a generic tools approach versus a customised system. These approaches are discussed further under ‘Questions to be considered’.
Student artefacts are collected into the portfolio application and usually have the ability to be eventually showcased as a webpage, or on a CD or DVD. Lorenzo and Ittelson (date) also discuss the specific use of e-portfolios in higher education, claiming that they help develop critical thinking (by encouraging reflexivity) and both writing and multi-media skills. However, these claims are of course dependent on the way that the e-portfolio application is set up by the institution. The last claim, particularly, is not entirely borne out by the increasing use of e-portfolios that are part of an LMS such as Blackboard or WebCT. These e-portfolio products do not require, or indeed even allow, students to develop serious multi-media skills.
The use of an e-portfolio as a reflective tool has its theoretical roots in ideas such as Biggs’ (2003) notion of ‘constructive alignment’. This constructivist theory moves beyond criterion-referenced assessment to the notion that everything in a curriculum, including the assessment task(s) and the teaching methods, should be aligned with the intended learning outcomes. In an e-portfolio this can be visually demonstrated by means of matrices and tables that show the student exactly where each quiz, clinical skill, piece of written work or oral presentation fits into the skills base or conceptual framework that the unit, module or course is intended to develop. Students can then reflect on how s/he is meeting, exceeding or failing to meet these standards of competencies, and mentor(s) can give feedback.
Barrett draws a distinction between “the portfolio as process (collection, selection, reflection, direction, presentation)” and the presentation and display aspect of the portfolio application, what she calls “the portfolio as product (the notebook, the website, the CD-ROM or the DVD and the technological tools used to create the portfolio-as-product)”. (2004, n.p.)In practice these two aspects are often conflated, but students may require training to distill the aspects of their ‘e-dentity’ (see Ittelson, 2001) that they want to present for different audiences, such as peers, tutors, assessors, accreditation or professional bodies, potential employers or even the students themselves in a reflective process.
Questions
The questions raised by a plan to introduce e-portfolios university-wide are closely inter-related which may make it difficult to untangle the implications of some answers from other answers. Key questions that should be considered are:
1What is the lifecycle of the e-portfolio space?
Will it be made available to students after they have left the institution (as the basis for construction of a CV and/or PDP), and if so for how long and under what circumstances?
What are the costs of this availability?
If the e-portfolio is to be used as a presentation tool, transferability of data becomes an issue if the student will not have access to the presentation they have created on the institution’s servers at some time in the future. The collection needs to be able to be exported as a webpage or database that can be imported and presented in other software outside the e-portfolio application. It can be taken away from the institution on a CD or DVD, but needs to be able to be updated. This is not possible with some commercially available systems,
2Who gets access to an e-portfolio?
Is the material owned by the students or by the institution?
Should there be areas that are completely private to the student?
Who owns the completed e-portfolio? (This clearly influences the answers to the questions in 1 above)
This second group of questions have particular significance if the e-portfolio is to be used for assessment (when staff need at least some level of access), but less if it is to be used by students for only presentation of their achievements (when it is clear that it is the student should decide what artefacts are included and who should have access at what level).
The amount of server space required by completed e-portfolios can be enormous, especially if they are to be stored over several years and the setting up of servers with the database software is another technical issue to be considered. These issues are not discussed in the literature. The ready availability of server space in institutions may account for the lack of literature, but it is something that needs to be discussed with the university community early in the process.
Uses of e-portfolios
In this project a number of e-portfolio sites were identified. However, access to most of them for assessment and evaluation was impossible, as they are not available outside the institution. Where possible, information about their use was included in this report, using material published on the institution’s website. There were a small number which are either showcasing their students’ achievements with publicly-available portfolios or making a guided tour available; links to these are provided in the table in Appendix 1. In a very few cases (The Faculty of Medicine at Newcastle in the UK and the Open Source Portfolio (OSP)) a live site was available for testing; again, links to these are in Appendix 1. However, the links to the testing site for the OSP were broken so no assessment of this was possible.
In Australia some universities are using e-portfolios: Wollongong (WebCT/Blackboard), Melbourne (Blackboard) and QUT. UNE is also using an e-portfolio system (probably the WebCT one), in a very positivist way - it is described on the website as "a web-based records management system", and is tied to seven graduate attributes. A reflective journal can also be submitted to be considered for a university-wide achievement award. Most importantly for the purposes of this project, the medical faculty at UNSW has created a comprehensive and complex e-portfolio system called EMed, which is it has used successfully for several years. We attended a demonstration of this system, which is impressive. Appendix 4 is a table of the matrix of capabilities that are assessed by staff and reflected on by students in the EMed system. (Note, however, that UNSW does have not have graduate entry, so the competencies and concepts in its curriculum may not be relevant to the curriculum used in the Faculty of Medicine at University of Sydney.)
Assessment using e-portfolios
There are many ways that e-portfolios can be used for both formative and summative assessment. Lorenzo & Ittelson (2005c) list the following:
- to assess students against specified learning objectives using a rubric to assess against a matrix of performance criteria (not a showcase – internal use)
- to help student analyse patterns in their own learning (formative)
- to measure against state-based standards
- as a collection of “digital artefacts” + reflective essay, all of which is given feedback by faculty (Johns Hopkins) – used for university’s review of its own programs
- as part of a learning plan with analysis of evidence and outcomes (different to the first as students set own outcomes and decide on the evidence)
One important use of e-portfolios that Lorenzo & Ittelson don’t mention in detail (although it could be considered as a subset of their first point) is as a facility to give both students on work placements and their supervisors a place to journal and give feedback on skill acquisition. The use of e-portfolios in this way (ie to assess as well as catalogue skills acquisition) is more often required in practice-based disciplines.
Reliability in relation to high stakes assessment is an issue and needs to be considered; in particular the numbers of assessors needed, staff development needs for assessment and issues relating to pass/fail or grading criteria.
The Australian Information and Communications Technology in Education Committee (AICTEC) maintains The Technical Standards website, “a gateway to standards, protocols and specifications relevant to learning, education and training”. Under their auspices, the Education Network Australia (edna), a joint initiative of the state and territory governments and the federal government, has collected standards created for e-portfolios around the world. This can be seen at
Value and costs of e-portfolio systems
Considerations in the choice and tailoring of e-portfolio systems:
- value-add to student learning
- acceptability to students and staff
- infrastructure costs: integration, sustainability (cost-effectiveness)
- maintenance and support costs
- Ease of use, including ability to transfer grades from an LMS into the e-portfolio system
- Choice of tools
The range of student artefacts that could be housed in an e-portfolio system is substantial. The following indicates the ranges of artefact:
- Actual marks/grades
- Detailed tabulation of practical training/clinical skills etc against a matrix
- Reflexive/reflective work (eg blogs, comments on assessments, responses to peer mentor, or faculty reviews)
- Presentation of particular achievements, eg graphic designs, artwork, creative writing
- Video/audio of student-patient interactions with reflective commentary
- Peer reviews of joint projects
- Faculty feedback on any of the above
- Student claims of prior learning, skills gained outside/prior to enrolment in the course, voluntary activities etc
Once an institution has decided to acquire an e-portfolio system, generally speaking there are two options: you can buy a licence for an existing system, or you can use Open Source software, which is free but basic and will need customising. The disadvantage of the latter solution is the maintenance model costs which can not be managed by annual licence costs.
Licensed options
Licensed options include tools already integrated with learning management systems (which have little software set up costs as it is already embedded), or third party products which can be integrated through Powerlinks or Building Blocks. Advantages are the ease of integration with existing tools making use of the system easier for non-specialist user-teachers, and a maintenance model which is covered by annual licence costs. Examples are the Blackboard eportfolio and iWebfolio, which are reviewed in Appendix 1.
Open Source options
Although there are no software purchase costs, set up costs include an operating system installed on a server, integration costs and customisation costs. Ongoing costs involve resolving how to address maintenance and upgrading.
Case studies of institutions who have built their portfolios using OSP can be seen at The costs of doing this are unknown.
It is also being used at Massey University in New Zealand:
Reviews of the systems that have been built from OSP at Portland State and Indiana University have been included in the review in Appendix 1.
Summary
There are many decisions that need to be taken before e-portfolios are implemented, including;
- the learning uses they are to be put to, and forms of assessment that will be incorporated
- ownership of the artefacts and final product,
- how the data will be managed and the user-support required
- setup, maintenance and upgrade costs (especially costs of maintaining space for students over the life of their studies and for some time after graduation).
There is no doubt that e-portfolios are becoming mainstream in higher education systems worldwide. They have particular application in integrated programs, they have more appeal and a clearer pedagogical application when a clear, detailed matrix of skills, competencies or conceptual frameworks is available throughout a program of study. Linking the e-portfolio to generalised Graduate Attributes is less useful for student learning than being able to connect to detailed Learning Outcomes. Johns Hopkins built an e-portfolio system for their Master of Arts in Teaching program five years ago, basing it on an earlier paper portfolio. (This system is now a commercial product and is reviewed in Appendix 1.) The department director is quoted as saying that Digital Portfolios “enhance our ability to look at our own program” (J. Nunn, quoted in Lorenzo & Ittelson 2005b). This may an unintended benefit of installing an e-portfolio system, that, like an eLearning system, it may reveal and make transparent parts of the curriculum and the student experience that have previously been hidden from view.
References
Acker, S. (2005) Overcoming Obstacles to Authentic e-portfolio Assessment. Campus Technology, 14 March 2005. Accessed at on 18.06.2007
Barrett, H. (2004) Selecting e-portfolio software. In E-portfolios for Learning, June 01, 2004. Accessed at on 18.06.2007
Biggs, J. (2003) Aligning teaching for constructive learning. Discussion Paper from The Higher Education Academy, UK. Accessed at on 18.06.2007
Gibson, D. & Barrett, H. (2003) Directions in electronic portfolio development. Contemporary Issues in Technology and Teacher Education, [Online serial], 2(4). Accessed at on 18.06.2007
Lorenzo, G. and Ittelson, J. (2005a) An overview of e-portfolios. ELI Paper 1: Educause Learning Initiative, July 2005, Educause. Accessed at on 18.06.2007
Lorenzo, G. and Ittelson, J. (2005b) An overview of institutional portfolios. ELI Paper 2: Educause Learning Initiative, September 2005, Educause. Accessed at 18.06.2007
Lorenzo, G. and Ittelson, J. (2005c) Demonstrating and assessing student learning with e-portfolios. ELI Paper 3: Educause Learning Initiative, October 2005, Educause. Accessed at on 18.06.2007
Ittelson, J. (2001) Building an e-dentity for each student. EDUCAUSE Quarterly, 24, 4, 2001.
Ward, R. (2006) Editorial. Centre for Recording Achievement Newsletter, December 2006. Accessed at on 18.06.2007
Webpages with background and other useful information
Helen Barrett’s site, especially where e-portfolios are demonstrated/
Helen Barrett’s presentation on the practical implications of planning for e-portfolios
1
June2007/MHW
Appendix 1: E-portfolios
Newcastle (UK): (Test site available - login details on request)
The University of Newcastle Medical School has developed an e-portfolio site that is especially suited to the specific assessment requirements of clinical skills for medical students.
Type of system / Generic e-portfolio site - has been developed at Newcastle and can be configured to suit. Creation date unknown but nothing added to site since 2004Aim and users / “…to satisfy both requirements for PDP in the wider HE sector and increasing demands within Medicine for reflective clinical practice and to develop life-long learners with skills and experience appropriate for appraisal, assessment and on-going professional revalidation.”
Used by medical and dental students at Newcastle (UK) and possibly another university
Content /
Assessment / The PDP and reflective learning diary are organized around specific assessable student outcomes. These are grouped as:
What the doctor is able to do
Clinical skills
Practical procedures
Patient investigation
Patient management
Health promotion and disease prevention
Communication
Data and information handling skills
How the doctor approaches practice
Understanding of basic and clinical sciences and underlying principles
Appropriate attitudes, ethical understanding and legal responsibilities
Appropriate decision making, clinical reasoning and judgement
The doctor as a professional
Personal development
Professional development
Ease of use / The student enters the details of each clinical skill.
The system is complex, but comprehensive record of everything the student has done.
Ability to cross-reference data: eg under communication could link a skill to a conference presentation
Evaluation / Documents of several evaluation presentations at
Costs / Unknown
Maintenance
Model / Unknown
Emed – UNSW Department of Medical Education
Contact Chris Hughes