Learning Analytics

Q1.1 Introduction

Q1.2 Learning Analytics is a new scientific domain that aims to use educational data produced by students and teachers through their browsing and interaction behaviour in order to produce new insights into the learning and teaching processes, and to improve them. A typical example for Learning Analytics is the analysis of data logs from online learning systems (e.g. LMS). This analysis is then shown to users in a visualisation which makes them aware of their performance and progress. Sometimes this is combined with the possibility to compare yourself to other peers, in order to position yourself in a group of learners. The data is also useful to teachers to see how a group of learners develops shared knowledge. The survey below will take around 12mins to complete. It aims to explore the expectations and priorities that stakeholders put into Learning Analytics. Questions are organised in seven parts (including some personal info) along our Learning Analytics framework model (below), which consists six key dimensions (outer circle) and related sub-domains (inner circle).

Learning Analytics framwork (Greller & Drachsler 2011)

Q2.1 Personal Information (page 1 of 7)

Q2.2 Which education sector do you work in?

Primary education (1)

Secondary education (2)

Adult education (3)

Vocational education (4)

Further education (5)

Higher education (6)

Commercial training provider (7)

Other (8) ______

Q2.3 What is your role in your institution (multiple answers possible)

Student (1)

Teacher (2)

System Administrator (3)

Support staff (4)

Librarian (5)

Researcher (6)

Learning Designer (7)

Senior Manager (8)

Other (please specify) (9) ______

Q3.1 Stakeholders (page 2 of 7)

Q3.2 In your opinion, who will benefit the most from Learning Analytics?Please rank the following answers according to your opinion (drag&drop)

______Learners (1)

______Teachers (2)

______Education Institution (School, College, University, etc.) (3)

______Parents (4)

______Third party service providers (companies) (5)

______Other: please specify (6)

Q3.3 Which relationships between stakeholder groups do you think Learning Analytics will strengthen most?Tick a max of 5 potential relationships (e.g. learner to learner)

to / to / to / to
teacher (1) / learner (1) / institution (1) / parents (1)
teacher (1) /  /  /  / 
learner (2) /  /  /  / 
institution (3) /  /  /  / 
parents (4) /  /  /  / 

Q3.4 Feel free to give us any comments that come to your mind

Q4.1 Objectives (page 3 of 7)

Q4.2 In your opinion, how much will Learning Analytics influence the following areas?Please rate

don't know (1) / not at all (2) / a little (3) / very much (4)
morepersonalised learning experiences for learners (1) /  /  /  / 
more timely information about the learning progress (2) /  /  /  / 
more detailed information about the learners' context (3) /  /  /  / 
better integration of IT services within schools/institutions (4) /  /  /  / 
easier grading possibilities for teachers (5) /  /  /  / 
more neutral and objective assessment of learning (6) /  /  /  / 
pedagogic improvements and innovation (7) /  /  /  / 
better insight by institutions on what's happening in each course (8) /  /  /  / 
more effective and more economic teaching structures (9) /  /  /  / 
additional and new learning services (10) /  /  /  / 
stimulate reflection of learners on their progress (11) /  /  /  / 
predict potential drop out of students (12) /  /  /  / 
detecting motivation and attitude of learners (13) /  /  /  / 

Q4.3 Which added innovation can Learning Analytics bring to educational technologies?Please rank the following items by using drag&drop

______more adaptive systems (1)

______content recommendations (2)

______recommendation of peers (3)

______datavisualisation of learning (e.g., via dashboards) (4)

______peer comparison (5)

______clustering of students (6)

______detecting undesirable student behaviour (7)

______other (please specify) (8)

Q4.4 What should the main objective for Learning Analytics be?

stimulating stakeholders to reflect on their performance and learning behaviour (1)

predict learner performance and lead to automatic system adaptation (2)

make invisible information about learners visible (3)

Q4.5 When thinking of impact , how much attention should Learning Analytics pay to...?

______Pedagogic creativity (1)

______Performance Standards (2)

______Educational experimentation (3)

______Quality Assurance (4)

______Innovation (5)

______Mainstream activities (6)

Q4.6 Feel free to give us any comments that come to your mind

Q5.1 Educational Data (page 4 of 7)

Q5.2 How important do you consider the following attributes of educational datasets?

don't know (1) / not important (2) / important (3) / highly important (4)
shared within an institution (1) /  /  /  / 
open across institutions for sharing (2) /  /  /  / 
version-controlled (3) /  /  /  / 
anonymised (4) /  /  /  / 
in standard format (5) /  /  /  / 
added context information (6) /  /  /  / 
containing meta information (7) /  /  /  / 

Q5.3 Which IT systems does your organization use for teaching and learning?

Learning management system (e.g., Blackboard, Moodle) (1)

Course management system (2)

Student information system (3)

E-portfolio system (e.g., Mahara) (4)

Social networking platform (5)

Intranet (6)

Wiki platform (7)

Mobile platform (e.g., Apps, e-books) (8)

Sensor data (e.g., location data) (9)

External services (e.g., Google docs, Facebook, Twitter, slideshare, iTunes U) (10)

other (please specify) (11) ______

Q5.4 Would you share educational datasets of learners openly in an international research community, provided these datasets are anonymised according to standard principles?

Yes (1)

No (2)

Don't know (3)

Q5.5 Feel free to give us any comments that come to your mind

Q6.1 Methods (page 5 of 7)

Q6.2 Learning Analytics is based on algorithms (formulas), methods, and theories that translate data into meaningful information. How hopeful are you that these methods produce an accurate picture of learning in the following areas?

______a comprehensive view on somebody's learning progress (1)

______an accurate view of a learners engagement with a subject discipline (2)

______an accurate prediction on how a learner will perform (3)

______a good assessment of a learners current state of knowledge (4)

______a suitable prediction of relevant learning resources (5)

______a suitable prediction of relevant peers (6)

______an accurate comparison between peer learners (7)

Q6.3 Feel free to give us any comments that come to your mind

Q7.1 Constraints (page 6 of 7)

Q7.2 In your opinion, how much will Learning Analytics influence the following subjects?

don't know (1) / not at all (2) / a little (3) / very much (4)
extended privacy regulations (1) /  /  /  / 
ethical policies (2) /  /  /  / 
data ownership (3) /  /  /  / 
data openness (4) /  /  /  / 
transparency of education (5) /  /  /  / 

Q7.3 Learning Analytics utilisesdata which is protected by data protection legislation. Do you think a national standard anonymisation process would alleviate fears of data abuse?

Yes (1)

No (2)

Don't know (3)

Q7.4 Does your institution have and operate ethical guidelines that regulate the use of student data (for example in research)?

Yes (1)

No (2)

Don't know (3)

Q7.5 Should institutions internally share all the data they hold on students with all members of staff (academic, technical, administrative, research)?

Yes (1)

No (2)

Don't know (3)

Comment (4) ______

Q7.6 How much do you feel Learning Analytics and automated data gathering affect the privacy and rights of individuals?

don't know (1) / not at all (2) / a little (3) / very much (4)
Breaches of privacy and intrusion in personal affairs (1) /  /  /  / 
Ethical principles of sex, political and religious beliefs, ethnic origin (2) /  /  /  / 
Ownership rights and intellectual property (3) /  /  /  / 
Freedom of expression (4) /  /  /  / 

Q7.7 Feel free to give us any comments that come to your mind

Q8.1 Competences (page 7 of 7)

Q8.2 How important do you consider the following skills to be present in learners to benefit from Learning Analytics?

don't know (1) / not important (2) / important (3) / highly important (4)
Numerical skills (1) /  /  /  / 
IT literacy (2) /  /  /  / 
Critical reflection (3) /  /  /  / 
Evaluation skills (4) /  /  /  / 
Ethical thinking (5) /  /  /  / 
Analytical skills (6) /  /  /  / 
Self-directedness (7) /  /  /  / 

Q8.3 Do you think the majority of learners would be competent enough to learn from their Learning Analytics reports?

Yes (1)

No, they would need support (2)

Don't know (3)

Q8.4 How sure can we be that end users will draw the right conclusions from their data, and decide on the best course of action?

______learner determining their next step (1)

______teachers to determine the next step for their students (2)

______Institutions to take positive policy decisions (3)

Q8.5 Feel free to give us any comment that comes to your mind