Overview

Background

 

Each of our VLE platforms records a large amount of data about student interactions, but the ability to report on this, and in particular to display this information to students is not well developed.  Basic course reporting facilities exist for staff which allow them to see student behaviour on an individual basis, or interaction with a particular course element. There is little or no comparative reporting capability however (e.g. student X relative to the average for the course). Blackboard sell a companion Learn Analytics product, but it is separate to the VLE, and exclusively geared towards staff viewing student activities, and not students getting feedback on their own activity. It is also significantly more heavy-weight than we need.

This project will build on work carried out in Jan - July 2014 (https://www.projects.ed.ac.uk/project/tel009). The TEL009 project report provided a “wish list” of types of data and display options which may be included in the development. This project will continue to make use of staff and student consultation events to steer the technical and associated non-technical developments, to ensure that there is transparency and clarity on the aims and objectives of the project throughout. This will also help ensure that the key information and appropriate display formats are prioritised. 

 

The project will set up a steering group to advise the prohect team on prioritisation and QA. At End of Planning the project team will undertake an Analysis Stage where the cost and risks of the deliverables associated with the objectives in the "wish list" will be fleshed out. The project team will then work closely with the Steering Group to prioritise and plan and pilot the deliverables. The Steering Group will also oversee the companion Moodle VLE project with which IS Applications is not involved.

 

Scope

Work to make student analytic data from the LEARN environment available to students.

Data to be viewed by students within the LEARN environment.

Data to be made visible will already be collected in LEARN..

Data from one course able to be accessed from one view.

Incorporation of  Student VLE functionality into new and existing  LEARN courses.

Staff control of data visibility at tool or graded event level, within a course. 

Business analysis to identify changes to processes and define requirements.

 

 

Out of Scope

 

Any work to produce views of student analytic information in Moodle.

Collection of and making available of any data not already collected within LEARN

Making LEARN data visible outwith  LEARN.

Granularity of control at individual student level, i.e. visibility of reports will only be at course level, not student by student. 

On-demand refresh of data.

Programme-level views.

 

Objectives and Deliverables

 

 DescriptionPriorityNotes
O1Work with stakeholders to prioritise and QA project deliverables.  
D1Set up Steering Group.MTo oversee Moodle project in tandem with TEL017
D2Produce costings and risks associated with deliverables to inform prioritisation.M 
    
O2To allow a student to view information about their own learning activity and grades within LEARN. Some already possible, but  with no context.
D3Process to allow a student to view their own grades within a new view.MAs prioritised with steering group
D4Process to allow a student to view their own login activity within a new view.M 
D5Process to allow students to view their own learning activity.HDWhich "learning activity/activities" to be prioritised  with steering group
    
O3Anonymising of data visible to a student which relates to activity or grades of other students. if data is not labelled with user - is it effectively anonymous?
D6Process to anonymise student dataMHow often  this will run, and when, to be agreed.
    
    
O4To allow a student to view  their own grades or learning activity against that of the cohort. Define learning activity
D7Process to allow students to view their own grades against the cohort.MAs prioritised with steering group ( just as against avergare?)
D8Process to allow students to view their own login activity against the cohort.M 
D9Process to allow students to view their own learning activity against that of the cohort.,HDAs prioritised with steering group 
    
O5A single destination where  grade and activity data for each course is made available to students.  
D10A single  destination  within LEARN  for each course  allowing students to view learning data  for that course.HDAs prioritised with steering group
    
O6To allow staff to select  which data will be viewable by   students  and when.  
D11Admin interface to allow staff to control visibility of data to student cohort. ( yes/no/when)MOptions as  prioritised with steering group 
    
O7To evaluate student activity, satisfaction and feedback and evaluate staff satisfaction and feedback after piloting  (in order to ensure solution delivered is acceptable to students. Staff may include personal tutors as well as course owner.
D12Deliver prototype solution for evaluation.M 
D13Hold evaluation events with stakeholders.M 
D14Pilot student VLE solution with one or more courses. D(information  in pilot might not be delivered through LEARN)
D15Make student data available to students on one or more pilot courses.MFor evaluation purposes.
D16Report on results of pilot evaluationsMTo feed into changes required for  final deliverables.
    
O8Data to be retained for student and staff viewing (for a period to be agreed)  
D16Procedure to store data for completed courses.HDAs prioritised with steering group
D17Procedure to retrieve data for students from their completed courses.HDAs prioritised with steering group
    
O9Incorporation of  Student VLE functionality into new and existing  LEARN courses.  
D18Procedure to add Student VLE functionality to existing coursesMAt the discretion of course owner.
D19Procedure to add Student VLE functionality to new  coursesMAt the discretion of course owner.
    
O10Business analysis to identify changes to processes and define  and  requirements,M 
D20Report assisting project sponsor and steering group to prioritise requirements.M 

 

 

Benefits

 

Some of the benefits can only be assessed some time after the project has delivered.  A benefits realisation and assessment exercise will be carried out by TEL in the academic year 2015/16

  1. Meet the requirements from students and schools for more information about student activity and performance.
  2. Make more effective use of data which already exists, and making it more transparent to the students.
  3. Providing feedback to students via a view of their progress and activity through the course, to help inform their reflection and longitudinal engagement.
  4. Providing feedback to students through access to contextualised information about their performance within the cohort.
  5. Empowering students to identify key indicators of performance and take appropriate actions, including requests for interventions.
  6. Supporting dialogue between students and tutors on activity, performace and progress.
  7. Helping staff identify possible problem areas of their courses and thus enhance course and activity design.
  8. Improving general student satisfaction with feedback.

Success Criteria

 

  1. Evidence that course organisers have chosen to use the new functionality. (This will be assessed by TEL post-project delivery)
  2. A single process  delivered enabling for course organisers to make learning activity information available to students.
  3. Students can view their grades  and logins for a particular course within the LEARN environment.
  4. Students can compare their grades and logins  for a particular course against the anonymised grades of the other students on that course.
  5. Students can view their learning activity for a particular course.
  6. Students can view their learning activity for a particular course against anonymised data representing the learning activity of the other students on that course,
  7. Staff able to tailor visibility  (and timing of visibility) of learning data.
  8. Process to allow monitoring, analysis and evaluation of student activity on pilot courses.
  9. No impact on current performance of LEARN.
  10. No impact on security or privacy of student data.
  11. Existing staff processes used to examine VLE data to continue working as expected.
  12. Student VLE functionality incorporated into set up of courses.