Overview
Background
Each of our VLE platforms records a large amount of data about student interactions, but the ability to report on this, and in particular to display this information to students is not well developed. Basic course reporting facilities exist for staff which allow them to see student behaviour on an individual basis, or interaction with a particular course element. There is little or no comparative reporting capability however (e.g. student X relative to the average for the course). Blackboard sell a companion Learn Analytics product, but it is separate to the VLE, and exclusively geared towards staff viewing student activities, and not students getting feedback on their own activity. It is also significantly more heavy-weight than we need.
This project will build on work carried out in Jan - July 2014 (https://www.projects.ed.ac.uk/project/tel009). The TEL009 project report provided a “wish list” of types of data and display options which may be included in the development. This project will continue to make use of staff and student consultation events to steer the technical and associated non-technical developments, to ensure that there is transparency and clarity on the aims and objectives of the project throughout. This will also help ensure that the key information and appropriate display formats are prioritised.
The project will set up a steering group to advise the prohect team on prioritisation and QA. At End of Planning the project team will undertake an Analysis Stage where the cost and risks of the deliverables associated with the objectives in the "wish list" will be fleshed out. The project team will then work closely with the Steering Group to prioritise and plan and pilot the deliverables. The Steering Group will also oversee the companion Moodle VLE project with which IS Applications is not involved.
Scope
Work to make student analytic data from the LEARN environment available to students.
Data to be viewed by students within the LEARN environment.
Data to be made visible will already be collected in LEARN..
Data from one course able to be accessed from one view.
Incorporation of Student VLE functionality into new and existing LEARN courses.
Staff control of data visibility at tool or graded event level, within a course.
Business analysis to identify changes to processes and define requirements.
Out of Scope
Any work to produce views of student analytic information in Moodle.
Collection of and making available of any data not already collected within LEARN
Making LEARN data visible outwith LEARN.
Granularity of control at individual student level, i.e. visibility of reports will only be at course level, not student by student.
On-demand refresh of data.
Programme-level views.
Objectives and Deliverables
| Description | Priority | Notes | |
|---|---|---|---|
| O1 | Work with stakeholders to prioritise and QA project deliverables. | ||
| D1 | Set up Steering Group. | M | To oversee Moodle project in tandem with TEL017 |
| D2 | Produce costings and risks associated with deliverables to inform prioritisation. | M | |
| O2 | To allow a student to view information about their own learning activity and grades within LEARN. | Some already possible, but with no context. | |
| D3 | Process to allow a student to view their own grades within a new view. | M | As prioritised with steering group |
| D4 | Process to allow a student to view their own login activity within a new view. | M | |
| D5 | Process to allow students to view their own learning activity. | HD | Which "learning activity/activities" to be prioritised with steering group |
| O3 | Anonymising of data visible to a student which relates to activity or grades of other students. | if data is not labelled with user - is it effectively anonymous? | |
| D6 | Process to anonymise student data | M | How often this will run, and when, to be agreed. |
| O4 | To allow a student to view their own grades or learning activity against that of the cohort. | Define learning activity | |
| D7 | Process to allow students to view their own grades against the cohort. | M | As prioritised with steering group ( just as against avergare?) |
| D8 | Process to allow students to view their own login activity against the cohort. | M | |
| D9 | Process to allow students to view their own learning activity against that of the cohort., | HD | As prioritised with steering group |
| O5 | A single destination where grade and activity data for each course is made available to students. | ||
| D10 | A single destination within LEARN for each course allowing students to view learning data for that course. | HD | As prioritised with steering group |
| O6 | To allow staff to select which data will be viewable by students and when. | ||
| D11 | Admin interface to allow staff to control visibility of data to student cohort. ( yes/no/when) | M | Options as prioritised with steering group |
| O7 | To evaluate student activity, satisfaction and feedback and evaluate staff satisfaction and feedback after piloting (in order to ensure solution delivered is acceptable to students. | Staff may include personal tutors as well as course owner. | |
| D12 | Deliver prototype solution for evaluation. | M | |
| D13 | Hold evaluation events with stakeholders. | M | |
| D14 | Pilot student VLE solution with one or more courses. | D | (information in pilot might not be delivered through LEARN) |
| D15 | Make student data available to students on one or more pilot courses. | M | For evaluation purposes. |
| D16 | Report on results of pilot evaluations | M | To feed into changes required for final deliverables. |
| O8 | Data to be retained for student and staff viewing (for a period to be agreed) | ||
| D16 | Procedure to store data for completed courses. | HD | As prioritised with steering group |
| D17 | Procedure to retrieve data for students from their completed courses. | HD | As prioritised with steering group |
| O9 | Incorporation of Student VLE functionality into new and existing LEARN courses. | ||
| D18 | Procedure to add Student VLE functionality to existing courses | M | At the discretion of course owner. |
| D19 | Procedure to add Student VLE functionality to new courses | M | At the discretion of course owner. |
| O10 | Business analysis to identify changes to processes and define and requirements, | M | |
| D20 | Report assisting project sponsor and steering group to prioritise requirements. | M |
Benefits
Some of the benefits can only be assessed some time after the project has delivered. A benefits realisation and assessment exercise will be carried out by TEL in the academic year 2015/16
- Meet the requirements from students and schools for more information about student activity and performance.
- Make more effective use of data which already exists, and making it more transparent to the students.
- Providing feedback to students via a view of their progress and activity through the course, to help inform their reflection and longitudinal engagement.
- Providing feedback to students through access to contextualised information about their performance within the cohort.
- Empowering students to identify key indicators of performance and take appropriate actions, including requests for interventions.
- Supporting dialogue between students and tutors on activity, performace and progress.
- Helping staff identify possible problem areas of their courses and thus enhance course and activity design.
- Improving general student satisfaction with feedback.
Success Criteria
- Evidence that course organisers have chosen to use the new functionality. (This will be assessed by TEL post-project delivery)
- A single process delivered enabling for course organisers to make learning activity information available to students.
- Students can view their grades and logins for a particular course within the LEARN environment.
- Students can compare their grades and logins for a particular course against the anonymised grades of the other students on that course.
- Students can view their learning activity for a particular course.
- Students can view their learning activity for a particular course against anonymised data representing the learning activity of the other students on that course,
- Staff able to tailor visibility (and timing of visibility) of learning data.
- Process to allow monitoring, analysis and evaluation of student activity on pilot courses.
- No impact on current performance of LEARN.
- No impact on security or privacy of student data.
- Existing staff processes used to examine VLE data to continue working as expected.
- Student VLE functionality incorporated into set up of courses.