Closure Report

Project Summary

Background

This project will review the current use of the University’s main virtual learning environment, Blackboard Learn, and support the adoption of a standard course presentation across the institution, as outlined in proposals to the Senate Learning & Teaching Committee on 20th September 2017. Feedback from the Learning and Technologies Advisory Group (LTAG) indicates some work on the graphic design would be welcome, and Headscape have identified the need for the VLE services to be more clearly identified with the University. The proposals support the University’s Learning and Teaching Strategy with the aim to improve NSS scores as a result of consistency in VLE course outlines. Whilst the main focus of the project is to improve Undergraduate courses in the VLE, we expect the project to deliver benefits that can be applied to all types of course, including blended and fully online. The project will run 2017-19.

Inconsistency in the use of Virtual Learning Environments (VLEs) has led to dissatisfaction amongst students across the HE sector and analysis of the NSS data indicates that the introduction of a standard may address some areas of dissatisfaction (Reed et al, 2015). The largest VLE used at Edinburgh is Blackboard Learn, used by over 80% of our academic courses. Consultations in 2016-17 with staff and students by the Learn Service team and the commissioned Headscape review in 2016 indicate that inconsistencies across all our internal and external systems cause frustration and dissatisfaction for students.

A recent ISG innovation fund project looking at students’ perceptions of Learn indicates that accessing course content and information in a clear and consistent way is a priority for students. This is supported by wider research on students’ negative experiences of digital technology in Higher Education (Selwyn, 2016) as well as the importance of ‘hygiene factors’; the extrinsic factors that can impact on satisfaction and well-being in places of work and study.

The recent change in NSS questions to shift focus toward how IT ‘supports students’ learning’ rather than on access alone means that addressing these inconsistencies in online student experience needs to be a priority.

Research indicates that minimal VLE standard is a hygiene factor that can address areas of dissatisfaction as well as provide the foundation upon which learning and teaching innovation may flourish (Reed et al, 2015).

Many other Universities have already implemented standards or thresholds for VLE usage, including: Surrey, York, Oxford, Edge Hill, Swansea, Aston, Newcastle, Sheffield Hallam, York St John, Surrey, Aberystwyth, South Wales, Cardiff, Leicester, Northampton and Bedfordshire.

Scope

This project aims to improve the digital experience for all students and users of the VLE, specifically by addressing two key priorities in the 2017 Learning and Teaching Strategy:

 

  1. Improve the digital experience for all students and users of the VLE by reviewing and enhancing ‘the experience of students on joint Honours programmes’ and thereby reducing the citation of Learn as a source of dissatisfaction.
  2. reviewing the way our ‘digital estates support high quality learning and teaching and interaction between staff and students’.

It will do this by evaluating current inconsistencies in the way that courses are presented and navigated in Blackboard Learn and seek to adopt a consistent standard for all programmes of study, including Joint Honours programmes.

Progress and outputs from the VLE Standards work will be included in institutional reporting to the Quality Assurance Agency Scotland (QAA).

This project will also work closely with the teams working on the new lecture recording and Resource Lists services, the leads of which will be part of the project steering group, to ensure that they are integrated into the standard VLE course presentation. The  project will aim to build on lessons learned from the VLE Consolidation projects and outputs from Service Excellence and the User Centred Portal project by delivering cultural change via better more consistent use of the VLE.

 

Objectives and Deliverables

Objective / Deliverable Ref

Description

Priority

(MoSCow)

Achieved?

O1

Baseline VLE usage in UoE.

 

 

D1.1

Establish project governance.

MUST

Yes

D1.2

Conduct consultation exercise to benchmark current VLE usage and identify challenges in adopting standards.

MUST

Partial

D1.3

Pilot across one or more schools / programmes to assess feasibility and resourcing for standard course design in Learn. 

SHOULD

Yes

D1.4

Complete Equality Impact Assessment (EQIA).

MUST

No

D1.5

Recommendations Report July 2018 outlining options for next steps to roll out to wider institution.

MUST

No

D1.6

Agree plan based on 2017/18 review and pilot(s).

MUST

No

D1.7

Agree and document VLE Standard policy/guidance.

SHOULD

No

     

 

O2

Adoption of an institution-wide VLE standard.

 

 

D2.1

Creation and management of  VLE Standard templates within the VLE.

MUST

No

D2.2

Agree Communications plan for rollout of new standards.​

MUST

No

D2.3

Agree Support and Training plan(s) for rollout of new standards.​

MUST

No

D2.4

Rollout of standards across agreed set of courses.

MUST

No

D2.5

Embed agreed standard templates into IS VLE service and activities including subsequent upgrades, training and advice.

MUST

No

D2.6

Ongoing review and consultation of adoption of VLE standards and technical assets.

MUST

No

     

 

O3

Deliver a more consistent and improved VLE  Experience.

 

 

D3.1

Review baseline measures for areas to be targetted including reducing mentions of VLE as source of dissatisfaction.

MUST

No

D3.2

Set targets for improvements in agreed areas including Resource Lists and Lecture Recording.

MUST

No

D3.4

Review and set targets to reduce staff dissatisfaction in VLE use.

SHOULD

No

Benefits and Success Criteria

Develop a deeper understanding of our user requirements and priorities with regards to use of the VLE

    • Acquired via desk research and user consultation across all schools/colleges, Measured through collation of the following and presented as published report
    • Previous UK and International research
    • User Experience analysis
    • Consultation/feedback gathered via multiple methods, including surveys and focus groups

 

  • Greater consistency in course presentations within the VLE leading to student satisfaction with VLE and location of key course information within the VLE.
    • Measured by NSS, case studies and other surveys – with fewer incidences of students citing the VLE and navigation of courses as a source of frustration
    • Comparison will be undertaken with existing baseline and available evaluations

 

  • Better student experience for those studying on Joint Honours programmes, especially where those programmes are across two schools
    • Measured via targeted piloting of VLE Standards on Joint Programmes and evaluated via case studies and focus groups indicating that consistency has resolved a source of student dissatisfaction on Joint Honours programmes.

 

  • Supporting and enhancing the requirements of the Accessible and Inclusive Learning Policy (2016) by extending to the online domain.
    • Measured by potential updating of existing policy or drafting of new one to demonstrate support for this standard.

 

  • Enable greater scaffolding and support for academic and professional staff to develop courses in a more consistent way, and create a greater understanding of how the VLE works to support their course context

    • Measured by increase of training sessions attendances, support material/guidance, example courses and templates
    • Positive feedback from school support staff , Case studies and evaluation that templates have improved support for course development within the VLE.
    • Additional training materials and support documents (materials can be aligned to promote consistency and be made relevant and accessible to more staff and students)

 

  • Schools can benefit from approaches already being adopted by other schools/units/programmes as key elements and exemplary behaviours will become more visible
    • Statistics from tool review to see trends in course structure and usage compared to courses prior to template (Quantitative)
    • Improved connections within Learn User Group and other Learning Technology initiatives for sharing focussed interests in course development (Qualitative)

Project Milestones

Target Date

Title

Stage

Achieved?

15-Dec-2017

End of Planning

Plan

Yes

19-Jan-2018

Establish Project Board

Execute

Yes

18-May-2018

Complete Consultation exercise

Execute

Yes

25-May-2018

New Templates Available

Deliver

Yes

15-Jun-2018

Recommendations Report

Deliver

No

22-Jun-2018

Agree Terms of Reference for Pilot(s) Semester 1 (2018/19)

Deliver

Yes

14-Dec-2018

Complete Pilot(s) Semester 1 (2018/19)

Execute

Partial

25-Jan-2019

Review Pilot(s) and Agree plan for Rollout

Execute

No

25-Jan-2019

Agree Communications Plan

Execute

No

25-Jan-2019

Agree Support & Training Plan

Execute

No

24-May-2019

Institutional Templates Available

Deliver

No

28-Jun-2019

Complete Rollout across institution

Deliver

No

30-Aug-2019

Post Rollout Support

Deliver

No

13-Sep-2019

Delivery Sign Off Review

Deliver

No

27-Sep-2019

Project Closure

Close

No

 

Analysis of Resource Usage:

Staff Usage Estimate: 160.25 days

Staff Usage Actual: 78.1 days

Other Resource Estimate: £xxx

Other Resource Actual: £xxx

Other Resource Variance: xx%

Outcome

Following the initial research & consultation exercises it became apparent that the scope of this project needed to be increased in order to deliver the project goal.  In addition to the project scope increasing, original milestones became irrelevant and new ones needed to be created and a new project brief needed to be written.  Given the scale of change it was agreed with the Project Sponsor and Head of Project Services that best way forward would be to close TEL049 and start up a new re-defined project with a rewritten Project Brief, scope, and milestones.

Explanation for variance

As above, this project was not completed and a new project will be opened.

Key Learning Points

- The solution to improve consistency in the VLE cannot be to just deliver a standardised template

- Communication will be key to the project and a communications plan needs to be carefully devised to ensure users are bought in to the solution

- More needs to be done to understand the student experience before a suitable solution can be delivered

Outstanding Issues

- More consultation needs to be done with users (staff and students) to identify requirements

- Current 'quality' and 'usage' of Learn needs to be baselined in order to measure the effectiveness of the project

- Staff need to be well supported throughout the change - the current training isn't fit for purpose and needs to be revisited

Project Info

Project
VLE Standards
Code
TEL049
Programme
ISG - Technology Enhanced Learning (TEL)
Management Office
ISG PMO
Project Manager
Laura Woods-Dunlop
Project Sponsor
Melissa Highton
Current Stage
Close
Status
Withdrawn
Start Date
16-Oct-2017
Planning Date
n/a
Delivery Date
n/a
Close Date
24-Aug-2018
Programme Priority
2
Overall Priority
Normal
Category
Discretionary

Documentation

Close