Overview

Background

Over the years, we have invested in our physical teaching spaces to ensure they are highly accessible, easy to use and that we have the capacity we need.  Since 2016, significant investment in the Virtual Learning Environment (VLE) has been undertaken to help improve the staff and student digital experience via the VLE Excellence programme.

The VLE Excellence programme has been developed to make a similar investment in our virtual teaching spaces along with laying foundations for the future by taking advantage of new technologies, gaining a better understanding what students need and expect from a digital learning experience, and embedding good practices to support users across the institution.

To date, this has been delivered through several large-scale projects:

• TEL038: VLE Consolidation – Consolidation of bespoke VLE’s into the University’s centrally supported VLE, Advanced Learn. 
• TEL058: Learn to the Cloud – Moving Learn to cloud hosting to improve availability and out of hours support.  This was completed in July 2019. 
• TEL057: Learn Foundations – Setting and supporting consistent standards for Learn use via the Learn Foundations project. This was completed in January 2022. 
• VLE000: Learn Ultra – Upgrading the existing version of Learn to the most up to date version, Learn Ultra.  This project is in its 3^rd and final year (due to close August 2024) where it will focus on embedding and evaluation of the change delivered by the project.    

As we move forward with VLE Excellence programme, the next step in the journey is to look at optimising the assessment and feedback workflows within the VLE via the LOUISA project.

Learn Optimised for In-course Submission and Assessment – LOUISA

In 2018, the Learn Foundations project was initiated as part of the institutional response to the Quality Assurance Agency Enhancement Themes (Evidence for Enhancement: Improving the Student Experience).

During the Learn Foundations project, a programme of user experience (UX) was undertaken with both staff and students to understand where the key pain points were when it came to experiences with the Learn Virtual Learning Environment (VLE):

• To understand the student perspective, this programme conducted qualitative face-to-face research with 34 different students from 15 different Schools and collected over 4,000 responses from students in quantitative research which provided valuable insights into the needs and expectations of our student audiences. 
• To investigate staff needs, 15 qualitative research sessions with different types of staff from nine different Schools were undertaken and 264 staff responses were received through qualitative research which has created a narrative of what the potential staff challenges may be. 

This programme of UX research looked at many aspects of the user experience of both staff and students when using the VLE, identifying multiple areas of pain across multiple user journeys in a variety of areas such as: accessing or logging into Learn, accessing lecture notes, submitting assignments, and receiving feedback. 

Why LOUISA?

The LOUISA project is named after a pioneering Scottish advocate for woman’s university education – Louisa Stevenson, who received an honorary LLD (Doctor of Laws) from the University of Edinburgh in 1906 and was one of the co-founders of the Queen Margaret University in Edinburgh.

Given the scale and scope of the project, it was felt that the project name should be reflective, easy to remember and links in with the sister project FLORA (Formal exams, Learning, Online Rubrics and Assessment) that will look at the digital exam landscape within the University of Edinburgh

Background

LOUISA will build on the existing knowledge gathered on pain points in relation to assessment and feedback to help understand changes required to enhance and provide a more consistent student and staff experience moving forward.  Findings identified via Learn Foundations included:

• On various occasions students struggled with the workflow to submit coursework;
• Inconsistencies and complexities within the submission processes across the University;
• Complexity of workflows and differences between and even within Schools resulted in a complex, disjointed experience for students, which did not facilitate development of or support for systems to tailor for their needs; Assignments were found to be a major pain point for both students and staff. Many people remarked on inconsistent processes from course to course and School to School. 
• Similarly, students’ concerns around the consistency of feedback were mirrored in staff members’ comments. A complex set of emotions and experiences around the issue of feedback were identified and it was noted that there is a challenge around providing feedback in a consistent format for students.  
• Staff members often put a lot of effort to provide good feedback and feel frustrated if students don’t always read it. 
• The ongoing challenge of providing high quality, targeted staff development support for appropriate, consistent use of learning technology tools when everyone is using the tools in diverse ways. (Appropriate use of learning technology is one of the core areas in gaining HEA accreditation)

The LOUISA project will support quality learning and teaching by removing unnecessary complexities, particularly in the administration of assessment and the return of feedback to students whilst underpinning key elements of the Enhancement Led Institutional Review (ELIR) 2020 reflective analysis, the University’s Strategy 2030, the Digital Strategy and the Curriculum Transformation Programme by enhancing the student experience and providing a more consistent approach to assessment and feedback when using Learning Technology.

As part of the institutional response to the 2020 ELIR reflective analysis, LOUISA supports the Assessment and Feedback Principles and Priorities, in particular principle 7.1: Learning technologies shall be used to facilitate efficient, user-friendly and effective assessment, marking and feedback for students and staff and will work closely with Curriculum Transformation and the  assessment and feedback  strategy group where LTW has representation.  The project will also engage with the final report produced for the Quality Enhancement and Standards Review (QESR) completed in November 2023 due to the additional recommendation on assessment and feedback.

The LOUISA project will support in the delivery of the University’s overarching Strategy 2030 by feeding into the following:

• We aim to achieve excellence in all that we do, always being principled, considerate, and respectful; 
• Our teaching is relevant to society, and we are diverse, inclusive, and accessible to all; 
• We are ambitious, bold and act with integrity, always being willing to listen; 
• We are a place of transformation and of self-improvement, driven to achieve benefit for individuals, communities, societies, and our world. 

We will have student representatives on the project board along with students working in the project as part of the project team, gaining professional skills, working collaboratively on authentic tasks, and engaging in complex professional thinking. 

Business Analysis of current assessment and feedback workflows 

In 2022 (and continued into 2023), business analysis was undertaken via the Learn Ultra Upgrade project to gain an understanding of the current landscape across the University around assessment and feedback within the VLE. 

During this analysis, a review of existing tools and technologies was undertaken to understand what tools/technologies had been adopted and why along with undertaking a gap analysis.  From this analysis, the feedback received from both staff and students via the UX (user experience) activities in recent years, it was evident that the same challenges still exist and cause frustration for users.

LOUISA will build upon both the business analysis and the user research undertaken via the VLE Excellence programme, looking at the pain points identified and understanding where improvements or efficiencies can be made.

Policies 

There are several existing academic policies which relate to use of Learn and will form a consideration (where needed) for the work being undertaken via LOUISA, including: 

• Taught Assessment Regulations; 
• Undergraduate Progression Boards Policy; 
• Special Circumstances Policy; 
• Assessment and Feedback Principles & Priorities; 
• Accessible and Inclusive Learning Policy; 
• Learning Analytics; 

We will work closely with the Educational Technology Policy Officer who will be reviewing the Learning Analytics policy to ensure that the work of this project supports the implementation of this policy.

Vision 

Our vision is in 3 parts around data, workflows, and user experience:

• To make the experience of assessment and feedback in the virtual learning environment more usable, accessible, inclusive, and equitable.
• We will have a limited set of defined workflows that support a consistent experience, and all core assessment tools will be integrated with Learn and the Learn Gradebook, where possible, providing a rich dataset for reporting and understanding
• All unnecessary complexities and inconsistencies in workflows around assessment and feedback within the VLE are designed out so that students and teachers find it easy to submit, assess and receive or provide feedback on assessments in a consistent manner. Teaching support staff will find it easy to transfer knowledge of assessment and feedback processes between schools.

Principles  

The following principles form the foundations of the LOUISA project:

• Remove unnecessary complexities and inconsistencies; 
• Bring together all the information that staff and students need to do a task relating to assessment and feedback; 
• Use simple language, without jargon or acronyms where necessary; 
• We will have more user-friendly processes and efficient systems to support our work;
• Have training available to support staff in their use of the VLE for assessment and feedback;
• Use accessible design from the ground up;
• Students will experience our systems as accessible, inclusive, connected, and easy to use.,
• All assessment is in Learn;
• Have an ‘EDI lens’ on use of assessment tools to ensure that students are not disadvantaged by any systems or processes in this project.

Scope

LOUISA is a 3-year project, initiated in the 23/24 academic year focusing on:

In Scope

• All courses on the VLE (on-campus and online along with undergraduate and postgraduate);
• All coursework contained within VLE;
• Creation of consistent approaches to assessment and feedback within the VLE;
• Collaboration with the FLORA project to have the right assessments on the right platform(s).
• Review, design, and delivery of a suite of training courses to support with assessment and feedback within the VLE;
• Review of both VLE-native and VLE-integrated assessment and feedback tools to support key assessment practices;
• User experience review of current assessment and feedback workflows;
• Programme of communications and engagement to gain buy-in;
• Management and support of both the Learning Analytics in UltRA (LAURA) and Scoping AI Developments in EdTech at Edinburgh (SADIE) projects via the LOUISA governance structure:
  1. Ongoing review of AI assessment tool features being proposed by vendors and technology partners as part of the SADIE project;
  2. Review of learning analytics within the portfolio of learning technology as part of the LAURA project.

Out of Scope

• Other VLEs (such as Moodle) – where appropriate, solutions should be platform agnostic;
• Development or procurement of new tools or systems to support with assessment and feedback;
• Remote proctoring and invigilation;
• Monitoring of staff performance;
• Changes (or developments) to existing integrations and tools.

Objectives and Deliverables

To support with the delivery of LOUISA, the above Objectives and Deliverables will be split across the following projects:

Benefits

Upon the delivery of LOUISA, the following benefits should be achieved: 

• A deeper understanding of the assessment and feedback landscape across the University;
• Improved student experience – tackling a key area of usability concern identified via Learn Foundations in relation to assessment and feedback; 
• Improved support for academic and administrative colleagues in Schools where support resources and systems can be optimised for a smaller number of workflows; 
• Improved understanding of the assessment landscape in the wider HE community. 
• Enhanced accessibility for staff and students whilst using the VLE for the purposes of assessment and feedback, supporting the requirements of the Accessible and Inclusive Learning Policy (2023); 
• A consistent approach to assessment and feedback with the VLE and integrated tools; 
• Increased utility of Learn VLE delivering a return in investment (supporting more of the business and providing better reporting);
• Colleagues across Schools will gain insights into the variety of ways assessment is delivered and managed along with having the opportunity to share insights and practice through sharing of workflows, experience, and good practice.

Success Criteria

Upon the delivery of the LOUISA project, the following success criteria should be achieved: 

• The removal of unnecessary complexities found within School workflows to better enhance the student experience along with saving on time and effort in Schools, improve support service efficiencies and improve the student experience of assessment across the University; 
• The successful delivery of a new minimum standards within the VLE for assessment and feedback that meets business requirements; 
• A new and consistent approach to assessment and feedback within the VLE;
• An understanding as to institutional requirements for assessment and feedback to answer the following question: Do the current learning technologies fit all requirements or is a review required? 
• Learn being the home of all assessment information with other tools (such as Turnitin) being used for originality scores.  Digital exams in the right place.
• Adoption of preferred workflows by Schools; 
• Positive feedback from staff and students. 

Project Milestones