Closure Report

Project Summary

Background

Improving the student experience is a strategic goal for the University and the College of Sciences and Engineering. Assessment and marking has been noted as an area for improvement for the School of Biological Sciences in the exam review and the school National Student Survey(NSS).

The School of Biological Sciences have been using the centrally supported Questionmark Perception (QMP) online application for exams with multiple choice questions but no online application is used to support short and long essay questions. These exams are currently handwritten. Exams for Year 3 & 4 in the School of Biological Sciences both include short answers and a long essay. There are no multiple choice questions exam in Year 4.

The School of Biological Sciences identified this lack in online assessment for exams in Year 3 & 4 as an area that could be improved and that would lead to a better student experience. Online essays tend to be shorter than handwritten ones but of better quality thanks to the ability to edit the draft. It removes any issue with deciphering handwriting for markers. It also allows for better tracking/monitoring of progress made with marking. It supports better and faster feedback to the student.

The School of Biological Sciences selected the ExamOnline application from Intelligent Assessment Technologies because it supports the inclusion of hand drawn diagrams. This feature is of critical importance because all essay exams in that field are expected support the inclusion of hand drawn diagrams.

Scope

The scope of this project was :

  • to support a trial run of ExamOnline in exam conditions in the School of Biological Sciences
  • to gather and assess user feedback on their experience with ExamOnline
  • to evaluate the suitability and benefits of using ExamOnline for the assessment and marking of Year 3 & 4 biology exams.
  • to provide recommendations for possible next steps

Objectives and deliverables

# Description Priority Outcome
Objective 1 To run a mock exam using ExamOnline in exam conditions
D1.1 Obtain trial license to ExamOnline Mandatory Achieved
D1.2 Book suitable student lab Mandatory Achieved
D1.3

Recruit student volunteers

Mandatory Achieved
D1.4 Deploy ExamOnline Mandatory Achieved
D1.5 Secure local IT support Mandatory Achieved
Objective 2

To gather user feedback and assess user experience

D2.1 Select parameters and success criteria for evaluation Mandatory Achieved
D2.2 Capture feedback from IT support staff Mandatory Achieved
D2.3 Observe and report on student experience during the trial  Mandatory Achieved
D2.4 Develop a survey to capture the student experience of ExamOnline for taking an exam and of receiving feedback Mandatory Achieved
D2.5 Survey students Mandatory Achieved
D2.6 Capture feedback from academic staff Mandatory Partially achieved
Objective 3 To evaluate ExamOnline and its suitability for Year 3 & 4 exams
D3.1 Evaluation report Mandatory Achieved
Objective 4 To recommendations for possible next steps
D4.1 Suggest next steps Mandatory Achieved

 

Analysis of Resource Usage:

Staff Usage Estimate: 10 days

Staff Usage Actual: 10 days

Staff Usage Variance: 0%

 

Outcome

Key Learning Points

  • Effective involvement from the local IT support team was critical in delivering the trial.
  • This project was intiated after the trial preparations were already underway within Biological Sciences which limited the scope of the project. The main concerns were identified as risks from the start, i.e. the focus of the trial restricted to one course, and not enough time to effectively plan the evaluation or to put in place the required resources. These proved to be significant issues throughout the project. It has been challenging to balance the need for a solution with wide appeal and a tailored solution. The availability of the project manager was an issue in the post evaluation phase.

Recommendations for future similar projects:

  • Involvement of ISG at an earlier stage. More time in the initial stages would have been beneficial in helping the stakeholders understand better the evaluation process and success criteria, and promote an active participation. This would have prevented re-drafting of the evaluation reports in the later stages of the project.
  • Pre-planning briefing for non-ISG staff on ISG procedures would have set more realistic expectations.

 

Outstanding Issues

Recommendations for next steps include further trials within other courses, schools and colleges. No ownership has been assigned as to who should be taking this forward.

 

 

Project Info

Project
ExamOnline Trial
Code
SCE011
Programme
SCE Portfolio Projects (OTHSCE)
Management Office
ISG PMO
Project Manager
Muriel Mewissen
Project Sponsor
David Cavanagh
Current Stage
Close
Status
Closed
Start Date
20-Oct-2016
Planning Date
n/a
Delivery Date
n/a
Close Date
28-Aug-2017
Programme Priority
1
Overall Priority
Normal
Category
Discretionary