Improving the student experience is a strategic goal for the University and the College of Sciences and Engineering. Assessment and marking has been noted as an area for improvement for the School of Biological Sciences in the exam review and the school National Student Survey(NSS).
The School of Biological Sciences have been using the centrally supported Questionmark Perception (QMP) online application for exams with multiple choice questions but no online application is used to support short and long essay questions. These exams are currently handwritten. Exams for Year 3 & 4 in the School of Biological Sciences both include short answers and a long essay. There are no multiple choice questions exam in Year 4.
The School of Biological Sciences identified this lack in online assessment for exams in Year 3 & 4 as an area that could be improved and that would lead to a better student experience. Online essays tend to be shorter than handwritten ones but of better quality thanks to the ability to edit the draft. It removes any issue with deciphering handwriting for markers. It also allows for better tracking/monitoring of progress made with marking. It supports better and faster feedback to the student.
The School of Biological Sciences selected the ExamOnline application from Intelligent Assessment Technologies because it supports the inclusion of hand drawn diagrams. This feature is of critical importance because all essay exams in that field are expected support the inclusion of hand drawn diagrams.
The scope of this project was :
- to support a trial run of ExamOnline in exam conditions in the School of Biological Sciences
- to gather and assess user feedback on their experience with ExamOnline
- to evaluate the suitability and benefits of using ExamOnline for the assessment and marking of Year 3 & 4 biology exams.
- to provide recommendations for possible next steps
Objectives and deliverables
|Objective 1||To run a mock exam using ExamOnline in exam conditions|
|D1.1||Obtain trial license to ExamOnline||Mandatory||Achieved|
|D1.2||Book suitable student lab||Mandatory||Achieved|
Recruit student volunteers
|D1.5||Secure local IT support||Mandatory||Achieved|
To gather user feedback and assess user experience
|D2.1||Select parameters and success criteria for evaluation||Mandatory||Achieved|
|D2.2||Capture feedback from IT support staff||Mandatory||Achieved|
|D2.3||Observe and report on student experience during the trial||Mandatory||Achieved|
|D2.4||Develop a survey to capture the student experience of ExamOnline for taking an exam and of receiving feedback||Mandatory||Achieved|
|D2.6||Capture feedback from academic staff||Mandatory||Partially achieved|
|Objective 3||To evaluate ExamOnline and its suitability for Year 3 & 4 exams|
|Objective 4||To recommendations for possible next steps|
|D4.1||Suggest next steps||Mandatory||Achieved|
Analysis of Resource Usage:
Staff Usage Estimate: 10 days
Staff Usage Actual: 10 days
Staff Usage Variance: 0%
Key Learning Points
- Effective involvement from the local IT support team was critical in delivering the trial.
- This project was intiated after the trial preparations were already underway within Biological Sciences which limited the scope of the project. The main concerns were identified as risks from the start, i.e. the focus of the trial restricted to one course, and not enough time to effectively plan the evaluation or to put in place the required resources. These proved to be significant issues throughout the project. It has been challenging to balance the need for a solution with wide appeal and a tailored solution. The availability of the project manager was an issue in the post evaluation phase.
Recommendations for future similar projects:
- Involvement of ISG at an earlier stage. More time in the initial stages would have been beneficial in helping the stakeholders understand better the evaluation process and success criteria, and promote an active participation. This would have prevented re-drafting of the evaluation reports in the later stages of the project.
- Pre-planning briefing for non-ISG staff on ISG procedures would have set more realistic expectations.
Recommendations for next steps include further trials within other courses, schools and colleges. No ownership has been assigned as to who should be taking this forward.