Overview

Background

This is the third year of work in implementing and embedding Practique's use in the College. This is viewed as a year of stabilisation, extending the practitioners/disciplines using the software and enabling the college to work towards the benefits of working from one consolidated platform. Overall this project is expected to:

  • have a positive impact on Student Experience given the feedback from students already taking some exams in Practique
  • provide a recurring benefit of reduction in administration time once the solutions are live
  • provide the College with more of its exams in one consolidated platform

This project is building on the work already delivered over the last two years in:

  • MVM107 Procurement of online application
  • MVM118 Deployment of FRY-IT Practique for MBChB
  • MVM119 Data Creation and Migration into Practique for MBChB
  • MVM129 Support for OSCEs in Practique for MBChB

 

Scope

This project will implement 3 new aspects of Practique:

  • MBChB Admissions
  • OSCEs for Vets
  • Written Exams for BMTO

 

Objectives and Deliverables

Practique Roll-Out Priority (MoSCoW) Owner
O1 Gap analysis between current processes and Practique    
O1D1 Exam Fit analysis session for Admissions M - Completed  
O1D2 Exam Fit report for Admissions M Fry (Supplier)
O1D3 Exam Fit analysis session for Vets M - Completed  
O1D4 Exam Fit report for Vets M Fry (Supplier)
O1D5 Exam Fit analysis session for BMTO M - Completed  
O1D6 Exam Fit report for BMTO M Fry (Supplier)
O2 Implement new instances    
O2D1 Delivery of Admissions Instance M - Completed Fry (Supplier)
O2D2 Delivery of Vets/BMTO Instance

M

Fry (Supplier)
O2D3 Training Day each for Admissions, Vets and BMTO M Fry (Supplier)/UoE Staff
O2D4 Creation of content for Admissions, Vets and BMTO in Practique M School Staff
O2D5 SSO to be implemented for new instances (as per current) S Project Manager/UoE and Supplier tech staff
O2D6 Pilot for Admissions S - Completed School Staff/Fry (Supplier)
O2D7 Pilot for Vets  S School Staff/Fry (Supplier)
O2D8 Support calls with supplier for Admissions, Vets and BMTO following respective training days to go Live M School Staff/Fry (Supplier)
O2D9 Build of ipads for Admissions and Vets M IT Lead
O3 Delivery using Practique    
O3D1 Admissions Assessment Days for MBChB M School Staff/Fry (Supplier)/IT Lead
O3D2 OSCEs for Vets M School Staff/Fry (Supplier)/IT Lead
O3D3 Written Exams for BMTO M School Staff/Fry (Supplier)/IT Lead

Benefits

Supplier anticipated Impact and Benefits are:

Our end-to-end e-assessment management service means you can have all your assessments managed from one integrated platform including Admissions/Structured Interviews, OSCEs, MCQs, EMQs, SAQs and VSAQs. Some key benefits are:   

For the student 

  • Give your students feedback more quickly and when it matters with  automatically generated feedback from exams
  • Improve the outcomes of your students by having better quality feedback  that is automatically linked to their assessments

For the faculty and leadership team

  • Gain an overview of the assessment and student performance to drive  better learning
  • Improve the quality and reliability of your assessments with inbuilt analytics  and reporting
  • Access to new and innovative assessment types to ensure that you can trial  and introduce new assessment types as the medical assessment landscape  continues to change
  • Remote access for all users allowing multi-site involvement, secure exam  bank management and remote question authoring
  • Demonstrate to the GMC that assessments meet gold standard practice

For the administration

  • Reduce the need for administrative tasks and increase time spent on  activities that add value
  • Remote access for all users allowing multi-site involvement, secure exam  bank management and remote question authoring
  • Immediate results processing - Save hours of time with electronic results  capture and inbuilt analytics. Our customers have reported saving between  50 and 80 person hours per OSCE exam using Practique
  • Benefit from the extensive experience of the Fry e-assessment team who  have been supporting thousands of candidates take OSCEs (over 50% of UK  medical graduates have sat a summative OSCE on Practique)

Benefits anticipated by Sponsor and Senior User for Admissions:

The introduction of a new admissions process, with the incorporation of face to face assessments, carries a risk in terms of credibility of Edinburgh medical school, and the ability to deliver the process.

The use of e-assessments is likely to confer significant benefits:

  • The lack of need for multiple paper copies of the assessment process improves security
  • The software is intuitive to use, and will help assessors focus on the assessment process itself, rather than concerns about the technology
  • It will allow rapid uptake of outcomes, and will speed up the process, allowing us to inform applicants of our decision earlier. We hope that this will increase conversion rates
  • It will support assessors, giving them a positive experience, making them more likely to volunteer in the future
  • It will allow timely and uniform feedback to candidates, to a level agreed prior to commencement of the assessments.
  • It will liberate the admissions team from having to upload the scores from paper copies
  • It will reduce the risk of transcription errors.

Benefits anticipated by Senior User for Vets:

The R(D)SVS is aware of the many benefits experienced by MBChB colleagues having attended and discussed extensively prior to the project, and therefore anticipates:

  • Significant time saving for the assessment team
  • Preparation activity – thousands (estimated 40k) of printed mark sheets previously used no longer need to be printed and stored.
  • Exam day activity - reduction in collecting papers and organising examiners
  • Post activity – will be automated therefore expect similar benefits to MBChB team including less stressful organisation, examiner experience benefits etc.
  • Due to immediate availability of data and reports after OSCEs take place, School will be able to spend more time reviewing the results before sending to students

 

Benefits anticipated by Senior User for BMTO:

  • For 2018/19 there are approximately 4800 UG student course enrolments within BMS.  A new system would reduce risk and allow a more robust way of managing our current multiple choice question assessment and examination processes.
  • There would be a time saving for both academic and professional services staff when setting and approving assessments as well as a reduction in time for returning feedback for some assessment types.
  • Exam paper creation using data gathered within the system.  Currently not possible other than by manual keying in Word question banks – not possible at present as there is no resource within the BMTO to undertake this time consuming task. 
  •  Exam paper approval process within one system removing the use of email and password protected files. 
  •  Remove risk of correct answer tags being left in exam papers.
  •  A new online system would allow for standard setting which is not currently possible.
  •  Delivery of timely, detailed and bespoke feedback to students on their performance in MCQs/EMQs.
  •  Potential to use one system to deliver online tests rather than using multiple different systems.
  •  Potential to move to online delivery of more exams/assessments rather than paper based.
  •  BMS UG students are a potential growth area within CMVM.  This investment would facilitate future growth.

 

Success Criteria

  • Admissions: Would use again - Successful use of Practique for MBChB's Admissions Assessment days. Given this is new for the Medical School, it is expected that there will be opportunities to learn for following years - what went well and should be maintained, what could have gone better and should be changed for next year.
  • Vets: Would use again - Successful use of Practique for OSCEs. Given this is new software for Vets, it is anticipated that the School will wish to expand its use
  • BMTO: Would use again - Successful production of August written exam using Practique. It is anticipated that if successful, the School will wish to expand its use.

Project Milestones

 

Stage Milestone Due Date
Execute Exam Fit Sessions 11-Oct-2019
Execute Delivery of Admissions Exam Fit report 01-Nov-2019
Execute Admissions Pilot 06-Nov-2019
Plan Project brief approval 08-Nov-2019
Execute Delivery of BMTO Exam Fit report 08-Nov-2019
Execute Admissions Training Day 18-Nov-2019
Execute Delivery of Vets Exam Fit Report 15-Nov-2019
Execute Admissions Business Ready checkpoint 02-Dec-2019
Execute Admissions LIVE 09-Dec-2019
Execute BMTO Training Day 24-Feb-2020
Execute Vets Training Day 25-Feb-2020
Execute Vets Pilot 17-Mar-2020
Deliver Vets LIVE 29-May-2020
Deliver Delivery of all Practique elements 17-Jul-2020
Deliver BMTO LIVE 17-Jul-2020
Close Project Closure approved 31-Jul-2020

Project Info

Project
Practique Roll-Out
Code
MVM136
Programme
MVM Learning and Teaching (MVMLAT)
Management Office
ISG PMO
Project Manager
Susan Ridder-Patrick
Project Sponsor
Susan Rhind
Current Stage
Close
Status
Closed
Project Classification
Grow
Start Date
20-Sep-2019
Planning Date
10-Oct-2019
Delivery Date
07-May-2021
Close Date
21-May-2021
Overall Priority
Normal
Category
Discretionary