Overview

Background

This is an opportunity to extend MBChB’s use of Practique beyond written exams and into OSCEs (Objective Structured Clinical Examination) for the next academic year 2018-19. Utilising the same team who deployed Practique seamlessly into MBChB (over 80 written exams executed already), should ensure that the continued development of support for our student body and the MBChB assessment team, academics and administrative functions continues.

Previous completed projects that should be referenced are:

https://www.projects.ed.ac.uk/project/mvm107  Procurement

https://www.projects.ed.ac.uk/project/mvm118  Implementation, contract and support

https://www.projects.ed.ac.uk/project/mvm119  Data migration and creation

 

Scope

The activities in this project will include:

  • Extend deployment of  Practique from FRY-IT - cloud based servers hosted on Amazon Web Servers - to execute all OSCE exams plus the existing written paper based assessments throughout MBChB
  • Provision of training for MBChB users, examiners and IS support staff (where appropriate)
  • Configuration and setup of UoE platforms that will support running the assessments
  • Support a training session for the project core team on July 10, 2018
  • Develop a pilot to enable MBChB examiners to assess Practique prior to exams
  • Communication to all stakeholders
  • Exam day preparation and support
  • Support MBChB change - not just the technical elements surrounding the introduction of a new exam application, but the structural, process and people based change that will emerge alongside the technical delivery
  • Progress the Practique User Group - these colleagues will work with FRY-IT to shape future releases, MBChB requirements and review all change emerging across the platform,   this group meet twice a year consisting of representatives from all of the supplier clients within the UK, such as Imperial, Leeds, Brighton etc
    • Support developing requirements for improved reporting (commenced) which is priority number 1 for UoE at the PUG
  • Maintain the relationship for the project with the supplier - currently monthly meetings with assessment team alongside adhoc as required
  • Develop a strategy and planning approach with the assessment team to consider the next three years use of Practique
  • Submission to and presentation for UoE learning and teaching conference on June 20 https://www.ed.ac.uk/institute-academic-development/learning-teaching/cpd/workshops/learning-teaching-conference
  • Continued support of all testing for future releases (V 5.4 released in mid June, V6 in late August) with some specific UoE requirements contained within
  • Assist MBChB in updating the current blueprint, particularly within the definition of skills
  • In parallel the written exams will be introducing a new question type - Very Short Answer Question (VSAQ)
    • Support the assessment team in constructing, testing and then creating the new guidelines for module organisers to create new questions
  • Continue developing engagement within the Medical School to review requirements surrounding their use of a variety of assessment tools
    • Commenced presentation and response to Clinical Brain Science who currently use QMP
    • Planned meeting with the Teaching Organisations who currently use elements of Speedwell (BMTO and MTO)
  • Manage contract updates with Legal Services, Procurement and suppliers:-
    • Existing Fry-IT contract will require amending along with finance
    • Existing Speedwell contract for scanners to process marks
  • Decommission scanners and remove support:-
    • Remove physical servers once confirmed that they are not used by anyone else
    • Review and potentially remove MVM IS Support from a need to provide support for this function
  • Creation of mobile device management platform for configuration of iPads

 

This project follows these previous completed projects:- MVM107, MVM119 and MVM118.

 

https://www.projects.ed.ac.uk/unpublished/project/mvm118/risks/7 This risk from MVM118 articulates the issue of MBChB executing the type of structural changes that are required, but which the introduction of Practique has accelerated. I have closed it as part of that project, acknowledging that MBChB have made huge strides in this space. However, challenges do remain which the team are aware of, but note that it's mainly out of the power of this project to influence.

Further significant changes are likely to be 'anthropological rather than technological’, so we may introduce technological change, but we need humans to execute it, hopefully, we will determine what we want the technology to do, not the other way round!

 

Objectives and Deliverables

 

No.

Description

Priority     

Owner     
O1  Deploy an updated assessment and feedback function within Practique covering OSCE's for MBChB.    

D1

 Deliver signed contract change

  M

Project

D2

 Execute training programme

  M

Project
D3  Create and deliver detailed implementation framework for OSCE assessments in 2018-19 academic year   M Project
D4  Support the creation and import of OSCE exam content   M

Project

D5  Decommission all scanners, support and update existing contracts   M Project
D6  Manage the update of new blueprint requirements   M Project 
D7  Provide exam day support   M Project
O2  Ensure written exam support is not affected by the introduction of OSCE    
D8  Maintain monthly supplier meetings and organise support of PUG attendance   M Project
D9  Ensure MBChB assessment team are fully briefed on any exam day updates   M Project
D10  Manage the release and testing for all Practique release updates for the duration of the project   M Project
D11  Manage the update of existing blueprint entries - British Medical Council updates + UoE requirements   M Project
D12  Assist in the development and production of a Practique strategy document - where it might be developed in next 3 years   M Project

 

 

Benefits

 

Improving assessment and feedback of the students is considered the number one priority for the Medical School. This will be improved further by the enhancements and level of support for Practique provided by this project. It is a strategic target to improve student satisfaction and it is believed that this tool will greatly enhance the student experience. Whilst it remains too early to measure changes in student satisfaction, the introduction of Practique for written exams in the 2017-18 academic year has provided the MBChB assessment team with many examples of positive student responses.

This second phase will more directly impact OSCE examiners, who will no longer have to manage a significant amount of physical paper. However, students will also receive Practique candidate reports similar to those received for written, so it remains a great opportunity to continue developing constructive feedback for them.

It is anticipated that there will be a further benefit from decommissioning physical scanner equipment along-with their support and contract costs. Currently we have no confirmation of existing costs attributed to them, but have requested the information.

As MBChB continue to adopt Practique, it is expected that the assessment team workload might be re-prioritised as exam assessment construction, execution and reporting is managed within one system.

Project Service engagement with MBChB and Fry-IT is enabling us to help prioritise development which delivers the most effective change for the benefit of the UoE.

Success Criteria

Supporting the execution of a series of OSCE exams during 2018-19 with quality data and improved best practise.

Project Milestones

as of July 18th, 2018

Target Date Previous Date Title Stage Complete  
20-Apr-2018 No date available Purchase iPads and charging trolleys Initiate Yes

view

03-Jul-2018 29-Jun-2018 Confirm build of iPads for UoE usage Plan Yes

view

10-Jul-2018 No date available OSCE training day with Fry-IT and MBChB core project team members Plan Yes

view

23-Aug-2018 No date available Update existing Practique contract Execute No

view

28-Sep-2018 No date available Pilot commences Execute No

view

19-Oct-2018 No date available Western Hospital wifi upgraded Execute No

view

16-Nov-2018 No date available Low stakes go live Execute No

view

25-Jan-2019 No date available Year 5 go-live Deliver No

view

15-Feb-2019 No date available Year 6 go-live Deliver No

view

05-Apr-2019 No date available Assess phase 2 deliveries Deliver No view
11-Jun-2019 No date available Close Project Close No

view

Project Info

Project
Support for OSCEs in Practique for MBChB
Code
MVM129
Programme
MVM Learning and Teaching (MVMLAT)
Management Office
ISG PMO
Project Manager
Paul De'Ath
Project Sponsor
David Kluth
Current Stage
Close
Status
Closed
Project Classification
Grow
Start Date
01-Jun-2018
Planning Date
20-Jul-2018
Delivery Date
05-Apr-2019
Close Date
11-Jun-2019
Programme Priority
1
Overall Priority
Normal
Category
Discretionary