This is an opportunity to extend MBChB’s use of Practique beyond written exams and into OSCEs (Objective Structured Clinical Examination) for the next academic year 2018-19. Utilising the same team who deployed Practique seamlessly into MBChB (over 80 written exams executed already), should ensure that the continued development of support for our student body and the MBChB assessment team, academics and administrative functions continues.
Previous completed projects that should be referenced are:
https://www.projects.ed.ac.uk/project/mvm118 Implementation, contract and support
https://www.projects.ed.ac.uk/project/mvm119 Data migration and creation
The activities in this project will include:
- Extend deployment of Practique from FRY-IT - cloud based servers hosted on Amazon Web Servers - to execute all OSCE exams plus the existing written paper based assessments throughout MBChB
- Provision of training for MBChB users, examiners and IS support staff (where appropriate)
- Configuration and setup of UoE platforms that will support running the assessments
- Support a training session for the project core team on July 10, 2018
- Develop a pilot to enable MBChB examiners to assess Practique prior to exams
- Communication to all stakeholders
- Exam day preparation and support
- Support MBChB change - not just the technical elements surrounding the introduction of a new exam application, but the structural, process and people based change that will emerge alongside the technical delivery
- Progress the Practique User Group - these colleagues will work with FRY-IT to shape future releases, MBChB requirements and review all change emerging across the platform, this group meet twice a year consisting of representatives from all of the supplier clients within the UK, such as Imperial, Leeds, Brighton etc
- Support developing requirements for improved reporting (commenced) which is priority number 1 for UoE at the PUG
- Maintain the relationship for the project with the supplier - currently monthly meetings with assessment team alongside adhoc as required
- Develop a strategy and planning approach with the assessment team to consider the next three years use of Practique
- Submission to and presentation for UoE learning and teaching conference on June 20 https://www.ed.ac.uk/institute-academic-development/learning-teaching/cpd/workshops/learning-teaching-conference
- Continued support of all testing for future releases (V 5.4 released in mid June, V6 in late August) with some specific UoE requirements contained within
- Assist MBChB in updating the current blueprint, particularly within the definition of skills
- In parallel the written exams will be introducing a new question type - Very Short Answer Question (VSAQ)
- Support the assessment team in constructing, testing and then creating the new guidelines for module organisers to create new questions
- Continue developing engagement within the Medical School to review requirements surrounding their use of a variety of assessment tools
- Commenced presentation and response to Clinical Brain Science who currently use QMP
- Planned meeting with the Teaching Organisations who currently use elements of Speedwell (BMTO and MTO)
- Manage contract updates with Legal Services, Procurement and suppliers:-
- Existing Fry-IT contract will require amending along with finance
- Existing Speedwell contract for scanners to process marks
- Decommission scanners and remove support:-
- Remove physical servers once confirmed that they are not used by anyone else
- Review and potentially remove MVM IS Support from a need to provide support for this function
- Creation of mobile device management platform for configuration of iPads
This project follows these previous completed projects:- MVM107, MVM119 and MVM118.
https://www.projects.ed.ac.uk/unpublished/project/mvm118/risks/7 This risk from MVM118 articulates the issue of MBChB executing the type of structural changes that are required, but which the introduction of Practique has accelerated. I have closed it as part of that project, acknowledging that MBChB have made huge strides in this space. However, challenges do remain which the team are aware of, but note that it's mainly out of the power of this project to influence.
Further significant changes are likely to be 'anthropological rather than technological’, so we may introduce technological change, but we need humans to execute it, hopefully, we will determine what we want the technology to do, not the other way round!
Objectives and Deliverables
|O1||Deploy an updated assessment and feedback function within Practique covering OSCE's for MBChB.|
|Deliver signed contract change||
Execute training programme
|D3||Create and deliver detailed implementation framework for OSCE assessments in 2018-19 academic year||M||Project|
|D4||Support the creation and import of OSCE exam content||M||
|D5||Decommission all scanners, support and update existing contracts||M||Project|
|D6||Manage the update of new blueprint requirements||M||Project|
|D7||Provide exam day support||M||Project|
|O2||Ensure written exam support is not affected by the introduction of OSCE|
|D8||Maintain monthly supplier meetings and organise support of PUG attendance||M||Project|
|D9||Ensure MBChB assessment team are fully briefed on any exam day updates||M||Project|
|D10||Manage the release and testing for all Practique release updates for the duration of the project||M||Project|
|D11||Manage the update of existing blueprint entries - British Medical Council updates + UoE requirements||M||Project|
|D12||Assist in the development and production of a Practique strategy document - where it might be developed in next 3 years||M||Project|
Improving assessment and feedback of the students is considered the number one priority for the Medical School. This will be improved further by the enhancements and level of support for Practique provided by this project. It is a strategic target to improve student satisfaction and it is believed that this tool will greatly enhance the student experience. Whilst it remains too early to measure changes in student satisfaction, the introduction of Practique for written exams in the 2017-18 academic year has provided the MBChB assessment team with many examples of positive student responses.
This second phase will more directly impact OSCE examiners, who will no longer have to manage a significant amount of physical paper. However, students will also receive Practique candidate reports similar to those received for written, so it remains a great opportunity to continue developing constructive feedback for them.
It is anticipated that there will be a further benefit from decommissioning physical scanner equipment along-with their support and contract costs. Currently we have no confirmation of existing costs attributed to them, but have requested the information.
As MBChB continue to adopt Practique, it is expected that the assessment team workload might be re-prioritised as exam assessment construction, execution and reporting is managed within one system.
Project Service engagement with MBChB and Fry-IT is enabling us to help prioritise development which delivers the most effective change for the benefit of the UoE.
Supporting the execution of a series of OSCE exams during 2018-19 with quality data and improved best practise.
as of July 18th, 2018
|Target Date||Previous Date||Title||Stage||Complete|
|20-Apr-2018||No date available||Purchase iPads and charging trolleys||Initiate||Yes|
|03-Jul-2018||29-Jun-2018||Confirm build of iPads for UoE usage||Plan||Yes|
|10-Jul-2018||No date available||OSCE training day with Fry-IT and MBChB core project team members||Plan||Yes|
|23-Aug-2018||No date available||Update existing Practique contract||Execute||No|
|28-Sep-2018||No date available||Pilot commences||Execute||No|
|19-Oct-2018||No date available||Western Hospital wifi upgraded||Execute||No|
|16-Nov-2018||No date available||Low stakes go live||Execute||No|
|25-Jan-2019||No date available||Year 5 go-live||Deliver||No|
|15-Feb-2019||No date available||Year 6 go-live||Deliver||No|
|05-Apr-2019||No date available||Assess phase 2 deliveries||Deliver||No||view|
|11-Jun-2019||No date available||Close Project||Close||No|