Overview

Background

Assessment and ranking of student attainment in the University’s medical school is carried out both for internal purposes (e.g. the MBChB degree, honours awards), for onward progression (e.g. applications for Foundation Training) and to support long-term sector planning (the UK Clinical Aptitude Test follow-up study).  All these information needs are currently managed via a number of complex Excel spreadsheets that require a large amount of administration and detailed user knowledge.  The spreadsheets are heavily formulated and inter-linked, and rely upon a master data source (the ‘Assessment Engine’) that is due to be retired in the course of the 2017/2018 academic year. 

The current system carries a very high administrative burden, and there is a gap in support following the departure of the reporting manager, who created the Excel spreadsheets.  This increases the likelihood of the serious risk of user error and the reputational damage that would arise from an incorrect degree result being published, or ranking for Foundation Training applications being incorrect. 

Maintaining the existing system is not an option.  There is therefore a need to update and improve the approach to the sourcing of data, cohort identification, assessment and ranking calculations and the onward dissemination of these outputs. 

Scope

This project will deliver a business systems analysis of the processes and data required for the following assessments and rankings:

Intercalated Honours selection;

Foundation Application Educational Performance Measures;

Award of MBChB with Honours;

UK Clinical Aptitude Test (UKCAT);

The analysis will define the as-is and to-be processes, data, sources and systems required to support the above information needs.  It will recommend a technology choice for the preferred solution, and define the development and data migration projects required to deliver that solution.

A preliminary mapping of the as-is spreadsheet-based process has already been carried out by the Digital Learning Applications & Media team, based upon review of the existing spreadsheets.  This will be used as the basis of, and further elaborated in, the business analysis.

Out of Scope

No other existing or planned types of assessments or rankings will be within the scope of this project.  However, it is assumed that the solution identified will be capable of extension to include other assessment rankings within the University’s College of Medicine & Veterinary Medicine.  Stakeholders in Veterinary Medicine and in Biomedical Sciences will be informed of the project and solution, but will not be required to contribute objectives or requirements.

Objectives & Deliverables

The College has an overall objective of using the new system for the in-scope rankings in 2017/2018.   Historical data (2010/2011 – 2016/2017) used by the current process is being migrated from its current store (the ‘Assessment Engine’) to the Euclid system.  At the same time two projects (MVM118 and MVM119) are being initiated to implement a new examinations system (‘Practique’) and migrate the data that it requires from legacy systems.  The first assessment data to be mastered in ‘Practique’ will be the end-of-semester exams of December 2017.  It is assumed that the solution proposed by MVM121 will need to be ready to consume this data.  In order to ensure successful control of dependencies, awareness of and co-ordination with the timescales of MVM118 and MVM119 will be an objective of project management activities in the present project and its successors.

Benefits

There are no direct financial benefits arising from this business analysis. 

However the following overall benefits can be assumed to arise from the delivery of the proposed solution:

• Reduction in person-hours spent administering the production of the assessment rankings; 
• Improvement in timeliness of response to external requests, e.g. for the UKCAT dataset; 
• Reduction of likelihood of reputational risk arising from the provision or publication of inaccurate assessments and rankings. 

The preferred solution product of this analysis will quantify these benefits.

Success Criteria

The following success criteria have been initially identified for the overall delivery of the solution, and should therefore be supported by the business analysis:

• The tool should eliminate, or at least reduce to an acceptable minimum, the amount of human intervention required in identifying and selecting student cohorts and their assessments. 
• The tool should support amendment and update of parameters such as weighting factors without requiring technically skilled intervention.
• The tool should be capable of extension to include other assessments and rankings used by the Colleges. 

Project Milestones

Target Date	Previous Date	Title	Stage	Complete
26-May-2017	19-May-2017	Project Brief sign-off	Plan	No	view
26-May-2017	19-May-2017	Planning sign-off	Plan	No	view
04-Aug-2017	No date available	Delivery	Deliver	No	view
31-Aug-2017	No date available	Project Closure	Close	No	view