Success Criteria

Efforts were made to gather a set of requirements and success criteria for an online application for the School of Biological Sciences against which the student experience and software would be evaluated. Time was very limited as the project started less than a month before the trial. This was noted as risk 2. A suitable draft list was produced but it would have benefited from further refinements, and from the input of a larger group of staff to better represent the varied, and potential different, needs with the school and college, see sce011-_success_criteria_-_0.2.pdf

It was not possible to assess all the success criteria during this short evaluation. Unlike a formal procurement, we did not have access to all staff or information required. For example, affordability could not be measured against a budget as no fund is currently available for this. Similarly, assessing the SLA information would have required a support team to be involved but ownership of the service is not determined. In some cases, the evidence is only anecdotal and would benefit from a more extensive and formal evaluation. Some of the functionality that could not be tested during the trial including providing access to feedback within the application, granting access to external markers and moderation of the marking process because of access/security restrictions.

The full review of success criteria is available at: sce011-_success_criteria_-_ltw_ede_evaluation_-_1.2.pdf

 

The evaluation shows that ExamOnline fully meets fully about half of the must have requirements, with another 20% of must have requirements being partially met. Three quarters of the should have requirement are fully met.

 

We note that this only provides an overview rather than a formal scoring of the application. In this limited excercise, all requirements have been accounted for without the usual weighting that would be included in a procurement excercise and that would capture the relative importance given to a requirement by the various stakeholders/areas of business. In this case, some of the requirements that are not met are of particular importance to deployment of the application and the student experience like the lack of support for accessibility and poor support for EASE or group managemet or error messages present during testing. These need to be given proper consideration.

 

Requirement Priority   Fully met Partially met Not met Not assessed
Must Have 21 11 4 5 1
Should Have 16 12 1 1 2
Could Have 2 2 0 0 0

 

LTW EDE also analysed the feedback provided by students, the results of which is presented in this report: ltw_ede_examonline_evaluation.pdf

 

Project Info

Project
ExamOnline Trial
Code
SCE011
Programme
SCE Portfolio Projects (OTHSCE)
Management Office
ISG PMO
Project Manager
Muriel Mewissen
Project Sponsor
David Cavanagh
Current Stage
Close
Status
Closed
Start Date
20-Oct-2016
Planning Date
n/a
Delivery Date
n/a
Close Date
28-Aug-2017
Programme Priority
1
Overall Priority
Normal
Category
Discretionary