Closure Review
Project Summary:
Contributors
Role | Department | Name |
Project Manager (Owner) | IS Apps Project Services | Jamie Thin |
Programme Manager | IS Apps Project Services | Jamie Thin |
Production Management Coordinator | IS Apps Apps Mgmnt | Suran Perera |
Business Area Manager |
Timetabling Unit, Academic Registry | Scott Rosie |
Project Sponsor | VP Office | VP Prof Nigel Brown |
Other document contributors | Project Team Project Board Business and Technical User group |
|
Project Review
Were the project goals met? Yes - mostly (Note: this is a multi-year project and not all the benefits will be realised until after next year's extended implementation project) Original objectives from the project brief for STU215 - Minimum Change - School timetables will be prepared and managed in the new shared system (including lectures, tutorials, labs, etc.) - Generic course-level timetables can be produced based on student numbers - Timetable clash highlighting will be used to identify conflicts - Generic course timetables will be available via the web - Standard reporting tools will be available (Excel reporting from Scientia Enterprise client delivered as part of minimum change, full reporting solution will be delivered as part of the STU226 Extended Implementation) - Room maintenance activities will be able to be flagged in the new shared system - The system will supply data to digital signage units e.g. OneLan/iGel - There will be integration with SITS for student numbers and programme/course info (scope change - Programme of Study data will follow in STU226 - Extended implementation) - It will be integrated with IDM accessing teaching staff and user details - Room information and old room bookings, drawn from EBIS, will be integrated into the system - There will be integration with EEMec/EEVeC to provide an information feed from the MVM timetabling system (scope change - moved to STU226 - Extended implementation project) - The new system will go-live in March 2012 (Client applications launched on 26-March, Web applications on 17-April 2012) (notes added in Italics , where brief changed or where some work has been re-planned to be delivered by the next extended Timetabling project - STU226 - Extended Implementation) Were the project deliverables fully or partially accomplished? Mostly met - partial with respect to Reporting , not met wrt Programme of Study data and integration with EEMec/EEVeC, which were pushed into Year 2 Extended Implementation Comparison against the Business Requirements identified during Procurement in 2011 (these for the multi-year project not just STU215) 63 out of 176 requirements completely implemented (by STU215 project) 28 out of 176 requirements partly implemented 85 out of 176 requirements not yet implemented (21 of the original bus requirements that were not implemented or only partly implemented are now marked as 'out-of-scope' and won't be delivered. Our understanding of what is now needed has moved on.) If partial, what are the reasons for this? A better Reporting solution was in development by Scientia , so it made sense to wait and implement the new Reporting solution (once it was released in July 2012, rather than deliver a depreciated product) . There was a late clarification to enforce authentication via Shibboleth (in August 2011, in line with a clarification to UoE policy by ITI), this required additional work by both UoE and Scientia. Did the project deliver a solution to the problems being addressed? Yes - the leagacy Room booking system was replaced with a new Web Room Booking system now integrated with the new Timetabling system Have the desired operational results been achieved? Yes - WRB is easy to use and available to all staff via EASE Some additional work for Course secretaries / room bookers who now need to interact with a more complex system to make their room bookings. Were additional results achieved besides the original success criteria? Easier for School Timetablers and Room Gatekeepers to make changes to get better fit of events to rooms Additional items which were delivered (and which were not identified as part of the original scope) to assist the business were: the post login plugin to pull data from Central Auth, the use of Shibboleth, the pre-loading of activity templates and the EBIS room booking migration. All of these items provided "added value" to users in that they avoided the need for the business to have to handle them manually (bar Shibboleth which was a policy issue to bring the integration in-line with the recently updated UoE IS Policy). Does the Project Sponsor agree that this project can be closed at this time? Yes - Year 1 of the multi-year project
Cost Summary | ||
---|---|---|
Staff Resources Estimated on Project Brief (Days) | Provide a breakdown by resource area/team based on the estimate provided with the project brief 600 IS Apps days in 11/12 split by IS Apps core project team : Jamie Thin PM 0.5 FTE Sarah Gormley Project Officer 0.5 FTE Dawn Nicholls Business Analyst 0.5 FTE Craig Middlemass Business Analyst 0.5 FTE Mark Ritchie Senior Supplier 0.2 FTE Project Services 250 days Dev Tech 100 days Dev Team 100 days Config Team 40 days Apps Mgmnt 30 days Tech Mgmnt 20 days Service Mgmnt (Multimedia / MyEd) 10 days un-allocated contingency 50 days | |
Other Resources Estimated on Brief (Money) | Did not have to dip into the contingency and able to carry over £290k from 2011-12 to 2012-13 Budget for Scientia Consultancy in 11/12 was 44 days , 36 days used to date. | |
Actual Staff Resources Used (% Variance) | Provide a breakdown by resource area/team to enable comparison with the original estimate 602 days delivered in 11/12 , and 40 days delivered in 12/13 | |
Proj Services | 320 days (+28%) | |
Dev Tech | 155 days (+55%) | |
Dev Team | 55 days (-45%) | |
Config Team | 0 days (n/a) | |
Directors Office | 30 days (-26%) | |
Apps Mgmnt | 18 days (-40%) | |
Tech Mgmnt | 20 days (0%) | |
Multimedia / MyEd | 7 days (-30%) | |
Total across all IS Teams (excl. ITI for build of the Virtual servers) = 642 days | ||
Actual Other Resources Used (%Variance) | Actual days spend on Scientia Consultancy in 11/12 was 36 days (8 days under budget) Payments made to Scientia Ltd matched the payment schedule agreed at the start of the contract. | |
Planned Delivery Date (From Project Brief) | March 2012 | |
Actual Delivery Date | Phased deliveries: 26-March 2012 for back-end database and client applications for School Timetablers and Room Gatekeepers 17-April for Web Room Booking (staff) and Web Timetables 23-May for Book Study Space (students) 28-May for Outgoing Interfaces (onelan and iGel) 6-June for Outgoing Interfaces (PADS for Business School) 19-July for Incoming interfaces (course and staff updates) |
Project Manager's Commentary on Reasons For Variance From Plans |
---|
For most projects the Project Brief should provide a reasonably accurate and detailed estimate of the staff and non staff costs for the project. Project Management research suggests that the actual resources used should typically be within 20% of this figure. --- to 23-Nov-2012 : 642 days IS Apps days spent to date , this was against proposal budget of 716 days, and TOR budget of 600 days Project has been delivered on budget , 11% under proposal, 7% over on TOR estimate (IS Apps days) Project has been delivered within hard milestones of original project brief ( in practice the project has been delivered in multiple phases with re-scheduling of some phases for Web applications and incoming interfaces , but impact on the business of these changes have been minimal) Reasons for Variance: The infrastructure required to run the Scientia components was more complex to deliver than initially estimated. 155 days Dev Tech resource was required compared to estimate of 100 days. Less Dev and Config team resource was spent than expected , as the MVM integration work was pushed into Year 2 and most of the config work was done by Scientia consultants (55 days Dev team resource compared to 100 days at estimate on Brief) Training was more onerous for the Timetabling Unit to deliver than expected - this was due to the large number of potential Enterprise client users (300+) and the complexity of the client application (2 FTE added to the Timetabling Unit on 18-month fixed-term contracts). Post go-live support was also more onerous than expected due to the large number of client users (300+ distributed across all schools and support groups) |
Key Learning Points | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
1. What went well? Feedback from the Project Board: "Being realistic about objectives and timescales" "Allocating some resources to Schools to acknowledge the demands on their workloads during transistion" Feedback from the Project Manager: Little churn in core project team (dedicated resources booked for duration of the project) Good working together between IS Apps and the Timetabling Unit - with very active involvement from the Academic champion to steer the project team and to champion requirements of School users Feedback from the User group: "Good quality info has been available for schools with good support and training""Early detailed discussions with stakeholders has been vital. Planning and communication of plans / delivery/issues has also been very important" "The team have done outstandingly well in the time. 95% of [centrally bookable] rooms requested have been confirmed back to gatekeepers, whereas it was only about 8% last year at the same point in time. Amazing!" " I have found attending the User Group very useful and I would be happy to be involved in the next stage of the project. The meetings were very well organised, clear agenda and follow-up." Feedback from the Project Board: "Major success is effectiveness of Project Board, despite its size (necessary because of complexity of culture change involved and range of stakeholders): it has engaged stakeholders across the University. This is a tribute to its chair, the Academic Champion and the support provided by Information Services and the Timetabling Unit."
2. What didn't go so well? Sudden and unexpected death of the key contact at Scientia (Dinesh Vaswani), though Scientia responded quickly by allocating an equally experienced consultant to lead the Scientia activities. Issues with Scientia Enterprise Client application running on Windows XP Work on Incoming Interfaces was pushed back to allow higher priority work to be done first. (Outgoing interfaces) IS Apps Dev Services resource managers and staff were very unhappy with the lack of advance bookings for STU215. (the project implementation was fast-tracked through IS Apps and this was in conflict with certain existing processes). Rio Watt (Academic Registry) noted that the Project Services costs were high. An increase in 28% over the estimated figure at the time the project was initiated. The Timetabling Unit costs also increased by more than 25% with addition of 2 x 18-month fixed term posts. The initial estimates under-estimated the complexity of the delivery , and the large number of school timetablers and room gatekeepers (over 200 Enterprise client users). Compared to other comparitive Scientia HE customers - the UoE has a much more devolved culture with in some cases up to 30 school timetablers and room gatekeepers across a single school - this creates a large demand for training and support. Closure was delayed by resource conflict within the IS Production Management team to complete last steps in the build of the Disaster Recovery servers at Appleton Tower. Feedback from the User group: "System more complicated to use than we had been led to believe - some staff had real issues with this. However support & help was excellent" "The end users need the right desktop platform ie. Windows 7" "The full workload for the initial entry of data was a bit of a shock for users. However, this will not be the case next year." Feedback from the Project Board: "A lesson that should be picked up is that of the constant concern over IS resources. It is difficult to judge whether there is a general problem of over-commitment or whether IS is crying wolf – they seem to deliver when the resource is actually needed."3. If you had a project like this again, what would you improve and how? Plan to deliver in multiple phases to smooth project team and school / business area workload. Project was initially planned to be delivered in 2 phases : 1) Client and Web applications and Incoming Interfaces by end of March 2012 2) Outgoing interfaces on 28-May 2012 in practice as the tasks and dependencies were better known, the delivery plan was re-scheduled and split into 4 delivery phases: 1) Client Applications (26-March) 2) Web Applications (17-April and 23-May) 3) Outgoing Interfaces (28-May and 6-June) 4) Incoming Interfaces (19-July) | ||||||||||
Outstanding Issues | ||||||||||
Open issues in STu215 as recorded by the project team in the JIRA issue tracking system. | ||||||||||
Key | ||||||||||
2 support issues (housekeeping) will be passed to support team | ||||||||||
STU215-37 Non Key Properties Config | ||||||||||
STU215-107 Acceptable use of Org Exceptions Table | ||||||||||
Closure Review Contributors |
---|
Project Team Project Board Business and Technical User group |
Analysis of Resource Usage:
Staff Usage Estimate: 600 days
Staff Usage Actual: 642 days
Staff Usage Variance: -7%
Other Resource Estimate: 653000 days
Other Resource Actual: 524450 days
Other Resource Variance: 20%
Explanation for variance:
Key Learning Points:
Outstanding issues: