Closure Review

Project Summary:

Project Review

The project started in December 2015 and was scoped, estimated and resourced to completion in August 2016, with the development phases split into Work Package 1 (WP1) and Work Package 2 (WP2). Development was due to start at the beginning of March 2016, but was delayed as the senior developer was required on the higher priority TTU011 Timetabling project. A contractor was brought in to complete the development, under the supervision of the senior developer.

Throughout the project, there have been demands and conflicts on the senior developer’s time. It was anticipated that STU251 would be completed by August 2016, and the contractor was planned to do other work at that point. However during development and testing there were numerous issues to resolve and this resulted in missing the August 2016 completion date, and with the contractor now working on another project, the original senior developer returned to STU251. WP1 and WP2 were reviewed and categorised into PG and UG JIRAs. UG was successfully deployed to LIVE in October 2016. In November 2016 the project was made a priority one to expedite the delivery of the PG work.

One of the development JIRAs required an extension to a text field length. Once development has started, it was realised that a database migration was required to resolve this. As there were five databases on that server, 4 of which belonged to CAM, it was agreed to extend the scope to migrate them all. The UG and PG databases were higher priority and successfully migrated to LIVE in November 2016, with the remaining migrated in March 2017.

The PG updates identified issues to be resolved during peer testing in December 2016 and January 2017, however the senior developer left the University in December and handed over to a third developer who started on STU251 in February 2017. The issues were resolved and deployment was arranged for March. In March a serious issue with the automated deployment tool Bamboo resulted in all deployments being cancelled for a number of weeks until this was resolved. This delayed the deployment of the PG updates until April.

There was scope change added to this project for XCRi-CAP (eXchanging Course Related Information, Course Advertising Profile, a UK standard for describing course marketing information), as part of JIRA 8: https://www.jira.is.ed.ac.uk/browse/STU251-8, and was subsequently de-scoped. CAM wanted to progress with interim solution via STU251 while Student Systems wanted to wait and put in better long-term solution as part of Service excellence. It was agreed to de-scope this from STU251 and include it in the next CAM project, starting in May 2017.

Objectives

The objectives were met with the exception of O.3. The original 88 day allocation was significantly exceeded. This was due to the length of time taken to complete development of the original JIRAs, and the post peer testing bug fixes that were required. Also adding to the time was the migration of the 4 underlying CAM databases from MySQL 5.1 to MySQL5.7, and the migration of the SRAREC database.

Deliverables

The original brief was to deliver the following:

Work Package 1: Priority JIRA’s, system outputs and exports.

Work Package 2: Admin system changes and improvements.

Scope & Additional Deliverables

The scope was initially set as in the section above, and amended in:

• PICCL 4 to include the upgrade of Campus Maps, A-Z, UG, PG databases. This was done in order to provide the fix for JIRA 13.
• PICCL 4 also includes the SRAREC database upgrade from version 5.2 to version 5.7.12
• PICCL 26 to include the change  for XCRI-CAP requirements. However this needed to involve Service Excellence and was subsequently de-scoped for this project.

Benefits

The project benefits were originally expected to be:

• Improvements to the CAM applications for both Administration and Student facing pages resulting in increased usability and functionality.
• Alternative balance of work between IS Applications Management and IS Software Development for calls which are enhancement requests rather than fixes to LIVE applications.
• Continuing the new method of working, whereby CAM define the requirement, test cases and success criteria in JIRA
• Implementation of Bamboo for deployments to improve the ability of support to administer and deploy changes outside of project work.

The above benefits were all realised, and in addition, the project delivered:

• A more stable platform for on-going CAM use through an upgrade of the SQL databases - this also means that a separate project to upgrade the 4 CAM databases is not needed
• A more stable platform for the SRA Tours Finders application through an upgrade of the SRAREC database - this also means that a separate project to upgrade the SRA databases is not needed.

Schedule

Analysis of Resource Usage:

Staff Usage Estimate: 88 days

Staff Usage Actual: 264 days

Staff Usage Variance: 200%

Explanation for variance:

A higher priority project (Timetabling) conflicted for the developer's time at the start of the project and this removed the developer resource early on. This resulted in the appointment of a contractor to complete the development. This appointment delayed the timescales by a month and a half, and increased the estimation due to handover, familiarisation with the application and University procedures and processes needing to be carried out and observed. The start of the development was scheduled for the middle of March, but did not begin until May.

At that point there was then effectively two developers working on the project as the original developer continued to provide support and fixes on a reduced scale to the contractor when needed.

A bug raised on LIVE early on was related to one of the development JIRAs that had been estimated, so it was agreed to be included in scope in this project. This turned out to be significantly more complex than originally thought and the resultant fix was a database migration of 5 databases to the MySQL enterprise architecture. A number of issues encountered in setting up the new MySQL 5.7 environments meant that additional effort was needed for resolution to enable a stable infrastructure.

Timescales were impacted with developer training, workshops, and writing initiative proposals between July and October 2016.

More development work was required to fix issues highlighted during testing (both peer and UAT) than estimated.

Throughout the project, there were JIRAs marked as complete and submitted for software development peer testing which were not in fact resolved, and required a senior developer to undertake re-work on the code in conjunction with the original developer.

The XML development JIRA required a different approach than what was adopted originally, and it came to light that it would also need involvement from Student Systems. As there was a lack of agreement about the way to proceed this delayed resolution.

Lack of testing / understanding of the application from the contractor resulted in senior developer taking time to explain or fix the problems with the code.

Throughout the project development effort exceeded the original estimations due to having to revisit and perform bug fixes or re-work after testing.

Key Learning Points:

• Regarding temporary staff: Documentation, procedures and quality assurance were lacking in this project.
  • Learning Point: Software Development (or appropriate team leader) to ensure regular reviews of code and documentation to quality assure that procedures are being followed. If remote working is taking place, then we should review the arrangements if this isn't working for the project.
• There were occasions where there was a large period of time where developers were unavailable due to training, workshops and writing proposals for innovation projects.
  • Learning point: Project team members to keep the project manager informed of anything that could or will take someone off the project for a length of time.
• Throughout the project, the business, programme and resource managers need to be kept up to date on progress and concerns, and this was not consistent.
  • Learning Point: Project Manager to escalate any constant or recurring issues that are impacting milestones  sooner to the programme or resource managers.
• Three point estimates did not reflect the work required for some JIRAs, a number were under-estimated. After starting development, or during testing,  it was found that some required a more complex solution, or the estimate did not take into account that there were other parts of the PG / UG applications that were affected by the update required. 
  • Learning Point: The worst case should be thought about in relation to particular tasks, or a group of related tasks, to consider "what happens if..." by the developer estimating the work - in some cases this may result in an additional task with the best case of 0, and other estimates accordingly.

Outstanding issues:

• It has been agreed to de-scope the jira: Create an xml feed of our live output JIRA 8 - https://www.jira.is.ed.ac.uk/browse/STU251-8 This also encompasses the XCRI-CAP work. It has been agreed that this will be undertaken in the next CAM project, starting in May 2017.