Closure Report

Project Summary

This project achieved its all of its objectives and deliverables:

• Infrastructure upgrade to new Virtual Machine (VM) for each of the eight institutions
• Upgrade of UniDesk to v7
• Development and deployment of QuickCalls

The institutions are all now using the upgraded version of UniDesk with its new features and our in-house development of QuickCalls. The feedback the project team has received from all institutions has been really positive on how the upgrade went, the latest version of UniDesk and the new QuickCalls application. The comparison graph below illustrates the increase in calls logged between 2017/2018.

The big increase in UoE is due to a number of new teams using QuickCalls since the relaunch:

• CRC (special collections)

• Vet Library

• Kings Buildings

• Student Systems

The annual upgrade was a major release moving to V7. In addition it was within the scope of this project to upgrade all UniDesk environments to new VMs. QuickCalls required to be developed in house as the underlying UniDesk architecture had been redesigned for V7, which removed the QuickCalls application. This was added to the scope of the project.

The upgrade commenced in August 2018 with planning, estimation and resourcing. QuickCalls was added into the scope resulting in lengthening discussions and estimations. An extra budget of 94 days was estimated for the QuickCalls development. This was incorporated within the estimate submitted at the planning stage.

Work on setting up the University of Edinburgh DEV environment and the QuickCalls specification write-up and development started in October 2017. The Ulster University UniDesk instance for the live environment had remained on an older version (V5.4) due to problems during the last upgrade with the Shibboleth integration. The plan was to resolve the authentication and bring them up to V5.7, then upgrade to V7 which effectively meant two upgrades for Ulster.  The infrastructure was ordered for all institutions, and QuickCalls development progressed.

Milestone changes were required in November, along with a change to a higher priority project. UoE LIVE deployment was now scheduled for 31/01/2018. The changes were due to:

• Scope creep and crucial decisions had to be made on the path to deployments partly led to delays in DEV sign off.
• The Dev Tech resource leaving the University and continuity of resource bookings not possible due to live Tristan issues that took priority.
• Shibboleth integration with the Django infrastructure for Quick Calls was an unknown and has some issues to resolve.
• Ulster were not able to use Shibboleth. Azure authentication had to be put in place, and this needed to be researched, estimated and actioned. This work was paid for by Ulster.
• UniDesk Process Manager advised the start of Semester 2 needsed to be protected due to the new PGT intakes. Deployments commenced from the end of January with Edinburgh going first.

It was also noted that in the previous year, deployments were carried out out of hours, so this was communicated to the members.

Training had been booked for 11/12/2018 with all the institutions as it was anticipated that the TEST environments would be ready by then. However due to the issues above, it was not possible to deliver the full TEST environments in time for the 11th, so a bare-bones UniDesk only upgrade minus the QuickCalls application was delivered. Durham requested their TEST environment be delayed due to their operational reasons.

Further scope creep occurred during January, it became apparent that there was an issue with the Problem Management module. Functionality had been removed from the V7 upgrade by TOPdesk, and not included in the release notes. Subsequently this missing functionality was not noticed when reviewing the release notes at the start, and the issue was only picked up on during UAT, and referred back to TOPdesk to provide a fix.

There was also an issue with the microservice API for QuickCalls in that it would not work with multiple institutions. Again, this was only noticed when migrated to TEST as the DEV environment is only used by one institution (UoE). A re-architecture of the API was required and carried out, this was done under the Digital Transformation API program.

The deployment date for UoE was now scheduled for 20/02/2018. This would leave one week less for bug fixes before roll out to other institutions, and also impacted the original deployment for Stirling. In order to avoid moving the deployment date for every institution after Stirling, Stirling was now rescheduled to the end of May. As we were waiting on a fix from TOPdesk, to make use of this spare deployment slot, the the schedule changed to allow the DEMO server to be built and deployed to LIVE. TOPdesk delivered the fix to problem management and UoE was deployed on 20/02/2018 along with QuickCalls. This was a successful deployment with no significant issues noted. The remaining TEST environments were handed over to members for their UAT, and TestRail was used to manage the testing. This was very well received by all members who found it intuitive to use and allowed the project team to see the test results and when they were completed.

In March, the deployment schedule was disrupted due to the pension’s dispute, which resulted in strike action being taken by UoE and a number of our member institutions. A change freeze was put in place and deployments were suspended for this period of the strike action starting 26/02/2018 to the 21/03/2018 due to the risk of not knowing which resources would be participating. We had built flexibility into our plan at the start, and this benefitted the project at this stage where we would only need to move Abertay from 28/02/2018 to 08/05/2018 and St Andrews by one day from 21/03/2018 to 22/03/2018. When the dispute came to an end there was "Action-Short-of-a-Strike" in place meaning no out of hours deployments were to be made. 

Planning has been difficult due to the continual changes with the rollout schedule, and issues raised with QuickCalls making it difficult to move bookings to when they are required without having to conflict with other projects. This led to delays in reassignments. The scheduling of upgrades for multiple environments was very complex and had to take into account scheduling here at Edinburgh as well as the service members. There were instances where a plan was distributed for signoff, issues were identified requiring rescheduling of dates and another plan had to be re-issued before the first was signed off.

With all the deployment completed, we received some very positive feedback from all eight institutions on the upgrade. They all felt it went smoothly and the transition to the new version of TopDesk was very well handled by the team. We have received the following favourable feedback:  

The table below shows which institutions were upgraded out of hours and in hours and the date completed.

After the deployments resumed, and an issue was discovered on QuickCalls for Sheffield. There were originally two development resources for QuickCalls - a senior developer and new start. One has now been seconded to EDINA and the other is fully booked on RES069. A conflict with RES069 occurred to secure resource on 20/04/2018. The Resource Manager indicated that all developers are now nearing over-allocation.

After Stirling and Napier deployments, further bugs were found with QuickCalls which related to the individual institutions and were not spotted during UAT. As both the original developers were not available, a new developer was assigned the task of putting the final fixes in place for QuickCalls.

The additional bug fixes for QuickCalls, and annual leave resulted in a moved project closure date to 18/07/2018. Further bugs on QuickCalls and resource availability due to annual leave and live issues needing resolved resulted in a closure date of 31/08/2018 then 21/09/2019.

Quickcalls is now a stable application that is used by seven out of eight of the institutions to log their calls. Whilst there have been some initial problems to overcome as it was rolled out, these have been dealt with quickly and we are happy to report no issues have since been raised. As with any new software build there have been a number of enhancements suggested and these have been collated and will continue to be collated for a future project. These are listed at the end of this report.

Objectives and Deliverables

All objectives and deliverables have been achieved.

Scope

There was a scope change to add in the QuickCalls development. This was added in at the planning stage.

Analysis of Resource Usage:

Staff Usage Estimate: 222 days

Staff Usage Actual: 364 days

Staff Usage Variance: +64%

Explanation for variance

• Scope Creep: QuickCalls was removed from the v7 UniDesk, so there was a new in-house development of the software that was to be rolled out to each member. This took longer than the original estimate as there were refinements made along the way. QuickCalls required re-development at most deployments to tweak it for that member. This has resulted in an extra development support bookings required, and delays to the project due to developer availability. However we now have a product that is very well received by every member, and is proving to be a stable and reliable platform used by 7 our of our 8 institutions.
• Shibboleth authentication being put onto Django infrastructure required additional research and time to put together
• Additional time required for re-planning deployments during strike action
• Further testing and integration and documentation work required on the Problem Management module that TOPdesk were required to provide a fix for, and the project team had to integrate, test and deploy
• Project duration being lengthened increased the effort for all teams
• Azure authentication had to be set up for Ulster and took longer than expected. This 9 days was paid for by Ulster as their set up differed from the other institutions meaning they could not use Shibboleth. 

The table below shows the estimated vs actual effort on the project per team:

Key Learning Points

Issue: The QuickCalls specification was not complete as there were a number of changes and updates as development progressed.

Key Learning point: More time should have been allocated at the start to get the specification correct. We needed to talk to more business users and to the institutions. We assumed that the institutions would use QuickCalls in the same way, but this was not the case.

Issue: Communication between the teams. Although there were weekly meetings held throughout the project, it was felt that communication was poor relating to but not limited to notification of project plan updates, the TEST environment that was needed for training, and the deployment dates of the institutions.

Key Learning point: Assumptions made that documentation would be viewed by project team, however PM has taken note to inform project team on any new updates made to any project documentation including milestones once complete by email or Hipchat. Project team to inform the PM early on of any key activities that may be impacted by changing milestones such as the issue with the training.

Issue: There were two parts to this project to be considered in DSOR. The TopDesk Upgrade, and the QuickCalls development. With eight institutions to rollout these changes to, we need to consider how we expect the deployment sign off review (DSOR) to be achieved.  How would DSOR work for all institutions and QuickCalls moving forward? For instance, the first institution to be deployed may be using QuickCalls for a month, the latest one may raise a bug straight after deployment. Should QuickCalls DSOR be considered per institution two weeks after each deployment or at the end of all the deployments? The application is not deployed each time, the same code is used by all institutions with an API link to the application.

Key Learning point: Raise this issue and the project planning stage, discuss with the team whether we need separate signoffs after each deployment or two at the end, one for Topdesk and one for Quickcalls.

Issue: Due to the number of institutions involved (and growing) the deployments can run over a number of months. At the start of the deployments we only had one DevTech resource which was a big risk on the deployment dates that were set. Any unexpected leave could result in missed deployments. Any annual leave would be difficult if not impossible to take. A second DevTech resource became available mid-way through the deployments after shadowing a deployment and delivering the Ulster TEST environment.  

Key Learning point: Working with multiple institutions we want to avoid numerous deployment date changes. Establish at the start who can take over in the event of absence and if any annual leave is required during the deployment phase. Ascertain if they need to shadow  deployments to any of the environments.

Issue: There were delays in the project where fixes were being applied to QuickCalls, some of which were institution specific, and delays to deployments where last minute bugs were found and fixes needed to be applied. QuickCalls was deployed once at the start and each institution uses an API to connect, but it was assumed that each institution used QuickCalls in the same way. The bulk of the development support was allocated at the start of the deployments to iron out any issues with less allocated as they progressed. 

Key Learning Point:  For future updates, we need to ensure that sufficient uniform development resource is available throughout the deployments for QuickCalls, and not load it from the start with it tailing off as deployments progress.

Issue: The barcode lookup API was not formally included in this project and led to issues needing resolved.

Key Learning Point:  We should have been more careful to include the development and provisioning of the barcode lookup API more formally within the project. The unmanaged development of this lead to problems due to lack of quality control, lack of deployment planning, and poor/missing requirements specification.

Issue: Problem Management module had functionality removed from version 7, and this was not included in the release notes. As a result the issue was not spotted until UAT where TopDesk needed to provide a fix, thus delaying the deployment for UoE.

Key Learning Point: TopDesk's release notes were not accurate. Service Management to be aware for future upgrades. 

Outstanding Issues

There are two outstanding issue:

https://www.jira.is.ed.ac.uk/browse/SMI018-94

SQL Server compatibility level

There is an issue with the UoE person import database table when SQL Server is set to SQL 2014 compatibility level. On testing it only affects the person import database table. There was no developer availability at the time to be able to fix this in this project, therefore It has been agreed with production and service management that we will upgrade UoE and all the other institution databases in DEV and TEST to SQL 2014 compatibility level, leaving out this database table for UoE which is set to SQL 2008 compatibility. This part will be fixed in the next UniDesk project. All other institutions are not affected as they don't use linked database tables, and this is where the error occurs.

Service Management and Production have agreed that the deployment to LIVE will be picked up under support once the CAB change freeze ends in on 01/10/18. This change can be applied with no downtime required.

Production have confirmed that this issue will have limited impact on future upgrades if left as it is. It is a compliance change as TopDesk recommend SQL Server compatibility level of 2014, but keeping it at 2008 level does not have a detrimental affect on the application.

Deployment of JIRA 99 for a QuickCalls fix for Durham

https://www.jira.is.ed.ac.uk/browse/SMI018-99

This JIRA was deployed before the change freeze but a bug was raised by Durham on the day the change freeze took affect. A fix has been applied to TEST but deployment was denied by CAB until after the change freeze. This will be picked up under support.

Should an issue be discovered after this deployment this will be handled as BAU for the national service.

QuickCalls Enhancement JIRAs

There were JIRAs raised for QuickCalls that were classed as enhancement JIRAs after UAT.

These are beyond the scope of SMI018 will be taken into the next QuickCalls enhancement project for review. They are listed below: 

Attachment	Size
unidesk_-_slide_quickcalls_use.png	62.76 KB