Closure Report

Project Summary

Background

The Website search function used across much of the University had not been actively managed for a significant period of time. In early 2012 it was identified that the free search service used at that time required to be changed in order to comply with the latest EU regulations. 

The process of changing search engine uncovered a number of issues that could not be resolved with the current solution. During year 2014/15 some further work was undertaken to better understand search engine requirements, progress with developing proof of concept work and deploy enhancements to the existing service. This work fed into the project requirements for the procurement of a new Search engine (UWS005).

Concurrently, delivery of EdWeb - the new central content management system (CMS) - and the Edinburgh Global Experience Language (EdGEL) - a design framework for branding and design guidelines for developing web applications within the University - provide opportunities to provide a better user experience, with the search engine being able to take advantage of and implement a better and more usable service.

The search engine procurement project (UWP014) was undertaken to gather requirements and procure a new University-wide search engine. The project resulted in the conclusion of a signed SaaS agreement with the approved supplier ready for implementation.

This project has was initiated to implement the new search engine with the following scope:

Implementation of the procured applications, documentation and services (as outlined in SaaS agreement) in accordance with the Business Requirements Specification.  

Analysis of Resource Usage:

• Staff Usage Estimate: Original estimate 130 days increasing to 185 days
• Staff Usage Actual: 204 days
• Staff Usage Variance: 74 days (57%)

Outcome

Priority – M = Must Have; S = Should Have; C = Could Have; W = Won't Have

• M = has to be satisfied for the final solution to be acceptable in terms of delivery dates, compliance, viability etc.
• S = high-priority requirement that should be included if possible -workarounds may be available
• C = a nice-to-have requirement
• W = will not be part of this project

O = Objective   D = Deliverable    AD = Additional Deliverable as a result of Project Sponsor funded scope increase 

Explanation for variance

The project has been impacted by a number of issues which have caused timescales to shift considerably.  These have included:

• Early on in the project the requirement for additional Development and Development Technology resource was identified following the decision to build the system using XML/JSON and for the front end to be developed with Python / Django due to the benefits this would bring (usable by any programming language which supports HTTP requests and XML DOM/JSON functionality, ability to reuse this interface with ease and flexibility, the API interfaces provide higher throughput capability).  This required an estimated 28 days for development time alone (contingency was used and budget was increased by 15 days).
• Training was scheduled later than expected by supplier which impacted planned dependant milestones.  This delayed the project milestones by around 3-4 weeks.
• The project required to push its  original Go Live back to adhere to a Change Freeze. This impacted milestones at the time by 2 weeks.
• Significant (and unplanned) discussions with key stakeholders were undertaken when it was established the previous search application which offered the feature of looking up telephones, and staff and student email addresses (and was due to be replicated for MVP) had a number of security and robustness issues. The telephone numbers and staff and student email addresses data resided within the  search01 server as a bundle of perl scripts, C binary, TSV files that had not been updated for a long period of time and unused LDAP lookups. This practice was not to be carried forward on the new application. This posed considerable challenges to the project and the solution involved separating features, and creating different API applications that exposed phone and student email address data.  See journal - https://www.projects.ed.ac.uk/unpublished/project/uwp016/journal/1 . The quality of the source data had a significant impact to the projects ability to reach milestones and progress. This originally pushed milestones out by 2 months and budget by 47 days however due to further complications it resulted in a further 7 month delay.
• The solution outlined above (building the telephone and email directory APIs) only resolved half of the problem.  Even though a means to access telephone and student email data had implemented, our supplier also needed to set the search data collections accordingly.  The supplier had not been clear or specific about the best integration method(s) between the telephone and email directory APIs and their system.  This meant adding time on the integration, implementation and configuration of the APIs in order to facilitate the integration, with subsequent UAT, deployments etc. (e.g. UWP016-179).  This had an impact on the originally planned “one-off” MVP release with potentially additional post-go-live fixes.  It resulted in an almost “phased MVP” release – “must-have” features going in the released MVP gradually and a temporary solution put in place to accommodate for the intermediate stage.
•  The solution, to separate the two features and create two API applications which exposed phone and student email data, was delayed whilst a planned deployment of changes to the Cookies Banner in EdGel was finalised.  This was due to issues identified within the API UAT.  This extended milestones by a further month. These included:
  • https://www.jira.is.ed.ac.uk/browse/UWP016-134
  • https://www.jira.is.ed.ac.uk/browse/UWP016-170
  • https://www.jira.is.ed.ac.uk/browse/UWP016-156
• Problems were identified when undertaking the overnight crawl (https://www.jira.is.ed.ac.uk/browse/UWP016-164) post deployment and this delayed closure by an estimated 2 months.
• There was challenges in relation to resource allocation from external suppliers throughout the project.  There were a number of changes to  the core project / development team.
• Availability of the UoE key project team members (due to changes to milestones, timeline and conflicts with other projects) meant breaks in bookings were frequent and milestones were pushed out due to this.

Key Learning Points

• The Project Team all worked very well / efficiently, especially in its use of the tools available to it, with use of HipChat, particularly during deployment being particularly effective. 
• The challenges outlined above in reasons for variance all had an impact on the project and resulted in significant changes to budget and timeline.  It also ultimately resulted in many of the "would like" deliverables having to be moved into the future development roadmap rather than being deployed within this project.  As there is consistency within the LTW team who were on the project team and who will be progressing the future developments some of the potential risks and issues will be avoidable going forward given their knowledge and understanding of the work undertaken to date.    
• The data source issue was not identified until development had just about started, this resulted in the significant scope creep.  If this could have been identified earlier it would have reduced the amount of additional development required and negated the significant re planning which had to be undertaken.
• It was established that having better understanding and deeper insight for the complexity of our public, and protected, web estate and its impact when introducing high-profile services.  The end user expectation that the Funnelback's service would perform at an equal or even more level than the previous Google-powered search was not possible immediately. This hasn't happened right away due to the complexity and breadth of the web state being underestimated by both Funnelback and ISG. To avoid this the project would have needed a lot more time before transitioning from the older service to ensure a higher service quality, while working with Funnelback in a more effective configuration.   This was limited as there was a need to replacement the Google search  system before the end of the Google contract.
• Some of the project team felt there was a lack of clarity on our high-profile golden source systems and what the most efficient way is to integrate with them to get access data. Others felt it was clear from these systems what their preferred integration means were, but rather there was not clarity on how these could be integrated with Funnelback i.e. although the supplier provide a list of different methods they lacked key features that allowed for a secure and smooth integration.
• EdSearch “middleware” tier development.  This will be progressed separately out with the scope of this project.

Attachment	Size
uwp016closure.png	11.59 KB