Closure Report

Project Summary

REACH is a research project within the Asthma UK Centre for Applied Research (AUKCAR). 

The objective was to launch a new UK wide volunteer registrar for Asthma patients who could sign-up and volunteer to take part in future research studies.

This project had a hard deadline to launch the new service by Oct 2017 , ahead of a funding review with the Funder.

The project went Live on Wed 4-Oct 2017.

Overall the ISG teams in IS LTW WGI and IS Apps  worked together well with the business partner in MVM / Usher / AUKCAR to deliver this project in a tight timescale. Big success.

The web front end was developed in Drupal 8, as IS LTW WGI team inherited a half-built site for AUKCAR that was started by a 3rd party commercial in Drupal 8.

The Python / Django development tools enabled fast development of the backend API integration and data management application

The puppet automation tools now in place for the Python infrastructure allowed the new instance of the application / database infrastructure to be quickly built and deployed to the shared infrastructure in DEV , TEST and LIVE

--

The REACH system is comprised of two new parts, embedded within the existing AUKCAR website:

· Front end  is part of the AUKCAR website. The Reach sub-site is  public facing.

· Back end is a stand-alone website and API. It is restricted to authenticated users only.

The front end was implemented in Drupal 8 as part of the existing AUKCAR website. This work was done by the WGI team within IS LTW directorate

The back end was implemented in Python 3.4 with Django 1.11, with  a MySQL database. This work, along with the integration, was done by Development Services within IS Applications directorate

The key features of the back end are:

1. An API component which receives volunteer registration data from the public-facing front end web form via a custom UoE Reach drupal oAuth module.

2. A secure management interface for the volunteer data, allowing it to be maintained by REACH staff / sys admins.

3. Data export features, which enables REACH staff to create files of volunteer data, to be provided to researchers.

Analysis of Resource Usage:

Staff Usage Estimate (IS LTW) : 16 days (WGI led by Arthur Wilson)

Staff Usage Actual (IS LTW) :  21 days

Staff Usage Variance(IS LTW):  +5 days ( +31%)

Staff Usage Estimate (IS Applications) : 90 days ( IS S/W dev team, IS Dev Tech, Project services - led by IS senior supplier Bill Lee)

Staff Usage Actual (IS Applications) : 87 days

Staff Usage Variance (IS Applications): -3 days (-3%)

Other Resource Estimate: 0

Other Resource Actual: 0

Other Resource Variance: n/a

Outcome

New REACH service was successfully launched on Wed 4-Oct 2017

Release 2b with Data export functionality was deployed to LIVE on 18-Dec 2017

Explanation for variance

Change of Scope 

Two objectives were de-scoped : O2 - Create application process for researchers who want access to the data and O5 - Implement support for project workflow

This reduced demand on IS Applications S/W Dev  team

The Reach business team will review their future requirements in 6 months time as the new service matures and AUKCAR business lead will consider whether  any of these features are still required .

If they are required , then AUKCAR  may fund a new follow-up  project.

Key Learning Points

ISG need advance warning of future project work to build capacity and to ensure teams area available to pick up new project work.

Most ISG resources are booked 3 months in advance.

There is an emerging pattern of increased demand from CMVM for  IT project work to support CMVM research projects.

It may be prudent for ISG to build capacity in this area so that ISG can meet future demand from CMVM.

Closure report has been shared with Head of CMVM IT - so that CMVM forward planning can include an estimate of future demand from research institutes across CMVM.

--

When AUKCAR first got in touch with ISG LTW WGI team - they were informed  that immediate resources weren't available to start a new project in the next 3 months.

As AUKCAR had an urgent hard milestone , they then went out to an external commercial supplier and agreed a contract of work with them.

In fact , the external supplier didn't have any better availability than ISG, and could not meet all the business and technical requirements of the project.

The External supplier started to build the Drupal 8 web-front-end , but could not meet all the AUKCAR requirements for a flexible registration web form nor the research project  data security requirements for volunteer medical information.

At this point AUKCAR terminated their contract with the external supplier and came back to ISG.

The then project sponsor at AUKCAR Colin Simpson noted that in retrospect AUKCAR may have been better advised to wait for availability from ISG (who are better placed to meet the IT requirements of a research project) , than go to a commercial web / media agency who over-promised and weren't familiar with the research project context.

There is a need for a more flexible resourcing model within ISG so that ISG can respond more quickly to CMVM requests in the narrow time frames of research projects , which need to start up project work within the life of a research project grant, and meet research funder reporting requirements.

--

On second request to ISG LTW WGI team - IS Applications got involved as the LTW WGI team could not deliver the secure back-end integration required for REACH.

Bill Lee got involved as IS Applications  Senior supplier as well as Paul Clark ( then Head of CMVM IT) .

Bill Lee , John Alison and Arthur Wilson pulled together a proposal for AUKCAR offering that the work could be completed by a single project team across the two ISG teams, with project management from IS project Services.

This is a good model for future project delivery for CMVM research projects.

Project Management provided by IS Project services, so the customer has a single point of contact for project issues, and reporting via the Projects website so there is visibility of issues and project status.

--

AUKCAR urged ISG LTW WGI team to pick up the existing partially built web-front end , and build the final solution on the initial work by MTC.

This was in response to the urgent Research funder deadline and to re-use what had already been built.

In retrospect this approach carried a number of risks for the project and future support and maintenance - as there were a number of undocumented features in the MTC site including an unusual footer, which then caused problems later.

Some features of the MTC Drupal environment build also caused problems when a Drupal core security patch was deployed to DEV, TEST and LIVE.

--

Due Diligence

For future projects - there should be some guidelines for due diligence when ISG adopts a web site or code from a 3rd party. There is already a process for security code review.

Typically this happens when support for an application that has been developed locally at a school or institute moves from the school to ISG. More rarely it involves an application developed by an external commercial provider.

Any 3rd party , be it a school, research institute or commercial supplier will apply different standards and build guidelines, and these are unlikely to match the standards used within ISG.

This due diligence template should be extended to cover web sites.

The ISG LTW WGI team took on the existing Drupal 8 site built by the external 3rd party (MTC)  on a 'best endeavour' basis.

In fact , as there were several undocumented features in the existing Drupal 8 REACH website , it would have been cleaner to build a brand new site in Drupal 8 for REACH.

Unplanned complications included different configuration and behaviour of the DEV, TEST and LIVE environments. 

Though it would have required extra work in the first release of work, it would have saved time and effort later when resolving problems, and would have simplified future support and maintenance.

This risk has now passed to the AUKCAR web editors as they maintain the front end website.

--

This observation from AUKCAR business lead ( Susan Buckingham) confirms this lesson.

Susan advises that for future projects when co-ordinating effort across  different ISG teams , there is a need to carefully manage the DEV , TEST and LIVE environments so that  end-to-end behaviour in DEV and TEST closely matches LIVE , otherwise testing in DEV and TEST is undermined. 

The source of many of these  issues was undocumented configuration passed to ISG with no formal handover from the previous external supplier (MTC).

--

There is a future action on the IS LTW WGI team to review the release process for web-front end changes from DEV to TEST to LIVE.

IS Applications always recommends a repeatable release process , so that a standard process is used to deploy to TEST, and repeated when deployed to LIVE.

Outstanding  Project Issues

None

Handover to Business as usual teams

Future Drupal 8 core patching:

1. the uoe_reach module built by IS apps has dependencies on php libraries (oauth)
2. the installation of this module, webform and the core drupal 8 modules need to be managed using composer
3. As future Drupal 8 core updates are released , there may may be a requirement for some on-going support from IS Apps directorate  to resolve any issues  with Drupal core updates and to keep the site secure ( this effort will be chargeable as part of the support effort covered by the OLA) 

The Drupal Core update process has been documented by IS LTW 

AUKCAR specific notes on Drupal 8 core and module patching/installing are available at:

https://www.wiki.ed.ac.uk/display/awdrupaldrops/Installing+and+patching+modules+in+Drupal+8

Recommendations for IS LTW / IS Applications future software development collaboration

1. When IS LTW and IS Applications work together , the project team should prioritise agreeing common good practice around version control and release management early in the project.

2. A review / meeting should be scheduled with IS LTW, IS Apps Dev services, IS Apps Production management and IS Apps Project services to review advice / agree improvements when IS LTW and IS Apps work together on website development projects. This will be done outside of the MVM315 project as part of ISG continuous improvement work. ( Action owner - Jamie Thin, IS Apps project services) 

3. IS Apps S/W Dev team are recommended to create a due diligence template to be used for code reviews (when adopting code from a 3rd party ) and add in some of the lessons learned from this MVM315 project and share this updated template with IS LTW ( ( Action owner - Bill Lee, IS Apps development services)