Closure Report

Project Summary

Learn is the main centrally supported Virtual Learning Environment, and is used to underpin over 4500 courses, including campus based, and wholly online Masters level courses.

As Learn is a top priority service for the University we aim to have an uptime of 99.7% (combined planned and unplanned availability). A key part of ensuring that the platform reaches the availability target is making sure that we are on a stable and supported version. The supplier recommends keeping up to date with service packs to reduce performance issues.

This project was for the planned annual upgrade of Learn VLE platform.

The project is scheduled for delivery into production in July to fit with teaching patterns.

Objectives & Deliverables

The deliverables were prioritised using the MoSCoW prioritisation method

M=Must Have; S=Should Have; C=Could Have; W=Want

Additional deliverables outwith the scope of the project

• Nik Pardoe (DevTech lead) has done some amazing work in streamlining the Learn upgrade processes and automated the Learn processes as much as possible resulting in an improved infrastructure and a sleeker deployment. The following improvements have been made:

  • Nik has created a health-check that reports back on all 6 servers - this can also help Moodle and will make troubleshooting the servers in service much easier
  • Nik has rebuilt KB3T and AT3T servers on both TEST and LIVE meaning their configuration is now the same as the other servers
  • Nik moved KB3T and AT3T servers into same firewall group as other servers resolving web url issues
  • Nik updated the Java version on Live
  • Time to upgrade: the upgrade to CU2 on the primary server KB1 went very quickly (from an expected 1hr30mins down to 29mins)
• After testing the feed in TEST Learn it was discovered that Learn was “soft deleting” user’s accounts unnecessarily. The data needed to be cleaned up in order to allow UAT to take place without any issues. It was agreed by the project team to bring the clean-up task on the data feed into the project so that UAT on TEST would be easier to undertake
• Added LTW tasks to implementation plan to give a fuller picture of what was needed at implementation – not just development tasks

LTW Deliverables

Success Criteria

Benefits

Benefits are logged on the projects website at https://www.projects.ed.ac.uk/unpublished/project/tel053/benefits

Analysis of Resource Usage:

Staff Usage Estimate: 72days (61d most likely + 11d contingency)

Staff Usage Actual: 71days

Staff Variance: -1.3%

Outcome

Explanation for variance

Cost IS APPS

Time

Issues

1. A showstopper issue came to light after the end of UAT just after we had approved acceptance on 18 June. WebDAV has not previously been part of the Learn UAT plans, something which DLA&M will update. It was found as part of a check on the issues with CU2 in Behind the Blackboard. The WebDAV application was no longer working in CU2 and following advice from Blackboard, LTW chose their recommended safer option to upgrade to CU3 which would fix the issue. This meant undertaking another upgrade 4 weeks before deployment. We moved to CU3 on DEV on 20th of June but received errors throughout the UAT process. This was due to a memory issue which was fixed and we moved to TEST on 25 June.  Unfortunately more issues were found which required to be escalated to Blackboard.  Despite their help we were unable to resolve the issues and LTW took the decision to roll back to CU2 accepting the WebDAV issue which was dealt with through comms as moving the deployment date would impact colleagues in the College of Medicine & Veterinary Medicine who start term at the beginning of August. WebDav is used by certain users to allow them to access Learn’s content collection through a networked drive, this allowed them to easily add files through dragging and dropping, edit and delete them. The current issue with WebDav stops users from being able to add new files via WebDav.The workaround given to users is that they are now able to use Drag & Drop via the GUI in Learn, a feature that came as part of the upgrade to Q4 2017 CU2. Full details of this issue can be found in piccl 10 .
2. When kicking off the upgrade on TEST it was discovered that KB3T and AT3T servers were not connecting. Receiving error reports and unable to resolve Nik rebuilt the 2 servers. There followed an issue with the web url – it discovered that these servers were sitting outside the firewall that the other 4 servers were in so Nik moved them into the same firewall
3. And when it came to the LIVE environment the set up was the same as TEST so the LIVE servers were rebuilt and moved into the same firewall as the KB & AT 1 and 2 servers – all are now working as they should be
4. Because of the delay caused by CU3 issue we ran over the assigned time for the developer and had to conflict for his time against HSS024
5. There were a couple of issues on deployment to Live but they were quickly fixed:
   • MediaHopper Quizzes were not appearing - traced to wrong Player being used - corrected by the Media Team in Test & Live instances of Learn. JIRA closed.
   • The LTI connection for Leganto in LIVE Learn failed following the upgrade displaying an error message when trying to access the link. After trying unsuccessfully to change the custom variables in the LTI registration to force it to see the correct user ID LTW contacted Blackboard who confirmed the variables were correct and recommended contacting Ex Libris and was escalated by Library Learning Services.  Was traced to a mismatch on userID - the fix had to be implemented within ALMA to force Leganto to see and use the custom user name. This change was made by Library Learning Services and tested by DLA&M & the Library Learning Services. JIRA closed.

Key Learning Points

1. We need to consider the affect our deployments have on sections such as Helpdesk. The Learn deployment was due on the same day as the MyEd deployment. Sue had checked with Paul and MarkL and there was no issue from the development side with them deploying on the same day. However Go CAB questioned this as they were concerned about the load on helpdesk. Outcome was that they recommended that (because of the user base Learn affects) MyEd change its deployment date.
2. The idea of getting service owners of various plugins and LTI connections into Learn to sign off as we moved through the environments was really good. LTW would like to expand that.
3. Getting the Learn User Group to do a community UAT similar to Moodle.
4. DLA&M would like to schedule the course feed rollover and template work better, this all seemed to happen around the same time and contributed to confusion for users.
5. Test plans to be reviewed each year by LTW. This should take place in February to ensure the UAT plan is okay, ideally it would be good to have someone who knows Learn well but isn’t part of the Learn Service team.
6. DLA&M would like to be able to lock down Live once deployed (i.e.not fully bring up) so they can do their checks without the user base contacting them immediately the system is Live.

What went well

1. As mentioned in the Additional Deliverables section the improvements that Nik has made to Learn
2. This has been an excellent project to work on from a teamwork point of view. The working relationship between Apps Man, TechMan, Dev Services, LTW and DevTech has been excellent. The issues in the upgrade made the project itself difficult but the teams worked so well together to get the issues fixed that we made an increasingly impossible deployment date on time
   • Quote from Steph  I would like to note down that both Nik and Chris have been working like legends on this for us, definitely a good project team effort.
   • Quote from Colin I would like to note the excellent supportive team work all round on this project dealing with obstacles as they came along; some great work to document processes for future projects too. Thanks everyone

Outstanding Issues

• A decision has also been taken on CU3 long term and LTW now do not consider this a priority and as they are unlikely to get a window at Christmas for rolling CU3 out they are prepared to live with CU2 for the coming year. This project can therefore be closed within normal timescales.  
• The following are to be taken forward under business as usual:
  • Shared courses: TEL053-8 has been closed but LTW are developing a test plan in service.
  • TEL053-10 (not working in Safari) - Fixes supplied but Blackboard not working so Steph is to go back to community and Southampton University. LTW were happy to go live with the issues as Steph is sending comms out to mitigate the issues  -  issue will move into service to be resolved
  • TEL053-9 (not in UK English) - Fixes supplied but Blackboard not working so Steph went back to community and Southampton University. LTW were happy to go live with the issue as Steph sent comms out to mitigate the issue. Taken into service to be resolved.
• Metrics gathering:  Nik will retrieve previous peak usage statistics for Steph as they move into October/November – their busiest period - this will done be in service

To note

• Load testing – note for future: DevTech had recommended that load testing was not warranted as CU2 was a minor update.  Mark L confirmed that the database on the shared infrastructure was well over spec’d and he had never seen the database stressed and Steph confirmed that CPUs don’t look bad in monitoring. Mark L recommended however that it would be worth doing if we were ever to do a major upgrade (see piccl 6)