Closure Report

Project Summary

Blackboard Collaborate is the University’s centrally supported system for providing an online collaborative learning experience. The tool is provided through the centrally supported VLE’s, Learn and Moodle, and also via an integration with MyEd. 

The Collaborate MyEd channel was developed in 2016 (TEL032) as a result of the move to Collaborate Ultra.  When Collaborate Ultra was adopted, the product was still very much in its infancy and significant new functionality has since been added, which is reflected in the Learn and Moodle integrations.    

There is now a lack of continuity (in Collaborate) between the above VLEs and MyEd in terms of user experience and cosmetic feel.  The Collaborate channel in MyEd is the only way for students to schedule their own Collaborate sessions, which are used as a secure online space for group work and/or interviews for research and dissertations. The MyEd integration is used extensively by Student Recruitment and Admissions (SRA) and Student Induction and therefore is often the first digital experience students have of the University.   In addition, the Collaborate channel in MyEd is often the only way non-teaching staff have access to Collaborate to use for hosting online training, virtual meetings or large live online events. 

The Collaborate service team have also determined that in order to keep the MyEd channel reasonably aligned with other integrations, a review and upgrade of the channel would be required every 2 years. 

Scope

The scope of this project encompassed the following:

• Analysing the optimum development path for delivering a modified and upgraded Collaborate channel
• Determining the new functionality to be included in the upgraded MyEd channel
• Developing, in the appropriate DEV environment, a new/replacement channel that will offer the desired functionality
• Promoting the developed channel into our TEST environment and making this available for testing
• Conducting user testing of the new channel and reporting any issues
• Preparing support and help documentation for support staff and users
• Deploying the upgraded channel into LIVE so that it is available to staff and student users
• Updating related technical documentation

Out of scope

• Any changes or development work on other offerings of Collaborate
• Any changes or upgrades to the infrastructure currently used

Objectives and Deliverables

Analysis of Resource Usage:

Staff Usage Estimate: 81 days

Staff Usage Actual: 108 days

Staff Usage Variance: 32%

Explanation for Variance

• Time

The project had an original, planned delivery date of 5th July 2022, and an actual delivery date of 5th July 2023. There were obviously several delays to getting the major stages completed, and these were attributable to resourcing problems across Software Development and the LTW team at different times of the gap between the two July dates.

The first delay was reported in issue 3. This was down to the demand on colleagues in Software Development and the difficulties in assigning them to scheduled project plans. In this case, the resource was assigned but conflicting demands, and delays, on other projects impacted these original bookings and, in moving them out, another member of the team had to be assigned. This meant that the original plan had to be updated. Delivery was still to be in July 2022 at this point

The next change - reported in issue 4 - saw the milestones being moved out again (with delivery in September 2022). The reason for this modification was the need to release booked resources to People & Money ahead of its final launch. The build sign-off was met and user acceptance testing due to be carried out, but the project was delayed because of issues when the new channel was being uploaded into TEST. Issue 5 explained this problem.

The difficulty with the environment was resolved and the new channel added into TEST. This allowed the UAT to be carried out and feedback reported back to the project team. However, we ran out of time to complete the required fixes before the deadline of the start of term to launch the new channel. The project team subsequently planned to launch the new channel in December 2022, after the end of the teaching session. The detail for this was recorded in issue 6 and the new dates were communicated via issue 7.

The remedial work and the subsequent retesting of the new channel were further delayed for a few months because of the demands on colleagues in Software Dev and LTW. This was eventually restarted in January 2023 but took a longer time to complete than planned because of an issue that emerged in moving the reworked channel into TEST. This was attributable to changes in the portal, and took several weeks to resolve. This round of date changes was summarised in issue 9, with the second round of testing due to finish by 24/05/23 and a deployment date of 13/06/23.

The user acceptance testing was completed as replanned, but the final sign-off of this was delayed by a few weeks because of the delays in hearing back from the supplier (Class) about an issue that the project team was keen to resolve. This was the functionality around the ability to download poll results from a recording of a session. It was not available in the original version of the channel and it was hoped that it might become something that users would have access to in the new channel. This final delay was reported in issue 10. This had delivery as 05/07/23 and closure as 20/07/23.

The former of these dates was met, but the delivery sign-off and closure were delayed for several more months because of a variety of issues that became apparent in the LIVE (not launched) channel. These have been problems round elements of the wider channel functionality rather than changes introduced by the development of a new channel but they have needed to be addressed to ensure that the channel works as expected and to re-assure users that the replacement channel is not problematic.

The fisrt of these was reported in issue 12 in July 2023 and was concerned with difficulties when deleting saved recordings and with emails to invited participants. These were resolved and a new sign-off date of 11/10/23 was established. This was not met because the user team were now reporting difficulties for some staff in loading the new channel. This was covered in issue 13 and the sign-off was moved out to late November. This was fixed but took longer than anticipated and the sign-off was moved out again to January 2024 under issue 15. This was not met because of staff absence and workload and so the sign-off was shifted again (issue 16) and was then revised once more to late March because another issue - an error message saying that the portlet could not contact Collaborate - was reported by the testing team. This was reported in issue 17 and the milestones moved in issue 18, with this finally being resolved this month, and the fix that has been deployed being successfully tested across the project team and subsequently signed off

• Effort

The original estimate (with the project finishing in 21/22) was for 81 days. By the end of the financial year 2021/22, a total of 54 days had been used by the project, and the remaining work, which would have to be done in 22/23, was optimistically estimated at being completed using another 9 days effort. However, this did not work out as expected and the longer lifespan of the project and the need to revise the remaining work, as well as the rework and testing by the development team meant that an additional 32 days [up to 12/04/23] had been used across the project, bringing the current total of effort used over the two financial years to 86 days. The remaining work on the project was estimated at a final extra figure of 14 days, bringing the new total effort required to 100 days. This requires an extra allocation of 37 days for the project, in addition to the 9 days allocated at the start of 22/23.

More effort has been used in 23/24 with the small fixes that have had to be developed and implemented. This has totalled 8 days in 23/24, bringing the final figure for effort used to 108 days.

• Scope

No changes were made to the scope of the project.

Key Learning Points

The experience of testing the updated channel in this project has emphasised the need to conduct regression testing as part of a project that is introducing new or amended functionality. While the latter was formally tested (and passed) as part of the project, the channel performance in the context of the wider operations was tested by the business users and that is where various issues were picked up. Software Development have also highlighted the importance of UAT in a project like this so that testing of functionality is carried out in the correct context to ensure the viability of what has been modified.

Outstanding Issues

There are no outstanding issues. The new channel has not yet been launched to a wider audience. When this happens, the old channel should be decommissioned. This will be co-ordinated between the Service owners in LTW and the MyEd team.