Completion Report

Background

This project was to implement and pilot the Totara LMS system which had been procured by CSG under a previous project (ref. CSG007), to help track and manage the learning and development activities of their staff.

Outcome

The user group decided that the pilot had not sufficiently met their success criteria and a decision was taken not to proceed past the pilot.

I enclose the message from Charles Hill, the project sponsor, below:

"Dear Colleagues

Following last week’s Totara Pilot review meeting (Tues 24^th Oct) I thought it would be helpful to formally advise you of the decision taken to withdraw the project.

At the meeting we received feedback from each of the 3 pilot groups, Estates, CBS, and Finance. We noted a score of 5, 7, and 7+ out of 10 from the groups based on the feedback received.

There were several points noted, below, which contributed to the decision:

1. The solution didn’t deliver at least 80% of the user requirements threshold.
2. A view (particularly from Estates) that Totara was too admin heavy, and require extensive administrative resourcing
3. Supplier not able to provide confidence in their ability to integrate MyEd Event Booking service with Totara
4. An emerging requirement to secure staff data ‘in transit’ to supplier which had not yet been fulfilled
5. The aim to move to a university wide learning management system in the next few years as part of the HR transformation programme
6. The Procurement Office were not able to participate in the pilot, as their resources had to be diverted to an eProcurement system migration project.  Their business requirements were similar to those of Finance. 
7. A concern (particularly from Estates) that staff training records would have to be migrated in the near term to Totara, and again in a few years to the new university wide system
8. The HR Learning and Development team did not support a wider rollout
9. An acceptance that CSG teams could run with existing solutions until the new university wide system comes on board
10. Continuing with the Totara solution would incur costs of extending  the Totara contract AND funding of a Grade 5 FTE post within HR to provide user support.

On behalf of Hugh (Edmiston) I would like to thank those involved in the pilot review, we know there was a lot of hard work put in. Of course it does mean that CSG are well positioned to contribute to the wider University Learning and Management system as part of the HR Transformation Programme.

Many thanks again,

Charles"

CSG are now more informed about their requirements to feed into the HR Transformation Programme work towards procuring a university wide system.  A review session was held and a summary of learning points shared with members of the HR Transformation Programme Team (see attached, below).

learning_from_the_csg_totara_lms_pilot.pptx

Objectives

Deliverables

Analysis of Resource Usage:

Staff Usage Estimate: 50 days (ISG)

Staff Usage Actual: 135 days (ISG)

Staff Usage Variance: 170%

Other Resource Estimate: 0 days

Other Resource Actual: 0 days

Other Resource Variance: 0%

Explanation for variance:

The project budget was increased from 50 days to 135 days, in April 2017.

The reasons for the increase were additional scope for the project, and a higher than expected resource requirement for project management, business analysis and software development. 

The additional scope for the project included:

• The Project Manager taking on the role of a Lead System Administrator during the pilot, as CSG were unable to supply resource to do so
• Making MyTraining a managed service with assignment of Business Service Owner / IS Service Owner and Service Manager roles for the system
• Development and testing of automated feed to manage CSG starters, leavers and changes in user data
• A feasibility study (with cost specification) to determine if Totara LMS can integrate with the University's Event Booking application with regard to course catalogues and the course booking process

The project team requested the business requirements to be re-validated which led to a higher than expected estimate for business analysis.  Additionally, the project management resource requirement was identified as being higher than expected following further review of the supplier's implementation plan and the requirements to set up and run the pilot.

Key Learning Points

Successes:

• The user group were committed and engaged in trying to make the solution work, both in advance of the pilot and during the pilot itself - thanks to all involved for their effort

• A number of the more straightforward business requirements and user scenarios were proved to work during the pilot

• Those end users who responded to the user survey (although response rate was fairly low) found the basic functions (adding training needs, marking courses as complete) to be simple and straightforward to use
• A summary of learning points has been shared with the HR Transformation programme

Issues:

• The training guides for system administrators were not issued by the supplier until late in the pilot, where they should have been issued immediately after the training
• The IS Project Manager fulfilled the role of a lead administrator / co-ordination point for the pilot, where this should have been fulfilled by a CSG colleague
• The feedback from the pilot showed most users found difficulties in interpreting the definition of 'external' training.  Some users found system navigation tricky, or difficulty in linking courses to their plans

Recommendations for similar projects:

• HR Transformation programme to review findings from Totara LMS pilot in the context of the requirements they develop for their procurement of a learning management system
• A periodic check to ensure the project still has the right level of priority at senior level, especially when the priority was decided a long time before the implementation activity
• The lead system administrator should be from the business, not IS, to ensure knowledge is built up within the user community for when the project completes
• All training materials should be made available on the day of the training session or immediately afterwards
• The effort involved in a temporary solution (before a university wide solution is brought in) should be considered and ensure it's appropriate for the length of time it will be in place

Outstanding issues:

• None noted

Attachment	Size
learning_from_the_csg_totara_lms_pilot.pptx	43.16 KB