Completion Report

Objectives  

PM Notes

Complete

To successfully procure a new Lecture Recording service.  This procurement phase will produce the procurement strategy, evaluation criteria, the selection schedule and an award.

Completed successfully with the award being made as scheduled in the original plan.

A successful procurement will perform an important step in satisfying these wider objectives.

Enhance student satisfaction with learning resources and academic support by automating the recording of lectures at scale across the institution and making them available to supplement face-to-face teaching.

We procured a system that has allowed the Implementation team to successfully pilot scheduling of lecture recording.

Reduce the risks around lecturing as a high-stakes activity, in particular supporting student well-being and reducing stress.

This is a longer-term objective and is being taken forward within the Implementation Programme.

Support the recruitment of international students for whom English is not their first language.

This is a longer-term objective and is being taken forward within the Implementation Programme.

Support the learning of students with specific disabilities.

Echo360 scored highest on accessibility.

Support pedagogical innovation by enabling staff to record their lectures and use contact time for other activities (flipped classroom).

Echo360 offered to be a partner in innovation and part of the contract includes a ‘Lighthouse Agreement’.  This was an important part of our evaluation and Echo360 provided the best offer.

Support sustainability and creativity through growing a repository of high quality recorded lectures that can be shared, re-used and re-mixed as appropriate.

The technology to support this objective is in place.

Support sustainability and simplification by delivering a solution within centrally managed teaching spaces that can be extended into locally managed teaching spaces through a ‘buy-in’ option.

College rooms are being included in the new service.  This will extend as we go through the implementation phases.

Support sustainability and cost-control by delivering a system that has stable running costs and a simplified support model that makes best use of existing resources and facilitates long term financial planning.

We picked a Software as a Service (SaaS) option. Being in the cloud for storage simplifies the service burden greatly.  Additionally, the contract is potentially for 9 years (3+3+3) which assists financial planning.

Enable more flexible use of the physical estate as lecture delivery will not be limited by physical room size.

This is a longer-term objective and will be taken forward within the Implementation Programme.  The technology platform is now in place to support this objective.

Create new opportunities for research, particularly in relation to learning analytics and teaching at scale.

Phase 2 of the implementation project contains a reporting workstream which will advance this objective.  Echo360 have committed to supporting this work.

Deliverables

Notes

Complete

Project governance arrangements.  The board will be appointed by the Project Sponsor and the administration will be supported by the LTW Director’s office.

Complete.

Procurement strategy including the appropriate sign off at Director level.

Complete.

Continued lecture recording consultation with University colleagues including staff and student representatives.

User consultation within MLE002 contributed to the priorities within the Competitive Dialogue process.  We have a student on the Competitive Dialogue team and staff (both academic and support) contributed to the evaluation.

Additionally, the Programme Structure for the Implementation was setup during MLE002 and includes an Academic User Groups and an Engagement Group.

A vision for the new service.

Complete.

A Communications Strategy and a Communications Plan. 

A Communications Officer has been employed for the implementation.  The feedback on this has been highly positive.

A Contract Notice to establish supplier interest to participate.  The contract notice will be referred to throughout the process and the information held within it is fixed once issued.  The Contract Notice will contain the following elements.

Complete

The Competitive Dialogue phase arrangements.  This will include participant, supplier and room preparation.

This went well and the suppliers (who were all fresh to the process too) praised the process when we were in dialogue.

A final Invitation to Tender (ITT). This will be built throughout the Competitive Dialogue process.

This worked well and the process allowed the team to produce a simplified, but highly targeted Invitation to Tender.

A contract award to the successful supplier.

Complete.

Support and maintenance arrangements.

Complete.

Implementation plan(s) from the supplier.

Complete.

Training plan(s) from the supplier.

Complete.

What went well

Further notes

Accelerated timeline with a Competitive Dialogue process that on paper should have taken longer than we had.

We pushed the compression of our Competitive Dialogue process to the limit.  Our implementation target was the start of term which was only eleven and a half months away. 

Our knowledge of the market helped. 

Our Sponsor, Business Lead and Project Manager along with members of the Competitive Dialogue team had been involved in the previous Media Hopper Procurement and Implementation.  We could supposition that only three serious lecture recording candidates would emerge and should be taken through the dialogue process. 

Along with our market knowledge we also knew our requirement.  It was to deliver lecture recording only, not media content management or a combination of the two.  Having been through a media procurement recently provided the team with a jump start. That said the process if taken from the template should have taken an extra 3-4 months and included an extra evaluation stage to reduce the suppliers during dialogue.

The project team successfully evaluated the risk of compressing the procurement timeline, against running a successful procurement and delivering an unchallenged award.

First time success with Competitive Dialogue.

A Competitive Dialogue process had not been run before within Information Services.  A fantastic job from the team to run the process that proceeded to plan and where stakeholders on both sides of the dialogue knew their role within the process.

Happy suppliers and a confident, knowledgeable project team.

It was the first time using this process and the suppliers reported that they liked the process.  Those on both sides of the dialogue grew to have a clear understanding of the expectations of the other.

Knowledge gained through dialogue allowed us to target key quality and cost criteria  

Areas where there was little or no disparity between the suppliers were given less weight in the Invitation to Tender (ITT).  Those quality elements that became the differentiators could be given a much stronger weighting.  We could not have done this without the dialogue sessions.

Commitment from the team.

Competitive Dialogue demands a great deal of time from the members of the dialogue group.  The team did an excellent job.

The demands on a person’s time was highlighted successfully at the beginning of the project which meant that where managers could not participate, the position and authority was successfully delegated to an experienced member of staff who could. 

Student participation in Competitive Dialogue.

We initially sought two student representatives for the Competitive Dialogue group.  One person did assist with the early work on User Evaluation but then couldn’t participate further.  The other person became a crucial part of the dialogue group.  They attended all the sessions in some capacity and provided a strong, honest voice during evaluation.

The success continues into implementation as the student became the Lecture Recording Intern and has now joined Information Services as a part time worker during the student’s final year.  They have also chosen to do her dissertation on elements of Lecture Recording.

Cloud vs On Premises

There are two elements to this.  Firstly, the dialogue process allowed the team to investigate the viability of each solution.  In the end we chose a cloud supplier.  The second element is that it flushed out the longer-term ambitions of our suppliers.  For example, during the second dialogue Echo360 announced they would only support a cloud installation and had instead accelerated their decision to be a cloud only supplier for future tenders.

Total Cost of Ownership (TCO)

With a 30% weighting for price TCO was very important to the outcome of the procurement.  The team did a good job through dialogue to tease out the requirements of running a software solution reliably on a podium PC versus running separate hardware recording kit.  As an example, Newcastle operate a software recorder, but have since installed a second PC in each room that is dedicated to the recording.  This risk factor was included in our TCO for rooms.  The software recorder was still cheaper, but the TCO scoring was fair.

Jump start for the Implementation

Names to faces, personal trust, a good deal of technical knowledge in both camps and clear expectations on delivery dates were all established during dialogue.  Additionally, the project scope for year one was largely agreed.

Good examples of work that was prioritised could be seen across Infrastructure where the email alias uun@ed.ac.uk was implemented, which significantly aided the implementation.  Additionally, a dedicated person was provided to configure network switches, essential to the successful outcome of the implementation. 

We knew about these priorities, because those team leaders spoke to the supplier during dialogue and tentative implementation plans were in place.

Simplified question sets

As noted we used simplified question sets, slimming down a lot of the standard questions used in previous procurements.  We covered all the points necessary, just in a simpler structure.  Our questions sets have been reviewed after the event by a team within IS and they have been taken forward as a template for future procurements.

Commercials were mainly completed before the ITT was released.

The dialogue process allowed much of the work on commercials to be done before the ITT was released so there were no nasty surprises in the costs put forward in the ITT.

Lessons

Further notes

No rest or reflection between dialogue sessions.

With the necessity to compress the timelines the team felt that it could have been beneficial to have at least one week between each of the dialogues.  This would give staff more time to reflect on the activities within the procurement and reset their minds as each dialogue session would cover a different agenda.

Compressed timelines and the demands that placed on staff

This process was hard on staff who still had to deal with Business as usual alongside this demanding process.   Staff were having to cover the increasing demands of the project with business as usual work. 

Although the procurement and implementation timeline given on the 26^th of September 2016 by University Court is unlikely to be repeated, if it should be then a review of key stakeholder’s responsibilities should be done to make sure a healthy work life balance is promoted.

The dialogue was comprehensive other than Terms and conditions which in the end delayed the contract.  

The timeline and complexity of the procurement gave the team a very narrow window within which to resource, analyse and prepare the process.

The team did a comprehensive job to mitigate many of the pitfalls after award on a more standard process such as technical and prices surprises.  Although legal were involved a stronger steer on terms and conditions would be recommended for future dialogue processes.  We would go as far to say a specific session should be run to cover terms and conditions only.

The process of reviewing a fixed number of suppliers once set, could not be altered.

To keep the timeline in line we omitted a step from the process which would have seen us reduce from 3 suppliers to 2.  A success in that we could hit our targeted timeline, but in future where time allows this step should be included as the team very quickly knew that only 2 suppliers were going to be viable for the University.