Completion Report

Project Summary:

The project had to procure a replacement for the current clickers system.

Initially, the project had systems with proprietary hand-held devices in scope, but the colleges later changed the requirement to be BYOD.

Scope

to work with Procurement and do Market Research to determine the best procurement model (EU, Open, Restricted, Framework) to establish a good team from IS Applications, USD, Desktop Services, and technicians and academics from each college to ensure procurement goes well to confirm that there is moneys for buying a product and - if appropriate - paying for the physical handsets (clickers) work with management to determine if a Senior Supplier is needed, if a Project Board is needed As part of Analysis work with Procurement to publish a PQQ (if that is the model) work with stakeholders to determine requirements and publish the ITT and T&C's As part of Acceptance work with Procurement to determine the selected suppliers answering the PQQ (if that is the model) work with stakeholders determine the winning supplier answering the ITT

As part of Delivery work with winning supplier to finalise the legal contract

 

Objectives

Enhance the current clicker technology by procuring the most appropriate technology for the University’s needs.

Expand the pedagogies available to lecturers by increasing the question types available to include ones that promote higher level skills.

Make it easier to expand the service by using supported hardware and licensing of seats to enable the use of students’ personal web enabled devices.

Allow for participation in interactive sessions independent of the actual location of the lecture – e.g. distance learning students watching a streamed lecture or virtual classroom session.

All objectives achieved.

 

Success Criteria

The current system does not allow tracking of the use of it beyond the numbers of handsets used and, in some cases, the numbers of courses using them. It is hoped that a new system with virtual clicker use would allow for more centralised collection of data and tracking of use.

  • Currently, across all three colleges, there are approximately 3500 clicker handsets in use. It would be expected that use of the new system would increase by at least 50%.
  • Use of clickers in HSS to increase from approximately 500 students to at least 1000 students.
  • Use of clicker system to become imbedded as a core teaching tool in all first year courses across Science and Engineering.
  • Increased use of the technology for more ad-hoc, one-off teaching sessions – currently only three mobile clicker kits are available to staff in Science and Engineering plus one central pool of clicker available for loan to all staff.
  • Reduction in time spent on maintenance of clickers by at least 50% with the uptake of virtual clickers using student devices.

 

These measurements of success will need to be undertaken after project is closed. However, the chosen system and the feedback we have already received leads us to expect all of the successes above are achievable.

Analysis of Resource Usage:

Staff Usage Estimate: 55 days

Staff Usage Actual: 63 days

Staff Usage Variance: 15%

Other Resource Estimate: 0 days

Other Resource Actual: 0 days

Other Resource Variance: 0%

Explanation for variance:

The contract negotiations took several months, which added a few more days into the budget for the project management.

Key Learning Points:

1 There should be quality thresholds for Quality, or at least specific sections with Quality.  Without this, there is a risk that low quality, low priced solutions can obtain the highest scores and win the contract.

 

2 It is recommended that when assessing software usability, a hands-on approach (e.g. live demos) should be done when scoring, as opposed to using a written evaluation approach.

 

3 Feedback from stakeholders highlights that our procurement process should consider a more hands-on approach for validating all of the written responses.  If not, then the option should be explicitly considered at the outset of the project.

 

4 It is recommended that the Procurement Overview at the start of the project covers the three points raised above.

 

5 It is recommended that the Project Sponsor is briefed at the start of the project and their role clearly understood.  This would cover:

  • when and if we can negotiate the process
  • confirming the budget is secured
  • the importance and implications of the Quality/Price Ratio
  • engaging with the project team through the project

 

6 The price-quality formula - as noted above, the Project Manager feels that procurement formula is incorrect and is prejudiced for cheaper, poor quality options.  Noted that the price/quality ratio can be changed as appropriate to the project.

 

7 Ensure the Finance review takes place at the start of Assessment.

Although it worked out, The Project Manager thinks we left it rather late to review the Financials in this procurement.

 

8 At the moment, much depends on the members of the team 'remembering' issues from previous procurements, such as those above. Much better to have a checklist of things to consider for project team and sponsors.

The Project Manager thinks it needs Procurement to get together with the Project Managers with the most experience of procurements in a dedicated workshop for a few hours, empowered to make decision that will be implemented.

The Lessons Learned from previous procurement have not been learned. Specific feedback from the OLLIE STU232 project for 1 and 6 above has not fed back to Procurement. It is recommended that the PM meets with IS Apps Senior Management to review the lessons learned from this project and previous procurement projects.

Further feedback from the team :-

The more I use TopHat, the more convinced I am that we came to the right conclusion, and I very much look forward to using it in my teaching next year. 

It was an interesting and rewarding process and I'm glad I was involved in it.

Outstanding issues:

No outstanding issues to report.