Completion Report

Project Summary:

1) UoE successfully completed KIS Return for 2012/3 (C12061) and 2013/4 (C13061) and thus meet the relevant HEFCE statutory regulations.  

2) Facility now exists in EUCLID CCAM for Schools to capture Breakdown of Assessment for each Course.

3) Facility now exists in EUCLID CCAM for schools to capture Breakdown of Teaching and Learning activities for each Course. This allows Schools to capture granular information (e.g. lecture hours, online activity hours) for Scheduled Learning and Teaching.

4) KIS Course information available on the BOXI Reporting Tool to allow analysis within the Schools. Course KIS related data available on the DRPS for applicants and other audiences.

5) All UG Courses have Assessment and Teaching/Learning data captured. The majority of PG Courses have been captured.

6) All business processes to support the creation/maintenance of KIS core data, and the generation of the return, are now fully managed within EUCLID and its underlying software (with the exception of accreditation data).

Benefits:

1) This was primarily a compliance project and thus the key benefit of enabling HEFCE compliance was met.

2) Schools are now required to input Breakdown of Assessment data and Breakdown of Teaching/Learning Activity data. This will provide valuable insight into how courses are delivered and assessed across the university, and allow comparison against other institutions delivery of programmes and courses. This centralised, comparable course information has not been available before within the University in an accessible format.

3) Schools are responsible for the creation and maintenance of the core KIS data.

4) Personal Tutors can now use KIS information to inform choices on electives, thereby enhancing the Student Experience.

Analysis of Resource Usage:

Staff Usage Estimate: 229 days

Staff Usage Actual: 73 days

Staff Usage Variance: -68%

Other Resource Estimate: 1 days

Other Resource Actual: 1 days

Other Resource Variance: 0%

Explanation for variance:

The original budget was 229 days. This revised down to 114 days at the mid-year review due to a large portion of the work being excluded and integrated into PCIM (details below). The budget was further revised to 83 days and delivered within 73 days.

The following items were descoped which constitutes the variance:

1) The following objective was not fully met: 'SACS to define a Long-term strategy regarding use of Programme and Course Information which will support/link into KIS and also the prospectus and DPS and DPT'. This was descoped from SAC002 and integrated into a separate 2013/4 project  PCIM (Programme and Course Information Management) Project. Further information on this project can be found at this location at https://www.wiki.ed.ac.uk/display/PCIM/Home.

2) Accreditation processes required to support the KIS return not fully integrated into EUCLID. There was a lack of build resource at the time to implement this change.

Key Learning Points:

What went well:

1. The key requirements were met within the timescale, and close working between Lesley Elliot, Karen Harris and later, Duncan Scott, ensured that the relevant lines of communication were maintained through-out the project.

2. Project delivered on time despite three separate Projects Managers over the lifetime of the 2-year  project.

3. All aspects that were delivered were done within EUCLID and following the same business processes across the schools.

4. Large amount of data inputted or bulk loaded into the system requiring a lot of effort upfront from the schools, SACS and the project team. However, going forward, the effort supplied by schools will be more focussed on course KIS maintenance and inputting data for new courses. In other words, the ongoing impact to the schools will have reduced and will level out.

5.    The majority of the key communication out to the business was managed by Karen Harris which made the project much more manageable.  

What didn't go so well:

1. Due to the sheer volume of data that needed to being collected form all the schools, the decision was made to use spreadsheets in year 1 and 2 to manage and bulk upload the core data into EUCLID. This was the correct decision (the alternative being manual keying into EUCLID for all courses), but did involve a lot of time and manual intervention to chase the schools and standardise the data for the bulk import.

2.Lack of development resource at the time within a transitional SSP meant that some aspects of the  project were descoped despite investing time in the business analysis (e.g accreditation).

3.      Some requirements/issues were missed during the Requirements analysis phase which could have been picked up earlier by going to a sample of schools with a working prototype. The risk at the time is that it could have opened a debate on the definition of the Scheduled Teaching and Learning categories, but this could've have been placed 'out of bounds' for any feedback session.These requirements were not critical to the project delivery but visibility of them will have improved preperation for the types of issues that will have been encountered during implementation (e.g. zero credit courses). All outstanding issues have been noted as user stories in 2013/4 SAC020 Statutory Returns Project and are detailed at the end of this report. 

4. User acceptance testing did pick up some items for change, but was too brief. However, even if the UAT was more comprehensive, there would have been no capacity to make changes to address any issues raised, so engagement with the users earlier, even in the later stages of development, would have been beneficial. 

5. The original solution delivered by Tribal had several issues which took a lot of time and effort to resolve. This should have been better tested internally by the vendor before delivery

6. Due to Tribal's late delivery of the solution, and lack of documentation about how they intended to provide their solution, temprary fields needed to be set up in SITS, and development resource required to migrate data and logic to the SITS fields once Tribal finally delivered their solution.

7. The CCAM changes exposed some underlying QA issues with the way courses are set up (e.g. zero credit courses. courses with 7.5, 12.5 credits), and the decision was made not to amend the screens to cater for these scenarios which is causing an issue in the schools.

What would you improve?

1. I would invest more time in speaking to a wider set of school course administrators when designing the screens to drive out usability and business process issues. The lesson learnt was to involve this subset of representative user's at an earlier stage and to validate the requirements and the business processes underlying the change by presenting a working prototype where possible. SSP are actively moving in this direction by adopting Agile methodologyies when implementing change. 

2. Greater involvement of SACS Support earlier in the project would have resulted in the person running the KIS return being fully aware from the outset what was being proposed and the timescales for development. This may been have exacerbated by changes in Project Management.

3. At times the project was a bit rushed, especially during the development of CCAM requirements, and perhaps stepping back for a period to reflect on what was trying to be achieved would have benefitted the project.

4.. Even more effort, perhaps at a higher level, is needed to insist that Tribal provide fit for purpose solutions for compliance requirements and provide visibility of proposed solution documentation in good time to inform the technical approach to take in the UoE implementation.

5. Compliance projects sometimes expose underlying issues with existing business processes which may not be easily rectified in the timescale for development but need to be factored in any estimation.

Outstanding issues: