Closure Report
Project Summary
The IT Implementation of the NHS Lothian Health Board Framework Agreement project (MVM319) was the next step from the report completed in the Lothian Health Board Framework Agreement Implementation project (MVM317) which explored actions that should be taken in the short, medium, and long term to move towards compliance with the Framework Agreement and the National Health Service Lothian Health Board (NHS-LHB) Requirements.
The primary deliverable of the project is a SharePoint site (referred to as the 'Resource') to act as a single point of digitised information to present relevant UoE IT guidance for use by all UoE stakeholders involved in conducting agreed joint research projects, where the Framework applies. The full development of the SharePoint site (O1) remained within College of Medicine and Veterinary Medicine (CMVM), and there was no requirement for involvement from the SharePoint Team or Software Development within IS Apps.
For an overview of the NHS Lothian Health Board Framework Agreement please see the Project Brief Overview.
The project began with a Discovery phase which gathered Subject Matter Experts (SMEs) from Data Managers in CMVM and Clinical Managers in the Academic and Clinical Central Office for Research and Development (ACCORD) to contribute to identifying the process researchers take in managing their research data against the NHSLHB Framework Agreement. This this allowed the project team to capture the IT information pain points researchers and SMEs were experiencing to ensure the final deliverable (the Resource) was supporting both the professional services members and researchers in answering questions around IT compliance for their research projects. After Discovery was completed, the SharePoint was drafted across two versions and were User tested by the SMEs to ensure the success criteria was met, and the content on the Resource was accurate. Alongside this phase, the project team drafted and agreed the documentation to support the Resource after project closure.
By the Closure of the project, all Must Have Deliverables were met and signed off by the project team, and all milestones were met within project budget. It was well received by all members of the College of Medicine and Veterinary Medicine.
Well done and thank you to the [project] team for helping to deliver this.
| Scope | Remained in Scope? |
|---|---|
| Produce a SharePoint resource tool on appropriate use of UoE recommended IT systems, services and processes that provides guidance for those UoE staff who contribute in agreed joint research activity with NHS-LHB, governed by The Framework. | Yes |
| Meet with Subject Matter Experts (SMEs) across CMVM and ACCORD to gather requirements and conduct analysis on research examples, IT and Data administrative processes, and assess the IT tools used for their projects | Yes |
| A due diligence of the IT tools was conducted in the project and are presented in D4.1 and D4.2. | Yes |
| Out of Scope | Remained Out of Scope? |
|---|---|
|
Any support, guidance or assistance with non-IT aspects for the operation or maintenance of the Framework, such as:
|
Yes |
|
Creation and management of an associated Data Asset Register, as referenced in MVM317.
|
Yes |
|
Developing a logging system for keeping track of research progress, including monitoring progress for the research cycle. |
Yes |
|
Developing online/digital Information Security questionnaire, as referenced in MVM317. |
Yes |
| Additional action following the gap analysis from MVM317. | Yes |
| Registering or tracking use of Bring your Own Device for those involved in operating the Framework Agreement project/s. | Yes |
Objectives and Deliverables
| Ref | Objective/Deliverable | Priority | Completed | Comment |
|---|---|---|---|---|
| O1 | Create a single point of digitised information to present relevant UoE IT guidance for use by all UoE stakeholders, involved in conducting agreed joint research projects, where the Framework applies. |
Yes |
||
| D1.1 | Create a SharePoint site that can be edited and maintained by relevant College of Medicine and Veterinary Medicine (CMVM) IS staff. | M | Yes |
https://uoe.sharepoint.com/sites/USDCMVM/NHSL/SitePages/Steps-to-comply.aspx Access Request Required |
| D1.2 | Technology Lead, to agree with CMVM IS Leads on the mechanism and agreed process/es for authoring and approving changes to the SharePoint site content. This should include who (from current CMVM IS staff) will be given permission to manage access to the SharePoint site and role-based permissions for initial authors. | M | Yes |
Review document here for reference. This document is a living document on a private Teams account managed by the College of Medicine and Veterinarian Medicine. |
| D1.3 | Once SharePoint site framework/templates are developed and process/es are in place to author content, test access controls work as intended. Establish foundation process for reviewing the Framework Agreement at regular intervals. | M | Yes |
The project team made a decision post-delivery to review the analytics of the SP Site every 6 months for changes to the Site. Please see D1.4 for more details. The legal representative for the Framework Agreement will get in touch with the Heads of IT at CMVM for any changes they need to be made aware of. The responsibility of updates to the SP Site is on the Owners of the Site, but they are not responsible for seeking updates to the Agreement as they should be made aware of this if changes are required. |
| D1.4 | Establish BAU analysis process for user feedback and site visits to be conducted at regular intervals to agreed Owners and Informed Area Experts identified. | S | Yes | Please see Page 7 of the Procedures and Process document attached to D1.5. |
| D1.5 | Establish a user feedback loop for site visitors to share their experience with the resource tool and provide Owners with both positive and negative insight. | S | Yes |
Review document here for reference. This document is a living document on a private Teams account managed by the College of Medicine and Veterinarian Medicine. |
| O2 | Re-assess the IT requirements in relation to the current Framework Agreement to ensure we are referring to current guidance, based on up-to-date NHS LHB policies, codes of practices and standards. | |||
| D2.1 | Obtain the latest version of NHS LHB policies, codes or practice and standards, as referenced in the Framework Agreement and compare it to previous versions. Document specific IT requirements and controls that must be adhered to which have changed, and compare against UoE recommended IT service, system and process controls. | M | Yes |
Used the most recent approved version of the Framework Agreement for the creation of the SharePoint site. The link to the Agreement is not shared on the Resource for legal reasons (confirmed by our Legal Stakeholder, Graham Ayres). If you wish to review the Framework Agreement, please reach out to the Project Sponsor for direction. Graham Ayres also signed off on the published content in the Resource prior to Delivery. |
| O3 | Conduct an information discovery exercise with the Framework subject matter experts, to create real user case examples for agreed joint research work, and to enhance knowledge for the project stakeholders. | Yes | ||
| D3.1 | With key project stakeholders and Framework subject matter experts - identify a small number of typical joint research projects examples for further review. | M | Yes |
Typical project examples were captured at the SME workshops. Miros for Governance and Practical sessions can be viewed below - please request access to review the content. Governance: https://miro.com/app/board/uXjVKrQueo4=/?share_link_id=529236283604 Practical: https://miro.com/app/board/uXjVKrQkhKM=/?share_link_id=985116155007 |
| D3.2 | Using the examples, map out The Framework joint-research project process flow, and key decision points, where specific IT requirements may be needed. Document outcomes. | M | Yes | To review the process and identified IT touchpoint please log into Miro and request access here: https://miro.com/app/board/uXjVKjHcZ2k=/?share_link_id=234795490698 |
| D3.3 | Using the examples, document which IT systems, services and processes were used at each decision point, and review outcomes of use, identifying any issues, challenges, or gaps that stakeholders encountered along the way. | M | Yes |
Documents were gathered from SMEs and saved to Teams Files for review and inclusion in the SharePoint site. For a final confirmed list please visit the resource: https://uoe.sharepoint.com/sites/USDCMVM/NHSL/SitePages/Using-software.aspx Access Request Required |
| D3.4 | Using documented outcomes from D3.1 and D3.2, assess if appropriate IT systems, services and/or processes were used to meet the current requirements of The Framework (O2). Document these where adjustments need to be made, to achieve or enable improved compliance. | M | Yes |
For collated documents you can review them in our Teams Files: Phase 1 Discovery Access is restricted to Project team. Please request Access to review documents. |
| D3.5 | Invite and collect feedback from SMEs on the architecture of the SharePoint site for both Version 1 and 2 prior to deployment. | M | Yes |
User testing was performed after SMEs workshops, with process agreement and SharePoint Architecture confirmed. Script and user responses can be reviewed here: User Testing Access is restricted to Project team. Please request Access to review documents. |
| O4 | Create a list of recommended UoE IT/IS services that can be used during operation of joint research project, operating under the Framework. | |||
| D4.1 | Using the deliverables from O3, identify the tools and services that can be recommend for use. | M | Access is restricted to members in the CMVM. For limited Access, please Request Access onthis Link to view IT systems and services allow or disallowed: https://uoe.sharepoint.com/sites/USDCMVM/NHSL/SitePages/Using-software.aspx | |
| D4.2 | Using the deliverables from O3, identify the tools and services that cannot be recommend for use. | M | Access is restricted to members in the CMVM. For limited Access, please Request Access onthis Link to view IT systems and services allow or disallowed: https://uoe.sharepoint.com/sites/USDCMVM/NHSL/SitePages/Using-software.aspx | |
| D4.3 | Collate guidance on the best way to interact with and make use of UoE IT/IS services, any costs associated with use/on-going use and how to obtain support, should this be required. | S | Information on this topic has been added to various pages in the Resource, and can be further explored on the landing page for compliance checks: https://uoe.sharepoint.com/sites/USDCMVM/NHSL/SitePages/Steps-to-comply.aspx | |
| D4.4 | Create relevant IT guidance for use to populate the SharePoint site using outcomes from previous deliverables to create content on SharePoint site ready for review. | M |
Key Pages meeting this deliverable are around data in UoE and NHS-LHB joint projects: https://uoe.sharepoint.com/sites/USDCMVM/NHSL/SitePages/Data.aspx https://uoe.sharepoint.com/sites/USDCMVM/NHSL/SitePages/Using-datastore-NHSL-proj-data.aspx |
|
| D4.5 | Link Service Level Definitions to IT/IS services in scope for use, where this is beneficial. | C | No |
As deliverable was Could Have and was not detrimental to the project the project team determined there was no value added to meeting Deliverable. There was no benefit to the Resource, and could confuse researchers unnecessarily. |
| O5 | Maintain clear and timely communication across Stakeholder Groups throughout the project lifecycle. | |||
| D5.1 | Create a communication plan and schedule early in the project to identify users, communication type, frequency, and communication owner. | M | Yes |
Access is restricted to Project team. Please request Access to review document. |
Benefits Identified and Realised
| Ref | Benefit | Realised | Comment |
|---|---|---|---|
| 1 | Mandatory compliance of the NHS Lothian Framework Agreement between NHS Lothian and the University Court.
|
Expected, Yes | Reviewed and approved by the appointed Solicitor of The Framework Agreement as a compliance tool for researchers on joint projects. |
| 2 | College business and research strategy requires compliance with the Framework Agreement.
|
Expected, Yes | Will be revisited in future 6 month review sessions by Owners based on visitor feedback and staff satisfaction in the content provided. |
| 3 | UoE colleagues will have easy access to and wider visibility of appropriate IT systems, services and enhanced processes to use. They will have more transparency on costs for using these systems and services, which will help planning and budget forecasting and control. | Unknown | Will be revisited in future 6 month review sessions by Owners based on visitor feedback. |
| 4 | While the resource is designed and purpose-build for researchers in CMVM, the final product could be an exemplar for other Colleges/Schools for assisting their researchers in maintaining compliance with similar frameworks. | Unknown | Noted in Outstanding Tasks |
Analysis of Resource Usage:
Staff Usage Estimate: 40 days
Staff Usage Actual: 32 days
Variance of Effort: - 8 days
Explanation for Variance
This project was allocated 40 days effort from the Core Fund in CMVM, used 32 days to complete the project. The 40 days did not include the effort provided by Subject Matter Experts (SMEs) or members in CMVM who supported the project on their own time within normal business duties.
The project closed under budget by 8 days effort. This was notably made possible by the determination, motivation, and collaboration of the project team. Each member responded very positively to the management of meeting, minutes, and actions, ensuring everyone worked well together.
Outcome
After initiation in June 2024, project Planning was completed in July 2024 with the expectation to Close the project by October 2024. Over the course of each Execution phase it became clear more time would be needed to reach the Deliverables as agreed by the project team, and Closure was pushed to January 2025. The project was divided by Phases: Phase 1 (Discovery), Phase 2 (Version 1 Output), Phase 3 (Version 2 Output), Phase 4 (UAT), Delivery, and Closure.
Discovery
The project team began with workshop meetings with SMEs across data governance and data application/use in the College of Medicine and Veterinary Medicine (CMVM) and Academic and Clinical Central Office for Research and Development (ACCORD). All Subject Matter Experts (SMEs) to the IT Implementation of the NHS Lothian Health Board Framework Agreement project (MVM319) were added to a Private Channel in our MS Team site for engagement and participation throughout the project lifecycle. This Channel was as much their space as it was ours, and SMEs did not hesitate to reach out there for any project queries, concerns, or support.
Prior to the workshops, SMEs were given details on the project Scope as well as a Plan on a Page.xlsx which was a simple visual for the project lifecycle at that time with details on the high-level tasks for the project team and SMEs in each project Phase. The document is set to View Only and also has the comprehensive list of identified SMEs for the Governance and Practical expertise areas. You will need to Request Access to view it. We also provided the SMEs with our understanding of the process to work as a starting point for workshop discussion - NHS LHB FA process discovery.pptx. This initial draft process mapping was developed by our Business and Technical Lead to help start the mapping sessions for our Discovery workshops.
The Governance workshop began with a walk-through with the Data Managers in CMVM of the current process for researchers in joint research projects as whole. This then allowed us to review Governance Requirements for research projects and identify the IT compliance steps needed for researchers against the whole process. As a note, there were no process maps fro the whole end to end lifecycle of joint research projects and each SME area (Data Managers and ACCORD Managers) had local information that was only shared when requested, causing researchers to ask multiple people the same question to ensure compliance for their projects. We identified the researcher and professional services staff pinch points as well as any additional non-IT issues logged. The Practical workshop ran similarly to the Governance workshop and met with Clinical Managers in ACCORD. We shared the process as captured in the previous workshop and logged any aspects not logged in the previous session that were special to ACCORD office. We again noted at IT touchpoints in the process and captured the requirements needed for any Compliance Resource built by the College.
A shorter final workshop with all SMEs took place at the end of Discovery to confirm our findings, the process captured, and sign off on the success criteria needed for the SharePoint Resource. Please see links in Deliverable 3.1 and 3.2 for workshop outcomes.
Success Criteria agreed was: i) Checklists-like option for researcher, ii) Something quick to review , iii) A reassurance that the researchers are aligning with the Framework Agreement
Building the SharePoint
After Discovery was signed off by the project team and SMEs agreed to the rough architecture needed for the Resource, work began on Version 1 of the Resource. The Business/Technical Lead and Project Sponsor first focused on creating content for the Resource outside of SharePoint by collating everything from Discovery on Word documents and editing content as needed. This then led to the Site Page creation and development of Version 1.
In September of 2024, a major technical incident at the university diverted attention from our project team to manage the situation, which caused a change in Milestones as resource was negatively impacted. It also became apparent that the Technical Lead needed support in building the SharePoint site and requested additional support from a key Computing Officer in the College with SharePoint development knowledge. The Change Request pushed our Delivery deadline back from November 2024 to January 2025 and Closure from December 2024 to January 2024.
The additional support from our Computing Officer ensured we could create a 'Test' site outside of the College SharePoint that could be used to track version history, and limit access to only the Project Team and SMEs. Nothing was published to the public at this point. The primary communication tool for the project team was MS Teams where we Posted themes for the SharePoint and collaborated in real time for decision making and edits to the Resource site.
UAT & Sign Off
Version 1 was completed by 31st October 2024 and User Acceptance Testing (UAT) began in November 2024. In Planning the project team expected the version development would happen prior to UAT and that a single instance of testing would occur before the Delivery. However, it became clear that two rounds of UAT - one for each version of the resource - would be the best method for this type of SharePoint (informational).
The project team agreed to the testing script which consisted of 5 questions on user experience and 5 questions on content review (veracity). Our approach was to have 1:1 or 2:1 MS Team video sessions with the PM and SME(s) where the SME was asked to share their screen so the PM could see their experience in engaging with the SharePoint site and answering the questions. They were encouraged to provide a 'train-of-thought' review as they attempted each question. You can see a PDF of the script here: 
testing_script.pdf.
Video recordings and transcripts of the sessions were provided to the project team to review user pain-points and feedback, but a collation of feedback was also saved to a excel spreadsheet which our Technical Lead and Technical Support used to apply Tester feedback through edits to the Site in preparation of Version 2 of the resource. An anonymised version of the captured feedback can be seen here: 
anon_uat_comments.xlsx.
At the end of each user session, SMEs were asked to grade the SharePoint site by the agreed success criteria from Discovery by giving a score from 1 to 10 ( a 1 being not reached at all, and 10 being the success criteria fully met). The overall Score for the Version 1 SharePoint Resource was 7.5 out of 10.
Version 2 was also provided for additional User Acceptance Testing in November 2024 after edits were applied from the cumulation of feedback gathered in UAT for Version 1. The process for UAT was exactly the same as Version 1 to keep consistency of questions, and grading of success criteria. The overall Score for the Version 2 SharePoint Resource was 8.5 out of 10.
Additional feedback was gathered for Version 2 which allowed the project team to make the final necessary edits to the resource. Once those edits were applied, the SMEs were given the final version (essentially the 3rd Version) in December and sought their Sign-Off for Delivery in January. All except two SMEs provided sign off - the two who did not sign off were extended annual or personal leave and could not provide sign off at the time. It should be noted that one of the two did give their approval on their return to work in January and was very happy with the product delivery to the College.
Delivery & Closure
As the SMEs had given their sign-off for the final SharePoint Resource, the project team met the agreed milestone for delivery on 8th January 2025. Delivery of the Resource was twofold i) the Technical Support published the site to the CMVM SharePoint Hub with the permission settings and active directory groups enabled, and ii) an email drafted by the PM and finalised by the Project Sponsor was sent by them to the College and all project Stakeholders. This officially marked the resource as delivered by the project team.
Prior to Closure, the project team was required to finalise SharePoint documentation (see Objective 1 above). Actions and feedback post-delivery fed into the process design for Change Requests and Feedback Gathering from site visitors, and was signed off by the project team as Standard Operating Procedure for the Site Owners group. The site was added as a resource to UoE Digital Research Services resource finder.
Between the Resource being shared on 7th January and the time of this Closure on 30th January, there had been 1,052 visits to the SharePoint Site, 138 unique visitors with an average visit time of about 6 minutes. These number will be reviewed against the site as a whole, and by Page, at the 6 month review conducted by the Site Owners group in BAU.
Closure of the project was met without resistance on 31st January when all Objectives, Deliverables, and the Closure Report were reviewed and signed off by the project team.
Congratulations on a great piece of work and a transformational step in the right direction. This is exactly what researchers need to comply with the tortuous requirements of the Framework Agreement.
Key Learning Points
| Ref | Date | Title | Impact | Theme |
|---|---|---|---|---|
| 1 | 28-Jan-2025 | Agree to the change and feedback process prior to Delivery | Should implement as good practice | Management of a Project |
| 2 | 28-Jan-2025 | UAT for information-based SharePoint sites | Should implement as good practice | Management of a Project |
| 3 | 28-Jan-2025 | Documentation for SharePoint IT Settings with clear owners and maintenance orders | Should implement as good practice | Support |
Outstanding Issues
| Owner | Issue / Action |
|---|---|
| Deputy Head of IT, CMVM | Set up communication point for feedback and change request decision making. For the first 6 months after project Closure, the MS Team Site used for the project will be used by the Deputy Head of IT and CMVM Computing Officers to discuss and decision-make on all matters relating to the change request and feedback submitted by visitors. At the first 6 month review session for the Resource SP Site's documentation and change review, the owners of this Issue will create and use an alternative tool for tracking and communicating Resource changes. It is understood that the members in ISG/Project Services will not be a part of this and will not action or support the changes (BAU) of the resource in any way. |
| Head of IT, CMVM | To decide by the next academic year if the Resource will be made available to the whole of the University in line with the proposed Benefit #4 (see above). |
| Deputy Head of IT, CMVM | Decision from the project to create a reference point for staff members to review PEN tested IT services will be owned by the IT team at CMVM and only shared as reference upon request when sponsors or joint researchers need evidence of services having been PEN tested. While originally the expectation was for the added deliverable (in scope) to be shared as a resource with data managers, the Deputy Head of IT will instead create a local living document for review as needed and upon request as the details for PEN testing an sometimes be sensitive data. The owner will draft this on their own time in the next few months. |
| Head of IT, CMVM | To send out communication updates on the project and SharePoint Resource to the CMVM Newsletter 'Scope' |
| Attachment | Size |
|---|---|
| testing_script.pdf | 54 KB |
| anon_uat_comments.xlsx | 24.63 KB |
 mvm_nhsl_it_ss_perms.docx | 2.11 MB |
| procedures_and_process.docx | 34.48 KB |