Project Brief

RSS044 - DataVault III

Approvals

Name Role Position Date
Tony Weir Project Sponsor Head, ISG IT Infrastructure 12/04/2019
Robin Rice Service Owner Data Librarian & Head, Research Data Support, L&UC 28/03/2019
Fraser Muir Senior User Chief Information Officer, CAHSS 29/03/2019
Anthony Davie Senior User IS Campus Leader, MVM 29/03/2019
Colin Higgs Senior User Computing Officer, School of Engineering 29/03/2019
Kirsty Lingstadt Senior Supplier Head, Digital Library & Deputy Director, L&UC 29/03/2019

David Fergusson

Senior Supplier Head, ITI Research Services Section 29/03/2019
Ianthe Sutherland Product Owner Development & Systems Manager, Digital Library 08/03/2019
Maurice Franceschi Programme Manager (RSS) ITI Portfolio Manager, ISG

07/03/2019

Distribution

Name Role Position Date
John Baker Senior Supplier Director, Digirati 15/04/2019
David Graham Senior Supplier Head, ITI Communication Infrastructure Services 15/04/2019
Graeme Wood Senior Supplier Head, ITI Enterprise Services 15/04/2019
Claire Young Senior Supplier Operations Manager, EDINA 15/04/2019

Background

The University of Edinburgh’s DataVault is a component of Research Data Management services which assists researchers to comply with funding requirements for the long-term retention of their research data. In 2015-16, the JISC funded DataVault project run by the Library and University Collections (L&UC) team at the University of Edinburgh, in collaboration with the University of Manchester, developed generic software designed to collect basic research metadata and deposit research datasets into an archive service. In parallel, the University’s ITI Research Services Section (RSS) defined an initial systems architecture to provide robust archive services.

Due to demand and the time to develop a UoE specific DataVault service, an interim service was launched in 2016. The interim service used the RSS Storage Manager facility to manage research data deposits on the disk-based DataStore system.

A project to deliver a full, support DataVault service began in 2017 and the service was launched in January 2019. This main objectives of the DataVault project (RSS022) were to deliver the following -

  • expand the interim service to include –

    • an improved (self-service) user interface
    • encryption of data
    • resilient storage on internal and external archive services
    • usage reporting
  • design a robust system architecture for the service
  • define the processes and policies of the full service
  • achieve GDPR compliance for the service
  • launch DataVault as a fully supported Research Data service

The latest project, DataVault III (RSS044), funded for FY2018-19, will run from February to September 2019 to deliver additional features to the DataVault service including –

  • an improved user experience (UX) and user interface (UI)
  • organisational structures and roles for data management
  • automated billing and usage reports
  • the migration of deposits from the interim DataVault service
  • storage of deposits up to 10TB
  • improved encryption management
  • reduced processing times and improved resilience

Project Scope

DataVault has been developed to provide a low-cost and reliable archiving service for the University’s research community in compliance with research data management and funding policies. The scope of the current project will concentrate on improving the service for the University’s research community.

Significant work on the design and development of the DataVault user interface was achieved in the previous project. Further work in this project will include a review and redesign of the user experience and interface based on feedback received during user testing and in the early adoption period of the current DataVault.

Initial DataVault requirements included the need for secure, data management within a research group or department, by incorporating organisational structures and roles, so that the University retains control of data access and retention if the original researcher is no longer a member of staff. This is a major feature to be delivered by this latest project.

Currently only deposits of 2TB can be stored and retrieved in DataVault. An increase to 10TB deposits is a key deliverable of this project and has been requested by several researchers interested in storing their data in the ‘vault’.

The DataVault has been designed to archive personal and sensitive data, within the scope of GDPR regulations, by (a) advising users to anonymise or pseudo-anonymise personal data and (b) encrypting all data prior to transmission and deposit, using SHA-256 protocols, to the archive subsystems. Further work will be undertaken in this project to improve encryption and key management processes.

Out of Scope

Expansion of the service to support the long-term needs of other groups within the University or elsewhere, outside the research community, is not within scope for this project. Future projects could be initiated to expand the service to provide, inter alia, -

  • archiving of University business data
  • providing a DataVault service to other institutions

Objectives & Deliverables

The following key objectives are planned for the next DataVault project, DV III, with starts in February 2019.

  • Increase maximum deposit sizes to 10TB (currently 2TB)
  • Improve release management
  • Introduce processing recovery procedures for partially failed dep
  • Implement organisational structures and roles for data management
  • Improve the website user experience (UX)
  • Provide storage auditing features
  • Implement automated billing
  • Improve retention date management features
  • Migrate deposits from the interim DataVault service
  • Improve existing vault webpages where feedback has been received e.g. remove encryption key cut-and-paste
  • Replace the PURE flat file interface with the PURE API for accessing metadata
  • Provide usage reports for Schools administrators
  • Simplify retention policy updates
  • Improve encryption key management
  • Optimise deposit and retrieval processing times
  • Improve resilience and recovery features
  • Provide workshops for researchers and support staff in all Colleges to present the features and benefits of using DataVault

More details on specific deliverables from the project are given in the table below.

Requirement Deliverable(s) MoSCoW
Increase maximum deposit sizes to 10TB (currently 2TB) A DataVault service handling data sets up to 10TB in size Should
Improve release management An automated and reliable deployment service for release to test, demonstration and live environments Should
Introduce processing recovery procedures A recovery subsystem that restarts processing from point of failure (or start of current stage at minimum) Must
Implement organisational structures and roles for data management

A user interface and back-end subsystem that supports a research organisation structure and the ability for a PI to share a data archive with colleagues and department supervisors to reassign ownership of deposits

Must
Review the website user experience (UX) A report on the user journey plus the suggested redesign of the user interface to simplify navigation and support additional features Should
Provide storage auditing features Regular data integrity checks (a) database records against archive storage data and (b) regular, random deposit retrievals to check datasets Must
Implement automated billing An automated billing system using usage statistics to provide DataVault users with invoices for costs of storage to replace the current manual methods Should
Improve retention date management and review features A subsystem that manages retention and review dates logically and effectively Must
Migrate deposits from the interim DataVault service To be done manually by retrieving from the interim vault and depositing to the DataVault Must
Improve vault webpage features and PURE interface Improvements to vault webpage; an improved metadata connection using PURE API Should
Provide usage reports for Schools administrators A reporting subsystem proving usage reports automatically to senior research staff Must
Simplify retention policy updates An automated data-driven subsystem that provides facilities for DataVault administrators to easily update retention policy information Must
Improve encryption key management An industry standard encryption and key management system integrated into DataVault Should
Optimise deposit and retrieval processing times Improved processing which reduces deposit and retrieval times by a significant margin Must

Benefits

The successful delivery of this project is expected to provide the following benefits –

  • Improved website features for users depositing, reviewing and retrieving research data sets to attract new users and retain existing ones
  • Recovery, from point of failure, of deposits during processing if impacted by planned or unplanned outages
  • The ability to share and take ownership of data sets within a research group
  • Clear usage and billing information for users and supervisors with costs retained at current low levels
  • Deposits in the interim DataVault moved to the new service, enabling decommissioning of the interim service
  • Increased compliance for research datasets to be archived, through improved management of research metadata
  • Better control of retention dates so that deposits can be reviewed and removed in a timely manner, reducing costs to data owners
  • Deposit and retrieval processing times reduced and so reducing risk of unplanned outages
  • Improved data security through more robust encryption key management and other features
  • Enable researchers to archive deposits up to 10TB thus increasing user community

Success Criteria

The following are the criteria to be met to ensure a successful completion of this project –

  • An increase in the usage of the DataVault service through the provision of additional features, improved resilience, reduced processing times, increased deposit sizes and regular data audits
  • Reduction in the loss of deposits in progress from unplanned outages
  • Research datasets being shared between researchers and colleagues which is a prerequisite for the transfer of data in the interim vault to new vault
  • Ownership of datasets taken over by senior or other colleagues when a principal investigator leaves University employment
  • Increased awareness of research archive costs through the provision of clear and regular billing information
  • Increased use of PURE to store research metadata, as a prerequisite for DataVault use

Project Milestones

The currently estimated major milestones, with delivery dates, are outlined in the following table to provide estimates when product releases and other deliverables will be made. Development work has been allocated to EDINA and Digirati, as appropriate.

Milestone Responsible Approval Date
Initial review of scope and requirements Pauline Ward Robin Rice 15/02/2019
Project brief and initial project plan approved Lawrence Stevenson Tony Weir Robin Rice 29/03/2019
System documentation complete Robin Taylor Tony Weir Senior Users 29/03/2019
Processing recovery procedures operational

Robin Taylor David Speed

Ianthe Sutherland 26/04/2019
Website user experience (UX) report, redesign and wireframes available Digirati Robin Rice 10/05/2019
Deposit and retrieval processing times improved Robin Taylor David Speed Ianthe Sutherland 24/05/2019
Organisational structures and roles for data management implemented Digirati Robin Rice 07/06/2019
Storage auditing reporting in place Robin Taylor David Speed Ianthe Sutherland 07/06/2019
Usage reports for Schools administrators available Digirati Robin Rice 21/06/2019

Deposits from the interim DataVault service migrated

Pauline Ward Robin Rice 05/07/2019
Improved release management in place Robin Taylor William Petit Ianthe Sutherland 05/07/2019
Improved retention date management in place Digirati Robin Rice 19/07/2019
End FY2018-19     31/07/2019
Improved vault webpage features and PURE connect Robin Taylor David Speed Ianthe Sutherland 02/08/2019
Automated billing launched Digirati Robin Rice 16/08/2019
Improved encryption key management integration Robin Taylor William Petit Ianthe Sutherland 30/08/2019
Maximum deposit sizes increased to 10TB Robin Taylor Ianthe Sutherland 27/09/2019
Project closed Lawrence Stevenson Tony Weir Senior Users 18/10/2019

Alignment with University Strategic Vision

This section defines how the areas of the upgraded service aligns with the University’s strategic vision and contribute to run, grow or transform services. DataVault III will extend the facilities of the existing service.

Vision Commentary
A unique Edinburgh offer for all our students  
  • All our undergraduates developed as student and/or researchers with clear, supported pathways through to Masters and PhD
N/A
  • All our students offered the opportunity to draw from deep expertise outside their core discipline

N/A

  • A highly satisfied student body with a strong sense of community
 
Strong and vibrant communities within and beyond the University – making the most of our unique offer of world-leading thinking and learning within one of the world’s most attractive cities N/A
A larger, more international staff who feel valued and supported in a University that is a great and collegial place to work, develop and progress DataVault supplements existing services to researchers
More postgraduate students – underpinned by the best support in the sector to ensure we attract the brightest and best regardless of ability to pay Not currently supported
A strong culture of philanthropic support focussed especially on our students and on outstanding research capabilities. N/A
Many more students benefiting from the Edinburgh experience (largely or entirely) in their own country – supported by deep international partnerships and world leading online distance learning N/A
Sustained world leading reputation for the breadth, depth and inter-disciplinary of our research supported by strong growth in research funding and strong international partnerships – drawing from well-established and less well-developed sources DataVault provides a unique service, not currently available to Universities in the UK
An estate that matches expectations, responds flexibly to changing student and staff needs, and showcases the University The DataVault service meet the needs for data retention required by funding bodies as well as meeting GDPR regulations
A deeper and earlier collaboration with industry, the public sector and the third sector – in terms of research; knowledge exchange; and in giving our students the best possible set of skills for their future DataVault has the potential to be extended to provide low-cost, secure storage for legacy data that require retention by other organisations in the University e.g. Finance and to external organisations seeking similar services
Digital Transformation The DataVault will facilitate the preservation and creation of metadata for research outputs, enhancing the prospects for re-use of that data and reducing risk of loss of data which might previously not have been backed up. It will also allow researchers to free up space on their active data storage for use in further work. 

Alignment with ISG Strategic Vision

This section defines how the areas of the upgraded service aligns with the ISG’s strategic vision.

Vision Commentary
Student Experience  
  • Student experience and the unique Edinburgh offer
N/A
  • Online and distance learning leaders
N/A
  • Library national and international leadership
DataVault development has been driven by the Library and University Collections team who recognised a gap in service provisioning in an area that could be exploited in the UK and overseas
Research and Innovation  
  • Research IT and Data Sciences
The service provides the University’s research community with ability to store completed research data cheaply, meet regulatory requirements but be accessible as and when required
  • Innovation
A market review has shown that DataVault provides a type of service that is currently not available but would benefit University researchers across the UK
  • Collaborative leadership and social responsibility
The University, along with the University of Manchester, are taking the lead in delivering a JISC funded version of DataVault for use in UK Universities
Service Excellence  
  • Process improvement, efficiency, quality and best practice
DataVault provides researchers with a secure central facility to improve their research process management for storing, retrieving and sharing past research data
  • Long-term IS strategic planning and linked professional services
DataVault is one of the strands of development to meet the goals of the Research Data Management Roadmap, which also includes complementary initiatives such as DataShare and Data Safe Heaven
  • Information Security
DataVault incorporates secure user account management by integrating with the University’s EASE and Shibboleth authentication services. The service also incorporates client-side encryption and decryption to ensure the secure transmission and storage of data

Impact

The impact of the continuing development of the DataVault service is outlined in this section where priority, budget and contribution to IS change are presented.

Priority & Funding

DataVault III is a project in the University’s ITI Research Services Section (RSS) programme for FY2018-19 funded by the Digital Research Services (DRS) Steering Group.

Budget

The table below shows funding that has been allocated to the project for this financial year plus the forecast for continuing the project into FY2019-20, but excludes FY2018-19 costs for previous DataVault project (RSS022)

 

Project Financial Summary FY2018-19 FY2019-20 Total
Approved Budget £155,000   £155,000
Revised Budget/Forecast £170,000 £28,000 £198,000
Capital £110,000   £110,000
Revenue (Revised) £60,000 £28,000 £98,000

IS Change Programme

The following table shows how the DataVault project contributes to the themes that underpin the University’s change programme.

IS Change Programme Theme Contribution
Project Management The ITI project management process is being used for this project.
Working Together The project team includes representatives for each of the University’s schools, members from digital library and research data services, internal (EDINA) and external (Digirati) development staff and infrastructure teams from research and enterprise services. The team was formed to ensure all the required skills were available to ensure a successful project. In particular, a wide-ranging requirements exercise was undertaken to ensure the right work and priorities were encapsulated in the project.
Standards and Technical Leadership DataVault is being developed, in collaboration with JISC, to provide a research data archiving and retrieval service that is will close a gap in service provisioning for UK Universities’ research communities.
Staff Learning and Development Some members of the team have developed their documentation and presentation skills to deliver various customer-facing activities such as presentations and workshops in the previous DataVault project. The new project gives them an opportunity to further develop these skills. The project team have benefited and will continue to benefit from planning and delivering a service using formal project management practices.
Service Based Culture Consultation with users and other stakeholders has been undertaken at every stage of requirements and design process. A web-based user interface has been developed to make the usability of the service as easy as possible. Support and assistance are provided by the Library Research Support services to users prior to, during and after they have used the service. Extensive web help pages, including a range of ‘how to’ videos have been developed and will continue to be improved in this project.
Equality and Diversity The DataVault website is be developed to incorporate accessibility features to ensure access is available to all that require to use the service. The University’s equality policies are followed by all team members in all communications, meetings and other channels used in the project.
Flexible Resourcing The project team is supplemented by both development staff from EDINA and an external design and development team from Digirati, a Glasgow-based software development company. Both these teams expand the resources and skills of the Digital Library development team.
Communication and Branding

A communication plan section is included in this project brief. The existing DataVault service and new releases to be delivered by this project are/will be communicated to the University’s research communities by digital and print media, by presentations and workshops and by the cascade of announcements by schools’ representatives on the project board.

The design and development of the DataVault website and supporting materials follows, and will continue to follow, the University’s branding guidelines and web design (EdGEL) framework.

Organisation

Work Breakdown

The following diagram shows the main components of work to be undertaken in the project

 

Resources & Skills

The following table defines the team members, role and skills and an estimate of the days allocated for work to the project. Days allocated for Digirati work still to be reviewed and agreed.

Team Member Role Days Allocated
Tony Weir Project Sponsor 4.0
Robin Rice Service Owner 18.0
Ianthe Sutherland Product Owner/Development Lead 25.0
Pauline Ward Business Analyst/Senior Tester 44.0
Anthony Davie Senior User (MVM) 4.0
Colin Higgs Senior User (CSE) 4.0
Fraser Muir Senior User (CAHSS) 4.0
Martin Donnelly Operations Manager (L&UC) 4.0
Jennifer Daub System Tester 6.0
Kerry Miller Schools Liaison/System Tester

8.0

Kirsty Lingstadt Senior Supplier (Digital Library)

8.0

Robin Taylor Lead Developer 44.0
Claire Young Senior Supplier (EDINA) 4.0
David Speed Developer (EDINA) 48.0 (25.0)
William Petit Developer (EDINA) 40.0 (35.0)
John Baker Senior Supplier (Digirati) 4.0
Sarah O’Sullivan Workstream Manager (Digirati)

12.0

Daniel Grant System Architect (Digirati)

12.0

Diego Lago UX/UI Designer (Digirati) 20.0
TBA UI Developer 42.0
David Fergusson

Senior Supplier (RSS)

4.0

Kevin Tomlinson System Administrator (RSS)

6.0

TBA System Advisor (RSS) 4.0
Graeme Wood Senior Supplier (ENT) 2.0
TBA System Administrator (ENT) 4.0
David Graham Senior Supplier (CIS)

2.0

TBA Network Administrator (CIS) 4.0
Lawrence Stevenson Project Manager 24.0
Maurice Franceschi Programme Manager 8.0

Projected Costs

In terms of current projected costs, the following expenditure has been estimated for the project as follows –

Category Expenditure FY2018-19 FY2019-20
Capital Digirati Development £73,000  
       
Revenue EDINA Development £25,200

£14,000

  Curation Consultancy £25,000  
  Thales SafeNet Licences £12,000  
  Oracle Cloud Archive Service (2-years)   £14,000
  Revenue subtotal £62,200 £28,000
       
Internal ENT SAN Increase to 10TB (per annum) £16,740  
       
  Total £151,940 £28,000

A capacity planning exercise, based on current and estimated growth data, will take place during the design phase of the project, to ascertain if and when additional compute power and/or storage will be required during the following 3-year period, in addition to the planned 10TB storage upgrade already planned.

Project Governance

Overall project governance will be provided by the project board whose members are listed in the next section. Day to day project management and delivery is the responsibility of the project team.

After initial planning and design, a backlog will be generated in Jira for the project plan and internal development will be run in sprints managed. The project plan will also be kept up to date for progress reporting purposes to the project board and other stakeholders.

Work packages to be developed by Digirati will be assigned to the nominated workstream manager to be managed internally. The workstream manager will provide regular project reports on a work package and raise any escalations that cannot be resolved internally.

Each work package, when delivered, must have clear documentation and, where appropriate, installation, configuration and test instructions. In some cases, Digirati staff will need to work with internal development staff to integrate and test work packages in the DataVault test environment.

Digirati will adhere to the current Data Vault development standards to ensure a product that can be maintained into the future. EDINA development staff will provide Digirati with the appropriate source management and deployment standards to ensure seamless integration of all code work packages that they deliver. University user and VPN accounts will be provided to Digirati staff where work package delivery makes this necessary.

The current system design is included in the systems documentation. DataVault III is an upgrade project so any systems design effort will be around the planned incremental work. On that basis, the project is divided into work packages to facilitate (a) assignment of work to EDINA or Digirati, (b) changes to scope and priorities and (c) the incremental design required within each work package.

The initial Digirati work package will review the current systems architecture against planned DV III work to provide design recommendations on the improvement and expansion to the system.

Where work impacts other teams, e.g. ENT or RSS, the internal development team will engage with the systems experts in those teams to ensure the integration of system design and implementation into the parts of their infrastructure which supports DataVault.

Project Board

The project sponsor is Tony Weir, ITI Director. Project board meetings will be scheduled monthly to review progress, major escalations and resolution. The following senior stakeholders will be invited to participate in project board meetings.

Role Name Position/Team
Project Sponsor Tony Weir Head, ISG IT Infrastructure
Service Owner Robin Rice Data Librarian & Head, Research Data Support, L&UC
Senior User Fraser Muir Chief Information Officer, CAHSS
Senior User Anthony Davie IS Campus Leader, MVM
Senior User Colin Higgs Computing Officer, School of Engineering
Senior Supplier Kirsty Lingstadt Head, Digital Library & Deputy Director, L&UC
Senior Supplier

David Fergusson

Head, ITI Research Services Section
Product Owner Ianthe Sutherland Development & Systems Manager, Digital Library
Project Manager

Lawrence Stevenson

ITI Project Manager, ISG

Programme Manager (RSS) Maurice Franceschi ITI Portfolio Manager, ISG

Project Team

The following diagram shows the key members of the DataVault III project team.

 

Outline Project Plan

The table below show the initial estimated duration for the main tasks in the project. Tasks to be allocated to EDINA for development are marked (E). Tasks to be allocated to Digirati are marked (D). Tasks in red indicate dates past the end of FY2018-19. During the planning and design phase, modules will be reviewed to ascertain if work can be overlapped or reduced in duration to see if additional releases can be delivered before the end of FY2018-19.

Task Duration Start Finish
Define initial scope, requirements and priorities 25

14/01/2019

15/02/2019
Project planning and design 30

18/02/2019

29/03/2019
Review system architecture and documents (D) 5 01/04/2019 05/04/2019
Implement resilience and recovery procedures (E) 20

01/04/2019

26/04/2019
Prepare UX proposal, design, wireframes (D) 15

08/04/2019

10/05/2019
Improve deposit and retrieval processing times (E) 20

29/04/2019

24/05/2019
Develop/integrate organisational structures and roles (D) 20

29/04/2019

07/06/2019
Implement storage auditing reporting (E) 10 27/05/2019 07/06/2019
Build/integrate usage reports for administrators (D) 10

27/05/2019

21/06/2019
Migrate deposits from the interim DataVault service 20 10/06/2019 05/07/2019
Improve release management processes (E) 20 10/06/2019 05/07/2019
Build/integrate improved retention date management (D) 20 10/06/2019 19/07/2019
Improve vault webpage and PURE interface (E) 20 08/07/2019 02/08/2019
Build/integrate automated billing (D) 20

08/07/2019

16/08/2019

Implement improved encryption key management (E) 20 05/08/2019 30/08/2019
Increase maximum deposit size to 10TB (E) 20

02/09/2019

27/09/2019
Close project 15 20/09/2019 18/10/2019

Assumptions

The following assumptions have been identified at the start of the project and may be expanded during the project.

  • Formal planning and design will be completed prior to any development work starting
  • EDINA resources will be available at the required level for the duration of the project
  • Digirati remote working will still underpin the momentum of the project
  • Research testers will be available during test periods
  • Senior project team members will be available for project boards, escalation and resolution of issues, etc. as required

Constraints  

The main constraint for the project is that some team members are not fully assigned to the project and are working on other projects and supporting BAU activities. This can be translated into a key risk were precedence for resources may go the BAU work or work on a higher priority project. Management commitment needs to be made to days allocated in the resource plan and to the overall commitment needed.

Additionally, it may be management decision to complete the project by the end of FY2018-19. If it is agreed to close the project in July 2019, the current outline plan shows that two work packages, Improved Encryption Key Management and Deposits up to 10TB, may not be delivered unless additional funding is obtained for the work.

Development resources on the project are coming from EDINA and Digirati, an external software development company. Work packages have been allocated to ensure work can be undertaken by both group in parallel to reduce the project timeline. However, some issues may arise in such areas as communication with remotely located teams and clashes for time to test on the single test environment available.

Dependencies

Unlike the previous project (e.g. TSM integration, cloud service procurement), there are limited external dependencies currently predicted. The following are the current dependencies that have been noted during project planning –

  • DV III will be at a release level that supports migration from the interim vault e.g. roles and organisation in place
  • Clarification of EDINA policy on deployment management e.g. move to new Puppet environment
  • Availability of the PURE API to improve the metadata interface
  • Support from ENT to upgrade SAN storage on live workers to 20TB (to support 10TB deposits)
  • Support from RSS, ENT and CIS to review processing times and identify bottlenecks in the process
  • Availability of a supported encryption and key management subsystem replacement e.g. Thales SafeNet

Risks

The initial project risks identified are as follows –

  • Current system documentation will need to be reviewed and updated to provide a firm base for formal design
  • The loss of a key project team member in April and their replacement not on board
  • EDINA development staff not available towards the end of the project
  • Budget will not cover Digirati costs for all work allocated to them
  • Additional licencing (e.g. TSM) or hosting costs (Oracle Cloud Archive) may be incurred as the service is expanded
  • Potentials users will not use the DataVault service e.g. user interface is too complex, processing takes too long, PURE metadata prerequisites

Issues

If they arise, issues will be recorded, on the Projects website, and tracked throughout the project. Issues that are not outside agreed tolerances will be resolved within the project team. Major issues need to be escalated to the project board for review and advice on resolution. Issue resolution that requires a major change to the project in terms of time, quality or cost needs to be approved under formal change management by the project board.

All project changes outside tolerances must be reviewed and approved by the project board. Project changes within tolerances can be managed with the project team.

Project work that impacts production systems or networks must be communicated with the requisite lead time to the ISG CAB for review and approval.

Tolerances

The following standard ITI project tolerances will be applied and these are outline below. Any deviation outside these tolerances must be reported for review and resolution to the Project Board, if constituted, or the Project Sponsor and Portfolio Manager otherwise.

  • Cost – over +/- 10% of original budget
  • Plan – slippage of over 4 weeks to current plan
  • Scope – extensive addition or reduction from current scope
  • Quality – refusal to approve a project work package

Lessons Learned

A log will be maintained on the Projects website to record lessons learned throughout the project.

The key observations from the previous DataVault project are summarised in the table below –

Observation Description Recommendations Impact
Limited knowledge transfer from interim system Development team for DV II not the same as used on interim system but there was limited knowledge transfer from interim system which put the new development team at a disadvantage. Minimal documentation produced from interim system. Ensure DV III architecture and operations documentation is of a detail and quality that benefits DV III developers. High
New development team engaged distinct for interim system team

Development for DV II was subcontracted to EDINA.  The lack of knowledge transfer and documentation provided a steep learning curve for the developers.

Ensure DV III development team includes EDINA developers, if possible. If not ensure L&UC development staff are involved in the final stages of DV II and the system is fully documented at the architecture and operations level. Medium
No detailed system design There was no appointed system architect, so no detailed system design was undertaken. A 3-page outline system design document was produced. Extensive design decisions were done 'on the fly' during the build phase of the project. System evolved rather than being designed. Ensure system architect in place and detailed design - functional and infrastructure - is completed in a distinct design phase prior to system build. High
Workflow has high working storage overheads and long processing times Workflow is inefficient with high working storage overheads and long processing times in what is essentially a system integration exercise e.g. encryption, TSM, Oracle. DV is fundamentally a secure archiving service. A detailed redesign exercise is recommended to ensure the service provides acceptable archive and retrieval times. The system design evolved and includes various steps that could be replaced by more efficient methods.  High
Planning moved to Jira when project was in flight A decision was made to use Jira (Scrum version) for planning by adding functional requirements directly into the Jira backlog in November 2017 (project started February 2017). Limited formal planning seems to have been in place. A formal plan was reverse-engineering from Jira in January 2018.  Jira is a short-term planning and tracking tool. It does not provide facilities for overall project planning such as scheduling, resourcing and dependencies. Use a formal planning process and tools for DV III. Jira can still be used for task level initiation and tracking. Medium
Adoption of Jira forced a Scrum based approach DataVault is not best managed overall as an Agile project for the following reasons - The system is quasi-regulatory (e.g. GDPR required). The target system was well defined. The project team was not 100% assigned to the project and had limited Agile/Scrum experience.

 

DV III project should be managed using formal requirements-planning-design-build-test-launch approach. Once complete, the plan can be added to Jira and a sprint or Kanban based delivery initiated. Medium
Long lead time identifying cloud supplier The selection of Oracle cloud as the external archive supplier took over 6 months using the G-Cloud framework. With the value of the contract (£13,500 over 3 years) a single supplier could have been engaged directly.

 

Ensure procurement approach is appropriate for the costs involved. High
TSM licences not available as agreed TSM licences from the RSS pool were to be used but when they were required they were not available. Unplanned expenditure (£10,600) was required to purchase the requisite licences.

 

Ensure commitment to supply licences is formally confirmed. Medium
Processing on DataStore impacted other users Processing on DataStore, a networked shared storage system, impacted other users including staff network drives. Testing had to be curtailed and a request to move to ENT SAN raised where performance and reliability improved. Ensure appropriate storage technology is used for DV working storage during the design phase. High
Design of the system infrastructure was ad hoc Design of the system infrastructure was done just prior to the provisioning of final test and production environments. The team had limited infrastructure design knowledge.

Ensure infrastructure design is included at the start of the project as part of the system design process.

Medium
Significant budget overspend on development resources Spending on development resources ate into the FY2018-19 project (DV III) budget potentially reducing the work that could be done on the next phase of DV.  Effective planning at the start of the project identifies resource and scheduling requirements that can be used to control project spend more efficiently. High
Three changes of project manager During the project, four different project managers were assigned to the project which contributed to delays in progress as handovers were completed Ensure a single project manager is assigned and retained on the project throughout. Medium

Communication

The following meetings and communication channels will be put in place to support the effective communication of project progress, events, product releases, and risk and issue management.

Forum Frequency Participants/Recipients Communication
Sprint Planning Start of sprint Robin Rice Ianthe Sutherland Robin Taylor David Speed Pauline Ward Lawrence Stevenson Updated plan and backlog, next sprint
Sprint Meetings Weekly or daily, as required Robin Rice Ianthe Sutherland Robin Taylor David Speed Pauline Ward Lawrence Stevenson Progress report, updated risks & issues, next steps
Sprint Retrospective End of sprint Robin Rice Ianthe Sutherland Robin Taylor David Speed Pauline Ward Lawrence Stevenson

Lessons learned, actions

External Workstream Reviews Fortnightly Digirati team Ianthe Sutherland Lawrence Stevenson Work package requirements, progress reports, and modules for test and integration
Project Board Monthly Tony Weir Robin Rice Fraser Muir Colin Higgs Anthony Davie Kirsty Lingstadt David Fergusson Ianthe Sutherland Lawrence Stevenson Maurice Franceschi Meeting minutes & progress report, escalations and resolutions
Project Report Monthly

Project Manager All stakeholders

Project website
Project announcements & updates ​​As required Project Manager or Service Owner, as appropriate

Email, IS Alerts, website announcements

ISG CAB As required Appropriate Senior Supplier or Project Manager Change requests

Project Timeline

 

AttachmentSize
Image icon rss044_plan.png129.95 KB
Image icon rss044_wbs.png231.73 KB
Image icon rss044_project_team.png260.59 KB

Project Info

Project
Data Vault III - Continuing Development
Code
RSS044
Programme
ITI - Research Services (RSS)
Management Office
ISG PMO
Project Manager
Lawrence Stevenson
Project Sponsor
Anthony Weir
Current Stage
Close
Status
Closed
Project Classification
Transform
Start Date
04-Feb-2019
Planning Date
29-Mar-2019
Delivery Date
14-Feb-2020
Close Date
21-Feb-2020
Programme Priority
1
Overall Priority
Normal
Category
Discretionary

Documentation

Plan