Closure Report

Project Summary

A review of the online student experience, conducted by Headscape in the spring of 2016, highlighted many examples of poor user experience (UX) arising from a lack of human-centred design, inconsistent standards, and lack of integration between services.  The insight provided by the study showed that the overall online experience provided by the University is substantially below student expectations.

A significant contributing factor to the current situation is the lack of user experience-related skills and processes with the University’s planning and development practices.

The Digital Transformation  programme has raised the level of skills and awareness of UX practices among staff across all disciplines involved in software procurement, development and service management, both within IS and across all areas of the University.

2018/19 DTI funding brings the opportunity to focus on what we are now looking at as a Digital Experience Standards Service that encompasses the design framework, EdGEL, while continuing to evolve and improve consultancy and training services.

 

Objectives & Deliverables

The deliverables were prioritised using the MoSCoW prioritisation method

M=Must Have; S=Should Have; C=Could Have; W=Want

No

Description

 

Delivered?

Reason for changes/non-delivery

Output

O1

O1 Pilot & evolve SDES process

 

Partial

 

 

D1

Identification of projects for retrospective review and/or in-flight pilot of SDES

M

Yes

 

DES Assessment Work – SharePoint

 

D2

Completed SDES reviews

M

Yes

 

DES Assessment Work – SharePoint

D3

Development of SDES materials based on insight gained in D2

M

Yes

A webpage on Student Digital Experience Standards is available at https://www.ed.ac.uk/information-services/user-experience/design-process/about  

However this is not the final home for the materials (see D4) and the wording in the Standards and the way they  are pitched will need to be reconsidered - project unable to carry over into 19/20

 

 

D4

Provision of direction/requirements to LTW WAC development of a revised Design Framework (EdGEL) website

M

Yes

Provision is delivered. Physical delivery was to have been carried out by WAC but a reconsideration of objectives within WAC meant this would not be completed within the timescales of this project and as project unable to carry over into 19/20. WAC will take forward under LTW. The wording in the Standards and the way they are pitched will need to be reconsidered.

A prototype of how the final documents may look is at https://dg0rl4.axshare.com/#g=1&p=design_system_prototype

It should be noted that this is in very early development and the content may change significantly – should also be a living document. Needs an owner to take forward.

 

Final report on self-assessment by Nicola Dobiecka

 

The EqIA can be found at http://www.docs.csg.ed.ac.uk/EqualityDiversity/EIA/IS-The_User_Experience_Consultancy_and_Training_Service.pdf.

 

SLA is not yet published as a Business Service Owner has still to be appointed but a copy can be found meantime at SLD - UX Services - Service Level Description v1.3.docx

 

 

D5

Revise and execute the Consultancy and Training rollout and communications plan produced as part of DTI009

M

No

This was deprioritised by WAC as it was clear there was no funding available to continue the UX Service as envisaged in the previous project. However there is no piccl to confirm this so cannot be put as withdrawn. However it is clear that the existing comms plan from 18/19 could not be executed because the funding restrictions have significantly changed the contents.

 

D6

User group set up

M

Yes

 

http://bit.ly/UoE-DES-UserGroup

D7

Harmonise activities across SDES and WAC digital consultancy services

M

Yes

This has been completed under WAC who are operating a UX Service for LTW only at present, and they are using DES. The standards are also the basis for development of EdWeb and MyEd, the two key services in WAC.

https://www.ed.ac.uk/information-services/user-experience/design-process/about 

D8

Planning round proposal: evidenced proposition for an ongoing service is presented via the IS Apps planning round providing a sound business case and sustainability plan for a transition to BAU operation

M

Yes

A proposal was drawn up and submitted but it was rejected. Planning round was challenging this year and this project did not make the final cut as other items were more pressing.

However LTW are running a UX service using the standards for LTW. The original scope was to provide the service for the whole of the University but the challenging planning round has not allowed this.

 

 

No

Description

 

Delivered?

Reason for changes/non-delivery

Output

O2

Applications Directorate Adoption

 

Partial

 

 

D1

Align the human centred design process with the project methodologies and project/programme management approaches supported by Applications Directorate  informing the evolution of the Student/Staff Digital Experience Standards (SDES) as required

M

Partial

Project unable to carry over into 19/20 due to challenging planning round

Link to prototype of project methodology for design thinking /human centred approach

 

 

Workshop held on 31 Jan 19 to influence scope:

Workshop notes Jan 19

 

 

D2

Align the human centred design process with the approaches to business requirements gathering supported by Applications Directorate  informing the evolution of the SDES as required

M

Partial

Project unable to carry over into 19/20 due to challenging planning round

UX consultant started to update existing HCD method but felt that this no longer applied.

Human centred design process prototype

This will be trialled by Digital Transformation programme going forward with a view to establishing a working methodology for human centred projects

 

D3

Align the human centred design process with the approaches to software design and development supported by Applications Directorate  informing the evolution of the SDES as required

M

No

Project unable to carry over into 19/20 due to challenging planning round

 

D4

Align the human centred design process with the approaches to software specification, procurement and implementation supported by Applications Directorate  informing the evolution of the SDES as required

S

No

Project unable to carry over into 19/20 due to challenging planning round

 

 

 

All deliverable documents can be found on the DT UX website DTI045 section

 

Success Criteria

Success Criteria as in Project Brief

Delivered

How delivered

WP1

  • Projects for retrospective review and/or in-flight pilots of SDES completed identifying refinements for SDES

 

Yes

DES Assessment Work – SharePoint

Final report on self-assessment by Nicola Dobiecka

  • Further development of the consultancy and training rollout and communications plan (drawn up under DTI009) to include the SDES

 

Not applicable

Due to restricted funding

  • Provide a sound business plan for a transition to BAU operation

 

No  - Rejected

 

WP2

  • Successful alignment of the human centred design process with the approaches to project methodologies and project/programme management, business requirements, software design and development gathering and software specification, procurement and implementation supported by Applications Directorate  

 

Partial

There were issues in this project round timings, availability and challenges around perspectives. Due to these delays we were unable to align the human centred design process with the approaches to project methodologies etc. as much as had been envisaged. However a  project methodology prototypes has been developed and the Digital Transformation project will trial the prototype going forward with projects under their 19/20 programme with the view to embedding the design thinking /human centred approach into the project approach. Following the outcome of these trials we hope to have finalised methodology which will then be rolled out to IS Applications staff via the PMO who are the owners of the Project Services processes and methodologies.

 

 

Benefits

Benefit as stated in project brief

Delivered

How delivered

Retrospective pilot of the SDES service (including proof of the service delivery model via case studies) will support standardisation User Experience research and design processes across the University and help to ensure that the products/services delivered meet users’ needs.

Yes

DES Assessment Work – SharePoint

Final report on self-assessment by Nicola Dobiecka

Raise awareness and promote adoption of user centred practices, including adoption of SDES

Partial

Ongoing work within UX services in WAC:  they highlight that adoption is also happening in Comms and Marketing, and in Student Systems as a result of the Standards pilot work. The Document Mgt team, BI/API Services, Learn and MyEd teams adopted HCD practices in their projects following their work together, although they are unlikely to be appraising their work against the Standards given their design maturity and the issues surfaced in the research. However, it’s feasible that they could achieve this through further engagement with the WAC UX Service.

 

Analysis of Resource Usage:

Staff Usage Estimate (at project brief) : 65 days

Staff Usage Actual: 46 days

Staff Usage Variance (from project brief): - 30%

Staff Usage Variance (from revised budget of 45 days due to project not continuing): +2%

 

 

Explanation for variance

Cost

Project Brief cost    

65 days        

When end of planning was finally approved on 7 Mar 2019 there was concern from WIS that the budget of 50 days was unrealistic. The budget was reviewed and approved at 65 days

Changes to costs

45 days        

However when it was clear that the project would not carry over into 2019/20 the budget was reduced on 23 July 19 to 45 days (piccl 7 refers)

Final Cost

46 days

Actual expenditure

 

Time

Milestones

Project Brief date

Actual Date

Reason for delay

Planning – original timeline

02-Jul-19

n/a

See point 1 below

Planning – revised & approved

22-02-19

07-03-19

 

Work package 1

14-Jun-19

 

 

Work package 2

28-Jun-19

 

 

Delivery

12-Jul-19

12-Jul-19

 

Close

22-Jul-19

31-Jul-19

Carried over into 19/20 with costs allocated under DT Digital Cultural, Adoption & Comms programme

 

Issues in delivering

  1. The original project scope was not clear so with discussions around the scope and sponsor leave at the beginning of August the end of planning milestone was moved to 24 August 2018 (piccl 1 refers).  The brief was submitted to WIS 31 August for final approval but was rejected as it was felt the plan was not sustainable as recruitment of SDES manager had been put on hold by senior management. Plus a review of the whole Digital Transformation portfolio had started and this had an extreme knock on effect on the resource challenges in DTI budgets within the UX team.  We could not get the brief signed off at that point 2018 (piccl 2 refers). There was also the question of whether funding for the project would continue. Following the DT review an agreement on how to proceed was agreed and the standards service manager role submitted to PRAF. A revised End of Planning date was agreed as 16 Nov 2018 2018 (piccl 3 refers).
  2. However the scope of the project became unclear after the DT review. Project sponsor wanted a large change to the scope which changed the remit of the project so many discussions were held to decide how to take the project forward. A workshop took place at the end of January to inform the scope and the end of planning was delayed to 22 Feb 2019 2018 (piccl 5 refers).   This milestone was missed due to outstanding actions on completing the brief (piccl 6 refers) and was delayed to 22 Mar 2019.
  3. When end of planning was finally signed off on 7 Mar 2019 Bill Lee and Defeng Ma were brought on board as stakeholders to input from a development perspective and suggested the current timescales were not achievable. PM revised the timescales and we waited for approval taking us into 19/20.
  4. However funding became restricted for 2019/20. Programme Manager had discussions with Finance in July to try and extend the timescales of the project but funding for Digital Transformation was confirmed by Finance on 22 July 19 as not available to take into the next financial year (20/21). Project therefore has to close by 31 July 19 without Bill and Defengs input and other activities not being completed. Instead programme manager is looking to create as part of the programme budget for next financial year a pocket for proposals to continue some of the work not completed within this project. See Proposals for 2019/20 section for a summary of these proposals.

 

 

Key Learning Points

  • It is too late to start to involve UX once a project has been initiated. Insight work and user research work needs to be completed before a project is even thought of in order to know what user issues are facing the university and it is this insight that should determine where budgets could be best allocated to resolve those issues. This means that senior management and budget holders need to be aware of the need for UX involvement before planning round proposals take place and understand the need for further UX involvement in project processes by way of design thinking processes and invoking Student Digital Experience Standards.

    • Recommendation is that UX discovery is carried out within the business and the insights from this research feed into planning round proposals. Project budgets should then be able to take UX involvement into account in project processes.
    • This is a big culture change for senior management and may require changes to their funding principles. 
  • Student Digital Experience Standards:  There is still an issue around the self-service aspect – UX team state that a lack of understanding on design processes means that it is difficult to produce effective self-service assessments. They also think that other places do not successfully don’t work fully self-serviced. From a UX user group presentation of 23 Jul 2019 they state that UoE are not ready for a wholly self-service model.

UX team feel that the current standards are meant to be used with a guide as current levels of UX knowledge is poor. However there is another Issue over UX resources being able to feed into project processes. The current proposal in WAC is to have the UX team to feed into WAC/LTW only so how do projects get the support required?

    • Recommendation: WAC continue to develop the self-assessment guidelines and determine how to provide expertise throughout the university so that we do not end up with siloed  knowledge again
    • Recommendation: train project managers in UX practices within project services so they understand how to utilise UX working practices in project work and determine how to include checklists for Student Digital Experience Standards into project processes and understand whether they have been appropriately assessed 
    • Recommendation: train business analysts in UX practices within project services so they understand how to utilise UX working practices in project work and determine whether Student Digital Experience Standards have been appropriately assessed. A toolkit was created for analysis in BI/MI so could this be adapted for UX?  
    • Recommendation: train development staff in UX practices within project services so they understand how to utilise UX working practices in project work and determine whether Student Digital Experience Standards have been appropriately assessed in development phases 
    • Recommendation:  Understanding of UX throughout university still required
  • From the same UX user group presentation of 19 Jul 2019 it had been suggested that the people they interviewed did not have a full understanding of the project processes and suggested that project activities are based around the downstream activities starting at development - this was questioned as to did they interview the right people? Also does it show that they have not entirely grasped how projects work?
    • Recommendation: more project training to UX and business level  members on what projects are and what involved in running a project

Proposals for 2019/20

  • The digital approach was not fully completed or tested. A project management methodology prototype encapsulating the human centred approach was drawn up but needs to be trialled. We were unable to get projects started with trialling (Estates and DT were interested) due to the extreme delays in the project. As feedback is needed to grow the initial prototype the first proposal is to pick a couple of projects and trial them with the project methodology prototype giving feedback to the DT programme and ultimately Project Services how the digital lifecycle process can be evolved feeding back into the Student Digital Experience Standards.

    • Example projects (e.g. a new project and an upgrade) should be made available on the projects website sample section- depicting what would be happening a different stages
  • WAC to map the gaps in the Student Digital Experience Standards and decide a plan for who will update them and when
  • WAC to look at Student Digital Experience Standards to provide more qualitative measures. They have implied that this will need expert input but Alex Carter has helped by discussing how checklists work in the project lifecycle
  • WAC to replace the existing UX website with a new repository for Student Digital Experience Standards within the EdGel proposed repository
  • Get a development perspective on the Standards and how they would work from Bill Lee and Defeng Ma

Next steps

Mark and Stefan to have conversation on the management and engagement.

 

Project Info

Project
Embed and Optimise UX Services
Code
DTI045
Programme
Digital Transformation - User Experience
Management Office
ISG PMO
Project Manager
Sue Woodger
Project Sponsor
Neil Allison
Current Stage
Close
Status
Closed
Project Classification
Transform
Start Date
22-Jun-2018
Planning Date
22-Mar-2019
Delivery Date
23-Aug-2019
Close Date
30-Aug-2019
Overall Priority
Normal
Category
Discretionary

Documentation

Close