Closure Report

Project Summary

Currently there is a gap in producing requirements for Business Intelligence deliverables - it is technical requirements rather than business requirements that are currently driving the endeavour. This impacts the existing BI/MI service, and will impact a proposed Enterprise Data Warehouse in the future.

This project provides a BI/MI Analysis Toolkit which we expect to consist of a set of templates, guidelines and any associated explanatory material that will describe how Business Analysis should be conducted in the Business Intelligence/ Management Information field and in future for the EDW. This will allow any future business driven project that wishes to adopt, or change, an element of BI/MI solution can do so with a full understanding of the business information required.

The BI/MI Analysis Toolkit will be used by Business Analysts in conjunction with end users (such as Management Information analysts, Management Information consumers, Business Managers, Users of operational systems etc.) and other stakeholders and the resulting analysis output will be used as input into the design phase, typically conducted by MI application designers and information architects.

Objectives & Deliverables

The deliverables were prioritised using the MoSCoW prioritisation method

M=Must Have; S=Should Have; C=Could Have; W=Want

No

Description

 

Delivered?

Reason for not delivering

Notes

O1

To create a BI/MI Analysis Toolkit by the end of July 2017

M

partial

 

Delivered, but not in original timescale.

O1D1

Definition on the contents and format of the BI/MI Analysis toolkit

M

Yes

 

 

O1D2

Creation of a BI/MI Analysis toolkit

M

Yes

 

V1 delivered by Simpsons.

V2 delivered by project team

01D3

Review of the BI/MI Analysis toolkit by appropriate stakeholders

M

Yes

 

 

O2

To deliver the templates into the BI/MI Analysis Toolkit which will form part of the overarching BI/MI Projects toolkit

M

Yes

 

 

O2D1

Decision on the format of the delivery

M

 Yes

 

Delivered in MS Word format

O2D2

BI/MI Analysis Toolkit delivered into overarching BI/MI Projects toolkit

S

 No

Taken forward into BAU

Comments on toolkit collated to be delivered under BAU

 

Success Criteria

Success Criteria as in Project Brief

Delivered

How delivered

The provision of a BI/MI Analysis Toolkit reviewed by UoE and signed off by both consultant and UoE

Yes

V1 signed off by UoE and consultant

V2 signed off by UoE – no input from consultant required

 

Benefits

Benefit as stated in project brief

Delivered

How delivered

The BI/MI Analysis Toolkit will enable future business driven BI/MI change projects to deliver best and more consistent results

Yes

Hard to quantify but the use of the BI/MI Analysis Toolkit should provide requirements based on business needs and not technical requirements leading to more meaningful reporting throughout the University

 

Analysis of Resource Usage:

Staff Usage Estimate: 36 days

Staff Usage Actual: 49 days

Staff Usage Variance: +36%

The project brief originally included the external consultants days(25d) . This was removed 20/8/17 for costs to be dealt with at programme level to leave estimate at 36d

External

Consultancy £25,550 (excl. VAT). Cost has been split 50/50 with Estates. Cost to project is therefore £12,750.

 

Explanation for variance

 

Cost

Project Brief cost

61d

 20/8/17 Consultants costs came out  to be dealt with at programme level (-25d)

Therefore original estimate is now 36d

Changes to costs

45d

10/1/18

Current budget exhausted. Project sponsor and programme manager have approved a revision of 45 days to complete.

Actual Cost

 49d

 

 

Time

Major Milestones

Project Brief date

Actual Date

Reason

Planning

26 Jun 17

23 Jun 17

 

BI/MI Analysis Toolkit created and reviewed

4 Aug 17

12 Jan 18

V1 from consultant  - See summary of delays section

 

Redrafting of toolkit as UoE toolkit

new

31 Jul 18

V2 in-house

 

Delivery of BI/MI Analysis Toolkit into overarching BI/MI Projects toolkit

15 Aug 17

31 Jul 18

 

Delivery Review

22 Aug 17

31 Jul 18

Signed off at same time as delivery into

Close

29 Aug 17

29 Aug 18

 

 

Summary of delays

  • The ITQ took longer than expected so the consultant was unable to start until 14 August.  This pushed milestones out by 9 weeks.
  • 7 September - Programme Manager contacted consultant over his plan - does not reference our Gantt chart and goes beyond the milestone dates that were set. Also highlighted that we were not receiving some form of weekly activity report as expected – we in fact only received three over the entire period that consultancy was involved.
  • Consultant was on-site 29/9 to run through his first cut of the toolkit and presented slides which programme and project managers fed back on (first review notes) and on 3/10 programme manager agreed the revised timescales for a 1st draft of MI/BI tool-kit with the consultant by COB Friday 6/10.
  • Consultants produced a first copy of the toolkit on 8 October but the links only took us to the original materials.  Following a call on 10 October the consultant provided a list of deliverables and new delivery dates: Forms by 10/10/2017; Draft Toolkit V1.0 “Over-Arching” Document 13/10/2017; Completed Toolkit V2.0 10/11/2017. Held a conference call with the consultant on 11 October to define and confirm understanding of what was to be in the toolkit
  • 16 October Programme Manager contacted CTO level at Simpsons Associates and informed that we were still seem to be a long way from achieving a successful outcome on this work.  He added that his core feeling is that the consultant understands what we want, but has difficulty in translating that into actual delivery and did not know whether he has too much other stuff on his plate or whether our experience is the “norm”.  Asked for CTO’S continued support in getting us to a successful outcome.
  • On 30 October the consultant confirmed that the delivery of work in progress was not achievable. Conference call on same day agreed dates of 1st draft Toolkit delivered to UoE by COB 10/ November and Final Toolkit sign-off by COB 24 November.  Programme Manager raised with Managing Director to make him aware as we were at a critical point in the delivery of the toolkit and Estates requirements. He said that we had to move the delivery dates back a few times, so were keen to ensure that we meet the latest dates agreed.
  • This version was not received until 22 November which we were informed was a final version – not a draft version. However it was poorly formed, did not link to UoE objectives, referred to other organisations (clearly cut and paste), repetition, irrelevant sections and overall extremely unsatisfactory (Link to full feedback). Programme Manager informed the consultant and their Managing Director on 27 November that a considerable amount of rectification would be required before we were prepared to sign it off and asked the MD for a recovery plan
  • A call on 1 December had MD confirming they had dropped the ball on this, and were committed to putting it right
  • 8 December consultants submitted a next version of the toolkit. Because reviewers felt that this latest version was a vast improvement on their previous two drafts we still did not have the confidence that a business analyst could pick this document up and carry out analysis in the BI/MI field. We fed our comments back to consultants and a new version (their version 004) was sent to us on 22 December.
  • However most people were on Christmas leave then so was picked up week beginning 8 January. The changes we requested were mostly made but the document still didn’t flow properly or explain the main techniques properly. The decision was made between project sponsor, programme manager and project manager to accept this as their final version and the project manager to create a new version of the toolkit with input from other project managers and business analysts. Delivery was now to be 23 March.
  • Toolkit was delayed due to PM taking on new work with higher priorities. PM then had family leave. As project should have closed previously PM was struggling to fit in with her higher priority projects. Closure moved to 27 June 18
  • Due to sponsor leave closure moved to 30 July 18

 

Outcome

Key Learning Points

Despite careful selection of our external consultancy partners, the expected benefits of adopting this approach did not materialise.    Although there were some useful outputs, the deliverables were considered by reviewers to be sub-standard, meaning that a fair degree of re-work was required in-house to bring them up-to standard.  The key learning points are:

a) not to use this particular consultancy in future

b) be as clear as possible, in terms of statement of work, what is required

c) manage the process closely through the use of timelines and deliverables (this would require concentrated UoE project management effort)

d) have realistic expectations on what the consultants can actually achieve

Outstanding Issues

  • Iain Fiddes is to discuss project with Alex Carter under BAU
  • The BI/MI Analysis Toolkit v2 has been reviewed by the following people and comments for its improvement have been collated and can be found on the DTI019 SharePoint site. These improvements will be implemented under BAU and it is expected that through use of the toolkit within projects further improvements can be made.
    • Geoff Cropley experienced BA – has knowledge of BI/MI
    • Nigel Hedley experienced BA – has knowledge of BI/MI
    • Elaine Wighton BA but completely new to BI/MI 
    • Defeng Ma, Rob Manley and Geir Granum (team leader and developers working in BI/MI)
  • BI/MI Analysis Toolkit v2 will go through IS governance review before being adopted

 

 

Project Info

Project
BI/MI Analysis Toolkit
Code
DTI019
Programme
Digital Transformation - Enterprise Data Warehouse
Management Office
ISG PMO
Project Manager
Sue Woodger
Project Sponsor
Andrew McFarlane
Current Stage
Close
Status
Closed
Project Classification
Grow
Start Date
01-May-2017
Planning Date
26-Jun-2018
Delivery Date
25-Jul-2018
Close Date
31-Aug-2018
Overall Priority
Normal
Category
Discretionary