Completion Report

Project Summary:

Success criteria

• Fully assess the impact of moving to  3.12  the timetabling service and make a clear recommendation about whether or not to proceed with the switch to 3.12

Stage 2

Analysis of Resource Usage:

Staff Usage Estimate: 141 days

Staff Usage Actual: 228 days

Staff Usage Variance: 62%

Other Resource Estimate: 1 days

Other Resource Actual: 1 days

Other Resource Variance: 0%

Explanation for variance:

Variance from original Project delivery dates

The original delivery milestone was 26 October and the original closure milestone was 9 November. The delivery began on 30 November (completed on 14 December) and closure is planned for 27 January.  

Variance from original project budget

The project team delivered the project in (est) 218 days rather than in the original estimate of 141 days - an overspend of 79 days

There were several reasons for the overspend:

1. Additional effort in Stage 1 (investigation of 3.12 and recommendation to proceed): 5 days
2. Additional effort to deliver 3.12 DEV, TEST and LIVE environments:20 days
   1. The planned "team build" of TEST, LIVE and TRN was disrupted by assigned technical architects being conflicted for by other project work leading to a longer duration than planned for the delivery of these 3.12 environments
   2. The quality assurance expected to derive from the planned "team build" and planned daily progress standups was not achieved as the technical architects were no able to work together as planned
3. Additional effort to deliver automated webapps:12 days​
   1. The duration of this task took four weeks longer than expected, which had a knock-on effect on the date when full user-acceptance testing could begin
4. Additional effort due to handover of webapps to production:3 days
   1. ​The project team did not identify any requirement for additional handover until late in the project
5. Additional effort in TEST due to image build issues:4 days
   1. ​Scientia instructed the project team to make configuration changes allowing Acceptance to be achieved  but these proved to be ineffective in LIVE
6. Additional effort in LIVE due to image build issues: 9 days
   1. ​The effort required to get a workaround for the image build issue in LIVE meant that the deployment did not complete in 3 days as expected but took more than 10 days, including one day's toil time awarded to the technical architect
   2. The workaround requires additional handover tasks for production
7. Additional effort required for post-deployment fixes in LIVE:16 days
   1. ​The unexpectedly extended deployment required handover from  one technical architect to another, leading to some confusion as to the state of which deployment steps had been completed successfully.
   2. Some deployment steps ( eg PADs) were not fully covered in the build instructions
8. Additional effort due to increased duration of project
   1. (project management, reporting): 10 days
   2. resourcing - 2 days

Planned effort not used to complete delivery of TRN(descoped): 3 days

Variance from original project scope

Add to scope

The project team  delivered the first stage of the annual roll-forward (SDB)  as planned and  in addition delivered the automation of the second stage of the roll-forward  (webapps).

Remove from scope

The project team did not deliver a 3.12 TRN environment. This was descoped largely due to the lack of remaining budget. The Timetabling  team will use TEST for training.

Key Learning Points:

1. Key technical resources taken off project at build stage disrupted plans to carry out team build of TEST, LIVE and TRN environments. (In the end, TRN was never completed) if this  approach is to be tried again the key resources must be protected.
2. Having built more than one 3.12 environment, we were able to show that image builds were an issue in more than one environment and that this was not an artifact of one environment having been built wrongly.
3. It took repeated attempts by the team sponsor and senior supplier to finally get a workaround for the (known) image build issues from Scientia. The workaround already existed and more than one other site had the same issues as we did, but the project team was not offered this as a solution until three months after reporting the issue to Scientia. The senior supplier is raising this with Scientia.
4. Deploying so close to Christmas meant that  as the deployment did not go well and took more than 10 days instead of 3 complete, the completion of the deployment was interrupted as key resources went on annual leave and this led to issues being missed.
5. Building on new 3.12 servers while old 3.10 servers ( and services) were available may have masked some incomplete deployments ( eg SSRS reports)
6. It isn’t actually enough to just compare 3.10 to 3.12.  The business needs to define what it sees as an acceptable time to complete the "student allocation whole sheet" test and measure the results for any upgrade against that criteria.
7. In the future check ECA (or any other site outwith edlan) can access Enterprise to rule out firewall issues.
8. Deployment plan incomplete - PADS not deployed to LIVE, scripts not deployed to LIVE, SSRS reports required repointing in LIVE, deletion group issues(I160203-0289), MyEd mywebtimetable not deployed with bamboo
9. We need to suceed better at ensuring changes made in production since the last projectrecords are understood and fed into the project. 
   1. recorded under https://www.wiki.ed.ac.uk/display/insite/Timetabling+List+of+Scheduled+Tasks
10. 16/17 OneLan and PADS were running in LIVE when they should have been shut off
11. UAT must be carried out using a variety of roles - such as admin and non-admin users.
12. The UAT plan did not identify a key bug which Scientia will not address until a later release. If we had identified this bug then the upgrade may have been cancelled.

Outstanding issues:

CMDB not populated

TAD not updated - this will be taken into scope of TTU012