Closure Review

Project Summary:

Objectives

The following objectives were identified at the planning stage of the project:

1. Bring production version of Enterprise applications up to v3.10, specifically with a view to resolving known issues in existing versions.
2. Improve client application performance by making SQL Server Express desktop application available for use on managed desktops with Enterprise v3.10.

The first of these was met, with the production versions upgraded as planned. However, while some issues were resolved, new issues emerged with a 'Function 9' error manifesting in the week after the upgrade was deployed into LIVE.

The second objective was not met after investigative work established that performance would not be improved by using the SQL Server Express client.

Deliverables

The following deliverables were identified at the planning stage of the project:

1. v3.10 Syllabus Plus is deployed in the Development, Test, Train and Live environments.
2. The latest versions of the Timetabling applications are deployed in Development, Test, Train and Live environments
3. The Scientia Database (SDB) is upgraded to v3.10 in Dev, Test, Train and Live
4. The results of re-running existing load tests on the Web Applications to ensure they are performant with this version of Syllabus Plus.
5. Mechanism to deploy the SQL Server Express application to managed user desktops for use with Enterprise.

Four of the five deliverables were delivered as part of the project, with number 5 not being implemented after the decision to not deploy SQL Server Express. It should also be added that number 1 was partially delivered, with the upgrade of the secondary LIVE server (at KB) not being completed at this time.

Analysis of Resource Usage:

Staff Usage Estimate: 96 days

Staff Usage Actual: 149 days

Staff Usage Variance: 55%

Other Resource Estimate: 0 days

Other Resource Actual: 0 days

Other Resource Variance: 0%

Explanation for variance:

Extra effort required

This project was estimated, at the planning stage, to require a total of 96 days effort, but has ended up needing a total of 149 days; an increase of 53 days.

There was three requests to increase the effort for the project, with the first in April 2014, when it became obvious that the project could not be completed in the original budget because of the reported issues with applying the upgrade and the LDAP plug-in. This problem meant that each instance had to be rebuilt, rather than have the upgrade simply applied to the existing deployment. This first request was for an additional 29 days, with 28 days for technical tasks and 1 extra day for project management. The second request for extra effort was in late May, and this was for an additional 20 days to cover extra work on the build and deployment of the LIVE server, investigative and support work for the Function 9 errors that came to light after the deployment to LIVE, and time for project management and QA. The final request for extra days was in July, and was for another 4 days to cover PM time and more work done on the secondary LIVE server.

The additional 53 days can be broken down across the teams in IS Apps as follows:

Development Technology - 45 days (to carry out the additional build work on each server)

Project Services - 4 days (because of extended duration of project)

Production Management - 3 days (to investigate the Function 9 errors)

All teams (QA) - 1 day (because of extended duration of project)

Extended milestones

The deployment of the upgrade to LIVE was originally planned for the end of January 2014, but was not achieved until late April; a delay of three months.

The most obvious reason for this late delivery was the additional work created by the problem encountered when applying the upgrade to a server that also featured the Enterprise Portal LDAP Plug-in. Some time was spent analysing and attempting to resolve the problem before it was decided that the best way round the issue was to rebuild each server and apply the upgrade by creating each server from scratch. This obviously required the additional effort (as noted above) but also extended the duration of each task, necessitating that each milestone be moved out accordingly. For example, the deployment of v3.10 to TEST was finally achieved on 14th March, but had been scheduled for 9th December (again, a slippage of three months). While some of the delay is directly attributable to the additional work required to rebuild the servers, some of the delay is also down to a combination of some confusion round the concurrent Roll-Forward project, a change of project manager (in January), the availability of resources once delays had been encountered and the dependency, at times, of colleagues with experience of the applications and their upgrade from from previous projects.

There has been a further delay in signing off the deployment to LIVE, with this first taking place in May, not long after the deployment of v3.10 to LIVE. This was not accepted by WIS because of two outstanding actions - repointing the feed to LEARN, and upgrading the secondary LIVE server at KB. The former has been done, but the latter has not been achieved because it has not been possible to establish v3.10 on the rebuilt server. Various attempts have been made to work around this (adding again to the increased effort noted above) but have not ultimately been successful, and so the deployment sign off has finally been achieved in July.

Key Learning Points:

Several important lessons were learnt through this upgrade and were brought up for discussion during the deployment sign-off in May. These included the following points:

• Should we carry out a clean install each time we upgrade a server? There was agreement that this would be the best option for future upgrades as it is a cleaner and simpler way of doing the upgrade and gets round any 'random' issues that may arise. Such an approach should be planned beforehand so that it is known that this is the way things will be done and estimates and plans will take this into consideration.
• Downtime for an upgrade is too long from the user perspective. Deploying the upgrade to LIVE meant that some aspects of the timetabling systems were not available to users for up to two working days. However much notice is given, this still creates a backlog of work for timetablers in the Schools and Colleges, and for staff in the Timetabling Unit. It would be beneficial if the necessary downtime was planned for a weekend, if possible.
• The overlap between the Roll-Forward and Upgrade projects caused some confusion as there was overlap between the required availability of TEST environments and resources to work on these. The former project should ideally be completed in November-December of one year, with work on the the upgrade starting in earnest early in the following calendar year.

Outstanding issues:

The outstanding issue is the upgrade of the secondary LIVE server at KB, which has not been delivered under this project. It has been decided to address this using Veeam replication software and this will be done in early 14-15 under a new Timetabling project (TTU007).