Closure Report
For guidance on how to complete the closure report template, please click here.
For staff in Information Services Group (ISG), please click here for guidance on how to complete the Closure Report following ISG guidelines.
** DRAFT**
Background
The Timetabling service is delivered via a number of applications and interfaces, some of which are delivered by our third party supplier (Scientia Ltd). Scientia release upgrades to the core applications each year. The current service is based on Enterprise Foundation (EF) v3.12.
This project has been initiated to carry out essential upgrades and changes to the Timetabling hardware and software stack, so that the service can continue to be supported during 18/19.
One of the main drivers was a bug fix in EF which was delivered in EF v3.13 for a write back locking issue.
Rather than upgrade in situ, this project will repeat the upgrade plan and approach used in TTU010. A suite of parallel environments will be built which will replace the existing v3.12 DEV, TEST, LIVE and MOD (the new modelling environment created by TTU013 in 2017) environments. The opportunity will be taken when building the parallel environments to upgrade to the latest supported versions as defined in the Scientia hardware and software requirements. Any end of life components identified in USG008 will also be upgraded.
EF v3.14 was released in April 2018 and it is expected that this is the version which will be upgraded to from v3.12. The preference is to upgrade to the latest version possible. However, there is the alternative option to upgrade to v3.13 which still contains the write back locking fix.
Scope
The scope will be limited to those activities identified as required to upgrade EF to v3.14 on the latest supported versions of hardware and software possible. The following will be included in this:
- Upgrade DEV, TEST, LIVE and MOD environments to Scientia Enterprise Timetabler to ET v3.14
- Upgrade to any newer versions possible from review of latest supported versions
- Upgrade Puppet automation tools to new version that is supported by ITI
- Check that the post login plugin integration - UoE modified version of C# code as provided by Scientia as part of original implementation project is still supportable and on supported platform
- UAT and regression testing of all incoming and outgoing data feeds and BI Suite reporting and Calsync to Office 365 calendar
Not in the scope of this project:
- 19/20 roll forward will be initiated in the same time period. Early SDB and Web Apps work will need to be closely aligned or brought in to this project depending on progress.
- Any change to the disaster recovery or resilience solutions or service levels for the Timetabling service.
- Disaster recovery testing
- Changes to the Timetabling service as part of support and which have not been logged with the project and applied over the period of the project will not be delivered by this project.
Project Summary
The project delivered the Upgrade to Scientia Enterprise EF v3.14 on 12-Nov 2019
The project took extra time to deliver as there were issues getting the legacy Scientia software to run on newer operating systems Windows 2016, some servers had to be kept at Windows 2012
The project was delayed by 12 months while these issues were worked through.
The original planned go-live was Oct 2018, then this moved to a target go-live of Feb 2019 - when this wasn't achieved , the project was re-scheduled to deploy to LIVE in Oct / Nov 2019.
The LIVE service was successfully upgraded in Nov 2019.
The new version EFv3.14 should reduce the impact of the system locking seen under the previous version of the Scientia software EF v3.12
Objectives and Deliverables
|
No |
Description |
Teams | Priority | Notes | Achieved | Comments | ||
| Stage 1 - Analysis / Design | ||||||||
|
O1 |
Ensure that the upgrade is fit for purpose, checking that changes to applications or infrastructure including downstream systems are not negatively impacted | Achieved | ||||||
| D1 | Review the Scientia hardware and software requirements and amend the existing build document as required | IS Apps | M | Identify all latest supported versions | Achieved | |||
|
D2 |
Review the release notes for v3.14 and amend the existing build document as required |
IS APPS | M | Achieved | ||||
| D3 | Agreed method of change control to operate for all environments during duration of this project | IS APPS/TTU | M | Aim is to restrict configuration changes in 3.12 environments to essential change only - so that these can be recorded and applied to 3.14 | Achieved | |||
|
D4 |
DEV build |
IS APPS | M | Include any upgrades to latest supported versions | Achieved | |||
| D5 | Puppet upgrade | IS APPS | M | Achieved | ||||
| D6 | Working version of the post login plugin integration | IS APPS | M | Check UoE modified version of C# code still works and is still supportable | Achieved | |||
| D7 | On DEV, carry out functional testing of the release and the critical fix for the system lock issue that has affected the LIVE service | TTU | M | Achieved | ||||
| D8 | Make a recommendation whether or not to proceed or not with 3.14 in the TEST, LIVE and Modelling environments | IS APPS/TTU | M |
If there are issues with v3.14 which are deemed too high risk, v3.13 can alternatively be considered. If the lock issue isn't resolved and there are no other perceived benefits from the release, a decision can be taken to withdraw the upgrade at this time. |
Achieved | MOD environment left at 3.12 as discussed with TTU | ||
| Stage 2 - Build / Integration / Acceptance | ||||||||
| O2 | Deliver the 3.14 software into each of the new environments ensuring that a consistent set of environments is left at the end of the project | Achieved | Some servers had to be kept in the Windows 2012 environment | |||||
| D9 | Build the remaining three parallel environments, with Scientia 3.14 software installed (TEST, LIVE and MOD) | IS APPS | M | Achieved | MOD environment left at 3.12 as discussed with TTU | |||
| D10 | Carry out functional testing on TEST and MOD | IS APPS/TTU | M | Achieved | MOD environment left at 3.12 as discussed with TTU | |||
| D11 | Prepare test plan and carry out full UAT on new LIVE environment | TTU | M | Signoff to include interfaces | Achieved | |||
| D12 | Agree load test scenarios and carry out load testing | IS APPS/TTU | M | Not done | Agreed to not carry out load testing due to time/budget, and difficulty to replicate realistic live load | |||
| Stage 3 - Delivery | ||||||||
| O3 | Switchover from current LIVE, TEST, DEV and TRN environments to new 3.14 environments | Achieved | Achieved with a Delay of 12 months | |||||
| D13 | Transfer data from current LIVE SDB into new 3.14 LIVE environment and switch production service to new 3.14 LIVE. | IS APPS | M | Achieved | Done 11 & 12-Nov 2019 | |||
| D14 | Transfer data from current LIVE SDB into new 3.14 TEST environment and switch test service to new 3.14 TEST. | IS APPS | M | Refresh test 3.14 environment with copy of data in Live | Not done | Was not essential to complete UAT - the original data from TEST 3.12 was used ; propose this task to refresh data in TEST 3.14 is picked up by TTU020 project in February 2020 | ||
| D15 | Transfer data from current LIVE SDB into new 3.14 DEV environment and switch development service to new 3.14 DEV. | IS APPS | M | Refresh dev 3.14 environment with copy of data in Live | Not done |
|
||
| D16 | Transfer data from current LIVE SDB into new 3.14 MOD environment and switch modelling service to new 3.14 MOD. | IS APPS | M | Not done | MOD environment left at 3.12 as discussed with TTU | |||
| O4 | Support users through the upgrade to v3.14 | Achieved | ||||||
| D17 |
Prepare training material / communications as appropriate for any new features |
TTU | M | Achieved | Done by TTU | |||
| D18 | Provide support over the period of deployment | IS APPS | M | Achieved | Done by IS Production & TTU017 project team | |||
| Stage 4 - Decommissioning | ||||||||
| O5 | Remove legacy servers | Not done | Legacy servers have been kept for investigation reasons. Will be done in consultation with IS Production | |||||
| D19 | Decommission 3.12 LIVE, TEST, DEV and MOD environments | IS APPS | M | At appropriate point after LIVE switchover | MOD 3.12 will not be decommissioned as still in use |
Analysis of Resource Usage:
IS Staff Usage Estimate:180 days
IS Staff Usage Actual: 244 days
IS Staff Usage Variance: 64 days (35%)
Other Resource Estimate: £xxx - Not applicable - Scientia support costs covered by the Scientia Annual Maintenance
Other Resource Actual: £xxx - Not applicable
Other Resource Variance: xx% - Not applicable
Explanation for Variance
Much Dev tech time was spent attempting to get the legacy Scientia components to run on Windows 2016 in TEST with the new ITI-supported versions of Shibboleth and Puppet.
Target Go-Live date was pushed back from Oct 2018 to Feb 2019 to Oct / Nov 2019
To meet the go-live date of Nov 2019 , the decision was taken by Project team , Project sponsor and IS Apps senior supplier to roll back some servers to Windows 2012 and older versions of Shibboleth and Puppet.
Outcome
The LIVE service was upgraded to EF v3.14 on 12-Nov 2019
Database server was upgraded to Windows 2016 and SQL server 2016
Portal server was kept at Windows 2012
Webserver was kept at Windows 2012.
The Webserver was also kept to a single VM server, rather than attempting to horizontally scale the Scientia web applications across 2 web servers.
Splitting the Scientia web apps across multiple servers created some difficulties with the Puppet automation scripts
Key Learning Points
Update the legacy Scientia Enterprise platform every 2 years.
Before TTU017 , the last Scientia upgrade to be applied to the LIVE service was in 2015 ( TTU010 , 4 years ago) - this was too long a gap and made the TTU017 upgrade tasks larger and more difficult.
Outstanding Issues
Database performance issue in LIVE and impact on availability of the Course Timetable Browser and the BI Suite Class list reports. Issue expected to be resolved after remedial work done w/c 16-Dec and w/c 6-Jan with addition of new index on the RDB. IS Production will continue to monitor the live service through the start of Semester 2 , w/c 13-Jan 2020