Closure Review

Project Summary:

Project Review

The Office of Lifelong Learning (OLL) replacement administration platform, CESAR (Continuing Education System for Administration and Registration) has been a very successful agile project.  CESAR went live on 19 January 2015 and has been well received by the OLL administration and academic teams.

A word from our sponsors -

"The CESAR project team were thorough in considering OLL’s business needs in order to best replace an existing and somewhat out-of-date system. Where OLL processes could be simplified through improved functionality, the team were proactive in providing solutions and the process to prioritise and discuss work at the beginning of each iteration allowed for a certain amount of fluidity in both our requirements and the results.

Communication from the project team was exceptional, they were responsive  to any potential issues and  maintained an open forum throughout each iteration, allowing the dialogue to continue, ensuring that the system was constantly being developed in line with OLL’s needs.

I’m certain I speak for both Bob and myself when I say the project was a pleasure to work on and the resulting system has exceeded our expectations. CESAR provides our staff with flexibility, easier access to information and the possibility to simplify and improve internal OLL processes."

Jemma Wallace

It is important that the investment both financially and in effort from OLL is recognised as crucial to ensuring that the new platform was delivered successfully.  OLL have made excellent business partners both through engagement with the agile methodology, freeing up two members of staff to become part of the project team and taking confidence in our early iterations, demonstrated by the two increases in funding beyond the agreed Terms of Reference estimate.

Objectives and Deliverables

Scope

The Minimum Viable Product (MVP) was attained in November 2014, further iterations have developed more features beyond what was required to replace OLLIE (the existing administration platform that has served OLL well for 10 years but is definitely in need of retirement along with the unsupported legacy hardware on which it runs).

Schedule

The schedule for this project has changed in line with the increases in estimate and realisation of the extent of features required to build a replacement to OLLIE.  Following the major replanning of the project to move to Agile and the end of foundation the project has run to time.  Originally the go live was penciled in for November 2014, however, upon approaching that time the decision was made between OLL and the Project Team to push the go live into February 2015 in order to avoid a launch at a busy period for OLL staff and to allow more features to be created before going live. Cesar sucessfully went live in February, and this was subsequently followed by putting the BI Universe reporting live. Some bug fixes were deployed post go live.

Analysis of Resource Usage:

Staff Usage Estimate: 500 days

Staff Usage Actual: 824 days

Staff Usage Variance: 65%

Other Resource Estimate: 1 days

Other Resource Actual: 1 days

Other Resource Variance: 0%

Explanation for variance:

The scope of the project went through 3 major transitions over the course of the project.

The original proposal estimated approximately 200 days, based on an existing analysis of the business processes supported by OLLIE.   OLL and the project team decided to undertake a more extensive business analysis and allow for a more extensive rewrite to support OLL's current business practices, using an agile methodology.  This produced a planning estimate of 500 days.

During the requirements gathering it was determined that the whole OLL lifecycle needed to be mapped to ensure that the OLLIE platform could be replaced without impairing OLL's ability to run their continuing education business.  This did result in a large investment of time for the foundation stage of the project roughly 100 days went into the foundation.  While at the time this was seen as a major spend the benefits of getting the analysis correct have meant a much improved system for OLL and the project has been able to run 18 iterations with 3 developers without having to stop.

Once the MVP had been established there was a re-estimate of the number of iterations to complete, OLL recognised the need to make MVP and funded an additional 160 days in May 2014 and in October 2014 OLL approached IS about running more iterations to deliver more feature and also act as a safety net for any changing coming out the go live, again funding an additional 4 iterations at 140 days.

Following go live, there have been some issues with the development of the BI universe for reporting, as well as training for OLL to use BI to create their custom reports. The complexity of getting the BI right coupled with the training requirements to allow OLL to build their own reports have subsequently pushed the project days out further, requiring an extra 18 days to be requested. It has subsequently been decided to start a new project to take the existing BI work to completion, allowing it to be scoped, developed and tested in a comprehensive and focused manor.

Data migration from OLLIE took a large amount of effort to bring it all across. Accuracy had to be maintained particularly with enrolment and payment information. The web application written was particularly useful for the business to use to import the data as it could be re-run with minimal effort.

A lot of refactoring also went into the OLLBook part of the project where we had to integrate CESAR and it's data with the customer facing side of OLLBook. This proved to be more time consuming than anticipated as OLLBook has no test coverage to ensure what was changed did not affect anything else. An API was written in CESAR for interaction between the two applications (as well as wpmserv) and OLLBook being customer facing for OLL, it was crucial to ensure same user experience post CESAR migration. OLLBook’s error handling was also improved.

Key Learning Points:

• Carrying out a lifecycle review and holding Agile user story workshops with the entire OLL team was successful and the project sponsor noted how given the chance to meet each other and reflect on what they did provided a fresh motivation for the department.  This was reflected in the 1,400 user stories that were collected during the workshops.  A key learning point here would be to review how agile workshops are run with a view to producing fewer and more focused user stories. 1,400 user stories takes a reasonable amount of effort to then filter down into the few hundred user stories that made up MVP and the raft of remaining stories that could be used to run future extension to CESAR. Whilst it’s clear there was value in running the workshops and consulting with a wide range of OLL staff, a more focused approach needed to be taken in the initial requirements gathering. Of 1400 stories collected, only approx. 170 stories were completed (including new stories added during the project).
• OLL dedicated two members of staff to become part of the project team, this was invaluable in getting user stories quickly prioritised and clarified.  Only needing to involve a small number of people in the prioritisation process has meant that less time has been expended on grooming the backlog and more time invested in development.  Getting this level of buy in from business partners up front could make Agile projects deliver more features by being able to dedicate more time to development than negotiation and clarifications.  Bob, Jemma and Amanda should be recognised for this dedication to investigating issues and returning them to the analysts and developers in very short timescales.
• Agile mentors are essential, Dawn Nicholls assisted in preparing the approach to the analysis workshops and gave initial direction to the team which was useful to get our heads into the Agile way and Bill Lee provided Agile and JIRA Agile training to the team so that everyone including the business partners had a common frame of reference and stuck to our methodology.
• Having a strong development lead with Agile experience is crucial.  During this project the development lead (John Allison) and the Agile mentor (Mike McMonagle) ensured that we were sticking to the methodology and also provided clear direction for the development, I believe Greg Carter may also have helped out in getting our automated deployment lines in order. Being able to pull on this experience has made the development run so much more smoothly.
• Consistency of staff throughout an Agile project is very important to maintain momentum, there were some shifts in project manager at the start of this project but once the core team was in place it proved invaluable to delivering a lot of user stories in the time available.  Chris Konczak, Thalia Nikolaidou, Sabrina Niziolek and John Allison are now extremely well versed on OLL processes and the CESAR development, if we didn't have that consistency throughout the project there would have been far more increases in budget down to integrating new team members and bringing people up to speed with a year's worth of conversations and learning as we go. Mention also has to be made to Peter Jackson from Dev Tech who helped throughout the project in setting up and managing the infrastructure and databases.
• The introduction of unit testing and test driven development had a very positive impact on the project. In particular:

1. It reduced the requirement on webdriver UI tests, which are time consuming to write and maintain, and slow to run.
2. It lead to a good, loosely coupled application architecture which was easy to scale up as the application grew more complex.
3. It allowed for aggressive and confident refactoring, when required.

Test Driven Development (TDD) is when the developer first writes a test case before any product development is undertaken. This test case will therefore fail as there is nothing to test against. The developer will then produce a minimum amount of code to develop the product functionality to get the test case to pass, then refactor the code as required.

• The quality and consistency of the core project team does mean that once the project is established that actual project management time can reduce.  Having the user stories defined well in advance and having the same project team with no conflicts for resource has made the project management less time consuming as reflected in the overall PM time of 89 days.
• Engaging Application Support and Technology Management, the proposed model for having both these teams in all meetings was found not to be necessary.  During the course of the project both teams were kept up to speed and invited to major iterations and preparations for go live, but didn't need to be included in all events.  This has meant that time allocated to these team was able to be repurposed to drive development.  It is important that these teams are included as ultimately they will support the software, however, on a long running Agile project it is possible to reduce the engagement to end of iteration reviews and handovers prior to going live, without compromising the ability for them to support the system.
• Under ISAPPS\SHARED a shared folder "IS-OLL" was created give IS and OLL staff a place to work on materials, such as word documents and spreadsheets.  This has been very effective as it has removed the need to send large documents and collaborative working on spreadsheets has been critical in whittling down the vast amount of information to the useable user stories that have driven the development.
• Development of the BI reporting suite was done as a side-line as there was resource / time / budget available at the time. There were no hard requirements set, only to do this on a best endeavours basis with a view that future changes would be for support. In hindsight this was the wrong thing to do, as we should have clearer definition of the deliverable, i.e. agreement to produce according to priority,  a set of reports within the agreed budget then stop when the budget runs out  or define what must be delivered. Otherwise scope creep and confusion result. The database design was complex in places, particularly in the financial model. This meant that developing the business objects universe was not trivial and the delivered product did not meet the requirement. These problems with BI development could have been avoided if the universe had been developed in-team, preferably concurrently with the application development. This didn’t happen because no team members had skills in BI. This would be a useful skill for more developers to have. Extra training was also required for the business in order for users to build the custom reports they required. The extent of the training involved was underestimated in the initial discussions.

Outstanding issues:

While no work is outstanding in an Agile project as the scope constantly managed there are several materials that should be retained for use in future CESAR developments these are:

• The JIRA backlog, these issues should be retained for use in any upcoming development or suitable archived for re-use
• The OLL_SHARED folders should be retained or archived, the contents of which again could be used for future development work or even to help in producing proposal for further extensions to CESAR
• Further BI development is required and this will be tackled in another project, agreed to be started as soon as possible -
  • Requirements for this new project need to be defined and subsequently estimated.
  • This, and the following bullet point would be a sponsor funded project.
• Additional development requested for CESAR needs to be quoted for, and integrated into this new project as a separate phase away from the reporting. The Jiras for this are:

HSS001-286

HSS001-291

HSS001-292

Only one of them needs to be actioned, which one will depend on the development time quoted.