Completion Report

Project Summary:

Background

This project was to develop a new application, within or integrated with EUCLID, to support the College of Arts, Humanities & Social Sciences’ (CAHSS) annual report writing processes for quality review at a Programme, School and College level.

This followed a previous project which analysed the existing systems in place across the CAHSS schools and reported to the CAHSS QA Committee.

A Senate QA Committee paper set out the new templates to support the annual programme and school QA review processes within the University, as part of the new University Quality Framework (UQF), and the templates were used to help inform the development of this system.  It was intended that the system could be used beyond the life of this project by other Colleges as well as CAHSS.

Objectives

Deliverables

Analysis of Resource Usage:

Staff Usage Estimate: 200 days

Staff Usage Actual: 273 days

Staff Usage Variance: 36.5%

Other Resource Estimate: 0 days

Other Resource Actual: 0 days

Other Resource Variance: 0%

Explanation for variance:

The project budget was increased from 200 days (5 iterations Agile project) to 275 days (8 iterations project).  A saving was made between the 7th and 8th iteration where they were effectively combined.

3 additional development iterations were approved in order for the project to complete the development of the software.

The reasons for the shortfall were:

Foundations stage

• Extension of Foundations Stage to present options analysis and make decision on optimum development platform
• Resolving gaps in EUCLID data and multiple definitions of same types of data

Technical building blocks in Iterations 1 and 2

• Emerging requirements to fulfil EdGEL integration
• Underestimation of effort for Python/Django for first time technical configuration

Iterations 3, 4 and 5

• Changes in business requirements, e.g. additional permissions for school quality directors, enabling workflow for school specific additions to core templates
• Underestimation of the time required to establish precedents for Python/Django development relating to some aspects of application functionality
• Changes required to data from EUCLID arising from emerging business requirements, e.g. school level mismatch with Central Auth, addition of subject areas

Key Learning Points

Successes:

• First application to be developed on Python/Django platform, and the first to integrate with EdGEL

• The user stories workshop had a wide representation of schools and roles which ensured there was sufficient input and thinking put into the requirements stage.  Inger Seiferheld (Business School) commented "I think this was particularly useful as it helped us all understand what we wanted the system to do".  The sponsor had commented positively on getting such a diverse range of schools and roles involved in the workshop.
• Good project communications were maintained through attending the College QA Committee, which the School Directors of Quality attend.  Several demonstrations were delivered towards the end of the development to the College and colleagues in Student Systems.
• The agile meetings framework was effective, with the product owner able to attend all the key meetings either in person or via Skype.
• Daily stand-ups were held in the collaborative space in Argyle House, and worked well.
• The developers built a good understanding of the requirements early on in the project and worked very effectively with the CAHSS team throughout.  The system demonstrations were expertly delivered.
• Both the product owner and the system administrator took an active role in the development and testing of the system; testing was undertaken promptly and effectively.
• A user group was successfully engaged to test the system and provided useful feedback for the development.
• Early bookings secured for software developers meant the project was able to adhere to the hard deadlines required by CAHSS in line with the University's requirement for annual quality reports.
• New view developed in EUCLID to provide required programme data as read-only
• Established precedents as part of first time technical configuration for Python/Django application
• Alastair Duthie (system administrator, CAHSS) commented "Thanks again for all your work on the project, which has been a great success".

Issues:

• It took a long time to get the appropriate people together to provide appropriate feedback and suggest recommendations for selecting the optimum development platform.  The decision also took longer because it was influenced by the business requirements so there was some waiting time for this to develop the user stories in the Foundations Stage.  Normally, this decision would have been made in the Planning Stage but it was quite particular to this project that the business requirements/user stories were required first.
• When Python was selected as the development platform, the project budget should have been reviewed (and likely increased to make allowances for this being the first development of this type)
• The development encountered technical challenges with emerging requirements to fulfil an EdGEL integration, and a higher effort requirement than anticipated for Python/Django first time technical configuration.  This meant a slow start to completing user stories in the first two iterations which made it harder to calculate Agile velocity accurately.
• The Bamboo service (used to automate deployments and release software) experienced a serious technical issue and took over a month to be restored.  This had a big impact on the middle part of the development, with the first release pushed back and a delay in promoting new developments.  Whilst development work continued, we were unable to release the system for testing and this created a backlog.
• The user story describing the requirement to upload documents was assigned a 'should-have' priority, but the second (and anticipated final) system release was unable to include this.  When signing off the maintenance iteration, further discussion confirmed this was actually a 'must-have' requirement and the project had to schedule a third release to include this.  However, the remaining project budget was able to accommodate this and only a small period of downtime for users was required.
• Having the additional development iterations meant a conflict for development resource with another project, and time spent negotiating availability and budgets.

Recommendations for similar projects:

• If the Foundations stage takes more effort than planned, then it is likely that the effort required for the Development Iterations will also be higher than planned.  It would therefore be prudent to review the must-have and should-have User Stories to confirm that these are still likely to be completed within the remaining project budget.
• Consider whether to temporarily stop development if a key service like Bamboo goes down and requires to be restored.
• Clarify the prioritisation of user stories in terms of the minimum viable product containing only 'must-haves'.  Review prioritisation regularly in case priorities change.
• Future projects integrating with EdGEL will have access to a more developed EdGEL framework/consultancy

Outstanding issues:

• With reference to Objective 3 (extracting data for BI), the lack of progress on that separate project presents a continuing risk to the system itself.
• The CAHSS system administrator does not view the current support mailbox as a long-term solution and would like to review support arrangements through further discussion with IS Helpline, who had categorised SPQS as a local service and, as such, were not keen to provide first-line support without a strong business case.
• An exercise had to be undertaken by the system administrator, in conjunction with schools, to update EUCLID data for programme directors.  Following this, the schools should ensure that all recording in EUCLID is correct and updated when needed, and that archived programmes are removed via Student Systems.