Completion Report

Project Summary:

The procurement and implementation of Reading Lists Management System was completed successfully following migration of data in May 2017 and re-launch of Resource Lists Service on 12^th July 2017.

The main driver for this project was to define requirements and go to market to procure a new Reading List software for our existing service known as Resource_Lists@Edinburgh and implement the chosen software within limited time period with the aspiration to be ready for the start of the new academic year in September 2017.

This project has been a successful collaboration from the teams within IS Library and University Collections, User Services, Learning Teaching and Web, and Applications, Divisions, and our Project Board representatives from Colleges and EUSA that we have been able to procure and implement this system on time, in advance of incumbent supplier contract expiring in July 2017. The project manager would like to thank the project team and colleagues for all their efforts in this achievement, and would also like to thank our academic colleagues, EUSA, and student representatives for their input and support throughout the project.

Scope

The scope of the project to procure and implement a new Reading List software.

• The procurement phase was an 'open process', i.e. one stage, straight to Invitation to Tender. No ESPD (European Single Procurement Document) was undertaken, however it was incorporated into the ITT Documentation as selection criteria (PASS / FAIL requirements). This was agreed with the UoE Procurement Office following a market analysis exercise. We anticipated that approx. 4 suppliers would bid for the contract, in practice, only 2 suppliers submitted their tenders for this contract.
• The procurement phase awarded the contract to the most economically advantageous tender (this means that the highest scoring tender / supplier over both cost and quality was awarded the contract).
• The procurement phase defined the high level implementation plan, including any plans for implementation and agreed with the successful supplier.
• The outcome of the award, resulted in the project completing an implementation phase migrating data to the new software and re-launch of the Resource List Service

Governance arrangements for the procurement and subsequent implementation, included:

• Structure of the Project Board and Core Project Team.

Please note that the management of Digitised Content was deemed out of scope for this project.

Benefits

The procurement part of the project will deliver some intangible benefits, including:

• Engagement with key stakeholder groups throughout the University.
• Increase relationship building between IS Applications,  IS Library & University Collections and Learning, Teaching and Web
• Efficient procurement process to ensure the University selects the right product
• Cost benefit due to the increased market pressure on the incumbent supplier

Project benefits have been reviewed and documented in the following document Reading List Project Expected Benefits which details key benefits description, value, timescale for delivery and owner for realisation of the benefit.

Analysis of Resource Usage:

Staff Usage Estimate: 100 days

Staff Usage Actual: 92 days

Staff Usage Variance: (8)%

Other Resource Estimate: 0 days

Other Resource Actual: 0 days

Other Resource Variance: 0%

Explanation for variance:

The original IS Apps budget approved for the overall project was 100 days - this was split across Procurement and Implementation activities

• Procurement - 70 days
• Implementation & Integration - 30 days

   

The actual IS Apps effort on closure of the overall project is as follows:

• Procurement - 67 days
• Implementation & Integration - 25 days

For the purposes of project documentation and information advised to Project Board resource costs for L&UC and other IS Areas have been recorded these are as follows:

• Resource costs for IS L&UC and other Areas as at end of June 2017 = 426 days
• This provides an overall project costs of circa 520 days effort.
• Any remaining IS Apps resource days (8 days) will be transferred back to ISG Portfolio.

Procurement Element

Key Learning Points:

• Having all stakeholders represented on procurement group worked well. This also helped make sure all key staff had been invested and involved from the beginning of the procurement process through implementation, development of new workflows and to rollout.
• Having a representative from the VLE team on the project group and getting VLE staff involved in scoring requirements raised awareness across LTW and resulted in LLS staff being invited to user groups and information sessions to promote the Resource Lists service.
• Start work on requirements much earlier in the process, consider number of questions and requirements as time consuming to score.

• Be realistic about the amount of time required to evaluate/score the requirements and also the number of evaluators.

• Provide clear guidelines on scoring requirements in advance to ensure consistent approach (i.e. how we score functionality scheduled for release but currently unavailable) Make sure we all use and understand the scores and definitions used in Award – any model answers should correspond to these scores.

• This software product fell in scope of the new threshold contract value within the Procurement Scotland Regulations 2016, thus requiring a tender process.  As a result, the project team indicated their view that the procurement process was useful in bringing the team together to seek a solution with input and collaboration with other business areas of the University.  The process aided the team to think clearly about what was actually required in terms of a potential product solution.  It was beneficial to highlight to the team how systems and processes within the University integrated together.  The outcome of the process reiterated that the solution acquired is with a supplier that can offer a sustainable and innovative product.
• The clarification stage of the tendering process was met with many questions posed to each of the bidding suppliers to this opportunity.  This resulted in extensive procurement administrative resource time and additional time from the evaluation team to complete their scoring.  It was noted that both suppliers were complacent with their responses to many of the questions posed e.g. lack of clarity in responses; incomplete responses; vague wording.
• The project team indicated their view that there were too many questions within the standard IT technical questions sets.  It may have been beneficial to have provided the project team with some supporting information at the beginning of the process to help them to understand why Edinburgh University needed these in depth questions included as part of the ITT process.
• The procurement project pre-ITT stage was met with input from multiple procurement leads. The lack of continuity and communication caused some impact to the Project Team in terms of receiving information about the procurement processes, key milestones, and roles and responsibility requirements of the key stakeholders within the procurement phase.
• The University Terms and Conditions submitted with the ITT were not specific to a Software as a Service Solution (SaaS).  SaaS terms and conditions required lengthy discussion and additional time to undertake negotiations between the supplier and both legal teams to reach a satisfactory conclusion.
• The Evaluation Team indicated that the Award Software evaluation tool was too slow to operate.  It is not clear whether this was due to an internal IT issue or programme software issue at the time of the evaluation stage.

   

Recommendations

• Where possible, continuity of a lead procurement expert to be involved with a project throughout the entire project phase.
• Introduction of University of Edinburgh SaaS Terms & Conditions to be included as standard in all future ITT’s.  These have now been drawn up by University of Edinburgh’s Legal Department for inclusion in future projects.

• IS Applications are reviewing the overall procurement question sets and will be making updates based on feedback from recent procurement projects completed.

Outstanding issues: None

Implementation Element

Key Learning Points:

• Moodle Service Manager highlighted how enjoyable this whole project has been for them. The team has been excellent to work with, they felt there is more shared knowledge across the teams and they did find Basecamp a useful tool to work with to see the kinds of questions raised.
• Collaborating with Steph Hay in Learning, Teaching and Web on user documentation and communications for Learn/Moodle rollovers and upgrades worked well to present consistent information to users about the integration with Leganto and the VLEs.
• The project team were given access to the Leganto sandbox and implementation form before the contract was signed. In retrospect, we were right to insist on this. Information on how data was to be migrated was included in the implementation form. As signing was delayed, any delay to accessing the system would have meant the project would been unable to complete the implementation and prepare date for export within the timescales. It would have been extremely useful to have been aware of data requirements much earlier in the process.
• In retrospect the Business Lead would have liked to have explored alternative timescales, running both systems simultaneously and a gradual migration to Leganto.

• The Library team did not realise until implementation what impact Leganto would have on current Course reserve or on existing course data in Alma.

• The Library team underestimated the amount of time that would be taken up by manually checking and handing back lists to course organisers. It is always difficult to predict how long these things take, they could maybe be more cautious/pessimistic next time.

• On a related issue, they could have perhaps have changed the timings of some stages in the process. Promoting the service and training staff on the use of Leganto started while the majority of lists had not yet been published and handed back. This caused a little bit of confusion: some staff had been introduced to the new system but were unable to find their lists as they hadn’t been published. It might have been better to delay the launch of Leganto to give us more time to check the lists. Tight timescales made it difficult to schedule all necessary tasks at the best possible time taking into consideration data clean-up, deadlines and end of existing software contracts.

Outstanding issues :

• Managing to maintain on-going commitment from Project Board representatives remains a challenge on these types of projects due to duration and/or other work commitments – raise with Mark Ritchie, head of Project Services to incorporate some elements for discussion within Project Sponsor training
• Tasks to be taken forward by Library team
  • Leganto reading lists share space with existing course reserve lists. The Library team need to de-duplicate lists and work out how we manage both reserve and resource lists going forward.
  • Linking to multiple course codes- Ex Libris currently circulating requirements document to Leganto user group.
  • Issues emerging as academic staff start to use Leganto:  when list is opened in Alma and Leganto at the same time (different people or the same person) changes made on one override changes on the other. ‘Send to library’ not working for academic staff who want to send their updated lists for review. Both issues are logged on Sales force.
  • Plan for reading list rollover

Recommendation

• Automation or scheduling of course information required. Library Learning services are adding too much data manually therefore the library team are looking to have this developed in a future project. Requirements to be considered for Enterprise APIs project under Digital Transformation

Project feedback has been reviewed with supplier regarding the following areas:

• Excellent support via Basecamp
• Lack of documentation available for product
• Assumed previous knowledge and experience of Alma
• Onsite training
• Workflows - Additional onsite training focusing on Leganto/Alma workflows would be more useful.
• Analytics - There was no dedicated training on analytics
• User and usability testing

.