Overview
Background
The External Examiners Online Reporting System (project STU235 https://www.projects.ed.ac.uk/project/stu235) will be delivered as a pilot for ten schools to use from September to December 2014 PGT Exam Boards. This new IT Tool will allow the collection and management of external examiner reports. It will also allow performing thematic analysis on the responses.
This new separate project will be opened in 2014/15 to support the pilot and its evaluation, so that a list of prioritised enhancements to the IT Tool can be built as per available budget. The enhancements will be deployed to the whole university in June 2015. IS resources will be required to manage this post go live phase: support software over the PGT Exam Board period, support the evaluation pilot which will be managed by Academic Services, build/test/deploy the rework and manage the project.
Reasons fro Taking Project Forward
The introduction of the pilot was stakeholder driven - it was requested from stakeholders at the College Quality Assurance Committee, CHSS and was mentioned also in Information Sessions to launch the project. It was approved by the Project Board at its first meeting. This approach was welcomed by the School Directors of Quality at their annual briefing session.
Delivery of Phase I will be significantly compromised (and potentially fail) if resource is not allocated to the support of the Pilot, its evaluation and rework as per pilot schools’ feedback.
Scope
This project will:
- Support the IT tools during the Sept-Dec 2014 PG Exam Board Period.
- Evaluate the impact and validate the benefits of the IT Tools
- Enhance the IT tools as per users’ feedback. Workshops will be held with user group representatives to finalise and prioritise the requirements. It will be implemented across the whole University in June 2015.
Objectives and Deliverables
Reference | Details |
O1 | The External Examiner Reporting System (EERS), which has been delivered end July 2014, is used by 10 Pilot Schools to conduct the end to end process: · External Examiners can submit reports on line and in time, and view responses. · School Key Contacts can distribute and track the External Examiners Report. · University Academic Response Coordinators can respond to the External Examiner report and create a single response. · School and College Key Contacts and Academic Response Coordinators can view a summary of the External Examiner issues and the responses so that it can be viewed and analysed. · The University can demonstrate to External Stakeholders it is compliant by scrupulously using the External Examiners’ reports to extract strategic themes. |
D1
| Support the IT Tools during its use by the Pilot Schools. This will be done by the IS Apps Production team and the Development team. |
D2 | Agreed five user stories carried over from phase 1 will start ahead of the start of pilot. They are estimated at 4 days to complete. |
O2 | The IT Tool is evaluated by Pilot Schools |
D3 | Enhancements are proposed by Users from the Pilot Schools. A list is produced, so that it can be reviewed and prioritised by User Representatives, and approved by the Board. IS Apps will support Academic Services in the evaluation of the Pilot. |
O3 | The IT Tool is enhanced with the prioritised enhancements |
D4 | The prioritised enhancements to the IT Tools are analysed, estimated and built in, as per IS Apps budget constraints, using an Agile approach. |
Out of Scope
- The management of External Examiners (nominations and appointments) will be done in 2014/15 under a separate project.
- There will be no integration with the Finance system.
- There will not be a student view of information at this point.
Benefits
In addition to the main benefits highlighted during the initial project (https://www.projects.ed.ac.uk/project/stu235/brief-0) this pilot evaluation and enhancements project will have other benefits:
· Ten schools will evaluate whether the pilot is used effectively by External Examiners and staff, and assess its impact on the University.
· It will contribute to ensuring that stakeholders buy in to the IT tools that are developed. As such the pilot can be a great approach to convincing the late adopters and bringing them along as supporters.
· This approach will enable to engage with users at an early stage. Staff will be able to try the process and feedback to the project team; this will enable to rally support for the IT Tools and address any highlighted bugs before the roll out.
· The schools will be able to propose further meaningful enhancements to the IT Tools, which will then be implemented by IS Apps, as per budget constraints.
· This rework will maximise its benefits once rolled out to the whole University. It will reduce the risk of the External Examiners IT Tool not delivering all its benefits.
Success Criteria
Ten schools representing all 3 colleges will engage with the new process and adopt the system. As agreed by the Project Board, tor this project to be deemed a success at least one pilot school from each of the 3 colleges must engage with the new process and successfully adopt the system. This will ensure a consistent approach to handling external examiner reports is achieved across the 3 colleges.
Metrics and the measures of the IT Tools have been presented to the Project Board in June 2014. They are documented in the document stored in External Examiner Pilot & Evaluation success criteria