Overview

Background

This is the third year of work in implementing and embedding Practique's use in the College. This is viewed as a year of stabilisation, extending the practitioners/disciplines using the software and enabling the college to work towards the benefits of working from one consolidated platform. Overall this project is expected to:

• have a positive impact on Student Experience given the feedback from students already taking some exams in Practique
• provide a recurring benefit of reduction in administration time once the solutions are live
• provide the College with more of its exams in one consolidated platform

This project is building on the work already delivered over the last two years in:

• MVM107 Procurement of online application
• MVM118 Deployment of FRY-IT Practique for MBChB
• MVM119 Data Creation and Migration into Practique for MBChB
• MVM129 Support for OSCEs in Practique for MBChB

Scope

This project will implement 3 new aspects of Practique:

• MBChB Admissions
• OSCEs for Vets
• Written Exams for BMTO

Objectives and Deliverables

Benefits

Supplier anticipated Impact and Benefits are:

Our end-to-end e-assessment management service means you can have all your assessments managed from one integrated platform including Admissions/Structured Interviews, OSCEs, MCQs, EMQs, SAQs and VSAQs. Some key benefits are:   

For the student 

• Give your students feedback more quickly and when it matters with  automatically generated feedback from exams
• Improve the outcomes of your students by having better quality feedback  that is automatically linked to their assessments

For the faculty and leadership team

• Gain an overview of the assessment and student performance to drive  better learning
• Improve the quality and reliability of your assessments with inbuilt analytics  and reporting
• Access to new and innovative assessment types to ensure that you can trial  and introduce new assessment types as the medical assessment landscape  continues to change
• Remote access for all users allowing multi-site involvement, secure exam  bank management and remote question authoring
• Demonstrate to the GMC that assessments meet gold standard practice

For the administration

• Reduce the need for administrative tasks and increase time spent on  activities that add value
• Remote access for all users allowing multi-site involvement, secure exam  bank management and remote question authoring
• Immediate results processing - Save hours of time with electronic results  capture and inbuilt analytics. Our customers have reported saving between  50 and 80 person hours per OSCE exam using Practique
• Benefit from the extensive experience of the Fry e-assessment team who  have been supporting thousands of candidates take OSCEs (over 50% of UK  medical graduates have sat a summative OSCE on Practique)

Benefits anticipated by Sponsor and Senior User for Admissions:

The introduction of a new admissions process, with the incorporation of face to face assessments, carries a risk in terms of credibility of Edinburgh medical school, and the ability to deliver the process.

The use of e-assessments is likely to confer significant benefits:

• The lack of need for multiple paper copies of the assessment process improves security
• The software is intuitive to use, and will help assessors focus on the assessment process itself, rather than concerns about the technology
• It will allow rapid uptake of outcomes, and will speed up the process, allowing us to inform applicants of our decision earlier. We hope that this will increase conversion rates
• It will support assessors, giving them a positive experience, making them more likely to volunteer in the future
• It will allow timely and uniform feedback to candidates, to a level agreed prior to commencement of the assessments.
• It will liberate the admissions team from having to upload the scores from paper copies
• It will reduce the risk of transcription errors.

Benefits anticipated by Senior User for Vets:

The R(D)SVS is aware of the many benefits experienced by MBChB colleagues having attended and discussed extensively prior to the project, and therefore anticipates:

• Significant time saving for the assessment team
• Preparation activity – thousands (estimated 40k) of printed mark sheets previously used no longer need to be printed and stored.
• Exam day activity - reduction in collecting papers and organising examiners
• Post activity – will be automated therefore expect similar benefits to MBChB team including less stressful organisation, examiner experience benefits etc.
• Due to immediate availability of data and reports after OSCEs take place, School will be able to spend more time reviewing the results before sending to students

Benefits anticipated by Senior User for BMTO:

• For 2018/19 there are approximately 4800 UG student course enrolments within BMS.  A new system would reduce risk and allow a more robust way of managing our current multiple choice question assessment and examination processes.
• There would be a time saving for both academic and professional services staff when setting and approving assessments as well as a reduction in time for returning feedback for some assessment types.
• Exam paper creation using data gathered within the system.  Currently not possible other than by manual keying in Word question banks – not possible at present as there is no resource within the BMTO to undertake this time consuming task. 
•  Exam paper approval process within one system removing the use of email and password protected files. 
•  Remove risk of correct answer tags being left in exam papers.
•  A new online system would allow for standard setting which is not currently possible.
•  Delivery of timely, detailed and bespoke feedback to students on their performance in MCQs/EMQs.
•  Potential to use one system to deliver online tests rather than using multiple different systems.
•  Potential to move to online delivery of more exams/assessments rather than paper based.
•  BMS UG students are a potential growth area within CMVM.  This investment would facilitate future growth.

Success Criteria

• Admissions: Would use again - Successful use of Practique for MBChB's Admissions Assessment days. Given this is new for the Medical School, it is expected that there will be opportunities to learn for following years - what went well and should be maintained, what could have gone better and should be changed for next year.
• Vets: Would use again - Successful use of Practique for OSCEs. Given this is new software for Vets, it is anticipated that the School will wish to expand its use
• BMTO: Would use again - Successful production of August written exam using Practique. It is anticipated that if successful, the School will wish to expand its use.

Project Milestones