Three courses offered by the School of Health and Social Care achieve high numbers of applications (Adult nursing, Mental Health Nursing and radiography). Candidates are invited to the University to undergo interviews and paper based diagnostic testing. These tests are marked by university staff. If a student fails the test then they are told at this stage that there application has not been successful.
The volume of applicants and subsequent marking of these paper tests consumes staff time and subsequent costs equates to the salary of a full time lecturer. Technology Enhanced Learning (TEL) having experience and knowledge in Computer Aided Assessment was asked if they could assist in part of reducing the overhead of the process. The idea being that candidates would undertake the CAA within a given time frame and unsuitable candidates ruled out before being invited to an open day saving time and costs interviewing and testing unsuitable candidates.
In this presentation we look at was undertaken in order to achieve this goal. From which system to use for the CAA, the conversion of the questions so that they would work on the CAA without losing validity and how the system was tested. We will discuss the risks that were identified and how these would be manged. These included but not limited to the potential for candidates to cheat either individually or in groups, system failure before, during and after testing, the level of digital literacy of candidates and how the test data was interpreted and presented to the admissions tutors. evidence that this initiative has led to better admissions decision making. The overhead on the TEL support team and is this sustainable and if this approach transferable to other areas of the university business.
Although there was a given pass rate for the tests, an issue of concern raised during the actual recruitment process was about tolerance of these grades and to what level are candidates scores reviewed against the ambiguity in the questions (eg If an answer to a maths question was 1.6 do we allow 1.60 because the system can’t automaticalty detect that this is also correct. Many of these concerns have now been addressed and incorporated into next year’s iteration of the test. Improvements include use of question banks rather than fixed questions. Clearer instructions to students on how to answer questions and a format of result data formulated and agreed on before testing starts.