Comparative Marking

Posted on: 12 December 2023 by Samuel Saunders and Anthony Topping in General

In this video podcast, Tony Topping, Senior Learning Technologist in the School of Engineering, spoke to Sam Saunders from the Centre for Innovation in Education about the purpose, benefits and process of comparative marking – where markers examine two pieces of work side-by-side and make a judgement on which is better based on a set of criteria.

Comparative marking is designed as an alternative to traditional rubric-based marking, where staff mark assessments or pieces of work against a rubric. By comparing submissions, markers can develop a more sophisticated sense of the quality of submissions, can find the marking process far quicker and simpler, and can account for the nuances in the submissions in a more complex way.

Tony also showcases a platform for completing comparative marking processes called RM Compare, which allows comparative judgements to be done in an adaptive way. Adaptive comparative marking presents the marker with pairs of submissions repeatedly, asking which is the better example and learning more about the submissions as the process develops. The more comparisons are made, the more often the marker is presented with two pieces of work that are deemed to be of similar quality, eventually leading to a very complex and sophisticated rank order of submissions based on the comparisons made. Multiple judges can be used on RM Compare, too, which develops a highly complex picture of the quality of submissions across a range of artefacts very quickly.

In Engineering, the platform is already being used in both summative and formative contexts – on both assessments, and in exercises where students rate each others’ work and example assessment submissions, in order to help develop their assessment literacy, their subject knowledge, and their engagement with the course.

Engineering also have the capacity to provide licences for the software, and anyone interested in hearing more can sign up for a workshop (schedule for January 16^th 2024) on the platform here. Further questions or comments can be directed to topping@liverpool.ac.uk or samuel.saunders@liverpool.ac.uk.