2024–ongoing
Comparative Judgement
A web app for ranking work by repeatedly picking which of two things is better. A collaboration between Liverpool’s Centre for Educational Development and Support, the School of Dentistry, and the School of Biosciences. Students use it to peer-assess and actually look at the work; staff use it to mark at scale; it has also been used with movie posters as an icebreaker. Dentistry is using it live.
Marking a pile of student posters usually means giving each one a number and hoping the numbers mean something. This does it the other way: keep asking which of these two is better, and a ranking shows up from the choices.
The app is for images, most often student posters. It has also been used with movie posters as a way in. Students peer-assess in a way that forces them to actually look at the work. Staff can mark large sets without drowning in 7-out-of-10s. It is a collaboration with CEDS, Dentistry, and Biosciences. Dentistry uses it live.
You compare two images, comment, score a short rubric, and get another pair. The pictures sit in the same zoom viewers museums use on paintings, so dense posters are readable. Comments can be summarised with GPT. Every click is written to a database and also queued for Google Sheets, because a spreadsheet of a marking run is something people actually trust.
It is not a polished product. The README is still the framework starter, and deploying it involves a small ritual so the host does not eat the poster uploads. What I like is the two posters side by side, and the ranking appearing from the clicks.
The mashup
Keep picking which poster is better, zoom in like a museum, and keep a spreadsheet people actually trust.
AI-generated guess of the mashup mix, 1 to 5. Click a stat.
Which is better? 5/5: Keep picking a winner, and a ranking emerges.
Tags
Built with
Papers
Papers and conference writeups I am on.
- The use of AI to summarize peer feedback
Short piece on using GPT to summarise the pile of comments this app collects on student posters.
- A Bespoke Comparative Judgement Tool: Feedback-rich assessment through simple comparisons
Workshop with Sean Goodman, piloting the tool with colleagues from around the university.
- Everything Compares to You: Development and Evaluation of an Online Comparative Judgment Application for Assessment
Conference digital showcase about building and evaluating the app.
Explore Further
DemoLive site
Directory listing for the tool. Demo login: demotask / demo.
CodeGitHub repo
Private for now. The live site is the public face of this.
LinkLiverpool Centre for Educational Development and Support
DocsOpenSeadragon
The museum-style zoom viewer used as a poster loupe.
Notes to self for media to add
- Hero: two posters (movie posters are safer) in OpenSeadragon, “A is better / B is better”, faint ranking ticks or a bell curve underneath. Caption energy: “marking by only ever picking a winner.”
- Screenshots: group login; side-by-side compare; zoomed comment screen; rubric dropdowns; dashboard cards; win-rate table; Thurstone numbers if a group has them on; an AI summary; Sheets log filling up; Thurstone toggle in group settings.
- Short demo: pick a winner → scribble a comment → score the rubric → cut to the ranking moving. Optional second beat: movie-poster ice breaker, then dentistry posters.
- YouTube Short: “We marked posters without giving anyone 7/10.” Two zooms, one click, scale appears. Or: ice breaker, then same tool for a dentistry cohort.
- Extra: diagram of pairs → comparison matrix → z-scores → ranking. Terminal of php deploy/reclaim.php or php spark google-sheets:flush. Before/after: lonely rubric vs a pile of pairwise wins.
- Privacy: blur real student posters / names unless there is clearance. Movie posters are the safer hero.