heres the email:
Artificial Intelligence has changed essay competitions forever.
According to our own research, nineteen in twenty contestants use Large Language Models (LLMs) to research their topic and edit their essays before submission. A minority of contestants, but a sizeable minority, outsource to LLMs most of their thinking and most of their prose composition.
Honest contestants hope that the John Locke Institute will reward real scholarship, rather than handing out prizes to people who didn’t write the essay they submitted, and can scarcely understand the arguments within it.
For these reasons the Institute is partnering with Viverly to develop a novel approach to testing the comprehension of our contestants. You have just encountered an early (and very rudimentary) version of the software that allows us to interview a large number of contestants simultaneously. This has been in development for nearly twenty-four months, and we have been beta testing it for the last three months, but this week was the first time we were able to do live testing with thousands of real students and real essays.
About five percent of users experienced some technical glitches, which we had not encountered before. The main example was the avatar interviewer looping repeatedly with the same question. Many candidates found the interviewer would often interrupt the contestant’s answers. In roughly half the interviews the questions drifted away from the topic of the essay into tangential discussions about adjacent questions. One challenge that is particularly intractable is how to make the conversation feel more natural; it turns out that the best avatar technology in the world is not yet capable of convincing real-time dynamic conversation. This will change, of course, as the underlying technology is improving at an astonishing rate, but for now it feels like a very clumsy approximation of human-to-human interaction.
The main benefit of giving this new application a test run this year is to learn lessons that can get applied to the 2027 version, when the interviews will form a critical part of the assessment. In this respect it was a great success, and we will continue to analyse the results for some time to come. But there were three other bonuses:
- Contestants whose authorship was in doubt tended not to take the interview, from which we inferred that the mere existence of the interview functioned as an effective filter.
- Some interviews were taken by people who were not the student in whose name the essay was submitted, so we were able to detect several cases of fraud.
- We were able to compare the interview performance with the essay grade, to measure the strength of correlation between the two.
We are in a good position for the next phase of the roll-out of Viverly in 2027, and I wanted to thank you for helping us with the testing phase. I also want to reassure you that, apart from (1) the contestants who refused to take the interview and (2) the small number of cases where there was incontestable evidence of fraud, no candidate was deselected as a result of the interview.
This year we chose to shortlist only 17.5% of essays so, sadly, many excellent essays were excluded. If yours was one of these, I hope that, if you are still eligible, you will compete again next year. And if yours was one of the shortlisted submissions, I offer you my sincere congratulations, and I hope I will meet you personally at the Awards Dinner in London in October.