Randomizing parts of grant decision-making increases diversity and reduces costs, argue some researchers.Credit: Andrew Angelov/Alamy
Rachel Heyard’s job was to find a signal in the noise, but the trouble was how often there wasn’t one. As a biostatistician in the data team at the Swiss National Science Foundation (SNSF) in Bern, Switzerland’s main public research funder, Heyard faced a tricky problem. Reviewers and panel members scored each grant application against criteria such as scientific relevance, feasibility and the applicant’s track record, grading the proposal on a scale from A to D. Her team’s task was to test how much those scores could be trusted: to determine which applications truly outranked the others in the competition for limited funds.
The strongest and the weakest proposals were sorted readily enough. But the rest piled up in the messy middle — what she calls “a cloud of B proposals” — clustered so tightly in score that panels struggled to differentiate them. To force a ranking, they sharpened the grades with pluses and minuses, splitting hairs that might not have existed.
Grant proposals drafted with AI help more likely to win NIH funding
The stakes were high at the SNSF, which in 2021 was handing out the equivalent of about US$1 billion annually to projects across every discipline. Each year, the cut-off between the financed proposals and the rejected ones ran straight through the crowded middle. Where a proposal fell could shape a laboratory’s future or a young researcher’s career.
Heyard helped to build a statistical framework to measure what the reviewers’ scores were actually worth. It treated each proposal’s standing not as a fixed rank, but as a range that reflected the uncertainty in a panel’s judgement. Instead of pretending that one proposal was exactly 23rd and another precisely 27th, the model asked whether those apparent ranks could really be distinguished.
Near the funding line, they often could not. When a proposal’s possible ranks crossed the threshold between funded and rejected, it fell into the grey zone, good enough to stay in contention but too close to its neighbours to justify a precise ordering.
Fortune favours the fundable
Starting in 2022, the SNSF stopped treating these rankings as exact, and implemented a new approach. A mathematical model identified the proposals that were too close to separate and sent them into a lottery — a tie-breaker the foundation had previously trialled. Peer review still set the bar to qualify, but among the proposals near the funding threshold, a financed project and a rejected one would now be divided by chance. “Instead of saying, ‘should we introduce randomness into science processes?’, we’re reframing this question into ‘are we confident that our decisions are precise enough?’” says Heyard, now a researcher at the University of Zurich’s Center for Reproducible Science and Research Synthesis in Switzerland. “And the answer to that question is often no.”
The research backs up Heyard’s opinion. When 43 reviewers judged the same 25 applications in a 2018 study1, they barely agreed. “We already know that the awards are a lottery,” says Tom Stafford, a cognitive scientist at the University of Sheffield, UK, who is on secondment at the Research on Research Institute in London, which studies how science is funded. “We should just make it a formal lottery, rather than an unofficial one.”
After judgement has reached its limit, handing a decision to chance might sound like surrender, but it is one of the oldest tools of fair government. Ancient Athens filled many public offices and juries by drawing lots, pulling out the names of eligible citizens using a stone slab called a kleroterion. What these funders propose might be less a break with tradition than a return to it.
From splitting hairs to drawing straws
The Research on Research Institute keeps a catalogue of such trials and has logged more than a dozen funders that have used or tested partial lotteries, with 11 catalogue entries since 2022. The practice remains uncommon in global terms but is no longer hypothetical, spanning national agencies, charities such as Wellcome in London and the Novo Nordisk Foundation in Hellerup, Denmark, as well as universities. Individual awards range from the equivalent of a few thousand US dollars to more than $1 million. Stafford and others in the field have no usable global data on how much money is disbursed through such draws, but the number of funders that are experimenting with this idea keeps growing.
The implementations also vary. The lightest is the tie-breaker, the SNSF’s model, in which chance decides between only those proposals that the reviewers genuinely cannot rank. A bolder version randomizes across everything that clears a quality bar. This is the route that New Zealand’s Health Research Council has taken since 2013 for its Explorer Grants, currently worth NZ$150,000 (US$87,000) each. The most radical system, adopted by some programmes in Germany, inverts the process entirely with a ‘lottery first’ model. Here, researchers draw for the right to submit a full proposal.
New peer-review trial lets grant applicants evaluate each other’s proposals
One of the most closely watched of these experiments comes from the United Kingdom. In 2022, the British Academy, the national organization for the humanities and social sciences, began randomizing its Small Research Grants among all proposals that pass a quality threshold — a trial since extended to 2028. The issue that the academy was trying to solve was not simply that the review board was overworked, but also that many researchers never applied at all. “Some were disqualifying themselves from even making an application,” says Ken Emond, the academy’s head of research funding, who notes that many wrongly assumed they stood little chance against those from bigger-name, research-heavy universities. A lottery, by contrast, signals that every qualified applicant has the same odds.
The programme’s numbers quickly changed after implementing this system. Applications nearly doubled, from 643 in 2022 to 1,257 in the 2025–26 round, and the pool broadened. The share of applications from Asian and Asian British researchers, for instance, climbed from 13% to 19%, and their share of awards went from 11% to 17%. Other factors played a part, Emond notes, but the results matched what supporters of this change had predicted.
A lottery also changes what a rejection means. Whereas the academy once gave no feedback at all, it can now tell applicants whether they fell short on quality or simply lost the draw. Some rejectees, Emond says, take their “passed the threshold” verdict as validation of their proposal when applying elsewhere.
More names in the hat
To Adrian Barnett, a statistician at the Queensland University of Technology in Brisbane, Australia, and one of the trial’s independent evaluators, the diversity gain is especially valuable because it comes at such a low cost. Once the academy has screened applications for quality, the proposals still in contention go to Barnett, who runs the draw. “I get the anonymized list, and I send them the winners and losers the next day,” he says. “Funders could gain in diversity without unpopular post hoc adjustments to scores. That’s a big gain for a simple change.”
Responses to the AI grant flood must prioritize fairness as part of excellence
