The algorithm used by the exams regulator Ofqual to determine A-level and GCSE grades in England following the cancellation of exams has been scrapped, prompting chaos and an urgent review by the UK’s national statistics regulator. The U-turn followed a series of warnings over its potential volatility and fairness.
31 March
With schools closed and the pandemic gathering pace, the education secretary, Gavin Williamson, warns the head of Ofqual the regulator “should ensure, as far as is possible, that qualification standards are maintained and the distribution of grades follows a similar profile to that in previous years”. In a letter to Sally Collier, Ofqual chief regulator, he says students should be issued with calculated results based on schools’ assessments of their ability in the relevant subjects, supplemented by other evidence. Ofqual begins to test a range of algorithms, simulating predictions of results for 2019 exams and comparing the outcomes to the actual results.
15 April
Ofqual launches a consultation on the arrangements for awarding GCSEs, AS and A-levels, which draws responses from more than 1,000 schools plus unions, teaching bodies and employers.
22 May
It publishes an analysis of responses, which includes the concern that “some arbitrary algorithm [is] making standardised adjustments”. Calls are made for transparency. One respondent says: “BAME – feedback from some local communities has noted that they are concerned that grade calculation will have a negative impact on their results.”
1-12 June
When schools begin submitting teacher-assessed grades, flaws in the algorithm become evident, an adviser who reportedly helped develop the Ofqual software told the Daily Mail. There was “always an understanding there would be winners and losers … there was a very specific point when it became doomed,” the source reportedly said.
Early July
Sir Jon Coles, a former director general at the Department for Education (DfE), writes to Williamson expressing concerns about the algorithm used by Ofqual. He warned it would be 75% accurate at best when applied to GCSE and A-level grading, according to the Times. Williamson holds a video conference with Coles to discuss the concerns but opts to press ahead.
External advisers meanwhile sound warnings to Ofqual that the formula was “volatile” and risked producing erratic outcomes, the Guardian reveals.
11 July
MPs on the education select committee raise concerns about the grading system in an interim report, warning that some pupils risked being “systematically disadvantaged by calculated grades”. It calls on Ofqual to publish details of the algorithm immediately, adding: “Ofqual should not be afraid of scrutiny or open debate over whether its model offers the fairest outcome for every pupil and provider.” But the exams body refuses, with Ofqual warning that publishing the model would help some schools to calculate awarded grades.
Mid-July
Coles also calls on the schools minister, Nick Gibb, who holds a meeting with Ofqual and senior officials at the DfE, to examine the concerns raised. “He felt that the model as devised would disadvantage young people from poorer families and so we discussed that in great detail and I was reassured that it would not,” Gibb said on Thursday.
21 July
Ofqual runs a summer symposium about the exam arrangements, as it comes under increasing pressure but continues to resist calls to publish the model it will rely on. However, it admits for the first time that it has tested 12 different approaches. Slide 17 of a slide pack used at the event states the statistical model it has selected is the “Direct Centre-level Performance (DCP) approach”.
Late July
The algorithm is finalised, with the diktat issued by Williamson on 31 March understood to have been the overriding factor that led Ofqual to reject warnings from its external advisers.
13 August
The failure of the approach becomes apparent when A-level grades are announced, with teachers in England having 39% of their assessments downgraded. A U-turn by Scottish authorities has already taken place. While Ofqual has yet to publish the actual algorithm, it publishes the rationale in a “Annex E” addition to a document first published on 7 July: “Requirements for the calculation of results in summer 2020.”
