“Reason is and ought only to be the slave of the passions,” wrote David Hume. Language, our instrument of reason, is saturated with value judgments. So what happens when computers – apparently the embodiment of pure mathematical rationality – start to use human language? They reproduce the traces of our passions, of course.
A thorough and elegant experiment reported in the journal Science this week shows this clearly. Researchers analysed a gigantic collection of English texts – more than 840bn instances of 2.3m words – and expressed mathematically how likely different words are to appear in the same contexts. This captures the largely unconscious web of associations around any given word with greater subtlety and fidelity than dictionary definitions can do, since people use words with much greater confidence than they can define them.
This technique also shows the value judgments embedded into ordinary language. Flowers are consistently easier to associate with pleasant concepts than insects; musical instruments than weapons. But others of these unconscious judgments are more worrying and in some cases reflect attitudes we would consciously reject. White-sounding names appear more pleasant than African American ones; young people’s names more than old people’s (a rather weaker effect). Male names are associated with careers, maths, and sciences, while female names are associated with family and with humanities. Bias of this sort both reflects and amplifies existing tendencies in society. This does not mean that it corresponds to underlying, essential inequalities. Different languages encode different systems of prejudice, as does the same language as it changes over time. But changes on that scale happen slowly, and the question meanwhile is what is to be done? Machine learning of this sort is becoming ubiquitous and at the same time much less visible. There is prejudice involved in the way these decisions are made by humans, too, but shuffling responsibility off onto computer programs is, in the phrase of one researcher, “money laundering for bias”.
Some people have proposed that the algorithms become self-correcting. If sexism is expressible as a mathematical relation between words in the system, and so transmitted by invisible algorithms, it could just as well be corrected algorithmically. Something similar already happens with the virtual keyboards on phones, which will write “duck” in place of a more common word. But to extend the principle would involve social engineering on a very large scale, carried out by private companies, not democratic governments. And differing societies, even different governments, could demand that the system reflect very different valuations. One can imagine a Russian government demanding that all the algorithmic associations of a concept like “gay” be tweaked to make them more negative.
The EU has taken a different approach. Next year a law will come into operation in member states which gives everyone a right to an explanation of any decision affecting them that has been reached algorithmically. There are considerable technical and even philosophical difficulties involved in realising any such aim. But computer systems are in some ways easier to interrogate for bias than human beings. We must take advantage of this fact.
