Behind the Screen review – inside the social media sweatshops

Sarah T Roberts’s vital new study demonstrates how online content moderation is a global industry that operates on the back of human exploitation

“All human life is there” used to be the proudest boast of the (mercifully) defunct News of the World. Like everything else in that organ, it wasn’t true: the NoW specialised in randy vicars, chorus girls, Tory spankers, pools winners, C-list celebrities and other minority sports. But there is a medium to which the slogan definitely applies – it’s called the internet.

The best metaphor for the net is to think of it as a mirror held up to human nature. All human life really is there. There’s no ideology, fetish, behaviour, obsession, perversion, eccentricity or fad that doesn’t find expression somewhere online. And while much of what we see reflected back to us is uplifting, banal, intriguing, harmless or fascinating, some of it is truly awful, for the simple reason that human nature is not only infinitely diverse but also sometimes unspeakably cruel.

In the early days of the internet and, later, the web, this didn’t matter so much. But once cyberspace was captured by a few giant platforms, particularly Google, YouTube, Twitter and Facebook, then it became problematic. The business models of these platforms depended on encouraging people to upload content to them in digital torrents. “Broadcast yourself”, remember, was once the motto of YouTube.

And people did – as they slit the throats of hostages in the deserts of Arabia, raped three-year-old girls, shot an old man in the street, firebombed the villages of ethnic minorities or hanged themselves on camera…

All of which posed a problem for the social media brands, which liked to present themselves as facilitators of creativity, connectivity and good clean fun, an image threatened by the tide of crud that was coming at them. So they started employing people to filter and manage it. They were called “moderators” and for a long time they were kept firmly under wraps, so that nobody knew about them.

That cloak of invisibility began to fray as journalists and scholars started to probe this dark underbelly of social media. What has emerged from their investigations is the realisation that content moderation is now a huge global industry employing at least 100,000 people across the world. Most of these are contract workers rather than staff members and they work in stressful conditions, with little job security or health insurance and suffer psychological damage from having to confront – and make decisions about – the horrific stuff that people post on social media platforms.

Sarah T Roberts is an academic at the University of California, Los Angeles, who has been exploring this netherworld of commercial content moderation for eight years. Her book Behind the Screen is the first extensive ethnographic study of those who inhabit it. Nobody who reads it will ever again view social media platforms in a tolerant light. For what it reveals is that the tech industry, like all the great industries of the past, prospers on the back of human exploitation. Our social media feeds are kept clean by the unsung and poorly remunerated labour of countless thousands of human beings who have to risk traumatisation so that we shall not.

In that sense, social media is rather like the fashion business, endlessly providing diversion and pleasure to western consumers by marking up garments that have been made in the sweatshops of India, Bangladesh, Pakistan and Vietnam.

Roberts’s research confirms what scholars such as Tarleton Gillespie had suggested – that far from being a way for social media platforms to demonstrate their sense of social responsibility, content moderation is actually the critical part of their operations, for without it their brands would be irreparably damaged by what unsuspecting users would find in their feeds.

After all, one of the platforms’ pitches to users is that they provide safer spaces than the unruly, uncurated web. As one former moderator replied when Roberts asked him what the platform would be like without moderation: “It would be 100% porn. One hundred per cent. It would be a train wreck. The awful part is, it has to be done. It can’t not be moderated. But there’s no good way to do it.”

A key question, which Roberts doesn’t really explore, is whether the moderation task is ultimately a futile, sisyphean one, given the scale at which the platforms operate. Something like 400 hours of video are uploaded to YouTube every minute, for example. The technocratic response to this challenge is a cheery assurance that, one day, AI will take care of most of it. Roberts and her interviewees don’t buy that and neither do I. And even if Facebook or YouTube employed half-a-million human moderators, it wouldn’t be enough, unless something changes.

One thing that could make a difference would be for the platforms to “throttle” uploads to a more manageable rate. Unsurprisingly, this is an option that is never discussed in the industry. As the computer scientist Hany Farid put it: “The content is just too valuable a commodity to the platforms; it is the bait that lures users in and keeps them coming back for updates to scroll, new pictures or videos to view, new posts to read and new advertisements to be served.”

Corporations such as Google and Facebook that live by surveillance capitalism may therefore be heading for some kind of existential crisis. On the one hand, they have to keep the supply of uploads coming, for they provide the feedstock for their advertising machines; on the other, if through their contractors they cannot manage the moderation task, then their brands will become polluted and users will start to desert their platforms.

But that’s their problem. For us, one of the most striking messages of this remarkable book is that governments and regulators should be investigating the appalling working conditions under which much content moderation is done. And if you’re the kind of person who balks at buying a T-shirt that has been made in an Asian sweatshop, you might think of giving up Facebook or switching to DuckDuckGo for searching. After all, the issues are much the same.

Behind the Screen is published by Yale University Press (£20). To order a copy go to or call 0330 333 6846. Free UK p&p over £15, online orders only. Phone orders min p&p of £1.99


John Naughton

The GuardianTramp

Related Content

Article image
It's Complicated review – 'online space is teenagers' only public space'

This study shows that's it's not technology that teenagers are 'addicted' to – it's friendship groups, says Carole Cadwalladr

Carole Cadwalladr

24, Mar, 2014 @9:05 AM

Article image
Anti-Social Media: How Facebook Disconnects Us and Undermines Democracy by Siva Vaidhyanathan – review
An excellent critique of the social media giant underlines the threat it poses to us all – and suggests how it can be tamed

John Naughton

25, Jun, 2018 @7:00 AM

Article image
The Twittering Machine by Richard Seymour review – our descent into a digital dystopia
A stark polemic argues that social media may have unleashed an age of nihilism from which there is little hope of escape

Peter Conrad

11, Aug, 2019 @6:00 AM

Article image
The great digital-age swindle… and the man fighting back
He was tour manager for the Band, producer of Mean Streets… so why, at nearly 70, is Jonathan Taplin taking on Facebook and co?

Andrew Anthony

21, May, 2017 @9:00 AM

Article image
Roger McNamee: ‘It’s bigger than Facebook. This is a problem with the entire industry’
Mark Zuckerberg’s mentor and an early investor in Facebook on why his book Zucked urges people to turn away from big tech’s toxic business model

Alex Hern

16, Feb, 2019 @1:00 PM

Article image
Consent of the Networked: The Worldwide Struggle for Internet Freedom by Rebecca MacKinnon – review

John Kampfner applauds a study of the threats to liberty in the online age

John Kampfner

12, Feb, 2012 @12:04 AM

Article image
The Internet’s Dirtiest Secrets review – the human toll of detoxifying social media
A masterful edition of Storyville exposed the awful plight of the moderators tasked with purging tech platforms of violent and sexually abusive images

Lucy Mangan

19, Mar, 2019 @10:25 PM

Article image
Coming to a screen near you… the 20 best new social apps
There are plenty of great social apps jostling to be the next smartphone hit. Here are some of the contenders

Stuart Dredge

22, May, 2016 @9:30 AM

Article image
Don’t Be Evil review – how the tech giants have become too big to fail
Rana Foroohar’s masterly critique of the internet pioneers who now dominate our world

John Naughton

03, Nov, 2019 @7:00 AM

Article image
An Ugly Truth by Sheera Frenkel and Cecilia Kang review – Facebook’s battle for domination
Russian hacking, smear campaigns and livestreamed massacres are the price of Mark Zuckerberg’s quest for connectivity, grippingly probed by two New York Times journalists

John Naughton

18, Jul, 2021 @6:00 AM