From frozen screens to the oblivious person on mute, the trials and tribulations of video calls became familiar challenges as the pandemic forced workers to communicate from their kitchen tables, makeshift offices and boxroom desks.
Now scientists have revealed why we often end up raising our voices at our colleagues: as video quality deteriorates, we speak louder and alter our gestures in an attempt to compensate.
“The gestures that we make while talking, as well as other visual signals such as facial expressions, are truly important and integrated aspects of how we communicate,” said Dr James Trujillo, the first author of the research at Radboud University Nijmegen, in the Netherlands.
“If your Zoom connection is giving you poor video, remember to articulate both your speech and gestures.”
He said previous research had shown that when we cannot hear someone well because of noise, we speak louder, while the team’s recent work had shown that when a video call is very noisy, people make their gestures more noticeable.
“It seems that there is a general tendency for us to put more effort into our speech and gestures when our communication is disrupted by something, such as noise or a poor video connection,” he said.
Writing in the Royal Society Open Science journal, Trujillo and colleagues reported how they analysed video calls between 20 pairs of participants. For each pair, participants sat in separate rooms and chatted to each other in casual, unscripted conversation over a Zoom-like video call for 40 minutes.
Over the course of the call, the quality of the video was changed in 10 steps between excellent and completely blurred, with half of the pairs experiencing improving video quality over the call, and half having it deteriorate. The video quality during the call at each step was the same for both participants.
The researchers tracked the gestures of participants and aspects of their speech during the calls.
It revealed that as video quality deteriorated, participants initially reduced their arm and body movements while conversing, but as the quality deteriorated further, they moved them more. However, the rate, speed and size at which they gestured all increased as video quality initially fell but then decreased as it deteriorated further That, said Trujillo, suggested gestures were, at least initially, becoming bigger but “cleaner”.
The team added that when gestures were not being used, speech was unaffected by decreases in visibility. But when gestures were used, volume increased by up to 5 decibels as video quality initially fell, and then remained at this higher level as video quality deteriorated further – in other words, when gestures ceased to be useful at all.
Trujillo said the results showed communication was not just about speech but that, in the case of spoken language, it was an integrated system of visual and auditory signals that work together to convey meaning.
“To compensate [for poorer video quality], people ‘exaggerate’ the form of their gestures in order to help their partner recognise the meaning of the gesture, even when they cannot see it as well as normal,” he said.
“Although speaking louder probably doesn’t help, the fact that people do it shows how integrated these systems are, particularly since people only speak louder when gestures are present,” Trujillo added. “They know that the gestures being produced are vital to their communication, but their partner is going to have a harder time seeing them. So they increase the strength of the other signal – speech.”
