Paralysed man uses ‘mindwriting’ brain computer to compose sentences

Man, known as T5, was able to write 18 words a minute with more than 94% accuracy on individual letters

A man who was paralysed from the neck down in an accident more than a decade ago has written sentences using a computer system that turns imagined handwriting into words.

It is the first time scientists have created sentences from brain activity linked to handwriting and paves the way for more sophisticated devices to help paralysed people communicate faster and more clearly.

The man, known as T5, who is in his 60s and lost practically all movement below his neck after a spinal cord injury in 2007, was able to write 18 words a minute when connected to the system. On individual letters, his “mindwriting” was more than 94% accurate.

Frank Willett, a research scientist on the project at Stanford University in California, said the approach opened the door to decoding other imagined actions, such as 10-finger touch typing and attempted speech for patients who had permanently lost their voices. “Instead of detecting letters, the algorithm would be detecting syllables, or rather phonemes, the fundamental unit of speech,” he said.

Amy Orsborn, an expert in neural engineering at the University of Washington in Seattle, who was not involved in the work, called it “a remarkable advance” in the field.

Scientists have developed numerous software packages and devices to help paralysed people communicate, ranging from speech recognition programs to the muscle-driven cursor system created for the late Cambridge cosmologist Stephen Hawking, who used a screen on which a cursor automatically moved over the letters of the alphabet. To select one, and to build up words, he simply tensed his cheek.

Hawking’s system was a big improvement on the painstaking process used by Jean-Dominique Bauby, the late editor-in-chief of French Elle, to write his 1997 memoir, The Diving Bell and the Butterfly. Bauby had a massive stroke in 1995 that left him “locked-in”, able to blink only his left eyelid. He dictated the entire book by having the freelance editor Claude Mendibil read the alphabet aloud and blinking when she reached the next letter he wanted. For all the progress made since then, researchers have long dreamed of more efficient systems that tapped directly into the brain.

In the case of T5, nine years after his accident he enrolled on a clinical trial called BrainGate2 to investigate the safety of brain-computer interfaces (BCIs). These are small, implantable computer chips that read electrical activity straight from the brain. The man had two computer chips, each the size of a baby aspirin and bearing 100 electrodes, placed on the left side of his brain where neurons send signals to control the right hand.

Willett and his colleagues asked T5 to imagine he was holding a pen above a pad of paper and then to try to write individual letters of the alphabet, even though he could not move his arm or hand. As he tried, they recorded activity from the brain region that would have controlled his movements.

The scientists found that more than 10 years after the accident, the man’s brain still produced distinct patterns of neural activity for each letter and various punctuation marks.

These recordings, and others taken as T5 tried to write out example sentences, were used to train an artificial intelligence algorithm. Many sessions later, the algorithm could predict in real time which letter the man was trying to write with 94.1% accuracy. When the scientists added autocorrect the accuracy rose to 99%.

During the sessions, T5 often felt that the imaginary pen in his hand was moving across the page and tracing out letters, and that he could “write” faster if he kept letters small, Willetts said.

Some letters were harder to distinguish than others, according to the study in Nature. The letters r, h and n, for example, require similar movements and so similar brain activity.

One unknown is how well the algorithm might work for languages not based on the Roman alphabet. For example, Tamil has 247 letters, many of which look similar, which could confuse the algorithm.

It will be some time before such brain-reading systems are ready for wider use. In an accompanying commentary, Orsborn and Pavithra Rajeswaran, also at the University of Washington, write that implantable BCIs “will need to provide tremendous performance and usability benefits to justify the expense and risks associated with implanting electrodes into the brain”.

Orsborn said: “This study is a remarkable advance for intracortical brain-computer interfaces because it achieves a huge leap in typing speed and accuracy.

“While this gives us many reasons to be optimistic about the future of BCIs, challenges remain to develop systems that we can use for daily life. For instance, they demonstrated their algorithms in one person for a brief time. How we extend this work to assure the algorithms work well every day and for everyone is an exciting new challenge.”


Ian Sample Science editor

The GuardianTramp

Related Content

Article image
Brain implant helps paralysed man regain partial control of his hand
Pioneering neural bypass allows chip in the brain to send signals to muscles in wrist and fingers, making it possible to swipe a card and play Guitar Hero

Ian Sample Science editor

13, Apr, 2016 @5:00 PM

Article image
Brain implant allows paralysed woman to control a robot with her thoughts

The BrainGate implant can decode a patient's brain signals and instruct a robotic arm to reach and grasp objects

Ian Sample, science correspondent

16, May, 2012 @5:00 PM

Article image
Human neurons transplanted into rats to help study brain disorders
Groundbreaking research could throw light on how cells from patients with neuropsychiatric disorders malfunction

Ian Sample Science editor

12, Oct, 2022 @3:07 PM

Article image
Mind over matter helps paralysed woman control robotic arm

Doctors in Pittsburgh stunned at ability of patient who has reached levels of performance never seen before

Ian Sample, science correspondent

17, Dec, 2012 @1:00 AM

Article image
Paralysed man moves arm using power of thought in world first
Neuroprosthetic procedure first in world to restore brain-controlled reaching and grasping in man paralysed from the neck down

Sarah Boseley Health editor

29, Mar, 2017 @8:38 AM

Article image
'Brain training' technique restores feeling and movement to paraplegic patients
Using VR technology, an exoskeleton and software, study claims participants have unexpectedly experienced partial recovery of movement and sensation

Tim Radford

11, Aug, 2016 @1:10 PM

Article image
Start running and watch your brain grow, say scientists

Aerobic exercise triggers new cell growth in region of brain linked to recollection, study finds

Ian Sample, science correspondent

18, Jan, 2010 @8:41 PM

Article image
Scientists create decoder to turn brain activity into speech
Technology could in effect give voice back to people with conditions such as Parkinson’s

Hannah Devlin Science correspondent

24, Apr, 2019 @5:11 PM

Article image
Paralysed people could get movement back through thought control
Brain implant could allow people to 'feel' the presence of infrared light and one day be used to move artificial limbs

Alok Jha, science correspondent

17, Feb, 2013 @7:10 PM

Article image
Ultrasound trial offers hope for brain cancer patients
New technique temporarily allows drugs to cross blood brain barrier to treat tumours

Linda Geddes

13, Oct, 2021 @5:00 PM