Feature

Freed from the brain

Patients with locked-in syndrome often have excellent cognitive abilities but are locked in their brain due to paralysis. Researchers at Brain Center Rudolf Magnus are developing brain-computer interfaces that allow these people to communicate again.

By Jolein de Rooij Image: Bram Belloni

The late, great scientist Stephen Hawking was also the world’s most famous ALS patient. At the end of his life, the discoverer of Hawking rays used his cheek muscles to select sentences, words, and letters and have them spoken by a speech synthesizer. However, Hawking ran a great risk of becoming ‘completely locked-in’. In anticipation of this fate, he began working with researchers at Intel, at the beginning of this century, to develop a system that translates the brain activity patterns behind facial expressions into actions. This system would never see the light of day.

Hawking would, therefore, have been very interested in the achievements of a research team at Brain Center Rudolf Magnus. In 2016, this team, led by professor Nick Ramsey, made headlines worldwide with a system that patients with locked-in syndrome (LIS) can use autonomously, at home, day and night, to call a carer and to communicate. A 58-year-old ALS patient learned to use the system to make ‘brain clicks’ with which she can select letters on a screen attached to her electric wheelchair. She does this by trying to touch her ring finger with her thumb. Even though this doesn’t result in a real movement of the hand, the system catches the motor signal and generates a click on the screen. ‘The greatest thing is that I can go outside again and that I can communicate,’ the first user told New Scientist in 2016.

No bells and whistles Since then, the team has been hard at work on new and improved methods of translating brain activity into all types of useful actions for people who have lost control of their muscles. Patients with locked-in syn

drome are so severely paralyzed that they can hardly move. Speaking is almost impossible. ‘Many people think that a life with such a severe disability is not worth living,’ says researcher Mariska van Steensel. ‘However, French and Swedish research has shown that the quality of life of these people is surprisingly high. But the ability to communicate is very important.’

BCI’s may make this possible. A BCI, or brain-computer interface, is a device that turns brainwaves into action. In the BCI created by Ramsey’s team, the electrodes are attached directly to the brain itself. This is done through a number of small holes in the skull, ensuring that the quality of the brainwaves and the reliability of the system are the highest. Two strips of four electrodes send signals to a transmitter that has been placed under the skin of the chest and sends signals wirelessly to a computer.

‘Ramsey’s research team is among the global pioneers in BCI,’ says Femke Nijboer, professor of health, medical, and neuropsychology at the University of Leiden. ‘The team combines fundamental research with practical research and has a strong methodological foundation. The great thing about the BCI system that they have developed is that the patient from the first experiment in 2016 is still using the system today. It also meets the most important user requirements for this type of system: no wires, no bells, and no whistles, just very practical.’

The ALS patient can use the BCI to call her carer, she can use it to communicate, and also to change the channel on her TV. All this is possible because, following an intense period of training, she was able to use the power of her brain to produce a ‘brain click’, which is reliable almost 90 percent of the time. Van Steensel: ‘She is currently averaging three letters per minute.’ A second person is now also training with the system.

fMRI as a predictor The success of the Utrecht research team is the result of over a decade of research into human brain mapping. Ramsey: ‘Brain functions are organized spatially within the brain. Different areas have different func

tions.’ For example, individual body parts have their own ‘control center’ in the brain, which is responsible for controlling and feeling movement.

To find positions on the surface of the brain that are particularly suited for BCI, many different areas of the brain had to be studied, for which the team required a steady stream of volunteers. They found them thanks to the cooperation of epilepsy patients, who came to Brain Center Rudolf Magnus to have temporary electrodes implanted under their skulls, directly on the brain, for diagnostic purposes. ‘For many years, we asked epilepsy patients to have an MRI before getting their implants,’ says Van Steensel. ‘We asked them to perform a task, like a specific hand movement.’ Using special technology – the functional MRI (fMRI) – their brain activity was visualized in a three-dimensional image. ‘It shows specific spots on the surface of the brain lighting up.’

After the diagnostic measurement electrodes were implanted, the team asked the test subjects to repeat the task. ‘This is how we tested to see whether fMRI can be used as a reliable predictor of the location where the most substantial changes in electric brainwaves take place when performing a certain task. This proved to be the case. We also asked test subjects to play a kind of game, in which they used these brainwaves to move a ball on a computer screen. This allowed us to study the extent to which patients were able to control the signal that they could send from a specific area of the brain.’

Smart signal processing In order to develop the world’s first BCI that could be used at home, the team made a number of smart choices. The team concentrated on a number of areas on the outside of the brain, where activity is relatively easy to interpret. This includes, for example, the area that controls the hands.

In addition, the team was looking for brain functions that can be controlled by will. ‘You want to be able to detect a signal

with which you can help people with severe disabilities,’ says Ramsey. ‘This means that the patient has to be able to control the signal and that noise has to be suppressed as much as possible.’

The signal must also be extremely reliable. Ramsey: ‘Our greatest challenge is preventing false positives. You have to make sure that the system doesn’t detect a false click when someone isn’t trying to click.’ This can be rather difficult, because the brain is ‘very unruly’, according to Ramsey. ‘It responds to everything,’ he says. ‘For example, the motor cortex becomes active as soon as you see someone else move or even think about movement. We’ve had to use quite a few tricks to filter out the activity that is generated on purpose.’ On top of that, the biofeedback that the user gets during the training period alters the signal. ‘This means we also have to refine the processing of the signal. So, the placement of a BCI really stimulates interaction between humans and machines.’

Speech using brain power The team is now also working on other ways to help LIS patients communicate. ‘In future versions of the BCI, we want to offer people even more control options,’ says Van Steensel. ‘We want to do this by making optimum use of the bounteous way in which the brain is organized. The area that controls the hand lends itself to this because it is relatively large, which makes it easier to distinguish between different movements.’ Ramsey: ‘Think of the surface

– p,k,a, and u – from the brainwaves of three different epilepsy patients. This research may make it possible for paralyzed people to learn to speak through a computer using brain power. ‘I think it might take another ten years before we become so good at it that we can try it with paralyzed patients,’ says Ramsey. ‘What we would like most of all, is to send the brainwaves generated by speaking out loud within the mind to a speech synthesizer, one by one.’

The potential applications seem endless.

‘What we would like most of all, is to send the brainwaves generated by speaking out loud within the mind to a speech synthesizer, one by one.’

of the brain as a screen that shows what the person is doing at that moment.’ In 2016, Ramsey’s team succeeded in distinguishing between four different signs in American Sign Language, thanks to smart signal-processing methods and intricate brain implants – again, in epilepsy patients. In addition, the researchers are focusing on the areas of the brain that control the tongue, the lips, the jaws, and the vocal cords. In 2017, Ramsey described how he could recognize four different phonemes For example, in 2016, a research team led by Professor Robert Gaunt of the University of Pittsburgh succeeded in having a 28-year-old man not only move a robot arm using a BCI, but also feel sensations of touch in his artificial fingers. These types of systems are currently too heavy and impractical for daily use. ‘Far in the future – I think thirty years is being optimistic – it will probably become possible to attach small computers the size of grains of rice to nerve-muscle connections, enabling paralyzed people to control their own muscles again,’ says Ramsey.

But in the short term, Ramsey also sees promising applications for BCIs on the horizon. ‘It is possible that, in the future, BCIs could offer practical treatment of strokes. Usually, a brain hemorrhage only takes place in one hemisphere of the brain. You then lose certain functions on one side of your body, because the pathways or gray matter in the corresponding hemisphere are damaged. However, there are indications that you can read a lot of information on the healthy side of the brain relating to what the injured side of the brain is trying to, albeit unsuccessfully. There is a chance that, in the future, people will be able to recover part of their lost functions following a stroke, thanks to BCIs.’

The least effort At the moment, many people with lockedin syndrome shy away from the brain surgery that is required for placement of a BCI. ‘That fear is understandable,’ says Van Steensel. ‘The only thing these people have is their ability to think. That makes taking the step to having brain surgery a difficult one.’ Femke Nijboer: ‘The real risk is much smaller than people think. The placement of a BCI is comparable to that of a cochlear implant (a system that communicates directly with the acoustic nerve), which is considered to be far more normal.’

Meanwhile, Ramsey’s research team is working on more intricate implants and smaller and more powerful transmitters, in collaboration with engineers at the Italian Consiglio Nazionale delle Ricerche (CNR). ‘In the future, operations on all fronts will become even less invasive,’ says Ramsey. For example, there are plans to place the implants not directly onto the brain, but on the cerebral membrane. That is the tough membrane between the skull and the brain. ‘That would mean it’s no longer brain surgery.’

At the same time, the team is continuing its research into the wishes of potential users. Van Steensel: ‘These types of aids can only be successful if they are actually of use to the target group. That is why an intrinsic part of all our research is to work with users to see what works best with the least amount of effort.’