FACESSVEP

The human brain has a remarkable ability to recognise and remember faces, even in different situations and across large time gaps. How do we do it? Professor Bruno Rossion tells us about his work in investigating how the human brain builds a visual representation of a face, research which could lead to new insights into how the brain functions

neuroscience offers enormous scope for research, with scientists using a wide variety of techniques to build a deeper understanding of how the human brain functions. As the Principal Investigator of the FACESSVEP project, Professor Bruno Rossion is looking at one specific aspect of brain function. “We specifically focus on a brain function that we think is fundamental: Perceptual Categorisation – how the brain organises sensory information and labels the world, for instance to tell apart cats from dogs, and do so for all instances of these animal species. We are concentrating on vision, the dominant modality in humans” he outlines. “Within the field of vision, we use the human face as our favourite model. We want to understand how the brain recognises people by their face – that’s the main goal of the project, it’s the model for us to understand brain function.”

This is a very rich model, as the visual system extracts a lot of sensory information from an individual’s face to categorize it according to gender, emotion, age and a whole host of other things. The main focus for the project however is facial identity. “How do we use the face to identify people?” says Professor Rossion. The human brain has a great capacity to recognise and remember faces, even in different situations and across large time gaps, and there are no animal species that have developed such a level of expertise. “We’re able to clearly individualise faces, draw distinctions between faces, and are

able to recognise that it’s the same face across changes of views and different lighting. Maybe you haven’t seen a person for five years but you still recognise them – how does the brain do it?” asks Professor Rossion.

There has been a great deal of research into whether we use specific facial features such as the eyes or the mouth to recognise an individual’s face. However, Professor Rossion believes the primary question is elsewhere: how we integrate an individual’s facial features into a visual image. “It’s a perceptual process – we generally call it “holistic” or “configural” processing. It’s really the ability of the human brain, of the visual system, to extract a single, unified representation of all these features at a glance,” he explains. “When you look at a face and recognise it, you don’t usually fixate on any of the features. Your first fixation, where your eye gaze will land, is on the top of the nose of the face, usually in the middle – in between these features.”

This is thought to be because the top of the nose is the best position to extract information on all the features at once and construct a holistic representation. This is the case both for people that we know well and those we are meeting for the first time; in fact, there is little evidence to suggest the visual system treats people differently on the basis of familiarity. “What is clear is that if you know somebody, the representation is richer and you’re able to recall a lot of semantic information about that person,” says Professor Rossion. “However, I’m interested in the extraction of the first perceptual image, an image that is sufficiently detailed to individualise the face, to make it a unique pattern in your brain. How do we do that?”

The visual system, broadly defined as the system that deals with the visual modality, plays a central role in this process. While the eye, the optic nerve and the primary visual cortex were long thought to be the main regions of the

This is a powerful source of motivation for the project’s work, with researchers using a method called fast periodic visual stimulation (FPVS) to investigate how humans build a visual representation of a face. This method involves repeatedly stimulating the visual system at a fixed frequency and recording the brain’s response using an electroencephalogram (EEG). “The idea is pretty old: if you

stimulate the brain with a light flickering on and off at say ten cycles per second, 10 Hertz – then the brain synchronises its activity exactly to that frequency,” explains Professor Rossion. “So if you put an electrode on an individual’s scalp, or inside their brain, you will see that after a few hundreds of milliseconds, populations of neurons will synchronise exactly to that frequency.”

across changes of views and different lighting.

brain that dealt with vision, it is now known that a much larger proportion of our brains are involved. “It’s estimated that up to 30-40 percent of the human brain deals with vision, either directly or indirectly,” outlines Professor Rossion. Enormous mental resources are required to deal with the complexity of the visual world, underlining the wider importance of research into the visual system. “If we understand how the visual system works, then we can gain deep insights into the working of the whole brain,” says Professor Rossion.

Fig 2 (a & b). A face picture presented at a fast rate of 5.88 Hz through sinusoidal contrast modulation, ensuring that the visual system is constantly stimulated for about one minute. Every 5 pictures (1.18 Hz), a new face identity is presented. Stimulation sequences can be presented at upright or - as a control - at inverted orientation, since the human brain finds it very difficult to identify inverted faces. c. An EEG spectrum gives responses at 1.18 Hz and harmonics (2.36 Hz, etc.), reflecting the discrimination of facial identities at this rate. This response is prominent over right occipito-temporal sites. d. Despite interindividual variability, the face identity discrimination response is obtained in a few minutes of stimulation in all individual participants tested (from Liu-Shuang, Norcia & Rossion, 2014, Neuropsychologia).

Fig 3. a. Sinusoidal contrast modulation of natural images at a fast 12.5 Hz rate (80 milliseconds between image onset). Faces embedded every 7 stimuli leads to large faceselective responses in the EEG spectrum (b) at the exact frequency rate of 12.5/7 = 1.78 Hz and harmonics (3.56 Hz, etc.). This right occipito-temporal face-selective response is obtained in a few minutes of stimulation in every individual brain. c. EEG signals over time reveal an ongoing synchronization to stimulation rate at 12.5 Hz, which can be selectively filtered out to reveal the specific complex response to faces. This response is made of 4 components from about 100ms onset to 500ms, revealing a prolonged selective processing of faces in the human brain (Retter & Rossion, 2016, Neuropsychologia).

This property offers a way to control brain activity, from which researchers can investigate which regions of the brain deal with visual information. “If I show you a light flickering at 10 Hertz, I get an electrical response from the brain at 10 Hertz, but I don’t get it from regions of the brain that don’t deal with vision. So the FPVS method is extremely specific,” says Professor Rossion. This phenomenon is fairly well-known, described in a paper published as long ago as 1934, but it has been relatively under-used; now Professor Rossion is using it in research with complex images. “We simply started by presenting images of different faces at 3.5 Hertz, for a duration of a minute,” he explains.

Researchers found that regions of the brain that deal with facial processing started to synchronise to that frequency. From here, Professor Rossion has been able to directly relate external stimulations to brain activity. “This is a major goal in research, we aim to develop intelligible relationships between the visual world and brain activity,” he says. By synchronising the brain to a specific frequency, researchers are able to almost take control of the brain. “We are able to concentrate all brain activity – everything that we are interested in – in that specific frequency. So it’s a very objective tool to understand brain function and it gives us enormous power, because we can just concentrate on the electrical response to that specific frequency,” explains Professor Rossion.

Different frequency rates of stimulation are being investigated within the project, as Professor Rossion believes it’s important to take the rate of stimulation into account when testing brain function. For instance, researchers have found that the best frequency to individualise faces, to build a visual representation, is at 6 hertz. “If I give you a second, and there are a number of faces to recognise in a crowd, we have found that you’re able to individualise up to six faces. If you have more faces you can’t do it in a second – it’s too many,” outlines Professor Rossion. “This is why we call 6 Hertz the optimal frequency for individualizing faces.”

This research is primarily centred on adults, but there are also workpackages in the project investigating facial recognition in children and infants. One of the great advantages of the FPVS technique is that researchers get a significant brain response in a very short amount of time, which means it can be used with infants, who might be rapidly distracted when other techniques are applied. “We’ve been able to test infants between 4-6 months old. We have studied their ability to tell various natural images of faces apart from complex natural images of objects. We have shown that they are able to do it at a single glance, that is when 6 images are presented every second,” says Professor Rossion. “In children, we have also started to look at the development of the ability to recognise the identity of faces.”

The question of whether adults, infants and children all recognise faces in the same way is still a source of debate. Professor Rossion believes the FPVS technique holds clear potential in this regard, yet it will be important to maintain a common standard across different populations in research, taking the typical adult system as the model. “If adults are able to individualise faces at 6 Hertz, we should not slow down the frequency when we test infants and children, we should also test them at 6 Hertz,” he stresses. “If they cannot respond at 6 Hertz it just means they cannot do it. So when we talk about facial identity recognition, we need to incorporate the rate at which you are able to do it in the definition of the function proper.”

A lot of cognitive processes are still evolving in infants and children, and debate continues over the age at which the facial identity recognition system becomes fully mature. There are differences in the way children and adults recognise faces, and our facial recognition abilities do change over time, but the underlying nature of these changes are unclear. “We don’t know if it’s a qualitative or quantitative change. Is the process the same in children and adults, and it improves? Or does a fundamentally different process take place?” says Professor Rossion. The application of new techniques could help answer these questions in future, believes Professor Rossion. “I’m very optimistic on this because we have a sensitive, objective and implicit measure of brain activity in our hands. But at the moment this question is unresolved,” he says.

The project’s research also extends to investigating clinical populations, whose facial recognition processes have been disturbed. Researchers are looking at facial recognition processes in patients with a deficit called prosopagnosia, where the ability to recognise faces is impaired following brain damage; Professor Rossion says this work has uncovered new potential avenues of investigation. “With the FPVS technique I gained facial identity recognition measures from every individual I tested outside the clinical setting, within a few minutes of testing. Then when I tested patients with prosopagnosia I got no response,” he says. This demonstrates that the technique is valid as a diagnostic tool, now researchers are looking to use it on other populations. “We intend to test a population with autism spectrum disorder,” outlines Professor Rossion.

This condition is characterised in some cases by an impaired ability to use facial

cues to interact with people. It is not known whether this is a perceptual problem, in that they fail to extract the information from the facial cues, or if they are in fact able to extract the information but they are not interested and don’t make use of it. “With the FPVS technique we should be able to answer that question. We’re not going to ask them anything, we’re just going to measure how their brain reacts to facial changes,” explains Professor Rossion. The technique has also just been applied to other processes, including reading. “Instead of presenting faces we present words and letters at a fixed rate, and measure the brain’s response at that rate,” says Professor Rossion. “While faces are processed primarily in the posterior region of the brain’s right hemisphere, visual letters primarily engage the left hemisphere, as soon as children learn these letters.”

Researchers are looking to build further on this work in future. With the project set to conclude in January 2017, Professor Rossion is keen to explore other areas in which the FPVS technique could prove relevant, including dyslexia. “We would like to test children who are suspected of having reading disorders, to see if we can develop diagnostic tools for dyslexia,” he outlines. Another area of interest is recording signals inside the brain; Professor Rossion says the development of a method of treating epilepsy refractory to medication offers a unique opportunity to gain new insights. “Clinicians are considering removing the part of the brain that generates an epileptic seizure. These patients are implanted with electrodes to test which part of the brain should be removed or not,” he explains.

This provides a great opportunity to investigate the brain, as signals recorded from within the brain are thought to much richer than those from outside. It is of course essential to get the consent of the patients first; with this proviso, Professor Rossion is keen to explore this further. “We have a unique opportunity to understand the brain by testing these individuals with this new approach,” he says.

Inside the brain. The fast periodic stimulation technique is applied while recording electrophysiological signals inside the brain of patients implanted for clinical monitoring of epileptic seizures. This rare approach allows identifying and quantifying the neural basis of face perception, with fast periodic visual stimulation (Jonas et al., 2016, PNAS)

Fast Periodic Visual Stimulation to Understand Human Brain Function (FACESSVEP)

The overarching goal of this research project is to understand how the human brain recognizes people by their face, a function for which artificial systems or other animal species can’t match human adult performance. The specific project is based on the visual stimulation of the brain with complex images presented at a fast periodic rate (e.g., 6 images/second) and the recording of electrophysiological brain activity exactly synchronized to this rate.

• Main funding of the project: European Research Council (ERC) – Starting Grant (Consolidator scheme) 2012-2017. • Fonds National de la recherche scientifique (FNRS) • Belgian Federal Grant Agency (BELSPO) • Louvain Fundation • Communauté Française de Belgique, ARC grant.

• Anthony Norcia, Stanford University, USA • Louis Maillard, Centre Hospitalier

Universitaire (CHRU), Nancy, France

Professor Bruno Rossion, IPSY/IONS Place Cardinal Mercier 10 bte L3.05.01 à 1348 Louvain-la-Neuve T: +32 1047 8788 E: bruno.rossion@uclouvain.be W: http://face-categorization-lab. webnode.com/research/steady-state-facepotentials-ssfp-/

Professor Bruno Rossion

Professor Bruno Rossion is director of research at the National Research Fund in Belgium (FNRS), University of Louvain in Belgium. He graduated from the University of Louvain and spent postdoctoral years at Brown University, USA. He has authored over 150 scientific publications in international peer-reviewed journals on the topic of face perception, using a diversity of approaches in systems and cognitive neuroscience.