Listening to books for research purposes

Researchers at FAU make language research more realistic within the context of an interdisciplinary project

Speaking and understanding language is one of the most complex abilities of the human brain. For a long time now, researchers have been trying to better understand these processes by observing what changes take place in the brain when people speak, read or hear language. However, experiments to date have been held under artificial conditions, participants have only been given individual words or sentences to read or listen to. An interdisciplinary team led by Dr. Achim Schilling and Dr. Patrick Krauss from the Cognitive Computational Neuroscience Group at FAU has now taken a significant step forward towards making this research more realistic by measuring brain activity in participants listening to an audiobook. They then processed the data using cutting edge methods from computer linguistics and machine learning. The results have been published in the journal ‘Language, Cognition and Neuroscience’.

Listening to audiobooks for research purposes almost sounds like being on holiday. However, it wasn’t quite as relaxing as it sounds for study participants. They had to concentrate on keeping their vision focused on one point and answer comprehension questions whilst ignoring the magnetoencelography machine above their head measuring the changes in their brain activity. However, their task was significantly more realistic than the methods usually used by language researchers.

‘We hoped our experiment would act as a proof of concept to demonstrate that useful data can be obtained using this method,’ explained cognition expert Dr. Patrick Krauss. It was successful. The data was not only usable, but also abundant. In one single measuring session, participants listened to audiobooks for forty minutes, which is equivalent to approximately 7000 words at a time. ‘As our study involved 15 participants, we were able to measure brain activity for 10 hours of spoken language, equivalent to approximately 100,000 words,’ says Krauss. ‘The data we obtained can be evaluated again and again. One single data set allows us to test a virtually unlimited number of hypotheses, as well as generating brand new hypotheses. This is a paradigm shift in cognitive neuroscience of language and offers an immeasurable advantage over previous methods.’ It was also the first time that researchers in Erlangen have succeeded in using continual, natural, spoken language as a stimulus instead of well-controlled, repeated individual stimuli.

Thanks to the study, conducted in collaboration with researchers from the Brain Language Lab at Freie Universität Berlin, the newly established Linguistics Lab as well as the Chair of Machine Intelligence at FAU and several institutes from Universitätsklinikum Erlangen, the team of researchers now have a data set at their disposal which is comparable to the resources available to other research fields. This dataset is set to expand further. In preparation for a follow-on study which is yet to be published the team has already monitored 50 participants whilst listening to other recordings. ‘This approach allows us to combine the advantages of imaging techniques with those of corpus linguistics,’ Krauss explains. ‘That opens a number of new, exciting doors for research.’

Further information:

DOI: 10.1080/23273798.2020.1803375

Dr. Patrick Krauss
Cognitive Computational Neuroscience Group
Chair of English Linguistics
patrick.krauss@fau.de