brain research paper

Search Menu
CNS Injury and Stroke
Epilepsy and Sleep
Movement Disorders
Multiple Sclerosis/Neuroinflammation
Neuro-oncology
Neurodegeneration - Cellular & Molecular
Neuromuscular Disease
Neuropsychiatry
Pain and Headache
Advance articles
Editor's Choice
Author Guidelines
Submission Site
Why publish with this journal?
Open Access
About Brain
Editorial Board
Advertising and Corporate Services
Journals Career Network
Self-Archiving Policy
Dispatch Dates
Terms and Conditions
Journals on Oxford Academic
Books on Oxford Academic

Latest Article

Editor's Choice

Publish with Brain

Enjoy benefits including a user-friendly submissions system, discoverability support, and a distinguished editorial team.

Learn more about reasons to publish your work with Brain

Latest posts on X

Altered brain glucose metabolism as a mechanism for delirium.

Delirium is a common condition with significant impact on patient outcome. This episode discussed potential intrinsic brain mechanisms that may underly delirium. The team explores the evidence that delirium is more than just a systemic process that extends into the brain but may involve pathophysiological alterations of brain function.

Are there differences in the immune response among individuals of Black ethnicity with multiple sclerosis?

This episode discussed racial differences in antibody response to natalizumab treatment for individuals of black ethnicity with multiple sclerosis. Further, they consider potential mechanisms which may underly differences and the implications on treatment.

Exploring a genetic basis for disordered speech and language: FOXP2 mutations and striatal neurons

This article explores a potential genetic basis for disordered speech by a mutation in the transcription factor FOXP2 . This was discovered in KE family members with speech disturbances was a landmark example of the genetic control of vocal communication in humans.

The BRAIN podcast

The BRAIN podcast is the official podcast of Brain and Brain Communications highlighting the rich and diverse neuroscience published in the journals.

Browse all episodes

Video Abstracts

Brain endothelium class I molecules regulate neuropathology in experimental cerebral malaria

Spike propagation mapping reveals effective connectivity and predicts surgical outcome in epilepsy

Watch more videos from Brain

Haemorrhage of human foetal cortex associated with SARS-CoV-2 infection

Diabetes and hypertension are related to amyloid-beta burden in the population-based Rotterdam Study

Neurogenetics header with pink flowers and one white flower in a field

Neurogenetics Collection

Explore recently published articles on the topic of neurogenetics from Brain . The articles are open access or freely available for a limited time.

Browse the collection

China Collection

Brain celebrates the Year of the Tiger with a collection of articles from our Chinese colleagues. The articles are open access or freely available for a limited time.

Neuroinflammation Collection

Explore nine recently published articles on neuroinflammation. The articles are open access or freely available for a limited time.

Browse the collection

COVID Collection

Discover Brain ’s COVID collection, bringing together a range of articles on the neuroscience, neurology, and neuropsychiatry of SARS-CoV-2 infection.

Brain on the OUPblog

Of language, brain health, and global inequities

Speech and language assessments have emerged as crucial tools in combatting one of the greatest public health challenges of our century, the growth of neurodegenerative disorders. In this blog post, Adolfo M. García explores how a lack of linguistic diversity in assessment methods threatens their potential for more equitable testing worldwide.

Blog post | Read article | All blog posts

An illustration of a human head surrounded by computer software.

It’s time to use software-as-medicine to help an injured brain

Multiple mild Traumatic Brain Injuries (“mTBIs”) can put military service members at an elevated risk of cognitive impairment. Service members and veterans were enrolled in a trial with a new type of brain training program, based on the science of brain plasticity and the discovery that intensive, adaptive, computerized training—targeting sensory speed and accuracy—can rewire the brain to improve cognitive function. The trial found that the training program significantly improved overall cognitive function.

Blog post | Read article | All blog posts

How air pollution may lead to Alzheimer’s disease

Air pollution harms billions of people worldwide. Over the past few decades, it has become widely recognized that outdoor air pollution is detrimental to respiratory and cardiovascular health, but recently scientists have come to acknowledge the damage it may cause on the brain as well.

When narcolepsy makes you more creative

Patients with narcolepsy are often lucid dreamers, and experience direct transitions from wakefulness into REM sleep. Lacaux et al. report that these patients perform better than healthy controls on creativity tests, supporting a role for REM sleep in creativity.

Blog post | Read article | All blog posts

Author resources

Learn about how to submit your article, our publishing process, and tips on how to promote your article.

Find out more

Want access to Brain ?

Fill out our simple online form to recommend Brain to your library. Your institution could also be eligible to free or deeply discounted access via our developing countries initiatives .

Recommend now

Introducing Brain Communications

Brain Communications is the open access sister journal to Brain , publishing high-quality preclinical and clinical studies related to diseases of the nervous system or maintaining brain health.

Related Titles

Cover image of current issue from Brain Communications

Contact the editorial office
Guarantors of Brain
Recommend to your Library

Affiliations

Online ISSN 1460-2156
Print ISSN 0006-8950
Copyright © 2024 Guarantors of Brain
About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Copyright © 2024 Oxford University Press
Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Loading metrics

Open Access

Peer-reviewed

Research Article

Relating Structure and Function in the Human Brain: Relative Contributions of Anatomy, Stationary Dynamics, and Non-stationarities

* E-mail: [email protected]

Affiliation Laboratoire d'Imagerie Fonctionnelle, UMR678, Inserm/UPMC Univ Paris 06, Paris, France

Arnaud Messé,
David Rudrauf,
Habib Benali,
Guillaume Marrelec

Published: March 20, 2014
https://doi.org/10.1371/journal.pcbi.1003530
Reader Comments

Investigating the relationship between brain structure and function is a central endeavor for neuroscience research. Yet, the mechanisms shaping this relationship largely remain to be elucidated and are highly debated. In particular, the existence and relative contributions of anatomical constraints and dynamical physiological mechanisms of different types remain to be established. We addressed this issue by systematically comparing functional connectivity (FC) from resting-state functional magnetic resonance imaging data with simulations from increasingly complex computational models, and by manipulating anatomical connectivity obtained from fiber tractography based on diffusion-weighted imaging. We hypothesized that FC reflects the interplay of at least three types of components: (i) a backbone of anatomical connectivity, (ii) a stationary dynamical regime directly driven by the underlying anatomy, and (iii) other stationary and non-stationary dynamics not directly related to the anatomy. We showed that anatomical connectivity alone accounts for up to 15% of FC variance; that there is a stationary regime accounting for up to an additional 20% of variance and that this regime can be associated to a stationary FC; that a simple stationary model of FC better explains FC than more complex models; and that there is a large remaining variance (around 65%), which must contain the non-stationarities of FC evidenced in the literature. We also show that homotopic connections across cerebral hemispheres, which are typically improperly estimated, play a strong role in shaping all aspects of FC, notably indirect connections and the topographic organization of brain networks.

Author Summary

By analogy with the road network, the human brain is defined both by its anatomy (the ‘roads’), that is, the way neurons are shaped, clustered together and connected to each others and its dynamics (the ‘traffic’): electrical and chemical signals of various types, shapes and strength constantly propagate through the brain to support its sensorimotor and cognitive functions, its capacity to learn and adapt to disease, and to create consciousness. While anatomy and dynamics are organically intertwined (anatomy contributes to shape dynamics), the nature and strength of this relation remain largely mysterious. Various hypotheses have been proposed and tested using modern neuroimaging techniques combined with mathematical models of brain activity. In this study, we demonstrate the existence (and quantify the contribution) of a dynamical regime in the brain, coined ‘stationary’, that appears to be largely induced and shaped by the underlying anatomy. We also reveal the critical importance of specific anatomical connections in shaping the global anatomo-functional structure of this dynamical regime, notably connections between hemispheres.

Citation: Messé A, Rudrauf D, Benali H, Marrelec G (2014) Relating Structure and Function in the Human Brain: Relative Contributions of Anatomy, Stationary Dynamics, and Non-stationarities. PLoS Comput Biol 10(3): e1003530. https://doi.org/10.1371/journal.pcbi.1003530

Editor: Claus C. Hilgetag, Hamburg University, Germany

Received: August 14, 2013; Accepted: February 8, 2014; Published: March 20, 2014

Copyright: © 2014 Messé et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This work is supported by the Inserm and the University Pierre et Marie Curie (Paris, France). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Coherent behavior and cognition involve synergies between neuronal populations in interaction [1] – [3] . Even at rest, in the absence of direct environmental stimulations, these interactions drive the synchronization of spontaneous activity across brain systems, shedding light on the large-scale anatomo-functional organization of the brain [4] . The study of such patterns of synchronization has known important developments due to recent methodological advances in brain imaging data acquisition and analysis. These advances have enabled investigators to estimate interactions in the brain by measuring functional connectivity (FC) from resting-state functional MRI (rs-fMRI). Analyses of FC at rest have supported the hypothesis that the brain is spatially organized into large-scale intrinsic networks [5] – [7] , e.g. the so-called resting-state networks [8] , [9] , such as the default mode network, which have been linked to central integrative cognitive functions [10] – [13] . The study of large-scale intrinsic networks from rs-fMRI has become a central and active area for neuroscience research. However, the mechanisms and factors driving FC, as well as their relative contribution to empirical data, are still highly debated [14] and remain to be elucidated.

Theoretical rationale and empirical findings support the hypothesis that FC is driven and shaped by structural connectivity (SC) between brain systems, i.e., by the actual bundles of white matter fiber connecting neurons [15] . As a first approximation, SC can be inferred from fiber tractography based on diffusion-weighted imaging (DWI) [16] – [19] . A recent study [20] , which focused on a small subset of robustly estimated structural connections, demonstrated the existence of a statistical, yet complex, correspondence between FC and specific features of SC (e.g., low vs. high fiber density, short vs. long fibers, intra vs. interhemispheric connections). However, a large part of FC cannot be explained by SC alone [21] . There appears that FC is the result of at least two main contributing factors: (i) the underlying anatomical structure of connectivity, and (ii) the dynamics of neuronal populations emerging from their physiology [3] . A key issue is to better understand the relative contributions of these two components to FC. Besides, recent studies using windowed analyses have suggested that FC estimated over an entire acquisition session (referred to as ‘stationary FC’ in the literature) breaks down into a variety of reliable correlation patterns (also referred to as ‘dynamic FC’ or ‘non-stationarities’) when estimated over short time windows (30 s) [14] , [22] . The authors advocated that FC estimated over short time windows (or windowed FC, for short) mostly reflects recurrent transitory patterns that are aggregated when estimating FC over a whole session. They further suggested that whole-session FC may only be an epiphenomenon without clear physiological underpinning, and not the reflection of an actual process with stationary FC [14] . This perspective remains to be reconciled with the fact that whole-session FC has been found to be highly reproducible, functionally meaningful and a useful biomarker in many pathological contexts [23] , [24] . Note that, in the recent literature of fMRI data analysis, stationarity implicitely refers to a stationary FC (i.e., the invariance of FC over time), to be contrasted with the more general notion of (strong) stationarity, where a model or process is stationary if its parameters remain constant over time [25] , [26] . SC being temporally stable at the scale of a whole resting state fMRI session (typically 10 min), we could expect SC to drive a stationary process (in the strong sense). Since SC is furthermore expected to drive FC, we can hypothesize that this stationary process contributes to generate a stationary FC.

In order to bring together the structural and dynamical components underlying FC, some studies have used computational models that incorporate SC together with biophysical models of neuronal activity to generate coherent brain dynamics [27] – [32] . This approach has yielded promising results for the understanding of the relationship between structure and function [17] , [33] , [34] . Here, we used a testbed of well-established generative models simulating neuronal dynamics combined with empirical measures, to investigate the relative contributions of anatomical connections, stationary dynamics, and non-stationarities to the emergence of empirical functional connectivity. In particular, we considered the following hypotheses: (H1) part of FC directly reflects SC; (H2) models of physiological mechanisms added to SC increase predictive power all the more as they are complex; (H3) part of the variance of FC that is unexplained by models is due to issues in the estimation of SC, e.g., problems with measuring homotopic connections; (H4) there is an actual stationary process reflected in whole-session FC that is not merely an artifact but substantially reflects the driving of the dynamics by SC.

In order to test these hypotheses and estimate the relative contribution of anatomy, stationary dynamics and non-stationarities to FC, we relied on the following approach. After T 1 -weighted MRI based parcellation of the brain ( N = 160 regions), SC was estimated using the proportion of white matter fibers connecting pairs of regions, based on probabilistic tractography of DWI data [35] . FC was measured on rs-fMRI data using Pearson correlation between the time courses of brain regions. We quantified the correlation between SC alone and FC as a reference, and also fed SC to generative neurodynamical models of increasing complexity: a spatial autoregressive (SAR) model [36] , analytic models with or without conduction delays [28] – [31] , [37] , and biologically constrained models [29] , [32] . Importantly, all these models were used in their stationary regime in the strong sense, since their parameters were not changed during the simulations. Of these models, only the SAR is explicitely associated with a stationary FC; other, more complex models, generate dynamics that are compatible with a non-stationary FC. We computed FC from data simulated by these models and compared the results to empirical FC. For each model, performance was quantified using predictive power [29] , for each subject as well as on the ‘average subject’ (obtained by averaging SC and empirical FC across subjects). Values for the model parameters were based on the literature, except for the structural coupling strength that was optimized in order to maximize each model's performance.

Predictive power of models

In agreement with H1, SC explained a significant amount of the variance of whole-session FC for all subjects, as did all generative models (permutation test, p <0.05 corrected) ( Figure 1 , panel A). Generative models predicted FC better than SC alone (paired permutation test, p <0.05 corrected). Predictive power obtained with the average subject ranged from 0.32 for SC alone to 0.43 for the SAR model ( Table 1 ). For a given model, predictive power was reproducible across subjects. Contrary to our hypothesis H2, generative models had similar performance, and complexity was not predictive of performance. The results remained unchanged when no global signal regression was applied ( Figure S1 ). Also, findings were found to be similar for SC alone and the SAR model at finer spatial scales ( N = 461 and N = 825 regions, Figure S2 ) and consistent with a replication dataset ( Figure S3 ). Most importantly, a large part of the variance ( R 2 ) in the empirical data (at least 82%) remained unexplained by this first round of simulations.

PPT PowerPoint slide
PNG larger image
TIFF original image

(A) Predictive power for all connections and when restricted to intra/interhemispheric, direct/indirect connections. For each type of connections and each model, we represented the individual predictive powers (bar chart representing means and associated standard deviations), as well as the predictive power for the average subject computed using the original SC (diamonds), or after adding homotopic connections (circles). Of note, SC alone has no predictive power (zero) for the subset of indirect connections, by definition. (B) Patterns of SC, empirical FC and FC simulated from the SAR model for the average subject and associated scatter plot of simulated versus empirical FC (solid line represents perfect match). SARh stands for the SAR model with added homotopic connections. Matrices were rearranged such that network structure is highlighted. Homologous regions were arranged symmetrically with respect to the center of the matrix; for instance, the first and last regions are homologous. (C) Similarity of functional brain networks across subjects (bar chart with means and associated standard deviations), for the average subject (diamonds), and when adding homotopic connections (circles) (left). Network maps for the average subject and empirical FC, as well as for FC simulated using either the SAR model with original SC or the SARh.

https://doi.org/10.1371/journal.pcbi.1003530.g001

https://doi.org/10.1371/journal.pcbi.1003530.t001

Role of homotopic connections

We reasoned (see hypothesis H3) that part of the unexplained variance could reflect issues with the estimation of SC from DWI, which can be expected because of limitations in current fiber tracking algorithms and the problem of crossing fibers [38] . We know for instance that many fibers passing through the corpus callosum are poorly estimated in diffusion imaging, in particular those connecting more lateral parts of the cerebral cortex [39] . Yet, the corpus callosum is the main interhemispheric commissure of the mammal brain, see [40] . It systematically connects homologous sectors of the cerebral cortex across the two hemispheres in a topographically organized manner, with an antero-posterior gradient, through a system of myelinated homotopic fibers or ‘homotopic connections’. The hypothesis of an impact of SC estimation problems on FC unexplained variance was supported by the observation that, in our results, intrahemispheric connections yielded on average a much higher predictive power (e.g., 0.59 for the SAR model) than interhemispheric connections (0.16 for the SAR model).

In order to further test the role of white matter connections in driving FC, we artificially set all homotopic connections to a constant SC value (0.5) for the average subject and reran all simulations. As a result, the predictive power strongly increased for all models ( Figure 1 , panels A and B), ranging from 0.39 for SC alone to 0.61 for the SAR model ( Table 1 ). Thus the variance unexplained (1- R 2 ) was reduced to 63%. Moreover, predictive power for intra and interhemispheric connections became equivalent (0.60 and 0.62, respectively). Interestingly, adding homotopic connections also led to a substantial increase in predictive power for indirect connections, that is, pairs of regions for which SC is zero (increasing from 0.07 to 0.45). The effect of adding interhemispheric anatomical connections on increasing predictive power was highly specific to homotopic connections. When applying the SAR model to the SC matrix with added homotopic connections and randomly permuting (10 000 permutations) the 80 corresponding interhemispheric connections (one region in one hemisphere was connected to one and only one region in the other hemisphere), the predictive power strongly decreased, even compared to results with the original SC ( Figure 2 , panel A). Moreover, we further assessed the specificity of this result by systematically manipulating SC. In three different simulations, we randomly removed, added, and permuted structural connections (10 000 times). In all cases, the predictive power decreased as a function of the proportion of connections manipulated ( Figure 2 , panel B). Moreover, changes induced by these manipulations remained small (<0.05), far below the changes that we were able to induce by adding homotopic connections. All in all, these results suggest that homotopic connections play a key role in shaping the network dynamics, in a complex and non-trivial manner.

(A) Predictive power of the SAR model with original SC (green), when adding homotopic connections (‘SARh’, red), or with shuffled homotopic connections (black). (B) Predictive power of the SAR model with original SC (red) and when SC values were randomly permuted, removed or added (from left to right). For each graph, predictive power was quantified as a function of the percentage of connections manipulated.

https://doi.org/10.1371/journal.pcbi.1003530.g002

Predicting functional brain networks

Beyond predicting the overall pattern of FC, we also assessed whether models could predict the empirical organization of FC into a set of intrinsic networks. Connectivity matrices were clustered into groups of non-overlapping brain regions showing high within-group correlation and low between-group correlation, and the resulting partitions into functional brain networks were compared between empirical and simulated FC using the adjusted Rand index (see Methods ). Again, the SAR model tended to perform best among all computational models ( Figure 1 , panel C).

Without adding homotopic connections in the SC matrix, the simulated networks highly differed from the empirical networks. In particular, most networks were found to be lateralized. After adding homotopic connections, the resemblence between simulated and empirical networks greatly improved. Networks were more often bilateral and overall consistent with the topography of empirical functional networks, including somatosensory, motor, visual, and associative networks. High FC between the amygdala and ventral-lateral sectors of the prefrontal cortex was also correctly predicted by the simulations. There were nevertheless some notable differences. First, the clustering of empirical FC yielded a long-range fronto-parieto-temporal association network ( Figure 1 , panel C, cyan) that was not observed in simulated FC as such. Second, a parieto-temporal cluster ( Figure 1 , panel C, red), which was associated with thalamo-striatal networks, was predicted by simulations but was not present in the empirical data. Third, a cluster encompassing the entire cingulate gyrus and precuneus ( Figure 1 , panel C, green) was predicted by simulations but was broken down into more clusters in the empirical data.

Stationary FC, non-stationary FC, and non-stationarities

The results above show that SC plays a causal role in FC, but one can still wonder what aspects of the underlying dynamics are the most directly related to this influence. A hypothesis is that SC, in combination with stable physiological processes (e.g., overall gain in synaptic transmission), drives a stationary regime of the dynamics. This hypothesis is supported by the finding that all models tested in this study, which were used in a stationary regime (in the strong sense), could explain significantly more variance than SC alone. Furthermore, the fact that the SAR could predict FC significantly better than all other models is evidence that this stationary regime is associated with stationary FC (paired permutation test, p <0.05 corrected).

But, clearly, many variations in the dynamical patterns of brain activity, be it in the process of spontaneous cognition, physiological regulation, or context-dependent changes, cannot be expected to be associated with a purely stationary FC. Modeling how the brain dynamics deal with endogenous and environmental contexts should require more complex models, either stationary or non-stationary, that are able to generate non-stationary (i.e., time-varying) patterns of FC. Given that at best 37% of the variance could be explained by the model of a purely stationary FC (the SAR), we can wonder why the models of higher complexity used in our simulation testbed did not perform better in predicting FC. One possible hypothesis is that the SAR model was favored in the simulations, because we estimated FC over about 10 minutes of actual brain dynamics. In such configuration, we can imagine that the non-stationarities of FC cancel out, the estimation effectively keeping the stationary part of FC. We thus wondered whether the more complex models would better perform when non-stationary FC had the potential of being more strongly reflected in the data. We approached this question by computing predictive power on windowed FC as a function of the length of the time-window used [22] , for all possible time-windows over which FC could be estimated and for all models. We also investigated the effect of simulation duration (see Methods ). We found that the relative performance of more complex models was still lower than that of the SAR model ( Figures 3 and S4 ). The average predictive power was lower for shorter time-windows and increased towards a limit for longer time-windows. The SAR model behaved like an ‘upper-bound’ for predictive power. The performance of all other models, irrespective of the size of the time-window, was between that of SC alone and that of the SAR model.

Predictive power as a function of time-window length across subjects (left) and of duration of simulated runs on the average subject (right). For color code see Figure 1 .

https://doi.org/10.1371/journal.pcbi.1003530.g003

A straightforward explanation is that the non-stationary patterns of FC, as generated by the simulation models, did not match the non-stationary patterns of the empirical FC as they unfolded during the acquisition in the brain of the participants. Context-dependent and transient dynamics are likely to be missed by models of the dynamics that cannot be contextually constrained in the absence of further information. It is thus difficult to infer how much of the 63% of unexplained variance remaining in whole-session FC actually reflect physiologically meaningful non-stationary FC, and more broadly, non-stationary dynamics.

In the present study, we investigated the respective contributions of anatomical connections, stationary dynamics, and non-stationarities to the emergence of empirical functional connectivity. We compared the performance of computational models in modeling FC and manipulated SC in order to analyze the impact of SC on FC, with and without the filter of combined physiological models of the dynamics.

The importance of white matter fiber pathways in shaping functional brain networks is a known fact, for a review, see [15] , [17] , [21] , [23] . Previous modeling studies have supported the importance of the underlying anatomical connections, i.e., SC, in shaping functional relationships among brain systems [16] , [41] , [42] . In agreement with our hypothesis H1, we showed that functional connectivity could at least in part be explained by structural connectivity alone. Adding homotopic connections in the matrix of SC, we found a slight increase in explained variance when considering the prediction of whole-session FC from SC alone (+4% of explained variance). In agreement with H2, adding models of physiological interactions above and beyond SC alone increased the explained variance in whole-session FC, by 8% for the best performing model, the SAR model, when no homotopic connections were added, and by 22% when homotopic connections were added. This latter fact, which strongly supports H3, suggests a complex interplay between anatomy as reflected by SC and physiological mechanisms in generating FC. This impact of SC manipulations on predicted FC pertained not only to direct but also to indirect connections. For indirect connections, whole-session FC was much better predicted after adding homotopic connections to SC than before adding them (0.45 versus 0.07 in predictive power). The problem of limited predictive power for FC based on SC when considering indirect connections has puzzled the field [43] . For this reasons many studies only assess the performance of models on direct connections. Here, we showed that a major factor in driving FC for indirect anatomical connections (+20% in explained variance) is the interplay between a subset of anatomical connections, i.e., homotopic connections (which are typically underestimated by DWI), and physiological parameters that generate the dynamics underlying FC, themselves conditioned by the possible interactions defined by SC.

Contrary to our expectation (see hypothesis H2), all models tended to perform similarly, irrespective of model complexity. The best performing model in most cases was the SAR model, a model of stationary FC driven by SC, with 63% of the variance remaining unexplained. It is likely that, above and beyond problems with the estimation of SC from DWI, and other incompressible sources of irrelevant noise, much of the unexplained variance in FC relates to non-stationary patterns in FC, and more generally to non-stationarities in the strong-sense. Such non-stationarities are difficult to model in the experimentally unconstrained resting-state and in the absence of further information regarding the specific parameters shaping FC. Irrespective of their complexity, computational models are only capable of generating prototypical brain activities, and not the subject-dependent activity that took place in the brain of the participants during scanning. The scientific necessity of modeling brain dynamics is hindered by such uncertainty and it will be a challenge to find solutions to approach this problem [26] , [44] . Even though one objective for neuroscience is to propose generative models that are capable of generating detailed neuronal dynamics, generative models cannot be informed by this unknown context and, as a consequence, cannot generate context-dependent activity in a manner that would be predictive of empirical data, in the absence of additional measures and experimental controls. Nevertheless, and perhaps for that very reason, the study of non-stationarity in FC should become of central interest for the field, as such non-stationarities could explain much of FC (up to 63% according to our simulation results), and thus reflect critical mechanisms for neurocognitive processing.

In the absence of adequate modeling principles, determining the precise contribution of non-stationarities to the unexplained variance in FC is impossible, as other confounding sources of unexplained variance are expected. As we showed, even naive manipulations aimed at estimating the impact of the known errors in DWI-based reconstruction of homotopic connections showed that such errors could cause 20% of the unexplained variance in predicting empirical FC. How DWI and fiber tracking should be used for an optimal estimation of structural connectivity is still a topic of intense debates [45] – [48] . It is likely that part of the unexplained variance in predicting FC will be reduced as better estimates of SC become available.

The model showing the best results, the SAR model, explicitly modeled a stationary process with a stationary FC. In line with our hypothesis H4, empirical FC is likely to incorporate stationary components driven by SC. Further knowledge about this stationary process might be gained by analyzing FC computed over much longer periods of time than is commonly performed (e.g., hours versus minutes). This stationary process is itself likely to be only locally stationary, as it might be expected that slow physiological cycles, from nycthemeral cycles to hormonal cycles, development, learning and aging, will modify the parameters controlling it.

In the present study, we did not take into account the statistical fluctuations induced by the fact that the time series were of finite length. Such a finiteness entails that even a model that is stationary in the strong sense could generate sample moments that fluctuate over time. For instance, the sample sum of square of a multivariate normal model with covariance matrix Σ computed from time series of size N is not equal to N Σ but is Wishart distributed with N -1 degrees of freedom and scale matrix Σ. This phenomenon will artificially increase the part of variance that cannot be accounted for by stationary models and, hence, play against stationary models. Since it is conversely very unlikely for a non-stationary model to generate sample moments that are constant over time, statistical fluctuations cannot at the same time artificially increase the part of variance that can be accounted for by stationary models. As a consequence, not considering these statistical fluctuations made us underestimate the part of variance that can be accounted for by models that are stationary in the strong sense. In other words, our estimate of the part of variance accounted for by a stationary model is a lower bound for the true value. We can therefore be confident that taking statistical fluctuations into account will only strengthen H4.

Our goal here was to investigate how current generative models of brain acticity fare in predicting the relationship between structure and function. The complexity of some of these models was such that the simulations included here were only possible thanks to a computer cluster. The behavior of all these models depends on the values of some parameters and, in the present study, we set these parameters in agreement with the literature. In what measure this choice affects how well models predict FC is unclear. Yet a full investigation of this issue remains beyond the scope of this study, since parameter optimization through extensive exploration of the parameter space for all models is at this stage unrealistic. Nevertheless, in order to get a sense of the sensitivity of our results to parameter values in a way that is compatible with the computational power available, we explored the behavior of the Fitzhugh-Nagumo, Wilson and Kuramoto models over a subset of the parameter space (see Figures S5 and S6 ). We found that parameter values had little influence on predictive power, which, in all cases, remained below that of the SAR, the simplest model tested.

We formulated H2 to test for the existence of a relationship between complexity and realism in the models that we used. Indeed, there should exist a very tight connection between the two, since the more complex generative models in our study have been designed to take biophysical mechanisms into account, with parameters that are physiologically relevant and values often chosen based on prior experimental results. Now, realism usually comes at the cost of complexity. As a consequence, it is often (implicitely) assumed that, among the models we selected, the more complex a model is, the more realistic it will also be and the better it will fit the data. This is the reason why we stated H2, based on such rationale inspired from the literature, in order to put such hypothesis to the test. The results show that for the models we used, with their sets of parameters, an increase in complexity was not associated with an increase in performance. This suggests that, for these models, complexity and realism are not quite as tightly connected as expected.

Given that the SAR model is the only model that does not include a step of hemodynamic modeling (Balloon-Windkessel), it cannot be ruled out that the superiority of the SAR reflects issues with this step. In order to check that this is not the case, we computed predictive power for all models without the hemodynamic model. The predictive power was largely insensitive to the presence of the hemodynamic model (see Figure S7 ). In particular, the SAR model remained overall an upper bound in terms of predictive power.

Finally, we should note that we relied on a definition of SC restricted to the white matter compartment. Although this is standard in the field, in reality, local intrinsic SC exists in the gray matter. However, current models generally make prior assumptions about such SC. Moreover, intrinsic SC currently remains impossible to measure reliably for the entire brain.

In spite of the complexity of the problems and the limitations of current modeling approaches, computational modeling of large-scale brain dynamics remains an essential scientific endeavor. It is key to better understand generative mechanisms and make progress in brain physiology, physiopathology and, more generally, theoretical neuroscience. It is also central to the endeavor of searching for accurate and meaningful biomarkers in aging and disease [49] . Moreover, computational modeling of FC opens the possibility of making inference on specific biophysical parameters, including inference about the underlying anatomical connectivity itself. In spite of their limited predictive powers, simpler models can be useful in this context. The SAR model, introduced in [36] , may appear well-suited to model essential stationary aspects of the generative mechanisms of FC. One interest of such a simple and analytically tractable model is that, beyond its very low computational burden, it could be the basis for straightforward estimation of the model parameters that can be used to compare clinical populations, and could constitute a potentially important biomarker of disease.

Ethics statement

All participants gave written informed consent and the protocol was approved by the local Ethics Committee of the Pitié-Salpêtrière Hospital (number: 44-08; Paris, France).

Twenty-one right-handed healthy volunteers were recruited within local community (11 males, mean age 22±2.4 years). Data were acquired using a 3 T Siemens Trio TIM MRI scanner (CENIR, Paris, France). For acquisition and preprocessing details, see Text S1 . For each subject, the preprocessing yielded three matrices: one of SC, one with the average fiber lengths, and one of empirical FC. These matrices were also averaged across subjects (‘average subject’).

Simulations

We used eight generative models with various levels of complexity: the SAR model, a purely spatial model with no dynamics that expresses BOLD fluctuations within one region as a linear combination of the fluctuations in other regions; the Wilson-Cowan system, a model expressing excitatory and inhibitory neuronal populations activity; the two rate models (with or without conduction delays), simplified versions of the Wilson-Cowan system obtained by considering exclusively the excitatory population; the Kuramoto model, which simulates neuronal activity using oscillators; the Fitzhugh-Nagumo model, which aims at reproducing complex behaviors such as those observed in conductance-based models; the neural-mass model, also based on conductance and with strong biophysiological constraints; and finally, the model of spiking neurons, the most constrained model in the current study which models neuron populations as attractors. For more details, see Text S2 .

All models took an SC matrix as input, and all but the SAR were taken as models of neuronal (rather than BOLD) activity. Simulated fMRI BOLD signal was obtained from simulated neuronal activity by means of the Balloon-Windkessel hemodynamic model [50] , [51] . Global mean signal was then regressed out from each region's time series. Finally, simulated FC was computed as Pearson correlation between simulated time series. For the SAR model, we directly computed simulated FC from the analytical expression of the covariance, see Equation (2) in Text S2 . All models had a parameter that represents the coupling strength between regions. This parameter was optimized separately for each model on the average subject to limit computational burden ( Text S2 ). After optimization, we generated three runs of 8 min BOLD activity and averaged the corresponding FCs to obtain the simulated FC for each dynamical model and each subject. For the average subject, simulated FC was obtained by feeding the average SC matrix to the different models.

Performance

Modeling performance was assessed using predictive power and similarity of spatial patterns. Predictive power was quantified for each subject and for the average subject by means of Pearson correlation between simulated and empirical FC [29] . Regarding the similarity of functional brain networks, SC, empirical FC and simulated FC were decomposed into 10 networks using agglomerative hierarchical clustering and generalized Ward criterion [52] . The resulting networks from SC and simulated FC were compared to the ones resulting from empirical FC using the adjusted Rand index [53] , [54] . The Rand index quantifies the similarity between two partitions of the brain into networks by computing the proportion of pairs of regions for which the two partitions are consistent (i.e., they are either in the same network for both partitions, or in a different network for both partitions). The adjustment accounts for the level of similarity that would be expected by chance only.

Analysis of dynamics

Empirical and simulated windowed FC were computed on individual subjects using sliding time-windows (increment of 20 s) of varying length (from 20 to 420 s by step of 20 s). Predictive power was computed as the correlation between any pair of time-windows of equal length corresponding to simulated and empirical windowed FC, respectively. This approach was only applied to the dynamical models; for SC alone and the SAR model, simulated FC remained, by definition, constant through time and, as a consequence, windowed FC was equaled to whole-session FC. The influence of simulated run duration on predictive power was also investigated. For each model, three runs of one hour were simulated on the average subject. Predictive power was then computed as a function of simulated run duration. For the same reason as above, SC alone and the SAR model did not depend on simulation duration.

Supporting Information

Performance of computational models when no global signal regression was performed. (A) Predictive power for all connections and when restricted to intra/interhemispheric, direct/indirect connections. For each type of connections and each model, we represented the individual predictive powers (bar chart representing means and associated standard deviations), as well as the predictive power for the average subject computed using the original SC (diamonds), or after adding homotopic connections (circles). Of note, SC alone has no predictive power (zero) for the subset of indirect connections, by definition. (B) Patterns of SC, empirical FC and FC simulated from the SAR model for the average subject and associated scatter plot of simulated versus empirical FC (solid line represents perfect match). SARh stands for the SAR model with added homotopic connections. Matrices were rearranged such that network structure is highlighted. Homologous regions were arranged symmetrically with respect to the center of the matrix; for instance, the first and last regions are homologous. (C) Similarity of functional brain networks across subjects (bar chart with means and associated standard deviations), for the average subject (diamonds), and when adding homotopic connections (circles) (left). Network maps for the average subject and empirical FC, as well as for FC simulated using either the SAR model with original SC or the SARh.

https://doi.org/10.1371/journal.pcbi.1003530.s001

Performance of SC alone and the SAR model at finer spatial scales. Predictive power for all connections and when restricted to intra/interhemispheric, direct/indirect connections. For each type of connections and each model, we represented the individual predictive powers (bar chart representing mean and associated standard deviation), as well as the predictive power of the average subject computed using the original SC (diamonds), or after adding homotopic connections (circles).

https://doi.org/10.1371/journal.pcbi.1003530.s002

Performance of computational models on the replication dataset. The replication dataset was from the study of Hagmann and colleagues [55] . Brain network was defined at low anatomical granularity (N = 66 regions), and connectivity measures were averaged over five healthy volunteer subjects. (A) Predictive power for all connections and when restricted to intra/interhemispheric, direct/indirect connections. For each type of connections and each model, we represented the individual predictive powers (bar chart representing means and associated standard deviations), as well as the predictive power for the average subject computed using the original SC (diamonds), or after adding homotopic connections (circles). Of note, SC alone has no predictive power (zero) for the subset of indirect connections, by definition. (B) Patterns of SC, empirical FC and FC simulated from the SAR model for the average subject and associated scatter plot of simulated versus empirical FC (solid line represents perfect match). SARh stands for the SAR model with added homotopic connections. Matrices were rearranged such that network structure is highlighted. Homologous regions were arranged symmetrically with respect to the center of the matrix; for instance, the first and last regions are homologous. (C) Similarity of functional brain networks across subjects (bar chart with means and associated standard deviations), for the average subject (diamonds), and when adding homotopic connections (circles) (left). Network maps for the average subject and empirical FC, as well as for FC simulated using either the SAR model with original SC or the SARh.

https://doi.org/10.1371/journal.pcbi.1003530.s003

Effect of time on performance. Predictive power of computational models as a function of the time-window length for each subject (graphs) and model (color).

https://doi.org/10.1371/journal.pcbi.1003530.s004

Exploration of the parameter space for the Fitzhugh-Nagumo model. (Left) Phase diagrams (i.e., x - y plane) for an uncoupled model ( k = 0) over various parameter values of α and β . The model operate mostly in an oscillatory regime for the range of parameter values investigated. (Right) Predictive power as a function of α and β . The black dot represents the parameter set used in our simulations, while the black square corresponds to the values from [28] . The values used in our simulations gave rise to higher predictive power than the parameters values from [28] . In any case, for the range of parameters considered, the predictive power always remained lower than that obtained with a SAR model.

https://doi.org/10.1371/journal.pcbi.1003530.s005

Effect of velocity on predictive power. Predictive power as a function of the coupling strength and velocity values in generative models. Black dots represent values used for subsequent simulations. These simulations show that the predictive power is little influenced by velocity. In any case, for the range of parameters considered, the predictive power also always remained lower than that obtained with a SAR model.

https://doi.org/10.1371/journal.pcbi.1003530.s006

Effect of the hemodynamic model. Predictive power for all connections and when restricted to intra/interhemispheric, direct/indirect connections. For each type of connections and each model, we represented the predictive power for the average subject computed using the BOLD signal (diamonds, solid line) or using the neuronal activity (circles, dashed line). Of note, the prediction differs slightly from that of the Figure 1 due to the stochastic component of most models at each run.

https://doi.org/10.1371/journal.pcbi.1003530.s007

Data and preprocessing.

https://doi.org/10.1371/journal.pcbi.1003530.s008

Computational models.

https://doi.org/10.1371/journal.pcbi.1003530.s009

Acknowledgments

The authors are thankful to Olaf Sporns (Department of Psychology, Princeton University, Princeton, USA) and Christopher J. Honey (Department of Psychological and Brain Sciences, Indiana University, Bloomington, USA) for providing the neural-mass model; to Gustavo Deco, Étienne Hugues and Joanna Cabral (Computational Neuroscience Group, Department of Technology, Universitat pompeu Fabra, Barcelona, Spain) for providing the Kuramoto and rate models as well as the spike model; and to Olaf Sporns and Patric Hagmann (Department of Radiology, University Hospital Center and University of Lausanne, Lausanne, Switzerland) for sharing their data for replication. We would also like to thank them for fruitful discussions. The authors are grateful to Stéphane Lehéricy and his team (Center for Neuroimaging Research, Paris, France) for providing them with the data, and especially to Romain Valabrègue for his help in handling coarse-grained distributed parallelization of computational tasks.

Author Contributions

Conceived and designed the experiments: AM DR HB GM. Performed the experiments: AM DR GM. Analyzed the data: AM. Wrote the paper: AM DR HB GM.

View Article
Google Scholar
40. Schmahmann JD, Pandya DN (2006) Fiber pathways of the brain. Oxford University Press.
52. Batageli V (1988) Generalized ward and related clustering problems. In: Bock HH, editor. Classification and Related Methods of Data Analysis. North-Holland. pp. 67–74.

Experimental Brain Research

Founded in 1966, Experimental Brain Research publishes original contributions on many aspects of experimental research of the central and peripheral nervous system. The scope of the journal encompasses neurophysiological mechanisms underlying sensory, motor, and cognitive functions in health and disease. This includes developmental, cellular, molecular, computational and translational neuroscience approaches. The journal does not normally consider manuscripts with a singular specific focus on pharmacology or the action of specific natural substances on the nervous system.

Experimental Brain Research is pleased to offer Registered Reports for authors submitting to the journal. Please see the full instructions here: https://www.springer.com/journal/221/updates/19883440

Article Types: Original Paper, Review, Mini-Review, Registered Reports, Case Study, Letter to the Editor Before submitting a manuscript, please consult "Submission Guidelines" at www.springer.com/221.

Winston Byblow,
Bill J. Yates

Latest issue

Volume 242, Issue 3

Latest articles

Unveiling the invisible: receivers use object weight cues for grip force planning in handover actions.

L. Kopnarski
C. Voelcker-Rehage

Sleep deprivation induces late deleterious effects in a pharmacological model of Parkinsonism

L. B. Lopes-Silva
D. M. G. Cunha
R. H. Silva

A computational model of motion sickness dynamics during passive self-motion in the dark

Aaron R. Allred
Torin K. Clark

Inhibition of fatty acid amide hydrolase reverses aberrant prefrontal gamma oscillations in the sub-chronic PCP model for schizophrenia

Alexandre Seillier

Effects of the perceived temporal distance of events on mental time travel and on its underlying brain circuits

Claudia Casadio
Ivan Patané
Francesca Benuzzi

Journal updates

Join experimental brain research as an associate editor (neuropsychology).

Experimental Brain Research is recruiting a new Associate Editor to serve on the editorial team. We welcome applications from researchers with expertise in neuropsychology, cognitive neuroscience and neurorehabilitation.

Important Update

It is with profound sadness that we announce that Dr. Francesca Frassinetti, an Associate Editor for Experimental Brain Research and Professor at Università di Bologna, died suddenly on February 6 in an automobile accident.

Follow us on Twitter

Thanks to our reviewers

A special note of thanks to our reviewers in the first half of 2020.

Journal information

Biological Abstracts
CAB Abstracts
Chemical Abstracts Service (CAS)
Current Contents/Life Sciences
Google Scholar
Japanese Science and Technology Agency (JST)
Norwegian Register for Scientific Journals and Series
OCLC WorldCat Discovery Service
Pathway Studio
Science Citation Index Expanded (SCIE)
TD Net Discovery Service
UGC-CARE List (India)

Rights and permissions

Springer policies

Find a journal
Publish with us
Track your research

MyU : For Students, Faculty, and Staff

New study reveals breakthrough in understanding brain stimulation therapies

For the first time, researchers show how the brain can precisely adapt to external stimulation.

MINNEAPOLIS/ST. PAUL (03/14/2024) — For the first time, researchers at the University of Minnesota Twin Cities showed that non-invasive brain stimulation can change a specific brain mechanism that is directly related to human behavior. This is a major step forward for discovering new therapies to treat brain disorders such as schizophrenia, depression, Alzheimer’s disease, and Parkinson’s disease.

The study was recently published in Nature Communications , a peer-reviewed, open access, scientific journal.

Researchers used what is called “transcranial alternating current stimulation” to modulate brain activity. This technique is also known as neuromodulation. By applying a small electrical current to the brain, the timing of when brain cells are active is shifted. This modulation of neural timing is related to neuroplasticity, which is a change in the connections between brain cells that is needed for human behavior, learning, and cognition.

“Previous research showed that brain activity was time-locked to stimulation. What we found in this new study is that this relationship slowly changed and the brain adapted over time as we added in external stimulation,” said Alexander Opitz, University of Minnesota biomedical engineering associate professor. “This showed brain activity shifting in a way we didn’t expect.”

This result is called “neural phase precession.” This is when the brain activity gradually changes over time in relation to a repeating pattern, like an external event or in this case non-invasive stimulation. In this research, all three investigated methods (computational models, humans, and animals) showed that the external stimulation could shift brain activity over time.

“The timing of this repeating pattern has a direct impact on brain processes, for example, how we navigate space, learn, and remember,” Opitz said.

The discovery of this new technique shows how the brain adapts to external stimulation. This technique can increase or decrease brain activity, but is most powerful when it targets specific brain functions that affect behaviors. This way, long-term memory as well as learning can be improved. The long-term goal is to use this technique in the treatment of psychiatric and neurological disorders.

Opitz hopes that this discovery will help bring improved knowledge and technology to clinical applications, which could lead to more personalized therapies for schizophrenia, depression, Alzheimer’s disease, and Parkinson’s disease.

In addition to Opitz, the research team included co-first authors Miles Wischnewski and Harry Tran. Other team members from the University of Minnesota Biomedical Engineering Department include Zhihe Zhao, Zachary Haigh, Nipun Perera, Ivan Alekseichuk, Sina Shirinpour and Jonna Rotteveel. This study was in collaboration with Dr. Jan Zimmermann, associate professor in the University of Minnesota Medical School.

This work was supported primarily by the National Institute of Health (NIH) along with the Behavior and Brain Research Foundation and the University of Minnesota’s Minnesota’s Discovery, Research, and InnoVation Economy (MnDRIVE) Initiative. Computational resources were provided by the Minnesota Supercomputing Institute (MSI).

To read the entire research paper titled, “Induced neural phase precession through exogenous electric fields”, visit the Nature Communications website .

Researchers using external stimulation during experiment

Rhonda Zurn, College of Science and Engineering, [email protected]

University Public Relations, [email protected]

NIH studies find severe symptoms of “Havana Syndrome,” but no evidence of MRI-detectable brain injury or biological abnormalities

Compared to healthy volunteers, affected U.S. government personnel did not exhibit differences that would explain symptoms.

Using advanced imaging techniques and in-depth clinical assessments, a research team at the National Institutes of Health (NIH) found no significant evidence of MRI-detectable brain injury, nor differences in most clinical measures compared to controls, among a group of federal employees who experienced anomalous health incidents (AHIs). These incidents, including hearing noise and experiencing head pressure followed by headache, dizziness, cognitive dysfunction and other symptoms, have been described in the news media as “Havana Syndrome” since U.S. government personnel stationed in Havana first reported the incidents. Scientists at the NIH Clinical Center conducted the research over the course of nearly five years and published their findings in two papers in JAMA today.

“Our goal was to conduct thorough, objective and reproducible evaluations to see if we could identify structural brain or biological differences in people who reported AHIs,” said Leighton Chan, M.D., chief, rehabilitation medicine and acting chief scientific officer, NIH Clinical Center, and lead author on one of the papers. “While we did not identify significant differences in participants with AHIs, it’s important to acknowledge that these symptoms are very real, cause significant disruption in the lives of those affected and can be quite prolonged, disabling and difficult to treat.”

Researchers designed multiple methods to evaluate more than 80 U.S. government employees and their adult family members, mostly stationed abroad, who had reported an AHI and compared them to matched healthy controls. The control groups included healthy volunteers who had similar work assignments but did not report AHIs. In this study, participants underwent a battery of clinical, auditory, balance, visual, neuropsychological and blood biomarkers testing. In addition, they received different types of MRI scans aimed at investigating volume, structure and function of the brain.

In this study, researchers obtained multiple measurements and used several methods and models to analyze the data. This was done to ensure the findings were highly reproducible, meaning similar results were found regardless of how many times participants were evaluated or their data statistically analyzed. Scientists also used deep phenotyping, which is an analysis of observable traits or biochemical characteristics of an individual, to assess any correlations between clinically reported symptoms and neuroimaging findings.

For the imaging portion of the study, participants underwent MRI scans an average of 80 days following symptom onset, although some participants had an MRI as soon as 14 days after reporting an AHI. Using thorough and robust methodology, which resulted in highly reproducible MRI metrics, the researchers were unable to identify a consistent set of imaging abnormalities that might differentiate participants with AHIs from controls.

“A lack of evidence for an MRI-detectable difference between individuals with AHIs and controls does not exclude that an adverse event impacting the brain occurred at the time of the AHI,” said Carlo Pierpaoli, M.D., Ph.D., senior investigator and chief of the Laboratory on Quantitative Medical Imaging at the National Institute of Biomedical Imaging and Bioengineering, part of NIH, and lead author on the neuroimaging paper. “It is possible that individuals with an AHI may be experiencing the results of an event that led to their symptoms, but the injury did not produce the long-term neuroimaging changes that are typically observed after severe trauma or stroke. We hope these results will alleviate concerns about AHI being associated with severe neurodegenerative changes in the brain.”

Similarly, there were no significant differences between individuals reporting AHIs and matched controls with respect to most clinical, research and biomarker measures, except for certain self-reported measures. Compared to controls, participants with AHIs self-reported significantly increased symptoms of fatigue, post-traumatic stress and depression. Forty-one percent of participants in the AHI group, from nearly every geographic area, met the criteria for functional neurological disorders (FNDs), a group of common neurological movement disorders caused by an abnormality in how the brain functions, or had significant somatic symptoms. FNDs can be associated with depression and anxiety, and high stress. Most of the AHI group with FND met specific criteria to enable the diagnosis of persistent postural-perceptual dizziness, also known as PPPD. Symptoms of PPPD include dizziness, non-spinning vertigo and fluctuating unsteadiness provoked by environmental or social stimuli that cannot be explained by some other neurologic disorder.

“The post-traumatic stress and mood symptoms reported are not surprising given the ongoing concerns of many of the participants,” said Louis French, Psy.D., neuropsychologist and deputy director of the National Intrepid Center of Excellence at Walter Reed National Military Medical Center and a co-investigator on the study. “Often these individuals have had significant disruption to their lives and continue to have concerns about their health and their future. This level of stress can have significant negative impacts on the recovery process.”

The researchers note that if the symptoms were caused by some external phenomenon, they are without persistent or detectable patho-physiologic changes. Additionally, it is possible that the physiologic markers of an external phenomenon are no longer detectable or cannot be identified with the current methodologies and sample size.

About the NIH Clinical Center: The NIH Clinical Center is the clinical research hospital for the National Institutes of Health. Through clinical research, clinician-investigators translate laboratory discoveries into better treatments, therapies and interventions to improve the nation's health. More information: https://clinicalcenter.nih.gov .

About the National Institute of Biomedical Imaging and Bioengineering (NIBIB): NIBIB’s mission is to improve health by leading the development and accelerating the application of biomedical technologies. The Institute is committed to integrating the physical and engineering sciences with the life sciences to advance basic research and medical care. NIBIB supports emerging technology research and development within its internal laboratories and through grants, collaborations, and training. More information is available at the NIBIB website: https://www.nibib.nih.gov .

About the National Institutes of Health (NIH): NIH, the nation's medical research agency, includes 27 Institutes and Centers and is a component of the U.S. Department of Health and Human Services. NIH is the primary federal agency conducting and supporting basic, clinical, and translational medical research, and is investigating the causes, treatments, and cures for both common and rare diseases. For more information about NIH and its programs, visit www.nih.gov .

NIH…Turning Discovery Into Health ®

Pierpaoli C, Nayak A, Hafiz R, et al. Neuroimaging Findings in United States Government Personnel and their Family Members Involved in Anomalous Health Incidents. JAMA. Published online March 18, 2024. doi: 10.1001/jama.2024.2424

Chan L, Hallett M, Zalewski C, et al. Clinical, Biomarker, and Research Tests Among United States Government Personnel and their Family Members Involved in Anomalous Health Incidents. JAMA. Published online March 10, 2024. doi: 10.1001/jama.2024.2413

Connect with Us

More Social Media from NIH

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
Sensors (Basel)

Brain-Computer Interface: Advancement and Challenges

M. f. mridha.

1 Department of Computer Science and Engineering, Bangladesh University of Business and Technology, Dhaka 1216, Bangladesh; db.ude.tbub@zorif (M.F.M.); [email protected] (S.C.D.); moc.liamg@ibakmdm (M.M.K.); [email protected] (A.A.L.)

Sujoy Chandra Das

Muhammad mohsin kabir, aklima akter lima, md. rashedul islam.

2 Department of Computer Science and Engineering, University of Asia Pacific, Dhaka 1216, Bangladesh

Yutaka Watanobe

3 Department of Computer Science and Engineering, University of Aizu, Aizu-Wakamatsu 965-8580, Japan; pj.ca.uzia-u@akatuy

Associated Data

There is no statement regarding the data.

Brain-Computer Interface (BCI) is an advanced and multidisciplinary active research domain based on neuroscience, signal processing, biomedical sensors, hardware, etc. Since the last decades, several groundbreaking research has been conducted in this domain. Still, no comprehensive review that covers the BCI domain completely has been conducted yet. Hence, a comprehensive overview of the BCI domain is presented in this study. This study covers several applications of BCI and upholds the significance of this domain. Then, each element of BCI systems, including techniques, datasets, feature extraction methods, evaluation measurement matrices, existing BCI algorithms, and classifiers, are explained concisely. In addition, a brief overview of the technologies or hardware, mostly sensors used in BCI, is appended. Finally, the paper investigates several unsolved challenges of the BCI and explains them with possible solutions.

1. Introduction

The quest for direct communication between a person and a computer has always been an attractive topic for scientists and researchers. The Brain-Computer Interface (BCI) system has directly connected the human brain and the outside environment. The BCI is a real-time brain-machine interface that interacts with external parameters. The BCI system employs the user’s brain activity signals as a medium for communication between the person and the computer, translated into the required output. It enables users to operate external devices that are not controlled by peripheral nerves or muscles via brain activity.

BCI has always been a fascinating domain for researchers. Recently, it has become a charming area of scientific inquiry and has become a possible means of proving a direct connection between the brain and technology. Many research and development projects have implemented this concept, and it has also become one of the fastest expanding fields of scientific inquiry. Many scientists tried and applied various communication methods between humans and computers in different BCI forms. However, it has progressed from a simple concept in the early days of digital technology to extremely complex signal recognition, recording, and analysis techniques today. In 1929, Hans Berger [ 1 ] became the first person to record an Electroencephalogram (EEG) [ 2 ], which shows the electrical activity of the brain that is measured through the scalp of a human brain. The author tried it on a boy with a brain tumor; since then, EEG signals have been used clinically to identify brain disorders. Vidal [ 3 ] made the first effort to communicate between a human and a computer using EEG in 1973, coining the phrase “Brain-Computer Interface”. The author listed all of the components required to construct a functional BCI. He made an experiment room that was separated from the control and computer rooms. In the experiment room, three screens were required; the subject’s EEG was to be sent to an amplifier the size of an entire desk in the control area, including two more screens and a printer.

The concept of combining brains and technology has constantly stimulated people’s interest, and it has become a reality because of recent advancements in neurology and engineering, which have opened the pathway to repairing and possibly enhancing human physical and mental capacities. The sector flourishing the most based on BCI is considered the medical application sector. Cochlear implants [ 4 ] for the deaf and deep brain stimulation for Parkinson’s illness are examples of medical uses becoming more prevalent. In addition to these medical applications, security, lie detection, alertness monitoring, telepresence, gaming, education, art, and human enhancement are just a few uses for brain–computer interfaces (BCIs), also known as brain–machine interfaces or BMIs [ 5 ]. Every application based on BCI follows different approaches and methods. Each method has its own set of benefits and drawbacks. The degree to which a performance can be enhanced while minute-to-minute and day-to-day volatility are reduced is crucial for the future of BCI technology. Such advancements rely on the capacity to systematically evaluate and contrast different BCI techniques, allowing for the most promising approaches to be discovered. In addition, this versatility around BCI technologies in different sectors and their applications can seem so complex yet so structured. Most of the BCI applications follow a standard structure and system. This basic structure of BCI consists of signal acquisition, pre-processing, feature extraction, classification, and control of the devices. The signal acquisition paves the way to connecting a brain and a computer and to gathering knowledge from signals. The three parts of pre-processing, feature extraction, and classification are responsible for making the associated signal more usable. Lastly, control of the devices points out the primary motivation: to use the signals in an application, prosthetic, etc.

The outstanding compatibility of various methods and procedures in BCI systems demands extensive research. A few research studies on specific features of BCI have also been conducted. Given all of the excellent BCI research, a comprehensive survey is now necessary. Therefore, an extensive survey analysis was attempted and focused on nine review papers featured in this study. Most surveys, however, do not address contemporary trends and application as well as the purpose and limits of BCI methods. Now, an overview and comparisons of the known reviews of the literature on BCI are shown in Table 1 .

A summary of recent surveys/reviews on various BCI technologies, signals, algorithms, classifiers, etc.

Abiri, R. et al. [ 6 ] evaluated the current review on EEG-based various experimental paradigms used by BCI systems. For each experimental paradigm, the researchers experimented with different EEG decoding algorithms and classification methods. The researchers overviewed the paradigms such as Motor imagery paradigms, Body kinematics, Visual P300, Evoked potential, and Error related potential and the hybrid paradigms analyzed with the classification methods and their applications. Researchers have already faced some severe issues while exploring BCI paradigms, including training time and fatigue, signal processing, and novel decoders; shared control to supervisory control in closed-loop; etc. Tiwari, N. et al. [ 7 ] provided a complete assessment of the evolution of BCI and a fundamental introduction to brain functioning. An extensive comprehensive revision of the anatomy of the human brain, BCI, and its phases; the methods for extracting signals; and the algorithms for putting the extracted information to use was offered. The authors explained the steps of BCI, which consisted of signal acquisition, feature extraction, and signal classification. As the human brain is complex, human-generated thoughts are non-stationary, and generated signals are nonlinear. Thus, the challenging aspect is to develop a system to find deeper insights from the human brain; then, BCI application will perform better with these deeper insights. Vasiljevic, G.A.M. et al. [ 8 ] presented a Systematic Literature Review (SLR) conclusion of BCI games employing consumer-grade gadgets. The authors analyzed the collected data to provide a comprehensive picture of the existing reality and obstacles for HCI of BCI-based games utilizing consumer-grade equipment. According to the observations, numerous games with more straightforward commands were designed for research objectives, and there was a growing amount of more user-friendly BCI games, particularly for recreation. However, this study is limited to the process of search and classification. Martini, M.L. et al. [ 9 ] investigated existing BCI sensory modalities to convey perspectives as technology improves. The sensor element of a BCI circuit determines the quality of brain pattern recognition, and numerous sensor modalities are presently used for system applications, which are generally either electrode-based or functional neuroimaging-based. Sensors differed significantly in their inherent spatial and temporal capabilities along with practical considerations such as invasiveness, mobility, and maintenance. Bablani, A. et al. [ 10 ] examined brain reactions utilizing invasive and noninvasive acquisition techniques, which included electrocorticography (ECoG), electroencephalography (EEG), magnetoencephalography (MEG), and magnetic resonance imaging (MRI). For operating any application, such responses must be interpreted utilizing machine learning and pattern recognition technologies. A short analysis of the existing feature extraction techniques and classification algorithms applicable to brain data has been presented in this study.

Fleury, M. et al. [ 11 ] described various haptic interface paradigms, including SMR, P300, and SSSEP, and approaches for designing relevant haptic systems. The researchers found significant trends in utilizing haptics in BCIs and NF and evaluated various solutions. Haptic interfaces could improve productivity and could improve the relevance of feedback delivered, especially in motor restoration using the SMR paradigm. Torres, E.P. et al. [ 12 ] conducted an overview of relevant research literature from 2015 to 2020. It provides trends and a comparison of methods used in new implementations from a BCI perspective. An explanation of datasets, emotion elicitation methods, feature extraction and selection, classification algorithms, and performance evaluation is presented. Zhang, X. et al. [ 13 ] discussed the classification of noninvasive brain signals and the fundamentals of deep learning algorithms. This study significantly gives an overview of brain signals and deep learning approaches to enable users to understand BCI research. The prominent deep learning techniques and cutting-edge models for brain signals are presented in this paper, together with specific ideas for selecting the best deep learning models. Gu, X. et al. [ 14 ] investigated the most current research on EEG signal detection technologies and computational intelligence methodologies in BCI systems that filled in the loopholes in the five-year systematic review (2015–2019). The authors demonstrated sophisticated signal detecting and augmentation technologies for collecting and cleaning EEG signals. The researchers also exhibited computational intelligence techniques, such as interpretable fuzzy models, transfer learning, deep learning, and combinations for monitoring, maintaining, or tracking human cognitive states and the results of operations in typical applications.

The study necessitated a compendium of scholarly studies covering 1970 to 2021 since we analyze BCI in detail in this literature review. We specialized in the empirical literature on BCI from 2000 to 2021. For historical purposes, such as the invention of BCI systems and their techniques, we selected some publications before 2000. Kitchenham [ 15 , 16 ] established the Systematic Literature Review (SLR) method, which is applied in the research and comprises three phases: organizing, executing, and documenting the review. The SLR methodologies attempted to address all possible questions that could arise as the current research progresses. The recent study’s purpose is to examine the findings of numerous key research areas. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were used to put together the essential materials for this study, which consists of four parts: identification, scanning, eligibility testing, and inclusion. We gathered 577 papers from a variety of sources and weeded out duplicates and similar articles. Finally, we carefully chose 361 articles and sources for monitoring and review. The PRISMA process is presented in Figure 1 .

An external file that holds a picture, illustration, etc.
Object name is sensors-21-05746-g001.jpg

The PRISMA process that is followed in this article.

However, this research looks at the present challenges and difficulties in this BCI field. Furthermore, this study generates ideas and suggestions for future research subjects. The following are the research’s total contributions:

The paper explicitly illustrates Brain-Computer Interface’s (BCI) present, past, and future trends and technologies.
The paper presents a taxonomy of BCI and elaborates on the few traditional BCI systems with workflow and architectural concepts.
The paper investigates some BCI tools and datasets. The datasets are also classified on different BCI research domains.
In addition, the paper demonstrates the application of BCI, explores a few unsolved challenges, and analyzes the opportunities.

After reading this section, one should understand BCI and how to get started with it. Our motivation to work with BCI started from a desire to learn more about this domain. Furthermore, the BCI has a bright future ahead of it, as it has a lot to offer in the medical field and in everyday life. BCI can change one’s incapability and can make life and work easy, as detailed in the following section. The applications, problems, future, and social consequences of BCI have also fueled our enthusiasm for this research.

The remainder of the paper is constructed as follows. The motivation of this work and diverse applications of BCI systems are illustrated in Section 2 . Section 3 describes the structure of BCI and briefly reviews the most popular techniques of BCI. In Section 5 , different categories of datasets available publicly are displayed. In Section 7 , the most widely used methods for signal enhancement and feature extraction of BCI are discussed. The most commonly known classifiers are reviewed in Section 8 . A broad discussion on the evaluation metrics for BCI is given in Section 9 . The challenges faced most commonly during the BCI process are reviewed in Section 10 . Lastly, this paper provides a conclusion in Section 11 .

2. Applications of BCI

BCIs may be used for various purposes and the application determines the design of a BCI. According to Nijholt [ 17 ], applications based on BCI have two methods of usability. One can command whether the other one can be observed or monitored. The majority of command applications concentrate on manipulating brain impulses using electrodes to control an external device. On the other hand, applications that involve observation focus on recognizing a subject’s mental and emotional state to behave appropriately depending on their surroundings. Some applications of BCI [ 18 ] based on usability are described below:

2.1. Biomedical Applications

The majority of BCI integrations and research have been focused on medical applications, with many BCIs aiming to replace or restore Central Nervous System (CNS) functioning lost with sickness or by accident. Other BCIs are more narrowly targeted. In diagnostic applications, on treatment and motor rehabilitation following CNS disease or trauma, BCIs for biological purposes are also employed in affective application domains. Biomedical technologies and applications can minimize extended periods of sickness, can provide supervision and protection by empowering persons with mobility difficulties, and can support their rehabilitation. The necessity to build accurate technology that can cope with potentially abnormal brain responses that might occur due to diseases such as brain stroke is a significant challenge in developing such platforms [ 19 ]. The following subsections go through each of these applications in further detail.

2.1.1. Substitute to CNS

These substitution means that it can repair or replace CNS functioning lost due to diseases such as paralysis and spinal cord injury due to stroke or trauma. In addition, due to changed brain functions, individuals with such illnesses might suffer and developing such technology can be difficult. Myoelectrics, known as a motor action potential, which captures electrical impulses in muscles, is now used in several robotic prosthetics. Bousseta, R. et al. [ 20 ] provided an experimental technology for controlling the movement of a robotic prosthetic arm with mental imagery and using cognitive tasks, which can move in four directions like left, right, up, and down.

2.1.2. Assessment and Diagnosis

The usage of BCIs in a clinical context can also help with assessment and diagnosis. Perales [ 21 ] suggested a BCI for assessing the attention of youngsters with cerebral palsy while playing games. Another research [ 22 ] looked into using BCI to capture EEG characteristics as a tool for diagnosing schizophrenia. There are also various diagnostic methods such as the detection of brain tumors [ 23 ], the identification of breast cancer [ 24 ], parkinson’s disease [ 25 ] etc. Diagnoses of several diseases in children including epilepsy, neurodegenerative disorders, motor disabilities, inattentiveness, or different types of ADHD [ 26 ] are possible. Assessment and diagnosis technologies are essential to patient well-being. Their functioning must be fine-tuned to guarantee that they are safe, acceptable, and accurate to industry standards.

2.1.3. Therapy or Rehabilitation

BCI is being used in therapeutic applications besides neurological application and prosthetics nowadays. Among the many applications, post-stroke motor rehabilitation shows promising results using BCI. Stroke is a disease that causes long-term disability to the human body and hampers all kinds of motor or vigorous activity due to an impediment of blood flow. Stroke rehabilitation application has promised to aid these activities or user imaginations through a robot or other types of machinery [ 27 , 28 , 29 ]. Some other applications treat neurological disorders such as Parkinson’s disease (PD), cluster headaches, tinnitus, etc. Deep Brain Stimulation (DBS) is an established treatment for PD as it delivers electrical impulses to a targeted area of the brain responsible for the symptoms [ 30 ]. Some stimulation BCI devices are used to process calmness during migraine attacks and cluster headaches. Lastly, a CNS disorder known as tinnitus is also in development to provide treatment by identifying brain patterns that are changed due to the disease [ 31 ]. Lastly, treatment for auditory verbal hallucinations (AVHs), best known as schizophrenia, is a possibility besides diagnosis [ 32 , 33 ].

2.1.4. Affective Computing

Users’ emotions and state of mind are observed in affective computing BCIs, with the possibility of altering their surrounding environment to improve or change that emotion. Ehrlich, S. et al. [ 34 ] created a closed-loop system in which music is generated and then replayed to listeners based on their emotional state. Human emotional states and sensory connections can be studied with a device that is related to BCI system. Patients suffering neurological diseases also can benefit from affective computing to help them convey their feelings to others [ 35 ].

2.2. Non-Biomedical Applications

BCI technologies have shown economic promise in recent years, notably in the field of non-biomedical applications. Most of these applications consist of entertaining applications, games, and emotional computation. In comparison, researchers focus on robustness and high efficiency in medical and military applications, and innovations targeted at leisure or lifestyle demand a greater emphasis on enjoyment and social elements. The most challenging aspect of this entertainment application is that it must be a user favorite to be commercially successful. As an example, some of the most popular forms of amusement are as follows:

2.2.1. Gaming

BCIs focused mainly on the gaming sector have grown in importance as a research topic. However, gaming BCIs are currently a poor substitute for standard game control methods [ 36 ]. BCI in gaming is an area where further research is needed to make games more user-friendly. In some cases, EEG data make BCI games more utilizable and increase engagement, and the system tracks each player’s enthusiasm level and activates dynamic difficulty adjustment (DDA) when the players’ excitement drops [ 37 ]. When developing such systems, fine-tuning the algorithms that regulate the game’s behavior is a big challenge. Some other games are based on BCI, as it is not visually intense and the graphics are not compatible with the recent generation. With setbacks, there is an engaging future for an Adaptation of P300 based Brain-Computer Interface for Gaming [ 38 ], which is gaining more popularity as these are very flexible to play.

2.2.2. Industry

EEG-based BCIs can also be used in industrial robotics, increasing worker safety by keeping people away from potentially demanding jobs. These technologies could substitute the time-consuming button and joystick systems used to teach robots in industrial applications; can detect when a person is too tired or ill to operate the machinery; and can take the necessary precautions to avoid injury, such as stopping the machinery [ 38 ].

2.2.3. Artistic Application

The four types of artistic applications recognized by BCIs are passive, selective, direct, and collaborative. Passive artistic BCIs need not require active user input to use the user’s brain activity to determine which pre-programmed responses to produce. Every user has had some limited control over the process within selective systems. Still, they will never be in charge of the creative product. Direct artistic BCIs provide users with far more flexibility, generally allowing them to choose items from extensive menus, such as brush type and managing brush stroke movements [ 39 ]. Lastly, the collaborative system is controlled by different users [ 40 ].

2.2.4. Transport

BCI is used in transportation monitoring which tracks awareness to assess driver weariness and to enhance airline pilot performances. In the BCI system, mistakes can be costly regarding lives and monetary obligations on the entities involved when such technologies are utilized in critical applications [ 41 , 42 ].

3. Structure of BCI

The BCI system operates with a closed-loop system. Every action taken by the user is met with some feedback. For example, an imagined hand movement might result in a command that causes a robotic arm to move. This simple movement of this arm needs a lot of processes inside it. It starts from the brain, which is one of our body’s most extensive and most complicated organs. It is made up of billions of nerves that link billions of synapses to communicate. The processes from taking signals from the human brain to transforming into a workable command are shown in Figure 2 and described below:

Signal acquisition: In the case of BCI, it is a process of taking samples of signals that measure the brain activity and turning them into commands that can control a virtual or real-world application. The various techniques of BCI for signal acquisition are described later.
Pre-processing: After the signal acquisition, the pre-processing of signals is needed. In most cases, the collected signals from the brain are noisy and impaired with artifacts. This step helps to clean this noise and artifacts with different methods and filtering. That is why it is named signal enhancement.
Feature extraction: The next stage is feature extraction, which involves analyzing the signal and extracting data. As the brain activity signal is complicated, it is hard to extract useful information just by analyzing it. It is thus necessary to employ processing algorithms that enable the extraction of features of a brain, such as a person’s purpose.
Classification: The next step is to apply classification techniques to the signal, free of artifacts. The classification aids in determining the type of mental task the person is performing or the person’s command.
Control of devices: The classification step sends a command to the feedback device or application. It may be a computer, for example, where the signal is used to move a cursor, or a robotic arm, where the signal is utilized to move the arm.

An external file that holds a picture, illustration, etc.
Object name is sensors-21-05746-g002.jpg

Basic architecture of a BCI system.

The basic architecture of the BCI system was explained in the preceding section. It prompts us to investigate the classification of BCI system. Based upon various techniques, BCI system is classified. The BCI techniques are discussed in following parts.

From the above Figure 3 , we can classify BCI from different aspects such as dependability, invasiveness, and autonomy.

Dependability: BCI can be classified as dependent or independent. Dependent BCIs necessitate certain types of motor control from the operator or healthy subjects, such as gaze control. On the other hand, independent BCIs do not enable the individual to exert any form of motor control; this type of BCI is appropriate for stroke patients or seriously disabled patients.
Invasiveness: BCI is also classified into three types according to invasiveness: invasive, partially invasive, and non-invasive. Invasive BCIs are by far the most accurate as they are implanted directly into the cortex, allowing researchers to monitor the activity of every neuron. Invasive varieties of BCI are inserted directly into the brain throughout neurosurgery. There are two types of invasive BCIs: single unit BCIs, which detect signals from a single place of brain cells, and multi-unit BCIs, which detect signals from several areas. Semi-invasive BCIs use Electrocorticography (ECoG), a kind of signal platform that enables electrodes to be placed on the attainable edge of the brain to detect electrical impulses originating from the cerebral cortex. Although this procedure is less intrusive, it still necessitates a surgical opening in the brain. Noninvasive BCIs use external sensing rather than brain implants. Electroencephalography (EEG), Magnetoencephalography (MEG), Positron emission tomography (PET), Functional magnetic resonance imaging (fMRI), and Functional near-infrared spectroscopy (fNIRS) are all noninvasive techniques used it to analyze the brain. However, because of the low cost and portability of the gear, EEG is the most commonly used.
Autonomy: BCI can operate either in a synchronous or asynchronous manner. Time-dependent or time-independent interactions between the user and system are possible. The system is known as synchronous BCI if the interaction is carried out within a particular amount of time in response to a cue supplied by the system. In asynchronous BCI, the subject can create a mental task at a certain time to engage with the system. Synchronous BCIs are less user-friendly than asynchronous BCIs; however, designing one is substantially easier than developing an asynchronous BCI.

An external file that holds a picture, illustration, etc.
Object name is sensors-21-05746-g003.jpg

The classification/taxonomy of the BCI system.

As the motive of this research work is to focus on advancements of BCI, the most advanced and mostly used techniques that is based on invasiveness are described in the following part. Based on invasiveness, BCI is classified into three categories that are more familiar. In the consequent section, we address these three categories and describe them elaborately.

3.1. Invasive

The types of BCI that are invasive are inserted directly into the brain with neurosurgery. Invasive BCIs seem to be the most accurate even though they are implanted directly into the cortex as it is allowed to track every neuron’s action. Invasive BCI also has two units rather than parts. The first unit is single-unit BCIs that detect signals from a single location of brain cells, whereas multi-unit BCIs detect numerous areas, the second unit [ 43 ]. However, the neurosurgery treatment has various flaws, such as the possibility of scar tissue formation. The body responds to the foreign object by forming a scar around the electrodes, leading the signal to deteriorate. Since neurosurgery is a dangerous and costly procedure, invasive BCI is mainly used on blind and paralyzed patients.

3.2. Partially Invasive

Although this approach is not as intrusive, it still involves brain surgery. Electrocorticography (ECoG) is a sort of partially invasive BCI monitoring system that places electrodes in the cortex surface of the brain to produce signals with electrical activity. For example, blinking allows your brain to discharge electrical activity. When investigating signals, though, these involuntary actions are generally not of interest since they are in the way of what we search for. It is a form of noise. ECoGs are less considered with noise than non-invasive BCI, making interpretation easier [ 44 ].

Electrocorticography (ECoG)

Electrocorticography (ECoG) [ 45 ] is an partially invasive method that measures the brain’s electrical activity. In another sense, the participant’s skull must be evacuated, and the electrodes must be placed right at the brain’s service. Consequently, this electrode is located on the skull. The particular resolution of the recorded signals is considerably better than EEG. The signal-to-noise ratio is superior compared with the closer proximity to cerebral activity. Furthermore, motion artifacts such as blinks and eye movement have a significantly lower impact on ECoG signals. However, ECoG would only be helpful in the accessible brain area and is close to impossible to utilize outside of a surgical setting [ 46 ].

3.3. Noninvasive

Noninvasive neuroimaging technologies have also been used as interfaces in human research. Noninvasive EEG-based BCIs account for the vast bulk of published BCI research. EEG-based noninvasive technologies and interfaces have been employed in a considerably more comprehensive range of applications. Noninvasive apps and technologies are becoming increasingly popular in recent years since they do not require any brain surgery. In the noninvasive mode, a headpiece or helmet-like electrode is utilized outside the skull to measure the signal by causing electrical activity in the brain. There are some well-known and widely used ways for measuring these electrical activity or potentials, such as Electroencephalography (EEG), Magnetoencephalography (MEG), Functional Magnetic Resonance Imaging (fMRI), Facial Near Infrared Spectroscopy (fNIRS), and Positron Emission Tomography (PET). An elaborate description of BCI techniques is given below:

3.3.1. Electroencephalography (EEG)

EG monitors electrical activity in the scalp generated by activating a few of the brain’s neurons. Several electrodes implanted on the scalp directly, mainly on the cortex, are often used to record these electrical activities quickly. For its excellent temporal resolution, ease of use, safety, and affordability, EEG is the most used technology for capturing brain activity. Active electrodes and passive electrodes are indeed the two types of electrodes that can be utilized. Active electrodes usually feature an integrated amplifier, whereas passive electrodes require an external amplifier to magnify the detected signals. The prime objective of implementing either embedded or external amplifiers is to lessen the impact of background noise and other signal weaknesses caused by cable movement. One of the issues with EEG is that it necessitates the use of gel or saline solutions to lower the resistance of skin-electrode contact. Furthermore, the signal quality is poor, and it is altered by background noise. The International 10–20 system [ 47 ] is often used to implant electrodes over the scalp surface for recording purposes. The electrical activities across various frequency bands are used to describe EEG in general.

3.3.2. Magnetoencephalography (MEG)

The magnetic fields created by current flow in the brain are measured using MEG (Magnetoencephalography). Electric fields have significantly more interrupted travel via the skull than magnetic fields, therefore it has superior spatial resolution than EEG. A functional neuroimaging technique is applied to measure and evaluate the brain’s magnetic field. MEG operates on the outside of the head and is now a part of the clinical treatment regularly. David Choen [ 48 , 49 ] was the first to invent it in 1968 by utilizing a conduction copper detector inside a shielded chamber to reduce background noise. Improved MEG signals have recently been produced using more sensitive sensors such as superconducting quantum interference devices (SQUID) [ 50 ]. MEG has become significant, especially for patients with epilepsy and brain tumors. It may aid in detecting regions of the brain with average function in individuals with epilepsy, tumors, or other mass lesions. MEG operates with magnetic waves rather than electrical waves so that it could contribute additional information to EEG. MEG is also capable of capturing signals with high temporal and spatial resolution. Therefore, to detect cerebral activity that creates tiny magnetic fields the scanners must be closer to the brain’s surface. As a result, specific sensors are required for MEG, such as superconducting quantum interference (SQUID) sensors [ 51 ].

3.3.3. Functional Magnetic Resonance Imaging (fMRI)

Noninvasive functional magnetic resonance imaging (fMRI) is used to evaluate the fluctuation in blood oxygen levels throughout brain activities. fMRI has an excellent spatial resolution, which makes it ideal for identifying active areas of the brain [ 52 ]. The time resolution of fMRI is comparatively low, ranging from 1 to 2 s [ 53 ]. It also has low resolution when it comes to head movements, which could result in artifacts. In the 1990s, functional magnetic resonance imaging (fMRI) was created. It is a noninvasive and safe technology that does not include the use of radiation, is simple to use, and has great spatial and temporal resolution. Hemoglobin in capillary red blood cells in the brain transports oxygen to the neurons. As a result of the increased demand for oxygen, blood flow increases. If haemoglobin is oxygenated, its magnetic properties vary. The MRI equipment, which is a cylindrical tube with a strong electromagnet, can determine which regions of the brain are activated because of this difference. That is how fMRI works. There is also a specific application or software known as diffusion MRI, which generates images from the data or results that use water molecules’ diffusion. Diffusion-weighted and diffusion tensor imaging (DWI/DTI) facilitates this exploration of the microarchitecture of the brain. Diffusion-weighted magnetic resonance imaging (DWI or DW-MRI) imaging renders picture variation depending on variances in the degree of diffusion of water particles inside the brain. Diffusion depicts the stochastic thermic mobility of particles. Diffusion inside the brain is defined by several agents, including representing particles beneath study, the temperature, and the microenvironmental structure in which the diffusion occurs [ 54 ]. Diffusion tensor imaging (DTI) investigates the three-dimensional form of the diffusion, also recognized as diffusion tensor. It is a powerful MRI modality that produces directional knowledge about the water motion in a voxel. It exhibits noninvasively microscopic tissue features that surpass the ability of any other imaging methods [ 55 ].

3.3.4. Functional Near-Infrared Spectroscopy (fNIRS)

The infrared radiation is projected into the brain using fNIRS equipment [ 53 , 56 ] to monitor improvements in specific wavelengths as the light is reflected. fNIRS often detects changes in regional blood volume and oxygenation. When a particular area of the brain works, it requires additional oxygen, which is given to the neurons via capillary red blood cells—the increased blood flow in the brain areas that would be most active at a given time. fMRI is a technique that monitors variations in oxygen levels caused by various activities. As a result, images with a high spatial resolution (1 cm) but lower temporal resolution (>2–5 s) could be obtained, comparable with standard functional magnetic resonance imaging.

3.3.5. Positron Emission Tomography (PET)

PET (positron emission tomography) is a sophisticated imaging tool for examining brain activities in real-time. It enables noninvasive measurement of cerebral blood flow, metabolism, and receptor binding in the brain. Due to the relatively high prices and complexity of the accompanying infrastructure, including cyclotrons, PET scanners, and radio chemistry laboratories, PET was previously only used in research. PET has been widely employed in clinical neurology in recent years due to technological improvements and the proliferation of PET scanners to better our understanding of disease etiology, to help in diagnosis, and to monitor disease progression and response to therapy [ 57 ]. PET medications such as radiolabeled choline, fluciclovine (18F-FACBC), and compounds targeting prostate-specific membrane antigen are now being researched and explored to improve noninvasive prostate cancer localization diagnostic performance [ 58 ].

4. Brain Control Signals

The brain-computer interface (BCI) is based on signal amplification that comes directly from the brain. Several of these signals are simple to extract, while others are more difficult and require additional preprocessing [ 53 ]. These control signals can be classified into one of three groups: (1) evoked signals, (2) spontaneous signals, and (3) hybrid signals. A detailed overview of the three categories is given below. The control signals classification is shown in Figure 4 .

An external file that holds a picture, illustration, etc.
Object name is sensors-21-05746-g004.jpg

The basic architecture of BCI control signals.

4.1. Visual Evoked Potentials

Electrical potentials evoked by short visual stimuli are known as VEPs. The visual cortex’s potentials are monitored, and the waveforms are derived from the EEG. VEPs are generally used to assess the visual pathways from the eye to the brain’s visual cortex. Middendorf et al. published a procedure for measuring the position of the user’s gaze using VEPs in 2000 [ 59 ]. The user is confronted with a screen that displays several virtual buttons that flash at varied rates. The frequency of the photic driving reflex over the user’s visual brain is determined after the user focuses their gaze on a button. Whenever the frequency of a shown button equals the frequency of the user, the system concludes that the user wants to pick it. Steady-State Evoked Potentials (SSEP) and P300 are two of the most well-evoked signals. External stimulation is required for evoked signals that can be unpleasant, awkward, and exhausting for the individual.

4.1.1. Steady-State Evoked Potential (SSEP)

SSEP signals are produced when a patient experiences periodic stimuli such as a flickering picture, modulated sound, or even vibrations [ 60 , 61 ]. The strength of the EEG signal in the brain must grow to meet the stimulus frequency. Signals in many brain locations are observed in terms of the sensory process. SSEP signals of different forms, such as steady-state visual potentials (SSVEPs), somatosensory SSEP, and auditory SSEP, are found. SSVEP is widely used in a variety of applications. These are normal brain reactions to repeating stimuli, which vary depending on the frequency with which they are presented. Although there are instances of BCI paradigms utilizing somatosensory (SSSEP) or auditory (SSAEP) stimuli, they are generally induced using visual stimuli (steady-state visually evoked potentials, SSVEP) [ 62 ].

4.1.2. P300 Evoked Potentials (P300)

The peaks in an EEG generated by infrequent visual, auditory, or somatosensory inputs are known as P300 evoked potentials. Without the need for training to use P300-based BCI systems. A matrix of symbols, in which selection is dependent on the participant’s gaze, is a prominent use of P300-based BCI systems. Such a signal is typically produced using an “odd-ball” paradigm. The user is asked to respond to a random succession of stimuli, which is less frequent than others [ 63 ]. The P300-based EEG waves are triggered when this unusual stimulus is significant to the person. P300 does not reasonably require any subject training, although, it does need repetitive stimulation, which may tire the subject and may cause inconsistencies.

4.2. Spontaneous Signals

With no external cues, the person produces random signals willingly. These signals are produced without any external stimuli (somatosensory, aural, or visual). Motor and sensorimotor rhythms, Slow Cortical Potentials (SCPs), and non-motor cognitive signals are some of the most prominent spontaneous signals [ 53 ].

4.2.1. Motor and Sensorimotor Rhythms

Motor activities are linked to motor and sensorimotor rhythms. Sensorimotor rhythms are rhythmic oscillations in electrophysiological brain activity in the mu (Rolandic band, 7–13 Hz) and beta (13–30 Hz) frequencies. Motor imagery is the process of converting a participant’s motor intentions into control signals employing motor imagery conditions [ 64 ]. The left-hand motion, in an instance, may result in EEG signals in the and rhythms and a decrease in certain motor cortex areas (8–12 Hz) and (18–26 Hz). Depending on the motor imagery rhythms, various applications can be used such as controlling a mouse or playing a game.

4.2.2. Slow Cortical Potentials (SCP)

SCP is indeed an EEG signal with a frequency less than 1 Hz [ 65 ]. It is a low-frequency potential observed in the frontal and central portions of the cortex and depolarization level variations throughout the cortical dendrites. SCP is a highly gradual change in brain activity, either positive or negative, that can only last milliseconds to several seconds. Through operant conditioning, the subject can control the movement of such signals. As a result, extensive training may be required in addition to that needed for motor rhythms. Many studies no longer choose SCP, and motor and sensorimotor rhythms have taken their place.

4.2.3. Non-Motor Cognitive Tasks

Cognitive objectives are utilized to drive the BCI in non-motor cognitive tasks. Several tasks, such as musical imagination, visual counting, mental rotation, and mathematical computation, might be completed [ 66 ]. Penny, W.D. et al. [ 67 ] used a pattern classifier with unclear parameters. The individual performed simple subtraction in one of their non-motor cognitive activities.

4.3. Hybrid Signals

The term “hybrid signals” refers to the utilization of a mixture of brain-generated signals for control. As a result, instead of measuring and using only one signal in the BCI system, a mix of signals is used. The fundamental goal of using two or more types of brain signals as input to a BCI system is to increase dependability while avoiding the drawbacks of each signal type [ 68 ].

Some research is addressed that the types of brain signals are classified into two categories [ 10 ]. These are event-related potentials and evoked brain potential. Three varieties are organized for evoked brain potential: Visual Evoked Potential (VEP), Tactile Evoked Potential (TEP), and Auditory Evoked Potential (AEP) [ 69 ].

While analyzing the literature on BCI systems, we discovered various often used datasets that researchers used while implementing these techniques. In terms of the research, EEG is now the most frequent method for collecting brain data in BCI. As this is a noninvasive method and has convenient handling for most datasets, an EEG signal is used. However, for a variety of reasons, EEG does not provide a comprehensive method of data collection. It needs a variety of fixed things to acquire the data. Firstly, the signal must be acquired and stored by some subject, participants, or patients. It is unsuitable when only one subject requires the same arrangement as multiple subjects to obtain data. After the subjects are prepared, the electrodes (a gear mounted on the scalp) are attached to the individuals to capture and measure data. This data collection method lasted for several sessions, with a particular recording period determined by the work’s purpose. The saved data in these sessions and recordings are primarily brain signals measured by a brain’s action on a sure thing, such as a video or a picture. EEG signals differ from one participant to the next and from one session to the next. In this section, the datasets as well as the subjects and electrodes, channels, and sessions are described. The explanation is tabulated in Table 2 , Table 3 , Table 4 , Table 5 , Table 6 , Table 7 and Table 8 . In Table 2 , some popular motor imagery datasets are illustrated. The most beneficial option for creating BCIs is motor imagery (MI) impulses captured via EEG, which offers a great degree of mobility. It enables people with motor disabilities to communicate with the device by envisioning motor movements without any external stimuli generated from the motor cortex. A few datasets based on error-related potentials (ErrPs) are exhibited in Table 3 . ErrPs is an EEG dataset that utilizes a P300-based BCI speller to boost the performance of BCIs. Detecting and fixing errors of the neuronal signature of a user’s knowledge linked to a brain pattern is known as error-related potentials (ErrPs). Affective computing improves human–machine communication by identifying human emotions. Some mostly used emotion recognition datasets are shown in Table 4 . Various EEG-based BCI devices can detect the user’s emotional states to make contact effortless, more useable, and practical. The emotions extracted in emotion-recognition datasets are valence, arousal, calm, positive, exciting, happy, sad, neutral, and fear. In addition, it is certainly clear by now that brain signals or memory are a mixed emotion. The part where all of these mixed emotions are gathered from different body parts is known as a miscellaneous part of the brain. Therefore, miscellaneous datasets include memory signals, brain images, brain signals, etc. Some miscellaneous datasets are represented in Table 5 . In EEG-based BCI, the signals can detect eye movement such as eye blinks, eye states, etc. The BCI datasets of eye blinks or movements include voluntary and involuntary eye states, blinks, and activities are illustrated in Table 6 . Subsequently, the electrical response in the brain to a specific motor or cognitive event such as a stimulus is known as an event-related potential (ERP). An unwanted sound, a sparking light, or a blinking eye can be an example of a stimulus. BCI utilizing ERPs attempts to track attention, weariness, and the brain’s reaction to this event-related stimulus. Table 7 is encapsulated with popular ERP datasets around. Moreover, the visual information-processing mechanism in the brain is reflected in Visually Evoked Potentials (VEPs). Flashing objects in the form of shifting colors or a reversing grid are frequent visual stimulators. The CRT/LCD monitor or flash tube/infrared diode (LED) is utilized for stimulus display in VEP-based BCIs. Frequently used VEP-based datasets with these utilized objects are represented in Table 8 .

A table of different types of motor imagery datasets of BCI.

A table of different types of Error-Related Potentials (ErrP) dataset of BCI.

A table of different types emotion recognition dataset of BCI.

A table of different types of miscellaneous datasets.

A table of different types of eye-blink or movement datasets in BCI.

A table of different types Event-Related Potential (ERP) datasets in BCI. These datasets are collected from [ 229 ].

A table of different types of Visually Evoked Potential (VEP) datasets in BCI. These datasets are collected from [ 229 ].

However, the dataset covers information recorded from the beginning of BCI. To extract information from datasets, feature extraction methods are necessary, which is reviewed in the following section.

6. Signal Preprocessing and Signal Enhancement

In most situations, the signal or data measured or extracted from datasets are filled with noise. With a natural human activity such as eye blinks and heartbeats, the collected data might become noisy. These noises are eliminated during the pre-processing step to produce clean data that may subsequently process the feature extraction and classification. This pre-processing unit is also known as signal enhancement since it cleans the signal in BCI. Some methods are used for signal enhancement in the BCI system, and these are explained elaborately in the following subsections.

6.1. Independent Component Analysis (ICA)

The noises and EEG signals are isolated in ICA by treating them as distinct entities. Furthermore, the data are retained during the removal of noises. This method divides the EEG data into spatially fixed and temporally independent components. In the case of computing and noise demonstrable, the ICA shows more efficiency [ 256 ].

6.2. Common Average Reference (CAR)

It is most commonly employed as a basic dimensionality reduction technique. This approach decreases noise across all recorded channels, but this does not address channel-specific noise and may inject noise into an otherwise clean channel. It is a spatial filter that can be thought of as the subtraction of shared EEG activity, retaining only the idle action of each EEG particular electrode [ 256 ].

6.3. Adaptive Filters

The adaptive filter is a computational device for mathematical processes. It connects the adaptive filter’s input/output signals iteratively. There are filter coefficients that are self-adjusted utilizing an adaptive algorithm. It works by altering signal properties depending on the characteristics of the signals under investigation [ 257 ].

6.4. Principal Component Analysis (PCA)

PCA is a technique for detecting patterns in data represented by a rotation of the coordinate axes. These axes are not aligned with single time points, but they depict a signal pattern with linear combinations of sets of time points. PCA keeps the axes orthogonal while rotating them to maximize variance along the first axis. It reduces feature dimensions and aids in data classification by completing ranking. In comparison with ICA, PCA compresses separate data better whether noise is eliminated with it or not [ 258 ].

6.5. Surface Laplacian (SL)

SL refers to a method of displaying EEG data with a high spatial resolution. SL can be generated using any EEG recording reference scheme as their estimates are reference-free. Based on the volume conductor’s exterior shape, it is a general estimate of the current density entering or exiting the scalp through the skull, and it does not require volume conduction details. The advantage of SL is that it improves the spatial resolution of the EEG signal. However, SL seems not to demand additional operative neuroanatomy premises as it is sensitive to spline patterns and artifacts [ 259 ].

6.6. Signal De-Noising

Artefacts frequently corrupt EEG signals taken from brain. These artifacts must be removed from EEG data to obtain valuable information from it. The technique of eliminating sounds or artefacts from EEG signals is known as de-noising [ 260 ]. Some de-noising methods are given below:

Wavelet de-noising and thresholding: The multi-resolution analysis is used to transfer the EEG signal to the discrete wavelet domain. The contrasting or adaptive threshold level is used to reduce particular coefficients associated with the noise signal [ 261 ]. Shorter coefficients would tend to define noise characteristics throughout time and scale in a well-matched wavelet representation. In contrast, threshold selection is one of the most critical aspects of successful wavelet de-noising. Thresholding can isolate the signal from the noise in this case; hence, thresholding approaches come in several shapes and sizes. All coefficients underneath a predetermined threshold value are set to zero in hard thresholding. Soft thresholding is a method of reducing the value of the remaining coefficients by a factor of two [ 262 ].
Empirical mode decomposition (EMD): It is a signal analysis algorithm for multivariate signals. It breaks the signal down into a series of frequency and amplitude-regulated zero-mean signals, widely known as intrinsic mode functions (IMFs). Wavelet decomposition, which decomposes a signal into multiple numbers of Intrinsic Mode Functions (IMFs), is compared by EMD. It decomposes these IMFs using a shifting method. An IMF is a function with a single maximum between zero crossings and a mean value of zero. It produces a residue after degrading IMFs. These IMFs are sufficient to characterize a signal [ 263 ].

Most of our datasets mentioned in the previous section are a part of various BCI paradigms and follow these signal enhancement techniques as well. The motor imagery datasets represent paradigms such as sensorimotor activity or rhythms. In addition, error-related potentials datasets and datasets such as event-related potentials or visually evoke potentials signify their own BCI paradigm. Some other paradigms, such as overt attention, eye movement, miscellaneous, and emotion recognition, identify their datasets. Indeed, these paradigms become bigger in number as the measurement of different brain movements and emotions are attempted. More than 100 BCI designs are required to use signal enhancement techniques to extract features from the signal. In comparison, Reference [ 264 ] shows that 32% of BCI designs use surface Laplacian (SL) to extract features, principal component analysis (PCA) or independent component analysis (ICA) was used in 22%, and common spatial patterns (CSP) and common average referencing (CAR) techniques are used in 14% and 11%, respectively.

7. Feature Extraction

Now, it is necessary to understand what the features represent, their qualities, and how to use them for a BCI system to select the best appropriate classifier. A classification system’s accuracy or efficiency is primarily determined by the feature(s) of the samples to be categorized [ 265 ]; therefore, feature extraction has been crucial stage in BCI. The majority of noninvasive BCI devices use neuroimaging techniques such as MEG and MRI. However, EEG is the most widely utilized method, owing to its high temporal resolution and inexpensive cost [ 266 ]. The EEG signal feature extraction method is one of the essential components of a BCI system because of its involvement in successfully executing the classification stage at discriminating mental states. Nevertheless, the feature extraction methods based on both EEG and ECoG are discussed elaborately in the subsequent section.

7.1. EEG-Based Feature Extraction

Typically, BCI focuses on identifying acquired events using various neuroimage techniques, the most common of which is electroencephalography (EEG). Since its involvement in successfully executing the classification stage at discriminating mental states, the EEG signal feature extraction method is one of the essential components of a BCI system. According to [ 267 ] on EEG, three types of feature extraction are discussed in detail in the following sections. These features are the time domain, the frequency domain, and the time–frequency domain. The following subsection address the feature domains elaborately.

7.1.1. Time Domain

The time–frequency domain integrates analyses in the time and frequency domains. It depicts the signal energy distribution in the Time–Frequency plane (t-f plane) [ 268 ]. When it comes to deciphering rhythmic information in EEG data, a time–frequency analysis comes in handy. EEG’s time-domain properties are straightforward to fix, but they have the disadvantage of containing non-stationary signals that alter over time. Features are usually derived using signal amplitude values in time-domain approaches that can be distorted by interference as noise during EEG recording.

Event related potentials: Event-related potentials (ERPs) are very low voltages generated in brain regions in reaction to specific events or stimuli. They are time-locked EEG alterations that provide a safe and noninvasive way to research psychophysiological aspects of mental activities. A wide range of sensory, cognitive, or motor stimuli can trigger event-related potentials [ 269 , 270 ]. ERPs are useful to measure the time to process a stimulus and a response to be produced. The temporal resolution of event-related potentials is remarkable, but it has a low spatial resolution. ERPs were used by Changoluisa, V. et al. [ 271 ] to build an adaptive strategy for identifying and detecting changeable ERPs. Continuous monitoring of the curve in ERP components takes account of their temporal and spatial information. Some limitations of ERPs are that it shows poor spatial resolution, whether it is suitable with temporal resolution [ 272 ]. Furthermore, a significant drawback of ERP is the difficulty in determining where the electrical activity originates in the brain.
− Mean absolute value: M A V = 1 N ∑ n = 1 N x n (1)
− Power: P = 1 N ∑ n = 1 N x n 2 (2)
− Standard deviation: S D = 1 N ∑ n = 1 N x ( n ) − μ n (3)
− Root mean square (RMS): RMS = 1 N ∑ i = 1 N x i 2 1 / 2 (4)
− Square root of amplitude (SRA): SRA = 1 N ∑ i = 1 N x i 2 (5)
− Skewness value (SV): SV = 1 N ∑ i = 1 N x l − x ¯ σ 3 (6)
− Kurtosis value (KV): KV = 1 N ∑ i = 1 N x l − x ¯ σ 4 (7)
Hjorth features: Bo Hjorth introduced the Hjorth parameters in 1970 [ 276 ]; the three statistical parameters employed in time-domain signal processing are activity, mobility, and complexity. Dagdevir, E. et al. [ 277 ] proposed a motor imagery-based BCI system where the features were extracted from the dataset using the Hjorth algorithm. The Hjorth features have advantages in real-time analyses as it has a low computation cost. However, it has a statistical bias over signal parameter calculation.
Phase lag index (PLI): The functional connectivity is determined by calculating the PLI for two pairs of channels. Since it depicts the actual interaction between sources, this index may help estimate phase synchronization in EEG time series. PLI measures the asymmetry of the distribution of phase differences between two signals. The advantage of PLI is that it is less affected by phase delays. It quantifies the nonzero phase lag between the time series of two sources, making it less vulnerable to signals. The effectiveness of functional connectivity features evaluated by phase lag index (PLI), weighted phase lag index (wPLI), and phase-locking value (PLV) on MI classification was studied by Feng, L.Z. et al. [ 278 ].

7.1.2. Frequency Domain

When analyzing any signal in terms of frequency instead of just time, the frequency domain properties are considered. Any signal’s frequency domain representation displays how much of it falls inside a specific frequency range. The frequency domain properties are commonly acquired using power spectral density (PSD). The discussion about these properties is presented below in the following section.

Fast fourier transform (FFT): The Fourier transform is a mathematical transformation that converts any time-domain signal into its frequency domain. Discrete Fourier Transform (DFT) [ 279 ], Short Time Fourier Transform (STFT) [ 280 , 281 ], and Fast Fourier Transform are the most common Fourier transform utilized for EEG-based emotion identification (FFT) [ 282 ]. Djamal, E.C. et al. [ 283 ] developed a wireless device that is used to record a player’s brain activity and extracts each action using Fast Fourier Transform. FFT is faster than any other method available, allowing it to be employed in real-time applications. It is a valuable instrument for signal processing at a fixed location. A limitation of FFT is that it can convert the limited range of waveform data and the requirement to add a window weighting function to the waveform to compensate for spectral leakage.

An external file that holds a picture, illustration, etc.
Object name is sensors-21-05746-g005.jpg

The basic structure of CSP [ 286 ].

In Figure 5 , CSP provides spatial filters that minimize the variance of an individual class while concurrently maximizing the variance of other classes. These filters are mainly used to choose the frequency from the multichannel EEG signal. After frequency filtering, spatial filtering is performed using spatial filters that are employed to extract spatial information from the signal. Spatial information is significantly necessary to differentiate intent patterns in multichannel EEG recordings for BCI. The performance of this spatial filtering depends on the operational frequency band of EEG. Therefore, CSP is categorized as a frequency domain feature. However, CSP acts as signal enhancement while it requires no preceding excerpt or information of sub-specific bands.

Higher-order Spectral (HOS): Second-order signal measurements include the auto-correlation function and the power spectrum. Second-order measures operate satisfactorily if the signal resembles a Gaussian probability distribution function. However, most of the real-world signals are non-Gaussian. Therefore, Higher-Order Spectral (HOS) [ 285 ] is an extended version of the second-order measure that works well for non-Gaussian signals, when it comes into the equation. In addition, most of the physiological signals are nonlinear and non-stationary. HOS are considered favorable to detect these deviations from the signal’s linearity or stationarity. It is calculated using the Fourier Transform at various frequencies. H O S = X ( K ) X ( l ) X ∗ ( k + l ) (8) where X ( K ) is the Fourier transform of the raw EEG signal x ( n ) and l is a shifting parameter.

7.1.3. Time–Frequency Domain

In the time-frequency domain, the signal is evaluated both in the time and frequency domains simultaneously. The wavelet transform is one of many advanced approaches for analyzing the time-frequency representation. There are some other widely used models for utilizing the time-frequency domain. These models are addressed with a proper explanation in the subsequent section.

The AR parameters are a p ( i ) , the observations are x ( n ) and the excitation white noise is v ( n ) . Lastly, the most challenging part of AR EEG modeling is choosing the correct model to represent and following the changing spectrum correctly.

Wavelet Transform (WT): The WT technique encodes the original EEG data using wavelets, which are known as simple building blocks. It looks at unusual data patterns using variable windows with expansive windows for low frequencies and narrow windows for high frequencies. In addition, WT is considered an advanced approach as it offers a simultaneous localization in the time-frequency domain, which is a significant advantage. These wavelets can be discrete or continuous and describe the signal’s characteristics in a time-domain frequency. The Discrete Wavelet Transform (DWT) and the Continuous Wavelet Transform (CWT) are used frequently in EEG analysis [ 289 ]. DWT is now a more widely used signal processing method than CWT as CWT is very redundant. DWT decomposes any signal into approximation and detail coefficients corresponding to distinct frequency ranges maintaining the temporal information in the signal. However, most researchers try all available wavelets before choosing the optimal one that produces the best results, as selecting a mother wavelet is challenging. In wavelet-based feature extraction, the Daubechies wavelet of order 4 (db4) is the most commonly employed [ 290 ].

7.2. ECoG-Based Features

Electrocorticography (ECoG) generates a reliable signal through electrodes placed on the surface of the human brain, which decodes movement, vision, and speech. Decoding ECoG signal processing gives immediate patient feedback and controls a computer cursor or perhaps an exoskeleton. The ECoG signal feature extraction approach is a crucial element of the BCI system since it is involved in accomplishing the classification phase during decoding. Some of the widely used feature extraction methods are discussed below.

7.2.1. Linear Filtering

It is typically employed to filter out noise in the form of signals that are not in the frequency range of the brain’s messages. Low-pass filters and high-pass filters are the two types of linear filters. This typical linear filtering is used to removed ECOG, EOG, and EMG artifacts from EEG signals. Low pass filtering is used to remove EMG artifacts, and high pass filtering is used to remove EOG artifacts [ 291 ]. These artifacts are noises produced by either physiological processes such as muscle, eye, or other biological movement or exogenous (external) sources such as machinery faults. There are three approaches for dealing with artifacts in EEG signal acquisition. Avoiding artifacts by keeping an eye on the subject’s movements and the machine’s operation. Contaminated trials are discarded due to artifact rejection. Pre-processing techniques are used to remove artifacts. The advantage of linear filtering is that signals are considered a controlled scaling of the signal’s frequency domain components. High pass filtering is used to raise the relative importance of the high-frequency components by reducing the features in the frequency domain’s center.

7.2.2. Spatial Filtering

Spatial filtering is a technique for improving decoding by leveraging information about the electrode positions. The spatial filter aims to lessen the influence of spatial distortion in the raw signal; various ECoG channels are treated as coordinates for multivariate data sampling through spatial filters. The filtering transforms that coordinate system to facilitate decoding. Spatial filtering can use to minimize data dimensionality or to increase the dissimilarity of various observations. The referencing systems used during ECoG recordings are frequently utilized for preliminary spatial filtering. Equation ( 10 ) determines the spatial filter [ 292 ].

where x ′ is the spatially filtered signal, x i is the EEG signal from channel i , and w i is the weight of that channel. With the aid of relevant information acquired from multiple EEG channels, spatial filtering contributes to recovering the brain’s original signal. Simultaneously, it reduces dimensionality by lowering EEG channel size to smaller spatially filtered signals.

Thus far, feature extraction involves extracting new features from existing ones to minimize feature measurement costs, to improve classifier efficiency, and to improve classification accuracy. Now in the following section, the extracted feature classifiers are briefly described.

8. BCI Classifiers

BCI always needs a subject to use its device, and similarly, the subject must produce several types of data to use a BCI device. In addition, to use a BCI system, the subject must develop various brain activity patterns that the system can recognize and convert into commands. To achieve this mentioned conversion, some regression or classification algorithms can be used. The classification step’s design comprises selecting one or more classification algorithms from a variety of options. In this section, some commonly known classifiers [ 293 ], which are classified in Figure 6 , as well as some new classifiers [ 294 ] are described below.

An external file that holds a picture, illustration, etc.
Object name is sensors-21-05746-g006.jpg

Classification of commonly used classifiers in BCI.

8.1. Linear Classifiers

Linear classifiers are discriminant algorithms that discriminate classes using linear functions. It is most likely the most widely used algorithm in BCI systems. Two types of linear classifiers are used during BCI design: linear discriminant analysis (LDA) and support vector machine (SVM).

8.1.1. Linear Discriminant Analysis (LDA)

The objective of Linear Discriminant Analysis is to separate data from diverse classes using a hyperplane. The side of hyperplane determinded through the category of a feature vector in a two-class problem. LDA requires that the data has a normal distribution and that both classes have the same covariance matrix. The separation hyper-plane is based on looking for a projection that maximizes the margin between the means of two classes while minimizing intraclass variance [ 295 ]. Furthermore, this classifier is straightforward to apply and generally produces excellent results and soundly implemented in various BCI system, including MI-based BCI, P300 speller, multiclass, and asynchronous BCI. The disadvantage of LDA is its linearity, which might lead to unsatisfactory results when faced with various nonlinear EEG data.

8.1.2. Support Vector Machine (SVM)

A Support Vector Machine (SVM) uses a discriminant hyperplane to identify classes. The determined hyperplane in SVM is the one that maximizes the margins, i.e., the distance between both the nearest training samples. The ability to generalize is believed to improve when margins are maximized [ 296 ]. Linear SVM [ 297 ] is a type of SVM that allows for classification utilizing linear decision bounds. This classifier has been used to solve a substantial number of synchronous BCI tasks with tremendous success. The SVM classifier also works by projecting the input vector X onto a scalar value f(X), as shown in Equation ( 11 ).

Gaussian SVM or RBF SVM is the term applied to the equivalent SVM. RBF and SVM have also produced remarkable outcomes in BCI applications. SVM is used to solve multiclass BCI problems that use the OVR approach, similar to LDA.

8.2. Neural Networks (NN)

Neural networks (NN) and linear classifiers are the two types of classifiers most usually employed in BCI systems, considering that a NN is a collection of artificial neurons that allows us to create nonlinear decision limits [ 298 ]. The multilayer perceptron (MLP) is the most extensively used NN for BCI, as described in this section. Afterward, it briefly discusses other neural network architectures utilized in BCI systems.

8.2.1. Deep Learning (DL) Models

Deep learning has been widely used in BCI applications nowadays compared with machine learning technologies because most BCI applications require a high level of accuracy. Deep learning models perform better in recognizing changing signals from the brain, which changes swiftly. Some popular DL models such as CNN, GNN, RNN, and LSTM are described below:

Convolutional Neural Network (CNN): A convolutional neural network (CNN) is an ANN intended primarily to analyze visual input used in image recognition and processing. The convolutional layer, pooling layer, and fully connected layer are the three layers that comprise CNN. Using a CNN, the input data may be reduced to instant response formations with a minimum loss, and the characteristic spatial relationships of EEG patterns can be recorded. Fatigue detection, sleep stage classification, stress detection, motor imagery data processing, and emotion recognition are among the EEG-based BCI applications using CNNs. In BCI, the CNN models are used in the input brain signals to exploit the latent semantic dependencies.
Generative Adversarial Network (GAN): Generative adversarial networks are a recent ML technique. The GAN used two ANN models for competing to train each other simultaneously. GANs allow machines to envision and develop new images on their own. EEG-based BCI techniques recorded the signals first and then moved to the GAN techniques to regenerate the images [ 299 ]. The significant application of GAN-based BCI systems is data augmentation. Data augmentation increases the amount of training data available and allows for more complicated DL models. It can also reduce overfitting and can increase classifier accuracy and robustness. In the context of BCI, generative algorithms, including GAN, are frequently used to rebuild or generate a set of brain signal recordings to improve the training set.
Recurrent Neural Network (RNN): RNNs’ basic form is a layer with the output linked to the input. Since it has access to the data from past time-stamps, and the architecture of an RNN layer allows for the model to store memory [ 300 , 301 ]. Since RNN and CNN have strong temporal and spatial feature extraction abilities in most DL approaches, it is logical to mix them for temporal and spatial feature learning. RNN can be considered a more powerful version of hidden Markov models (HMM), which classifies EEG correctly [ 302 ]. LSTM is a kind of RNN with a unique architecture that allows it to acquire long-term dependencies despite the difficulties that RNNs confront. It contains a discrete memory cell, a type of node. To manage the flow of data, LSTM employs an architecture with a series of “gates”. When it comes to modeling time series of tasks such as writing and voice recognition, RNN and LSTM have been proven to be effective [ 303 ].

8.2.2. Multilayer Perceptron (MLP)

An Multilayer Perceptron (MLP) [ 304 ] comprises multiple layers of neurons along with an input layer, one or more hidden layers, and an output layer. The input of each neuron is linked to the output of the neurons in the preceding layer. Meanwhile, the output layer neurons evaluate the classification of the input feature vector. MLP and neural networks can approximate, meaning they can compare continuous functions if they have sufficient neurons and layers. The challenging factor behind MLPs is that they are susceptible to over-training, particularly containing noisy and non-stationary data. As a result, significant selection and regularization of the architecture are necessary. Perceptron is a multilayer with no hidden layers comparable with LDA. It has been used in BCI applications on occasion [ 293 ]. Sunny, M.S.H. et al. [ 305 ] used Multilayer Perceptron (MLP) to distinguish distinct frequency bands from EEG signals to extract features more effectively.

8.2.3. Adaptive Classifiers

As new EEG data become accessible, adaptive classifiers’ parameters, such as the weights allocated to each feature in a linear discriminant hyperplane, are gradually re-estimated and updated. Adaptive classifiers can use supervised and unsupervised adaptation, that is, with or without knowledge of the input data’s real class labels. The true class labels of the receiving EEG signals are obtained using supervised adaptation. The classifier is either reassigned on the existing training data, enhanced with these updated, labeled incoming data, or updated solely on this new data. Supervised user testing is essential for supervised BCI adaptation. The label of the receiving EEG data is vague with unsupervised adaptation. As a result, unsupervised adaptation is based on class-unspecific adaptation, such as updating the generalized classes EEG data mean or a co-variance matrix in the classifier model or estimating the data class labels for additional training [ 306 ].

8.3. Nonlinear Bayesian Classifiers

This section discusses the Bayes quadratic and hidden Markov models (HMM), two Bayesian classifiers used in BCI. Although Bayesian graphical networks (BGN) have been used for BCI, they are not covered here since they are not widely used [ 307 ].

8.3.1. Bayes Quadratic

The objective of Bayesian classification is to provide the highest probability class to a feature vector. The Bayes rule is often used to calculate the a posteriori probability of a feature vector assigned to a single class. The class of this feature vector can be calculated by using the MAP (maximum a posteriori) rule with these probabilities. The Bayes quadratic assumption is that the data have a distinct normal distribution. The result is quadratic decision boundaries that justify the classifier’s name [ 308 ]. Although this classifier is not extensively utilized for BCI, it has been successfully used to classify motor imagery and mental tasks.

8.3.2. Hidden Markov Model

A Bayesian classifier that generates a nonlinear cost function is known as a Hidden Markov Model (HMM). An HMM is a statistical algorithm that calculates the chances of seeing a given set of feature variables [ 309 ]. These statistical probabilities from HMM are generally Gaussian Mixture Models (GMM) in case of BCI [ 310 ]. HMM may be used to categorize temporal patterns of BCI characteristics (Obermaier, B. et al. [ 302 ]), even raw EEG data, since the EEG elements required to control BCI have particular time sequences. Although HMM is not widely used in the BCI world, this research demonstrated that they could be helpful to classification on BCI systems such as EEG signals [ 311 ].

8.4. Nearest Neighbor Classifiers

In this section, some classifiers with distance vectors are described. Classifiers such as K nearest neighbors (KNN) and Mahalanobis distance are common among them as they are nonlinear discriminative classifiers [ 312 ].

8.4.1. K Nearest Neighbors

K nearest neighbor method aims to identify the dominant class amongst an unseen point within the dataset habituated for training. Nearest neighbors are typically estimated using a metric that has some intervals during the signal acquisition of BCI. KNN can construct nonlinear decision boundaries by evaluating any function with enough training data with an inflated k value. The usability of KNN algorithms is less in the BCI field as their condescending sensitivity hampers the capacity, which causes them to fail in multiple BCI research. KNN is efficient in BCI systems with some feature vectors, but low power can cause failure in BCI research [ 313 ].

8.4.2. Mahalanobis Distance

For each prototype of class c , Mahalanobis distance-based classifiers [ 314 ] assume a Gaussian distribution N ( c , M c ) . Subsequently, using the Mahalanobis distance d c , a feature vector x is allocated to the class that corresponds to the closest prototype ( x ).

This results in a basic yet reliable classifier; it has been shown to work in multiclass and asynchronous BCI systems. Considering its excellent results, it is still rarely mentioned in BCI literature [ 315 ].

8.5. Hybrid

In several BCI papers, classification is implemented with a single classifier. Furthermore, a current tendency is to combine many classifiers in various ways [ 316 ]. The following are indeed the classifier combination strategies utilized in BCI systems:

8.5.1. Boosting

Boosting is the process of using multiple classifiers in a cascade, and each focused on the errors made by the one before it. It can combine numerous weak classifiers to form a powerful one; thereforem it is unlikely to overtrain. Moreover, it is susceptible to mislabeling, illustrating why it failed in one BCI trial [ 293 ].

8.5.2. Voting

Multiple classifiers are employed for voting, each of which allocates the input feature vector to a class. The majority class becomes the final class. In BCI systems, voting is the most preferred process of combining classifiers due to its simplicity and efficiency [ 293 ].

8.5.3. Stacking

Stacking is the process of utilizing multiple classifiers to categorize the input feature vector. Level-0 classifiers are what it is named. Each one of these classifiers’ output would then feed into a “meta-classifier” (or “level-1 classifier”), which makes a final decision [ 293 ].

Aforementioned in this section, some other classifiers are utilized in the recent BCI research. Since 2016 transfer learning is used for using MI classification tasks [ 317 ]. Some ground-breaking architectures are established in recent years, such as EEG-inception, an end-to-end Neural network [ 318 ], cluster decomposing, and multi-object optimization-based-ensemble learning framework [ 319 ]; RFNet is a fusion network that learns from attention weights and used in embedding-specific features for decision making [ 179 ].

Now, a better understanding of the performance of commonly known classifiers with some popular datasets are given in Table 9 .

Comparison of classifiers based on popular datasets and features.

9. Evaluation Measurement

To evaluate the performance of BCI systems, researchers employed several evaluation metrics. The most common is accuracy, commonly known as error rate. Although accuracy is not always an acceptable criterion due to specific rigorous requirements, various evaluation criteria have been offered. An overview of BCI research evaluation criteria is provided below.

9.1. Generally Used Evaluation Metrics

In this section, we sorted the most commonly used evaluation metrics for measuring the BCI system performances. The evaluation measures are explained carefully in the following subsections.

9.1.1. The Confusion Matrix

The confusion matrix represents the relationship between the actual class’s user-intentioned output classes and the actual predicted class. True positives rate (TPR), False negative rate (FNR), False positives rate (FPR), Positive predictive value (PPV), and negative predictive value (PPV) are used to describe sensitivity or recall, specificity, (1-specificity), precision, etc. [ 325 ].

9.1.2. Classification Accuracy and Error Rate

Classification accuracy is one of the important metrics in BCI systems; this study evaluates performance using classification accuracy as well as sensitivity and specificity. This measure determines how frequently the BCI makes a right pick or what proportion of all selections are accurate. It is the most obvious indicator of BCI accomplishment, implying that it increase in a linear fashion with decision time, so it takes a long time. The following is the mathematical formula for calculating accuracy:

9.1.3. Information Transfer Rate

Shannon [ 326 ] proposed the Information Transfer Rate (ITR) as the rate that makes up both of these metrics. This rate represents the quantity of data that may pass through the system in one unit of time. In [ 327 ], the information transmission rate in bits per minute ( b i t s / m i n ) and accuracy (ACC) in percentage (%) were used to evaluate performance. They made demographic data (age and gender) as well as the performance outcomes of 10 participants, and the ITR was computed using the Formula ( 14 ), which is as follows:

where N is the number of targets and p is the classification accuracy (ACC). Based on four cursor movements and the choose command, this resulted in a N of 5. Bits per trial were used to compute B t .

According to ITR [ 328 ] also has some important parameters that are used to evaluate BCI. A description of them is given below:

Target detection accuracy: The accuracy of target identification may be enhanced by increasing the Signal-to-Noise Ratio (SNR) and the separability of several classes. Several techniques, such as trial averaging, spatial filtering, and eliciting increased task-related EEG signals, are employed in the preprocessing step to reduce the SNR. Many applications utilize trail averaging across topics to improve the performance of a single BCI. These mental states may be used to lower the SNR [ 53 ].
Number of classes: The number of classes is raised and more sophisticated applications are built with a high ITR. TDMA, FDMA, and CDMA are among the stimulus coding techniques that have been adopted for BCI systems [ 243 , 329 ]. P300, for example, uses TDMA to code the target stimulus. In VEP-based BCI systems, FDMA and CDMA have been used.
Target detection time: The detection time is when a user first expresses their purpose and when the system makes a judgment. One of the goals of BCI systems is to improve the ITR by reducing target detection time. Adaptive techniques, such as the “dynamic halting” method, might be used to minimize the target detection time [ 330 ].

9.1.4. Cohen’s Kappa Coefficient

Cohen’s Kappa measures the agreement between two observers; it measures the contract between the proper output and the command of BCI domain in a BCI-based AAC system. Cohen’s kappa coefficient resolves many of the accuracy measure’s objections [ 331 ]. The general agreement p 0 = A C C , which is equivalent to the classification accuracy and the chance agreement p e , with n i and n i i being the column i t h and row i t h , correspondingly, are used to calculate K .

where posteriori and priori probability are n : i , n i : respectively. The estimated kappa Coefficient K and standard error e ( K ) are acquired by

When there is no correlation between the expected and actual classes, the kappa coefficient becomes zero. A perfect categorization is indicated by a kappa coefficient of 1. If the Kappa value is less than zero, the classifier offers an alternative assignment for the output and actual classes [ 332 ].

9.2. Continuous BCI System Evaluation

Continuous BCI performance was measured using a variety of parameters. Different measures may be even more appropriate depending on whether the study is conducted online or offline. The section goes through some of the most commonly used metrics in this field, including the correlation coefficient, accuracy, and Fitts’s Law [ 333 ].

9.2.1. Correlation Coefficient

The correlation coefficient could be a useful statistic for determining whether an intracortical implant receives task-relevant neurons. There are two essential stipulations: one is scale-invariant, which implies that the cursor might miss the mark substantially while still generating high values if the sign of the actual and anticipated movements coincide [ 334 ]; the other is that a decoder can yield a high value if it simply generates a signal that fluctuates with the repetitions [ 333 ].

9.2.2. Accuracy

Task characteristics such as target size and dwell time have a significant impact on accuracy. As a result, it is more of a sign that the task was is good enough for the subject and modality than a performance measure [ 333 ].

9.2.3. Fitts’s Law

Fitts’s law asserts that the time taken for a person to move a mouse cursor to a targeted object of the target’s distance is divided by its size. The longer it takes, the greater the distance and the narrower the target [ 335 , 336 ]. Fitts’s law requires using a method to calculate the “index of difficulty” of a particular change.

9.3. User-Centric BCI System Evaluation

Users are an essential element of the BCI product life cycle. Their interactions and experiences influence whether BCI systems are acceptable and viable. The four criteria or User Experience (UX) factors are used to evaluate user-centric BCI systems. These are usability, affects, ergonomics, and quality of life, shown below in the following subsection.

9.3.1. Usability

The amount that can be utilized to fulfill specific objectives with effectiveness, efficiency, learnability, and satisfaction in a given context is referred to as usability [ 337 ]. In usability measure, we can include four metrics, such as,

Effectiveness or accuracy: It depicts the overall accuracy of the BCI system as experienced from the end user’s perspective [ 333 ].
Efficiency or information transfer rate: It refers to the speed and timing at which a task is accomplished. Therefore, it depicts the overall BCI system’s speed, throughput, and latency seen through the eyes of the end user’s perspective [ 333 ].
Learnability: The BCI system can make users feel as if they can use the product effectively and quickly learn additional features. Both the end-user and the provider are affected by learnability [ 338 ].
Satisfaction: It is based on participants’ reactions to actual feelings while using BCI systems, showing the user’s favorable attitude regarding utilizing the system. To measure satisfaction, we can use rating scales or qualitative methods [ 333 ].

9.3.2. Affect

Regarding BCIs, it might refer to how comfortable the system is, particularly for long periods, and how pleasant or uncomfortable the stimuli are to them. EEG event-related possibilities, spectral characteristics, galvanic skin responses, or heart rates could be used to quantitatively monitor user’s exhaustion, valence, and arousal levels [ 339 ].

9.3.3. Ergonomics

Ergonomics studies are the study of how people interact with their environments. The load on the user’s memory is represented by the cognitive task load, a multidimensional entity. In addition, physiological markers including eye movement, EEG, ERP, and spectral characteristics could also be employed to evaluate cognitive stress objectively [ 340 ].

9.3.4. Quality of Life

It expresses the user’s overall perception of the system’s utility and acceptance and its influence on their well-being. The Return on Investment (ROI) is an economic measure of the perceived benefit derived from it. The overall quality of experience is a measure of how satisfied a user is with their expertise [ 333 ].

Other assessment methods, such as Mutual Information, Written symbol rate (WSR), and Practical bit rate (PBR), are utilized to a lesser extent.

10. Limitations and Challenges

The brain-computer interface is advancing towards a more dynamic and accurate solution of the connection between brain and machine. Still, few factors are resisting achieving the ultimate goal. Therefore, we analyzed a few core research on BCI in this section and found the limitations exhibited in Table 10 . Then, we demonstrated the significant challenges of the BCI domain.

A summary of some research papers proposing new methods of BCI.

The challenges and difficulties of the BCI domain are divided into three categories: challenges based on usability, technical challenges, and ethical challenges. The rest of the section briefly explains these challenges.

10.1. Based on Usability

This section describes the challenges that users have in accepting BCI technology [ 350 ]. They include concerns relating to the requisite training for class discrimination.

10.1.1. Training Time

Usually, training a user, either leading the user through the procedure or the total quantity of the documented manual, takes time. The majority of the time, the user also requests the system to be simpler to use. The users often despise a complicated system that is difficult to manage. It is a challenging effort to create such a sophisticated, user-friendly system [ 351 ].

10.1.2. Fatigue

The majority of present BCIs generate a lot of fatigue since they need a lot of concentration, focus, and awareness to a rapid and intermittent input. In addition to the annoyance of weariness of electrodes, BCI may fail to operate because the user cannot maintain a sufficient degree of focus. As in BCI, mental activity is continually monitored and the user’s attention point alters the input. The concentration necessary for stimuli results in a combination of input and output [ 352 , 353 ]. Rather than relaxing, the user must concentrate on a single point as an input and then look at the outcome. At some point, the interaction has a forced quality to it, rather than the natural quality that would be there if the user could choose whatever part of the visual output to focus on [ 6 ].

10.1.3. Mobility to Users

Across most situations, users are not allowed to move around or to have mobility in BCIs. During the test application, users must stay motionless and quiet, ideally sitting down. However, in a real-world setting, a user may need to utilize BCI while walking down the street, for example, to manage a smartphone. Additionally, BCIs cannot ensure user comfort. Usually, the EEG headset is not lightweight and easy to carry, which hampers the user experience.

10.1.4. Psychophysiological and Neurological Challenges

Emotional and mental mechanisms, cognition-related neurophysiology, and neurological variables, such as functionality and architecture, play vital roles in BCI performance, resulting in significant intra- and inter-individual heterogeneity. Immediate brain dynamics are influenced by psychological elements such as attention; memory load; weariness; conflicting cognitive functions; and users’ specific characteristics such as lifestyle, gender, and age. Participants with weaker empathy engage less emotionally in a P300-BCI paradigm and generate larger P300 wave amplitudes than someone with greater empathy involvement [ 354 ].

10.2. Technical Challenges

Non-linearity, non-stationarity, and noise as well as limited training sets and the accompanying dimensionality curse are difficulties relating to the recorded electrophysiological characteristics of brain impulses.

10.2.1. Non-Linearity

The brain is a very complex nonlinear system in which chaotic neuronal ensemble activity may be seen. Nonlinear dynamic techniques can thus better describe EEG data than linear ones.

10.2.2. Non-Stationarity

The non-stationarity of electrophysiological brain signals to recognize human recognition is a significant challenge in developing a BCI system. It results in a constant shift in the signals utilized with time, either between or within transition time. EEG signal variability can be influenced by the mental and emotional state backdrop across sessions. In addition, various emotional states such as sadness, happiness, anxiety, and fear can vary on daily basis that reflects non-stationarity [ 355 ]. Noise is also a significant contribution to the non-stationarity problems that BCI technology faces. Noises and other external interferences are always present in raw EEG data of emotion recognition that is most robust [ 356 ]. It comprises undesired signals generated by changes in electrode location as well as noise from the surroundings [ 357 ].

10.2.3. Transfer Rate of Signals

In BCIs, the system must continuously adjust to the signals of the user. This modification must be made quickly and precisely. Current BCIs have an extremely slow information transfer rate, taking almost two minutes to “digitalize” a single phrase, for example. Furthermore, BCI accuracy does not always reach a desirable level, particularly in visual stimulus-based BCI. Actions must sometimes be repeated or undone, producing pain or even dissatisfaction in using interactive systems using this type of interface [ 358 ].

10.2.4. Signal Processing

Recently, a variety of decoding techniques, signal processing algorithms, and classification algorithms have been studied. Despite this, the information retrieved from EEG waves does not have a high enough signal-to-noise ratio to operate a device with some extent of liberty, such as a prosthetic limb. Algorithms that are more resilient, accurate, and quick are required to control BCI.

10.2.5. Training Sets

In BCI, the training process is mainly impacted by usability concerns, but training sets are tiny in most cases. Although the subjects find the training sessions time-consuming and challenging, they give the user the required expertise to interact with the system and to learn to manage their neurophysiological signals. As a result, balancing the technological complexity of decoding the user’s brain activity with the level of training required for the proper functioning of the interfaces is a crucial issue in building a BCI [ 359 ].

10.2.6. Lack of Data Analysis Method

The classifiers should be evaluated online since every BCI implementation is in an online situation. Additionally, it should be validated to ensure that it has low complexity and can be calibrated rapidly in real-time. Domain adaptation and transfer learning could be an acceptable solution for developing calibration-free BCIs, where even the integration of unique feature sets, such as covariance matrices with domain adaptation algorithms, can strengthen the invariance performance of BCIs.

10.2.7. Performance Evaluation Metrics

A variety of performance evaluation measures are used to evaluate BCI systems. However, when different evaluation metrics are used to assess BCI systems, it is nearly impossible to compare systems. As a result, the BCI research community should establish a uniform and systematic approach to quantify a particular BCI application or a particular metric. For example, to test the efficiency of a BCI wheelchair control, the number of control commands, categories of control commands, total distance, time consumed, the number of collisions, classification accuracy, and the average success rate need to be evaluated, among other factors [ 360 ].

10.2.8. Low ITR of BCI Systems

The information transfer rate is one of the extensively used processes for the performance evaluation metrics of BCI systems. The number of classes, target detection accuracy, and target detection time are all factors of this rate. By increasing the Signal-to-Noise Ratio (SNR), it can improve the target detection accuracy [ 53 , 328 ]. Several techniques are typically used for the preprocessing phase to optimize the SNR. When a high ITR has been attained, more complicated applications can be created by expanding the number of classes available. CDMA, TDMA, and FDMA [ 243 , 361 ] are only a few of the stimulus coding schemes that have already been developed for BCI systems. TDMA was used with P300 to code the required stimuli, while CDMA and FDMA have been used with BCIs that interact with VEP. Furthermore, the essential aspect of BCIs is reducing the target recognition period, which helps to increase the ITR. Adaptive techniques, such as “dynamic stopping”, could be an effective option for accomplishing this.

10.2.9. Specifically Allocated Lab for BCI Technology

Most of the BCI systems are trialed in a supervised lab rather than in the actual surroundings of the users. When designing a BCI system, it is essential to think about the environment in which the technology may be used. It is critical to thoroughly investigate the system’s requirements, environmental factors, circumstances, and target users mostly during the system design phase.

10.3. Ethical Challenges

There are many thoughts surrounding the ethical issues behind BCI as it considers physical, psychological, and social factors. In biological factors, BCI always finds a human body to identify signals that must be acquainted with electrodes. As humans need to wear these electrodes, it is always risky for them and can harm the human body to some worse extent. BCI also requires strict maintenance of the human body during signal acquisition, so the subject must sit for a long time in his place. Adding to that, a user or participant must act what the electrodes need, so they cannot do anything willingly. This fact can have a substantial impact on the human body.

11. Conclusions

The brain-computer interface is a communication method that joins the wired brain and external applications and devices directly. The BCI domain includes investigating, assisting, augmenting, and experimenting with brain signal activities. Due to transatlantic documentation, low-cost amplifiers, greater temporal resolution, and superior signal analysis methods, BCI technologies are available to researchers in diverse domains. Moreover, It is an interdisciplinary area that allows for biology, engineering, computer science, and applied mathematics research. However, an architectural and constructive investigation of the brain–computer interface is exhibited in this article. It is aimed at novices who would like to learn about the current state of BCI systems and methodologies. The fundamental principles of BCI techniques are discussed elaborately. It describes the architectural perspectives of certain unique taxons and gives a taxonomy of BCI systems. The paper also covered feature extraction, classification, evaluation procedures, and techniques as the research continues. It presents a summary of the present methods for creating various types of BCI systems. The study looks into the different types of datasets that are available for BCI systems as well. The article also explains the challenges and limitations of the described BCI systems, along with possible solutions. Lastly, BCI technology advancement is accomplished in four stages: primary scientific development, preclinical experimentation, clinical investigation, and commercialization. At present, most of the BCI techniques are in the preclinical and clinical phases. The combined efforts of scientific researchers and the tech industries are needed to avail the benefit of this great domain to ordinary people through commercialization.

Acknowledgments

We would like to thank Bangladesh University of Business & Technology (BUBT), University of Asia Pacific (UAP), and University of Aizu (UoA) for supporting this research. Also, special thanks to the Advanced Machine Learning lab, BUBT; Computer Vision & Pattern Recognition Lab, UAP; Database System Lab, UoA; for giving facilities to research and publish.

Author Contributions

Conceptualization, M.F.M.; Data curation, M.F.M., S.C.D., M.M.K. and A.A.L.; Formal analysis, M.F.M.; Investigation, M.R.I. and Y.W.; Methodology, M.F.M., S.C.D., M.M.K., A.A.L., M.R.I. and Y.W.; Software, S.C.D., M.M.K. and A.A.L.; Supervision, M.R.I.; Validation, M.F.M., M.R.I. and Y.W.; Visualization, M.F.M., S.C.D., M.M.K. and A.A.L.; Writing—original draft, M.F.M., S.C.D., M.M.K., A.A.L., M.R.I. and Y.W.; Writing—review & editing, M.F.M., M.R.I. and Y.W. All authors have read and agreed to the published version of the manuscript.

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Data availability statement, conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

To revisit this article, visit My Profile, then View saved stories .

Backchannel
Newsletters
WIRED Insider
WIRED Consulting

By Steven Levy

8 Google Employees Invented Modern AI. Here’s the Inside Story

Eight names are listed as authors on “Attention Is All You Need,” a scientific paper written in the spring of 2017. They were all Google researchers, though by then one had left the company. When the most tenured contributor, Noam Shazeer, saw an early draft, he was surprised that his name appeared first, suggesting his contribution was paramount. “I wasn’t thinking about it,” he says.

Image may contain Art Drawing Adult Person Face Head Photography and Portrait

It’s always a delicate balancing act to figure out how to list names—who gets the coveted lead position, who’s shunted to the rear. Especially in a case like this one, where each participant left a distinct mark in a true group effort. As the researchers hurried to finish their paper, they ultimately decided to “sabotage” the convention of ranking contributors. They added an asterisk to each name and a footnote: “Equal contributor,” it read. “Listing order is random.” The writers sent the paper off to a prestigious artificial intelligence conference just before the deadline—and kicked off a revolution.

Approaching its seventh anniversary, the “Attention” paper has attained legendary status. The authors started with a thriving and improving technology—a variety of AI called neural networks—and made it into something else: a digital system so powerful that its output can feel like the product of an alien intelligence . Called transformers, this architecture is the not-so-secret sauce behind all those mind-blowing AI products , including ChatGPT and graphic generators such as Dall-E and Midjourney. Shazeer now jokes that if he knew how famous the paper would become, he “might have worried more about the author order.” All eight of the signers are now microcelebrities. “I have people asking me for selfies—because I’m on a paper!” says Llion Jones, who is (randomly, of course) name number five.

Image may contain Art Drawing Adult Person Accessories Glasses Face Head Photography and Portrait

“Without transformers I don’t think we’d be here now,” says Geoffrey Hinton , who is not one of the authors but is perhaps the world’s most prominent AI scientist . He’s referring to the ground-shifting times we live in, as OpenAI and other companies build systems that rival and in some cases surpass human output.

All eight authors have since left Google. Like millions of others, they are now working in some way with systems powered by what they created in 2017. I talked to the Transformer Eight to piece together the anatomy of a breakthrough, a gathering of human minds to create a machine that might well save the last word for itself.

The story of transformers begins with the fourth of the eight names: Jakob Uszkoreit.

Uszkoreit is the son of Hans Uszkoreit, a well-known computational linguist. As a high school student in the late 1960s, Hans was imprisoned for 15 months in his native East Germany for protesting the Soviet invasion of Czechoslovakia. After his release, he escaped to West Germany and studied computers and linguistics in Berlin. He made his way to the US and was working in an artificial intelligence lab at SRI, a research institute in Menlo Park, California, when Jakob was born. The family eventually returned to Germany, where Jakob went to university. He didn’t intend to focus on language, but as he was embarking on graduate studies, he took an internship at Google in its Mountain View office, where he landed in the company’s translation group. He was in the family business. He abandoned his PhD plans and, in 2012, decided to join a team at Google that was working on a system that could respond to users’ questions on the search page itself without diverting them to other websites. Apple had just announced Siri, a virtual assistant that promised to deliver one-shot answers in casual conversation, and the Google brass smelled a huge competitive threat: Siri could eat up their search traffic. They started paying a lot more attention to Uszkoreit’s new group.

“It was a false panic,” Uszkoreit says. Siri never really threatened Google. But he welcomed the chance to dive into systems where computers could engage in a kind of dialog with us. At the time, recurrent neural networks—once an academic backwater—had suddenly started outperforming other methods of AI engineering. The networks consist of many layers, and information is passed and repassed through those layers to identify the best responses. Neural nets were racking up huge wins in fields such as image recognition, and an AI renaissance was suddenly underway. Google was frantically rearranging its workforce to adopt the techniques. The company wanted systems that could churn out humanlike responses—to auto-complete sentences in emails or create relatively simple customer service chatbots.

Hackers Found a Way to Open Any of 3 Million Hotel Keycard Locks in Seconds

Andy Greenberg

Apple Chip Flaw Leaks Secret Encryption Keys

Andrew Couts

Netflix’s 3 Body Problem Adapts the Unadaptable

Amit Katwala

Stardew Valley’s 1.6 Update Is Out&-Here Are Some of the Biggest Changes

Nena Farrell

But the field was running into limitations. Recurrent neural networks struggled to parse longer chunks of text. Take a passage like Joe is a baseball player, and after a good breakfast he went to the park and got two hits. To make sense of “two hits,” a language model has to remember the part about baseball. In human terms, it has to be paying attention. The accepted fix was something called “long short-term memory” (LSTM), an innovation that allowed language models to process bigger and more complex sequences of text. But the computer still handled those sequences strictly sequentially—word by tedious word—and missed out on context clues that might appear later in a passage. “The methods we were applying were basically Band-Aids,” Uszkoreit says. “We could not get the right stuff to really work at scale.”

Around 2014, he began to concoct a different approach that he referred to as self-attention. This kind of network can translate a word by referencing any other part of a passage. Those other parts can clarify a word’s intent and help the system produce a good translation. “It actually considers everything and gives you an efficient way of looking at many inputs at the same time and then taking something out in a pretty selective way,” he says. Though AI scientists are careful not to confuse the metaphor of neural networks with the way the biological brain actually works, Uszkoreit does seem to believe that self-attention is somewhat similar to the way humans process language.

Uszkoreit thought a self-attention model could potentially be faster and more effective than recurrent neural nets. The way it handles information was also perfectly suited to the powerful parallel processing chips that were being produced en masse to support the machine learning boom. Instead of using a linear approach (look at every word in sequence), it takes a more parallel one (look at a bunch of them together). If done properly, Uszkoreit suspected, you could use self-attention exclusively to get better results.

Not everyone thought this idea was going to rock the world, including Uszkoreit’s father, who had scooped up two Google Faculty research awards while his son was working for the company. “People raised their eyebrows, because it dumped out all the existing neural architectures,” Jakob Uszkoreit says. Say goodbye to recurrent neural nets? Heresy! “From dinner-table conversations I had with my dad, we weren’t necessarily seeing eye to eye.”

Uszkoreit persuaded a few colleagues to conduct experiments on self-attention. Their work showed promise, and in 2016 they published a paper about it. Uszkoreit wanted to push their research further—the team’s experiments used only tiny bits of text—but none of his collaborators were interested. Instead, like gamblers who leave the casino with modest winnings, they went off to apply the lessons they had learned. “The thing worked ,” he says. “The folks on that paper got excited about reaping the rewards and deploying it in a variety of different places at Google, including search and, eventually, ads. It was an amazing success in many ways, but I didn’t want to leave it there.”

Uszkoreit felt that self-attention could take on much bigger tasks. There’s another way to do this , he’d argue to anyone who would listen, and some who wouldn’t, outlining his vision on whiteboards in Building 1945, named after its address on Charleston Road on the northern edge of the Google campus.

Image may contain Art Drawing Face Head Person Photography Portrait and Adult

One day in 2016, Uszkoreit was having lunch in a Google café with a scientist named Illia Polosukhin. Born in Ukraine, Polosukhin had been at Google for nearly three years. He was assigned to the team providing answers to direct questions posed in the search field. It wasn’t going all that well. “To answer something on Google.com, you need something that’s very cheap and high-performing,” Polosukhin says. “Because you have milliseconds” to respond. When Polosukhin aired his complaints, Uszkoreit had no problem coming up with a remedy. “He suggested, why not use self-attention?” says Polosukhin.

Polosukhin sometimes collaborated with a colleague named Ashish Vaswani. Born in India and raised mostly in the Middle East, he had gone to the University of Southern California to earn his doctorate in the school’s elite machine translation group. Afterward, he moved to Mountain View to join Google—specifically a newish organization called Google Brain . He describes Brain as “a radical group” that believed “neural networks were going to advance human understanding.” But he was still looking for a big project to work on. His team worked in Building 1965 next door to Polosukhin’s language team in 1945, and he heard about the self-attention idea. Could that be the project? He agreed to work on it.

Together, the three researchers drew up a design document called “Transformers: Iterative Self-Attention and Processing for Various Tasks.” They picked the name “transformers” from “day zero,” Uszkoreit says. The idea was that this mechanism would transform the information it took in, allowing the system to extract as much understanding as a human might—or at least give the illusion of that. Plus Uszkoreit had fond childhood memories of playing with the Hasbro action figures. “I had two little Transformer toys as a very young kid,” he says. The document ended with a cartoony image of six Transformers in mountainous terrain, zapping lasers at one another.

There was also some swagger in the sentence that began the paper: “We are awesome.”

In early 2017, Polosukhin left Google to start his own company. By then new collaborators were coming onboard. An Indian engineer named Niki Parmar had been working for an American software company in India when she moved to the US. She earned a master’s degree from USC in 2015 and was recruited by all the Big Tech companies. She chose Google. When she started, she joined up with Uszkoreit and worked on model variants to improve Google search.

Another new member was Llion Jones. Born and raised in Wales, he loved computers “because it was not normal.” At the University of Birmingham he took an AI course and got curious about neural networks, which were presented as a historical curiosity. He got his master’s in July 2009 and, unable to find a job during the recession, lived on the dole for months. He found a job at a local company and then applied to Google as a “hail Mary.” He got the gig and eventually landed in Google Research, where his manager was Polosukhin. One day, Jones heard about the concept of self-attention from a fellow worker named Mat Kelcey, and he later joined up with Team Transformers. (Later, Jones ran into Kelcey and briefed him on the transformer project. Kelcey wasn’t buying it. “I told him, ‘I’m not sure that’s going to work,’ which is basically the biggest incorrect prediction of my life,” Kelcey says now.)

The transformer work drew in other Google Brain researchers who were also trying to improve large language models . This third wave included Łukasz Kaiser, a Polish-born theoretical computer scientist, and his intern, Aidan Gomez. Gomez had grown up in a small farming village in Ontario, Canada, where his family would tap maple trees every spring for syrup. As a junior at the University of Toronto, he “fell in love” with AI and joined the machine learning group—Geoffrey Hinton’s lab. He began contacting people at Google who had written interesting papers, with ideas for extending their work. Kaiser took the bait and invited him to intern. It wasn’t until months later that Gomez learned those internships were meant for doctoral students, not undergrads like him.

Kaiser and Gomez quickly understood that self-attention looked like a promising, and more radical, solution to the problem they were addressing. “We had a deliberate conversation about whether we wanted to merge the two projects,” says Gomez. The answer was yes.

The transformer crew set about building a self-attention model to translate text from one language to another. They measured its performance using a benchmark called BLEU, which compares a machine’s output to the work of a human translator. From the start, their new model did well. “We had gone from no proof of concept to having something that was at least on par with the best alternative approaches to LSTMs by that time,” Uszkoreit says. But compared to long short-term memory, “it wasn’t better.”

They had reached a plateau—until one day in 2017, when Noam Shazeer heard about their project, by accident. Shazeer was a veteran Googler—he’d joined the company in 2000—and an in-house legend, starting with his work on the company’s early ad system. Shazeer had been working on deep learning for five years and recently had become interested in large language models. But these models were nowhere close to producing the fluid conversations that he believed were possible.

As Shazeer recalls it, he was walking down a corridor in Building 1965 and passing Kaiser’s workspace. He found himself listening to a spirited conversation. “I remember Ashish was talking about the idea of using self-attention, and Niki was very excited about it. I’m like, wow, that sounds like a great idea. This looks like a fun, smart group of people doing something promising.” Shazeer found the existing recurrent neural networks “irritating” and thought: “Let’s go replace them!”

Shazeer’s joining the group was critical. “These theoretical or intuitive mechanisms, like self-attention, always require very careful implementation, often by a small number of experienced ‘magicians,’ to even show any signs of life,” says Uszkoreit. Shazeer began to work his sorcery right away. He decided to write his own version of the transformer team’s code. “I took the basic idea and made the thing up myself,” he says. Occasionally he asked Kaiser questions, but mostly, he says, he “just acted on it for a while and came back and said, ‘Look, it works.’” Using what team members would later describe with words like “magic” and “alchemy” and “bells and whistles,” he had taken the system to a new level.

“That kicked off a sprint,” says Gomez. They were motivated, and they also wanted to hit an upcoming deadline—May 19, the filing date for papers to be presented at the biggest AI event of the year, the Neural Information Processing Systems conference in December. As what passes for winter in Silicon Valley shifted to spring, the pace of the experiments picked up. They tested two models of transformers: one that was produced with 12 hours of training and a more powerful version called Big that was trained over three and a half days. They set them to work on English-to-German translation.

The basic model outperformed all competitors—and Big earned a BLEU score that decisively shattered previous records while also being more computationally efficient. “We had done it in less time than anyone out there,” Parmar says. “And that was only the beginning, because the number kept improving.” When Uszkoreit heard this, he broke out an old bottle of champagne he had lying around in his mountain expedition truck.

The last two weeks before the deadline were frantic. Though officially some of the team still had desks in Building 1945, they mostly worked in 1965 because it had a better espresso machine in the micro-kitchen. “People weren’t sleeping,” says Gomez, who, as the intern, lived in a constant debugging frenzy and also produced the visualizations and diagrams for the paper. It’s common in such projects to do ablations—taking things out to see whether what remains is enough to get the job done.

“There was every possible combination of tricks and modules—which one helps, which doesn’t help. Let’s rip it out. Let’s replace it with this,” Gomez says. “Why is the model behaving in this counterintuitive way? Oh, it’s because we didn’t remember to do the masking properly. Does it work yet? OK, move on to the next. All of these components of what we now call the transformer were the output of this extremely high-paced, iterative trial and error.” The ablations, aided by Shazeer’s implementations, produced “something minimalistic,” Jones says. “Noam is a wizard.”

Vaswani recalls crashing on an office couch one night while the team was writing the paper. As he stared at the curtains that separated the couch from the rest of the room, he was struck by the pattern on the fabric, which looked to him like synapses and neurons. Gomez was there, and Vaswani told him that what they were working on would transcend machine translation. “Ultimately, like with the human brain, you need to unite all these modalities—speech, audio, vision—under a single architecture,” he says. “I had a strong hunch we were onto something more general.”

In the higher echelons of Google, however, the work was seen as just another interesting AI project. I asked several of the transformers folks whether their bosses ever summoned them for updates on the project. Not so much. But “we understood that this was potentially quite a big deal,” says Uszkoreit. “And it caused us to actually obsess over one of the sentences in the paper toward the end, where we comment on future work.”

That sentence anticipated what might come next—the application of transformer models to basically all forms of human expression. “We are excited about the future of attention-based models,” they wrote. “We plan to extend the transformer to problems involving input and output modalities other than text” and to investigate “images, audio and video.”

A couple of nights before the deadline, Uszkoreit realized they needed a title. Jones noted that the team had landed on a radical rejection of the accepted best practices, most notably LSTMs, for one technique: attention. The Beatles, Jones recalled, had named a song “All You Need Is Love.” Why not call the paper “Attention Is All You Need”?

The Beatles?

“I’m British,” says Jones. “It literally took five seconds of thought. I didn’t think they would use it.”

They continued collecting results from their experiments right up until the deadline. “The English-French numbers came, like, five minutes before we submitted the paper,” says Parmar. “I was sitting in the micro-kitchen in 1965, getting that last number in.” With barely two minutes to spare, they sent off the paper.

Google, as almost all tech companies do, quickly filed provisional patents on the work. The reason was not to block others from using the ideas but to build up its patent portfolio for defensive purposes. (The company has a philosophy of “if technology advances, Google will reap the benefits.”)

When the transformer crew heard back from the conference peer reviewers, the response was a mix. “One was positive, one was extremely positive, and one was, ‘This is OK,’” says Parmar. The paper was accepted for one of the evening poster sessions.

By December, the paper was generating a buzz. Their four-hour session on December 6 was jammed with scientists wanting to know more. The authors talked until they were hoarse. By 10:30 pm, when the session closed, there was still a crowd. “Security had to tell us to leave,” says Uszkoreit. Perhaps the most satisfying moment for him was when computer scientist Sepp Hochreiter came up and praised the work—quite a compliment, considering that Hochreiter was the coinventor of long short-term memory, which transformers had just booted as the go-to hammer in the AI toolkit.

Transformers did not instantly take over the world, or even Google. Kaiser recalls that around the time of the paper’s publication, Shazeer proposed to Google executives that the company abandon the entire search index and train a huge network with transformers—basically to transform how Google organizes information. At that point, even Kaiser considered the idea ridiculous. Now the conventional wisdom is that it’s a matter of time .

A startup called OpenAI was much faster to pounce . Soon after the paper was published, OpenAI’s chief researcher, Ilya Sutskever—who had known the transformer team during his time at Google—suggested that one of its scientists, Alec Radford, work on the idea. The results were the first GPT products. As OpenAI CEO Sam Altman told me last year, “When the transformer paper came out, I don’t think anyone at Google realized what it meant.”

The picture internally is more complicated. “It was pretty evident to us that transformers could do really magical things,” says Uszkoreit. “Now, you may ask the question, why wasn’t there ChatGPT by Google back in 2018? Realistically, we could have had GPT-3 or even 3.5 probably in 2019, maybe 2020. The big question isn’t, did they see it? The question is, why didn’t we do anything with the fact that we had seen it? The answer is tricky.”

Many tech critics point to Google’s transition from an innovation-centered playground to a bottom-line-focused bureaucracy. As Gomez told the Financial Times , “They weren’t modernizing. They weren’t adopting this tech.” But that would have taken a lot of daring for a giant company whose technology led the industry and reaped huge profits for decades. Google did begin to integrate transformers into products in 2018, starting with its translation tool. Also that year, it introduced a new transformer-based language model called BERT, which it started to apply to search the year after.

But these under-the-hood changes seem timid compared to OpenAI’s quantum leap and Microsoft’s bold integration of transformer-based systems into its product line. When I asked CEO Sundar Pichai last year why his company wasn’t first to launch a large language model like ChatGPT, he argued that in this case Google found it advantageous to let others lead. “It’s not fully clear to me that it might have worked out as well. The fact is, we can do more after people had seen how it works,” he said.

There is the undeniable truth that all eight authors of the paper have left Google. Polosukhin’s company, Near, built a blockchain whose tokens have a market capitalization around $4 billion. Parmar and Vaswani paired up as business partners in 2021 to start Adept (estimated valuation of $1 billion) and are now on their second company, called Essential AI ($8 million in funding). Llion Jones’ Tokyo-based Sakana AI is valued at $200 million. Shazeer, who left in October 2021, cofounded Character AI (estimated valuation of $5 billion). Aidan Gomez, the intern in the group, cofounded Cohere in Toronto in 2019 (estimated valuation of $2.2 billion). Jakob Uszkoreit’s biotech company, Inceptive, is valued at $300 million. All those companies (except Near) are based on transformer technology.

Image may contain Art Drawing Face Head Person Photography and Portrait

Kaiser is the only one who hasn’t founded a company. He joined OpenAI and is one of the inventors of a new technology called Q* , which Altman said last year will “push the veil of ignorance back and the frontier of discovery forward.” (When I attempted to quiz Kaiser on this in our interview, the OpenAI PR person almost leaped across the table to silence him.)

Does Google miss these escapees? Of course, in addition to others who have migrated from the company to new AI startups. (Pichai reminded me, when I asked him about the transformer departures, that industry darling OpenAI also has seen defections: “The AI area is very, very dynamic,” he said.) But Google can boast that it created an environment that supported the pursuit of unconventional ideas. “In a lot of ways Google has been way ahead—they invested in the right minds and created the environment where we could explore and push the envelope,” Parmar says. “It’s not crazy that it took time to adopt it. Google had so much more at stake.”

Without that environment: no transformer. Not only were the authors all Google employees, they also worked out of the same offices. Hallway encounters and overheard lunch conversations led to big moments. The group is also culturally diverse. Six of the eight authors were born outside the United States; the other two are children of two green-card-carrying Germans who were temporarily in California and a first-generation American whose family had fled persecution, respectively.

Uszkoreit, speaking from his office in Berlin, says that innovation is all about the right conditions. “It’s getting people who are super excited about something who are at the right point in their life,” he says. “If you have that and have fun while you do it, and you’re working on the right problems—and you’re lucky—the magic happens.”

Something magical also happened between Uszkoreit and his famous father. After all those dinner table debates, Hans Uszkoreit, his son reports, has now cofounded a company that is building large language models. Using transformers, of course.

Let us know what you think about this article. Submit a letter to the editor at [email protected] . Updated March 21, 2024, 10 pm EDT: This article was updated to correct the spelling of Alec Radford's name.

You Might Also Like …

In your inbox: Will Knight's Fast Forward explores advances in AI

This shadowy firm enables businesses to operate in near-total secrecy

Scientists are inching closer to bringing back the woolly mammoth

The first rule of the Extreme Dishwasher Loading Facebook group is …

Phones for every budget: These devices stood up to WIRED’s testing

Lauren Goode

Robert Peck

Javier Bardem Is Menacing and Thrilling in Dune: Part Two&-and a Soulful Teddy Bear IRL

Hemal Jhaveri

How the Pentagon Learned to Use Targeted Ads to Find Its Targets&-and Vladimir Putin

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
My Account Login
Explore content
About the journal
Publish with us
Sign up for alerts
Consensus Statement
Open access
Published: 07 February 2019

POSITION PAPER

Challenges to curing primary brain tumours

Kenneth Aldape 1 ,
Kevin M. Brindle 2 ,
Louis Chesler 3 ,
Rajesh Chopra 3 ,
Amar Gajjar 4 ,
Mark R. Gilbert 5 ,
Nicholas Gottardo 6 ,
David H. Gutmann 7 ,
Darren Hargrave 8 ,
Eric C. Holland 9 ,
David T. W. Jones 10 ,
Johanna A. Joyce 11 ,
Pamela Kearns 12 ,
Mark W. Kieran 13 ,
Ingo K. Mellinghoff 14 ,
Melinda Merchant 15 ,
Stefan M. Pfister 16 ,
Steven M. Pollard 17 ,
Vijay Ramaswamy ORCID: orcid.org/0000-0002-6557-895X 18 ,
Jeremy N. Rich 19 ,
Giles W. Robinson ORCID: orcid.org/0000-0001-7441-9486 4 ,
David H. Rowitch 20 ,
John H. Sampson 21 ,
Michael D. Taylor 22 ,
Paul Workman 3 &
Richard J. Gilbertson ORCID: orcid.org/0000-0001-7539-9472 2 , 23

Nature Reviews Clinical Oncology volume 16 , pages 509–520 ( 2019 ) Cite this article

43k Accesses

492 Citations

128 Altmetric

Metrics details

Cancer models
Cancer therapy
Translational research

Despite decades of research, brain tumours remain among the deadliest of all forms of cancer. The ability of these tumours to resist almost all conventional and novel treatments relates, in part, to the unique cell-intrinsic and microenvironmental properties of neural tissues. In an attempt to encourage progress in our understanding and ability to successfully treat patients with brain tumours, Cancer Research UK convened an international panel of clinicians and laboratory-based scientists to identify challenges that must be overcome if we are to cure all patients with a brain tumour. The seven key challenges summarized in this Position Paper are intended to serve as foci for future research and investment.

Evolutionary trajectories of small cell lung cancer under therapy

Julie George, Lukas Maas, … Roman K. Thomas

KRAS mutation: from undruggable to druggable in cancer

Lamei Huang, Zhixing Guo, … Liwu Fu

Molecular principles of metastasis: a hallmark of cancer revisited

Jawad Fares, Mohamad Y. Fares, … Youssef Fares

Introduction

Brain tumours are among the most feared of all forms of cancer. More than two-thirds of adults diagnosed with glioblastoma — the most aggressive type of brain cancer — will die within 2 years of diagnosis 1 , 2 . Brain cancers are also the most common and most lethal of all paediatric solid tumours 3 . Furthermore, children with these tumours who survive and enter adulthood will often be affected by the long-term consequences of exposing the developing brain to medical interventions, including surgery, radiotherapy and/or chemotherapy 4 , 5 .

Brain tumours have proved challenging to treat, largely owing to the biological characteristics of these cancers, which often conspire to limit progress. First, by infiltrating one of the body’s most crucial organs, these tumours are often located beyond the reach of even the most skilled neurosurgeon. These tumours are also located behind the blood–brain barrier (BBB) — a system of tight junctions and transport proteins that protect delicate neural tissues from exposure to factors in the general circulation, thus also impeding exposure to systemic chemotherapy 6 , 7 . Furthermore, the unique developmental, genetic, epigenetic and microenvironmental features of the brain frequently render these cancers resistant to conventional and novel treatments alike 8 , 9 , 10 . These challenges are compounded by the rarity of brain tumours relative to many other forms of cancer, which limits the level of funding and interest from the pharmaceutical industry and attracts a relatively small and fragmented research community.

To begin to address these issues and improve the outcomes of patients with brain tumours, Cancer Research UK (CRUK) convened an international panel of brain cancer researchers with interests in neurobiology, preclinical tumour modelling, genomics, pharmacology, drug discovery and/or development, neuropathology, neurosurgery, imaging, radiotherapy and medical oncology, with the task of identifying the most important challenges that must be overcome if we are to eventually be in the position to cure all patients with a brain tumour. In this Position Paper, we summarize seven key challenges identified by the panel that should serve as the foci for future research and investment (Fig. 1 ). Each of these challenges is worthy of extensive discussion and review, which are beyond the scope of this manuscript. Therefore, we highlight these challenges as a ‘call-to-arms’, summarizing the nature and importance of each challenge rather than providing an exhaustive review of the current level of understanding.

The nonmalignant cellular composition of the ventricular–subventricular zone includes neural stem cells that divide and produce transit-amplifying cells. Transit-amplifying cells give rise to migratory neuroblasts 90 . Ependymal cells are also present in the neural stem cell niche. The niche is intimately associated with blood vessels and might also communicate with other cell types, including microglia and astrocytes. Malignant transformation of neural stem cells presumably leads to the premalignant expansion of transit-amplifying cells and migratory neuroblasts as the nonmalignant hierarchy begins to transform. This unregulated hierarchy generates the malignant brain tumour. As these lesions are treated, tumour cells are killed, with the ultimate goal of eventual cure. The blue panels below these cartoons denote the focus of key research questions at each stage, from nonmalignant brain tissue to the development of cancer and remission following successful treatment. The green panels depict how the seven challenges to progress relate to these specific stages of disease development.

Challenge 1: redesign research pipeline

Clinical trials have yet to reveal an effective therapy for most brain tumours. This harsh reality stems, in part, from an incomplete understanding of brain tumour biology and the existence of a disconnect between preclinical drug development and rigorous testing in the clinic. Each element of the brain tumour research pipeline, from basic neurobiology to clinical trials, requires careful scrutiny and increased investment, although the development of an overarching strategy that facilitates and promotes interdisciplinary research is equally important (Fig. 1 ). This strategy would bring an end to the previous ‘siloed’ organization of working practices, in which basic and clinical researchers performed their studies independently and collaborated only when laboratory research was judged to be ready for the clinic or when the laboratory is engaged to understand the reasons why a promising drug failed to achieve the expected level of efficacy in clinical trials. Much deeper, longitudinal collaboration is essential in order to drive progress as rapidly as possible.

Nascent attempts to unify the brain tumour research pipeline are underway. The international paediatric brain tumour community has demonstrated remarkable levels of collaborative activity over many years and is now working to discover and define the genomic subtypes of paediatric brain tumours and form multidisciplinary teams in order to design preclinical studies that better inform the design of clinical trials 11 , 12 . However, the community must go further, by forming international collaborations that extend beyond the borders of traditional research disciplines and by engaging the entire breadth of expertise available within the biological and physical sciences (see Challenge 2). Such congregations of experts could provide a more comprehensive understanding of the workings of the brain and how these processes are subverted during malignant transformation.

The community should also explore innovative methods of designing and delivering clinical trials. Interest among the brain tumour research community in the use of adaptive trial designs is particularly welcomed 13 , 14 . Such trials could provide prospective opportunities to modify ongoing studies and integrate the investigation of new hypotheses as data are accumulated and analysed (Fig. 2 ). This approach has distinct advantages over the less flexible, traditional trial designs, which are typically designed to test one or two primary hypotheses, typically in heavily pretreated patients over a number of years. However, several hurdles must be overcome before novel trial designs can be implemented successfully, including identifying a cadre of rational therapies to feed into these new trials; developing robust biomarker-based patient selection criteria; constructing systems that enable the provision of real-time, in-trial, biomarker and end point data; and formulating optimal mechanisms of generating and sharing complex research data among centres worldwide (Fig. 2 ). Notwithstanding these challenges, improving the current approach to clinical trials is an urgent objective for the community. In the context of small, genomically defined patient subgroups, international collaborative clinical trials are crucial. A comprehensive review of the current logistical, regulatory and financial hurdles that inhibit the initiation of such trials is therefore warranted.

Patients with different molecular subtypes of a specific type of brain tumour (represented as blue, red and green figures) are enrolled into the trial. Patients in the control group receive the optimal standard of care. New treatments can then be tested as comparators in groups A and B. Following a defined period of treatment, the responses of patients in these groups are compared with those of patients in the control group. Responding patients are represented by bold colours; nonresponding patients are coloured grey. In this example, new treatment B produced a biased favourable response in patients with a single disease subtype (blue figures) relative to the control treatment. No advantage of new treatment A is observed relative to the control treatment. Therefore, in the next phase of the study, the new treatment A group is discontinued, while the new treatment B arm proceeds, with enrichment for patients with tumours of the ‘blue’ subtype. The control group continues with randomized recruitment of patients with all disease subtypes. The success of this approach is dependent upon addressing the key challenges summarized in the thought bubbles emerging at each stage of the trial. Many of these challenges are yet to be addressed in patients with brain tumours.

Greater integration of research disciplines will also require a change in the current research culture. The principal metrics used to reward success in most academic environments are focused on competitive activities, such as publication in high-impact journals and the acquisition of grant funding. These metrics have some value in enabling the assessment and quantification of the extent of innovative thought and discovery, although they arguably also work against the development of the deeper levels of collaborative activity that will ultimately be required to cure patients with brain tumours. This change in academic culture must go beyond a cursory recognition of the value of ‘team science’. Rather, akin to the complex and multidisciplinary groups that operate in the pharmaceutical industry, new reward and promotion structures should be developed for laboratory and clinical researchers alike that inspire and encourage their participation in collaborative academic brain tumour research.

Challenge 2: use neuroscience research

We now have a considerable level of understanding of the cellular hierarchies and signals that generate and maintain brain activity in the absence of cancer 15 . Specialized glial cells serve as neural stem cells (NSCs) in both the embryonic and adult brain, in which they reside in neurogenic niches and produce daughter cells that differentiate into neurons, astrocytes and oligodendrocytes (Fig. 1 ). These niches also include supporting cells and secreted factors that are critical for the close regulation of both NSC self-renewal and the early stages of daughter cell differentiation: an intimate relationship between NSCs and blood vessels seems to be especially important in the neurogenic niche 16 .

Several lines of evidence suggest that brain tumours arise within, or are driven by, cells that recapitulate the neurogenic niche (Fig. 3 ). Stem-like cells have been isolated from paediatric and adult brain tumours 17 , 18 , 19 , and brain tumours have been shown to contain malignant niches that seem to recapitulate the micro-architectural features and signalling properties of the nonmalignant neurogenic niche 20 , 21 . Recurrent mutations in brain tumours can also perturb the signalling pathways that regulate brain development 12 , and subgroups of brain tumours have been shown to contain the transcriptomes and epigenomes of their originating parental NSCs that generated these tumours in the mouse brain 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 .

Brain tumours are thought to arise from a transformed and malignant brain tumour hierarchy generated from transformed neuronal stem cells. Nonmalignant and malignant stem cells in the brain are believed to reside in perivascular niches formed by the blood vessels of the brain. Unaddressed key research questions that could provide a better understanding of brain tumour development and treatment are shown in the thought bubbles.

Evidence that brain tumours are a consequence of aberrant brain development or repair underscores the importance of closer integration between neurobiological and cancer research; however, clear examples of such collaborations are rare. Only ~1% (US$260 million) of the $21 billion invested in neuroscience research by the US NIH in 2018 is aimed specifically at brain tumour research. Furthermore, although the PubMed database currently contains ~320,000 and 198,000 articles on neuroscience and brain tumour research, respectively, only 6,000 (~2% and ~3.5%, respectively) overlap. These observations are crude indicators of the level of interaction, but they suggest that the brain tumour research community is not capitalizing fully on the available research data. An increased focus on areas of common interest, such as immune dysregulation in patients with cancer and in those with dementia, should drive joint funding initiatives and ultimately result in progress for patients 30 , 31 .

Challenge 3: understand the TME

A thorough understanding of the properties and functions of the tumour microenvironment (TME) is required in order to obtain a complete understanding of brain tumour biology and treatment 9 , 32 (Fig. 4 ). As discussed with regard to Challenge 2, closer interactions with scientists investigating nonmalignant brain diseases will be especially important to this effort. Over many decades, research groups studying neurodegeneration and other nonmalignant brain diseases have generated a considerable level of understanding of the biology of the extracellular matrix, of specific brain-resident cellular populations (such as microglia, astrocytes and neurons) and of blood vessels (including those forming the BBB) that comprise the TME.

Brain tumours typically comprise a complex mixture of malignant cells and a great variety of nonmalignant cells. These include immune, vascular and nonmalignant brain cells, all of which are able to communicate with malignant cells and contribute to the development and progression of cancer. These features also form a key part of the response of a tumour to treatment. Key research questions focused on understanding how these elements affect tumour biology and responses to treatment are shown in the thought bubbles. BBB, blood–brain barrier; TAM, tumour-associated macrophage.

The immune and vascular components of the TME are likely to have particular relevance for improving the treatment of brain tumours 6 , 33 . For example, the discovery of the immune or glymphatic system of the brain and evidence that immunotherapies are effective treatments for an increasing number of cancers have prompted studies of the efficacy of such therapies in patients with brain tumours 31 , 34 , 35 , 36 . The major myeloid cell populations of the brain TME comprise macrophages and microglia — together referred to as tumour-associated macrophages (TAMs) 37 , 38 . Glioblastoma cells stimulate TAMs to produce immunosuppressive, tumour-promoting cytokines and enhanced apoptosis of T cells 39 . Glioblastoma cells can also inhibit the production of immunostimulatory cytokines and activate the recruitment of regulatory T cells, thus inhibiting antitumour immune responses 40 . These data suggest that inhibiting the activity of TAMs might provide therapeutic benefit in patients with brain tumours, although the clinical reality is likely to be more complex. Research conducted over the past decade suggests that re-educating TAMs to adopt phenotypes that are likely to prevent or inhibit the progression of brain tumours might be more effective than depleting all TAM populations 32 , 41 . A range of additional immune-based therapies are also currently in development, including vaccines, cellular therapies (such as chimeric antigen receptor (CAR) T cells) and immune-checkpoint inhibitors 9 , 31 , 34 , 40 , 41 , 42 .

Regardless of the type of immunotherapy that is being developed, several key challenges must be addressed if we are to deploy these agents effectively in the treatment of brain tumours (Fig. 4 ). Once again, a better understanding of tumour biology will be important to the selection of patients who are most likely to benefit from therapies and to improve the level of understanding of treatment-related toxicities (see Challenge 1). Addressing these challenges will not be straightforward for immunotherapies, not least because even the imprecise predictors of a response to immune-checkpoint inhibition in other cancers, such as tumour mutational burden and/or DNA methylation, remain to be proved in patients with brain tumours 43 . Furthermore, the identity of the optimal targets of other immunotherapies, such as CAR T cells, which might be patient specific, also remains unclear. Other important and unanswered questions surrounding the use of immunotherapies in patients with brain tumours include the degree to which the BBB will impede the access of cellular and molecular immunotherapies to tumours; whether immunotherapies will synergize with or counteract the effects of existing treatments, such as temozolomide, steroids and/or radiotherapy; and the type and severity of neurological adverse effects associated with immunotherapies and how best to prevent, monitor and manage these. Thus, although immunotherapy holds great promise as a new therapeutic approach for the treatment of patients with brain tumours, many aspects of the unique immunological and microenvironmental status of the brain present challenges that need to be overcome in order to develop and validate these therapies efficiently 9 .

As previously alluded to, the BBB is a major hurdle to the successful treatment of brain tumours 44 . This dynamic structure of tight junctions and molecular pumps comprises endothelial, ependymal and tanycytic cells that communicate with other brain cells and with circulating immune cells 45 , 46 , 47 , 48 . Single-cell transcriptomics has revealed a remarkable level of regional heterogeneity along the arteriovenous axis of the BBB, including a seamless continuum of endothelial cells versus a punctuated continuum of mural cells 49 . These insights could prove important in determining the functionality of the BBB in tumours located in different regions of the brain.

Discrete subtypes of brain tumours might also alter the BBB (Fig. 4 ). In this regard, the WNT subtype of medulloblastomas, which are highly sensitive to treatment, has been shown to secrete WNT antagonists that ‘re-educate’ the BBB, thus causing it to adopt a phenotype that is highly permeable to systemic chemotherapy 6 . Similarly, the therapeutic response of gliomas to temozolomide might be improved by use of this agent in combination with small-molecule inhibitors of WNT signalling 33 . Greater research efforts should be invested in cataloguing the integrity and other alterations of the BBB in each brain tumour subtype. This information might enable the more widespread use of systemic therapies that are currently regarded as BBB non-penetrating in those tumours that lack an intact BBB and support the use of WNT inhibitors and other approaches to enhance the delivery of chemotherapy to those tumours in which the BBB is intact 50 .

Challenge 4: develop preclinical models

The limited progress made in the treatment of brain tumours relates, in part, to the inaccuracy of preclinical models that have thus far failed to consistently show responses to agents with therapeutic activity in patients. Preclinical drug development pipelines that enable the accurate prediction of effective drugs are especially important for rare cancers, including brain tumours, owing to the low numbers of patients available for participation in clinical research 12 , 51 . Current pipelines are limited in their ability to identify new, more effective treatments of brain tumours for several reasons: they typically involve poorly characterized in vitro systems or subcutaneous tumour xenografts, rather than more accurate orthotopic models of brain tumours; they do not enable the assessment of benefit from new treatments in terms of survival relative to that provided by the existing combinations of neurosurgery, radiotherapy and/or chemotherapy; and finally, they usually lack rigorous characterization of other clinically important features, such as those of the BBB. Thus, curing all patients with a brain tumour will require a new approach that leverages an improved understanding of neuroscience and brain tumour biology to develop and deploy more accurate preclinical models.

A better understanding of the processes that drive the development and progression of brain tumours should promote the development of more accurate genetically engineered mouse models (GEMMs) and patient-derived xenografts (PDXs). We recommend that the brain tumour community maintains a centralized catalogue of all existing and newly developed preclinical models that have been subjected to a minimum agreed set of rigorous histological, genomic and imaging evaluations. Models from this catalogue should be made readily available and would provide the community with universally accessible, validated models that closely mimic the characteristics of human tumours. Initiatives such as the ITCC Paediatric Preclinical Proof of Concept Platform are already underway, with the intention of creating such a resource.

In addition to minimum standards of model characterization, the community should also adopt minimum standards for preclinical studies, thus enabling more effective comparisons of the results of different studies (Fig. 5 ). Of particular note, although patients with brain tumours receive complex, multimodality therapy, mice in preclinical studies typically receive monotherapies that are rarely comparable to the standard of care for the malignancy that is being modelled. Thus, efforts to establish preclinical ‘mouse hospitals’, in which potential new therapies are tested in the context of combination with neurosurgery, fractionated radiotherapy and/or conventional chemotherapy, should become the new standard 11 . Researchers with access to such preclinical hospitals should also perform robust pharmacokinetic (with particular attention to BBB penetration), pharmacodynamic and biomarker (tumour, liquid biopsy and imaging) studies for novel agents, thus maximizing the speed and likelihood of success of clinical translation (Fig. 5 ).

Schema depicting pathways for improvements in the preclinical development of new treatments for subsets of patients with brain tumours of distinct molecular and/or clinical subtypes (represented by different colours). This approach should include both patient-derived xenografts (PDXs) and genetically engineered mouse models (GEMMs) that accurately reflect the principle molecular and/or clinical subtypes. In order to provide the optimal level of clinical relevance, these models must also involve an assessment of responses to conventional combination therapies. These existing treatments could be included in adaptive preclinical trials designed to test novel treatments and inform the ultimate clinical trial design.

The failure of many drugs to penetrate the BBB to any clinically meaningful extent is an important reason for the poor treatment responses of patients with brain tumours to systemic therapies 52 . Functional imaging studies in particular can provide evidence of early treatment responses, although a failure to detect a response cannot distinguish between a drug that is ineffective and one that simply does not cross the BBB. Contrast enhancement of tumours on MRI, using various agents, can reveal disruption of the BBB, for example, following ultrasonography-mediated opening of the BBB 53 ; however, this approach does not provide direct evidence of drug delivery. More direct evidence can be provided by observing tumour accumulation on PET imaging of drugs labelled with a positron-emitting isotope, such as [methyl- 11 C] temozolomide in patients with gliomas 54 . However, conjugating all new drugs with such isotopes — particularly with 11 C, which will not perturb drug chemistry and thus drug–target interactions but has a sufficiently short half-life (20.3 minutes) — would be a formidable undertaking. Thus, the community should agree on acceptable preclinical and clinical standards for demonstrating successful drug delivery to brain tumours.

In addition to being grown as PDXs, patient-derived brain tumour cells can be consistently and successfully encouraged to propagate under NSC culture conditions as organoids 55 , 56 . These short-term culture systems have enormous potential for studying interactions between different cells and drug sensitivities and for interrogation using genome-wide interference and/or editing techniques 20 , 55 , 56 , 57 , 58 . Preclinical pipelines that deploy these novel screening approaches, together with the use of matched GEMMs and PDX models (ideally in humanized mice), should provide a more comprehensive and more accurate means of delivering optimized drugs and their associated biomarkers into clinical trials 11 .

Preclinical and clinical research efforts to evaluate the value of new treatments, relative to that of the conventional treatments of brain tumours, should also consider re-evaluating some long-established treatment approaches that might be of questionable value. For example, corticosteroids, such as dexamethasone, continue to be used perioperatively and might be continued throughout subsequent treatment. The original reasons for using dexamethasone in this way probably reflect antiproliferative activity that confers protection of nonmalignant cells from radiotherapy-induced and chemotherapy-induced genotoxic stress. However, both clinical data and preclinical data from mouse models suggest that corticosteroids might decrease the effectiveness of treatments and shorten the survival durations of patients with glioblastoma. Thus, greater efforts should be invested in identifying alternative agents, on the basis of robust preclinical and clinical evidence, such as anti-VEGF agents for the treatment of brain tumour-associated oedema 59 .

Challenge 5: drugging complex cancers

The so-called omics revolution is providing a comprehensive view of the genomes, epigenomes and transcriptomes of brain tumours; this additional information has enabled the segregation of histologically similar subsets of tumours into clinically and molecularly distinct subgroups. However, despite a few notable exceptions, such as inhibitors of mTOR or BRAF in selected patients 60 , 61 , genomics has yet to deliver on the promise of identifying new therapeutic targets in patients with brain tumours. Discovering new treatments of brain tumours will require a much deeper level of understanding of the biology of potential drug targets, combined with the application of cutting-edge drug discovery approaches to inhibit these targets.

Progress in drug development will not come easily. Alterations identified in brain tumours in the past decade include poorly understood targets that are frequently regarded as undruggable, including amplification of genes encoding transcription factors, such as MYC and MYCN ; gene translocations, such as FGFR–TACC and C11OF95–RELA 27 , 62 ; oncogene ‘enhancer hijacking’, for example, by GFI1 (ref. 63 ); and mutated histones, such as H3F3A and HIST1H3B variants 64 , 65 . Furthermore, these alterations exist in a shifting landscape of tumour heterogeneity, in which the populations of clones carrying each alteration fluctuate as the disease evolves in response to therapy 66 , 67 . This clonal evolution means that the proportion of tumour cells carrying a specific drug target can vary markedly during the course of disease, thus complicating efforts to target specific alterations. This heterogeneity can be measured in terms of mutational load, epigenetic alterations, rewiring of transcriptional circuits and microenvironmental influences and is widely considered to underpin the diversity of responses to treatment. The serial analysis of tumour biology using additional measures, such as advanced metabolic imaging and analyses involving serial sampling of circulating tumour DNA, is discussed with regard to Challenge 6 and might provide a more complete and accurate picture of tumour evolution.

Although daunting, the data provided by these novel approaches reflect an important principle that must be adopted by brain tumour researchers and clinicians — the notion that brain tumours are monogenetic and monoclonal must be dispelled. Comprehensive mapping of genomic evolution before, during and following treatment, including the use of single-cell sequencing approaches, should be used to better identify and prioritize treatment targets 68 , 69 . This objective will be best achieved in the context of prospective clinical trials, in which molecular imaging and other diagnostic approaches can be used to provide real-time assessments of disease evolution 70 . Prioritized targets should also be interrogated using approaches, such as deep mutational scanning, that can reveal information on the intrinsic properties of proteins and their function and how this changes with mutation 71 . Reverse translation efforts involving detailed mechanistic studies after a clinical trial of appropriately prioritized targets should include functional studies that enable the accurate definition of target function in tumours. Such serial analyses will likely be complex and resource-intensive and are probably best organized as part of a community-wide approach. This approach might be promoted in several ways: development of international tumour subtype-specific trials; the widespread availability of brain tumour stem cell banks (such as the UK glioma cellular genetics resource (GCGR) , the human glioblastoma cell culture (HGCC) resource in Sweden and the Stand Up to Cancer (SU2C) Canada Cancer Stem Cell Dream Team ); established efforts to understand complex cancer targets (such as the US National Cancer Institute RAS Initiative ); and closer collaborations with industry.

Challenge 6: precision medicine

The current classifications of brain tumours are based primarily on microscopic morphology and immunohistochemistry 72 . This approach enables the broad characterization of tumour type and a certain level of prognostication, although it fails to capitalize on the wealth of clinically relevant insights produced by the genomic subtyping of brain tumours. This dilemma is exemplified by medulloblastoma and ependymoma, both of which consist of clinically relevant subtypes with discrete cells of origin, driver mutations and global transcriptomic and epigenomic patterns that are yet to be incorporated into the WHO classification of central nervous system tumours 10 , 12 , 72 , 73 , 74 , 75 .

Concerns that genome-wide classification tools might prove impractical for the routine diagnosis of brain tumours have been dispelled by the successful large-scale methylome subtyping of formalin-fixed, paraffin-embedded tumours 75 , 76 . Therefore, the challenge to the brain tumour community is not whether but how to deploy diagnostic genomic profiling. In the UK, the centralization of genomic profiling through the National Health Service provides an enormous opportunity to begin this process at a population level. In doing so, it will be important to note that not all brain tumours are created equally: paediatric medulloblastomas and ependymomas comprise robust subgroups with distinct characteristics. However, the degree of diagnostic separation of other brain tumours, including gliomas in adults, is less clear.

The genomic classification of brain tumours will increase the level of diagnostic precision that is currently achievable, although intratumoural heterogeneity and the often limited amounts of tumour material available for analysis are likely to confound the accurate diagnosis of some tumours 68 , 77 . Therefore, the research community should aim to urgently investigate the use of more advanced imaging techniques, such as 13 C-hyperpolarized MRI, radiomics and sequencing of cell-free DNA obtained from plasma and/or cerebrospinal fluid, as ancillary approaches to the diagnosis and classification of brain tumours 70 , 78 , 79 , 80 . The challenge of validating and combining these new diagnostic modalities is enormous but also holds the potential to greatly improve the accuracy of brain tumour classification. Implementation of such approaches might also improve the assessment of novel therapies as part of the federal drug approval process. In this regard, the current standards, such as the Response Assessment in Neuro-Oncology criteria 81 , which describe complete and partial drug responses, are of limited value for assessing new treatments of slow-growing tumours, such as low-grade glioma. Early molecular and/or metabolic responses that are predictive of ultimate clinical benefit might, therefore, provide a more accurate means of evaluating the efficacy of treatments.

Once validated, new classification tools, such as those outlined above, must be adopted within the WHO guidelines in order to enable use by the wider community. A comprehensive strategy to achieve this goal should formally include genomic metrics, such as transcriptomic analysis, in WHO diagnosis and classification schemes; involve appropriate training of experienced and newly qualified pathologists in genomic methodologies; be accompanied by appropriate levels of investment in the resources (both capital and personnel) needed to deliver genomic assays routinely in diagnostic pathology labs; and incorporate standardized genomic characterization within both conventional and clinical trial management strategies. Histology will always have a place in the diagnosis of patients, although only when used in concert with advanced molecular and imaging-based diagnostics will this provide a comprehensive prediction of tumour behaviour.

In addition to developing advanced diagnostic approaches, we must also learn how best to integrate these rich data streams if they are to revolutionize the treatment of patients with cancer (Fig. 5 ). Continual, iterative and integrated analysis of these complementary and complex data streams should be made possible by machine learning, artificial intelligence and/or other mathematical and computational approaches. This will require a concerted effort to invent and deploy new analytical approaches and visualization technologies in order to generate a single flow of information that travels with a patient throughout his or her treatment journey, enabling better-informed management decisions and guiding the patient towards the maximum opportunity for cure (Fig. 6 ). Reporting the clinically relevant findings from such large-scale data sets to health-care professionals and patients will require a multidisciplinary approach, supported by appropriate information technology infrastructure, genetic counsellors, medical geneticists and clinical scientists. This longitudinal and integrated approach will also enable the development of the next generation of adaptive precision medicine clinical trials.

Serving as a ‘biological prism’, the use of artificial intelligence (AI) and machine learning (ML) approaches should enable the integration of separate data streams at critical points in the patient journey, providing an unprecedented level of comprehensive decision-making to guide the selection of the most appropriate treatment and support the management of patients with brain tumours. The knowledge gained through the management of each patient should be analysed iteratively over time, enabling constant refinement and improvements in clinical decision-making. ctDNA, circulating tumour DNA.

Challenge 7: reduce treatment for some

Certain brain tumours, particularly those arising in children, can be cured with aggressive surgery, radiotherapy and chemotherapy, although these cures often carry a considerable cost, particularly for young children who have lifelong neurocognitive and endocrine adverse effects 4 , 82 . Reducing the intensity of treatment in children with other cancers, such as acute lymphoblastic leukaemia, has enabled a reduction in the incidence of adverse effects while maintaining high cure rates; however, this approach has proved challenging to implement in patients with brain tumours, the majority of whom have disease progression following surgery alone. Identifying which patients might be cured with less-intense adjuvant radiotherapy and chemotherapy and to what extent the intensity of these modalities can be safely reduced remains a major challenge.

Medulloblastomas provide a clear example of how a detailed understanding of tumour biology can guide adjustments of treatment intensity. Historical attempts to limit the damaging effects of craniospinal radiotherapy on the developing brain by reducing the radiation dose have failed, largely owing to an inadequate understanding of medulloblastoma biology, which precluded the accurate selection of patients with truly low-risk disease 83 , 84 , 85 . A breakthrough was achieved with the recognition that almost all patients with the WNT subtype of medulloblastoma are cured 12 . This discovery has led to a series of highly selective ongoing studies testing reduced-intensity radiotherapy in patients with this disease subtype (NCT02724579, NCT02066220, NCT02212574 and NCT01878617). Evidence that WNT-subtype tumours (but not other forms of medulloblastoma) lack a BBB and are therefore remarkably vulnerable to systemic chemotherapy provides an explanation for the curability of these tumours, thus opening further options for alternatives to radiation-based treatments 6 . Paediatric low-grade gliomas provide another example of how molecularly guided treatments (such as inhibitors of activated BRAF and MEK) are enabling reductions in radiation dose 86 , 87 . Studies involving patients with other brain tumours are likely to identify similar approaches for the rational de-escalation of therapy.

A comprehensive approach to preventing and ameliorating the adverse effects associated with long-term treatment should include prospective functional and patient-reported outcome measures, as well as innovative predictive assessments of risk. Novel ways of supplementing these data are also emerging in the form of genetic polymorphisms and somatic genomic characteristics that might predict the risk of subsequent intellectual disability and/or hearing loss 88 , 89 .

Conclusions

The past 15 years have witnessed a revolution in our understanding of cancer. The integration of genomic and developmental biology has shown that morphologically similar cancers comprise discrete subtypes, driven by different genetic alterations, which likely arise from distinct cell lineages. These data help to explain why cancers once regarded as histologically homogeneous diseases have a discrepant range of characteristics. Improved understanding is also leading to the development of completely new treatment approaches for cancer, such as immunotherapies, and novel ways to test such therapies, such as adaptive trial designs. However, the successes achieved with these improvements have not occurred equally across all forms of cancer. Of particular note, the treatment of most childhood and adult brain tumours is at an impasse, with no new, more effective therapies being developed in the past 30 years. Thus, all available evidence suggests that the current preclinical and clinical research approaches to curing brain tumours are ineffective.

Herein, we have summarized seven major challenges to progress that must be overcome if we are to produce the sea change in brain tumour therapy that has long been awaited by patients and their families. Some of these challenges might seem to be self-evident; however, each will require a profound change in research and clinical practice and investment. In order to truly redesign the brain tumour research and treatment pipeline and to leverage the full spectrum of neuroscience research, modest improvements in the level of interdisciplinary collaborations among otherwise siloed research groups will not be sufficient. We envisage a much more substantial congregation of experts and infrastructure, focused on the task of curing brain tumours. This call reflects the need to practise brain tumour research differently and to identify, convene and support teams of scientists and clinicians who focus specifically and urgently on the need to improve the lives of patients with brain tumours. This approach must be different from the largely ad hoc and predominantly poorly structured brain tumour collaborations and programmes that currently exist in most academic centres. The harsh reality is that the current efforts at various universities and aligned clinical environments around the world have failed to adequately improve the understanding and treatment of patients with brain tumours. Nevertheless, this approach would probably be easiest to implement within the context of existing academic settings in order to engage with and closely incorporate disciplines not traditionally involved in brain tumour research, such as engineering, chemistry, physics and mathematics.

As we seek to better understand the role of the TME and aim to develop more predictive preclinical model systems, we need to take a long, hard look at the current model systems. Many of these clearly do not adequately reflect all the nonmalignant and malignant cell types that compose brain tumours and have failed to accurately predict treatments that will be effective for testing in clinical trials. Only systems that are proven to model clinically relevant aspects of tumour biology should be used as evidence to guide the initiation of clinical trials. Furthermore, through reverse translation, we should refine the use of or entirely remove models that fail to predict clinical efficacy.

Concerns relating to safety and patient acceptability limit the ability to obtain repeat biopsy samples from brain tumours in most patients, thus preventing longitudinal studies of tumour biology and treatment response and the deployment of a precision medicine approach. The use of liquid biopsies and advanced imaging approaches should be explored immediately to provide additional approaches to determine tumour response earlier in the course of treatment. Finally, the challenge most likely to yield patient benefit in the immediate future will be the use of advanced stratification approaches to reduce the risk of treatment-related toxicities, especially among children.

Gilbert, M. R. et al. A randomized trial of bevacizumab for newly diagnosed glioblastoma. N. Engl. J. Med. 370 , 699–708 (2014).

Article CAS PubMed PubMed Central Google Scholar

Chinot, O. L. et al. Bevacizumab plus radiotherapy–temozolomide for newly diagnosed glioblastoma. N. Engl. J. Med. 370 , 709–722 (2014).

Article CAS PubMed Google Scholar

Smith, M. A. & Reaman, G. H. Remaining challenges in childhood cancer and newer targeted therapeutics. Pediatr. Clin. North Am. 62 , 301–312 (2015).

Article PubMed Google Scholar

Brinkman, T. M. et al. Long-term neurocognitive functioning and social attainment in adult survivors of pediatric CNS tumors: results from the St Jude Lifetime Cohort Study. J. Clin. Oncol. 34 , 1358–1367 (2016).

Chemaitilly, W., Armstrong, G. T., Gajjar, A. & Hudson, M. M. Hypothalamic-pituitary axis dysfunction in survivors of childhood CNS tumors: importance of systematic follow-up and early endocrine consultation. J. Clin. Oncol. 34 , 4315–4319 (2016).

Phoenix, T. N. et al. Medulloblastoma genotype dictates blood brain barrier phenotype. Cancer Cell 29 , 508–522 (2016).

Gerstner, E. R. & Fine, R. L. Increased permeability of the blood-brain barrier to chemotherapy in metastatic brain tumors: establishing a treatment paradigm. J. Clin. Oncol. 25 , 2306–2312 (2007).

Mackay, A. et al. Integrated molecular meta-analysis of 1,000 pediatric high-grade and diffuse intrinsic pontine glioma. Cancer Cell 32 , 520–537 (2017).

Quail, D. F. & Joyce, J. A. The microenvironmental landscape of brain tumors. Cancer Cell 31 , 326–341 (2017).

Gilbertson, R. J. Mapping cancer origins. Cell 145 , 25–29 (2011).

Nimmervoll, B. et al. Establishing a preclinical multidisciplinary board for brain tumors. Clin. Cancer Res. 24 , 1654–1666 (2018).

Northcott, P. A. et al. Medulloblastomics: the end of the beginning. Nat. Rev. Cancer 12 , 818–834 (2012).

Chow, S. C. Adaptive clinical trial design. Annu. Rev. Med. 65 , 405–415 (2014).

Alexander, B. M. et al. Adaptive global innovative learning environment for glioblastoma: GBM AGILE. Clin. Cancer Res. 24 , 737–743 (2018).

Kriegstein, A. & Alvarez-Buylla, A. The glial nature of embryonic and adult neural stem cells. Annu. Rev. Neurosci. 32 , 149–184 (2009).

Bjornsson, C. S., Apostolopoulou, M., Tian, Y. & Temple, S. It takes a village: constructing the neurogenic niche. Dev. Cell 32 , 435–446 (2015).

Singh, S. K. et al. Identification of human brain tumour initiating cells. Nature 432 , 396–401 (2004).

Ignatova, T. N. et al. Human cortical glial tumors contain neural stem-like cells expressing astroglial and neuronal markers in vitro. Glia 39 , 193–206 (2002).

Taylor, M. D. et al. Radial glia cells are candidate stem cells of ependymoma. Cancer Cell 8 , 323–335 (2005).

Calabrese, C. et al. A perivascular niche for brain tumor stem cells. Cancer Cell 11 , 69–82 (2007).

Lathia, J. D., Mack, S. C., Mulkearns-Hubert, E. E., Valentim, C. L. L. & Rich, J. N. Cancer stem cells in glioblastoma. Genes Dev. 29 , 1203–1217 (2015).

Pei, Y. et al. WNT signaling increases proliferation and impairs differentiation of stem cells in the developing cerebellum. Development 139 , 1724–1733 (2012).

Goodrich, L. V., Milenkovic, L., Higgins, K. M. & Scott, M. P. Altered neural cell fates and medulloblastoma in mouse patched mutants. Science 277 , 1109–1113 (1997).

Chen, J. et al. A restricted cell population propagates glioblastoma growth after chemotherapy. Nature 488 , 522–526 (2012).

Gibson, P. et al. Subtypes of medulloblastoma have distinct developmental origins. Nature 468 , 1095–1099 (2010).

Johnson, R. A. et al. Cross-species genomics matches driver mutations and cell compartments to model ependymoma. Nature 466 , 632–636 (2010).

Parker, M. et al. C11orf95-RELA fusions drive oncogenic NF-kappaB signalling in ependymoma. Nature 506 , 451–455 (2014).

Mohankumar, K. M. et al. An in vivo screen identifies ependymoma oncogenes and tumor-suppressor genes. Nat. Genet. 47 , 878–887 (2015).

Tirosh, I. et al. Single-cell RNA-seq supports a developmental hierarchy in human oligodendroglioma. Nature 539 , 309–313 (2016).

Article PubMed PubMed Central CAS Google Scholar

Abbott, A. Is ‘friendly fire’ in the brain provoking Alzheimer’s disease? Nature 556 , 426–428 (2018).

Sampson, J. H., Maus, M. V. & June, C. H. Immunotherapy for brain tumors. J. Clin. Oncol. 35 , 2450–2456 (2017).

Quail, D. F. & Joyce, J. A. Microenvironmental regulation of tumor progression and metastasis. Nat. Med. 19 , 1423–1437 (2013).

Griveau, A. et al. A glial signature and Wnt7 signaling regulate glioma-vascular interactions and tumor microenvironment. Cancer Cell 33 , 874–889 (2018).

Louveau, A., Harris, T. H. & Kipnis, J. Revisiting the mechanisms of CNS immune privilege. Trends Immunol. 36 , 569–577 (2015).

Tivnan, A., Heilinger, T., Lavelle, E. C. & Prehn, J. H. M. Advances in immunotherapy for the treatment of glioblastoma. J. Neurooncol. 131 , 1–9 (2017).

Zacharakis, N. et al. Immune recognition of somatic mutations leading to complete durable regression in metastatic breast cancer. Nat. Med. 24 , 724–730 (2018).

Graeber, M. B., Scheithauer, B. W. & Kreutzberg, G. W. Microglia in brain tumors. Glia 40 , 252–259 (2002).

Ginhoux, F. et al. Fate mapping analysis reveals that adult microglia derive from primitive macrophages. Science 330 , 841–845 (2010).

Hambardzumyan, D., Gutmann, D. H. & Kettenmann, H. The role of microglia and macrophages in glioma maintenance and progression. Nat. Neurosci. 19 , 20 (2015).

Article CAS Google Scholar

Razavi, S.-M. et al. Immune evasion strategies of glioblastoma. Front. Surg. 3 , 11 (2016).

Article PubMed PubMed Central Google Scholar

Bowman, R. L. & Joyce, J. A. Therapeutic targeting of tumor-associated macrophages and microglia in glioblastoma. Immunotherapy 6 , 663–666 (2014).

Weller, M. et al. Vaccine-based immunotherapeutic approaches to gliomas and beyond. Nat. Rev. Neurol. 13 , 363–374 (2017).

Campbell, B. B. et al. Comprehensive analysis of hypermutation in human cancer. Cell 171 , 1042–1056 (2017).

Banks, W. A. From blood–brain barrier to blood–brain interface: new opportunities for CNS drug delivery. Nat. Rev. Drug Discov. 15 , 275 (2016).

Larochelle, C. et al. EGFL7 reduces CNS inflammation in mouse. Nat. Commun. 9 , 819 (2018).

Hawkins, B. T. & Davis, T. P. The blood-brain barrier/neurovascular unit in health and disease. Pharmacol. Rev. 57 , 173–185 (2005).

Daneman, R., Zhou, L., Kebede, A. A. & Barres, B. A. Pericytes are required for blood-brain barrier integrity during embryogenesis. Nature 468 , 562–566 (2010).

Abbott, N. J., Ronnback, L. & Hansson, E. Astrocyte-endothelial interactions at the blood-brain barrier. Nat. Rev. Neurosci. 7 , 41–53 (2006).

Vanlandewijck, M. et al. A molecular atlas of cell types and zonation in the brain vasculature. Nature 554 , 475–480 (2018).

Kung, Y. et al. Focused shockwave induced blood-brain barrier opening and transfection. Sci. Rep. 8 , 2218 (2018).

Pajtler, K. W. et al. Molecular classification of ependymal tumors across all CNS compartments, histopathological grades, and age groups. Cancer Cell 27 , 728–743 (2015).

Jacus, M. O. et al. Pharmacokinetic properties of anticancer agents for the treatment of central nervous system tumors: update of the literature. Clin. Pharmacokinet. 55 , 297–311 (2016).

Carpentier, A. et al. Clinical trial of blood-brain barrier disruption by pulsed ultrasound. Sci. Transl Med. 8 , 343re342 (2016).

Rosso, L. et al. A new model for prediction of drug distribution in tumor and normal tissues: pharmacokinetics of temozolomide in glioma patients. Cancer Res. 69 , 120–127 (2009).

Hubert, C. G. et al. A three-dimensional organoid culture system derived from human glioblastomas recapitulates the hypoxic gradients and cancer stem cell heterogeneity of tumors found in vivo. Cancer Res. 76 , 2465–2477 (2016).

Toledo, C. M. et al. Genome-wide CRISPR-Cas9 screens reveal loss of redundancy between PKMYT1 and WEE1 in glioblastoma stem-like cells. Cell Rep. 13 , 2425–2439 (2015).

Atkinson, J. M. et al. An integrated in vitro and in vivo high-throughput screen identifies treatment leads for ependymoma. Cancer Cell 20 , 384–399 (2011).

Housden, B. E. et al. Improved detection of synthetic lethal interactions in Drosophila cells using variable dose analysis (VDA). Proc. Natl Acad. Sci. USA 114 , E10755–E10762 (2017).

Pitter, K. L. et al. Corticosteroids compromise survival in glioblastoma. Brain 139 , 1458–1471 (2016).

Krueger, D. A. et al. Everolimus for subependymal giant-cell astrocytomas in tuberous sclerosis. N. Engl. J. Med. 363 , 1801–1811 (2010).

Kieran, M. W. et al. CNS tumours: the first study of dabrafenib in pediatric patients with BRAF V600–mutant relapsed or refractory low-grade gliomas [abstract]. Ann. Oncol. 27 (Suppl. 6), LBA19_PR (2016).

Google Scholar

Singh, D. et al. Transforming fusions of FGFR and TACC genes in human glioblastoma. Science 337 , 1231–1235 (2012).

Northcott, P. A. et al. Enhancer hijacking activates GFI1 family oncogenes in medulloblastoma. Nature 511 , 428–434 (2014).

Schwartzentruber, J. et al. Driver mutations in histone H3.3 and chromatin remodelling genes in paediatric glioblastoma. Nature 482 , 226–231 (2012).

Wu, X. et al. Clonal selection drives genetic divergence of metastatic medulloblastoma. Nature 482 , 529–533 (2012).

Morrissy, A. S. et al. Divergent clonal selection dominates medulloblastoma at recurrence. Nature 529 , 351–357 (2016).

Wang, J. et al. Clonal evolution of glioblastoma under therapy. Nat. Genet. 48 , 768–776 (2016).

Patel, A. P. et al. Single-cell RNA-seq highlights intratumoral heterogeneity in primary glioblastoma. Science 344 , 1396–1401 (2014).

Darmanis, S. et al. Single-cell RNA-seq analysis of infiltrating neoplastic cells at the migrating front of human glioblastoma. Cell Rep. 21 , 1399–1410 (2017).

Brindle, K. M., Izquierdo-Garcia, J. L., Lewis, D. Y., Mair, R. J. & Wright, A. J. Brain tumor imaging. J. Clin. Oncol. 35 , 2432–2438 (2017).

Fowler, D. M. & Fields, S. Deep mutational scanning: a new style of protein science. Nat. Methods 11 , 801 (2014).

Louis, D. N. et al. The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 131 , 803–820 (2016).

Pajtler, K. W. et al. The current consensus on the clinical management of intracranial ependymoma and its distinct molecular variants. Acta Neuropathol. 133 , 5–12 (2017).

Aldape, K., Zadeh, G., Mansouri, S., Reifenberger, G. & von Deimling, A. Glioblastoma: pathology, molecular mechanisms and markers. Acta Neuropathol. 129 , 829–848 (2015).

Sturm, D. et al. New brain tumor entities emerge from molecular classification of CNS-PNETs. Cell 164 , 1060–1072 (2016).

Capper, D. et al. DNA methylation-based classification of central nervous system tumours. Nature 555 , 469–474 (2018).

Reinartz, R. et al. Functional subclone profiling for prediction of treatment-induced intratumor population shifts and discovery of rational drug combinations in human glioblastoma. Clin. Cancer Res. 23 , 562–574 (2017).

Day, S. E. et al. Detecting response of rat C6 glioma tumors to radiotherapy using hyperpolarized [1–13C]pyruvate and 13C magnetic resonance spectroscopic imaging. Magn. Reson. Med. 65 , 557–563 (2011).

Wan, J. C. M. et al. Liquid biopsies come of age: towards implementation of circulating tumour DNA. Nat. Rev. Cancer 17 , 223–238 (2017).

Pentsova, E. I. et al. Evaluating cancer of the central nervous system through next-generation sequencing of cerebrospinal fluid. J. Clin. Oncol. 34 , 2404–2415 (2016).

Wen, P. Y. et al. Updated response assessment criteria for high-grade gliomas: response assessment in neuro-oncology working group. J. Clin. Oncol. 28 , 1963–1972 (2010).

Moxon-Emre, I. et al. Intellectual outcome in molecular subgroups of medulloblastoma. J. Clin. Oncol. 34 , 4161–4170 (2016).

Duffner, P. K. et al. Postoperative chemotherapy and delayed radiation in children less than three years of age with malignant brain tumors. N. Engl. J. Med. 328 , 1725–1731 (1993).

Packer, R. J. et al. Treatment of children with medulloblastomas with reduced-dose craniospinal radiation therapy and adjuvant chemotherapy: a Children’s Cancer Group Study. J. Clin. Oncol. 17 , 2127–2136 (1999).

Geyer, J. R. et al. Multiagent chemotherapy and deferred radiotherapy in infants with malignant brain tumors: a report from the Children’s Cancer Group. J. Clin. Oncol. 23 , 7621–7631 (2005).

Ater, J. L. et al. Randomized study of two chemotherapy regimens for treatment of low-grade glioma in young children: a report from the Children’s Oncology Group. J. Clin. Oncol. 30 , 2641–2647 (2012).

Krishnatry, R. et al. Clinical and treatment factors determining long-term outcomes for adult survivors of childhood low-grade glioma: a population-based study. Cancer 122 , 1261–1269 (2016).

Cole, P. D. et al. Polymorphisms in genes related to oxidative stress are associated with inferior cognitive function after therapy for childhood acute lymphoblastic leukemia. J. Clin. Oncol. 33 , 2205–2211 (2015).

Xu, H. et al. Common variants in ACYP2 influence susceptibility to cisplatin-induced hearing loss. Nat. Genet. 47 , 263–266 (2015).

Lim, D. A. & Alvarez-Buylla, A. The adult ventricular–subventricular zone (V-SVZ) and olfactory bulb (OB) neurogenesis. Cold Spring Harb. Perspect. Biol. 8 , a018820 (2016).

Download references

Acknowledgements

R.J.G. gratefully acknowledges financial support from the US NIH (grants P01CA96832 and R0CA1129541), Cancer Research UK, the Mathile Family Foundation, Cure Search and the Sohn Foundation.

Reviewer information

Nature Reviews Clinical Oncology thanks D. Reardon and other anonymous reviewer(s) for their contribution to the peer review of this work.

Author information

Authors and affiliations.

Department of Pathology, University Health Network, Toronto, Ontario, Canada

Kenneth Aldape

CRUK Cambridge Institute, Li Ka Shing Centre, Cambridge, UK

Kevin M. Brindle & Richard J. Gilbertson

The Institute of Cancer Research, London, UK

Louis Chesler, Rajesh Chopra & Paul Workman

Department of Oncology, St Jude Children’s Research Hospital, Memphis, TN, USA

Amar Gajjar & Giles W. Robinson

Center for Cancer Research, National Cancer Institute, Bethesda, MD, USA

Mark R. Gilbert

Telethon Kids Institute, Subiaco, WA, Australia

Nicholas Gottardo

Department of Neurology, Washington University School of Medicine, St Louis, MO, USA

David H. Gutmann

Great Ormond Street Hospital for Children, London, UK

Darren Hargrave

Human Biology Division, Fred Hutchinson Cancer Research Center, Seattle, WA, USA

Eric C. Holland

Pediatric Glioma Research Group, Hopp Children’s Cancer Center at the NCT Heidelberg, Heidelberg, Germany

David T. W. Jones

Ludwig Institute for Cancer Research, University of Lausanne, Lausanne, Switzerland

Johanna A. Joyce

Cancer Research UK Clinical Trials Unit, Institute of Cancer and Genomic Sciences, University of Birmingham, Birmingham, UK

Pamela Kearns

Dana–Farber/Boston Children’s Cancer and Blood Disorders Center and Harvard Medical School, Boston, MA, USA

Mark W. Kieran

Human Oncology and Pathogenesis Program and Department of Neurology, Memorial Sloan Kettering Cancer Center, New York, NY, USA

Ingo K. Mellinghoff

Oncology, AstraZeneca IMED Biotech Unit, Boston, MA, USA

Melinda Merchant

Division of Pediatric Oncology, Hopp Children’s Cancer Center at the NCT Heidelberg, Heidelberg, Germany

Stefan M. Pfister

Cancer Research UK Edinburgh Centre and Medical Research Council Centre for Regenerative Medicine, University of Edinburgh, Edinburgh, UK

Steven M. Pollard

Department of Paediatrics, The Hospital for Sick Children, Toronto, Ontario, Canada

Vijay Ramaswamy

Division of Regenerative Medicine, Department of Medicine, University of California, San Diego, CA, USA

Jeremy N. Rich

Department of Paediatrics, University of Cambridge and Wellcome Trust-MRC Stem Cell Institute, Cambridge, UK

David H. Rowitch

The Preston Robert Tisch Brain Tumor Center, Duke Cancer Center, Durham, NC, USA

John H. Sampson

The Arthur and Sonia Labatt Brain Tumour Research Centre and Division of Neurosurgery, The Hospital for Sick Children, Toronto, Ontario, Canada

Michael D. Taylor

CRUK Cambridge Institute and Department of Oncology, University of Cambridge, Hutchison/MRC Research Centre, Cambridge Biomedical Campus, Cambridge, UK

Richard J. Gilbertson

You can also search for this author in PubMed Google Scholar

Contributions

All authors made substantial contributions to researching data for the article, to discussions of content and to writing the manuscript. R.J.G. reviewed and edited the manuscript before submission.

Corresponding author

Correspondence to Richard J. Gilbertson .

Ethics declarations

Competing interests.

P.W. and R.C. are employees of The Institute of Cancer Research (ICR), which has a commercial interest in a range of drug targets. The ICR operates a Rewards to Inventors scheme whereby employees of the ICR may receive financial benefit following commercial licensing of a project. P.W. is a consultant/scientific advisory board member for NextechInvest, Storm Therapeutics, Astex Pharmaceuticals and CV6 and holds stock in Chroma Therapeutics, NextInvest and Storm Therapeutics; he is also a Non-Executive Director of Storm Therapeutics and the Royal Marsden NHS Trust and a Director of the non-profit Chemical Probes Portal. R.C. is an advisor to Syncona Limited and holds equity in Celgene Corporation, e-Therapeutics and Monte Rosa Therapeutics. The other authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Aldape, K., Brindle, K.M., Chesler, L. et al. Challenges to curing primary brain tumours. Nat Rev Clin Oncol 16 , 509–520 (2019). https://doi.org/10.1038/s41571-019-0177-5

Download citation

Published : 07 February 2019

Issue Date : August 2019

DOI : https://doi.org/10.1038/s41571-019-0177-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Patient-derived organoids in human cancer: a platform for fundamental research and precision medicine.

Shanqiang Qu
Rongyang Xu
Guanglong Huang

Molecular Biomedicine (2024)

ZDHHC5-mediated S-palmitoylation of FAK promotes its membrane localization and epithelial-mesenchymal transition in glioma

Zhangjie Wang

Cell Communication and Signaling (2024)

Redox-responsive polymer micelles co-encapsulating immune checkpoint inhibitors and chemotherapeutic agents for glioblastoma therapy

Zhiqi Zhang
Xiaoxuan Xu
Shenghong Ju

Nature Communications (2024)

The liver and muscle secreted HFE2-protein maintains central nervous system blood vessel integrity

Xue Fan Wang
Robin Vigouroux
Philippe P. Monnier

Cudraflavone B induces human glioblastoma cells apoptosis via ER stress-induced autophagy

Rongchuan Zhao
Yanxiang Liu

BMC Neuroscience (2023)

Quick links

Explore articles by subject
Guide to authors
Editorial policies

Open-source AI models released by Tokyo lab Sakana founded by former Google researchers

By Anna Tong

SAN JOSE (Reuters) - Sakana AI, a Tokyo-based artificial intelligence startup founded by two prominent former Google researchers, released AI models on Wednesday it said were built using a novel method inspired by evolution, akin to breeding and natural selection.

Sakana AI employed a technique called "model merging" which combines existing AI models to yield a new model, combining it with an approach inspired by evolution, leading to the creation of hundreds of model generations.

The most successful models from each generation were then identified, becoming the "parents" of the next generation.

The company is releasing the three Japanese language models and two are being open-sourced, Sakana AI founder David Ha told Reuters in online remarks from Tokyo.

The company's founders are former Google researchers Ha and Llion Jones.

Jones is an author on Google's 2017 research paper "Attention Is All You Need", which introduced the "transformer" deep learning architecture that formed the basis for viral chatbot ChatGPT, leading to the race to develop products powered by generative AI.

Ha was previously the head of research at Stability AI and a Google Brain researcher.

All the authors of the ground-breaking Google paper have since left the organisation.

Venture investors have poured millions of dollars in funding into their new ventures, such as AI chatbot startup Character.AI run by Noam Shazeer, and the large language model startup Cohere founded by Aidan Gomez.

Sakana AI seeks to put the Japanese capital on the map as an AI hub, just as OpenAI did for San Francisco and the company DeepMind did for London earlier. In January Sakana AI said it had raised $30 million in seed financing led by Lux Capital.

(Reporting by Anna Tong in San Jose; Editing by Clarence Fernandez)

Words reading "Artificial intelligence AI", miniature of robot and toy hand are picture in this illustration taken December 14, 2023. REUTERS/Dado Ruvic/Illustration/file photo

IMAGES

How to Write a Professional Paper Using Psychology Research Topics
(PDF) Review paper on Blue Brain Technology
Deep image reconstruction from human brain activity
Erika Howe Featured in the Journal Experimental Brain Research
Visual System In The Brain Research Paper
Brain Tumor Detection And Classification Using Clustering And

VIDEO

10 Lines on The Brain/ 12 Lines Essay on Brain/ The Brain Essay in english
Master the Brainstorming Session: Preparation and Research
Challenges & Opportunities in Brain Research Strategy in Canada

COMMENTS

Brain Research
We recognize how technology has changed the way scientific breakthroughs are communicated and Brain Research is committed to serving as a dynamic journal …. View full aims & scope. Between 1981 and 1988 Brain Research shared its volume numbering system with Brain Research Review, Developmental Brain Research and Molecular Brain Research. $3080.
Brain Health: What is brain health and why is it important?
The human brain is the command centre for the nervous system and enables thoughts, memory, movement, and emotions by a complex function that is the highest product of biological evolution. Maintaining a healthy brain during one's life is the uppermost goal in pursuing health and longevity. As the population ages, the burden of neurological ...
What We Know About the Brain Structure-Function Relationship
1. Introduction. How the human brain works is still an open question, as is its implication with brain architecture: the non-trivial structure-function relationship. The development of neuroimaging techniques, the improvement on their processing methods and the unfolding of computational neuroscience field have driven brain research focused ...
Brain
Multiple mild Traumatic Brain Injuries ("mTBIs") can put military service members at an elevated risk of cognitive impairment. Service members and veterans were enrolled in a trial with a new type of brain training program, based on the science of brain plasticity and the discovery that intensive, adaptive, computerized training—targeting ...
Protocol of the Healthy Brain Study: An accessible resource for ...
The endeavor to understand the human brain has seen more progress in the last few decades than in the previous two millennia. Still, our understanding of how the human brain relates to behavior in the real world and how this link is modulated by biological, social, and environmental factors is limited. To address this, we designed the Healthy Brain Study (HBS), an interdisciplinary ...
Relating Structure and Function in the Human Brain: Relative ...
Author Summary By analogy with the road network, the human brain is defined both by its anatomy (the 'roads'), that is, the way neurons are shaped, clustered together and connected to each others and its dynamics (the 'traffic'): electrical and chemical signals of various types, shapes and strength constantly propagate through the brain to support its sensorimotor and cognitive ...
Brain
Non-neuronal brain cells modulate behaviour. A single gene in astrocytes can constrain repetitive behaviours, indicating that these cells are regulators of behavioural disruption in conditions ...
The brain
We are pleased to acknowledge the financial support of the Shanghai Research Center for Brain Science and Brain-Inspired Intelligence, United Imaging Healthcare Technology Group Co. Ltd, the ...
From Brain Science to Artificial Intelligence
During the past five years, important achievements in brain research have been made with the support of brain research projects. The development of optogenetics has made it possible to manipulate neural activities at a single-cell resolution [40].Large-scale manipulation can be further accomplished using advanced beam-modulation techniques [41], [42].
Top 100 in Neuroscience
Top 100 in Neuroscience. This collection highlights our most downloaded* neuroscience papers published in 2021. Featuring authors from around the world, these papers showcase valuable research ...
Generalized fear after acute stress is caused by change in neuronal
To understand the mechanisms underlying generalization of fear responses, we studied contextual fear. Pairing of an unconditioned stimulus (US; foot shock) and the conditioning stimulus (CS) leads to conditioned fear when testing occurs later in the same context (context A) (2, 13, 14).Increasing the intensity of the US produces fear generalization even when testing occurs later in a new ...
Brain Research Bulletin
The Brain Research Bulletin (BRB) is an open access, peer-reviewed journal which publishes novel, scientifically accurate research across all aspects of neuroscience research. The journal sections include Aging Neuroscience, Computational Neuroscience, Mechanisms of Behavior & Cognition, …. From January 1, 2023, Brain Research Bulletin will ...
Topics and trends in artificial intelligence assisted human brain research
Topical proportions and trends. The topical intervals in Table 4 and the developmental trends in Fig 2 clarify different groups of topics with different degrees of prominence. First, there are eight frequently discussed topics in the dataset, each with a proportion over 4% and accounting for 42.21% in total.
Home
Founded in 1966, Experimental Brain Research publishes original contributions on many aspects of experimental research of the central and peripheral nervous system. The scope of the journal encompasses neurophysiological mechanisms underlying sensory, motor, and cognitive functions in health and disease. This includes developmental, cellular ...
New study reveals breakthrough in understanding brain stimulation
For the first time, researchers show how the brain can precisely adapt to external stimulationMINNEAPOLIS/ST. PAUL (03/14/2024) — For the first time, researchers at the University of Minnesota Twin Cities showed that non-invasive brain stimulation can change a specific brain mechanism that is directly related to human behavior. This is a major step forward for discovering new therapies to ...
Brain‐inspired intelligence‐driven scientific research
The integration of brain-inspired AI into scientific research, while opening new horizons, also presents challenges related to data quality and stewardship, AI model interpretability, and the need for better understanding and collaboration between scientists and AI experts (Figure 1). The interplay between neuroscience and AI technology has ...
(PDF) The Brain and How it Functions
The brain is the primary function of the entire body, However, in this article, I will be focusing on the nervous system and the relationship between mind and body. This article focuses on "the ...
COVID's toll on the brain: new clues emerge
Research from neuroscientist Matthew Campbell and neurologist Colin Doherty at Trinity College Dublin has found that the blood-brain barrier, which separates the brain from the rest of the body ...
Human brains preserve in diverse environments for at least 12 000 years
2. Results. We identified a total of 4405 preserved human brains from 213 unique sources, which were reported from every world region except Antarctica (figure 1a; see electronic supplementary material, dataset S1).These brains are universally described as discoloured and shrunken (i.e. reduced in volume compared to the brains of living humans), albeit with reported variations in their degree ...
Brain Development and the Role of Experience in the Early Years
An account of brain development in the early years of childhood is only complete if we first examine the origins of this process during the prenatal months. Brain development is a protracted process that begins about 2 weeks after conception and continues into young adulthood 20 years later. Brain development that occurs during the prenatal ...
Guide for authors
The Neuroscience Peer Review Consortium Brain Research is a member of the Neuroscience Peer Review Consortium (NPRC). The NPRC has been formed to reduce the time expended and, in particular, the duplication of effort by, and associated burden on reviewers involved in the peer review of original neuroscience research papers.
Human Brain Mapping
Human Brain Mapping. Human Brain Mapping is a functional neuroanatomy and neuroimaging journal where all disciplines of neurology collide to advance the field. The journal offers basic, clinical, technical and theoretical research in the rapidly expanding field of human brain mapping. Proudly accessible, every issue is open to the world.
(PDF) Brain Cancer Treatment; A Systematic Review
Results: Statistical analysis showed that 50% of the papers used a virus, 36% used polymers, and 14% used cells as carriers to transfect the genes as a therapeutic agent in brain tumor models ...
NIH studies find severe symptoms of "Havana Syndrome," but no evidence
Using advanced imaging techniques and in-depth clinical assessments, a research team at the National Institutes of Health (NIH) found no significant evidence of MRI-detectable brain injury, nor differences in most clinical measures compared to controls, among a group of federal employees who experienced anomalous health incidents (AHIs).
Brain-Computer Interface: Advancement and Challenges
Abstract. Brain-Computer Interface (BCI) is an advanced and multidisciplinary active research domain based on neuroscience, signal processing, biomedical sensors, hardware, etc. Since the last decades, several groundbreaking research has been conducted in this domain. Still, no comprehensive review that covers the BCI domain completely has been ...
8 Google Employees Invented Modern AI. Here's the Inside Story
The paper was accepted for one of the evening poster sessions. By December, the paper was generating a buzz. Their four-hour session on December 6 was jammed with scientists wanting to know more.
Challenges to curing primary brain tumours
Challenge 1: redesign research pipeline. Clinical trials have yet to reveal an effective therapy for most brain tumours. This harsh reality stems, in part, from an incomplete understanding of ...
Behavioural Brain Research
Behavioural Brain Research is an international, interdisciplinary journal dedicated to the publication of articles in the field of behavioural neuroscience, broadly defined. Contributions from the entire range of disciplines that comprise the neurosciences, behavioural sciences or cognitive …. View full aims & scope.
Open-source AI models released by Tokyo lab Sakana founded by ...
Ha was previously the head of research at Stability AI and a Google Brain researcher. All the authors of the ground-breaking Google paper have since left the organisation.

Latest Article

Editor's Choice

Publish with Brain

Latest posts on X

Are there differences in the immune response among individuals of Black ethnicity with multiple sclerosis?

Exploring a genetic basis for disordered speech and language: FOXP2 mutations and striatal neurons

The BRAIN podcast

Video Abstracts

Brain endothelium class I molecules regulate neuropathology in experimental cerebral malaria

Spike propagation mapping reveals effective connectivity and predicts surgical outcome in epilepsy

Haemorrhage of human foetal cortex associated with SARS-CoV-2 infection

Diabetes and hypertension are related to amyloid-beta burden in the population-based Rotterdam Study

Neurogenetics Collection

China Collection

Neuroinflammation Collection

COVID Collection

Brain on the OUPblog

Of language, brain health, and global inequities

It’s time to use software-as-medicine to help an injured brain

How air pollution may lead to Alzheimer’s disease

When narcolepsy makes you more creative

Author resources

Want access to Brain ?

Introducing Brain Communications

Top trending articles

Related Titles

Affiliations

This Feature Is Available To Subscribers Only

Relating Structure and Function in the Human Brain: Relative Contributions of Anatomy, Stationary Dynamics, and Non-stationarities

Author Summary

Introduction

Predictive power of models

Role of homotopic connections

Predicting functional brain networks

Stationary FC, non-stationary FC, and non-stationarities

Ethics statement

Simulations

Performance

Analysis of dynamics

Supporting Information

Acknowledgments

Author Contributions

Experimental Brain Research

Latest issue

Latest articles

Sleep deprivation induces late deleterious effects in a pharmacological model of Parkinsonism

A computational model of motion sickness dynamics during passive self-motion in the dark

Inhibition of fatty acid amide hydrolase reverses aberrant prefrontal gamma oscillations in the sub-chronic PCP model for schizophrenia

Effects of the perceived temporal distance of events on mental time travel and on its underlying brain circuits

Journal updates

Important Update

Follow us on Twitter

Thanks to our reviewers

Journal information

New study reveals breakthrough in understanding brain stimulation therapies

Read more stories:

Related news releases

You are here

NIH studies find severe symptoms of “Havana Syndrome,” but no evidence of MRI-detectable brain injury or biological abnormalities

Connect with Us

Brain-Computer Interface: Advancement and Challenges

Sujoy Chandra Das

Yutaka Watanobe

Associated Data

1. Introduction

2. Applications of BCI

2.1. Biomedical Applications

2.1.1. Substitute to CNS

2.1.2. Assessment and Diagnosis

2.1.3. Therapy or Rehabilitation

2.1.4. Affective Computing

2.2. Non-Biomedical Applications

2.2.1. Gaming

2.2.2. Industry

2.2.3. Artistic Application

2.2.4. Transport

3. Structure of BCI

3.1. Invasive

3.2. Partially Invasive

Electrocorticography (ECoG)