Reconstructing computational system dynamics from neural data with recurrent neural networks

Amit, D. J. & Brunel, N. Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cereb. Cortex 7, 237–252 (1997).

Article  CAS  PubMed  Google Scholar 

Brunel, N. Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci. 8, 183–208 (2000).

Article  CAS  PubMed  Google Scholar 

Carnevale, F., de Lafuente, V., Romo, R., Barak, O. & Parga, N. Dynamic control of response criterion in premotor cortex during perceptual detection under temporal uncertainty. Neuron 86, 1067–1077 (2015).

Article  CAS  PubMed  Google Scholar 

Deco, G. & Rolls, E. T. in Creating Brain-Like Intelligence (eds Sendhoff, B. et al.) 31–50 (Springer, 2009).

Durstewitz, D. Self-organizing neural integrator predicts interval times through climbing activity. J. Neurosci. 23, 5342–5353 (2003).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Durstewitz, D., Huys, Q. J. M. & Koppe, G. Psychiatric illnesses as disorders of network dynamics. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 6, 865–876 (2021).

PubMed  Google Scholar 

Durstewitz, D., Seamans, J. K. & Sejnowski, T. J. Neurocomputational models of working memory. Nat. Neurosci. 3, 1184–1191 (2000).

Article  CAS  PubMed  Google Scholar 

Goel, A. & Buonomano, D. V. Timing as an intrinsic property of neural networks: evidence from in vivo and in vitro experiments. Philos. Trans. R. Soc. Lond. B Biol. Sci. 369, 20120460 (2014).

Article  PubMed  PubMed Central  Google Scholar 

Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl Acad. Sci. USA 79, 2554–2558 (1982).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Izhikevich, E. M. Dynamical Systems in Neuroscience (MIT Press, 2007).

Machens, C. K., Romo, R. & Brody, C. D. Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307, 1121–1124 (2005).

Article  CAS  PubMed  Google Scholar 

Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013). A milestone in RNN-based analysis of neural data, in which task-trained RNNs were used to elucidate potential dynamical mechanisms of context-dependent decision-making, involving the context-dependent integration of evidence by approximate line attractors, similar to the patterns observed in the actual experimental data.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Miller, P. Dynamical systems, attractors, and neural circuits. F1000Res. 5, F1000 (2016).

Article  PubMed  PubMed Central  Google Scholar 

Rinzel, J. & Ermentrout, G. B. in Methods of Neuronal Modeling: From Synapses to Networks (eds Koch, C. & Segev, I.) 251–292 (MIT Press, 1998).

Wang, X.-J. Synaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory. J. Neurosci. 19, 9587–9603 (1999).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Wang, X.-J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).

Article  CAS  PubMed  Google Scholar 

Wilson, H. R. Spikes, Decisions, and Actions: The Dynamical Foundations of Neuroscience (Oxford Univ. Press, 1999).

Wilson, H. R. & Cowan, J. D. Excitatory and inhibitory interactions in localized populations of model neurons. Biophys. J. 12, 1–24 (1972).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Branicky, M. S. Universal computation and other capabilities of hybrid and continuous dynamical systems. Theor. Comput. Sci. 138, 67–100 (1995).

Article  Google Scholar 

Koiran, P., Cosnard, M. & Garzon, M. Computability with low-dimensional dynamical systems. Theor. Comput. Sci. 132, 113–128 (1994).

Article  Google Scholar 

Siegelmann, H. & Sontag, E. D. On the computational power of neural nets. J. Comput. Syst. Sci. 50, 132–150 (1995).

Article  Google Scholar 

Bhalla, U. S. & Iyengar, R. Emergent properties of networks of biological signaling pathways. Science 283, 381–387 (1999).

Article  CAS  PubMed  Google Scholar 

Bhalla, U. S. & Iyengar, R. Robustness of the bistable behavior of a biological signaling feedback loop. Chaos 11, 221–226 (2001).

Article  CAS  PubMed  Google Scholar 

Durstewitz, D. & Gabriel, T. Dynamical basis of irregular spiking in NMDA-driven prefrontal cortex neurons. Cereb. Cortex 17, 894–908 (2007).

Article  PubMed  Google Scholar 

Durstewitz, D. & Seamans, J. K. The computational role of dopamine D1 receptors in working memory. Neural Netw. 15, 561–572 (2002).

Article  PubMed  Google Scholar 

Mackey, M. C. & Glass, L. Oscillation and chaos in physiological control systems. Science 197, 287–289 (1977).

Article  CAS  PubMed  Google Scholar 

Sherman, A. Dynamical systems theory in physiology. J. Gen. Physiol. 138, 13–19 (2011).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Machado, T. A., Kauvar, I. V. & Deisseroth, K. Multiregion neuronal activity: the forest and the trees. Nat. Rev. Neurosci. 23, 683–704 (2022).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Paulk, A. C. et al. Large-scale neural recordings with single neuron resolution using Neuropixels probes in human cortex. Nat. Neurosci. 25, 252–263 (2022).

Article  CAS  PubMed  Google Scholar 

Steinmetz, N. A. et al. Neuropixels 2.0: a miniaturized high-density probe for stable, long-term brain recordings. Science 372, eabf4588 (2021).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Urai, A. E., Doiron, B., Leifer, A. M. & Churchland, A. K. Large-scale neural recordings call for new insights to link brain and behavior. Nat. Neurosci. 25, 11–19 (2022).

Article  CAS  PubMed  Google Scholar 

Vogt, N. Massively parallel intracellular recordings. Nat. Methods 16, 1079–1079 (2019).

Article  CAS  PubMed  Google Scholar 

Brunton, S. L., Proctor, J. L. & Kutz, J. N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 113, 3932–3937 (2016). Introduces the sparse identification of non-linear dynamical systems (SINDy) framework for DS reconstruction that delivers an interpretable representation of the dynamics, based on a known function library, and can be trained in a very efficient way.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Champion, K., Lusch, B., Kutz, J. N. & Brunton, S. L. Data-driven discovery of coordinates and governing equations. Proc. Natl Acad. Sci. USA 116, 22445–22451 (2019). The first study to combine autoencoders with a DS reconstruction model (SINDy) in order to find suitable low-dimensional latent representations and coordinate transformations on which the dynamics can be efficiently learned.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Durstewitz, D. A state space approach for piecewise-linear recurrent neural networks for identifying computational dynamics from neural measurements. PLoS Comput. Biol. 13, e1005542 (2017).

Article  PubMed  PubMed Central  Google Scholar 

Hernandez, D. et al. Nonlinear evolution via spatially-dependent linear dynamics for electrophysiology and calcium data. Neurons Behav. Data Anal. Theory 3, 3 (2020).

Kass, R. E., Eden, U. T. & Brown, E. N. Analysis of Neural Data (Springer, 2014).

Kim, T. D., Luo, T. Z., Pillow, J. W. & Brody, C. D. Inferring latent dynamics underlying neural population activity via neural differential equations. In Proc. 38th International Conference on Machine Learning (eds Meila, M. & Tong, Z.) 5551–5561 (PMLR, 2021).

Koppe, G., Toutounji, H., Kirsch, P., Lis, S. & Durstewitz, D. Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI. PLoS Comput. Biol. 15, e1007263 (2019).

Article  CAS  PubMed  PubMed Central  Google Scholar 

Kramer, D., Bommer, P. L., Tombolini, C., Koppe, G. & Durstewitz, D. Reconstructing nonlinear dynamical systems from multi-modal time series. In Proc. 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 11613–11633 (PMLR, 2022). Develops an architecture specifically for DS reconstruction that enables the exploitation of many statistically different data modalities simultaneously for reconstruction, such as neural recordings and behavioural responses.

Pandarinath, C. et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nat. Methods 15, 805–815 (2018). Takes previous statistical inference frameworks for RNNs from neural data one step further, situating them in a deep variational autoencoder structure that also allows for the inference of unobserved inputs to a given target area.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Paninski, L. & Cunningham, J. P. Neural data science: accelerating the experiment-analysis-theory cycle in large-scale neuroscience. Curr. Opin. Neurobiol. 50, 232–241 (2018).

Article  CAS  PubMed  Google Scholar 

Alligood, K. T., Sauer, T. D. & Yorke, J. A. Chaos: An Introduction to Dynamical Systems (Springer, 1996).

Perko, L. Differential Equations and Dynamical Systems Vol. 7 (Springer, 2001).

Strogatz, S. H. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering (CRC, 2018).

Vyas, S., Golub, M. D., Sussillo, D. & Shenoy, K. V. Computation through neural population dynamics. Annu. Rev. Neurosci. 43, 249–275 (2020).

Article  CAS  PubMed  PubMed Central 

Comments (0)

No login
gif