Most of our existing plasticity models are essentially a mathematical description of how to move a single parameter w, the synaptic weight, of a synapse up and down. We are good at casting this into a differential equation framework that works for both rate-based and spiking neurons and can be fitted to data from STDP experiments.
However, chemical synapses are complicated dynamical systems in their own right with a potentially high-dimensional state space and possibly complex latent dynamics. But what is the role if this synaptic complexity? Is it just an epiphenomenon of how biology implements LTP and LTD or do the complicated temporal dynamics do something essential for learning (Redondo and Morris, 2011; Ziegler et al. 2015; Zenke et al. 2015)? Unlikely. A body of theoretical work suggests that synaptic complexity may be crucial for memory retention (Fusi et al., 2005; Lahiri and Ganguli, 2013; Benna and Fusi, 2016).
Until recently, most existing theoretical work was focused on memory retention at the single synapse level. However, just a while ago we have shown that “synaptic intelligence”, i.e. specific complex synaptic dynamics are can, in fact, be sufficient to avoid catastrophic forgetting in deep neural networks. However, this only gives us a glimpse of what complex synaptic dynamics could in principle be capable of and opens the door to many exciting future research directions.
- Benna, M.K., and Fusi, S. (2016). Computational principles of synaptic memory consolidation. Nature Neuroscience 19, 1697–1706.
- Fusi, S., Drew, P.J., and Abbott, L.F. (2005). Cascade models of synaptically stored memories. Neuron 45, 599–611.
- Lahiri, S., and Ganguli, S. (2013). A memory frontier for complex synapses. In Advances in Neural Information Processing Systems, (Tahoe, USA: Curran Associates, Inc.), pp. 1034–1042.
- Redondo, R.L., and Morris, R.G.M. (2011). Making memories last: the synaptic tagging and capture hypothesis. Nat Rev Neurosci 12, 17–30.
- Zenke, F., Agnes, E.J., and Gerstner, W. (2015). Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nat Commun 6.
- Ziegler, L., Zenke, F., Kastner, D.B., and Gerstner, W. (2015). Synaptic Consolidation: From Synapses to Behavioral Modeling. J Neurosci 35, 1319–1334.
- Zenke, F.*, Poole, B.*, and Ganguli, S. (2017). Continual Learning Through Synaptic Intelligence. In ICML, pp. 3987–3995.