Update (2020-12-30) Now published:

Cramer, B., Stradmann, Y., Schemmel, J., and Zenke, F. (2020). The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks. IEEE Transactions on Neural Networks and Learning Systems 1–14. https://ieeexplore.ieee.org/document/9311226

A range of exciting developments around surrogate gradient methods has put researchers in the position to build sophisticated functional spiking neural networks that compute specific functions. A major hurdle to further accelerate this line of research is the lack of free spike-based benchmark datasets that are not saturated by existing methods, computationally tractable, and solve problems of general interest. This lack makes it is difficult to compare the different emerging approaches quantitatively. Spurred by this immediate need for suitable benchmarks, we collaborated with Benjamin Cramer, Yannik Stradmann, and Johannes Schemmel at the University of Heidelberg to develop two supervised datasets that satisfy these requirements. They can be downloaded freely from compneuro.net. An initial characterization using both conventional and spiking neural network classifiers can be found in the accompanying preprint.