Using Connectome Features to Constrain Echo State Networks
share
We report an improvement to the conventional Echo State Network (ESN), which already achieves competitive performance in one-dimensional time series prediction of dynamical systems. Our model – a 20%-dense ESN with reservoir weights derived from a fruit fly connectome (and from its bootstrapped distribution) – yields superior performance on a chaotic time series prediction task, and furthermore alleviates the ESN's high-variance problem. We also find that an arbitrary positioning of weights can degrade ESN performance and variance; and that this can be remedied in particular by employing connectome-derived weight positions. Herein we consider four connectome features – namely, the sparsity, positioning, distribution, and clustering of weights – and construct corresponding model classes (A, B, B_2, C) from an appropriate null model ESN; one with its reservoir layer replaced by a fruit fly connectivity matrix. After tuning relevant hyperparameters and selecting the best instance of each model class, we train and validate all models for multi-step prediction on size-variants (50, 250, 500, and 750 training input steps) of the Mackey-Glass chaotic time series; and compute their performance (Mean-Squared Error) and variance across train-validate trials.
READ FULL TEXT