Adaptive state-feedback echo state networks for temporal sequence learning

Abstract Echo State Networks are recurrent neural networks that leverage a random reservoir’s dynamics, training only a simple readout layer. This approach is computationally efficient but limits network performance. By introducing state or output-feedback connections, the reservoir dynamics can be shaped to enhance the network’s ability to capture complex temporal dependencies. Here we propose AFRICO (Adaptive Feedback, Readout and Input with Connectivity Optimisation), a novel training framework for Echo State Networks that adapts input and state-feedback weights via an Extended Kalman Filter, followed by optimisation of a sparse readout layer that selectively connects reservoir states to the output. The key novelty lies in jointly adapting input and state-feedback pathways to shape reservoir dynamics, while separately constructing a sparse task-specific readout. This approach enables the Echo State Network to capture both the internal dynamics and output mapping of the target system. We evaluate AFRICO on synthetic linear and nonlinear systems, as well as in vivo electrophysiological recordings from fly photoreceptors, demonstrating its versatility for general-purpose input–output time-series modelling across both engineering and biological domains. Across varied hyperparameter initialisations, AFRICO achieves up to 88% reduction in Normalised Mean Squared Error compared to Echo State Networks with fixed output-feedback, while maintaining modest computational effort relative to fully trained Recurrent Neural Networks.