Tel.: +49 541 969-7090
Institute of Cognitive Science,
49090 Osnabrück, Germany
Active dendrites implement probabilistic temporal logic gates
I’m interested in learning algorithms for neural networks both as models of computation in the brain and as machine learning and artificial intelligence tools. During my PhD I hope to contribute to the understanding of unsupervised dynamics in neural nets and how they facilitate discovery of immediate and latent features of its input, as well as maintenance of relevant information from the past: memory.
Delay-coupled reservoirs facilitate memory dependent prediction and forecasting of time-series via complex, non-linear delay-coupled dynamics. While the mathematics and theoretical understanding of these systems is hard, applying them in practice is much easier. How can we use deeper insights about the system’s dynamics to apply the algorithm to a wider array of machine learning problems?
Training a recurrent neural network to maintain a longer short-term memory of relevant information in an input sequence via backpropagation is difficult due to properties of gradient-based, iterative optimization. The seminal Hochreiter and Schmidhuber paper “Long short-term memory”  solves the problem by introducing a much more complex network architecture, the LSTM cell, that has proved wildly successful in deep learning architectures on problems ranging from translating language to recognizing objects in images. In its footsteps, other algorithms have been developed to ensure backpropagation works for recurrent networks. Can we instead use unsupervised methods to take care of maintaining memory in a recurrent network? How do the brains plasticity mechanisms help solve this task, and what is the objective a plastic and dynamic network optimizes?
P Nieters, J Leugering, G Pipa
IBM Journal of Research and Development 61 (2/3), 8: 7-8: 9
Neuromorphic computation in multi-delay coupled models