Information Geometry of Learning Systems

Intelligent behaviour can only emerge through sensorimotor interactions, incorporating the brain, the body, and the environment. This crucial understanding has many far-reaching implications and led to the field of embodied intelligence. What does this understanding mean for learning? I am firmly convinced that the efficiency of learning strongly depends on the coupling of the system's control architecture, its brain, with its embodiment constraints. In the main part of my talk, I will outline an information-geometric approach to
such a coupling and present results on the design of embodied systems with concise control architectures. This formalises the notion of "cheap control" within the field of embodied intelligence. While most of the results are based on the class of restricted Boltzmann machines, I will conclude with ongoing research related to more general architectures. In particular, I will present initial results on the natural gradient method for deep learning in embodied agents.