Information field theory for collective behavior

Proposed project for an Emmy Noether research group at the Goethe University Frankfurt

Julian C. Schäfer-Zimmermann

Max Planck Institute of Animal Behavior

Department for the Ecology of Animal Societies

Communication and Collective Movement (CoCoMo) Group

Short biography

• My background:
• Studied physics at the TU Berlin, Bachelor's and Master's theses at the German Research Centre for Geosciences on computational methods in physical oceanography
• Scholarship from the German National Academic Foundation (Studienstiftung des deutschen Volkes)
• PhD at the Max-Born-Institute Berlin on artificial intelligence in ultrafast nonlinear quantum optics under Prof. Thomas Möller
• Worked part-time in the industry as machine-learning engineer
• First PostDoc at ETH Zürich on computer vision in the nanostructures and ultrafast x-ray science group of Prof. Daniela Rupp
• I led a small team of researchers and student researchers and supervised four Master's theses (One jointly with IBM Zürich) and three semester projects
• Second PostDoc at the Max Planck Institute of Animal Behavior on large-scale deep learning and bioacoustics in the movement ecology group of Dr. Ariana Strandburg-Peshkin
• Organized and funded a small international workshop on bioacoustics, supervise two PhD students as part of the European Bioacoustic AI network (Horizon EU program)


• My vision for this project is to bring together all these experiences in a research group that focuses on gathering strength and insight from interdisciplinary approaches, with the aim of advancing the relatively new research field of 'Physics of Behaviour'*

Goethe University as host institute

• Goethe University is ideal because:
• Complex & Dynamical Systems & Machine Learning Group of Prof. Claudius Gros at Goethe Uni
• The soon starting Goethe Center for Mathematics and Modeling in the Natural Sciences (GCM2NS)
• The interdisciplinary Frankfurt Institute for Advanced Studies (FIAS)
• Frankfurt is ideal because:
• The interdisciplinary Center for Critical Computational studies (C3S)
• Senckenberg Biodiversity and Climate Research Centre
• What I bring to the Goethe University:
• I add the perspective of the emerging research field of Physics of Behavior
• I extend the Goethe University network to the Max Planck Institute of Animal Behavior and the Uni Konstanz, home of the Centre for the Advanced Study of Collective Behaviour, where I remain affiliated as a guest scientist and collaborator

Collective behavior, phase change, and criticality

Approaches to quantitative collective behavior

These models are only concerned with movement

Goal of this project

My research focuses on understanding the internal motivations of animal behavior directly from multimodal data. I model the dynamics of entire animal groups using an approach that renders these internal states visible.

From data to behavior

The animal collective

Phase 1:

From the animal collective to the behavioral neural network

Phase 2:

From the neural network to information field theory

Phase 2:

From the neural network to information field theory

The approach and novelty of this project

1. Learning behaviors directly from data by training a behavioral neural network (BNN), which assumes nothing
• This is a hypothesis-free abductivist approach to behavioral science
2. Define biologically relevant situations that show phase transitions or tuning in and out of a state of criticality, such as foraging or predator evasion
• Infer the behavioral states for the situations from the BNN
3. Learn a social Hamiltonian for each of these situations
• Pairwise and, later, higher order interactions via Graph Neural Networks
4. Recreate the situation using the information-field theory approach
• How is each behavioral state from each individual affecting every other behavioral state from every other individual
• These are predictive models, with which we can also predict behaviors for new and/or rare situations (Hypothesis generation)

The approach and novelty of this project

5. Once this is established, we can:
• Explore the collective behavior of other species / systems (Max Planck Institute of Animal Behavior (MPIAB) as collaborator)
• I will remain a guest scientist at MPIAB
• Derive world models with agents based on the learned behaviors of the neural network
• Backwards engineer analytical - biologically interpretable - models from the learned Hamiltonians
• Go back to the field and test hypotheses in playback experiments

Timeline

Risk mitigation

1. What if the BNN learns nothing
• We can kickstart phase two using:
• Classical ethological descriptors (e.g., "sleeping" or "foraging")
• Video as additional input modality for the BNN (this substantially decreases dataset temporal coverage)
• Synthetic data from established movement models
• Modify the collars worn by the meerkats (sampling frequencies, modalities, ...) and do an experimental campaign
• I followed up a comment by Prof. Meyer in terms of funding for this to ensure this remains a possibility
2. What if the modelling in the configuration space yields nothing
• We can fallback to node-based mean-field modelling
• Directly derive an analytical model from the behavioral states of the BNN

Final words

My research focuses on understanding the internal motivations of animal behavior directly from multimodal data. I model the dynamics of entire animal groups using an approach that renders these internal states visible.

The team

Thank you for you attention

Information field theory for collective behavior

Proposed project for an Emmy Noether research group at the Goethe University Frankfurt

Julian C. Schäfer-Zimmermann

Max Planck Institute of Animal Behavior

Department for the Ecology of Animal Societies

Communication and Collective Movement (CoCoMo) Group