Julian | Researcher (AI in sciences)

I started as a trained audio technician, worked in the film industry, and then studied physics, where I did a Bachelor’s and Master’s thesis in Geophysics, started my Ph.D. program in classical nanoparticle research, worked briefly as a machine-learning engineer, and completed my doctorate in nonlinear quantum optics, and deep learning. Then, I did my first PostDoc at ETH Zurich, leading a small team working on deep learning in coherent diffraction imaging. Currently, I am doing my second PostDoc in an animal ecology group building Transformer-based models for bioacoustics.

I was awarded “Outstanding Reviewer” by IoP Nanotechnology. I have peer-reviewed multiple Journals for IoP and Springer Nature, including “Machine Learning: Science and Technology” and “Nature Communications.”

selected publications

animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics

Julian C Schäfer-Zimmermann , Vlad Demartsev , Baptiste Averly , and 8 more authors

arXiv preprint arXiv:2406.01253, 2024

Abs HTML Code

Bioacoustic research, vital for understanding animal behavior, conservation, and ecology, faces a monumental challenge: analyzing vast datasets where animal vocalizations are rare. While deep learning techniques are becoming standard, adapting them to bioacoustics remains difficult. We address this with animal2vec, an in- terpretable large transformer model, and a self-supervised training scheme tailored for sparse and unbalanced bioacoustic data. It learns from unlabeled audio and then refines its understanding with labeled data. Furthermore, we introduce and publicly release MeerKAT: Meerkat Kalahari Audio Transcripts, a dataset of meerkat (Suri- cata suricatta) vocalizations with millisecond-resolution annotations, the largest labeled dataset on non-human terrestrial mammals currently available. Our model outperforms existing methods on MeerKAT and the publicly available NIPS4Bplus birdsong dataset. Moreover, animal2vec performs well even with limited labeled data (few-shot learning). animal2vec and MeerKAT provide a new reference point for bioacoustic research, enabling scientists to analyze large amounts of data even with scarce ground truth information.
Finding the semantic similarity in single-particle diffraction images using self-supervised contrastive projection learning

Julian Zimmermann , Fabien Beguet , Daniel Guthruf , and 2 more authors

npj Computational Materials, Feb 2023

Abs HTML Code

Single-shot coherent diffraction imaging of isolated nanosized particles has seen remarkable success in recent years, yielding in-situ measurements with ultra-high spatial and temporal resolution. The progress of high-repetition-rate sources for intense X-ray pulses has further enabled recording datasets containing millions of diffraction images, which are needed for the structure determination of specimens with greater structural variety and dynamic experiments. The size of the datasets, however, represents a monumental problem for their analysis. Here, we present an automatized approach for finding semantic similarities in coherent diffraction images without relying on human expert labeling. By introducing the concept of projection learning, we extend self-supervised contrastive learning to the context of coherent diffraction imaging and achieve a dimensionality reduction producing semantically meaningful embeddings that align with physical intuition. The method yields substantial improvements compared to previous approaches, paving the way toward real-time and large-scale analysis of coherent diffraction experiments at X-ray free-electron lasers.
Deep neural networks for classifying complex features in diffraction images

Julian Zimmermann , Bruno Langbehn , Riccardo Cucini , and 13 more authors

Phys Rev E, Jun 2019

Abs HTML Code

Intense short-wavelength pulses from free-electron lasers and high-harmonic-generation sources enable diffractive imaging of individual nanosized objects with a single x-ray laser shot. The enormous data sets with up to several million diffraction patterns present a severe problem for data analysis because of the high dimensionality of imaging data. Feature recognition and selection is a crucial step to reduce the dimensionality. Usually, custom-made algorithms are developed at a considerable effort to approximate the particular features connected to an individual specimen, but because they face different experimental conditions, these approaches do not generalize well. On the other hand, deep neural networks are the principal instrument for today’s revolution in automated image recognition, a development that has not been adapted to its full potential for data analysis in science. We recently published [Langbehn et al., Phys. Rev. Lett. 121, 255301 (2018)PRLTAO0031-900710.1103/PhysRevLett.121.255301] the application of a deep neural network as a feature extractor for wide-angle diffraction images of helium nanodroplets. Here we present the setup, our modifications, and the training process of the deep neural network for diffraction image classification and its systematic bench marking. We find that deep neural networks significantly outperform previous attempts for sorting and classifying complex diffraction patterns and are a significant improvement for the much-needed assistance during postprocessing of large amounts of experimental coherent diffraction imaging data.
Coherent diffractive imaging of single helium nanodroplets with a high harmonic generation source

Daniela Rupp , Nils Monserud , Bruno Langbehn , and 14 more authors

Nat. Commun., Sep 2017

Abs HTML

Coherent diffractive imaging of individual free nanoparticles has opened routes for the in situ analysis of their transient structural, optical, and electronic properties. So far, single-shot single-particle diffraction was assumed to be feasible only at extreme ultraviolet and X-ray free-electron lasers, restricting this research field to large-scale facilities. Here we demonstrate single-shot imaging of isolated helium nanodroplets using extreme ultraviolet pulses from a femtosecond-laser-driven high harmonic source. We obtain bright wide-angle scattering patterns, that allow us to uniquely identify hitherto unresolved prolate shapes of superfluid helium droplets. Our results mark the advent of single-shot gas-phase nanoscopy with lab-based short-wavelength pulses and pave the way to ultrafast coherent diffractive imaging with phase-controlled multicolor fields and attosecond pulses.Diffraction imaging studies of free individual nanoparticles have so far been restricted to XUV and X-ray free - electron laser facilities. Here the authors demonstrate the possibility of using table-top XUV laser sources to image prolate shapes of superfluid helium droplets.