Publications

2024

1. animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics

   Julian C Schäfer-Zimmermann ,  Vlad Demartsev ,  Baptiste Averly , and 8 more authors

   arXiv preprint arXiv:2406.01253, 2024

   Abs HTML Code

   Bioacoustic research, vital for understanding animal behavior, conservation, and ecology, faces a monumental challenge: analyzing vast datasets where animal vocalizations are rare. While deep learning techniques are becoming standard, adapting them to bioacoustics remains difficult. We address this with animal2vec, an in- terpretable large transformer model, and a self-supervised training scheme tailored for sparse and unbalanced bioacoustic data. It learns from unlabeled audio and then refines its understanding with labeled data. Furthermore, we introduce and publicly release MeerKAT: Meerkat Kalahari Audio Transcripts, a dataset of meerkat (Suri- cata suricatta) vocalizations with millisecond-resolution annotations, the largest labeled dataset on non-human terrestrial mammals currently available. Our model outperforms existing methods on MeerKAT and the publicly available NIPS4Bplus birdsong dataset. Moreover, animal2vec performs well even with limited labeled data (few-shot learning). animal2vec and MeerKAT provide a new reference point for bioacoustic research, enabling scientists to analyze large amounts of data even with scarce ground truth information.

2023

1. Finding the semantic similarity in single-particle diffraction images using self-supervised contrastive projection learning

   Julian Zimmermann ,  Fabien Beguet ,  Daniel Guthruf , and 2 more authors

   npj Computational Materials, Feb 2023

   Abs HTML Code

   Single-shot coherent diffraction imaging of isolated nanosized particles has seen remarkable success in recent years, yielding in-situ measurements with ultra-high spatial and temporal resolution. The progress of high-repetition-rate sources for intense X-ray pulses has further enabled recording datasets containing millions of diffraction images, which are needed for the structure determination of specimens with greater structural variety and dynamic experiments. The size of the datasets, however, represents a monumental problem for their analysis. Here, we present an automatized approach for finding semantic similarities in coherent diffraction images without relying on human expert labeling. By introducing the concept of projection learning, we extend self-supervised contrastive learning to the context of coherent diffraction imaging and achieve a dimensionality reduction producing semantically meaningful embeddings that align with physical intuition. The method yields substantial improvements compared to previous approaches, paving the way toward real-time and large-scale analysis of coherent diffraction experiments at X-ray free-electron lasers.

2022

1. The Scatman: an approximate method for fast wide-angle scattering simulations

   A Colombo ,  J Zimmermann ,  B Langbehn , and 8 more authors

   J. Appl. Crystallogr., Sep 2022

   Abs HTML Code

   Single-shot coherent diffraction imaging (CDI) is a powerful approach to characterize the structure and dynamics of isolated nanoscale objects such as single viruses, aerosols, nanocrystals and droplets. Using X-ray wavelengths, the diffraction images in CDI experiments usually cover only small scattering angles of a few degrees. These small-angle patterns represent the magnitude of the Fourier transform of the 2D projection of the sample’s electron density, which can be reconstructed efficiently but lacks any depth information. In cases where the diffracted signal can be measured up to scattering angles exceeding ∼10°, i.e. in the wide-angle regime, some 3D morphological information of the target is contained in a single-shot diffraction pattern. However, the extraction of the 3D structural information is no longer straightforward and defines the key challenge in wide-angle CDI. So far, the most convenient approach relies on iterative forward fitting of the scattering pattern using scattering simulations. Here the Scatman is presented, an approximate and fast numerical tool for the simulation and iterative fitting of wide-angle scattering images of isolated samples. Furthermore, the open-source software implementation of the Scatman algorithm, PyScatman, is published and described in detail. The Scatman approach, which has already been applied in previous work for forward-fitting-based shape retrieval, adopts the multi-slice Fourier transform method. The effects of optical properties are partially included, yielding quantitative results for small, isolated and weakly interacting samples. PyScatman is capable of computing wide-angle scattering patterns in a few milliseconds even on consumer-level computing hardware, potentially enabling new data analysis schemes for wide-angle coherent diffraction experiments.

2021

1. Probing ultrafast electron dynamics in helium nanodroplets with deep learning assisted diffraction imaging

   Julian Claudius Zimmermann

   Technische Universität Berlin , Sep 2021

   Abs HTML

   Coherent diffraction imaging (CDI) of single particles in free flight enables studying the structural composition of fragile nano-scaled matter. Such experiments demand high-intensity extreme ultra- violet (XUV) or X-ray light pulses, until recently only achievable at large-scale free-electron laser facilities. In this thesis, data from the first time-resolved infrared (IR), XUV pump-probe single-shot single- particle CDI experiment on superfluid helium nanodroplets using a high harmonic generation based laser source is presented and analyzed. Two configurations of the experiment were carried out, where the first configuration uses an IR beam intensity of « 2 ˆ 1014 W cm´2 and the second configuration uses a lower IR laser intensity of « 9 ˆ 1012 W cm´2. These both configurations yielded drastically different observations. Using the first, called intense, IR pulse configuration, ultrafast fragmentation dynamics, and collectively enhanced ionization of helium nanodroplets on the ps scale were observed. Whereas the second, called moderate, IR pulse configuration showed that superfluid helium nanodroplets exhibit a substantial decrease in scattering strength in the presence of the IR pulse on the fs scale. This decrease in scattering strength is the primary experimental finding of this thesis and is attributed to an ultrafast nonlinear electronic change of the refractive properties of the nanodroplets. For the analysis, concepts from effective medium theory were employed to build a parameter- free and simplistic model based on microscopic non-perturbative time-dependent Schrödinger equation calculations on atomic helium and the macroscopic and classical Clausius-Mossotti relation. Scattered fields of this so constructed effective medium were then calculated using Mie scattering theory. This so-constructed model provides a consistent qualitative description, which is in good agreement with the experimental observations. Furthermore, this thesis is interdisciplinary, wherein the second part two novel approaches from supervised and unsupervised deep learning are adapted, expanded, and validated with CDI data from a single-shot CDI experiment within the wide-angle X-ray scattering regime on individual rotating helium nanodroplets obtained in 2015 at the FERMI-facility in Trieste, Italy. Supervised and unsupervised deep learning are paradigms in machine-learning that refer to having ground truth label information or not. The here introduced techniques produce in both paradigms new state-of-the-art results and establish novel ways of how researchers can analyze large amounts of data in finite time.

2019

1. Deep neural networks for classifying complex features in diffraction images

   Julian Zimmermann ,  Bruno Langbehn ,  Riccardo Cucini , and 13 more authors

   Phys Rev E, Jun 2019

   Abs HTML Code

   Intense short-wavelength pulses from free-electron lasers and high-harmonic-generation sources enable diffractive imaging of individual nanosized objects with a single x-ray laser shot. The enormous data sets with up to several million diffraction patterns present a severe problem for data analysis because of the high dimensionality of imaging data. Feature recognition and selection is a crucial step to reduce the dimensionality. Usually, custom-made algorithms are developed at a considerable effort to approximate the particular features connected to an individual specimen, but because they face different experimental conditions, these approaches do not generalize well. On the other hand, deep neural networks are the principal instrument for today’s revolution in automated image recognition, a development that has not been adapted to its full potential for data analysis in science. We recently published [Langbehn et al., Phys. Rev. Lett. 121, 255301 (2018)PRLTAO0031-900710.1103/PhysRevLett.121.255301] the application of a deep neural network as a feature extractor for wide-angle diffraction images of helium nanodroplets. Here we present the setup, our modifications, and the training process of the deep neural network for diffraction image classification and its systematic bench marking. We find that deep neural networks significantly outperform previous attempts for sorting and classifying complex diffraction patterns and are a significant improvement for the much-needed assistance during postprocessing of large amounts of experimental coherent diffraction imaging data.

2018

1. Three-Dimensional Shapes of Spinning Helium Nanodroplets

   Bruno Langbehn ,  Katharina Sander ,  Yevheniy Ovcharenko , and 23 more authors

   Phys. Rev. Lett., Dec 2018

   Abs HTML

   A significant fraction of superfluid helium nanodroplets produced in a free-jet expansion has been observed to gain high angular momentum resulting in large centrifugal deformation. We measured single-shot diffraction patterns of individual rotating helium nanodroplets up to large scattering angles using intense extreme ultraviolet light pulses from the FERMI free-electron laser. Distinct asymmetric features in the wide-angle diffraction patterns enable the unique and systematic identification of the three-dimensional droplet shapes. The analysis of a large data set allows us to follow the evolution from axisymmetric oblate to triaxial prolate and two-lobed droplets. We find that the shapes of spinning superfluid helium droplets exhibit the same stages as classical rotating droplets while the previously reported metastable, oblate shapes of quantum droplets are not observed. Our three-dimensional analysis represents a valuable landmark for clarifying the interrelation between morphology and superfluidity on the nanometer scale.

2. XUV double-pulses with femtosecond to 650 ps separation from a multilayer-mirror-based split-and-delay unit at FLASH

   Mario Sauppe ,  Dimitrios Rompotis ,  Benjamin Erk , and 28 more authors

   J. Synchrotron Radiat., Dec 2018

   Abs HTML

   Extreme ultraviolet (XUV) and X-ray free-electron lasers enable new scientific opportunities. Their ultra-intense coherent femtosecond pulses give unprecedented access to the structure of undepositable nanoscale objects and to transient states of highly excited matter. In order to probe the ultrafast complex light-induced dynamics on the relevant time scales, the multi-purpose end-station CAMP at the free-electron laser FLASH has been complemented by the novel multilayer-mirror-based split-and-delay unit DESC (DElay Stage for CAMP) for time-resolved experiments. XUV double-pulses with delays adjustable from zero femtoseconds up to 650 picoseconds are generated by reflecting under near-normal incidence, exceeding the time range accessible with existing XUV split-and-delay units. Procedures to establish temporal and spatial overlap of the two pulses in CAMP are presented, with emphasis on the optimization of the spatial overlap at long time-delays via time-dependent features, for example in ion spectra of atomic clusters.

3. Single-shot diffractive imaging of individual helium nanodroplets with intense multicolor XUV pulses

   Nils Monserud ,  Bruno Langbehn ,  Mario Sauppe , and 17 more authors

   Optics InfoBase Conference Papers, Dec 2018

   Abs

   We report on single-shot coherent diffractive imaging of isolated helium nanodroplets obtained with intense multicolor XUV pulses from a high harmonic source. The wide-angle scattering patterns yield the droplets’ shapes and refractive indices.

2017

1. Coherent diffractive imaging of single helium nanodroplets with a high harmonic generation source

   Daniela Rupp ,  Nils Monserud ,  Bruno Langbehn , and 14 more authors

   Nat. Commun., Sep 2017

   Abs HTML

   Coherent diffractive imaging of individual free nanoparticles has opened routes for the in situ analysis of their transient structural, optical, and electronic properties. So far, single-shot single-particle diffraction was assumed to be feasible only at extreme ultraviolet and X-ray free-electron lasers, restricting this research field to large-scale facilities. Here we demonstrate single-shot imaging of isolated helium nanodroplets using extreme ultraviolet pulses from a femtosecond-laser-driven high harmonic source. We obtain bright wide-angle scattering patterns, that allow us to uniquely identify hitherto unresolved prolate shapes of superfluid helium droplets. Our results mark the advent of single-shot gas-phase nanoscopy with lab-based short-wavelength pulses and pave the way to ultrafast coherent diffractive imaging with phase-controlled multicolor fields and attosecond pulses.Diffraction imaging studies of free individual nanoparticles have so far been restricted to XUV and X-ray free - electron laser facilities. Here the authors demonstrate the possibility of using table-top XUV laser sources to image prolate shapes of superfluid helium droplets.