From scientific imaging to digital specimens

Interview for the Imaging Technologies & Innovation Coordinator role at Senckenberg

Julian C. Schäfer-Zimmermann

Max Planck Institute of Animal Behavior

A brief introduction

My path has not been linear.

Trained audio, image and video technician; worked in film
Physics at TU Berlin; computational oceanography and optics
PhD in nonlinear quantum optics and AI (computer vision)
ETH Zurich: deep learning for coherent X-ray imaging; small team lead
MPI Animal Behavior: self-supervised learning for animal communication and collective behaviour

The domains changed — the core problem stayed the same.

How can we turn complex, high-dimensional measurements into knowledge that people can actually find, trust and use?

What I learned

Acquisition matters as much as analysis

Good downstream analysis starts with robust capture, calibration and documentation.

Similarity is key for accessibility

In X-ray imaging, we made millions of diffraction patterns semantically explorable by physical similarity.

Sparsity is the scientific norm

In all my career, I found that rare cases are often the most interesting ones.

For me, the usefulness of AI and technology in general is not defined by an abstract benchmark, but by the people that use it for things previously not possible.

How I understand the job

Not mainly

buying one more instrument
building one isolated database
training one impressive AI model
maximising the number of image files

Main task

turn distributed imaging expertise into a reliable institutional service
connect acquisition, metadata, provenance, quality control and publication
make workflows useful for curators, collection managers and researchers
define when automation helps — and when humans must remain in the loop

The goal is not simply more images. The goal is trusted, reusable digital knowledge.

High-level work plan

0–100 days
listen, map, measure

visit collections and imaging sites
map devices, workflows, identifiers and storage paths
understand SeSam / JACQ / AQUiLA / DINA / GBIF / BioCASe / DiSSCo interfaces and constraints
define baseline metrics together with the users; such as throughput, failure modes, metadata completeness

3–6 months
define the common layer

minimum Digital Specimen in the Senckenberg context
modality-specific acceptance criteria
prioritization framework with curators and collection managers
select three contrasting pilot workflows

6–12 months
pilot phase

flat high-throughput objects
high-value complex objects using photogrammetry, z-stack, or micro-CT
novel methods pilot (e.g., Gaussian splatting)
automated quality control plus human review loop (e.g., as part of an active learning pipeline)

12–24 months
scale into service

thoroughly documented procedures
software bridge for integration with DiSSCo and GBIF, regardless of previous storage system
dashboard for quality and progress
roadmap for wider rollout across collections and sites

Gaussian splatting for iridescent specimens

Challenge

Structural colour and metallic reflections change with viewing angle. Conventional texture mapping may smear, suppress or misplace that appearance.

→

Low-cost capture

Short orbit video with a phone or existing macro camera; fixed specimen and repeatable lighting. Camera poses are estimated automatically — no pre-measured camera path.

→

Reflectance-aware 3DGS

A Gaussian representation can encode view-dependent radiance and render interactive novel views in real time — potentially preserving iridescence better than a static texture.

Scientific boundary: the splat is an appearance-preserving Digital Media Object, not automatically a metrically validated mesh or a physical reflectance spectrum. Retain photogrammetry, structured light or CT when geometry is the primary measurement.

This is not a replacement for photogrammetry. Jewel beetles and similar specimens are difficult because their colour is structural and changes with viewing and illumination angle. Standard photogrammetry is strongest when corresponding surface points look similar across views; that assumption is weakened by moving highlights and iridescence.

Gaussian splatting is interesting because its representation can encode view-dependent appearance. A curator or researcher could rotate an interactive digital object and see colour shifts that a conventional static texture tends to average away. Capture could be a short, overlapping video with a phone or an existing macro camera. The camera positions do not need to be measured in advance, but the capture still needs sharp frames, sufficient overlap, locked focus and exposure, a stationary specimen and controlled lighting.

I would benchmark rather than promise: conventional photogrammetry, standard 3D Gaussian Splatting, and one method designed for reflective surfaces. The result would initially be an appearance-preserving digital surrogate for research communication, teaching and virtual access. If metric shape is required, I would pair it with a calibrated photogrammetric mesh, structured light or CT. This makes the project inexpensive, technically current, easy to evaluate and directly relevant to Senckenberg's heterogeneous collections.

Gaussian splatting of semi-transparent Cicada shell

How I would like to work

With respect

Curators and collection staff define scientific meaning. Technology should reduce friction, not bypass expertise.

With evidence

Before scaling, measure failure modes, costs and quality. A beautiful image is not enough if it is not linked and trustworthy.

With pragmatism

Build modularly, document well, keep humans in the loop, and make the first working service small enough to actually work.

Thank you.