animal2vec workshop · MPIABDay 3 · 09:30–10:15
What does one point mean?
Build the analysis dataset before you draw the map.
Julian C. Schäfer-Zimmermann
Max Planck Institute of Animal Behavior
Department for the Ecology of Animal Societies
Communication and Collective Movement (CoCoMo) Group
animal2vec workshop · MPIABDay 3 · 09:30–10:15

Every vector needs an observation record

row_id source_file start_s end_s unit context
event_001 rec_017.wav 12.30 12.48 event 10 s segment
event_002 rec_017.wav 16.10 16.22 event 10 s segment
event_003 rec_104.wav 4.80 5.16 event 10 s segment
Feature matrixRow i: D numbers
→
Stable row_idExplicit link, not position guessing
→
Metadata recordAudio, intervals, biological and technical fields

Illustrative schema—not actual participant recordings.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

The unit determines the scientific comparison

Call-level question

One interval per call. Compare examples while tracking individual, session, and detector origin.

Soundscape question

One fixed window per observation. Keep duration and sampling policy consistent.

“One row per frame” and “one independent biological observation” are not the same thing.

Which level will your result generalize to: another frame, recording, individual, or site?

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Pool only the frames assigned to your interval

# Z: [T, D]; t: frame-center timestamps in recording seconds
# start_s and end_s come from a declared annotation/detection policy
inside = (t >= start_s) & (t < end_s)
if not inside.any():
    raise ValueError("No frame centers fall inside this interval")
z_event = Z[inside].mean(axis=0)   # [D]
Use seconds from the returned timestamps. Keep the half-open interval convention.

Very short intervals may contain no frame center: flag them, rather than silently producing NaNs.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Your detector helps decide which observations exist

All recordingsCalls + background + artefacts
→
Event selectionAnnotations or detector threshold
→
Embedding datasetOnly the selected intervals

Missed calls

Will be absent from a prediction-selected repertoire.

False positives

May appear as a convincing “new cluster.”

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Keep the original recording clock

Segment startAbsolute offset in the recording
→
Frame centerComputed from frontend geometry
→
Returned timestampCenter + segment offset
Do not reset time to zero in every chunk or guess it from the number of rows.

Store both the source interval and the extraction segment/context policy.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Three operations often called “normalization”

Operation What it changes Use it deliberately
Waveform normalization The model’s audio input Part of the extraction pipeline
L2-normalize each vector Each observation’s overall vector length Makes direction, rather than magnitude, central
Scale each feature column Relative weight of different dimensions Changes distances and fitted projections
Do not add all three automatically. Save the chosen preprocessing policy.
animal2vec workshop · MPIABDay 3 · 09:30–10:15

“Similar” depends on the distance you choose

Interactive example requires JavaScript.

For your question, could vector magnitude carry useful information—or a nuisance factor?

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Join metadata by identity, not by row order

features
  row_id  → vector

metadata
  row_id  → file, interval,
            individual, site,
            device, session

Check the join.

Are IDs unique?
Did any rows disappear?
Are there unexpected duplicates?
Can three random rows be traced to audio?

Keep missing metadata explicitly missing; do not invent individuals, sites, or devices.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Save enough information to repeat the extraction

Model / processing

Checkpoint hash, package snapshot, layer policy, rate, channel, segment length, aggregation.

Observation / selection

row_id, source file, interval, annotation or detector origin, threshold/fusion settings.

analysis_bundle/
  embeddings.npy        # rows identified by rows.csv
  rows.csv              # row_id, source interval, metadata
  extraction.json       # checkpoint and processing provenance

Proposed export convention for the practical—not a built-in CLI output schema.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

More rows do not automatically mean more evidence

Many frames from one call

Useful temporal detail.
Not many independent calls.

Many calls from one deployment

Useful repeated observations.
Not many independent sites.

Split and report at the level relevant to the scientific claim.
animal2vec workshop · MPIABDay 3 · 09:30–10:15

Same coordinates; a different explanation

Interactive example requires JavaScript.

Reveal the colours by call group, site, and device. Which interpretation survives?

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Freeze a comparison policy before comparing methods

Same observationsStable row IDs and inclusion rules
→
Same representationCheckpoint, unit, preprocessing
→
Different analysisProjection / retrieval / clustering
Change one decision at a time and record what changed.

A method comparison is uninterpretable when the underlying observations also change unnoticed.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Ready for nearest-neighbour search?

Check Required answer
What is a row? One declared frame, event, window, or recording
Which space? One checkpoint and one extraction/preprocessing policy
Where is the sound? Source file + interval + stable row ID
What could confound similarity? At least two biological or technical metadata fields

Pick one query observation. Explain what would count as a useful neighbour.