animal2vec workshop · MPIABDay 1 · 09:15–10:35
animal2vec
without math
From a noisy recording to a research-ready prediction.
Julian C. Schäfer-Zimmermann
Max Planck Institute of Animal Behavior
Department for the Ecology of Animal Societies
Communication and Collective Movement (CoCoMo) Group
animal2vec workshop · MPIABDay 1 · 09:15–10:35

Start with the output you need

Where did a call occur?

A start time, an end time, and a label.

What sounds are present?

One or more labels for a recording.

Which examples are similar?

Neighbours in a learned representation.

What should I annotate next?

Candidates for human review.

Choose one of these questions for your own recordings.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

The difficult part is not an isolated clean call

Paper Fig. 1a, cropped: alarm calls amid scratching, bumping, and digging.
Paper Fig. 1a, cropped: alarm calls amid scratching, bumping, and digging.

Find the vocalization.

Ignore the recorder.

Keep the timing.

Which strong acoustic events in this figure are not the target vocalizations?

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Rare can mean few events—or little time

Paper Fig. 2, cropped. Event counts and durations are different imbalances.
Paper Fig. 2, cropped. Event counts and durations are different imbalances.
MeerKAT, as reported
1,068 h

184 h with complete event labels

A short-note call has a median duration of 37 ms.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Learn, specialize, apply

PretrainUnlabelled audio
Learn useful representations
Fine-tuneAnnotated audio
Learn the target labels
InferNew audio
Apply fixed weights
Only the first two stages change the learned model.
animal2vec workshop · MPIABDay 1 · 09:15–10:35

A neural network is a chain of learned transformations

Numbers inSamples from a microphone
Many adjustable stepsFilters and transformations
Numbers outRepresentations or scores

Before training

The adjustable values do not yet solve the task.

After training

The values have been changed using a training objective.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Training adjusts the model; inference does not

Run examplesProduce predictions
Measure mismatchAgainst a training target
Adjust weightsRepeat over many batches
The target is different in self-supervised pretraining and labelled fine-tuning.

When you run the same checkpoint on a new WAV file, which box is absent?

animal2vec workshop · MPIABDay 1 · 09:15–10:35

The input is a waveform—not a picture

Example: 8 kHz recording
8,000

amplitude samples per second

10 seconds → 80,000 samples

A spectrogram is a view of the audio

Useful for us to inspect frequencies and timing. It is not the input image expected by this model.

The sample rate is part of the model/data contract.
animal2vec workshop · MPIABDay 1 · 09:15–10:35

Learned filters turn samples into local features

Paper Fig. 1b, cropped: clean examples illustrate spectral diversity.
Paper Fig. 1b, cropped: clean examples illustrate spectral diversity.

Look locally first.

A SincNet-style filterbank learns frequency bands. Convolutions turn nearby samples into a shorter sequence of feature vectors.

A feature vector is a list of numbers describing a local part of the signal.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

A model frame is not a waveform sample

80,000 samples10 seconds at 8 kHz
Convolutional frontendStride reduces sequence length
About 2,000 framesOne learned vector per position
MeerKAT baseline: a nominal 5 ms frame hop. Always use the returned timestamps.
animal2vec workshop · MPIABDay 1 · 09:15–10:35

Local evidence becomes contextual evidence

Local question

What pattern is present around this position?

Contextual question

How does that pattern relate to the rest of this segment?

Local feature sequenceOne vector at each position
Transformer layersExchange information across positions
Contextual embeddingsA representation at each position
animal2vec workshop · MPIABDay 1 · 09:15–10:35

Attention: which other positions matter here?

Interactive example requires JavaScript.

Move the selected frame. Which parts of the segment could change its interpretation?

animal2vec workshop · MPIABDay 1 · 09:15–10:35

An embedding exists before the label decision

Illustration: rows are time positions; columns are learned dimensions.

A description in numbers.

The classification head reads this description and produces scores for the trained labels.

Embedding ≠ call label.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Several labels may be true at the same time

Output label Illustrative score Interpretation
Close call 0.82 Evidence for this call type
Alarm call 0.07 Little evidence for this label
Focal 0.76 The caller may be the focal animal
Independent sigmoid scores—not a single winner-takes-all choice.

Illustrative numbers, not measured checkpoint outputs.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Four related outputs, four different questions

Representation / output One row or answer per… What it supports
Frame scores Model position × label When is label evidence high?
Fused events Contiguous active interval Where does a predicted event start/end?
Recording-level labels Recording × label Which labels occur somewhere?
Embeddings Model position × feature How can sounds be represented or compared?
A recording-level classifier does not automatically provide event boundaries.
animal2vec workshop · MPIABDay 1 · 09:15–10:35

How can a model learn without call annotations?

Interactive example requires JavaScript.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

The teacher is a moving reference—not an annotator

Student weightsChanged by the training objective
Running averageSmoothly tracks the student
Teacher weightsProduce targets from the full input
The teacher is updated from the student’s weights. It has no secret biological labels.
animal2vec workshop · MPIABDay 1 · 09:15–10:35

The training scheme is designed for difficult audio

Self-supervised learning

Use the much larger pool of unlabelled recordings.

Mixed examples

The paper uses between-classes learning to expose the model to mixtures.

Imbalanced labels

Focal loss reduces the emphasis on already-easy supervised examples.

These are learning choices—not filters you turn on while running an existing model.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Fine-tuning gives the representation a specific task

Pretrained encoderLearned from unlabelled audio
New labelled taskCall types and event intervals
Fine-tuned checkpointEncoder + trained output head

Human work

Define meaningful labels and annotate representative data.

Model work

Learn how those labels relate to the audio.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

What did the original study demonstrate?

Paper Fig. 4, cropped: precision–recall curves on MeerKAT.
Paper Fig. 4, cropped: precision–recall curves on MeerKAT.

1% of fine-tuning labels

Micro-average AP: 0.83

100% of fine-tuning labels

Micro-average AP: 0.91

What stayed the same?

Pretraining used the full 1,068 h.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

A strong published result is not a transfer guarantee

Evidence supplied by the paper

MeerKAT experiments and the separately pretrained birdsong transfer experiments.

Evidence your project still needs

Performance on recordings representative of your species, site, devices, and target labels.

Name one way your data differ from the data used to train the model you plan to use.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Paper, workshop package, and “modern” recipe

Name Meaning in these slides
animal2vec, original paper Scientific method and reported experiments
a2v2 Python package The supplied native-PyTorch workshop implementation
configs/MeerKAT/ Animal2Vec 1.0 reproduction-oriented configurations
configs/modern/ A different, opt-in architecture/checkpoint family
The modern fine-tuning recipe is CLS-based. The event runner requires a frame head.
animal2vec workshop · MPIABDay 1 · 09:15–10:35

Put the complete inference path together

Interactive example requires JavaScript.

At which stage do we obtain embeddings? At which stage can a threshold change the result?

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Long recordings are processed in bounded segments

Long WAV fileOriginal time origin
Separate model passes10 s segments by default
Concatenated outputsTimestamps include segment offsets
Concatenating outputs does not create attention across segment boundaries.
animal2vec workshop · MPIABDay 1 · 09:15–10:35

Three audio choices to check before inference

Sample rate

Inference resamples to the checkpoint’s rate. It cannot restore frequencies lost by that conversion.

Channels

The default averages channels. A selected channel uses a zero-based index.

Model vocabulary

A checkpoint only scores the labels its output head was trained to predict.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

“Good enough” depends on the research question

Missed events

Could hide rare calls, interactions, or behavioural contexts.

False positives

Could inflate call counts and create unnecessary review work.

Choose an operating point using labelled validation data—not appearance alone.

Inspect false negatives as well as the events the model already found.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Explain the pipeline to a colleague

A new WAV file enters a fixed checkpoint.
What happens next?

Explain

Samples → local features → contextual embeddings → scores → intervals.

Distinguish

Learning weights, selecting a threshold, and interpreting a prediction.

Two minutes in pairs. Identify one remaining uncertainty.

animal2vec workshop · MPIABDay 1 · 09:15–10:35

Next: make a prediction you can inspect

SelectCompatible frame checkpoint + WAV
RunRead the CLI arguments
InspectAudio + predicted intervals
Keep the checkpoint, settings, source filename, and output together.

After the break: prepared CoCoMo environment → CLI tour → first inference.

animal2vec workshop · MPIABDay 1 · 14:30–15:00

Threshold: the score stays, the decision changes

Interactive example requires JavaScript.

a2v2-infer MODEL.native.pt AUDIO.wav events.tsv \
  --threshold 0.50 --method avg --fusion-window-seconds 0

Raise the threshold: which weak events vanish, and which runs fragment?

animal2vec workshop · MPIABDay 1 · 14:30–15:00

Pool → threshold → join adjacent active frames

Interactive example requires JavaScript.

a2v2-infer MODEL.native.pt AUDIO.wav events.tsv \
  --threshold 0.50 --method max --fusion-window-seconds 0.10

Average can attenuate short peaks. Maximum can extend or merge nearby events. Inspect both timing and false positives.

animal2vec workshop · MPIABDay 2 · 09:30–11:00

Three linked objects: audio, events, manifest

project/
  wav/clip01.wav
  lbl/clip01.h5
  manifests/train.tsv
  manifests/valid.tsv

WAV: the recorded signal

HDF5: sample-index event intervals

TSV: files in each split

train.tsv · first line: audio root · remaining fields separated by a literal TAB

/datasets/project
wav/clip01.wav	80000

The row means 80,000 waveform samples—not 80,000 model frames. Set task.data to the manifest directory.

animal2vec workshop · MPIABDay 2 · 09:30–11:00

An event label is an interval in waveform samples

# Toy vocabulary: task.unique_labels = ["call_A", "call_B"]
# 8 kHz audio; call_A from 1.00 s up to (not including) 1.15 s
start_frame_lbl = [8000]
end_frame_lbl   = [9200]
lbl_cat         = [0]

Required arrays

Same length. Starts inclusive; ends exclusive. Label indices follow the configured vocabulary.

Do not infer “background”

A fully reviewed no-event file can have empty arrays. Missing annotations are not known negatives.

Training requires the configured audio rate. min_label_size filters HDF5 file bytes; 3032 is not a minimum number of calls.

animal2vec workshop · MPIABDay 2 · 11:15–12:15

Adapt the recipe—not the checkpoint’s geometry

What to edit Relevant fields What to preserve / coordinate
Data and task task.data, unique_labels
dataset.train_subset / valid_subset
Label order, complete annotations, held-out groups
Starting model model.w2v_path
task.sample_rate / conv_feature_layers
Rate and frontend must match the pretrained checkpoint
Compute and output distributed_world_size, max_tokens
update_freq, checkpoint.save_dir
Match the actual launch; use a new run directory
Short-run timing max_update, freeze_finetune_updates
warmup, validation and saving intervals
A 200-update run must not inherit a 10,000-update freeze schedule
Paper recipe ≠ smoke recipe. max_tokens is a waveform-sample budget, not a file count.
animal2vec workshop · MPIABDay 2 · 14:15–15:00
Beyond the label
What an embedding is—and what it is not.
Julian C. Schäfer-Zimmermann
Max Planck Institute of Animal Behavior
Department for the Ecology of Animal Societies
Communication and Collective Movement (CoCoMo) Group
animal2vec workshop · MPIABDay 2 · 14:15–15:00

A representation is not a prediction table

Embeddings

[T, D]
T time positions; D learned features.
Useful for comparison and downstream analysis.

Label scores

[T, C]
The same T positions; C trained labels.
Useful for the supervised detection task.

The baseline has 1,024 feature dimensions. That does not mean 1,024 call categories.
animal2vec workshop · MPIABDay 2 · 14:15–15:00

The coordinates are learned—not named acoustic measurements

Illustrative matrix; not a measured activation pattern.

Dimension 42 is not automatically “pitch.”

Information may be distributed across many dimensions. A coordinate can encode mixtures of signal and context.

What would you need to show before calling a feature “individual identity”?

animal2vec workshop · MPIABDay 2 · 14:15–15:00

Two different kinds of averaging

Several layersRepresentations at the same time position
Layer averageStill one vector per model frame
Temporal aggregationOne vector for a chosen interval
The runner already averages selected layers. It does not automatically average over a whole recording.
animal2vec workshop · MPIABDay 2 · 14:15–15:00

What the workshop API returns

Attribute Shape / type Meaning
result.embeddings [T, D] Layer-averaged frame representations
result.probabilities [T, C] Independent sigmoid label scores
result.timestamps [T] Frame-center times in recording seconds
result.events Tuple of intervals Pooled, thresholded, fused predictions

Arrays are returned as CPU tensors. Exact T comes from the audio geometry and endpoint trimming.

animal2vec workshop · MPIABDay 2 · 14:15–15:00

Choose the unit before you make one vector

Unit Question it can support What may be lost or mixed
Frame How does the sound change over time? Many strongly correlated observations
Annotated or predicted event How do individual calls compare? Within-call temporal order after pooling
Fixed window What is in a standard duration of audio? Calls and background in the same window
Whole recording How do broad soundscapes compare? Rare brief events can be diluted
animal2vec workshop · MPIABDay 2 · 14:15–15:00

Temporal mean pooling discards order

Interactive example requires JavaScript.

Can these two frame sequences have the same mean but different biological interpretations?

animal2vec workshop · MPIABDay 2 · 14:15–15:00

Pretrained and fine-tuned spaces answer different objectives

Pretrained representation

Learned to match teacher targets from unlabelled audio.

Fine-tuned representation

Further shaped by the chosen supervised labels.

The paper reports linear probing of both frozen representations—not proof that every useful factor is preserved.
animal2vec workshop · MPIABDay 2 · 14:15–15:00

Use the extraction path that actually exists

Supported by InferenceRunner

A native fine-tuning checkpoint with a frame head.
Returns scores, embeddings, times, and events.

A different path is required

A pure pretraining checkpoint or the checked-in modern CLS classifier.
Do not pass either to the event runner.

The archive has no dedicated embedding-export CLI. Use the Python API in the practical session.

animal2vec workshop · MPIABDay 2 · 14:15–15:00

Extract all three aligned arrays in one pass

from a2v2 import InferenceRunner
from a2v2.data import load_audio

runner = InferenceRunner("MODEL.native.pt")
waveform, sample_rate = load_audio("AUDIO.wav")
result = runner.run_tensor(waveform, sample_rate)

Z = result.embeddings.numpy()     # [T, D]
t = result.timestamps.numpy()     # [T], seconds
p = result.probabilities.numpy()  # [T, C]

Use the provided native frame checkpoint. Replace the two illustrative filenames.

animal2vec workshop · MPIABDay 2 · 14:15–15:00

An event embedding can contain surrounding context

Audio segmentTarget call + neighbours + background
Contextual frame vectorsDepend on the model’s input context
Pooled event vectorSelect frames inside the interval
Cropping after inference is not necessarily equivalent to rerunning the model on an isolated call.
animal2vec workshop · MPIABDay 2 · 14:15–15:00

Comparable vectors need a comparable extraction policy

Record this Why it matters
Checkpoint identity and hash Different learned coordinate systems
Layers and temporal aggregation Different definitions of the representation
Sample rate, channel, segment length Different input and context
Source file and interval Return to the actual audio
Do not concatenate vectors from independently trained checkpoints as if their coordinates aligned.
animal2vec workshop · MPIABDay 2 · 14:15–15:00

Three ways to use the representation

Retrieve

Find examples near a known call or acoustic pattern.

Explore

Inspect structure, candidate groups, and unusual observations.

Predict

Train a separate classifier or regression model on labelled outcomes.

Retrieval and exploration can be unlabelled. A labelled downstream classifier is supervised learning.

animal2vec workshop · MPIABDay 2 · 14:15–15:00

Leave this afternoon with a traceable analysis dataset

ExtractZ, timestamps, scores
Define intervalsFrame / event / window
Save togetherVectors + row IDs + metadata
Tomorrow, every plotted point must lead back to an identifiable piece of audio.

Choose one unit and one metadata field you will inspect first.

animal2vec workshop · MPIABDay 3 · 09:30–10:15
What does one point mean?
Build the analysis dataset before you draw the map.
Julian C. Schäfer-Zimmermann
Max Planck Institute of Animal Behavior
Department for the Ecology of Animal Societies
Communication and Collective Movement (CoCoMo) Group
animal2vec workshop · MPIABDay 3 · 09:30–10:15

Every vector needs an observation record

row_id source_file start_s end_s unit context
event_001 rec_017.wav 12.30 12.48 event 10 s segment
event_002 rec_017.wav 16.10 16.22 event 10 s segment
event_003 rec_104.wav 4.80 5.16 event 10 s segment
Feature matrixRow i: D numbers
Stable row_idExplicit link, not position guessing
Metadata recordAudio, intervals, biological and technical fields

Illustrative schema—not actual participant recordings.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

The unit determines the scientific comparison

Call-level question

One interval per call. Compare examples while tracking individual, session, and detector origin.

Soundscape question

One fixed window per observation. Keep duration and sampling policy consistent.

“One row per frame” and “one independent biological observation” are not the same thing.

Which level will your result generalize to: another frame, recording, individual, or site?

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Pool only the frames assigned to your interval

# Z: [T, D]; t: frame-center timestamps in recording seconds
# start_s and end_s come from a declared annotation/detection policy
inside = (t >= start_s) & (t < end_s)
if not inside.any():
    raise ValueError("No frame centers fall inside this interval")
z_event = Z[inside].mean(axis=0)   # [D]
Use seconds from the returned timestamps. Keep the half-open interval convention.

Very short intervals may contain no frame center: flag them, rather than silently producing NaNs.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Your detector helps decide which observations exist

All recordingsCalls + background + artefacts
Event selectionAnnotations or detector threshold
Embedding datasetOnly the selected intervals

Missed calls

Will be absent from a prediction-selected repertoire.

False positives

May appear as a convincing “new cluster.”

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Keep the original recording clock

Segment startAbsolute offset in the recording
Frame centerComputed from frontend geometry
Returned timestampCenter + segment offset
Do not reset time to zero in every chunk or guess it from the number of rows.

Store both the source interval and the extraction segment/context policy.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Three operations often called “normalization”

Operation What it changes Use it deliberately
Waveform normalization The model’s audio input Part of the extraction pipeline
L2-normalize each vector Each observation’s overall vector length Makes direction, rather than magnitude, central
Scale each feature column Relative weight of different dimensions Changes distances and fitted projections
Do not add all three automatically. Save the chosen preprocessing policy.
animal2vec workshop · MPIABDay 3 · 09:30–10:15

“Similar” depends on the distance you choose

Interactive example requires JavaScript.

For your question, could vector magnitude carry useful information—or a nuisance factor?

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Join metadata by identity, not by row order

features
  row_id  → vector

metadata
  row_id  → file, interval,
            individual, site,
            device, session

Check the join.

Are IDs unique?
Did any rows disappear?
Are there unexpected duplicates?
Can three random rows be traced to audio?

Keep missing metadata explicitly missing; do not invent individuals, sites, or devices.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Save enough information to repeat the extraction

Model / processing

Checkpoint hash, package snapshot, layer policy, rate, channel, segment length, aggregation.

Observation / selection

row_id, source file, interval, annotation or detector origin, threshold/fusion settings.

analysis_bundle/
  embeddings.npy        # rows identified by rows.csv
  rows.csv              # row_id, source interval, metadata
  extraction.json       # checkpoint and processing provenance

Proposed export convention for the practical—not a built-in CLI output schema.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

More rows do not automatically mean more evidence

Many frames from one call

Useful temporal detail.
Not many independent calls.

Many calls from one deployment

Useful repeated observations.
Not many independent sites.

Split and report at the level relevant to the scientific claim.
animal2vec workshop · MPIABDay 3 · 09:30–10:15

Same coordinates; a different explanation

Interactive example requires JavaScript.

Reveal the colours by call group, site, and device. Which interpretation survives?

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Freeze a comparison policy before comparing methods

Same observationsStable row IDs and inclusion rules
Same representationCheckpoint, unit, preprocessing
Different analysisProjection / retrieval / clustering
Change one decision at a time and record what changed.

A method comparison is uninterpretable when the underlying observations also change unnoticed.

animal2vec workshop · MPIABDay 3 · 09:30–10:15

Ready for nearest-neighbour search?

Check Required answer
What is a row? One declared frame, event, window, or recording
Which space? One checkpoint and one extraction/preprocessing policy
Where is the sound? Source file + interval + stable row ID
What could confound similarity? At least two biological or technical metadata fields

Pick one query observation. Explain what would count as a useful neighbour.

animal2vec workshop · MPIABDay 3 · 11:00–11:45
Explore the structure.
Test the interpretation.
An unsupervised-learning toolbox without unnecessary mathematics.
Julian C. Schäfer-Zimmermann
Max Planck Institute of Animal Behavior
Department for the Ecology of Animal Societies
Communication and Collective Movement (CoCoMo) Group
animal2vec workshop · MPIABDay 3 · 11:00–11:45

Start with the question—not the plotting function

Your question First tool Its immediate output
What resembles this example? Nearest neighbours A ranked set of observations
What directions contain variation? PCA Linear coordinates and explained variance
What is locally neighbouring? UMAP A neighbourhood-oriented layout
Can examples be grouped? Clustering Candidate assignments
Which examples are unusual? Outlier scoring A ranking for inspection
animal2vec workshop · MPIABDay 3 · 11:00–11:45

Nearest neighbours: retrieve first, interpret second

Interactive example requires JavaScript.

In your real dataset: inspect the audio, remove self-matches, and avoid near-duplicate segments.

animal2vec workshop · MPIABDay 3 · 11:00–11:45

PCA chooses directions that retain variation

Interactive example requires JavaScript.

animal2vec workshop · MPIABDay 3 · 11:00–11:45

A PCA map is a summary, not the complete space

What to read

Which differences appear along each axis?
How much variance do these axes retain?

What not to conclude

Large variation is not automatically important biology.
Two dimensions need not preserve all useful distinctions.

scikit-learn PCA centers columns; it does not automatically standardize their scales.
animal2vec workshop · MPIABDay 3 · 11:00–11:45

UMAP begins from relationships between neighbours

Interactive example requires JavaScript.

The graph below is a schematic. No UMAP fit is computed in this slide.

animal2vec workshop · MPIABDay 3 · 11:00–11:45

A UMAP island is not a discovered call category

Useful

Explore local neighbourhoods.
Inspect examples.
Compare metadata and parameter choices.

Not established by the map

Biological category count.
Meaning of empty gaps.
Original-space density or global distance.

Check stability and the original feature space before treating a visual separation as evidence.
animal2vec workshop · MPIABDay 3 · 11:00–11:45

k-means: ask for K groups, then get K groups

Interactive example requires JavaScript.

Cluster assignments are fitted in the 12-D feature space. The PCA coordinates only display them.

animal2vec workshop · MPIABDay 3 · 11:00–11:45

Density-based grouping can leave points unassigned

Interactive example requires JavaScript.

Grey observations are marked noise. “Noise” here is an algorithmic status—not a biological annotation.

animal2vec workshop · MPIABDay 3 · 11:00–11:45

A candidate group needs an audit

Inspect Why
Typical examples and boundary cases A centroid alone can hide variation
Individual / session / device / site Technical or sampling factors may explain the group
Different parameters and resamples A fragile partition may not support a stable interpretation
Independent annotated observations Test the proposed biological interpretation

What observation would make you reject your current interpretation of a cluster?

animal2vec workshop · MPIABDay 3 · 11:00–11:45

Outliers are candidates for inspection

Interactive example requires JavaScript.

Rare vocalization? Corrupted file? Unusual background? Annotation mistake? Listen before naming it.

animal2vec workshop · MPIABDay 3 · 11:00–11:45

Turn exploration into an annotation decision

Retrieve / group / rankGenerate candidates
Listen and annotateTest the interpretation
Update the datasetAdd examples and document changes

Seek coverage

Inspect diverse and underrepresented regions.

Keep a comparison sample

Do not review only what the current model considers unusual.

animal2vec workshop · MPIABDay 3 · 11:00–11:45

One visible structure can support several stories

Interactive example requires JavaScript.

State a hypothesis with no labels. Then reveal site, device, and call group in that order.

animal2vec workshop · MPIABDay 3 · 11:00–11:45

Change a decision; check whether the finding survives

Decision to perturb Question to ask
Model / layer choice Does the proposed structure depend on one representation?
Aggregation / normalization / metric Does the observation definition change the conclusion?
Projection / clustering settings / seed Is the visible or assigned structure stable?
Held-out sessions / individuals / sites Does the interpretation survive a new context?
Keep exploratory findings separate from confirmatory evidence.
animal2vec workshop · MPIABDay 3 · 11:00–11:45

Next: the mystery-dataset exercise

Before metadata reveal

Describe the structure.
Inspect examples.
Write a testable hypothesis.

After metadata reveal

Compare alternative explanations.
Revise the hypothesis.
Identify the next validation step.

Your deliverable is an argued interpretation—not just a plot.
animal2vec workshop · MPIABDay 3 · 11:00–11:45

Use the map to ask better questions

A neighbour is a retrieval result.
A cluster is a candidate grouping.
An outlier is a review priority.

A biological conclusion requires evidence beyond the geometry.

Which analysis will you try first—and what would convince you that it is useful?