animal2vec workshop · MPIABDay 3 · 09:30–10:15

What does one point mean?

How to analyze embeddings.

Julian C. Schäfer-Zimmermann

Max Planck Institute of Animal Behavior
Department for the Ecology of Animal Societies
Communication and Collective Movement (CoCoMo) Group

animal2vec workshop · MPIAB06 · Embeddings to an analysis dataset

PCA: keep the direction with most variation

Input: one vector per observation72 toy vectors · principal component analysis

→

Output: fewer coordinatesHere: 2 features → 1 retained component

A cloud and a projection axis

Teaching example

Synthetic numbers; projection and variance are calculated.

Choose a projection

Axis angle 0°

Reveal example labels

Variance retained—

Variance discarded—

High variance is not the same as useful biological information. PCA does not use the labels.

Suggested demonstration (3–4 minutes): rotate the candidate axis, press Find PC1, then Project points. Run Play for the same sequence. Reveal labels: these toy groups differ mainly along the narrow direction, so retaining most variance need not retain their separation.

PCA centers observations and finds orthogonal directions of decreasing variance. The first direction here is computed from the 2 x 2 covariance matrix; its eigenvalue determines the displayed retained-variance fraction. Lines show orthogonal projection. Grey dots retain the unprojected locations as a visual reference. In high dimensions the same idea retains several components, often two for plotting or more for analysis.

This numerical dataset is two-dimensional to make the operation visible; it is not a projection of animal2vec output. Feature scaling is fixed. Standard PCA centers but does not automatically standardize each feature. Fit preprocessing on training data when assessing held-out prediction; a descriptive all-data map should be described as such.

Source: official PCA documentation. This is new teaching material for presentation 06, not a PCA experiment reported in the animal2vec paper.

animal2vec workshop · MPIAB06 · Embeddings to an analysis dataset

UMAP: turn neighborhoods into a map

Input: 90 window vectorsNew toy dataset · 8 features per 200 ms window

→

Output: a 2D neighborhood mapUniform Manifold Approximation and Projection

Same observations, new display coordinates

Teaching example

Follow the construction

Reveal example labels

Stage—

Graph mismatch / pair—

Computed, small full-batch teaching solver—not umap-learn. No animal2vec outputs are used.

Suggested demonstration (4 minutes): start with the unlabelled vectors. Select Neighbors and click two different points. Then show Graph and Layout. Play steps through all stages, then replays saved numerical optimizer iterations. Reveal labels after the map is constructed; labels were never input to the calculation.

The input is a new, fixed synthetic table of 90 eight-dimensional window vectors, not 90 adjacent frames from the previous waveform example. The initial scatter is a calculated two-component PCA display. Neighbors are selected in all eight features, not from distances in that display. Each point has a local distance scale; directed memberships are combined using fuzzy union. The plot suppresses edges of weight <= 0.15 solely for readability. The optimization uses all graph weights.

Implementation boundary: a transparent small-scale solver computes exact neighbors, locally scaled memberships, and the full undirected fuzzy-set cross-entropy. Low-dimensional memberships use 1/(1+a*d^(2b)), with a and b fitted from min_dist and spread. L-BFGS-B optimizes all pairs. This is a numerical teaching model of the official objective, not umap-learn (which uses stochastic updates and negative sampling). Its snapshots are actual stored iterations, not invented motion. It is not a production implementation or a performance comparison.

Source: official explanation of UMAP. Neighborhood weights are not probabilities of call labels. Map axes have no direct acoustic meaning. A 2D map inevitably distorts some relationships.

animal2vec workshop · MPIAB06 · Embeddings to an analysis dataset

UMAP: settings change what you see

Hold the observations fixed90 windows · same eight input features

→

Change one decision at a timeNeighborhood size · packing · initialization

One dataset, several layout choices

Teaching example

Compare settings

Neighborhood size

min_dist: packing control

Initialization seed

10-neighbor overlap—

Show example labels

Smaller min_dist encourages tighter packing; it does not prove cleaner classes. Small full-batch teaching solver.

Suggested demonstration (4 minutes): hold the seed and min_dist fixed while changing neighborhood size. Then hold the neighborhood size fixed while changing min_dist. Finally change the seed. The selected row stays selected. Axes use common display limits across the twelve computed layouts. The smooth transitions between final layouts are illustrative interpolation, not a training trajectory.

Neighborhood size changes the local graph scale. min_dist changes the smooth low-dimensional membership curve and encourages different packing; it is not a hard lower bound obeyed by every pair. Seed changes the random initial layout. Larger neighborhoods incorporate broader structure but do not guarantee faithful global distances. Absolute axis directions, island separation and visible density should not be read as direct biological measurements.

These results use the same full-batch teaching solver described on the previous slide. It is not umap-learn and its parameter sensitivities should not be taken as production benchmarks. The readout is the mean fraction of ten original-space neighbors recovered among ten 2D neighbors, excluding self. It is a descriptive preservation check, not classification accuracy or proof of a correct map. Neither parameter changes nor label recoloring modify the source vectors.

Sources: official UMAP parameters and UMAP clustering cautions. UMAP is a dimension-reduction method, not a class-discovery oracle.

animal2vec workshop · MPIAB06 · Embeddings to an analysis dataset

Clustering: assign, average, repeat

Input: unlabelled observations78 synthetic vectors · two features

→

Output: k group assignmentsGroup numbers are not biological class names

K-means on a visible 2D toy space

Teaching example

Choose the assumptions

Number of groups, k

Starting centers

Iteration—

Within-group sum of squares—

Reveal example labels

K-means assigns every vector. Smaller inertia alone does not tell you the correct number of call types.

Suggested demonstration (4 minutes): step through Initialize, Assign and Move centers. Only the diamonds move; observations do not. Run to convergence. Change k from three to two and four, then change the starting centers. Reveal reference labels only after discussing the fitted groups.

K-means alternates nearest-center assignment and arithmetic means in a Euclidean feature space. Its objective is the sum of squared distances to assigned centers. Each displayed state is numerically computed; transitions of centers only interpolate between exact states. The implementation retains an empty center if needed. Examples were checked against scikit-learn using identical initial centers.

The vectors shown are genuinely two-dimensional toy inputs, not a UMAP projection being silently used for clustering. On real data choose and record the analysis space: original embeddings or a justified reduction, usually more than two components. k is specified, results can depend on initialization, and compact approximately spherical groups suit the objective better than curved or strongly unequal-density groups. A cluster may reflect device or site rather than call type.

Source: official clustering guide, K-means. Lower inertia with larger k is not evidence by itself of a better biological partition.

animal2vec workshop · MPIAB06 · Embeddings to an analysis dataset

Density clustering: grow connected regions

Input: points and a distance86 synthetic 2D observations

→

Output: groups and unassigned pointsUse local density, not a requested number of groups

DBSCAN: look around one point

Teaching example

Set the neighborhood rule

Radius ε (feature units)

Minimum points, including self

Points inside radius—

Selected point—

“Unassigned” is an algorithmic status. It does not mean acoustic background/noise.

Suggested demonstration (4 minutes): use radius 0.70 and minimum five points. Play the expansion, then show the completed result and click a core point, border point and unassigned point. Reduce the radius to 0.40 or 0.22 and run again. The curved groups show a different assumption from K-means.

Core points have at least min_samples observations within epsilon, counting themselves. Clusters expand through connected core points and attach adjacent border points. A non-core point cannot extend the expansion on its own. Early unassigned visits can later become border points; the animation shows that state evolution. The final labels and core status were checked against scikit-learn DBSCAN.

This example uses the full 2D toy feature space, so the drawn radius is a genuine Euclidean neighborhood. In a projection of high-dimensional embeddings, a drawn 2D circle would generally not be that neighborhood. Density depends on sampling; many adjacent frames can create artificial abundance. DBSCAN uses one global radius, so groups of very different densities can be challenging.

HDBSCAN is a related extension that examines a hierarchy of density scales and selects persistent groups rather than requiring one epsilon. This slide implements DBSCAN, not HDBSCAN. Source: DBSCAN guide and HDBSCAN guide.

animal2vec workshop · MPIAB06 · Embeddings to an analysis dataset

Retrieval: find examples near a query

Input: one query and a reference setSame 90 windows · eight features · query excluded

→

Choose a point, retrieve its neighbors

Teaching example

Choose a query and distance

Exclude the query’s recording

or click a point

Rank · distance

Click a ranked result to inspect its source above.

Similarity is not a class prediction. Synthetic records; no audio is loaded.

Suggested demonstration (3 minutes): select call_A and press Retrieve. Five connections and ranked IDs appear in order. Click a result to inspect its source record, then switch Euclidean/Cosine or exclude the query recording. Clicking any scatter point makes it the new query.

Distances use the original eight-dimensional synthetic vectors, not the two PCA coordinates shown for orientation. Euclidean distance is sensitive to vector magnitude; cosine distance compares direction. Neither is automatically the right biological notion of similarity. This exercise excludes self and optionally all observations from the same fictitious recording. A practical dataset may also need exclusion of overlapping windows or the same event.

This is unsupervised retrieval, not a k-nearest-neighbor classifier: there is no majority vote. Known toy labels only color the plot and help interpret matches. Source records link each row to a file and interval, echoing the identity/provenance contract of presentation 06. These filenames are deliberately synthetic and have no attached recordings, so no audio playback is offered.

Source: official nearest-neighbor guide. Return to the real audio and metadata before accepting a match. For very large archives, approximate search trades retrieval exactness for speed; this demonstration uses exact distances.

animal2vec workshop · MPIAB06 · Embeddings to an analysis dataset

Outliers: unusual relative to their neighbors

Input: observations and local neighbors87 synthetic 2D vectors · dense and sparse groups

→

Output: a review priorityHigh score → inspect audio and metadata

A sparse group is not necessarily an outlier

Teaching example

Inspect local density

Neighbors used by LOF

Selected LOF score—

Own local density—

Mean neighbor density—

Show top three for review

A high LOF score is not a probability of novelty, an error diagnosis, or a reason to delete the observation.

Suggested demonstration (4 minutes): inspect Dense and Sparse. Both can score near one because their density resembles that of nearby points. Inspect Isolated, then vary the neighborhood size. Reveal the top three to make a review shortlist. The rare small group can be treated differently when the neighborhood extends beyond the group.

LOF estimates local reachability density using k-nearest neighbors, excluding self, and compares a point with its neighbors. The displayed score equals mean neighbor reachability density divided by own reachability density. It is the positive LOF, equivalent to negative of scikit-learn negative_outlier_factor_. Around one means comparable local densities; higher means relatively less dense. Values were checked against scikit-learn.

The dashed circle indicates the selected point’s kth-neighbor distance, not a decision boundary. Top three is a review budget, not a calibrated threshold or assumed contamination rate. Outlier status depends on sampling, representation, distance, preprocessing and neighborhood size. A candidate could be an artifact, a rare call, a new context or a systematic domain shift. Do not remove it without review.

Source: official LocalOutlierFactor documentation. These are numerical toy vectors, not actual animal2vec embeddings. For real high-dimensional data compute scores in the declared analysis space, not from visual isolation in a UMAP plot.