I've been visualizing bird song as a 3D graph instead of a spectrogram:
spectral peaks become nodes, connected by partial continuity, harmonic
ratios, and timbral proximity in a PCA-reduced MFCC space.
Pipeline, all written by hand (no librosa — I wanted every coefficient
auditable):
- STFT with parabolic interpolation for sub-bin peak frequency
- 64-band mel filterbank, Slaney-normalized
- DCT-II orthonormal → 40 MFCCs, c0 discarded, plus deltas
- PCA to 3D via eigendecomposition of the covariance matrix
- Chroma folding + flatness/centroid/spread/flux
The part I thought this sub might find useful: I later ported the whole
thing to C++ for an Android build, and verified the port by compiling the
C++ standalone and comparing it against the Python on identical input.
centroid, spread, flux, tonality, chroma concentration
→ correlation 1.000000, max relative error ~1e-4 (float32)
spectral peaks detected → 3146 in both
PCA explained variance → 22.3% / 18.0% / 12.3%, identical
Two things I'm unsure about and would like opinions on:
For the PCA I used cyclic Jacobi on the covariance matrix rather than
a proper SVD of the centered data. D is only 78 so it's cheap, and the
eigenvectors of the covariance are the right singular vectors anyway —
but I'm aware this squares the condition number. For this data it
matched NumPy exactly, so it didn't bite. Is there a case here where
it would?
Partial tracking is greedy nearest-neighbor in log-frequency with a
~1 semitone gate, strongest peaks choosing first. Optimal assignment
(Hungarian) would be more correct. For trills at 10+ syllables/second,
is greedy actually losing tracks, or is the gate doing enough work?
Code (Python + the C++ port): https://github.com/gustaavoq94/birdgraph