Five techniques and where each fits
- FFT (frequency analysis)
- Short-Time Fourier Transform splits audio into energy per frequency band. Bass (< 250Hz), mid (< 4kHz), high (< 16kHz) can be driven separately — "color shifts on bass," "sparkle on hi-hats." In the browser, AnalyserNode.getByteFrequencyData() exposes it immediately.
- Loudness (RMS / peak)
- Instantaneous volume. RMS tracks perceived loudness, peak catches transients. Mapping volume to brightness or scale is the simplest and most common reactive layer — the default baseline for live VJing.
- BPM detection (beat tracking)
- Estimates song tempo. Dance 120–140, hip hop ~90, drum-and-bass ~170. With BPM in hand, LFOs sync to the beat and per-beat cuts become possible. Accuracy drops in crowd-noisy clubs.
- Onset detection
- Spots sudden rises — kicks, snares, claps. Best for strobes, inversions, hard cuts. Typical algorithms watch spectral flux or energy deltas.
- LFO (Low-Frequency Oscillator)
- Runs independently of audio — sine, triangle, or saw at 0.1Hz to a few Hz. The only way to get "alive but not audio-reactive" motion. Blended with reactive layers, produces the natural-drift-plus-peaks feel that reads best over long sets.
Implementation environments
Professional rigs live mostly in Max/MSP (Cycling '74) and TouchDesigner (Derivative). Both ship FFT, onset detection, and BPM modules natively; wiring them to a visual node is enough to build reactive behavior. Max/MSP sits deeply inside Ableton Live as Max for Live — the native tool of artists like Robert Henke / Monolake.
In the browser, start from AnalyserNode in the Web Audio API. FFT, RMS, and waveform data are all available from plain JavaScript, feeding naturally into Three.js or Hydra. Butterchurn (jberg, a WebGL port of MilkDrop) is close to a complete audio-reactive visualizer out of the box, driven by preset files alone.