Skip to main content

Audio visualization

Remotion has APIs for visualizing audio, for example for creating audiograms or music visualizers.

The @remotion/media-utils package provides helper functions for reading and processing audio. Using the getAudioData() API you can read audio, and using the useAudioData() helper hook you can load this audio data directly into your component.

Using the visualizeAudio() API, you can get an audio spectrum for the current frame, with the numberOfSamples parameter being an option to control the amount of detail you need.

Refer to the documentation of the above mentioned functions to learn more.

tsx
import {useAudioData, visualizeAudio} from '@remotion/media-utils';
import {Audio, staticFile, useCurrentFrame, useVideoConfig} from 'remotion';
 
const music = staticFile('music.mp3');
 
export const MyComponent: React.FC = () => {
  const frame = useCurrentFrame();
  const {width, height, fps} = useVideoConfig();
  const audioData = useAudioData(music);
 
  if (!audioData) {
    return null;
  }
 
  const visualization = visualizeAudio({
    fps,
    frame,
    audioData,
    numberOfSamples: 16,
  }); // [0.22, 0.1, 0.01, 0.01, 0.01, 0.02, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
 
  // Render a bar chart for each frequency, the higher the amplitude,
  // the longer the bar
  return (
    <div>
      <Audio src={music} />
      {visualization.map((v) => {
        return (
          <div style={{width: 1000 * v, height: 15, backgroundColor: 'blue'}} />
        );
      })}
    </div>
  );
};
tsx
import {useAudioData, visualizeAudio} from '@remotion/media-utils';
import {Audio, staticFile, useCurrentFrame, useVideoConfig} from 'remotion';
 
const music = staticFile('music.mp3');
 
export const MyComponent: React.FC = () => {
  const frame = useCurrentFrame();
  const {width, height, fps} = useVideoConfig();
  const audioData = useAudioData(music);
 
  if (!audioData) {
    return null;
  }
 
  const visualization = visualizeAudio({
    fps,
    frame,
    audioData,
    numberOfSamples: 16,
  }); // [0.22, 0.1, 0.01, 0.01, 0.01, 0.02, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
 
  // Render a bar chart for each frequency, the higher the amplitude,
  // the longer the bar
  return (
    <div>
      <Audio src={music} />
      {visualization.map((v) => {
        return (
          <div style={{width: 1000 * v, height: 15, backgroundColor: 'blue'}} />
        );
      })}
    </div>
  );
};

Working with large files​

useAudioData() loads the entire audio file into memory.
This is fine for small files, but for large files, it can be slow and consume a lot of memory.

Use useWindowedAudioData() to only load a portion of the audio around the current frame.
The tradeoff is that this API only works with .wav files.

See also​