Skip to content

@remotion/studio: Use streaming audio waveform in timeline#6931

Merged
JonnyBurger merged 4 commits intomainfrom
studio/streaming-audio-waveform
Mar 30, 2026
Merged

@remotion/studio: Use streaming audio waveform in timeline#6931
JonnyBurger merged 4 commits intomainfrom
studio/streaming-audio-waveform

Conversation

@JonnyBurger
Copy link
Copy Markdown
Member

Summary

  • Replace getAudioData() (Web Audio API, decodes entire file into memory) with mediabunny streaming that computes peaks on-the-fly at 100 samples/sec
  • Replace individual <div> bars with canvas-based pixel rendering via putImageData
  • Waveform is centered vertically with clipping indicators at both edges

Test plan

  • Open Studio with a composition that has audio sequences
  • Verify waveform renders correctly in the timeline
  • Verify waveform appears for trimmed/offset audio (startFrom)
  • Verify volume envelope line still renders for dynamic volume
  • Verify large audio files don't cause memory spikes

🤖 Generated with Claude Code

Replace the old waveform implementation (which decoded entire audio files
into memory via Web Audio API) with a streaming approach using mediabunny.
Peaks are computed on-the-fly at 100 samples/sec and rendered to canvas
instead of individual DOM elements. Waveform is centered vertically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown
Contributor

vercel bot commented Mar 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
bugs Ready Ready Preview, Comment Mar 30, 2026 3:33pm
remotion Ready Ready Preview, Comment Mar 30, 2026 3:33pm

Request Review

@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog bot commented Mar 30, 2026

Reviewed PR #6931. Submitted feedback with 3 inline comments: AudioWaveformBar.tsx is now dead code and should be deleted, redundant width parameter in drawBars, and unbounded peaksCache. Overall the streaming approach via mediabunny is a solid improvement.

Task list (7/7 completed)
  • Read the PR diff and identify major areas of change
  • Review AudioWaveform.tsx changes
  • Review draw-peaks.ts (new file)
  • Review load-waveform-peaks.ts (new file)
  • Review parse-color.ts (new file)
  • Impact analysis: check for stale references to removed code
  • Submit review or report progress
Pullfrog  | [View workflow run](https://github.com/remotion-dev/remotion/actions/runs/23752613333/job/69198598033) | Triggered by [Pullfrog](https://pullfrog.com) | Using `Big Pickle` (free) | [𝕏](https://x.com/pullfrogai)

Copy link
Copy Markdown
Contributor

@pullfrog pullfrog bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean rewrite — streaming peaks via mediabunny + canvas putImageData is a solid improvement over decoding entire files into memory. Two actionable items below, one dead-code cleanup and one minor redundancy.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Big Pickle (free) | 𝕏


const TARGET_SAMPLE_RATE = 100;

const peaksCache = new Map<string, Float32Array>();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this module-level Map grows without bound across the Studio session lifetime. Not critical for a dev tool, but for long sessions with many audio files it could accumulate. A simple LRU or size cap would help, though this is fine to defer.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — deferring for now since it's a dev-only tool.

Pullfrog  | View workflow run | Triggered by Pullfrog | Using Big Pickle (free) | 𝕏

@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog bot commented Mar 30, 2026

Addressed review feedback: deleted unused AudioWaveformBar.tsx (confirmed no remaining imports) and acknowledged the deferred nit about the unbounded peaksCache map. Pushed 4393a5c.

Task list (3/3 completed)
  • Delete unused AudioWaveformBar.tsx file
  • Review if load-waveform-peaks.ts cache comment needs action (nit/defer)
  • Verify changes, commit, push, reply to comments
Pullfrog  | [View workflow run](https://github.com/remotion-dev/remotion/actions/runs/23752893068/job/69199617176) | Triggered by [Pullfrog](https://pullfrog.com) | Using `Big Pickle` (free) | [𝕏](https://x.com/pullfrogai)

@JonnyBurger JonnyBurger merged commit 25dd2c3 into main Mar 30, 2026
22 checks passed
@JonnyBurger JonnyBurger deleted the studio/streaming-audio-waveform branch March 30, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant