`@remotion/elevenlabs`: Add package for converting ElevenLabs output to captions by tiwariaayu · Pull Request #6930 · remotion-dev/remotion

tiwariaayu · 2026-03-30T14:27:42Z

Summary

Adds @remotion/elevenlabs package for converting ElevenLabs Speech to Text API output to Caption objects
elevenLabsTranscriptToCaptions() — converts word-level transcript output to captions, analogous to @remotion/openai-whisper
API must be called with timestamps_granularity: "word" — function throws a helpful error with docs link if the response is missing word timing
Docs include a full fetch() example, API reference with argument/return type docs

Test plan

Full snapshot test with real ElevenLabs transcript (324 words, "Gimme Gimme Gimme")
Invalid input throws error with helpful message
Empty words array returns empty captions
All 3 tests pass

🤖 Generated with Claude Code

pullfrog · 2026-03-30T14:27:48Z

Reviewed PR #6930. Submitted 4 inline comments covering: test runner mismatch (vitest should be bun:test), missing array-length validation in the conversion function, broken import sort order in api.tsx, and a misleading comment referencing OpenAI Whisper.

Task list (7/7 completed)

Read the full diff to understand the scope of changes
Review core package code (types, implementation, exports)
Review tests
Review package configuration (package.json, tsconfig, eslint)
Review documentation changes
Review integration with existing packages (CLI, create-video, docs)
Self-critique and submit review

^{｜ [View workflow run](https://github.com/remotion-dev/remotion/actions/runs/23750015059/job/69189094498) ｜ Triggered by [Pullfrog](https://pullfrog.com) ｜ Using `Big Pickle` (free) ｜ [𝕏](https://x.com/pullfrogai)}

vercel · 2026-03-30T14:28:00Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
bugs	Ready	Preview, Comment	Mar 31, 2026 0:45am
remotion	Ready	Preview, Comment	Mar 31, 2026 0:45am

pullfrog

The package structure, docs integration, and conversion logic are solid overall. A few issues need attention — primarily the test runner mismatch (vitest vs bun:test) and a missing array-length validation that could produce NaN timestamps silently.

^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using Big Pickle (free) ｜ 𝕏}

packages/elevenlabs/src/test/elevenlabs-api-to-captions.test.ts

packages/elevenlabs/src/elevenlabs-api-to-captions.ts

packages/docs/components/TableOfContents/api.tsx

packages/elevenlabs/src/elevenlabs-api-to-captions.ts

The original implementation only handled the TTS alignment format. This adds support for the ElevenLabs Speech to Text API output, which is the actual analogue to `@remotion/openai-whisper`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This was based on a misunderstanding of the issue — the request was for Speech to Text support (like @remotion/openai-whisper), not Text to Speech alignment. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The inner slider circle used box-content which made its total rendered size 24x24 (20px content + 4px borders), filling the entire track with no visible gap. Removing it uses the default border-box from Tailwind's base reset, making the circle 20x20 total with symmetric 4px gaps. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

[elevenlabs]: Add package for converting ElevenLabs output to captions

696e6b3

vercel bot deployed to Preview – bugs March 30, 2026 14:28 View deployment

vercel bot had a problem deploying to Preview – remotion March 30, 2026 14:29 Failure

pullfrog bot reviewed Mar 30, 2026

View reviewed changes

fix

35d680f

vercel bot deployed to Preview – bugs March 30, 2026 14:39 View deployment

vercel bot had a problem deploying to Preview – remotion March 30, 2026 14:41 Failure

Fix according to pullfrog

26b77d5

vercel bot deployed to Preview – bugs March 30, 2026 15:56 View deployment

vercel bot had a problem deploying to Preview – remotion March 30, 2026 15:57 Failure

JonnyBurger changed the title ~~[elevenlabs]: Add package for converting ElevenLabs output to captions~~ @remotion/elevenlabs: Add package for converting ElevenLabs output to captions Mar 31, 2026

vercel bot deployed to Preview – bugs March 31, 2026 11:19 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:19 Failure

Remove elevenLabsApiToCaptions() TTS alignment code

87ffb17

This was based on a misunderstanding of the issue — the request was for Speech to Text support (like @remotion/openai-whisper), not Text to Speech alignment. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:25 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:26 Failure

Assert captions snapshot explicitly, bump AvailableFrom to 4.0.442

6df80be

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:28 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:29 Failure

Fix redundant docs description, add fetch() example

ee6ddfc

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:34 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:35 Failure

Improve docs and error message for elevenLabsTranscriptToCaptions()

03d8d69

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:36 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:36 Failure

Bump AvailableFrom to 4.0.443

3d67897

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:43 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:43 Failure

Mention @remotion/elevenlabs alongside @remotion/openai-whisper in docs

f9c27d4

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:44 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:44 Failure

Include JSON files in tsconfig to fix tsgo build

60af4c9

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:48 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:49 Failure

vercel bot deployed to Preview – bugs March 31, 2026 11:51 View deployment

List all caption sources in @remotion/captions docs

21cdd32

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:53 View deployment

vercel bot had a problem deploying to Preview – remotion March 31, 2026 11:55 Failure

Fix broken openai-whisper links in elevenlabs docs

f80a61f

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:57 View deployment

Revert unrelated Switch.tsx change

4fceff7

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 11:57 View deployment

JonnyBurger enabled auto-merge (squash) March 31, 2026 11:58

vercel bot deployed to Preview – remotion March 31, 2026 11:59 View deployment

Fix @remotion/elevenlabs version to match monorepo (4.0.442)

52a4cea

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vercel bot deployed to Preview – bugs March 31, 2026 12:44 View deployment

vercel bot deployed to Preview – remotion March 31, 2026 12:45 View deployment

JonnyBurger merged commit 700a574 into remotion-dev:main Mar 31, 2026
16 of 17 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

`@remotion/elevenlabs`: Add package for converting ElevenLabs output to captions#6930

`@remotion/elevenlabs`: Add package for converting ElevenLabs output to captions#6930
JonnyBurger merged 16 commits intoremotion-dev:mainfrom
tiwariaayu:#6927

tiwariaayu commented Mar 30, 2026 •

edited by JonnyBurger

Loading

Uh oh!

pullfrog bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

vercel bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

pullfrog bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

tiwariaayu commented Mar 30, 2026 • edited by JonnyBurger Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

pullfrog bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pullfrog bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tiwariaayu commented Mar 30, 2026 •

edited by JonnyBurger

Loading

pullfrog bot commented Mar 30, 2026 •

edited

Loading

vercel bot commented Mar 30, 2026 •

edited

Loading