Skip to content

fix(audio): catch ort init panic so speaker init can't crash tokio worker#3290

Merged
louis030195 merged 1 commit into
mainfrom
fix/ort-init-panic-no-worker-crash
May 8, 2026
Merged

fix(audio): catch ort init panic so speaker init can't crash tokio worker#3290
louis030195 merged 1 commit into
mainfrom
fix/ort-init-panic-no-worker-crash

Conversation

@louis030195

Copy link
Copy Markdown
Collaborator

Problem

ort 2.0.0-rc.10 panics from inside its global OnceLock when GetApi(ORT_API_VERSION) returns NULL — the failing line is expect(\"Failed to initialize ORT API\") at ort-2.0.0-rc.10/src/lib.rs:188. On Windows, that panic bubbled out of Session::builder() inside screenpipe_audio::speaker::create_session, unwound the tokio worker, and showed up in Sentry as SCREENPIPE-APP-9X / 9Y — 42 events from 42 distinct users on screenpipe-app@2.4.160 in ~13 hours, all on Windows 10.0.26100.

Stacktrace from Sentry:

ort::util::OnceLock<T>::try_init_inner
ort::session::Session::builder
screenpipe_audio::speaker::create_session
screenpipe_audio::speaker::models::get_or_download_model
…
panic on thread 'tokio-rt-worker' at lib.rs:188:57: Failed to initialize ORT API

Fix

Wrap the body of create_session in std::panic::catch_unwind and convert the panic payload (&'static str or String) into an anyhow::Error. The retry/recovery loop in speaker/models.rs already handles the Err branch — failed init now degrades to a clean error instead of crashing the runtime.

The change is in crates/screenpipe-audio/src/speaker/mod.rs only. No dependency changes; Cargo.lock untouched. The default panic = \"unwind\" profile is preserved (workspace Cargo.toml has no panic = \"abort\"), so catch_unwind will actually catch.

Verification

cargo test -p screenpipe-audio --lib speaker:: — 15 tests pass, 6 new:

test speaker::tests::catch_panic_into_error_passes_through_ok ... ok
test speaker::tests::catch_panic_into_error_passes_through_err ... ok
test speaker::tests::catch_panic_into_error_catches_str_panic ... ok
test speaker::tests::catch_panic_into_error_catches_string_panic ... ok
test speaker::tests::catch_panic_into_error_simulates_ort_api_init_panic ... ok
test speaker::tests::create_session_returns_err_for_missing_path ... ok

test result: ok. 15 passed; 0 failed

catch_panic_into_error_simulates_ort_api_init_panic panics with the literal string ort raises (\"Failed to initialize ORT API\") and asserts the wrapper turns it into an anyhow::Error containing \"ort session init panicked\" and the original payload.

cargo check -p screenpipe-audio — clean (no new warnings).

Confidence: 7/10

The defensive conversion is unambiguous and tested against the exact panic message. What's harder to prove from this side is the root environmental cause on Windows (likely VC++/ORT DLL version skew vs the bundled binary) — but that's a separate fix; this PR just stops the panic from killing the tokio worker so users can continue running.


auto-generated by issue-solver pipe

louis030195 added a commit that referenced this pull request May 8, 2026
Two `assert!` / `sqlx::query` blocks in worker_integration.rs were
unformatted, breaking `cargo fmt --check` in CI. Pre-existing on main
since 3f44640 (the redact destructive-flag refactor). Unblocks
Clippy & Format on every open PR — most visibly #3290 (catch ort init
panic), which has been held up entirely by this fmt drift.

No semantic change; pure rustfmt output.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@louis030195 louis030195 force-pushed the fix/ort-init-panic-no-worker-crash branch from a974f7b to b5517e7 Compare May 8, 2026 21:04
ort 2.0.0-rc.10 panics from inside its global OnceLock at lib.rs:188
when GetApi(ORT_API_VERSION) returns NULL — the path screenpipe-audio
exercises is `expect("Failed to initialize ORT API")`. On Windows that
panic blew through Session::builder(), unwound the tokio-rt-worker, and
showed up in Sentry as SCREENPIPE-APP-9X / 9Y (42 events from 42 distinct
users on screenpipe-app@2.4.160 in ~13 hours).

Wrap create_session in catch_unwind and convert any panic payload
(&'static str or String) into an anyhow::Error. Callers already handle
the Err branch (models.rs has retry/backoff), so failed init now returns
a clean error instead of crashing the worker.
@louis030195 louis030195 force-pushed the fix/ort-init-panic-no-worker-crash branch from b5517e7 to 72591cd Compare May 8, 2026 21:21
@github-actions

github-actions Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor

Diarization eval results

Source: crates/screenpipe-audio-eval/evals/ · VoxConverse dev (CC-BY-4.0) + composed workday templates

fixture DER VAD FA VAD FN boundary err (s) continuity predicted / true spk
interrupted_meeting 0.917 0 0.916 20.435 0 2 / 5
long_silence_day 0.988 0 0.971 15.251 0 2 / 10
abjxc 0.11 0.068 0.002 0.066 n/a 2 / 1
bxpwa 0.999 0.043 0.994 4.492 0 2 / 5
dhorc 0.798 0.136 0.736 8.425 0.111 2 / 4

DER, VAD FA, VAD FN, boundary err: lower is better. Continuity: higher is better, 1.0 = same hyp cluster across all silence gaps. Composed workday rows (interrupted_meeting, long_silence_day) are the headline — they exercise screenpipe-shaped data. Raw VoxConverse rows score broadcast-quality stems for comparison. See crates/screenpipe-audio-eval/evals/README.md for methodology.

Transcription quality

Source: LibriSpeech test-clean (CC-BY-4.0) · whisper-tiny · normalized lowercased word-level Levenshtein

model utterances WER CER throughput (samples/s)
tiny 50 0.085 0.033 70004

WER + CER on read-aloud speech. whisper-tiny is the smallest model and gives a useful regression watch even though larger models score better. See README for normalization rules.

@louis030195 louis030195 merged commit 14a441f into main May 8, 2026
16 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant