feat(ai): upgrade Chirpy on-device AI with proper APIs, download UX, and streaming by jamesarich · Pull Request #5579 · meshtastic/Meshtastic-Android

jamesarich · 2026-05-22T23:37:29Z

Motivation

Chirpy's on-device AI (Gemini Nano) was using deprecated Firebase AI APIs that stopped working in recent releases. The implementation lacked download progress feedback, had choppy streaming, and the model would sometimes identify as "Gemma" instead of Chirpy.

Approach

Full audit against the Firebase AI Logic SDK at our release tag (firebase-ai:17.12.0 + firebase-ai-ondevice:16.0.0-beta02), then a ground-up rework of the assistant and its UX:

API migration and reliability:

Migrated from deprecated static FirebaseAIOnDevice.checkStatus()/.download() to instance-based model.onDeviceExtension?.checkStatus()/.download()/.warmUp()
Added OnDeviceConfig with InferenceMode.ONLY_ON_DEVICE, OnDeviceModelOption.STABLE, tuned temperature/topK/maxTokens
Exponential backoff retry for transient model failures
Response cleanup (strip markdown artifacts from on-device output)

Download state UX:

New ModelReadiness sealed interface (Checking, Downloading(progress), Available, Unavailable)
Real determinate progress from DownloadStatus.DownloadInProgress
Sheet UI branches on readiness state with appropriate messaging

Expressive M3 FAB (ChirpyFab.kt):

Animated states: pulse when available, spin when checking, progress arc when downloading
Replaces inline FABs in both docs screens

Streaming and intro fixes:

80ms throttle on partial token emissions prevents Markdown re-render jank
Lightweight fast path for short prompts (skips doc bundle loading)
Identity reinforcement so model always identifies as Chirpy
Tightened intro prompt to be brief and punchy

Test fix:

Moved StoreForwardPacketHandlerImplTest from commonTest to jvmTest to avoid a Wire 6 @JvmField + KMP androidHostTest binary incompatibility (tests pass on jvmTest; the metadata-vs-field mismatch only affects androidHostTest)
Migrated test helpers from Type.ADAPTER.encode() to instance encode()

Key files

androidApp/src/google/.../GeminiNanoDocAssistant.kt -- core AI implementation
feature/docs/.../ui/ChirpyFab.kt -- new animated FAB
feature/docs/.../navigation/DocsNavigation.kt -- state wiring, intro logic
feature/docs/.../model/DocModels.kt -- ModelReadiness sealed interface
core/data/src/jvmTest/.../StoreForwardPacketHandlerImplTest.kt -- moved from commonTest

Notes for reviewers

App Check integration is left as a TODO breadcrumb in GoogleAiModule.kt -- needs research on impact to fdroid/sideload channels before enabling
Hybrid cloud fallback was investigated but Gemini on-device does NOT support tools/grounding, so we stay fully on-device
The Wire @JvmField issue is a known KMP interop problem; moving to jvmTest is the cleanest fix since these tests have no Android dependencies

Replace the binary isSupported polling loop with reactive modelStatus StateFlow collection. ChirpyAssistantSheet now shows: - Loading spinner during Checking state - Download progress bar during Downloading state - Normal chat UI when Available - Hidden when Unavailable Thread modelReadiness through ChirpyUiState, DocsBrowserScreen, and DocsPageRouteScreen to the sheet composable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Add ModelReadiness sealed interface to DocModels.kt with states: Checking, Downloading (with progress), Available, Unavailable - Add modelStatus: StateFlow<ModelReadiness> to AIDocAssistant interface - Implement modelStatus in KeywordFallbackAssistant (always Unavailable) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ackoff - Migrate from deprecated FirebaseAIOnDevice static API to instance-based model.onDeviceExtension for checkStatus/download/warmUp - Add ModelReadiness sealed interface (Checking/Downloading/Available/Unavailable) with determinate download progress tracking - Surface download state in UI with progress bar and percentage - Add warmUp() call after model becomes available - Add exponential backoff retry (up to 3 attempts) for BUSY/BATTERY errors - Add defensive response cleanup (strip backtick fences from on-device output) - Configure OnDeviceConfig with STABLE model option, temperature 0.7, topK 40 - Fix system instruction to honestly describe knowledge sources - Log usageMetadata (prompt/response token counts) for debugging - Add App Check TODO breadcrumb for future hybrid/cloud adoption - Extract Chirpy sheet states into separate composables for detekt compliance Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Add ChirpyFab composable with M3 Expressive motion patterns: - Available: gentle breathing pulse (alive/ready feel) - Checking: spinning indeterminate ring with reduced opacity icon - Downloading: determinate circular progress arc with spring animation - Unavailable: hidden (no FAB rendered) - Show FAB during Checking/Downloading states (not just Available) - Fix first-request failure: AutoIntroduceChirpy now waits for ModelReadiness.Available before firing intro inference - Fix choppy streaming: throttle partial emissions to ≤1 per 80ms to reduce Markdown re-render pressure on the UI thread - Replace hardcoded strings with proper string resources - Remove unused imports from screen composables Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Add lightweight fast path in answerStream: prompts <100 chars with no page context and empty history skip the expensive bundle load + page ranking pipeline entirely, sending just the question with system instruction - Tighten intro prompt to 'Say hi in 1-2 sentences' for concise output - Intro now generates in <1s vs ~3-5s previously Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Wire 6 generates @JvmField on companion ADAPTER properties. In KMP, androidHostTest compiles commonMain against common metadata (emitting a getter call) but at runtime the JVM class has a field (no getter), causing NoSuchMethodError. Fix: - Move StoreForwardPacketHandlerImplTest to jvmTest source set (pure Kotlin tests with no Android deps, avoids the metadata/JVM mismatch) - Migrate test helpers to use instance encode() (Wire Message method) instead of Companion.ADAPTER.encode() Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-05-22T23:37:43Z

🖼️ Preview staleness check — advisory

This PR modifies UI composables but does not update any *Previews.kt files.

Previews power screenshot tests and in-app docs screenshots. Keeping them current ensures visual regression coverage stays accurate.

Changed UI files:

feature/docs/src/commonMain/kotlin/org/meshtastic/feature/docs/ui/ChirpyAssistantSheet.kt
feature/docs/src/commonMain/kotlin/org/meshtastic/feature/docs/ui/ChirpyFab.kt
feature/docs/src/commonMain/kotlin/org/meshtastic/feature/docs/ui/DocsBrowserScreen.kt
feature/docs/src/commonMain/kotlin/org/meshtastic/feature/docs/ui/DocsPageRouteScreen.kt

What to check:

Pattern	Preview file convention
`feature/{name}/…/ui/` or `component/`	`feature/{name}/…/*Previews.kt`
`core/ui/…/`	`core/ui/…/` (previews colocated)

Adding previews checklist:

Create or update a *Previews.kt file in the same module with @PreviewLightDark
Add @Suppress("PreviewPublic") if the preview is consumed by screenshot-tests
Add corresponding @PreviewTest function in screenshot-tests/src/screenshotTest/
Run ./gradlew :screenshot-tests:updateDebugScreenshotTest to generate reference images

If this PR does not require preview updates (e.g., logic-only change, non-visual refactor), add the skip-preview-check label to dismiss.

github-actions · 2026-05-22T23:37:43Z

📄 Docs staleness check — advisory

This PR modifies user-facing UI source files but does not update any page under docs/en/user/ or docs/en/developer/.

⚠️ Doc changes propagate to 3 consumers: in-app docs browser, Jekyll site (GitHub Pages), and meshtastic.org (Docusaurus sync). Updating a page in docs/en/ automatically flows to all three.

Changed source files:

feature/docs/src/commonMain/kotlin/org/meshtastic/feature/docs/ui/ChirpyAssistantSheet.kt
feature/docs/src/commonMain/kotlin/org/meshtastic/feature/docs/ui/ChirpyFab.kt
feature/docs/src/commonMain/kotlin/org/meshtastic/feature/docs/ui/DocsBrowserScreen.kt
feature/docs/src/commonMain/kotlin/org/meshtastic/feature/docs/ui/DocsPageRouteScreen.kt

What to check:

Changed area	Likely doc page
`feature/messaging/`	`docs/en/user/messages-and-channels.md`
`feature/node/`	`docs/en/user/nodes.md` or `docs/en/user/node-metrics.md`
`feature/map/`	`docs/en/user/map-and-waypoints.md`
`feature/connections/`	`docs/en/user/connections.md`
`feature/settings/`	`docs/en/user/settings-radio-user.md` or `docs/en/user/settings-module-admin.md`
`feature/firmware/`	`docs/en/user/firmware.md`
`feature/intro/`	`docs/en/user/onboarding.md`
`feature/discovery/`	`docs/en/user/discovery.md`
`feature/docs/`	Internal docs infrastructure
`core/ui/`	`docs/en/developer/codebase.md` or component-specific user pages

New page checklist (if adding a new doc page):

Create the .md file in docs/en/user/ or docs/en/developer/ with last_updated frontmatter
Register in DocBundleLoader.kt with string resources (in-app browser)
Jekyll and Docusaurus sync pick up new pages automatically — no config change needed

If this PR does not require a doc update (e.g., internal refactor, bug fix, test change), add the skip-docs-check label to dismiss this check.

Cross-platform note: This check is advisory while doc coverage matures. Both Android and Apple repos use the same skip-docs-check label and advisory severity. See meshtastic/design standards for shared conventions.

jamesarich and others added 6 commits May 22, 2026 15:50

github-actions Bot added the enhancement New feature or request label May 22, 2026

jamesarich added this pull request to the merge queue May 22, 2026

Merged via the queue into main with commit d870141 May 22, 2026
18 checks passed

jamesarich deleted the jamesarich/install-firebase-skills branch May 22, 2026 23:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(ai): upgrade Chirpy on-device AI with proper APIs, download UX, and streaming#5579

feat(ai): upgrade Chirpy on-device AI with proper APIs, download UX, and streaming#5579
jamesarich merged 6 commits into
mainfrom
jamesarich/install-firebase-skills

jamesarich commented May 22, 2026

Uh oh!

github-actions Bot commented May 22, 2026

Uh oh!

github-actions Bot commented May 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

jamesarich commented May 22, 2026

Motivation

Approach

Key files

Notes for reviewers

Uh oh!

github-actions Bot commented May 22, 2026

🖼️ Preview staleness check — advisory

Uh oh!

github-actions Bot commented May 22, 2026

📄 Docs staleness check — advisory

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant