Bug type
Regression (worked before, now fails)
Beta release blocker
No
Summary
After upgrading beyond 2026.4.21, an always-on Raspberry Pi Slack gateway saw Slack replies increase from same-minute responses to multi-minute delays, with Slack Socket Mode missed pongs and gateway event-loop stalls observed in logs.
Steps to reproduce
- Run OpenClaw as a gateway on a Raspberry Pi-class device using Slack Socket Mode.
- Configure a Slack channel route to an agent using
google/gemini-2.5-flash.
- Use a newer OpenClaw version such as
2026.4.29.
- Send a Slack message to the configured channel.
- Observe multi-minute delay and missed Slack pong warnings in gateway logs.
- Downgrade the same setup to
2026.4.21.
- Observe that the worst multi-minute delay is no longer present (although unmodified
2026.4.21 still has a separate smaller per-message ensureOpenClawModelsJson(...) delay).
Expected behavior
On the same Raspberry Pi Slack gateway setup, Slack replies should remain approximately same-minute or faster, as observed before the regression and after downgrading/backporting the fast model-resolution path.
Expensive plugin runtime or provider/model discovery work should not block the Slack gateway event loop long enough to miss Socket Mode pongs.
Actual behavior
On newer 2026.4.x builds after 2026.4.21, Slack replies were delayed by minutes. The gateway logs showed missed Slack Socket Mode pongs during long gateway/agent-preparation windows.
Examples of observed user-visible delays:
Neerav [6:46 AM]
Hi, this is a test
Gabriel [6:52 AM]
Hi there! How can I help you today, Neerav?
Neerav [8:29 AM]
When is Arsenal's next game?
Gabriel [8:32 AM]
Arsenal's next game is against Fulham on Saturday, May 2, 2026, in the Premier League. The match is scheduled to kick off at 5:30 PM BST (4:30 PM UTC).
After downgrading to 2026.4.21, the worst multi-minute behavior was not observed in the same way:
Neerav [9:18 AM]
Who will Manchester city play against next?
Gabriel [9:19 AM]
Manchester City will play against Everton next, on Monday, May 4, 2026, at 3:00 PM (English Premier League).
OpenClaw version
2026.4.21 and higher versions
Operating system
Raspberry Pi OS (Raspberry Pi gateway device, Linux-based)
Install method
npm global install, running via openclaw-gateway.service
Model
google/gemini-2.5-flash
Provider / routing chain
openclaw -> google/gemini-2.5-flash via direct Google Gemini API (google-generative-ai)
Additional provider/model setup details
- Slack channel route was bound to a
gemini agent.
- Effective model provider was
google.
- Effective model was
gemini-2.5-flash.
- The gateway process had Google Gemini API credentials available via environment.
- This did not appear to be NVIDIA/OpenRouter/intermediary model latency; the largest delays were before the model attempt or during gateway/runtime setup.
Logs, screenshots, and evidence
### Missed Slack Pong Warning
socket-mode:SlackWebSocket:1 A pong wasn't received from the server before the timeout of 5000ms!
### Version Comparison Evidence
During local package comparison, `2026.4.22+` contained bundled runtime root/mirror code not present in `2026.4.21`. On this Pi gateway, newer versions correlated with heavy gateway/agent-preparation work, missed pongs, and multi-minute Slack replies.
### Downgraded `2026.4.21` Still Had A Separate Per-Message Delay
Temporary tracing on unmodified `2026.4.21` showed this message-path step taking about 32 seconds:
12:16:38.120 embeddedRunner.ensureModels.before
12:17:10.232 embeddedRunner.ensureModels.after
Missed pongs happened during the same window:
12:17:03 socket-mode:SlackWebSocket:1 A pong wasn't received from the server before the timeout of 5000ms!
12:17:05 socket-mode:SlackWebSocket:1 A pong wasn't received from the server before the timeout of 5000ms!
The traced code path was:
embeddedRunner.ensureModels.before
await ensureOpenClawModelsJson(params.config, agentDir)
embeddedRunner.ensureModels.after
### Fast Model Resolution Workaround
The newer codebase has a fast embedded model resolution path that tries model resolution with Pi discovery skipped and only falls back to `ensureOpenClawModelsJson(...)` if the fast path misses:
const dynamicModelResolution = await resolveModelAsync(provider, modelId, agentDir, params.config, { skipPiDiscovery: true });
const { model, error, authStorage, modelRegistry } =
dynamicModelResolution.model
? dynamicModelResolution
: await (async () => {
await ensureOpenClawModelsJson(params.config, agentDir);
return await resolveModelAsync(provider, modelId, agentDir, params.config);
})();
Backporting this narrow behavior to `2026.4.21` removed `ensureOpenClawModelsJson(...)` from the warm Slack message path.
Warm trace after the workaround:
12:28:04.731 Slack handler received message
12:28:05.943 runEmbeddedPiAgent entered
12:28:05.976 embeddedRunner.resolveModel.fast.before
12:28:07.319 embeddedRunner.resolveModel.fast.after
12:28:07.409 embeddedAttempt.start
12:28:09.003 embeddedAttempt.tools.create.after
12:28:09.556 embeddedAttempt.systemPrompt.build.after
12:28:12.468 embeddedRunner.attemptBackend.after
12:28:13.280 Slack dispatch completed
Total gateway time after the workaround: about 8.55 seconds.
No `embeddedRunner.ensureModels.*` lines appeared in the warm trace.
No missed Slack pong warnings appeared in the warm trace.
### User-Visible Behavior After Workaround
Neerav [12:22 PM]
Does Fulham have any suspensions or injuries currently?
Gabriel [12:22 PM]
To give you the most accurate and up-to-date information, I'll need to check a specific source for Fulham's current injuries and suspensions. These things can change very rapidly in football!
Looking at the search results, it seems "Premier Injuries" and "Transfermarkt" are good sources for this kind of information. I'll check them for you now.
Impact and severity
Affected: Raspberry Pi Slack gateway users on newer 2026.4.x versions after 2026.4.21.
Severity: High for always-on Slack gateway use on resource-constrained hardware.
Frequency: Multiple observed Slack messages on the affected Pi setup showed multi-minute latency before downgrade. The per-message ensureOpenClawModelsJson(...) delay was directly observed on 2026.4.21 before the fast-path workaround.
Consequence: Slack replies were delayed by minutes and Slack Socket Mode missed pongs during gateway preparation windows, making the bot feel unreliable.
Additional information
The investigation suggests two separate but related performance concerns:
- Versions after
2026.4.21 introduced heavier bundled plugin runtime root/mirror behavior. On this Pi gateway, that correlated with multi-minute Slack replies and missed pongs.
- Unmodified
2026.4.21 still ran ensureOpenClawModelsJson(...) on the message path, which took about 32 seconds on the Pi. A fast model-resolution-first path (which is present in newer OpenClaw versions) fixed that portion when backported.
Request (feel free to ignore):
- Please check whether
2026.4.22+ bundled runtime root/mirror behavior can be cached, startup-only, asynchronous, or otherwise prevented from blocking Slack gateway responsiveness on low-resource devices.
- Please consider a low-resource gateway mode that avoids expensive plugin/provider/model discovery during normal message handling.
Bug type
Regression (worked before, now fails)
Beta release blocker
No
Summary
After upgrading beyond
2026.4.21, an always-on Raspberry Pi Slack gateway saw Slack replies increase from same-minute responses to multi-minute delays, with Slack Socket Mode missed pongs and gateway event-loop stalls observed in logs.Steps to reproduce
google/gemini-2.5-flash.2026.4.29.2026.4.21.2026.4.21still has a separate smaller per-messageensureOpenClawModelsJson(...)delay).Expected behavior
On the same Raspberry Pi Slack gateway setup, Slack replies should remain approximately same-minute or faster, as observed before the regression and after downgrading/backporting the fast model-resolution path.
Expensive plugin runtime or provider/model discovery work should not block the Slack gateway event loop long enough to miss Socket Mode pongs.
Actual behavior
On newer
2026.4.xbuilds after2026.4.21, Slack replies were delayed by minutes. The gateway logs showed missed Slack Socket Mode pongs during long gateway/agent-preparation windows.Examples of observed user-visible delays:
After downgrading to
2026.4.21, the worst multi-minute behavior was not observed in the same way:OpenClaw version
2026.4.21 and higher versions
Operating system
Raspberry Pi OS (Raspberry Pi gateway device, Linux-based)
Install method
npm global install, running via
openclaw-gateway.serviceModel
google/gemini-2.5-flashProvider / routing chain
openclaw -> google/gemini-2.5-flashvia direct Google Gemini API (google-generative-ai)Additional provider/model setup details
geminiagent.google.gemini-2.5-flash.Logs, screenshots, and evidence
Impact and severity
Affected: Raspberry Pi Slack gateway users on newer
2026.4.xversions after2026.4.21.Severity: High for always-on Slack gateway use on resource-constrained hardware.
Frequency: Multiple observed Slack messages on the affected Pi setup showed multi-minute latency before downgrade. The per-message
ensureOpenClawModelsJson(...)delay was directly observed on2026.4.21before the fast-path workaround.Consequence: Slack replies were delayed by minutes and Slack Socket Mode missed pongs during gateway preparation windows, making the bot feel unreliable.
Additional information
The investigation suggests two separate but related performance concerns:
2026.4.21introduced heavier bundled plugin runtime root/mirror behavior. On this Pi gateway, that correlated with multi-minute Slack replies and missed pongs.2026.4.21still ranensureOpenClawModelsJson(...)on the message path, which took about 32 seconds on the Pi. A fast model-resolution-first path (which is present in newer OpenClaw versions) fixed that portion when backported.Request (feel free to ignore):
2026.4.22+bundled runtime root/mirror behavior can be cached, startup-only, asynchronous, or otherwise prevented from blocking Slack gateway responsiveness on low-resource devices.