Skip to content

[Bug]: Raspberry Pi Slack gateway latency regression after 2026.4.21 #75872

@neeravmakwana

Description

@neeravmakwana

Bug type

Regression (worked before, now fails)

Beta release blocker

No

Summary

After upgrading beyond 2026.4.21, an always-on Raspberry Pi Slack gateway saw Slack replies increase from same-minute responses to multi-minute delays, with Slack Socket Mode missed pongs and gateway event-loop stalls observed in logs.

Steps to reproduce

  1. Run OpenClaw as a gateway on a Raspberry Pi-class device using Slack Socket Mode.
  2. Configure a Slack channel route to an agent using google/gemini-2.5-flash.
  3. Use a newer OpenClaw version such as 2026.4.29.
  4. Send a Slack message to the configured channel.
  5. Observe multi-minute delay and missed Slack pong warnings in gateway logs.
  6. Downgrade the same setup to 2026.4.21.
  7. Observe that the worst multi-minute delay is no longer present (although unmodified 2026.4.21 still has a separate smaller per-message ensureOpenClawModelsJson(...) delay).

Expected behavior

On the same Raspberry Pi Slack gateway setup, Slack replies should remain approximately same-minute or faster, as observed before the regression and after downgrading/backporting the fast model-resolution path.

Expensive plugin runtime or provider/model discovery work should not block the Slack gateway event loop long enough to miss Socket Mode pongs.

Actual behavior

On newer 2026.4.x builds after 2026.4.21, Slack replies were delayed by minutes. The gateway logs showed missed Slack Socket Mode pongs during long gateway/agent-preparation windows.

Examples of observed user-visible delays:

Neerav [6:46 AM]
Hi, this is a test

Gabriel [6:52 AM]
Hi there! How can I help you today, Neerav?
Neerav [8:29 AM]
When is Arsenal's next game?

Gabriel [8:32 AM]
Arsenal's next game is against Fulham on Saturday, May 2, 2026, in the Premier League. The match is scheduled to kick off at 5:30 PM BST (4:30 PM UTC).

After downgrading to 2026.4.21, the worst multi-minute behavior was not observed in the same way:

Neerav [9:18 AM]
Who will Manchester city play against next?

Gabriel [9:19 AM]
Manchester City will play against Everton next, on Monday, May 4, 2026, at 3:00 PM (English Premier League).

OpenClaw version

2026.4.21 and higher versions

Operating system

Raspberry Pi OS (Raspberry Pi gateway device, Linux-based)

Install method

npm global install, running via openclaw-gateway.service

Model

google/gemini-2.5-flash

Provider / routing chain

openclaw -> google/gemini-2.5-flash via direct Google Gemini API (google-generative-ai)

Additional provider/model setup details

  • Slack channel route was bound to a gemini agent.
  • Effective model provider was google.
  • Effective model was gemini-2.5-flash.
  • The gateway process had Google Gemini API credentials available via environment.
  • This did not appear to be NVIDIA/OpenRouter/intermediary model latency; the largest delays were before the model attempt or during gateway/runtime setup.

Logs, screenshots, and evidence

### Missed Slack Pong Warning


socket-mode:SlackWebSocket:1 A pong wasn't received from the server before the timeout of 5000ms!


### Version Comparison Evidence

During local package comparison, `2026.4.22+` contained bundled runtime root/mirror code not present in `2026.4.21`. On this Pi gateway, newer versions correlated with heavy gateway/agent-preparation work, missed pongs, and multi-minute Slack replies.

### Downgraded `2026.4.21` Still Had A Separate Per-Message Delay

Temporary tracing on unmodified `2026.4.21` showed this message-path step taking about 32 seconds:


12:16:38.120 embeddedRunner.ensureModels.before
12:17:10.232 embeddedRunner.ensureModels.after


Missed pongs happened during the same window:


12:17:03 socket-mode:SlackWebSocket:1 A pong wasn't received from the server before the timeout of 5000ms!
12:17:05 socket-mode:SlackWebSocket:1 A pong wasn't received from the server before the timeout of 5000ms!


The traced code path was:


embeddedRunner.ensureModels.before
await ensureOpenClawModelsJson(params.config, agentDir)
embeddedRunner.ensureModels.after


### Fast Model Resolution Workaround

The newer codebase has a fast embedded model resolution path that tries model resolution with Pi discovery skipped and only falls back to `ensureOpenClawModelsJson(...)` if the fast path misses:


const dynamicModelResolution = await resolveModelAsync(provider, modelId, agentDir, params.config, { skipPiDiscovery: true });
const { model, error, authStorage, modelRegistry } =
  dynamicModelResolution.model
    ? dynamicModelResolution
    : await (async () => {
        await ensureOpenClawModelsJson(params.config, agentDir);
        return await resolveModelAsync(provider, modelId, agentDir, params.config);
      })();


Backporting this narrow behavior to `2026.4.21` removed `ensureOpenClawModelsJson(...)` from the warm Slack message path.

Warm trace after the workaround:


12:28:04.731 Slack handler received message
12:28:05.943 runEmbeddedPiAgent entered
12:28:05.976 embeddedRunner.resolveModel.fast.before
12:28:07.319 embeddedRunner.resolveModel.fast.after
12:28:07.409 embeddedAttempt.start
12:28:09.003 embeddedAttempt.tools.create.after
12:28:09.556 embeddedAttempt.systemPrompt.build.after
12:28:12.468 embeddedRunner.attemptBackend.after
12:28:13.280 Slack dispatch completed


Total gateway time after the workaround: about 8.55 seconds.

No `embeddedRunner.ensureModels.*` lines appeared in the warm trace.
No missed Slack pong warnings appeared in the warm trace.

### User-Visible Behavior After Workaround


Neerav [12:22 PM]
Does Fulham have any suspensions or injuries currently?

Gabriel [12:22 PM]
To give you the most accurate and up-to-date information, I'll need to check a specific source for Fulham's current injuries and suspensions. These things can change very rapidly in football!

Looking at the search results, it seems "Premier Injuries" and "Transfermarkt" are good sources for this kind of information. I'll check them for you now.

Impact and severity

Affected: Raspberry Pi Slack gateway users on newer 2026.4.x versions after 2026.4.21.

Severity: High for always-on Slack gateway use on resource-constrained hardware.

Frequency: Multiple observed Slack messages on the affected Pi setup showed multi-minute latency before downgrade. The per-message ensureOpenClawModelsJson(...) delay was directly observed on 2026.4.21 before the fast-path workaround.

Consequence: Slack replies were delayed by minutes and Slack Socket Mode missed pongs during gateway preparation windows, making the bot feel unreliable.

Additional information

The investigation suggests two separate but related performance concerns:

  1. Versions after 2026.4.21 introduced heavier bundled plugin runtime root/mirror behavior. On this Pi gateway, that correlated with multi-minute Slack replies and missed pongs.
  2. Unmodified 2026.4.21 still ran ensureOpenClawModelsJson(...) on the message path, which took about 32 seconds on the Pi. A fast model-resolution-first path (which is present in newer OpenClaw versions) fixed that portion when backported.

Request (feel free to ignore):

  • Please check whether 2026.4.22+ bundled runtime root/mirror behavior can be cached, startup-only, asynchronous, or otherwise prevented from blocking Slack gateway responsiveness on low-resource devices.
  • Please consider a low-resource gateway mode that avoids expensive plugin/provider/model discovery during normal message handling.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingregressionBehavior that previously worked and now fails

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions