Skip to content

Gateway Heartbeat timer stops after 1-2 triggers (introduced in v2026.3.8) #45772

@Suidge

Description

@Suidge

Bug Description

Gateway heartbeat feature stops working after triggering 1-2 times. The timer does not reschedule after firing, causing heartbeat to permanently stop.

Timeline

  • 2026-03-09 16:50: User asked about OpenClaw v2026.3.8 release notes
  • 2026-03-09 17:02:12: Config hot reload (lossless-claw plugin update)
  • 2026-03-09 17:02:14: Heartbeat first triggered (never worked correctly from the start)
  • 2026-03-09 to 2026-03-13: Heartbeat intervals chaotic (minutes to hours, not periodic)
  • 2026-03-14 10:14: Heartbeat triggered last time
  • 2026-03-14 10:14 to 12:46: No heartbeat for 2.5 hours
  • 2026-03-14 12:46: Gateway restarted
  • 2026-03-14 12:46:00: Heartbeat triggered once after restart
  • 2026-03-14 13:16: Expected heartbeat (MISSING)
  • 2026-03-14 13:37: Still no heartbeat (51 minutes since last trigger)

Evidence

Log excerpts (2026-03-14):

2026-03-14T10:11:18.478+08:00 [heartbeat] started
2026-03-14T10:14:11.872+08:00 [heartbeat] started
# GAP: 2.5 hours with NO heartbeat
2026-03-14T12:46:00.236+08:00 [heartbeat] started (after Gateway restart)
# NO subsequent triggers (expected every 30m)

Configuration at time of failure:

{
  "agents": {
    "defaults": {
      "heartbeat": {
        "every": "30m",
        "activeHours": {
          "start": "07:00",
          "end": "24:00",
          "timezone": "Asia/Shanghai"
        },
        "target": "last",
        "directPolicy": "allow"
      }
    },
    "list": [
      {
        "id": "agent-qwen",
        "heartbeat": {}
      }
    ]
  }
}

Root Cause Analysis

1. Heartbeat never worked correctly

From first activation on 2026-03-09, intervals were chaotic (not periodic as configured with every: "1h")

2. Hot reload may corrupt timer

Config hot reloads on 2026-03-14 (10:05 and 10:13) were followed by heartbeat stopping completely

3. Timer not rescheduling

After each trigger, the timer should reschedule for next interval. Instead, it fires 1-2 times then stops permanently.

4. Gateway restart only temporarily fixes

Restart causes one trigger, then timer stops again.

🔍 CRITICAL: Related to Lossless-Claw Context Engine

Timeline correlation:

Gateway log (2026-03-09 17:02):

17:02:12 [reload] config change detected (plugins.installs.lossless-claw.version...)
17:02:22 [lcm] Plugin loaded (enabled=true, db=/Users/neoshi/.openclaw/lcm.db)
17:02:23 [lcm] Plugin loaded (repeated)
17:02:30 [lcm] Plugin loaded (repeated)
17:02:32 [lcm] Plugin loaded (repeated)
17:02:33 [lcm] Plugin loaded (repeated)

Related Issues:

Lossless-Claw / Context Engine:

Issue Created Title
#40232 Mar 8 Plugin context engines can resolve before plugin registration
#44511 Mar 13 LosslessClaw + v2026.3.11: Tasks stop silently
#45059 Mar 13 ContextEngine plugin access intermittently unavailable
#42628 Mar 11 context-engine auto-compaction does not persist compactionCount

Heartbeat:

Issue Created Title
#39185 Mar 7 Heartbeat subsystem ignores HEARTBEAT.md
#41879 Mar 10 isHeartbeatEnabledForAgent ignores agents.defaults.heartbeat
#42495 Mar 11 Heartbeat polls overwrite session displayName
#43728 Mar 12 Heartbeat config written to invalid location
#45422 Mar 13 heartbeat.model configuration not working
#45772 Mar 14 Gateway Heartbeat timer stops (this issue)

Hypothesis:

The lossless-claw context engine registration timing bug (#40232) may be causing heartbeat scheduler to fail:

  1. Context engine resolves before plugin registration completes
  2. Heartbeat scheduler depends on context engine state
  3. State inconsistency causes timer to not reschedule after trigger
  4. Repeated plugin reload (seen in logs) may corrupt scheduler state

Expected Behavior

Heartbeat should trigger periodically (every 30m or 1h as configured) during activeHours (07:00-24:00).

Actual Behavior

  • Heartbeat triggers 1-2 times after Gateway start/restart
  • Timer does not reschedule
  • No further triggers until next restart
  • Intervals are chaotic even when working

Impact

  • Heartbeat-dependent features (daily calendar checks, weather reports, email monitoring) do not work
  • Users must manually restart Gateway or use cron-based workaround

Environment

  • OpenClaw version: v2026.3.8 (issue introduced in this version)
  • OS: macOS (Darwin 25.3.0, arm64)
  • Node: v25.8.1
  • First occurrence: 2026-03-09 (coincides with v2026.3.8 update + lossless-claw hot reload)
  • Plugin: @martian-engineering/lossless-claw (repeatedly loaded during hot reload)

Workaround

Use cron job instead of built-in heartbeat:

{
  "schedule": {
    "kind": "cron",
    "expr": "0,30 7-23 * * *",
    "tz": "Asia/Shanghai"
  },
  "payload": {
    "kind": "agentTurn",
    "message": "Execute heartbeat tasks"
  }
}

Request

  1. Investigate Gateway heartbeat timer implementation — check setInterval/setTimeout logic
  2. Check if hot reload properly resets/reschedules timer — config changes should not break timer
  3. Investigate lossless-claw context engine registration timing (Plugin context engines can resolve before plugin registration (Lossless Claw / contextEngine slot) #40232) — may be related to heartbeat scheduler failure
  4. Verify every duration parsing — ensure correct interval calculation
  5. Consider adding heartbeat health monitoring — alert if no triggers in 2x expected interval
  6. Review plugin reload behavior — repeated plugin load (seen in logs) may corrupt scheduler state

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions