Skip to content

fix(whatsapp): resolve outbound PN to LID via auth-dir forward mapping#74925

Merged
mcaxtr merged 1 commit intoopenclaw:mainfrom
edenfunf:fix/whatsapp-outbound-lid-resolution
May 7, 2026
Merged

fix(whatsapp): resolve outbound PN to LID via auth-dir forward mapping#74925
mcaxtr merged 1 commit intoopenclaw:mainfrom
edenfunf:fix/whatsapp-outbound-lid-resolution

Conversation

@edenfunf
Copy link
Copy Markdown
Contributor

Summary

  • Problem: Proactive WhatsApp sends to phone numbers whose contact internally uses LID end up in a sender-only ghost chat. The message shows ✓✓ but never reaches the recipient (issue [Bug]: WhatsApp outbound messages create ghost chats due to missing LID→PN resolution in createWebSendApi #67378).
  • Why it matters: Silent data loss with a positive-success signal — agents report "sent" while users never see the message. Hard to debug for operators because the failure is invisible to the recipient and the gateway logs success.
  • What changed: Outbound active-listener send paths (sendMessage / sendPoll / sendComposingTo) now resolve a phone number to {lid}@lid via lid-mapping-{phoneDigits}.json in the account auth dir when present, falling back to {digits}@s.whatsapp.net otherwise. New helpers readLidForwardMapping + toWhatsappJidWithLid are added beside the existing readLidReverseMapping / jidToE164 to keep the LID handling symmetric.
  • What did NOT change (scope boundary): outbound-base.ts:131 and send.ts outer entries keep toWhatsappJid because those produce redacted log strings only; the actual delivery flows through active.sendMessage(to, ...). sendReaction's chatJid keeps toWhatsappJid because callers pass already-formed JIDs (group / DM / LID). The companion lid-mapping-{lid}_reverse.json regeneration after re-link, mentioned at the end of [Bug]: WhatsApp outbound messages create ghost chats due to missing LID→PN resolution in createWebSendApi #67378, is left out as a separate fix.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Root Cause

  • Root cause: toWhatsappJid in extensions/whatsapp/src/targets-runtime.ts is purely lexical — it maps any phone number to {digits}@s.whatsapp.net with no awareness of LID. The reverse direction (LID JID → PN) was already wired through readLidReverseMapping reading lid-mapping-{lid}_reverse.json, but the forward direction (PN → LID) was missing entirely on the outbound path. When Baileys' contact uses LID internally, sending to s.whatsapp.net creates a separate sender-side thread.
  • Missing detection / guardrail: No assertion that proactive sends route to a delivery-confirmable JID. WhatsApp Web does not surface the ghost-chat condition as an error.
  • Contributing context (if known): Baileys 7.0 stores both directions under the lid-mapping keyspace — [pnUser]: lidUser and [lidUser_reverse]: pnUser — but only the reverse half had a code path here.

Regression Test Plan

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file:
    • extensions/whatsapp/src/text-runtime.test.tsdescribe("toWhatsappJidWithLid (issue #67378)", ...)
    • extensions/whatsapp/src/inbound/send-api.test.tsdescribe("createWebSendApi LID resolution (issue #67378)", ...)
  • Scenario the test should lock in: Given a lid-mapping-{phoneDigits}.json containing a LID, outbound sendMessage / sendPoll / sendComposingTo must call sock.sendMessage (or sendPresenceUpdate) with {lid}@lid. With no forward mapping, the legacy {digits}@s.whatsapp.net JID must still be used. Without authDir, behavior must be identical to before this PR.
  • Why this is the smallest reliable guardrail: The bug surfaces inside createWebSendApi, which is exactly where the new tests assert. The unit test on toWhatsappJidWithLid pins down the helper contract; the integration test confirms the authDir is threaded all the way through and that all three outbound entry points use it.
  • Existing test that already covers this (if any): None — all prior createWebSendApi tests instantiate without authDir, so they never exercised LID resolution.
  • If no new test is added, why not: N/A — both layers added.

User-visible / Behavior Changes

  • Proactive WhatsApp sends to LID-addressed contacts now actually reach the recipient. No config flag, no migration — behavior changes only for accounts whose Baileys auth dir already contains forward LID mappings (i.e., contacts have been observed at least once on inbound).
  • Accounts with no LID mappings see no behavior change.

Diagram (if applicable)

Before:
  message tool ── "+15555550000" ──> toWhatsappJid ──> 15555550000@s.whatsapp.net
                                                       └─> WA Web ──> ghost chat (sender-only) ✗

After (with authDir + forward mapping present):
  message tool ── "+15555550000" ──> toWhatsappJidWithLid ──> reads lid-mapping-15555550000.json
                                                              └─> 987654@lid ──> WA Web ──> recipient ✓

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No (same Baileys send call, only the JID argument changes)
  • Command/tool execution surface changed? No
  • Data access scope changed? No (reads files already created and used by Baileys in the same auth dir)

Repro + Verification

Environment

Steps

  1. Start with an OpenClaw account paired to WhatsApp Web; receive any message from a contact so Baileys writes the forward and reverse lid-mapping-*.json for them.
  2. Have an agent invoke the message tool to send to that contact's phone number.
  3. Observe the outbound JID Baileys is asked to send to and where the message lands.

Expected

  • Outbound JID: {lid}@lid. Message arrives in the recipient's existing thread on Android/iOS and on the recipient's WhatsApp.

Actual (before this PR)

  • Outbound JID: {digits}@s.whatsapp.net. A ghost chat appears only on the sender's WA Web; recipient never sees the message.

Evidence

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

The new text-runtime.test.ts and send-api.test.ts LID describe blocks fail against main and pass against this branch.

Human Verification (required)

  • Verified scenarios: Helper-level (4 cases for toWhatsappJidWithLid) and integration-level (5 cases for createWebSendApi) covering: PN with mapping, PN without mapping, numeric LID values, already-formed JIDs (group, s.whatsapp.net, lid), legacy path with no authDir, and sendPoll / sendComposingTo parity with sendMessage.
  • Edge cases checked: Forward file containing numeric vs string LID, whatsapp: prefix, group JIDs passing through unchanged, missing forward file falling back to PN, no-authDir regression guard.
  • What you did NOT verify: Multi-device LID ({lid}:N@lid) — Baileys' on-disk format does not store device specificity for the forward direction, so the LID JID emitted is device-less, matching the working pattern the reporter applied to the compiled bundle. @hosted.lid outbound — same pattern would apply but was not in scope here.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes. No API change visible to callers; authDir is opt-in and defaults to legacy PN-only behavior.
  • Config/env changes? No
  • Migration needed? No — Baileys already writes the required mapping files; this PR only consumes them.

Risks and Mitigations

  • Risk: A forward mapping file might exist for a phone number whose LID is no longer the recipient's active LID (e.g., contact migrated devices). Sending to the stale LID could itself create a delivery hole.
    • Mitigation: This is no worse than the inbound behavior, which already trusts the same files for LID→PN. Baileys keeps the mapping in sync via storeLIDPNMappings whenever a fresh pair is observed. No new write path introduced here.
  • Risk: Tests can race if multiple cases reuse a single tmp dir.
    • Mitigation: Each withTempDir / beforeEach creates a fresh mkdtempSync and the afterEach removes it.

@openclaw-barnacle openclaw-barnacle Bot added channel: whatsapp-web Channel integration: whatsapp-web size: S labels Apr 30, 2026
@clawsweeper
Copy link
Copy Markdown
Contributor

clawsweeper Bot commented Apr 30, 2026

Codex review: needs maintainer review before merge.

Summary
This PR threads WhatsApp authDir into createWebSendApi, adds LID forward-mapping resolution for outbound message/poll/composing sends, exports and tests the helper, and adds a changelog entry.

Reproducibility: yes. source-level. Current main routes proactive phone targets through toWhatsappJid without any account authDir, so existing Baileys lid-mapping-{phoneDigits}.json files cannot influence outbound JID selection.

Real behavior proof
Override: The PR carries proof: override, so the external real-behavior proof gate is explicitly overridden and no contributor proof action is required.

Next step before merge
No repair lane is needed because the prior changelog blocker is fixed and there is no discrete actionable defect for automation to repair.

Security
Cleared: The diff only changes WhatsApp recipient JID selection, reads existing Baileys auth-dir files, and adds tests/changelog; it introduces no dependency, workflow, secret, command-execution, or package-resolution change.

Review details

Best possible solution:

Merge the focused WhatsApp plugin fix after normal maintainer and CI gates, and track the re-link reverse-mapping concern separately if maintainers want that follow-up.

Do we have a high-confidence way to reproduce the issue?

Yes, source-level. Current main routes proactive phone targets through toWhatsappJid without any account authDir, so existing Baileys lid-mapping-{phoneDigits}.json files cannot influence outbound JID selection.

Is this the best way to solve the issue?

Yes. Resolving phone-number recipients against the same account auth directory inside the active send API is the narrow maintainable fix for message, poll, and composing delivery; the separate reverse-mapping re-link behavior should stay as follow-up work.

What I checked:

  • Current main outbound path is PN-only: On current main, createWebSendApi has no authDir parameter and sendMessage, sendPoll, and sendComposingTo send phone targets through toWhatsappJid, which always builds {digits}@s.whatsapp.net. (extensions/whatsapp/src/inbound/send-api.ts:61, 92284bc46043)
  • Current main only reads reverse LID mapping: Current targets-runtime.ts has readLidReverseMapping for inbound lid-mapping-{lid}_reverse.json lookup but no forward PN-to-LID resolver. (extensions/whatsapp/src/targets-runtime.ts:79, 92284bc46043)
  • PR head implements active-send LID resolution: The PR head adds authDir?: string, defines resolveOutboundJid, and uses it for sendMessage, sendPoll, and sendComposingTo while leaving reactions on the existing JID contract. (extensions/whatsapp/src/inbound/send-api.ts:50, d8d3ec5c4b68)
  • PR head adds focused tests: The PR adds helper and createWebSendApi coverage for mapped PN sends, missing mapping fallback, poll, composing presence, newsletter pass-through, already-formed JIDs, and no-authDir legacy behavior. (extensions/whatsapp/src/inbound/send-api.test.ts:380, d8d3ec5c4b68)
  • Baileys persistence contract matches the PR: Baileys 7.0.0-rc.9 LIDMappingStore.storeLIDPNMappings stores lid-mapping keys as [pnUser]: lidUser and [lidUser_reverse]: pnUser; useMultiFileAuthState persists keys as {type}-{id}.json, matching lid-mapping-{phoneDigits}.json.
  • Changelog finding is resolved: The PR head now includes a single-line user-facing WhatsApp fix entry under the active changelog section with issue and PR references plus contributor credit. (CHANGELOG.md:486, d8d3ec5c4b68)

Likely related people:

  • steipete: Recent path history shows Peter Steinberger repeatedly touching WhatsApp send/runtime code around provider-accepted auto-replies, helper exports, and plugin seam refactors; the existing reverse LID helper is in the same target normalization module. (role: recent maintainer / adjacent LID owner; confidence: high; commits: eab402493429, d647ba1c6fb6, f0000ab72d01; files: extensions/whatsapp/src/inbound/send-api.ts, extensions/whatsapp/src/inbound/monitor.ts, extensions/whatsapp/src/targets-runtime.ts)
  • mcaxtr: Marcus Castro has recent merged work on WhatsApp account connection lifecycle and reply/quoting behavior near this path, and also performed maintainer follow-up on this PR branch. (role: active listener lifecycle / outbound delivery maintainer; confidence: medium; commits: aa023e428306, f5f0235bb18a, d8d3ec5c4b68; files: extensions/whatsapp/src/inbound/send-api.ts, extensions/whatsapp/src/inbound/monitor.ts, extensions/whatsapp/src/inbound/send-api.test.ts)
  • vincentkoc: Current-main shallow blame points the central send API and target helper lines to Vincent Koc’s latest snapshot commit, and GitHub path history shows recent WhatsApp target handling work for newsletter sends. (role: recent adjacent maintainer; confidence: medium; commits: 3a12a7a7e6, 0fad53a19281; files: extensions/whatsapp/src/inbound/send-api.ts, extensions/whatsapp/src/targets-runtime.ts)

Remaining risk / open question:

  • No local tests or live WhatsApp delivery proof were run in this read-only pass; merge should still rely on CI and the maintainer proof: override decision.
  • The linked report also mentions reverse-mapping regeneration after re-link; this PR intentionally leaves that separate concern out of scope.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 92284bc46043.

@mcaxtr mcaxtr self-assigned this May 7, 2026
@mcaxtr mcaxtr force-pushed the fix/whatsapp-outbound-lid-resolution branch from 496cfcb to d8d3ec5 Compare May 7, 2026 03:33
@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 7, 2026
@mcaxtr mcaxtr added the proof: override Maintainer override for the external PR real behavior proof gate. label May 7, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 7, 2026
@mcaxtr mcaxtr force-pushed the fix/whatsapp-outbound-lid-resolution branch 3 times, most recently from 483d3d6 to 958cfd1 Compare May 7, 2026 03:48
openclaw#67378)

When an agent proactively sends a WhatsApp message to a phone number
whose contact uses LID internally, the outbound JID was always built as
`{digits}@s.whatsapp.net`. WhatsApp Web routes that to a sender-only
ghost chat that never reaches the recipient — the message appears sent
(double check) but is invisible on Android/iOS and on the recipient's
device.

Baileys writes a per-account forward mapping `lid-mapping-{pnUser}.json`
in the auth dir whenever an LID-PN pair is observed (see
Signal/lid-mapping.js storeLIDPNMappings). Inbound already consults this
directory through readLidReverseMapping for the LID->PN direction;
outbound had no symmetric helper.

Add `readLidForwardMapping` and `toWhatsappJidWithLid(num, { authDir })`
in targets-runtime, mirroring the existing reverse helper. Thread the
account's `authDir` from the inbound monitor into `createWebSendApi`,
and route sendMessage / sendPoll / sendComposingTo through a new
`resolveOutboundJid` that uses the LID JID when a forward mapping
exists, falling back to PN-only behavior otherwise.

sendReaction's chatJid still uses the existing PN-aware `toWhatsappJid`
because callers pass an already-formed JID (group, DM, or LID); the
participant key follows the same legacy contract.

`outbound-base.ts` and `send.ts` outer entries are unchanged: those
`toWhatsappJid` calls only feed redacted log strings; the actual send
goes through `active.sendMessage(to, ...)` which now does LID
resolution.
@mcaxtr mcaxtr force-pushed the fix/whatsapp-outbound-lid-resolution branch from 958cfd1 to 5f51cb7 Compare May 7, 2026 03:49
@mcaxtr mcaxtr merged commit fcdfa30 into openclaw:main May 7, 2026
16 checks passed
@mcaxtr
Copy link
Copy Markdown
Member

mcaxtr commented May 7, 2026

Merged via squash.

Thanks @edenfunf!

rogerdigital pushed a commit to rogerdigital/openclaw that referenced this pull request May 7, 2026
openclaw#74925)

Merged via squash.

Prepared head SHA: 5f51cb7
Co-authored-by: edenfunf <146086744+edenfunf@users.noreply.github.com>
Co-authored-by: mcaxtr <7562095+mcaxtr@users.noreply.github.com>
Reviewed-by: @mcaxtr
steipete pushed a commit that referenced this pull request May 7, 2026
#74925)

Merged via squash.

Prepared head SHA: 5f51cb7
Co-authored-by: edenfunf <146086744+edenfunf@users.noreply.github.com>
Co-authored-by: mcaxtr <7562095+mcaxtr@users.noreply.github.com>
Reviewed-by: @mcaxtr

(cherry picked from commit fcdfa30)
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
openclaw#74925)

Merged via squash.

Prepared head SHA: 5f51cb7
Co-authored-by: edenfunf <146086744+edenfunf@users.noreply.github.com>
Co-authored-by: mcaxtr <7562095+mcaxtr@users.noreply.github.com>
Reviewed-by: @mcaxtr
rogerdigital pushed a commit to rogerdigital/openclaw that referenced this pull request May 9, 2026
openclaw#74925)

Merged via squash.

Prepared head SHA: 5f51cb7
Co-authored-by: edenfunf <146086744+edenfunf@users.noreply.github.com>
Co-authored-by: mcaxtr <7562095+mcaxtr@users.noreply.github.com>
Reviewed-by: @mcaxtr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: whatsapp-web Channel integration: whatsapp-web proof: override Maintainer override for the external PR real behavior proof gate. size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: WhatsApp outbound messages create ghost chats due to missing LID→PN resolution in createWebSendApi

2 participants