Skip to content

feat(dingtalk): proactive messaging, media pipeline, card throttle + quoted-reply extraction#12769

Open
PeterGuy326 wants to merge 4 commits into
NousResearch:mainfrom
PeterGuy326:feat/dingtalk-proactive-messaging
Open

feat(dingtalk): proactive messaging, media pipeline, card throttle + quoted-reply extraction#12769
PeterGuy326 wants to merge 4 commits into
NousResearch:mainfrom
PeterGuy326:feat/dingtalk-proactive-messaging

Conversation

@PeterGuy326

@PeterGuy326 PeterGuy326 commented Apr 20, 2026

Copy link
Copy Markdown

Hi maintainers,

First off — huge thanks for building Hermes Agent. The design quality and the way platform adapters are factored out have made integration genuinely pleasant on our side.

This PR comes from the official DingTalk (钉钉) team at Alibaba. For context, DingTalk is one of the largest enterprise collaboration platforms in China, serving hundreds of millions of users across organizations of every size, and we've been integrating Hermes as a first-class option inside our workspace. Everything in this PR is upstream-worthy work we did during that integration — we'd love to give it back rather than carry a long-lived fork.

你好 maintainers,

我们来自阿里巴巴钉钉(DingTalk)官方团队。钉钉是中国最主流的企业协作平台之一,服务数亿用户。在将 Hermes 接入钉钉工作台的过程中,我们对钉钉适配器做了一轮系统性增强 —— 补齐了主动消息下发、多媒体上传链路、互动卡片限流、引用消息解析等生产级能力,并修复了几个会影响多适配器共存稳定性的关键 Bug。希望这些改动能让社区里其他想接钉钉的同学少踩坑,也欢迎基于反馈继续迭代。


Summary

Major capability uplift for the DingTalk platform adapter — a 15-dimension enhancement covering proactive messaging, media pipeline, ACL, throttling, and inbound quoted-reply handling. This brings the DingTalk integration to production-grade for first-class workspace deployments. It also fixes a latent websockets monkey-patch that broke dingtalk-stream reconnects when multiple adapters ran in the same gateway process.

Nine commits, scoped:

  • feat(dingtalk): proactive messaging, media pipeline, card throttle, sethome DM fix — core capability buildout: outbound proactive send, media (image/file/audio/video) upload pipeline, interactive card throttling, and home channel DM routing.
  • fix(gateway): hydrate yaml HOME_CHANNEL to env + fix platform display names — makes yaml-configured HOME_CHANNEL actually reachable at runtime.
  • fix(send_message): document media support in schema + wire bare-path extraction — tool schema now advertises media params; gateway accepts bare paths.
  • fix(dingtalk): sampleImageMsg.photoURL expects raw @MediaID, not CDN URL — matches the DingTalk OpenAPI contract.
  • feat(dingtalk): session queue + global card QPS bucket — per-session serialization + global 20 QPS card throttle to respect DingTalk rate limits.
  • fix: keep websockets.connect() sync so dingtalk-stream's async-with works — a stray async def wrapper was globally monkey-patching websockets.connect, returning a coroutine instead of the connect object. That broke async with websockets.connect(uri) inside the dingtalk-stream SDK. Fix: keep the wrapper sync-returning.
  • test(dingtalk): cover quoted-reply extraction paths + fix(dingtalk): clarify quoted-reply placeholder copy for LLM handling — inbound repliedMsg recursive parser, with placeholders for msgTypes DingTalk stream delivers metadata-only for (picture/file/audio/video/interactiveCard).
  • feat(dingtalk): forward-compatible extraction for quoted interactiveCard — today DingTalk stream delivers repliedMsg for interactiveCards with only 4 metadata fields (msgId/senderId/msgType/createdAt), no content. We are pushing the DingTalk IM team to populate content.text / content.markdown / content.title / content.summary server-side. Once shipped, Hermes picks it up with zero code change — the extractor already reads those keys in priority order.

Test plan

  • pytest tests/gateway/test_dingtalk.py — 17/17 passing (13 original + 4 forward-compat)
  • Live-verified inbound text + quoted-text replies on an authorized DingTalk workspace
  • Live-verified proactive send via send_message_tool (markdown, text, card)
  • Live-verified gateway process stability with multiple adapters running concurrently (validates the websockets monkey-patch fix)
  • Reviewer to spot-check whether the extraction fallback strings ([图片], [interactiveCard消息], etc.) match project copy conventions

Notes for reviewers

  • The interactiveCard forward-compat branch is intentionally defensive — it's the only shape the DingTalk IM team has hinted at supporting, but the priority order (text → markdown → title → summary) is our guess. Happy to adjust based on what the IM team actually ships.
  • No breaking changes to existing platform adapters; all changes are additive or scoped to the DingTalk adapter.
  • Since this comes from the DingTalk official team, we're committed to maintaining the adapter going forward — happy to be on-call for follow-up issues or future API changes on our side.

@meng93

meng93 commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

good change

…tool media

- Add DingTalk access-token management (_dingtalk_fetch_access_token,
  _dingtalk_fetch_oapi_token) with process-wide cache, 5 min safety
  margin, and asyncio.Lock for concurrent safety.
- Add _dingtalk_upload_media() — upload local files to /media/upload
  (legacy OAPI token) with mime auto-detection and 20 MB guard.
- Add _dingtalk_classify_chat_id() — route on chat_id shape:
  'cidXXX==' → group /groupMessages/send, plain staffId or
  'user:<staffId>' → 1:1 /oToMessages/batchSend.
- Add dingtalk_send_proactive() orchestrator — text + media delivery.
- In send(), fall back to dingtalk_send_proactive() when no session
  webhook is cached (proactive/cron/cross-platform delivery).
- Fix /sethome DM chat_id: persist 'user:<staffId>' instead of DM
  conversation_id so proactive DM delivery works (gateway/run.py).
- Enhance send_message_tool.py: MEDIA:<path> tag extraction, native
  DingTalk/Feishu routing with media_files support.
- Update test assertions for proactive-send error path.
@spike2204

Copy link
Copy Markdown

Hi @alt-glitch, thanks so much for the quick labels and review across all 8 PRs — really appreciate the attention! 🙏

I wanted to provide some context on these PRs and how they relate to the earlier series (#14333#14336).

Who we are

We're from the DingTalk (钉钉) team at Alibaba — the same team that builds and maintains the DingTalk open platform, the dingtalk-stream SDK, and the AI Card APIs that these PRs integrate with. We've been running Hermes internally with DingTalk as the primary gateway, and these PRs represent battle-tested fixes and features from our production deployment.

Why the re-submission

The original 4 PRs (#14333#14336 by @PeterGuy326) were stacked — each one included all changes from prior PRs, making them hard to review independently. With 4.6k+ open PRs in this repo, stacked PRs are easily buried or create merge conflicts.

These 8 PRs are intentionally fine-grained replacements, each:

  • ✅ Self-contained against main (no inter-PR dependencies)
  • ✅ Independently reviewable and mergeable in any order
  • ✅ Focused on a single concern (one bug fix, one feature, etc.)

PR mapping

Original stacked PR → New fine-grained PRs
#14333 (reliability fixes) #17364 (websockets proxy) + #17365 (card QPS) + #17366 (inbound queue)
#14334 (quoted reply) #17368 (quoted-reply extraction + 12 tests)
#14335 (media pipeline) #17369 (rich-media inbound pipeline)
#14336 (proactive send) #17367 (gateway infra) + #17370 (proactive messaging) + #17371 (send_message media)

Regarding overlaps with other PRs

Next steps

Once these 8 PRs are reviewed, we'll close the original stacked PRs (#14333#14336). We're also happy to:

  • Address any review feedback promptly
  • Add more tests if needed
  • Help with any merge conflict resolution

Looking forward to getting these merged — they'll significantly improve the DingTalk adapter's reliability and feature parity. Let us know if you'd prefer a different organization! 🚀

@jeikl

jeikl commented May 7, 2026

Copy link
Copy Markdown

Hi @alt-glitch, thanks so much for the quick labels and review across all 8 PRs — really appreciate the attention! 🙏

I wanted to provide some context on these PRs and how they relate to the earlier series (#14333#14336).

Who we are

We're from the DingTalk (钉钉) team at Alibaba — the same team that builds and maintains the DingTalk open platform, the dingtalk-stream SDK, and the AI Card APIs that these PRs integrate with. We've been running Hermes internally with DingTalk as the primary gateway, and these PRs represent battle-tested fixes and features from our production deployment.

Why the re-submission

The original 4 PRs (#14333#14336 by @PeterGuy326) were stacked — each one included all changes from prior PRs, making them hard to review independently. With 4.6k+ open PRs in this repo, stacked PRs are easily buried or create merge conflicts.

These 8 PRs are intentionally fine-grained replacements, each:

  • ✅ Self-contained against main (no inter-PR dependencies)
  • ✅ Independently reviewable and mergeable in any order
  • ✅ Focused on a single concern (one bug fix, one feature, etc.)

PR mapping

Original stacked PR → New fine-grained PRs
#14333 (reliability fixes) #17364 (websockets proxy) + #17365 (card QPS) + #17366 (inbound queue)
#14334 (quoted reply) #17368 (quoted-reply extraction + 12 tests)
#14335 (media pipeline) #17369 (rich-media inbound pipeline)
#14336 (proactive send) #17367 (gateway infra) + #17370 (proactive messaging) + #17371 (send_message media)

Regarding overlaps with other PRs

Next steps

Once these 8 PRs are reviewed, we'll close the original stacked PRs (#14333#14336). We're also happy to:

  • Address any review feedback promptly
  • Add more tests if needed
  • Help with any merge conflict resolution

Looking forward to getting these merged — they'll significantly improve the DingTalk adapter's reliability and feature parity. Let us know if you'd prefer a different organization! 🚀

老哥 你们出个官方的hermes 分支吧 官方的钉钉插件不支持发送图片太难受了

@HongHaiyang

Copy link
Copy Markdown

Would be great if this could be merged soon — as a daily user, I really need this feature.

Resolves pyproject.toml conflict in [project.optional-dependencies]:
- Align to upstream's exact-pin policy (2026-05-12 supply-chain tightening)
- Keep PR-added media-parsing deps with explicit pins:
    python-docx==1.2.0, pdfplumber==0.11.9, openpyxl==3.1.5
- Adopt upstream pins for shared deps:
    dingtalk-stream==0.24.3, alibabacloud-dingtalk==2.2.42,
    qrcode==7.4.2, lark-oapi==1.5.3

Regenerated uv.lock; uv lock --check passes.
Tests: tests/gateway/test_dingtalk.py 76/76,
       tests/hermes_cli/test_dingtalk_auth.py 15/15.
@PeterGuy326 PeterGuy326 requested a review from a team May 14, 2026 07:16

@austinpickett austinpickett left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix merge conflicts and use .github/PULL_REQUEST_TEMPLATE.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/dingtalk DingTalk adapter type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants