🐛 fix: reduce agent document context latency by tjx666 · Pull Request #15436 · lobehub/lobehub

tjx666 · 2026-06-03T12:36:06Z

💻 Change Type

🔗 Related Issue

N/A

🔀 Description of Change

This PR reduces chat preparation and send-persistence latency when an agent has many attached documents, while keeping send-time hydration correct.

Adds a context-specific agentDocument.getContextDocuments endpoint for chat context hydration.
Avoids loading documents.content for progressive documents; only policyLoad === 'always' documents keep full content.
Carries contentCharCount so progressive index rendering can still show document sizes without requiring full content.
Narrows skill runtime document loading to skill-managed documents.
Replaces a full document load used for a boolean server-agent check with hasDocuments.
Prefetches agent document context and available-agent context before send-time preparation; send-time ensureAgentDocuments remains the correctness fallback.
Isolates available-agent prefetch in ConversationProvider so SWR refreshes do not re-render ChatList / VirtualizedList and disturb scroll anchoring.
Handles empty topic-reference input without making an unnecessary reference lookup.
Adds a server-side simple new-topic fast path that resolves session context, creates the topic, creates the user and assistant messages, and optionally touches the agent in one DB service call.
Keeps simple-turn group messages detached from session rows so the raw SQL fast path matches MessageModel group-message normalization.
Hardens SVG sanitization by stripping event-handler attributes before and after DOMPurify, covering DOM adapter differences seen in CI.
Updates stale database path references from src/database to packages/database.
Updates agent validation guidance in AGENTS.md.

Security review: this only changes open-source agent document/context/store/server persistence behavior and does not expose cloud-specific business logic.

🧪 How to Test

Tested locally
Added/updated tests
No tests needed

Commands run:

bunx vitest run --silent='passed-only' src/server/services/agentDocuments/index.test.ts src/services/agentDocument.test.ts src/services/chat/chat.test.ts src/store/agent/slices/agent/action.test.ts src/store/chat/slices/topic/action.test.ts src/store/chat/slices/aiChat/actions/__tests__/streamingExecutor.test.ts

cd packages/database
bunx vitest run --silent='passed-only' src/models/agentDocuments/__tests__/agentDocument.test.ts

cd packages/context-engine
bunx vitest run --silent='passed-only' src/providers/__tests__/AgentDocumentInjector.test.ts

cd packages/utils
bunx vitest run --coverage --silent='passed-only' src/client/sanitize.test.ts

bunx vitest run --silent='passed-only' src/features/Conversation/ConversationProvider.test.tsx src/store/agent/slices/agent/action.test.ts src/services/chat/mecha/contextEngineering.test.ts src/store/agent/slices/knowledge/action.test.ts src/store/agent/slices/plugin/action.test.ts

bunx vitest run --silent='passed-only' src/server/services/aiChat/index.test.ts src/server/routers/lambda/__tests__/aiChat.test.ts src/store/chat/slices/aiChat/actions/__tests__/conversationLifecycle.test.ts

rg -n 'src/database|lobehub/src/database|\.\/src\/database|\.\/lobehub\/src\/database' -S --glob '!node_modules/**' --glob '!*.lock' --glob '!public/**'

Also checked VS Code diagnostics and git diff --check.

📸 Screenshots / Videos

N/A

📝 Additional Information

This does not add a post-runtime document refresh. The optimization is prefetch plus a send-time fallback: if the cache is warm, chat preparation uses it; if not, ensureAgentDocuments still hydrates the required context before sending.

For ordinary new-topic sends, the server persistence fast path is intentionally scoped to the simplest case only; other sends keep the existing multi-step flow.

vercel · 2026-06-03T12:36:13Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
lobehub	Ready	Preview, Comment	Jun 4, 2026 8:27am

sourcery-ai

Sorry @tjx666, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

codecov · 2026-06-03T12:39:44Z

Codecov Report

❌ Patch coverage is 96.78801% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.94%. Comparing base (a3a08c2) to head (da3429b).
⚠️ Report is 12 commits behind head on canary.

Additional details and impacted files

@@             Coverage Diff             @@
##           canary   #15436       +/-   ##
===========================================
- Coverage   89.70%   70.94%   -18.76%     
===========================================
  Files         862     3225     +2363     
  Lines      105731   320651   +214920     
  Branches    10246    28223    +17977     
===========================================
+ Hits        94841   227475   +132634     
- Misses      10714    92999    +82285     
- Partials      176      177        +1

Flag	Coverage Δ
app	`61.71% <96.76%> (?)`
database	`92.53% <98.05%> (+0.03%)`	⬆️
packages/agent-runtime	`80.48% <ø> (ø)`
packages/builtin-tool-lobe-agent	`18.52% <ø> (ø)`
packages/context-engine	`84.18% <66.66%> (+0.01%)`	⬆️
packages/conversation-flow	`91.29% <ø> (ø)`
packages/file-loaders	`87.89% <ø> (ø)`
packages/memory-user-memory	`74.99% <ø> (ø)`
packages/model-bank	`99.99% <ø> (ø)`
packages/model-runtime	`84.51% <ø> (ø)`
packages/prompts	`72.49% <ø> (ø)`
packages/python-interpreter	`92.90% <ø> (ø)`
packages/ssrf-safe-fetch	`0.00% <ø> (ø)`
packages/types	`35.36% <ø> (ø)`
packages/utils	`84.98% <95.83%> (-3.79%)`	⬇️
packages/web-crawler	`88.08% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Store	`68.53% <92.85%> (∅)`
Services	`54.76% <66.66%> (∅)`
Server	`72.09% <98.40%> (∅)`
Libs	`57.15% <ø> (∅)`
Utils	`81.48% <100.00%> (-18.52%)`	⬇️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 01eae69a0c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

tjx666 · 2026-06-03T15:49:33Z

@codex review latest code

chatgpt-codex-connector · 2026-06-03T15:54:47Z

Codex Review: Didn't find any major issues. Nice work!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

tjx666 · 2026-06-03T19:06:46Z

@codex review it

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a4e2df9a99

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-06-03T19:12:21Z

+              "payload"."provider",
+              "payload"."parent_id",
+              ${this.userId},
+              "resolved_context"."session_id",


Preserve group-message scoping in fast-path inserts

When a group chat send takes this simple fast path with both groupId and sessionId, this direct insert persists resolved_context.session_id on the group messages. The regular MessageModel path intentionally clears sessionId whenever groupId is set (packages/database/src/models/message.ts:1691-1692), so these rows can later match ordinary session-scoped queries in addition to group queries, polluting conversation history. Please mirror that normalization for both simple fast paths by writing NULL for session_id when groupId is present.

Useful? React with 👍 / 👎.

- cli-backend-testing: use $CLI shorthand consistently, remove redundant "When Server Restart is NOT Needed" section and Credential Isolation table (already covered by the restart table and cli skill) - zustand: compress "Slices That Don't Need set" from 61 to 22 lines by merging overlapping Rules of thumb / Do-Don't sections

sourcery-ai Bot reviewed Jun 3, 2026

View reviewed changes

dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jun 3, 2026

vercel Bot deployed to Preview June 3, 2026 12:39 View deployment

chatgpt-codex-connector Bot reviewed Jun 3, 2026

View reviewed changes

Comment thread src/store/agent/slices/agent/action.ts Outdated

vercel Bot deployed to Preview June 3, 2026 13:17 View deployment

vercel Bot deployed to Preview June 3, 2026 16:00 View deployment

github-advanced-security AI found potential problems Jun 3, 2026

View reviewed changes

Comment thread packages/utils/src/client/sanitize.ts Fixed

Comment thread packages/utils/src/client/sanitize.ts Fixed

tjx666 force-pushed the fix/agent-doc-context-prefetch branch from 72187e2 to 0f9c5d1 Compare June 3, 2026 16:53

vercel Bot deployed to Preview June 3, 2026 16:59 View deployment

tjx666 added 11 commits June 4, 2026 01:13

🐛 fix: reduce agent document context hydration payload

25f39a4

⚡ perf: prefetch agent document context

f95ddcb

⚡ perf: trim agent context editor data

735fdab

⚡ perf: slim agent document context query

7eeaa96

⚡ perf: prefetch available agent context

315f6ee

🐛 fix: invalidate available agent context cache

1b80b6a

📝 docs: clarify agent validation guidance

ba10143

✅ test: cover agent document context size

967a001

✅ test: mock available agent context limit

89647df

🐛 fix: strip svg event handler attributes

d3273c3

🐛 fix: isolate available agent prefetch

c347f24

tjx666 force-pushed the fix/agent-doc-context-prefetch branch from 0f9c5d1 to c347f24 Compare June 3, 2026 17:13

vercel Bot deployed to Preview June 3, 2026 18:08 View deployment

tjx666 added 3 commits June 4, 2026 02:51

🐛 fix: handle empty topic references

bebcac4

⚡️ perf: fast path simple chat persistence

553abb1

🔧 chore: update database path references

a4e2df9

dosubot Bot removed the size:L This PR changes 100-499 lines, ignoring generated files. label Jun 3, 2026

dosubot Bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Jun 3, 2026

vercel Bot deployed to Preview June 3, 2026 19:08 View deployment

chatgpt-codex-connector Bot reviewed Jun 3, 2026

View reviewed changes

🐛 fix: normalize group simple turn sessions

21f272f

vercel Bot deployed to Preview June 4, 2026 03:32 View deployment

tjx666 added 3 commits June 4, 2026 14:14

⚡️ perf: preserve fast path timing markers

c6ff844

⚡️ perf: unify timing stage markers

c794142

vercel Bot deployed to Preview June 4, 2026 07:02 View deployment

♻️ refactor: express ai chat fast paths with drizzle

da3429b

tjx666 merged commit bab3ff4 into canary Jun 4, 2026
34 of 35 checks passed

tjx666 deleted the fix/agent-doc-context-prefetch branch June 4, 2026 08:23

vercel Bot deployed to Preview June 4, 2026 08:27 View deployment

tjx666 mentioned this pull request Jun 4, 2026

♻️ refactor(ai-chat): remove simple turn fast path #15471

Merged

12 tasks

This was referenced Jun 9, 2026

🚀 release: 20260610 #15619

Closed

🚀 release: 20260610 #15641

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

🐛 fix: reduce agent document context latency#15436

🐛 fix: reduce agent document context latency#15436
tjx666 merged 19 commits into
canaryfrom
fix/agent-doc-context-prefetch

tjx666 commented Jun 3, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Jun 3, 2026 •

edited

Loading

Uh oh!

sourcery-ai Bot left a comment

Uh oh!

codecov Bot commented Jun 3, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

tjx666 commented Jun 3, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jun 3, 2026

Uh oh!

Uh oh!

Uh oh!

tjx666 commented Jun 3, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Jun 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

tjx666 commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💻 Change Type

🔗 Related Issue

🔀 Description of Change

🧪 How to Test

📸 Screenshots / Videos

📝 Additional Information

Uh oh!

vercel Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sourcery-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

tjx666 commented Jun 3, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jun 3, 2026

Uh oh!

Uh oh!

Uh oh!

tjx666 commented Jun 3, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tjx666 commented Jun 3, 2026 •

edited

Loading

vercel Bot commented Jun 3, 2026 •

edited

Loading

codecov Bot commented Jun 3, 2026 •

edited

Loading