Skip to content

feat: bookmark knowledge base extraction and search#16

Merged
0xNyk merged 1 commit into
mainfrom
feat/bookmark-kb
Mar 16, 2026
Merged

feat: bookmark knowledge base extraction and search#16
0xNyk merged 1 commit into
mainfrom
feat/bookmark-kb

Conversation

@0xNyk

@0xNyk 0xNyk commented Mar 16, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add bookmark-kb command (bkb alias) — turns X bookmarks into a searchable, categorized knowledge base
  • extract: Fetch bookmarks via OAuth, extract structured knowledge via Grok AI (batched, incremental, canonical topic list for consistency)
  • search: Local substring search with --topic, --min-importance, --limit, --json filters; --remote for semantic search via xAI Collections
  • sync: Export topic markdown files locally; --cloud flag uploads to xAI Collections
  • topics / status: View discovered topics and KB stats (--json supported)
  • Fix collections.ts: .env.local priority, skip placeholder keys, correct Management API field names (collection_name), direct document upload to collections endpoint

Files changed

File Change
lib/bookmark_kb.ts New — main implementation (547 lines)
lib/collections.ts Fixed env loading, API field names, added collectionsUploadDocument
lib/costs.ts Added bookmark_kb_extract cost rate
xint.ts Added routing + engagement policy

Test plan

  • bookmark-kb extract --limit 5 — extracts knowledge from 5 bookmarks
  • bookmark-kb search "agent" — returns matching extractions with evaluation + source links
  • bookmark-kb search "agent" --topic "ai agents" --min-importance 3 --json — filtered JSON output
  • bookmark-kb sync — exports 7 topic .md files locally
  • bookmark-kb sync --cloud — uploads 7/7 to xAI Collections
  • bookmark-kb topics / topics --json — topic list with counts
  • bookmark-kb status / status --json — KB stats with sync info
  • Re-running extract skips already-processed tweets
  • bun run typecheck passes

Add `bookmark-kb` command that turns X bookmarks into a searchable,
categorized knowledge base using Grok AI extraction.

Subcommands:
- extract: fetch bookmarks, extract knowledge via Grok (batched, incremental)
- search: local substring search with topic/importance filters
- sync: export as markdown files, optionally upload to xAI Collections (--cloud)
- topics: list discovered topics with counts
- status: show KB stats and sync info

Also fixes collections.ts: env key loading (.env.local priority, skip
placeholders), correct Management API field names (collection_name),
and direct document upload to collections endpoint.
@0xNyk 0xNyk merged commit b686de8 into main Mar 16, 2026
@0xNyk 0xNyk deleted the feat/bookmark-kb branch March 16, 2026 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant