Skip to content

✨ feat(desktop): gate screen capture on macOS permission and add overlay hint#14097

Merged
Innei merged 3 commits into
canaryfrom
feat/screen-capture-ux
Apr 23, 2026
Merged

✨ feat(desktop): gate screen capture on macOS permission and add overlay hint#14097
Innei merged 3 commits into
canaryfrom
feat/screen-capture-ux

Conversation

@Innei

@Innei Innei commented Apr 23, 2026

Copy link
Copy Markdown
Member

πŸ’» Change Type

  • ✨ feat
  • πŸ› fix
  • ♻️ refactor
  • πŸ’„ style
  • πŸ‘· build
  • ⚑️ perf
  • βœ… test
  • πŸ“ docs
  • πŸ”¨ chore

πŸ”€ Description of Change

Two small UX improvements for the desktop Screen Capture (Shield Composer) flow:

  1. macOS Screen Recording permission gate β€” before opening the overlay, check getScreenCaptureStatus() on macOS. If not granted, show a native dialog explaining the requirement with an Open Settings button that deep-links to System Settings via requestScreenCaptureAccess(). Non-macOS platforms are unaffected. New i18n keys under dialog.screenCaptureAccess.* (en + zh-CN).

  2. Overlay hint pill β€” add a bottom-left hint that teaches the three core interactions:

    [ Hover β€’ Select active window β”‚ Drag β€’ Capture a region β”‚ Esc β€’ Exit ]
    

    The pill reuses the existing WindowTag visual language (slate bg, blur(12px), pill radius, same border/shadow), so it feels like the same building block as the hovered-window label. Single pill with three Key β€’ Label groups separated by thin vertical rules. Font stack simplified to system-ui, sans-serif.

    Visibility: hidden during drag and after a selection is committed so it doesn't compete with the ChatPanel. Capture safety is already handled by the existing withOverlayHidden (overlay window opacity is forced to 0 before each native capture), so the hint never appears in the screenshot.

πŸ§ͺ How to Test

  • Tested locally
  • Added/updated tests
  • No tests needed

Permission gate (macOS):

  1. Revoke Screen Recording permission for the app in System Settings
  2. Trigger a capture β€” dialog should appear with Open Settings / Later
  3. Click Open Settings β†’ lands in the correct pane
  4. Grant permission β†’ next capture opens the overlay normally

Overlay hint:

  1. Trigger capture β€” hint visible at bottom-left
  2. Hover a window β€” tag appears at top of window, hint remains
  3. Start a drag selection β€” hint disappears
  4. After a selection commits β€” hint stays hidden, ChatPanel shown
  5. Capture a screenshot β€” verify the PNG does not contain the hint

Tests:

  • ScreenCaptureManager.test.ts β€” permission gate paths
  • Overlay hint is pure rendering; no new unit tests added

πŸ“ Additional Information

No breaking changes. All new code is macOS-gated (permission) or overlay-scoped (hint).

Innei added 2 commits April 23, 2026 22:56
Prompt a native dialog before opening the capture overlay when macOS
Screen Recording permission is missing, with an Open Settings button
that deep-links to System Settings.
Bottom-left pill with three grouped hints (hover to pick a window, drag
to crop a region, Esc to exit), sharing the WindowTag pill language.
Hidden during drag and after a selection so it doesn't clutter.
@vercel

vercel Bot commented Apr 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lobehub Error Error Apr 23, 2026 4:14pm

Request Review

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @Innei, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. electron Electron/Desktop app specific i18n labels Apr 23, 2026
@codecov

codecov Bot commented Apr 23, 2026

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 86.11%. Comparing base (e4b15ca) to head (910a6c2).
⚠️ Report is 2 commits behind head on canary.

Additional details and impacted files
@@             Coverage Diff             @@
##           canary   #14097       +/-   ##
===========================================
+ Coverage   67.31%   86.11%   +18.80%     
===========================================
  Files        2148      608     -1540     
  Lines      184525    51200   -133325     
  Branches    22595     8641    -13954     
===========================================
- Hits       124213    44092    -80121     
+ Misses      60187     6983    -53204     
  Partials      125      125               
Flag Coverage Ξ”
app ?
database 92.23% <ΓΈ> (ΓΈ)
packages/agent-runtime 79.82% <ΓΈ> (ΓΈ)
packages/context-engine 83.10% <ΓΈ> (ΓΈ)
packages/conversation-flow 92.40% <ΓΈ> (ΓΈ)
packages/file-loaders 87.02% <ΓΈ> (ΓΈ)
packages/memory-user-memory 74.74% <ΓΈ> (ΓΈ)
packages/model-bank 99.89% <ΓΈ> (ΓΈ)
packages/model-runtime 84.07% <ΓΈ> (ΓΈ)
packages/prompts 70.14% <ΓΈ> (ΓΈ)
packages/python-interpreter 92.90% <ΓΈ> (ΓΈ)
packages/ssrf-safe-fetch 0.00% <ΓΈ> (ΓΈ)
packages/utils 88.41% <ΓΈ> (ΓΈ)
packages/web-crawler 88.66% <ΓΈ> (ΓΈ)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Ξ”
Store βˆ… <ΓΈ> (βˆ…)
Services βˆ… <ΓΈ> (βˆ…)
Server βˆ… <ΓΈ> (βˆ…)
Libs βˆ… <ΓΈ> (βˆ…)
Utils 93.47% <ΓΈ> (+13.37%) ⬆️
πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The first test case was timing out (~9.5s) because execAgent makes a
real HTTP request to market.lobehub.com via MarketService.getLobehubSkillManifests().
Mock MarketService to return empty skill manifests, eliminating the
network dependency that caused the cold-start timeout in CI.
@Innei Innei force-pushed the feat/screen-capture-ux branch from 7b2387f to cb8d74d Compare April 23, 2026 16:03
@Innei Innei merged commit 7955a43 into canary Apr 23, 2026
30 of 31 checks passed
@Innei Innei deleted the feat/screen-capture-ux branch April 23, 2026 16:06
arvinxx added a commit that referenced this pull request Apr 27, 2026
# πŸš€ LobeHub v2.1.53 (20260427)

**Release Date:** April 27, 2026
**Since v2.1.52:** 194 merged PRs Β· 17 contributors

> Introduce Heterogeneous Agent β€” Claude Code and Codex run as
first-class desktop runtimes, paired with a new Agent Signal package,
sharper desktop UX, and a wave of flagship model additions.

---

## ✨ Highlights

- **Introduce Heterogeneous Agent** β€” Claude Code and Codex run as
first-class desktop agents: subagent rendering, partial-message
streaming, multi-turn resume, terminal error surfacing, rich tool
inspectors, and runtime polish. (#14162, #13754, #14067, #14001, #13970,
#13942)
- **Screen capture & Quick Chat tray** β€” New desktop screen capture
overlay (macOS permission-gated) with Quick Chat tray and upload
pipeline improvements; chat input auto-focuses on overlay mount.
(#13818, #14097, #14105)
- **Desktop topic & tab UX** β€” Dedicated topic popup window with
cross-window sync, Cmd+W/Cmd+T tab shortcuts, TabBar polish, recent
working directories expanded to 20, and human approval notifications.
(#13957, #13983, #13972, #14036, #14092)
- **Git workflow built-in** β€” One-click pull/push from the branch chip,
ahead/behind badge, and submodule/worktree repo detection. (#14041,
#13980, #13978)
- **Agent Signal package** β€” New `@lobechat/agent-signal` runtime for
dynamic memory feedback signals, with OTel metrics and self-iteration in
Lab. (#14157, #14170, #14159, #14169, #14187)
- **New models** β€” Claude Opus 4.7 with `xhigh` effort tier, GPT-5.5,
DeepSeek V4 Flash/Pro with reasoning slider, Kimi K2.6, MiMo-V2.5/Pro,
gpt-image-2, Qwen3.6 Flash/Plus, and Pixverse-c1. (#13903, #14147,
#14114, #14004, #14089, #14039, #13923)
- **New providers** β€” OpenCode Zen, OpenCode Go, and Azure OpenAI Router
runtime. (#13943, #14064, #13823)
- **Mobile settings overhaul** β€” Full settings menu and responsive
profile layout for mobile. (#14019)

---

## πŸ—οΈ Heterogeneous Agent

- Claude Code runtime, working-directory awareness, and sidebar polish.
(#13970)
- CC subagent rendering with persistent streamed text; parallel-tool
orphan fix. (#14001, #13968, #14024)
- Per-step usage persisted to each step assistant message. (#13964)
- Per-phase workflow expand defaults; full-expand toggle with
three-level expansion. (#14171, #13906)
- Hetero-mode actions bar; tool inspector polish. (#13963, #14034,
#14030)
- Codex desktop integration with rich tool rendering and devtools
preview. (#14067, #14100)
- Codex terminal error surfacing and CLI output tracing. (#14166)
- Tighten `isCanUseVision` default and add aggregator fallback. (#14172)
- Persist `ccSessionId` in topic metadata for CC multi-turn resume.
(#13902)
- CC account card, topic filter, and integration polish. (#13955,
#13942, #13950)
- Token-level deltas streamed via `--include-partial-messages`. (#13929)

---

## 🧠 Agent Signal & Self-Iteration

- New `@lobechat/agent-signal` package with dynamic feedback signals.
(#14157)
- AgentSignalRuntime wired through agent-tracing and observability-otel
metrics. (#14170, #14159)
- Self-iteration feature flag added to Lab; front-side flag check.
(#14169, #14186)
- Signal policy for receiving memory feedback dynamically. (#14187)

---

## πŸ’¬ Conversation

- Queue follow-up sends during running CC turns. (#14179)
- Persist per-topic chat scroll position; pin user message + fold long
messages. (#14191, #14056)
- Inline resend when editing last user message. (#14080)
- Disable first-block markdown streaming to prevent flicker. (#14193,
#13904)
- Prevent Markdown stream replay when vlist remounts streaming items.
(#14086)
- Stop repinning after manual scroll; unify scroll-to-user + spacer
hooks. (#14099, #14132)

---

## πŸ“± Platforms & Integrations

### Desktop / Electron

- Screen capture overlay, Quick Chat tray, and upload pipeline
improvements. (#13818)
- macOS permission gate for screen capture; auto-focus chat panel input.
(#14097, #14105)
- Dedicated topic popup window with cross-window sync. (#13957)
- TabBar polish: `+` button for new topic, dark theme blend, close icon
by default. (#13972, #14203, #13973)
- Recent working directories expanded from 5 to 20; submodule/worktree
repo detection. (#14036, #13978)
- Cmd+W / Cmd+T tab shortcuts and global shortcut consolidation.
(#13983, #13880)
- Linux icon configuration; human approval desktop notifications.
(#14042, #14092)

### Git Workflow

- One-click pull/push from branch chip; ahead/behind badge with
refactored GitCtr. (#14041, #13980)

### Mobile

- Full settings menu and responsive profile layout. (#14019)
- Agent route added to mobile router; mobile agent topic route
registered. (#14103, #14158)
- Session list skeleton row layout corrected. (#14040)

### Bot / Messaging

- DM strategy support; bot emoji and markdown render optimization.
(#14201, #14091, #14140)
- Slack webhook fix; bot platform setup guide reference. (#14052,
#14121)

---

## πŸ€– Models & Providers

### New models

- **Claude Opus 4.7** with `xhigh` effort tier; strip temperature/top_p.
(#13903, #13909)
- **GPT-5.5**. (#14147)
- **DeepSeek V4** Flash/Pro cards with reasoning slider; cache-hit and
Pro discount pricing. (#14114, #14209, #14196, #14131)
- **Kimi K2.6** model with LobeHub-hosted card. (#14004, #14006)
- **MiMo-V2.5 / V2.5-Pro**. (#14089)
- **gpt-image-2**, **Qwen3.6 Flash/Plus**, **Pixverse-c1**. (#14039,
#13923)

### New providers

- **OpenCode Zen** and **OpenCode Go** with env-var support. (#13943,
#14064)
- **Azure OpenAI Router** runtime support. (#13823)
- Model alias mapping for image and video runtimes. (#13896)
- Seedance video models migrated to Dreamina. (#14144)

### Runtime reliability

- Sanitize invalid tool_call arguments to unbreak strict providers.
(#14033)
- Tolerate null `function.name` in streaming tool_call deltas. (#14139)
- Preserve Gemini 3 `thoughtSignature` in `call_tools_batch`
normalization. (#14032)
- Downgrade `image_url` parts when target model lacks vision. (#14029)
- Preserve Cloudflare provider error context. (#14136)
- Use `safety_identifier` for OpenAI Responses API. (#14148)
- Unwrap underlying PG error in `formatErrorEventData`. (#14038)

---

## πŸ–₯️ User Experience

- **Onboarding** β€” Preset agent naming suggestions, structured hunk ops
for `updateDocument`, persona analytics snapshot, footer promotion
pipeline, wrap-up button. (#13931, #13989, #13930, #13853, #13934)
- **Document workflow** β€” Agent documents promoted as primary workspace
panel; history management and compare workflow; web-crawl docs
associated with agent documents. (#13924, #13725, #13893)
- **cmdk** β€” Agent identity surfaced on topic search results;
topic/message search scoped to current agent. (#14204, #13960)
- **Floating chat panel** and workspace improvements. (#13887)
- **Topic completion status** with dropdown action and filter. (#14005)

---

## πŸ”§ Tooling

- Redis-backed feature flag provider for runtime config. (#14098)
- Vite upgraded to 8.0.0 with Rolldown strict execution order. (#12720,
#14058)
- `@lobechat/model-bank` automated npm release with provenance. (#14015,
#14017, #14018)
- Skill activation fallback when `activateTools` cannot find identifier.
(#14010)
- Cron tool: timezone and existing jobs injected into system prompt;
clarified `lobe-gtd` and `lobe-cron` descriptions. (#14012, #14013)

---

## πŸ”’ Security & Reliability

- **Security:** uuid bumped to v14 (advisory). (#14083)
- **Security:** validate avatar URL and scope old-avatar deletion to
owner. (#13982)
- **Security:** clear OIDC sessions on better-auth signout; return 401
(not 500) for expired OIDC JWT. (#13916, #14014)
- **Reliability:** scope pending-approval check to current assistant
turn. (#14182)
- **Reliability:** sanitize heterogeneous-agent attachment cache
filenames. (#13937)
- **Reliability:** reduce subagent task status error noise. (#14026)

---

## πŸ‘₯ Contributors

Huge thanks to **17 contributors** who shipped **194 merged PRs** this
week.

@hardy Β· @shaun0927 Β· @hezhijie0327 Β· @sxjeru Β· @arvinxx Β· @Innei Β·
@tjx666 Β· @lijian Β· @neko Β· @rdmclin2 Β· @AmAzing129 Β· @sudongyuer Β·
@CanisMinor Β· @rivertwilight

Plus @lobehubbot and renovate[bot] for maintenance.

---

**Full Changelog**:
v2.1.52...v2.1.53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

electron Electron/Desktop app specific i18n size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant