Skip to content

Inline image previews corrupt terminal rendering in cmux/Ghostty #4208

@garizs

Description

@garizs

Summary

Inline image previews rendered by Pi's built-in TUI renderer corrupt the terminal display inside cmux. The same image workflow is fine in iTerm2.

This looks like Pi's current direct Kitty graphics path is fragile when running inside cmux (which exposes itself as Ghostty / libghostty). A small local probe extension that renders the same ImageContent via Kitty U=1 Unicode placeholders rendered cleanly in the same cmux session.

Environment

  • Pi: 0.73.0
  • cmux: 0.64.1 (81) [55400432c]
  • macOS: macOS on Apple Silicon
  • Terminal where it fails: cmux terminal pane
  • Terminal where it works: iTerm2
  • In cmux, Pi detects the terminal as Ghostty/Kitty-capable. Observed env in the Pi session before testing:
    • TERM=xterm-256color
    • TERM_PROGRAM=ghostty
    • GHOSTTY_RESOURCES_DIR=/Applications/cmux.app/Contents/Resources/ghostty
    • TMUX unset
    • CMUX_WORKSPACE_ID set

Relevant Pi extensions in the user's setup include custom cmux status integration, pi-permission-system, pi-interview, etc., but the repro is specifically the built-in read tool returning an image and Pi rendering that image in the tool result.

Reproduction

  1. Start Pi inside a cmux terminal pane.
  2. Ask Pi to read a local PNG image, e.g. /var/folders/.../clipboard-....png.
  3. Pi calls the built-in read tool.
  4. read returns an ImageContent block.
  5. Pi's built-in TUI tool-result renderer displays the image inline.

Actual behavior

The terminal rendering becomes visually corrupted in cmux:

  • stale terminal regions are duplicated/repainted;
  • large chunks of previous output appear in the wrong places;
  • the image preview initially appears in a corrupted layout and later disappears after several seconds;
  • cmux refresh-surfaces did not fix the corrupted display.

Forcing Pi to use an iTerm-like env (TERM_PROGRAM=iTerm.app, ITERM_SESSION_ID=..., unsetting GHOSTTY_RESOURCES_DIR) did not fix the issue in cmux; the image rendered incorrectly / did not render properly in a different way.

Expected behavior

Inline image previews in tool results should render stably inside cmux, or Pi should choose a safer rendering path for cmux/Ghostty.

Diagnostic result

I created a small local Pi extension that listens for tool_result, extracts any ImageContent, and renders it in a Pi widget using Kitty Unicode placeholder mode (U=1) instead of Pi's built-in direct Kitty renderer.

In the same cmux session, with the same image content:

  • Pi built-in renderer: corrupts cmux display.
  • Local U=1 placeholder probe renderer: image rendered cleanly without corrupting the terminal.

This suggests the issue is likely in Pi/pi-tui's direct Kitty image rendering path for Ghostty/cmux, not in the image data itself.

Suspected cause

Pi's current built-in image renderer appears to use direct Kitty graphics commands for Ghostty/cmux, roughly the a=T,f=100,q=2,c=...,r=... path. This seems to be fragile in cmux's libghostty surface during TUI rerenders/scrollback updates.

A safer path may be to use Kitty Unicode placeholders (U=1) with stable image IDs for Ghostty/cmux, similar to the strategy used by rielj/pi-image-preview.

Request

Could Pi/pi-tui either:

  1. switch Ghostty/cmux image rendering from direct Kitty placements to stable-ID U=1 placeholder rendering, or
  2. expose a global ImageContent display renderer hook so extensions can replace only the image display path without overriding built-in tools like read?

I prefer not to override the built-in read tool because Pi frequently improves built-in tool behavior, and overriding read would pin/fork that behavior just to replace the image renderer.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions