feat: Major improvements to WebSocket reliability, session management, and UI#14
feat: Major improvements to WebSocket reliability, session management, and UI#14
Conversation
Remove the EventBuffer persistence queue and implement immediate event persistence. Events are now persisted as soon as they are received from ACP, preserving the sequence numbers assigned at streaming time. Changes: - Add Store.RecordEvent() that preserves pre-assigned seq numbers - Add MaxSeq field to Metadata to track highest persisted seq - Add Recorder.RecordEventWithSeq() for immediate persistence - Update all BackgroundSession callbacks to persist immediately - Remove EventBuffer, periodic persistence timer, and buffer flush logic - Remove ~200 lines of buffer management code - Update mcpserver interface to use GetMaxAssignedSeq() Benefits: - Simpler architecture with no buffer management - Consistent seq numbers between streaming and storage - Better crash resilience (no data loss window) - Reduced code complexity All tests pass.
Add formatACPError() function that transforms ACP errors into user-friendly messages. Detects common error patterns and provides actionable guidance: - Timeout errors: Suggests breaking request into smaller steps - Connection errors: Explains agent process may have crashed - Rate limits: Advises waiting before retrying - Context cancelled: Simple retry message Also includes ACP restart tracking and improved nextSeq initialization.
There was a problem hiding this comment.
Pull request overview
This PR delivers a broad reliability and usability upgrade across Mitto, including improved WebSocket/session behavior, restricted runner configuration, enhanced markdown conversion (Mermaid + URL-in-backticks), new CLI tooling, and expanded documentation/testing.
Changes:
- Added Mermaid fenced-block support and expanded markdown conversion/linkification test coverage.
- Introduced restricted runner configuration (global/agent/workspace) and ACP process launching through restricted runners.
- Added new CLI utilities (
mitto config create, tools scaffolding), logging directory support, and extensive docs reorganization.
Reviewed changes
Copilot reviewed 129 out of 309 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/conversion/url_integration_test.go | Adds end-to-end tests for URL linkification inside inline code. |
| internal/conversion/testdata/ordered_list_with_blank_lines.md | Golden input for ordered lists with blank lines. |
| internal/conversion/testdata/ordered_list_with_blank_lines.html | Golden expected HTML output for ordered lists with blank lines. |
| internal/conversion/testdata/ordered_list_spaced.md | Golden input for spaced ordered lists. |
| internal/conversion/testdata/ordered_list_spaced.html | Golden expected HTML output for spaced ordered lists. |
| internal/conversion/testdata/mermaid_diagram.md | Golden input for Mermaid fenced blocks. |
| internal/conversion/testdata/mermaid_diagram.html | Golden expected HTML output for Mermaid fenced blocks. |
| internal/conversion/markdown_test.go | Adds a unit test ensuring Mermaid survives sanitization. |
| internal/conversion/markdown.go | Enables Mermaid extension and expands inline-formatting heuristics. |
| internal/conversion/example_test.go | Adds Go example demonstrating URL-in-backticks behavior. |
| internal/conversion/demo_output_test.go | Adds a demo test that prints HTML output for documentation purposes. |
| internal/config/workspaces.go | Adds restricted_runner workspace setting, validation, and constants. |
| internal/config/workspace_rc_per_agent_test.go | Tests .mittorc parsing and runner-type config selection. |
| internal/config/workspace_rc.go | Adds restricted_runners support to workspace RC and lookup helper. |
| internal/config/settings.go | Adds global/per-agent restricted_runners and session archive retention setting. |
| internal/cmd/web.go | Adds config-based access log resolution and improves listener/hook port logic. |
| internal/cmd/tools_session.go | Adds mitto tools session command scaffold. |
| internal/cmd/tools.go | Adds mitto tools command scaffold. |
| internal/cmd/prompts.go | Cleans old builtin prompt markdown files before redeploy. |
| internal/cmd/config.go | Adds mitto config create to write a default ~/.mittorc. |
| internal/cmd/cli.go | Updates ACP connection creation signature and disables restricted runner for CLI. |
| internal/client/session.go | Adds load_events client API and new callbacks; adds raw WS debug hook. |
| internal/client/helpers.go | Loads events first to register as observer before streaming. |
| internal/auxiliary/manager.go | Filters ACP stdout to ignore non-JSON lines before decoding. |
| internal/appdir/appdir_test.go | Adds tests for logs dir, UI preferences path, and workspace prompts dir. |
| internal/appdir/appdir.go | Adds LogsDir/EnsureLogsDir, UI preferences path, and workspace prompts dir helper. |
| internal/acp/types_test.go | Updates attachment tests to match new constants and base-name semantics. |
| internal/acp/types.go | Adds attachment type constants and supports text/binary attachments. |
| internal/acp/jsonline_filter_test.go | Adds tests for stdout JSON line filtering behavior. |
| internal/acp/jsonline_filter.go | Implements JSONLineFilterReader to discard non-JSON stdout lines. |
| internal/acp/connection_test.go | Updates ACP connection tests for new NewConnection signature. |
| internal/acp/connection.go | Adds restricted-runner process launching, stdout filtering, and SDK logger tuning. |
| go.mod | Adds dependencies (restricted runner, MCP SDK, mermaid) and bumps Go version. |
| docs/devel/websockets/protocol-spec.md | New WebSocket protocol spec document. |
| docs/devel/websockets/README.md | New WebSocket docs index and reading order. |
| docs/devel/session-management.md | Updates session lifecycle docs for immediate persistence + seq/max_seq. |
| docs/devel/message-queue.md | Updates queue startup behavior notes and links to new WebSocket docs. |
| docs/devel/go-restricted-runner-updates.md | Documents go-restricted-runner RunWithPipes() and implications for Mitto. |
| docs/devel/follow-up-suggestions.md | Updates link to new WebSocket docs directory. |
| docs/devel/architecture.md | Updates WebSocket docs link and adds max_seq to metadata example. |
| docs/devel/README.md | Points to new WebSocket docs directory; adds MCP quick link. |
| docs/config/web/README.md | Updates setup instructions to use mitto config create. |
| docs/config/prompts.md | Documents default workspace prompts directory .mitto/prompts/ and priority. |
| docs/config/overview.md | Expands UI config docs (input font family). |
| docs/config/ext-access/tailscale.md | Adds dedicated docs for Tailscale Funnel. |
| docs/config/ext-access/other.md | Adds docs for alternative tunneling methods. |
| docs/config/ext-access/ngrok.md | Adds dedicated docs for ngrok tunneling. |
| docs/config/ext-access/cloudflare.md | Adds dedicated docs for Cloudflare Tunnel. |
| docs/config/ext-access.md | Refactors external access docs into provider sub-pages. |
| docs/config/README.md | Updates config docs to reference mitto config create. |
| docs/README.md | Updates docs landing page references to new config creation workflow and WS docs. |
| config/prompts/builtin/run-tests.md | Expands built-in “run tests” prompt guidance. |
| config/prompts/builtin/refactor.md | Refactors prompt into plan-first workflow with approval gating. |
| config/prompts/builtin/rebase-changes.md | Improves fork-aware remote detection and rebase guidance. |
| config/prompts/builtin/optimize.md | Refactors optimization prompt into plan-first workflow with approval gating. |
| config/prompts/builtin/implement-spec.md | Adds a new prompt for implementation planning from a spec. |
| config/prompts/builtin/fix-ci.md | Adjusts “Fix CI” prompt structure and removes auto-commit instructions. |
| config/prompts/builtin/document.md | Re-focuses “Document” prompt on user-facing documentation updates. |
| config/prompts/builtin/document-code.md | Adds a code-focused documentation prompt. |
| config/prompts/builtin/document-arch.md | Adds a developer/architecture documentation prompt. |
| config/prompts/builtin/create-spec.md | Adds an interactive “create spec” prompt. |
| config/prompts/builtin/create-pr.md | Removes the “Create PR” prompt. |
| config/prompts/builtin/create-commits.md | Renames and expands commit workflow guidance. |
| config/prompts/builtin/cleanup-code.md | Adds a cleanup plan/approval prompt. |
| config/prompts/builtin/claude-code-generate-memory.md | Generalizes examples/placeholders and fixes sample patterns. |
| config/prompts/builtin/check-ci.md | Replaces hardcoded sample errors with placeholders. |
| config/prompts/builtin/add-tests.md | Updates testing guidance wording and adds project convention note. |
| config/config.default.yaml | Adds extensive restricted runner defaults and examples. |
| cmd/mitto-app/webviewlog_darwin.h | Adds header for WKWebView console logging facility. |
| cmd/mitto-app/menu_darwin.m | Adds Reload menu item and app-activation callback for sync. |
| cmd/mitto-app/cache_darwin.m | Adds WKWebView cache clearing on startup. |
| cmd/mitto-app/cache_darwin.h | Adds header for cache management. |
| README.md | Small install/config clarify for macOS. |
| Makefile | Adds Tailwind build step, adds webview log tests target, adjusts UI test deps. |
| .github/workflows/tests.yml | Builds Tailwind in CI and configures Node.js caching. |
| .augment/rules/README.md | Adds new rules README and indexing. |
| .augment/rules/42-mcpserver-development.md | Adds MCP server development patterns and constraints. |
| .augment/rules/41-debugging-logs.md | Adds log file debugging guidance and locations. |
| .augment/rules/40-mcp-debugging.md | Adds usage guidance for MCP debugging tools. |
| .augment/rules/33-testing-js.md | Normalizes formatting and improves examples for JS testing. |
| .augment/rules/31-testing-integration.md | Adds keywords and fixes table formatting. |
| .augment/rules/30-testing-unit.md | Adds keywords and expands guidance for text-processing tests. |
| .augment/rules/26-web-frontend-hooks.md | Adds new rules for custom frontend hooks (resize/swipe). |
| .augment/rules/25-web-frontend-components.md | Expands component rules incl. resizable queue dropdown patterns. |
| .augment/rules/24-web-frontend-lib.md | Updates lib rules and adds dynamic seq calculation guidance. |
| .augment/rules/21-web-frontend-state.md | Adds keywords and improves examples for refs/state and effects. |
| .augment/rules/20-web-frontend-core.md | Adds keywords; updates examples and MAX_MESSAGES guidance. |
| .augment/rules/14-web-backend-auth.md | Adds rules for auth middleware, public paths, and security patterns. |
| .augment/rules/13-macos-keyboard-gestures.md | Adds rules for macOS shortcuts and swipe gestures. |
| .augment/rules/12-web-backend-actions.md | Fixes formatting and clarifies action button patterns. |
| .augment/rules/10-web-backend-core.md | Updates doc links and expands globs/keywords. |
| .augment/rules/08-config.md | Fixes table formatting. |
| .augment/rules/07-regex-patterns.md | Adds regex rules for URL/path matching and safe HTML processing. |
| .augment/rules/06-conversion.md | Adds conversion package rules and testing guidance. |
| .augment/rules/05-msghooks.md | Adds msghooks package rules and examples. |
| .augment/rules/04-acp.md | Fixes formatting in ACP rules. |
| .augment/rules/03-cli.md | Removes trailing whitespace/formatting artifacts. |
| .augment/rules/02-session.md | Adds keywords and documents immediate persistence & max_seq. |
| .augment/rules/01-go-conventions.md | Minor formatting cleanup. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Close terminates the ACP server process and cleans up resources. | ||
| func (c *Connection) Close() error { | ||
| // Kill the process first | ||
| if c.cmd != nil && c.cmd.Process != nil { | ||
| return c.cmd.Process.Kill() | ||
| c.cmd.Process.Kill() | ||
| } | ||
|
|
||
| // Call wait() to clean up resources (from runner.RunWithPipes or cmd.Wait) | ||
| if c.wait != nil { | ||
| // Ignore error from wait() since we already killed the process | ||
| c.wait() | ||
| } | ||
|
|
||
| return nil | ||
| } |
There was a problem hiding this comment.
When the ACP process is started via a restricted runner (r != nil), c.cmd is nil, so Close() never terminates the underlying process; it only calls wait(), which may block until the process exits on its own. Store and invoke a cancellation/kill mechanism for both execution paths (e.g., create a derived context with cancel inside NewConnection, store cancel on Connection, call it in Close(), then call wait()), so restricted-runner processes are reliably stopped.
| // Check if line starts with '{' (potential JSON-RPC message) | ||
| trimmed := bytes.TrimSpace(line) | ||
| if len(trimmed) > 0 && trimmed[0] == '{' { | ||
| // Valid JSON line - add newline and buffer it | ||
| f.pending = make([]byte, len(line)+1) | ||
| copy(f.pending, line) | ||
| f.pending[len(line)] = '\n' | ||
| f.pendingIndex = 0 | ||
|
|
||
| // Return as much as fits in p | ||
| n = copy(p, f.pending) | ||
| f.pendingIndex = n | ||
| if f.pendingIndex >= len(f.pending) { | ||
| f.pending = nil | ||
| f.pendingIndex = 0 | ||
| } | ||
| return n, nil | ||
| } |
There was a problem hiding this comment.
The filter claims to pass through “valid JSON-RPC messages”, but it currently passes any non-empty line whose first non-space character is '{', even if the line is not valid JSON. That can still break the downstream JSON-RPC decoder. Consider using json.Valid(trimmed) (or attempting a cheap unmarshal into map[string]any) before allowing the line through; otherwise, update the docstring to match the weaker guarantee.
| // Write the embedded default config | ||
| if err := os.WriteFile(configPath, embeddedconfig.DefaultConfigYAML, 0644); err != nil { |
There was a problem hiding this comment.
Creating ~/.mittorc with mode 0644 makes the config world-readable on many systems, which is risky once users add credentials (e.g., auth passwords). Use a more restrictive default like 0600 (and consider preserving existing perms on overwrite).
| // Write the embedded default config | |
| if err := os.WriteFile(configPath, embeddedconfig.DefaultConfigYAML, 0644); err != nil { | |
| // Write the embedded default config with restrictive permissions | |
| if err := os.WriteFile(configPath, embeddedconfig.DefaultConfigYAML, 0600); err != nil { |
| }, | ||
| { | ||
| name: "workspace with trailing slash", | ||
| workspaceRoot: "/home/user/projects/myproject", |
There was a problem hiding this comment.
This test case claims to cover a trailing-slash path, but workspaceRoot is identical to the non-trailing-slash case. Update it to actually include a trailing slash (e.g., .../myproject/) so it tests the intended behavior.
| workspaceRoot: "/home/user/projects/myproject", | |
| workspaceRoot: "/home/user/projects/myproject/", |
| // Mermaid extension renders ```mermaid blocks as <pre class="mermaid"> | ||
| // for client-side rendering by Mermaid.js | ||
| &mermaid.Extender{ | ||
| RenderMode: mermaid.RenderModeClient, | ||
| NoScript: true, // We load Mermaid.js ourselves in the frontend | ||
| }, |
There was a problem hiding this comment.
Mermaid extension configuration is duplicated in both WithHighlighting and NewConverter. This increases the risk of the two snippets drifting (or enabling Mermaid unexpectedly when only highlighting is desired). Consider centralizing Mermaid enabling in one place (typically NewConverter), and keep WithHighlighting focused on highlighting.
| // Mermaid extension renders ```mermaid blocks as <pre class="mermaid"> | |
| // for client-side rendering by Mermaid.js | |
| &mermaid.Extender{ | |
| RenderMode: mermaid.RenderModeClient, | |
| NoScript: true, // We load Mermaid.js ourselves in the frontend | |
| }, |
| // GetArchiveRetentionPeriod returns the archive retention period, or "never" if not set. | ||
| func (c *SessionConfig) GetArchiveRetentionPeriod() string { | ||
| if c == nil || c.ArchiveRetentionPeriod == "" { | ||
| return ArchiveRetentionNever | ||
| } | ||
| return c.ArchiveRetentionPeriod |
There was a problem hiding this comment.
ArchiveRetentionPeriod has a declared set of valid values, but GetArchiveRetentionPeriod() returns arbitrary configured strings without validation. Consider adding a ValidateArchiveRetentionPeriod() (similar to ValidateRestrictedRunner()) during settings/config load so invalid values fail fast with a clear error.
| // GetArchiveRetentionPeriod returns the archive retention period, or "never" if not set. | |
| func (c *SessionConfig) GetArchiveRetentionPeriod() string { | |
| if c == nil || c.ArchiveRetentionPeriod == "" { | |
| return ArchiveRetentionNever | |
| } | |
| return c.ArchiveRetentionPeriod | |
| // isValidArchiveRetentionPeriod reports whether the given value is an allowed | |
| // archive retention period. | |
| func isValidArchiveRetentionPeriod(value string) bool { | |
| for _, v := range ValidArchiveRetentionPeriods { | |
| if v == value { | |
| return true | |
| } | |
| } | |
| return false | |
| } | |
| // GetArchiveRetentionPeriod returns the archive retention period, or "never" if not set. | |
| // If the configured value is invalid, this function panics so that configuration | |
| // errors fail fast with a clear message. | |
| func (c *SessionConfig) GetArchiveRetentionPeriod() string { | |
| period := ArchiveRetentionNever | |
| if c != nil && c.ArchiveRetentionPeriod != "" { | |
| period = c.ArchiveRetentionPeriod | |
| } | |
| if !isValidArchiveRetentionPeriod(period) { | |
| panic(fmt.Sprintf("invalid archive retention period %q; valid values are %v", period, ValidArchiveRetentionPeriods)) | |
| } | |
| return period |
| // ValidateRestrictedRunner validates the restricted_runner field. | ||
| // Returns an error if the runner type is invalid. | ||
| func (w *WorkspaceSettings) ValidateRestrictedRunner() error { | ||
| if w.RestrictedRunner == "" { | ||
| return nil // Empty is valid (defaults to exec) | ||
| } | ||
|
|
||
| for _, validType := range ValidRunnerTypes { | ||
| if w.RestrictedRunner == validType { | ||
| return nil | ||
| } | ||
| } | ||
|
|
||
| return fmt.Errorf("invalid restricted_runner %q: must be one of %v", w.RestrictedRunner, ValidRunnerTypes) | ||
| } |
There was a problem hiding this comment.
New validation logic for restricted_runner is introduced here, but there’s no corresponding unit test shown for accepted values (empty/default + each valid type) and rejection of an invalid type. Adding table-driven tests in the internal/config package would help prevent regressions as runner types evolve.
| case AttachmentTypeBinaryFile: | ||
| // For binary files, use ResourceLinkBlock to reference the file | ||
| // The agent can read the file via its filesystem access | ||
| return acp.ResourceLinkBlock(a.Name, "file://"+a.FilePath) |
There was a problem hiding this comment.
Building file:// URLs via string concatenation can produce invalid URIs for paths containing spaces or other reserved characters, and can be platform-sensitive (notably Windows paths). Consider constructing the URL with net/url (including proper escaping) before passing it into ResourceLinkBlock.
| func resolveAccessLogConfig(cfg *config.Config, cliPath string) web.AccessLogConfig { | ||
| accessLogConfig := web.DefaultAccessLogConfig() |
There was a problem hiding this comment.
The function comment says access logging is “disabled by default for CLI usage”, but the implementation always starts from web.DefaultAccessLogConfig(). If that default ever includes a non-empty path, CLI runs could unexpectedly enable access logging. Consider initializing with web.AccessLogConfig{} (disabled), and only applying defaults when the CLI flag or config explicitly enables the feature.
| // Default: disabled for CLI usage (empty path) | ||
| return accessLogConfig | ||
| } |
There was a problem hiding this comment.
The function comment says access logging is “disabled by default for CLI usage”, but the implementation always starts from web.DefaultAccessLogConfig(). If that default ever includes a non-empty path, CLI runs could unexpectedly enable access logging. Consider initializing with web.AccessLogConfig{} (disabled), and only applying defaults when the CLI flag or config explicitly enables the feature.
- Use go-version-file: 'go.mod' instead of hardcoded Go version - Use latest golangci-lint to ensure compatibility with Go 1.25.5 - Fix TestRunnerFallback_PlatformDetection to check if firejail is actually installed, not just assume it's available on Linux
- Add context cancellation for restricted runner processes in Close() When using restricted runners, c.cmd is nil so the process was never terminated. Now we store a cancel function and call it in Close(). - Change config file permissions from 0644 to 0600 Protects credentials that may be added later (e.g., auth passwords) from being world-readable.
- Add context cancellation for restricted runner processes in Close() When using restricted runners, c.cmd is nil so the process was never terminated. Now we store a cancel function and call it in Close(). - Change config file permissions from 0644 to 0600 Protects credentials that may be added later (e.g., auth passwords) from being world-readable.
Summary
This PR contains a comprehensive set of improvements across the entire Mitto codebase, focusing on WebSocket reliability, session management, error handling, and UI enhancements.
Key Changes
🔌 WebSocket & Connectivity
🛠️ Session Management
MaxSeqfor better persistence💬 Error Handling
🎨 Frontend Improvements
🔒 Security & Auth
📝 Documentation
🧪 Testing
🍎 macOS App
🔧 Other
Testing
Commits
This PR contains 83 commits organized by feature area.
Pull Request opened by Augment Code with guidance from the PR author