This document provides a high-level introduction to the dump CLI tool, its architecture, and core capabilities. The dump tool aggregates content from local files, URLs, and tmux panes into structured formats (XML or Markdown) for consumption by Large Language Models.
For detailed information on specific subsystems:
The dump tool serves as a context aggregator for LLM workflows. It recursively traverses directories, fetches web content via the Exa API, and captures tmux terminal panes, producing unified output suitable for LLM context windows. The tool implements concurrent processing with bounded worker pools and respects .gitignore rules while providing extensive filtering capabilities.
Sources: main.go1-24 README.md1-21 CLAUDE.md1-13
The application is implemented as a single-file Go program with a clear separation between acquisition, processing, and output stages.
Sources: main.go26-43 main.go99-125 main.go631-802 main.go804-825
The tool acquires content from three distinct sources, each with its own processing mechanism.
| Source | Entry Function | Data Structure | Concurrency Model |
|---|---|---|---|
| Local Files | processDirectory() | Item | Per-directory goroutines |
| URLs | fetchURLContent() | Item | Bounded worker pool (max 3) |
| Tmux Panes | capturePaneContent() | TmuxPaneItem | Bounded worker pool (max 6) |
Local files are processed through processDirectory216-330 which uses filepath.WalkDir to recursively traverse directories. The function respects .gitignore patterns via the go-gitignore library and filters files based on glob patterns and extensions with OR semantics.
Sources: main.go216-330 main.go47-67 main.go69-76 main.go189-214
URLs are fetched concurrently through fetchURLsConcurrently332-372 which implements a bounded worker pool with rate limiting (350ms between requests). The function calls fetchURLContent526-582 to interact with the Exa API endpoint at https://api.exa.ai/contents. The API key is read from the EXA_API_KEY environment variable.
Sources: main.go332-372 main.go526-582 main.go45 main.go126-142
Tmux panes are captured via fetchTmuxConcurrently470-524 which resolves selectors like current, all, %1, or 0.1 through resolveTmuxSelectors392-437 Content is captured using capturePaneContent454-467 which executes tmux capture-pane with configurable history lines.
Sources: main.go470-524 main.go392-437 main.go440-451 main.go454-467 main.go375-389
The processing pipeline applies multiple filtering stages to content before output.
.gitignore filesgobwas/globSources: main.go47-67 main.go69-76 main.go78-97 main.go169-187 main.go263-281
The tool supports two output formats controlled by the --out-fmt flag.
| Format | Flag Value | File Tag | Tmux Tag | Tree Tag |
|---|---|---|---|---|
| XML (default) | xml | <document path='...'> | <tmux_pane id='...'> | <tree path='...'> |
| Markdown | md | ```path | ```shell | ```tree |
The formatItem144-154 function handles both file and URL output, using <document> or custom XML tags. The formatTmuxItem156-166 function formats tmux panes with session metadata. Tree visualization is rendered by formatTreeOutput621-629 using formatTreeNode594-619 for recursive traversal.
Sources: main.go144-154 main.go156-166 main.go594-629
The CLI is built using the Cobra framework with the main command defined in rootCmd804-825
All flags are registered in init827-848 and processed in runDump631-802 The tool defaults to the current directory when no sources are specified.
Sources: main.go26-41 main.go804-825 main.go827-848 main.go631-802
The tool implements concurrent processing for performance while maintaining bounded resource usage.
| Goroutine Pool | Max Workers | Implementation | Purpose |
|---|---|---|---|
| Directory Processing | Unlimited (1 per dir) | sync.WaitGroup | Parallel directory traversal |
| URL Fetching | 3 | Bounded worker pool | Rate-limited API requests |
| Tmux Capture | 6 | Bounded worker pool | Parallel pane capture |
The runDump631-802 function orchestrates three concurrent operations:
fetchURLsConcurrentlyfetchTmuxConcurrentlyResults are streamed through channels and written to stdout as they become available.
Sources: main.go631-802 main.go332-372 main.go470-524 main.go698-751
The tool relies on three external Go libraries:
| Package | Purpose | Usage Location |
|---|---|---|
github.com/spf13/cobra | CLI framework | main.go23 main.go804-825 |
github.com/gobwas/glob | Glob pattern matching | main.go21 main.go78-88 |
github.com/sabhiram/go-gitignore | Gitignore parsing | main.go22 main.go69-76 |
External system dependencies:
EXA_API_KEY environment variable for URL fetchingPATH for pane capture functionalitySources: main.go3-24 main.go45 go.mod
Refresh this wiki