feat: add --compact flag to strip metadata from JSON output#193
Conversation
Add a --compact global flag to both jtk and cfl that strips verbose metadata from JSON output. Useful for LLM/agent consumers where every token has a cost. When --compact is passed with --output json, the following are stripped: - Null-valued fields (e.g., dozens of null customfield_* entries) - avatarUrls objects (4 size variants per user, ~200 chars each) - API self-link URLs (/rest/... references back to the API) - Confluence _links and _expandable metadata objects Non-null fields, descriptions, content, and all meaningful data are preserved. The flag is opt-in and has no effect on table/plain output. Measured: 12% reduction on a 10-issue Jira search (392K → 342K chars).
Address review feedback: - Maps left empty after all fields are pruned (e.g., a user object with only avatarUrls + self) are now dropped entirely - Add test for top-level JSON arrays (covers commands that pass slices to v.JSON()) - Add comment explaining the JSON round-trip design choice
TDD AssessmentCore pruning logic ( Two gaps worth noting: 1. Array items that prune to
case []any:
result := make([]any, 0, len(val))
for _, item := range val {
result = append(result, pruneValue(item)) // no isEmpty check
}
return resultIf you have an array of user objects that only contained This is either a silent bug or an intentional choice to preserve array length/indices. Either way it's undocumented and untested — worth a comment in the code and a test case covering it. 2. It's true by implementation (only Neither gap is a blocker — just items to address before they become bugs in the field. |
monit-reviewer
left a comment
There was a problem hiding this comment.
Automated PR Review
Reviewed commit: 069a681
Summary
| Reviewer | Findings |
|---|---|
| harness-engineering:enforcement-reviewer | 1 |
| harness-engineering:knowledge-reviewer | 1 |
| security:code-auditor | 1 |
harness-engineering:enforcement-reviewer (1 findings)
shared/view/view.go:337
isEmpty() returns true for zero-length slices, so originally-empty arrays like
"labels": []are silently dropped from compact output. An empty array is semantically distinct from an absent field —[]means 'no items' while absence means 'field not fetched'. The PR description says compact strips 'null fields and metadata', but empty arrays are neither. Consider only dropping values that were non-empty before pruning, or tracking whether a value was emptied by pruning vs. originally empty.
harness-engineering:knowledge-reviewer (1 findings)
💡 Suggestion - shared/view/view.go:326
Array items that become empty maps after pruning (e.g., a user object containing only avatarUrls + self) are retained as
{}in the output array. This is inconsistent with map-value handling where empty pruned maps are dropped. Could result in[{}, {"id": "2"}]noise. Consider filtering empty maps from the array result, same as the map-value path does.
security:code-auditor (1 findings)
💡 Suggestion - shared/view/view.go:312
The self-link heuristic (
strings.Contains(s, "/rest/")) won't match Confluence v2 API URLs which use/wiki/api/v2/...paths. If Confluence v2 endpoints are used, theirselflinks will survive compact stripping. Consider also matching/api/v2/or/wiki/patterns, or just stripping all HTTP(S) URL values underselfkeys unconditionally.
2 info-level observations excluded. Run with --verbose to include.
1 PR discussion thread considered.
Completed in 6m 05s | $0.87
| Field | Value |
|---|---|
| Reviewers | hybrid-synthesis, database:sql-reviewer, security:code-auditor, harness-engineering:knowledge-reviewer, harness-engineering:enforcement-reviewer, harness-engineering:architecture-reviewer, harness-engineering:legibility-reviewer |
| Reviewed by | pr-review-daemon · monit-pr-reviewer |
| Duration | 6m 05s (Reviewers: 1m 46s · Synthesis: 58s) |
| Cost | $0.87 |
| Tokens | 307.7k in / 21.1k out |
| Turns | 21 |
- Preserve originally-empty collections: [] means "no items" and {}
means "empty object" — both are semantically distinct from an absent
field. Only drop collections that became empty due to pruning.
- Broaden self-link stripping to match any HTTP(S) URL, not just /rest/
paths. Covers Confluence v2 (/wiki/api/v2/...) endpoints.
- Drop empty array items after pruning (consistent with map-value path).
- Add tests: originally-empty array preservation, Confluence v2 self
URLs, table/plain no-op contracts.
|
Both addressed in 69254e4:
Also fixed the |
All 3 findings addressed in 69254e4 — see thread replies.
Summary
--compactglobal flag to bothjtkandcflthat strips verbose metadata from JSON outputviewpackage so both tools get it automaticallyWhat
--compactstrips"customfield_10088": null(20-30 per Jira issue)"avatarUrls": {"16x16": "...", "24x24": "...", ...}per user"self": "https://...atlassian.net/rest/api/3/..."on every nested object"_links","_expandable"objectsWhat it preserves
Everything meaningful: summaries, descriptions, status, assignee names, comments, page content, etc. Only API plumbing metadata is removed.
Usage
The flag has no effect on
tableorplainoutput formats.Motivation
We run an AI assistant (Opus orchestrator + MCP tool servers) that calls
jtkandcfl~200-400 times/day. Cache-write tokens (from tool output entering the LLM context) account for 73% of our API cost. The top offenders by output volume:jtk issues getcfl page viewjtk issues searchA 10-12% reduction across these calls meaningfully reduces token costs without any risk to answer quality.
Test plan
make test)make build)jtk issues get/searchwith real Jira data shows 10-12% size reduction