Skip to content

Combined JSON output can emit duplicate JSON when hotspot analysis lacks Git repo #294

@BartWaardenburg

Description

@BartWaardenburg

Summary

During the nixpkgs packaging work for fallow 2.65.0, the sandboxed upstream test run exposed a JSON contract bug in combined mode. Some combined-mode invocations outside a Git repository can write two JSON documents to stdout: first a structured hotspot error, then the normal combined report.

That makes --format json output invalid for agents and CI parsers because stdout is no longer a single JSON value.

Observed output shape

{
  "error": true,
  "message": "hotspot analysis requires a git repository",
  "exit_code": 2
}
{
  "schema_version": 3,
  "version": "2.65.0",
  "check": { ... },
  "dupes": { ... },
  "health": { ... }
}

The failing test parser reported:

failed to parse JSON: trailing characters at line 6 column 1

Where it surfaced

This reproduced while building the nixpkgs derivation with the upstream GitHub release source and full test suite. The failures were in combined-mode integration tests whose fixture projects live in temporary directories outside any Git checkout. Examples:

  • include_entry_exports_works_in_combined_mode
  • include_entry_exports_via_config_file_in_combined_mode
  • combined_mode_runs_successfully
  • combined_mode_config_enabled_coverage_gaps_stays_out_of_health_section
  • combined_mode_hidden_coverage_gap_gate_does_not_fail

The nixpkgs package works around this by initializing $TMPDIR as a harmless parent Git repo before running checks, so the package can still run the full upstream test suite. That workaround should not be required for normal JSON contract safety.

Expected behavior

When --format json is active, stdout should always contain exactly one JSON document. For combined mode, nested health/hotspot failures should not directly emit their own standalone JSON before the combined report.

Possible acceptable fixes:

  • treat missing Git history for hotspot analysis as unavailable data in the combined report, with hotspot_count: null or equivalent warning metadata;
  • include one top-level structured error and stop before writing the combined report;
  • route nested analysis errors through the combined envelope without writing a second JSON document.

Why this matters

AGENTS.md tells agents to use --format json and parse stdout as machine-readable output. Emitting two JSON documents breaks that contract and can cause agents/CI integrations to reject otherwise valid analysis output.

Acceptance criteria

  • Combined-mode --format json emits exactly one JSON document when run outside a Git repository.
  • No standalone nested health/hotspot error JSON is written before the combined report.
  • Add/adjust an integration test that runs the affected combined-mode path from a temp project outside any Git repository.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions