# Inquest

Inquest provides a database of test results which can be used as part of developer workflow to ensure/check things like:

• No commits without having had a test failure, test fixed cycle.
• No commits without new tests being added.
• What tests have failed since the last commit (to run just a subset).
• What tests are currently failing and need work.

Test results are inserted using subunit (and thus anything that can output subunit or be converted into a subunit stream can be accepted).

Inquest started as a Rust port of the Python testrepository tool, originally written by Robert Collins. It has since grown its own features and on-disk format.

Features:

• Fast, native binary with no Python runtime required
• SQLite-backed repository with rich run metadata
• Timeout support (per-test, overall, no-output)
• Auto-detection of project type
• Optional backward compatibility with testrepository's on-disk format
• Configuration via inquest.toml (TOML) or .testr.conf (legacy INI)

Build from source:

cargo build --release

The binary will be available at target/release/inq.

Create a config file inquest.toml:

test_command = "cargo test $IDOPTION"
test_id_option = "--test $IDFILE"

Create a repository:

inq init

Run tests and load results:

inq run

Query the repository:

inq stats
inq last
inq failing
inq slowest

Re-run only failing tests:

inq run --failing

List available tests:

inq list-tests

Delete a repository:

rm -rf .inquest

Auto-detect the project type and generate an inquest.toml configuration file.

Initialize a new test repository in the current directory. Creates a .inquest/ directory with a SQLite metadata database and runs directory.

Execute tests using the command defined in inquest.toml and load the results into the repository.

Options:

• --failing: Run only the tests that failed in the last run
• --partial: Partial run mode (update failing tests additively)
• --force-init: Create repository if it doesn't exist
• --auto: Auto-detect project type and generate inquest.toml if missing
• --load-list <FILE>: Run only tests listed in the file (one per line)
• -j, --parallel <N>: Run tests in parallel across N workers
• --until-failure: Run tests repeatedly until they fail
• --max-iterations <N>: Maximum iterations for --until-failure
• --isolated: Run each test in a separate process
• --subunit: Show output as a subunit stream
• --all-output: Show output from all tests (not just failures)
• --test-timeout <TIMEOUT>: Per-test timeout (5m, auto, or disabled)
• --max-duration <DURATION>: Overall run timeout
• --no-output-timeout <DURATION>: Kill test process if no output for this duration
• TESTFILTER: Regex patterns to filter which tests to run
• -- TESTARGS: Additional arguments passed through to the test command

Run tests with output formatted for a CI provider. Wraps inq run with CI-friendly defaults and emits structured output that GitHub Actions or GitLab CI can render natively: foldable log groups per failing test, inline annotations on the changed lines, and a markdown job summary.

Options:

• --format <auto|github|gitlab|plain>: Output format. auto (default) detects GITHUB_ACTIONS=true or GITLAB_CI=true from the environment and falls back to plain otherwise.
• --retry <N>: Re-run failing tests up to N times. Tests that pass on retry are reported as warnings (the run still succeeds) rather than errors. Opt-in; defaults to 0 so the first run is the honest signal.
• --order <ORDER>: Test ordering. Defaults to frequent-failing-first when the repository has run history (so known-bad tests fail fast), otherwise discovery. Accepts the same vocabulary as inq run --order.
• -j, --parallel <N>: Parallel worker count.
• --test-timeout <TIMEOUT> / --max-duration <DURATION>: Timeout overrides, same syntax as inq run.
• -s, --starting-with <PREFIX>: Restrict to tests with the given prefix.
• TESTFILTER: Regex patterns to filter which tests to run.
• -- TESTARGS: Additional arguments passed through to the test command.

See README.gh-action.md for a workflow example using the bundled jelmer/inquest composite action, plus how to cache the .inquest directory across runs so the ordering heuristics have history to work with.

Re-run exactly the tests of a previous run, in the same order, forwarding the same ---style test arguments captured in the run's metadata. Without an argument, re-runs the latest run.

inq rerun 7    # repeat run 7
inq rerun -1   # repeat the latest run

Load test results from stdin in subunit format.

my-test-runner | inq load

Options:

• --partial: Partial run mode (update failing tests additively)
• --force-init: Create repository if it doesn't exist

Show results from a test run, including timestamp, counts, and list of failing tests.

Options:

• -r, --run <ID>: Run ID to show (defaults to latest; supports negative indices like -1, -2)
• --subunit: Output results as a subunit stream
• --no-output: Don't show test output/tracebacks for failed tests

Show detailed information about a test run, including git commit, command, duration, exit code, and concurrency.

Options:

• -r, --run <ID>: Run ID to show (defaults to latest; supports negative indices)

Show only the failing tests from the last run. Exits with code 0 if no failures, 1 if there are failures.

Options:

• --list: List test IDs only, one per line (for scripting)
• --subunit: Output results as a subunit stream

Show currently in-progress test runs. Tails the in-progress subunit stream on disk to report live test counts (passed/failed), percent complete versus the historical test count, elapsed wall-clock time, and an ETA derived from each test's historical duration. Uses lock files to track active runs.

Print the resolved effective configuration. Combines values from the config file (inquest.toml, .inquest.toml, or .testr.conf), CLI overrides, and built-in defaults. Each value is annotated with its source ([config], [cli], or [default]), making it easy to see what inq run would actually use.

Options accept the same overrides as inq run: --test-timeout, --max-duration, --no-output-timeout, --order, and -j/--parallel.

Show repository statistics including total test runs, latest run details, and total tests executed.

Show the slowest tests from the last run, sorted by duration.

Options:

• -n, --count <N>: Number of tests to show (default: 10)
• --all: Show all tests (not just top N)

Rank tests by flakiness across recorded runs. Flakiness is measured by pass↔fail transitions in consecutive runs in which the test was recorded, so chronically broken tests rank low and genuinely flapping tests rank high.

Options:

• -n, --count <N>: Number of tests to show (default: 10)
• --all: Show all candidate tests (not just top N)
• --min-runs <N>: Minimum runs a test must appear in to be ranked (default: 5)

Each row reports flake% (transitions / max(1, runs - 1)), fail% (failures / runs), the raw transition count, and the number of recorded runs.

Group a run's failing tests by their common failure cause. Each failure's traceback is normalized (file paths, line numbers, hex addresses, temp dirs, durations are replaced with placeholders), anchored at the last line that names the failure (e.g. AssertionError:, panicked at, FAILED), and clustered with other failures whose normalized tails share a common suffix. Each cluster is reported with the longest line sequence shared by every member, so you see the unique failure once and the count of tests it covers.

Options:

• -r, --run <ID>: Run ID to summarize (defaults to latest; supports negative indices)
• --samples <N>: Maximum sample test IDs to list per group (default: 5)

Show logs and tracebacks for specific tests, matched by glob-style patterns.

Options:

• -r, --run <ID>: Run ID to show logs from (defaults to latest)

Example:

inq log 'test.module.TestCase.*'

Print the test IDs assigned to one shard of an M-way balanced split. Distributed CI nodes can each call inq shard N/M to claim a disjoint, load-balanced slice of the suite. Historical durations from the repository are used to balance shards when available; without history, the split degrades to round-robin.

The partition is deterministic given the same suite and history, so two nodes calling inq shard 1/4 and inq shard 2/4 produce disjoint shards whose union is the full suite.

Options:

• --group-regex <REGEX>: Override the config's group_regex. Pass an empty string to disable grouping for this command only.
• --zero-indexed: Treat the shard index as 0-based (i.e. 0/4..3/4).

Example (GitHub Actions matrix):

strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: inq shard ${{ matrix.shard }}/4 > shard.txt
  - run: inq run --load-list shard.txt

Drop older test runs from the repository. Exactly one selection mode must be given:

• --keep <N>: Keep the N most recent runs and prune the rest
• --older-than <DURATION>: Prune runs older than the given duration. Accepts s, m, h, d, and w suffixes (e.g. 30d, 2w, 12h)
• --run <ID>: Prune the named run by ID. May be given multiple times
• --all: Prune every run in the repository

Pass --dry-run to see which runs would be pruned without modifying the repository. In-progress runs are always skipped.

List all available tests by querying the test command with the list option from configuration.

Analyze test isolation issues using bisection to find which tests cause a target test to fail when run together but pass in isolation.

This command:

1. Runs the target test in isolation to verify it passes alone
2. Runs all tests together to verify the failure reproduces
3. Uses binary search to find the minimal set of tests causing the failure
4. Reports which tests interact with the target test

Example:

inq analyze-isolation test.module.TestCase.test_flaky

Upgrade a legacy .testrepository/ directory to the new .inquest/ format. Only available when built with the testr feature.

Generate shell completions for the given shell. Supported shells: bash, zsh, fish, elvish, powershell.

To enable completions:

# Zsh (add to ~/.zshrc)
eval "$(inq completions zsh)"

# Bash (add to ~/.bashrc)
eval "$(inq completions bash)"

# Fish (add to ~/.config/fish/config.fish)
inq completions fish | source

All commands support:

• -C, --directory <PATH>: Specify repository path (defaults to current directory)

inq looks for configuration in the following files (in order of priority): inquest.toml, .inquest.toml, .testr.conf. The TOML format is preferred; .testr.conf (INI format with a [DEFAULT] section) is supported for backward compatibility.

Key options:

• test_command: Command to run tests (required)
• test_id_option: Option format for running specific tests (e.g., --test $IDFILE)
• test_list_option: Option to list all available tests
• test_id_list_default: Default value for $IDLIST when no specific tests
• group_regex: Regex to group related tests together during parallel execution
• test_run_concurrency: Command to determine concurrency level (e.g., nproc)
• filter_tags: Tags to filter test results by (for parallel execution)
• instance_provision: Command to provision test instances (receives $INSTANCE_COUNT)
• instance_execute: Command template for running tests in an instance (receives $INSTANCE_ID)
• instance_dispose: Command to clean up test instances (receives $INSTANCE_ID)

The following variables are available for use in test_command:

• $IDOPTION: Expands to the test_id_option with actual test IDs
• $IDFILE: Path to a temporary file containing test IDs (one per line)
• $IDLIST: Space-separated list of test IDs
• $LISTOPT: Expands to the test_list_option

test_command = "cargo test $IDOPTION"
test_id_option = "--test $IDFILE"
test_list_option = "--list"

test_command = "pytest $IDOPTION"
test_id_option = "--test-id-file=$IDFILE"
test_list_option = "--collect-only -q"

Use the gotest-run wrapper (ships with python-subunit):

test_command = "gotest-run $LISTOPT $IDOPTION"
test_id_option = "--id-file $IDFILE"
test_list_option = "--list"

inq auto generates this automatically when it sees a go.mod. The wrapper enumerates tests via go test -json -list ... for --list, fans out one go test -json -run <regex> invocation per affected package for --id-file, and otherwise runs the whole tree. Subtests are created at runtime by t.Run and aren't statically discoverable, so they're absent from listings — but executing them by ID works (e.g. inq run --failing correctly re-runs pkg::TestX/sub_one).

Drive the build tool through jvmtest-subunit (ships with python-subunit):

test_command = "jvmtest-subunit $LISTOPT $IDOPTION"
test_id_option = "--id-file $IDFILE"
test_list_option = "--list"

inq auto generates this from pom.xml or build.gradle{,.kts}/settings.gradle{,.kts}. The wrapper:

• auto-detects Maven vs Gradle from the working directory;
• spawns the build tool and watches its reports directory live so per-class results stream into inq run as each test class finishes;
• enumerates tests for --list by walking src/test/java and src/test/kotlin for conventionally-named test classes (*Test/*Tests/*TestCase/*IT);
• translates --id-file IDs (Class or Class::method) to the build tool's selection syntax — -Dtest=Class1,Class2#methodA+methodB for Maven, --tests Class1 --tests Class2.methodA for Gradle;
• forwards build-tool stdout/stderr so users still see compile errors and progress.

inq run --failing, inq run --isolated, and inq list-tests all work — methods created at runtime (@ParameterizedTest, @TestFactory dynamic tests) aren't statically discoverable and so are absent from listings, but executing them by ID works.

The wrapper takes optional flags for non-default layouts: jvmtest-subunit --tool gradle --reports-dir build/custom-reports, or pass extra build args after --: jvmtest-subunit -- -Dmaven.test.skip=false.

Vitest ships a built-in TAP reporter; pipe it through tap2subunit (ships with python-subunit):

test_command = "vitest run --reporter=tap | tap2subunit"

inq auto generates this when it finds a vitest.config.* file or vitest in package.json dependencies.

Jest has no built-in machine-readable reporter, so the standard approach is to install jest-junit as a devDependency and convert its XML output:

test_command = "jest --ci --reporters=jest-junit; junitxml2subunit junit.xml"

inq auto generates this when it finds a jest.config.* file, a jest block in package.json, or jest in package.json dependencies. The ; (rather than &&) ensures junitxml2subunit still runs when tests fail.

test_command = "cargo test --quiet $IDOPTION"
test_id_option = "$IDLIST"
test_list_option = "--list"

# Use system CPU count for parallel execution
test_run_concurrency = "nproc"

# Group tests by module (keeps related tests together)
group_regex = "^(.*)::[^:]+$"

The bundled jelmer/inquest composite action installs inq, restores and saves the .inquest history cache, and runs inq ci:

- uses: actions/checkout@v4
- uses: jelmer/inquest@main

See README.gh-action.md for the full reference, including inputs and outputs, a manual setup recipe (no action dependency), test-history caching, flake retries, sharding across matrix jobs, and GitLab CI.

Inquest has its own on-disk format, stored in a .inquest/ directory:

• metadata.db: SQLite database storing run metadata (git commit, command, duration, exit code, concurrency)
• runs/: Directory containing individual test run files in subunit v2 binary format
• runs/N.lock: Lock files for tracking in-progress test runs

When built with the testr feature, inquest can also read and write the legacy .testrepository/ format used by the Python testrepository tool. Use inq upgrade to migrate.

Inquest is under BSD / Apache 2.0 licences. See the file COPYING in the source for details.

Inquest started as a Rust port of the Python testrepository tool, originally written by Robert Collins. It has since diverged with its own features:

• New on-disk format (.inquest/) using SQLite for metadata, with richer run information (git commit, command, duration, exit code, concurrency)
• Run lock files and inq running command for tracking in-progress test runs
• Auto-detection of project type (inq auto)
• Timeout support: per-test timeouts, overall run timeouts, and no-output timeouts with automatic process killing
• inq log command for viewing logs/tracebacks of individual tests by glob pattern
• inq info command for detailed run metadata
• Test filtering by regex patterns on the command line
• Pass-through arguments to the underlying test command via --

• Original Python version: https://github.com/testing-cabal/testrepository
• Subunit: http://subunit.readthedocs.io/