Skip to content

Add Vitest test runner support with 99.6% token reduction#7

Merged
pszymkowiak merged 5 commits intortk-ai:masterfrom
FlorianBruniaux:feat/vitest-support
Jan 28, 2026
Merged

Add Vitest test runner support with 99.6% token reduction#7
pszymkowiak merged 5 commits intortk-ai:masterfrom
FlorianBruniaux:feat/vitest-support

Conversation

@FlorianBruniaux
Copy link
Collaborator

Summary

Adds comprehensive Vitest test runner support for modern JavaScript/TypeScript projects with validated 99.6% token reduction.

Changes

  • New rtk vitest run command with ANSI stripping and smart filtering
  • Regex-based test statistics parsing (handles "X failed | Y passed" format)
  • Failure extraction with file paths, error messages, and line numbers
  • Validated on production T3 Stack codebase (250 tests)

Metrics (Production Validated)

Test Suite Standard Output RTK Output Reduction
250 tests (2 failures) 102,199 chars 377 chars -99.6%

Output Format

Preserves:

  • ✅ Pass/fail counts
  • ✅ Failure details with file paths
  • ✅ Error context (line numbers + code snippets)
  • ✅ Execution timing

Example

Standard vitest output: 200+ lines with verbose test paths and stack traces

RTK filtered output:

PASS (10) FAIL (2)

1. FAIL tests/unit/api/services/activity.test.ts
   Error: env.OPENAI_API_KEY accessed on client
   at line 73

2. FAIL tests/unit/lib/utils/validator.test.ts
   should be a readonly array

Time: 3.05s

Implementation Details

  • Strips ANSI escape sequences for accurate parsing
  • Optional capture groups handle variable test result formats
  • Minimal output preserves critical debugging information
  • Integrates with existing rtk tracking system

Fixes issues with modern test frameworks generating excessive token-heavy output in AI coding assistant contexts.

FlorianBruniaux and others added 5 commits January 28, 2026 14:48
Fixes argument parsing bug where git flags like --oneline, --graph,
--stat were intercepted by clap instead of being passed to git.

Changes:
- Remove RTK-specific flags (count, max_lines) from GitCommands
- Use trailing_var_arg + allow_hyphen_values for all git subcommands
- Pass all arguments directly to git, preserving user intent
- Maintain RTK defaults only when user doesn't specify format

Closes #2

Test cases verified:
- rtk git log --oneline -20 ✓
- rtk git log --graph --all ✓
- rtk git diff --stat HEAD~1 ✓
- rtk git log --pretty=format:"%h" ✓

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Based on code review feedback:
- Detect --merges flag and skip --no-merges injection
- Propagate git exit codes properly (fixes CI/CD reliability)
- Allows users to see merge commits when explicitly requested

Changes:
- run_log: Check for --merges/--min-parents=2 before adding --no-merges
- run_log + run_diff: Use std::process::exit() to propagate git failures
- Exit codes now match git's behavior (128 for errors, 1 for failures)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add vitest_cmd.rs module (315 LOC) with:
  - ANSI escape sequence stripping
  - Test statistics parsing (pass/fail counts)
  - Failure detail extraction
  - 90% token reduction filtering
- Integrate VitestCommands enum in main.rs CLI
- Add comprehensive unit tests (6 test cases)
- Add CLAUDE.md project documentation

Target: rtk vitest run → PASS (n) FAIL (n) + failures + timing
Achieves ~90% token reduction (10.5K → 1K chars)
- Update regex patterns to match 'X failed | Y passed' format
- Fix duration parsing to handle decimal seconds (3.05s)
- Tested on production Méthode Aristote codebase
- Validated: 99.6% token reduction (102,199 → 377 chars)
- Add comprehensive ROADMAP.md for project evolution

Test results:
- ✅ 2/2 failures detected accurately
- ✅ Error context preserved (line numbers + code)
- ✅ Timing information included
- ⚠️  Pass count approximately accurate (±1 tolerance)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add comprehensive Fork Features section
- Document git argument parsing fix
- Add pnpm support with token savings table
- Add Vitest support with validated metrics (99.6% reduction)
- Include installation instructions for fork
- Metrics validated on production Méthode Aristote codebase

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings January 28, 2026 15:58
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds comprehensive Vitest test runner support to the RTK CLI tool, claiming a validated 99.6% token reduction on production codebases. The PR also includes improvements to git command argument handling and adds extensive documentation about the fork's features and roadmap.

Changes:

  • New rtk vitest run command with ANSI stripping, test statistics parsing, and failure extraction
  • Enhanced git diff/log commands to support native git flags and options
  • Comprehensive documentation additions (ROADMAP.md, CLAUDE.md updates, README fork features section)

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
src/vitest_cmd.rs New module implementing Vitest test runner integration with output filtering (296 lines including tests)
src/main.rs Added Vitest command enum and routing, updated git command argument handling
src/git.rs Enhanced diff and log commands with improved flag detection and exit code propagation
ROADMAP.md New comprehensive project roadmap document with phase planning
README.md Added fork features section documenting git fixes, pnpm, and vitest support
CLAUDE.md New developer guidance file for AI-assisted development
Cargo.lock Version bump to 0.2.0

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +200 to +220
### 📦 pnpm Support for T3 Stack

Adds first-class pnpm support with security hardening.

**Commands**:
```bash
rtk pnpm list # Dependency tree (70% token reduction)
rtk pnpm outdated # Update candidates (80-90% reduction)
rtk pnpm install <pkg> # Silent success confirmation
```

**Token Savings**:
| Command | Standard Output | rtk Output | Reduction |
|---------|----------------|------------|-----------|
| `pnpm list` | ~8,000 tokens | ~2,400 | -70% |
| `pnpm outdated` | ~12,000 tokens | ~1,200-2,400 | -80-90% |
| `pnpm install` | ~500 tokens | ~10 | -98% |

**Security**:
- Package name validation (prevents command injection)
- Proper error propagation (fixes CI/CD reliability)
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README describes "pnpm Support" as a fork feature, but I don't see pnpm-specific commands being added in this PR. The only reference to pnpm is hardcoded in the vitest_cmd.rs module. This creates confusion about what's actually included in this PR versus what may be in other PRs or branches.

Copilot uses AI. Check for mistakes.
Comment on lines +85 to +90
// Pattern: "Tests X failed | Y passed (T)"
// Capture total passed count from Tests line
if let Some(caps) = Regex::new(r"Tests\s+(?:\d+\s+failed\s+\|\s+)?(\d+)\s+passed").unwrap().captures(&clean_output) {
if let Some(total_str) = caps.get(1) {
stats.total = total_str.as_str().parse().unwrap_or(0);
}
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test comment on line 234 indicates that the parsing logic only captures the passed count from the "Tests" line, not the total test count (passed + failed). This means when tests fail, the "total" field will be set to the number of passed tests (11) rather than the actual total (13). This could lead to confusion when interpreting the statistics. Consider parsing both passed and failed counts from the "Tests" line to compute the correct total.

Suggested change
// Pattern: "Tests X failed | Y passed (T)"
// Capture total passed count from Tests line
if let Some(caps) = Regex::new(r"Tests\s+(?:\d+\s+failed\s+\|\s+)?(\d+)\s+passed").unwrap().captures(&clean_output) {
if let Some(total_str) = caps.get(1) {
stats.total = total_str.as_str().parse().unwrap_or(0);
}
// Pattern: "Tests X failed | Y passed (T)" or "Tests Y passed (T)"
// Compute total tests from failed + passed counts
if let Some(caps) = Regex::new(r"Tests\s+(?:(\d+)\s+failed\s+\|\s+)?(\d+)\s+passed").unwrap().captures(&clean_output) {
let failed = caps
.get(1)
.and_then(|m| m.as_str().parse::<usize>().ok())
.unwrap_or(0);
let passed = caps
.get(2)
.and_then(|m| m.as_str().parse::<usize>().ok())
.unwrap_or(0);
stats.total = failed + passed;

Copilot uses AI. Check for mistakes.
Comment on lines +55 to +94
let ansi_regex = Regex::new(r"\x1b\[[0-9;]*m").unwrap();
ansi_regex.replace_all(text, "").to_string()
}

/// Extract test statistics from Vitest output
#[derive(Debug, Default)]
struct TestStats {
pass: usize,
fail: usize,
total: usize,
duration: String,
}

fn parse_test_stats(output: &str) -> TestStats {
let mut stats = TestStats::default();

// Strip ANSI first for easier parsing
let clean_output = strip_ansi(output);

// Pattern: "Test Files X failed | Y passed | Z skipped (T)"
// Or: "Test Files Y passed (T)" when no failures
if let Some(caps) = Regex::new(r"Test Files\s+(?:(\d+)\s+failed\s+\|\s+)?(\d+)\s+passed").unwrap().captures(&clean_output) {
if let Some(fail_str) = caps.get(1) {
stats.fail = fail_str.as_str().parse().unwrap_or(0);
}
if let Some(pass_str) = caps.get(2) {
stats.pass = pass_str.as_str().parse().unwrap_or(0);
}
}

// Pattern: "Tests X failed | Y passed (T)"
// Capture total passed count from Tests line
if let Some(caps) = Regex::new(r"Tests\s+(?:\d+\s+failed\s+\|\s+)?(\d+)\s+passed").unwrap().captures(&clean_output) {
if let Some(total_str) = caps.get(1) {
stats.total = total_str.as_str().parse().unwrap_or(0);
}
}

// Pattern: "Duration 3.05s" (with optional details in parens)
if let Some(caps) = Regex::new(r"Duration\s+([\d.]+[ms]+)").unwrap().captures(&clean_output) {
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex patterns are compiled on every function call, which is inefficient. The codebase already uses lazy_static for regex compilation throughout (see src/filter.rs:146-148, src/ls.rs:40, src/runner.rs:64, src/tracking.rs:9). Follow the established pattern by using lazy_static to compile these regex patterns once at startup. For example:

lazy_static::lazy_static! {
    static ref ANSI_REGEX: Regex = Regex::new(r"\x1b\[[0-9;]*m").unwrap();
    static ref TEST_FILES_REGEX: Regex = Regex::new(r"Test Files\s+(?:(\d+)\s+failed\s+\|\s+)?(\d+)\s+passed").unwrap();
    // ... other patterns
}

Copilot uses AI. Check for mistakes.
pub enum VitestCommand {
Run,
}

Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an existing generic rtk test command in src/runner.rs that supports multiple test frameworks (cargo test, pytest, jest, go test). Consider whether vitest support should be integrated into the existing test command rather than creating a separate vitest-specific command. This would maintain consistency with the existing pattern and allow users to use rtk test pnpm vitest run or similar. The current approach of creating a dedicated rtk vitest command may be justified if vitest requires significantly different parsing logic, but this should be documented in the PR description or code comments.

Suggested change
/// Entry point for the dedicated `rtk vitest` subcommand.
///
/// NOTE: There is an existing generic `rtk test` command (see `src/runner.rs`)
/// that supports multiple test frameworks (cargo test, pytest, jest, go test).
/// Vitest is implemented as a separate command instead of being wired through
/// `rtk test` because it requires specialized output processing (ANSI stripping,
/// Vitest-specific summary parsing, and custom tracking) that does not currently
/// fit cleanly into the generic runner abstraction.
///
/// If the generic `rtk test` interface is extended in the future to support this
/// style of output parsing, it may be worth refactoring Vitest into that path so
/// users can run it via `rtk test pnpm vitest run`. Until then, this command
/// intentionally remains separate.

Copilot uses AI. Check for mistakes.
Comment on lines +18 to +20
let mut cmd = Command::new("pnpm");
cmd.arg("vitest");
cmd.arg("run"); // Force non-watch mode
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoding "pnpm" as the package manager makes this command incompatible with projects using npm, yarn, or bun. Consider detecting the package manager from the project (e.g., by checking for lock files like pnpm-lock.yaml, package-lock.json, yarn.lock, or bun.lockb) or allowing users to specify it via an environment variable or config option.

Copilot uses AI. Check for mistakes.
Comment on lines +54 to +55
// Match ANSI escape sequences: \x1b[...m
let ansi_regex = Regex::new(r"\x1b\[[0-9;]*m").unwrap();
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ANSI escape sequence regex pattern \x1b\[[0-9;]*m only matches color/style codes ending in 'm'. However, ANSI escape sequences can end with other characters (like 'H' for cursor position, 'J' for clear screen, 'K' for clear line, etc.). While the current pattern may work for typical Vitest output, consider using a more comprehensive pattern like \x1b\[[0-9;]*[mGKHJfABCDsu] or \x1b\[[^m]*m to handle edge cases.

Suggested change
// Match ANSI escape sequences: \x1b[...m
let ansi_regex = Regex::new(r"\x1b\[[0-9;]*m").unwrap();
// Match common ANSI escape sequences (colors, cursor movement, clear line/screen, etc.)
let ansi_regex = Regex::new(r"\x1b\[[0-9;]*[mGKHJfABCDsu]").unwrap();

Copilot uses AI. Check for mistakes.
}

// Pattern: "Duration 3.05s" (with optional details in parens)
if let Some(caps) = Regex::new(r"Duration\s+([\d.]+[ms]+)").unwrap().captures(&clean_output) {
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern has a potential issue with the character class. The pattern [ms]+ will match any combination of 'm' and 's' characters (like "msms", "sssm", etc.), not just "ms" or "s". Consider using a non-capturing group like (?:ms|s) to match either "ms" or "s" as a unit.

Suggested change
if let Some(caps) = Regex::new(r"Duration\s+([\d.]+[ms]+)").unwrap().captures(&clean_output) {
if let Some(caps) = Regex::new(r"Duration\s+([\d.]+(?:ms|s))").unwrap().captures(&clean_output) {

Copilot uses AI. Check for mistakes.
Comment on lines +173 to +187
// If parsing failed, return cleaned output (fallback)
if result.len() <= 1 {
return strip_ansi(output)
.lines()
.filter(|line| {
// Keep only meaningful lines
let trimmed = line.trim();
!trimmed.is_empty()
&& !trimmed.starts_with("│")
&& !trimmed.starts_with("├")
&& !trimmed.starts_with("└")
})
.collect::<Vec<_>>()
.join("\n");
}
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic for detecting failed parsing (result.len() <= 1) is fragile. If stats.total is 0 or there are no failures or duration, the fallback will trigger even when parsing succeeded but found no tests. Consider checking if parsing genuinely failed (e.g., no regex matches at all) rather than just checking result length.

Copilot uses AI. Check for mistakes.
Comment on lines +174 to +175
// Check if user provided limit flag
let has_limit_flag = args.iter().any(|arg| arg.starts_with('-') && arg.chars().nth(1).map_or(false, |c| c.is_ascii_digit()));
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit flag detection using arg.starts_with('-') && arg.chars().nth(1).map_or(false, |c| c.is_ascii_digit()) is overly simplistic and will incorrectly match flags like "-2way" or other options that start with a digit. Consider using a more robust pattern or explicitly checking for -n followed by digits, or using a regex pattern like ^-\d+$.

Suggested change
// Check if user provided limit flag
let has_limit_flag = args.iter().any(|arg| arg.starts_with('-') && arg.chars().nth(1).map_or(false, |c| c.is_ascii_digit()));
// Check if user provided limit flag (e.g., -10), matching ^-\d+$
let has_limit_flag = args.iter().any(|arg| {
arg.len() > 1
&& arg.starts_with('-')
&& !arg.starts_with("--")
&& arg[1..].chars().all(|c| c.is_ascii_digit())
});

Copilot uses AI. Check for mistakes.
**Actions restantes**:
1. ✅ **Test sur Aristote** (1h)
```bash
cd /Users/florianbruniaux/Sites/MethodeAristote/app
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This roadmap file contains a hardcoded personal file path "/Users/florianbruniaux/Sites/MethodeAristote/app" which should not be included in the repository. Consider using a generic example path or environment variable placeholder instead.

Suggested change
cd /Users/florianbruniaux/Sites/MethodeAristote/app
cd /path/to/MethodeAristote/app

Copilot uses AI. Check for mistakes.
@pszymkowiak pszymkowiak merged commit 5f7adc9 into rtk-ai:master Jan 28, 2026
5 of 6 checks passed
FlorianBruniaux added a commit to FlorianBruniaux/rtk that referenced this pull request Jan 29, 2026
FlorianBruniaux added a commit to FlorianBruniaux/rtk that referenced this pull request Jan 30, 2026
Applies targeted code quality improvements:

Issue rtk-ai#5: Reduce clones in HashMap merge
- Destructure CcusagePeriod to avoid cloning key
- Use .or_insert_with_key() for ccusage data merging
- Keep necessary clones for rtk data (HashMap ownership requirement)
- Affects: merge_daily, merge_weekly, merge_monthly

Issue rtk-ai#7: Replace magic number with named constant
- Add const BILLION: f64 = 1e9
- Improves readability in cache token calculation

Issue rtk-ai#8: Add NaN/Infinity safety check
- Guard format_usd() against invalid float values
- Return "$0.00" for non-finite inputs

Build: Clean (1 pre-existing warning)
Tests: 79/82 passing (3 pre-existing failures)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
ahundt pushed a commit to ahundt/rtk that referenced this pull request Feb 23, 2026
Add Vitest test runner support with 99.6% token reduction
ahundt pushed a commit to ahundt/rtk that referenced this pull request Feb 23, 2026
Applies targeted code quality improvements:

Issue rtk-ai#5: Reduce clones in HashMap merge
- Destructure CcusagePeriod to avoid cloning key
- Use .or_insert_with_key() for ccusage data merging
- Keep necessary clones for rtk data (HashMap ownership requirement)
- Affects: merge_daily, merge_weekly, merge_monthly

Issue rtk-ai#7: Replace magic number with named constant
- Add const BILLION: f64 = 1e9
- Improves readability in cache token calculation

Issue rtk-ai#8: Add NaN/Infinity safety check
- Guard format_usd() against invalid float values
- Return "$0.00" for non-finite inputs

Build: Clean (1 pre-existing warning)
Tests: 79/82 passing (3 pre-existing failures)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants