Skip to content

Bump script timeout to 60 minutes per target#42

Merged
aallan merged 1 commit into
mainfrom
fix/script-timeout
Apr 8, 2026
Merged

Bump script timeout to 60 minutes per target#42
aallan merged 1 commit into
mainfrom
fix/script-timeout

Conversation

@aallan

@aallan aallan commented Apr 8, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Owner

Summary

  • 30-minute timeout was too short for slower flagship models (kimi-k2.5 timed out)
  • Bumped to 60 minutes as a safety net against hangs
  • Individual problem timeouts are a separate concern for the automated runs roadmap

Test plan

  • Verified kimi-k2-turbo-preview completes well within 60 minutes

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores
    • Extended the default execution timeout limit for benchmark operations from 30 minutes to 60 minutes, allowing longer-running operations to complete without interruption.

@aallan @claude
Bump script timeout from 30 to 60 minutes per target
8ef277e 
30 minutes was too aggressive for slower flagship models like
kimi-k2.5. 60 minutes is a reasonable safety net without silently
masking performance issues.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Apr 8, 2026 •
edited
Loading

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c523101d-cdeb-4535-b1e3-11e8d1ff0fa6

📥 Commits

Reviewing files that changed from the base of the PR and between 6cc3bac and 8ef277e.

📒 Files selected for processing (1)
  • scripts/run_full_benchmark.py
📝 Walkthrough

Walkthrough

The default execution timeout for benchmark target execution in the _run() function has been increased from 1800 seconds (30 minutes) to 3600 seconds (60 minutes), with the function signature and docstring updated accordingly.

Changes

Cohort / File(s) Summary
Timeout Configuration
scripts/run_full_benchmark.py		 Updated default timeout parameter in _run() function from 1800 to 3600 seconds; docstring updated to reflect the new default timeout for per-target execution.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Suggested labels

ci

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: increasing the script timeout from 30 to 60 minutes per target execution.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/script-timeout

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented Apr 8, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.94%. Comparing base (6cc3bac) to head (8ef277e).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main      #42   +/-   ##
=======================================
  Coverage   84.94%   84.94%           
=======================================
  Files          10       10           
  Lines        1116     1116           
=======================================
  Hits          948      948           
  Misses        168      168
Flag Coverage Δ
python 84.94% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aallan aallan merged commit 63c1f5d into main Apr 8, 2026
10 checks passed
@aallan aallan deleted the fix/script-timeout branch April 8, 2026 13:41
@coderabbitai coderabbitai Bot mentioned this pull request Apr 13, 2026
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@aallan