feat(logs): support --since <humantime>, use pager by default#204
feat(logs): support --since <humantime>, use pager by default#204
--since <humantime>, use pager by default#204Conversation
Summary of ChangesHello @gaojunran, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly upgrades the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request significantly enhances the logs command by introducing time-based filtering with humantime, and improving user experience by using a pager by default. The implementation of time-based filtering using a binary search on log files is a clever optimization. The code is well-structured, and the accompanying documentation and test updates are comprehensive. I've identified a few potential panic points and areas where memory usage could be optimized in streaming scenarios. Addressing these will further improve the robustness of the new features.
src/cli/logs.rs
Outdated
|
|
||
| match pager_result { | ||
| Ok(mut child) => { | ||
| let stdin = child.stdin.as_mut().expect("Failed to open pager stdin"); |
There was a problem hiding this comment.
Using expect() here can cause the program to panic if it fails to get the pager's standard input. It would be more robust to handle this case gracefully, for instance by using if let Some(stdin) = child.stdin.as_mut() and falling back to printing directly to standard output if it's None, similar to how a failure to spawn the pager is already handled.
src/cli/logs.rs
Outdated
|
|
||
| match pager_result { | ||
| Ok(mut child) => { | ||
| let stdin = child.stdin.take().expect("Failed to open pager stdin"); |
There was a problem hiding this comment.
src/cli/logs.rs
Outdated
| pager_args.push("+G"); | ||
| } | ||
|
|
||
| pager_args.push(path.to_str().unwrap()); |
There was a problem hiding this comment.
Using unwrap() here can cause a panic if the log file path contains non-UTF-8 characters. While this might be unlikely for daemon IDs, it's safer to handle this possibility. Consider using to_string_lossy() to avoid a potential panic.
| pager_args.push(path.to_str().unwrap()); | |
| pager_args.push(path.to_string_lossy().as_ref()); |
src/cli/logs.rs
Outdated
| let lines: Vec<String> = reader.lines().map_while(Result::ok).collect(); | ||
| let parsed = parse_log_lines(&lines); |
There was a problem hiding this comment.
Reading the entire file into memory with collect() can lead to high memory usage for large log files. To make this a true streaming implementation, consider processing the file line-by-line instead of collecting all lines into a vector first. You could adapt the logic from parse_log_lines to be stateful and process lines as they are read.
src/cli/logs.rs
Outdated
| let lines: Vec<String> = reader.lines().map_while(Result::ok).collect(); | ||
| let parsed = parse_log_lines(&lines); |
There was a problem hiding this comment.
There was a problem hiding this comment.
Pull request overview
This PR refactors the logs command to enhance usability by adding human-readable time parsing, implementing automatic paging, and organizing log-related tests into a dedicated file.
Changes:
- Added support for
--sinceand--untilwith multiple time formats (humantime, time-only, full datetime) - Implemented automatic pager usage by default when viewing logs in interactive terminals
- Added
--rawand--no-pagerflags for more control over output formatting - Moved log-related tests from
test_e2e.rsto dedicatedtest_e2e_logs.rsfile - Renamed
--from/--toto--since/--until(breaking change) - Changed
-nfrom having a default value of 100 to being optional (breaking change) - Added
humantimedependency for relative time parsing
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
src/cli/logs.rs |
Major refactor: added time parsing (humantime, time-only, datetime), implemented pager support, added raw output mode, implemented binary search for efficient time-range filtering |
tests/test_e2e_logs.rs |
New file containing comprehensive tests for logs command including tail, follow, time filters, raw output, and pager functionality |
tests/test_e2e.rs |
Removed log-related tests that were moved to dedicated file |
docs/guides/logs.md |
Updated documentation with new time filter examples and pager usage |
docs/cli/logs.md |
Auto-generated documentation updates for new flags and options |
pitchfork.usage.kdl |
Updated usage specification for logs command with new flags |
docs/cli/commands.json |
Auto-generated JSON schema updates for CLI commands |
Cargo.toml |
Added humantime = "2" dependency |
Cargo.lock |
Lockfile updates for humantime dependency |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 9 changed files in this pull request and generated 9 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // If the interpreted time for today is in the future, assume the user meant yesterday | ||
| if dt > now | ||
| && let Some(yesterday) = today.pred_opt() | ||
| { | ||
| naive_dt = NaiveDateTime::new(yesterday, time); | ||
| dt = Local | ||
| .from_local_datetime(&naive_dt) | ||
| .single() | ||
| .ok_or_else(|| miette::miette!("Invalid or ambiguous datetime: '{}'", s))?; | ||
| } | ||
| return Ok(dt); |
There was a problem hiding this comment.
The logic for assuming "yesterday" when a time-only input is in the future (lines 949-958) might be confusing in edge cases. For example, if a user queries at 23:59 for logs "since 23:58", they would get logs from yesterday at 23:58, not from one minute ago today. Consider documenting this behavior more explicitly or reconsidering the heuristic.
There was a problem hiding this comment.
wrong explanation. It should be querying 23:59 at 23:58, and I think starting from yesterday is reasonable
| } | ||
| } | ||
| } | ||
| None => { |
There was a problem hiding this comment.
When encountering a log line without a timestamp (line 717-719), the binary search advances low forward. This assumes that lines without timestamps should be included in the results. However, this could lead to incorrect behavior if there are many consecutive lines without timestamps at the beginning of the file, potentially causing the search to skip over valid timestamped lines. Consider documenting this assumption or handling this case more explicitly.
| None => { | |
| None => { | |
| // Lines without a parseable timestamp are treated as if they should be | |
| // included in the final results. To preserve that behavior, the binary | |
| // search treats them as "within range" and simply advances `low` past | |
| // the current line, continuing the search from the next line. | |
| // | |
| // This means that if a file starts with many non-timestamped lines, | |
| // the binary search may converge to an offset after those lines. Callers | |
| // relying on different semantics (e.g. always starting at byte 0 when | |
| // the target time is before the first timestamp) must handle that | |
| // explicitly outside this function. |
There was a problem hiding this comment.
We assume every line have timestamp because it is stored by pitchfork itself
src/cli/logs.rs
Outdated
| if Command::new("bat").arg("--version").output().is_ok() { | ||
| "bat".to_string() |
There was a problem hiding this comment.
The check for bat availability (Command::new("bat").arg("--version").output().is_ok()) is performed synchronously every time a pager is needed, which could add latency. Consider caching this check or using a more efficient detection method, especially since the pager selection logic is duplicated three times in the code.
src/cli/logs.rs
Outdated
| if let Ok(output) = Command::new("tput").arg("lines").output() | ||
| && output.status.success() | ||
| && let Ok(s) = String::from_utf8(output.stdout) | ||
| && let Ok(h) = s.trim().parse::<usize>() | ||
| { | ||
| return Some(h); | ||
| } |
There was a problem hiding this comment.
The tput command is Unix-specific and not available on Windows. This will cause the terminal height detection to silently fail on Windows, falling back to the default of 24 lines. Consider using a cross-platform solution like the terminal_size crate or adding platform-specific implementations.
There was a problem hiding this comment.
we can consider Windows later because on Windows there are so many things to consider
|
bugbot run verbose=true |
|
Bugbot request id: serverGenReqId_feeefa0a-964d-4053-a499-4d2b32d379c7 |
jdx
left a comment
There was a problem hiding this comment.
Code Review (AI-generated)
Nice feature set — the --since humantime support, pager integration, and --raw mode are solid additions. A few things worth addressing:
1. Pager logic duplicated 3x
The pager selection + spawn logic (including the bat --version subprocess probe) is copy-pasted across output_with_pager, stream_logs_to_pager, and open_file_in_pager. Should be extracted into a shared helper.
2. Output formatting duplicated 5+ times
The if single_daemon { ... } else { ... } println pattern appears in output_logs, output_with_pager (Ok + Err branches), stream_logs_to_pager, and stream_logs_direct. A format_log_line helper would clean this up.
3. stream_logs_to_pager collects everything into memory
The default path (no -n, no time filter) collects all log lines into a Vec before writing to the pager. This defeats the purpose of streaming and could be problematic for large log files. Consider a streaming merge approach.
4. edim() in pager output path
edim() is applied to timestamps when piping to pager stdin. This works for less -R / bat, but if $PAGER is set to something that doesn't handle ANSI codes, it'll show garbage.
5. Binary search edge case with non-timestamped lines
In binary_search_log_position, the None (no parseable timestamp) case always advances low. This could skip valid lines that follow a multi-line log entry's continuation line sitting at the search boundary.
6. bat doesn't get +G equivalent
less gets +G to start at the end of output, but bat users will always start at the top. Minor UX inconsistency.
7. parse_time_input fallthrough precedence
If parse_datetime happens to match an input like "10:30", it won't reach the parse_time_only branch. The precedence chain works but could surprise — worth a comment.
8. &PathBuf → &Path
read_lines_in_time_range takes &PathBuf — idiomatic Rust prefers &Path.
9. has_time_filter passed redundantly
print_raw_lines receives from, to, and has_time_filter, but has_time_filter is just from.is_some() || to.is_some(). Could derive it internally.
Overall the features are good. Main structural concern is the duplication — extracting pager + formatting helpers would cut ~100 lines. The binary search edge case (5) is the most important correctness item.
|
@jdx ready for a review |
jdx
left a comment
There was a problem hiding this comment.
This review was AI-generated using Claude Code.
Nice work on this refactor! The streaming merger with min-heap and binary search for time filtering are well-designed. A few items to address:
Bug
Regex capture group issue in StreamingLogParser::next() (~line 618):
let re = regex!(r"^(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) (\w)+ (.*)$");(\w)+ captures only the last character of the log level. Should be (\w+):
let re = regex!(r"^(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) (\w+) (.*)$");Suggestions
-
bat detection (~line 443):
output().is_ok()can return true even when bat isn't found on some systems. Consider:Command::new("bat").arg("--version").output().map(|o| o.status.success()).unwrap_or(false)
-
Raw mode multi-daemon sorting:
print_raw_lineswithout time filter outputs files sequentially rather than interleaved by timestamp (unlike non-raw mode). Is this intentional? -
Help text formatting: The
--sincelong_help uses inline-separators which render as one long line. Consider actual newlines for readability.
Minor
- Verify all imports in the
use std::io::{...}block are needed - The spawned thread for pager streaming can't propagate errors back (acceptable for CLI, just noting)
Overall this is solid - the architecture is clean and tests are comprehensive. The regex bug should be fixed before merge.
|
@jdx ready for a review |
|
bugbot run |
jdx
left a comment
There was a problem hiding this comment.
This review was generated by Claude Code.
Nice work on this. The streaming merger, binary search for time filtering, and pager abstraction are well-designed. A few items:
1. Use crossterm::terminal::size() instead of spawning tput
get_terminal_height() spawns a tput subprocess on every invocation. crossterm is already a dependency — use it directly:
fn get_terminal_height() -> Option<usize> {
crossterm::terminal::size().ok().map(|(_, h)| h as usize)
}2. Binary search skips continuation lines incorrectly
In binary_search_log_position, when a line has no parseable timestamp (multi-line log continuation), the None arm always advances low. If the search lands on a continuation line right at the boundary of the target time, this can overshoot and skip the entry it belongs to. Consider scanning forward to the next timestamped line before making the comparison, rather than blindly advancing.
3. Yesterday fallback in parse_time_input applies to --until too
If --since 23:59 at 00:01 → yesterday makes sense. But --until 23:59 at 00:01 should mean today at 23:59, not yesterday. The fallback should only apply for --since.
4. Single-file pager path skips formatting
When there's one daemon and no -n/--since/--until, open_file_in_pager sends the raw file directly to the pager — no edim() on timestamps, and the daemon label token is visible. Every other code path formats the output. This is a UX inconsistency users will notice.
|
@jdx Addressed 1, 3 and 4. For problem 4, I used to send the raw file to the pager to get a higher speed. But it can not add style to the timestamp part, which is bad in UX. So I rollbacked this quick path after a decision, and if user meet the case to open a large log, they should do logrotate (which will be implemented later) or just use For problem 2, as every log is created by pitchfork itself, we can assume every line has a timestamp to reduce complexity on binary search. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 9 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
btw, the CI keep failing on the same test, but I cannot reproduce it in my local environment. And the github action for the same branch in my forked repo runs successfully: https://github.com/gaojunran/pitchfork/actions/runs/21559857662/job/62122424249 I think it might be caused by the rust cache? not sure and wait for your idea. |
## Summary - Fixes intermittent `test_stop_already_stopped` failure in CI (seen in #204 and potentially other PRs) - Root cause: test asserted daemon status immediately after `stop` returned, before the supervisor fully persisted the state file — under CI load, reading a partially-written state file caused a TOML parse error - Adds `wait_for_status` helper to `TestEnv` that polls daemon status with retries (up to 2s) - Applies the fix to all three stop command tests: `test_stop_transitions_to_stopped`, `test_stop_kills_child_processes`, and `test_stop_already_stopped` ## Test plan - [x] All 115 tests pass locally with `cargo nextest run` - [x] The three fixed tests pass reliably when run together (`cargo nextest run test_stop`) - [ ] CI passes on this branch 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Low risk: test-only changes that add a small retry loop to tolerate CI timing/state persistence delays; no production code paths are modified. > > **Overview** > Makes stop-related E2E tests less flaky under CI load by adding `TestEnv::wait_for_status` (polls `pitchfork status` up to ~2s) and using it to assert the final `stopped` state. > > Updates `test_stop_transitions_to_stopped`, `test_stop_kills_child_processes`, and `test_stop_already_stopped` to wait for `stopped` after `stop`, avoiding transient reads while the supervisor is still writing the state file. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit ba00400. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
|
bugbot run |
## Summary - Fixes intermittent `test_stop_already_stopped` failure in CI (seen in #204) - Root cause: test asserted daemon status immediately after `stop` returned, before the supervisor fully persisted the state file — under CI load, reading a partially-written state file caused a TOML parse error (`None` status) - Adds `wait_for_status` helper to `TestEnv` that polls daemon status with retries (up to 5s) - Applies the fix to all three stop command tests ## Test plan - [x] All 115 tests pass locally with `cargo nextest run` - [x] The three fixed tests pass reliably when run together - [ ] CI passes 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Low risk: test-only change that increases retry window for daemon status polling; primary impact is slightly longer worst-case test runtime on failures. > > **Overview** > Extends the `TestEnv::wait_for_status` polling window from 2s to 5s (25 x 200ms) and updates the related docstring/error message, to better tolerate CI delays when persisting daemon state. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 21a6a09. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
docs/guides/logs.md:1
- The long help text for the
--sinceflag lacks proper formatting with newlines or bullet points between format types. The text runs together making it harder to read. Consider adding newlines (\n) between the format types to improve readability in the help output.
# Log Management
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| l.2.push('\n'); | ||
| l.2.push_str(&line); |
There was a problem hiding this comment.
When appending continuation lines to multi-line log entries, consider whether the message originally included a trailing newline. Currently, this unconditionally adds \n before appending the continuation line, which may result in extra newlines if the original message already had one. Verify that the log parsing regex and line handling correctly trim newlines before this point to avoid duplicate blank lines in output.
| l.2.push('\n'); | |
| l.2.push_str(&line); | |
| if !l.2.ends_with('\n') { | |
| l.2.push('\n'); | |
| } | |
| let trimmed = line.trim_end_matches('\n'); | |
| l.2.push_str(trimmed); |
| return false; | ||
| } | ||
|
|
||
| let terminal_height = get_terminal_height().unwrap_or(24); |
There was a problem hiding this comment.
The magic number 24 is used as a fallback terminal height. Consider defining this as a named constant (e.g., DEFAULT_TERMINAL_HEIGHT) to improve code readability and make it easier to adjust if needed.
|
|
||
| // Start searching from the byte just before `pos`. | ||
| let mut search_pos = pos.saturating_sub(1); | ||
| const CHUNK_SIZE: usize = 8192; |
There was a problem hiding this comment.
The chunk size of 8192 bytes for reading log files is a magic number. Consider defining this as a module-level constant with a descriptive name like LOG_SEARCH_CHUNK_SIZE to clarify its purpose and make it easier to tune performance.
| arg <FROM> | ||
| flag "-t --tail --follow" help="Show logs in real-time" | ||
| flag "-s --since" help="Show logs from this time" { | ||
| long_help "Show logs from this time\n\nSupports multiple formats: - Full datetime: \"YYYY-MM-DD HH:MM:SS\" or \"YYYY-MM-DD HH:MM\" - Time only: \"HH:MM:SS\" or \"HH:MM\" (uses today's date) - Relative time: \"5min\", \"2h\", \"1d\" (e.g., last 5 minutes)" |
There was a problem hiding this comment.
The long help text lacks proper formatting with bullet points appearing as plain text. Consider adding newlines between the format types (e.g., \n - Full datetime:...) to improve readability in the CLI help output.
## 🤖 New release * `pitchfork-cli`: 1.2.0 -> 1.3.0 <details><summary><i><b>Changelog</b></i></summary><p> <blockquote> ## [1.3.0](v1.2.0...v1.3.0) - 2026-02-01 ### Added - *(list)* show available daemons and align logics with TUI ([#206](#206)) - *(logs)* support `--since <humantime>`, use pager by default ([#204](#204)) - support `pitchfork.local.toml` ([#198](#198)) - impl `stop --all` ([#195](#195)) - beautify web ui ([#191](#191)) - add ready_cmd option ([#187](#187)) ### Fixed - refactor the logic of stopping a daemon and add tests ([#192](#192)) ### Other - re-order code to suit for multi-frontend structure ([#197](#197)) - *(deps)* update rust crate xx to v2.3.1 ([#203](#203)) - *(deps)* update rust crate clap to v4.5.56 ([#202](#202)) - *(ci)* run linting on all files in CI ([#196](#196)) - Update README.md logo ([#184](#184)) </blockquote> </p></details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/). <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Low risk: this PR only updates version numbers and release notes/docs, with no runtime code changes. > > **Overview** > Releases **v1.3.0** by bumping the crate version from `1.2.0` to `1.3.0` (including `Cargo.lock`) and updating the `pitchfork.usage.kdl`/generated CLI docs to match. > > Updates `CHANGELOG.md` with the `1.3.0` release notes and links to the included feature/fix PRs. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit d278b90. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Refactor
logscommand in CLI to make it fully usable. I will deal with TUI and Web UI later.