Feature to monitor client process system usage#617
Merged
Conversation
5bb5294 to
f964ae3
Compare
hoytak
reviewed
Feb 4, 2026
hoytak
reviewed
Feb 4, 2026
hoytak
reviewed
Feb 4, 2026
hoytak
reviewed
Feb 4, 2026
hoytak
reviewed
Feb 4, 2026
hoytak
reviewed
Feb 4, 2026
hoytak
reviewed
Feb 11, 2026
hoytak
approved these changes
Feb 27, 2026
Collaborator
hoytak
left a comment
There was a problem hiding this comment.
LGTM; some possible adjustments but these can be done later.
hoytak
pushed a commit
that referenced
this pull request
Feb 28, 2026
Introduces a client benchmark utility to track system resource usage (CPU, memory, disk I/O, and network I/O) of a process, so we don't need to write scripts to capture usage stats according to different OS standards. This becomes extremely helpful when I benchmark on Python notebook instances, e.g. Google Colab, where system monitor is not easily accessible or when running a separate monitor script is not easy. # Usage # Users can enable monitoring by setting `HF_XET_SYSTEM_MONITOR_ENABLED` to true, set usage sample interval using `HF_XET_SYSTEM_MONITOR_SAMPLE_INTERVAL`, this outputs metrics to the tracing stream at `INFO` level by default. In addition, these metrics can be redirected to a separate file by setting sample log path using `HF_XET_SYSTEM_MONITOR_LOG_PATH`. # Output # The stats are output in JSON format, which can be queried using tools like `jq`, e.g. 1. Trace of peak memory usage: `jq '.memory.peak_used_bytes' [HF_XET_SYSTEM_MONITOR_LOG_PATH]` 2. Trace of disk write speed: `jq '.disk.average_write_speed' [HF_XET_SYSTEM_MONITOR_LOG_PATH]` 3. Trace of network receive speed: `jq '.network.average_rx_speed' [HF_XET_SYSTEM_MONITOR_LOG_PATH]`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces a client benchmark utility to track system resource usage (CPU, memory, disk I/O, and network I/O) of a process, so we don't need to write scripts to capture usage stats according to different OS standards. This becomes extremely helpful when I benchmark on Python notebook instances, e.g. Google Colab, where system monitor is not easily accessible or when running a separate monitor script is not easy.
Usage
Users can enable monitoring by setting
HF_XET_SYSTEM_MONITOR_ENABLEDto true, set usage sample interval usingHF_XET_SYSTEM_MONITOR_SAMPLE_INTERVAL, this outputs metrics to the tracing stream atINFOlevel by default. In addition, these metrics can be redirected to a separate file by setting sample log path usingHF_XET_SYSTEM_MONITOR_LOG_PATH. Example:Output
The stats are output in JSON format, which can be queried using tools like
jq, e.g.jq '.memory.peak_used_bytes' [HF_XET_SYSTEM_MONITOR_LOG_PATH]jq '.disk.average_write_speed' [HF_XET_SYSTEM_MONITOR_LOG_PATH]jq '.network.average_rx_speed' [HF_XET_SYSTEM_MONITOR_LOG_PATH]