Skip to content

fix: handle missing pss attribute in benchmark on Windows#4160

Merged
johanneskoester merged 3 commits into
snakemake:mainfrom
PatrikMrnka:fix/benchmark-pss-windows
May 29, 2026
Merged

fix: handle missing pss attribute in benchmark on Windows#4160
johanneskoester merged 3 commits into
snakemake:mainfrom
PatrikMrnka:fix/benchmark-pss-windows

Conversation

@PatrikMrnka

@PatrikMrnka PatrikMrnka commented Apr 19, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #4159

On Windows, psutil.Process.memory_full_info() does not include the pss
field (Proportional Set Size is a Linux-only metric). In
BenchmarkTimer._update_record(), the line pss += meminfo.pss raises
AttributeError, which is silently caught by work():

except AttributeError:
    pass  # skip, process died in flight

This causes every polling cycle to be discarded, so all benchmark metrics
(max_rss, max_vms, max_uss, io_in, io_out, etc.) are written as NA.

Changes

  • snakemake/benchmark.py: Replace meminfo.pss with
    getattr(meminfo, "pss", 0) — one-line change.
  • tests/test_benchmark_pss.py: Unit test that reproduces the bug
    (mock meminfo without pss attribute). Confirms the fix on all
    platforms.

Testing

  • test_update_record_without_pss — fails before fix, passes after
  • test_update_record_with_pss — passes both before and after (Linux
    regression check)

No impact on Linux/macOS — pss attribute exists there, so getattr
returns it normally.

Summary by CodeRabbit

  • Bug Fixes

    • Improved memory-metrics collection to tolerate missing platform-specific memory fields, preventing failures and ensuring consistent resource reporting across systems.
  • Tests

    • Added regression tests validating metric collection both when the optional memory field is present and when it is absent (Windows-like), ensuring reliability across environments.

Review Change Stack

On Windows, psutil.memory_full_info() does not include the 'pss' field
(Linux-only metric). Accessing meminfo.pss raised AttributeError, which
was silently caught by work(), causing all benchmark metrics to be NA.

Use getattr(meminfo, 'pss', 0) instead of direct attribute access.

Fixes snakemake#4159
@coderabbitai

coderabbitai Bot commented Apr 19, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2dd58b25-20b4-47a4-be7b-f2006ad368c7

📥 Commits

Reviewing files that changed from the base of the PR and between 59a9984 and 9e8c5e0.

📒 Files selected for processing (2)
  • src/snakemake/benchmark.py
  • tests/test_benchmark_pss.py
💤 Files with no reviewable changes (1)
  • src/snakemake/benchmark.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/test_benchmark_pss.py

📝 Walkthrough

Walkthrough

Safely aggregates meminfo.pss using getattr to avoid AttributeError on platforms lacking pss, and adds regression tests that mock memory_full_info() with and without pss to validate metric collection.

Changes

Benchmark memory attribute fix

Layer / File(s) Summary
Safe pss aggregation
src/snakemake/benchmark.py
BenchmarkTimer._update_record() uses getattr(meminfo, "pss", 0) when accumulating PSS to handle missing pss attribute without raising AttributeError.
Regression tests for pss presence/absence
tests/test_benchmark_pss.py
New tests mock psutil.Process.memory_full_info() with and without pss, call BenchmarkTimer._update_record(), and assert RSS/VMS/USS/PS S metrics are populated as expected.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related issues

  • #4159 — Matches: this PR implements the suggested getattr fix to avoid AttributeError when meminfo.pss is missing on Windows.
  • #3665 — Matches: addresses the same BenchmarkTimer._update_record pss handling and adds tests for missing pss.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the core fix: handling the missing pss attribute on Windows in benchmark operations.
Description check ✅ Passed The description covers the issue, changes made, and testing approach, though it does not include explicit QC checkboxes or AI-assistance disclosure from the template.
Linked Issues check ✅ Passed The PR fully addresses issue #4159 by implementing the suggested fix with getattr() and adding comprehensive regression tests for both Windows and Linux scenarios.
Out of Scope Changes check ✅ Passed All changes are directly scoped to fixing the benchmark pss attribute issue: one-line fix in benchmark.py and new test module, with no extraneous modifications.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
tests/test_benchmark_pss.py (2)

45-61: Strengthen the regression assertion for the bug being fixed.

The key behavioral guarantee of the fix is that BenchmarkRecord.data_collected flips to True (and max_pss is 0) when pss is absent — that's precisely what was false before the fix and produced NA for every column. Asserting max_rss is not None would also pass if, say, a future refactor collected RSS but still swallowed the AttributeError elsewhere. Consider tightening:

Proposed assertion tightening
     timer._update_record()

-    assert record.max_rss is not None, "max_rss is None — metrics not collected"
-    assert record.max_vms is not None, "max_vms is None"
-    assert record.max_uss is not None, "max_uss is None"
-    assert record.max_rss == pytest.approx(100.0, abs=0.1)
+    assert record.data_collected, "benchmark metrics were not collected"
+    assert record.max_rss == pytest.approx(100.0, abs=0.1)
+    assert record.max_vms == pytest.approx(200.0, abs=0.1)
+    assert record.max_uss == pytest.approx(80.0, abs=0.1)
+    assert record.max_pss == 0, "pss should default to 0 when attribute is missing"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/test_benchmark_pss.py` around lines 45 - 61, The test
test_update_record_without_pss is too weak: it only asserts max_rss is not None
but should assert that BenchmarkRecord.data_collected becomes True and that
max_pss is set to 0 when pss is absent; update the test to check
record.data_collected is True and record.max_pss == 0 (in addition to existing
RSS/VMS/USS assertions) so the regression that previously left columns as NA is
prevented — locate the assertions in test_update_record_without_pss and add
checks for BenchmarkRecord.data_collected and max_pss.

49-54: Consider factoring out the timer construction to reduce duplication.

Both tests build a BenchmarkTimer via __new__ and manually wire the same four attributes. A small fixture/helper (e.g. _make_timer(proc)) would keep the tests focused on the behavioral difference (pss present vs absent) rather than boilerplate. Also, instantiating through __new__ bypasses __init__ — if future construction logic is added (e.g. additional state used by _update_record), these tests will silently go stale. A light alternative is patching psutil.Process and calling the real constructor.

Also applies to: 68-73

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/test_benchmark_pss.py` around lines 49 - 54, Factor out the repeated
BenchmarkTimer construction into a small helper (e.g. _make_timer(proc)) that
creates a BenchmarkRecord, constructs a BenchmarkTimer via its real constructor
(BenchmarkTimer(proc)) rather than using __new__, and returns the wired timer;
in the tests that currently do manual wiring (creating BenchmarkRecord(),
setting timer.pid, timer.main, timer.bench_record, timer.procs) replace that
block with calls to _make_timer(proc). If calling the real constructor requires
isolating psutil.Process behavior, patch psutil.Process in the test (or use a
fixture) so the real __init__ runs safely; update both test sites (the
pss-present and pss-absent tests) to use the new helper.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/test_benchmark_pss.py`:
- Around line 45-61: The test test_update_record_without_pss is too weak: it
only asserts max_rss is not None but should assert that
BenchmarkRecord.data_collected becomes True and that max_pss is set to 0 when
pss is absent; update the test to check record.data_collected is True and
record.max_pss == 0 (in addition to existing RSS/VMS/USS assertions) so the
regression that previously left columns as NA is prevented — locate the
assertions in test_update_record_without_pss and add checks for
BenchmarkRecord.data_collected and max_pss.
- Around line 49-54: Factor out the repeated BenchmarkTimer construction into a
small helper (e.g. _make_timer(proc)) that creates a BenchmarkRecord, constructs
a BenchmarkTimer via its real constructor (BenchmarkTimer(proc)) rather than
using __new__, and returns the wired timer; in the tests that currently do
manual wiring (creating BenchmarkRecord(), setting timer.pid, timer.main,
timer.bench_record, timer.procs) replace that block with calls to
_make_timer(proc). If calling the real constructor requires isolating
psutil.Process behavior, patch psutil.Process in the test (or use a fixture) so
the real __init__ runs safely; update both test sites (the pss-present and
pss-absent tests) to use the new helper.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: bbd7c6b4-63af-4a0a-94a9-98f1bc7f6ac9

📥 Commits

Reviewing files that changed from the base of the PR and between d8faabd and 59a9984.

📒 Files selected for processing (2)
  • src/snakemake/benchmark.py
  • tests/test_benchmark_pss.py

Copilot AI review requested due to automatic review settings May 29, 2026 12:28
@johanneskoester johanneskoester enabled auto-merge (squash) May 29, 2026 12:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes issue #4159 where benchmark metrics on Windows were all NA because psutil.Process.memory_full_info() does not expose a pss attribute on that platform, causing an AttributeError that was silently swallowed by BenchmarkTimer.work(). The fix safely accesses the attribute via getattr with a default of 0, and adds regression tests covering both platforms.

Changes:

  • Use getattr(meminfo, "pss", 0) in BenchmarkTimer._update_record() to avoid AttributeError on Windows.
  • Add unit tests reproducing the bug and verifying Linux-like behavior remains intact.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/snakemake/benchmark.py Safely read meminfo.pss (default 0) so non-Linux platforms still record other memory metrics.
tests/test_benchmark_pss.py New regression tests for _update_record() with and without a pss attribute.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@johanneskoester johanneskoester disabled auto-merge May 29, 2026 14:08
@johanneskoester johanneskoester merged commit da52080 into snakemake:main May 29, 2026
60 checks passed
johanneskoester pushed a commit that referenced this pull request May 29, 2026
🤖 I have created a release *beep* *boop*
---


##
[9.21.1](v9.21.0...v9.21.1)
(2026-05-29)


### Bug Fixes

* add default json function to benchmarks
([#4128](#4128))
([41fab22](41fab22))
* do not rerun when checkpoint job missing but downstream file exists
([#4124](#4124))
([a060b93](a060b93))
* ensure that error logs contain all available details
([#4183](#4183))
([74a86e9](74a86e9))
* handle missing pss attribute in benchmark on Windows
([#4160](#4160))
([da52080](da52080))
* implement Resources.setdefault
([#3968](#3968))
([2413e99](2413e99))
* reporting remote nodes number
([#3978](#3978))
([8c534f0](8c534f0))
* resolve pathvars before constructing storage queries
([#3969](#3969))
([bd15237](bd15237))
* storage temp() file cleanup with RemoteProvider
([#4189](#4189))
([898bad1](898bad1))
* tolerate FileNotFoundError in drop_iocache
([#4153](#4153))
([#4191](#4191))
([ce26b28](ce26b28))


### Documentation

* Added guide on debugging workflows
([#4029](#4029))
([3d052ae](3d052ae))
* **cli:** Remove broken ref bold markup
([#4204](#4204))
([1200ebf](1200ebf))
* remove duplicated resources attribute in rules.rst
([#4190](#4190))
([6c8ecdd](6c8ecdd))
* **rules:** Update script type hint advice
([#4193](#4193))
([6108712](6108712))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark directive produces NA for all metrics on Windows (AttributeError on meminfo.pss)

3 participants