Skip to content

Fix memory and swap handling#138573

Open
ash-darin wants to merge 19 commits intoelastic:mainfrom
ash-darin:main
Open

Fix memory and swap handling#138573
ash-darin wants to merge 19 commits intoelastic:mainfrom
ash-darin:main

Conversation

@ash-darin
Copy link
Copy Markdown

@ash-darin ash-darin commented Nov 25, 2025

This fixes the copy/paste errors of #42725 and adds swap handling
This fixes #42157 for swap

This assigns correct "total" and "free" variables to functions, which seem to have been mixed up.

Additionally this provides zeroing of total/free variables for swap, as expexted by getUsage for swap and analog to Mem for memory. which was apparently not done for swap, though it is expected:

// Swap can similarly be reported as negative and in
// those cases, we force it to zero in which case we can no longer correctly report the used swap
// as (total-free) and should report it as zero.

This ought prevent observed errors of the form:

java.lang.IllegalArgumentException: Values less than -1 bytes are not supported: -206108094464b at org.elasticsearch.server@8.17.3/org.elasticsearch.common.unit.ByteSizeValue.<init>(ByteSizeValue.java:95) at org.elasticsearch.server@8.17.3/org.elasticsearch.common.unit.ByteSizeValue.ofBytes(ByteSizeValue.java:52) at org.elasticsearch.server@8.17.3/org.elasticsearch.monitor.os.OsStats$Swap.getUsed(OsStats.java:224) at

Please backport to 8.x!

Fixed sloppy use of variable usage (free/total mixed up)
set swap variables to zero as expected by getUsed function
@elasticsearchmachine elasticsearchmachine added v9.3.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Nov 25, 2025
@ash-darin ash-darin marked this pull request as ready for review November 25, 2025 12:43
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Nov 25, 2025

public Swap(long total, long free) {
assert total >= 0 : "expected total swap to be positive, got: " + total;
assert free >= 0 : "expected free swap to be positive, got: " + total;
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reports "total" instead of "free"

this.total = in.readLong();
assert this.total >= 0 : "expected total swap to be positive, got: " + total;
this.free = in.readLong();
assert this.free >= 0 : "expected free swap to be positive, got: " + total;
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reports "total" instead of "free"

total = 0;
}
if (adjustedTotal < 0) {
logger.error("negative adjusted total memory [{}] found in memory stats", total);
Copy link
Copy Markdown
Author

@ash-darin ash-darin Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reports "total" instead of "adjustedTotal"

adjustedTotal = 0;
}
if (free < 0) {
logger.error("negative free memory [{}] found in memory stats", total);
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reports "total" instead of "free"

assert total >= 0 : "expected total swap to be positive, got: " + total;
assert free >= 0 : "expected free swap to be positive, got: " + total;
assert free >= 0 : "expected free swap to be positive, got: " + free;
if (total < 0) {
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is expected by getUsage func, but apparently not done

@john-wagster john-wagster added the :Core/Infra/Metrics Metrics and metering infrastructure label Dec 2, 2025
@elasticsearchmachine elasticsearchmachine added Team:Core/Infra Meta label for core/infra team and removed needs:triage Requires assignment of a team area label labels Dec 2, 2025
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@ash-darin
Copy link
Copy Markdown
Author

@john-wagster Is there anything I missed in creation of this PR that I should have observed?

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 27, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 78fe23fa-78ad-4cff-b14e-9132c90ec79c

📥 Commits

Reviewing files that changed from the base of the PR and between f264c1d and eee7015.

📒 Files selected for processing (1)
  • server/src/main/java/org/elasticsearch/monitor/os/OsStats.java

📝 Walkthrough

Walkthrough

Fixed assertions and runtime validation in OsStats.java for swap and memory stats. Swap constructors and deserialization now use the correct assertion message referencing free, log an error and clamp total/free to 0 when negative values are encountered, and perform validation on local deserialized values before assigning fields. Mem logging was corrected so error messages report the actual adjustedTotal and free values (previously logged total).

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • 🛠️ Update Documentation: Commit on current branch
  • 🛠️ Update Documentation: Create PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
server/src/main/java/org/elasticsearch/monitor/os/OsStats.java (1)

217-222: ⚠️ Potential issue | 🟠 Major

Missing clamping in Swap(StreamInput in) when assertions disabled.

The Swap(long, long) constructor now clamps negative values to 0 when assertions are disabled (lines 205-212), but Swap(StreamInput in) only has assertions without the protective clamping. Compare with Mem(StreamInput in) (lines 296-319) which includes both assertions AND clamping.

If a negative swap value arrives via deserialization with assertions disabled, the runtime IllegalArgumentException can still occur.

Proposed fix
     public Swap(StreamInput in) throws IOException {
-        this.total = in.readLong();
+        long total = in.readLong();
         assert this.total >= 0 : "expected total swap to be positive, got: " + total;
-        this.free = in.readLong();
+        if (total < 0) {
+            logger.error("negative total swap [{}] deserialized in swap stats", total);
+            total = 0;
+        }
+        this.total = total;
+        long free = in.readLong();
         assert this.free >= 0 : "expected free swap to be positive, got: " + free;
+        if (free < 0) {
+            logger.error("negative free swap [{}] deserialized in swap stats", free);
+            free = 0;
+        }
+        this.free = free;
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@server/src/main/java/org/elasticsearch/monitor/os/OsStats.java` around lines
217 - 222, The Swap(StreamInput in) constructor only asserts non-negative values
but does not clamp them, so when assertions are disabled a negative deserialized
value can cause later IllegalArgumentException; update Swap(StreamInput in) to
mirror Mem(StreamInput in) by reading total and free, assert they are >= 0, then
clamp any negative total/free to 0 (like the Swap(long, long) constructor does)
so deserialized negative swap values are safely normalized.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@server/src/main/java/org/elasticsearch/monitor/os/OsStats.java`:
- Around line 217-222: The Swap(StreamInput in) constructor only asserts
non-negative values but does not clamp them, so when assertions are disabled a
negative deserialized value can cause later IllegalArgumentException; update
Swap(StreamInput in) to mirror Mem(StreamInput in) by reading total and free,
assert they are >= 0, then clamp any negative total/free to 0 (like the
Swap(long, long) constructor does) so deserialized negative swap values are
safely normalized.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 5fd1aa39-a896-43cc-96c0-9271f719c64e

📥 Commits

Reviewing files that changed from the base of the PR and between cd38464 and a52d5a8.

📒 Files selected for processing (1)
  • server/src/main/java/org/elasticsearch/monitor/os/OsStats.java

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@server/src/main/java/org/elasticsearch/monitor/os/OsStats.java`:
- Around line 227-230: In the Swap(StreamInput) deserialization block inside
class OsStats the statement `free = 0` is missing a trailing semicolon which
prevents compilation; add the semicolon so the assignment reads `free = 0;`
immediately after the existing logger.error call that references `free` (the
negative free swap check) so the Swap(StreamInput) method compiles correctly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: ce806616-5e68-4730-8fe0-a90886a0b96e

📥 Commits

Reviewing files that changed from the base of the PR and between a52d5a8 and f264c1d.

📒 Files selected for processing (1)
  • server/src/main/java/org/elasticsearch/monitor/os/OsStats.java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/Metrics Metrics and metering infrastructure external-contributor Pull request authored by a developer outside the Elasticsearch team Team:Core/Infra Meta label for core/infra team v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Negative used memory causes IllegalArgumentException in _cat/nodes

3 participants