Skip to content

perf(logging): strip debug log messages at compile-time (backport #4365)#4375

Merged
mergify[bot] merged 2 commits intov1.xfrom
mergify/bp/v1.x/pr-4365
Oct 31, 2024
Merged

perf(logging): strip debug log messages at compile-time (backport #4365)#4375
mergify[bot] merged 2 commits intov1.xfrom
mergify/bp/v1.x/pr-4365

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Oct 31, 2024

partially close: #2847

Logging debug messages can lead to significant memory allocations, especially when outputting variable values. In Go, even if log_level is not set to debug, these allocations can still occur because the program evaluates the debug statements regardless of the log level.

To prevent unnecessary memory usage, you can strip out all debug-level code from the binary at compile time using build flags. This approach improves the performance of CometBFT by excluding debug messages entirely, even when log_level is set to debug. This technique is ideal for production environments that prioritize performance optimization over debug logging.

Note: Compiling CometBFT with this method will completely disable all debug messages. If you require debug output, avoid compiling the binary with these settings.

In order to build a binary stripping all debug log messages (e.g. log.Debug()) from the binary, use the nodebug tag:

COMETBFT_BUILD_OPTIONS=nodebug make install

once it's installed you can run a kvstore example with log_level = "debug" set in the config.toml and you will see that no debug messages are shown in the log.

cc: @ValarDragon

PR checklist

  • Tests written/updated
  • Changelog entry added in .changelog (we use unclog to manage our changelog)
  • Updated relevant documentation (docs/ or spec/) and code comments

This is an automatic backport of pull request #4365 done by [Mergify](https://mergify.com).

partially close: #2847

Logging debug messages can lead to significant memory allocations,
especially when outputting variable values. In Go, even if log_level is
not set to debug, these allocations can still occur because the program
evaluates the debug statements regardless of the log level.

To prevent unnecessary memory usage, you can strip out all debug-level
code from the binary at compile time using build flags. This approach
improves the performance of CometBFT by excluding debug messages
entirely, even when `log_level` is set to `debug`. This technique is
ideal for production environments that prioritize performance
optimization over debug logging.

> Note: Compiling CometBFT with this method will completely disable all
debug messages. If you require debug output, avoid compiling the binary
with these settings.

In order to build a binary stripping all debug log messages (e.g.
`log.Debug()`) from the binary, use the `nodebug` tag:
```
COMETBFT_BUILD_OPTIONS=nodebug make install
```

once it's installed you can run a `kvstore` example with `log_level =
"debug"` set in the `config.toml` and you will see that no debug
messages are shown in the log.

cc: @ValarDragon

#### PR checklist

- [ ] ~~Tests written/updated~~
- [X] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [X] Updated relevant documentation (`docs/` or `spec/`) and code
comments

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
(cherry picked from commit 51e9948)
@mergify mergify bot requested a review from a team as a code owner October 31, 2024 13:09
@mergify mergify bot requested a review from a team October 31, 2024 13:09
@andynog andynog added P:operator-experience Priority: Improve experience for operators logs Anything relating to logging labels Oct 31, 2024
Copy link
Collaborator

@andynog andynog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mergify mergify bot merged commit a02eea3 into v1.x Oct 31, 2024
@mergify mergify bot deleted the mergify/bp/v1.x/pr-4365 branch October 31, 2024 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logs Anything relating to logging P:operator-experience Priority: Improve experience for operators

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant