Skip to content

log: reopen disk files on SIGHUP#848

Merged
mmatczuk merged 5 commits intomainfrom
mmt/hup_reopen
Jun 25, 2024
Merged

log: reopen disk files on SIGHUP#848
mmatczuk merged 5 commits intomainfrom
mmt/hup_reopen

Conversation

@mmatczuk
Copy link
Contributor

No description provided.

Allow to close the underlying file.
@mmatczuk mmatczuk requested a review from Choraden as a code owner June 24, 2024 13:37
Comment on lines +41 to +45
defer func() {
logger.Close()
}()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just defer logger.Close()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually maybe we should be printing the error to stderr

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 👍

log/rotate.go Outdated
Comment on lines +61 to +79
for range ch {
if err := w.Reopen(); err != nil {
fmt.Fprintf(os.Stderr, "failed to rotate log file: %v\n", err)
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this goroutine ends. What closes the channel?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we close it in RotatableFile.Close()?

@Choraden
Copy link
Contributor

Let's comment that forwarder supports SIGHUP log rotation in --log-file flag usage.

log/rotate.go Outdated
Comment on lines +43 to +46
select {
case w.ch <- closeSignal:
default:
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand, it is to protect consecutive Close calls, but what If the channel is occupied with SIGHUPs and you only call close once. It's a rare case, but not impossible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This protects against blocking on Close() when Close is called multiple times after to Goroutine exited.

Note there is a race condition between Close and Reopen (and write) using atomics.
We could solve it by locking but then all logging would go under RW lock read path.
Given the use case I do not think it's a right approach.

There is not much we can do here tbh given this design - that's why there was no cancelation in the 1st place - it's a virtual problem.

Some of the common aids do not work here

  • Closing the channel causes panics in signal and Subsequent calls to Close
  • Using separate channel has the same problem
  • Increasing channel buffer size does not work as you can have multiple signals queued in theory

Maybe you can suggest something that I'm missing.

Copy link
Contributor

@Choraden Choraden Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about use sync.Once to send closeSignal

mmatczuk added 2 commits June 25, 2024 13:26
Add RotatableFile that allows for file reopening files on SIGHUP.
This allows Forwarder to work with tools like logrotate.
@mmatczuk mmatczuk merged commit cd3b2e7 into main Jun 25, 2024
@mmatczuk mmatczuk deleted the mmt/hup_reopen branch June 25, 2024 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants