Skip to content

feat(recipe/tableflip): Add Graceful shutdown to ensure continuous up…#2710

Merged
ReneWerner87 merged 1 commit intogofiber:masterfrom
juls0730:patch-1
Dec 1, 2024
Merged

feat(recipe/tableflip): Add Graceful shutdown to ensure continuous up…#2710
ReneWerner87 merged 1 commit intogofiber:masterfrom
juls0730:patch-1

Conversation

@juls0730
Copy link
Contributor

@juls0730 juls0730 commented Dec 1, 2024

In the original Cloudflare Tableflip http example, they use a graceful shutdown method to ensure a continuous service uptime. In contrast, with the original fiber recipe, if you send the upgrade signal to the fiber process, current connections will die out (you can test with ab -c 50 -n 100000 http://127.0.0.1:8080/version and then send an upgrade signal while the test is running and see how the service dies). This commit fixes this issue with the original recipe and now actually allows for a continuous service uptime.

Summary by CodeRabbit

  • New Features
    • Introduced a timeout mechanism for graceful application shutdown, ensuring processes exit properly within 30 seconds.
    • Added functionality to wait for existing connections to drain before termination, improving overall stability during shutdown.

@welcome
Copy link

welcome bot commented Dec 1, 2024

Thanks for opening this pull request! 🎉 Please check out our contributing guidelines.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 1, 2024

Walkthrough

The changes in this pull request focus on enhancing the shutdown process of the application in tableflip/main.go. A timeout mechanism is introduced to ensure the application exits gracefully after the upg.Exit() channel is closed. Specifically, a time.AfterFunc is implemented to log a message and forcefully terminate the process if it does not shut down within 30 seconds. Additionally, the application now waits for existing connections to drain by calling app.ShutdownWithContext(context.Background()) before terminating.

Changes

File Change Summary
tableflip/main.go Added a timeout mechanism with time.AfterFunc for graceful shutdown; included app.ShutdownWithContext(context.Background()) to drain connections before termination.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Server
    participant Main

    User->>Main: Initiate shutdown
    Main->>Server: Call Shutdown()
    Server->>Main: Wait for connections to drain
    Main->>Main: Start timeout (30 seconds)
    Main->>Main: Log message if not shut down
    Main->>Main: Force exit if timeout reached
Loading

🐇 "In the meadow where bunnies play,
A timeout helps us end the day.
With graceful exits, we hop along,
Ensuring our process won't go wrong!
So here's to changes, bright and clear,
A smoother shutdown, let's all cheer!" 🐇


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (2)
tableflip/main.go (2)

51-57: Consider making the shutdown timeout configurable

The timeout implementation is good, but could be improved for better maintainability and observability.

Consider these improvements:

const (
        version = "v0.0.1"
+       shutdownTimeout = 30 * time.Second
)

// ... rest of the code ...

        // Make sure to set a deadline on exiting the process
        // after upg.Exit() is closed. No new upgrades can be
        // performed if the parent doesn't exit.
-       time.AfterFunc(30*time.Second, func() {
-               log.Println("Graceful shutdown timed out")
+       time.AfterFunc(shutdownTimeout, func() {
+               log.Printf("Process did not exit within %s, forcing exit", shutdownTimeout)
                os.Exit(1)
        })

Line range hint 41-42: Consider handling errors from app.Listener

The app.Listener is started in a goroutine but any errors it encounters would be silently ignored. This could mask important issues during operation.

Consider using an error channel to handle potential errors:

+       errChan := make(chan error, 1)
-       go app.Listener(ln)
+       go func() {
+               if err := app.Listener(ln); err != nil {
+                       log.Printf("Listener error: %v", err)
+                       errChan <- err
+                       upg.Stop()
+               }
+       }()

        // tableflip ready
        if err := upg.Ready(); err != nil {
                panic(err)
        }

-       <-upg.Exit()
+       select {
+       case <-upg.Exit():
+       case err := <-errChan:
+               log.Printf("Shutting down due to listener error: %v", err)
+       }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 6dbfc60 and d1ffcca.

📒 Files selected for processing (1)
  • tableflip/main.go (1 hunks)

…time

In the original Cloudflare Tableflip http example, they use a graceful shutdown method to ensure a continuous service uptime. In contrast, with the original fiber recipe, if you send the upgrade signal to the fiber process, current connections will die out (you can test with `ab -c 50 -n 100000 http://127.0.0.1:8080/version` and then send an upgrade signal while the test is running and see how the service dies). This commit fixes this issue with the original recipe and now actually allows for a continuous service uptime.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (1)
tableflip/main.go (1)

52-62: Consider adding debug logs for shutdown sequence

To aid in debugging shutdown-related issues, consider adding log statements to track the shutdown progress.

 <-upg.Exit()
+log.Println("Received exit signal, initiating graceful shutdown")
 
 // Make sure to set a deadline on exiting the process
 // after upg.Exit() is closed. No new upgrades can be
 // performed if the parent doesn't exit.
 time.AfterFunc(30*time.Second, func() {
     log.Println("Graceful shutdown timed out")
     os.Exit(1)
 })
 
 // Wait for connections to drain.
+log.Println("Waiting for connections to drain")
 app.ShutdownWithContext(context.Background())
+log.Println("Shutdown complete")
🧰 Tools
🪛 golangci-lint (1.62.2)

62-62: Error return value of app.ShutdownWithContext is not checked

(errcheck)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between d1ffcca and fb4d6c8.

📒 Files selected for processing (1)
  • tableflip/main.go (2 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
tableflip/main.go

62-62: Error return value of app.ShutdownWithContext is not checked

(errcheck)

🔇 Additional comments (2)
tableflip/main.go (2)

4-4: LGTM: Required imports added

The new imports are necessary for the graceful shutdown implementation.

Also applies to: 10-10


53-59: LGTM: Well-implemented timeout mechanism

The 30-second timeout is a good practice to prevent the process from hanging indefinitely during shutdown.

Comment on lines +61 to +62
// Wait for connections to drain.
app.ShutdownWithContext(context.Background())
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Enhance error handling and context management

The shutdown implementation could be improved in two ways:

  1. Handle the error returned by ShutdownWithContext
  2. Use a timeout context instead of context.Background()

Here's the suggested implementation:

 // Wait for connections to drain.
-app.ShutdownWithContext(context.Background())
+ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
+defer cancel()
+if err := app.ShutdownWithContext(ctx); err != nil {
+    log.Printf("Error during shutdown: %v", err)
+}

Note: The context timeout (25s) is set slightly shorter than the process timeout (30s) to ensure orderly shutdown.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Wait for connections to drain.
app.ShutdownWithContext(context.Background())
// Wait for connections to drain.
ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
defer cancel()
if err := app.ShutdownWithContext(ctx); err != nil {
log.Printf("Error during shutdown: %v", err)
}
🧰 Tools
🪛 golangci-lint (1.62.2)

62-62: Error return value of app.ShutdownWithContext is not checked

(errcheck)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my implementation is identical to how upstream handles it in their reference implementation, so YOLO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juls0730, understood. Thank you for the clarification.


✏️ Learnings added
Learnt from: juls0730
PR: gofiber/recipes#2710
File: tableflip/main.go:61-62
Timestamp: 2024-12-01T01:15:47.983Z
Learning: In the GoFiber `tableflip` recipe (`tableflip/main.go`), the implementation matches the upstream reference implementation. Future code suggestions should consider maintaining this alignment to ensure consistency.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

@ReneWerner87 ReneWerner87 merged commit bc8858b into gofiber:master Dec 1, 2024
@welcome
Copy link

welcome bot commented Dec 1, 2024

Congrats on merging your first pull request! 🎉 We here at Fiber are proud of you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants