/tag-and-rerun-ci

Summary of Changes

Hello @fzyzcjy, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request performs a targeted refactoring to improve the handling of RAII guards associated with HTTP response bodies. It introduces a new generic utility, AttachedBody, which centralizes the logic for attaching a lifetime-managed object to an axum::response::Response. This change streamlines the implementation of resource management patterns, such as decrementing worker load or returning tokens, by abstracting the body-wrapping mechanism. This refactoring is a prerequisite for future enhancements to the ManualPolicy by providing a cleaner and more consistent way to manage resources tied to response lifetimes.

Highlights

• New Generic AttachedBody Utility: Introduced a new generic AttachedBody<T> struct in sgl-model-gateway/src/utils/http_utils.rs that wraps an axum::body::Body and holds an attached type T. This utility centralizes the logic for associating a resource (like a RAII guard) with the lifetime of an HTTP response body, ensuring the resource is dropped when the body is fully consumed or the connection closes.
• Refactored Worker Load Guards: The WorkerLoadGuard in sgl-model-gateway/src/core/worker.rs has been refactored to utilize the new AttachedBody utility. This change removes the custom GuardedBody and MultiGuardedBody implementations, simplifying the code for decrementing worker load when a response stream completes.
• Refactored Token Guards in Middleware: The TokenGuard in sgl-model-gateway/src/middleware.rs has been updated to leverage the AttachedBody utility. The previous TokenGuardBody struct was removed, and TokenGuardBody is now a type alias for AttachedBody<TokenGuard>, streamlining the mechanism for returning tokens to a bucket upon stream completion.
• Code Simplification and Modularity: This refactoring effort reduces code duplication by consolidating generic body-wrapping logic into a single utility, improving modularity and maintainability across the codebase. It also removes several http_body::Body implementations that are now handled by the new AttachedBody.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

done