Skip to content

🧹 chore: Improve Compress middleware RFC compliance#3745

Merged
ReneWerner87 merged 8 commits intomainfrom
review-compress-middleware-for-rfc-compliance
Sep 16, 2025
Merged

🧹 chore: Improve Compress middleware RFC compliance#3745
ReneWerner87 merged 8 commits intomainfrom
review-compress-middleware-for-rfc-compliance

Conversation

@gaby
Copy link
Member

@gaby gaby commented Sep 15, 2025

Summary

  • skip compression when Content-Encoding already set
  • add Accept-Encoding to Vary header when compressing
  • tokenize Vary merging and avoid appending when header is *
  • recompute strong ETags after compression using shared generator
  • skip compression for status codes with no body or range/206 responses
  • honor Cache-Control: no-transform on requests and responses
  • propagate compression headers on HEAD responses while omitting the body
  • emit Vary: Accept-Encoding even when compression is skipped, including when the request lacks Accept-Encoding
  • cover Vary header, pre-encoded responses, ETag recalculation, no-body skips, range requests, no-transform, HEAD metadata, and skip-branch Vary merging in tests
  • document compression behavior in middleware docs and whats_new

Related #3383

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 15, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Adds runtime compression behavior: skip rules (existing Content-Encoding, range/206, no body, Cache-Control: no-transform), HEAD negotiation mirroring encoded headers without body, recomputation of strong ETag for compressed payloads, Vary: Accept-Encoding merging, etag helpers, expanded tests, and docs/release-note updates.

Changes

Cohort / File(s) Summary of changes
Docs: compress behavior & release notes
docs/middleware/compress.md, docs/whats_new.md
New Behavior section describing skip conditions, HEAD handling, ETag/Vary rules; clarified zstd support and detailed compression behaviors in release notes.
Middleware: compression control & headers
middleware/compress/compress.go
Added header-token parsing, shouldSkip logic (status/no-body/Range/Cache-Control: no-transform), appendVaryAcceptEncoding, skip-path checks (including HEAD), Content-Encoding bypass, and recomputation of strong ETag when compression occurs.
Middleware: ETag helpers
middleware/etag/etag.go
Added Generate and GenerateWeak functions using a pre-initialized CRC32 table and centralized ETag generation; replaced per-call CRC setup.
Tests: compression behavior coverage
middleware/compress/compress_test.go
Large suite of tests added covering Vary merging (*/lists), Accept-Encoding negotiation, skip scenarios (existing Content-Encoding, Range/206, NoContent/NotModified, no-transform), strong vs weak ETag handling, and HEAD vs GET metadata parity; added etag test helper import.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant C as Client
  participant M as Compress Middleware
  participant H as Next Handler
  participant E as ETag Helper

  C->>M: Request (Accept-Encoding?, Range?, Cache-Control)
  M->>H: Call next handler
  H-->>M: Response (status, headers, body)

  alt Skip compression (Content-Encoding present, Range/206, no body, no-transform, HEAD)
    M->>M: append/merge Vary: Accept-Encoding (when appropriate)
    M-->>C: Return original response (headers/body as-is or headers-only for HEAD)
  else Compress
    M->>M: Negotiate encoding & compress body
    alt strong ETag present
      M->>E: Generate(compressedBody)
      E-->>M: New strong ETag
      M->>M: Set ETag header
    end
    M->>M: Ensure Vary: Accept-Encoding
    alt HEAD request
      M->>M: Set compressed Content-Length, omit body
      M-->>C: Return headers only
    else GET/others
      M-->>C: Return compressed response body
    end
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

📒 Documentation

Suggested reviewers

  • sixcolors
  • efectn
  • ReneWerner87

Poem

I twitched my whiskers at headers that vary,
Re-tagged the bytes so caches won't tarry.
Skip or compress, I check every flag—
For HEAD I whisper, for GET I wag.
Hoppity hop! gzip, br, zstd in the bag. 🐇✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.85% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The title concisely identifies the primary intent of the changeset: updating the Compress middleware for RFC compliance, which aligns with the PR’s code, tests, and documentation changes. It is a short, single-sentence phrase that accurately reflects the main change (Vary/ETag behavior, skip conditions, HEAD handling). The emoji and "chore:" prefix are stylistic but do not misrepresent the change.
Description Check ✅ Passed The PR description provides a clear summary of the behavioral changes (Vary/Accept-Encoding handling, ETag recomputation, skip conditions, no-transform, HEAD metadata), notes tests and docs updates, and references the related issue #3383. It does not fully follow the repository template: an explicit "Fixes #" line is missing and the "Changes introduced" checklist and "Type of change" selections are not filled out. Overall the description is mostly complete and actionable for reviewers, so I mark it as pass while recommending template completion.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch review-compress-middleware-for-rfc-compliance

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gaby gaby changed the title feat(compress): add Vary header, skip re-encoding, and support HEAD 🧹 chore: Improve Compress middleware RFC compliance Sep 15, 2025
@codecov
Copy link

codecov bot commented Sep 15, 2025

Codecov Report

❌ Patch coverage is 91.78082% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.02%. Comparing base (ba701cd) to head (5970397).
⚠️ Report is 184 commits behind head on main.

Files with missing lines Patch % Lines
middleware/etag/etag.go 76.92% 4 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3745      +/-   ##
==========================================
- Coverage   92.02%   92.02%   -0.01%     
==========================================
  Files         115      115              
  Lines       11585    11680      +95     
==========================================
+ Hits        10661    10748      +87     
- Misses        670      675       +5     
- Partials      254      257       +3     
Flag Coverage Δ
unittests 92.02% <91.78%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gaby gaby added this to v3 Sep 15, 2025
@gaby gaby added this to the v3 milestone Sep 15, 2025
@gaby gaby moved this to In Progress in v3 Sep 15, 2025
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.50.

Benchmark suite Current: 5970397 Previous: 4c73b2d Ratio
Benchmark_Compress/Gzip 429.8 ns/op 0 B/op 0 allocs/op 200.3 ns/op 0 B/op 0 allocs/op 2.15
Benchmark_Compress/Gzip - ns/op 429.8 ns/op 200.3 ns/op 2.15
Benchmark_Compress/Deflate 432.2 ns/op 0 B/op 0 allocs/op 223.2 ns/op 0 B/op 0 allocs/op 1.94
Benchmark_Compress/Deflate - ns/op 432.2 ns/op 223.2 ns/op 1.94
Benchmark_Compress/Brotli 437.5 ns/op 0 B/op 0 allocs/op 188.5 ns/op 0 B/op 0 allocs/op 2.32
Benchmark_Compress/Brotli - ns/op 437.5 ns/op 188.5 ns/op 2.32
Benchmark_Compress/Zstd 437 ns/op 0 B/op 0 allocs/op 227.2 ns/op 0 B/op 0 allocs/op 1.92
Benchmark_Compress/Zstd - ns/op 437 ns/op 227.2 ns/op 1.92
Benchmark_Compress_Levels/Gzip_LevelDefault 435.4 ns/op 0 B/op 0 allocs/op 212.6 ns/op 0 B/op 0 allocs/op 2.05
Benchmark_Compress_Levels/Gzip_LevelDefault - ns/op 435.4 ns/op 212.6 ns/op 2.05
Benchmark_Compress_Levels/Gzip_LevelBestSpeed 435.4 ns/op 0 B/op 0 allocs/op 213.5 ns/op 0 B/op 0 allocs/op 2.04
Benchmark_Compress_Levels/Gzip_LevelBestSpeed - ns/op 435.4 ns/op 213.5 ns/op 2.04
Benchmark_Compress_Levels/Gzip_LevelBestCompression 434.4 ns/op 0 B/op 0 allocs/op 205.7 ns/op 0 B/op 0 allocs/op 2.11
Benchmark_Compress_Levels/Gzip_LevelBestCompression - ns/op 434.4 ns/op 205.7 ns/op 2.11
Benchmark_Compress_Levels/Deflate_LevelDefault 440.6 ns/op 0 B/op 0 allocs/op 223.5 ns/op 0 B/op 0 allocs/op 1.97
Benchmark_Compress_Levels/Deflate_LevelDefault - ns/op 440.6 ns/op 223.5 ns/op 1.97
Benchmark_Compress_Levels/Deflate_LevelBestSpeed 439 ns/op 0 B/op 0 allocs/op 233.7 ns/op 0 B/op 0 allocs/op 1.88
Benchmark_Compress_Levels/Deflate_LevelBestSpeed - ns/op 439 ns/op 233.7 ns/op 1.88
Benchmark_Compress_Levels/Deflate_LevelBestCompression 436.5 ns/op 0 B/op 0 allocs/op 231.3 ns/op 0 B/op 0 allocs/op 1.89
Benchmark_Compress_Levels/Deflate_LevelBestCompression - ns/op 436.5 ns/op 231.3 ns/op 1.89
Benchmark_Compress_Levels/Brotli_LevelDefault 438.8 ns/op 0 B/op 0 allocs/op 192.7 ns/op 0 B/op 0 allocs/op 2.28
Benchmark_Compress_Levels/Brotli_LevelDefault - ns/op 438.8 ns/op 192.7 ns/op 2.28
Benchmark_Compress_Levels/Brotli_LevelBestSpeed 438 ns/op 0 B/op 0 allocs/op 197 ns/op 0 B/op 0 allocs/op 2.22
Benchmark_Compress_Levels/Brotli_LevelBestSpeed - ns/op 438 ns/op 197 ns/op 2.22
Benchmark_Compress_Levels/Brotli_LevelBestCompression 466.9 ns/op 16 B/op 0 allocs/op 203.6 ns/op 6 B/op 0 allocs/op 2.29
Benchmark_Compress_Levels/Brotli_LevelBestCompression - ns/op 466.9 ns/op 203.6 ns/op 2.29
Benchmark_Compress_Levels/Brotli_LevelBestCompression - B/op 16 B/op 6 B/op 2.67
Benchmark_Compress_Levels/Zstd_LevelDefault 437.5 ns/op 0 B/op 0 allocs/op 230.8 ns/op 0 B/op 0 allocs/op 1.90
Benchmark_Compress_Levels/Zstd_LevelDefault - ns/op 437.5 ns/op 230.8 ns/op 1.90
Benchmark_Compress_Levels/Zstd_LevelBestSpeed 438.8 ns/op 0 B/op 0 allocs/op 240.5 ns/op 0 B/op 0 allocs/op 1.82
Benchmark_Compress_Levels/Zstd_LevelBestSpeed - ns/op 438.8 ns/op 240.5 ns/op 1.82
Benchmark_Compress_Levels/Zstd_LevelBestCompression 439.7 ns/op 0 B/op 0 allocs/op 235.6 ns/op 0 B/op 0 allocs/op 1.87
Benchmark_Compress_Levels/Zstd_LevelBestCompression - ns/op 439.7 ns/op 235.6 ns/op 1.87
Benchmark_Compress_Parallel/Gzip 208 ns/op 0 B/op 0 allocs/op 93.69 ns/op 0 B/op 0 allocs/op 2.22
Benchmark_Compress_Parallel/Gzip - ns/op 208 ns/op 93.69 ns/op 2.22
Benchmark_Compress_Parallel/Deflate 209.1 ns/op 0 B/op 0 allocs/op 103.4 ns/op 0 B/op 0 allocs/op 2.02
Benchmark_Compress_Parallel/Deflate - ns/op 209.1 ns/op 103.4 ns/op 2.02
Benchmark_Compress_Parallel/Brotli 210.1 ns/op 0 B/op 0 allocs/op 87.73 ns/op 0 B/op 0 allocs/op 2.39
Benchmark_Compress_Parallel/Brotli - ns/op 210.1 ns/op 87.73 ns/op 2.39
Benchmark_Compress_Parallel/Zstd 209.3 ns/op 0 B/op 0 allocs/op 106 ns/op 0 B/op 0 allocs/op 1.97
Benchmark_Compress_Parallel/Zstd - ns/op 209.3 ns/op 106 ns/op 1.97
Benchmark_Compress_Levels_Parallel/Gzip_LevelDefault 212.2 ns/op 0 B/op 0 allocs/op 94.87 ns/op 0 B/op 0 allocs/op 2.24
Benchmark_Compress_Levels_Parallel/Gzip_LevelDefault - ns/op 212.2 ns/op 94.87 ns/op 2.24
Benchmark_Compress_Levels_Parallel/Gzip_LevelBestSpeed 215.7 ns/op 0 B/op 0 allocs/op 93.71 ns/op 0 B/op 0 allocs/op 2.30
Benchmark_Compress_Levels_Parallel/Gzip_LevelBestSpeed - ns/op 215.7 ns/op 93.71 ns/op 2.30
Benchmark_Compress_Levels_Parallel/Gzip_LevelBestCompression 209.2 ns/op 0 B/op 0 allocs/op 92.96 ns/op 0 B/op 0 allocs/op 2.25
Benchmark_Compress_Levels_Parallel/Gzip_LevelBestCompression - ns/op 209.2 ns/op 92.96 ns/op 2.25
Benchmark_Compress_Levels_Parallel/Deflate_LevelDefault 208.7 ns/op 0 B/op 0 allocs/op 104.7 ns/op 0 B/op 0 allocs/op 1.99
Benchmark_Compress_Levels_Parallel/Deflate_LevelDefault - ns/op 208.7 ns/op 104.7 ns/op 1.99
Benchmark_Compress_Levels_Parallel/Deflate_LevelBestSpeed 208.8 ns/op 0 B/op 0 allocs/op 109.4 ns/op 0 B/op 0 allocs/op 1.91
Benchmark_Compress_Levels_Parallel/Deflate_LevelBestSpeed - ns/op 208.8 ns/op 109.4 ns/op 1.91
Benchmark_Compress_Levels_Parallel/Deflate_LevelBestCompression 208.5 ns/op 0 B/op 0 allocs/op 104 ns/op 0 B/op 0 allocs/op 2.00
Benchmark_Compress_Levels_Parallel/Deflate_LevelBestCompression - ns/op 208.5 ns/op 104 ns/op 2.00
Benchmark_Compress_Levels_Parallel/Brotli_LevelDefault 208.8 ns/op 0 B/op 0 allocs/op 90.16 ns/op 0 B/op 0 allocs/op 2.32
Benchmark_Compress_Levels_Parallel/Brotli_LevelDefault - ns/op 208.8 ns/op 90.16 ns/op 2.32
Benchmark_Compress_Levels_Parallel/Brotli_LevelBestSpeed 208.9 ns/op 0 B/op 0 allocs/op 90.42 ns/op 0 B/op 0 allocs/op 2.31
Benchmark_Compress_Levels_Parallel/Brotli_LevelBestSpeed - ns/op 208.9 ns/op 90.42 ns/op 2.31
Benchmark_Compress_Levels_Parallel/Brotli_LevelBestCompression 233.5 ns/op 2 B/op 0 allocs/op 100.1 ns/op 1 B/op 0 allocs/op 2.33
Benchmark_Compress_Levels_Parallel/Brotli_LevelBestCompression - ns/op 233.5 ns/op 100.1 ns/op 2.33
Benchmark_Compress_Levels_Parallel/Brotli_LevelBestCompression - B/op 2 B/op 1 B/op 2
Benchmark_Compress_Levels_Parallel/Zstd_LevelDefault 209.7 ns/op 0 B/op 0 allocs/op 107.5 ns/op 0 B/op 0 allocs/op 1.95
Benchmark_Compress_Levels_Parallel/Zstd_LevelDefault - ns/op 209.7 ns/op 107.5 ns/op 1.95
Benchmark_Compress_Levels_Parallel/Zstd_LevelBestSpeed 210 ns/op 0 B/op 0 allocs/op 109.2 ns/op 0 B/op 0 allocs/op 1.92
Benchmark_Compress_Levels_Parallel/Zstd_LevelBestSpeed - ns/op 210 ns/op 109.2 ns/op 1.92
Benchmark_Compress_Levels_Parallel/Zstd_LevelBestCompression 214.4 ns/op 0 B/op 0 allocs/op 108.4 ns/op 0 B/op 0 allocs/op 1.98
Benchmark_Compress_Levels_Parallel/Zstd_LevelBestCompression - ns/op 214.4 ns/op 108.4 ns/op 1.98

This comment was automatically generated by workflow using github-action-benchmark.

@gaby gaby marked this pull request as ready for review September 15, 2025 05:35
@gaby gaby requested a review from a team as a code owner September 15, 2025 05:35
@gaby gaby requested review from ReneWerner87, Copilot, efectn and sixcolors and removed request for Copilot September 15, 2025 05:35
@gaby
Copy link
Member Author

gaby commented Sep 15, 2025

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly improves the RFC compliance of the Compress middleware, addressing many edge cases and improving caching behavior. The changes are well-structured, and the addition of comprehensive tests is excellent. I have a couple of suggestions to improve the implementation further. One is a high-severity fix for handling the Vary: * header, and the other is a medium-severity suggestion regarding a potential performance regression in the etag middleware due to the removal of bytebufferpool. Overall, this is a great contribution to improve Fiber's correctness and robustness.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (3)
docs/whats_new.md (1)

1130-1134: Doc-code mismatch: clarify when ETag is recalculated.

You state that strong ETags are recomputed “for compressed payloads.” In code, ETags are recomputed unconditionally after invoking the compressor, even if no compression actually occurs (e.g., small bodies). Please either clarify the doc or gate recomputation in code to only when Content-Encoding is set.

middleware/etag/etag.go (2)

45-45: Use canonical header casing for ETag.

[]byte("Etag") is non‑canonical. Prefer "ETag" to match fiber.HeaderETag conventions and avoid accidental mismatches with other code using canonical names.

Apply this diff:

-	normalizedHeaderETag := []byte("Etag")
+	normalizedHeaderETag := []byte("ETag")

78-84: Minor: avoid shadow naming with package name.

Local var etag shadows the package name etag in middleware/compress. Not a bug here, but renaming to tag/value reduces confusion in call sites and stack traces.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between aedffa1 and bb68ec6.

📒 Files selected for processing (5)
  • docs/middleware/compress.md (1 hunks)
  • docs/whats_new.md (1 hunks)
  • middleware/compress/compress.go (2 hunks)
  • middleware/compress/compress_test.go (2 hunks)
  • middleware/etag/etag.go (2 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
docs/**

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Review and update the contents of the docs folder if necessary when modifying code

Files:

  • docs/middleware/compress.md
  • docs/whats_new.md
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: Format Go code using gofumpt (enforced via make format)
Ensure code passes golangci-lint checks (enforced via make lint)
Optimize struct field alignment using betteralign (enforced via make betteralign)
Modernize Go code using gopls modernize (enforced via make modernize)

Files:

  • middleware/etag/etag.go
  • middleware/compress/compress.go
  • middleware/compress/compress_test.go
🧠 Learnings (2)
📚 Learning: 2024-10-02T23:03:31.727Z
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/store.go:164-167
Timestamp: 2024-10-02T23:03:31.727Z
Learning: Unit tests in this project use testify require.

Applied to files:

  • middleware/compress/compress_test.go
📚 Learning: 2024-07-26T21:00:12.902Z
Learnt from: sixcolors
PR: gofiber/fiber#2922
File: middleware/cors/utils.go:63-71
Timestamp: 2024-07-26T21:00:12.902Z
Learning: The project uses the testify/assert package for assertions in unit tests.

Applied to files:

  • middleware/compress/compress_test.go
🧬 Code graph analysis (3)
middleware/etag/etag.go (2)
middleware/compress/compress.go (1)
  • New (54-129)
middleware/etag/config.go (1)
  • Config (8-22)
middleware/compress/compress.go (3)
ctx_interface_gen.go (1)
  • Ctx (18-430)
constants.go (11)
  • StatusNoContent (56-56)
  • StatusResetContent (57-57)
  • StatusNotModified (67-67)
  • StatusPartialContent (58-58)
  • HeaderRange (236-236)
  • HeaderCacheControl (168-168)
  • HeaderVary (187-187)
  • HeaderAcceptEncoding (192-192)
  • HeaderContentEncoding (212-212)
  • HeaderETag (181-181)
  • MethodHead (6-6)
middleware/etag/etag.go (1)
  • Generate (17-29)
middleware/compress/compress_test.go (3)
middleware/compress/compress.go (1)
  • New (54-129)
middleware/etag/etag.go (2)
  • New (41-114)
  • Generate (17-29)
constants.go (12)
  • MethodGet (5-5)
  • HeaderVary (187-187)
  • HeaderContentEncoding (212-212)
  • HeaderETag (181-181)
  • HeaderContentType (216-216)
  • MIMETextPlainCharsetUTF8 (35-35)
  • HeaderContentLength (214-214)
  • MethodHead (6-6)
  • StatusNoContent (56-56)
  • StatusNotModified (67-67)
  • StatusPartialContent (58-58)
  • HeaderCacheControl (168-168)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Compare
🔇 Additional comments (4)
middleware/etag/etag.go (1)

16-39: ETag helpers look good.

Strong/weak helpers are clean and reusable. Prebuilt CRC table is a good micro-optimization.

docs/middleware/compress.md (1)

13-18: Behavior section reads well and aligns with RFCs.

Nice clarity on skip conditions, HEAD semantics, and Vary/ETag behavior.

middleware/compress/compress.go (2)

103-111: Skip path and Vary handling LGTM.

Early return with appendVaryAcceptEncoding covers RFC-compliant caching behavior for non-encoded variants.


121-125: HEAD metadata propagation: good implementation.

Capturing encoded length, resetting body, and setting Content-Length matches RFC semantics.

Copilot AI review requested due to automatic review settings September 15, 2025 05:57
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the Compress middleware to achieve better RFC compliance by implementing proper cache behavior, content negotiation, and response header management. The changes ensure compression works correctly with caching proxies and follows HTTP standards.

Key changes:

  • Implements proper Vary header management with Accept-Encoding
  • Adds compression skip conditions for various response scenarios
  • Recomputes strong ETags after compression using shared ETag generation

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
middleware/etag/etag.go Extracts ETag generation functions to be shared across middlewares
middleware/compress/compress.go Adds RFC-compliant compression logic with proper header handling
middleware/compress/compress_test.go Comprehensive test coverage for new compression behaviors
docs/whats_new.md Documents the compression improvements
docs/middleware/compress.md Updates middleware documentation with behavior details

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (3)
middleware/compress/compress.go (2)

12-19: Fix: hasToken won’t compile and misiterates slice (duplicate).

Replace SplitSeq and index iteration; trim spaces; prefer stdlib.

Apply:

@@
-func hasToken(header, token string) bool {
-	for part := range strings.SplitSeq(header, ",") {
-		if strings.EqualFold(utils.Trim(part, ' '), token) {
-			return true
-		}
-	}
-	return false
-}
+func hasToken(header, token string) bool {
+	for _, part := range strings.Split(header, ",") {
+		if strings.EqualFold(strings.TrimSpace(part), token) {
+			return true
+		}
+	}
+	return false
+}

Also drop the now‑unused utils import:

@@
-import (
-	"strings"
-
-	"github.com/gofiber/fiber/v3"
-	"github.com/gofiber/fiber/v3/middleware/etag"
-	"github.com/gofiber/utils/v2"
-	"github.com/valyala/fasthttp"
-)
+import (
+	"strings"
+
+	"github.com/gofiber/fiber/v3"
+	"github.com/gofiber/fiber/v3/middleware/etag"
+	"github.com/valyala/fasthttp"
+)

111-116: Only recompute strong ETag when compression actually happened (duplicate).

Gate on Content-Encoding and avoid clobbering app ETags; also guard nil.

Apply:

@@
-		if tag := c.GetRespHeader(fiber.HeaderETag); tag != "" && !strings.HasPrefix(tag, "W/") {
-			c.Set(fiber.HeaderETag, string(etag.Generate(c.Response().Body())))
-		}
+		if ce := c.GetRespHeader(fiber.HeaderContentEncoding); ce != "" {
+			if tag := c.GetRespHeader(fiber.HeaderETag); tag != "" && !strings.HasPrefix(tag, "W/") {
+				if newTag := etag.Generate(c.Response().Body()); len(newTag) != 0 {
+					c.Set(fiber.HeaderETag, string(newTag))
+				}
+			}
+		}
middleware/compress/compress_test.go (1)

352-374: Strong ETag recomputation test LGTM. Add guard for non-compress case (duplicate).

Add a test ensuring strong ETag stays unchanged when compressor skips (tiny body).

Apply:

+func Test_Compress_Strong_ETag_Unchanged_When_Not_Compressed(t *testing.T) {
+	t.Parallel()
+	app := fiber.New()
+	app.Use(New())
+	app.Get("/", func(c fiber.Ctx) error {
+		c.Set(fiber.HeaderETag, "\"abc\"")
+		return c.SendString("tiny") // below compression threshold
+	})
+	req := httptest.NewRequest(fiber.MethodGet, "/", nil)
+	req.Header.Set("Accept-Encoding", "gzip")
+	resp, err := app.Test(req, testConfig)
+	require.NoError(t, err)
+	require.Equal(t, "", resp.Header.Get(fiber.HeaderContentEncoding))
+	require.Equal(t, "\"abc\"", resp.Header.Get(fiber.HeaderETag))
+	require.Equal(t, "Accept-Encoding", resp.Header.Get(fiber.HeaderVary))
+}
🧹 Nitpick comments (2)
middleware/compress/compress_test.go (2)

397-434: Add HEAD coverage for early-return branches.

Please add HEAD tests for (a) pre‑encoded responses and (b) no-transform skip to ensure body is empty and Content-Length equals would‑be length.

Example additions:

+func Test_Compress_Head_Metadata_Preencoded(t *testing.T) {
+	t.Parallel()
+	app := fiber.New()
+	app.Use(New())
+	handler := func(c fiber.Ctx) error {
+		c.Set(fiber.HeaderContentEncoding, "gzip")
+		return c.SendString("hello")
+	}
+	app.Get("/", handler)
+	app.Head("/", handler)
+	getReq := httptest.NewRequest(fiber.MethodGet, "/", nil)
+	getResp, _ := app.Test(getReq, testConfig)
+	headReq := httptest.NewRequest(fiber.MethodHead, "/", nil)
+	headResp, _ := app.Test(headReq, testConfig)
+	require.Empty(t, mustReadAll(t, headResp.Body))
+	require.Equal(t, getResp.Header.Get(fiber.HeaderContentLength), headResp.Header.Get(fiber.HeaderContentLength))
+}
+
+func Test_Compress_Head_Metadata_NoTransform(t *testing.T) {
+	t.Parallel()
+	app := fiber.New()
+	app.Use(New())
+	handler := func(c fiber.Ctx) error {
+		c.Set(fiber.HeaderCacheControl, "no-transform")
+		return c.SendString("hello")
+	}
+	app.Get("/", handler)
+	app.Head("/", handler)
+	req := httptest.NewRequest(fiber.MethodHead, "/", nil)
+	req.Header.Set("Accept-Encoding", "gzip")
+	resp, _ := app.Test(req, testConfig)
+	require.Empty(t, mustReadAll(t, resp.Body))
+	require.NotEmpty(t, resp.Header.Get(fiber.HeaderContentLength))
+}

Helper:

func mustReadAll(t *testing.T, r io.Reader) []byte { b, err := io.ReadAll(r); require.NoError(t, err); return b }

580-614: Also assert Vary on no-transform skips.

Spec requires emitting Vary: Accept-Encoding even when skipping.

Apply:

@@
-			resp, err := app.Test(req, testConfig)
+			resp, err := app.Test(req, testConfig)
 			require.NoError(t, err, "app.Test(req)")
 			require.Equal(t, "", resp.Header.Get(fiber.HeaderContentEncoding))
+			require.Equal(t, "Accept-Encoding", resp.Header.Get(fiber.HeaderVary))
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bb68ec6 and a49f3f8.

📒 Files selected for processing (3)
  • middleware/compress/compress.go (2 hunks)
  • middleware/compress/compress_test.go (2 hunks)
  • middleware/etag/etag.go (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • middleware/etag/etag.go
🧰 Additional context used
📓 Path-based instructions (1)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: Format Go code using gofumpt (enforced via make format)
Ensure code passes golangci-lint checks (enforced via make lint)
Optimize struct field alignment using betteralign (enforced via make betteralign)
Modernize Go code using gopls modernize (enforced via make modernize)

Files:

  • middleware/compress/compress_test.go
  • middleware/compress/compress.go
🧠 Learnings (2)
📚 Learning: 2024-10-02T23:03:31.727Z
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/store.go:164-167
Timestamp: 2024-10-02T23:03:31.727Z
Learning: Unit tests in this project use testify require.

Applied to files:

  • middleware/compress/compress_test.go
📚 Learning: 2024-07-26T21:00:12.902Z
Learnt from: sixcolors
PR: gofiber/fiber#2922
File: middleware/cors/utils.go:63-71
Timestamp: 2024-07-26T21:00:12.902Z
Learning: The project uses the testify/assert package for assertions in unit tests.

Applied to files:

  • middleware/compress/compress_test.go
🧬 Code graph analysis (2)
middleware/compress/compress_test.go (3)
middleware/compress/compress.go (1)
  • New (50-125)
middleware/etag/etag.go (2)
  • New (44-117)
  • Generate (18-32)
constants.go (11)
  • HeaderVary (187-187)
  • HeaderContentEncoding (212-212)
  • HeaderETag (181-181)
  • HeaderContentType (216-216)
  • MIMETextPlainCharsetUTF8 (35-35)
  • HeaderContentLength (214-214)
  • MethodHead (6-6)
  • StatusNoContent (56-56)
  • StatusNotModified (67-67)
  • StatusPartialContent (58-58)
  • HeaderCacheControl (168-168)
middleware/compress/compress.go (3)
ctx_interface_gen.go (1)
  • Ctx (18-430)
constants.go (11)
  • StatusNoContent (56-56)
  • StatusResetContent (57-57)
  • StatusNotModified (67-67)
  • StatusPartialContent (58-58)
  • HeaderRange (236-236)
  • HeaderCacheControl (168-168)
  • HeaderVary (187-187)
  • HeaderAcceptEncoding (192-192)
  • HeaderContentEncoding (212-212)
  • HeaderETag (181-181)
  • MethodHead (6-6)
middleware/etag/etag.go (1)
  • Generate (18-32)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: repeated
  • GitHub Check: unit (1.25.x, windows-latest)
  • GitHub Check: Compare
🔇 Additional comments (4)
middleware/compress/compress.go (2)

21-35: Skip matrix LGTM.

Covers no-body/1xx/204/205/206/304, Range, and no-transform on req/resp.


37-47: Vary handling LGTM.

Tokenized merge with star short‑circuit and case‑insensitive match is correct.

middleware/compress/compress_test.go (2)

192-209: Vary: Accept-Encoding happy-path test LGTM.


229-247: Star semantics test LGTM. Correctly preserves Vary: * without modification.

@gaby
Copy link
Member Author

gaby commented Sep 15, 2025

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly improves the RFC compliance of the Compress middleware. The changes are well-structured, covering various aspects like handling Vary and ETag headers, skipping compression under appropriate conditions, and correctly processing HEAD requests. The addition of comprehensive tests ensures the new logic is robust. I have a couple of suggestions to further improve the code.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
middleware/etag/etag.go (1)

91-103: If-None-Match handling breaks for comma-separated tags and “*”; unify matching with a token scan.

Current weak-branch equality checks only the prefix segment and miss list cases like If-None-Match: W/"a", W/"b". It also ignores “*”. Replace both branches with a single loop that:

  • trims OWS around tokens,
  • strips W/ per token (weak comparison),
  • handles “*”,
  • compares against the server tag with any W/ stripped.

Apply:

-    // Check if client's ETag is weak
-    if bytes.HasPrefix(clientEtag, weakPrefix) {
-      // Check if server's ETag is weak
-      if bytes.Equal(clientEtag[2:], etag) || bytes.Equal(clientEtag[2:], etag[2:]) {
-        // W/1 == 1 || W/1 == W/1
-        c.RequestCtx().ResetBody()
-
-        return c.SendStatus(fiber.StatusNotModified)
-      }
-      // W/1 != W/2 || W/1 != 2
-      c.Response().Header.SetCanonical(normalizedHeaderETag, etag)
-
-      return nil
-    }
-
-    if bytes.Contains(clientEtag, etag) {
-      // 1 == 1
-      c.RequestCtx().ResetBody()
-
-      return c.SendStatus(fiber.StatusNotModified)
-    }
-    // 1 != 2
-    c.Response().Header.SetCanonical(normalizedHeaderETag, etag)
-    return nil
+    // RFC 9110 weak comparison across a comma-separated list; honor "*".
+    // Normalize server tag by stripping W/ if present.
+    st := etag
+    if bytes.HasPrefix(st, weakPrefix) {
+      st = st[2:]
+    }
+    h := clientEtag
+    for {
+      // skip OWS
+      for len(h) > 0 && (h[0] == ' ' || h[0] == '\t') {
+        h = h[1:]
+      }
+      if len(h) == 0 {
+        break
+      }
+      // split at comma
+      tok := h
+      if i := bytes.IndexByte(h, ','); i >= 0 {
+        tok, h = h[:i], h[i+1:]
+      } else {
+        h = nil
+      }
+      tok = bytes.TrimSpace(tok)
+      // "*" matches any current representation
+      if len(tok) == 1 && tok[0] == '*' {
+        c.RequestCtx().ResetBody()
+        return c.SendStatus(fiber.StatusNotModified)
+      }
+      // strip weak prefix per token
+      if bytes.HasPrefix(tok, weakPrefix) {
+        tok = tok[2:]
+      }
+      if bytes.Equal(tok, st) {
+        c.RequestCtx().ResetBody()
+        return c.SendStatus(fiber.StatusNotModified)
+      }
+    }
+    // No match; set server's ETag.
+    c.Response().Header.SetCanonical(normalizedHeaderETag, etag)
+    return nil

Please add tests for:

  • If-None-Match: W/"x", W/"y" with server W/"y" and with server "y"
  • If-None-Match: "*" (GET/HEAD) returning 304
  • OWS around tokens
♻️ Duplicate comments (1)
middleware/etag/etag.go (1)

17-31: Generate: pooling + defensive copy look correct.

Using bytebufferpool with an immediate defer and returning a fresh copy prevents use-after-put; nice.

🧹 Nitpick comments (2)
middleware/etag/etag.go (2)

33-40: Micro: avoid an extra allocation in GenerateWeak.

Pre-size the result to len("W/")+len(tag) to cut one growth step.

Apply:

 func GenerateWeak(body []byte) []byte {
   tag := Generate(body)
   if tag == nil {
     return nil
   }
-  return append(weakPrefix, tag...)
+  out := make([]byte, 0, len(weakPrefix)+len(tag))
+  out = append(out, weakPrefix...)
+  out = append(out, tag...)
+  return out
 }

47-47: Naming: consider a package-level canonical key.

normalizedHeaderETag is effectively a constant. Hoist to package scope to reuse across handlers.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2aef70a and fd6b7fb.

📒 Files selected for processing (1)
  • middleware/etag/etag.go (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: Format Go code using gofumpt (enforced via make format)
Ensure code passes golangci-lint checks (enforced via make lint)
Optimize struct field alignment using betteralign (enforced via make betteralign)
Modernize Go code using gopls modernize (enforced via make modernize)

Files:

  • middleware/etag/etag.go
🧬 Code graph analysis (1)
middleware/etag/etag.go (2)
middleware/compress/compress.go (1)
  • New (50-129)
middleware/etag/config.go (1)
  • Config (8-22)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Compare
  • GitHub Check: unit (1.25.x, windows-latest)
  • GitHub Check: unit (1.25.x, macos-latest)
  • GitHub Check: repeated
🔇 Additional comments (2)
middleware/etag/etag.go (2)

12-15: Good: move CRC table to a package-level var.

Precomputing CRC-32Q once avoids per-call allocations and speeds up checksums.


80-85: Config-driven strong/weak generation looks right.

Clean separation via Generate/GenerateWeak.

@gaby
Copy link
Member Author

gaby commented Sep 16, 2025

@codex Review pull request

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex Review: Here are some suggestions.

Reply with @codex fix comments to fix any unresolved comments.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
middleware/compress/compress_test.go (1)

396-416: Thanks for adding the “not compressed” strong ETag guard.

🧹 Nitpick comments (3)
middleware/compress/compress_test.go (3)

191-208: Good: validates Vary addition on compress path.

You can DRY response-body handling and ensure bodies are closed by adding a tiny helper and using t.Cleanup across tests.

+// helper near the top of the file
+func readAllAndClose(t *testing.T, rc io.ReadCloser) []byte {
+	t.Helper()
+	b, err := io.ReadAll(rc)
+	require.NoError(t, err)
+	t.Cleanup(func() { _ = rc.Close() })
+	return b
+}

417-453: HEAD metadata propagation validated; body omitted as intended.

Consider asserting the 200 status on both GET and HEAD for completeness.

-	getResp, err := app.Test(getReq, testConfig)
+	getResp, err := app.Test(getReq, testConfig)
 	require.NoError(t, err, "app.Test(getReq)")
+	require.Equal(t, fiber.StatusOK, getResp.StatusCode)
@@
-	headResp, err := app.Test(headReq, testConfig)
+	headResp, err := app.Test(headReq, testConfig)
 	require.NoError(t, err, "app.Test(headReq)")
+	require.Equal(t, fiber.StatusOK, headResp.StatusCode)

598-634: Add Vary assertion to no‑transform skip for consistency with other skip paths.

Apply this diff inside the subtest assertions:

-			resp, err := app.Test(req, testConfig)
+			resp, err := app.Test(req, testConfig)
 			require.NoError(t, err, "app.Test(req)")
 			require.Equal(t, "", resp.Header.Get(fiber.HeaderContentEncoding))
+			require.Equal(t, "Accept-Encoding", resp.Header.Get(fiber.HeaderVary))

Also consider using the readAllAndClose helper across tests to avoid leaking bodies.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between dcf0ff1 and 5970397.

📒 Files selected for processing (2)
  • middleware/compress/compress.go (2 hunks)
  • middleware/compress/compress_test.go (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • middleware/compress/compress.go
🧰 Additional context used
📓 Path-based instructions (1)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: Format Go code using gofumpt (enforced via make format)
Ensure code passes golangci-lint checks (enforced via make lint)
Optimize struct field alignment using betteralign (enforced via make betteralign)
Modernize Go code using gopls modernize (enforced via make modernize)

Files:

  • middleware/compress/compress_test.go
🧬 Code graph analysis (1)
middleware/compress/compress_test.go (2)
middleware/compress/compress.go (1)
  • New (54-125)
middleware/etag/etag.go (2)
  • New (43-116)
  • Generate (18-31)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: unit (1.25.x, windows-latest)
  • GitHub Check: Compare
  • GitHub Check: unit (1.25.x, macos-13)
  • GitHub Check: repeated
🔇 Additional comments (17)
middleware/compress/compress_test.go (17)

13-13: Import etag helper is appropriate for strong-ETag recomputation assertions.


209-227: Correct handling of Vary: "*" semantics (no merge).


228-246: Good: preserves list and respects terminal “*”.


247-265: Nice tokenization check; avoids substring collisions.


266-290: Pre‑encoded response skip path covered (Vary still emitted).


291-310: Skip + Vary “*” preserved.


311-330: Skip + Vary list with “*” preserved.


331-350: Skip + Vary merge with non‑token substring handled.


351-374: Strong ETag recalculation assertion looks right (ETag of compressed bytes).


375-395: Weak ETag passthrough verified.


454-474: No Content skip path behaves correctly (no Content‑Encoding, ETag preserved).


475-496: Not Modified skip path covered; headers left intact.


497-516: Range request skip path covered; Vary emitted.


517-535: Good: even without Accept‑Encoding request header, Vary is emitted.


536-556: Range + Vary “*”: star preserved.


557-576: Range + existing Vary token merging validated.


578-597: Partial Content skip path validated.

@ReneWerner87 ReneWerner87 merged commit 6d1f7b2 into main Sep 16, 2025
14 of 15 checks passed
@ReneWerner87 ReneWerner87 deleted the review-compress-middleware-for-rfc-compliance branch September 16, 2025 19:49
@github-project-automation github-project-automation bot moved this from In Progress to Done in v3 Sep 16, 2025
Abhirup-99 pushed a commit to Abhirup-99/fiber that referenced this pull request Nov 3, 2025
@gaby gaby added the 📜 RFC Compliance Feature, implementation, or contribution adheres to relevant RFC standards. label Nov 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex 📜 RFC Compliance Feature, implementation, or contribution adheres to relevant RFC standards. 🧹 Updates v3

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants