🧹 chore: Improve static middleware security#3595
Conversation
|
""" WalkthroughA new Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant FiberApp
participant StaticMiddleware
participant sanitizePath
participant FileSystem
Client->>FiberApp: HTTP Request (GET /static/...)
FiberApp->>StaticMiddleware: Serve static file
StaticMiddleware->>sanitizePath: Validate and clean path
alt Path is invalid
sanitizePath-->>StaticMiddleware: Error
StaticMiddleware-->>FiberApp: Return 404/400
else Path is valid
sanitizePath-->>StaticMiddleware: Sanitized path
StaticMiddleware->>FileSystem: Attempt to serve file
FileSystem-->>StaticMiddleware: File or error
StaticMiddleware-->>FiberApp: Return file or error
end
FiberApp-->>Client: HTTP Response
Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Summary of Changes
Hello @gaby, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly enhances the security of static file serving by introducing stricter path sanitization and expanding test coverage against various path traversal vulnerabilities. The changes aim to prevent attackers from accessing files outside the intended static root directory, making the application more robust against common web exploits.
Highlights
- Enhanced Path Sanitization: I've implemented a new
sanitizePathfunction within the static file serving middleware. This function rigorously validates and cleans requested paths by handling URL unescaping, rejecting null bytes (\x00), and preventing directory traversal attempts (e.g.,..,:). If an invalid path is detected, it returns a guaranteed-missing path (/__fiber_invalid__) to ensure a 404 response, preventing unauthorized file access. - Robust Path Traversal Testing: I've added comprehensive test suites,
Test_Static_PathTraversalandTest_Static_PathTraversal_WindowsOnly, to cover a wide array of malicious path traversal patterns. These tests include various encodings (e.g.,%2e,%2f,%252e), null bytes, double slashes, and platform-specific attempts (like backslashes on Windows), ensuring the new sanitization logic is effective. - Improved Test Environment Cleanup: In
client/client_test.go, I've enhancedTest_Client_SetProxyURLby addingt.Cleanupto properly restore environment variables related to proxy settings after the test runs, ensuring test isolation and preventing side effects.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3595 +/- ##
==========================================
- Coverage 90.94% 90.92% -0.02%
==========================================
Files 111 111
Lines 11195 11230 +35
==========================================
+ Hits 10181 10211 +30
- Misses 761 764 +3
- Partials 253 255 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Code Review
This pull request enhances security for the static file middleware by adding stricter path sanitization and comprehensive tests for path traversal vulnerabilities. The changes are a significant improvement. However, the review identified a few issues in the new sanitizePath function:
- A critical bug in handling root paths for
fs.FS. - An overly strict validation that rejects valid paths containing
... - Some redundant code that can be cleaned up.
Additionally, a minor copy-paste error was found in the new test assertions. Addressing these points will make the implementation more robust and correct.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces significant security enhancements to the static file middleware by adding a robust path sanitization function and comprehensive path traversal tests. The new sanitizePath function effectively mitigates various attack vectors, including URL-encoded and null-byte injection attacks. The accompanying tests are thorough and cover a wide range of malicious patterns for both POSIX-like and Windows systems. My review includes a suggestion to simplify the new sanitization logic by removing some redundant checks, which will improve maintainability without affecting security.
There was a problem hiding this comment.
Pull Request Overview
This PR enhances the security of the static middleware by implementing stricter path sanitization to prevent directory traversal attacks. The changes focus on preventing malicious file access attempts through various encoding and path manipulation techniques.
Key Changes:
- Added a new
sanitizePathfunction with comprehensive path validation and cleaning - Enhanced path traversal attack prevention through URL decoding, null byte detection, and directory traversal pattern blocking
- Expanded test coverage with extensive path traversal test cases for both Unix and Windows environments
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| middleware/static/static.go | Implements new sanitizePath function and integrates it into the path rewriting logic |
| middleware/static/static_test.go | Adds comprehensive path traversal tests and refactors hardcoded paths to use constants |
| client/client_test.go | Fixes a minor spelling error in a comment |
Comments suppressed due to low confidence (2)
middleware/static/static_test.go:955
- The test expects an "Are you a hacker?" message for 400 status codes, but this message is not defined anywhere in the static middleware code. The middleware returns "/fiber_invalid" path which would result in a 404, not a 400 with this specific message.
require.Contains(t, string(body), "Are you a hacker?",
middleware/static/static_test.go:1061
- Same issue as above - the test expects an "Are you a hacker?" message for 400 status codes, but this message is not implemented in the middleware. This will cause test failures.
require.Contains(t, string(body), "Are you a hacker?",
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (5)
middleware/static/static.go (3)
42-44: Remove overly strict..check before path cleaningThe check for
..in the unescaped path is overly strict and will reject valid paths. For example,/assets/css/../js/app.jsis a valid path that should resolve to/assets/js/app.js. Thepathpkg.Cleanfunction on line 47 is designed to safely resolve..segments.Remove the premature
..check:- if strings.Contains(s, "\x00") || strings.Contains(s, "..") || strings.Contains(s, ":") { + if strings.Contains(s, "\x00") || strings.Contains(s, ":") {
51-53: Fix fs.FS root path handlingWhen the cleaned path is empty for an
fs.FS, the function returns/. However, the root of anfs.FSshould be represented by.(a single dot), not/. This will cause errors when attempting to open the root path.Apply this fix:
if s == "" { - return []byte("/"), nil + s = "." }
59-66: Remove redundant validation checksThis else block contains redundant checks that can be removed:
strings.Contains(s, "..")will always be false afterpathpkg.Cleanon a rooted path!strings.HasPrefix(s, "/")will also always be false for the same reasonSimplify the validation:
} else { - // verify no traversal after cleaning - if strings.Contains(raw, "..") || strings.Contains(s, "..") { - return nil, errors.New("invalid path") - } - if !strings.HasPrefix(s, "/") { - s = "/" + s - } + // pathpkg.Clean with a rooted path already ensures no traversal }middleware/static/static_test.go (2)
955-957: Fix copy-paste error in assertion messageThe assertion message incorrectly states it expects a "Cannot GET" message when it's actually checking for "Are you a hacker?" message.
} else { require.Contains(t, string(body), "Are you a hacker?", - "Blocked traversal should have a Cannot GET message for %s", path) + "Blocked traversal should have an 'Are you a hacker?' message for %s", path) }
1061-1063: Fix copy-paste error in assertion messageSame issue as in the non-Windows test - the assertion message doesn't match what's being checked.
} else { require.Contains(t, string(body), "Are you a hacker?", - "Blocked traversal should have a Cannot GET message for %s", path) + "Blocked traversal should have an 'Are you a hacker?' message for %s", path) }
🧹 Nitpick comments (1)
middleware/static/static.go (1)
42-44: Consider platform-specific handling for colon characterThe rejection of paths containing
:might be overly restrictive on non-Windows systems. On Unix-like systems, colons are valid filename characters. Consider making this check Windows-specific.+ import "runtime" // reject any null bytes or traversal attempts - if strings.Contains(s, "\x00") || strings.Contains(s, ":") { + rejectChars := "\x00" + if runtime.GOOS == "windows" { + rejectChars += ":" + } + if strings.ContainsAny(s, rejectChars) { return nil, errors.New("invalid path") }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
client/client_test.go(1 hunks)middleware/static/static.go(3 hunks)middleware/static/static_test.go(10 hunks)
🧰 Additional context used
🧠 Learnings (3)
📓 Common learnings
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:20-23
Timestamp: 2024-10-08T19:06:06.583Z
Learning: Unit tests for key length enforcement in both `EncryptCookie` and `DecryptCookie` functions have been added to ensure robust validation and prevent potential runtime errors.
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:20-23
Timestamp: 2024-07-01T03:44:03.672Z
Learning: Unit tests for key length enforcement in both `EncryptCookie` and `DecryptCookie` functions have been added to ensure robust validation and prevent potential runtime errors.
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:51-54
Timestamp: 2024-10-08T19:06:06.583Z
Learning: Unit tests for key length enforcement in `DecryptCookie` have been added to ensure consistency and security in the encryption processes.
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:51-54
Timestamp: 2024-07-01T03:33:22.283Z
Learning: Unit tests for key length enforcement in `DecryptCookie` have been added to ensure consistency and security in the encryption processes.
client/client_test.go (3)
Learnt from: gaby
PR: gofiber/fiber#3170
File: ctx.go:1825-1826
Timestamp: 2024-10-16T14:00:50.724Z
Learning: In the Fiber framework, the `IsProxyTrusted()` function returns `true` when `TrustProxy` is `false`, maintaining compatibility with version 2 behavior.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/config.go:122-122
Timestamp: 2024-10-08T19:06:06.583Z
Learning: In `DefaultErrorHandler(c *fiber.Ctx, err error)`, since `c` is a pointer to an interface, we need to dereference `*c` when calling interface methods like `SendStatus`.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/config.go:122-122
Timestamp: 2024-09-25T16:18:34.719Z
Learning: In `DefaultErrorHandler(c *fiber.Ctx, err error)`, since `c` is a pointer to an interface, we need to dereference `*c` when calling interface methods like `SendStatus`.
middleware/static/static_test.go (16)
Learnt from: sixcolors
PR: gofiber/fiber#3051
File: middleware/session/session.go:215-216
Timestamp: 2024-06-30T00:38:06.580Z
Learning: Parallel tests for `Session.Save` already exist in the `middleware/session/session_test.go` file, specifically in the `Test_Session_Save` and `Test_Session_Save_Expiration` functions.
Learnt from: sixcolors
PR: gofiber/fiber#3051
File: middleware/session/session.go:215-216
Timestamp: 2024-10-08T19:06:06.583Z
Learning: Parallel tests for `Session.Save` already exist in the `middleware/session/session_test.go` file, specifically in the `Test_Session_Save` and `Test_Session_Save_Expiration` functions.
Learnt from: ReneWerner87
PR: gofiber/fiber#3161
File: app.go:923-932
Timestamp: 2024-11-15T07:56:21.623Z
Learning: In the Fiber framework, breaking changes are acceptable when moving from version 2 to version 3, including modifications to method signatures such as in the `Test` method in `app.go`.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/middleware_test.go:400-407
Timestamp: 2024-09-25T17:09:03.756Z
Learning: In the `Test_Session_Next` function in `middleware/session/middleware_test.go`, the variable `doNext` is properly synchronized with the `muNext` mutex for both read and write access.
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:22-25
Timestamp: 2024-10-08T19:06:06.583Z
Learning: The `encryptcookie_test.go` file contains unit tests that validate key lengths for both `EncryptCookie` and `DecryptCookie` functions, ensuring that invalid key lengths raise appropriate errors.
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:22-25
Timestamp: 2024-07-02T13:29:56.992Z
Learning: The `encryptcookie_test.go` file contains unit tests that validate key lengths for both `EncryptCookie` and `DecryptCookie` functions, ensuring that invalid key lengths raise appropriate errors.
Learnt from: efectn
PR: gofiber/fiber#3162
File: hooks_test.go:228-228
Timestamp: 2024-12-13T08:14:22.851Z
Learning: In Go test files, prefer using the `require` methods from the `testify` package for assertions instead of manual comparisons and calls to `t.Fatal` or `t.Fatalf`.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/store.go:164-167
Timestamp: 2024-10-02T23:03:31.727Z
Learning: Unit tests in this project use testify require.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/store.go:164-167
Timestamp: 2024-10-08T19:06:06.583Z
Learning: Unit tests in this project use testify require.
Learnt from: sixcolors
PR: gofiber/fiber#2922
File: middleware/cors/utils.go:63-71
Timestamp: 2024-07-26T21:00:12.902Z
Learning: The project uses the testify/assert package for assertions in unit tests.
Learnt from: sixcolors
PR: gofiber/fiber#2922
File: middleware/cors/utils.go:63-71
Timestamp: 2024-10-08T19:06:06.583Z
Learning: The project uses the testify/assert package for assertions in unit tests.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/middleware_test.go:190-191
Timestamp: 2024-10-12T10:01:44.206Z
Learning: When testing session `IdleTimeout` expiration, it's acceptable to use `time.Sleep` to simulate the passage of time in tests.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/session/middleware_test.go:190-191
Timestamp: 2024-09-25T17:05:06.991Z
Learning: When testing session `IdleTimeout` expiration, it's acceptable to use `time.Sleep` to simulate the passage of time in tests.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/csrf/csrf_test.go:188-193
Timestamp: 2024-09-25T15:57:10.221Z
Learning: In the Fiber framework tests, using `ctx.Response.Header.Cookie` may not be suitable for parsing cookies from the response header, as it requires a `*Cookie` and fills it rather than returning a string value; thus, manual parsing of the `Set-Cookie` header may be necessary.
Learnt from: sixcolors
PR: gofiber/fiber#3016
File: middleware/csrf/csrf_test.go:188-193
Timestamp: 2024-10-08T19:06:06.583Z
Learning: In the Fiber framework tests, using `ctx.Response.Header.Cookie` may not be suitable for parsing cookies from the response header, as it requires a `*Cookie` and fills it rather than returning a string value; thus, manual parsing of the `Set-Cookie` header may be necessary.
Learnt from: gaby
PR: gofiber/fiber#3170
File: ctx_test.go:1721-1724
Timestamp: 2024-10-16T12:12:30.506Z
Learning: In the Go unit tests in `ctx_test.go`, it is acceptable to use invalid CIDR notation such as `"0.0.0.1/31junk"` for testing purposes.
🔇 Additional comments (2)
client/client_test.go (1)
1606-1606: LGTM!The typo correction in the comment is appropriate.
middleware/static/static_test.go (1)
909-1091: Excellent comprehensive security test coverage!The path traversal tests are thorough and cover a wide range of attack vectors including:
- URL-encoded sequences (single and double encoding)
- Null byte injection
- Mixed forward/backward slashes
- Platform-specific patterns (Windows drive letters, UNC paths)
- System file access attempts
This provides strong validation of the security improvements.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
middleware/static/static.go (2)
42-42: Reconsider overly strict path traversal checks.The current implementation rejects all paths containing
"..", which breaks legitimate navigation like/assets/css/../js/app.js. Thepath.Cleanfunction safely resolves..segments, so this premature rejection is unnecessary and harmful.The colon check may also be overly restrictive, potentially breaking legitimate Unix filenames.
Consider this more targeted approach:
- if strings.Contains(s, "\x00") || strings.Contains(s, "..") || strings.Contains(s, ":") { + if strings.Contains(s, "\x00") { return nil, errors.New("invalid path") }
51-53: Fix root path handling for fs.FS.For
fs.FS, the root should be represented as".", not"/". The current implementation will cause fasthttp to fail when opening the root path.if s == "" { - return []byte("/"), nil + s = "." }
🧹 Nitpick comments (1)
middleware/static/static.go (1)
20-61: Strong security improvements with room for refinement.The centralized sanitization approach significantly enhances security by preventing path traversal and double-encoding attacks. However, consider balancing security strictness with legitimate use cases to avoid breaking valid file requests.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
middleware/static/static.go(3 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:20-23
Timestamp: 2024-10-08T19:06:06.583Z
Learning: Unit tests for key length enforcement in both `EncryptCookie` and `DecryptCookie` functions have been added to ensure robust validation and prevent potential runtime errors.
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:20-23
Timestamp: 2024-07-01T03:44:03.672Z
Learning: Unit tests for key length enforcement in both `EncryptCookie` and `DecryptCookie` functions have been added to ensure robust validation and prevent potential runtime errors.
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:51-54
Timestamp: 2024-10-08T19:06:06.583Z
Learning: Unit tests for key length enforcement in `DecryptCookie` have been added to ensure consistency and security in the encryption processes.
Learnt from: gaby
PR: gofiber/fiber#3056
File: middleware/encryptcookie/utils.go:51-54
Timestamp: 2024-07-01T03:33:22.283Z
Learning: Unit tests for key length enforcement in `DecryptCookie` have been added to ensure consistency and security in the encryption processes.
middleware/static/static.go (1)
Learnt from: gaby
PR: gofiber/fiber#3170
File: ctx_test.go:1721-1724
Timestamp: 2024-10-16T12:12:30.506Z
Learning: In the Go unit tests in `ctx_test.go`, it is acceptable to use invalid CIDR notation such as `"0.0.0.1/31junk"` for testing purposes.
🪛 GitHub Check: unit (1.24.x, macos-latest)
middleware/static/static.go
[failure] 46-46:
declared and not used: raw
🪛 GitHub Check: lint
middleware/static/static.go
[failure] 46-46:
declared and not used: raw (typecheck)
🪛 GitHub Check: unit (1.24.x, ubuntu-latest)
middleware/static/static.go
[failure] 46-46:
declared and not used: raw
🪛 GitHub Check: modernize
middleware/static/static.go
[failure] 46-46:
declared and not used: raw
[failure] 46-46:
declared and not used: raw
🪛 GitHub Check: govulncheck-check
middleware/static/static.go
[failure] 46-46:
declared and not used: raw
🪛 GitHub Actions: Modernize Lint
middleware/static/static.go
[error] 46-46: Go compiler error: declared and not used variable 'raw'.
🪛 GitHub Actions: golangci-lint
middleware/static/static.go
[error] 46-46: golangci-lint: declared and not used: raw (typecheck)
🪛 GitHub Actions: Run govulncheck
middleware/static/static.go
[error] 46-46: Go compiler error: declared and not used variable 'raw'.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: repeated
- GitHub Check: Compare
🔇 Additional comments (2)
middleware/static/static.go (2)
4-9: LGTM! Clean import additions.The new imports are necessary for the sanitization functionality and the path alias prevents naming conflicts.
157-162: Excellent integration of path sanitization.The integration cleanly handles sanitization errors by returning a guaranteed-missing path, allowing the file handler to naturally respond with 404. This approach maintains the middleware's flow while ensuring security.
|
/gemini review |
There was a problem hiding this comment.
Code Review
The pull request improves the security of the static middleware by adding stricter sanitization of requested paths and expanding path traversal tests. The changes include a new sanitizePath function to validate and clean requested paths, and additional tests to cover various malicious patterns.
Summary
Replaces #3105