Skip to content

schema: add support for multipart files#17

Merged
ReneWerner87 merged 3 commits intomainfrom
multipart-support
Feb 13, 2025
Merged

schema: add support for multipart files#17
ReneWerner87 merged 3 commits intomainfrom
multipart-support

Conversation

@efectn
Copy link
Copy Markdown
Member

@efectn efectn commented Feb 11, 2025

This PR adds support for multipart files to Gorilla decoder as a part of gofiber/fiber#2002

To-Do List:

  • Create benchmark.
  • Test required and default with multipart files.
  • Add testcases for the case when multipart file is member of slices of struct.

Summary by CodeRabbit

  • New Features

    • Now supports multipart file uploads, enabling form submissions to include file data processed seamlessly alongside regular inputs.
  • Refactor

    • Improved parsing and handling logic for complex field inputs, resulting in more reliable data processing.
  • Tests

    • Added new tests to ensure robust handling of multipart file data and verify that all fields are accurately decoded.
    • Introduced benchmark tests to measure performance for multipart file handling and decoding.

@efectn efectn requested a review from a team as a code owner February 11, 2025 22:12
@efectn efectn requested review from ReneWerner87, gaby and sixcolors and removed request for a team February 11, 2025 22:12
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Feb 11, 2025

Walkthrough

This pull request refines the logic for handling multipart file fields across the codebase. In cache.go, the parsing logic is adjusted in the parsePath method by introducing an extra multipart field check. In decoder.go, the Decode and decode methods are updated to accept multipart file uploads via an additional variadic parameter, along with new helper functions to recognize and process multipart file types. Additionally, a new test in decoder_test.go verifies that multipart file decoding works as expected.

Changes

File(s) Change Summary
cache.go Modified parsePath method to include an isMultipartField check, refining the handling of slice of struct fields with multipart data.
decoder.go, decoder_test.go Updated Decode and decode methods to support multipart file uploads by adding a variadic parameter for files; introduced helper functions isMultipartField and handleMultipartField; added a new test TestDecoderMultipartFiles to validate multipart file decoding.

Sequence Diagram(s)

sequenceDiagram
    participant C as Caller
    participant D as Decoder
    participant H as Helper (isMultipartField/handleMultipartField)
    
    C->>D: Call Decode(dst, src, files)
    D->>D: Process src and initialize multipartFiles if provided
    D->>D: Iterate over fields in dst for decoding
    alt Field is multipart type
        D->>H: Check using isMultipartField(fieldType)
        H-->>D: Return true
        D->>H: Call handleMultipartField(field, multipart files)
        H-->>D: Set multipart field value
    else Field is standard type
        D->>D: Perform regular decoding
    end
    D-->>C: Return decoded object
Loading

Suggested labels

🧹 Updates

Suggested reviewers

  • sixcolors
  • ReneWerner87

Poem

I'm a bunny coding through the night,
Hopping over bugs with all my might.
Multipart files now hop in line,
In every decode they truly shine.
With carrots and code, I celebrate this flight! 🐇
Happy hops to a cleaner, brighter code light!

✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@efectn efectn marked this pull request as draft February 11, 2025 22:13
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 11, 2025

Codecov Report

Attention: Patch coverage is 93.93939% with 4 lines in your changes missing coverage. Please review.

Project coverage is 88.55%. Comparing base (f00d979) to head (34c1119).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
decoder.go 93.75% 3 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #17      +/-   ##
==========================================
+ Coverage   87.77%   88.55%   +0.77%     
==========================================
  Files           4        4              
  Lines         867      926      +59     
==========================================
+ Hits          761      820      +59     
  Misses         90       90              
  Partials       16       16              
Flag Coverage Δ
unittests 88.55% <93.93%> (+0.77%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
decoder.go (3)

83-93: Possible collision when inserting file paths into src.
Currently, the code overwrites entries in src with an empty slice whenever a key from multipartFiles already exists in src. This has potential to discard existing data if there's a parameter collision.

 for path := range multipartFiles {
-    src[path] = []string{""}
+    if _, exists := src[path]; !exists {
+        src[path] = []string{""}
+    }
 }

104-104: Remove or replace debug print.
fmt.Println(parts) appears to be a leftover debugging statement. Consider removing or using structured logging.

- fmt.Println(parts)
+ // fmt.Println(parts) // remove or replace with logger if needed

105-115: Discourage printing errors directly.
Using fmt.Print(err) in production code could clutter output. Prefer proper logging or returning the error in a structured way.

} else if !d.ignoreUnknownKeys {
-    fmt.Print(err)
+    // log.Printf("parsePath error: %v", err)
    errors[path] = UnknownKeyError{Key: path}
}
🧰 Tools
🪛 GitHub Check: codecov/patch

[warning] 107-108: decoder.go#L107-L108
Added lines #L107 - L108 were not covered by tests

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f00d979 and cf1bef5.

📒 Files selected for processing (3)
  • cache.go (1 hunks)
  • decoder.go (6 hunks)
  • decoder_test.go (2 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
decoder_test.go

2718-2718: Error return value of decoder.Decode is not checked

(errcheck)

🪛 GitHub Check: lint
decoder_test.go

[failure] 2718-2718:
Error return value of decoder.Decode is not checked (errcheck)

🪛 GitHub Actions: golangci-lint
decoder_test.go

[error] 2718-2718: Error return value of decoder.Decode is not checked (errcheck)

🪛 GitHub Check: codecov/patch
decoder.go

[warning] 107-108: decoder.go#L107-L108
Added lines #L107 - L108 were not covered by tests


[warning] 351-352: decoder.go#L351-L352
Added lines #L351 - L352 were not covered by tests

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: unit (1.23.x, windows-latest)
  • GitHub Check: unit (1.22.x, windows-latest)
🔇 Additional comments (6)
decoder.go (4)

11-11: Import is consistent with usage.
No immediate issues with adding "mime/multipart" as it’s necessary for handling FileHeader.


306-310: Definition of multipart field types.
These constants clearly define multipart file pointer types, which is good for readability. No further issues noted.


346-369: Logic for checking multipart field type looks good.
All recognized multipart field forms are covered and no redundant checks are found.

🧰 Tools
🪛 GitHub Check: codecov/patch

[warning] 351-352: decoder.go#L351-L352
Added lines #L351 - L352 were not covered by tests


400-404: Early return is intentional for multipart fields.
If a struct field is a multipart field, we skip normal decoding. This appears to be by design.

cache.go (1)

66-73: Conditional check for isMultipartField.
Skipping slices of structs if they’re multipart-typed is consistent with the new decoding logic.

decoder_test.go (1)

2672-2795: General test coverage for multipart file decoding is well-structured.
Adequately verifies that nested multipart fields are assigned correctly.

🧰 Tools
🪛 golangci-lint (1.62.2)

2718-2718: Error return value of decoder.Decode is not checked

(errcheck)

🪛 GitHub Check: lint

[failure] 2718-2718:
Error return value of decoder.Decode is not checked (errcheck)

🪛 GitHub Actions: golangci-lint

[error] 2718-2718: Error return value of decoder.Decode is not checked (errcheck)

@efectn efectn marked this pull request as ready for review February 12, 2025 13:39
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (6)
decoder_test.go (3)

2844-2898: Improve error handling in benchmark.

Move the error check inside the benchmark loop to catch errors in intermediate iterations.

 for i := 0; i < b.N; i++ {
-    err = decoder.Decode(&s, data, fileHeaders)
+    if err := decoder.Decode(&s, data, fileHeaders); err != nil {
+        b.Fatalf("Failed to decode at iteration %d: %v", i, err)
+    }
 }
-
-if err != nil {
-    b.Fatalf("Failed to decode: %v", err)
-}

2900-2943: Add descriptive names to test cases.

Consider adding descriptive names to test cases to improve readability and debugging.

 tc := []struct {
+    name     string
     typ      reflect.Type
     input    map[string][]string
     expected bool
 }{
     {
+        name:     "plain string type",
         typ:      reflect.TypeOf(string("")),
         expected: false,
     },
     {
+        name:     "string slice type",
         typ:      reflect.TypeOf([]string{}),
         expected: false,
     },
     // ... rest of test cases
 }

-for _, tt := range tc {
+for _, tt := range tc {
+    t.Run(tt.name, func(t *testing.T) {
         if isMultipartField(tt.typ) != tt.expected {
             t.Errorf("Expected %v, got %v", tt.expected, isMultipartField(tt.typ))
         }
+    })
 }

2981-3055: Restructure test into sub-tests for better organization.

Consider using sub-tests to group related test cases and improve test output.

 func TestHandleMultipartField(t *testing.T) {
     t.Parallel()

     // ... setup code ...

-    ok := handleMultipartField(rv.FieldByName("F"), files["f"])
-    if !ok {
-        t.Error("Expected handleMultipartField to return true")
-    }
-    // ... more tests ...

+    tests := []struct {
+        name     string
+        field    string
+        wantOk   bool
+        validate func(*testing.T, S)
+    }{
+        {
+            name:   "single file pointer",
+            field:  "F",
+            wantOk: true,
+            validate: func(t *testing.T, s S) {
+                if s.F == nil {
+                    t.Error("Expected F to be a file header, got nil")
+                }
+                if s.F.Filename != "test.txt" {
+                    t.Errorf("Expected F.Filename to be 'test.txt', got %s", s.F.Filename)
+                }
+            },
+        },
+        // ... more test cases
+    }
+
+    for _, tt := range tests {
+        t.Run(tt.name, func(t *testing.T) {
+            s := S{}
+            rv := reflect.ValueOf(&s).Elem()
+            if got := handleMultipartField(rv.FieldByName(tt.field), files["f"]); got != tt.wantOk {
+                t.Errorf("handleMultipartField() = %v, want %v", got, tt.wantOk)
+            }
+            tt.validate(t, s)
+        })
+    }
decoder.go (3)

304-307: Consider using const declarations for type safety.

The type declarations could be made more type-safe using const declarations.

-var (
+const (
     multipartFileHeaderPointerType      = reflect.TypeOf(&multipart.FileHeader{})
     sliceMultipartFileHeaderPointerType = reflect.TypeOf([]*multipart.FileHeader{})
-)

311-349: Simplify function structure with early returns.

The function could be simplified by using early returns to reduce nesting.

 func handleMultipartField(field reflect.Value, files []*multipart.FileHeader) bool {
     fieldType := field.Type()
     if !isMultipartField(fieldType) {
         return false
     }

     // Skip if files are empty and field is multipart
     if len(files) == 0 {
         return true
     }

     // Check for *multipart.FileHeader
     if fieldType == multipartFileHeaderPointerType {
         field.Set(reflect.ValueOf(files[0]))
         return true
     }

     // Check for []*multipart.FileHeader
     if fieldType == sliceMultipartFileHeaderPointerType {
         field.Set(reflect.ValueOf(files))
         return true
     }

     // Check for *[]*multipart.FileHeader
     if fieldType.Kind() != reflect.Pointer {
         return false
     }
     
     fieldType = fieldType.Elem()
     if fieldType != sliceMultipartFileHeaderPointerType {
         return false
     }

     if field.IsNil() {
         field.Set(reflect.New(fieldType))
     }
     
     field.Elem().Set(reflect.ValueOf(files))
     return true
 }
🧰 Tools
🪛 GitHub Check: codecov/patch

[warning] 348-348: decoder.go#L348
Added line #L348 was not covered by tests


83-93: Add input validation for files parameter.

Consider validating the files parameter to ensure consistent behavior.

 func (d *Decoder) Decode(dst interface{}, src map[string][]string, files ...map[string][]*multipart.FileHeader) error {
+    if src == nil {
+        return errors.New("schema: source map is nil")
+    }
+
     var multipartFiles map[string][]*multipart.FileHeader
 
     if len(files) > 0 {
+        if files[0] == nil {
+            return errors.New("schema: files map is nil")
+        }
         multipartFiles = files[0]
     }
 
     // Add files as empty string values to src in order to make path parsing work easily
     for path := range multipartFiles {
+        if path == "" {
+            return errors.New("schema: empty path in files map")
+        }
         src[path] = []string{""}
     }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cf1bef5 and 34c1119.

📒 Files selected for processing (2)
  • decoder.go (6 hunks)
  • decoder_test.go (2 hunks)
🧰 Additional context used
🪛 GitHub Check: codecov/patch
decoder.go

[warning] 106-107: decoder.go#L106-L107
Added lines #L106 - L107 were not covered by tests


[warning] 348-348: decoder.go#L348
Added line #L348 was not covered by tests

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: unit (1.23.x, windows-latest)
  • GitHub Check: unit (1.22.x, windows-latest)
🔇 Additional comments (1)
decoder_test.go (1)

2672-2842: LGTM! Comprehensive test coverage for multipart file decoding.

The test thoroughly covers:

  • Single file fields
  • File slice fields
  • Nested struct fields
  • Required field validation
  • Empty file handling
  • Various pointer combinations

@ReneWerner87 ReneWerner87 added the ✏️ Feature New feature or request label Feb 13, 2025
@ReneWerner87 ReneWerner87 merged commit 9a2749c into main Feb 13, 2025
14 checks passed
@ReneWerner87 ReneWerner87 deleted the multipart-support branch February 13, 2025 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

✏️ Feature New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants