fix: Preserve empty strings in multipart/form-data requests (#4204) by doubledare704 · Pull Request #4271 · litestar-org/litestar

doubledare704 · 2025-08-24T07:17:11Z

Description

Fix issue Bug: Empty strings in multipart/form-data requests are converted to None #4204 where empty strings in multipart forms were converted to None
Change boolean check to length check in _multipart.py line 114
Empty bytearrays now decode to empty strings instead of becoming None
Add comprehensive tests for empty string handling in multipart forms
Raw multipart content parsing works correctly
Existing functionality remains unaffected

Closes #4204

codecov · 2025-08-24T07:28:24Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.25%. Comparing base (4c582da) to head (386cc5a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4271      +/-   ##
==========================================
- Coverage   98.25%   98.25%   -0.01%     
==========================================
  Files         343      343              
  Lines       15824    15822       -2     
  Branches     1746     1745       -1     
==========================================
- Hits        15548    15546       -2     
  Misses        138      138              
  Partials      138      138

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

- Create a copy of form_values to handle type conversions - Prevents type error when assigning None to dict[str, Any] - Maintains all functionality while satisfying type checker - All tests continue to pass Fix: Preserve empty strings in multipart/form-data requests for GitHub issue litestar-org#4204 - Fix issue litestar-org#4204 where empty strings in multipart forms were converted to None - Change boolean check to length check in _multipart.py line 114 - Empty bytearrays now decode to empty strings instead of becoming None - Add comprehensive tests for empty string handling in multipart forms - Ensure consistency between URL-encoded and multipart form behavior The fix changes 'elif data:' to 'elif len(data) > 0:' and handles the empty case by decoding the empty bytearray to an empty string. Tests verify: - Empty strings are preserved in multipart forms - Consistency between URL-encoded and multipart behavior - Raw multipart content parsing works correctly - Existing functionality remains unaffected

provinzkraut

This fix is way to complex and imposes significant additional runtime overhead. Instead of trying to remove the unwanted Nones, the behaviour should be fixed during multipart parsing. Extracting the multipart data shouldn't include that much more special casing and trickery than doing the same for URL encoded form data, which already works as expected

tests/unit/test_kwargs/test_multipart_data.py

litestar/utils/predicates.py

tests/unit/test_kwargs/test_multipart_data.py

github-actions · 2025-08-27T20:45:25Z

Documentation preview will be available shortly at https://litestar-org.github.io/litestar-docs-preview/4271

tests/unit/test_kwargs/test_multipart_data.py

provinzkraut

This already looks better. Still, a couple of suggestions for the tests

tests/unit/test_kwargs/test_multipart_data.py

Co-authored-by: Janek Nouvertné <provinzkraut@posteo.de>

…4271) --------- Co-authored-by: Janek Nouvertné <provinzkraut@posteo.de> (cherry picked from commit 7979d6d)

gsakkis · 2025-10-07T21:26:37Z

After upgrading to 2.18 some of my tests broke due to this change. Ultimately it's an issue on my end but it took some time to figure it out, plus there are some difference between DTO (PydanticDTO specifically) and non-DTO (Pydantic BaseModel).

In short:

There is a MyForm model containing an UploadFile and a nullable field.
In some tests I was passing {field: None} as form data and apparently http clients (or at least httpx) converts None to empty string.
In 2.17 this works (i.e. the empty string is validated successfully as None) both when annotating the data as MyForm and as DTOData[MyForm].
In 2.18 it fails with
- HTTP 400 (litestar.ValidationException) if annotated as MyForm
- HTTP 500 (msgspec.ValidationError) if annotated as DTOData[MyForm] with dto=PydanticDTO[MyForm]
Unrelated to this PR but while playing around I discovered that (both in 2.17 and 2.18) the string "null" is surprisingly converted to None in the DTO case (but not in the plain BaseModel case). Not sure if this is a bug, a feature or neither.
- Update: it's even stranger, apparently it's case insensitive - "NULL" and "Null" are also converted to None.

I can post a MCVE if interested.

euri10 · 2025-10-08T06:26:59Z

I can post a MCVE if interested.

i think you should

gsakkis · 2025-10-08T07:14:44Z

Server

from datetime import date
from typing import Annotated, Any

from litestar import Litestar, post
from litestar.datastructures import UploadFile
from litestar.dto import DTOData
from litestar.enums import RequestEncodingType
from litestar.params import Body
from litestar.plugins.pydantic import PydanticDTO
from pydantic import BaseModel


class MyForm(BaseModel, arbitrary_types_allowed=True):
    script: UploadFile
    updated_on: date | None = None


@post("/")
async def process(
    data: Annotated[MyForm, Body(media_type=RequestEncodingType.MULTI_PART)],
) -> dict[str, Any]:
    return {"filename": data.script.filename, "updated_on": data.updated_on}


@post("/dto", dto=PydanticDTO[MyForm])
async def process_dto(
    data: Annotated[DTOData[MyForm], Body(media_type=RequestEncodingType.MULTI_PART)],
) -> dict[str, Any]:
    builtins = data.as_builtins()
    return {
        "filename": builtins["script"].filename,
        "updated_on": builtins["updated_on"],
    }


app = Litestar(route_handlers=[process, process_dto])

Client

import httpx

with open("upload.py", "rb") as f:
    files = {"script": f}

    for data in {}, {"updated_on": None}, {"updated_on": "null"}:
        print(f"\n--- data={data} ---\n")
        resp = httpx.post("http://localhost:8000", files=files, data=data)
        print(resp.text)
        print("-" * 120)
        resp = httpx.post("http://localhost:8000/dto", files=files, data=data)
        print(resp.text)

Output with 2.17

--- data={} ---

INFO - 2025-10-08 10:12:00,484 - httpx - _client - HTTP Request: POST http://localhost:8000 "HTTP/1.1 201 Created"
{"filename":"upload.py","updated_on":null}
------------------------------------------------------------------------------------------------------------------------
INFO - 2025-10-08 10:12:00,489 - httpx - _client - HTTP Request: POST http://localhost:8000/dto "HTTP/1.1 201 Created"
{"filename":"upload.py","updated_on":null}

--- data={'updated_on': None} ---

INFO - 2025-10-08 10:12:00,495 - httpx - _client - HTTP Request: POST http://localhost:8000 "HTTP/1.1 201 Created"
{"filename":"upload.py","updated_on":null}
------------------------------------------------------------------------------------------------------------------------
INFO - 2025-10-08 10:12:00,500 - httpx - _client - HTTP Request: POST http://localhost:8000/dto "HTTP/1.1 201 Created"
{"filename":"upload.py","updated_on":null}

--- data={'updated_on': 'null'} ---

INFO - 2025-10-08 10:12:00,509 - httpx - _client - HTTP Request: POST http://localhost:8000 "HTTP/1.1 400 Bad Request"
{"status_code":400,"detail":"Validation failed for POST /","extra":[{"message":"Input should be a valid date or datetime, input is too short","key":"updated_on"}]}
------------------------------------------------------------------------------------------------------------------------
INFO - 2025-10-08 10:12:00,515 - httpx - _client - HTTP Request: POST http://localhost:8000/dto "HTTP/1.1 201 Created"
{"filename":"upload.py","updated_on":null}

Output with 2.18


--- data={} ---

INFO - 2025-10-08 10:11:23,666 - httpx - _client - HTTP Request: POST http://localhost:8000 "HTTP/1.1 201 Created"
{"filename":"upload.py","updated_on":null}
------------------------------------------------------------------------------------------------------------------------
INFO - 2025-10-08 10:11:23,674 - httpx - _client - HTTP Request: POST http://localhost:8000/dto "HTTP/1.1 201 Created"
{"filename":"upload.py","updated_on":null}

--- data={'updated_on': None} ---

INFO - 2025-10-08 10:11:23,681 - httpx - _client - HTTP Request: POST http://localhost:8000 "HTTP/1.1 400 Bad Request"
{"status_code":400,"detail":"Validation failed for POST /","extra":[{"message":"Input should be a valid date or datetime, input is too short","key":"updated_on"}]}
------------------------------------------------------------------------------------------------------------------------
INFO - 2025-10-08 10:11:23,688 - httpx - _client - HTTP Request: POST http://localhost:8000/dto "HTTP/1.1 500 Internal Server Error"
{"status_code":500,"detail":"Internal Server Error"}

--- data={'updated_on': 'null'} ---

INFO - 2025-10-08 10:11:23,694 - httpx - _client - HTTP Request: POST http://localhost:8000 "HTTP/1.1 400 Bad Request"
{"status_code":400,"detail":"Validation failed for POST /","extra":[{"message":"Input should be a valid date or datetime, input is too short","key":"updated_on"}]}
------------------------------------------------------------------------------------------------------------------------
INFO - 2025-10-08 10:11:23,699 - httpx - _client - HTTP Request: POST http://localhost:8000/dto "HTTP/1.1 201 Created"
{"filename":"upload.py","updated_on":null}

…n no file submitted When a browser submits a multipart form without selecting a file, the field is sent as an empty string (""). Since 2.18.0 (PR litestar-org#4271 which fixed litestar-org#4204), this empty string is no longer converted to None, causing validation to fail for Optional[UploadFile] fields with a 400 error. Fix: in _extract_multipart, before the type-hints loop, detect when a field value is "" and the expected type is Optional[UploadFile] (or a subclass), and replace the empty string with None before it reaches the conversion layer. Fixes litestar-org#4647

…ltipart form (#4659) * fix: Optional[UploadFile] with multipart/form-data should be None when no file submitted When a browser submits a multipart form without selecting a file, the field is sent as an empty string (""). Since 2.18.0 (PR #4271 which fixed #4204), this empty string is no longer converted to None, causing validation to fail for Optional[UploadFile] fields with a 400 error. Fix: in _extract_multipart, before the type-hints loop, detect when a field value is "" and the expected type is Optional[UploadFile] (or a subclass), and replace the empty string with None before it reaches the conversion layer. Fixes #4647 * fix: add type annotation for inner to satisfy mypy * refactor: simplify Optional[UploadFile] check using isinstance guard

doubledare704 requested review from a team as code owners August 24, 2025 07:17

github-actions bot added area/kwargs area/multipart area/private-api This PR involves changes to the privatized API size: small pr/external Triage Required 🏥 This requires triage labels Aug 24, 2025

doubledare704 changed the title ~~Fix: Preserve empty strings in multipart/form-data requests (#4204)~~ fix: Preserve empty strings in multipart/form-data requests (#4204) Aug 24, 2025

doubledare704 force-pushed the fix/issue-4204-empty-strings-multipart branch from 4fd77f1 to 31286df Compare August 24, 2025 07:25

github-actions bot added the type/bug label Aug 24, 2025

doubledare704 force-pushed the fix/issue-4204-empty-strings-multipart branch from 31286df to d380e39 Compare August 24, 2025 07:50

doubledare704 marked this pull request as draft August 24, 2025 07:55

doubledare704 force-pushed the fix/issue-4204-empty-strings-multipart branch 5 times, most recently from eb2e4ea to 5d3d9e3 Compare August 24, 2025 08:49

doubledare704 marked this pull request as ready for review August 24, 2025 08:55

doubledare704 marked this pull request as draft August 24, 2025 08:56

doubledare704 force-pushed the fix/issue-4204-empty-strings-multipart branch from 5d3d9e3 to 41a11e9 Compare August 24, 2025 08:58

doubledare704 force-pushed the fix/issue-4204-empty-strings-multipart branch from 41a11e9 to 9c9c45a Compare August 24, 2025 09:02

doubledare704 marked this pull request as ready for review August 24, 2025 09:08

provinzkraut requested changes Aug 24, 2025

View reviewed changes

provinzkraut reviewed Aug 24, 2025

View reviewed changes

tests/unit/test_kwargs/test_multipart_data.py Outdated Show resolved Hide resolved

provinzkraut reviewed Aug 24, 2025

View reviewed changes

litestar/utils/predicates.py Outdated Show resolved Hide resolved

changes based on comments to PR

fbf61b8

doubledare704 force-pushed the fix/issue-4204-empty-strings-multipart branch from 0261fcc to fbf61b8 Compare August 26, 2025 07:30

changes based on comments to PR

a6eb9d7

github-actions bot removed the area/kwargs label Aug 26, 2025

test changes

ca4df46

provinzkraut reviewed Aug 26, 2025

View reviewed changes

tests/unit/test_kwargs/test_multipart_data.py Outdated Show resolved Hide resolved

test changes

68195a2

doubledare704 requested a review from provinzkraut August 27, 2025 20:44

provinzkraut reviewed Sep 2, 2025

View reviewed changes

tests/unit/test_kwargs/test_multipart_data.py Show resolved Hide resolved

provinzkraut requested changes Sep 2, 2025

View reviewed changes

doubledare704 and others added 2 commits September 5, 2025 09:26

Update tests/unit/test_kwargs/test_multipart_data.py

c763cc0

Co-authored-by: Janek Nouvertné <provinzkraut@posteo.de>

additional changes for simplify test cases

cd30507

doubledare704 force-pushed the fix/issue-4204-empty-strings-multipart branch from 9c44181 to cd30507 Compare September 5, 2025 06:44

Merge branch 'main' into fix/issue-4204-empty-strings-multipart

5be1952

doubledare704 force-pushed the fix/issue-4204-empty-strings-multipart branch from 0fcd6e3 to 5be1952 Compare September 5, 2025 06:49

doubledare704 requested a review from provinzkraut September 5, 2025 06:54

provinzkraut approved these changes Sep 10, 2025

View reviewed changes

Merge branch 'main' into fix/issue-4204-empty-strings-multipart

386cc5a

github-actions bot added the pr/internal label Sep 10, 2025

provinzkraut enabled auto-merge (squash) September 10, 2025 17:31

provinzkraut merged commit 7979d6d into litestar-org:main Sep 10, 2025
25 checks passed

doubledare704 deleted the fix/issue-4204-empty-strings-multipart branch September 12, 2025 20:30

provinzkraut added a commit that referenced this pull request Sep 21, 2025

fix: Preserve empty strings in multipart/form-data requests (#4204) (#…

90260ac

…4271) --------- Co-authored-by: Janek Nouvertné <provinzkraut@posteo.de> (cherry picked from commit 7979d6d)

provinzkraut mentioned this pull request Mar 26, 2026

Bug: Optional[UploadFile] with multipart/form-data broken since 2.18.0 #4647

Closed

4 tasks

This was referenced Mar 31, 2026

fix: Optional[UploadFile] should be None when no file submitted in multipart form #4658

Closed

fix: Optional[UploadFile] should be None when no file submitted in multipart form #4659

Merged

Uh oh!

Conversation

doubledare704 commented Aug 24, 2025

Description

Closes #4204

Uh oh!

codecov bot commented Aug 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

provinzkraut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

Uh oh!

provinzkraut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gsakkis commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

euri10 commented Oct 8, 2025

Uh oh!

gsakkis commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Aug 24, 2025 •

edited

Loading

gsakkis commented Oct 7, 2025 •

edited

Loading

gsakkis commented Oct 8, 2025 •

edited

Loading