-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Closed
Description
Current file upload MIME parsing in Twisted Web requires, based on the interface, requires parsing the whole thing into memory. So for large uploads this can result in very large memory usage. The API for users specifically involves accessing the mapping request.args.
How to solve this?
- Long term, one could do full-on async streaming parsing, but that is a large project. It also doesn't actually solve the problem in the short term, since for backwards compatibility
request.argswould still need to exist. - On-demand parsing only when
request.argsis accessed. This is not ideal since it makes it requires blocking, which can be meaningful for large uploads. - Allow users to opt-out so they can just do their own parsing (or use the streaming future API, if that ever happens). Tahoe-LAFS already wants to do this, for example, and right now overrides the parsing logic already in a pretty hacky way.
I propose to implement option 3.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels