Skip to content

fix: code crash with a geojson > 2.5Mo after a conflict#3141

Merged
yohanboniface merged 2 commits into
masterfrom
fix-conflict-big-file
Jan 8, 2026
Merged

fix: code crash with a geojson > 2.5Mo after a conflict#3141
yohanboniface merged 2 commits into
masterfrom
fix-conflict-big-file

Conversation

@yohanboniface

Copy link
Copy Markdown
Member

In case of merge conflict, we try to resolve the conflict and then we replace the geojson sent by the user by the merged one. Until now we were doing it by relying on the InMemoryUploadedFile API, but when the sent file is larger than 2.5Mo Django use another class (NamedTemporaryFile), which has another API. So we cannot simply replace the file attribute, which is not the same in both cases.

In case of merge conflict, we try to resolve the conflict and then
we replace the geojson sent by the user by the merged one. Until
now we were doing it by relying on the InMemoryUploadedFile API, but
when the sent file is larger than 2.5Mo Django use another class
(NamedTemporaryFile), which has another API. So we cannot simply
replace the file attribute, which is not the same in both cases.
@yohanboniface

Copy link
Copy Markdown
Member Author

Eg. of traceback from osmfr server when the original issue happens:

Jan 08 11:50:57 umap uvicorn[2474707]: Internal Server Error: /fr/map/989221/datalayer/update/ffd9967f-860a-49ee-becc-71590bb464e8/
Jan 08 11:50:57 umap uvicorn[2474707]: Traceback (most recent call last):
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/asgiref/sync.py", line 555, in thread_handler
Jan 08 11:50:57 umap uvicorn[2474707]:     raise exc_info[1]
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 42, in inner
Jan 08 11:50:57 umap uvicorn[2474707]:     response = await get_response(request)
Jan 08 11:50:57 umap uvicorn[2474707]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/asgiref/sync.py", line 555, in thread_handler
Jan 08 11:50:57 umap uvicorn[2474707]:     raise exc_info[1]
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
Jan 08 11:50:57 umap uvicorn[2474707]:     response = await wrapped_callback(
Jan 08 11:50:57 umap uvicorn[2474707]:                ^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/asgiref/sync.py", line 504, in __call__
Jan 08 11:50:57 umap uvicorn[2474707]:     ret = await asyncio.shield(exec_coro)
Jan 08 11:50:57 umap uvicorn[2474707]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/asgiref/current_thread_executor.py", line 40, in run
Jan 08 11:50:57 umap uvicorn[2474707]:     result = self.fn(*self.args, **self.kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/asgiref/sync.py", line 559, in thread_handler
Jan 08 11:50:57 umap uvicorn[2474707]:     return func(*args, **kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/sentry_sdk/integrations/django/views.py", line 87, in sentry_wrapped_callback
Jan 08 11:50:57 umap uvicorn[2474707]:     return callback(request, *args, **kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/views/decorators/cache.py", line 80, in _view_wrapper
Jan 08 11:50:57 umap uvicorn[2474707]:     response = view_func(request, *args, **kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/views/generic/base.py", line 105, in view
Jan 08 11:50:57 umap uvicorn[2474707]:     return self.dispatch(request, *args, **kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/views/generic/base.py", line 144, in dispatch
Jan 08 11:50:57 umap uvicorn[2474707]:     return handler(request, *args, **kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/umap/views.py", line 1343, in post
Jan 08 11:50:57 umap uvicorn[2474707]:     return super().post(request, *args, **kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/views/generic/edit.py", line 206, in post
Jan 08 11:50:57 umap uvicorn[2474707]:     return super().post(request, *args, **kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/views/generic/edit.py", line 151, in post
Jan 08 11:50:57 umap uvicorn[2474707]:     return self.form_valid(form)
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/umap/views.py", line 1346, in form_valid
Jan 08 11:50:57 umap uvicorn[2474707]:     self.object = form.save()
Jan 08 11:50:57 umap uvicorn[2474707]:                   ^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/forms/models.py", line 554, in save
Jan 08 11:50:57 umap uvicorn[2474707]:     self.instance.save()
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/umap/models.py", line 536, in save
Jan 08 11:50:57 umap uvicorn[2474707]:     super(DataLayer, self).save(**kwargs)
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/db/models/base.py", line 902, in save
Jan 08 11:50:57 umap uvicorn[2474707]:     self.save_base(
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/db/models/base.py", line 1008, in save_base
Jan 08 11:50:57 umap uvicorn[2474707]:     updated = self._save_table(
Jan 08 11:50:57 umap uvicorn[2474707]:               ^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/db/models/base.py", line 1132, in _save_table
Jan 08 11:50:57 umap uvicorn[2474707]:     (getattr(self, f.attname) if raw else f.pre_save(self, False)),
Jan 08 11:50:57 umap uvicorn[2474707]:                                           ^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/db/models/fields/files.py", line 338, in pre_save
Jan 08 11:50:57 umap uvicorn[2474707]:     file.save(file.name, file.file, save=False)
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/db/models/fields/files.py", line 99, in save
Jan 08 11:50:57 umap uvicorn[2474707]:     self.name = self.storage.save(name, content, max_length=self.field.max_length)
Jan 08 11:50:57 umap uvicorn[2474707]:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/core/files/storage/base.py", line 49, in save
Jan 08 11:50:57 umap uvicorn[2474707]:     name = self._save(name, content)
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/core/files/storage/filesystem.py", line 115, in _save
Jan 08 11:50:57 umap uvicorn[2474707]:     content.temporary_file_path(),
Jan 08 11:50:57 umap uvicorn[2474707]:     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]:   File "/srv/umap/venv/lib/python3.12/site-packages/django/core/files/uploadedfile.py", line 84, in temporary_file_path
Jan 08 11:50:57 umap uvicorn[2474707]:     return self.file.name
Jan 08 11:50:57 umap uvicorn[2474707]:            ^^^^^^^^^^^^^^
Jan 08 11:50:57 umap uvicorn[2474707]: AttributeError: '_io.BytesIO' object has no attribute 'name'

Comment thread umap/views.py Outdated

@davidbgk davidbgk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be pertinent to bump FILE_UPLOAD_MAX_MEMORY_SIZE a bit too?

Co-authored-by: David Larlet <3556+davidbgk@users.noreply.github.com>
@yohanboniface

Copy link
Copy Markdown
Member Author

Would it be pertinent to bump FILE_UPLOAD_MAX_MEMORY_SIZE a bit too?

Humm, what for ? 🤔

@davidbgk

davidbgk commented Jan 8, 2026

Copy link
Copy Markdown
Contributor

Would it be pertinent to bump FILE_UPLOAD_MAX_MEMORY_SIZE a bit too?

Humm, what for ? 🤔

Not sure what's more efficient on today's hardware.

@yohanboniface yohanboniface merged commit 2410e2e into master Jan 8, 2026
3 checks passed
@yohanboniface yohanboniface deleted the fix-conflict-big-file branch January 8, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants