Safe JSON serializer & content-length fix (issue with Routes in Revit French)#3161
Conversation
Add a Unicode-safe JSON serializer (_safe_json_dumps) to handler.py to avoid json.dumps UnicodeDecodeError on IronPython by escaping non-ASCII as \uXXXX. Use json.dumps(..., ensure_ascii=True) with a fallback to _safe_json_dumps when encoding errors occur. Update HttpRequestHandler._write_response to always compute a bytes body, set a correct Content-Length header, skip duplicate Content-Length from response headers, and write the encoded body. Add unit tests covering response framing (Content-Length and body bytes) and Unicode-safe JSON serialization/parse_response behavior.
There was a problem hiding this comment.
PR Summary:
- Adds
_safe_json_dumps()fallback serializer to handleUnicodeDecodeErrorin IronPython 2.7 on French/accented Revit strings. - Updates
HttpRequestHandler._write_responseto always encode the body as bytes and set a correctContent-Lengthheader, skipping any duplicate from response headers. - Adds two unit test files covering response framing and Unicode JSON serialization.
Review Summary:
The Content-Length framing fix is a solid, correct improvement and the overall approach (try json.dumps(ensure_ascii=True), fall back to _safe_json_dumps) is sound. However, there is a critical functional bug: the fallback function _safe_json_dumps checks isinstance(obj, str) which, in IronPython 2.7, will never match unicode objects — precisely the type that carries accented Revit strings. The docstring even incorrectly claims IronPython 2.7 unifies str and unicode (they are distinct; only IronPython 3 unifies them). This means the fallback is unreachable for the exact inputs it was designed to handle. Additionally, the new \n body written for all None-data responses (including 204 No Content) violates RFC 7230. The IPY2712 repo guideline was the key lens for catching the unicode type gap.
Follow-up suggestions:
@devloaifix the identified issues: addunicodeto theisinstancecheck in_safe_json_dumps(usingpyrevit.compat.PY2), guard the\nfallback against no-body status codes (204, 304), and handleinf/nanfloats.
Handle float edge cases in _safe_json_dumps by detecting NaN and infinite values (using math.isnan/math.isinf) and returning "null" so the output remains valid JSON. Adds the math import and the corresponding check in pyrevitlib/pyrevit/routes/server/handler.py.
|
@jmcouffin Bug fixes to the routes API to work properly with Revit, French, and other multilingual Revit installations with non English characters. |
Apply consistent string quoting, spacing, and formatting across routes/server and tests. Changes include normalizing single to double quotes, consistent use of join and string concatenation, cleaning up pylint comment formatting, minor refactors (parentheses, trailing commas, line breaks), and small-safe-json-dumps tweaks. Tests updated to match the formatting changes. No functional behavior changes intended.
Make the _safe_json_dumps docstring a raw string literal to avoid accidental escape processing (e.g., \u sequences) on IronPython. Also adjust the unit test docstring capitalization for clarity in test_routes_server_unicode.py.
There was a problem hiding this comment.
Pull request overview
This PR addresses IronPython Unicode serialization failures in the routes server (notably seen with non-ASCII route data in localized Revit environments) and fixes HTTP response framing by ensuring response bodies are written as bytes with an accurate Content-Length.
Changes:
- Added a Unicode-safe JSON serializer fallback (
_safe_json_dumps) and integrated it intoparse_responsewhenjson.dumpsraises Unicode encoding/decoding errors. - Updated
HttpRequestHandler._write_responseto always compute a bytes body, send a single correctContent-Length, and write the encoded payload. - Added unit tests for Unicode-safe serialization and response framing (
Content-Lengthand written bytes).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
pyrevitlib/pyrevit/routes/server/handler.py |
Adds _safe_json_dumps and uses it as a fallback in parse_response to avoid IronPython Unicode errors. |
pyrevitlib/pyrevit/routes/server/server.py |
Ensures _write_response writes bytes and sets a correct Content-Length, avoiding duplicate Content-Length headers. |
pyrevitlib/pyrevit/unittests/test_routes_server_unicode.py |
New tests covering Unicode-safe JSON serialization and parse_response behavior with accented characters. |
pyrevitlib/pyrevit/unittests/test_routes_server_response.py |
New tests validating response framing (Content-Length, bytes body, header override behavior). |
…N fallback add Routes Module Tests button under pyRevitDev > Debug > Unit Tests discover and run all pyrevit.unittests.test_routes_* modules print detailed unittest error/failure tracebacks in button output harden RequestHandler.parse_response JSON fallback for IronPython runtime exceptions preserve numeric serialization for Python 2 long values via numbers.Integral make routes response tests IronPython-safe (handler patching + method binding compatibility) make unicode parse-response tests deterministic for fallback/exception string behavior
|
Added test menu for some of the routes code that I touched (additional coverage needed in future for pre-existing code): Tests pass: |
|
Thanks @vinnividivicci |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26071+0706-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26073+1714-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26073+2220-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26074+1648-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26076+0001-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26077+2319-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26079+2333-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26080+1314-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26083+2031-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26083+2042-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26083+2045-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26083+2048-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26083+2106-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26083+2130-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26086+2004-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26088+1318-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26089+1231-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26090+0549-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26090+1533-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26090+1536-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26090+1540-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26090+1540-wip |
|
📦 New work-in-progress (wip) builds are available for 6.1.0.26090+1556-wip |
|
📦 New public release are available for 6.2.0.26090+1754 |
Safe JSON serializer & content-length fix (issue with Routes in Revit French)
Description
Add a Unicode-safe JSON serializer (_safe_json_dumps) to handler.py to avoid json.dumps UnicodeDecodeError on IronPython by escaping non-ASCII as \uXXXX. Use json.dumps(..., ensure_ascii=True) with a fallback to _safe_json_dumps when encoding errors occur. Update HttpRequestHandler._write_response to always compute a bytes body, set a correct Content-Length header, skip duplicate Content-Length from response headers, and write the encoded body. Add unit tests covering response framing (Content-Length and body bytes) and Unicode-safe JSON serialization/parse_response behavior.
Checklist
Before submitting your pull request, ensure the following requirements are met:
pipenv run black {source_file_or_directory}Related Issues
If applicable, link the issues resolved by this pull request:
Additional Notes
Include any additional context, screenshots, or considerations for reviewers.
Thank you for contributing to pyRevit! 🎉