Skip to content

slow escaping of byte strings in Python 3 #366

@dimitri-yatsenko

Description

@dimitri-yatsenko

When I insert large blobs, e.g. a 10 MB byte string into a longblob field, it takes several seconds to prepare the value for insertion. Profiling indicates that most of that time is spent in the escape_bytes function in converters.py. Here is that function for reference.

def escape_bytes(value, mapping=None):
    # escape_bytes is calld only on Python 3.
    return escape_str(value.decode('ascii', 'surrogateescape'), mapping)

When I performed the same insertion through another interface, the escaping was about 100 times faster.

Is there a way to speed this up?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions