Skip to content

Inefficient merging of data #61

Description

@wonder-sk

In make_work_packages(), this bit can be very inefficient for larger files:

geodiff.create_changeset(wp_gpkg_base, master_gpkg_output, wp_changeset_base_output)

it creates big diff files, because the master gpkg has typically much more data than a wp gpkg. This should be avoided, and we should probably just concatenate diffs that we will apply to the master gpkg.

Another source of slowness are list_changes() calls, especially:

        geodiff.list_changes(wp_changeset_base_output, wp_changeset_base_output_json)

... which convert large diff from the previous step to even larger JSONs and that takes a lot of time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions