Skip to content

Implement de-duplication ID cache overflow to disk #97

@rfecher

Description

@rfecher

This is to support de-duplication filtering of huge query results and to safeguard against running out of memory.

In order to provide de-duplication (for example when polygon geometries have multiple insertion IDs) we cache insertion IDs as query results are streamed back. However, if the results are on the order of millions it is possible that this cache can exceed a reasonable memory footprint. We should safeguard against this unlikely situation by overflowing the deduplication cache to disk in the rare occurrence that the cached result IDs are huge.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions