This is to support de-duplication filtering of huge query results and to safeguard against running out of memory.
In order to provide de-duplication (for example when polygon geometries have multiple insertion IDs) we cache insertion IDs as query results are streamed back. However, if the results are on the order of millions it is possible that this cache can exceed a reasonable memory footprint. We should safeguard against this unlikely situation by overflowing the deduplication cache to disk in the rare occurrence that the cached result IDs are huge.
This is to support de-duplication filtering of huge query results and to safeguard against running out of memory.
In order to provide de-duplication (for example when polygon geometries have multiple insertion IDs) we cache insertion IDs as query results are streamed back. However, if the results are on the order of millions it is possible that this cache can exceed a reasonable memory footprint. We should safeguard against this unlikely situation by overflowing the deduplication cache to disk in the rare occurrence that the cached result IDs are huge.