Skip to content

PSRSimpleCache - Performance issue #1035

@norberttech

Description

@norberttech

Because of the internal index in PSRSimpleCache every single time we want to add something into the cache, we need to perform following operations:

  • check if an index exists in the cache
  • read index from the cache (if it exists)
  • merge that index with new rows
  • write that index into the cache

so only after that, we can put actual rows into the cache. The biggest bottleneck is here:

https://github.com/flow-php/flow/blob/1.x/src/core/etl/src/Flow/ETL/ExternalSort/CacheExternalSort.php#L42-L44

All the above operations on cache will be executed at least as many times as many rows we have. So 10k rows will generate around 40k hits to cache storage. This problem does not exists with the LocalFIlesystemCache because instead of checking if the index exists, it's simply trying to create or open it and then just appending new id at the end of it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions