Skip to content

could add a MinHashLSHDeleteSession, similar as MinHashLSHInsertionSession #247

@rocke2020

Description

@rocke2020

Dear Zhu,

Thanks for your codes, and we starts to use with redis.
In our usage, we have a situations to update keys, that's to delete unneeded keys and insert new keys.
Yes, you created the MinHashLSHInsertionSession to batch insertion which is very useful for redis as storage.
Now our bottleneck the one by one deletion is very slow with redis. You can see the simple demo codes and the slow screenshots: deleting 77 keys using 50 mintues with remote redis!

Could you add batch del feature such as MinHashLSHDeleteSession for redis storage?
thanks!!

Image

def del_key_not_in_input_keys(keys_to_del, lsh: MinHashLSH):
    """ """
    logger.info(f'Starts to del {len(keys_to_del)} keys not in input_keys')
    try:
        for key_to_del in tqdm(keys_to_del):  # type: ignore
            lsh.remove(key_to_del)
    except Exception as e:
        logger.error(f"LSH remove {keys_to_del = }\n{e}")

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions