-
Notifications
You must be signed in to change notification settings - Fork 315
could add a MinHashLSHDeleteSession, similar as MinHashLSHInsertionSession #247
Copy link
Copy link
Closed
Labels
Description
Dear Zhu,
Thanks for your codes, and we starts to use with redis.
In our usage, we have a situations to update keys, that's to delete unneeded keys and insert new keys.
Yes, you created the MinHashLSHInsertionSession to batch insertion which is very useful for redis as storage.
Now our bottleneck the one by one deletion is very slow with redis. You can see the simple demo codes and the slow screenshots: deleting 77 keys using 50 mintues with remote redis!
Could you add batch del feature such as MinHashLSHDeleteSession for redis storage?
thanks!!
def del_key_not_in_input_keys(keys_to_del, lsh: MinHashLSH):
""" """
logger.info(f'Starts to del {len(keys_to_del)} keys not in input_keys')
try:
for key_to_del in tqdm(keys_to_del): # type: ignore
lsh.remove(key_to_del)
except Exception as e:
logger.error(f"LSH remove {keys_to_del = }\n{e}")Reactions are currently unavailable
