Skip to content

[PATCH][273] Allow prepickle to be False for Redis storage#274

Merged
ekzhu merged 13 commits intoekzhu:masterfrom
Varun0157:redis-prepickle
Nov 11, 2025
Merged

[PATCH][273] Allow prepickle to be False for Redis storage#274
ekzhu merged 13 commits intoekzhu:masterfrom
Varun0157:redis-prepickle

Conversation

@Varun0157
Copy link
Copy Markdown
Contributor

@Varun0157 Varun0157 commented Nov 8, 2025

As per #273, we change the prepickle assignment to only default to True if the argument is not specified. Earlier, we were assigning it to True if it was ever falsy.
This pattern aligns with the experimental code, which seems to allow for prepickle to be False in the expected manner.

In order to test the change, we alter one of the (recently added) existing unit tests to work with prepickle=False for a Redis store.

@ekzhu ekzhu changed the title [BREAKING][273] Allow prepickle to be False for Redis storage [PATCH][273] Allow prepickle to be False for Redis storage Nov 10, 2025
@ekzhu
Copy link
Copy Markdown
Owner

ekzhu commented Nov 10, 2025

Thanks for the patch. I removed the word "BREAKING" from the title as this is a bug fix that enables specified behavior.

There are some conflicts with the master branch.

@Varun0157
Copy link
Copy Markdown
Contributor Author

Varun0157 commented Nov 10, 2025

Thanks for the patch. I removed the word "BREAKING" from the title as this is a bug fix that enables specified behavior.

There are some conflicts with the master branch.

Thanks for the change! I've resolved the conflicts.

@Varun0157 Varun0157 requested a review from ekzhu November 10, 2025 08:24
bhimrazy and others added 2 commits November 10, 2025 21:15
…zhu#271)

* Migrate build system to pyproject.toml and uv

* Update Readme docs for the uv build

* python version update and readme fixes as per review feedback

* update pyprojct config and remove setup py and all

* Fix formatting and punctuation in pyproject.toml

* Update ignore list in ruff configuration to include E731

* style: fix import ordering with ruff (I001)

* style: remove trailing whitespace (W291, W293)

* refactor: apply pyupgrade fixes - remove useless object inheritance and encode UTF-8 (UP004, UP012, UP026, UP032)

* style: fix import ordering (I001)

* refactor: apply minor pyupgrade fixes (UP005, UP034, W292)

* style: remove more trailing whitespace

* fix formatting

* style: reorganize imports to use collections.abc for type hints

* style: ignore deprecated imports (UP035)

* style: reorganize imports to use collections.abc for type hints

* ignore N801 and N803

* style: move version import to top of file (E402)

* style: move version import above weighted_minhash imports

* style: remove Python version check for AsyncMinHashLSH import

* style: reorder import statements for consistency

* style: ignore F841 for unused local variable assignments

* style: ignore UP028 for using `yield from` instead of `for` loop

* style: add additional ignore rules for ruff linter

* style: revert to Optional and Union syntax instead of | operator

* style: add ignore rule for undefined name `buffer` in ruff linter

* style: update type hints to use built-in list and tuple syntax

* style: add ignore rule for invalid function name in ruff linter

* style: add ignore rule for super-call-with-parameters in ruff linter

* style: add ignore rule for ambiguous variable name in ruff linter

* style: update ruff linter ignore rules for yield-in-for-loop and printf-string-formatting

* style: add ignore rule for non-lowercase-variable-in-function in ruff linter

* fix tab-indentation (W191)

* style: update workflows to use 'uv' for package management and installation

* style: update import statement for MinHash to correct module path

* style: reorganize steps in pypi.yml for clarity and consistency

* style: enhance pypi.yml workflow by adding package checks and default branch handling

* style: update pypi.yml to remove default branch reference and clean up steps

* style: rename workflow to 'Build and Publish Package' and update steps for clarity

* style: add comment to clarify condition for manual trigger in PyPI publish step

* comment trigger for build

* style: update license format in pyproject.toml and clean up ignored lint rules

* drop 3.8 as it's already EOL

* update

* comment environment temporarily

* chore: switch to hatchling for build system and update project configuration

* update lock file

* revert almost to original origianl

* add GitHub Actions workflow for building and verifying package

* add pytest configuration to pyproject.toml

* docs: update README to include contributing guidelines and development setup instructions

* activate env

* test extra tests to verify

* ci: update Python version to 3.12 in CI workflows for Cassandra and MongoDB

* update lock

* fix headings

* add pytest-rerunfailures dependency and flaky test marker

* okay; mongo and cassandra tests are also passing let's remove the pr trigger

* update

* exclude formatting for `datasketch/hyperloglog_const.py` adding major file changes

* fix type checks

* apply suggestiosn to use `3.10` in ci for compatibility

* fix formatting

* apply suggestions for version

* updated lock file

* update ordering in classifiers

* fix: SIM401

* fix: SIM300

* fix: SIM210

* fix SIM117

* fix: S108

* fix S101

* fix : RUF005

* fix: RET508

* fix ISC003

* fix ERA001

* fix D405

* fix SIM108

* fix SIM103

* fix : D413

* fix: D202

* fix: D212

* fix D208

* fix RUF023

* fix RUF023

* fix RUF023

* fix D403

* fix D209

* updated pyproect toml in ref with: https://github.com/microsoft/agent-framework/blob/1aaf37dab8ee4534ae2000082a08ba112173cc4c/python/pyproject.toml#L116

* fix RET505

* fix D412

* add ignore and tdo

* remove unused imports from multiple files

* fix F401

* add todo ignores

* fix D200

* add ignores

* update

* fix D415

* update

* fix RET504

* fix RET503

* fix B023

* fix T201, replace print with logger

* fix SIM102

* fix SIM212

* fix SIM212

* fix SIM118

* update

* fix D415

* add ignore todo for pickle

* fix E501

* update

* fix B007

* fix RUF013

* fix RUF059

* fix RUF059

* fix F841

* fix stack level warning

* fix D404

* ad ignore

* refactor: replace ABCMeta with ABC for compatibility

* apply formatting

* chore: add ruff checks workflow and remove lint step from test workflow

* update

* update readme

* feat: add link-check job to workflow for README.md validation

* update

* fix dead links

* add ignore

* trigger all tests

* all tests passing, revert temp trigger

* update to 3.10

---------

Co-authored-by: Sarang Pratham <sarangpratham720@gmail.com>

remove excess files
@ekzhu ekzhu merged commit b27cb86 into ekzhu:master Nov 11, 2025
7 of 8 checks passed
bhimrazy added a commit to bhimrazy/datasketch that referenced this pull request Nov 11, 2025
…#274)

* implement buffered removal for Redis storage

* implement MinHashLSHDeletionSession

* add unit tests for session deletion

* add unit test for redis deletion session

* pass config and name (actual params of constructor) instead of *args

* re-write comment for clarity

* only default to True for redis if prepickle is `None`.

* account for byte storage of keys

* [BREAKING] Migrate to uv package management and also update CI/CD (ekzhu#271)

* Migrate build system to pyproject.toml and uv

* Update Readme docs for the uv build

* python version update and readme fixes as per review feedback

* update pyprojct config and remove setup py and all

* Fix formatting and punctuation in pyproject.toml

* Update ignore list in ruff configuration to include E731

* style: fix import ordering with ruff (I001)

* style: remove trailing whitespace (W291, W293)

* refactor: apply pyupgrade fixes - remove useless object inheritance and encode UTF-8 (UP004, UP012, UP026, UP032)

* style: fix import ordering (I001)

* refactor: apply minor pyupgrade fixes (UP005, UP034, W292)

* style: remove more trailing whitespace

* fix formatting

* style: reorganize imports to use collections.abc for type hints

* style: ignore deprecated imports (UP035)

* style: reorganize imports to use collections.abc for type hints

* ignore N801 and N803

* style: move version import to top of file (E402)

* style: move version import above weighted_minhash imports

* style: remove Python version check for AsyncMinHashLSH import

* style: reorder import statements for consistency

* style: ignore F841 for unused local variable assignments

* style: ignore UP028 for using `yield from` instead of `for` loop

* style: add additional ignore rules for ruff linter

* style: revert to Optional and Union syntax instead of | operator

* style: add ignore rule for undefined name `buffer` in ruff linter

* style: update type hints to use built-in list and tuple syntax

* style: add ignore rule for invalid function name in ruff linter

* style: add ignore rule for super-call-with-parameters in ruff linter

* style: add ignore rule for ambiguous variable name in ruff linter

* style: update ruff linter ignore rules for yield-in-for-loop and printf-string-formatting

* style: add ignore rule for non-lowercase-variable-in-function in ruff linter

* fix tab-indentation (W191)

* style: update workflows to use 'uv' for package management and installation

* style: update import statement for MinHash to correct module path

* style: reorganize steps in pypi.yml for clarity and consistency

* style: enhance pypi.yml workflow by adding package checks and default branch handling

* style: update pypi.yml to remove default branch reference and clean up steps

* style: rename workflow to 'Build and Publish Package' and update steps for clarity

* style: add comment to clarify condition for manual trigger in PyPI publish step

* comment trigger for build

* style: update license format in pyproject.toml and clean up ignored lint rules

* drop 3.8 as it's already EOL

* update

* comment environment temporarily

* chore: switch to hatchling for build system and update project configuration

* update lock file

* revert almost to original origianl

* add GitHub Actions workflow for building and verifying package

* add pytest configuration to pyproject.toml

* docs: update README to include contributing guidelines and development setup instructions

* activate env

* test extra tests to verify

* ci: update Python version to 3.12 in CI workflows for Cassandra and MongoDB

* update lock

* fix headings

* add pytest-rerunfailures dependency and flaky test marker

* okay; mongo and cassandra tests are also passing let's remove the pr trigger

* update

* exclude formatting for `datasketch/hyperloglog_const.py` adding major file changes

* fix type checks

* apply suggestiosn to use `3.10` in ci for compatibility

* fix formatting

* apply suggestions for version

* updated lock file

* update ordering in classifiers

* fix: SIM401

* fix: SIM300

* fix: SIM210

* fix SIM117

* fix: S108

* fix S101

* fix : RUF005

* fix: RET508

* fix ISC003

* fix ERA001

* fix D405

* fix SIM108

* fix SIM103

* fix : D413

* fix: D202

* fix: D212

* fix D208

* fix RUF023

* fix RUF023

* fix RUF023

* fix D403

* fix D209

* updated pyproect toml in ref with: https://github.com/microsoft/agent-framework/blob/1aaf37dab8ee4534ae2000082a08ba112173cc4c/python/pyproject.toml#L116

* fix RET505

* fix D412

* add ignore and tdo

* remove unused imports from multiple files

* fix F401

* add todo ignores

* fix D200

* add ignores

* update

* fix D415

* update

* fix RET504

* fix RET503

* fix B023

* fix T201, replace print with logger

* fix SIM102

* fix SIM212

* fix SIM212

* fix SIM118

* update

* fix D415

* add ignore todo for pickle

* fix E501

* update

* fix B007

* fix RUF013

* fix RUF059

* fix RUF059

* fix F841

* fix stack level warning

* fix D404

* ad ignore

* refactor: replace ABCMeta with ABC for compatibility

* apply formatting

* chore: add ruff checks workflow and remove lint step from test workflow

* update

* update readme

* feat: add link-check job to workflow for README.md validation

* update

* fix dead links

* add ignore

* trigger all tests

* all tests passing, revert temp trigger

* update to 3.10

---------

Co-authored-by: Sarang Pratham <sarangpratham720@gmail.com>

remove excess files

* align with ruff formatting

---------

Co-authored-by: Bhimraj Yadav <bhimrajyadav977@gmail.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants