The lock file was updated by running

poetry update redis

inside components/job-orchestration after changing the redis version in pyproject.toml

🧹 Nitpick (assertive)

Optional: Document the rationale for pinning to >=6.4.0 to prevent accidental downgrades.

Add an inline comment referencing the deadlock context (issue #1059). This helps future maintainers avoid “optimizing” the version downwards and reintroducing the issue.

Apply this diff:

 [tool.poetry.dependencies]
 python = ">= 3.9"
@@
 PyYAML = "^6.0.1"
-redis = "^6.4.0"
+# NOTE: 6.4.0+ required to avoid Celery Redis backend deadlock (see #1059).
+redis = "^6.4.0"

In components/job-orchestration/pyproject.toml around line 24, the redis
dependency is pinned to "^6.4.0" but lacks an explanation; update that line to
include an inline comment stating the rationale: we require redis >=6.4.0 to
avoid a known deadlock bug (reference issue #1059) so maintainers don’t
accidentally downgrade; ensure the comment is concise, mentions "deadlock (issue
#1059)" and why the minimum is required, and keep formatting valid for TOML.

💡 Verification agent

#!/bin/bash
set -euo pipefail

echo "==> Searching for potentially incompatible redis APIs (StrictRedis, redis.asyncio direct usage)"
rg -n -C2 --type=python -P '(from\s+redis\s+import\s+StrictRedis\b|redis\.StrictRedis\b|redis\.asyncio\b)' || echo "No deprecated/fragile redis APIs found."

echo
echo "==> Locating poetry.lock files and checking resolved versions for redis and celery"
fd -a -t f '^poetry\.lock$' | while read -r lock; do
  echo "--- $lock ---"
  echo "[[package]] name = \"redis\" entries near version:"
  rg -n -C3 -F 'name = "redis"' "$lock" || echo "redis not found in $lock"
  echo
  echo "[[package]] name = \"celery\" entries near version:"
  rg -n -C3 -F 'name = "celery"' "$lock" || echo "celery not found in $lock"
  echo
done

echo
echo "==> Verifying the dependency declaration in components/job-orchestration/pyproject.toml"
rg -n -C2 -F 'redis = "^6.4.0"' components/job-orchestration/pyproject.toml

Does Celery 5.3.6 officially support redis-py 6.x (including kombu’s redis transport)?

Incompatible Celery/Kombu with redis-py 6.x detected

Celery 5.3.6 (via Kombu 5.3.x) officially supports only redis-py 4.x, not 6.x. Locking redis = "^6.4.0" risks breaking the Redis transport.

Please apply one of the following critical fixes:

• Pin redis to the Kombu-supported range:
  • In components/job-orchestration/pyproject.toml (line 24), change
  redis = "^6.4.0"
  to something like
  redis = ">=4.5.2,<5.0"
• OR upgrade Celery/Kombu to a release that explicitly supports redis-py 6.x:
  • Bump celery (and implicitly Kombu) in pyproject.toml, then run poetry lock --no-update redis to ensure redis-py resolves to a compatible version.
• After making either change, regenerate your lockfile and verify:
  • poetry.lock should show redis 4.x under components/job-orchestration/poetry.lock
  • celery should remain at 5.3.6 (if not upgrading) or move to your target version if you choose to bump.

In components/job-orchestration/pyproject.toml around line 24, the redis
dependency is pinned to an incompatible 6.x (redis = "^6.4.0") which breaks
Celery/Kombu; either change that line to a Kombu-supported range (for example
redis = ">=4.5.2,<5.0") OR upgrade celery/kombu to a version that explicitly
supports redis-py 6.x, then regenerate the lockfile; after making the change run
poetry lock (or poetry lock --no-update redis if you upgraded Celery) and verify
components/job-orchestration/poetry.lock shows redis 4.x (and celery remains at
5.3.6 if you did not upgrade).

I was looking at the lock file and realized that Celery actually specifies a version constraint on the "extra" redis-py. Since we're not using redis-py directly in our code, we should install redis-py as an extra of celery (akin to pip install celery[redis]).

We can do this by removing redis from pyproject.toml and updating the celery install to:

-celery = "^5.3.6"
+celery = {extras = ["redis"], version = "^5.3.6"}

Note that Celery locks to redis-py <= v5.2.1 rather than v6.4.0. I tested the change locally and it also seems to resolve the issue (at least no freezes after 200 queries).

🛠️ Refactor suggestion

Optionally enforce redis-py >= 6.4.0 to prevent accidental downgrades (only if Kombu in this Celery series supports redis 6.x).

If your lock check confirms Kombu in Celery ^5.5.3 supports redis 6.x, consider adding an explicit lower bound to avoid future downgrades when the lock is refreshed. If this conflicts during resolution, keep relying solely on celery[redis] and skip this.

Apply this diff only if resolution succeeds:

 [tool.poetry.dependencies]
 python = ">= 3.9"
 Brotli = "^1.1.0"
 celery = {extras = ["redis"], version = "^5.5.3"}
+# Enforce min redis-py that includes the deadlock fix (issue #1059; redis-py PR 3677).
+redis = ">=6.4.0,<7"

Follow up:

• Run: poetry lock --no-interaction
• Re-verify the lockfile contains redis >= 6.4.0 and Kombu consistent with Celery.

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
[tool.poetry.dependencies]
python = ">= 3.9"
Brotli = "^1.1.0"
celery = {extras = ["redis"], version = "^5.5.3"}
# Enforce min redis-py that includes the deadlock fix (issue #1059; redis-py PR 3677).
redis = ">=6.4.0,<7"

In components/job-orchestration/pyproject.toml around line 14, the celery[redis]
extra may allow redis-py to be downgraded; if Kombu used by Celery ^5.5.3 is
compatible with redis-py 6.x, add an explicit lower bound by changing the
dependency to require redis >=6.4.0 (e.g. add a redis = ">=6.4.0" entry or
adjust celery extras accordingly), then run `poetry lock --no-interaction` and
verify the lockfile contains redis >= 6.4.0 and that Kombu resolution remains
consistent; if dependency resolution fails or Kombu is incompatible, leave
celery[redis] as-is and do not apply the change.

💡 Verification agent

#!/bin/bash
set -euo pipefail

echo "==> Checking resolved versions in poetry.lock (all occurrences)"
fd -a -t f '^poetry\.lock$' | while read -r lock; do
  echo
  echo "--- $lock ---"
  rg -n -C2 -F 'name = "celery"' "$lock" || true
  rg -n -C2 -F 'name = "kombu"' "$lock" || true
  rg -n -C2 -F 'name = "redis"' "$lock" || true
done

echo
echo "==> Confirming the dependency declaration that drives resolution"
rg -n -C2 -F 'celery = {extras = ["redis"]' components/job-orchestration/pyproject.toml || true

echo
echo "==> Guard-rail: scan for fragile/legacy redis APIs in code"
rg -n -C2 --type=python -P '(from\s+redis\s+import\s+StrictRedis\b|redis\.StrictRedis\b|redis\.asyncio\b)' || echo "No legacy redis APIs found."

echo
echo "==> Summary check"
if fd -a -t f '^poetry\.lock$' | xargs rg -nP 'name = "redis"\n.*\nversion = "([0-9.]+)"' -U | awk -F'"' '{print $6}' | awk 'BEGIN{ok=0} {split($0,a,"."); if (a[1]>=6 && (a[2]>4 || (a[2]==4 && a[3]>=0))) ok=1} END{exit ok?0:1}'; then
  echo "redis-py appears to be >= 6.4.0 in the lock."
else
  echo "redis-py is below 6.4.0 or not found at the expected place in the lock; please investigate."
  exit 1
fi

Celery[redis] is locking redis-py at 5.2.1, below the required 6.4.0. Critical dependency bump needed.

Please update the dependency declaration and lockfile so that redis-py >= 6.4.0 is actually pulled in, and ensure Kombu remains compatible with the newer client. For example:

• In components/job-orchestration/pyproject.toml (around line 14), either:

  • Add an explicit top-level dependency

    redis = ">=6.4.0"

    alongside the existing celery = {extras = ["redis"], version = "^5.5.3"}, or
  • Bump Celery (and thus its bundled Kombu) to a version whose extras allow redis-py 6.x.

• Run poetry lock and verify in components/job-orchestration/poetry.lock that:

  • name = "redis" shows version = ">=6.4.0" (or a specific 6.4.x+ entry), and
  • name = "kombu" remains at a version known to support redis-py 6.x (e.g. Kombu 5.5.4+).

After locking, test the PubSub use cases to confirm the deadlock fix is in effect.

In components/job-orchestration/pyproject.toml around line 14, Celery's redis
extra is pulling redis-py 5.2.1 which is below the required 6.4.0; fix by adding
an explicit top-level dependency redis = ">=6.4.0" next to celery = {extras =
["redis"], version = "^5.5.3"} (or alternatively bump Celery to a version whose
extras allow redis-py 6.x), then run `poetry lock` to regenerate
components/job-orchestration/poetry.lock and verify the lockfile contains a
redis entry at 6.4.0+ and that kombu remains at a version compatible with
redis-py 6.x (adjust Celery/Kombu if needed), finally run your PubSub tests to
confirm the deadlock is resolved.