fix(dsl): use workflow-safe logging to avoid temporal deadlocks#2170
fix(dsl): use workflow-safe logging to avoid temporal deadlocks#2170daryllimyt merged 4 commits intomainfrom
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc71868bc1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
1 issue found across 9 files
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tracecat/registry/sync/tarball.py">
<violation number="1" location="tracecat/registry/sync/tarball.py:222">
P2: The new uncompressed_size calculation performs synchronous filesystem traversal/stat calls inside an async function, which can block the event loop for large site-packages trees. Offload this calculation to a thread, similar to the tarball creation and hashing steps.
(Based on your team's feedback about offloading heavy sync file I/O in async functions.) [FEEDBACK_USED]</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
279d6ce to
197a1cc
Compare
|
@cubic rereview |
|
@codex rereview |
@daryllimyt I have started the AI code review. It will take a few minutes to complete. |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
1 issue found across 5 files
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tracecat/dsl/workflow_logging.py">
<violation number="1" location="tracecat/dsl/workflow_logging.py:74">
P2: Process logging loses structured fields by flattening them into the message string. Loguru supports structured fields via bind/kwargs, so this adapter should pass fields to the process logger instead of discarding them.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Checklist
uv run pytest tests)?pre-commit run --all-files)?Description
This PR fixes
TMPRL1101workflow deadlock risk caused by using process Loguru logger inside Temporal workflow threads.Changes:
WorkflowRuntimeLoggeradapter intracecat/dsl/workflow_logging.py.info->debug) in hot paths.scripts/temporal/repro_workflow_logging_deadlock.py.ctx_logger/process_loggerwiring fromDSLWorkflow.Related Issues
N/A
Screenshots / Recordings
N/A
Steps to QA
uv run ruff check tracecat/dsl/workflow.py tracecat/dsl/scheduler.py tracecat/dsl/workflow_logging.py tests/unit/test_dsl_workflow_logging.py scripts/temporal/repro_workflow_logging_deadlock.pyuv run basedpyright tracecat/dsl/workflow.py tracecat/dsl/scheduler.py tracecat/dsl/workflow_logging.py tests/unit/test_dsl_workflow_logging.py scripts/temporal/repro_workflow_logging_deadlock.pyTRACECAT__SERVICE_KEY=dummy uv run pytest tests/unit/test_dsl_workflow_logging.pyTRACECAT__SERVICE_KEY=dummy uv run pytest tests/unit/test_schedule_role_healing.pyTRACECAT__SERVICE_KEY=dummy uv run pytest tests/unit/test_dsl_scheduler_determinism.pyuv run python scripts/temporal/repro_workflow_logging_deadlock.py --mode both --block-seconds 2.5TMPRL1101in workflow task history.TMPRL1101.Summary by cubic
Fixes Temporal workflow deadlocks (TMPRL1101) by switching DSL logs to a workflow‑safe logger, deferring formatting with lazy callables, and gating formatting behind level checks to prevent blocking in workflow threads. Adds tests and a hardened local repro script to reliably reproduce and validate the fix.
Bug Fixes
Refactors
Written for commit b04e23d. Summary will update on new commits.