src/snakemake/dag.py (3)

2152-2157: jobs is None condition appears unreachable.

The condition jobs is None on line 2156 can never be True because this function is always called with jobs or self.jobs (lines 2178, 2206), which ensures the argument is never None. The comment mentions that "jobs (if not None) are finished jobs", but this distinction is lost due to the or pattern.

If the intent is to skip the is_output_present check when the caller passes specific finished jobs, consider passing a separate flag or restructuring the call pattern.

---

2215-2217: Repeated map() call inside loop is inefficient.

map(lambda job: job.rule, consumers_prior) creates a new iterator for each consumer in consumers_posterior. Consider precomputing a set before the loop.

+            rules_prior = {job.rule for job in consumers_prior}
             for consumer in consumers_posterior:
-                if consumer.rule not in map(lambda job: job.rule, consumers_prior):
+                if consumer.rule not in rules_prior:
                     maybe_has_incomplete_consumers = True

---

2190-2223: Consider adding a maximum iteration safeguard.

The while True loop relies on maybe_has_incomplete_consumers eventually becoming False to terminate. While this should converge in normal cases, complex checkpoint dependencies or edge cases could potentially cause excessive iterations.

Consider adding a maximum iteration limit with a warning, similar to other Snakemake safeguards:

MAX_CHECKPOINT_UPDATE_ROUNDS = 100  # or configurable
i = 1
while True:
    if i > MAX_CHECKPOINT_UPDATE_ROUNDS:
        logger.warning(f"Checkpoint update did not converge after {MAX_CHECKPOINT_UPDATE_ROUNDS} rounds")
        break
    ...

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Learnt from: kdm9
Repo: snakemake/snakemake PR: 3562
File: src/snakemake/checkpoints.py:90-90
Timestamp: 2025-05-06T01:37:23.382Z
Learning: In Snakemake checkpoints implementation, tracking only the first missing output for each checkpoint is sufficient, because if one output is missing, all outputs for that checkpoint are considered incomplete. This was the behavior before PR #3562 and maintained in the pluralized `checkpoints.get()` implementation.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3140
File: snakemake/dag.py:1308-1308
Timestamp: 2024-10-14T09:42:11.571Z
Learning: In `snakemake/dag.py`, avoid flagging missing lines or indentation issues when there is no clear syntax or logical error to prevent false positives.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3148
File: snakemake/dag.py:1332-1336
Timestamp: 2024-11-12T12:08:20.342Z
Learning: In `snakemake/dag.py`, when code is outdated and will disappear upon resolving merge conflicts, avoid making code review suggestions on that code.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3117
File: tests/test_wrapper/Snakefile:11-11
Timestamp: 2024-10-06T14:09:54.370Z
Learning: Changes made within test cases, such as in `tests/test_wrapper/Snakefile`, are for testing purposes and do not require updates to the project documentation.

src/snakemake/dag.py (1)

2225-2227: Consider optimizing repeated map creation.

The map() object is recreated for each consumer in the loop, which is inefficient. Consider creating a set of prior rules once before the loop.

Apply this diff to optimize:

+        prior_rules = {job.rule for job in consumers_prior}
         for consumer in consumers_posterior:
-            if consumer.rule not in map(lambda job: job.rule, consumers_prior):
+            if consumer.rule not in prior_rules:
                 maybe_has_incomplete_consumers = True

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Learnt from: kdm9
Repo: snakemake/snakemake PR: 3562
File: src/snakemake/checkpoints.py:90-90
Timestamp: 2025-05-06T01:37:23.382Z
Learning: In Snakemake checkpoints implementation, tracking only the first missing output for each checkpoint is sufficient, because if one output is missing, all outputs for that checkpoint are considered incomplete. This was the behavior before PR #3562 and maintained in the pluralized `checkpoints.get()` implementation.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3140
File: snakemake/dag.py:1308-1308
Timestamp: 2024-10-14T09:42:11.571Z
Learning: In `snakemake/dag.py`, avoid flagging missing lines or indentation issues when there is no clear syntax or logical error to prevent false positives.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3148
File: snakemake/dag.py:1332-1336
Timestamp: 2024-11-12T12:08:20.342Z
Learning: In `snakemake/dag.py`, when code is outdated and will disappear upon resolving merge conflicts, avoid making code review suggestions on that code.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3117
File: tests/test_wrapper/Snakefile:11-11
Timestamp: 2024-10-06T14:09:54.370Z
Learning: Changes made within test cases, such as in `tests/test_wrapper/Snakefile`, are for testing purposes and do not require updates to the project documentation.

Learnt from: lczech
Repo: snakemake/snakemake PR: 3113
File: snakemake/scheduler.py:912-914
Timestamp: 2024-10-04T16:12:18.927Z
Learning: In `snakemake/scheduler.py`, avoid suggesting the use of `asyncio.gather` in the `jobs_rewards` method due to overhead concerns and the need for immediate results.

src/snakemake/dag.py (1)

2228-2230: Inefficient membership test with map().

Using map() inside the in operator creates a new map object on each iteration and performs a linear search through it. This is inefficient, especially with many consumers.

Precompute the set of prior rules outside the loop:

+            prior_consumer_rules = {job.rule for job in consumers_prior}
             for consumer in consumers_posterior:
-                if consumer.rule not in map(lambda job: job.rule, consumers_prior):
+                if consumer.rule not in prior_consumer_rules:
                     maybe_has_incomplete_consumers = True

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Learnt from: kdm9
Repo: snakemake/snakemake PR: 3562
File: src/snakemake/checkpoints.py:90-90
Timestamp: 2025-05-06T01:37:23.382Z
Learning: In Snakemake checkpoints implementation, tracking only the first missing output for each checkpoint is sufficient, because if one output is missing, all outputs for that checkpoint are considered incomplete. This was the behavior before PR #3562 and maintained in the pluralized `checkpoints.get()` implementation.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3140
File: snakemake/dag.py:1308-1308
Timestamp: 2024-10-14T09:42:11.571Z
Learning: In `snakemake/dag.py`, avoid flagging missing lines or indentation issues when there is no clear syntax or logical error to prevent false positives.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3148
File: snakemake/dag.py:1332-1336
Timestamp: 2024-11-12T12:08:20.342Z
Learning: In `snakemake/dag.py`, when code is outdated and will disappear upon resolving merge conflicts, avoid making code review suggestions on that code.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3117
File: tests/test_wrapper/Snakefile:11-11
Timestamp: 2024-10-06T14:09:54.370Z
Learning: Changes made within test cases, such as in `tests/test_wrapper/Snakefile`, are for testing purposes and do not require updates to the project documentation.

Learnt from: lczech
Repo: snakemake/snakemake PR: 3113
File: snakemake/scheduler.py:912-914
Timestamp: 2024-10-04T16:12:18.927Z
Learning: In `snakemake/scheduler.py`, avoid suggesting the use of `asyncio.gather` in the `jobs_rewards` method due to overhead concerns and the need for immediate results.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3600
File: src/snakemake/jobs.py:960-964
Timestamp: 2025-05-23T09:40:24.474Z
Learning: In the `cleanup` method of the `Job` class in `src/snakemake/jobs.py`, files in the `to_remove` list should be formatted with `fmt_iofile` without specifying `as_output=True` or `as_input=True` parameters, as these files should be displayed as generic files rather than specifically as output files.

Learnt from: kdm9
Repo: snakemake/snakemake PR: 3562
File: src/snakemake/checkpoints.py:90-90
Timestamp: 2025-05-06T01:37:23.382Z
Learning: In Snakemake checkpoints implementation, tracking only the first missing output for each checkpoint is sufficient, because if one output is missing, all outputs for that checkpoint are considered incomplete. This was the behavior before PR #3562 and maintained in the pluralized `checkpoints.get()` implementation.

src/snakemake/dag.py (1)

2213-2214: Consider checking for both added and removed target inputs.

The current check only detects when checkpoint_target inputs are removed (resolved), but doesn't detect when new ones are added. While new consumer rules appearing (line 2230) might cover this indirectly, an explicit check would be more robust:

if target_inputs_prior != target_inputs_posterior:
    maybe_has_incomplete_consumers = True

This would catch both checkpoint resolution (removal) and new checkpoint dependencies (addition).

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Learnt from: kdm9
Repo: snakemake/snakemake PR: 3562
File: src/snakemake/checkpoints.py:90-90
Timestamp: 2025-05-06T01:37:23.382Z
Learning: In Snakemake checkpoints implementation, tracking only the first missing output for each checkpoint is sufficient, because if one output is missing, all outputs for that checkpoint are considered incomplete. This was the behavior before PR #3562 and maintained in the pluralized `checkpoints.get()` implementation.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3148
File: snakemake/dag.py:1332-1336
Timestamp: 2024-11-12T12:08:20.342Z
Learning: In `snakemake/dag.py`, when code is outdated and will disappear upon resolving merge conflicts, avoid making code review suggestions on that code.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3140
File: snakemake/dag.py:1308-1308
Timestamp: 2024-10-14T09:42:11.571Z
Learning: In `snakemake/dag.py`, avoid flagging missing lines or indentation issues when there is no clear syntax or logical error to prevent false positives.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3140
File: snakemake/dag.py:1308-1308
Timestamp: 2024-10-14T09:42:11.571Z
Learning: In `snakemake/dag.py`, avoid flagging missing lines or indentation issues when there is no clear syntax or logical error to prevent false positives.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3148
File: snakemake/dag.py:1332-1336
Timestamp: 2024-11-12T12:08:20.342Z
Learning: In `snakemake/dag.py`, when code is outdated and will disappear upon resolving merge conflicts, avoid making code review suggestions on that code.

Learnt from: lczech
Repo: snakemake/snakemake PR: 3113
File: snakemake/scheduler.py:912-914
Timestamp: 2024-10-04T16:12:18.927Z
Learning: In `snakemake/scheduler.py`, avoid suggesting the use of `asyncio.gather` in the `jobs_rewards` method due to overhead concerns and the need for immediate results.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3600
File: src/snakemake/jobs.py:960-964
Timestamp: 2025-05-23T09:40:24.474Z
Learning: In the `cleanup` method of the `Job` class in `src/snakemake/jobs.py`, files in the `to_remove` list should be formatted with `fmt_iofile` without specifying `as_output=True` or `as_input=True` parameters, as these files should be displayed as generic files rather than specifically as output files.

Learnt from: kdm9
Repo: snakemake/snakemake PR: 3562
File: src/snakemake/checkpoints.py:90-90
Timestamp: 2025-05-06T01:37:23.382Z
Learning: In Snakemake checkpoints implementation, tracking only the first missing output for each checkpoint is sufficient, because if one output is missing, all outputs for that checkpoint are considered incomplete. This was the behavior before PR #3562 and maintained in the pluralized `checkpoints.get()` implementation.

src/snakemake/dag.py (1)

2231-2234: Inefficient map object creation inside loop

Line 2232 creates a new map object on each iteration to extract rules from consumers_prior, which is inefficient.

♻️ Proposed refactor

+            prior_rules = set(job.rule for job in consumers_prior)
             for consumer in consumers_posterior:
-                if consumer.rule not in map(lambda job: job.rule, consumers_prior):
+                if consumer.rule not in prior_rules:
                     maybe_has_incomplete_consumers = True

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Learnt from: kdm9
Repo: snakemake/snakemake PR: 3562
File: src/snakemake/checkpoints.py:90-90
Timestamp: 2025-05-06T01:37:23.382Z
Learning: In Snakemake checkpoints implementation, tracking only the first missing output for each checkpoint is sufficient, because if one output is missing, all outputs for that checkpoint are considered incomplete. This was the behavior before PR #3562 and maintained in the pluralized `checkpoints.get()` implementation.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3148
File: snakemake/dag.py:1332-1336
Timestamp: 2024-11-12T12:08:20.342Z
Learning: In `snakemake/dag.py`, when code is outdated and will disappear upon resolving merge conflicts, avoid making code review suggestions on that code.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3140
File: snakemake/dag.py:1308-1308
Timestamp: 2024-10-14T09:42:11.571Z
Learning: In `snakemake/dag.py`, avoid flagging missing lines or indentation issues when there is no clear syntax or logical error to prevent false positives.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3140
File: snakemake/dag.py:1308-1308
Timestamp: 2024-10-14T09:42:11.571Z
Learning: In `snakemake/dag.py`, avoid flagging missing lines or indentation issues when there is no clear syntax or logical error to prevent false positives.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3148
File: snakemake/dag.py:1332-1336
Timestamp: 2024-11-12T12:08:20.342Z
Learning: In `snakemake/dag.py`, when code is outdated and will disappear upon resolving merge conflicts, avoid making code review suggestions on that code.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3117
File: tests/test_wrapper/Snakefile:11-11
Timestamp: 2024-10-06T14:09:54.370Z
Learning: Changes made within test cases, such as in `tests/test_wrapper/Snakefile`, are for testing purposes and do not require updates to the project documentation.

Learnt from: lczech
Repo: snakemake/snakemake PR: 3113
File: snakemake/scheduler.py:912-914
Timestamp: 2024-10-04T16:12:18.927Z
Learning: In `snakemake/scheduler.py`, avoid suggesting the use of `asyncio.gather` in the `jobs_rewards` method due to overhead concerns and the need for immediate results.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3600
File: src/snakemake/jobs.py:960-964
Timestamp: 2025-05-23T09:40:24.474Z
Learning: In the `cleanup` method of the `Job` class in `src/snakemake/jobs.py`, files in the `to_remove` list should be formatted with `fmt_iofile` without specifying `as_output=True` or `as_input=True` parameters, as these files should be displayed as generic files rather than specifically as output files.

Learnt from: kdm9
Repo: snakemake/snakemake PR: 3562
File: src/snakemake/checkpoints.py:90-90
Timestamp: 2025-05-06T01:37:23.382Z
Learning: In Snakemake checkpoints implementation, tracking only the first missing output for each checkpoint is sufficient, because if one output is missing, all outputs for that checkpoint are considered incomplete. This was the behavior before PR #3562 and maintained in the pluralized `checkpoints.get()` implementation.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3107
File: snakemake/logging.py:510-515
Timestamp: 2025-01-14T14:04:30.554Z
Learning: In Snakemake's logging handlers, KeyboardInterrupt and SystemExit exceptions should be ignored (pass) rather than re-raised, as these are control flow signals that should not disrupt the logging process. The exception variable in the final except block should be kept for potential debugging purposes.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3171
File: snakemake/cli.py:106-145
Timestamp: 2024-10-29T09:26:26.636Z
Learning: In the `snakemake` codebase, exception chaining (using `raise ... from ...`) is avoided due to backward compatibility reasons. Do not suggest using exception chaining in `snakemake` codebase.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3117
File: snakemake/deployment/conda.py:0-0
Timestamp: 2024-10-06T14:09:26.494Z
Learning: In `snakemake/deployment/conda.py`, the exception handling in the `_get_version` method is intentional. The method raises a `WorkflowError` when the version cannot be determined, and this behavior is desired. Do not suggest modifying this exception handling.

Learnt from: johanneskoester
Repo: snakemake/snakemake PR: 3643
File: src/snakemake/sourcecache.py:474-492
Timestamp: 2026-01-08T11:13:20.523Z
Learning: In Python files under src/snakemake, ensure that the per-run cache_path for source files does not change within a single workflow run, so that HostingProviderFile._hosted_repos can be keyed reliably by repository name. This invariant should be documented and enforced (e.g., by freezing cache_path after initialization, avoiding mutations during a run, and ensuring any caching uses a run-scoped key). This pattern applies to all modules in the Snakemake source cache area where per-run caching and repository hosting are implemented.

src/snakemake/dag.py (2)

2215-2216: ⚠️ Potential issue | 🟠 Major

Incomplete convergence detection: only checks removed inputs.

The condition target_inputs_prior.difference(target_inputs_posterior) > 0 only detects when checkpoint target inputs are removed. It does not detect when inputs are added, which could also indicate incomplete checkpoint resolution requiring another iteration.

🐛 Proposed fix

-                if len(target_inputs_prior.difference(target_inputs_posterior)) > 0:
+                if target_inputs_prior != target_inputs_posterior:
                     maybe_has_incomplete_consumers = True

This checks for any change (additions or removals) in checkpoint target inputs.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/snakemake/dag.py` around lines 2215 - 2216, The current convergence check
only flags removed inputs by using
target_inputs_prior.difference(target_inputs_posterior); change it to detect any
change between target_inputs_prior and target_inputs_posterior (additions or
removals) — e.g., use symmetric difference or compare the sets for inequality —
so the maybe_has_incomplete_consumers flag is set whenever the sets differ
(refer to variables target_inputs_prior, target_inputs_posterior and the
maybe_has_incomplete_consumers assignment in the same block).

---

2243-2247: ⚠️ Potential issue | 🟠 Major

Replace exit(1) with exception for proper error handling.

Using exit(1) terminates the entire Python process, which is problematic when Snakemake is used as a library or in contexts where graceful error handling is expected. This is inconsistent with Snakemake's error handling patterns elsewhere in the codebase.

🐛 Proposed fix

             if i > MAX_CHECKPOINT_UPDATE_ROUNDS:
-                logger.error(
-                    f"Checkpoint update did not converge after {MAX_CHECKPOINT_UPDATE_ROUNDS} rounds"
-                )
-                exit(1)
+                raise WorkflowError(
+                    f"Checkpoint dependency update did not converge after {MAX_CHECKPOINT_UPDATE_ROUNDS} rounds. "
+                    "This may indicate a cyclic checkpoint dependency or a bug in checkpoint resolution."
+                )

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/snakemake/dag.py` around lines 2243 - 2247, The code currently calls
exit(1) when the checkpoint update loop exceeds MAX_CHECKPOINT_UPDATE_ROUNDS
(the block in src/snakemake/dag.py checking if i >
MAX_CHECKPOINT_UPDATE_ROUNDS); replace exit(1) with raising an exception (e.g.,
raise RuntimeError(f"Checkpoint update did not converge after
{MAX_CHECKPOINT_UPDATE_ROUNDS} rounds")) so callers can catch and handle the
error instead of forcibly terminating the process. Ensure the raised exception
includes the same descriptive message and remove the exit(1) call.

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

src/snakemake/dag.py (1)

2207-2210: Consider adding strict=True to zip() for defensive coding.

Per static analysis hint, adding strict=True ensures the iteration fails fast if checkpoint_jobs and results ever have mismatched lengths due to a bug, rather than silently dropping elements.

♻️ Proposed fix

         results = await asyncio.gather(
             *(is_output_new_present(job) for job in checkpoint_jobs)
         )
-        return [job for job, done in zip(checkpoint_jobs, results) if done]
+        return [job for job, done in zip(checkpoint_jobs, results, strict=True) if done]

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/snakemake/dag.py` around lines 2207 - 2210, Update the comprehension that
pairs checkpoint_jobs with results to use zip(..., strict=True) so a length
mismatch raises immediately; specifically modify the expression in the return
statement that currently does return [job for job, done in zip(checkpoint_jobs,
results) if done] to use zip(checkpoint_jobs, results, strict=True), leaving the
awaited asyncio.gather call and the is_output_new_present references unchanged.

src/snakemake/dag.py (2)

2264-2266: ⚠️ Potential issue | 🟡 Minor

The non-convergence error underreports the retry count.

After for i in range(1, 101) exhausts, i is 100, so {i-1} reports 99 rounds instead of 100.

🐛 Suggested fix

         raise WorkflowError(
-            f"Checkpoint dependency update did not converge after {i-1} rounds. "
+            f"Checkpoint dependency update did not converge after {i} rounds. "
             "This may indicate a cycle in checkpoint dependencies."
         )

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/snakemake/dag.py` around lines 2264 - 2266, The error message in the
raise within src/snakemake/dag.py underreports the number of rounds because it
uses {i-1} after the for i in range(1, 101) loop; change the f-string in the
WorkflowError to report the actual attempted rounds (use i instead of i-1 or
compute a rounds variable before raising) so the message correctly states 100
rounds when the loop exhausts; update the raise in that block (the WorkflowError
construction) accordingly.

---

2233-2258: ⚠️ Potential issue | 🟠 Major

Requeue consumers when a newly discovered checkpoint is immediately evicted.

_evicted_checkpoint_outputs is folded into created_output only before the first affected_jobs set is built. If replace_job() introduces a checkpoint whose output already exists on disk, update() can evict that checkpoint in the same round and append its outputs to _evicted_checkpoint_outputs. At that point the consumer still carries a new checkpoint_target, but affected_jobs is rebuilt only from checkpoint jobs that remain in self.jobs, so the next iteration can exit without ever re-running that consumer against the newly known output.

🔁 Suggested direction

         for i in range(1, 101):
             logger.debug(f"Checkpoint dependency update round {i}")

+            next_affected_jobs = set()
             no_new_deps = True
             for affected_job in affected_jobs:
                 updated_job = await affected_job.updated()
-                if no_new_deps and checkpoint_target_inputs_updated(
-                    affected_job, updated_job
-                ):
+                if checkpoint_target_inputs_updated(affected_job, updated_job):
                     no_new_deps = False
+                    next_affected_jobs.add(updated_job)
                 await self.replace_job(affected_job, updated_job, recursive=False)

             await self.update_needrun()
+            checkpoints_created_output.update(self._evicted_checkpoint_outputs)
             completed_checkpoint_jobs = await get_completed_checkpoint_jobs(self.jobs)
             flag_checkpoints_as_completed(completed_checkpoint_jobs)
-            affected_jobs = get_checkpoint_affected_jobs(completed_checkpoint_jobs)
+            affected_jobs = next_affected_jobs | get_checkpoint_affected_jobs(
+                completed_checkpoint_jobs
+            )

You also need the same _evicted_checkpoint_outputs fold before the initial if not affected_jobs fast path.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/snakemake/dag.py` around lines 2233 - 2258, Fold any outputs evicted
during update into the created-output set both before the initial affected_jobs
fast-path and after each rescan so consumers that gain a new checkpoint_target
and are immediately evicted get re-evaluated; specifically, call
checkpoints_created_output.update(self._evicted_checkpoint_outputs) immediately
before computing affected_jobs (before the initial if not affected_jobs) and
again right after completed_checkpoint_jobs = await
get_completed_checkpoint_jobs(self.jobs) (inside the loop, before recomputing
affected_jobs), so replace_job/updated evictions are visible when building
get_checkpoint_affected_jobs.

src/snakemake/dag.py (1)

2206-2209: Consider adding strict=True to zip() for defensive coding.

While checkpoint_jobs and results are guaranteed to have matching lengths by construction (since results comes from asyncio.gather over the same checkpoint_jobs), adding strict=True provides an explicit safety net against future refactoring that might break this invariant.

♻️ Suggested change

             results = await asyncio.gather(
                 *(is_output_new_present(job) for job in checkpoint_jobs)
             )
-            return [job for job, done in zip(checkpoint_jobs, results) if done]
+            return [job for job, done in zip(checkpoint_jobs, results, strict=True) if done]

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/snakemake/dag.py` around lines 2206 - 2209, The zip between
checkpoint_jobs and results should be made defensive: change the comprehension
that currently does return [job for job, done in zip(checkpoint_jobs, results)
if done] to use zip(checkpoint_jobs, results, strict=True) to ensure mismatched
lengths raise immediately; update the call site where asyncio.gather(...)
produces results (and the related is_output_new_present usage) so the returned
pairwise filtering uses zip(..., strict=True).

src/snakemake/dag.py (1)

2234-2243: ⚠️ Potential issue | 🔴 Critical

Evicted checkpoint outputs still don't schedule a second pass.

When a checkpoint was evicted earlier (Lines 1361-1373), its job and dependency edges are already gone. Folding _evicted_checkpoint_outputs into created_output here therefore does not produce any affected_jobs, so this path can hit Line 2242 and return with jobs that still carry checkpoint_target inputs unresolved. The same blind spot exists for evicted outputs discovered inside later rounds, because they are only folded once before the loop. Seed reevaluation from jobs with checkpoint_target inputs whenever evicted outputs are merged, and repeat that merge inside each round.

🔧 Minimal shape of the missing seed

+        def get_jobs_with_checkpoint_targets():
+            return {
+                job
+                for job in self.jobs
+                if any(is_flagged(f, "checkpoint_target") for f in job.input)
+            }
+
+        def fold_evicted_checkpoint_outputs():
+            evicted, self._evicted_checkpoint_outputs = (
+                self._evicted_checkpoint_outputs,
+                set(),
+            )
+            checkpoints_created.update(evicted)
+            return evicted
+
-        evicted, self._evicted_checkpoint_outputs = self._evicted_checkpoint_outputs, set()
-        checkpoints_created.update(evicted)
-        affected_jobs = get_checkpoint_affected_jobs(completed_checkpoint_jobs)
+        evicted = fold_evicted_checkpoint_outputs()
+        affected_jobs = get_checkpoint_affected_jobs(completed_checkpoint_jobs)
+        if evicted:
+            affected_jobs |= get_jobs_with_checkpoint_targets()
...
             await self.update_needrun()
+            evicted = fold_evicted_checkpoint_outputs()
             completed_checkpoint_jobs = await get_completed_checkpoint_jobs(self.jobs)
             update_checkpoints_created(completed_checkpoint_jobs)
             affected_jobs = get_checkpoint_affected_jobs(completed_checkpoint_jobs)
+            if evicted:
+                affected_jobs |= get_jobs_with_checkpoint_targets()

Also applies to: 2258-2262

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/snakemake/dag.py` around lines 2234 - 2243, The merge of
_evicted_checkpoint_outputs into checkpoints_created happens only once, so if
evicted outputs remove jobs earlier their dependent jobs with checkpoint_target
inputs never get re-seeded; move the swap-and-merge logic so it runs whenever
evicted outputs are discovered (i.e., perform evicted,
self._evicted_checkpoint_outputs = self._evicted_checkpoint_outputs, set() and
checkpoints_created.update(evicted) inside each DAG traversal round), and after
each such merge call get_checkpoint_affected_jobs(completed_checkpoint_jobs or
completed_checkpoint_jobs + evicted) to compute affected_jobs and enqueue them
instead of returning early; ensure code paths that currently return False (when
not affected) do not exit the round prematurely but continue until no new
evicted outputs are produced.

src/snakemake/dag.py (1)

132-138: Use a mutable type for the DAG adjacency tables.

These fields are deleted and updated throughout DAG (30+ mutation sites), so annotating them as Mapping[...] makes the types lie about the implementation and weakens static checking. Use MutableMapping[...] or the concrete defaultdict[...] shape on the private attributes instead.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/snakemake/dag.py` around lines 132 - 138, The type annotations for the
adjacency tables _dependencies and depending are wrong: they are mutated
extensively but are declared as immutable Mapping[...] which weakens static
checking; update their annotations to a mutable type (e.g., MutableMapping[Job,
MutableMapping[Job, Set[str]]] or the concrete defaultdict shape) on the private
attributes in class DAG so the type reflects mutation sites, and keep the
runtime construction (defaultdict(partial(defaultdict, set))) unchanged; change
only the type hints for _dependencies and depending to MutableMapping (or the
explicit defaultdict nested mapping) to satisfy the type checker and callers.