Files that changed from the base of the PR and between 756dc70 and 972b1ca.

• snakemake/jobs.py (3 hunks)
• snakemake/persistence.py (4 hunks)
• snakemake/rules.py (9 hunks)

snakemake/persistence.py (1)

587-587: ⚠️ Potential issue

Avoid using functools.lru_cache on instance methods to prevent memory leaks.

Applying @lru_cache to instance methods can lead to memory leaks because the self instance is part of the cache key, and the cache can grow indefinitely with different instances. Consider refactoring _params to avoid using @lru_cache on an instance method.

Apply this diff to remove the decorator:

-    @lru_cache()
     def _params(self, job: Job):

If caching is necessary, consider using an instance-level caching mechanism or limiting the cache size.

Likely invalid or redundant comment.

snakemake/rules.py (12)

699-699: Correct Initialization of non_derived_items Parameter

The optional parameter non_derived_items: typing.List[typing.Any] = None is properly initialized to None, which is a good practice to avoid mutable default arguments.

---

709-709: Initialization of is_derived Variable

The variable is_derived is appropriately initialized to False, ensuring that it has a defined state before usage.

---

722-723: Conditional Check for Deriving Functions

The code correctly checks if non_derived_items is not None before determining if the item is a deriving function:

if non_derived_items is not None:
    is_derived = self._is_deriving_function(item)

---

746-751: Proper Use of zip for Unpacking

The use of zip to construct apply_results ensures that each list is correctly aligned, facilitating the unpacking process in subsequent iterations:

apply_results = zip(
    [None] * len(item),
    item,
    [_is_callable] * len(item),
    [is_derived] * len(item),
)

---

754-757: Building apply_results from Dictionary Items

When item is a dictionary, the comprehension accurately constructs apply_results by pairing keys and values along with the callable and derived status:

apply_results = [
    (name, item, _is_callable, is_derived)
    for name, item in item.items()
]

---

759-759: Handling Single Items in apply_results

For single items, apply_results is correctly formed as a list containing a single tuple:

apply_results = [(name, item, _is_callable, is_derived)]

---

761-761: Correct Unpacking in Loop

The for-loop accurately unpacks name, item, from_callable, and is_derived from apply_results, which is essential for processing each element:

for name, item, from_callable, is_derived in apply_results:

---

785-786: Appending Non-Derived Items

Non-derived items are properly appended to non_derived_items when they are identified as such:

if not is_derived and non_derived_items is not None:
    non_derived_items.append(concrete)

---

848-859: Addition of _is_deriving_function Class Method

The new class method _is_deriving_function effectively identifies deriving functions based on the presence of specific parameters, enhancing the classification of function types:

@classmethod
def _is_deriving_function(cls, func):
    if is_callable(func):
        func_params = get_function_params(func)
        return (
            "input" in func_params
            or "output" in func_params
            or "threads" in func_params
            or "resources" in func_params
        )
    return False

---

891-891: Initialization of non_derived_params List

The list non_derived_params is correctly initialized to collect non-derived parameters during parameter expansion:

non_derived_params = []

---

912-912: Passing non_derived_items to _apply_wildcards

The inclusion of non_derived_items=non_derived_params in the _apply_wildcards method call allows for tracking of non-derived parameters:

non_derived_items=non_derived_params,

---

925-925: Returning Non-Derived Parameters

The expand_params method now returns a tuple containing both params and non_derived_params, which enhances the method's functionality:

return params, non_derived_params

snakemake/jobs.py (4)

225-225: Addition of _non_derived_params to __slots__ and initialization

The attribute _non_derived_params is appropriately added to __slots__ and initialized to None in the constructor. This maintains consistency and optimizes memory usage in classes using __slots__.

Also applies to: 279-279

---

424-424: Lazy initialization of params using _expand_params

The modification to the params property to lazily initialize _params by calling _expand_params() enhances code maintainability and ensures that parameters are expanded only when needed.

---

427-432: Introduction of non_derived_params property for better encapsulation

The new non_derived_params property provides access to _non_derived_params with lazy initialization, following the same pattern as the params property. This improves code clarity and encapsulation.

---

433-437: ⚠️ Potential issue

Ensure Rule.expand_params returns both params and non_derived_params

The _expand_params method expects self.rule.expand_params() to return a tuple of (params, non_derived_params). Please verify that the expand_params method in the Rule class has been updated accordingly to return both values to prevent a TypeError.

To verify, check the implementation of expand_params in the Rule class to ensure it returns a tuple containing both params and non_derived_params.