Skip to content

@pattern Decorator#115

Merged
fritz-astronomer merged 3 commits intomainfrom
patterns
Dec 9, 2025
Merged

@pattern Decorator#115
fritz-astronomer merged 3 commits intomainfrom
patterns

Conversation

@fritz-astronomer
Copy link
Contributor

@fritz-astronomer fritz-astronomer commented Nov 21, 2025 •
edited
Loading

Why

I have been having conversations with folks along the lines of "rule composability".

I am considering the idea that things like task_common_args could be a "Pattern", which I think fits with how creating translations is discussed.

  1. Translations have rulesets,
  2. rulesets have rules,
  3. rules have patterns.

Ultimately the complexity of a translation is in the "number of patterns" that need to be covered.

WDYT?

Caveats

This addition doesn't enforce any structure of the pattern, nor that an @rule uses them
(you can have a gigantic rule that does a million things - they are supposed to do one thing, but not required to)

This just gives an opportunity to document inputs/outputs and encourages composability and reuse

Example

Before / After

Is fairly similar - as this addition is just an indication / hint towards structure.

from orbiter.objects.operators.bash import OrbiterBashOperator
+ from orbiter.rules import task_rule, pattern

+ @pattern(params_doc={"command": "Operator.bash_command"})
- command_pattern_params_doc={"command": "Operator.bash_command"}
def command_pattern(val: dict) -> dict:
    if 'command' in val:
        return {"bash_command": val['command']}
    else:
        return {}

- @task_rule(params_doc={"id": "Operator.task_id"} | command_pattern_params_doc)  # params_doc composition
+ @task_rule(params_doc={"id": "Operator.task_id"} | command_pattern.params_doc)  # params_doc composition
def bash_rule(val: dict) -> OrbiterBashOperator | None:
    return OrbiterBashOperator(
        task_id=val['id'],
        **command_pattern(val),                              # pattern output can be added, or not
    ) if 'id' in val and 'command' in val else None          # rules still produce one something or nothing
>>> bash_rule(val={"id": "foo", "command": "echo 'hello world'"})
foo_task = BashOperator(task_id='foo', bash_command="echo 'hello world'")

@fritz-astronomer fritz-astronomer self-assigned this Nov 21, 2025
Copilot AI reviewed Nov 21, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a @pattern decorator to support rule composability by allowing reusable transformation patterns to be defined and documented. The decorator enables functions to be wrapped with metadata about their parameters, encouraging modular design in rule definitions.

Key Changes:

  • Added @pattern decorator and Pattern class for composable rule components
  • Implemented documentation support via params_doc parameter
  • Provided example usage showing pattern integration with rules

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codecov-commenter
Copy link

codecov-commenter commented Nov 21, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.74%. Comparing base (b3ee237) to head (ea1c7b1).
⚠️ Report is 74 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #115      +/-   ##
==========================================
+ Coverage   78.92%   82.74%   +3.82%     
==========================================
  Files          35       51      +16     
  Lines        1362     1971     +609     
==========================================
+ Hits         1075     1631     +556     
- Misses        287      340      +53

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@fritz-astronomer
Apply suggestion from @Copilot
cc46f7b 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: fritz-astronomer <80706212+fritz-astronomer@users.noreply.github.com>
@fritz-astronomer
Copy link
Contributor Author

fritz-astronomer commented Dec 2, 2025 •
edited
Loading

@pgagnon @odaneau-astro @aoelvp-astro
what do yall think? Helpful? Unnecessary?

It'd be an opt-in addition

@fritz-astronomer fritz-astronomer marked this pull request as ready for review December 9, 2025 18:36
@fritz-astronomer fritz-astronomer merged commit d556f54 into main Dec 9, 2025
8 checks passed
@fritz-astronomer fritz-astronomer deleted the patterns branch December 9, 2025 21:50
@pgagnon
Copy link
Member

pgagnon commented Dec 10, 2025

Was any other syntax other than @task_rule(params_doc={"id": "Operator.task_id"} | command_pattern.params_doc) # params_doc composition (i.e. decorator-based) considered? I'm worried this could become unwieldy if we're composing a lot of patterns together.

@fritz-astronomer
Copy link
Contributor Author

fritz-astronomer commented Dec 10, 2025

@pgagnon here's a very recent example of it getting towards 'unwieldy'.
https://github.com/astronomer/orbiter-translations/blob/main/orbiter_translations/autosys/jil_base.py#L703-L722

In my recent ruleset development seems like a @rule has between 1->10 patterns, though each pattern may have a few patterns of it's own and maybe up to a few levels of that. Because the params docs compose, it doesn't feel too bad, to me.

I don't think I mind it, as is, but don't have any particular ideas for other implementations. Am certainly open to suggestions, or a point towards a similar coding pattern that can be emulated or used as inspiration.

My only other thoughts were maybe around things like structures in docstrings, or some introspection-majick to automatically find and compose things together.

I do think the problem is worth solving: ending up with a solid and structured summary of which input produced which output - which can be fairly dynamic 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@pgagnon pgagnon Awaiting requested review from pgagnon

@odaneau-astro odaneau-astro Awaiting requested review from odaneau-astro

@aoelvp-astro aoelvp-astro Awaiting requested review from aoelvp-astro

Assignees

@fritz-astronomer fritz-astronomer

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@fritz-astronomer @codecov-commenter @pgagnon