feat(core/pipeline): allow dependency across source/conditions/targets#2800
feat(core/pipeline): allow dependency across source/conditions/targets#2800
Conversation
b9bc2a4 to
16f992c
Compare
16f992c to
b7ca498
Compare
Signed-off-by: Olblak <me@olblak.com>
Signed-off-by: Olblak <me@olblak.com>
Signed-off-by: Olblak <me@olblak.com>
|
Thanks for the pullrequest, this one is really awesome but I need a bit more time to review it properly, I am adding commits as I am reviewing it |
|
I am having many warning messages like
|
|
@olblak hiw did you get those warning? Could you share a pipeline or something so that I can investigate? |
|
Note for myself as it don't have to be done in this pullrequest but I am wondering if you can generated the graphviz graphs directly using https://github.com/goccy/go-graphviz |
The problem is well explained on https://stackoverflow.com/questions/76648255/how-to-range-over-a-slice-of-structs-that-contain-mutexes-in-go |
Signed-off-by: Olblak <me@olblak.com>
|
I think this commit fixes the mutex issue |
Signed-off-by: Olblak <me@olblak.com>
Signed-off-by: Olblak <me@olblak.com>
Signed-off-by: Olblak <me@olblak.com>
Signed-off-by: Olblak <me@olblak.com>
d98725b to
e3e48ee
Compare
That's a good point. I think having a graph functionality could also be useful on generating some summary, with what ran and what didn't. But then maybe dot is not the ideal choice and we should use mermaid-js so that it's easy to but in github? But this should be new issues / new pr. Here the goal is really to help debug and visualize dependencies |
…nto lp/cross-type-deps
I think I would move the graph option behind the experimental flag so we can change it later. |
Signed-off-by: Olblak <me@olblak.com>
Signed-off-by: Olblak <me@olblak.com>
|
I just fixed the |
Signed-off-by: Olblak <me@olblak.com>

Overview
This pull request introduces support for dependencies across sources, conditions, and targets, making the dependency model more flexible.
Current Behavior:
New Behavior:
With this change, sources, conditions, and targets can depend on any other resource type. This is a stepping stone towards resolving issues like issue #2718 and issue #159.
Implementation
To achieve this, I've extended the current dependsOn specification. It can now be defined using the format:
This change allows merging the three resource-specific dependency Directed Acyclic Graphs (DAGs) into a single unified DAG. The DAG is built using a custom node type:
To ensure backward compatibility, the existing dependencies (i.e., DependsOn) are preserved. In addition, the sourceID is passed as a dependency for conditions and targets, and conditionIDs are fed into the targets as dependencies in the DAG.
A traversal of the DAG using
DescendantsFlowprocesses each resource in the correct order once its dependencies are completed.Expected Impact
This PR is expected to be a no-op, meaning that while the core pipeline logic has been updated, there should be no impact on the output or behavior of existing pipelines.
updatecli manifest show -graphIn order to help debugging the dependencies tree,
I' ve added an option to the
showcommand to display the DAG in graphviz format, this allows to get interesting information on a pipeline: