Skip to content

Add labels and fix argo#1360

Merged
savingoyal merged 11 commits intoNetflix:masterfrom
dhpollack:add-labels-and-fix-argo
May 23, 2023
Merged

Add labels and fix argo#1360
savingoyal merged 11 commits intoNetflix:masterfrom
dhpollack:add-labels-and-fix-argo

Conversation

@dhpollack
Copy link
Contributor

@dhpollack dhpollack commented Apr 18, 2023

Revises: #1236

Fixes error related to null labels trying to get added to a dictionary in argo workflows.

Refactored based on comments from @saikonen

Now one can only add labels via the environmental variable METAFLOW_KUBERNETES_LABELS with the format label=val,label2=val2. Json is no longer supported. And the decorator does not allow labels either.

@dhpollack
Copy link
Contributor Author

fyi, I test this with a vanilla argo-workflows create command and everything was fine.

Copy link
Collaborator

@saikonen saikonen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an additional note, the newly added Sensors for argo-workflows from #1271 would also require labeling. If you have time, this could be done in the scope of this PR.

@dhpollack
Copy link
Contributor Author

As an additional note, the newly added Sensors for argo-workflows from #1271 would also require labeling. If you have time, this could be done in the scope of this PR.

I personally would like to get this merged as quickly as possible. I can look into this but I haven't used Sensors so I would also need to look into that. If it's easy, I can do it, but otherwise I would like to create a new PR.

@saikonen
Copy link
Collaborator

We can do another PR for the sensors, they are just another template that is being compiled as part of argo-workflows create so should be straightforward to add the labels support for those. I might have time today to look into it.

Could you resolve the conflicts with this branch so it is ready for merging?

saikonen
saikonen previously approved these changes Apr 26, 2023
@dhpollack
Copy link
Contributor Author

We can do another PR for the sensors, they are just another template that is being compiled as part of argo-workflows create so should be straightforward to add the labels support for those. I might have time today to look into it.

Could you resolve the conflicts with this branch so it is ready for merging?

Conflicts resolved. Thanks for your help.

@dhpollack
Copy link
Contributor Author

Tested the latest with:

METAFLOW_KUBERNETES_LABEL="" python hello.py run --with kubernetes
METAFLOW_KUBERNETES_LABEL="label1=val1" python hello.py run --with kubernetes
METAFLOW_KUBERNETES_LABEL="label1=val1,label2=val2" python hello.py run --with kubernetes
METAFLOW_KUBERNETES_LABEL="label=creation" python hello.py argo-workflows create
METAFLOW_KUBERNETES_LABEL="label=trigger" python hello.py argo-workflows trigger  # label has no effect here, the creation label is used
METAFLOW_KUBERNETES_LABEL="" python hello.py argo-workflows create
python hello.py run --with kubernetes:labels=l1=v1  # this also fails as expected

@dhpollack
Copy link
Contributor Author

Sensor Labels: dhpollack#1

@savingoyal savingoyal merged commit 74a8790 into Netflix:master May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants