Skip to content

[WB-6946] Improve handling of Run objects when service is not enabled#3362

Merged
kptkin merged 20 commits intomasterfrom
WB-6946-1
Mar 25, 2022
Merged

[WB-6946] Improve handling of Run objects when service is not enabled#3362
kptkin merged 20 commits intomasterfrom
WB-6946-1

Conversation

@kptkin
Copy link
Copy Markdown
Collaborator

@kptkin kptkin commented Mar 10, 2022

Fixes WB-6946
Fixes #NNNN

Description

What does the PR do?

Handle the non-service case: if the user tries to share a run we make all the log and finish a no-op

Testing

How was this PR tested?
Added new functional test + testing user warnings/errors in a unit-test

Checklist

  • Name PR "[WB-NNNN][WB-MMMM] Add support for..." similar to entries in CHANGELOG.md
  • Include reference to internal ticket "Fixes WB-NNNN" (and github issue "Fixes #NNNN" if applicable)

@kptkin kptkin changed the title init version to deal with attach in non-service case [WB-6946] attach follow up for the non-service case Mar 10, 2022
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 10, 2022

Codecov Report

Merging #3362 (9509373) into master (c6c950d) will increase coverage by 0.05%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3362      +/-   ##
==========================================
+ Coverage   81.51%   81.56%   +0.05%     
==========================================
  Files         234      234              
  Lines       28744    28760      +16     
==========================================
+ Hits        23431    23459      +28     
+ Misses       5313     5301      -12     
Flag Coverage Δ
functest 58.66% <97.01%> (+0.07%) ⬆️
unittest 71.71% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
wandb/sdk/lib/_wburls_generated.py 0.00% <ø> (ø)
wandb/sdk/lib/wburls.py 100.00% <ø> (ø)
wandb/errors/__init__.py 92.10% <100.00%> (+0.43%) ⬆️
wandb/sdk/wandb_run.py 90.01% <100.00%> (+0.15%) ⬆️
wandb/filesync/step_prepare.py 92.85% <0.00%> (-1.43%) ⬇️
wandb/sdk/lib/git.py 76.35% <0.00%> (ø)
wandb/__init__.py 92.30% <0.00%> (+0.96%) ⬆️
wandb/sdk/lib/sock_client.py 94.73% <0.00%> (+1.75%) ⬆️
wandb/sdk/internal/meta.py 90.74% <0.00%> (+3.08%) ⬆️
... and 1 more

@kptkin kptkin requested a review from raubitsj March 17, 2022 22:46
@kptkin kptkin marked this pull request as ready for review March 17, 2022 23:56
@kptkin kptkin requested a review from dmitryduev March 17, 2022 23:56
Copy link
Copy Markdown
Member

@dmitryduev dmitryduev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small suggestion, otherwise LGTM.
Also, we should either uncomment all those noop decorators or rm the comments :)

Comment thread tests/wandb_run_test.py
Comment on lines +54 to +59
_init_pid = run._init_pid
run._init_pid = _init_pid + 1
with pytest.raises(MultiprocessError) as excinfo:
run.log(dict(this=1))
assert "log() does not support multiprocessing" in str(excinfo.value)
assert "`log` does not support multiprocessing" in str(excinfo.value)
run._init_pid = _init_pid
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice trick!

Comment thread wandb/sdk/wandb_run.py
Comment on lines +191 to +192
class Dummy:
...
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a hacky hack, like it!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why hacky hack? 😂
how would you do it?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just don't think I've seen such tricks before, just an expression of appreciation :)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh i see... got ya... so thanks 😊

Comment thread wandb/sdk/wandb_run.py Outdated
return False

@Attach._attach
# @_run_decorator._noop
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried uncommenting these, does it break anything?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah... it works!
@raubitsj prefers not to add it to all the functions, since he is worried that it will break users code that worked although it shouldn't have (log already had this logic before)

Comment thread wandb/sdk/wandb_run.py
if settings and settings["strict"]:
wandb.termerror(message, repeat=False)
raise errors.MultiprocessError(
f"`{func.__name__}` does not support multiprocessing"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's ping the user to use service here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good... I can add a link to our doc

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait... i have the link in the message wburls.get("multiprocess")
I just need to add a section about service in there

@kptkin
Copy link
Copy Markdown
Collaborator Author

kptkin commented Mar 25, 2022

One small suggestion, otherwise LGTM. Also, we should either uncomment all those noop decorators or rm the comments :)

Yeah, I can remove it and just add it if we want to support other functions... (i was just keeping it so it would be easier for me to change later... but it is pure laziness ...)

@kptkin kptkin added this to the sdk-2022-04.1 milestone Mar 25, 2022
@kptkin kptkin merged commit a71977e into master Mar 25, 2022
@kptkin kptkin deleted the WB-6946-1 branch March 25, 2022 19:35
@raubitsj raubitsj changed the title [WB-6946] attach follow up for the non-service case [WB-6946] Improve handling of Run objects when service is not enabled Apr 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants