Skip to content

Fix console logging with very long print out when using wandb-service#3574

Merged
raubitsj merged 5 commits intomasterfrom
WB-9397
Apr 28, 2022
Merged

Fix console logging with very long print out when using wandb-service#3574
raubitsj merged 5 commits intomasterfrom
WB-9397

Conversation

@kptkin
Copy link
Copy Markdown
Collaborator

@kptkin kptkin commented Apr 26, 2022

Fixes WB-9397

Description

What does the PR do?

When using service and console logging with large input the data is not sent properly and get corrupted as a result the execution fails and the program hangs. If we replace send with sendall we are guaranteed either all data is sent or we get None.

Testing

How was this PR tested?

Resolved test: https://github.com/wandb/client/blob/master/tests/test_redir.py#L287. We can also add a dedicated functional test with service, but it seems to be already covered by the linked test.

@kptkin kptkin changed the title Wb 9397 fix data corruption due to console logging of a very long input Apr 26, 2022
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 26, 2022

Codecov Report

Merging #3574 (1b57ab2) into master (4f21ae3) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3574      +/-   ##
==========================================
+ Coverage   81.54%   81.56%   +0.02%     
==========================================
  Files         240      240              
  Lines       29508    29502       -6     
==========================================
+ Hits        24061    24064       +3     
+ Misses       5447     5438       -9     
Flag Coverage Δ
functest 57.61% <100.00%> (+0.03%) ⬆️
unittest 71.79% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
wandb/sdk/service/streams.py 94.22% <ø> (-0.03%) ⬇️
wandb/sdk/lib/redirect.py 87.13% <100.00%> (+0.42%) ⬆️
wandb/sdk/lib/sock_client.py 92.50% <100.00%> (ø)
wandb/sdk/lib/git.py 76.19% <0.00%> (ø)
wandb/filesync/step_prepare.py 94.80% <0.00%> (+1.29%) ⬆️
wandb/sdk/internal/meta.py 90.68% <0.00%> (+3.10%) ⬆️

@kptkin kptkin changed the title fix data corruption due to console logging of a very long input fix(sdk) support for console logging with very long print out Apr 26, 2022
@kptkin kptkin changed the title fix(sdk) support for console logging with very long print out fix(service) support for console logging with very long print out Apr 26, 2022
@kptkin kptkin changed the title fix(service) support for console logging with very long print out fix(sdk-service) support for console logging with very long print out Apr 26, 2022
@kptkin kptkin requested a review from raubitsj April 26, 2022 04:13
@kptkin kptkin added this to the sdk-2022-05.2 milestone Apr 26, 2022
@kptkin kptkin modified the milestones: sdk-2022-05.2, sdk-2022-05.1 Apr 26, 2022
@raubitsj raubitsj self-assigned this Apr 26, 2022
@kptkin kptkin changed the title fix(sdk-service) support for console logging with very long print out [WB-9397] support for console logging with very long print out Apr 26, 2022
@kptkin kptkin changed the title [WB-9397] support for console logging with very long print out [WB-9397] fixes data corruption when console logging with very long print out Apr 26, 2022
@raubitsj raubitsj merged commit b4b660a into master Apr 28, 2022
@raubitsj raubitsj deleted the WB-9397 branch April 28, 2022 21:45
@raubitsj raubitsj changed the title [WB-9397] fixes data corruption when console logging with very long print out Fix console logging with very long print out when using wandb-service May 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants