Skip to content

Fix _graph_to_futures bug for futures-based dependencies#4178

Merged
jrbourbeau merged 4 commits intodask:masterfrom
rjzamora:fix-future-deps
Oct 23, 2020
Merged

Fix _graph_to_futures bug for futures-based dependencies#4178
jrbourbeau merged 4 commits intodask:masterfrom
rjzamora:fix-future-deps

Conversation

@rjzamora
Copy link
Member

@rjzamora rjzamora commented Oct 22, 2020

closes #4177
depends on dask#6761

After an offline discussion with @madsbk and @jrbourbeau , I decided to revise this PR into a "standalone" fix (that does not depend on changes in dask/dask). The new fix will address the problem, but will also materialize the underlying graph in cases with futures. Therefore, this should be viewed as a temporary change, that will (hopefully) be replaced with ongoing HLG work relatively soon.

@rjzamora rjzamora marked this pull request as ready for review October 22, 2020 18:52
@rjzamora rjzamora changed the title [WIP] Fix _graph_to_futures bug for futures-based dependencies Fix _graph_to_futures bug for futures-based dependencies Oct 22, 2020
@rjzamora
Copy link
Member Author

Anyone with distributed knowledge know if the windows CI failure is likely to be related to these changes? (maybe @quasiben)

@quasiben
Copy link
Member

I think the failure is unrelated but I kicked the CI hoping that a restart comes back green

Copy link
Contributor

@madsbk madsbk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, simple fix and the regression test is very helpful when developing the permanent solution.

@madsbk
Copy link
Contributor

madsbk commented Oct 23, 2020

@rjzamora before merging, let's make sure that futures in tuples is even allowed: #4183

@rjzamora
Copy link
Member Author

Judging by @martindurant 's comment, it looks like we should merge this temporary fix, and then try to modify tuple usage in streamz before adding any follow-up changes that may (or may not) dissallow their current usage.

Copy link
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @rjzamora! I'm going to merge this as it restores our previous behavior. We can continue discussing if we want to keep inspecting non-task tuples for Futures over in #4183

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Failing to find dependencies in tuple of futures

4 participants