Make Pipinstall plugin default name unique#5127
Make Pipinstall plugin default name unique#5127jrbourbeau wants to merge 5 commits intodask:mainfrom
Pipinstall plugin default name unique#5127Conversation
|
@jrbourbeau sorry I messed up the merge commit. I pushed another one to fix it |
|
I sent in a comment by e-mail yesterday and just got a bounce message. That comment is below: My experience with this is that often I realize that I wanted to upload a different set of packages (maybe adding one more or changing a version, for example). What I really want in these cases is to replace the previous plugin, not add another one. In this situation I think that we want to keep the uniform name, but use the semantics in the NannyPlugin around repeats. Otherwise I could imagine being surprised that I have ten different pip installs happening each time :) |
I think this difference between how worker and nanny plugins behave will end up causing confusion. I'm curious why worker plugins have the repeat behave they do to begin with. Maybe we can update how worker plugins behave and always overwrite worker plugins with the same name. FWIW we're adding names to scheduler plugins (xref #5120) and I think there would be value in all our plugin systems having consistent behavior around repeats Also, I just noticed that we always overwrite the worker plugin stored on the scheduler distributed/distributed/scheduler.py Line 6952 in 33b795f which means that new workers which arrive can end up with a different set of worker plugins than older workers |
Currently the default name is
"pip"which results innot registering the second plugin. This PR ensures a unique name is generated for
PipInstallplugins which are installing different packages