Skip to content

Dont start http server in Scheduler.__init__#4928

Closed
fjetter wants to merge 1 commit intodask:mainfrom
fjetter:dont_start_http_server_init
Closed

Dont start http server in Scheduler.__init__#4928
fjetter wants to merge 1 commit intodask:mainfrom
fjetter:dont_start_http_server_init

Conversation

@fjetter
Copy link
Member

@fjetter fjetter commented Jun 18, 2021

This should deal with the CI port already in use warning situation reported in #4806

Highlights

  • We have a few places where we start up a cluster (e.g. spec cluster or gen_cluster) where we do not close the scheduler if workers fail to start. This would leave ports still allocated
  • Previously HTTP server sockets were opened during Scheduler.__init__ which made some tests really awkward. This PR moves the start of the HTTP server to the Scheduler.start method.
  • A few tests intentionally start two clusters/schedulers where I circumvented this warning by simply allowing random ports. This revealed a shortcoming of our API since there is currently no way to not start an HTTP server
  • I added a filterwarnings statement in our setup.cfg to err whenever this warning is raised in our test setup. This is very strict but I would prefer keeping it and removing it again if it causes any issues. Preferably we'd have a test fixture which checks if the test left any open socks but I'm not aware of any code which can do this without root

Supersedes #4896 and #4921

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants