Skip to content

Implement dask-ctl discovery#468

Merged
jacobtomlinson merged 8 commits intodask:mainfrom
jacobtomlinson:implement-discovery
Sep 2, 2022
Merged

Implement dask-ctl discovery#468
jacobtomlinson merged 8 commits intodask:mainfrom
jacobtomlinson:implement-discovery

Conversation

@jacobtomlinson
Copy link
Copy Markdown
Member

@jacobtomlinson jacobtomlinson commented May 10, 2022 •
edited
Loading

Implemented the discover entrypoint for dask-ctl so that clusters can be managed via dask-ctl.

>>> from dask_kubernetes.experimental import KubeCluster

>>> cluster = KubeCluster(name="myawesomecluster")
$ daskctl cluster list                         
                                                                                                                                                                                    
  Name               Address                 Type                                                   Discovery                  Workers   Threads   Memory       Created    Status   
 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
  myawesomecluster   tcp://localhost:51967   dask_kubernetes.experimental.kubecluster.KubeCluster   kubecluster_experimental   3         36        279.11 GiB   Just now   Running  
>>> from dask_ctl import get_cluster

>>> cluster = get_cluster("myawesomecluster")

>>> cluster
KubeCluster(myawesomecluster, 'tcp://localhost:59357', workers=3, threads=36, memory=279.11 GiB)

It required detecting the current namespace so I cleaned up the existing namespace discovery utility and reused it a few times throughout the codebase. So now we check for a service account for in-cluster config, then fall back to getting the namespace from the current kubectl context, then finally fall back to "default".

Closes #481

@jacobtomlinson jacobtomlinson mentioned this pull request May 10, 2022
Closed
@jacobtomlinson jacobtomlinson merged commit 6f59850 into dask:main Sep 2, 2022
@jacobtomlinson jacobtomlinson deleted the implement-discovery branch September 2, 2022 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

operator

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Implement dask-ctl discovery

1 participant

@jacobtomlinson