Skip to content

Add list resources command#673

Merged
torgermiller merged 16 commits intomainfrom
add-list-resources-command
Jun 17, 2025
Merged

Add list resources command#673
torgermiller merged 16 commits intomainfrom
add-list-resources-command

Conversation

@torgermiller
Copy link
Contributor

@torgermiller torgermiller commented Jun 11, 2025

Add new command to list resource names.

Example uses:

Get all resource names:
sdpctl sites resources [site-id]

Get all resource names from resolver esx and resource folders

sdpctl sites resources [site-id] --resolver "esx" --resource "folde
rs"

Get all resource names from resolvers esx and aws and resources folders and lbs
resources [site-id] --resolver "esx&aws" --resource "folders&lbs"

Example output:

Name       Resolver    Type       Gateway Name
----       --------    ----       ------------
folder1    esx         folders    gateway-site1
folder2    esx         folders    gateway-site1
folder3    esx         folders    gateway-site1

@torgermiller torgermiller force-pushed the add-list-resources-command branch from d361d6f to 0617f8c Compare June 11, 2025 20:10
@torgermiller torgermiller marked this pull request as ready for review June 11, 2025 20:13
@mandopaloooza
Copy link
Collaborator

mandopaloooza commented Jun 13, 2025

❯ go run main.go sites resources 8a4add9e-0e99-4bb1-949c-c9faf9a49ad4
Querying resource names...

Name    Resolver    Type    Gateway Name
----    --------    ----    ------------
No resources found in the site

It spent about 30 seconds until the results came back. Does it take this long because sdpctl is waiting for the server response (i.e. it is a compute-intensive task in the Controller)?

@torgermiller
Copy link
Contributor Author

❯ go run main.go sites resources 8a4add9e-0e99-4bb1-949c-c9faf9a49ad4
Querying resource names...

Name    Resolver    Type    Gateway Name
----    --------    ----    ------------
No resources found in the site

It spent about 30 seconds until the results came back. Does it take this long because sdpctl is waiting for the server response (i.e. it is a compute-intensive task in the Controller)?

The API endpoint requires a resolver and resource type be specified. So, the only way to get all of the resources is to try every combination of resolver and resource type. I suspect the O^2 nature of this to be the issue.

If you specify a resolver and resource type when running the command it's fast though.

I think the most reasonable option to improve the performance would be changing the API to allow a request without any filtering. Although, I guess a problem with changing the API would be backwards compatibility. We'd still need the current approach for backwards compatibility for a long time.

There are three possible solutions client side but I don't like any of them:

  1. Run many requests simultaneously. I worry about this destabilizing the controller or hitting rate limits.
  2. Maintain a mapping list so only valid combinations are tried. Currently, all combinations are tried (even invalid ones). But, I felt that maintaining a valid combination list would be more trouble than it was worth. Especially given that it's probably best solved server side with a way to query for all resources.
  3. Mirror the API and require a resolver type and resource type be specified. Personally I much prefer being able to scan for all resources even if it takes a while.

So, I agree the performance isn't the best but I'm not really sure of a reasonable way to improve it. If I had to pick one I'd go with option 1 I guess.

Copy link
Collaborator

@mandopaloooza mandopaloooza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@torgermiller Thanks for the explanation!

I'm not necessarily looking for the fix in this PR. I think we can ship it as it is and improve later on if we receive feedback that it needs to run faster. I highly doubt anyone would complain about the speed as long as it returns the correct outputs 😄

@torgermiller torgermiller merged commit 4f1d0b2 into main Jun 17, 2025
3 checks passed
@torgermiller torgermiller deleted the add-list-resources-command branch June 17, 2025 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants