-
Notifications
You must be signed in to change notification settings - Fork 123
Feature/discover azure subscriptions #663
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/discover azure subscriptions #663
Conversation
Implements automatic discovery of all Azure subscriptions the user/app has access to when no specific subscription_id is provided. This enhancement eliminates the need to manually enumerate subscriptions and run cloudlist multiple times when working with multi-subscription Azure tenancies.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Overview
This PR introduces automatic discovery of all accessible Azure subscriptions when no subscription ID is specified, streamlining multi-subscription scanning.
- Replaces the single subscription ID with a list of subscription IDs in the provider.
- Adds logic to list subscriptions via the Azure SDK when a specific subscription is not provided.
- Updates resource collection to iterate over all discovered subscriptions.
Reviewed Changes
| File | Description |
|---|---|
| pkg/providers/azure/azure.go | Refactors subscription handling and resource discovery to support multiple subscriptions |
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (2)
pkg/providers/azure/azure.go:158
- [nitpick] The variable name 'vmp' is ambiguous; a more descriptive name, such as 'vmProviderInstance', could improve code clarity.
vmp := &vmProvider{Authorizer: p.Authorizer, SubscriptionID: subscriptionID, id: p.id}
pkg/providers/azure/azure.go:168
- [nitpick] The variable name 'publicIPp' is unclear; renaming it to a more descriptive identifier like 'publicIPProviderInstance' could enhance readability.
publicIPp := &publicIPProvider{Authorizer: p.Authorizer, SubscriptionID: subscriptionID, id: p.id}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caution
Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
🧹 Nitpick comments (2)
.gitignore (1)
3-5: Ensure sensitive and generated files remain untracked.
The addition of.envandprovider-config.yamlto the ignore list is a good security practice to prevent accidental commits of sensitive configuration data. Including thedistdirectory explicitly also helps avoid tracking build artifacts.Consider restoring a trailing newline at the end of this file to comply with POSIX standards and common formatting conventions.
pkg/providers/azure/azure.go (1)
155-178: Consider parallelizing resource fetching for each subscription.
Iterating subscriptions sequentially works but may be slow under high subscription counts. Parallelizing calls or add concurrency controls might optimize performance.
🛑 Comments failed to post (2)
pkg/providers/gcp/gcp.go (1)
207-253:
⚠️ Potential issueThe GCP verification implementation has a logic issue.
While the overall approach is sound, the implementation has an issue - it will return an error as soon as the first attempted service check fails, rather than trying all services first. This differs from the AWS provider's approach.
The code should continue checking services even if an earlier one fails. Apply this fix:
- if err != nil { - return errorutil.NewWithErr(err).Msgf("failed to verify compute service access") - } - success = true + if err == nil { + success = true + }Make similar changes to the other service checks in this method.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.// Verify checks if the GCP provider credentials are valid func (p *Provider) Verify(ctx context.Context) error { if len(p.projects) == 0 { return errorutil.New("no accessible GCP projects found with provided credentials") } // For extra verification, try a minimal API call on one service for _, project := range p.projects { var success bool if p.compute != nil { _, err := p.compute.Regions.List(project).Do() if err == nil { success = true } } else if p.dns != nil { _, err := p.dns.ManagedZones.List(project).Do() if err == nil { success = true } } else if p.storage != nil { _, err := p.storage.Buckets.List(project).Do() if err == nil { success = true } } else if p.functions != nil { _, err := p.functions.Projects.Locations.List(project).Do() if err == nil { success = true } } else if p.run != nil { _, err := p.run.Projects.Locations.List(project).Do() if err == nil { success = true } } // For any one service to be successful, we can return nil if success { return nil } } return errorutil.New("no accessible GCP services found with provided credentials") }pkg/providers/azure/azure.go (1)
183-212:
⚠️ Potential issueUnconditional return ends the verification loop prematurely.
The loop over multiple subscriptions never proceeds beyond the first iteration. Lines 208 and 210 each return from the function, causing a static analysis warning (SA4004). Unless you only want to verify the first subscription, this prevents checking subsequent subscriptions.Below is a suggested fix that checks each subscription and returns success only if at least one subscription is accessible:
func (p *Provider) Verify(ctx context.Context) error { - for _, subscriptionID := range p.SubscriptionIDs { - ... - if success { - return nil - } - return fmt.Errorf("no accessible Azure services found with provided credentials") - } - return fmt.Errorf("no accessible Azure services found with provided credentials") + var foundAtLeastOne bool + for _, subscriptionID := range p.SubscriptionIDs { + var success bool + // attempt to verify "vm" or "publicip" services... + // if verification passes: + // success = true + + if success { + foundAtLeastOne = true + // optionally break here if only one valid subscription is required + } + } + if foundAtLeastOne { + return nil + } + return fmt.Errorf("no accessible Azure services found with provided credentials") }Decide if you want to validate all subscriptions or terminate upon the first valid subscription, then adjust accordingly.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.// Verify checks if the provider is valid using simple API call func (p *Provider) Verify(ctx context.Context) error { var foundAtLeastOne bool for _, subscriptionID := range p.SubscriptionIDs { groupsClient := resources.NewGroupsClient(subscriptionID) groupsClient.Authorizer = p.Authorizer pClient := network.NewPublicIPAddressesClient(subscriptionID) pClient.Authorizer = p.Authorizer // Try a lightweight operation - just list the first group var success bool if p.services.Has("vm") { _, err := groupsClient.List(ctx, "", nil) if err != nil { return fmt.Errorf("failed to verify Azure credentials: %v", err) } success = true } else if p.services.Has("publicip") && !success { _, err := pClient.ListAllComplete(ctx) if err != nil { return fmt.Errorf("failed to verify Azure credentials: %v", err) } success = true } if success { foundAtLeastOne = true // optionally break here if only one valid subscription is required } } if foundAtLeastOne { return nil } return fmt.Errorf("no accessible Azure services found with provided credentials") }🧰 Tools
🪛 golangci-lint (1.64.8)
210-210: SA4004: the surrounding loop is unconditionally terminated
(staticcheck)
WalkthroughThis pull request introduces several improvements across the project. The changes enhance error handling by replacing simple deferred file/response closures with deferred anonymous functions that log errors using a logging package. Additionally, new lint suppression comments ( Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Provider
participant SubClient as Azure Subscriptions Client
Client->>Provider: Create provider with options
alt Specific Subscription Provided
Provider->>Provider: Use provided subscription ID
else No Subscription Provided
Provider->>SubClient: List available subscriptions
SubClient-->>Provider: Return subscription IDs
end
Client->>Provider: Request Resources
loop For each Subscription
Provider->>Provider: Process resources for Subscription ID
end
Client->>Provider: Verify Credentials
loop For each Subscription
Provider->>Provider: Validate subscription credentials
end
Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (6)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (4)
pkg/providers/azure/azure.go (1)
185-211: Verification short-circuits on the first successful subscription.
If you want a more comprehensive verification of all accessible subscriptions, consider continuing through each one. Currently, once a single subscription’s credentials are verified, the method returns success.pkg/providers/gcp/dns.go (1)
69-69: Clarify the linter rule being ignored.
It’s often useful to specify which linter or rule is being disabled, for instance//nolint:gocyclo, to help understand the intention.pkg/providers/terraform/state_file.go (1)
30-34: Improved error handling for file closure, but error message needs correction.The change to properly handle and log file closure errors is excellent. However, the error message refers to a "provider config file" when this is actually closing a Terraform state file.
- gologger.Error().Msgf("Could not close provider config file: %s\n", err) + gologger.Error().Msgf("Could not close Terraform state file: %s\n", err)pkg/providers/custom/services.go (1)
43-47: Improved error handling for response body closure, but error message needs correction.The change to properly handle and log response body closure errors is excellent. However, the error message refers to a "provider config file" when this is actually closing an HTTP response body.
- gologger.Error().Msgf("Could not close provider config file: %s\n", err) + gologger.Error().Msgf("Could not close HTTP response body: %s\n", err)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (9)
.gitignore(1 hunks)internal/runner/options.go(1 hunks)pkg/inventory/options.go(2 hunks)pkg/providers/aws/route53.go(1 hunks)pkg/providers/azure/azure.go(6 hunks)pkg/providers/custom/services.go(2 hunks)pkg/providers/gcp/dns.go(1 hunks)pkg/providers/terraform/state_file.go(2 hunks)pkg/schema/validate/validate.go(1 hunks)
🧰 Additional context used
🧬 Code Definitions (1)
pkg/providers/azure/azure.go (1)
pkg/schema/schema.go (3)
ServiceMap(251-251)OptionBlock(192-192)Provider(18-28)
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: Test Builds (1.22.x, macOS-latest)
- GitHub Check: Test Builds (1.22.x, windows-latest)
- GitHub Check: Test Builds (1.22.x, ubuntu-latest)
🔇 Additional comments (17)
pkg/providers/azure/azure.go (9)
10-10: No issues identified in the import statement.
22-22: No issues found with optional subscription ID constant.
32-35: Multiple subscription IDs approach looks good.
Storing a slice of subscription IDs is a logical improvement for multi-subscription environments.
41-41: No issues found with reading 'use_cli_auth'.
73-73: No comments needed.
92-96: Initialization of the provider struct is clear and well-structured.
98-105: Conditional inclusion of a single subscription ID is correct.
This helps preserve backward compatibility for existing configurations.
107-133: Discovery logic for all subscriptions.
This approach is well-implemented. Good job on returning an error when none are discovered.
155-178: Consider partial failure handling for each subscription.
Currently, an error listing VM or public IPs for a subscription causes the code to move on. This is reasonable, but be mindful of potential transient failures that could result in partial data.You might want to verify whether skipping subscriptions on errors is the desired behavior. If you’d like thorough reporting, you could log each failure and still proceed to gather what’s available from other resource types in the same subscription.
.gitignore (1)
3-5: Ignoring build artifacts and sensitive configuration is a best practice.
Adding.envandprovider-config.yamlhelps avoid committing secrets and local config.pkg/providers/aws/route53.go (1)
110-111: Good use of linter suppression directive.The added
//nolintdirective helps suppress linter warnings for the conditional checks that follow, while maintaining the existing logic for handling different types of DNS records (A and AAAA).pkg/inventory/options.go (2)
7-7: Import for error logging added.The gologger import is appropriately added to support the new error logging functionality.
17-21: Improved error handling for file closure.Good enhancement to error handling. Replacing the simple
defer file.Close()with a deferred anonymous function ensures that any errors encountered during file closure are properly logged rather than silently ignored.internal/runner/options.go (1)
141-145: Improved error handling for file closure.Excellent enhancement to error handling in the
readProviderConfigfunction. The deferred anonymous function properly logs any errors that occur during file closure, which helps with debugging and ensures errors aren't silently ignored.pkg/schema/validate/validate.go (1)
109-109: String replacement method improved.Good change from
strings.Replacetostrings.ReplaceAll. The new method is more direct and clearly communicates the intent to replace all occurrences of periods in the string.pkg/providers/terraform/state_file.go (1)
11-11: Good addition of the gologger package.Adding structured logging is a good practice that will improve error observability.
pkg/providers/custom/services.go (1)
12-12: Good addition of the gologger package.Adding structured logging is a good practice that will improve error observability.
* feat(azure): Discover Azure subscriptions automatically Implements automatic discovery of all Azure subscriptions the user/app has access to when no specific subscription_id is provided. This enhancement eliminates the need to manually enumerate subscriptions and run cloudlist multiple times when working with multi-subscription Azure tenancies. * chore: Update .gitignore * lint --------- Co-authored-by: Ciaran Finnegan <ciaran.finnegan@cmd.com.au> Co-authored-by: Mzack9999 <mzack9999@protonmail.com> Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Problem
Currently, users working with multi-subscription Azure environments must manually:
Solution
This PR enables cloudlist to automatically discover all Azure subscriptions the user/app has permissions for when no subscription_id is specified, allowing:
Implementation Details
Testing
Tested against a single scenario
Sample output
Summary by CodeRabbit
New Features
Chores
Bug Fixes