Skip to content

Conversation

@ciaran-finnegan
Copy link
Contributor

@ciaran-finnegan ciaran-finnegan commented Mar 8, 2025

Problem

Currently, users working with multi-subscription Azure environments must manually:

  • Enumerate each subscription individually
  • Run cloudlist multiple times with different subscription IDs
  • Manage results from separate operations

Solution

This PR enables cloudlist to automatically discover all Azure subscriptions the user/app has permissions for when no subscription_id is specified, allowing:

  • Single-operation scanning across all accessible subscriptions
  • Consistent resource discovery without manual subscription enumeration
  • Improved workflow for organisations with Azure tenancies with multiple subscriptions

Implementation Details

  • Uses Azure SDK subscription client to retrieve accessible subscriptions
  • Maintains backward compatibility with existing configuration options

Testing

Tested against a single scenario

  • Multiple subscriptions
  • Reader permission
  • CLI auth

Sample output

./cloudlist -pc provider-config.yaml -p azure

  _______             _____     __ 
 / ___/ /__  __ _____/ / (_)__ / /_
/ /__/ / _ \/ // / _  / / (_-</ __/
\___/_/\___/\_,_/\_,_/_/_/___/\__/ 

                projectdiscovery.io

[INF] Current cloudlist version 1.2.1 (latest)
[INF] Listing subscriptions from provider: azure
[INF] Discovered subscription: adb71dea-10fb-42ff-9afd-redacted
[INF] Discovered subscription: cfb2dbf7-9908-41d9-a544-redacted
[INF] Listing assets from provider: azure services: vm,publicip id: 
[INF] Processing subscription: adb71dea-10fb-42ff-9afd-redacted
etc.

Summary by CodeRabbit

  • New Features

    • Enhanced Azure support to manage multiple subscription IDs for improved resource aggregation.
  • Chores

    • Updated project configurations to automatically ignore sensitive and provider-specific configuration files.
  • Bug Fixes

    • Improved error handling during file and response body closures for better feedback on potential issues.
    • Updated string manipulation method in validation to ensure correct DNS name checks.

Ciaran Finnegan added 2 commits March 8, 2025 11:02
Implements automatic discovery of all Azure subscriptions the user/app has access to when no specific subscription_id is provided. This enhancement eliminates the need to manually enumerate subscriptions and run cloudlist multiple times when working with multi-subscription Azure tenancies.
@ehsandeep ehsandeep requested a review from Copilot March 8, 2025 21:48
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overview

This PR introduces automatic discovery of all accessible Azure subscriptions when no subscription ID is specified, streamlining multi-subscription scanning.

  • Replaces the single subscription ID with a list of subscription IDs in the provider.
  • Adds logic to list subscriptions via the Azure SDK when a specific subscription is not provided.
  • Updates resource collection to iterate over all discovered subscriptions.

Reviewed Changes

File Description
pkg/providers/azure/azure.go Refactors subscription handling and resource discovery to support multiple subscriptions

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

pkg/providers/azure/azure.go:158

  • [nitpick] The variable name 'vmp' is ambiguous; a more descriptive name, such as 'vmProviderInstance', could improve code clarity.
vmp := &vmProvider{Authorizer: p.Authorizer, SubscriptionID: subscriptionID, id: p.id}

pkg/providers/azure/azure.go:168

  • [nitpick] The variable name 'publicIPp' is unclear; renaming it to a more descriptive identifier like 'publicIPProviderInstance' could enhance readability.
publicIPp := &publicIPProvider{Authorizer: p.Authorizer, SubscriptionID: subscriptionID, id: p.id}

@ehsandeep ehsandeep requested a review from dogancanbakir March 27, 2025 14:34
@dogancanbakir dogancanbakir requested a review from Mzack9999 March 27, 2025 17:49
@Mzack9999 Mzack9999 closed this Mar 28, 2025
@Mzack9999 Mzack9999 reopened this Mar 28, 2025
@projectdiscovery projectdiscovery deleted a comment from coderabbitai bot Mar 28, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.

Actionable comments posted: 2

🧹 Nitpick comments (2)
.gitignore (1)

3-5: Ensure sensitive and generated files remain untracked.
The addition of .env and provider-config.yaml to the ignore list is a good security practice to prevent accidental commits of sensitive configuration data. Including the dist directory explicitly also helps avoid tracking build artifacts.

Consider restoring a trailing newline at the end of this file to comply with POSIX standards and common formatting conventions.

pkg/providers/azure/azure.go (1)

155-178: Consider parallelizing resource fetching for each subscription.
Iterating subscriptions sequentially works but may be slow under high subscription counts. Parallelizing calls or add concurrency controls might optimize performance.

🛑 Comments failed to post (2)
pkg/providers/gcp/gcp.go (1)

207-253: ⚠️ Potential issue

The GCP verification implementation has a logic issue.

While the overall approach is sound, the implementation has an issue - it will return an error as soon as the first attempted service check fails, rather than trying all services first. This differs from the AWS provider's approach.

The code should continue checking services even if an earlier one fails. Apply this fix:

-			if err != nil {
-				return errorutil.NewWithErr(err).Msgf("failed to verify compute service access")
-			}
-			success = true
+			if err == nil {
+				success = true
+			}

Make similar changes to the other service checks in this method.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

// Verify checks if the GCP provider credentials are valid
func (p *Provider) Verify(ctx context.Context) error {
	if len(p.projects) == 0 {
		return errorutil.New("no accessible GCP projects found with provided credentials")
	}

	// For extra verification, try a minimal API call on one service
	for _, project := range p.projects {
		var success bool
		if p.compute != nil {
			_, err := p.compute.Regions.List(project).Do()
			if err == nil {
				success = true
			}
		} else if p.dns != nil {
			_, err := p.dns.ManagedZones.List(project).Do()
			if err == nil {
				success = true
			}
		} else if p.storage != nil {
			_, err := p.storage.Buckets.List(project).Do()
			if err == nil {
				success = true
			}
		} else if p.functions != nil {
			_, err := p.functions.Projects.Locations.List(project).Do()
			if err == nil {
				success = true
			}
		} else if p.run != nil {
			_, err := p.run.Projects.Locations.List(project).Do()
			if err == nil {
				success = true
			}
		}
		// For any one service to be successful, we can return nil
		if success {
			return nil
		}
	}
	return errorutil.New("no accessible GCP services found with provided credentials")
}
pkg/providers/azure/azure.go (1)

183-212: ⚠️ Potential issue

Unconditional return ends the verification loop prematurely.
The loop over multiple subscriptions never proceeds beyond the first iteration. Lines 208 and 210 each return from the function, causing a static analysis warning (SA4004). Unless you only want to verify the first subscription, this prevents checking subsequent subscriptions.

Below is a suggested fix that checks each subscription and returns success only if at least one subscription is accessible:

 func (p *Provider) Verify(ctx context.Context) error {
-    for _, subscriptionID := range p.SubscriptionIDs {
-        ...
-        if success {
-            return nil
-        }
-        return fmt.Errorf("no accessible Azure services found with provided credentials")
-    }
-    return fmt.Errorf("no accessible Azure services found with provided credentials")
+    var foundAtLeastOne bool
+    for _, subscriptionID := range p.SubscriptionIDs {
+        var success bool
+        // attempt to verify "vm" or "publicip" services...
+        // if verification passes:
+        //   success = true
+
+        if success {
+            foundAtLeastOne = true
+            // optionally break here if only one valid subscription is required
+        }
+    }
+    if foundAtLeastOne {
+        return nil
+    }
+    return fmt.Errorf("no accessible Azure services found with provided credentials")
 }

Decide if you want to validate all subscriptions or terminate upon the first valid subscription, then adjust accordingly.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

// Verify checks if the provider is valid using simple API call
func (p *Provider) Verify(ctx context.Context) error {
	var foundAtLeastOne bool
	for _, subscriptionID := range p.SubscriptionIDs {
		groupsClient := resources.NewGroupsClient(subscriptionID)
		groupsClient.Authorizer = p.Authorizer

		pClient := network.NewPublicIPAddressesClient(subscriptionID)
		pClient.Authorizer = p.Authorizer

		// Try a lightweight operation - just list the first group
		var success bool
		if p.services.Has("vm") {
			_, err := groupsClient.List(ctx, "", nil)
			if err != nil {
				return fmt.Errorf("failed to verify Azure credentials: %v", err)
			}
			success = true
		} else if p.services.Has("publicip") && !success {
			_, err := pClient.ListAllComplete(ctx)
			if err != nil {
				return fmt.Errorf("failed to verify Azure credentials: %v", err)
			}
			success = true
		}

		if success {
			foundAtLeastOne = true
			// optionally break here if only one valid subscription is required
		}
	}
	if foundAtLeastOne {
		return nil
	}
	return fmt.Errorf("no accessible Azure services found with provided credentials")
}
🧰 Tools
🪛 golangci-lint (1.64.8)

210-210: SA4004: the surrounding loop is unconditionally terminated

(staticcheck)

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 28, 2025

Walkthrough

This pull request introduces several improvements across the project. The changes enhance error handling by replacing simple deferred file/response closures with deferred anonymous functions that log errors using a logging package. Additionally, new lint suppression comments (//nolint) are added to specific provider methods, and the Azure provider is modified to support multiple subscription IDs. Updates to the .gitignore file now prevent tracking of environment and config files, and a minor string replacement update is applied in the validator.

Changes

File(s) Change Summary
.gitignore Added new entries: .env and provider-config.yaml; explicitly marked cloudlist as added.
internal/runner/options.go, pkg/inventory/options.go, pkg/providers/custom/services.go, pkg/providers/terraform/state_file.go Replaced simple defer close calls with deferred anonymous functions that log errors on closure failures. (State_file.go additionally adds a new gologger import.)
pkg/providers/aws/route53.go, pkg/providers/gcp/dns.go Added //nolint comments before resource record condition checks to suppress linter warnings.
pkg/providers/azure/azure.go Changed SubscriptionID string to SubscriptionIDs []string in the Provider struct and updated the New, Resources, and Verify methods to support multiple subscriptions.
pkg/schema/validate/validate.go Replaced strings.Replace with strings.ReplaceAll in the isDNSName method for improved string manipulation.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Provider
    participant SubClient as Azure Subscriptions Client
    Client->>Provider: Create provider with options
    alt Specific Subscription Provided
         Provider->>Provider: Use provided subscription ID
    else No Subscription Provided
         Provider->>SubClient: List available subscriptions
         SubClient-->>Provider: Return subscription IDs
    end
    Client->>Provider: Request Resources
    loop For each Subscription
         Provider->>Provider: Process resources for Subscription ID
    end
    Client->>Provider: Verify Credentials
    loop For each Subscription
         Provider->>Provider: Validate subscription credentials
    end
Loading

Possibly related PRs

  • DNSSimple provider #665: The changes in the main PR regarding the .gitignore file are related to the retrieved PR, as both PRs add the same entries for .env and provider-config.yaml, and both involve modifications to the cloudlist entry.

Suggested reviewers

  • dogancanbakir
  • Mzack9999

Poem

I'm a bouncy rabbit in a world of code so bright,
Hopping through errors with logging done just right.
Azure subscriptions now flourish in a multi-hued sky,
And linting whispers "all is well" as I merrily hop by.
With each new change, my paws tap a joyful beat,
Celebrating robust fixes with a heart so light and sweet.
Hop on, my friend—let’s code with boundless delight!


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between e585702 and 444f78b.

📒 Files selected for processing (1)
  • .gitignore (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • .gitignore
⏰ Context from checks skipped due to timeout of 90000ms (6)
  • GitHub Check: Test Builds (1.22.x, macOS-latest)
  • GitHub Check: Test Builds (1.22.x, windows-latest)
  • GitHub Check: release-test
  • GitHub Check: Lint Test
  • GitHub Check: Analyze (go)
  • GitHub Check: Test Builds (1.22.x, ubuntu-latest)

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai plan to trigger planning for file edits and PR creation.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
pkg/providers/azure/azure.go (1)

185-211: Verification short-circuits on the first successful subscription.
If you want a more comprehensive verification of all accessible subscriptions, consider continuing through each one. Currently, once a single subscription’s credentials are verified, the method returns success.

pkg/providers/gcp/dns.go (1)

69-69: Clarify the linter rule being ignored.
It’s often useful to specify which linter or rule is being disabled, for instance //nolint:gocyclo, to help understand the intention.

pkg/providers/terraform/state_file.go (1)

30-34: Improved error handling for file closure, but error message needs correction.

The change to properly handle and log file closure errors is excellent. However, the error message refers to a "provider config file" when this is actually closing a Terraform state file.

- gologger.Error().Msgf("Could not close provider config file: %s\n", err)
+ gologger.Error().Msgf("Could not close Terraform state file: %s\n", err)
pkg/providers/custom/services.go (1)

43-47: Improved error handling for response body closure, but error message needs correction.

The change to properly handle and log response body closure errors is excellent. However, the error message refers to a "provider config file" when this is actually closing an HTTP response body.

- gologger.Error().Msgf("Could not close provider config file: %s\n", err)
+ gologger.Error().Msgf("Could not close HTTP response body: %s\n", err)
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between 9306e22 and e585702.

📒 Files selected for processing (9)
  • .gitignore (1 hunks)
  • internal/runner/options.go (1 hunks)
  • pkg/inventory/options.go (2 hunks)
  • pkg/providers/aws/route53.go (1 hunks)
  • pkg/providers/azure/azure.go (6 hunks)
  • pkg/providers/custom/services.go (2 hunks)
  • pkg/providers/gcp/dns.go (1 hunks)
  • pkg/providers/terraform/state_file.go (2 hunks)
  • pkg/schema/validate/validate.go (1 hunks)
🧰 Additional context used
🧬 Code Definitions (1)
pkg/providers/azure/azure.go (1)
pkg/schema/schema.go (3)
  • ServiceMap (251-251)
  • OptionBlock (192-192)
  • Provider (18-28)
⏰ Context from checks skipped due to timeout of 90000ms (3)
  • GitHub Check: Test Builds (1.22.x, macOS-latest)
  • GitHub Check: Test Builds (1.22.x, windows-latest)
  • GitHub Check: Test Builds (1.22.x, ubuntu-latest)
🔇 Additional comments (17)
pkg/providers/azure/azure.go (9)

10-10: No issues identified in the import statement.


22-22: No issues found with optional subscription ID constant.


32-35: Multiple subscription IDs approach looks good.
Storing a slice of subscription IDs is a logical improvement for multi-subscription environments.


41-41: No issues found with reading 'use_cli_auth'.


73-73: No comments needed.


92-96: Initialization of the provider struct is clear and well-structured.


98-105: Conditional inclusion of a single subscription ID is correct.
This helps preserve backward compatibility for existing configurations.


107-133: Discovery logic for all subscriptions.
This approach is well-implemented. Good job on returning an error when none are discovered.


155-178: Consider partial failure handling for each subscription.
Currently, an error listing VM or public IPs for a subscription causes the code to move on. This is reasonable, but be mindful of potential transient failures that could result in partial data.

You might want to verify whether skipping subscriptions on errors is the desired behavior. If you’d like thorough reporting, you could log each failure and still proceed to gather what’s available from other resource types in the same subscription.

.gitignore (1)

3-5: Ignoring build artifacts and sensitive configuration is a best practice.
Adding .env and provider-config.yaml helps avoid committing secrets and local config.

pkg/providers/aws/route53.go (1)

110-111: Good use of linter suppression directive.

The added //nolint directive helps suppress linter warnings for the conditional checks that follow, while maintaining the existing logic for handling different types of DNS records (A and AAAA).

pkg/inventory/options.go (2)

7-7: Import for error logging added.

The gologger import is appropriately added to support the new error logging functionality.


17-21: Improved error handling for file closure.

Good enhancement to error handling. Replacing the simple defer file.Close() with a deferred anonymous function ensures that any errors encountered during file closure are properly logged rather than silently ignored.

internal/runner/options.go (1)

141-145: Improved error handling for file closure.

Excellent enhancement to error handling in the readProviderConfig function. The deferred anonymous function properly logs any errors that occur during file closure, which helps with debugging and ensures errors aren't silently ignored.

pkg/schema/validate/validate.go (1)

109-109: String replacement method improved.

Good change from strings.Replace to strings.ReplaceAll. The new method is more direct and clearly communicates the intent to replace all occurrences of periods in the string.

pkg/providers/terraform/state_file.go (1)

11-11: Good addition of the gologger package.

Adding structured logging is a good practice that will improve error observability.

pkg/providers/custom/services.go (1)

12-12: Good addition of the gologger package.

Adding structured logging is a good practice that will improve error observability.

@ehsandeep ehsandeep linked an issue Mar 29, 2025 that may be closed by this pull request
@ehsandeep ehsandeep merged commit 60fb263 into projectdiscovery:dev Mar 29, 2025
9 checks passed
visnetodev pushed a commit to visnetotest/cloudlist that referenced this pull request Dec 7, 2025
* feat(azure): Discover Azure subscriptions automatically

Implements automatic discovery of all Azure subscriptions the user/app has access to when no specific subscription_id is provided. This enhancement eliminates the need to manually enumerate subscriptions and run cloudlist multiple times when working with multi-subscription Azure tenancies.

* chore: Update .gitignore

* lint

---------

Co-authored-by: Ciaran Finnegan <ciaran.finnegan@cmd.com.au>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

discover azure subscriptions

4 participants