Skip to content

[IDE-5654] Add a new endpoint for secrets scanning#816

Merged
alonam merged 11 commits intomainfrom
alonam/scan_secrets_endpoint
Feb 24, 2026
Merged

[IDE-5654] Add a new endpoint for secrets scanning#816
alonam merged 11 commits intomainfrom
alonam/scan_secrets_endpoint

Conversation

@alonam
Copy link
Contributor

@alonam alonam commented Feb 15, 2026

What problem are you trying to solve?

This PR adds a new /scan-secrets endpoint to the static analyzer server that scans source code for secrets like API keys, tokens, and credentials using configurable detection rules.

What is your solution?

  • Follows existing endpoint patterns (async with spawn_blocking, TraceSpan integration)
  • Uses Result-based error handling consistent with analyze endpoint
  • Generic request model SecretScanRequest<T> matches AnalysisRequest<T> pattern
  • Properly separates serialization errors from results

Alternatives considered

We considered integrating secret scanning into the /analyze endpoint. However, for performance reasons, we chose to keep them separate, since secret scanning may call a third-party endpoint to validate detected secrets, which could slow down the overall static analysis process.

Future Work

  • Cache secret rules

@alonam alonam added enhancement New feature or request ide labels Feb 15, 2026
@datadog-datadog-prod-us1

This comment has been minimized.

@alonam alonam changed the title [IDE-5590] Add endpoint for secrets scanning [IDE-5590] Add a new endpoint for secrets scanning Feb 16, 2026
@alonam alonam requested a review from jamesphlewis February 18, 2026 14:52
@alonam alonam marked this pull request as ready for review February 18, 2026 14:53
@alonam alonam requested a review from a team as a code owner February 18, 2026 14:53
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99fa425f66

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}

// Build the scanner with the provided rules
let scanner = secrets::scanner::build_sds_scanner(&rules, request.use_debug);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid panic on malformed scan rules

This endpoint forwards untrusted request rules directly into build_sds_scanner, but scanner construction is panic-based (.build().expect(...) in crates/secrets/src/scanner.rs), so a malformed rule (for example an invalid regex pattern) can make /scan-secrets panic instead of returning a structured error response. Because scan_secrets also does .await.unwrap(), that panic propagates through the handler path and turns a bad client payload into a server-side failure rather than a normal validation error.

Useful? React with 👍 / 👎.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed the signature of build_sds_scanner to return Result

@alonam alonam changed the title [IDE-5590] Add a new endpoint for secrets scanning [IDE-5654] Add a new endpoint for secrets scanning Feb 18, 2026
return Err("No rules provided".to_string());
}

if rules.len() > MAX_RULES_COUNT {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be a bit more optimized if you checked the length of request.rules before attempting deserialization

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to check rules count before deserializing

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SecretScanRequest<T = serde_json::Value> {
pub filename: String,
pub code: String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit, but I would probably rename this to text or data or something. Just helps to quickly communicate the intent (I am guessing we'll scan plain text as well)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed field to data

.map_err(|e| format!("Failed to parse rules: {}", e))?;

// Build the scanner with the provided rules
let scanner = secrets::scanner::build_sds_scanner(&rules, request.use_debug);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Codex already caught this. It might be worth changing the signature of build_sds_scanner to return a Result. This method is only called in one place in the codebase besides here, so it is should be an easy change and would allow you to avoid dealing with a panic. It would only require a map_err to String in here.

The alternative would be to rely on Tokio catching the panic in the endpoint and then using unwrap_or_else to map it to a more appropriate error.

Since the latter approach depends on the panic configuration being unwind, I'd go for the first suggestion.

Copy link
Contributor Author

@alonam alonam Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree we should fix it. I'd like to fix it in coordination with the code owner on a separate PR, to make sure he/she is aware and agrees with the change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense, but then I would at least consider using here the unwrap_or_else alternative I mentioned in the endpoint. I'll leave my suggestion there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR updated, I changed the signature of build_sds_scanner to return Result

errors,
})
})
.await
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we won't be refactoring the build_sds_scanner in this PR I think we should consider this in here:

.unwrap_or_else(|_| {
    json!(SecretScanResponse {
        rule_responses: vec![],
        errors: vec!["Internal error: scanner failed to initialize".to_string()],
    })
})

To be honest, I’d consider changing the build_sds_scanner signature in this PR since the impact is negligible. That said, I understand if you’d prefer to address it in a separate PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with all of this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's go with Plan A, to change build_sds_scanner signature. I'll update the PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR updated, changed the signature of build_sds_scanner to return Result

@alonam alonam removed the enhancement New feature or request label Feb 24, 2026
Copy link
Contributor

@robertohuertasm-datadog robertohuertasm-datadog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@alonam alonam merged commit b0dddc4 into main Feb 24, 2026
100 of 101 checks passed
@alonam alonam deleted the alonam/scan_secrets_endpoint branch February 24, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants