[IDE-5654] Add a new endpoint for secrets scanning#816
Conversation
This comment has been minimized.
This comment has been minimized.
crates/bins/src/bin/datadog_static_analyzer_server/endpoints.rs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 99fa425f66
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
|
|
||
| // Build the scanner with the provided rules | ||
| let scanner = secrets::scanner::build_sds_scanner(&rules, request.use_debug); |
There was a problem hiding this comment.
Avoid panic on malformed scan rules
This endpoint forwards untrusted request rules directly into build_sds_scanner, but scanner construction is panic-based (.build().expect(...) in crates/secrets/src/scanner.rs), so a malformed rule (for example an invalid regex pattern) can make /scan-secrets panic instead of returning a structured error response. Because scan_secrets also does .await.unwrap(), that panic propagates through the handler path and turns a bad client payload into a server-side failure rather than a normal validation error.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
changed the signature of build_sds_scanner to return Result
crates/bins/src/bin/datadog_static_analyzer_server/endpoints.rs
Outdated
Show resolved
Hide resolved
| return Err("No rules provided".to_string()); | ||
| } | ||
|
|
||
| if rules.len() > MAX_RULES_COUNT { |
There was a problem hiding this comment.
This would be a bit more optimized if you checked the length of request.rules before attempting deserialization
There was a problem hiding this comment.
Updated to check rules count before deserializing
| #[derive(Debug, Clone, Serialize, Deserialize)] | ||
| pub struct SecretScanRequest<T = serde_json::Value> { | ||
| pub filename: String, | ||
| pub code: String, |
There was a problem hiding this comment.
Small nit, but I would probably rename this to text or data or something. Just helps to quickly communicate the intent (I am guessing we'll scan plain text as well)
There was a problem hiding this comment.
Renamed field to data
| .map_err(|e| format!("Failed to parse rules: {}", e))?; | ||
|
|
||
| // Build the scanner with the provided rules | ||
| let scanner = secrets::scanner::build_sds_scanner(&rules, request.use_debug); |
There was a problem hiding this comment.
I think Codex already caught this. It might be worth changing the signature of build_sds_scanner to return a Result. This method is only called in one place in the codebase besides here, so it is should be an easy change and would allow you to avoid dealing with a panic. It would only require a map_err to String in here.
The alternative would be to rely on Tokio catching the panic in the endpoint and then using unwrap_or_else to map it to a more appropriate error.
Since the latter approach depends on the panic configuration being unwind, I'd go for the first suggestion.
There was a problem hiding this comment.
Agree we should fix it. I'd like to fix it in coordination with the code owner on a separate PR, to make sure he/she is aware and agrees with the change.
There was a problem hiding this comment.
That makes sense, but then I would at least consider using here the unwrap_or_else alternative I mentioned in the endpoint. I'll leave my suggestion there.
There was a problem hiding this comment.
PR updated, I changed the signature of build_sds_scanner to return Result
| errors, | ||
| }) | ||
| }) | ||
| .await |
There was a problem hiding this comment.
Given that we won't be refactoring the build_sds_scanner in this PR I think we should consider this in here:
.unwrap_or_else(|_| {
json!(SecretScanResponse {
rule_responses: vec![],
errors: vec!["Internal error: scanner failed to initialize".to_string()],
})
})To be honest, I’d consider changing the build_sds_scanner signature in this PR since the impact is negligible. That said, I understand if you’d prefer to address it in a separate PR.
There was a problem hiding this comment.
Agree with all of this
There was a problem hiding this comment.
Ok, let's go with Plan A, to change build_sds_scanner signature. I'll update the PR
There was a problem hiding this comment.
PR updated, changed the signature of build_sds_scanner to return Result
What problem are you trying to solve?
This PR adds a new
/scan-secretsendpoint to the static analyzer server that scans source code for secrets like API keys, tokens, and credentials using configurable detection rules.What is your solution?
spawn_blocking,TraceSpanintegration)SecretScanRequest<T>matchesAnalysisRequest<T>patternAlternatives considered
We considered integrating secret scanning into the
/analyzeendpoint. However, for performance reasons, we chose to keep them separate, since secret scanning may call a third-party endpoint to validate detected secrets, which could slow down the overall static analysis process.Future Work