[Cloud Security] [Fleet] Add cloud connector access scope for input or package level credential definitions#250280
Conversation
|
Pinging @elastic/fleet (Team:Fleet) |
|
Pinging @elastic/contextual-security-apps (Team:Cloud Security) |
|
Hi @seanrathier it would be great if you can give us more information to maybe test this locally what integrations are expected to be at the input/package level? thanks a lot |
|
|
||
| // Input mode: update the correct stream | ||
| const { inputIndex, streamIndex } = target; | ||
| if (inputIndex === -1 || streamIndex === -1) { |
There was a problem hiding this comment.
Hmm why would index be -1 here? Maybe store -1 as store as
const INVALID_INDEX = -1There was a problem hiding this comment.
Added the index constant, but now the function has been deduped.
| updatePolicyWithNewCredentials, | ||
| updatePolicyWithExistingCredentials, | ||
| } = useCloudConnectorSetup(input, newPolicy, updatePolicy); | ||
| } = useCloudConnectorSetup(input, newPolicy, updatePolicy, packageInfo); |
There was a problem hiding this comment.
Nit: Can we derive input from newPolicy? I wonder if remove input and create a util to extract from newPolicy?
There was a problem hiding this comment.
My bad, did not notice the input was still in the params and unused. It has been removed
...components/fleet_extensions/cloud_connector/aws_cloud_connector/aws_cloud_connector_form.tsx
Show resolved
Hide resolved
x-pack/platform/plugins/shared/fleet/server/services/secrets/cloud_connector.ts
Show resolved
Hide resolved
| packageInfo: PackageInfo | ||
| ): NewPackagePolicy => { | ||
| const mode = detectStorageMode(packageInfo); | ||
| const { target } = resolveVarTarget(policy, mode); |
There was a problem hiding this comment.
It looks like this function is really about updating vars based on their scope, not just a “target.”
Some ideas to make this clearer:
- Rename target → scope throughout.
- Rename updatePolicyVarsAtTarget → updatePolicyVarsByScope or updatePackagePolicyVarsScope.
Add a brief comment explaining the two var scopes:
- Package scope: policy.vars (global to the package)
- Input/stream scope: policy.inputs[].streams[].vars
There was a problem hiding this comment.
Thanks for bringing this also to my attention, I found a way deduplicate updating vars, server side and client side.
Changed updatePolicyVarsAtTarget -> updatePolicyVarsByScope
Rename target → scope throughout.
I changed mode to scope, but target makes sense to me. I dedicated most of the code review changes on this comment 😅
The target is there mostly for the inputs to return the index of the enabled input. For consistency I would like to keep this as target
const scope = getCredentialStorageScope(packageInfo);
const { target } = resolveVarTarget(policy, scope);
| const packageVars: RegistryVarsEntry[] = packageInfo.vars || []; | ||
| const supportedVarNames = getAllSupportedVarNames(); | ||
|
|
||
| const hasPackageLevelCredentials = packageVars.some((varDef) => |
There was a problem hiding this comment.
The name detectStorageMode is a bit unclear about what’s being checked. It seems this function is really determining where connector credentials are stored—package/global vs input/stream level.
Maybe consider a name like getConnectorVarScope or getConnectorCredentialScope to make the intent clearer.
There was a problem hiding this comment.
changed to updatePolicyVarsByScope
const updatePolicyVarsByScope = (
policy: NewPackagePolicy,
updatedVars: PackagePolicyConfigRecord,
packageInfo: PackageInfo
): NewPackagePolicy => {
const scope = getCredentialStorageScope(packageInfo);
const { target } = resolveVarTarget(policy, scope);
return applyVarsAtTarget(policy, updatedVars, target);
};
| const enabledInputIndex = packagePolicy.inputs.findIndex((input) => input.enabled); | ||
| if (enabledInputIndex === -1) { | ||
| return { | ||
| target: { mode: 'input', inputIndex: -1, streamIndex: -1 }, |
There was a problem hiding this comment.
Storing -1 as const INVALID_INDEX or INDEX_NOT_FOUND
There was a problem hiding this comment.
| // Look for tenant_id using schema-defined keys | ||
| let tenantIdVar: PackagePolicyConfigRecordEntry | undefined; | ||
| for (const key of tenantIdKeys) { | ||
| if (vars[key]) { |
There was a problem hiding this comment.
I noticed there are quite a few for/map loops here (O(n) operations) to access or update vars both on the FE and BE. Wondering if there’s a way to optimize some of these lookups to O(1), e.g., by indexing vars by key or using a lookup map. This could improve performance especially in case there is scale in the increase fields for each cloud provider and processing Cloud connector API request.
There was a problem hiding this comment.
I was going to come back to this a create a helper, thanks for pointing this out.
I created a new function in the var_accessor called findFirstVarEntry to reduce the repeating for-loop to get the first var that matches any key.
const tenantIdVar = findFirstVarEntry(vars, tenantIdKeys);
const clientIdVar = findFirstVarEntry(vars, clientIdKeys);
const azureCredentials = findFirstVarEntry(vars, connectorIdKeys);
I need to give some thought about replacing the for-loop though.
varKeys is typically 2-4 items (credential field names like ['role_arn', 'aws.credentials.role_arn'])
Building a lookup map would add complexity and the performance difference is small for these arrays
Omolola-Akinleye
left a comment
There was a problem hiding this comment.
Looks good! Just have some feedback on naming and mainly the use of for-loops.
f4e7020 to
0f2dcb3
Compare
You can test this locally using the CSPM or Cloud Asset Discovery integrations. Credential ScopesCSPM and Cloud Asset Discovery has credentials in the input level, while the AWS integration has credentials in the package level This change is needed before we move the Cloud Connector UI to Fleet to have the CC enabled for AWS and other integration. This change also reduces the current complexity around checking for preferred credentials var names and the aliases with the prefixes. |
| * @param packageInfo - The package info containing var definitions | ||
| * @returns The storage scope ('package' or 'input') | ||
| */ | ||
| export function getCredentialStorageScope(packageInfo: PackageInfo): CloudConnectorVarStorageMode { |
There was a problem hiding this comment.
nit: suggestion on the naming getCredentialVariablesScope
There was a problem hiding this comment.
😅 I am not going to change this since its a :nit, I just changed it to getCredentialStorageScope
| const enabledInputIndex = packagePolicy.inputs.findIndex((input) => input.enabled); | ||
| if (enabledInputIndex === INVALID_INDEX) { | ||
| return { | ||
| target: { mode: 'input', inputIndex: INVALID_INDEX, streamIndex: INVALID_INDEX }, | ||
| vars: undefined, | ||
| }; | ||
| } |
There was a problem hiding this comment.
With that model It will never be possible to have multiple enabled input with cloud connector variables? I do not think a package exists with that currently
There was a problem hiding this comment.
For Cloud Connectors, I don't see how it is possible, ATM, to have multiple enabled inputs with each having their own access credentials. We can only have one cloud connector id per policy.
There was a problem hiding this comment.
I think there is a potential future case where a multi-cloud integration, such as Sentinel Cloud, might need to support Cloud Connector. Sentinel Cloud integrations already enable AWS, GCP, and Azure inputs with credentials. While this is not something we need to account for right now, it’s possible that if this use case becomes relevant in the future, we would need to refactor the Cloud Connector model to better support multi-cloud enabled inputs .
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Public APIs missing exports
Page load bundle
Unknown metric groupsAPI count
References to deprecated APIs
Unreferenced deprecated APIs
History
cc @seanrathier |
…iew_cps * commit '5f7fec57cb01883038810bd735a0666683b49904': (116 commits) [Security Solution][Attacks/Alerts][Setup and miscellaneous] Advanced setting to control feature visibility (elastic#250157) (elastic#250830) Fix synthtrace `fetch` usage (elastic#250950) [APM] Add Nodes and Edges components and selection logic (elastic#250937) [Docs] Update alerting-settings.md and add serverless value for one parameter (elastic#250842) [Agent Builder] filestore: initial implementation (elastic#250043) [CPS] Support CPS in Vega ESQL (elastic#250693) Adjustments to cascade document esql helpers (elastic#250560) [Security Solutions] Trial Companion - adds ai chat and elastic agent detectors (elastic#250908) [Obs Presentation] Code Scanning Alert Fixes (elastic#250858) [performance] add return and refresh render scenarios to dashboard journeys (elastic#250939) skip failing test suite (elastic#245458) Add Cloud Forwarder onboarding tile to O11y Solution (elastic#250325) [Traces] Remove APM unified trace waterall embeddable registration (elastic#250808) [Discover] [Metrics] Fix: metrics grid titles do not update on order change (elastic#250963) [a11y] Fix Eui modal title annoucment (elastic#250459) [Cloud Security] [Fleet] Add cloud connector access scope for input or package level credential definitions (elastic#250280) [WorkplaceAI] SharePoint Online stack connector (elastic#248737) [Response Ops][Task Manager] Update functions do not handle API key invalidation (elastic#249109) [Osquery] Remove @kbn/timelines-plugin dependency from osquery plugin (elastic#250055) [One Discover][Logs UX] Update OpenTelemetry Semantic Conventions (elastic#250346) ...
…r package level credential definitions (elastic#250280)
Summary
Implements a scope-aware solution for Cloud Connector credential variables, addressing the inconsistency where credentials could be stored at either package-level (policy.vars) or input/stream-level (policy.inputs[].streams[].vars).
Allows to easily support integration with credential aliases.
Key Changes
Test Plan
Checklist
release_note:breakinglabel should be applied in these situations.release_note:*label is applied per the guidelinesbackport:*labels.Identify risks
None