[RFC] Stage 0: Introducing new fields in ECS vulnerability field set by smriti0321 · Pull Request #2331 · elastic/ecs

smriti0321 · 2024-04-05T17:01:29Z

RFC to add new fields in ECS vulnerability field set

Have you signed the contributor license agreement?Yes
Have you followed the contributor guidelines?Yes
For proposing substantial changes or additions to the schema, have you reviewed the RFC process?Yes
If submitting code/script changes, have you verified all tests pass locally using make test?
If submitting schema/fields updates, have you generated new artifacts by running make and committed those changes?
Is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed.
Have you added an entry to the CHANGELOG.next.md?

#1685

RFC to add new fields in ECS vulnerability field set

ebeahan

Added some initial comments.

I think it's reasonable to target stage 1 or even stage 2 for this initial PR.

rfcs/0000-rfc-template.md

ebeahan · 2024-04-08T15:51:14Z

rfcs/0000-rfc-template.md

+vulnerability.firstfound	| date | First time a vulnerability was found on the asset
+vulnerability.lastfound	| date | Last time a vulnerability was found on the asset.
+vulnerability.lastscanned	date | Last time a scan was performed on the asset. It's important as some companies are scanning on a quarterly basis. If lastfound and lastscanned are close, it means it's still an active vulnerability. 
+vulnerability.age	| date | Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates. 


Suggested change

vulnerability.age | date | Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates.

vulnerability.age | long | Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates.

Based on the description, I think a numeric type is a better fit.

ebeahan · 2024-04-08T15:53:21Z

rfcs/0000-rfc-template.md

+vulnerability.lastfound	| date | Last time a vulnerability was found on the asset.
+vulnerability.lastscanned	date | Last time a scan was performed on the asset. It's important as some companies are scanning on a quarterly basis. If lastfound and lastscanned are close, it means it's still an active vulnerability. 
+vulnerability.age	| date | Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates. 
+vulnerability.uid	| keyword | It's extremely import to be able to deduplicate different scans. It's often that we have different scanners showing the same vulnerability on the same asset. 


Maybe event.id ?

I don't think event.id is good as a server A with vulnerability 1 will always have the same uid, in different scans and different vendors. This field will basically be the fingerprint of hostname + CVE ID.
I'm wondering what will happen if we have one detection affecting multiple vulnerabilities. Should this be an array or should we advice to split per CVE and not detection?

I'm wondering what will happen if we have one detection affecting multiple vulnerabilities. Should this be an array or should we advice to split per CVE and not detection?

@clement-fouque from a practitioner's perspective, what would make the most sense to you in ur day to day work? I am guessing this isn't much of an issue now since you primarily use Qualys (not multiple vendors).

@tinnytintin10 sorry I missed your comment. I believe it's an exception to have only one scanner. Usually, there are multiple scanners (e.g. network + agent).

I believe an array of ids would make the most sense. Qualys usually have one detection for multiple CVEs. If we are able to generate a fingerprint of CVE + Asset identifier (Serial number, instance ID, or hostname if the others are not available), we should be able to deduplicate vulnerabilities coming from different scanners.

clement-fouque

@smriti0321 I've added multiple comments. I'm happy to discuss and bring more details if needed.

clement-fouque · 2024-04-15T12:01:52Z

rfcs/0000-rfc-template.md

 -->

 ## Fields
+The `vulnerabilities` fields being proposed are as follows:


Why is it vulnerabilities and not vulnerability ?

rfcs/0000-rfc-template.md

clement-fouque · 2024-04-15T12:11:14Z

rfcs/0000-rfc-template.md

+vulnerability.patch.exists | boolean | The “patch” field refers to whether a security fix or update (commonly known as a patch) is available to address the identified vulnerability. It indicates whether the software vendor or developer has released a solution to mitigate the vulnerability.
+vulnerability.patch.name | text | Name of the patch
+vulnerability.patch.code | keyword | Associated patch code for example ESA-2020-13
+vulnerability.evidence	| text | A demonstration of the validity of a vulnerability claim, e.g. app.any.run replaying the exploitation of the vulnerability.


I'm not sure vulnerability.evidence will be heavily used. I think we can leverage the vulnerability.reference (or whatever the name of the field is 😉)

clement-fouque · 2024-04-15T12:12:02Z

rfcs/0000-rfc-template.md

+vulnerability.patch.name | text | Name of the patch
+vulnerability.patch.code | keyword | Associated patch code for example ESA-2020-13
+vulnerability.evidence	| text | A demonstration of the validity of a vulnerability claim, e.g. app.any.run replaying the exploitation of the vulnerability.
+vulnerability.status | keyword | The status field helps security teams track vulnerabilities, prioritize actions, and communicate their progress effectively


Should we provide examples such as: New, Active, Fixed, Ignored, ... ?

clement-fouque · 2024-04-15T12:14:54Z

rfcs/0000-rfc-template.md

+vulnerability.patch.code | keyword | Associated patch code for example ESA-2020-13
+vulnerability.evidence	| text | A demonstration of the validity of a vulnerability claim, e.g. app.any.run replaying the exploitation of the vulnerability.
+vulnerability.status | keyword | The status field helps security teams track vulnerabilities, prioritize actions, and communicate their progress effectively
+vulnerability.tags	| keyword | This is different from cloud provider assigned resource tags, this is specifically for vulnerability. Vulnerability tags serve as a way to add custom metadata to vulnerabilities, enhancing their context and aiding in search and automation. 


What would be the added value of this field compared to the default tags one? We can create specific combinaison like vulnerability:exploited. The only added value I see is for dashboarding as it makes things much easier.

clement-fouque · 2024-04-15T12:20:38Z

rfcs/0000-rfc-template.md

+vulnerability.exploitability.confidence	| keyword | Confidence measure the credibility of existence and exploitability
+vulnerability.exploitability.first_seen	| date | First time of exploitability
+vulnerability.exploitability.last_seen	| date | Last time of exploitability
+vulnerability.affected.* | object | The affected field is a JSON array containing objects that describes the affected package versions, meaning those that contain the vulnerability.


It would be interesting to provide examples on the expected format.

I'm not saying we should change but it's something to consider: with some scanners, we have this information in raw text only. We don't have it in a JSON format that we can separate.

Also, why capture it here as opposed to using the package.* fieldset?

Maybe capture it in both places? The benefit I see of capturing it using the package.* fields would be the ease in which it would enable correlations. Question for me is if we can capture an array of packages (package : [{...}, {...}, ...])

cc @oren-zohar @smriti0321

the original intention was to use the affected field somewhat like it's used in ossf, as in - indicate package versions which introduce / fix vulnerabilities.

i think we can use package.* as the vulnerable package, but it'd be nice if we also provided the patched version.

rfcs/0000-rfc-template.md

clement-fouque · 2024-04-15T12:30:44Z

rfcs/0000-rfc-template.md

+vulnerability.lastfound	| date | Last time a vulnerability was found on the asset.
+vulnerability.lastscanned	date | Last time a scan was performed on the asset. It's important as some companies are scanning on a quarterly basis. If lastfound and lastscanned are close, it means it's still an active vulnerability. 
+vulnerability.age	| date | Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates. 
+vulnerability.uid	| keyword | It's extremely import to be able to deduplicate different scans. It's often that we have different scanners showing the same vulnerability on the same asset. 


I don't think event.id is good as a server A with vulnerability 1 will always have the same uid, in different scans and different vendors. This field will basically be the fingerprint of hostname + CVE ID.
I'm wondering what will happen if we have one detection affecting multiple vulnerabilities. Should this be an array or should we advice to split per CVE and not detection?

tinnytintin10

@smriti-elastic it looks like you updated the template file directly as opposed to creating a new rfc file. Besides that, not many changes/issues. Thanks!

tinnytintin10 · 2024-05-29T22:48:59Z

rfcs/0000-rfc-template.md

 ## Fields
+The `vulnerabilities` fields being proposed are as follows:
+
+Field | Type | Description /Use Case


The table isn't rendering for me :/ feel free to use this table markdown:

Details

Field Type Description / Use Case

vulnerability.vendor.id keyword A vulnerability doesn't have necessarily a CVE associated with it. I don't know if it makes sense to separate vulnerability ID (like CVEs) to the vendor/detection IDs.

vulnerability.title keyword Title/Name/Short Description for vulnerability, to be used in flyout and dashboards.

vulnerability.mitigation text Explains how to fix or mitigate the problem. Could be used to store resolution from the scanner vendor or document mitigation in place.

vulnerability.published date The “published” field indicates the date when information about a specific vulnerability was publicly disclosed or made available. It represents the moment when details about the vulnerability were shared with the security community, vendors, and the public. This field helps security professionals track the timeline of vulnerability awareness. Example value: If a critical vulnerability (e.g., CVE-2024-1234) affecting widely used software is disclosed on February 1, 2024, the published date for that CVE would be February 1, 2024.

vulnerability.patch.* object

vulnerability.patch.exists boolean The “patch” field refers to whether a security fix or update (commonly known as a patch) is available to address the identified vulnerability. It indicates whether the software vendor or developer has released a solution to mitigate the vulnerability.

vulnerability.patch.name text Name of the patch.

vulnerability.patch.code keyword Associated patch code, for example, ESA-2020-13.

vulnerability.evidence text A demonstration of the validity of a vulnerability claim, e.g., app.any.run replaying the exploitation of the vulnerability.

vulnerability.status keyword The status field helps security teams track vulnerabilities, prioritize actions, and communicate their progress effectively.

vulnerability.tags keyword This is different from cloud provider assigned resource tags. This is specifically for vulnerability. Vulnerability tags serve as a way to add custom metadata to vulnerabilities, enhancing their context and aiding in search and automation.

vulnerability.firstfound date First time a vulnerability was found on the asset.

vulnerability.lastfound date Last time a vulnerability was found on the asset.

vulnerability.lastscanned date Last time a scan was performed on the asset. It's important as some companies are scanning on a quarterly basis. If lastfound and lastscanned are close, it means it's still an active vulnerability.

vulnerability.age date Number of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be the difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates.

vulnerability.uid keyword It's extremely important to be able to deduplicate different scans. It's often that we have different scanners showing the same vulnerability on the same asset.

vulnerability.type keyword To conclude if the vulnerability is confirmed or potential.

vulnerability.exploitability.* object

vulnerability.exploitability.exploited boolean To indicate if the vulnerability has been exploited or not.

vulnerability.exploitability.reference keyword Exploitability database, for example, CSA-KEV.

vulnerability.exploitability.confidence keyword Confidence measures the credibility of existence and exploitability.

vulnerability.exploitability.first_seen date First time of exploitability.

vulnerability.exploitability.last_seen date Last time of exploitability.

vulnerability.affected.* object The affected field is a JSON array containing objects that describe the affected package versions, meaning those that contain the vulnerability.

vulnerability.affected.package array Package field is a JSON object identifying the affected code library or command provided by the package.

vulnerability.affected.serverity array This field applies to a specific package, in cases where affected packages have differing severities for the same vulnerability.

vulnerability.affected.versions array Affected version in whatever version syntax is used by the given package ecosystem.

rfcs/0000-rfc-template.md

tinnytintin10 · 2024-05-29T23:02:52Z

rfcs/0000-rfc-template.md

+vulnerability.lastfound	| date | Last time a vulnerability was found on the asset.
+vulnerability.lastscanned	date | Last time a scan was performed on the asset. It's important as some companies are scanning on a quarterly basis. If lastfound and lastscanned are close, it means it's still an active vulnerability. 
+vulnerability.age	| date | Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates. 
+vulnerability.uid	| keyword | It's extremely import to be able to deduplicate different scans. It's often that we have different scanners showing the same vulnerability on the same asset. 


I'm wondering what will happen if we have one detection affecting multiple vulnerabilities. Should this be an array or should we advice to split per CVE and not detection?

@clement-fouque from a practitioner's perspective, what would make the most sense to you in ur day to day work? I am guessing this isn't much of an issue now since you primarily use Qualys (not multiple vendors).

tinnytintin10 · 2024-05-29T23:04:23Z

rfcs/0000-rfc-template.md

+vulnerability.exploitability.confidence	| keyword | Confidence measure the credibility of existence and exploitability
+vulnerability.exploitability.first_seen	| date | First time of exploitability
+vulnerability.exploitability.last_seen	| date | Last time of exploitability
+vulnerability.affected.* | object | The affected field is a JSON array containing objects that describes the affected package versions, meaning those that contain the vulnerability.


Also, why capture it here as opposed to using the package.* fieldset?

tinnytintin10 · 2024-05-29T23:07:08Z

rfcs/0000-rfc-template.md

+vulnerability.exploitability.confidence	| keyword | Confidence measure the credibility of existence and exploitability
+vulnerability.exploitability.first_seen	| date | First time of exploitability
+vulnerability.exploitability.last_seen	| date | Last time of exploitability
+vulnerability.affected.* | object | The affected field is a JSON array containing objects that describes the affected package versions, meaning those that contain the vulnerability.


Maybe capture it in both places? The benefit I see of capturing it using the package.* fields would be the ease in which it would enable correlations. Question for me is if we can capture an array of packages (package : [{...}, {...}, ...])

cc @oren-zohar @smriti0321

rfcs/text/0045-additional-vulnerability-fields.md

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

rfcs/text/0045-additional-vulnerability-fields.md

smriti0321 · 2024-07-23T14:37:35Z

rfcs/text/0045-additional-vulnerability-fields.md

+| `vulnerability.last_found` | date | Last time a vulnerability was found on the asset, in ISO 8601 format: 2016-05-23T08:05:34.853Z |
+| `vulnerability.last_scanned` | date | Last time a scan was performed on the asset. It's important as some companies are scanning on a quarterly basis. If lastfound and lastscanned are close, it means it's still an active vulnerability, in ISO 8601 format: 2016-05-23T08:05:34.853Z|
+| `vulnerability.age` | int | Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the first_found and last_found dates. |
+| `vulnerability.uid` | keyword | It's extremely important to be able to deduplicate different scans. It's often that we have different scanners showing the same vulnerability on the same asset. |


Resolving this comment- #2331 (comment)

rfcs/text/0045-additional-vulnerability-fields.md

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

trisch-me

lgtm.

nicpenning · 2024-08-02T19:30:23Z

Does vulnerability.exploitability.confidence == EPSS?

Tenable Nessus has now implemented CVSS 4.0 and EPSS scoring within the last week, so this is an awesome RFC to see these fields come alive!

smriti0321 · 2024-09-02T14:01:10Z

@tinnytintin10 Requested changes are taken care of, could you please approve this?

clement-fouque · 2024-09-05T12:04:39Z

Does vulnerability.exploitability.confidence == EPSS?

No, it's different. The confidence is tied to the trust that we have in the source of the information. For EPSS, we can debate but I would say high as it’s a well known source that is gathering data from other providers.

But if it’s just a Twitter account claiming a specific vulnerability is exploited, the confidence will be low as we are not able to corroborate this information.

I think I having 4 levels is good and enough: Low, Medium, High, Very high (or something similar).

nicpenning · 2024-09-05T12:37:43Z

That makes sense. Where would EPSS live then? A new field?

clement-fouque · 2024-09-05T19:42:56Z

That makes sense. Where would EPSS live then? A new field?

It's a different scoring system than CVSS (which have several versions). For each vulnerability, we should have multiple scoring systems. I'm not sure about the best way to represent it, an array is probably good.

FYI, this is how Qualys is representing those: https://github.com/elastic/integrations/blob/b698afea8af60f3fd52978813bc740681764dbba/packages/qualys_vmdr/data_stream/asset_host_detection/sample_event.json#L68-L85

tinnytintin10

LGTM

@trisch-me

* Add .caseless subfield to process.name & process.executable (elastic#2341) Adds a subfield to the process.name and process.executable fields to improve the compatibility of data sources like System, Sysmon, etc., with our Elastic Defend data, which enables us to handle language limitations in KQL more effectively. * Revert "Add .caseless subfield to process.name & process.executable" (elastic#2350) This reverts commit 7815b3f from elastic#2341. This is being reverted due to storage concerns. The goal will be to advance the native querying capabilities (ES|QL, KQL) of the Elastic stack such that this extra normalized multi-field is not necessary. In the meantime, localized overrides of the ECS field definition will be used to add the additional multi-field where needed. The downside of localized overrides are that it creates inconsistency across usages of the this field. * [RFC] Apple Platform specific fields (elastic#2338) Adds RFS stage 0 --------- Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Add renovate.json (elastic#2352) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update template fields (elastic#2354) Update some templated fields that were missed before merging the RFC * Pin dependencies (elastic#2355) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency PyYAML to v6.0.2 (elastic#2356) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency gitpython to v3.1.43 (elastic#2358) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency yamllint to v1.35.1 (elastic#2361) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update stale PR message (elastic#2369) Add a friendlier stale PR message, based from the [Beats stale message](https://github.com/elastic/beats/blob/main/.github/stale.yml#L63-L74). This will hopefully also prompt contributors to respond, so we'll be better able to track PRs people are still interested in contributing. * Update actions/checkout action to v4 (elastic#2362) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update actions/github-script action to v7 (elastic#2363) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update actions/setup-python action to v5 (elastic#2364) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update actions/stale action to v9 (elastic#2365) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency mock to v5 (elastic#2367) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency ubuntu to v22 (elastic#2368) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency autopep8 to v1.7.0 (elastic#2359) Update dependency autopep8 to v1.7.0 --------- Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency autopep8 to v2 (elastic#2366) * Update dependency autopep8 to v2 --------- Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * add license header (elastic#2377) * Update actions/setup-python digest to f677139 (elastic#2374) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * [RFC] Stage 0: Introducing new field in rule namespace (elastic#2330) * Update 0000-rfc-template.md Updating the temaplate for RFC Stage 0 for adding 2 new rule fields: rule.tags and rule.remediation * Update 0000-rfc-template.md Incorporating review comments. * Renaming the template file with recommended name * Resolving conflicts * Removing Tag Field * Resolving comments from @trisch-me * Moving file to rfcs/text folder as per @trisch-me comment. using next number in series. * I saw number 44 was used in a recent RFC, using next number in series --------- Co-authored-by: Eric Beahan <eric.beahan@elastic.co> Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> * [RFC] Stage 2: Adding Apple Platform specific fields (elastic#2370) Updating the RFC and moving it to stage two. * code blocks specified language yaml (elastic#2380) Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * trim trailing whitespace in schema (elastic#2379) Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * [RFC] Stage 0: Introducing new fields in ECS vulnerability field set (elastic#2331) * RFC to add new fields in ECS vulnerability field set RFC to add new fields in ECS vulnerability field set * Moving to separate file * set title and add stage 0 PR # * clean up fields table markdown * Moving to (rfcs/text) and renaming file to next number in series. * Resolving the comments from @trisch-me * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> * Making changed to the date format as per comments from @trisch-me * Resolving @trisch-me comments * Resolving latest comments * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> --------- Co-authored-by: Eric Beahan <eric.beahan@elastic.co> Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> * Fix type in code signature (elastic#2382) Change the type of code_signature.flags to keyword, which is what it should be. Also add a unit test that will verify all types are valid. * Enforce yamllint in CI (elastic#2381) Start running and enforcing yamllint checks in CI. * Add Stage0 RFC for new fields for fileless execution on Linux (elastic#2322) * Add support for settings * Fix settings merging * Restrict test workflow * Fix merge conflicts * Less restrictive * Add docker files and pipeline * Make building more restrictive * Simplify build workflow * Update tagging strategy * Removing unused variable * Kick? * Anchors aren't supported 😭 * Fix role name * Test branch name * Remove extra default update (#3) * Remove extra default update * Fix role name * Add support for a top-level type (#4) * Add support for a top-level type * Actually, don't need to be all the complicated * Type needs to be nested within the field name (#5) * Add documention for parameters field (#6) * Add undocumented field argument * Remove the PR template --------- Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com> Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co> Co-authored-by: Thijs Xhaflaire <thijsxhaflaire31@hotmail.com> Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Stefan Bischof <bipolis@bipolis.org> Co-authored-by: Smriti <152067238+smriti0321@users.noreply.github.com> Co-authored-by: Eric Beahan <eric.beahan@elastic.co> Co-authored-by: Michal Stanek <75310947+stanek-michal@users.noreply.github.com>

@trisch-me

* Add .caseless subfield to process.name & process.executable (elastic#2341) Adds a subfield to the process.name and process.executable fields to improve the compatibility of data sources like System, Sysmon, etc., with our Elastic Defend data, which enables us to handle language limitations in KQL more effectively. * Revert "Add .caseless subfield to process.name & process.executable" (elastic#2350) This reverts commit 7815b3f from elastic#2341. This is being reverted due to storage concerns. The goal will be to advance the native querying capabilities (ES|QL, KQL) of the Elastic stack such that this extra normalized multi-field is not necessary. In the meantime, localized overrides of the ECS field definition will be used to add the additional multi-field where needed. The downside of localized overrides are that it creates inconsistency across usages of the this field. * [RFC] Apple Platform specific fields (elastic#2338) Adds RFS stage 0 --------- Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Add renovate.json (elastic#2352) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update template fields (elastic#2354) Update some templated fields that were missed before merging the RFC * Pin dependencies (elastic#2355) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency PyYAML to v6.0.2 (elastic#2356) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency gitpython to v3.1.43 (elastic#2358) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency yamllint to v1.35.1 (elastic#2361) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update stale PR message (elastic#2369) Add a friendlier stale PR message, based from the [Beats stale message](https://github.com/elastic/beats/blob/main/.github/stale.yml#L63-L74). This will hopefully also prompt contributors to respond, so we'll be better able to track PRs people are still interested in contributing. * Update actions/checkout action to v4 (elastic#2362) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update actions/github-script action to v7 (elastic#2363) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update actions/setup-python action to v5 (elastic#2364) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update actions/stale action to v9 (elastic#2365) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency mock to v5 (elastic#2367) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency ubuntu to v22 (elastic#2368) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency autopep8 to v1.7.0 (elastic#2359) Update dependency autopep8 to v1.7.0 --------- Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * Update dependency autopep8 to v2 (elastic#2366) * Update dependency autopep8 to v2 --------- Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * add license header (elastic#2377) * Update actions/setup-python digest to f677139 (elastic#2374) Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * [RFC] Stage 0: Introducing new field in rule namespace (elastic#2330) * Update 0000-rfc-template.md Updating the temaplate for RFC Stage 0 for adding 2 new rule fields: rule.tags and rule.remediation * Update 0000-rfc-template.md Incorporating review comments. * Renaming the template file with recommended name * Resolving conflicts * Removing Tag Field * Resolving comments from @trisch-me * Moving file to rfcs/text folder as per @trisch-me comment. using next number in series. * I saw number 44 was used in a recent RFC, using next number in series --------- Co-authored-by: Eric Beahan <eric.beahan@elastic.co> Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> * [RFC] Stage 2: Adding Apple Platform specific fields (elastic#2370) Updating the RFC and moving it to stage two. * code blocks specified language yaml (elastic#2380) Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * trim trailing whitespace in schema (elastic#2379) Co-authored-by: Michael Wolf <michael.wolf@elastic.co> * [RFC] Stage 0: Introducing new fields in ECS vulnerability field set (elastic#2331) * RFC to add new fields in ECS vulnerability field set RFC to add new fields in ECS vulnerability field set * Moving to separate file * set title and add stage 0 PR # * clean up fields table markdown * Moving to (rfcs/text) and renaming file to next number in series. * Resolving the comments from @trisch-me * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> * Making changed to the date format as per comments from @trisch-me * Resolving @trisch-me comments * Resolving latest comments * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> --------- Co-authored-by: Eric Beahan <eric.beahan@elastic.co> Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> * Fix type in code signature (elastic#2382) Change the type of code_signature.flags to keyword, which is what it should be. Also add a unit test that will verify all types are valid. * Enforce yamllint in CI (elastic#2381) Start running and enforcing yamllint checks in CI. * Add Stage0 RFC for new fields for fileless execution on Linux (elastic#2322) * Add support for settings * Fix settings merging * Restrict test workflow * Fix merge conflicts * Less restrictive * Add docker files and pipeline * Make building more restrictive * Simplify build workflow * Update tagging strategy * Removing unused variable * Kick? * Anchors aren't supported 😭 * Fix role name * Test branch name * Remove extra default update (#3) * Remove extra default update * Fix role name * Add support for a top-level type (#4) * Add support for a top-level type * Actually, don't need to be all the complicated * Type needs to be nested within the field name (#5) * Add documention for parameters field (#6) * Add undocumented field argument * Remove the PR template --------- Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com> Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co> Co-authored-by: Thijs Xhaflaire <thijsxhaflaire31@hotmail.com> Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co> Co-authored-by: Michael Wolf <michael.wolf@elastic.co> Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com> Co-authored-by: Stefan Bischof <bipolis@bipolis.org> Co-authored-by: Smriti <152067238+smriti0321@users.noreply.github.com> Co-authored-by: Eric Beahan <eric.beahan@elastic.co> Co-authored-by: Michal Stanek <75310947+stanek-michal@users.noreply.github.com>

RFC to add new fields in ECS vulnerability field set

81eed70

RFC to add new fields in ECS vulnerability field set

smriti0321 requested a review from a team as a code owner April 5, 2024 17:01

ebeahan reviewed Apr 8, 2024

View reviewed changes

smriti0321 requested review from clement-fouque, oren-zohar, orouz and tinnytintin10 April 9, 2024 08:09

clement-fouque self-assigned this Apr 15, 2024

clement-fouque reviewed Apr 15, 2024

View reviewed changes

orouz mentioned this pull request May 7, 2024

Adopt ECS fields for Findings and Vulnerabilities elastic/cloudbeat#2187

Open

1 task

tinnytintin10 requested changes May 29, 2024

View reviewed changes

ebeahan added 4 commits June 26, 2024 15:02

Moving to separate file

86282bb

Merge branch 'main' into vulnerability-fields

c46c24d

set title and add stage 0 PR #

de65f45

clean up fields table markdown

26b1ae8

smriti0321 requested a review from tinnytintin10 July 2, 2024 15:07

Moving to (rfcs/text) and renaming file to next number in series.

9ef32b4

smriti0321 requested a review from trisch-me July 4, 2024 13:23

trisch-me reviewed Jul 15, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 15, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 15, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 15, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 15, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 15, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 15, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

smriti0321 and others added 3 commits July 16, 2024 14:02

Resolving the comments from @trisch-me

70310cc

Update rfcs/text/0045-additional-vulnerability-fields.md

4f36f2b

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

Update rfcs/text/0045-additional-vulnerability-fields.md

8082829

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

smriti0321 requested a review from trisch-me July 16, 2024 12:04

trisch-me reviewed Jul 17, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 17, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 17, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me and others added 2 commits July 17, 2024 14:16

Merge branch 'main' into vulnerability-fields

290dd36

Making changed to the date format as per comments from @trisch-me

cc6731c

smriti0321 requested a review from trisch-me July 18, 2024 12:08

trisch-me reviewed Jul 23, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

trisch-me reviewed Jul 23, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

Resolving @trisch-me comments

6515f59

smriti0321 requested a review from trisch-me July 23, 2024 13:39

smriti0321 commented Jul 23, 2024

View reviewed changes

Resolving latest comments

7028a9e

trisch-me reviewed Jul 23, 2024

View reviewed changes

rfcs/text/0045-additional-vulnerability-fields.md Outdated Show resolved Hide resolved

Update rfcs/text/0045-additional-vulnerability-fields.md

b13209c

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

trisch-me approved these changes Jul 23, 2024

View reviewed changes

norrietaylor mentioned this pull request Aug 2, 2024

Extend Vulnerability Fieldset #1685

Open

Merge branch 'main' into vulnerability-fields

5c24d41

tinnytintin10 approved these changes Sep 13, 2024

View reviewed changes

Merge branch 'main' into vulnerability-fields

4e35b30

smriti0321 merged commit 8be4ed7 into elastic:main Sep 17, 2024

lksnyder0 mentioned this pull request Oct 9, 2024

elastic main huntresslabs/ecs#8

Merged

	vulnerability.age \| date \| Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates.
	vulnerability.age \| long \| Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates.

Field	Type	Description / Use Case
vulnerability.vendor.id	keyword	A vulnerability doesn't have necessarily a CVE associated with it. I don't know if it makes sense to separate vulnerability ID (like CVEs) to the vendor/detection IDs.
vulnerability.title	keyword	Title/Name/Short Description for vulnerability, to be used in flyout and dashboards.
vulnerability.mitigation	text	Explains how to fix or mitigate the problem. Could be used to store resolution from the scanner vendor or document mitigation in place.
vulnerability.published	date	The “published” field indicates the date when information about a specific vulnerability was publicly disclosed or made available. It represents the moment when details about the vulnerability were shared with the security community, vendors, and the public. This field helps security professionals track the timeline of vulnerability awareness. Example value: If a critical vulnerability (e.g., CVE-2024-1234) affecting widely used software is disclosed on February 1, 2024, the published date for that CVE would be February 1, 2024.
vulnerability.patch.*	object
vulnerability.patch.exists	boolean	The “patch” field refers to whether a security fix or update (commonly known as a patch) is available to address the identified vulnerability. It indicates whether the software vendor or developer has released a solution to mitigate the vulnerability.
vulnerability.patch.name	text	Name of the patch.
vulnerability.patch.code	keyword	Associated patch code, for example, ESA-2020-13.
vulnerability.evidence	text	A demonstration of the validity of a vulnerability claim, e.g., app.any.run replaying the exploitation of the vulnerability.
vulnerability.status	keyword	The status field helps security teams track vulnerabilities, prioritize actions, and communicate their progress effectively.
vulnerability.tags	keyword	This is different from cloud provider assigned resource tags. This is specifically for vulnerability. Vulnerability tags serve as a way to add custom metadata to vulnerabilities, enhancing their context and aiding in search and automation.
vulnerability.firstfound	date	First time a vulnerability was found on the asset.
vulnerability.lastfound	date	Last time a vulnerability was found on the asset.
vulnerability.lastscanned	date	Last time a scan was performed on the asset. It's important as some companies are scanning on a quarterly basis. If lastfound and lastscanned are close, it means it's still an active vulnerability.
vulnerability.age	date	Number of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be the difference between the lastfound date and the published date (preferred). It could also be the difference between the firstfound and lastfound dates.
vulnerability.uid	keyword	It's extremely important to be able to deduplicate different scans. It's often that we have different scanners showing the same vulnerability on the same asset.
vulnerability.type	keyword	To conclude if the vulnerability is confirmed or potential.
vulnerability.exploitability.*	object
vulnerability.exploitability.exploited	boolean	To indicate if the vulnerability has been exploited or not.
vulnerability.exploitability.reference	keyword	Exploitability database, for example, CSA-KEV.
vulnerability.exploitability.confidence	keyword	Confidence measures the credibility of existence and exploitability.
vulnerability.exploitability.first_seen	date	First time of exploitability.
vulnerability.exploitability.last_seen	date	Last time of exploitability.
vulnerability.affected.*	object	The affected field is a JSON array containing objects that describe the affected package versions, meaning those that contain the vulnerability.
vulnerability.affected.package	array	Package field is a JSON object identifying the affected code library or command provided by the package.
vulnerability.affected.serverity	array	This field applies to a specific package, in cases where affected packages have differing severities for the same vulnerability.
vulnerability.affected.versions	array	Affected version in whatever version syntax is used by the given package ecosystem.

Conversation

smriti0321 commented Apr 5, 2024 • edited by norrietaylor Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ebeahan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clement-fouque left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tinnytintin10 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

trisch-me left a comment

Choose a reason for hiding this comment

Uh oh!

nicpenning commented Aug 2, 2024

smriti0321 commented Apr 5, 2024 •

edited by norrietaylor

Loading