[Security Solution][Detections] Update the MITRE ATT&CK model to v11.3 by banderror · Pull Request #137122 · elastic/kibana

banderror · 2022-07-25T21:22:51Z

Related to: elastic/detection-rules#2073 (comment), #89876

Summary

Here we regenerate the MITRE ATT&CK model in the code based on the official MITRE content:

we update to the version ATT&CK-v11.3 (see [Rule Tuning] Missing MITRE ATT&CK Mappings detection-rules#2073 (comment))
this corresponds to the https://raw.githubusercontent.com/mitre/cti/ATT&CK-v11.3/enterprise-attack/enterprise-attack.json content

Also, this PR fixes the model regeneration script (check the comment below).

elasticmachine · 2022-07-25T21:22:54Z

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine · 2022-07-25T21:22:54Z

Pinging @elastic/security-solution (Team: SecuritySolution)

banderror · 2022-07-25T21:28:07Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

This script didn't work out of the box and I have to add this condition to it:

|| obj.x_mitre_is_subtechnique === undefined

because it looks like the official metadata has been changed since we used the script last time, and now a lot of techniques don't have x_mitre_is_subtechnique field in them.

banderror · 2022-07-25T21:32:10Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

Instead of finding the parent tactic by doing text transformations:

const tactic = tactics.find( (tactic) => tactic.name === startCase(camelCase(technique.tactics[0])) );

we can leverage the tactic's short name that's available in the source json metadata.

I think this is a more robust way to do that.

dplumlee

lgtm!

kibana-ci · 2022-07-26T11:44:57Z

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	5.5MB	5.5MB	+43.6KB

History

💚 Build #60621 succeeded 658267f3cbe6a5c70192dce03bad628c6e2af2d0
💔 Build #60614 failed 4b21b01bfb111dd66fa41dfb7af76be8b1fba793

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @banderror

xcrzx · 2022-07-26T13:13:23Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+// Every release we should update the version of MITRE ATT&CK content and regenerate the model in our code.
+// This version must correspond to the one used for prebuilt rules in https://github.com/elastic/detection-rules.
+// This version is basically a tag on https://github.com/mitre/cti/tags, or can be a branch name like `master`.


Thanks for clarifying that 👍

@xcrzx Yeah, sure! Don't think this is enough though. Ideally, we need to make this responsibility visible in our GH board and checklists and have a recurring ticket for it or try to automate it somehow as suggested in #89876.

vitaliidm

thanks @banderror for improving extract script

Left a couple of comments

vitaliidm · 2022-07-26T14:36:54Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+  const tactics = mitreData
+    .filter((obj) => obj.type === 'x-mitre-tactic')
+    .reduce((acc, item) => {
+      const { id, reference } = getIdReference(item.external_references);


if id and reference are empty. Do we still want to add item in tactics array?

No, I don't think we want it. I have a bunch of concerns about reliability of this script. But I don't want to make any further changes in this PR to not accidentally break anything. Opening a ticket for follow-up changes would be a better option IMHO.

vitaliidm · 2022-07-26T14:59:30Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+      let tactics = [];
+      const { id, reference } = getIdReference(item.external_references);
+      if (item.kill_chain_phases != null && item.kill_chain_phases.length > 0) {
+        item.kill_chain_phases.forEach((tactic) => {
+          tactics = [...tactics, tactic.phase_name];
+        });
      }


looks like can be extracted to another method, as it identical to this

Let's address it as part of a follow-up ticket. I can create one.

vitaliidm · 2022-07-26T15:05:31Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+    .filter((obj) => obj.type === 'x-mitre-tactic')
+    .reduce((acc, item) => {


nit: by moving filter condition inside reduce, could save one traverse through tactics

Let's address it as part of a follow-up ticket.

terrancedejesus

Great work and thank you for being so swift!

#151931) ## Summary Updates MITRE ATT&CK mappings to `v12.1`, see `detection-rules` repo update here: elastic/detection-rules#2422. Last update was to `v11.3` in #137122. To update, I modified https://github.com/elastic/kibana/blob/1a19148c1818b9af3b7735a0b6001bbb6bd8d7ba/x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js#L22 to point to the `ATT&CK-v12.1` tag. Then ran `yarn extract-mitre-attacks` from the root `security_solution` plugin directory, and then `node scripts/i18n_check.js --fix` from Kibana root to regen the i18n files. ### Checklist - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)

elastic#151931) ## Summary Updates MITRE ATT&CK mappings to `v12.1`, see `detection-rules` repo update here: elastic/detection-rules#2422. Last update was to `v11.3` in elastic#137122. To update, I modified https://github.com/elastic/kibana/blob/1a19148c1818b9af3b7735a0b6001bbb6bd8d7ba/x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js#L22 to point to the `ATT&CK-v12.1` tag. Then ran `yarn extract-mitre-attacks` from the root `security_solution` plugin directory, and then `node scripts/i18n_check.js --fix` from Kibana root to regen the i18n files. ### Checklist - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) (cherry picked from commit bbfa43a)

…o v12.1 (#151931) (#152006) # Backport This will backport the following commits from `main` to `8.6`: - [[Security Solution][Detections] Updates MITRE ATT&CK mappings to v12.1 (#151931)](#151931)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Garrett Spong <spong@users.noreply.github.com>

…o v12.1 (#151931) (#152007) # Backport This will backport the following commits from `main` to `8.7`: - [[Security Solution][Detections] Updates MITRE ATT&CK mappings to v12.1 (#151931)](#151931)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Garrett Spong <spong@users.noreply.github.com>

banderror requested a review from a team as a code owner July 25, 2022 21:22

banderror self-assigned this Jul 25, 2022

banderror requested a review from a team as a code owner July 25, 2022 21:22

banderror requested a review from vitaliidm July 25, 2022 21:22

banderror commented Jul 25, 2022

View reviewed changes

banderror requested review from Mikaayenson, dplumlee and terrancedejesus July 25, 2022 21:32

banderror mentioned this pull request Jul 25, 2022

[Rule Tuning] Missing MITRE ATT&CK Mappings elastic/detection-rules#2073

Merged

dplumlee approved these changes Jul 25, 2022

View reviewed changes

banderror added 3 commits July 26, 2022 12:35

Fix the script and regenerate the model

4241c4d

Remove unused keys from the localization dictionaries

c9f8c0c

Regenerate the model based on ATT&CK-v11.3

1bda53f

banderror force-pushed the update-mitre-attack-model branch from 658267f to 1bda53f Compare July 26, 2022 10:36

banderror changed the title ~~[Security Solution][Detections] Update the MITRE ATT&CK model~~ [Security Solution][Detections] Update the MITRE ATT&CK model to v11.3 Jul 26, 2022

xcrzx approved these changes Jul 26, 2022

View reviewed changes

vitaliidm reviewed Jul 26, 2022

View reviewed changes

terrancedejesus approved these changes Jul 27, 2022

View reviewed changes

banderror merged commit cc634ed into elastic:main Jul 27, 2022

kibanamachine added the backport:skip This PR does not require backporting label Jul 27, 2022

banderror deleted the update-mitre-attack-model branch July 27, 2022 13:20

spong mentioned this pull request Feb 22, 2023

[Security Solution][Detections] Updates MITRE ATT&CK mappings to v12.1 #151931

Merged

1 task

banderror mentioned this pull request Sep 12, 2023

[Security Solution] Update MITRE ATT&CK mappings to the next version #166152

Open

		.filter((obj) => obj.type === 'x-mitre-tactic')
		.reduce((acc, item) => {

Conversation

banderror commented Jul 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

elasticmachine commented Jul 25, 2022

Uh oh!

elasticmachine commented Jul 25, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dplumlee left a comment

Choose a reason for hiding this comment

Uh oh!

kibana-ci commented Jul 26, 2022

💚 Build Succeeded

Metrics [docs]

Async chunks

History

Uh oh!

Choose a reason for hiding this comment

Uh oh!

banderror Jul 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vitaliidm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

terrancedejesus left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

banderror commented Jul 25, 2022 •

edited

Loading

banderror Jul 27, 2022 •

edited

Loading