Skip to content

[7.x] [i18n] remove i18n html extractor (#115004)#115596

Merged
kibanamachine merged 1 commit intoelastic:7.xfrom
kibanamachine:backport/7.x/pr-115004
Oct 19, 2021
Merged

[7.x] [i18n] remove i18n html extractor (#115004)#115596
kibanamachine merged 1 commit intoelastic:7.xfrom
kibanamachine:backport/7.x/pr-115004

Conversation

@kibanamachine
Copy link
Copy Markdown
Contributor

Backports the following commits to 7.x:

@kibanamachine kibanamachine added the backport This PR is a backport of another PR label Oct 19, 2021
@kibanamachine kibanamachine enabled auto-merge (squash) October 19, 2021 16:08
@kibanamachine
Copy link
Copy Markdown
Contributor Author

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / general / X-Pack Detection Engine API Integration Tests.x-pack/test/detection_engine_api_integration/security_and_spaces/tests/create_threat_matching·ts.detection engine api security and spaces enabled create_threat_matching tests with auditbeat data should be able to execute and get 10 signals when doing a specific query

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]     │
[00:00:00]       └-: detection engine api security and spaces enabled
[00:00:00]         └-> "before all" hook in "detection engine api security and spaces enabled"
[00:00:00]         └-: 
[00:00:00]           └-> "before all" hook in ""
[00:10:11]           └-: create_threat_matching
[00:10:11]             └-> "before all" hook in "create_threat_matching"
[00:10:26]             └-: tests with auditbeat data
[00:10:26]               └-> "before all" hook for "should be able to execute and get 10 signals when doing a specific query"
[00:10:26]               └-> "before all" hook for "should be able to execute and get 10 signals when doing a specific query"
[00:10:26]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Loading "mappings.json"
[00:10:26]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Loading "data.json.gz"
[00:10:26]                 │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [auditbeat-8.0.0-2019.02.19-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:10:26]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Created index "auditbeat-8.0.0-2019.02.19-000001"
[00:10:26]                 │ debg [x-pack/test/functional/es_archives/auditbeat/hosts] "auditbeat-8.0.0-2019.02.19-000001" settings {"index":{"lifecycle":{"name":"auditbeat-8.0.0","rollover_alias":"auditbeat-8.0.0"},"mapping":{"total_fields":{"limit":"10000"}},"number_of_replicas":"1","number_of_shards":"1","query":{"default_field":["tags","message","agent.version","agent.name","agent.type","agent.id","agent.ephemeral_id","client.address","client.mac","client.domain","client.geo.continent_name","client.geo.country_name","client.geo.region_name","client.geo.city_name","client.geo.country_iso_code","client.geo.region_iso_code","client.geo.name","cloud.provider","cloud.availability_zone","cloud.region","cloud.instance.id","cloud.instance.name","cloud.machine.type","cloud.account.id","container.runtime","container.id","container.image.name","container.image.tag","container.name","destination.address","destination.mac","destination.domain","destination.geo.continent_name","destination.geo.country_name","destination.geo.region_name","destination.geo.city_name","destination.geo.country_iso_code","destination.geo.region_iso_code","destination.geo.name","ecs.version","error.id","error.message","error.code","event.id","event.kind","event.category","event.action","event.outcome","event.type","event.module","event.dataset","event.hash","event.timezone","file.path","file.target_path","file.extension","file.type","file.device","file.inode","file.uid","file.owner","file.gid","file.group","file.mode","group.id","group.name","host.hostname","host.name","host.id","host.mac","host.type","host.architecture","host.os.platform","host.os.name","host.os.full","host.os.family","host.os.version","host.os.kernel","host.geo.continent_name","host.geo.country_name","host.geo.region_name","host.geo.city_name","host.geo.country_iso_code","host.geo.region_iso_code","host.geo.name","http.request.method","http.request.body.content","http.request.referrer","http.response.body.content","http.version","log.level","network.name","network.type","network.iana_number","network.transport","network.application","network.protocol","network.direction","network.community_id","observer.mac","observer.hostname","observer.vendor","observer.version","observer.serial_number","observer.type","observer.os.platform","observer.os.name","observer.os.full","observer.os.family","observer.os.version","observer.os.kernel","observer.geo.continent_name","observer.geo.country_name","observer.geo.region_name","observer.geo.city_name","observer.geo.country_iso_code","observer.geo.region_iso_code","observer.geo.name","organization.name","organization.id","os.platform","os.name","os.full","os.family","os.version","os.kernel","process.name","process.args","process.executable","process.title","process.working_directory","server.address","server.mac","server.domain","server.geo.continent_name","server.geo.country_name","server.geo.region_name","server.geo.city_name","server.geo.country_iso_code","server.geo.region_iso_code","server.geo.name","service.id","service.name","service.type","service.state","service.version","service.ephemeral_id","source.address","source.mac","source.domain","source.geo.continent_name","source.geo.country_name","source.geo.region_name","source.geo.city_name","source.geo.country_iso_code","source.geo.region_iso_code","source.geo.name","url.original","url.full","url.scheme","url.domain","url.path","url.query","url.fragment","url.username","url.password","user.id","user.name","user.full_name","user.email","user.hash","user.group.id","user.group.name","user_agent.original","user_agent.name","user_agent.version","user_agent.device.name","user_agent.os.platform","user_agent.os.name","user_agent.os.full","user_agent.os.family","user_agent.os.version","user_agent.os.kernel","agent.hostname","error.type","cloud.project.id","kubernetes.pod.name","kubernetes.pod.uid","kubernetes.namespace","kubernetes.node.name","kubernetes.container.name","kubernetes.container.image","file.origin","raw","file.selinux.user","file.selinux.role","file.selinux.domain","file.selinux.level","user.audit.id","user.audit.name","user.effective.id","user.effective.name","user.effective.group.id","user.effective.group.name","user.filesystem.id","user.filesystem.name","user.filesystem.group.id","user.filesystem.group.name","user.saved.id","user.saved.name","user.saved.group.id","user.saved.group.name","user.selinux.user","user.selinux.role","user.selinux.domain","user.selinux.level","user.selinux.category","source.path","destination.path","auditd.message_type","auditd.session","auditd.result","auditd.summary.actor.primary","auditd.summary.actor.secondary","auditd.summary.object.type","auditd.summary.object.primary","auditd.summary.object.secondary","auditd.summary.how","auditd.paths.inode","auditd.paths.dev","auditd.paths.obj_user","auditd.paths.obj_role","auditd.paths.obj_domain","auditd.paths.obj_level","auditd.paths.objtype","auditd.paths.ouid","auditd.paths.rdev","auditd.paths.nametype","auditd.paths.ogid","auditd.paths.item","auditd.paths.mode","auditd.paths.name","auditd.data.action","auditd.data.minor","auditd.data.acct","auditd.data.addr","auditd.data.cipher","auditd.data.id","auditd.data.entries","auditd.data.kind","auditd.data.ksize","auditd.data.spid","auditd.data.arch","auditd.data.argc","auditd.data.major","auditd.data.unit","auditd.data.table","auditd.data.terminal","auditd.data.grantors","auditd.data.direction","auditd.data.op","auditd.data.tty","auditd.data.syscall","auditd.data.data","auditd.data.family","auditd.data.mac","auditd.data.pfs","auditd.data.items","auditd.data.a0","auditd.data.a1","auditd.data.a2","auditd.data.a3","auditd.data.hostname","auditd.data.lport","auditd.data.rport","auditd.data.exit","auditd.data.fp","auditd.data.laddr","auditd.data.sport","auditd.data.capability","auditd.data.nargs","auditd.data.new-enabled","auditd.data.audit_backlog_limit","auditd.data.dir","auditd.data.cap_pe","auditd.data.model","auditd.data.new_pp","auditd.data.old-enabled","auditd.data.oauid","auditd.data.old","auditd.data.banners","auditd.data.feature","auditd.data.vm-ctx","auditd.data.opid","auditd.data.seperms","auditd.data.seresult","auditd.data.new-rng","auditd.data.old-net","auditd.data.sigev_signo","auditd.data.ino","auditd.data.old_enforcing","auditd.data.old-vcpu","auditd.data.range","auditd.data.res","auditd.data.added","auditd.data.fam","auditd.data.nlnk-pid","auditd.data.subj","auditd.data.a[0-3]","auditd.data.cgroup","auditd.data.kernel","auditd.data.ocomm","auditd.data.new-net","auditd.data.permissive","auditd.data.class","auditd.data.compat","auditd.data.fi","auditd.data.changed","auditd.data.msg","auditd.data.dport","auditd.data.new-seuser","auditd.data.invalid_context","auditd.data.dmac","auditd.data.ipx-net","auditd.data.iuid","auditd.data.macproto","auditd.data.obj","auditd.data.ipid","auditd.data.new-fs","auditd.data.vm-pid","auditd.data.cap_pi","auditd.data.old-auid","auditd.data.oses","auditd.data.fd","auditd.data.igid","auditd.data.new-disk","auditd.data.parent","auditd.data.len","auditd.data.oflag","auditd.data.uuid","auditd.data.code","auditd.data.nlnk-grp","auditd.data.cap_fp","auditd.data.new-mem","auditd.data.seperm","auditd.data.enforcing","auditd.data.new-chardev","auditd.data.old-rng","auditd.data.outif","auditd.data.cmd","auditd.data.hook","auditd.data.new-level","auditd.data.sauid","auditd.data.sig","auditd.data.audit_backlog_wait_time","auditd.data.printer","auditd.data.old-mem","auditd.data.perm","auditd.data.old_pi","auditd.data.state","auditd.data.format","auditd.data.new_gid","auditd.data.tcontext","auditd.data.maj","auditd.data.watch","auditd.data.device","auditd.data.grp","auditd.data.bool","auditd.data.icmp_type","auditd.data.new_lock","auditd.data.old_prom","auditd.data.acl","auditd.data.ip","auditd.data.new_pi","auditd.data.default-context","auditd.data.inode_gid","auditd.data.new-log_passwd","auditd.data.new_pe","auditd.data.selected-context","auditd.data.cap_fver","auditd.data.file","auditd.data.net","auditd.data.virt","auditd.data.cap_pp","auditd.data.old-range","auditd.data.resrc","auditd.data.new-range","auditd.data.obj_gid","auditd.data.proto","auditd.data.old-disk","auditd.data.audit_failure","auditd.data.inif","auditd.data.vm","auditd.data.flags","auditd.data.nlnk-fam","auditd.data.old-fs","auditd.data.old-ses","auditd.data.seqno","auditd.data.fver","auditd.data.qbytes","auditd.data.seuser","auditd.data.cap_fe","auditd.data.new-vcpu","auditd.data.old-level","auditd.data.old_pp","auditd.data.daddr","auditd.data.old-role","auditd.data.ioctlcmd","auditd.data.smac","auditd.data.apparmor","auditd.data.fe","auditd.data.perm_mask","auditd.data.ses","auditd.data.cap_fi","auditd.data.obj_uid","auditd.data.reason","auditd.data.list","auditd.data.old_lock","auditd.data.bus","auditd.data.old_pe","auditd.data.new-role","auditd.data.prom","auditd.data.uri","auditd.data.audit_enabled","auditd.data.old-log_passwd","auditd.data.old-seuser","auditd.data.per","auditd.data.scontext","auditd.data.tclass","auditd.data.ver","auditd.data.new","auditd.data.val","auditd.data.img-ctx","auditd.data.old-chardev","auditd.data.old_val","auditd.data.success","auditd.data.inode_uid","auditd.data.removed","auditd.data.socket.port","auditd.data.socket.saddr","auditd.data.socket.addr","auditd.data.socket.family","auditd.data.socket.path","geoip.continent_name","geoip.city_name","geoip.region_name","geoip.country_iso_code","hash.blake2b_256","hash.blake2b_384","hash.blake2b_512","hash.md5","hash.sha1","hash.sha224","hash.sha256","hash.sha384","hash.sha3_224","hash.sha3_256","hash.sha3_384","hash.sha3_512","hash.sha512","hash.sha512_224","hash.sha512_256","hash.xxh64","event.origin","user.entity_id","user.terminal","process.entity_id","socket.entity_id","system.audit.host.timezone.name","system.audit.host.hostname","system.audit.host.id","system.audit.host.architecture","system.audit.host.mac","system.audit.host.os.platform","system.audit.host.os.name","system.audit.host.os.family","system.audit.host.os.version","system.audit.host.os.kernel","system.audit.package.entity_id","system.audit.package.name","system.audit.package.version","system.audit.package.release","system.audit.package.arch","system.audit.package.license","system.audit.package.summary","system.audit.package.url","system.audit.user.name","system.audit.user.uid","system.audit.user.gid","system.audit.user.dir","system.audit.user.shell","system.audit.user.user_information","system.audit.user.password.type","fields.*"]},"refresh_interval":"5s"}}
[00:10:27]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Indexed 1751 docs into "auditbeat-8.0.0-2019.02.19-000001"
[00:10:27]                 │ info [x-pack/test/functional/es_archives/auditbeat/hosts] Indexed 1 docs into "winlogbeat-8.0.0-2019.02.19-000001"
[00:10:27]               └-> should be able to execute and get 10 signals when doing a specific query
[00:10:27]                 └-> "before each" hook: global before each for "should be able to execute and get 10 signals when doing a specific query"
[00:10:27]                 └-> "before each" hook for "should be able to execute and get 10 signals when doing a specific query"
[00:10:27]                   │ info [o.e.x.i.a.TransportPutLifecycleAction] [node-01] adding index lifecycle policy [.siem-signals-default]
[00:10:27]                   │ info [o.e.c.m.MetadataIndexTemplateService] [node-01] adding index template [.siem-signals-default] for index patterns [.siem-signals-default-*]
[00:10:27]                   │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [.siem-signals-default-000001] creating index, cause [api], templates [.siem-signals-default], shards [1]/[1]
[00:10:27]                   │ info [o.e.x.i.IndexLifecycleTransition] [node-01] moving index [.siem-signals-default-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [.siem-signals-default]
[00:10:32]                 │ proc [kibana]   log   [16:56:03.146] [info][eventLog][plugins] event logged: {"@timestamp":"2021-10-19T16:56:03.140Z","event":{"provider":"alerting","action":"execute-start","kind":"alert","category":["siem"],"start":"2021-10-19T16:56:03.140Z"},"kibana":{"saved_objects":[{"rel":"primary","type":"alert","id":"6eb9c410-30fd-11ec-a675-e77d0fd48b79","type_id":"siem.signals"}],"task":{"scheduled":"2021-10-19T16:56:00.021Z","schedule_delay":3119000000},"server_uuid":"5b2de169-2785-441b-ae8c-186a1936b17d","version":"7.16.0"},"rule":{"id":"6eb9c410-30fd-11ec-a675-e77d0fd48b79","license":"basic","category":"siem.signals","ruleset":"siem"},"message":"alert execution start: \"6eb9c410-30fd-11ec-a675-e77d0fd48b79\"","ecs":{"version":"1.8.0"}}
[00:10:36]                 │ proc [kibana]   log   [16:56:07.092] [info][plugins][securitySolution] [+] Finished indexing 88  signals searched between date ranges [
[00:10:36]                 │ proc [kibana]   {
[00:10:36]                 │ proc [kibana]     "to": "2021-10-19T16:56:05.059Z",
[00:10:36]                 │ proc [kibana]     "from": "1900-01-01T00:00:00.000Z",
[00:10:36]                 │ proc [kibana]     "maxSignals": 100
[00:10:36]                 │ proc [kibana]   }
[00:10:36]                 │ proc [kibana] ] name: "Query with a rule id" id: "6eb9c410-30fd-11ec-a675-e77d0fd48b79" rule id: "rule-1" signals index: ".siem-signals-default"
[00:10:36]                 │ proc [kibana]   log   [16:56:07.101] [info][eventLog][plugins] event logged: {"@timestamp":"2021-10-19T16:56:03.140Z","event":{"provider":"alerting","action":"execute","kind":"alert","category":["siem"],"start":"2021-10-19T16:56:03.140Z","outcome":"success","end":"2021-10-19T16:56:07.100Z","duration":3960000000},"kibana":{"saved_objects":[{"rel":"primary","type":"alert","id":"6eb9c410-30fd-11ec-a675-e77d0fd48b79","type_id":"siem.signals"}],"task":{"scheduled":"2021-10-19T16:56:00.021Z","schedule_delay":3119000000},"alerting":{"status":"ok"},"server_uuid":"5b2de169-2785-441b-ae8c-186a1936b17d","version":"7.16.0"},"rule":{"id":"6eb9c410-30fd-11ec-a675-e77d0fd48b79","license":"basic","category":"siem.signals","ruleset":"siem","name":"Query with a rule id"},"message":"alert executed: siem.signals:6eb9c410-30fd-11ec-a675-e77d0fd48b79: 'Query with a rule id'","ecs":{"version":"1.8.0"}}
[00:10:36]                 └- ✖ fail: detection engine api security and spaces enabled  create_threat_matching tests with auditbeat data should be able to execute and get 10 signals when doing a specific query
[00:10:36]                 │      Error: expected undefined to be truthy
[00:10:36]                 │       at Assertion.assert (/dev/shm/workspace/parallel/20/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:10:36]                 │       at Assertion.ok (/dev/shm/workspace/parallel/20/kibana/node_modules/@kbn/expect/expect.js:122:8)
[00:10:36]                 │       at Function.ok (/dev/shm/workspace/parallel/20/kibana/node_modules/@kbn/expect/expect.js:531:15)
[00:10:36]                 │       at Context.<anonymous> (test/detection_engine_api_integration/security_and_spaces/tests/create_threat_matching.ts:172:43)
[00:10:36]                 │       at runMicrotasks (<anonymous>)
[00:10:36]                 │       at processTicksAndRejections (node:internal/process/task_queues:96:5)
[00:10:36]                 │       at Object.apply (/dev/shm/workspace/parallel/20/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:10:36]                 │ 
[00:10:36]                 │ 

Stack Trace

Error: expected undefined to be truthy
    at Assertion.assert (/dev/shm/workspace/parallel/20/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.ok (/dev/shm/workspace/parallel/20/kibana/node_modules/@kbn/expect/expect.js:122:8)
    at Function.ok (/dev/shm/workspace/parallel/20/kibana/node_modules/@kbn/expect/expect.js:531:15)
    at Context.<anonymous> (test/detection_engine_api_integration/security_and_spaces/tests/create_threat_matching.ts:172:43)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.apply (/dev/shm/workspace/parallel/20/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @Bamieh

@kibanamachine kibanamachine merged commit 8e09daa into elastic:7.x Oct 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants