Skip to content

[8.19] [Alerting] Add snapshot telemetry for _ignored fields (#221480)#222150

Merged
maryam-saeidi merged 3 commits intoelastic:8.19from
maryam-saeidi:backport/8.19/pr-221480
Jun 3, 2025
Merged

[8.19] [Alerting] Add snapshot telemetry for _ignored fields (#221480)#222150
maryam-saeidi merged 3 commits intoelastic:8.19from
maryam-saeidi:backport/8.19/pr-221480

Conversation

@maryam-saeidi
Copy link
Copy Markdown
Member

Backport

This will backport the following commits from main to 8.19:

Questions ?

Please refer to the Backport tool documentation

Closes elastic#220815

## Summary

This PR adds telemetry for _ignored fields. The goal is to have an alert
based on this field and get notified in case a user hits the mapping
limit, but by default, we don't expect this to happen.

This PR adds `count_ignored_fields_by_rule_type` field that counts the
number of _ignored fields per rule type.

In the future, we can extend the telemetry data to also include the
actual number of mappings over the limit (there is a [feature
request](elastic/elasticsearch#68947) for
adding field count information to index API)

### How to test

- Add a lot of dynamic fields as mentioned here:
elastic#216719
- Create a rule with a custom threshold rule with multiple group by
fields to generate an alert with _ignored field
- Run the following API and check the value of
`count_ignored_fields_by_rule_type`
  ```
  POST kbn:/internal/telemetry/clusters/_stats?apiVersion=2
  {
    "unencrypted": true,
    "refreshCache": true
  }
  ```

<details>
<summary> Here is what it looks like:</summary>

![image](https://github.com/user-attachments/assets/bb71bde1-9002-4f96-9b84-3e4c7b6f0aed)

![image](https://github.com/user-attachments/assets/725fda41-454b-4ed0-a2dc-40796729bc19)

</details>

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit cde7a86)

# Conflicts:
#	x-pack/platform/plugins/shared/alerting/server/usage/lib/get_telemetry_from_alerts.test.ts
@maryam-saeidi maryam-saeidi added the backport This PR is a backport of another PR label Jun 2, 2025
@maryam-saeidi maryam-saeidi enabled auto-merge (squash) June 2, 2025 08:26
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 2, 2025

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #5 / kibana index telemetry should return empty results and log debug log if query throws search_phase_execution_exception error
  • [job] [logs] Jest Tests #5 / kibana index telemetry should return empty results and log debug log if query throws search_phase_execution_exception error
  • [job] [logs] Jest Tests #5 / kibana index telemetry should return empty results and log warning if query throws error
  • [job] [logs] Jest Tests #5 / kibana index telemetry should return empty results and log warning if query throws error
  • [job] [logs] Jest Tests #5 / kibana index telemetry should return total alert counts and alert counts by rule type id
  • [job] [logs] Jest Tests #5 / kibana index telemetry should return total alert counts and alert counts by rule type id

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id before after diff
alerting 91 92 +1

Total ESLint disabled count

id before after diff
alerting 98 99 +1

History

@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id before after diff
alerting 91 92 +1

Total ESLint disabled count

id before after diff
alerting 98 99 +1

History

@maryam-saeidi maryam-saeidi merged commit 46bbf17 into elastic:8.19 Jun 3, 2025
8 checks passed
@maryam-saeidi maryam-saeidi deleted the backport/8.19/pr-221480 branch June 3, 2025 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants