Gap reason UI by nkhristinin · Pull Request #260095 · elastic/kibana

nkhristinin · 2026-03-27T19:14:22Z

Gap Reason UI

Overview

This PR adds API-level filtering by gap reason and the Rule Settings UI for controlling which gap reasons are included in gap monitoring and auto-fill. The gap reason detection logic and the "Reason" column in the gaps table were shipped in a previous PR (#258231).

CleanShot.2026-03-29.at.20.48.11.mp4

Feature Flag

Gated behind gapReasonDetectionEnabled (default: false). When disabled:

The "Include disabled gaps" checkbox is hidden from the Rule Settings modal
API filtering by excludedReasons still works at the schema level but has no practical effect since no reasons are written

Changes

Rule Settings Modal

New "Gap detection scope" section with a checkbox to include/exclude gaps caused by disabled rules (hidden when feature flag is off)
Saves excludedReasons to both the gap auto-fill scheduler saved object and securitySolution:excludedGapReasons UI setting. The value is stored in two places because the gap auto-fill scheduler can be available for people with free tiers.

Bulk Fill Modal

Shows an info callout when rule_disabled gaps are excluded: "Gaps caused by disabled rules will not be filled. You can change this in Rule Settings."

Gap table

Add reason filter which by default get values from the rule setting modal

API Filtering

getRuleIdsWithGaps — accepts excluded_reasons parameter to filter out gaps by reason type
getGapsSummaryByRuleIds — accepts excluded_reasons parameter for summary calculations
findRules route — reads EXCLUDED_GAP_REASONS_KEY from UI settings and passes it to gap filtering
Bulk fill gaps — respects excludedReasons when scheduling gap fills
Gap auto-fill scheduler — stores and applies excludedReasons (persisted in saved object)
buildGapsFilter — extended to support reason-based filtering in ES queries

UI Settings

securitySolution:excludedGapReasons — new advanced setting (readonly, array type) controlling which gap reasons are excluded from monitoring and auto-fill. Default: ['rule_disabled']

How to Test

Prerequisites

Enable the feature flag in kibana.dev.yml:

xpack.securitySolution.enableExperimental: ['gapReasonDetectionEnabled']

Test 1: `rule_disabled` reason

Create a detection rule with 1 minute interval and 1 second lookback
Enable the rule and let it run successfully at least once
Disable the rule
Wait 5 minutes
Enable the rule
Go to the rule details page → Gaps tab
Expected: A gap appears with reason "Rule was disabled"

Test 2: `rule_did_not_run` reason (Kibana downtime)

Create a detection rule with 1 minute interval and 1 second lookback
Enable the rule and successfully at least once
Kill Kibana (stop the process)
Wait 5 minutes
Start Kibana again
Go to the rule details page → Gaps tab
Expected: A gap appears with reason "Rule did not run"

…nto gap-reason-detected

…tion_tests/ci_checks

…nto gap-reason-detected

nkhristinin · 2026-03-27T19:14:45Z

/ci

nkhristinin · 2026-03-27T19:16:11Z

@elasticmachine merge upstream

elasticmachine · 2026-03-27T19:16:17Z

There are no new commits on the base branch.

nkhristinin · 2026-03-27T19:16:17Z

/ci

…tion_tests/ci_checks

…mportable/exportable SO types (elastic#260280) ## Summary Closes elastic/kibana-team#3162 The \`validateNameTitleFieldTypes\` check in \`kbn-check-saved-objects-cli\` enforces that any SO type with a \`name\` or \`title\` mapping field must use \`type: text\`, citing Search API compatibility. There are two distinct cases where the check was incorrect: ### Case 1: Types not searchable via the management page The check was applied unconditionally to all SO types, but the Saved Objects management page search only queries types where \`management.importableAndExportable === true\`. The registry treats the absence of that flag as \`false\`, so types without it set are never included in management search and the full-text requirement does not apply to them. Flagging such types also creates pressure to change \`keyword\` → \`text\`, which is a **breaking Elasticsearch mapping change** (reindex required) that is entirely unnecessary. Note: the \`hidden\` flag alone is **not** the right exemption criterion — the management find route explicitly includes hidden types that are also \`importableAndExportable: true\` (via \`includedHiddenTypes\`), so those remain subject to the check. ### Case 2: Pre-existing keyword fields from shipped model versions Even for importable/exportable types, if \`name\` or \`title\` was already shipped as \`keyword\` in a previous model version, **there is nothing the developer can do to fix it** without a full reindex. Failing the CI check in this case blocks the PR with no actionable resolution. ## Changes - Split \`validateNameTitleFieldTypes\` into two focused functions: - \`validateNameTitleFieldTypesNewType\` (in \`new_type_utils.ts\`) — throws for any wrong field type on importable/exportable types; no baseline exists for new types so the full error always applies - \`validateNameTitleFieldTypesExistingType\` (in \`existing_type_utils.ts\`) — warns for pre-existing wrong field types, fails only for newly introduced ones - Shared helpers \`getInvalidNameTitleFields\` and \`isSearchableViaManagement\` extracted to \`common_utils.ts\` - Exemption condition uses \`!== true\` (not \`=== false\`) to also cover types where \`management\` is absent, matching the registry's \`?? false\` default - \`createMockType\` in tests now defaults to \`importableAndExportable: true\` so the positive (participates-in-management-search) path is exercised correctly - Tests updated: exemption test covers \`importableAndExportable: false\`; a new test confirms that hidden types with \`importableAndExportable: true\` are still subject to the check ## Context This was first surfaced by elastic#260095, which adds model version 2 to \`gap_auto_fill_scheduler\` — a type with \`importableAndExportable\` not set to \`true\` and a pre-existing \`name: keyword\` mapping from model version 1. ## Test plan - [ ] Existing tests in \`new_type.test.ts\` still pass - [ ] Tests in \`existing_type.test.ts\` confirm: warning for pre-existing keyword fields, fail for newly introduced ones, exempt for non-importable/exportable types, and fail for hidden types that are importable/exportable --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

darnautov

Overall LGTM, just one concern about duplicating excludedReasons schema

darnautov · 2026-04-01T16:21:42Z

...erting/server/application/gaps/methods/get_rule_ids_with_gaps/schemas/get_rules_with_gaps.ts

+  excludedReasons: schema.maybe(
+    schema.arrayOf(
+      schema.oneOf([
+        schema.literal(gapReasonType.RULE_DISABLED),
+        schema.literal(gapReasonType.RULE_DID_NOT_RUN),
+      ]),
+      { maxSize: Object.values(gapReasonType).length }
+    )


I noticed it's been duplicated in other schemas, should we consolidate it?

maximpn

@nkhristinin Thanks for addressing my comments 🙏

I've tested the functionality locally. It works as expected.

There are a couple of comments related to my previous comments regarding exclude reasons schema and deduplication. These should be easy to address. Besides that it looks good so approving.

maximpn · 2026-04-02T07:50:38Z

...ion/gaps/auto_fill_scheduler/methods/create/schemas/create_gap_auto_fill_scheduler_schema.ts

+      schema.oneOf([
+        schema.literal(gapReasonType.RULE_DISABLED),
+        schema.literal(gapReasonType.RULE_DID_NOT_RUN),
+      ])


Suggested change

])

]),

{

maxSize: Object.values(gapReasonType).length,

}

maximpn · 2026-04-02T07:56:09Z

x-pack/platform/plugins/shared/alerting/server/lib/rule_gaps/build_gaps_filter.ts

+  const excludedReasonsFilter =
+    excludedReasons && excludedReasons.length > 0
+      ? `NOT (${excludedReasons
+          .map((reason) => `${GAP_REASON_TYPE_FIELD}: "${reason}"`)
+          .join(' OR ')})`
+      : null;


Let's have excluded reasons deduplication. Typing doesn't prevent having an array with the same values.

Suggested change

const excludedReasonsFilter =

excludedReasons && excludedReasons.length > 0

? `NOT (${excludedReasons

.map((reason) => `${GAP_REASON_TYPE_FIELD}: "${reason}"`)

.join(' OR ')})`

: null;

const excludedReasonsSet = new Set(excludedReasons ?? []);

const excludedReasonsFilter =

excludedReasonsSet.size > 0

? `NOT (${Array.from(excludedReasonsSet)

.map((reason) => `${GAP_REASON_TYPE_FIELD}: "${reason}"`)

.join(' OR ')})`

: null;

…mportable/exportable SO types (elastic#260280) ## Summary Closes elastic/kibana-team#3162 The \`validateNameTitleFieldTypes\` check in \`kbn-check-saved-objects-cli\` enforces that any SO type with a \`name\` or \`title\` mapping field must use \`type: text\`, citing Search API compatibility. There are two distinct cases where the check was incorrect: ### Case 1: Types not searchable via the management page The check was applied unconditionally to all SO types, but the Saved Objects management page search only queries types where \`management.importableAndExportable === true\`. The registry treats the absence of that flag as \`false\`, so types without it set are never included in management search and the full-text requirement does not apply to them. Flagging such types also creates pressure to change \`keyword\` → \`text\`, which is a **breaking Elasticsearch mapping change** (reindex required) that is entirely unnecessary. Note: the \`hidden\` flag alone is **not** the right exemption criterion — the management find route explicitly includes hidden types that are also \`importableAndExportable: true\` (via \`includedHiddenTypes\`), so those remain subject to the check. ### Case 2: Pre-existing keyword fields from shipped model versions Even for importable/exportable types, if \`name\` or \`title\` was already shipped as \`keyword\` in a previous model version, **there is nothing the developer can do to fix it** without a full reindex. Failing the CI check in this case blocks the PR with no actionable resolution. ## Changes - Split \`validateNameTitleFieldTypes\` into two focused functions: - \`validateNameTitleFieldTypesNewType\` (in \`new_type_utils.ts\`) — throws for any wrong field type on importable/exportable types; no baseline exists for new types so the full error always applies - \`validateNameTitleFieldTypesExistingType\` (in \`existing_type_utils.ts\`) — warns for pre-existing wrong field types, fails only for newly introduced ones - Shared helpers \`getInvalidNameTitleFields\` and \`isSearchableViaManagement\` extracted to \`common_utils.ts\` - Exemption condition uses \`!== true\` (not \`=== false\`) to also cover types where \`management\` is absent, matching the registry's \`?? false\` default - \`createMockType\` in tests now defaults to \`importableAndExportable: true\` so the positive (participates-in-management-search) path is exercised correctly - Tests updated: exemption test covers \`importableAndExportable: false\`; a new test confirms that hidden types with \`importableAndExportable: true\` are still subject to the check ## Context This was first surfaced by elastic#260095, which adds model version 2 to \`gap_auto_fill_scheduler\` — a type with \`importableAndExportable\` not set to \`true\` and a pre-existing \`name: keyword\` mapping from model version 1. ## Test plan - [ ] Existing tests in \`new_type.test.ts\` still pass - [ ] Tests in \`existing_type.test.ts\` confirm: warning for pre-existing keyword fields, fail for newly introduced ones, exempt for non-importable/exportable types, and fail for hidden types that are importable/exportable --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

denar50

Thanks for addressing my comments, I tested locally. LGTM!

…p-reason-ui

leathekd · 2026-04-02T13:42:20Z

src/platform/plugins/shared/telemetry/schema/oss_platform.json

+          "items": {
+            "type": "keyword",
+            "_meta": {
+              "description": "Non-default value of setting."


Can you improve this description? We hope to one day use these descriptions as documentation in Telemetry.

leathekd

Schema LGTM

nkhristinin · 2026-04-02T16:02:43Z

@elasticmachine merge upstream

elasticmachine · 2026-04-02T17:27:46Z

💚 Build Succeeded

Buildkite Build
Commit: e00c933
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-260095-e00c933c1a11
Security Deployment

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	9297	9298	+1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/management-settings-ids`	149	150	+1

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id	before	after	diff
`alerting`	1	2	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	11.6MB	11.6MB	+3.3KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`alerting`	20.1KB	20.2KB	+57.0B
`securitySolution`	121.2KB	121.3KB	+53.0B
total			+110.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/management-settings-ids`	150	151	+1
`alerting`	899	901	+2
total			+3

History

💚 Build #421294 succeeded 9379881
💔 Build #421242 failed 163a0e4
💔 Build #421214 failed 9d9a270
💔 Build #421141 failed 41cc85d

cc @nkhristinin

nkhristinin and others added 24 commits March 10, 2026 14:09

Add last enabled set for a rule

8cf72fe

omit some fields for testing

1011d70

add last enabled to fixiture

2c97657

add field to transfrom rule domain to rule attributes

c3c8186

add gap reason detection

5f1fcdc

Changes from yarn openapi:bundle

3714e84

add ui

45d35ee

update shchema and add ff

d805e8d

Merge branch 'gap-reason-detected' of github.com:nkhristinin/kibana i…

1a3b353

…nto gap-reason-detected

Merge branch 'main' into gap-reason-detected

e88efe9

Changes from node scripts/jest_integration -u src/core/server/integra…

28a3957

…tion_tests/ci_checks

add comment test

cdbf688

fix tests

5bbcdd9

Merge branch 'gap-reason-detected' of github.com:nkhristinin/kibana i…

4becaaa

…nto gap-reason-detected

Changes from make api-docs

5e675fb

Merge branch 'main' into gap-reason-detected

137acc9

update fixtures

6c5dd2c

add unit tests for gap reason

bf62851

rename function in tranform to gap

5ee2da7

Merge branch 'main' into gap-reason-detected

a688b23

clear whole gap

e0b2142

fix test files

6dbf44f

Add exlcude logic for gap auto scheduler

fe013eb

Add api filtering for gap reason

ab86335

Merge branch 'main' into gap-reason-ui

4301378

Changes from node scripts/jest_integration -u src/core/server/integra…

601a83a

…tion_tests/ci_checks

darnautov approved these changes Apr 1, 2026

View reviewed changes

nkhristinin added 5 commits April 2, 2026 08:20

remove ?. and default values for scheduler

530ae78

move schema to one varible

8a9e68a

add tests

c2910e9

improve tests

f6cabf6

early return

aa32922

nkhristinin requested a review from denar50 April 2, 2026 08:23

nkhristinin added 2 commits April 2, 2026 11:06

PR fixes

943d7e4

Get default value from UI settings

41cc85d

maximpn approved these changes Apr 2, 2026

View reviewed changes

remove default ovveride

9d9a270

denar50 approved these changes Apr 2, 2026

View reviewed changes

Changes from node scripts/eslint_all_files --no-cache --fix

163a0e4

jesuswr approved these changes Apr 2, 2026

View reviewed changes

nkhristinin added 2 commits April 2, 2026 14:19

fix tests

c485a73

Merge branch 'gap-reason-ui' of github.com:nkhristinin/kibana into ga…

9379881

…p-reason-ui

leathekd reviewed Apr 2, 2026

View reviewed changes

update description for excluded gap reasons

52b6281

nkhristinin requested a review from leathekd April 2, 2026 14:02

Changes from node scripts/telemetry_check

e556a34

leathekd approved these changes Apr 2, 2026

View reviewed changes

nkhristinin enabled auto-merge (squash) April 2, 2026 15:03

Merge branch 'main' into gap-reason-ui

e00c933

nkhristinin merged commit 63785b2 into elastic:main Apr 2, 2026
18 checks passed

kibanamachine added the v9.4.0 label Apr 2, 2026

nastasha-solomon mentioned this pull request Apr 11, 2026

[UI copy]: New gap reason and error gap fill status strings — Security Solution Detection Engine elastic/docs-content#5866

Open

-      ])
+        ]),
+        {
+          maxSize: Object.values(gapReasonType).length,
+        }

Conversation

nkhristinin commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Gap Reason UI

Overview

Feature Flag

Changes

Rule Settings Modal

Bulk Fill Modal

Gap table

API Filtering

UI Settings

How to Test

Prerequisites

Test 1: rule_disabled reason

Test 2: rule_did_not_run reason (Kibana downtime)

Uh oh!

nkhristinin commented Mar 27, 2026

Uh oh!

nkhristinin commented Mar 27, 2026

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

nkhristinin commented Mar 27, 2026

Uh oh!

darnautov left a comment

Choose a reason for hiding this comment

Uh oh!

darnautov Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

maximpn left a comment

Choose a reason for hiding this comment

Uh oh!

maximpn Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

maximpn Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

denar50 left a comment

Choose a reason for hiding this comment

Uh oh!

leathekd Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

leathekd left a comment

Choose a reason for hiding this comment

Uh oh!

nkhristinin commented Apr 2, 2026

Uh oh!

elasticmachine commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Any counts in public APIs

Async chunks

Page load bundle

API count

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

nkhristinin commented Mar 27, 2026 •

edited

Loading

Test 1: `rule_disabled` reason

Test 2: `rule_did_not_run` reason (Kibana downtime)

elasticmachine commented Apr 2, 2026 •

edited

Loading