Skip to content

fix(check-saved-objects): only apply name/title field type check to importable/exportable SO types#260280

Merged
gsoldevila merged 5 commits intoelastic:mainfrom
gsoldevila:fix/check-so-changes-exempt-hidden-types
Mar 31, 2026
Merged

fix(check-saved-objects): only apply name/title field type check to importable/exportable SO types#260280
gsoldevila merged 5 commits intoelastic:mainfrom
gsoldevila:fix/check-so-changes-exempt-hidden-types

Conversation

@gsoldevila
Copy link
Copy Markdown
Member

@gsoldevila gsoldevila commented Mar 30, 2026

Summary

Closes https://github.com/elastic/kibana-team/issues/3162

The `validateNameTitleFieldTypes` check in `kbn-check-saved-objects-cli` enforces that any SO type with a `name` or `title` mapping field must use `type: text`, citing Search API compatibility. There are two distinct cases where the check was incorrect:

Case 1: Types not searchable via the management page

The check was applied unconditionally to all SO types, but the Saved Objects management page search only queries types where `management.importableAndExportable === true`. The registry treats the absence of that flag as `false`, so types without it set are never included in management search and the full-text requirement does not apply to them.

Flagging such types also creates pressure to change `keyword` → `text`, which is a breaking Elasticsearch mapping change (reindex required) that is entirely unnecessary.

Note: the `hidden` flag alone is not the right exemption criterion — the management find route explicitly includes hidden types that are also `importableAndExportable: true` (via `includedHiddenTypes`), so those remain subject to the check.

Case 2: Pre-existing keyword fields from shipped model versions

Even for importable/exportable types, if `name` or `title` was already shipped as `keyword` in a previous model version, there is nothing the developer can do to fix it without a full reindex. Failing the CI check in this case blocks the PR with no actionable resolution.

Changes

  • Split `validateNameTitleFieldTypes` into two focused functions:
    • `validateNameTitleFieldTypesNewType` (in `new_type_utils.ts`) — throws for any wrong field type on importable/exportable types; no baseline exists for new types so the full error always applies
    • `validateNameTitleFieldTypesExistingType` (in `existing_type_utils.ts`) — warns for pre-existing wrong field types, fails only for newly introduced ones
  • Shared helpers `getInvalidNameTitleFields` and `isSearchableViaManagement` extracted to `common_utils.ts`
  • Exemption condition uses `!== true` (not `=== false`) to also cover types where `management` is absent, matching the registry's `?? false` default
  • `createMockType` in tests now defaults to `importableAndExportable: true` so the positive (participates-in-management-search) path is exercised correctly
  • Tests updated: exemption test covers `importableAndExportable: false`; a new test confirms that hidden types with `importableAndExportable: true` are still subject to the check

Context

This was first surfaced by #260095, which adds model version 2 to `gap_auto_fill_scheduler` — a type with `importableAndExportable` not set to `true` and a pre-existing `name: keyword` mapping from model version 1.

Test plan

  • Existing tests in `new_type.test.ts` still pass
  • Tests in `existing_type.test.ts` confirm: warning for pre-existing keyword fields, fail for newly introduced ones, exempt for non-importable/exportable types, and fail for hidden types that are importable/exportable

…name/title field type check

The validateNameTitleFieldTypes check enforces that name and title mapping
fields use type:text for Search API compatibility. However, SO types that
are hidden or not importable/exportable are never exposed via the Search
API, so this requirement does not apply to them.

Passing registeredType (already available in both callers) to the function
and returning early for internal-only types avoids forcing unnecessary
breaking Elasticsearch mapping changes (keyword→text requires reindex)
on types that have no Search API surface.

Closes elastic/kibana-team#3162

Made-with: Cursor
@gsoldevila gsoldevila added Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Mar 30, 2026
…rd name/title fields

If a name or title mapping field was already shipped as keyword in a
previous model version, changing it to text requires reindexing which
is not possible without a dedicated migration. The check can therefore
not be enforced retroactively.

When a baseline snapshot is available (existing types), the check now
distinguishes pre-existing issues (warn via log) from newly introduced
ones (fail). New types have no baseline and still fail on incorrect
field types, as they can be fixed before shipping.

Made-with: Cursor
… two functions and use importableAndExportable as the sole exemption criterion

The management find route filters types exclusively by `importableAndExportable`, not by
`hidden`. This commit aligns the exemption logic with that reality:

- Replaces the single `validateNameTitleFieldTypes` with two focused functions:
  `validateNameTitleFieldTypesNewType` (throws on any wrong type) and
  `validateNameTitleFieldTypesExistingType` (warns on pre-existing, throws on newly
  introduced).
- Extracts `getInvalidNameTitleFields` and `isSearchableViaManagement` as private
  helpers to keep each function concise.
- Fixes the exemption condition from `=== false` (missed undefined/unset) to
  `!== true`, matching the registry's `?? false` default.
- Updates tests: `createMockType` now defaults to `importableAndExportable: true`
  so the positive path is exercised correctly; exemption tests cover the authoritative
  `importableAndExportable` criterion rather than `hidden`.

Made-with: Cursor
…o their respective utils files

- `validateNameTitleFieldTypesNewType` → `new_type_utils.ts` (new file)
- `validateNameTitleFieldTypesExistingType` → `existing_type_utils.ts`
- `getInvalidNameTitleFields` and `isSearchableViaManagement` remain in `common_utils.ts`
  and are now exported so the utils files can import them

Made-with: Cursor
@gsoldevila gsoldevila marked this pull request as ready for review March 30, 2026 15:48
@gsoldevila gsoldevila requested a review from a team as a code owner March 30, 2026 15:48
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/kibana-core (Team:Core)

@gsoldevila gsoldevila changed the title fix(check-saved-objects): exempt hidden/non-importable SO types from name/title field type check fix(check-saved-objects): only apply name/title field type check to importable/exportable SO types Mar 30, 2026
@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

Copy link
Copy Markdown
Contributor

@hammad-nasir-elastic hammad-nasir-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

return;
}

const invalidFields = getInvalidNameTitleFields(to);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the original from.name mapping was type text but then changed to keyword? I think that might put that scenario in the preExisting path even though the "pre-existing" was valid before as text.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point.

I merged the PR to unblock folks, but we can treat this scenario in a follow-up.

Note also that this kind of change (text => keyword) is not allowed as it is considered a breaking change, but I think it's worth making the logic more consistent.

@gsoldevila gsoldevila merged commit 42fe0ed into elastic:main Mar 31, 2026
15 checks passed
gsoldevila added a commit that referenced this pull request Apr 1, 2026
…ly introduced problem (#260499)

## Summary

Follow-up to #260280, addressing the edge case raised in [this review
comment](https://github.com/elastic/kibana/pull/260280/changes/BASE..8b520af6534a998b849dd3567b3b659c470e60ea#r3016163498).

### The bug

The pre-existing vs. newly-introduced classification in
`validateNameTitleFieldTypesExistingType` was based on whether the field
*key* was present in the baseline snapshot (`'properties.name.type' in
from.mappings`). This correctly catches `absent → keyword`, but silently
misclassifies `text → keyword` as pre-existing — because the key is
already in `from.mappings`, even though the field was previously valid.

The result: a developer who downgrades a `name`/`title` field from
`text` to `keyword` gets no warning and no failure from the CI check.

### The fix

Both `preExisting` and `newlyIntroduced` are now derived from the set of
fields that were **already invalid in `from`**, not merely present:

```ts
const alreadyInvalidInFrom = new Set(
  getInvalidNameTitleFields(from).map(({ fieldName }) => fieldName)
);
const preExisting     = invalidInTo.filter(({ fieldName }) =>  alreadyInvalidInFrom.has(fieldName));
const newlyIntroduced = invalidInTo.filter(({ fieldName }) => !alreadyInvalidInFrom.has(fieldName));
```

This handles all four scenarios correctly:

| `from` type | `to` type | Result |
|---|---|---|
| absent | `keyword` | ❌ fail (newly introduced) |
| `text` | `keyword` | ❌ fail (newly introduced — was this PR's bug) |
| `keyword` | `keyword` | ⚠️ warn (pre-existing, cannot be fixed without
reindex) |
| `keyword` | `text` | ✅ nothing (fixed) |

## Test plan

- [ ] New test `'should throw when a name or title field is downgraded
from text to keyword'` in `existing_type.test.ts` covers the regression
case
- [ ] All 40 tests in the `validate_changes` suite pass

Made with [Cursor](https://cursor.com)
jeramysoucy pushed a commit to jeramysoucy/kibana that referenced this pull request Apr 1, 2026
…mportable/exportable SO types (elastic#260280)

## Summary

Closes elastic/kibana-team#3162

The \`validateNameTitleFieldTypes\` check in
\`kbn-check-saved-objects-cli\` enforces that any SO type with a
\`name\` or \`title\` mapping field must use \`type: text\`, citing
Search API compatibility. There are two distinct cases where the check
was incorrect:

### Case 1: Types not searchable via the management page

The check was applied unconditionally to all SO types, but the Saved
Objects management page search only queries types where
\`management.importableAndExportable === true\`. The registry treats the
absence of that flag as \`false\`, so types without it set are never
included in management search and the full-text requirement does not
apply to them.

Flagging such types also creates pressure to change \`keyword\` →
\`text\`, which is a **breaking Elasticsearch mapping change** (reindex
required) that is entirely unnecessary.

Note: the \`hidden\` flag alone is **not** the right exemption criterion
— the management find route explicitly includes hidden types that are
also \`importableAndExportable: true\` (via \`includedHiddenTypes\`), so
those remain subject to the check.

### Case 2: Pre-existing keyword fields from shipped model versions

Even for importable/exportable types, if \`name\` or \`title\` was
already shipped as \`keyword\` in a previous model version, **there is
nothing the developer can do to fix it** without a full reindex. Failing
the CI check in this case blocks the PR with no actionable resolution.

## Changes

- Split \`validateNameTitleFieldTypes\` into two focused functions:
- \`validateNameTitleFieldTypesNewType\` (in \`new_type_utils.ts\`) —
throws for any wrong field type on importable/exportable types; no
baseline exists for new types so the full error always applies
- \`validateNameTitleFieldTypesExistingType\` (in
\`existing_type_utils.ts\`) — warns for pre-existing wrong field types,
fails only for newly introduced ones
- Shared helpers \`getInvalidNameTitleFields\` and
\`isSearchableViaManagement\` extracted to \`common_utils.ts\`
- Exemption condition uses \`!== true\` (not \`=== false\`) to also
cover types where \`management\` is absent, matching the registry's \`??
false\` default
- \`createMockType\` in tests now defaults to \`importableAndExportable:
true\` so the positive (participates-in-management-search) path is
exercised correctly
- Tests updated: exemption test covers \`importableAndExportable:
false\`; a new test confirms that hidden types with
\`importableAndExportable: true\` are still subject to the check

## Context

This was first surfaced by
elastic#260095, which adds model version
2 to \`gap_auto_fill_scheduler\` — a type with
\`importableAndExportable\` not set to \`true\` and a pre-existing
\`name: keyword\` mapping from model version 1.

## Test plan

- [ ] Existing tests in \`new_type.test.ts\` still pass
- [ ] Tests in \`existing_type.test.ts\` confirm: warning for
pre-existing keyword fields, fail for newly introduced ones, exempt for
non-importable/exportable types, and fail for hidden types that are
importable/exportable

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
jeramysoucy pushed a commit to jeramysoucy/kibana that referenced this pull request Apr 1, 2026
…ly introduced problem (elastic#260499)

## Summary

Follow-up to elastic#260280, addressing the edge case raised in [this review
comment](https://github.com/elastic/kibana/pull/260280/changes/BASE..8b520af6534a998b849dd3567b3b659c470e60ea#r3016163498).

### The bug

The pre-existing vs. newly-introduced classification in
`validateNameTitleFieldTypesExistingType` was based on whether the field
*key* was present in the baseline snapshot (`'properties.name.type' in
from.mappings`). This correctly catches `absent → keyword`, but silently
misclassifies `text → keyword` as pre-existing — because the key is
already in `from.mappings`, even though the field was previously valid.

The result: a developer who downgrades a `name`/`title` field from
`text` to `keyword` gets no warning and no failure from the CI check.

### The fix

Both `preExisting` and `newlyIntroduced` are now derived from the set of
fields that were **already invalid in `from`**, not merely present:

```ts
const alreadyInvalidInFrom = new Set(
  getInvalidNameTitleFields(from).map(({ fieldName }) => fieldName)
);
const preExisting     = invalidInTo.filter(({ fieldName }) =>  alreadyInvalidInFrom.has(fieldName));
const newlyIntroduced = invalidInTo.filter(({ fieldName }) => !alreadyInvalidInFrom.has(fieldName));
```

This handles all four scenarios correctly:

| `from` type | `to` type | Result |
|---|---|---|
| absent | `keyword` | ❌ fail (newly introduced) |
| `text` | `keyword` | ❌ fail (newly introduced — was this PR's bug) |
| `keyword` | `keyword` | ⚠️ warn (pre-existing, cannot be fixed without
reindex) |
| `keyword` | `text` | ✅ nothing (fixed) |

## Test plan

- [ ] New test `'should throw when a name or title field is downgraded
from text to keyword'` in `existing_type.test.ts` covers the regression
case
- [ ] All 40 tests in the `validate_changes` suite pass

Made with [Cursor](https://cursor.com)
eokoneyo pushed a commit to davismcphee/kibana that referenced this pull request Apr 2, 2026
…ly introduced problem (elastic#260499)

## Summary

Follow-up to elastic#260280, addressing the edge case raised in [this review
comment](https://github.com/elastic/kibana/pull/260280/changes/BASE..8b520af6534a998b849dd3567b3b659c470e60ea#r3016163498).

### The bug

The pre-existing vs. newly-introduced classification in
`validateNameTitleFieldTypesExistingType` was based on whether the field
*key* was present in the baseline snapshot (`'properties.name.type' in
from.mappings`). This correctly catches `absent → keyword`, but silently
misclassifies `text → keyword` as pre-existing — because the key is
already in `from.mappings`, even though the field was previously valid.

The result: a developer who downgrades a `name`/`title` field from
`text` to `keyword` gets no warning and no failure from the CI check.

### The fix

Both `preExisting` and `newlyIntroduced` are now derived from the set of
fields that were **already invalid in `from`**, not merely present:

```ts
const alreadyInvalidInFrom = new Set(
  getInvalidNameTitleFields(from).map(({ fieldName }) => fieldName)
);
const preExisting     = invalidInTo.filter(({ fieldName }) =>  alreadyInvalidInFrom.has(fieldName));
const newlyIntroduced = invalidInTo.filter(({ fieldName }) => !alreadyInvalidInFrom.has(fieldName));
```

This handles all four scenarios correctly:

| `from` type | `to` type | Result |
|---|---|---|
| absent | `keyword` | ❌ fail (newly introduced) |
| `text` | `keyword` | ❌ fail (newly introduced — was this PR's bug) |
| `keyword` | `keyword` | ⚠️ warn (pre-existing, cannot be fixed without
reindex) |
| `keyword` | `text` | ✅ nothing (fixed) |

## Test plan

- [ ] New test `'should throw when a name or title field is downgraded
from text to keyword'` in `existing_type.test.ts` covers the regression
case
- [ ] All 40 tests in the `validate_changes` suite pass

Made with [Cursor](https://cursor.com)
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Apr 2, 2026
…mportable/exportable SO types (elastic#260280)

## Summary

Closes elastic/kibana-team#3162

The \`validateNameTitleFieldTypes\` check in
\`kbn-check-saved-objects-cli\` enforces that any SO type with a
\`name\` or \`title\` mapping field must use \`type: text\`, citing
Search API compatibility. There are two distinct cases where the check
was incorrect:

### Case 1: Types not searchable via the management page

The check was applied unconditionally to all SO types, but the Saved
Objects management page search only queries types where
\`management.importableAndExportable === true\`. The registry treats the
absence of that flag as \`false\`, so types without it set are never
included in management search and the full-text requirement does not
apply to them.

Flagging such types also creates pressure to change \`keyword\` →
\`text\`, which is a **breaking Elasticsearch mapping change** (reindex
required) that is entirely unnecessary.

Note: the \`hidden\` flag alone is **not** the right exemption criterion
— the management find route explicitly includes hidden types that are
also \`importableAndExportable: true\` (via \`includedHiddenTypes\`), so
those remain subject to the check.

### Case 2: Pre-existing keyword fields from shipped model versions

Even for importable/exportable types, if \`name\` or \`title\` was
already shipped as \`keyword\` in a previous model version, **there is
nothing the developer can do to fix it** without a full reindex. Failing
the CI check in this case blocks the PR with no actionable resolution.

## Changes

- Split \`validateNameTitleFieldTypes\` into two focused functions:
- \`validateNameTitleFieldTypesNewType\` (in \`new_type_utils.ts\`) —
throws for any wrong field type on importable/exportable types; no
baseline exists for new types so the full error always applies
- \`validateNameTitleFieldTypesExistingType\` (in
\`existing_type_utils.ts\`) — warns for pre-existing wrong field types,
fails only for newly introduced ones
- Shared helpers \`getInvalidNameTitleFields\` and
\`isSearchableViaManagement\` extracted to \`common_utils.ts\`
- Exemption condition uses \`!== true\` (not \`=== false\`) to also
cover types where \`management\` is absent, matching the registry's \`??
false\` default
- \`createMockType\` in tests now defaults to \`importableAndExportable:
true\` so the positive (participates-in-management-search) path is
exercised correctly
- Tests updated: exemption test covers \`importableAndExportable:
false\`; a new test confirms that hidden types with
\`importableAndExportable: true\` are still subject to the check

## Context

This was first surfaced by
elastic#260095, which adds model version
2 to \`gap_auto_fill_scheduler\` — a type with
\`importableAndExportable\` not set to \`true\` and a pre-existing
\`name: keyword\` mapping from model version 1.

## Test plan

- [ ] Existing tests in \`new_type.test.ts\` still pass
- [ ] Tests in \`existing_type.test.ts\` confirm: warning for
pre-existing keyword fields, fail for newly introduced ones, exempt for
non-importable/exportable types, and fail for hidden types that are
importable/exportable

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Apr 2, 2026
…ly introduced problem (elastic#260499)

## Summary

Follow-up to elastic#260280, addressing the edge case raised in [this review
comment](https://github.com/elastic/kibana/pull/260280/changes/BASE..8b520af6534a998b849dd3567b3b659c470e60ea#r3016163498).

### The bug

The pre-existing vs. newly-introduced classification in
`validateNameTitleFieldTypesExistingType` was based on whether the field
*key* was present in the baseline snapshot (`'properties.name.type' in
from.mappings`). This correctly catches `absent → keyword`, but silently
misclassifies `text → keyword` as pre-existing — because the key is
already in `from.mappings`, even though the field was previously valid.

The result: a developer who downgrades a `name`/`title` field from
`text` to `keyword` gets no warning and no failure from the CI check.

### The fix

Both `preExisting` and `newlyIntroduced` are now derived from the set of
fields that were **already invalid in `from`**, not merely present:

```ts
const alreadyInvalidInFrom = new Set(
  getInvalidNameTitleFields(from).map(({ fieldName }) => fieldName)
);
const preExisting     = invalidInTo.filter(({ fieldName }) =>  alreadyInvalidInFrom.has(fieldName));
const newlyIntroduced = invalidInTo.filter(({ fieldName }) => !alreadyInvalidInFrom.has(fieldName));
```

This handles all four scenarios correctly:

| `from` type | `to` type | Result |
|---|---|---|
| absent | `keyword` | ❌ fail (newly introduced) |
| `text` | `keyword` | ❌ fail (newly introduced — was this PR's bug) |
| `keyword` | `keyword` | ⚠️ warn (pre-existing, cannot be fixed without
reindex) |
| `keyword` | `text` | ✅ nothing (fixed) |

## Test plan

- [ ] New test `'should throw when a name or title field is downgraded
from text to keyword'` in `existing_type.test.ts` covers the regression
case
- [ ] All 40 tests in the `validate_changes` suite pass

Made with [Cursor](https://cursor.com)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants