Skip to content

[RAC][Rule Registry] Index bootstrapping: make index upgrade logic compatible with adding field aliases and runtime fields to old indices for backwards compatibility #110795

@banderror

Description

@banderror

Parent ticket: #101016
Related to: #109293

Summary

(Address before the next release that makes any mapping changes, possibly as soon as 7.15.1)

Work on improving the index upgrade logic because upgrading the mappings in place is potentially incompatible with the plan to add field aliases and runtime fields to old indices for backwards compatibility. If the mapping on a single index can change over time, it's hard to define what aliases/runtime fields would need to be added to make it compatible with the new schema.

Open questions:

  • Should we update the mapping in place if we're only adding a new field? Pro: limits oversharding. Con: divides alerts in the index into old/new, harder to apply some types of backwards compatibility changes
  • Right now the in-place index upgrade only applies new mappings. Can/should we update index settings as well?

Background

The background for this is our discussions with @kobelb (see #109276 (comment) and above comments) on the "compatibility" of the current index upgrade logic with the ideas for backwards compatibility (#109293).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Team: SecuritySolutionSecurity Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.Team:Detection AlertsSecurity Detection Alerts Area TeamTeam:Detections and RespSecurity Detection Response TeamTheme: raclabel obsoletebugFixes for quality problems that affect the customer experience

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions