Skip to content

[Ingest Manager] Fails when transform is absent in ElasticSearch but present in Ingest Manager #80697

@nnamdifrankie

Description

@nnamdifrankie

Kibana version:

7.10, 8.0

Elasticsearch version:

7.10, 8.0

Server OS version:
N/A

Browser version:

N/A

Browser OS version:
N/A

Original install method (e.g. download page, yum, from source, etc.):
Download page, source

Describe the bug:

It is possible for a record for the transform to exist in the Fleet registry but not exist in Elastic Search. This can happen when the install did not complete successfully. Another case is when the transform is deleted from ElasticSearch outside of Fleet.

Steps to reproduce:
Since this is only expected to happen in exceptional cases, we will have to induce it using approach #2.

  1. Start Kibana/ES
  2. Upgrade to the latest Endpoint Package.
  3. Ingest some metadata documents and ensure that it is visible in Endpoint App.
  4. Delete the transform using ES API or DevTools.
  5. Attempt to force install the endpoint package. This should try to remove the old package that is part of the Fleet registry and this will not find the Transform as expected in ES and should produce the error.

Expected behavior:

  1. Fleet should ignore 404 for missing transform. Future improvement in the Transform API should remove the need for the Fleet to remove the index when the Transform is removed from the package.

Screenshots (if relevant):

Errors in browser console (if relevant):

[resource_not_found_exception] Transform with id [endpoint.metadata_current-default-0.16.0] could not be found response from /_transform/endpoint.metadata_current-default-0.16.0: {"error":{"root_cause":[{"type":"resource_not_found_exception","reason":"Transform with id [endpoint.metadata_current-default-0.16.0] could not be found"}],"type":"resource_not_found_exception","reason":"Transform with id [endpoint.metadata_current-default-0.16.0] could not be found"},"status":404}

Provide logs and/or server output (if relevant):

Any additional context:

Metadata

Metadata

Assignees

Labels

Feature:FleetFleet team's agent central management projectTeam: SecuritySolutionSecurity Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.Team:Defend Workflows“EDR Workflows” sub-team of Security SolutionTeam:FleetTeam label for Observability Data Collection Fleet teamv7.11.0v8.0.0

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions