@ishleenk17 - What happens to the existing user who faces the field conflicts ?

@ishleenk17 - What happens to the existing user who faces the field conflicts ?

They will have to upgrade to this version if they think it's a deal breaker for them.

@ishleenk17 - What happens to the existing user who faces the field conflicts ?

They will have to upgrade to this version if they think it's a deal breaker for them.

Does it requires removing the conflicting field and re-indexing the existing data ? Are we considering this as a breaking change ? Otherwise the changes looks good !

@ishleenk17 - What happens to the existing user who faces the field conflicts ?

They will have to upgrade to this version if they think it's a deal breaker for them.

Does it requires removing the conflicting field and re-indexing the existing data ? Are we considering this as a breaking change ? Otherwise the changes looks good !

Yes, I think this would be categorized as a breaking change since it's going to be a mapping change.
@andrewkroh: How do we handle such a scenario?
do we document them?

I would categorize this as a bug. ECS was not adhered here and the default ES dynamic mappings prevailed.

When the new package version is installed Fleet will _rollover the backing data stream index so that the new mappings are applied. From a data indexing point of view there should be no issues.

I would categorize this as a bug. ECS was not adhered here and the default ES dynamic mappings prevailed.

When the new package version is installed Fleet will _rollover the backing data stream index so that the new mappings are applied. From a data indexing point of view there should be no issues.

Thanks for the clarity @andrewkroh

@andrewkroh - I am trying to understand what happens when we try to search or perform aggregations against the data after the new change.

Will the type conflict still exists ?

Observations :

• Applied an aggregation query against the datastream which has a type of long in older indices and with type of keyword in newer indices after _rollover.
• Getting two different type for same field error.

Correct me if I am wrong,

• This happens because we are searching for the data from older indices which has a different field type.
• This can be avoided either by searching for the data from the newer index only or by re-indexing the older index with newer mapping.

That is correct. Any functions that require knowing the type can have issues. Queries without aggs (like Discover) should work. The options for resolution are reindexing, deleting, closing, or adding a runtime field in the old indices to shadow the field with the correct type.

If we go by this document, I think we should consider this as a bug fix and not a breaking change.
There are lesser chances that the error.code field will be used in aggregation queries.

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

🤖 GitHub comments

🌐 Coverage report

Package microsoft_sqlserver - 2.1.1 containing this change is available at https://epr.elastic.co/search?package=microsoft_sqlserver

Package system - 1.36.2 containing this change is available at https://epr.elastic.co/search?package=system