Pinging @elastic/es-core-infra (Team:Core/Infra)

Hi @pkoutsovasilis, I've created a changelog YAML for you.

@juliaElastic / @nchaulet mind taking a look? Adding you both I see you've edited this file in the past, and I see that you're on the fleet team. I'll remove core/infra as this isn't in our purview. Thank you!

@elasticmachine test this please

💚 Backport successful

Status	Branch	Result
✅	9.0
✅	8.18

Hello @pkoutsovasilis / @nchaulet / @juliaElastic
As a workaround, would we have any escape hatch to do a put mapping adding the complete field to the system index?
Would passing an header to the PUT .fleet-agents/_mapping bypass the system indices check? I'm asking as few users might not be able to upgrade and I was looking for a quick solution which doesn't involve upgrading.

@lucabelluccini 👋 Apologies, I’m not sure how I missed this earlier.
From my testing, using PUT .fleet-agents/_mapping isn’t a viable workaround — Elasticsearch rejects the request with an error like:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_state_exception",
        "reason": "Cannot update mappings in [.fleet-agents-7]: system indices can only use mappings from their descriptors, but the mappings in the request [{\"properties\":{\"local_metadata\":{\"properties\":{\"panos\":{\"type\":\"keyword\"}}}}}] did not match those in the descriptor(s)"
      }
    ],
    "type": "illegal_state_exception",
    "reason": "Cannot update mappings in [.fleet-agents-7]: system indices can only use mappings from their descriptors, but the mappings in the request [{\"properties\":{\"local_metadata\":{\"properties\":{\"panos\":{\"type\":\"keyword\"}}}}}] did not match those in the descriptor(s)"
  },
  "status": 500
}

Unless there’s an undocumented escape hatch, I don’t believe system indices can be updated this way - they’re locked to the mappings defined in their descriptors.

Hello @pkoutsovasilis - Thanks.
Yep that's what I thought... I think there's an header which we could pass, but I think in order to avoid any possible messing with system indices, we'll recommend to upgrade to a fixed version.

Hello @pkoutsovasilis - we need to port this to 8.19.x branch and 8.17.x branches - I think this is mandatory given our new release cadence.

@lucabelluccini thanks for catching that. I am gonna try to add the respective backport labels, but I don't have experience with this repo. I will also tag @ebeahan for awareness here

and nothing happens after adding the labels... @michel-laterman @juliaElastic @nchaulet guys any ideas?

Not sure about the auto backporting. If needed, this is the guidance for manually triggering them:

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 127651

I'm guessing the automated backporting only works on version labels added before the merge?

version labels added before the merge?

Yes that's right. You'll need to create the backport PRs manually now, unfortunately.

@rjernst could you tell me, if you know, which gh label I should add in my backport PRs so the automated tool doesn't generate a new changelog for them? or maybe I should allow the automation to handle the changelog and I include only the index change there?

Backport PRs just need backport and auto-merge-without-approval. If you add eg >xxx labels I think that will generate a changelog.

@rjernst thanks for the guidance, one of the backports is already merged. But the other one fails on the CI #132903 for reasons unknown to me 🙁 Do you happen to know anybody that could help me with that?

ok all backports are merged, after some retrying magic, thanks everybody for your help and guidance