I should have thought of this when reviewing the earlier PR, but do you think it would be worthwhile to put in a check after line 166 in this file that if the merged model is equal to the parsed model, then we return early? If the update is a no-op then we don't need to do the extra work of performing a validation call or updating the indices.

Stakes of actually performing the validation and update on not updated model are higher than in case described in your previous comment and I agree that we should return early. Specially considering the fact it is so easy to implement.

This change is done. Also in #142597

I don't think we want to allow modelId to be updated. I don't think there's really a valid use case for changing the model associated with an endpoint, since doing so will cause the behaviour of the endpoint to change, and the InferenceServiceConfiguration in Ai21Service lists it as not updatable:

                configurationMap.put(
                    ServiceFields.MODEL_ID,
                    new SettingsConfiguration.Builder(SUPPORTED_TASK_TYPES).setDescription(
                        "Refer to the AI21 models documentation for the list of available inference models."
                    )
                        .setLabel("Model")
                        .setRequired(true)
                        .setSensitive(false)
                        .setUpdatable(false)
                        .setType(SettingsConfigurationFieldType.STRING)
                        .build()
                );

This comment applies to many other service settings fields, in particular ones for similarity, dimensions and embedding type for dense embedding service settings. I think we need to consider each service setting field carefully when deciding whether to allow it to be updated. The InferenceServiceConfiguration for a given service may be incomplete or incorrect, so I'm not sure it's safe to just go by what those say.

Good catch on the updatable flag. I completely missed that. You’re right that it needs to be set appropriately for any setting we intend to allow updates on.
Historically it was set to false across the board because updateServiceSettings effectively didn’t support updates for most services (it was returning this in the majority of cases). This PR changes that behavior and makes updates possible, which is why these flags now matter. InferenceServiceConfigurations are often incomplete and incorrect and I, as an author of Ai21 chat integration didn't mean that model must be immutable.

I don't think we want to allow modelId to be updated.
doing so will cause the behaviour of the endpoint to change

I respectfully disagree here. Changing the modelId will indeed change the behavior of the endpoint. That’s inherently true for any configuration change that points to a different underlying model. Conceptually, updating modelId would mean swapping the provider endpoint model within the same elastic endpoint, which seems like a reasonable and potentially useful operation for customers.
As long as the updated configuration results in a valid and functional underlying endpoint, I don’t think we should artificially restrict this capability on our side. The behavior change is explicit and intentional from the user’s perspective.

The reason we don't want to allow all fields to be updatable is that users can easily break their system. For example if a user ingests some documents with model_id_a and then changes it to model_id_b and ingests more documents, the embeddings could be incompatible. For chat completion and other task types it's not quite as bad but I still think we should consider some fields as immutable. If the user wants to use a different one, then they can create a new endpoint to do that.

Generally most things within service_settings probably shouldn't be updatable. Like Donal mentioned, we shouldn't allow updates for these fields in particular but there are probably more:

similarity
dimensions
embedding type
model id

Understood. Will make the changes making those fields immutable post-creation.

Here's the PR where model ID is immutable:
#142597