It has been reported that using the inference APIs with openAI's o3 results in an error because we're always sending the temperature parameter, and o3 doesn't support it.

Unfortunately, I don't think there' a "good" way to detect this, so we will have to find a work around here.

A couple ideas:

1. default to not sending the temperature parameter

To add more consistency across model and providers that don't have the same value for their default temperature, we defaulted to force to 0 the temperature when unspecified:

One option could be to revert this and get back to defaulting to undefined. However I don't think this is ideal, given that API consumers that still specify a temperature value, which will cause models not supporting the parameter to return an error during the call.

2. Evict/ignore the temperature parameter depending on the model name

Basically, just check the model's name, maintain a list of patterns/names of models not supporting temperature, and if that matches, omit the parameter when calling the connector.

This would be somewhat similar to what we do to "detect" native function calling:

https://github.com/elastic/kibana/blob/main/x-pack/platform/plugins/shared/inference/server/chat_complete/utils/function_calling_support.ts#L11-L26

This meant maintaining (yet another) hardcoded list we will have to maintain, but I feel like this still might be preferable in terms of developer and user experience.

3. Add the parameter in the connector config

Another option would be to add an option in the openAI connector configuration such as "no temperature", so that the customer can manually specify, when creating their connector, that the model/provider does not support the temperature parameter.

It's still not ideal for 2 reasons:

• require input from the customer
• technically, connectors are not bound to a specific model, as you can override the default modelId when calling the connector, so that's not a 1-1 relationship, and attaching that "temperature support" at the connector level isn't really reflecting the reality.