Describe the bug
When using AzureOpenAiChatModel with GPT-o4-mini, the stop parameter is sent to Azure even if it is not explicitly set in the builder.
This causes a 400 error from Azure: "Unsupported parameter: 'stop' is not supported with this model."
Possible workaround
After applying a workaround for the likewise unsupported attribute "logitBias"
model.javaClass.getDeclaredField("logitBias").let {
it.isAccessible = true
it.set(model, null)
}
I tried the same with the "stop" attribute, which did not work.
I was only able to prevent this by removing the parameter inside a listener callback at runtime,
since this is the only way to access the finalChatRequest
class RemoveListener: ChatModelListener {
override fun onRequest(requestContext: ChatModelRequestContext) {
val parameters = requestContext.chatRequest().parameters()
parameters.javaClass.getDeclaredField("stopSequences").let {
it.isAccessible = true
it.set(parameters, null)
}
}
}
fun main() {
AzureOpenAiChatModel.builder()
.endpoint("...")
.apiKey("...")
.deploymentName("o4-mini")
.listeners(listOf(RemoveListener()))
.build()
}
Log and Stack trace
Exception in thread "main" dev.langchain4j.exception.InvalidRequestException: Status code 400, "{
"error": {
"message": "Unsupported parameter: 'stop' is not supported with this model.",
"type": "invalid_request_error",
"param": "stop",
"code": "unsupported_parameter"
}
}"
at dev.langchain4j.internal.ExceptionMapper$DefaultExceptionMapper.mapHttpStatusCode(ExceptionMapper.java:71)
at dev.langchain4j.model.azure.AzureOpenAiExceptionMapper.mapException(AzureOpenAiExceptionMapper.java:19)
at dev.langchain4j.internal.ExceptionMapper.withExceptionMapper(ExceptionMapper.java:31)
at dev.langchain4j.model.azure.AzureOpenAiChatModel.doChat(AzureOpenAiChatModel.java:200)
at dev.langchain4j.model.chat.ChatModel.chat(ChatModel.java:46)
at dev.langchain4j.model.chat.ChatModel.chat(ChatModel.java:76)
at com.rhenus.the.mentor.utils.OpenAITestKt.main(OpenAITest.kt:21)
at com.rhenus.the.mentor.utils.OpenAITestKt.main(OpenAITest.kt)
To Reproduce
- Create an AzureOpenAiChatModel using the builder
- Do not set .stopSequences(...) in the parameters
- Call .chat(...)
- Observe the 400 error from Azure
Expected behavior
If stopSequences is not set, the stop parameter should not be included in the request payload.
Please complete the following information:
- LangChain4j version: langchain4j-1.1.0 respectively langchain4j-azure-open-ai-1.1.0-rc1
- LLM(s) used: o4-mini
- Java version: Temurin JDK 21 (Kotlin 2.2.0)
- Spring Boot version (if applicable): 3.5.3
Additional context
relates to #3258
Describe the bug
When using AzureOpenAiChatModel with GPT-o4-mini, the stop parameter is sent to Azure even if it is not explicitly set in the builder.
This causes a 400 error from Azure: "Unsupported parameter: 'stop' is not supported with this model."
Possible workaround
After applying a workaround for the likewise unsupported attribute "logitBias"
I tried the same with the "stop" attribute, which did not work.
I was only able to prevent this by removing the parameter inside a listener callback at runtime,
since this is the only way to access the
finalChatRequestLog and Stack trace
To Reproduce
Expected behavior
If stopSequences is not set, the stop parameter should not be included in the request payload.
Please complete the following information:
Additional context
relates to #3258