Skip to content

[BUG] AzureOpenAiChatModel sends 'stop' parameter even when not set #3453

Description

@m-feilen

Describe the bug
When using AzureOpenAiChatModel with GPT-o4-mini, the stop parameter is sent to Azure even if it is not explicitly set in the builder.
This causes a 400 error from Azure: "Unsupported parameter: 'stop' is not supported with this model."

Possible workaround
After applying a workaround for the likewise unsupported attribute "logitBias"

model.javaClass.getDeclaredField("logitBias").let {
    it.isAccessible = true
    it.set(model, null)
}

I tried the same with the "stop" attribute, which did not work.
I was only able to prevent this by removing the parameter inside a listener callback at runtime,
since this is the only way to access the finalChatRequest

class RemoveListener: ChatModelListener {
    override fun onRequest(requestContext: ChatModelRequestContext) {
        val parameters = requestContext.chatRequest().parameters()
        parameters.javaClass.getDeclaredField("stopSequences").let {
            it.isAccessible = true
            it.set(parameters, null)
        }
    }
}

fun main() {
    AzureOpenAiChatModel.builder()
            .endpoint("...")
            .apiKey("...")
            .deploymentName("o4-mini")
            .listeners(listOf(RemoveListener()))
            .build()
}

Log and Stack trace

Exception in thread "main" dev.langchain4j.exception.InvalidRequestException: Status code 400, "{
  "error": {
    "message": "Unsupported parameter: 'stop' is not supported with this model.",
    "type": "invalid_request_error",
    "param": "stop",
    "code": "unsupported_parameter"
  }
}"
	at dev.langchain4j.internal.ExceptionMapper$DefaultExceptionMapper.mapHttpStatusCode(ExceptionMapper.java:71)
	at dev.langchain4j.model.azure.AzureOpenAiExceptionMapper.mapException(AzureOpenAiExceptionMapper.java:19)
	at dev.langchain4j.internal.ExceptionMapper.withExceptionMapper(ExceptionMapper.java:31)
	at dev.langchain4j.model.azure.AzureOpenAiChatModel.doChat(AzureOpenAiChatModel.java:200)
	at dev.langchain4j.model.chat.ChatModel.chat(ChatModel.java:46)
	at dev.langchain4j.model.chat.ChatModel.chat(ChatModel.java:76)
	at com.rhenus.the.mentor.utils.OpenAITestKt.main(OpenAITest.kt:21)
	at com.rhenus.the.mentor.utils.OpenAITestKt.main(OpenAITest.kt)

To Reproduce

  • Create an AzureOpenAiChatModel using the builder
  • Do not set .stopSequences(...) in the parameters
  • Call .chat(...)
  • Observe the 400 error from Azure

Expected behavior
If stopSequences is not set, the stop parameter should not be included in the request payload.

Please complete the following information:

  • LangChain4j version: langchain4j-1.1.0 respectively langchain4j-azure-open-ai-1.1.0-rc1
  • LLM(s) used: o4-mini
  • Java version: Temurin JDK 21 (Kotlin 2.2.0)
  • Spring Boot version (if applicable): 3.5.3

Additional context
relates to #3258

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1Highest priorityP2High prioritybugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions