I'm concerned about a footgun with surprising behaviour where a user passes in a normal client side tool without using our tool helpers, where if the model tries to call it, you'd probably get a weird error.

How would a user pass the client side tools without using tool helpers? It is basically only possible with the tool helpers feature.

cause now you can do tool_runner(..., tools=[{'name': 'get_weather', 'type': 'custom', 'input_schema': ...}]) right?

Ah, I see now. The server tool use blocks have other types, not 'tool_use', so our tool runner doesn't see them. For example, the web search tool use block will send 'server_tool_use'

sorry no I mean that you could do this

    runner = client.beta.messages.tool_runner(
        max_tokens=1024,
        model="claude-3-5-sonnet-latest",
        tools=[{"name": "get_weather", "input_schema": {...}}],
        messages=[{"role": "user", "content": "What is the weather in SF?"}],
    )

which form the anthropic API perspective is the exact same as @beta_tool(), so it'll try to call that tool and then we'll get a weird error

In that case, the tool runner will send an error message as a tool call result back to Anthropic, and the user will probably see a final message from Anthropic saying that the user-provided tool wasn't successfully called. Would it be sufficient if we add a warning there, so it would be clear to the user that something is wrong?