Defining the problem space for asynchronous tool execution #843
Replies: 3 comments 19 replies
-
|
Regarding problems (1) and (2), is there something in the MCP protocol that requires tool calls to be synchronous? I think that tool calls using the HTTP transport are inherently async, so I'm not clear what protocol changes are needed for this. |
Beta Was this translation helpful? Give feedback.
-
|
Ya, we actually spoke about that on the call working group call today, NO, that's not a requirement, so for item 2 in particular that's not an issue. For point 1, it's a bit more like, if you have a multi hour call, the clients generally will sit there waiting for it to come back. So I think point 2 is not really an issue, but point 1 continues to be |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I think we can drop point 2 once we have more documentation and a proper way to handle this on the client side. We might also need to update the SDK, since I don't feel it's properly handle today |
Beta Was this translation helpful? Give feedback.
-
|
Regarding (1), consider adding a tool annotation that declares the maximum expected duration. This information also could go into the description, but implementations can benefit structured access at the protocol level. |
Beta Was this translation helpful? Give feedback.
-
|
We had a good conversation around this in the working group call today - there are a few issues, and a few things that came up. First, there is bluriness between "sessions" from the mcp server perspective, and "sessions" from the client perspective. Maybe they are the same thing, but if you fire a tool call with the intention of waiting for it to come back, and there's a delay of some sort, how do you correlate the tool call with the client (inference?) session when it finally returns. Maybe this isn't an issue, it's a funny one. Second, there was discussion about transport protocol being intimately connected to sessions, which could be an issue long term. Third, it was brought up that polling should never be a requirement for checking for tool completion - it might be one model, but should never be the only model. That is to say, to really allow this to scale we'll want some form of callback at some point so the client doesn't have to be connected and can get alerted to a tool completion. I'll summarize the high level issue I see - the clients as they exist today will wait for a tool response before moving on with their inference session. So, you really CAN'T just let a multi hour tool call sit there holding onto the inference session. So, there are a lot of different ways to solve this, but I'll throw out one which could work with the current protocol (with maybe a few small change) - a tool call happens - the server starts the call, and if it finishes within 10 seconds just returns a response. If it goes further, the call returns a resource pointer. The client can then poll that resource for the results to the call. If the client is smart enough it could setup a subscription to that resource, and that essentially create a model WITHIN A SESSION to have long lived tool calls. I do not believe this is MxN compatible, because most clients would not immediately know to subscribe to the resource to see when it finishes, so a small change that could be made is something like a "pending" resource that would indicate to the client that it will eventually satisfy and it should subscribe to it. A2A has a good solution to this where a task call can include a web hook call back - that solves the async issue, though its not very MCP-ish and doesn't work great with think desktop clients - I think a better model that fits in with MCP would be alternate notification transport mechanisms. So, an example would be being able to register for a webhook callback for any notification. That callback could include some sort of token to resume an MCP session and maybe some sort of correlation ID supplied by the client, so in that way an agent could fire a call, wait for the callback, correlate the callback with a local inference session, and then pull the remote resource. I think this is actually a nice model (and @evalstate brought up the correlation id stuff, that could be generally useful within elicitation and other interactions) and this builds some abstractions around transports that could be really useful downstream. |
Beta Was this translation helpful? Give feedback.
-
|
it's the last part, btw, that is why I'm a bit drawn to it - I hate how everything ends up getting a new set of messages (think elicitation), so being able to piggy back on an existing mechanism and no new messages is a big plus for me (speaking from the backend perspective here) |
Beta Was this translation helpful? Give feedback.
-
|
I don't actually love the proposal that's floating out there, because it actually intentionally sidesteps the subscriptions if I'm reading it right? Whereas we really want to do this using all the primitives we already have, that's why I like a resource return that's a pending resource, rather than doing this over the top in metadata like the proposal says |
Beta Was this translation helpful? Give feedback.
-
|
On a related note, a few folks have been discussing streaming tool calls with partial results #776.
|
Beta Was this translation helpful? Give feedback.
-
|
I've created this doc which has summed up all the approaches that are out there with their pros and cons
This is option 2 in the doc and what I suggested in my first RFC |
Beta Was this translation helpful? Give feedback.
-
|
@bzsurbhi I like your proposal, I think something got lost in the long conversation - the benefit of resources here is that you can either poll or subscribe for updates. So, that completely fits into the current model. Also, discussion about progress and stuff got intermingled, which is orthogonal. Also, someone in your conversation said it wasn't backwards compatible which is completely incorrect - net new fields are absolutely backwards compatible. I think your proposal needs some documentation / clarification on what the resource looks like if it's in a pending state (ie whats the empty state) and documentation that you can subscribe to a pending state resource and that status changes MUST trigger a notification. I think that's the bare minimum set of changes to support async tool results |
Beta Was this translation helpful? Give feedback.
-
|
@evalstate what do you think about a small pr to allow clients to include a client_session_id that round trips through the server on requests, included in tool returns, elicitation requests, etc |
Beta Was this translation helpful? Give feedback.
-
|
Highlighting another relevant discussion here: #823 I think there are a few things to work through before we can get to a solution, and would propose we continue the discussion here and in the CWG Discord to form a proposal). Some immediate questions below: - note a large number of these are choices the Host application developer would need to make, but helpful for framing the solutions/giving guidance in the docs.
Whilst I think this is very important, it's also the case that we've got this far (~9 months) without it, and the current default of generally stateless Servers (or stateful, but STDIO local servers like basic-memory) is working OK. But as we see more remote Servers, and now DXT distribution etc. this is going to need attention. I propose continuing this thread and CWG discussions; and in the meantime I'll figure out with the other moderators how much overlap/underlap this has with the other initiatives. |
Beta Was this translation helpful? Give feedback.
-
|
Ya, I mean, I think this can be very simple - at the end of the day, session id is assigned by the server, and for security reasons that must always happen that way. There is NECESSARILY an orthogonality between a server assigned ID that has timeouts and the like, and a client supplied ID that is purely a pass through from the server perspective, ie the server never looks at this, never contemplates it, just passes it back through as a form of client supplied correlation id. It's different than more complex discussions about coordinating client and server sessions, and honestly just a lot simpler |
Beta Was this translation helpful? Give feedback.
-
|
That way, it could be used to handle single tool correlation, but also longer client side context / session correlation. I'd write it up something like on the metadata of various server calls (tool calls) and it would stream back through into the meta. And, the description would be something like "The correlation_id MUST be returned in any call that is directly spawned by a client call. If a tool execution results in an elicitation, both the elicitation and the tool call result MUST include the correlation id. If a client subscribes to a resource change and the subscription request call has a correlation_id, it MUST be supplied as part of any notification change event." Something like that, just keeping this VERY separate from sessions |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Motivation
There is a growing consensus within the community that the current request/response model for
tools/executepresents significant challenges for real-world applications. This theme is evident across several ongoing discussions and proposals (e.g., #617, #651, #650, #543)The goal of this new discussion is to consolidate these concerns and formally define the problem space we need to solve.
Problem Statement
The fundamental issue is that many critical tool interactions do not fit the simple, blocking request/response paradigm. This is especially true for tools that trigger jobs in cloud environments or interact with complex enterprise systems. We can break the problem down into the following key areas:
1. Unresponsive Clients and Degraded User Experience
Interactive clients (IDEs, desktop apps) must remain responsive at all times. A
tools/executecall that blocks the client while waiting for a response is not viable. For any operation that takes more than a few hundred milliseconds, the user interface will "freeze". This is a critical usability failure.sequenceDiagram participant User participant IDE_Client participant MCP_Server User->>IDE_Client: "Analyze workspace for tech debt" IDE_Client->>MCP_Server: tools/execute (long-running task) Note over IDE_Client: Chat thread is blocked. User-->>IDE_Client: Tries to send a new chat (fails) MCP_Server-->>IDE_Client: Response (after several minutes) Note over IDE_Client: Chat thread is unblocked.2. Inefficient Agentic Systems
Advanced AI agents often need to orchestrate multiple tools to achieve a goal. A blocking execution model forces these tool calls into a slow, sequential chain. An agent's total execution time becomes the sum of all individual tool latencies, crippling its ability to perform complex, multi-source tasks in a timely manner.
sequenceDiagram participant Agent participant Salesforce_Server participant Slack_Server participant GitHub_Server Agent->>Salesforce_Server: tools/execute (createAccount) Salesforce_Server-->>Agent: Response (OK) Agent->>Slack_Server: tools/execute (createChannel) Slack_Server-->>Agent: Response (OK) Agent->>GitHub_Server: tools/execute (grantAccess) GitHub_Server-->>Agent: Response (OK) Note right of Agent: Total time = Latency(SF) + Latency(Slack) + Latency(GitHub)3. Incompatibility with Long-Running and Indefinite Jobs
Many modern workflows involve operations that are inherently durational and cannot be expected to complete quickly, if at all. This is especially true for jobs offloaded to cloud services.
sequenceDiagram participant Client participant MCP_Server participant Cloud_Service Client->>MCP_Server: tools/execute (transcribe video) MCP_Server->>Cloud_Service: Start transcription job Note over Client, MCP_Server: Client holds connection open, waiting... Client--xMCP_Server: Connection times out after 60s Note over Client: Request failed. User is confused. Note over Cloud_Service: Job continues running in the background, but result can never be delivered.4. Lack of State and Certainty After Disconnection
In a distributed system, network connections are unreliable. If a client disconnects after sending a
tools/executerequest, it has no way of knowing the state of that operation. This is especially problematic for non-idempotent actions.sequenceDiagram participant Mobile_Client participant MCP_Server Mobile_Client->>MCP_Server: tools/execute (provisionServer) Note over Mobile_Client: Connection drops Mobile_Client--xMCP_Server: Disconnected MCP_Server-->>MCP_Server: Processes request, provisions server Note over Mobile_Client: State is unknown. Did it work? Should I retry? Note over MCP_Server: Server has no way to report completion to the disconnected client.This discussion aims to gather community input to ensure we have a comprehensive definition of these problems before we move toward architecting a solution.
Beta Was this translation helpful? Give feedback.
All reactions