Skip to content

Background tasks in PromptStreamConsumer can be garbage-collected mid-run (silent prompt-execution drops) #819

@kratos0718

Description

@kratos0718

Summary

PromptStreamConsumer (futureagi/sockets/prompt_stream_consumer.py) launches its core work with bare asyncio.create_task(...) and discards the returned task. The event loop only keeps a weak reference to a task, so a fire-and-forget task can be garbage-collected before it finishes running. When that happens the user's prompt execution is silently dropped — no result is streamed back and no error is raised.

This affects all three streaming entry points:

  • handle_execute_template — line 120 → execute_template_async
  • handle_improve_prompt — line 224 → execute_improve_prompt_async
  • handle_generate_prompt — line 315 → execute_generate_prompt_async

Why it matters

These tasks are the request — they run the prompt and stream the response over the WebSocket. If the task is collected mid-run, the client just sees the stream stall after execution_started with nothing after it. It's intermittent and load-dependent (GC-timing), which makes it the kind of "works on my machine, flaky in prod" bug that's painful to track down.

This is the documented failure mode in the CPython asyncio docs for create_task:

Important: Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn't referenced elsewhere may get garbage collected at any time, even before it's done.

Suggested fix

Keep a strong reference to each spawned task until it completes (and cancel any still-pending tasks on disconnect). I've got a small patch ready and will open a PR.

How I found it

Ran my static analyzer (codehound) over the backend — it flags discarded create_task/ensure_future results (check CH006). It also flagged a related spot in model_hub/utils/kb_helpers.py (deprecated get_event_loop() + a fire-and-forget ensure_future) which I can follow up on separately to keep this PR focused.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions