Skip to content

[Bug]: asyncio.gather without return_exceptions discards results on failure #2744

@nidhi-singh02

Description

@nidhi-singh02

Description

In tools/web_tools.py, four asyncio.gather() calls do not use return_exceptions=True. If any single LLM summarization task raises an exception, the entire gather fails and all other successfully completed results are lost.

Affected Locations

File Line Context
tools/web_tools.py 486 Chunk summarization gather
tools/web_tools.py 1055 Firecrawl result processing
tools/web_tools.py 1230 Tavily crawl result processing
tools/web_tools.py 1455 Crawl result processing

Impact

When extracting content from multiple web pages in parallel, a transient failure on one page (e.g., rate limit, timeout) causes the agent to lose all results, even from pages that were successfully fetched and summarized.

Fix

Add return_exceptions=True to all four asyncio.gather() calls and filter out BaseException instances from results with appropriate logging.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium — degraded but workaround existstool/webWeb search and extractiontype/bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions