Skip to content
This repository was archived by the owner on Apr 26, 2024. It is now read-only.
This repository was archived by the owner on Apr 26, 2024. It is now read-only.

Waiting for streams to catch up when processing HTTP replication can get stuck timing out #15136

Closed
Closed
Waiting for streams to catch up when processing HTTP replication can get stuck timing out#15136
Labels
A-WorkersProblems related to running Synapse in Worker Mode (or replication)O-UncommonMost users are unlikely to come across this or unexpected workflowS-MajorMajor functionality / product severely impaired, no satisfactory workaround.T-DefectBugs, crashes, hangs, security vulnerabilities, or other reported issues.
@squahtx

Description

@squahtx
squahtx
opened on Feb 22, 2023

#14820 (introduced in v1.76.0) added code to wait for streams to catch up when receiving HTTP replication responses.

We've seen that code hit a failure mode where all waits on the backfill stream start timing out, on a deployment where backfill is very rare. It's not clear how Synapse entered that state or how to reproduce it yet.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-WorkersProblems related to running Synapse in Worker Mode (or replication)O-UncommonMost users are unlikely to come across this or unexpected workflowS-MajorMajor functionality / product severely impaired, no satisfactory workaround.T-DefectBugs, crashes, hangs, security vulnerabilities, or other reported issues.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions