Skip to content

Publishing needs to be fully synchronous #293

@KyleMaas

Description

@KyleMaas

Okay, so looking through some of the problems here, publishing really needs to be a fully synchronous operation that does not return until all indexes are synced. See

seq := pl.byAuthor.Seq()
and https://github.com/ssbc/go-ssb/blob/master/multilogs/userfeeds.go for why this is the case.

The reasons this needs to be fixed are numerous:

  1. The fact that it's not is causing all kinds of problems with the tests. If the expectation from the folks writing the tests is that it's a synchronous operation, and they know how it's implemented internally, then user expectations are likely to be the same. This has the potential to cause difficult-to-trace issues in any other project using go-ssb components in the future.
  2. Publishing a second message to the publish log by the same author requires that the previous message be indexed (see Possible race condition in the publish log causing a forked feed #292). Which means unless publishing is synchronous, we'd be relying on the caller to ensure that all indexes are synced before calling publish, and everyone using the publishing mechanism needs to know this and do this every time they go to publish. This is (a) difficult, or (b) impossible to ensure, especially in a multithreaded environment.
  3. There really needs to be a mutex on the log so that no new publishing can happen while waiting on the indexes to catch up. This needs to be done in one place. Within the publish log seems to me to be the most logical place to implement that.

This would fix #292 as well as #289, and probably several of the other flaky tests in #237. This is core enough functionality and broken enough to cause a lot of issues to due race conditions pretty much everywhere, especially in the test suite where it often publishes a bunch of messages in quick succession.

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is neededquestionFurther information is requested

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions