Skip to content

Crashes under runnerv2 due to project tracking semantics #676

@dpcollins-google

Description

@dpcollins-google

Under dataflow RunnerV2, after trySplit returns successfully, the runner does not immediately check the progress accounted for by testing the RestrictionTracker. Instead, it destroys the RestrictionTracker and tries to construct a new one using the already claimed restriction instead. The runner will not do this if the SDF implements the @Getsize annotated method, in which case it will be used instead. As far as I can tell, this particular interplay is undocumented, but it should remain true for this public API.

Our I/O will crash when constructed with an already completed restriction to avoid constructing a restriction tracker in an unusable state. We should implement GetSize so that we don't need to allow for partially unusable RestrictionTrackers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    🚨This issue needs some love.api: pubsubliteIssues related to the googleapis/java-pubsublite API.triage meI really want to be triaged.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions