-
Notifications
You must be signed in to change notification settings - Fork 24
Description
Under dataflow RunnerV2, after trySplit returns successfully, the runner does not immediately check the progress accounted for by testing the RestrictionTracker. Instead, it destroys the RestrictionTracker and tries to construct a new one using the already claimed restriction instead. The runner will not do this if the SDF implements the @Getsize annotated method, in which case it will be used instead. As far as I can tell, this particular interplay is undocumented, but it should remain true for this public API.
Our I/O will crash when constructed with an already completed restriction to avoid constructing a restriction tracker in an unusable state. We should implement GetSize so that we don't need to allow for partially unusable RestrictionTrackers.