What happened?

Runners can ensure compatibility across multiple jobs runs of encoded schemas via encoding positions. Dataflow runner in particular sets encoding position overrides to maintain compatibility across pipelines.

RowCoderGenerator utilizes these to map encoded index to the current Row representation. However it neglects to do so for writing the null bit set here and when reading the null bit set here

This leads to corruption when reading with encodedPositionOverrides that do not match the field ordering and additionally corrupts state when writing the null bit set based upon field ordering instead of encoding position.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner