Skip to content

bind variables not pruned when doing cross-shard inserts #14077

@harshit-gangal

Description

@harshit-gangal

Overview of the Issue

  • Do a large (i.e. large number of values) cross-shard insert on a sharded table that inserts a subset of the rows on each shard
  • Capture the network traffic between vtgate and each primary vttablet's gRPC port
  • Observe from the network traffic that the full bind variable map is sent to every shard, even though the insert will only reference a subset of those variables for that shard.
  • This behaviour can be confirmed by inspection of the vtgate executor, plan_execute and engine insert code
  • The overhead here gets worse as the number of rows and the number of shards grow. Vtgate spends huge amounts of CPU time in gRPC marshalling when dispatching an insert like this.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions