Skip to content

VStream does not work when keyspaces_to_watch is set on vtgate #8772

@5antelope

Description

@5antelope

Overview of the Issue

VStream relies on topo server to create a tablet picker:

tp, err := discovery.NewTabletPicker(vs.ts, []string{vs.vsm.cell}, sgtid.Keyspace, sgtid.Shard, vs.tabletType.String())

After #7873, when we pass keyspaces_to_watch flag, VTGate will initialize a keyspaceFilteringServer which will return error when we try to get topo server out of it:

func (ksf keyspaceFilteringServer) GetTopoServer() (*topo.Server, error) {

This could be problematic if there is a lot of of keyspaces, and the vtgate has to keep connections to all of them in order to support vstream() endpoint.

@setassociative suggested providing a readonly topo interface in keyspaceFilteringServer, instead of raising error directly - which I think makes sense, it can:

  • avoid the online DDL issue that we were trying to address initially (i don't fully grok the problem yet, but a readonly topo interface that avoids writes seems to be good enough for the specific use case)
  • allow vstream with keyspaces_to_watch flag

Reproduction Steps

Launch VTGate with keyspaces_to_watch flag and hit VStream endpoint on vtgate

Binary version

I'm on top of 4358c6

commit 4358c6a85e607c2676135599f36ef3b53705f004 (upstream/main)
Merge: 6afc58d9df 92e07c2293
Author: Harshit Gangal <harshit@planetscale.com>
Date:   Wed Aug 25 00:25:56 2021 +0530

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions