Feature Description
When using MoveTables to move data from one sharded keyspace to another, it would be great if you could have the source and target keyspaces have different primary Vindexes.
Currently, Vitess lets you do this, but the results end up being wrong because of the way the target workflow tablet selection works.
For example, given two sharded keyspaces with the same shard spec -80,80- but different primary Vindexes, MoveTables will create VReplication tasks in the target shards which stream only the corresponding source shards. This means that some data may live in source shard -80 which needs to go to target 80-, but won't.
Use Case(s)
When migrating to Vitess, users may have a custom manual sharding setup. Ideally, they would be able to migrate to Vitess by setting up a source keyspace with shards corresponding to the shards of their manual setup, and then use MoveTables to move that data into a target keyspace. (I am hand-waving the challenge of getting Vitess to route queries to the right source shard.) They might want the target keyspace to use a Vitess-native Vindex like hash, rather than whatever custom sharding strategy they employ in the source keyspace.
Another use case is if a user is already in Vitess, and finds out that their primary Vindex is suboptimal for some reason. Perhaps it results in hot shards. They would like to change the primary Vindex to get better performance, data distribution, etc.
Feature Description
When using
MoveTablesto move data from one sharded keyspace to another, it would be great if you could have the source and target keyspaces have different primary Vindexes.Currently, Vitess lets you do this, but the results end up being wrong because of the way the target workflow tablet selection works.
For example, given two sharded keyspaces with the same shard spec
-80,80-but different primary Vindexes,MoveTableswill create VReplication tasks in the target shards which stream only the corresponding source shards. This means that some data may live in source shard-80which needs to go to target80-, but won't.Use Case(s)
When migrating to Vitess, users may have a custom manual sharding setup. Ideally, they would be able to migrate to Vitess by setting up a source keyspace with shards corresponding to the shards of their manual setup, and then use
MoveTablesto move that data into a target keyspace. (I am hand-waving the challenge of getting Vitess to route queries to the right source shard.) They might want the target keyspace to use a Vitess-native Vindex likehash, rather than whatever custom sharding strategy they employ in the source keyspace.Another use case is if a user is already in Vitess, and finds out that their primary Vindex is suboptimal for some reason. Perhaps it results in hot shards. They would like to change the primary Vindex to get better performance, data distribution, etc.