My perspective on migration to v3:

Since envoy will call different handlers for v2 vs v3, the control plane has to make sure it handles both handlers.
Since we use the canonical go-control-plane, the auto generated proto code enables services/methods for v2 & v3. However, the separate services have to be registered with the grpc server.
This is additional work for operators of control planes.
since the go grpc boiler plate works with strong types, it creates Send/Recv separately for v2 and v3. This is where the complexity starts. Although most fields are compatible, the deprecated fields make it hard to make a generic solution in the go-control-plane.
As a result the go-control-plane puts the onus of handling separate versions on the private control planes.
At Lyft, we operate a private control plane.
We need a deployment strategy to granularly migrate to v3 without changing too much code in one go. Like mentioned above, the onus of handling the separate versions is on the private control plane. One way to pursue the migration would be to make sure we serve the same response to v2/v3 requests and avoid creating a layer that understands the version semantics and changing typeurl manually. I think Support mixed v2/v3 xDS request/responses #10776 handles the use case. (We understand that for this to work, we we cannot have any deprecated fields in configs) . This approach helps in rolling back easily if anything goes wrong.
If (3) goes well, we can change all code in control plane to v3. All v3 sidecars will be current. Any lagging envoys at this point can still operate as long as control plane doesn't send new v3 fields.
When all envoys and their bootstrap configs are updated to use v3 resource version, we can call the migration done. Due to unavoidable circumstances, there are going to be envoy sidecars or configs in the fleet which are few weeks old.
migration of rds/eds is easier since they can be easily rolled forward/backward by control plane code. Migration of cds/lds is more tricky since they live in bootstrap configs. And rollback will take time to deploy and cause disruptions. It would be great if there was a way to handle this use case.

v2 -> v3 API postmortem #10943

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions