Skip to content

azure: support multiple pods subnets#11268

Merged
tgraf merged 1 commit intocilium:masterfrom
DataDog:azure-ipam-with-multi-subnets
May 6, 2020
Merged

azure: support multiple pods subnets#11268
tgraf merged 1 commit intocilium:masterfrom
DataDog:azure-ipam-with-multi-subnets

Conversation

@bpineau
Copy link
Copy Markdown
Contributor

@bpineau bpineau commented Apr 30, 2020

Support pods attached to secondary interfaces, and spread over several
subnets: inject explicit routes and rules to forward pods traffic to
their subnet gateway band back, set MTU, etc. the same way Cilium does
for AWS/ENI.

We mostly need to propagate subnets gw details (collecting addressPrefix
from instances manager so we can infer the subnets gateways addresses)
down to API / CRD, which can then be leveraged by the CNI plugin as we
do with ENIs.

While at it, also support k8s clusters and vpc/subnets running on
different resource groups.

Signed-off-by: Benjamin Pineau benjamin.pineau@datadoghq.com

Azure: support multiple pods subnets, and networks in different resource groups

@bpineau bpineau requested review from a team as code owners April 30, 2020 20:58
@bpineau bpineau requested a review from a team April 30, 2020 20:58
@maintainer-s-little-helper
Copy link
Copy Markdown

maintainer-s-little-helper Bot commented Apr 30, 2020

Please set the appropriate release note label.

@ungureanuvladvictor ungureanuvladvictor added the integration/cloud Related to integration with cloud environments such as AKS, EKS, GKE, etc. label Apr 30, 2020
@coveralls
Copy link
Copy Markdown

coveralls commented Apr 30, 2020 •
edited
Loading

Coverage Status

Coverage decreased (-0.04%) to 44.422% when pulling b8c9023 on DataDog:azure-ipam-with-multi-subnets into 4bdb361 on cilium:master.

@qmonnet qmonnet added the release-note/minor This PR changes functionality that users may find relevant to operating Cilium. label May 1, 2020
@qmonnet
Copy link
Copy Markdown
Member

qmonnet commented May 1, 2020

test-me-please

@qmonnet qmonnet self-assigned this May 1, 2020
@qmonnet
Copy link
Copy Markdown
Member

qmonnet commented May 1, 2020

test-me-please

@aanm aanm requested a review from a team May 1, 2020 16:55
ungureanuvladvictor
ungureanuvladvictor requested changes May 1, 2020
View reviewed changes
Comment thread plugins/cilium-cni/cilium-cni.go Outdated
Comment thread pkg/ipam/crd.go Outdated
Comment thread pkg/azure/api/api.go Outdated
Comment thread pkg/azure/api/api.go Outdated
@bpineau bpineau force-pushed the azure-ipam-with-multi-subnets branch from 945a51f to f9a4d97 Compare May 2, 2020 00:41
@ungureanuvladvictor
Copy link
Copy Markdown
Member

ungureanuvladvictor commented May 2, 2020

test-me-please

@qmonnet qmonnet requested review from a team and removed request for a team May 2, 2020 11:00
tgraf
tgraf suggested changes May 4, 2020
View reviewed changes
Comment thread plugins/cilium-cni/cilium-cni.go Outdated
Copy link
Copy Markdown
Contributor

@tgraf tgraf May 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename this function if we don't exclusively use it for ENI.

Copy link
Copy Markdown
Contributor Author

@bpineau bpineau May 4, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Relatedly, most of that function meat recently moved to pkg/aws/eni/routing package, which is in no way specific to AWS.

Mind if I rename that to, say, pkg/nic/routing for instance (or another name suggestion @tgraf )?

And while at it, also hook in the newly introduced routes deletion on endpoint removal - perhaps material for a follow up PR?

Copy link
Copy Markdown
Contributor

@tgraf tgraf May 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bpineau Good point. I think it should be moved into pkg/datapath/linux/routing.

@bpineau
azure: support multiple pods subnets
b8c9023 
Support pods attached to secondary interfaces and spread over several
subnets: inject explicit routes and rules to forward pods traffic to
their subnet gateway and back, set MTU, etc. the same way Cilium does
for AWS/ENI.

We mostly need to propagate subnets gw details (collecting addressPrefix
from instances manager so we can infer the subnets gateways addresses)
down to API / CRD, which can then be leveraged by the CNI plugin as we
do with ENIs.

While at it, also support k8s clusters and vpc/subnets running on
different resource groups.

Signed-off-by: Benjamin Pineau <benjamin.pineau@datadoghq.com>
@bpineau bpineau force-pushed the azure-ipam-with-multi-subnets branch from f9a4d97 to b8c9023 Compare May 4, 2020 19:23
@qmonnet
Copy link
Copy Markdown
Member

qmonnet commented May 5, 2020

test-me-please

@qmonnet qmonnet requested a review from tgraf May 5, 2020 12:38
@tgraf tgraf merged commit dd99958 into cilium:master May 6, 2020
@tklauser tklauser mentioned this pull request May 6, 2020
Merged
@christarazi christarazi mentioned this pull request May 6, 2020
Merged
@bpineau bpineau mentioned this pull request May 10, 2020
Merged
bpineau added a commit to DataDog/cilium that referenced this pull request May 14, 2020
@bpineau
cilium: gc routes on endpoint deletion on Azure
f8f1f0d 
Also garbage collect routes on endpoint deletion when running
Azure IPAM mode (the same way we do on AWS instances).

And rename aws/eni/routing to pkg/datapath/linux/routing:
per discussion in cilium#11268 : the routing rules we use here are in
no way specific to AWS ENI; they are for also used to setup
Azure IPAM routes.

Signed-off-by: Benjamin Pineau <benjamin.pineau@datadoghq.com>
aanm pushed a commit that referenced this pull request May 15, 2020
@bpineau @aanm
cilium: gc routes on endpoint deletion on Azure
fac1802 
Also garbage collect routes on endpoint deletion when running
Azure IPAM mode (the same way we do on AWS instances).

And rename aws/eni/routing to pkg/datapath/linux/routing:
per discussion in #11268 : the routing rules we use here are in
no way specific to AWS ENI; they are for also used to setup
Azure IPAM routes.

Signed-off-by: Benjamin Pineau <benjamin.pineau@datadoghq.com>
jrajahalme pushed a commit that referenced this pull request May 20, 2020
@bpineau @jrajahalme
cilium: gc routes on endpoint deletion on Azure
a44e24f 
[ upstream commit fac1802 ]

Also garbage collect routes on endpoint deletion when running
Azure IPAM mode (the same way we do on AWS instances).

And rename aws/eni/routing to pkg/datapath/linux/routing:
per discussion in #11268 : the routing rules we use here are in
no way specific to AWS ENI; they are for also used to setup
Azure IPAM routes.

Signed-off-by: Benjamin Pineau <benjamin.pineau@datadoghq.com>
Signed-off-by: Jarno Rajahalme <jarno@covalent.io>
aanm pushed a commit that referenced this pull request May 25, 2020
@bpineau @aanm
cilium: gc routes on endpoint deletion on Azure
af2f682 
[ upstream commit fac1802 ]

Also garbage collect routes on endpoint deletion when running
Azure IPAM mode (the same way we do on AWS instances).

And rename aws/eni/routing to pkg/datapath/linux/routing:
per discussion in #11268 : the routing rules we use here are in
no way specific to AWS ENI; they are for also used to setup
Azure IPAM routes.

Signed-off-by: Benjamin Pineau <benjamin.pineau@datadoghq.com>
Signed-off-by: Jarno Rajahalme <jarno@covalent.io>
jrajahalme pushed a commit that referenced this pull request May 28, 2020
@bpineau @aanm
cilium: gc routes on endpoint deletion on Azure
b970f4c 
[ upstream commit fac1802 ]

Also garbage collect routes on endpoint deletion when running
Azure IPAM mode (the same way we do on AWS instances).

And rename aws/eni/routing to pkg/datapath/linux/routing:
per discussion in #11268 : the routing rules we use here are in
no way specific to AWS ENI; they are for also used to setup
Azure IPAM routes.

Signed-off-by: Benjamin Pineau <benjamin.pineau@datadoghq.com>
Signed-off-by: Jarno Rajahalme <jarno@covalent.io>
christarazi pushed a commit that referenced this pull request May 28, 2020
@bpineau @christarazi
cilium: gc routes on endpoint deletion on Azure
b5ab80d 
[ upstream commit fac1802 ]

Also garbage collect routes on endpoint deletion when running
Azure IPAM mode (the same way we do on AWS instances).

And rename aws/eni/routing to pkg/datapath/linux/routing:
per discussion in #11268 : the routing rules we use here are in
no way specific to AWS ENI; they are for also used to setup
Azure IPAM routes.

Signed-off-by: Benjamin Pineau <benjamin.pineau@datadoghq.com>
Signed-off-by: Jarno Rajahalme <jarno@covalent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ungureanuvladvictor ungureanuvladvictor ungureanuvladvictor approved these changes

+1 more reviewer

@tgraf tgraf tgraf approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@qmonnet qmonnet

Labels

integration/cloud Related to integration with cloud environments such as AKS, EKS, GKE, etc. release-note/minor This PR changes functionality that users may find relevant to operating Cilium.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@bpineau @coveralls @qmonnet @ungureanuvladvictor @tgraf