Skip to content

add L3 dependent L4 policies#1217

Closed
amreshakim wants to merge 4 commits intomasterfrom
wip-pr-789
Closed

add L3 dependent L4 policies#1217
amreshakim wants to merge 4 commits intomasterfrom
wip-pr-789

Conversation

@amreshakim
Copy link
Copy Markdown
Contributor

@amreshakim amreshakim commented Jul 28, 2017

WIP

#1217

Signed-off-by: Amre Shakimov amre@covalent.io

@amreshakim amreshakim requested a review from tgraf July 28, 2017 16:01
@amreshakim amreshakim added kind/enhancement This would improve or streamline existing functionality. wip labels Jul 28, 2017
Comment thread pkg/policy/api/rule.go
// member will have no effect on the rule.
// - All members of this structure are evaluated independently, i.e. L4 ports
// allowed with ToPorts do not depend on a match of the FromEndpoints in the
// same IngressRule.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to also add a comment which says that if multiple members are set, all of them need to match in order for the rule to take effect. And also add something like: The exception to this rule is the FromRequires field, the effects of any Requires field in any rule will apply to all other rules as well.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread pkg/policy/rule.go
return 1
}
l4Filter := CreateL4Filter(r, p, dir, proto)
l4Filter := CreateL4Filter(fromEndpoints, r, p, dir, proto)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may need further thought. Imagine the case where we have the following rules:

  ingress:
  - fromEndpoints:
    - matchLabels:
        id: app2
    toPorts:
    - ports:
      - port: "80"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/public"
  - toPorts:
    - ports:
      - port: "80"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/public"

I think we would agree that if we have two rules which both cover port 80/tcp. One is restricted to fromEndpoints, one is not. The one without fromEndpoints should take precedence as it also includes the more restrictive rule.

I thik we need to clear existing fromEndpoints conditions if we encounter a unrestricted toPorts and we need to skip conditional toPorts if we already have an unrestricted L4 rule.

L7 rule will potentially need to be stored per fromEndpoints as the rules should not apply to other endpoints. So while we want to potentially skip/clear fromEndpoints restrictions on pure L4 allow/disallow. We always need to store the L7 rules per fromEndpoints. We will need to encode logic in the proxy which allows matching on the source identity.

What do you think?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick review!

I think we would agree that if we have two rules which both cover port 80/tcp. One is restricted to fromEndpoints, one is not. The one without fromEndpoints should take precedence as it also includes the more restrictive rule.

Hrm, to me this is not as obvious. For some reason, I'd expect the opposite actually. Or, perhaps, "unexpected behavior"? :)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have a disjunction between rules ("or"). So if rule A is less restrictive than rule B, then rule A should have precedence. But I wouldn't talk about "precedence" because there's no order or priority.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

personally i'd like to have an option to disallow overlapping rules to avoid being overly permissive unintentionally, but in general i agree with @rlenglet .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with that, in particular if used in combination with the Cilium REST API. We can reject rules there. As for k8s, the best we could do is report ignored/rejected rules in the Status field though.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

L7 rule will potentially need to be stored per fromEndpoints as the rules should not apply to other endpoints. So while we want to potentially skip/clear fromEndpoints restrictions on pure L4 allow/disallow. We always need to store the L7 rules per fromEndpoints. We will need to encode logic in the proxy which allows matching on the source identity.

@tgraf just to make sure I understand this correctly. If the rule has a number of rules with explicit FromEps lists, we should make sure L7 rules are not merged. However, do we also clear all existing FromEps->L7 entries if there is a rule without explicit set of FromEps?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the logic should be:

  1. Any L7 rule with 1..n combined FromEndpoints must be appended to a list which is maintained for each endpoint in the FromEndpoints, e.g. if we have the independent rules: r1 from {a,b} and r2 from {b,c}, then b must be able to do {r1, r2}.
  2. Any L7 rule without an explicit combined FromEndpoints must be appended to all lists maintained and also to a wildcard list. We may also maintain the wildcard list only and then append the wildcard list to all maintained lists at the end. E.g. if we have an additional, unconstrained r3 then {a, b, c and d} must be able to do r3.

@amreshakim amreshakim force-pushed the wip-pr-789 branch 3 times, most recently from 83f8fb9 to 36edc48 Compare July 31, 2017 23:47
Comment thread pkg/policy/l4.go Outdated
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment on exported type L7DataMap should be of the form "L7DataMap ..." (with optional leading article)

Comment thread pkg/policy/l4.go Outdated
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exported const WildcardEndpointSelector should have comment (or a comment on this block) or be unexported

Comment thread pkg/policy/l4.go Outdated
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exported type L7Rules should have comment or be unexported

@amreshakim amreshakim requested review from aanm and ianvernon August 1, 2017 22:36
@amreshakim amreshakim removed the wip label Aug 2, 2017
Copy link
Copy Markdown
Member

@ianvernon ianvernon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly documentation nitpicks. If possible, I recommend adding a test for these types of policy to our CI (in tests directory).

Comment thread pkg/policy/l4.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

L7 rules

Comment thread pkg/policy/l4.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add documentation for this type?

Comment thread pkg/policy/l4.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the documentation to include context for the parameter you added, as well as for direction / protocol strings.

Comment thread pkg/policy/l4.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please name the import for logrus to "log" for consistency.

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name the import to "log" so that logging statements are "log.Debugf..." for consistency.

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

L7

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though this function is not exported, I think it needs documentation.

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EP -> endpoint for consistency

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ep -> endpoint for consistency

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to above - even though this function is not visible outside of this package, some documentation would be very helpful.

Copy link
Copy Markdown
Member

@aanm aanm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed over slack. Can't add the Hash method in vendor like that.

Comment thread pkg/policy/l4.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the users print cilium endpoint get XXX they will see this in the output. It might be confusing:

"policy": {
    "allowed-consumers": [
      1,
      268,
      269
    ],
    "build": 60,
    "id": 269,
    "l4": {
      "egress": [],
      "ingress": [
        "{\n  \"Port\": 9080,\n  \"Protocol\": \"TCP\",\n \"FromEndpoints\":\"There will be duplicated and lots of verbose that here\"  \"L7Parser\": \"\",\n  \"L7RedirectPort\": 0,\n  \"L7Rules\": null,\n  \"Ingress\": true\n}"
      ]
    }
  },

I suggest you append json:"-" at the end to omit this field from the output

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nit, why not?

		wildcardEp := L7Rules{}
		for _, existingL7Rules := range v.L7RulesPerEp {
			wildcardEp = append(wildcardEp, existingL7Rules...)
		}
                v.L7RulesPerEp[WildcardEndpointSelector] = wildcardEp

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work as you're expecting, replace with
v.L7RulesPerEp[hash] = append(ep, newL7Rules...)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Copy Markdown
Contributor Author

@amreshakim amreshakim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved Hash to endpoint

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread pkg/policy/rule.go Outdated
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@amreshakim amreshakim force-pushed the wip-pr-789 branch 2 times, most recently from 124d0b2 to 0269e7f Compare August 3, 2017 00:34
@aanm
Copy link
Copy Markdown
Member

aanm commented Aug 3, 2017

@amreshakim I forgot to ask you, can you also add a runtime test for this?

@amreshakim
Copy link
Copy Markdown
Contributor Author

@aanm for true "e2e" runtime tests, I believe, there is some DP work that needs to be done. There is a unit test I am fixing right now that is expected to fail (network_policy_test.go) which covers bulk of the changes here.
I am happy to add more tests after the feature is fully in. @tgraf, who will be working on the DP side of things? I am happy to volunteer but it might be too big of a piece to bite, perhaps?

WIP

Signed-off-by: Amre Shakimov <amre@covalent.io>
Signed-off-by: Amre Shakimov <amre@covalent.io>
Signed-off-by: Amre Shakimov <amre@covalent.io>
@amreshakim amreshakim force-pushed the wip-pr-789 branch 2 times, most recently from a3795cb to cffacd9 Compare August 3, 2017 07:26
@tgraf
Copy link
Copy Markdown
Contributor

tgraf commented Aug 3, 2017

I am happy to add more tests after the feature is fully in. @tgraf, who will be working on the DP side of things? I am happy to volunteer but it might be too big of a piece to bite, perhaps?

The datapath should be done. I can rebase on top of this and push them into this branch. The work is currently parked in PR #1064 and needs a rebase. I will take care of it.

Signed-off-by: Amre Shakimov <amre@covalent.io>
@amreshakim
Copy link
Copy Markdown
Contributor Author

@aanm @tgraf I fixed most of the runtime tests and I think it also gives a pretty good coverage for this change. I will wait for your reviews to close the loop on all remaining tests.

Copy link
Copy Markdown
Contributor

@tgraf tgraf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amreshakim LGTM!

Comment thread pkg/policy/l4.go
l4.L7RulesPerEp[hash] = append(l4.L7RulesPerEp[hash], l7rules...)
}
} else if l4.Ingress {
// for ingress only, if there are no explicit fromEps, have a 'special' wildcard endpoint
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amreshakim Why is this for ingress only?

Copy link
Copy Markdown
Member

@joestringer joestringer Sep 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, seems like it may just be because egress L7Filters were not supported when this was written.

@tgraf
Copy link
Copy Markdown
Contributor

tgraf commented Oct 2, 2017

Obsoleted by #1599

@tgraf tgraf closed this Oct 2, 2017
@tgraf tgraf deleted the wip-pr-789 branch June 19, 2018 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement This would improve or streamline existing functionality.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants