Fix isValidIP in iptable-start.sh and add unit test for it.#13563
Merged
istio-testing merged 9 commits intoistio:masterfrom May 1, 2019
sbezverk:fix_13562
Merged
Fix isValidIP in iptable-start.sh and add unit test for it.#13563istio-testing merged 9 commits intoistio:masterfrom sbezverk:fix_13562
istio-testing merged 9 commits intoistio:masterfrom
sbezverk:fix_13562
Conversation
Contributor
Author
|
/cc @incfly |
Member
|
Can we add some tests to this ? even though it's bash this is simple function so should be possible |
Contributor
Author
|
@howardjohn what would trigger test? please suggest. |
Member
|
Can probably add a make target? not sure how/if we have other tests like this |
tiswanso
reviewed
Apr 24, 2019
Contributor
There was a problem hiding this comment.
digging in more, I don't think this really fixes it. The *s will glob any char. IMO, moving to something like this is better:
if [[ ${ip_cidr} =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\/?[0-9]*$ ]]; then
true
elif ...
Contributor
There was a problem hiding this comment.
Also, need to fix other places where it tries to strip the pattern--e.g. isIPv4()
I'm worried the ipv6 matching is buggy too
Contributor
|
RE: testing... quickest way to create a test would probably be to add a flag to the script to "test" itself and a function in there to call the util functions with a set of params and check results. That could be invoked under Staying with bash, an alternative would be to pull out the utils into another file and have some go test code somewhere that uses exec to source and exec util functions and check results. Of course, creating another file would require changes in all the places this script is packaged (init-container dockerfile, & maybe mesh expansion stuff). |
|
@tiswanso re creating new util functions in separate file while staying with bash. I think we can workaround by adding a top level function for all the default intialization in iptable-start.sh. And then outside case $1 in
test-case)
do_nothing # just to get those util functions defined in the bash to be sourced.
*)
do_all_default_setup_as_todayAnd the in the go test, we can run the iptable-start.sh as "test-case" option. |
|
Could we continue making progress on this fix? |
Contributor
Author
|
@incfly I will complete either over the weekend or Monday the latest, on training this week.. |
Contributor
Author
|
I think we should use these 2 regex, they are not ideal but I think it is as close as we could get using bash without making regex completely unreadable. |
tiswanso
reviewed
Apr 26, 2019
Contributor
There was a problem hiding this comment.
this won't match "::1" so I think you just change it to
ipv6regexp="^([0-9a-fA-F]{0,4}:){0,7}[0-9a-fA-F]{1,4}$"
Contributor
Author
There was a problem hiding this comment.
Fixed, thank you for catching it
Contributor
There was a problem hiding this comment.
Actually, per our slack discussion the way you have regex makes sense for this file since localhost wouldn't be used in the included/excluded ranges and any addresses would lead with a uni/any/multi-cast or linklocal routing prefix.
Contributor
Author
There was a problem hiding this comment.
removed in the latest revision..
|
Thanks for the fix. Let's ensure this time we have it well tested. I can help to look into what can be done to add a test target. Shouldn't be hard, I think. |
Contributor
Author
incfly
reviewed
Apr 28, 2019
There was a problem hiding this comment.
can you ensure the debian/rpm/sidecar docker include validations.sh script?
Contributor
Author
There was a problem hiding this comment.
I could not find a place where istio-iptables.sh get explicitely added to the image, so I suspect whole tools/packaging/common folder is copied for every image flavour. If you think it is not the case please point me where copying of the file happens.
There was a problem hiding this comment.
code search on istio-iptables.sh shows multiple references.
https://github.com/istio/istio/search?q=istio-iptables.sh&unscoped_q=istio-iptables.sh
This includes init container, Debian package and rpm package. I don't know all the build details, but I think at least validations.sh must be able to be referenced by istio-iptables.sh in all those environments.
Also you reference validations.sh as . ./tools/packaging/common/validations.sh. This will work with Git workdir, but I doubt istio-iptable.sh will be invoked and has also such reletive path in environments I mentioned above.
I'm not sure how the presubmit test using init container docker image can pass if validation.sh can't be resolved. might not be a problem thought, just not clear to me.
Figuring this out might be a bit of complicated... So thats why i initially suggested to add another command level option in this script just for test. look at usage().
There was a problem hiding this comment.
By the way, splitting script to different files is definitely great practice, IMO, just some additional use cases & build stuff need to verify.
Contributor
Author
There was a problem hiding this comment.
@incfly If validations.sh were missing in any image it would have never passed CI, as I think docker images are build for e2e tests.
Contributor
There was a problem hiding this comment.
There is no good reason to add complexity or have validation as a separate file.
I'm already concerned that the validation script is too complicated already - and the fact it broke stuff proves it.
We get the IP from a trusted source - it doesn't need validation, and there are far simpler way to detect if it is ipv6 or ipv4. Just check if it has a ":" in it, or doesn't have "."
Over complex code leads to bugs, validating the output of ip commands is not needed.
Contributor
Author
There was a problem hiding this comment.
maybe in this specific case we get it from the trusted source, but people tend to re-use already developed functions, so there is a good chance it would be reused and with proposed approach it will not validate. Besides this version of validation comes with unit testing which is covering pretty much any variations, so the chances of introducing a bug aI would say fairly low. If you do not like complexity, I am afraid then whole Istio code base needs to be revisited as it is far from being simple.
bin/check_shell_scripts.sh
Outdated
There was a problem hiding this comment.
Personally I think this is okay, but I guess this shellcheck was used for lint.
https://github.com/istio/istio/blob/master/Makefile#L446, add a test target to the TEST_OBJ or existing common_test might fit your intent better (functionality test), rather than style.
There was a problem hiding this comment.
TEST_OBJ eventually will be part of make test, and that will run on prow & circle ci.
Contributor
Author
Now it runs as a part of |
howardjohn
reviewed
Apr 28, 2019
Makefile
Outdated
Member
There was a problem hiding this comment.
Might just be my display but the indent looks off? I think make is pretty strict about the indent
Contributor
Author
There was a problem hiding this comment.
Double checked and looks good on my side and if there were a problem make would have complained about it.
linsun
approved these changes
Apr 29, 2019
Member
linsun
left a comment
There was a problem hiding this comment.
/approve
thanks for having good tests for this.
|
Vadliation.sh does not to include in debian pkg. Ways to reproduce make sidecar.deb
dpkg-deb -c $GOPATH/out/linux_amd64/release/istio-sidecar.deb
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./usr/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./usr/local/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./usr/local/bin/
-rwxr-x--- 0/0 25132952 2019-04-29 10:15 ./usr/local/bin/envoy
-rwxr-x--- 0/0 2189 2019-04-25 16:40 ./usr/local/bin/istio-node-agent-start.sh
-rwxr-xr-x 0/0 21341 2019-04-29 10:10 ./usr/local/bin/istio-iptables.sh
-rwxr-x--- 0/0 18887544 2019-04-24 14:38 ./usr/local/bin/node_agent
-rwxr-x--- 0/0 40033110 2019-04-24 14:38 ./usr/local/bin/pilot-agent
-rwxr-xr-x 0/0 4479 2019-04-26 10:15 ./usr/local/bin/istio-start.sh
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./usr/share/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./usr/share/doc/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./usr/share/doc/istio-sidecar/
-rw-r--r-- 0/0 138 2019-04-29 10:15 ./usr/share/doc/istio-sidecar/changelog.gz
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./var/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./var/lib/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./var/lib/istio/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./var/lib/istio/envoy/
-rw-r----- 0/0 174 2019-04-25 16:40 ./var/lib/istio/envoy/envoy_bootstrap_drain.json
-rw-r----- 0/0 8588 2019-04-25 16:40 ./var/lib/istio/envoy/envoy_bootstrap_tmpl.json
-rw-r----- 0/0 3562 2019-04-25 16:40 ./var/lib/istio/envoy/sidecar.env
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./lib/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./lib/systemd/
drwxr-xr-x 0/0 0 2019-04-29 10:15 ./lib/systemd/system/
-rw-r----- 0/0 250 2019-04-25 16:40 ./lib/systemd/system/istio-auth-node-agent.service
-rw-r----- 0/0 222 2019-04-25 16:40 ./lib/systemd/system/istio.service |
|
I tried that on your branch. |
costinm
reviewed
Apr 29, 2019
Contributor
There was a problem hiding this comment.
Wait, let's not make this complicated. Include it in this file, we don't want 2 files.
Contributor
There was a problem hiding this comment.
If you want to make it testable - add another option so iptables can be sourced ( i.e. doesn't do anything ), and include it from the test.
Contributor
Author
There was a problem hiding this comment.
@costinm I am sorry but I disagree, modifying "main"application logic with extra parameter just to be able to test it, does not make too much sense to me. Having separate function file allows flexible testing and usage.
costinm
reviewed
Apr 29, 2019
Contributor
costinm
left a comment
There was a problem hiding this comment.
/hold
This will break mesh expansion at least - and probably everything else ( packaging ). If tests don't catch it - we have a problem...
Contributor
Author
|
@costinm very surprised, how it could break anything if 1. it runs perfectly fine in my test bed, 2. all CI tests are passing?!?! |
Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com>
Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com>
Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com>
Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com>
Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com>
tiswanso
reviewed
Apr 30, 2019
tiswanso
reviewed
Apr 30, 2019
tiswanso
reviewed
Apr 30, 2019
tests/scripts/scripts_test.sh
Outdated
| # list of good ipv6 addresses | ||
| test_good_ipv6=("2001:db8:1::1" "fe80::1" "fe80::5054:ff:fe6c:1c4d" "2001:470:b16e:81::11" "1111:a2a2:b3b3:c4c4:d5d5:e5e5:f6f6:a8a8") | ||
| # list of invalid ipv6 addresses | ||
| test_bad_ipv6=("::1" "1111:b2b21::1" "1111:ab2g::1" "1111:2222:3333:::1" "1111:2222:3333:4444:5555:6666:7777:8888:1") |
Contributor
There was a problem hiding this comment.
Should add a "" string test since in current diffs, the isIPv6() will pass with that and that seems to be in the first array element of OUTBOUND_IP_RANGES_EXCLUDE when a -x is given as arg
tiswanso
reviewed
Apr 30, 2019
Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com>
Contributor
Author
|
@costinm could you please check PR and if you are ok, please remove |
costinm
reviewed
Apr 30, 2019
| echo ' outbound traffic (i.e. "*") is being redirected (default to $ISTIO_SERVICE_EXCLUDE_CIDR).' | ||
| echo ' -k: Comma separated list of virtual interfaces whose inbound traffic (from VM)' | ||
| echo ' will be treated as outbound (optional)' | ||
| echo ' -t: Unit testing, only functions are loaded and no other instructions are executed.' |
Contributor
There was a problem hiding this comment.
Just use an environment variable - it's not something to expose to users.
Contributor
Author
There was a problem hiding this comment.
it is a possibility but that introduces more than 1 control of the script, right now everything runs around while getops and it kind of make sense, adding another control of the script flow will introduce more complexity and what prevent a user to set the variable and get the same result?
Contributor
There was a problem hiding this comment.
Except on mesh expansion - where everything is using environment variables from sidecar.env.
And except that this exposes an internal testing detail to the user.
costinm
reviewed
Apr 30, 2019
| # | ||
| function isIPv4() { | ||
| if [ "$1" != "${1#*[0-9].[0-9].[0-9].[0-9]}" ]; then | ||
| local ipv4regexp="^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$" |
Contributor
There was a problem hiding this comment.
How about removing this ( we didn't have this validation before ), and treat it as !isPIv6 ?
Contributor
Author
There was a problem hiding this comment.
if something makes something more robust at very little cost/impact, why not to add it?
Contributor
There was a problem hiding this comment.
Because it's not making things more robust - and it's not little cost.
The pod and config will still be bad and user will have no idea about it - unlike Galley validation, the config is applied and user doesn't have any signal it happened ( nobody reads the logs from iptables script ). And adding all this complex shell for no benefit - on a critical script that too few
people understand - is IMHO a high cost.
costinm
reviewed
Apr 30, 2019
| number_of_parts=0 | ||
| number_of_skip=0 | ||
| IFS=':' read -r -a addr <<< "$1" | ||
| if [ ${#addr[@]} -eq 0 ]; then |
Contributor
There was a problem hiding this comment.
Can we simplify this ? It's really just v6 or v4 - and the presence of a : is sufficient.
We can assume the OS returns a valid IP - again, we didn't see any problems before having a validation
for v4.
Contributor
Author
There was a problem hiding this comment.
If user specifies incorrect include/exclude ranges then it will cause failure. You did not see problems before because it is not yet widely used and there is probably one or two tests specifying include/exclude ranges in pod's yaml. Having good sanity checks will protect from unexplainable crash loops.
Contributor
There was a problem hiding this comment.
If we decide to officially support this ( right now it's a pretty experimental annotation - not part of the official API ) - Galley can validate as much as it needs. When we switch to CNI as sidecar and possibly reimplement this in go - we can do validation ( assuming this annotation can be supported - since we may switch off iptables and use veth pair ).
If you feel it's important enough - you can probably add a validation on the Pod object in Galley and reject it.
Don't forget that by the time iptable.sh is run - the user has no way to fix the problem even if validation fails. So one way or another - things will not work - either crash loop or ignored config is bad.
Contributor
Author
There was a problem hiding this comment.
If you could point me to galley code, I would be happy to add validation and then retire bash validation.
Also I can refactor istio-iptables.sh in go if you are ok with it.
costinm
reviewed
Apr 30, 2019
| @@ -193,7 +230,7 @@ for range in "${OUTBOUND_IP_RANGES_EXCLUDE[@]}"; do | |||
| if isValidIP "$r"; then | |||
Contributor
There was a problem hiding this comment.
Simplify: if isIPv6... else ...
Contributor
There was a problem hiding this comment.
Or even better: just let user to explicitly configure OUTBOUND_IP6_RANGES_EXCLUDE
Contributor
Author
There was a problem hiding this comment.
and if use by mistake puts garbage or invalid addresses? I have hard time understanding why you are so against of fairly common sanity verification, it is not in datapath so no performance hit, it runs only once on pod start up. what is the reason?
Contributor
There was a problem hiding this comment.
The shell script is getting very complex and hard to maintain.
We can do validation in galley - when the config is validated. Or when we switch iptable initialization to go - we can do more validation.
The script is already extremely complex and risky to change - very few people understand it.
Doing regex matches and validating IPv6 addresses in shell script is neither clean nor
needed.
Contributor
|
fyi, this successful e2e-simple run is on k8s ipv6 (KDC DinD) run with 13563, 13117, and some testcase fix diffs (https://github.com/tiswanso/istio/tree/simpletest_ipv6): |
costinm
reviewed
Apr 30, 2019
Contributor
costinm
left a comment
There was a problem hiding this comment.
/lgtm
/approve
Approving since there is a bug affecting some users - but please address my comments in separate PR.
Collaborator
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: costinm, incfly, linsun, sbezverk The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Contributor
Author
|
/hold cancel |
Contributor
Author
|
/test all |
Contributor
Author
|
/test e2e-simpleTests |
Contributor
Author
|
/retest |
Collaborator
|
@sbezverk: The following test failed, say
DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
geeknoid
pushed a commit
that referenced
this pull request
May 1, 2019
* Add two sample deployments for user guide of Istio Vault integration (#12917) * prevent duplicate inbound listeners (#12937) * respect locality weight set from ServiceEntry (#12714) * respect the lb weight setting from users * add ut * fix golint * add locality lb setting test * fix lint * update test case * update test case * lint * Auto bind to services for Sidecar listeners with specific ports (#12724) * auto bind to TCP services for egress ports in Sidecar Signed-off-by: Shriram Rajagopalan <rshriram@gmail.com> * fix test Signed-off-by: Shriram Rajagopalan <rshriram@gmail.com> * Cleanup gateway vhost config gen (#12847) * check match direction * Cleanup http route generation * undo pickMatching change * golangbot comments * address review comments * fix validation bug * gofmt * check for intersection duplicates * Add wildcard route fallthrough (Fixes ALLOW_ANY, 404s) (#12916) * Add wildcard route fallthrough Currently, ALLOW_ANY doesn't actually allow any external traffic if there is an http service already present on a port. This change adds a wildcard PassthroughCluster as the final route, allowing external traffic even if there is already a service on the port. Additionally, in REGISTRY_ONLY mode, we will return a 404 error if there is already an http service. This is misleading, as it can be conflated with a 404 error returned from the actual service. When in REGISTRY_ONLY mode, we instead return a 502 error to indicate the request is blocked. * add unit tests * Remove node-level flag * Fix tests * Change Ip Address to readable format in accesslog from stdio/stackdriver adapter (#12850) (#12936) * Change Ip Address to readable format in accesslog from stdio adapter * Add a check to validate it's an IP Address before calling ip.string function * Fix formatting error * Fix test * Correct stringify function in instanceUtil.go too for IP address * Fix based on review * Fix based on review * Fix based on review * use only ipv4 for pilot and zipkin (#12997) * do ipv4 lookups for pilot and zipkin Signed-off-by: Shriram Rajagopalan <rshriram@gmail.com> * update goldens Signed-off-by: Shriram Rajagopalan <rshriram@gmail.com> * Cherry pick cert file config from master to release-1.1 (#12707) * Cherry pick from master: Configuration: no longer hardcode mesh certs (#12189) * Configuration: Pilot-Agent: no longer hardcode certs to watch. Pilot-Discovery: no longer hardcode Envoy listener cert paths. * Address demands of golangcibot overlord * Change usages of github.com/stretchr/testify/require to github.com/stretchr/testify/assert * Address code style violation * Revert temporary api changes. Set cert paths in envoy node metadata and use them when setting up listeners * Use envoy node metadata cert paths (if available) when constructing clusters * Rename constants to make golint happy * Fix imports * Ignore ordering in test * Pass around proxy instead of proxy.Metadata (cherry picked from commit 7c34274) * goimports file * Add support for datadog tracing (on release-1.1 branch) (#12687) * Add support for datadog tracing. Signed-off-by: Caleb Gilmour <caleb.gilmour@datadoghq.com> * Use $(HOST_IP) instead of special-casing empty address value Signed-off-by: Caleb Gilmour <caleb.gilmour@datadoghq.com> * fix validation logic so that port.name is no longer a valid PortSelector (#13054) * Add x alias to experimental istioctl command (#11801) * Add x alias to experimental istioctl command I'm super lazy and experimental is far too much effort to type Signed-off-by: Liam White <liam@tetrate.io> * Add exp as an additional alias Signed-off-by: Liam White <liam@tetrate.io> * Update tracing_datadog_golden.json (#13082) * Add jitter in CSR request (#12805) * Add jitter in CSR request * Add log * Fix comments * Fix test * Fix test * Fix comment * 'istioctl proxy-config clusters' cluster type column rendering (#12458) (#12730) * update sds secret mount. (#12733) * Copy data from right place (#12762) * Fix updateClusterInc for overlapping ports (#12766) * Fix updateClusterInc for overlapping ports It is possible that a service will have multiple ports, with the same port number. The typical example here is kube-dns, which uses port 53 for UDP and TCP. When we do an incremental push, we would select the first port to match the port number, which would sometimes causes us to ignore the correct port. This fix searches through all matching ports. * Ensure port number matches as well * Add unit tests * remove dead code * Allow overriding of registry locality (#13077) Also fixes bug where non-kube envs could override to something that parsed incorrectly Signed-off-by: Liam White <liam@tetrate.io> * mixer: add support for standard CRDs for compiled-in adapters (#12815) * cherry pick subset of #12689 Signed-off-by: Kuat Yessenov <kuat@google.com> * add support for compiled in adapters Signed-off-by: Kuat Yessenov <kuat@google.com> * patch log line Signed-off-by: Kuat Yessenov <kuat@google.com> * parse cert to get expire time (#13145) * parse cert * cleanup * unit test coverage * missing file * address comments * rebase and address comment * Installing istio for perf testing (#13159) * Perf scripts * gsutil * WD * perf running and geting metrics * Perf * perf * perf * Perf * remove * qq * Appsv1 pilot (#13050) * appsv1 for Pilot * appsv1 for Pilot * appsv1 for Pilot * dep update * fix test * fix test * fix test * fix test * fix test * typo * typo * typo * typo * typo * update go-control-plane (#13154) Signed-off-by: Kuat Yessenov <kuat@google.com> * added sidecar.istio.io/rewriteAppProbers annotation (#13112) * pilot: registered sidecar.istio.io/rewriteAppProbers annotation * pilot: checked from sidecar.istio.io/rewriteAppProbers too * pilot: added webhook inject tests TestWebhookInject_http_probe_rewrite_enabled_via_annotation case is a modification of TestWebhookInject_http_probe_rewrite case. The difference is rewriteAppHTTPProbe is false in template, but set to true in annotation. TestWebhookInject_http_probe_rewrite_disabled_via_annotation case is a modification of TestWebhookInject case. The difference is rewriteAppHTTPProbe is true in template, but set to false in annotation. * fixed linter issue in test * added http probe test for kubeinject case * added tests and fixed login upon checking RewriteAppHTTPProbe setting * Add more tests in app_probe_test.go * renamed RewriteAppProbers to RewriteAppHTTPProbers * fixed test case for webhook injection * add description to rewriteAppHTTPProbers annotation * updated tests in app probe to sync with recent master change * change validateBool to alwaysValidFunc as per review * Export inject.injectionData() (#12426) * Registrator should use master version (#13083) * dependencies: update cel-go and remove protoc-gen-docs (#12711) * experiment with COMPAT Signed-off-by: Kuat Yessenov <kuat@google.com> * get errors Signed-off-by: Kuat Yessenov <kuat@google.com> * get errors Signed-off-by: Kuat Yessenov <kuat@google.com> * stop validation Signed-off-by: Kuat Yessenov <kuat@google.com> * remove hack Signed-off-by: Kuat Yessenov <kuat@google.com> * testing Signed-off-by: Kuat Yessenov <kuat@google.com> * only access log Signed-off-by: Kuat Yessenov <kuat@google.com> * debugging Signed-off-by: Kuat Yessenov <kuat@google.com> * debugging Signed-off-by: Kuat Yessenov <kuat@google.com> * debugging Signed-off-by: Kuat Yessenov <kuat@google.com> * debugging Signed-off-by: Kuat Yessenov <kuat@google.com> * debugging Signed-off-by: Kuat Yessenov <kuat@google.com> * add runtimeconfig Signed-off-by: Kuat Yessenov <kuat@google.com> * add a benchmark Signed-off-by: Kuat Yessenov <kuat@google.com> * cel_perf Signed-off-by: Kuat Yessenov <kuat@google.com> * update cel Signed-off-by: Kuat Yessenov <kuat@google.com> * update examples Signed-off-by: Kuat Yessenov <kuat@google.com> * remove unnecessary dependencies Signed-off-by: Kuat Yessenov <kuat@google.com> * Fixing copy for helm, one more time. (#13186) * Run goimports on generated file (#13195) * Enable disabled mixer tests in New Test Framework (#13151) * Enable disabled mixer tests in NF * Change tests config to new style * Change tests config to new style * Change tests config to new style * Fix config for native policybackend * Fix report test * Reduce Pilot resource requests for demo (#12477) * Reduce Pilot resource requests for demo * Add limits as well * Added data source for Galley dashboard (#13041) Fixes: #13040 * fix values for pod anti-affinity. (#12798) * Add sensible defaults to istio-gateways (#12315) * report succeed after validation (#13165) * report succeed after validation * review comments * Change exposed port of istio-pilot in consul (#13170) `15003` and `15005` are never used in pilot under consul env. It would be confusing to expose the two ports. Instead, ``` --grpcAddr string Discovery service grpc address (default ":15010") --secureGrpcAddr string Discovery service grpc address, with https (default ":15012") ``` we know `15010` and `15012` are still using. * Cherrypick: Add wildcard route fallthrough (Fixes ALLOW_ANY, 404s) (#12916) (#12973) * Add wildcard route fallthrough (Fixes ALLOW_ANY, 404s) (#12916) * Add wildcard route fallthrough Currently, ALLOW_ANY doesn't actually allow any external traffic if there is an http service already present on a port. This change adds a wildcard PassthroughCluster as the final route, allowing external traffic even if there is already a service on the port. Additionally, in REGISTRY_ONLY mode, we will return a 404 error if there is already an http service. This is misleading, as it can be conflated with a 404 error returned from the actual service. When in REGISTRY_ONLY mode, we instead return a 502 error to indicate the request is blocked. * add unit tests * Remove node-level flag * Fix tests * Use new env var framework * Fix long line * Run format and linter * CEL checker mutex (#13192) * checker mutex Signed-off-by: Kuat Yessenov <kuat@google.com> * deadlock Signed-off-by: Kuat Yessenov <kuat@google.com> * Integration testing for Locality Load Balancing (#13084) * Initial testing functionality Signed-off-by: Liam White <liam@tetrate.io> * appease the linting gods Signed-off-by: Liam White <liam@tetrate.io> * Fall back to bootstrap locality as a last resort Signed-off-by: Liam White <liam@tetrate.io> * Move service instance check after we set them... Signed-off-by: Liam White <liam@tetrate.io> * Add EDS test Signed-off-by: Liam White <liam@tetrate.io> * Reorganise tests to run in parallel Signed-off-by: Liam White <liam@tetrate.io> * Move to pilot directory Signed-off-by: Liam White <liam@tetrate.io> * minor Infof fixes Signed-off-by: Liam White <liam@tetrate.io> * fix package name Signed-off-by: Liam White <liam@tetrate.io> * Increase propagation sleep and add warning Signed-off-by: Liam White <liam@tetrate.io> * [test-framework] Support helm values containing spaces (#13127) * Support helm values containing spaces in integration test framework For a helm template command, e.g., "helm template --set key1=value1 --set key2=value2", the existing integration test framework assumes the values do not contain spaces and splits the command argument using the space character before executing the helm command. Thus, the existing implementation does not support helm values (e.g., certificates) containing spaces. This PR adds the support of helm values that contain spaces. * Revised to use array based on review comments * Adding servicegraph testing to postsubmit (#13190) * Adding servicegraph testing to postsubmit * m * perf * change * pod * fix * Adding E2E Test for kiali (#11448) * Add Kiali E2E Test * Minor Fixings on Kiali E2E Test * Remove unused mixer.enabled value (#13214) This is not a functional change; this value is never used so it is misleading/confusing. mixer.policy.enabled and mixer.telemetry.enabled are used. * Adding aliases for OWNERS (#13194) * Fixing copy for helm, one more time. * Adding aliases for test group. Setting up labels and no parent_owners * prow * owners * fix(helm/sidecar-injector-configmap): run as root (#13217) * Destination host cannot be * (#13222) * destination host cannot be * * fix test * Fixing helm order (#13224) * Fixing copy for helm, one more time. * Fix order of the helm command * automating Mixer samples (#13196) * move samples Signed-off-by: Kuat Yessenov <kuat@google.com> * add metric samples Signed-off-by: Kuat Yessenov <kuat@google.com> * add samples Signed-off-by: Kuat Yessenov <kuat@google.com> * add samples Signed-off-by: Kuat Yessenov <kuat@google.com> * typo Signed-off-by: Kuat Yessenov <kuat@google.com> * typo Signed-off-by: Kuat Yessenov <kuat@google.com> * add samples Signed-off-by: Kuat Yessenov <kuat@google.com> * add samples Signed-off-by: Kuat Yessenov <kuat@google.com> * add samples Signed-off-by: Kuat Yessenov <kuat@google.com> * add samples Signed-off-by: Kuat Yessenov <kuat@google.com> * Add upstream idle_timeout to cluster definition (#13146) * fix lint (#12988) * update certificates with expiration time 100 years (#13233) * update certificates with expiration time 100 years * update testdata/local/etc/certs * Cherry pick #13233 to fix expired certificates (#13234) * update certificates with expiration time 100 years * update testdata/local/etc/certs * fix original destination bug (#13011) * fix original destination bug * add ut * fix original destination bug (#13242) * fix original destination bug * add ut * Fix #11818 fix workloadSelector for Sidecars (#12666) * Fix test error in mixer/adapter/bypass * Fixes #11818. Extend ServiceRegistries to return workload labels independent of Services * Added test for getting workload labels from consul registry * Removed expected errors and results for now from MemoryRegistry.GetProxyWorkloadLabels() * Added LDS test for consumer without Service and workload specific Sidecar * Removed unnecessary fake for service_accounts * Fix #12957. Match EnvoyFilter.workloadSelector against Pod labels, instead of Service labels * Don't dump config in EnvoyFilter LDSTest * Added missing test data * Implemented review comments. * Added test for generation of inbound listeners for proxies without services. * Add ingress to Sidecar configuration for consumer-only Sidecar.workloadSelector test * Formatted imports based review comments * Only log at debug level if ServiceRegistries fail at determining workload labels * Right place to copy data from (#13149) * Right place to copy data from * Copy riught place * align init role label. (#13172) * Remove --platform option (#13187) * Fix #10380: Remove hardcoded sidecar template for istioctl kube-inject (#10830) * Remove the hardcoded sidecar template for * Remove deprecated flags in istioctl kube-inject * update testdata after rebase * add rule for kubeinject.go in codecov.threshold * push client the new root cert when it's changed (#13163) * refresh root * refresh root * unit test * add logs * address comment * more comment * address comment * use port 80 for HTTP from details, for TLS origination (#13206) Istio now can rewrite the port to 443 and perform TLS origination no need to use port 443 for HTTP traffic * Implement `role` field in AuthorizationPolicy (#13181) * Add check for role in ServiceRoleBinding * Implement global role * Add integration tests for SDS-Vault mTLS flow and SDS-Citadel mTLS flow (#13199) * Add integration tests for SDS-Vault mTLS flow and SDS-Citadel mTLS flow Add integration tests for SDS-Vault mTLS flow and SDS-Citadel mTLS flow. The mutual TLS connection uses the certificates issued by SDS-Vault CA flow and SDS-Citadel CA flow. * Use the flag EnableCDSPrecomputation() * Address review comments * Ignore missing resources on kubectl delete (#13225) This makes it so tests won't fail on cleanup for resources that are already deleted. * [Testing] Cleanup PortForwarder (#13250) * Add generated LICENSES.txt to gitignore (#13209) * remove myself from owners (#13231) Signed-off-by: Kuat Yessenov <kuat@google.com> * Fix ingress sds memory leak (#13251) * use syncmap to avoid race conditions * Do not let ingress gateway agent send SDS response if secret is not ready. * fix test * add test * add liveness probe for citadel. (#12734) * Make 15020 first port in ingressgateway service (#12668) * Make 15020 first port in ingressgateway service Fixes: #12503 * Updated test utils to use NodePort for port 80 Test utils were dependent on the ordering of the ports to work, updated it so that they use the NodePort for port 80 explicitly. * Fixed lint issues * add upstream_transport_failure_reason to access log (#12434) * add upstream_transport_failure_reason to access log Signed-off-by: Lizan Zhou <lizan@tetrate.io> * update proxy to latest Signed-off-by: Lizan Zhou <lizan@tetrate.io> * fix Signed-off-by: Lizan Zhou <lizan@tetrate.io> * fix format Signed-off-by: Lizan Zhou <lizan@tetrate.io> * Allow Locality Distribute without outlierDetection (#12965) * Enable distribute locality LB without outlier detection Failover needs outlier detection to mark hosts unhealthy and fall down to the next priority, but this is not needed for distribute. * fix testS * Fix integration test errors and refactor security integration tests (#13253) * Fix integration test errors and refactor security integration tests - Fix the failure of integration tests when --istio.test.nocleanup=false, which is the default test setting. The failures of integration tests when --istio.test.nocleanup=false are caused by that the errors during cleaning up tests are treated as test failures while the actual tests have succeeded when --istio.test.nocleanup=true. - Organize security integration tests under testss/integration/security. - Refactor the code to share common utility functions and remove duplicate code. - Misc fixes. * Address review comments * Use a const to represent the test policy directory * Address review comments * Fixes the multicluster e2e test (#13246) The secret was being created after the apps where deployed on the remote. This was causes the test to never think the apps successfully deployed since the envoy sidecar was continually restarting. * pre-check: fix a logic error (#13278) `getNameSpace()` always returns an object, even if namespace does not exist. Checking the error status is safer. * patch deprecated field (#13266) * patch deprecated field Signed-off-by: Kuat Yessenov <kuat@google.com> * ge11 Signed-off-by: Kuat Yessenov <kuat@google.com> * typo Signed-off-by: Kuat Yessenov <kuat@google.com> * Remove kubectl from dockerfile prereqs since it pulls it (#13256) * Fixing EDS unit tests (#12995) The current EDS test is incorrect and passes because the check calls time out rather than sucessfully completing. This PR fixes the problem and add one more test. fixes issue #12994 * Skip validating non ingress gateway secret at secret fetcher. (#13281) * use syncmap to avoid race conditions * Do not let ingress gateway agent send SDS response if secret is not ready. * fix test * add test * Skip validating non ingress gateway secret * Fix labels on manifests (#11788) * add missing labels on mixer resources * update istio chart helper to match other charts * disable a test (#13295) Signed-off-by: Kuat Yessenov <kuat@google.com> * Import istio/proxy for 1.1.3 (#13296) * Update proxy version to 1.1.3 (#13300) * move to newer grafana (#13273) * rbac: fix a data race in listener generation (#13308) * Include js/css files into static folder (#12983) * Include js/css files Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com> * Append version to file Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com> * ignore assets.gen.go in code coverage Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com> * remove assets.gen.go from codecov test Signed-off-by: clyang82 <clyang@cn.ibm.com> * remove skipped test from .cov file Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com> * fix check chell issue Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com> * fix shell check issue Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com> * Fix galley integration test race (#13303) * [Test Framework] Integrate apps with Galley (#13115) The most recent refactoring broke the apps component when Pilot is being used with Galley. The apps register their services with the ServiceManager directly. When Pilot is configured with Galley, however, it doesn't use the ServiceManager, which means that the app services are never properly registered with Pilot. - Changed the Pilot and Apps component to require Galley to be configured, to avoid confusion. - Removed the ServiceManager altogether - Galley is used for service registration. Fixes #13090 * Fix again helm copy, was reverted during merge from release 1.1 (#13337) * Fixing copy for helm, one more time. * Fixing copy again for master * Update OpenShift dependencies; Drop [deprecated] legacy schema (#13160) * Extend istioctl mocking library to allow mocking of authn etc (#13118) * Fixing iptabes ranges (#13291) * Fixing iptabes ranges Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fix shellcheck errors Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fixing ci failures #1 Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fixing ci failures #2 Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fixing ci failures #3 Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * Addressing comments Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * Don't apply locality label unless provided (#13297) * Single Template injection spec fully at runtime (#13147) * Template injection spec fully at runtime This eliminates the need to have two layers of templates, which adds a lot of complexity to the template. * Get tests working and rebase on removal of hardcoded template * Remove unused vars * Fix istioctl tests * Report circleci status to testgrid k8s dump (#13340) The dump script often fails for the same reason the test fails. The dump script should probably be hardened, but in the mean time we can just make sure we report the failure (high priority) before we dump the state. * use syncmap in secretcache (#13333) * Gateway names cannot contain dot (#13351) * Sidecar Capture mode NONE to use Bind (#13202) * Sidecar Capture mode NONE to use Bind Signed-off-by: Shakti shaktiprakash.das@salesforce.com * Added test, updated comments * format file * Add integration tests for RBAC v2 (#13353) * Implement RBAC v2 intergration test * Add Galley to app for security tests * Disable locality LB tests (#13305) * [Galley] Add NotReadyEndpoints to Synthetic ServiceEntry (#13255) * [Galley] Add NotReadyEndpoints to Synthetic ServiceEntry The k8s Endpoints NotReadyAddresses are used by Pilot to create inbound ports. Without these ports, the endpoints will never become "ready". Supports #10589 * addressing comments * remove unneeded ClusterRole and ClusteRroleBinding in gateway (#13292) * Initial RPM packaging (#13088) This adds the make targets `rpm/istio` and `rpm/proxy` for creating rpm's for Istio components. Artifacts will be created in `$ISTIO_OUT/rpm`. It also creates a target `rpm/builder-image`, which creates a docker builder image necessary to build istio and proxy rpm's. All RPM's will have as the version number whatever is defined at `VERSION` variable. So, a typical usage will be `make VERSION=1.1.0 rpm/istio`. * Simplified issue templates. (#13380) * [Testing] Minor improvements to kube utilities (#13377) * spiffe: fix a data race in writing trust domain. (#13343) * min ring size for hash lb policy was getting to zero in default case instead of doc'd 1024 (#13275) * appsv1 mixer (#13164) * Fix security tests (#13368) They try to read testdata/testdata/... instead of testdata/... before this change. * Adding exec permissions to files. (#13401) * Fixing copy for helm, one more time. * Adding permissions * Add locality failover integration testing (#13252) * Add locality failover integration testing Signed-off-by: Liam White <liam@tetrate.io> * Fix up prioritized integration test Signed-off-by: Liam White <liam@tetrate.io> * Fix panic in loadbalancer and more failover tests Signed-off-by: Liam White <liam@tetrate.io> * Add no test check Signed-off-by: Liam White <liam@tetrate.io> * stop doing dumb things with errors Signed-off-by: Liam White <liam@tetrate.io> * Fix description of failover tests Signed-off-by: Liam White <liam@tetrate.io> * fix function signature change Signed-off-by: Liam White <liam@tetrate.io> * Use better practice framework usage Signed-off-by: Liam White <liam@tetrate.io> * turn on locality in makefile Signed-off-by: Liam White <liam@tetrate.io> * Enable more linters and fix warnings/errors. (#13245) * Enable next step for perf testing (#13381) * Fixing copy for helm, one more time. * Next step for perf was added * Fix MCP dial-out mode. (#13399) * Fix MCP dial-out mode. + The MCP dial-out mode sends an initial trigger response to trigger proper server/client communication. This is needed under certain scenarios. The original code expected a NACK response to this using a synchronous wait. However, this caused problems as the NACK can be sent *after* the actual resource requests are enqueued in the gRPC stream. This PR fixes the issue by making the handling of the trigger response in-line, as part of regular stream handling. + Adding a new dial-out integration tests capturing the basic scenario. + Adding a sleep in the Galley integration component, as the component startup is inherently racy. There is a race between setting the os signal event handlers during startup and applicatrion of configuration (and subsequent event trigger). The stop-gap solution is to sleep. The right solution is to go back to the correct ordering model for the startup of Galley. * Add an explicit name to the trigger collection to avoid collisions. * Fix lint issues. * Fix lint issues. * Remove failing test case. * Update code coverage. * Fix bug causing deleted endpoints to not be updated (#13402) (#13403) * Fix cluster name, the value in aggregate map must match the cluster ID. * Address review comments, add few more comments * Broken productpage css and glyphicons fonts (#13314) * productpage css and fonts broken #13244 * remove .DS_Store * Update bookinfo image tags to 1.12.0 * update tests * Fixes panic in pilot agent when provided with custom cert paths. (#13409) * Configure logging level in proxy and control plane (#12639) (#13369) * configure proxy log level via helm values for sidecar and gateways * configure istio control plane log level via helm * min ring size for hash lb policy was getting to zero in default case instead of doc'd 1024 (#13284) * [Testing] Improve logging for echo application (#13376) * [Testing] Improve logging for echo application * switch to use Cobra * Add istioctl completion to the 'istioctl' make target. (#13024) Signed-off-by: Jason Clark <jason.clark.oss@gmail.com> * [Testing] Adding integration test instructions (GKE) (#13404) * [Testing] Adding integration test instructions (GKE) These started as a copy of of the ones under e2e. Removed instructions specific to the old test framework. Also cleaned up other instructions and added a script to simplify creation of a cluster. * Fixing spellcheck errors. * Add integration tests that detect race condition (#13342) * Add integration tests that detect race condition * Address review comments * Remove log level * Change to reuse e2e-suite.sh * Address review comments * Fix a duplicate * Fix envvar linter use. (#13411) - envvar linter now fails with an error code when it finds problems. - Stop running the linter over useless directories - Only try to lint go files - Fix discovered unregistered uses of env vars in the code base. * replace ayj with ozevren as istioctl owner (#13335) * [Testing] Various fixes for structpath (#13375) * Fix a linter warning. (#13426) * Refactor integration tests of Citadel (#13304) * Refactor integration tests of Citadel - Citadel is a security component -> organize Citadel integration tests under the security integration tests folder. - The common utility functions are refactored into the util folder. * Fix lint error * Fix manual injection when webhook disabled (#13434) * Fix manual injection when webhook disabled If webook is disabled, then Helm values for the webhook will not be exposed. This means that in the ConfigMap that stores the values, we will not have the rewriteAppHTTPProbe variable, causing errors. By defaulting this to false, we keep the same behavior but succeed in the case when the config is not present. I also verified this is the only case of this bug, all other variables read in the injection template are from global. * Fix linter and check nil * Add field to explicitly define Istio kind for config data (#13347) * Add field to explicitely define Istio kind for config data * Lint * Add missing space in log statement (#10982) * Add missing space in log statement Previously, the log statement was: "Failed to generate bootstrap configopen /etc..." (note that there's no space in configopen). This commit fixes the statement, so it reads "Failed to generate bootstrap config: open /etc/..." * Add missing spaces to all Debuga,Infoa,Warna,Errora,Fatala statements * add CRD sample for rate limiting task (#13370) Signed-off-by: Kuat Yessenov <kuat@google.com> * Fix make test-bins (#13124) Prior to this PR, make test-bins produces no action. * Scrape internal Grafana metrics. (#12509) * [Test Framework] Fix forward echo timeout (#13459) This was using picoseconds instead of microseconds * Fix test flakes in pkg/cache. (#13454) - Run the expirstion tests with no background evicter goroutine, which eliminates the non-determinism. - Stop using time.UTC(), turns out its unnecessary when using time.UnixNanos - Correctly initialize the base nanosecond value when using the caches with no evicter goroutine. - Add a missing delay in the test for EvictExpired uncovered by setting the base nanosecond value above. * Add integration test for outboundTrafficPolicy (#13099) * Add outboundTrafficPolicy integration test * Run format * Fix lint errors * Fix call validation * Fix native and comment why we can't use native * Remove all checks except count * Remove Servicegraph, and therefore addons. (#12470) Servicegraph was deprecated but available in 1.1, with a plan to remove in subsequent releases. * [Test Framework] Support Pilot mesh config (#13460) * Refactor authentication plugin code to support future policy versions (#13441) * Refactor authentication plugin code to support future policy versions * Consolidate support functions * Lint * Fix import * Rename Applier to PolicyApplier * Fix EnableFallthroughRoute for HTTPS traffic (#13440) HTTPS traffic does not go through the route config like http, so the fix to allow outbound traffic properly is not applied. Instead, we can do the same thing at the listener level. Because we cannot do a direct response here, we can't return a 502 in the case of REGISTRY_ONLY, but we can still block the traffic (same behavior as if we had no listener on the port). * New prow e2e Multi-cluster test for Split Horizon EDS (#12709) * Add an e2e testing environment and tests for split horizon multicluster * Temporarily run the new mc test instead of old one * Revert "Temporarily run the new mc test instead of old one" This reverts commit 8634ae1. * Revert "Revert "Temporarily run the new mc test instead of old one"" This reverts commit 39e007c. * return errors if the split horizon test runs without auth and automatic sidecar injection * remove the separate prow test for split horizon, add it to the multicluster test * move the auth-enable flag from the prow script into tests/istio.mk * remove the flat network multicluster test until it will be fixed * fix the comment of KubeCommand * TestRemoteInstanceAcessible -> TestRemoteInstanceAccessible * add use-automatic-injection flag to the split horizon test * Revert "add use-automatic-injection flag to the split horizon test" This reverts commit c488cd8. * for split horizon check that the framework's automatic injection is not set * add the split horizon flat to e2e README * use strings.Contains instead of strings.IndexOf >= 0 * do not redefine err * use "naked" return consistently * return error if some pods are not running * Revert "remove the separate prow test for split horizon, add it to the multicluster test" This reverts commit c2d0ece. * istio-pilot-e2e-split-horizon-eds.sh -> e2e-split-horizon-eds.sh * Revert "Revert "Revert "Temporarily run the new mc test instead of old one""" This reverts commit 46c4a98. * reduce timeout from 50 to 15 * [Code Mauve]: Get TcpMetrics test working again in new test framework (#13247) * Code Mauve: Get Tcp test working again in new test framework Code Mauve: Get Tcp test working again in new test framework Code Mauve: Get Tcp test working again in new test framework Fix based on reviews Fix based on reviews Fix based on reviews Fix based on reviews Fix formatting error Fix failing codecoverage and unit test on circle as they are getting killed because of short timeout Trying to fix circleci tests Trying to apply gateway file in bookinfo namespace only * Fix linting error * deploy bookinfo in its own namespace for all mixer tests * deploy bookinfo in its own namespace for loadbalancing test too * Fix perfcheck script (#13461) * Make sure all flags get pulled during init. (#13513) * Make sure all flags get pulled during init. * Fix lint errors. * Fix example_test. * Sleep to prevent test flakes in outbound traffic (#13514) * Fix configz test failures (#13478) * Fix configz test failures * Dynamically assign port * [Test Framework] Expand capability of Echo component (#13175) * [Test Framework] Expand capability of Echo component The Echo component API was essentially a rewrite of the Apps component, but allows the test author more flexibility in the behavior of the application instances. This PR merges the functionality of the Apps component (including running on Kubernetes as well as running natively with a sidecar) into the Echo component. Once this lands, we can remove the Apps component entirely. * addressing comments * various fixes * attempt to update golangcilint (#13525) Signed-off-by: Kuat Yessenov <kuat@google.com> * Adding unit tests for gateway (#12792) * Adding unit tests for gateway * Fixing the lint issue * Fixing the copyright year * Making changes suggested in the reviews. Changes the name of a function and location of another. * Support using the kiali-viewer role directly from Helm chart configuration (#13528) * Support using the kiali-viewer role * Switch to viewOnlyMode name to be consistent with Kiali operator name * multicluster: fix panic caused by invalid kubeconfig (#13451) * multicluster: fix panic caused by invalid kubeconfig * fix comment * [WIP] Preventing duplicate route entries (#13431) Addresses issue #13267 Adds unit tests * Fix bug causing deleted endpoints to not be updated (#13402) (#13403) (#13470) * Fix cluster name, the value in aggregate map must match the cluster ID. * Address review comments, add few more comments * Fix SE with non FQDN hosts (#13447) * Adding the missing validation pieces for CORS (#12840) * Adding the missing validation pieces for CORS Includes new unit test case * Allow for http/https schemes specified * Making "*" the only host with wildcard allowed for allow-origin * Allow port number in CORS Allow Origin Having a port number in "Allow Origin" is accepted according to the spec. * Using strings.TrimPrefix as suggested by lint * fix panic (#13548) * Fix RBAC integration tests (#13384) * update to go1.12 (#13531) * update CI image to 1.12 Signed-off-by: Kuat Yessenov <kuat@google.com> * fix coverage test Signed-off-by: Kuat Yessenov <kuat@google.com> * fix coverage once again Signed-off-by: Kuat Yessenov <kuat@google.com> * rbac: refactor filter generation and split filter logic (#13488) * move istioctl completion generator to its own target (#13567) * Fix potential fd leak (#13310) * update jinja and urllib3 (#13585) * set GOGC (#13587) Signed-off-by: Kuat Yessenov <kuat@google.com> * make GC more aggressive (#13596) Signed-off-by: Kuat Yessenov <kuat@google.com> * Fix bug: when pod occur later than sidecar connection, the sidecar in… (#13229) * Fix bug: when pod occur later than sidecar connection, the sidecar inbound listener will not be pushed * fix comments: only do a full push to the added sidecar * optimize: do not record workloads that have no sidecars or have not connected with pilot * add istioctl experimental auth for checking TLS/JWT/RBAC setting on a pod. (#12774) * add istioctl auth command for checking TLS/JWT/RBAC setting on a pod. This is still experimental and under active undevelopment, not ready for production use. * fix lint. * Support reading from file, refine the help message. * update cluster to show 'none' for certs. * update google.golang.org/grpc * add whitespace in column * add unit test * fix lint * fix lint * make --remote and --s as default for istioctl version command (#13389) * make remote and short as default * fix lint * remove unused permission in istio_init. (#12978) * Update UsingGKE.md (#13560) To avoid confusion, per the gcloud SDK documentation: https://cloud.google.com/sdk/gcloud/reference/#--project, project ID instead of project name should be used for the project flag. * Fix several lint issues on Citadel Agent. (#13558) * Fix several error handling and lint. * Small fix. * Small fix * fix broken links in readme. (#13610) * Cache values config in sidecar injector (#13480) Values were read each time during injection, rather than cached like mesh config and the injection template. * Add integration tests for Istio authorization for groups and list claims (#13557) * Add integration tests for Istio authorization for groups and list claims * Separate RBAC tests to avoid interference from each other * Add headers from the test options * Fix lint errors * Add headers in the native environment * Add headers in echo component * Refactor the test structure * add MacOS support KinD (#13583) * Do not use sh in istioctl. (#13395) * Do not use sh in istioctl. Co-authored-by: Jakob Schmid <jakob.schmid@sap.com> * Fix lint errors. Co-authored-by: Jakob Schmid <jakob.schmid@sap.com> * For RBAC v2, add integration tests for authorization of groups and list claims (#13628) * For RBAC v2, add integration tests for authorization of groups and list claims * Add to-do * cleans up unnecessary left over comment (#13137) * Adding a unit test case Adds a unit test case aand cleans up unnecessary left over comment * Removing the extraneous comment * Remove trailing tab chars from each line ending. (#13570) Trailing tabs were left in the rendered template, having the yaml linter throw warnings. * show detailed mcp resource information in ctrlz page (#12999) Signed-off-by: clyang82 <clyang@cn.ibm.com> * Re-enable Mixer validation (#13379) * cleaning up mixer validation Signed-off-by: Kuat Yessenov <kuat@google.com> * fixes Signed-off-by: Kuat Yessenov <kuat@google.com> * fix mixer tests Signed-off-by: Kuat Yessenov <kuat@google.com> * fix galley test Signed-off-by: Kuat Yessenov <kuat@google.com> * less diff Signed-off-by: Kuat Yessenov <kuat@google.com> * no edge case possible Signed-off-by: Kuat Yessenov <kuat@google.com> * fixing the adapter dependencies Signed-off-by: Kuat Yessenov <kuat@google.com> * enable validation Signed-off-by: Kuat Yessenov <kuat@google.com> * goimports Signed-off-by: Kuat Yessenov <kuat@google.com> * missed an adapter Signed-off-by: Kuat Yessenov <kuat@google.com> * edge case Signed-off-by: Kuat Yessenov <kuat@google.com> * coverage Signed-off-by: Kuat Yessenov <kuat@google.com> * Pass componentLogLevel to Envoy to disable deprecation warnings (#13182) Istio users do not care about Envoy features we choose to use the are deprecated, but we spam their logs with thousands of warnings about deprecations. This turns off these messages, and allows proxy log level to be tuned by operators to their preferences (including re-enabling deprecation warnings if they wish). * Add Redis Ratelimiting tests in new test framework (#11209) * Add Redis Ratelimiting tests in new test framework Fix based on reviews Fix based on reviews Fix based on reviews Also, deploy tiller before deploying redis Fix based on reviews Increase timeout value for fetching values from prometheus to make it more reliable Wait for Tiller to start before using it Increase test timeout as Redis tests have increased the overall runtime of the tests Increase reliability of rate limiting tests fix failing test fix failing test fix failing test Try to decrease runtime of the test Fix descriptors.go after rebase Fix lint error Add debugging steps in original redisquota tests Debugging failure in ratelimit test when running in prow Debug test in prow Fix conflict error Fix errors Fix config for redis using new style crds Refactored to reduce setup time nit fix Fix golang errors Fix golang errors Fix errors in config Fix errors in config and golang errors Fix errors in config Fix errors in config Fix golang errors Fix TestRateLimiting_DefaultLessThanOverride test Formatting file Refactor common code Fix golang errors Fix golang errors Fix tests Reduce timeouts * Fix golang error * fix typo in pilot/cmd/pilot-agent/status/ready/probe.go (#12321) * Try out a template experiment. * Another template update. * Template tweakathon. * Skip failing test case (#13669) This test breaks all commits, likely cause by TLS certs expired. This means all past commits will no longer pass tests, and all new commits will be blocked. We should disable this test for now until it can be properly fixed. * Stop using task lists since they cause GitHub to mark issues as 0/7 completed... * correct example text for istioctl authn tls-check command (#13561) * Fix integration tests and user guide of SDS Vault CA flow (#13685) * Fix integration tests and user guide of SDS Vault CA flow Tests under tests/integration/security/sds_vault_flow/ fail because the cluster hosting the test Vault server was deleted. This PR: - Fixes the failed integration tests to use a new Vault server. - Fixes an example Vault CA server config used in user guide. * Address review comments * fix namespace parsing in istioctl validate (#13624) * fix namespace parsing in istioctl validate Signed-off-by: Kuat Yessenov <kuat@google.com> * merge fix Signed-off-by: Kuat Yessenov <kuat@google.com> * revert yaml v3 change Signed-off-by: Kuat Yessenov <kuat@google.com> * manually transform map interface Signed-off-by: Kuat Yessenov <kuat@google.com> * restore extra field test Signed-off-by: Kuat Yessenov <kuat@google.com> * bootstrap: add test to confirm ISTIO_META_ envvar (#13645) ISTIO_META_key=val env variable can be encoded into node metadata as "key" to "val" Signed-off-by: Yuchen Dai <silentdai@gmail.com> * release: Update latest stable Istio CNI SHA (#13556) * Lock down development of installer (#13350) * Lock down development of installer All development should be done on the istio-installer repo. * Top level owners * Update url * Fix isValidIP in iptable-start.sh and add unit test for it. (#13563) * refactoring validations Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * addressing comments Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * Addressing more comments Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fix typo Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * shellcheck error Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fixing unbound variable error Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fixing CI failure Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * shell check Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * dealing with ipv6 special case Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fix broken links. (#13741) * add samples validation linter (#13736) * add samples validation linter Signed-off-by: Kuat Yessenov <kuat@google.com> * typo Signed-off-by: Kuat Yessenov <kuat@google.com> * print deprecation warnings Signed-off-by: Kuat Yessenov <kuat@google.com> * english Signed-off-by: Kuat Yessenov <kuat@google.com> * [Testing] Refactoring Echo application (#13586) The echo application has gotten hard to follow. - Restructuring code into more sensible packages. - Adding readiness. The echo app will now return 503s until all ports are up. - Propagating timeouts throughout the call chain. - Use Cobra in client main. This also required that all uses of the client switch to using double dashes for flags (they were previously using single dashes). Fixes #13553 * Fix unit tests of Vault CA integration (#13683) * Fix unit tests of Vault CA integration Tests under security/pkg/nodeagent/caclient/providers/vault/ fail because the cluster hosting the test Vault server was deleted. This PR: - Fixes the failed tests to use a new Vault server. - Moves the tests using real Vault server to integration tests. * Add a documentation * Opt in the test framework and label the test as post-submit * Fix istioctl test (#13750) * Fixing copy for helm, one more time. * Fixing test * Refactor Test Framework API Surface, and add complete Galley component methods. (#13738) * Implement Missing Galley functionality and more framework tests. + Adding missing methods for Galley Kubernetes component. + Tests for creating/deleting namespaces. + Tests for Galley snapshot reading. * Remove accidental edit. * Remove unused field. * Add a new Yaml resource tracker utility to yml package. * Fixup tests. * fix lint errors. * more lint fixes. * Remove offending test. * Update Galley code to use tracker Re-disable conversion test for Kubernetes environment. * Refactor API surface and a test for framework.Suite * Update Readme file. * code review feedback. * Fixup new tests. * Add straggler test. * Extend fake policy backend for OOP adapter integration test (#13729) * extend fake policy backend for out of process adapter integration test * Make valid duration and valid count configurable
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Serguei Bezverkhi sbezverk@cisco.com
Closes: #13562