* Clean up legacy Discovery v1 server

Right now we have two discovery servers - v1 and v2. The v1 server
basically does nothing, except expose a single unused debugging
endpoint.

This PR kills the v1 discovery server, and moves the only part that is
used (setting up the profile endpoints) to its appropriate location in
InitDebug.

* Fix integ tests

Includes a new image for prow based on ubuntu bionic.

Needed by #18577

* QuotaSpec and QuotaSpecBinding only triggers push for sidecar

* Dertermine push types according to config types updated and proxy type

* Move filter into a separate file and add test

* fix ut

* protect by PILOT_SCOPE_PUSHES

* push to proxy if its service updated

* EDS must be paired with CDS

* Add comment

* fix ut

* go format

#18574)

* wip

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* add test cases

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* revert controller change

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* add debug error

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* try deleting completely

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* trigger full push for ss3 config update

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* change sse to trigger full push

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* lint

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* review comments

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* split delete in to two functions

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* address comments

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* clean up ports

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* add collector endpoint version for zipkin

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* change api in tests

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* change back api in tests

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* move to v2 spans

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* fix readiness probe regression

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* fix unit test

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* envoy api update

* run go mod tidy

* Initial super-rough ugly iteration

* refactor a bit so we're at least not broken

* Cleanup a bit

* add new message, cleanup

* add test cases

* Message fixes

* Simplify handling of regular vs synthetic service entries

* Reorder args from PR feedback

* fix node agent crash

* revert change to mixer_test.go

* Fix version

* Update istio api version to the latest.

* Update test data to test the feature

* Fix indentation and config.

* Fix indentation round 2

* Indentation fix round 3.

* Update crds for tests.

* remove yaml separator

* Add loadbalancer mandatory attribute for test data.

* Incorporate code review comments.

* Updated to reflect existing behavior

* Update import

* Restore to original order.

* Fix unit tests.

* Fix lint errors.

* Check in make gen output

* Update istio api and redo make gen

* Testing with rolled back api version

* Keeping the old crd gen file.

* fix json field names in tests and update to latest api

* fix go sum

* fix json naming in conformance tests

* use 1.4 so it can pick up patches

* move grafana to latest

…eport disabled (#18717)

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* skip syncing root cert if not in self signed CA mode

* lint

* lint

* revise

* update test

gencheck is optional so even with recent changes this was not caught
from #18713

* Add new operator annotations to the analyze whitelist

* Handle 'Any' resource type

* rename istioctl auth to authz to focus on authorization

* fix conflict

* address comments

* update to use authZ

Fixes #18699. Also makes the printed message a bit clearer.

* fix(stackdriver): pull mesh uid from config

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* rerun make gen

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* Incremental changes:

- remove the SDS startup in pilot - the feedback is to start with SDS in
agent, pilot will only sign certs.
- cleanup a bit the startup
- use port 8080 for http - istioctl expects it.
- add the original Dockerfile - while it is getting integrated into the
istio build system ( can be useful after as well, it's very easy)

* Add a cloudbuilder for istiod, add missing files

* More cleanup, remove controversial dockerfile

* Use upstream galley server, reuse kube

* Make format

* Fix lint

* Add missing trust domain

* Post merge fixes

* Lint

* Fix nil check for interface

* Lint

* Initial SDS agent and support

* Improved startup config, use injection

* Tested SDS with Istiod and google plugin

* Format and lint

* More lint fighting, cleanup

* Change the template to use K8S-signed cert with local SDS

* More fixes, better debug

* Lint and fixes

* More lint

* Simpler/safer code. Istiod will not support citadel agent

* Restore previous code to disable sds

* Another attempt to format code and fight linter

* Indent and fix sds default

* Fix another crazy startup issue.

The watcher is not only watching - but also kicks the first start.
This code needs to be mostly replaced once SDS and istiod are enabled.

Also add a simple way to patch galley options for fine tunning. Like
values.yaml subset needed by injection, it's part of the transition to
mesh.yaml

* Add explicit option to use local JWT

* format

* Review feedback, renames and more comments

* Test failure

* Signature change in security

* Fixes for crash and canary mode

* Remove dep on bootstrap

* update sha for automtls

* run make gen

* Add flag to control output format for analyze

See #18306. On its own this doesn't do anything, just adds a flag and
sets a default output format to the current format.

* Add MarshalJSON function for Message struct

* Add Json as an output format option to analyze

* Add YAML as an output format to analyze

* Update style re: PR feedback

* Use Unstructured for json marshaling messages

* Indent JSON output by default in analyze

Signed-off-by: Kuat Yessenov <kuat@google.com>

* add support for http_proxy protocol

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* add http_proxy to protocol enum

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* remove unnecessary comment

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* update ishttp function

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* skip lint for naming

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

Currently the mTLS analyzer uses the same message for all conflicts. While this is technically correct, it's not easy to determine why there's a policy conflict, especially in the case of a missing sidecar. This commit introduces a new message specifically for the situation where the host does not have a sidecar and a destination rule enforces mTLS.

Showing the namespace is redundant, as the relevant namespace will either be mentioned in the name of the policy object or the destination rule.

Note that this will cause duplicate messages to appear since we still iterate over all namespaces (which is needed to find all misconfigurations). This isn't a problem now that we dedupe messages (#19419).

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

GH Issue: #19380

* Ingress SDS

* Cleanup startup of istiod

* Move istiod to pilot/bootstrap

* More fixes, dup removal

* More dups, revert some of the renames since all are in same package now

* More cleanup for istiod

* Adjustments for SDS

* More dup removal and comments

* typo

* Another round of cleanups

* Change the order back

* Reuse the HTTPS and GRPC servers for galley

* Tests - if k8s missing, don't start galley

* revert again the ingress sds

* Address comments on galley startup, don't start CA unless a token is mounted

* format

* Need to start it

* Reorder the startup, so wait happens before listening and galley can
sync with localhost

* Fix annoying log loop message.

Galley is slower to startup, pilot keeps logging.

* When starting in-process galley we seem to have dashboard test problems.

Add an env to enable/disable the galley - as well as using the local
galley if enabled.

* Format

* Format

* Revert fancy spam reduction, race conditions

* Ordering - sidecar is using the dns certs.

Eventually it'll use the same port/server.

* More details in logs

* Review comments

* Lint, default AUDIENCE is empty

* Fix callout for galley

* Missed fix

* Remove duplicated start

* Lint

* Remove the standalone istiod.

Code is moved to costinm/istiod package - will be developed separatedly,
and added back after the other components are removed.

Istiod is supposed to be a super-clean version, with only minimal
mesh.yaml config.

* Add a small log, to know this is started (and only once).

It was too silent.

* Address comments

* Make it off by default

* Remove POD_NAMESPACE as well

* Removed standalone istiod, based on feedback.

Will still be built in a private repo, but now the focus is on the
pilot and minimal profile.

* Revert future work

* Lint

* Fix format

* More lint

* Lint

* format

* Fix default name.

We are now using .svc, for consistency with K8S which requires this
pattern for webhooks.

* Another small fix for ingress controller and istiod.

If only istiod is installed the logs will be spammed with status update
messages - this change delays taking over the 'update ingress status'
until the address is available.

* format

* Revert "format"

This reverts commit 5fc67f5.

* format again

* Lint

* format again

* Format after conflicts

* Revert processing2 changes since galley is no longer loaded.

Webhook may still be loaded.

* Multiple round of conflict resolution lost 3 lines..

* Fix `make docker` on a Mac to build linux binaries

* Revert back to single target and fix macro

…19303)

* fix galley validatingwebhookconfiguration deletion issue.

* fix ut.

* feat(bootstrap): add support for AWS platform to bootstrap

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* fix typo

* Allow analyzing content passed via stdin

* Set finalizer for open files in analyze

* Allow interactive input via stdin with analyze

* Add util funcs for mesh cfg and istio-system ns

* Add IsSystemNamespace

* Add unit test

* Move context test fixture to avoid import cycle

* Don't use istio-system at all

* Lint fixes

* Fix the fix

* add copyright header

* mtls ignores system namespaces, fix a test case

* Fix syntax in test yaml

* Add test case

* wip first ver applier.

* add test case.

* format and make gen

* alawys add peers mtls

* remove jwt location

* comment

* sorting in the contructor.

* gofmt

* unit test

* remove the comments

* fmt

* Adding ServiceAssociation Analyzer

* Approaching the problem of reliying on containerPort

* Only analyze deployments that are in the mesh

Right now running unit tests causes a change to the repo, it seems like
the golden file was updated in 997c137
but nothing else was. It doesn't look like we need the files at all so
let's get rid of them.

…19373)

* Add fallback to alpha JWT policy if RequestAuthentication is not found

* Add test when alpha policy doesn't have JWT

* Add more TODO comment for AuthNFilter

* Change log info to debug:

* Fix rebase

* Lint

* Small workaround to allow istiod to not use values

* Add more info for SDS

* Replaced with 'env'

* Use the default

* Format

* revert, separate pr

"Enmesh" is not user-friendly (or at least, not commonly-understood).

* Add more validation to RequestAuthentication resource

* Fix refactor

* Add validation for default policy

* Address comments

* Soft graduate istioctl analyze

* Graduate analyze out of experimental

* Fix istioctl root getDefaultNamespace to respect --context

* Fix dashboard test for slightly tweaked message

* tweak working of validate deprecation notice

Missing the 's' in 'apis'. This update matches the changes to the injection-template.yaml in #18485.