WIP: combine galley start mode and add liveness/readiness probe#3
WIP: combine galley start mode and add liveness/readiness probe#3ayj wants to merge 1 commit intoayj:masterfrom
Conversation
| versionPath = "/version" | ||
| ) | ||
|
|
||
| func startSelfMonitoring(stop <-chan struct{}, port uint) { |
There was a problem hiding this comment.
We should be able to share the promhttp handler between the validation and server paths. Consider moving startSelfMonitoring to galley/pkg/server/monitoring.go?
There was a problem hiding this comment.
Now there is no metrics available in server, so keep the logics for validation only move it once server have added metrics part?
There was a problem hiding this comment.
We'll be adding metrics shortly. Since you're already moving this code around we might as well put it the common place now :)
There was a problem hiding this comment.
Done~ Move to galley/pkg/server/galleymonitor.go
|
|
||
| var sa = server.DefaultArgs() | ||
| var vc = validation.DefaultWebhooktParams() | ||
|
|
There was a problem hiding this comment.
We could simplify this a bit by hooking PersistantFlags.StringVar() directly to the server and validation args. e.g.
func GetRootCmd(...) {
var (
serverArgs = server.DefaultArgs()
validationARgs = validadtion.DefaultArgs()
)
rootCmd := &cobra.Command{ ... }
rootCmd.PersistentFlags().BoolVar(&serverArgs.server, "server", "Run galley server mode", server.Args.server)
// etc
}This would also keep the server.DefaultArgs() and default flag values in sync.
| rootCmd.PersistentFlags().BoolVar(&flags.noServer, "no-server", false, | ||
| "Run galley server mode") | ||
| rootCmd.PersistentFlags().BoolVar(&flags.noValidation, "no-validation", false, | ||
| "Run galley validation mode") |
There was a problem hiding this comment.
Can we reverse the polarity of no-server and no-validation` to avoid double negatives? I think the command line usage would read a bit better, e.g.
galley --no-server=true --no-validation=false
// vs.
galley --enable-server=false --enable-validation| //Run liveness and readiness check | ||
| probeCheckController := galleyProbe.NewProbeCheckController(flags.livenessProbeOptions,flags.readinessProbeOptions) | ||
| probeCheckController.Run() | ||
| galleyStop := make(chan struct{}) |
There was a problem hiding this comment.
Should we propagate the galleyStop channel to server.RunServer and validation.RunValidation?
There was a problem hiding this comment.
in server.RunServer and validation.RunValidation they will call os.Exit(-1) when they encounter errors. So I think we can use this channel to exit if none of them has errors.
| noValidation bool | ||
| noServer bool | ||
| livenessProbeOptions *probe.Options | ||
| readinessProbeOptions *probe.Options |
There was a problem hiding this comment.
I would declare livenessProbeOptions and readinessProbeOptions as value types instead of pointers and move it to within the scope of GetRootCmd (line ~56). This would eliminate lines 104-105 and the use of global variables.
| "Maximum size of individual gRPC messages") | ||
| rootCmd.PersistentFlags().UintVarP(&sa.MaxConcurrentStreams, "server.maxConcurrentStreams", "", sa.MaxConcurrentStreams, | ||
| "Maximum number of outstanding RPCs per connection") | ||
| rootCmd.PersistentFlags().BoolVarP(&sa.Insecure, "insecure", "", sa.Insecure, |
There was a problem hiding this comment.
Missing description for insecure flag
| readinessProbeController.Start() | ||
| readinessProbe.SetAvailable(nil) | ||
| } | ||
|
|
There was a problem hiding this comment.
Can we align this with how mixer uses the readiness probe package? Specifically, the probe package itself seems to already handle updating the health file. We only need to invoke SetAvailable(<status>) based on server and validation readiness.
There was a problem hiding this comment.
Mixer have not used this part actually because they have not defined liveness and readiness probe options in their helm/docker file. the security part is relying on this mechanism now: https://github.com/istio/istio/blob/master/security/pkg/probe/controller.go , so we are now under the same way with it.
There was a problem hiding this comment.
currently I have used the same logic to detect the liveness and readiness for galley:
func (c *ProbeCheckController) checkLiveness() error {
content := []byte(`ok`)
if err := ioutil.WriteFile(c.livenessProbeOptions.Path, content, 0644); err != nil {
log.Errorf("Health check update of %q failed: %v", c.livenessProbeOptions.Path, err)
return err
}
return nil
}
func (c *ProbeCheckController) checkReadiness() error {
content := []byte(`ok`)
if err := ioutil.WriteFile(c.readinessProbeOptions.Path, content, 0644); err != nil {
log.Errorf("Health check update of %q failed: %v", c.readinessProbeOptions.Path, err)
return err
}
return nil
}So in case there is error, the probe package itself will get the error and remove the heath file.
So is above logic valid to check the galley liveness and readiness? (Just copy the logic from validation code)?
There was a problem hiding this comment.
See istio#7986 (release-1.0 branch). This change should also be merged into master. This decouples the config reconciliation loop from the health check updates. To do this, I used the istio.io/pkg/probe package instead of writing the file directly. We should use the same mechanism for the galley service health checks.
| // limitations under the License. | ||
| package validation | ||
|
|
||
| import ( |
There was a problem hiding this comment.
FYI, you'll need to run make format and make lint to help fix the formatting and linter issues before the PR can be merged.
|
fyi @irisdingbj |
* Fixing iptabes ranges Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fix shellcheck errors Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fixing ci failures #1 Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fixing ci failures #2 Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * fixing ci failures #3 Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com> * Addressing comments Signed-off-by: Serguei Bezverkhi <sbezverk@cisco.com>
Fix domainname of galley in envoy configmap istio-control/istio-discovery
* Installer CRD proto definition * Review comments * Review comments * Add docs and examples * Review comments * Review comments * Review comments * License text * Change IstioInstall to IstioControlPlane * Add components struct layer
No description provided.