This repository was archived by the owner on Sep 30, 2024. It is now read-only.
batches: Remote Mount #7 - Executor#40931
Merged
Merged
Conversation
Contributor
Author
|
Note to self so I do not have to dig thru comments/commits.
|
Contributor
Author
|
@eseliger I believe I added/updated the monitoring and pushed it into this PR. Is this correct way to do this? Do I need to do anything with Grafana itself or is this sufficient? |
eseliger
approved these changes
Oct 4, 2022
eseliger
left a comment
Member
There was a problem hiding this comment.
last set of (very minor) comments, let's go 
Comment on lines
+452
to
+455
| // src_apiworker_apiclient_queue_total | ||
| // src_apiworker_apiclient_queue_duration_seconds_bucket | ||
| // src_apiworker_apiclient_queue_errors_total | ||
| func (codeIntelligence) NewExecutorAPIQueueClientGroup(containerName string) monitoring.Group { |
Member
There was a problem hiding this comment.
This will "wipe" the history of these graphs, as the new name of the metrics will only be available from now on, hiding all previously collected data. I don't think this is a terribly big problem for this particular graph, but wanted to call it out, for future monitoring work.
Contributor
Author
There was a problem hiding this comment.
Is there anyway to preserve the data when migrating metrics?
# Conflicts: # enterprise/cmd/executor/internal/apiclient/baseclient.go # enterprise/cmd/executor/internal/apiclient/queue/client.go # enterprise/cmd/executor/internal/config/config.go # enterprise/cmd/executor/main.go # enterprise/cmd/frontend/internal/executorqueue/queues/batches/transform.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #31792.
Previous part: #40924
The following changes have been made to the overall logic flow.
*btypes.BatchSpecWorkspaceExecutionJobto aapiclient.Job, the batch spec's workspace files metadata is stored in the job'sVirtualMachineFilesfield (previouslyVirtualMachineFileswas a map of strings, but has been updated to a struct to accommodate retrieving files from the files API)apiclient.Jobto specify where the workspace files will be written to (WorkspaceFilesDirectory). This directory is also being provided as a command/flag tobatch exec(a separate PR will come for thesrc-cliside of things).apiclient.Job, theVirtualMachineFilesthat do not contain the embedded value of the file (instead have aBucketandKey) are retrieved from the files API (.executors/files/batches/{spec}/{file}). These files are written to to the location specified inWorkspaceFilesDirectory.apiclientpackage. The queueClienthas been moved into its own package a new package (files) has been added to handle the files API.BaseClienthas been refactor to be more generic for both queue and files clientssrc batch execis called with the VM full prepared.Test plan
Added Go Unit Tests