This repository was archived by the owner on Sep 30, 2024. It is now read-only.
feat(sg): sqlite-backed local store for sg analytics#63578
Merged
Conversation
dcff2bd to
f5bda09
Compare
bfbb9f9 to
94bf52b
Compare
jhchabran
approved these changes
Jul 4, 2024
|
|
||
| func AddMeta(ctx context.Context, meta map[string]any) { | ||
| invc, ok := ctx.Value(invocationKey).(invocation) | ||
| if !ok { |
Contributor
There was a problem hiding this comment.
Note for later: I totally agree that we don't want to bork the behaviour if somehow the context is missing that. But on the other hand, this shouldn't happen. In an ideal world, if we had tests for each individual command and pretend to run them, it would fail in tests, but not when running in production mode.
Or it could fail in dev mode only for example.
Let's not derail from the original intent though, just thinking out loud here 🙏
burmudar
reviewed
Jul 4, 2024
Contributor
Author
added a 1-retry mechanism to all queries, this should be exceedingly rare though, so once retry should cover it |
ba5cfe4 to
ef35f8d
Compare
Reads events from the local analytics db and writes them to events table
in the bigquery `sg_analytics` dataset
```
{
"uuid": "01908369-9f55-7ed0-b8ff-79a804e4ead5",
"user_id": "anonymous",
"inserted_at": "2024-07-05 15:03:25.108429 UTC",
"recorded_at": "2024-07-05 15:01:00.578734 UTC",
"command": "sg live",
"version": "unknown",
"flags_and_args": "{\"args\":[\"dotcom\"],\"flags\":{\"live\":null,\"sg\":[\"disable-analytics\"]}}",
"duration": "0-0 0 0:0:4",
"error": "",
"data": null,
"metadata": "{\"cancelled\":false,\"failed\":false,\"panicked\":false,\"success\":true}"
}, {
"uuid": "0190836b-e1e3-7b2b-8002-e5eba10f2a27",
"user_id": "anonymous",
"inserted_at": "2024-07-05 15:04:22.344947 UTC",
"recorded_at": "2024-07-05 15:03:33.360211 UTC",
"command": "sg bazel configure",
"version": "unknown",
"flags_and_args": "{\"args\":[],\"flags\":{\"bazel\":null,\"configure\":null,\"sg\":[\"disable-analytics\"]}}",
"duration": "0-0 0 0:0:9",
"error": "signal: killed",
"data": null,
"metadata": "{\"cancelled\":true,\"failed\":true,\"panicked\":false,\"success\":false}"
}]
```
The following event attributes are puslished:
- command
- version
- flags_ang_args
- error
- recorded_at
- metadata: which contains cancelled, failed, panicked and sucess
(should this possibly be stored under status instead?)
The metadata field can and _should_ be expanded.
* Need to expand what is added to metadata
* Need to use and expand what is put into data
Tested locally and
https://console.cloud.google.com/bigquery?referrer=search&project=sourcegraph-local-dev&ws=!1m10!1m4!4m3!1ssourcegraph-local-dev!2ssg_analytics!3sevents!1m4!1m3!1ssourcegraph-local-dev!2sbquxjob_761f4a78_1908370012d!3sus-central1
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
ef35f8d to
93a55c9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes existing
sg analyticscommand and replaces it with a one-per-invocation sqlite backed approach. This is a local storage for invocation events before theyre pushed to bigqueryTest plan
Changelog