This repository was archived by the owner on Sep 30, 2024. It is now read-only.
feat(sg): send analytic events to bigquery#63674
Merged
burmudar merged 9 commits intoJul 9, 2024
Merged
Conversation
The following fields in the events table are populated - command, version, flags_and_args, error, duration and recorded_at
burmudar
commented
Jul 5, 2024
jhchabran
reviewed
Jul 5, 2024
| metadata map[string]any | ||
| } | ||
|
|
||
| func (i invocation) GetStartTime() *time.Time { |
Contributor
There was a problem hiding this comment.
Aren't these funcs an UnmarshalJSON() on the metadata type in disguise?
Contributor
Author
There was a problem hiding this comment.
This is an excellent point 🙏🏼
Contributor
Author
There was a problem hiding this comment.
Still have to look into this
- use background context - set events array capacity - comment Save method
jhchabran
approved these changes
Jul 8, 2024
burmudar
added a commit
that referenced
this pull request
Jul 9, 2024
Reads events from the local analytics db and writes them to events table
in the bigquery `sg_analytics` dataset
```
{
"uuid": "01908369-9f55-7ed0-b8ff-79a804e4ead5",
"user_id": "anonymous",
"inserted_at": "2024-07-05 15:03:25.108429 UTC",
"recorded_at": "2024-07-05 15:01:00.578734 UTC",
"command": "sg live",
"version": "unknown",
"flags_and_args": "{\"args\":[\"dotcom\"],\"flags\":{\"live\":null,\"sg\":[\"disable-analytics\"]}}",
"duration": "0-0 0 0:0:4",
"error": "",
"data": null,
"metadata": "{\"cancelled\":false,\"failed\":false,\"panicked\":false,\"success\":true}"
}, {
"uuid": "0190836b-e1e3-7b2b-8002-e5eba10f2a27",
"user_id": "anonymous",
"inserted_at": "2024-07-05 15:04:22.344947 UTC",
"recorded_at": "2024-07-05 15:03:33.360211 UTC",
"command": "sg bazel configure",
"version": "unknown",
"flags_and_args": "{\"args\":[],\"flags\":{\"bazel\":null,\"configure\":null,\"sg\":[\"disable-analytics\"]}}",
"duration": "0-0 0 0:0:9",
"error": "signal: killed",
"data": null,
"metadata": "{\"cancelled\":true,\"failed\":true,\"panicked\":false,\"success\":false}"
}]
```
The following event attributes are puslished:
- command
- version
- flags_ang_args
- error
- recorded_at
- metadata: which contains cancelled, failed, panicked and sucess
(should this possibly be stored under status instead?)
The metadata field can and _should_ be expanded.
* Need to expand what is added to metadata
* Need to use and expand what is put into data
Tested locally and
https://console.cloud.google.com/bigquery?referrer=search&project=sourcegraph-local-dev&ws=!1m10!1m4!4m3!1ssourcegraph-local-dev!2ssg_analytics!3sevents!1m4!1m3!1ssourcegraph-local-dev!2sbquxjob_761f4a78_1908370012d!3sus-central1
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reads events from the local analytics db and writes them to events table in the bigquery
sg_analyticsdatasetExample of published events
Fields that are published
The following event attributes are puslished:
The metadata field can and should be expanded.
What is missing?
Test plan
Tested locally and https://console.cloud.google.com/bigquery?referrer=search&project=sourcegraph-local-dev&ws=!1m10!1m4!4m3!1ssourcegraph-local-dev!2ssg_analytics!3sevents!1m4!1m3!1ssourcegraph-local-dev!2sbquxjob_761f4a78_1908370012d!3sus-central1
Changelog