Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

[Backport 5.2] telemetry-export: 5.2.1 default enablement (#57605)#57681

Merged
keegancsmith merged 1 commit into
5.2from
backport-57605-to-5.2
Oct 18, 2023
Merged

[Backport 5.2] telemetry-export: 5.2.1 default enablement (#57605)#57681
keegancsmith merged 1 commit into
5.2from
backport-57605-to-5.2

Conversation

@bobheadxi

@bobheadxi bobheadxi commented Oct 17, 2023

Copy link
Copy Markdown
Member

This change implements the license-based enforcement of telemetry export as outlined in the plan for the 5.2.1 release.

Most of this functionality has been tested in various environments for some time: dotcom, s2, rctest, cody-dev, demo, and 9 trial Cloud instances (with more to be enabled before the final 5.2.1 release, see https://github.com/sourcegraph/customer/issues/2416), processing about 2 million events in the last 7 days (a small set of events currently instrumented in the new system). The following changes are made to how this feature is enabled:

  1. An export enablement mode is added to package internal/licensing that evaluates what we export based on the configured license, configured based on license plan or tags in the license, but the levels are unlikely to change):
    • all events (default going forward)
    • cody-only (Feature=~'cody|cody.*'), since Cody events are a special case covered by terms of use
    • nothing (air-gapped instance)
  2. When persisting events to the export queue, the enablement mode is checked to decide if anything gets written to the database.
  3. Exporter background workers are now enabled by default, checking the mode on each iteration to decide if we need to run the exporter at all. We also create a new connection to Telemetry Gateway on each run now, to ensure Telemetry Gateway downtime does not break instances.

The feature flag telemetry-export and env var TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR remain configurable, but in a default-enabled state - in 5.2.2, the ability to disable these capabilities via configuration will be removed, and the license-based mechanism will become the only way to disable this feature, as requested in the telemetry export rollout plan. Parts that require follow-up changes are denoted with TODO(5.2.2) comments.

A changelog entry and docs are added in https://github.com/sourcegraph/sourcegraph/pull/57606

Part of https://github.com/sourcegraph/sourcegraph/issues/56816

  • New unit/integration test coverage
  • Existing tests pass with minimal changes
  • Manual testing (run again on backport branch)
    • Comment out sg.config.yaml line TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR: "http://127.0.0.1:10080" and add TELEMETRY_GATEWAY_EXPORTER_EXPORT_INTERVAL: "30s", exporter starts by default. Run some searches to generate events, and see that worker doesn't export anything, as the default mode for an older license is to disable export (seen in logs and metrics)
    • Set dev-private license to a new, real one from sourcegraph.com, remove all mentions of SOURCEGRAPH_LICENSE_GENERATION_KEY from sg.config.yaml, repeat above, events now export (seen in logs and metrics):
      [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:129 events exported {"maxBatchSize": 10000, "succeeded": 1}

Co-authored-by: Joe Chen joe@sourcegraph.com
(cherry picked from commit 20c771b)

cc @sourcegraph/release-guild

@cla-bot cla-bot Bot added the cla-signed label Oct 17, 2023
@bobheadxi bobheadxi changed the title telemetry-export: 5.2.1 default enablement (#57605) [Backport 5.2] telemetry-export: 5.2.1 default enablement (#57605) Oct 17, 2023
@bobheadxi bobheadxi added the backport/improvement Final touching of existing features label Oct 17, 2023
@bobheadxi bobheadxi force-pushed the backport-57605-to-5.2 branch 3 times, most recently from 6423a7d to 94f8aca Compare October 17, 2023 22:19
@bobheadxi bobheadxi marked this pull request as ready for review October 17, 2023 22:23
@bobheadxi

bobheadxi commented Oct 17, 2023

Copy link
Copy Markdown
Member Author

I've run through the manual test plan again on this branch and verified everything is intact

@bobheadxi bobheadxi requested review from a team and keegancsmith October 17, 2023 22:24
@sourcegraph-bot

sourcegraph-bot commented Oct 17, 2023

Copy link
Copy Markdown
Contributor

Codenotify: Notifying subscribers in CODENOTIFY files for diff f4b23d6...35f4eca.

Notify File(s)
@efritz enterprise/cmd/worker/internal/telemetrygatewayexporter/BUILD.bazel
enterprise/cmd/worker/internal/telemetrygatewayexporter/exporter.go
enterprise/cmd/worker/internal/telemetrygatewayexporter/telemetrygatewayexporter.go
@unknwon internal/license/BUILD.bazel
internal/license/license_test.go
internal/license/licensetest/BUILD.bazel
internal/license/licensetest/licensetest.go
internal/licensing/BUILD.bazel
internal/licensing/tags.go
internal/licensing/telemetryexport.go
internal/licensing/telemetryexport_test.go

@sourcegraph-bot

sourcegraph-bot commented Oct 17, 2023

Copy link
Copy Markdown
Contributor

📖 Storybook live preview

This change implements the license-based enforcement of telemetry export as outlined in [the plan for the 5.2.1 release](https://docs.google.com/document/d/1Z1Yp7G61WYlQ1B4vO5-mIXVtmvzGmD7PqYHNBQV-2Ik/edit#bookmark=id.w10771x4dx7y).

Most of this functionality has been tested in various environments for some time: dotcom, s2, rctest, cody-dev, demo, and 9 trial Cloud instances (with more to be enabled before the final 5.2.1 release, see sourcegraph/customer#2416), processing about 2 million events in the last 7 days (a small set of events currently instrumented in the new system). The following changes are made to how this feature is enabled:

1. An export enablement mode is added to package `internal/licensing` that evaluates what we export based on the configured license, configured based on license plan or tags in the license, but the levels are unlikely to change):
   - all events (default going forward)
   - cody-only (`Feature=~'cody|cody.*'`), since Cody events are a special case covered by terms of use
   - nothing (air-gapped instance)
3. When persisting events to the export queue, the enablement mode is checked to decide if anything gets written to the database.
4. Exporter background workers are now enabled by default, checking the mode on each iteration to decide if we need to run the exporter at all. We also create a new connection to Telemetry Gateway on each run now, to ensure Telemetry Gateway downtime does not break instances.

The feature flag `telemetry-export` and env var `TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR` remain configurable, but in a default-enabled state - in 5.2.2, the ability to disable these capabilities via configuration will be removed, and the license-based mechanism will become the only way to disable this feature, as requested in the [telemetry export rollout plan](https://docs.google.com/document/d/1Z1Yp7G61WYlQ1B4vO5-mIXVtmvzGmD7PqYHNBQV-2Ik/edit#bookmark=id.w10771x4dx7y). Parts that require follow-up changes are denoted with `TODO(5.2.2)` comments.

A changelog entry and docs are added in https://github.com/sourcegraph/sourcegraph/pull/57606

Part of https://github.com/sourcegraph/sourcegraph/issues/56816

- [x] New unit/integration test coverage
- [x] Existing tests pass with minimal changes
- [x] Manual testing:
  - Comment out `sg.config.yaml` line `TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR: "http://127.0.0.1:10080"` and add `TELEMETRY_GATEWAY_EXPORTER_EXPORT_INTERVAL: "30s"`, exporter starts by default. Run some searches to generate events, and see that worker doesn't export anything, as the default mode for an older license is to disable export (seen in logs and metrics)
  - Set `dev-private` license to a new, real one from sourcegraph.com, remove all mentions of `SOURCEGRAPH_LICENSE_GENERATION_KEY` from `sg.config.yaml`, repeat above, events now export (seen in logs and metrics):
    `[         worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:129 events exported {"maxBatchSize": 10000, "succeeded": 1}`

---------

Co-authored-by: Joe Chen <joe@sourcegraph.com>
(cherry picked from commit 20c771b)
@bobheadxi bobheadxi force-pushed the backport-57605-to-5.2 branch from 94f8aca to 35f4eca Compare October 18, 2023 02:32
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants