-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: refactor actual tags assigned to tests #100605
Copy link
Copy link
Closed
Labels
C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-testengTestEng TeamTestEng Team
Description
Today, when working with roachtest tags:
- If a roachtest has no user assigned tag, the "default" tag is added
- All "default" tagged roachtests are assumed to run on GCE
- Said roachtests run daily, unless user tagged "weekly"
- Only tests tagged "aws" will run on AWS (recently updated from a gnarly name regex)
This is confusing in several ways:
- Inconsistency in selecting jobs for different clouds. GCE and local is assumed, AWS is explicit
- Cadence is specified by "weekly" or "aws-weekly" (gce again implied)
- If a user adds any "tag", it will have the unexpected consequence of stopping the test from being assigned the "default" tag, which would then exclude it from running in GCE
The existing tags should be refactored to make it easy for a user or TC job to select relevant roachtests. Some things that can be implemented:
- Explicitly naming clouds (no more assume GCE)
- Adopt a "cloud-cadence" tag and assign to all tests. e.g.
aws-weekly,gce-daily. This would allow us to categorise tests easily according to existing TeamCity jobs (gce/aws daily/weekly) - Sensible tags added at runtime (like what we do now for "default"). e.g. If no cloud-cadence tags, add "all-daily" tag to run on as default.
We could also more narrowly selecting relevant tests, and skip logging"Ignored" tests in the TC UI. roachtest run tag:aws-daily would show only those that are supposed to run.
Jira issue: CRDB-26519
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-testengTestEng TeamTestEng Team