Skip to content

jobs: enhance the InfoStorage API#98993

Merged
craig[bot] merged 4 commits intocockroachdb:masterfrom
knz:20230319-job-info-storage
Mar 21, 2023
Merged

jobs: enhance the InfoStorage API#98993
craig[bot] merged 4 commits intocockroachdb:masterfrom
knz:20230319-job-info-storage

Conversation

@knz
Copy link
Copy Markdown
Contributor

@knz knz commented Mar 19, 2023

Needed for #98459.
Epic: CRDB-23559

@knz knz requested a review from adityamaru March 19, 2023 21:55
@knz knz requested a review from a team as a code owner March 19, 2023 21:55
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Contributor

@adityamaru adityamaru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this, just a couple of non-blocking comments!

infoStorage := job2.InfoStorage(txn)
return infoStorage.GetLast(ctx, kPrefix, func(key, value []byte) error {
i++
require.Equal(t, key, kC)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think kC only has one revision, could we write a few more revisions before iterating?

var iterConfig string
switch iterMode {
case iterateAll:
iterConfig = `ORDER BY info_key ASC, written DESC` // with no LIMIT.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given that for a job_id there will only ever be one row per info_key we could drop the secondary sort on written, right?

@knz knz force-pushed the 20230319-job-info-storage branch from b996ef6 to 220083b Compare March 20, 2023 21:15
Copy link
Copy Markdown
Contributor Author

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @adityamaru)


pkg/jobs/job_info_storage.go line 161 at r3 (raw file):

Previously, adityamaru (Aditya Maru) wrote…

given that for a job_id there will only ever be one row per info_key we could drop the secondary sort on written, right?

Discussed internally here https://cockroachlabs.slack.com/archives/C01RX2G8LT1/p1679345511855089 - in summary it doesn't matter here.
I'm going to keep it with an explanatory comment.


pkg/jobs/job_info_storage_test.go line 162 at r3 (raw file):

Previously, adityamaru (Aditya Maru) wrote…

I think kC only has one revision, could we write a few more revisions before iterating?

Done.

@knz
Copy link
Copy Markdown
Contributor Author

knz commented Mar 20, 2023

TFYR!

bors r=adityamaru

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 20, 2023

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 21, 2023

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 21, 2023

Build succeeded:

@craig craig bot merged commit b806772 into cockroachdb:master Mar 21, 2023
craig bot pushed a commit that referenced this pull request Apr 3, 2023
98459: server,autoconfig: automatic configuration via config tasks r=adityamaru a=knz

Epic: CRDB-23559
Informs #98431.
All commits but the last are from #98993.

This change introduces "auto config tasks", a mechanism through which
configuration payloads ("tasks") can be injected into a running SQL
service.

This is driven via the "auto config runner" job that was introduced in
the previous commit. The job listens for the arrival of new task
definitions via a `Provider` interface. When new tasks are known, and
previous tasks have completed, the runner creates a job for the first
next task.

Release note: None

100476: server/drain: shut down SQL subsystems gracefully before releasing table leases r=JeffSwenson,rytaft a=knz

Needed for #99941 and #99958.
Epic: CRDB-23559

See individual commits for details.

100511: sqlccl: deflake TestGCTenantJobWaitsForProtectedTimestamps r=adityamaru,arulajmani a=knz

Fixes #94808

The tenant server must be shut down before the tenant record is removed; otherwise the tenant's SQL server will self-terminate by calling Stop() on its stopper, which in this case was shared with the parent cluster.

Release note: None

Co-authored-by: Raphael 'kena' Poss <knz@thaumogen.net>
@knz knz deleted the 20230319-job-info-storage branch April 6, 2023 21:37
@knz
Copy link
Copy Markdown
Contributor Author

knz commented Apr 7, 2023

blathers backport 23.1

@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Apr 7, 2023

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from fa90381 to blathers/backport-release-23.1-98993: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 23.1 failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants