Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

gitserver: Experiment with janitor#62619

Closed
eseliger wants to merge 1 commit into
mainfrom
es/05-13-gitserverexperimentwithjanitor
Closed

gitserver: Experiment with janitor#62619
eseliger wants to merge 1 commit into
mainfrom
es/05-13-gitserverexperimentwithjanitor

Conversation

@eseliger

@eseliger eseliger commented May 13, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Just experimenting with a new janitor implementation, nothing concrete to learn here yet.

What's missing:

  • Currently, when GC fails multiple times, we will perform a full reclone. Should we even do that? The new Janitor needs a lot of memory.We could also alert and admins can use the reclone graphQL mutation to trigger a new repo clone
  • SG Maintenance had a lock around maintenance operations so that not even git performs any operations; namely writing the gc.pid file.
  • Who schedules the jobs? With fixed concurrency, it would be great if a user-triggered aggressive optimization would take one of those slots, and not fill an additional slot. Also, would be nice if we didn't lose those jobs across restarts. So maybe we should just use a dbworker with concurrency, and a filter for "does this belong on this shard?". For that to work, we would need to be able to trust the database (at least mostly) that the shard_id column is correct (and we do a recheck and requeue the job if the assignment is wrong).

@cla-bot cla-bot Bot added the cla-signed label May 13, 2024
This was referenced May 13, 2024
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@eseliger Graphite App

eseliger commented May 13, 2024 •
edited
Loading

Copy link
Copy Markdown
Member Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @eseliger and the rest of your teammates on Graphite Graphite

@github-actions github-actions Bot added team/product-platform team/source Tickets under the purview of Source - the one Source to graph it all labels May 13, 2024
github-advanced-security[bot]
github-advanced-security AI found potential problems May 13, 2024
View reviewed changes
Comment thread cmd/gitserver/internal/janitor/cleanup.go Fixed
@eseliger eseliger force-pushed the es/05-13-gitserverreducenumberofgitconfiginvocations branch from 65cc349 to 1fe128f Compare May 13, 2024 03:13
@eseliger eseliger force-pushed the es/05-13-gitserverexperimentwithjanitor branch from 3cdddbe to 79e64b5 Compare May 13, 2024 03:13
@eseliger eseliger force-pushed the es/05-13-gitserverreducenumberofgitconfiginvocations branch from 1fe128f to 79cba1f Compare May 13, 2024 03:29
@eseliger eseliger force-pushed the es/05-13-gitserverexperimentwithjanitor branch from 79e64b5 to af8bc59 Compare May 13, 2024 03:29
@eseliger eseliger force-pushed the es/05-13-gitserverreducenumberofgitconfiginvocations branch from 79cba1f to 804d532 Compare May 13, 2024 13:15
@eseliger eseliger force-pushed the es/05-13-gitserverexperimentwithjanitor branch from af8bc59 to d20e3ac Compare May 13, 2024 13:15
@eseliger eseliger force-pushed the es/05-13-gitserverreducenumberofgitconfiginvocations branch from 804d532 to 0b9d527 Compare May 13, 2024 14:45
Base automatically changed from es/05-13-gitserverreducenumberofgitconfiginvocations to main May 13, 2024 14:54
@eseliger eseliger force-pushed the es/05-13-gitserverexperimentwithjanitor branch 4 times, most recently from 1de8b49 to 886eacd Compare May 20, 2024 00:23
@eseliger eseliger force-pushed the es/05-13-gitserverexperimentwithjanitor branch 3 times, most recently from 7173b4a to 3673ca8 Compare May 24, 2024 04:02
@eseliger eseliger force-pushed the es/05-13-gitserverexperimentwithjanitor branch from 3673ca8 to ae6579a Compare May 31, 2024 02:18
@eseliger
gitserver: Experiment with janitor
30942f8 
Just experimenting with a new janitor implementation, nothing concrete to learn here yet.
@eseliger eseliger force-pushed the es/05-13-gitserverexperimentwithjanitor branch from ae6579a to 30942f8 Compare June 3, 2024 04:54
@eseliger eseliger closed this Sep 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

cla-signed team/product-platform team/source Tickets under the purview of Source - the one Source to graph it all

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@eseliger @github-advanced-security