gitserver: Implement Perforce changelist ID mapper as standalone job#62868
Conversation
|
This stack of pull requests is managed by Graphite. Learn more about stacking. |
This PR moves the changelist mapper to be a proper worker job and be monitored by the usual worker monitoring. It also makes use of well-tested gitserver APIs instead of running git commands itself. This also moves from a reactive system that purely resides in memory to a system that occasionally makes sure all perforce depots are properly mapped. This should be slightly more DB queries to find the depots that have a change, but it shouldn't be a huge burden, and this is another good indicator that events can be useful :) Test plan: I've added some "integration-ish" tests, and ran it locally to verify that it works. It will be good to also implement an integration test, once we have the tools for that.
1cc7cef to
cc40a4f
Compare
There was a problem hiding this comment.
Great changes - moving into worker, checking the database for changed repos, using gitserver APIs. The potential delay introduced by not queuing this up immediately after a repo sync might cause som UI jank, as users will see git commit hashes until the mapping is done, but as long as SRC_PERFORCE_CHANGELIST_MAPPER_INTERVAL is kept low, the UI impact should not be any worse than the previous implementation, and will actually be more reliable, since previously the delay depended on the size of the queue. Some documentation about that environment variable encouraging admins to leave it low (and reassuring them of the low impact this process has) might be in order.
reference: internal discussion thread
|
plus a lot to this being another example of how useful a |
|
Implemented https://github.com/sourcegraph/sourcegraph/pull/62871 to alleviate the indexing latency concerns. |
This PR moves the changelist mapper to be a proper worker job and be monitored by the usual worker monitoring.
It also makes use of well-tested gitserver APIs instead of running git commands itself.
This also moves from a reactive system that purely resides in memory to a system that occasionally makes sure all perforce depots are properly mapped. This should be slightly more DB queries to find the depots that have a change, but it shouldn't be a huge burden, and this is another good indicator that events can be useful :)
Test plan:
I've added some "integration-ish" tests, and ran it locally to verify that it works.
It will be good to also implement an integration test, once we have the tools for that.