Feature Description

In order to control the background tasks' resource usage, we plan to add an extra quota resource limit layer to the background jobs. That is, for foreground jobs, on the tikv side, the scheduler only controls the enqueue/deque order. Thus, running tasks is not under control, and tasks from different routine(thread pool) are not controlled either. But for the background jobs, the extra quota limiter layer can ensure the running tasks(in different pools) can be preempted early enough, so their resource usage can appeal to the setting:

• Control the resource usage of all background tasks by the Resource Limiter : The rate limit is dynamically adjusted to the value via the formula TiKVTotalRUCapcity - sum(RUCostRateOfForgroundTasks), with a fine-grained adjusting duration, we can ensure the foreground tasks' RU is always enough(or near the system's maximum if the foreground requirement reaches the maximum quota), so the background tasks' impact on foregroup tasks should be very low; on the other hand, when the foreground resource consumption is low, the controller should increase the limit threshold, so background jobs can take advantage of the remaining resources.
• The local resource manager will statics RU consumption of background jobs via the Resource Limiter : We will do statistics and report the resource consumption to the global resource manager. In the first stage, we only do statistics globally but control it locally.
• Feedback mechanism: It's better to give feedback on how fast the limiter layer executes tasks on tikv to the upper layer like tidb, so that the upper layer task framework can adjust the number of tasks.

Task Breakdown

Management

• Interface
  • Add SQL syntax to config resource group background setting parser,ddl: add background job syntax and meta management for resource control #45189
  • Add session variable to set the task type of current session *: label external the request source via sessionvar #44770
  • All Background REF_GROUP set to Default, and relative API
• Label the task as a background task and the {group_name@lightning @nolouch @HuSharp
  • Lightning Label the lightning request source and background job #44769
    - [ ] TiDB @nolouch
    - [ ] Local @HuSharp
  • BR
  • DDL / ANALYZE?
  • Verify seperate binary @HuSharp
• Integrate into another component

TiKV Resource Limiter tikv/tikv#14900

• Resource Group background-related metadata management @glorv resource_control: support manage resource group background metadata tikv/tikv#14921
• Limiter Future Interface @glorv resource_control: wrap resource limiter in future tikv/tikv#14931
• Background resource limiter and quota auto adjustment. resource_control: support auto adjust background quota limit tikv/tikv#14980
• Background tasks' RU consumption report @nolouch