Skip to content

Raft peers may get stuck in busy apply state post TiKV startup #18233

New issue
New issue
Closed
Closed
Raft peers may get stuck in busy apply state post TiKV startup#18233
#19199 (+2)
Assignees
LykxSassinator
Labels
affects-7.1This bug affects the 7.1.x(LTS) versions.affects-7.5This bug affects the 7.5.x(LTS) versions.affects-8.1This bug affects the 8.1.x(LTS) versions.affects-8.5This bug affects the 8.5.x(LTS) versions.impact/upgradereport/customerCustomers have encountered this bug.severity/minortype/bugThe issue is confirmed as a bug.
@hbisheng

Description

@hbisheng
hbisheng
opened on Feb 20, 2025

Bug Report

We observed that after a TiKV instance restarted, some peers got stuck in the "busy apply" state for 2 minutes while there was no Raft log lag. As a result, the server reported the "Server is busy" error and prevented PD from transferring leaders to the TiKV instance.

Image

Image

During this time, Raftstore had no operations, indicating that the peers were not performing any actions.

Image

One potential cause is that the peers went into hibernation before successfully exiting the "busy apply" state.

Image

What version of TiKV are you using?

v8.5.1

Metadata

Metadata

Assignees

Labels

affects-7.1This bug affects the 7.1.x(LTS) versions.affects-7.5This bug affects the 7.5.x(LTS) versions.affects-8.1This bug affects the 8.1.x(LTS) versions.affects-8.5This bug affects the 8.5.x(LTS) versions.impact/upgradereport/customerCustomers have encountered this bug.severity/minortype/bugThe issue is confirmed as a bug.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions