Skip to content

The jemalloc's background thread is disabled #9812

@CalvinNeo

Description

@CalvinNeo

Bug Report

Please answer these questions before submitting your issue. Thanks!

The current implementation is

    auto * malloc_conf = getenv("MALLOC_CONF");

    if (!malloc_conf && !old_b)
    {
        // enable background thread
    }

In previous version, there is not MALLOC_CONF set, so the background thread is enabled by default.

However, after pingcap/tiup@bac3c2e (pingcap/tiup#2437), TiFlash is bootstrapped with a given MALLOC_CONF env string which does not specify background_thread at all. So the background thread will not be enabled.

If the background thread is not enabled, dirty pages could not be purged in the expected time, if using jemalloc 5.2.1.

There are mainly several ways to dirty pages in jemalloc:

  • Manual purge called by mallctl
  • The value of dirty_decay_ms is changed
  • A counter named decay_ticker will decrease from 1000, on alloc/dealloc events. Once it goes to 0, it would check if the current epoch should be advanced according to deadline. If so, a purge will trigger.
  • The background thread is enabled, and it would trigger purge in a given interval.

The problem is, if there is no frequent alloc/dealloc events after one dealloc, the decay_ticker will not be decreased. thus the epoch is not checked, and the memory is not released.

So, we'd better enable background_thread again.

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiFlash version? (Required)

Metadata

Metadata

Assignees

Labels

affects-6.1This bug affects the 6.1.x(LTS) versions.affects-6.5This bug affects the 6.5.x(LTS) versions.affects-7.1This bug affects the 7.1.x(LTS) versions.affects-7.5This bug affects the 7.5.x(LTS) versions.affects-8.1This bug affects the 8.1.x(LTS) versions.affects-8.5This bug affects the 8.5.x(LTS) versions.component/storageseverity/majortype/bugThe issue is confirmed as a bug.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions