Skip to content

Feature Request: Prevent the MaxReplicationLagModule from flooding the vttablet log #14874

@ejortegau

Description

@ejortegau

Feature Description

If the MaxReplicationLagModuleis used (e.g., because you enable the Transaction Throttler), the vttablet logs are FULL with messages like these:

...
I0102 08:27:27.659354   39655 max_replication_lag_module.go:384] rate was: not changed from: 3215 to: 3215
alias: us_east_1b-0170349799 lag: 0s
last change: 0.1s rate: 0 good/bad?  skipped b/c:  good/bad: 485/0
state (old/tested/new): I/I/I
lag before: n/a (n/a ago) rates (primary/replica): 0/0 backlog (old/new): 0/0
reason: skipping this replica because we're waiting for the next lag record from the 'replica under test': us_east_1b-0170354798
I0102 08:27:27.684415   39655 max_replication_lag_module.go:384] rate was: not changed from: 3215 to: 3215
alias: us_east_1d-0170385729 lag: 0s
last change: 0.1s rate: 0 good/bad?  skipped b/c:  good/bad: 485/0
state (old/tested/new): I/I/I
lag before: n/a (n/a ago) rates (primary/replica): 0/0 backlog (old/new): 0/0
reason: skipping this replica because we're waiting for the next lag record from the 'replica under test': us_east_1b-0170354798
I0102 08:27:28.546290   39655 max_replication_lag_module.go:384] rate was: not changed from: 3215 to: 3215
alias: us_east_1b-0170354798 lag: 0s
last change: 1.0s rate: 0 good/bad?  skipped b/c:  good/bad: 485/0
state (old/tested/new): I/I/I
lag before: n/a (n/a ago) rates (primary/replica): 0/0 backlog (old/new): 0/0
reason: waiting 4.0 more seconds to see if the lag has changed
I0102 08:27:28.547133   39655 max_replication_lag_module.go:384] rate was: not changed from: 3215 to: 3215
alias: us_east_1d-0170385965 lag: 0s
last change: 1.0s rate: 0 good/bad?  skipped b/c:  good/bad: 485/0
state (old/tested/new): I/I/I
lag before: n/a (n/a ago) rates (primary/replica): 0/0 backlog (old/new): 0/0
reason: skipping this replica because we're waiting for the next lag record from the 'replica under test': us_east_1b-0170354798
...

A log record is generated every time that the max rate is recalculated - which happens quite often, hence the log is very noisy.

Request: do not pollute the log with MaxReplicationLagModule.recalculateRate() info.

Use Case(s)

Looking for anything in the log when the transaction throttler is enabled is difficult due to the amount of noise generated by this logging.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions