Skip to content

The threshold of schema apply diff/full is fixed and short #6532

@zanmato1984

Description

@zanmato1984

Enhancement

The threshold of schema apply diff/full is hard coded as 100. This value could be tight when users are doing highly frequent DDL operations. The problem of this value being small is that, once apply full is triggered, and unluckily there exists tons of tables, the apply diff process could take a long time (for a real world case, 1 to 2 minutes for 60k tables). And given that schema apply process will block read (from getting the schema version) during the whole applying period, this 1 to 2 minutes could result in many read failures.

We may be able to enhance this in some way below:

  1. Make the global schema version an atomic variable to avoid the schema lock, so that schema apply won't block it for long. But we may need to verify the correctness carefully.
  2. Make the threshold relative to current schema version, like 10%, so that for a cluster with too many tables, the apply diff becomes more eager.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions