Support throttling vstreamer copy table work on source tablets#9923
Support throttling vstreamer copy table work on source tablets#9923rohit-nayak-ps merged 17 commits intovitessio:mainfrom
Conversation
5d4a46a to
b0e7f4c
Compare
f924c86 to
ca8468c
Compare
5c27419 to
4bbabfb
Compare
94b8a91 to
f33ee77
Compare
Signed-off-by: Matt Lord <mattalord@gmail.com>
f33ee77 to
ac8564a
Compare
Signed-off-by: Matt Lord <mattalord@gmail.com>
7af1c00 to
e89f98b
Compare
Signed-off-by: Matt Lord <mattalord@gmail.com>
e89f98b to
9de8182
Compare
Signed-off-by: Matt Lord <mattalord@gmail.com>
fed244e to
4fca26b
Compare
Signed-off-by: Matt Lord <mattalord@gmail.com>
42e2108 to
07bd5c9
Compare
And make it a gauge so that it's always showing the current value as we allow this to be changed in the running process via /debug/env. Signed-off-by: Matt Lord <mattalord@gmail.com>
2fce166 to
55cee55
Compare
55cee55 to
2c9b869
Compare
Signed-off-by: Matt Lord <mattalord@gmail.com>
2c9b869 to
e60a3ce
Compare
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Document work in vitessio/vitess#9923 Signed-off-by: Matt Lord <mattalord@gmail.com> * Move to two dashes for long flag names Signed-off-by: Matt Lord <mattalord@gmail.com> * Add detailed flag info and related info Signed-off-by: Matt Lord <mattalord@gmail.com>
|
Starting this discussion here to follow up on @deepthi's DM observation that these flag names are too long. @mattlord and I did note that, but we thought more descriptive the better since they would anyway reside in scripts or config files. However as @deepthi points out this may contribute to hitting OS command line length limits and also shorter the better, for recall purposes. @mattlord, thoughts? To start off suggestions!: |
|
I'm not opposed to keeping the full name of the innodb variable in the flag for clarity. At a minimum, we can drop |
|
IMO related flags should all have the same prefix. I would have used The I don't really agree that shorter is easier to remember, but I don't feel too strongly here. I'm not sure that OS limits are a major concern, but it's possible (1 MiB on macOS and 2MiB on Linux): Please note that the entire --help output — which includes the help text — is currently less than 60KiB: IMO what we really need is a project/ticket to define some Vitess policies for command-line options/flags and then apply them to all flags across all of vitess. Want me to create an issue for that? I don't mind being the one to work on it either — the lack of a policy is the real issue IMO (don't feel too strongly on the specifics, but the lack of uniformity and predictability is bad for users). |
|
Let's create an issue/task saying "we need to define a policy", and once there is a concrete proposal it can become an RFC. The current state is not good. We also have some flags with dashes and some with underscores. |
Description
This provides mechanisms to limit the impact of large vreplication copy table (row streamer) workflow phases on production source tablets.
It supports throttling based on the InnoDB history list length and the MySQL replication lag seen (if the source is a replica). This gives vreplication source tablets an opportunity to catch up in between copy table phase cycles (copy, catchup, fastforward). You can control this using two new vttablet flags:
Related Issue(s)
Checklist