Skip to content

*: implement basic flow control#733

Merged
asddongmen merged 14 commits intopingcap:masterfrom
asddongmen:simpler-memory-control
Dec 31, 2024
Merged

*: implement basic flow control#733
asddongmen merged 14 commits intopingcap:masterfrom
asddongmen:simpler-memory-control

Conversation

@asddongmen
Copy link
Collaborator

@asddongmen asddongmen commented Dec 27, 2024

ref #736

What is changes?

This PR introduces path-level and area-level flow control mechanisms.
For detailed explanations of these concepts, please refer to the documentation added in this PR.

Testing

Manual Testing: Verified the functionality through manual testing.

As shown in the diagram below, a path is paused when its used memory exceeds 20% of the maximum quota.

image

Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>

DS: fix panic

Signed-off-by: dongmen <414110582@qq.com>

DS: fix bug

Signed-off-by: dongmen <414110582@qq.com>

eventcollector: adjust some codes

Signed-off-by: dongmen <414110582@qq.com>

eventcollector: fix sentResolvedTs being update unexpectedly

Signed-off-by: dongmen <414110582@qq.com>

eventbroker: update some comments

Signed-off-by: dongmen <414110582@qq.com>

eventbroker: fix unit test

Signed-off-by: dongmen <414110582@qq.com>

small changes

Signed-off-by: dongmen <414110582@qq.com>

workload: fix workload

Signed-off-by: dongmen <414110582@qq.com>

code adjust

Signed-off-by: dongmen <414110582@qq.com>

*: implement a simpler flow control

Signed-off-by: dongmen <414110582@qq.com>
@asddongmen asddongmen changed the title DS: add pathSizeHeap *: implement simple flow control Dec 27, 2024
Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
@asddongmen asddongmen mentioned this pull request Dec 30, 2024
5 tasks
@asddongmen asddongmen marked this pull request as ready for review December 30, 2024 03:45
@asddongmen asddongmen changed the title *: implement simple flow control *: implement basic flow control Dec 30, 2024
Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
// Caller should avoid to use this method if the remote EventService maybe offline forever.
// Caller should avoid to use this method when the remote EventService is offline forever.
// And this method may be deprecated in the future.
// FIXME: Add a checking mechanism to avoid sending request to offline EventService.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we also create a issue for this? to prevent too much FIXME and TODO in the code base ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea.

Comment on lines 669 to 675
d.eventServiceInfo.RLock()
defer d.eventServiceInfo.RUnlock()

if d.eventServiceInfo.serverID == "" || !d.eventServiceInfo.readyEventReceived {
log.Panic("should not happen: reset dispatcher before receiving ready signal")
// Just ignore the request if the dispatcher is not ready.
return
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks this can be split to a method

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good advice, I will do it in a separate PR.

Signed-off-by: dongmen <414110582@qq.com>
Signed-off-by: dongmen <414110582@qq.com>
@asddongmen asddongmen merged commit 353751c into pingcap:master Dec 31, 2024
@asddongmen asddongmen deleted the simpler-memory-control branch January 10, 2025 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants