Skip to content

overlay: add byte limit to flow control#3737

Merged
latobarita merged 13 commits intostellar:masterfrom
marta-lokhova:flow_control_bytes_MAY
May 20, 2023
Merged

overlay: add byte limit to flow control#3737
latobarita merged 13 commits intostellar:masterfrom
marta-lokhova:flow_control_bytes_MAY

Conversation

@marta-lokhova
Copy link
Contributor

@marta-lokhova marta-lokhova commented May 16, 2023

Functionally the change is equivalent to #3607, but opening a new PR since with the latest refactor the code is sufficiently different.

One more TODO I'm currently working on is proper handling of Soroban txMaxSize upgrades, but we can start reviewing this change already (the upgrade handling is behind the vNext flag anyways)

@marta-lokhova marta-lokhova force-pushed the flow_control_bytes_MAY branch from 3da16d1 to 6012cdb Compare May 16, 2023 16:17
@marta-lokhova marta-lokhova requested review from SirTyson and mbsdf May 16, 2023 16:37
@marta-lokhova
Copy link
Contributor Author

(note: still need to update vNext XDR, so the build is currently failing)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heads up: this const and capacity configs are placeholders for now. I'll update those once I finish simulation tuning

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why minus send_more_batch_size?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is max classic tx size? do we read it from somewhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a const. In the future, the actual max tx size value will be derived from this const and Soroban network config.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can send_more_batch_size be less than max_tx_size?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, batch size is just a frequency of data requests.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what it means if less than batch_size?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this enable/disable flag for flow control in bytes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this pointer is set if a connection decides to use fc in bytes

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be nice to add a description in this PR to explain peer flood capacity, batch size for both sender and receiver (maybe a simple graph will be better?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

send more and send more extended, how they are different?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

send_more is for current fc (messages only), while extended includes byte counts

@marta-lokhova marta-lokhova force-pushed the flow_control_bytes_MAY branch from c9d5a09 to ceb3bef Compare May 19, 2023 00:06
Copy link
Contributor

@SirTyson SirTyson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions but LGTM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add OUTBOUND_TX_QUEUE_BYTE_LIMIT here too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth the overhead to add an extension switch here to avoid SendMoreExtenededExteneded in the future?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for overlay, we normally introduce new messages instead of extending old ones, since older overlay versions become obsolete and this approach makes it easy to deprecate old behavior (note that this is different from protocol, where we need to support old behavior forever)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why do we have a message count reading capacity but not a byte based reading capacity?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question. the reading capacity is a purely internal concept, needed to ensure fairness across all of our connections (what we used to do was read from a peer while there was any data available, potentially starving other peers). the reason I decided to keep only the message counting for now was because the largest message we allow in overlay is 16MB anyway, so it seemed like it wasn't super useful to try to enforce the existing byte limits, since they would be way off when we receive larger messages like tx sets. That being said, we might do something better than this in the future, but for now I'm leaving it as-is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Double negation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This value seems cryptic, I'd add some comment saying something like "must be a value > 0 && value != 100" with a brief explanation as to why.

@marta-lokhova marta-lokhova force-pushed the flow_control_bytes_MAY branch from 5ca9b25 to 784ad75 Compare May 19, 2023 22:25
"can't be greater than PEER_FLOOD_READING_CAPACITY_BYTES";
throw std::runtime_error(msg);
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link

@mbsdf mbsdf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good to me. Only have a few comments.

PS: already reviewed #3607 and reviewing this PR was much easier after overlay refactoring #3686

@marta-lokhova marta-lokhova force-pushed the flow_control_bytes_MAY branch from 80ac752 to da8e7c7 Compare May 20, 2023 00:39
@mbsdf
Copy link

mbsdf commented May 20, 2023

r+ da8e7c7

@marta-lokhova
Copy link
Contributor Author

r+ f012569

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants