A0-4318: rate-limiter for all #1857
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- parametrized by TimeProvider instead of taking `now: Instant` as argument for `rate_limit` - simplified rate_limit method - re-aligned tests of TokenBucket - TokenBucket can be set to rate_per_second = 0 - no data will be read
- seperate RateLimiter implementations for [`tokio::io::AsyncRead`] and [`futures::AsyncRead`] - using [`std::sync::Mutex`] for TokenBucket in RateLimiter - it should allow for `global` rate-limit, not just per connection
… substrate's [`sc_client::network::NetworkWorker`]
- s/alephbft_bit_rate/alephbft_network_bit_rate
…ces `TokenBucket`
…src/network/build/mod.rs`]
… [`finality-aleph/src/network/build/transport.rs`]
…port.rs`] - removed references to [`libp2p`] from [`finality-aleph/src/network/build/mod.rs`]
…d of `requeste` and `available`. It should be also easier now to transform it into wait-free version.
… - so far only tested locally using 7 instances of aleph-node.
…>` in `network/substrate.rs`
…n `finality-aleph/src/network/build/mod.rs`
…ant>>` in TokenBucket
- added ShareTokenBucket, AsyncTokenBucket and ShredBandwidthManager - used for sharing bandwidth between multiple connections
…and Socket-based networks (sync + alephbft) - new default values for rate-limiters (sync + alephbft). 768Kib for alephbft and 5Mib for sync. These values were tested using t3a.xlarge aws instances - nodes were not able to handle more.
…docker/docker_entrypoint.sh to allow configuration of rate-limiting for both sync and alephbft networks
…ated with impls of AsyncRead, etc.
… types in its hierarchy). It is more readable this way - previous version was using `Clone`, which could be confusing regarding if bandwidth was shared or each node uses separate rate.
fixxxedpoint
force-pushed
the
A0-4318_rate_limited_substrate_network
branch
from
d12abfd
to
1fbf91b
Compare
timorleph
approved these changes
Nov 15, 2024
| ), | ||
| Dial = impl Send, | ||
| ListenerUpgrade = impl Send, | ||
| Error = impl Send, |
There was a problem hiding this comment.
I don't think there is anything wrong here, but the impl Send everywhere looks very intimidating. Perhaps a short comment explaining why it's necessary would be nice – I expect something like "We want this to be Send, but the only way to actually tell the compiler that is to insist that every template argument is Send.".
Actually, this sounds wrong. What does the compiler say if you only put the last Send in? Or are these impls needed for some other reason, e.g. later usage requiring sending (of _everything?)?
lesniak43
requested changes
Nov 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Implementation of a shared-bandwidth rate-limiter for both substrate-based and alephbft networks.
In a next PR I will introduce a new e2e-test that should help to verify these changes.
Type of change
Checklist: