Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(congestion control): add CongestionController trait and example impl #907

Merged
merged 1 commit into from
Mar 11, 2024

Conversation

ytakano
Copy link
Contributor

@ytakano ytakano commented Feb 18, 2024

Add CongestionController trait for congestion controllers of TCP.

By default, this PR uses NoControl, which do not control congestion,
and the PR does not affect conventional behavior.

To use Cubic or Reno congestion controllers,
socket-tcp-cubic or socket-tcp-reno features must be set or
tcp::Socket::new_with_cc() must be used.

Users can implement custom congestion controllers
by using CongestionController trait and tcp::Socket::new_with_cc().

Cubic or Reno can be tested as follows.

$ cargo test --features socket-tcp-cubic
$ cargo test --features socket-tcp-reno

@ytakano ytakano changed the title feat(congestion control): add CongestionControler tairt and example impl feat(congestion control): add CongestionController tairt and example impl Feb 18, 2024
Copy link

codecov bot commented Feb 18, 2024

Codecov Report

Attention: Patch coverage is 98.41270% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 79.96%. Comparing base (f0d1fd9) to head (833399f).
Report is 4 commits behind head on main.

Files Patch % Lines
src/socket/tcp/congestion.rs 97.67% 1 Missing ⚠️
src/socket/tcp/congestion/no_control.rs 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #907      +/-   ##
==========================================
+ Coverage   79.92%   79.96%   +0.04%     
==========================================
  Files          80       82       +2     
  Lines       28234    28378     +144     
==========================================
+ Hits        22566    22693     +127     
- Misses       5668     5685      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ytakano ytakano marked this pull request as draft February 18, 2024 11:24
@ytakano
Copy link
Contributor Author

ytakano commented Feb 18, 2024

Avoid cbrt() and powi() for no_std.

@pothos
Copy link
Contributor

pothos commented Feb 18, 2024

Just for your reference, there was also #450 and https://github.com/cbranch/smoltcp/commits/cbranch/cc (other non-upstreamed things are also public: https://github.com/cbranch/smoltcp/branches/stale, and if you are looking into BBR there is https://github.com/quinn-rs/quinn/tree/0.8.0)

@ytakano
Copy link
Contributor Author

ytakano commented Feb 18, 2024

When I was testing smoltcp, it occupied traffic bandwidth, and other TCP connections starved due to the lack of congestion control.

By the way, I think delay based congestion control mechanisms like BBR should be also implemented by using CongestionController,
because it can know time and bytes of sending/receiving data
and RTT by using following methods.

pub trait CongestionController: Default {
    #[allow(unused_variables)]
    fn on_ack(&mut self, now: Instant, len: usize, rtt: &RttEstimator) {}

    #[allow(unused_variables)]
    fn post_transmit(&mut self, now: Instant, len: usize) {}
}

This is similar to quinn.
https://github.com/quinn-rs/quinn/blob/0.8.0/quinn-proto/src/congestion.rs

I know theory of BBR, but I cannot evaluate it sufficiently because it requires emulation of delays.

@ytakano ytakano marked this pull request as ready for review February 18, 2024 14:21
src/socket/tcp.rs Outdated Show resolved Hide resolved
src/socket/tcp.rs Outdated Show resolved Hide resolved
src/socket/tcp.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@whitequark whitequark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we shouldn't add this with non-additive features. I think it's OK to have an enum of all possible congestion control methods, with some sensible default selection, but still allowing two crates that both use smoltcp to compile without needing to coordinate their feature sets.

@ytakano
Copy link
Contributor Author

ytakano commented Feb 21, 2024

I think we shouldn't add this with non-additive features. I think it's OK to have an enum of all possible congestion control methods, with some sensible default selection, but still allowing two crates that both use smoltcp to compile without needing to coordinate their feature sets.

Right.
A general enum type of congestion control can be prepared.
I implemented it before, but discarded.
I will fix it later.

Thank you for your review.

@whitequark
Copy link
Contributor

To expand a bit, I'm thinking that the enum could include all of the necessary state, therefore making tcp::Socket as much bigger as the biggest possible to choose congestion controller (0 bytes if none are enabled).

The way to create it would probably be methods on the CongestionController enum, whose fields are all private. Then a Default implementation would pick the "best" one available if several are enabled. (The decision on which congestion controller is the "best" one is probably contentious but I don't expect it to matter that much since most people will pick one.)

Once this PR is in, we should strongly recommend picking a congestion controller in the README, to be a good netizen.

@ytakano
Copy link
Contributor Author

ytakano commented Feb 21, 2024

I have defined set_congestion_control() and get_congestion_control() methods for tcp::Socket as follows.

pub enum CongestionControl {
    None,
    Cubic,
    Reno,
}

impl tcp::Scoket {
    /// Set an algorithm for congestion control.
    ///
    /// The default setting is `CongestionControl::None`, indicating that no congestion control is applied.
    /// Options `CongestionControl::Cubic` and `CongestionControl::Reno` are also available.
    ///
    /// `CongestionControl::Cubic` represents a modern congestion control algorithm designed to
    /// be more efficient and fair compared to `CongestionControl::Reno`.
    /// It is the default choice for Linux, Windows, and macOS.
    ///
    /// `CongestionControl::Reno` is a classic congestion control algorithm valued for its simplicity.
    /// Despite having a lower algorithmic complexity than `Cubic`,
    /// it is less efficient in terms of bandwidth usage.
    pub fn set_congestion_control(&mut self, congestion_control: CongestionControl) { /* omitted */ }

    /// Return the current congestion control algorithm.
    pub fn get_congestion_control(&self) -> CongestionControl { /* omitted */ }
}

CongestionController enum, whose fields are all private.

Variants of a public enum cannot be private.
So, I have defined these 2 methods and CongestionControl enum to tcp::Socket.

If the best performance is required, I think it should be a type parameter of tcp::Socket.

pub struct Socket<'a, CC: CongestionController> {
    congestion_controller: CC
}

However, it is less compatible.
See #907 (comment)

If it is a type parameter, the required bytes of the empty congestion controller must be 0 as follows.

let socket = tcp::Socket<EmptyCongestionController>();

@whitequark
Copy link
Contributor

Yeah it's not actually possible to use a type parameter here. A enum is perfectly fine.

src/socket/tcp.rs Outdated Show resolved Hide resolved
src/socket/tcp.rs Outdated Show resolved Hide resolved
src/socket/tcp.rs Outdated Show resolved Hide resolved
src/socket/tcp.rs Outdated Show resolved Hide resolved
src/socket/tcp/congestion_controller.rs Outdated Show resolved Hide resolved
src/socket/tcp/congestion_controller.rs Outdated Show resolved Hide resolved
src/socket/tcp/congestion_controller/cubic.rs Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
src/socket/tcp/congestion.rs Outdated Show resolved Hide resolved
src/socket/tcp.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@whitequark whitequark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, I am happy with the functionality now, and I only have a few comments about naming.

src/socket/tcp/congestion.rs Show resolved Hide resolved
src/socket/tcp/congestion.rs Outdated Show resolved Hide resolved
src/socket/tcp/congestion.rs Outdated Show resolved Hide resolved
@ytakano
Copy link
Contributor Author

ytakano commented Feb 28, 2024

Now we can test Cubic and Reno by specifying the features as follows.

$ cargo test --features=socket-tcp-reno
$ cargo test --features=socket-tcp-cubic

Copy link
Contributor

@whitequark whitequark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for this excellent work!

@Dirbaio Once this is in, I think we ought to consider recommending enabling a congestion controller in the docs, to be a good netizen.

@whitequark whitequark changed the title feat(congestion control): add CongestionController tairt and example impl feat(congestion control): add CongestionController trait and example impl Feb 28, 2024
@whitequark
Copy link
Contributor

Can you squash everything into one commit please?

@ytakano
Copy link
Contributor Author

ytakano commented Mar 11, 2024

Squashed.

@whitequark
Copy link
Contributor

I think this includes some unrelated commits?

- add CongestionController trait
- implement Cubic and NoControl

Signed-off-by: Yuuki Takano <[email protected]>

feat(socket/tcp): add congestion_controller.rs

Signed-off-by: Yuuki Takano <[email protected]>

feat(cubic): add test for TCP Cubic

Signed-off-by: Yuuki Takano <[email protected]>

feat(cubic): do not update congestion as much as possible

If the last update time of the cwnd was less than 100ms ago,
don't update the congestion window.

Signed-off-by: Yuuki Takano <[email protected]>

feat(cubic): implement slow start for TCP Cubic

Signed-off-by: Yuuki Takano <[email protected]>

fix(cubic): use MSS as the minimum cwnd

Signed-off-by: Yuuki Takano <[email protected]>

feat(tcp): add "socket-tcp-cubic" and "socket-tcp-reno" features

- "socket-tcp-cubic": use Cubic as a default congestion controller
- "socket-tcp-reno": use Reno as a default congestion controller

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): prepare new() for Socket<'a, DefaultCC>

All tests are passed because of this fix.

$ cargo test
$ cargo test --features socket-tcp-cubic

Signed-off-by: Yuuki Takano <[email protected]>

feat(tcp, reno): add Reno of a congestion controller

Signed-off-by: Yuuki Takano <[email protected]>

feat(cubic): do not use cbrt() and powi() for no_std

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): update for coverage test

Signed-off-by: Yuuki Takano <[email protected]>

fix(cubic): do not use `abs()` for no_std

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): make some types regading congestion control private

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): make `CongestionController` `pub(super)`

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): make modules of congestion controllers private

- `cubic`, `reno` and `no_control` modules are now private.
- `ActiveCC` is now `pub(super)`

Signed-off-by: Yuuki Takano <[email protected]>

fix: typo

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): use `defmt::Format` if `defmt` feature is specified

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): introduce `GenericController` for congestion control

Remove `socket-tcp-cubic` and `socket-tcp-reno` features
for congestion control.
Instead of the features, `set_congestion_control()`
is defined to set an algorithm of congestion control.

Signed-off-by: Yuuki Takano <[email protected]>

feat(tcp): add a test for `(set|get)_congestion_control()`

Signed-off-by: Yuuki Takano <[email protected]>

fix: restore unessesory change

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): rename congestion_controller to congestion

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): update the document about `Cubic`

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): do not use `Default` trait for congestion controller

Because `Default` trait requires `Size`,
returning `&dyn Trait` is rejected by the Rust compiler.

Signed-off-by: Yuuki Takano <[email protected]>

chore(tcp): `#[allow(unused_variables)]` is moved on the trait

Signed-off-by: Yuuki Takano <[email protected]>

feat(tcp): add `socket-tcp-cubic` feature for Cubic

Cubic uses f64 and it is inefficient on some platforms like STM32.
So, Cubic is disabled by default.

Signed-off-by: Yuuki Takano <[email protected]>

fix: typo

Signed-off-by: Yuuki Takano <[email protected]>

Update src/socket/tcp.rs

Co-authored-by: Catherine <[email protected]>

feat(tcp): add `socket-tcp-reno` feature for Reno

Signed-off-by: Yuuki Takano <[email protected]>

feat(tcp): use GenericController::inner_*() methods

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): choose the best congestion controller

If Cubic is enabled, it is used as a default controller.
If Reno is enabled, it is used as a default controller.

If Cubic and Reno are eanabled, Cubic is a default controlerr,
but it can be specified by `set_congestion_control()` method.

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): rename CongestionController to Controller

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): rename GenericController to AnyController

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): add a document to `AnyController::new()`

Signed-off-by: Yuuki Takano <[email protected]>

fix(tcp): use Controller instead of CongestionController

CongestionController has been renamed.

Signed-off-by: Yuuki Takano <[email protected]>
@ytakano
Copy link
Contributor Author

ytakano commented Mar 11, 2024

Sorry. How about this?

@whitequark whitequark added this pull request to the merge queue Mar 11, 2024
@whitequark
Copy link
Contributor

Thanks!

@whitequark
Copy link
Contributor

@Dirbaio Once this is merged I think we should strongly recommend that Reno be selected unless there is an extremely good reason not to use any congestion control.

Merged via the queue into smoltcp-rs:main with commit 4c27918 Mar 11, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants