feat: congestion control for catchunks #132

AnoldGH · 2025-02-21T02:01:11Z

Related to #97

Summary
This PR introduces congestion control mechanism for catchunks

Improvements

Commit 0e215a2 adds a general interface that supports multiple congestion window strategies, including fixed window, AIMD, and CUBIC
Commit 48afe7c refactors the existing codebase to add a retransmission queue for better congestion control

Pending Changes/Additions

TryStore as in ac07cca
~~Argument parsing for congestion window parameters~~ (need to change Client APIs)

Introduce "std/object/congestion" subpackage providing a unified interface for windowed congestion control and three window strategies: fixed, AIMD, and CUBIC Introduce a basic interface for RTT estimators

Copilot

Pull Request Overview

This PR introduces a congestion control mechanism for catchunks by adding multiple congestion window strategies and integrating a retransmission queue. Key changes include:

Adding a new RTTEstimator interface for RTT measurement.
Implementing three congestion window strategies: Fixed, AIMD, and CUBIC.
Refactoring the client consumer to leverage the new congestion control mechanism with retransmission queue support.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
std/object/congestion/rtt_estimator.go	Adds the RTTEstimator interface for RTT estimation.
std/object/congestion/congestion_window_fixed.go	Introduces a fixed congestion window implementation.
std/object/congestion/congestion_window_cubic.go	Implements the CUBIC congestion window strategy with updates.
std/object/congestion/congestion_window_aimd.go	Implements the AIMD congestion control mechanism.
std/object/congestion/congestion_window.go	Provides the generic CongestionWindow interface.
std/object/client_consume_seg.go	Refactors client consumption logic to use congestion control.

std/object/congestion/congestion_window_fixed.go

std/object/congestion/congestion_window_cubic.go

Co-authored-by: Copilot <[email protected]>

std/object/client_consume_seg.go

zjkmxy · 2025-04-29T05:05:07Z

std/object/congestion/congestion_window.go

+type CongestionWindow interface {
+	String() string
+
+	EventChannel() <-chan WindowEvent		// where window events are emitted


Is it used anywhere?

Is it used anywhere?

It's not used for now. I put the code there so that it is easier to build custom listeners.
By default (no channel specified) the event is passed to the logger by log.Debug

Then, in your code, this channel will always block and the window outputs every event to the log.

Yes, since we don't have any listeners at the moment. I feel it is reasonable to output to the log in this case so that we don't block the whole thing.

I don't think there would be Listeners at all.... Let's just put them to logging.

std/object/client_consume_seg.go

zjkmxy · 2025-04-29T05:07:23Z

std/object/congestion/rtt_estimator.go

+)
+
+// RTTEstimator provides an interface for estimating round-trip time.
+type RTTEstimator interface {


This interface has not been implemented I guess?

This interface has not been implemented I guess?

Yeah it hasn't... I remembered thinking if a TCP-like RTT estimator is enough due to the presence of cachings and multipath, etc. The CUBIC implementation will not enter the TCP-friendliness mode if an RTT estimator is not present.

AnoldGH · 2025-04-30T18:35:56Z

Oh I missed the congestion module

std/object/client_consume_state.go

zjkmxy

LGTM
I'm not familiar with the algorithms and it seems the RTT estimator is not done. Merge to unblock.

Copilot

Pull Request Overview

This PR introduces a congestion control mechanism for catchunks by adding an interface for multiple congestion window strategies and refactoring the client segment fetching logic to support retransmission.

Added new interfaces for RTT estimation and congestion window strategies (Fixed, AIMD, CUBIC).
Refactored client consumption state and segment fetching to use a structured fetching window and a retransmission queue.

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
std/object/congestion/rtt_estimator.go	New interface for round-trip time estimation.
std/object/congestion/congestion_window_fixed.go	Implements a fixed window congestion control strategy.
std/object/congestion/congestion_window_cubic.go	Implements CUBIC congestion control with custom update logic.
std/object/congestion/congestion_window_aimd.go	Implements AIMD congestion control strategy.
std/object/congestion/congestion_window.go	Defines the common CongestionWindow interface and associated signals.
std/object/client_consume_state.go	Refactors window tracking from raw indices to a FetchWindow structure.
std/object/client_consume_seg.go	Updates segment fetching logic with a retransmission queue and window-based flow control.
std/object/client_consume.go	Uses the new FetchWindow structure for client consumption state.

std/object/congestion/congestion_window_cubic.go

Copilot · 2025-05-01T05:17:37Z

std/object/client_consume_seg.go

+		log.Debug(nil, "Checking for work")
+
+		// check if the window is full
+		if s.IsCongested() {
+			log.Debug(nil, "Window full", "size", s.window.Size())


[nitpick] Consider passing a non-nil context or identifier to log.Debug for improved traceability instead of using nil.

Suggested change

log.Debug(nil, "Checking for work")

// check if the window is full

if s.IsCongested() {

log.Debug(nil, "Window full", "size", s.window.Size())

log.Debug(s.id, "Checking for work")

// check if the window is full

if s.IsCongested() {

log.Debug(s.id, "Window full", "size", s.window.Size())

Valid comment but I don't think it's that important.

AnoldGH force-pushed the congestion branch 2 times, most recently from bc237b7 to 48afe7c Compare February 28, 2025 23:56

AnoldGH added 2 commits February 28, 2025 16:14

feat: add congestion control subpackage

4e88721

Introduce "std/object/congestion" subpackage providing a unified interface for windowed congestion control and three window strategies: fixed, AIMD, and CUBIC Introduce a basic interface for RTT estimators

chunks: refactor to support a retransmission queue

e061112

AnoldGH force-pushed the congestion branch from 3bbffeb to e061112 Compare March 1, 2025 00:17

AnoldGH marked this pull request as ready for review March 1, 2025 00:18

zjkmxy added 2 commits March 1, 2025 07:32

Merge branch 'main' into congestion

4026c51

Merge branch 'main' into congestion

e504590

zjkmxy requested a review from Copilot April 16, 2025 07:00

Copilot AI reviewed Apr 16, 2025

View reviewed changes

std/object/congestion/congestion_window_fixed.go Outdated Show resolved Hide resolved

std/object/congestion/congestion_window_cubic.go Outdated Show resolved Hide resolved

AnoldGH and others added 2 commits April 16, 2025 22:48

Update std/object/congestion/congestion_window_fixed.go

5a1869c

Co-authored-by: Copilot <[email protected]>

Merge branch 'main' into congestion

e24bd7f

zjkmxy requested changes Apr 29, 2025

View reviewed changes

refactor ConsumerState.wnd into a FetchWindow struct

2a45ce7

AnoldGH force-pushed the congestion branch from 4490b1f to 2a45ce7 Compare April 30, 2025 06:39

AnoldGH added 2 commits April 29, 2025 23:54

use functions for shared lock-updates & remove unused variables

2f0140a

format using goimports

2c9150f

format congestion using goimports

5fc5576

zjkmxy reviewed Apr 30, 2025

View reviewed changes

std/object/client_consume_state.go Outdated Show resolved Hide resolved

AnoldGH added 2 commits April 30, 2025 21:02

remove unused definitions

d8b2238

remove unnecessary atomics

755d2f3

zjkmxy approved these changes May 1, 2025

View reviewed changes

zjkmxy requested a review from Copilot May 1, 2025 05:16

Copilot AI reviewed May 1, 2025

View reviewed changes

zjkmxy merged commit 232ed54 into named-data:main May 1, 2025
9 checks passed

feat: congestion control for catchunks #132

feat: congestion control for catchunks #132

Uh oh!

Conversation

AnoldGH commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zjkmxy Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

AnoldGH Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

zjkmxy Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

AnoldGH Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zjkmxy Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zjkmxy Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

AnoldGH Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

AnoldGH commented Apr 30, 2025

Uh oh!

Uh oh!

zjkmxy left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Copilot AI May 1, 2025

Choose a reason for hiding this comment

Uh oh!

zjkmxy May 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

AnoldGH commented Feb 21, 2025 •

edited

Loading

AnoldGH Apr 29, 2025 •

edited

Loading