Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: stream block updates from pool, use in builder #272

Merged
merged 1 commit into from
Sep 1, 2023

Conversation

dancoombs
Copy link
Collaborator

Closes #146

@dancoombs dancoombs force-pushed the danc/local-server branch 2 times, most recently from 4d08016 to 962879c Compare July 28, 2023 19:08
@dancoombs dancoombs force-pushed the danc/local-server branch 2 times, most recently from 9877ee0 to 1cde0bc Compare July 31, 2023 22:39
Base automatically changed from danc/local-server to feat/local-server August 1, 2023 00:00
@dancoombs dancoombs force-pushed the danc/pool-stream branch 2 times, most recently from 2500ca3 to 26b5a2f Compare August 1, 2023 14:39
@dancoombs dancoombs force-pushed the danc/pool-stream branch 2 times, most recently from 0b86bf4 to 4a479ec Compare August 1, 2023 15:00
@dancoombs dancoombs changed the title feat: stream block updates from pool, use in builder [WIP] feat: stream block updates from pool, use in builder Aug 23, 2023
@dancoombs dancoombs marked this pull request as draft August 23, 2023 01:27
@dancoombs dancoombs marked this pull request as ready for review August 23, 2023 13:49
@dancoombs dancoombs changed the title [WIP] feat: stream block updates from pool, use in builder feat: stream block updates from pool, use in builder Aug 23, 2023
Copy link
Collaborator

@0xfourzerofour 0xfourzerofour left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks, solid.

src/builder/bundle_sender.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@dphilipson dphilipson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Some sophisticated wiring in here. A few comments inline.

src/builder/bundle_sender.rs Outdated Show resolved Hide resolved
src/op_pool/mempool/mod.rs Outdated Show resolved Hide resolved
src/op_pool/server/remote/server.rs Show resolved Hide resolved
src/builder/bundle_sender.rs Outdated Show resolved Hide resolved
src/op_pool/server/local/server.rs Outdated Show resolved Hide resolved
src/op_pool/server/remote/client.rs Outdated Show resolved Hide resolved
@dancoombs dancoombs force-pushed the danc/pool-stream branch 4 times, most recently from f214510 to 8b58032 Compare September 1, 2023 01:12
@@ -41,20 +41,36 @@ pub async fn run(bundler_args: NodeCliArgs, common_args: CommonArgs) -> anyhow::
let builder_url = builder_args.url(false);

let (tx, rx) = mpsc::channel(1024);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if all the channels are the same size we should probably make this configurable and use a default channel size

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we have this problem all over the codebase. Definitely something we should clean up. I think we should just have a concept of an unbounded_channel that we use in most places.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeh I think we could clean up the thread messaging a bit by using some type of event passing structure that can handle easy iterations of the codebase.

I am going to play around with it over the weekend and see what I can come up with.

pub struct Engine<E, A> {
    /// The set of collectors that the engine will use to collect events.
    collectors: Vec<Box<dyn Collector<E>>>,

    /// The set of executors that the engine will use to execute actions.
    executors: Vec<Box<dyn Executor<A>>>,

    /// The capacity of the event channel.
    event_channel_capacity: usize,

    /// The capacity of the action channel.
    action_channel_capacity: usize,
}

Something like this then we can have a collector trait that listens for new block events and exectutors that can process the events and bundle the user ops.

will most likely take a while to get right but will let you know how I go

src/op_pool/server/remote/server.rs Show resolved Hide resolved
Copy link
Collaborator

@dphilipson dphilipson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I don't have any specific ideas on how to improve it at the moment- the stream abstraction sounds pretty reasonable. Thanks for putting so much thought into this 🔥 !

src/op_pool/server/remote/client.rs Outdated Show resolved Hide resolved
@dancoombs dancoombs merged commit 781543c into feat/local-server Sep 1, 2023
5 checks passed
@dancoombs dancoombs deleted the danc/pool-stream branch September 1, 2023 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants