docs: comments about consensus concurrency control

README.md

@@ -51,6 +51,13 @@ processing of some requests by immediately returning a future that will be
 executed on the caller's task. Although all requests are still received by
 the application task, not all request processing needs to happen on the
 application task.
+At this level the developer must pay closer attention to utilising Tower
+layers to control the concurrency of the individual services mentioned above.
+In particular the `Consensus` service should be wrapped with
+`ServiceBuilder::concurrency_limit` of 1 to avoid a potential reordering of
+consensus message effects caused by concurrent execution, as well as
+`ServiceBuilder::buffer` to avoid any deadlocks in message handling in `Connection`
+due to the limited concurrency.
 
 3. At the highest level of complexity, application developers can implement
 multiple distinct `Service`s and manually control synchronization of shared
@@ -65,4 +72,4 @@ services, etc.
 
 [ABCI]: https://docs.tendermint.com/master/spec/abci/
 [Tower]: https://docs.rs/tower
-[svc]: https://docs.rs/tower/0.4.6/tower/trait.Service.html
+[svc]: https://docs.rs/tower/0.4.6/tower/trait.Service.html

examples/kvstore_34/main.rs

@@ -158,6 +158,9 @@ async fn main() {
     // Hand those components to the ABCI server, but customize request behavior
     // for each category -- for instance, apply load-shedding only to mempool
     // and info requests, but not to consensus requests.
+    // Note that this example use synchronous execution in `Service::call`, wrapping the end result in a ready future.
+    // If `Service::call` did the actual request handling inside the `async` block as well, then the `consensus` service
+    // below should be wrapped with a `ServiceBuilder::concurrency_limit` to avoid any unintended reordering of message effects.
     let server_builder = Server::builder()
         .consensus(consensus)
         .snapshot(snapshot)

examples/kvstore_37/main.rs

@@ -165,6 +165,9 @@ async fn main() {
     // Hand those components to the ABCI server, but customize request behavior
     // for each category -- for instance, apply load-shedding only to mempool
     // and info requests, but not to consensus requests.
+    // Note that this example use synchronous execution in `Service::call`, wrapping the end result in a ready future.
+    // If `Service::call` did the actual request handling inside the `async` block as well, then the `consensus` service
+    // below should be wrapped with a `ServiceBuilder::concurrency_limit` to avoid any unintended reordering of message effects.
     let server_builder = Server::builder()
         .consensus(consensus)
         .snapshot(snapshot)

examples/kvstore_38/main.rs

@@ -207,6 +207,9 @@ async fn main() {
     // Hand those components to the ABCI server, but customize request behavior
     // for each category -- for instance, apply load-shedding only to mempool
     // and info requests, but not to consensus requests.
+    // Note that this example use synchronous execution in `Service::call`, wrapping the end result in a ready future.
+    // If `Service::call` did the actual request handling inside the `async` block as well, then the `consensus` service
+    // below should be wrapped with a `ServiceBuilder::concurrency_limit` to avoid any unintended reordering of message effects.
     let server_builder = Server::builder()
         .consensus(consensus)
         .snapshot(snapshot)

src/buffer4/service.rs

@@ -62,6 +62,12 @@ where
     /// [`poll_ready`] but will not issue a [`call`], which prevents other senders from issuing new
     /// requests.
     ///
+    /// # A note on the scope of `bound`
+    ///
+    /// Note that `bound` will only limit the rate of the _submission_ of [Message]s to the [Worker],
+    /// not their _execution_. If the execution itself is asynchronous, concurrency should be further
+    /// controlled by applying an appropriate [tower::Layer] on the returned service component.
+    ///
     /// [`Poll::Ready`]: std::task::Poll::Ready
     /// [`call`]: crate::Service::call
     /// [`poll_ready`]: crate::Service::poll_ready