From e2c5022a161eb4b6dd5f3493f7a0cecc86e83325 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 09:43:40 +0100 Subject: [PATCH 01/24] modify prerequisites --- NewExercises/agenda-2-day.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/NewExercises/agenda-2-day.md b/NewExercises/agenda-2-day.md index f00ea13..97cc2c7 100644 --- a/NewExercises/agenda-2-day.md +++ b/NewExercises/agenda-2-day.md @@ -1,8 +1,8 @@ ### Prerequisites - * VS 2019 and .net472 - * Visual Studio Code and TLA+ extension - * NServiceBus 7 + * VS 2019 or later + * netcoreapp3.1 * CosmosDB Emulator + * Visual Studio Code and TLA+ extension ### Agenda From b0e818963bbdc0cacd5e8418ba0be80fe0cd9ac5 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 09:56:36 +0100 Subject: [PATCH 02/24] editorial --- NewExercises/Exercise-1/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/NewExercises/Exercise-1/README.md b/NewExercises/Exercise-1/README.md index 07ee233..02a4e2d 100644 --- a/NewExercises/Exercise-1/README.md +++ b/NewExercises/Exercise-1/README.md @@ -2,12 +2,12 @@ Let's start simple. The solution contains a very simple "e-commerce" web application that sells pierogi. -Users have to log in by providing their name. Then, they can place orders by adding items to a shopping cart and finally submitting the shopping cart for processing. +Users must log in by providing their name. Once they are authenticated, they can place items in their shopping cart. Once their shopping cart is ready, they can submit the order for processing. -The domain model of the application consists of two aggregates: `ShoppingCart` and `Order`. The shopping cart is created immediately when a user navigates to the shopping page. Items are added to the cart. Finally, when the user decides to submit the order, the `Order` aggregate is created based on the contents of the `ShoppingCart` aggregate. +The domain model of the application consists of two [aggregates](https://martinfowler.com/bliki/DDD_Aggregate.html): `ShoppingCart` and `Order`. The shopping cart is immediately created when a user navigates to the shopping page. Items are added to the cart. Finally, when the user decides to submit the order, the `Order` aggregate is created based on the contents of the `ShoppingCart` aggregate. Order processing happens asynchronously via a batch job but is not shown in this simplistic example. -Try placing few orders to get the feeling of the application. Maybe put some breakpoints and see the flow. +Try placing few orders to get a sense of how the application works. Consider placing some breakpoints, or even breaking the application, to observe the flow. -Perhaps try to break the application? What happens if you open the same order in two browser tabs and click `Submit` on both pages? Can you explain the behavior? \ No newline at end of file +Consider what happens if you open the same shopping cart in two browser tabs and click `Submit` on both pages? Can you explain the behavior? \ No newline at end of file From 0c392b010605ac76c99b6e6f1ef133fb9bea4a50 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 10:10:42 +0100 Subject: [PATCH 03/24] editorial --- NewExercises/Exercise-2/README.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/NewExercises/Exercise-2/README.md b/NewExercises/Exercise-2/README.md index abf3bc4..8c5332d 100644 --- a/NewExercises/Exercise-2/README.md +++ b/NewExercises/Exercise-2/README.md @@ -1,14 +1,15 @@ # Exercise 2 - State-based deduplication -The code in the previous exercise was very naive as I did not guard against attempts at duplicate submission. Such guard can be seens as basic business validation +The code in the previous exercise was very naive as it did not prevent duplicate submissions. Such a guard can be seen as a basic business rule: > Order can't be submitted more than once -but it also can be seen as the most basic form of request deduplication. Regardless how we label it, we need to add code that protects our shopping carts from double submission. +This business rule also represents the most basic form of request deduplication. Regardless of how we label it, we need to add code that protects our shopping carts from duplicate submissions. - Add a boolean flag (property) `Submitted` to the `ShoppingCart` class. -- In the `ApplicationServices` class modify the `SubmitOrder` method to check if the cart is not submitted before submitting it. If so, throw an exception. +- In the `ApplicationServices`-class, modify the `SubmitOrder` method to check if the cart wasn't already submitted. If so, throw an exception. - If not, set that property to `true`. -- In the `repository.Put` call pass the cart as another parameter to ensure the change in the submitted flag is persisted. This method uses Cosmos DB's `TransactionalBatch` feature to make sure that modifications to all passed objects are done atomically. +- In the `repository.Put`-method, pass the cart as a parameter to ensure the submitted flag is persisted. This method uses [Cosmos DB's `TransactionalBatch` feature](https://docs.microsoft.com/en-us/azure/cosmos-db/sql/transactional-batch) to ensure that modifications to all passed objects are persisted [atomically](https://en.wikipedia.org/wiki/Atomicity_(database_systems)). -Did that help? We hope so. Now let's try to break the application again. How about adding a simulated pause (e.g. via `await Task.Delay(3000)`) right before the `repository.Put` call? Can you try the double submission hack using two browser tabs. \ No newline at end of file +This helped to mitigate the behavior we perceived previously. +Next, try adding a simulated pause (e.g. via `await Task.Delay(3000)`) right before the `repository.Put` call. Now, try the duplicate order submission scenario using two browser tabs. What behavior do you observe? \ No newline at end of file From 9239251c18fde0f32c1ef3015b6d025bd98c66cf Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 10:17:35 +0100 Subject: [PATCH 04/24] hint at what we're trying to prove --- NewExercises/Exercise-2/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/NewExercises/Exercise-2/README.md b/NewExercises/Exercise-2/README.md index 8c5332d..5ddd464 100644 --- a/NewExercises/Exercise-2/README.md +++ b/NewExercises/Exercise-2/README.md @@ -12,4 +12,4 @@ This business rule also represents the most basic form of request deduplication. - In the `repository.Put`-method, pass the cart as a parameter to ensure the submitted flag is persisted. This method uses [Cosmos DB's `TransactionalBatch` feature](https://docs.microsoft.com/en-us/azure/cosmos-db/sql/transactional-batch) to ensure that modifications to all passed objects are persisted [atomically](https://en.wikipedia.org/wiki/Atomicity_(database_systems)). This helped to mitigate the behavior we perceived previously. -Next, try adding a simulated pause (e.g. via `await Task.Delay(3000)`) right before the `repository.Put` call. Now, try the duplicate order submission scenario using two browser tabs. What behavior do you observe? \ No newline at end of file +Next, try adding a simulated pause (e.g. via `await Task.Delay(3000)`) right before the `repository.Put` call, to help us simulate concurrent submissions. Now, try the duplicate order submission scenario using two browser tabs. What behavior do you observe? \ No newline at end of file From 7fedaa1a48352770c41d9e015ade9a59cb4c8da3 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 10:29:59 +0100 Subject: [PATCH 05/24] editorial --- NewExercises/Exercise-2/README.md | 2 +- NewExercises/Exercise-3/README.md | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/NewExercises/Exercise-2/README.md b/NewExercises/Exercise-2/README.md index 5ddd464..542423b 100644 --- a/NewExercises/Exercise-2/README.md +++ b/NewExercises/Exercise-2/README.md @@ -11,5 +11,5 @@ This business rule also represents the most basic form of request deduplication. - If not, set that property to `true`. - In the `repository.Put`-method, pass the cart as a parameter to ensure the submitted flag is persisted. This method uses [Cosmos DB's `TransactionalBatch` feature](https://docs.microsoft.com/en-us/azure/cosmos-db/sql/transactional-batch) to ensure that modifications to all passed objects are persisted [atomically](https://en.wikipedia.org/wiki/Atomicity_(database_systems)). -This helped to mitigate the behavior we perceived previously. +This helped mitigate the behavior we perceived previously. Next, try adding a simulated pause (e.g. via `await Task.Delay(3000)`) right before the `repository.Put` call, to help us simulate concurrent submissions. Now, try the duplicate order submission scenario using two browser tabs. What behavior do you observe? \ No newline at end of file diff --git a/NewExercises/Exercise-3/README.md b/NewExercises/Exercise-3/README.md index e83cf8f..2b12d3c 100644 --- a/NewExercises/Exercise-3/README.md +++ b/NewExercises/Exercise-3/README.md @@ -1,14 +1,14 @@ # Exercise 3 - Optimistic concurrency -What you have seen in the previous exercise is the effect of lack of concurrency control in our application. Concurrency control is the basis of all other, more advanced, consistency control techniques that we will encounter during the workshops. +What we've observed in the previous exercise is the effect of a lack of [concurrency control](https://en.wikipedia.org/wiki/Concurrency_control) in our application. Concurrency control is the basis of all consistency control techniques that we will encounter during the workshop. -Consistency control is needed because the modification of the shopping cart is based on a *view* (or *snapshot*) of that cart that has been obtained sometime in the past. The shopping cart might have changed between the time when the *snapshot* was taken and the time when the changes are to be applied. +The modification of the shopping cart is based on a *view* (or *snapshot*) that was obtained sometime in the past. The shopping cart might have changed between the time when the *snapshot* was taken and the time when the changes submitted, which is why consistency control is required. -Concurrency control is essential in any distributed system in which modifications to the data are computed in a different process from the one that keeps the data. This pretty much means *any* modern application unless it is based purely on stored procedures. And even in that case, depending on the transaction isolation level, concurrency control might be necessary. +Concurrency control is essential in any distributed system in which data modifications are applied in a different process from the one that keeps the data. This applies to *any* modern application unless it is purely based on stored procedures. Even in the latter case, depending on the transaction isolation level, concurrency control might be necessary. -Let's add optimistic concurrency control - implemented in CosmosDB via version numbers (`ETag`) passed to the update request. If the version is included, the update succeeds only if the version of the document in the database matches the version included in the request. If not then the whole `TransactionalBatch` is rejected. +Let's add optimistic concurrency control - implemented in CosmosDB through version numbers (`ETag`) passed to the put request. If the version is included, the update will only succeed when the version of the document in the database matches the version included in the request. If not, the whole `TransactionalBatch` is rejected. -- In the `ApplicationServices` class modify the `SubmitOrder` method to include the version properties in the `repository.Put` call +- In the `ApplicationServices` class, modify the `SubmitOrder` method to include the version properties in the `repository.Put` call - the version of the cart is already available as the `version` variable - pass `null` as the version of the order (can you explain why?) - hint: the concurrency-friendly `Put` API expects a collection of `(Entity, string)` tuples From 24487507bb26ca86df80a895d1962a512eb2795c Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 13:27:35 +0100 Subject: [PATCH 06/24] editorial --- NewExercises/Exercise-4/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/NewExercises/Exercise-4/README.md b/NewExercises/Exercise-4/README.md index c3ad574..bbb210f 100644 --- a/NewExercises/Exercise-4/README.md +++ b/NewExercises/Exercise-4/README.md @@ -1,17 +1,17 @@ # Exercise 4 - Make it asynchronous -Up until now we were dealing with a simple web application backed up by a single data store (Cosmos DB). Let's pretend now that our pierogi-selling application achieved great market success and we need to handle many more customers. One common way of adjusting the application for scalability needs is making part of the processing asynchronous. This is where messaging becomes really hepful. +Up until now, we were dealing with a simple web application using a single data store (Cosmos DB). In the meantime, our pierogi-selling application has been overwhelmed with sales and we need to cater for more customers. A common way to adjust applications for scalability purposes is to defer processing and have this handled asynchronously by a dedicated process. This is where you want to start making use of messaging. -In this exercise we are going to move the logic responsible for creating the `Order` aggregate from the web application to a backend service. Both components are going to communicate using durable asynchronous message queue. +In this exercise we are going to move the logic responsible for creating the `Order` aggregate from the web application to a dedicated backend service. Both components are going to communicate using a durable message queue. Let's get our hands dirty. - Move the `Order` class from the `WebFrontend` project to the `Orders` project. -- In the `ApplicationServices` class change the type used by the `GetOrders` method from `Order` to `ShoppingCart`. From now on this method will only list shopping carts. Change the name to `GetShoppingCarts`. -- In the Orders project find the `SubmitOrderHandler` class. Notice it implements `IHandleMessages` in order to tell the endpoint that it can handle messages of type `SubmitOrder`. +- In the `ApplicationServices` class, change the type used by the `GetOrders` method from `Order` to `ShoppingCart`. From now on this method will only list shopping carts. Rename the method to `GetShoppingCarts`. +- In the Orders project, find the `SubmitOrderHandler` class. Notice it implements `IHandleMessages` in order to tell the endpoint that it can handle messages of type `SubmitOrder`. - Move the code that creates and saves the order from the `SubmitOrder` method to the `Handle` method of `SubmitOrderHandler`. - You can now remove the `Task.Delay`. We won't need it any more. - Consider logging something at the end of the `Handle` method e.g. `log.Info("Order submitted "+ order.Id);`. - Remember that in the `SubmitOrder` method you still need to save the cart after the flag is is set. -- In the `SubmitOrder` method, after the call to `repository.Put` to save the state of the cart, add the code to send the `SubmitOrder` message. Use the `session` field of type `IMessageSession` and its `Send` method. Set the properties of the `SubmitOrder` message based on the shopping cart. +- In the `SubmitOrder` method, after the call to `repository.Put` to save the cart, add code to send the `SubmitOrder` message. Use the `session` field of type `IMessageSession` and its `Send` method. Set the properties of the `SubmitOrder` message based on the shopping cart. - Run the application and check if you can submit an order. Make sure that both `WebFrontend` and `Orders` are selected as start projects. From c4a76e136cb351a054f4f1bd8ed00c03d6c025db Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 13:35:22 +0100 Subject: [PATCH 07/24] editorial --- NewExercises/Exercise-5/README.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/NewExercises/Exercise-5/README.md b/NewExercises/Exercise-5/README.md index b7d48bb..f8a178c 100644 --- a/NewExercises/Exercise-5/README.md +++ b/NewExercises/Exercise-5/README.md @@ -1,11 +1,13 @@ -# Exercide 5 - Monkeys of Chaos +# Exercise 5 - Monkeys of Chaos -In this exercise we are going to use the principles of [chaos engineering](https://en.wikipedia.org/wiki/Chaos_engineering) to ensure our system is robust. Instead of testing the behavior of our system system in different failure modes based on pure chance, we are going to build a *chaos monkey* -- a piece of code that is going to introduce a certain category of anomalies into our system **on purpose**. This will allow our code to cope with such anomalies better when they occur in production (according to Murphy's laws it is going to happen on Friday afternoon just before your long-planned vacation). +In this exercise we are going to use the principles of [chaos engineering](https://en.wikipedia.org/wiki/Chaos_engineering) to ensure our system is robust. Instead of testing the behavior of our system in different failure modes based on pure chance, we are going to build a *chaos monkey* -- a piece of code that is going to introduce a certain series of anomalies into our system **on purpose**. This will allow improve the system's ability to cope with such anomalies when they occur in production (according to [Murphy's law](https://en.wikipedia.org/wiki/Murphy%27s_law), this will happen on Friday afternoon just before your long-planned vacation). -In NServiceBus, the appropriate extension point for this task is the `Behavior` class. NServiceBus handles incoming and outgoing messages by passing them through processing pipelines composed of `Behaviors`. Each behavior can execute arbitrary code and pass invocation to behaviors further down the pipeline. Here's an example behavior: +In NServiceBus, the [appropriate extension point](https://docs.particular.net/nservicebus/pipeline/manipulate-with-behaviors) for these types of concerns, is the `Behavior` class. NServiceBus handles incoming and outgoing messages by passing them through [processing pipelines](https://docs.particular.net/nservicebus/pipeline/) composed of `Behaviors`. Each behavior can execute arbitrary code and passes the invocation to behaviors further down the pipeline. + +Here's an example behavior: ``` -class MyBehavior : Behavior //T defines in which part of the pipeline the behavior is injected +class MyBehavior : Behavior // T defines in which part of the pipeline the behavior is injected { public override async Task Invoke(IOutgoingLogicalMessageContext context, Func next) { @@ -20,12 +22,12 @@ class MyBehavior : Behavior //T defines in which } ``` -Create a behavior in the outgoing pipeline that duplicates the send invocation +Create a behavior in the outgoing pipeline that duplicates the send invocation: - In the `WebFrontend` project create a new class derived from `Behavior` -- In the `Invoke` method call `await next()` to create a behavior that does nothing but just forwards the invocation -- In the `Program` class of `WebFrontend` project register the behavior with `EndpointConfiguration` via `Pipeline.Register` API (e.g. after the call to `endpointConfiguration.SendOnly()`) +- In the `Invoke` method, call `await next()` to create a behavior that does nothing but just forwards the invocation +- In the `Program` class of `WebFrontend` project, register the behavior with `EndpointConfiguration` via `Pipeline.Register` API (e.g. after the call to `endpointConfiguration.SendOnly()`) - Run the solution to check if messages continue to flow. Put a breakpoint in the new behavior to verify that it is invoked -- In the behavior class add an instance field `failed` to ensure that only the first message triggers the failure -- Before the `next()` call add code in the behavior that checks if `failed` flag is not set. If it is not, set the flag to `true` and throw new `Exception`. This will ensure that the first attempt to send a message after the web application is started is always going to fail. +- In the behavior class, add an instance field `failed` to ensure that only the first message triggers the failure +- Before the `next()` call, add code that checks whether the `failed` flag is set. If it is not, set the flag to `true` and throw a new `Exception`. This will ensure that the first attempt to send a message after the web application is started is always going to fail. -Now try placing the order and see what happens. Has the backend received the message? What does the customer think about their pierogi order? +Now try placing the order and observe what happens. Has the backend received the message? What does the customer believe happened to their pierogi order? From b10ae07d15b9f03dab36bfeafb21286a322e37c5 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 13:45:23 +0100 Subject: [PATCH 08/24] editorial --- NewExercises/Exercise-5/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/NewExercises/Exercise-5/README.md b/NewExercises/Exercise-5/README.md index f8a178c..d9a6a7c 100644 --- a/NewExercises/Exercise-5/README.md +++ b/NewExercises/Exercise-5/README.md @@ -23,11 +23,11 @@ class MyBehavior : Behavior // T defines in whic ``` Create a behavior in the outgoing pipeline that duplicates the send invocation: -- In the `WebFrontend` project create a new class derived from `Behavior` -- In the `Invoke` method, call `await next()` to create a behavior that does nothing but just forwards the invocation -- In the `Program` class of `WebFrontend` project, register the behavior with `EndpointConfiguration` via `Pipeline.Register` API (e.g. after the call to `endpointConfiguration.SendOnly()`) -- Run the solution to check if messages continue to flow. Put a breakpoint in the new behavior to verify that it is invoked -- In the behavior class, add an instance field `failed` to ensure that only the first message triggers the failure +- In the `WebFrontend` project create a new class derived from `Behavior` and implement its members +- In the `Invoke` method, call `await next()` to create an empty behavior that just forwards the invocation +- In the `Program` class of the `WebFrontend` project, register the behavior on the `endpointConfiguration`-variable using the `Pipeline.Register`-API (e.g. after the call to `endpointConfiguration.SendOnly()`) +- Run the solution to check if messages continue to flow. Place a breakpoint in the new behavior to verify that it is invoked +- In the behavior class, add an instance field named `failed` to ensure that only the first message triggers the failure - Before the `next()` call, add code that checks whether the `failed` flag is set. If it is not, set the flag to `true` and throw a new `Exception`. This will ensure that the first attempt to send a message after the web application is started is always going to fail. Now try placing the order and observe what happens. Has the backend received the message? What does the customer believe happened to their pierogi order? From ba973ad52d73065b4191993c2312dc555766740e Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 14:14:18 +0100 Subject: [PATCH 09/24] editorial --- NewExercises/Exercise-6/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/NewExercises/Exercise-6/README.md b/NewExercises/Exercise-6/README.md index 251b100..4c42ace 100644 --- a/NewExercises/Exercise-6/README.md +++ b/NewExercises/Exercise-6/README.md @@ -1,11 +1,11 @@ # Exercise 6 - The USB Rule -It is a well-known fact that the classis USB plug never fits the first time. You *always* have to turn it around but it does not fit either. Then you turn it around again and it fits. Scientists are still working on explaining that phenomenon ;-) +It is a well-known fact that the classic USB plug never fits the first time. You *always* end up having to turn it around only to conclude that it still doesn't fit. Third time's a charm, so when you turn it around again, somehow it fits. Scientists are still working on explaining that phenomenon ;-) -Back to being serious, what we have observed in the previous exercise was a lost message. Because we first update the state and then send a message, if the latter fails, the state is already modified. The customer sees their shopping cart as submitted even though the submission message never went out. What can we do? +On a serious note, what we have observed in the previous exercise was a lost message, or what we also call a **phantom record**. We updated the state first and then sent the message. If the latter fails, the state is already modified. At that point, the view shows a submitted shopping cart even though the submission message never went out. Therefore, the submitted cart represents a phantom record, because the according message was lost. What can we do? -We can use the USB principle and try to turn the order of operations upside-down. If we fist try to send the message and then update the state, we will make sure that if the send fails, the customer will see the correct state of the shopping cart and hopefully will retry the submission. +We can use the USB principle and try to switch the order of the operations. If we first try to send the message and then update the state, we ensure that if the send fails, the customer will continue to see an unsubmitted shopping cart, at which point they will hopefully try to submit the order again. -- Go to the `ApplicationServices` class and the `SubmitOrder` method and reverse the order of `repository.Put` and `session.Send`. This should make sure that the state of the cart remains not `Submitted` if the message sending failed. +- In the `ApplicationServices` class, change the `SubmitOrder` method to reverse the order of `repository.Put` and `session.Send`. This should ensure that the cart never appears as `Submitted` if the sending the message fails. - Check if the system can handle the simulated broker failures correctly. - Now go on add add few (4-5) more orders. Let's see what happens. \ No newline at end of file From ee04602c9f3d9066897ef1bce677b598c7e30a75 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 14:28:28 +0100 Subject: [PATCH 10/24] editorial --- NewExercises/Exercise-7/README.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/NewExercises/Exercise-7/README.md b/NewExercises/Exercise-7/README.md index 20dc53f..86d4fce 100644 --- a/NewExercises/Exercise-7/README.md +++ b/NewExercises/Exercise-7/README.md @@ -1,22 +1,21 @@ # Exercise 7 - If in doubt, try again -What you have seen in the previous exercise were ghost messages. These messages carry information about state changes that have not been persisted. Ghost messages are as bad as missing messages because they can easily turn a system that was supposed to be eventually consistent into an immediately inconsistent one. We need to solve this problem. +What you have observed in the previous exercise were **ghost messages**. These messages carry information about state changes that were never applied. Ghost messages are equally harmful as phantom records. They can easily turn a system that was supposed to be eventually consistent into an immediately inconsistent one. We need to solve this problem. -The solution is to go back to the previous save-fist, send-later approach (remember the USB principle?) but this time allow re-sending the submission message. To make sure it is safe to do it we'll introduce another state of the shopping cart -- `Accepted`. We mark the cart as accepted before attempting to send a message and we mark it as submitted after we know at least one copy of the message has been sent. +The solution is to revert our changes to a the save-first, send-later approach (remember the USB principle?). However, this time we allow re-sending the submit message. To ensure it is safe to do it we'll introduce a new status of the shopping cart -- `Accepted`. We mark the cart as accepted before attempting to send a message and we mark it as submitted once we know at least one copy of the message has been successfully sent. -- Notice we have added a new `Accepted` flag to the shopping cart. It will be used in the submission logic. -- Go to the `ApplicationServices` class and change the logic of the `SubmitOrder` method to do the following: - - If the cart is submitted throw an exception (as previously). - - If the cart is accepted +- Notice we have added a new `Accepted` flag to the shopping cart. It will be used in the submit logic. +- In the `ApplicationServices` class and change the logic of the `SubmitOrder` method to do the following: + - If the cart is submitted, throw an exception (as previously). + - If the cart is accepted: - send the `SubmitOrder` message, - set the state of the shopping cart to submitted, - save the state of the shopping cart. - - If the cart is not yet accepted + - If the cart is not yet accepted: - set the state of the shopping cart to accepted, - save the state of the shopping cart, - send the `SubmitOrder` message, - set the state of the shopping cart to submitted, - save the state of the shopping cart. -- Check what are the consequences of this behavior to the `Orders` service. -Consider what would happen if `AddItem` was called when the shopping cart is in the accepted state. +Consider the consequences of this behavior to the`Orders` service. What would happen if `AddItem` was called when the shopping cart is in the accepted state? From b91d2a3c19234d2df4b0554f2d1af3351a91c0fa Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 15:04:11 +0100 Subject: [PATCH 11/24] editorial --- NewExercises/Exercise-8/README.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/NewExercises/Exercise-8/README.md b/NewExercises/Exercise-8/README.md index a8627aa..54960a2 100644 --- a/NewExercises/Exercise-8/README.md +++ b/NewExercises/Exercise-8/README.md @@ -2,17 +2,17 @@ In the previous exercise we allowed the customer to retry submitting their order in case there is an error. That's a good change. But the system is still far from ideal. Is pushing the burden of handling our system failures on the customer OK? Certainly not. -The solution is based on two observations +The solution is based on two observations: - The messaging infrastructure is usually much more reliable (higher availability) than databases. If well maintained, it almost always accepts sent messages. -- The message processing using a durable messaging infrastructure has built-in retry capability. +- Message processing solutions that use durable messaging infrastructure have a built-in retry capability. -We are going to create a solution in which the payload of the submit request is encapsulated into a message that is sent to the local queue. The complex business logic of the `SubmitOrder` is moved to a message handler to allow for automatic retries in case of failures. +We are going to create a solution in which the payload of the submit request is encapsulated into a message that is sent to the local queue. The complex business logic of the `SubmitOrder` is moved to a dedicated message handler to allow for automatic retries in case of failures. Let's write some code. -- In the `Messages` project create a new message class `SendSubmitOrder` with two `string` properties: `Customer` and `CartId`. -- In the `Program` class in the section where NServiceBus is configured remove the call to `SendOnly`. We need to make the `WebFrontend` and active endpoint to process the `SendSubmitOrder` messages. -- In the same piece of code make sure the `repository` is available to NServiceBus handlers by adding following code +- In the `Messages` project, create a new message class `SendSubmitOrder` with two `string` properties: `Customer` and `CartId`. +- In the `Program` class, in the section where NServiceBus is configured, remove the call to `SendOnly`. We need to configure the `WebFrontend` as a full endpoint to process the `SendSubmitOrder` messages. +- In the same piece of code, make sure the `repository` is available to NServiceBus handlers by adding following code: ```c# endpointConfiguration.RegisterComponents(c => @@ -21,18 +21,18 @@ endpointConfiguration.RegisterComponents(c => }); ``` -- In the `WebFrontend` project add a handler for the `SendSubmitOrder` message, similar to the `SubmitOrderHandler` in the `Orders` project +- In the `WebFrontend` project, add a handler for the `SendSubmitOrder` message, similar to the `SubmitOrderHandler` in the `Orders` project - Remember to implement the `IHandleMessages` interface. - Add a `repository` parameter of type `Repository` to the constructor and store the value in an instance field - Move the code from the `SubmitOrder` method to the `Handle` method of the new handler. - - Replace the parameter references to references to the incoming message + - Replace the parameter references to reference properties of the incoming message - Replace the `session.Send` call with `context.Send` - - Pass an instance of `SendOptions` to the send method. Configure the send options to use immediate dispatch via `sendOptions.RequireImmediateDispatch()` before sending. -- Change the code from the `SubmitOrder` method + - Pass an instance of `SendOptions` to the send method. Configure the send options to perform an immediate dispatch via `sendOptions.RequireImmediateDispatch()` before sending. +- Change the code in the `SubmitOrder` method - Remove existing code - Add a call to `session.SendLocal` passing an instance of a `SendSubmitOrder` class. -- Does it stil make sense to throw exception when the cart is already marked as submitted and `SendSubmitOrder` is received? Why? -- Why do you think `RequireImmediateDispatch` is needed? +Does it still make sense to throw an exception when the cart is marked as submitted? Why? +Why do you think `RequireImmediateDispatch` is needed? Consider the output of the `Orders` backend application. What did you notice? \ No newline at end of file From ee6f79ee8f82c0bbb655b2b201995e3c2a3ade05 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 15:07:14 +0100 Subject: [PATCH 12/24] change to match current code in the exercise --- NewExercises/Exercise-8/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/NewExercises/Exercise-8/README.md b/NewExercises/Exercise-8/README.md index 54960a2..d3fc01d 100644 --- a/NewExercises/Exercise-8/README.md +++ b/NewExercises/Exercise-8/README.md @@ -15,9 +15,9 @@ Let's write some code. - In the same piece of code, make sure the `repository` is available to NServiceBus handlers by adding following code: ```c# -endpointConfiguration.RegisterComponents(c => +endpointConfiguration.RegisterComponents(collection => { - c.RegisterSingleton(repository); + collection.AddSingleton(repository); }); ``` From 88269e8ec7f73cf36f5ffcf66c6cce887fb0bd21 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Mon, 21 Mar 2022 15:19:00 +0100 Subject: [PATCH 13/24] Revert "change to match current code in the exercise" This reverts commit d6f76af20bcb0fac0122e3ca74fcf00b3e33629d. --- NewExercises/Exercise-8/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/NewExercises/Exercise-8/README.md b/NewExercises/Exercise-8/README.md index d3fc01d..54960a2 100644 --- a/NewExercises/Exercise-8/README.md +++ b/NewExercises/Exercise-8/README.md @@ -15,9 +15,9 @@ Let's write some code. - In the same piece of code, make sure the `repository` is available to NServiceBus handlers by adding following code: ```c# -endpointConfiguration.RegisterComponents(collection => +endpointConfiguration.RegisterComponents(c => { - collection.AddSingleton(repository); + c.RegisterSingleton(repository); }); ``` From 7c19147cc6c6ef4fb3f9c1e9083a849674df08ac Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 09:41:07 +0100 Subject: [PATCH 14/24] editorial --- NewExercises/Exercise-9/README.md | 10 +++++----- NewExercises/Exercise-9/follow-up.md | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/NewExercises/Exercise-9/README.md b/NewExercises/Exercise-9/README.md index e108beb..1193c94 100644 --- a/NewExercises/Exercise-9/README.md +++ b/NewExercises/Exercise-9/README.md @@ -1,18 +1,18 @@ # Exercise 9: Message duplication on the receiver side -In this exercise we are going to experience the message duplication first-hand. We are going to use Azure Storage Queues as a messaging infrastructure but the behavior we are going to observe is not specific to this technology. +In this exercise we are going to experience message duplication first-hand. We'll use [Azure Storage Queues](https://docs.microsoft.com/en-us/azure/storage/queues/storage-queues-introduction) as a messaging infrastructure but the behavior we will observe is not specific to this technology. -Provide configuration options: +In the solution, complete the following configuration options: - Search for "TODO" strings - - Enter ASQ connection string value + - Specify the ASQ connection string - Set the endpoint name value to "Orders-_yourname_" -The code in this exercise is very similar to previous exercises but uses a real message queue that we have set up in the cloud. Start by running the solution (`Frontend` and `Orders` projects). +The code in this exercise is very similar to the previous exercises but uses a real message queue that we set up in the cloud. Start by running the solution (`Frontend` and `Orders` projects). - Create one order - Add some *pierogi* with meat to the order. Is everything OK? -Stop and go back to Visual Studio. Find the `AddItemHandler` class. We are going to simulate a slow database by adding a `Task.Delay(40000)` before the final log statement of the `Handle` method. +Stop and go back to Visual Studio. Find the `AddItemHandler` class. We are going to simulate lag on the database by inserting a `Task.Delay(40000)` statement, right before the final log statement of the `Handle` method. - Run the solution - Create one order diff --git a/NewExercises/Exercise-9/follow-up.md b/NewExercises/Exercise-9/follow-up.md index 365b7f6..caef750 100644 --- a/NewExercises/Exercise-9/follow-up.md +++ b/NewExercises/Exercise-9/follow-up.md @@ -1,5 +1,5 @@ # Exercise 2: Message duplication on the receiver side -The NServiceBus Azure Storage Queues transport uses [Peek-Lock](https://docs.microsoft.com/en-us/rest/api/servicebus/peek-lock-message-non-destructive-read) API to receive messages without destroying them. The lock is held for 30 seconds after which the message becomes visible again. Because the `Orders` endpoint is multi-threaded, it immediately fetches the message again, in another thread. +The NServiceBus Azure Storage Queues transport uses the [Peek-Lock](https://docs.microsoft.com/en-us/rest/api/servicebus/peek-lock-message-non-destructive-read) API to receive messages without destroying them. The lock hides the message on the queue from competing consumers and is held for 30 seconds, after which the message becomes visible to other consumers. Since the `Orders` endpoint is multi-threaded, it immediately fetches the message again, in another thread. -As a result, the message is never ever going to be consumed. This is of course an extreme case. In real world system everything would be fine in the testing environment (where the load is small). It would even work in production for a couple of weeks until a higher load is experienced and the database starts to lag behind. A couple of messages are processed slower than usual and they are picked up again, adding even more load to an already struggling system. +As a result, the message is never going to be consumed. This showcases an extreme scenario. In a real world system, everything would be fine in the testing environment (where the load is small). It would even work in production for a couple of weeks until a higher load is experienced and the database starts to lag. A couple of messages are processed slower than usual and later picked up again, adding even more load to an already struggling system. From bb8e89b1bd44815cdc6dfe7b3a5a5facc30b795e Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 09:41:16 +0100 Subject: [PATCH 15/24] editorial --- NewExercises/Exercise-10/README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/NewExercises/Exercise-10/README.md b/NewExercises/Exercise-10/README.md index 25c25f2..57d6d31 100644 --- a/NewExercises/Exercise-10/README.md +++ b/NewExercises/Exercise-10/README.md @@ -1,7 +1,9 @@ # Exercise 10: Primary key-based deduplication -Create-type operation can be de-duplicated based on the ID of the entity/aggregate to be created. Add such logic to the order. And test. +Create-type operations can be deduplicated based on the ID of the entity/aggregate to be created. -- Go to the `SubmitOrderHandler` class and change the `Guid`-based order ID generation strategy with the value of the `CartId` property of the `SubmitOrder` message. +Try to add such deduplication logic to the order and test the behavior. + +- In the `SubmitOrderHandler` class, change the `Guid`-based order ID generation strategy with the value of the `CartId` property of the `SubmitOrder` message. - Run the solution to see the result -- Modify the code of the `SubmitOrderHandler` to discard the message if an order already exists by using `repository.Get` method. Remember to check the `version` part of the return because the `Get` method always returns a non-null item. \ No newline at end of file +- Modify the code of the `SubmitOrderHandler` to discard the message if an order already exists by using `repository.Get` method. Remember to check the `version` because the `Get` method always returns a non-null item. \ No newline at end of file From 576d54e65208fe2d46af96cafb5b668215bdae71 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 09:41:24 +0100 Subject: [PATCH 16/24] editorial --- NewExercises/Exercise-11/README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/NewExercises/Exercise-11/README.md b/NewExercises/Exercise-11/README.md index a8746ed..1f4803a 100644 --- a/NewExercises/Exercise-11/README.md +++ b/NewExercises/Exercise-11/README.md @@ -4,7 +4,7 @@ When writing integration tests for message-based systems it's common to make ass ### Goal -The goal of this exercise is to write a single integration test that depends on the result of processing the last message in a conversation. The conversation starts with `PlaceOrder` command that has a 1 in 20 chance of triggering the final `FinalizeOrder` command or re-sending the same command with `1` seconds delay: +The goal of this exercise is to write a single integration test that depends on the processing result of the last message in a conversation. The conversation starts with the `PlaceOrder` command that has a 1 in 20 chance of triggering the final `FinalizeOrder` command or re-sending the same command with `1` seconds delay: ```csharp if (new Random().Next(0, 20) == 0) @@ -43,11 +43,12 @@ static IEnumerable TestCases => Enumerable.Range(1, 25).Select(n => $"{n ### Step 1 -Can we solve the problem with a simple patch? Can we add `Task.Delay` in our test to make it pass consistently and not make it take a couple of minutes to pass? +Can we solve the problem with a simple patch? Can we add `Task.Delay` in our test to make it pass consistently? ### Step 2 -Let's add behavior to the message processing pipeline in our endpoint so that for every incoming message and all messages that are generated we capture their identifiers and sent this data off to a dedicated queue for further processing using `TracingBehavior`: +Let's add behavior to the message processing pipeline in our endpoint to capture identifiers for every incoming message and its resulting outgoing messages and send this data off to a dedicated queue for further processing. +This `TracingBehavior` is registered on the endpoint: ```csharp (endpoint, store) = await Program.StartEndpoint(c => @@ -56,16 +57,16 @@ Let's add behavior to the message processing pipeline in our endpoint so that fo }); ``` -TASK: Add a missing piece of logic to the `TracingBehavior` to capture outgoing message identifiers in the `OutgoingMessageIds` property of the `TracingMessage`. Run the test in the `Debug` mode to make sure the data is being captured. +TASK: Add the missing logic to the `TracingBehavior` to capture outgoing message identifiers in the `OutgoingMessageIds` property of the `TracingMessage`. Run the test in `Debug` mode to ensure the data is captured. ### Step 3 Tracing messages sent to the `trace` queue will be processed by a dedicated endpoint encapsulated in the `Tracer` class. -TASK: Create an instance of the `Tracer` class in the `Setup` method of the test and make sure it's properly clean-up in the `Cleanup` +TASK: Create an instance of the `Tracer` class in the `Setup` method of the test and make sure it's properly cleaned up in the `Cleanup` method. ### Step 4 -`Tracer` provides logic to setup conversation tracking and later to wait until the conversation finishes. In order, to set up, the conversation one needs to call the `Prepare` method which returns `conversationId` and `SendOptions` tuple. +`Tracer` provides logic to set up conversation tracking and wait until the conversation finishes. In order to set up, the conversation one needs to call the `Prepare` method which returns `conversationId` and `SendOptions` tuple. TASK: Extend the test code by calling the `tracer.Prepare` and `tracer.WaitUntilFinished` to make sure that test asserts are called at the right time. From d1fb7dc95ef6e884cb7fc681a4821a87d44b8d9d Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 10:58:11 +0100 Subject: [PATCH 17/24] editorial --- NewExercises/Exercise-12/README.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/NewExercises/Exercise-12/README.md b/NewExercises/Exercise-12/README.md index 46dac78..4e84ae2 100644 --- a/NewExercises/Exercise-12/README.md +++ b/NewExercises/Exercise-12/README.md @@ -1,11 +1,13 @@ -NOTE: This and couple of following exercises use automated tests for show how our system behaves in various scenarios that might happen in messaging systems. +NOTE: Starting this exercise, we use automated tests to showcase how the system behaves in various scenarios that can occur in messaging systems. -Our system has been extended with new functionality. After order has been placed, we can book a payment for a given order or cancel a payment that has already been booked. So far the deduplication mechanism depended on the idempotence of the operations conducted in the message handling logic. If we mark the payment as booked, we can do it multiple times and the result is the same as if we marked it once. Similarly, when we mark the payment as non-booked, we can repat the operation multiple times and still the payment stays non-booked. +Our system was extended with new functionality. After an order has been placed, a payment for a given order can be booked or cancelled. So far, the deduplication mechanism depended on the idempotency of the message handling logic. +Marking the payment as booked, is an operation that has the same observable result, regardless of how many times we execute it. +Similarly, when we mark the payment as not-booked, we can repeat the operation multiple times without unexpected side effects. -Let's see what happens when some of these messages get reordered: +Let's see what happens when some of these messages are reordered: -* Open `IntegrationTests.cs` in the `Test` project and naviage to `ChangeStatus` test -* Use `SendInOrder` utility method to simulate scenario in which oder is placed, payment is booked and later cancelled but the `BookPayment` command in duplicated and the duplicate arrives as the last message: +* Open `IntegrationTests.cs` in the `Test` project and navigate to the `ChangeStatus` test +* Use the `SendInOrder` method to simulate a scenario in which an order is placed, the payment is booked and later cancelled. We want the `BookPayment` command to be duplicated and the duplicate to arrive as the last message: ```csharp await SendInOrder(new IMessage[] { @@ -16,11 +18,11 @@ await SendInOrder(new IMessage[] } ); ``` -* Run `ChangeStatus` test and check if the assertion holds. If not then can you tell what is wrong? -* Add `List` property to `Order` enity called `ProcessedMessages` +* Run the `ChangeStatus` test and check if the assertion holds. If not, can you tell what's wrong? +* Add a `List` property called `ProcessedMessages` to the `Order` entity ```csharp public List ProcessedMessages { get; set; } = new List(); ``` -* Go to the `BookPaymentHandler` and `CancelPaymentHandler` and modify the deduplication logic to use message Ids. At the end of the message processing in the handler, but before the `Order` is persisted, include the Id of the message being processed in the `ProcessedMessages` collection. At the beginning of the handler code check if the message already exists in the collection. If so, return without doing anything in the handler. +* In the `BookPaymentHandler` and `CancelPaymentHandler`, modify the deduplication logic to use message Ids. At the end of the message processing logic, right before the `Order` is persisted, include the Id of the message being processed in the `ProcessedMessages` collection. At the beginning of the handler, check whether the message already exists in the collection. If so, exit the handler, discarding the message. What aspect of a message handler is missing in this exercise? From a75e012347ae85d306b32890595220f8ad8a7771 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 11:22:57 +0100 Subject: [PATCH 18/24] editorial --- NewExercises/Exercise-14/README.md | 34 ++++++++++++++++-------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/NewExercises/Exercise-14/README.md b/NewExercises/Exercise-14/README.md index e496ada..ae1b1f9 100644 --- a/NewExercises/Exercise-14/README.md +++ b/NewExercises/Exercise-14/README.md @@ -1,8 +1,9 @@ -## State based message generation +## State-based message generation -Now that we can reliably calculate value of all the payments made by a customer the business wants to put that to a good use. Our team needs to add a small feature i.e. when a customer goes over 100 USD in total paymets for the first time we want to send them a coupon. +Now that we can reliably calculate the total of all payments made by a customer, the business wants to put this information to good use. +Our team needs to add a small feature i.e. when a customer exceeds 100 USD in total payments, we want to send them a coupon. -* Go to the `IssueCouponCardAfterFirst100USDSpent` test and define the follwong sequence of message processing. What could be a production scenario in which this could happen? +* In the `IssueCouponCardAfterFirst100USDSpent` test, define the following sequence of message processing: ```csharp new IMessage[] { @@ -12,12 +13,13 @@ new IMessage[] { bookSecondPayment } ``` - -* Run the test and check if the asserition holds -* Put logic in the `DropMessagesBehavior` to ensure that the `GrantCoupon` message is skipped (dropped) the first time it is sent. This simulates situation when `PaymentBookHandler` failes on sending out `GrantCoupon` and the incoming message is retried. Use the same method as in [Exercise 5](../Exercise-5/README.md): +What could be a production scenario in which this could happen? + +* Run the test and check if the assertion holds +* Add logic in the `DropMessagesBehavior` to ensure that the `GrantCoupon` message is discarded the first time it is sent. This simulates a situation when `PaymentBookHandler` fails when sending out `GrantCoupon` and the incoming message is retried. Use the same method as in [Exercise 5](../Exercise-5/README.md): * Add a boolean flag `dropped` - * In the `Invoke` method check if the `context.Message.Instance` is `GrantCoupon`. If so, and the flag is `false` then flip the flag and return. Otherwise invoke `await next()`. This logic will ensure that only the first instance of the `GrantCoupon` message is dropped. -* To account for this simulated failure we should add another copy of the `bookFirstPayment` to the messages list for the test: + * In the `Invoke` method, check if the `context.Message.Instance` is `GrantCoupon`. If so, and the flag is `false`, flip the flag and return. Otherwise, invoke `await next()`. This logic will ensure that only the first instance of the `GrantCoupon` message is dropped. +* To account for this simulated failure, we will add an additional copy of the `bookFirstPayment` to the messages list for the test: ```csharp new IMessage[] { @@ -29,14 +31,14 @@ new IMessage[] { } ``` -* With that change the test should pass again. Re-run the test. Does it work? Can you tell why? Try placing some breakpoints in the `PaymentBookedHandler`: - * in line 18 to check how many times the handler is invoked - * in line 36 to see how many times the business logic is invoked - * in line 38 to see how many times the `GrantCoupon` message is generated +* With that change in place, the test should pass again. Re-run the test. Does it work? Can you tell why? Try placing some breakpoints in the `PaymentBookedHandler`: + * on line 18 to check how many times the handler is invoked + * on line 36 to see how many times the business logic is invoked + * on line 38 to see how many times the `GrantCoupon` message is generated * Can you explain the behavior? -* Use `public List OutgoingMessages = new List();` property in the `Payments` entity to store the outoging messages. -* Instead of sending the message immediately in line 38, add it to the `OutgoingMessages` collection. -* Make sure that items in the `OutgoingMessages` are _always_ sent out (including the times when duplicates arrive). Add following code to the bottom of the handler: +* Use a `public List OutgoingMessages = new List();` property in the `Payments` entity to store the outgoing messages. +* Instead of sending the message immediately, on line 38, add it to the `OutgoingMessages` collection. +* Make sure that items in the `OutgoingMessages` are _always_ sent out (including the cases in which a duplicate is detected). Add the following code to the bottom of the handler: ```csharp foreach (var outgoingMessage in payments.OutgoingMessages) @@ -44,4 +46,4 @@ foreach (var outgoingMessage in payments.OutgoingMessages) await context.SendImmediately(outgoingMessage); } ``` -* Run all the test in the `Tests` project +* Run all the tests in the `Tests` project. From 1bccb604f8fe435da28d791f9e14d8cc1608a046 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 11:29:23 +0100 Subject: [PATCH 19/24] editorial --- NewExercises/Exercise-13/README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/NewExercises/Exercise-13/README.md b/NewExercises/Exercise-13/README.md index 4bacbca..6292bb1 100644 --- a/NewExercises/Exercise-13/README.md +++ b/NewExercises/Exercise-13/README.md @@ -1,10 +1,11 @@ ## Deterministic message identifiers -Due to considerable sucess of our the business, the system has been extended with new `Marketing` endpoint, reponsible for tracking value of payments booked for any given customer. When status of an order is changed, either `PaymentBooked` or `PaymentCancelled` event is published. +Due to the success of our the business, the system has been extended with a new `Marketing` endpoint, responsible for tracking the total expenditure of our customers (the total of all payments). +When the status of an order is changed, either `PaymentBooked` or `PaymentCancelled` event is published. -Unfortunatelly, our production support team claims that once in a while the calculated value for a customer does not match the total from all the payments. Let's see if we can reproducte such a scenario. +Unfortunately, our support team claims that once in a while, the calculated total expenditure per customer does not match the actual total of all payments. Let's see if we can reproduce such a scenario. -* Go to the `BookPaymentHandler` or `CancelPaymentHandler` and take a look at the logic. Notice that it is almost the same as in the previous exercise but it also includes publishing of events. Notice that the basic structure remains the same regardless what type of duplication mechanism is used: +* Go to the `BookPaymentHandler` or `CancelPaymentHandler` and consider the logic. Notice that it is mostly identical to the logic in the previous exercise but it includes events being published. Notice that the basic structure remains the same regardless of the deduplication mechanism used: ``` if (!IsDuplicate()) @@ -16,7 +17,7 @@ if (!IsDuplicate()) SendAndPublishMessages()' ``` -* Go to `TrackTotalPaymentsValue` test and check if it passes. Why does it fail? Open the results of the test run and check what is are the `messageId` values for both duplicates of the `BookPayment` message. Why are they different? Doest it resemble any situation we have seen before? -* In the `BookPaymentHandler` and `CancelPaymentHandler` use `PublishWithId` extension method and use `Utils` class to ensure that the published messages have Ids that are deterministically derived from the incoming message Id. -* Is our deterministic Id generation strategy good? +* Check if the `TrackTotalPaymentsValue` test passes. Why does it fail? Open the results of the test run and check the `messageId` values for both duplicates of the `BookPayment` message. Why are they different? Does this resemble any situation we've seen before? +* In the `BookPaymentHandler` and `CancelPaymentHandler`, use the `PublishWithId` extension method and the `Utils` class to ensure that the published messages have Ids that are deterministically derived from the incoming message Id. +* Is our deterministic Id generation strategy a good solution? * Does the `ChangeStatus` test still pass? From 6afe4fd1e34b1fa551bbb9cf229cd4fb57e3aeab Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 14:53:17 +0100 Subject: [PATCH 20/24] editorial --- NewExercises/Exercise-15/README.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/NewExercises/Exercise-15/README.md b/NewExercises/Exercise-15/README.md index 37df77b..02e5f5d 100644 --- a/NewExercises/Exercise-15/README.md +++ b/NewExercises/Exercise-15/README.md @@ -1,31 +1,32 @@ # Exercise 15: Generic outbox -In the previous exercise we implemented the Outbox pattern inline in the handler of the `AddItem` message. While it did solve our problem, it is not ideal from the code reuse perspective. Who would like to have the same code copied over and over again? In this exercise we are going to extract that piece of code and make it more generic. In order to achieve this we will use the *behavior* extension system of NServiceBus; the same one that we previously used to simulate various failure conditions. +In the previous exercise, we've implemented the Outbox pattern inline in the handler of the `AddItem` message. While it did solve our problem, it is not ideal from the code reusability perspective. +Who wants the same code duplicated everywhere? In this exercise we will extract that piece of code and make it more generic. In order to achieve this we will use the *behavior* extension system of NServiceBus; the same one that we previously used to simulate various failure conditions. -Before we start coding, let's open the completed exercise 14 and the solution for the exercise 15 side-by-side. Go to the `BookPaymentHandler` and `AddItemHandler`. Take a look at the code. It looks almost exactly the same, right? That's not coincidence. Both handlers execute different business logic but have the same structure. That is a sign of an opportunity for code sharing. Taking advantage of that opportunity is the point of this exercise. +Before we start coding, let's open the completed exercise 14 and the solution for the exercise 15 side-by-side. Go to the `BookPaymentHandler` and `AddItemHandler`. Take a look at the code. It looks almost exactly the same, right? That's not by coincidence. Both handlers execute different business logic but have the same structure. That's an opportunity for code sharing, which is the point of this exercise. Now let's get our hands dirty. -- Open the `OutboxBehavior` class. You should be by now familiar with behaviors. They are a way of expessing cross-cutting concerns in NServiceBus. This behavior does not do anything right now but in the course of this exercise we are going to fill it with code. -- In the `OutboxBehavior`, before passing the invocation further in line 27, load the order from the repository and add it to the context. +- Open the `OutboxBehavior` class. By now, you should be familiar with behaviors. They provide a framework to express cross-cutting concerns in NServiceBus. This behavior doesn't do anything right now, but in the course of this exercise we'll extend it. +- In the `OutboxBehavior`, before passing the invocation further on line 27, load the order from the repository and add it to the context. - Load the order based on the `OrderId` property of the `orderMessage` - Put it in the handling context: `context.Extensions.Set(order);` -- Persist the changes done to the order via `orderRepository.Store` after the call to `next()` (this line invokes the message handler) -- Replace usages of repository in the `AddItemHandler` with usages of the `Order` instance managed by the `OutboxBehavior` - - Replace the repository usage with retrieving the `Order` from the context: `var order = context.Extensions.Get();` - - Remove the second first to `orderRepository.Store` as it is handled by the behavior +- Persist the changes by calling `orderRepository.Store` after the call to `next()` (calling `next()` invokes the message handler) +- Replace all usages of repository in the `AddItemHandler` with usages of the `Order` instance managed by the `OutboxBehavior` + - Replace the repository usage by retrieving the `Order` from the context: `var order = context.Extensions.Get();` + - Remove the second call to `orderRepository.Store` as it is handled by the behavior - Run the code -With this change we moved most of the code responsible for loading and storing the entity to the `OutboxBehavior` and the handler can focus on the actual business logic. But there is some stuff left so let's continue. +With this change, we moved most of the code responsible for loading and storing the entity to the `OutboxBehavior` and the handler can focus on the actual business logic. But there is some stuff left so let's continue. -- Move the code responsible for pushing out generated messages to the `OutboxBehavior` +- Move the code responsible for publishing messages to the `OutboxBehavior` - Remove (cut) the last section of code from the `AddItemHandler` where messages are published and cleared from the collection - Add (paste) that code just below the `orderRepository.Store` in the `Invoke` method of the `OutboxBehavior` -Looks better, doesn't it? Event better, it compiles and works. Let's now take care of the last bit of the handler that is related to the deduplication -- the check if a message has been processed. +Looks better, doesn't it? Even better, it compiles and works. Now, let's take care of the bits of code in the handler that's' related to deduplication -- the check if a message has been processed. Remove the `!order.ProcessedMessages.Contains(context.MessageId)` from the handler and move it to the `OutboxBehavior` to guard the calls to `next()` and `orderRepository.Store`. Voila! ## It is alive! -The solution works perfectly. You can be proud of yourselves. Let's take a moment to apprieciate that. What we have implemented is in fact the cutting-edge deduplication approach used by multiple commercial and open-source frameworks. That has been a long and tough journey but we made it! From now one we are going to be entering a much less known territory and the algorithms we are going to talk about are not yet available from production-ready tools. But that's good, right? \ No newline at end of file +The solution works perfectly. You can be proud of yourselves. Let's take a moment to appreciate that. What we have implemented is in fact the cutting-edge deduplication approach used by multiple commercial and open-source frameworks. That has been a long and tough journey but we made it! Moving forward, we're going to enter a much less known territory and the algorithms we are going to talk about are not yet available in production-ready tools. But that's good, right? \ No newline at end of file From c4f80d414c06c99ee90d030c3ed361f1c7a55e78 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 14:53:23 +0100 Subject: [PATCH 21/24] editorial --- model-checking/README.md | 47 ++++++++++++++++++++-------------------- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/model-checking/README.md b/model-checking/README.md index 603697d..7c2fc71 100644 --- a/model-checking/README.md +++ b/model-checking/README.md @@ -2,14 +2,15 @@ - [Visual Studio Code](https://code.visualstudio.com/) - [TLA+ extensions](https://github.com/alygin/vscode-tlaplus/wiki/How-to-Install) +- [JAVA](https://github.com/tlaplus/vscode-tlaplus/wiki/Installing-Java) ## Setup -In the exercise we won't check for deadlocks and we want to prevent the model checker from checking that. In Visual Studio Code go to TLA extension settings and in `Tla Plus > Tlc > Model checker` field specify `-deadlock` parameter. +In this exercise, we won't check for deadlocks, therefore we want to prevent the model checker from checking these. In Visual Studio Code, go to the TLA extension's settings and in the `Tla Plus > Tlc > Model checker` field, specify the `-deadlock` parameter. ## Introduction -`MessageHandler.tla` holds a TLA+ specification of a message processing handler. It models an environment with no distributed transactions available i.e. messaging infrastructure and the business data store operations are performed without any atomicity guarantees. +`MessageHandler.tla` holds a TLA+ specification of a message processing handler. It models an environment without distributed transactions i.e. messaging infrastructure and the business data store operations are performed without atomicity guarantees. ## Exercise 1 @@ -17,13 +18,13 @@ Model-checking with TLA+ requires two elements, a specification (`MessageHandler To run the model checker we need to: * Parse the specification: Ctrl+Shift+P -> `TLA+: parse module` - * Check the model: Ctrl+Shift+P -> `TLA+: check the model with TLC` + * Check the model: Ctrl+Shift+P -> `TLA+: check model with TLC` - This opens `TLA+ model checking` window that shows the model checking status and the final result. + This opens the `TLA+ model checking` window which shows the model checking status as well as the final result. ## Exercise 2 -Let's verify that the model does not allow for ghost messages using `NoGhostMessages` formula: +Let's verify that the model does not allow for ghost messages using the `NoGhostMessages` formula: ```tla+ NoGhostMessages == \A m \in processed : @@ -31,7 +32,7 @@ NoGhostMessages == \A m \in processed : \/ \E chg \in db : chg.id = m.id ``` -In the Visual Studio code: +In Visual Studio Code: * Open `MessageHandler.cfg` and add `NoGhostMessages` in the `INVARIANTS` section of the file. * Parse and model-check the specification. * The check fails with: @@ -41,7 +42,7 @@ In the Visual Studio code: ## Exercise 3 -Let's verify that the model does not allow any outgoing messages to be lost using `NoLostMessages` formula: +Let's verify that the model does not allow any outgoing messages to be lost using the `NoLostMessages` formula: ```tla+ NoLostMessages == \A m \in processed : @@ -50,11 +51,11 @@ NoLostMessages == \A m \in processed : ``` * Open `MessageHandler.cfg` and add `NoLostMessages` in the `INVARIANTS` sections. - * Parse and model check the specification. + * Parse and model-check the specification. * The check fails with: > Invariant NoLostMessages is violated. - * Analyze the trace to understand what happened - * Let's patch the problem temporarily and change the atomicity of the steps (this models 2PC) to prevent outgoing message loss + * Analyze the trace to understand what happened. + * Let's patch the problem temporarily and change the atomicity of the steps (this models 2PC) to prevent outgoing message loss. ```tla+ UpdateDbAndSend: (* update data base and send output messages - can fail *) @@ -66,7 +67,7 @@ UpdateDbAndSend: (* update data base and send output messages - can fail *) ## Exercise 4 -Let's verify that the model does not allow for duplicated processing of the same message using `NoDuplicatedProcessings` formula: +Let's verify that the model does not allow for duplicate processing of the same message using `NoDuplicatedProcessings` formula: ```tla+ NoDuplicatedProcessings == \A a \in db: @@ -85,14 +86,14 @@ end if; ``` ## Exercise 5 -Now we want to keep current properties but remove the atomicity between database updates and sending out messages: - * First, we will make a change to the specification. Currently, our model allows for a message to be partially processed due to failures. We will change it so that every message is eventually processed - possibly with some failures. +Now, we want to keep current properties but remove the atomicity between database updates and sending out messages: + * First, we will make a change to the specification. Currently, our model allows for a message to be partially processed due to failures. We will change this so that every message is eventually processed - possibly with some failures. * Add `Fails(messageId)` definition at the top of the `define` section. ```tla+ Fails(messageId) == IF failures[messageId] <= MaxFailures THEN {TRUE, FALSE} ELSE {FALSE} ``` - * Change the `Fail` macro to update the number of failures for a given message and jump back to the `MainLoop` label + * Change the `Fail` macro to update the number of failures for a given message and jump back to the `MainLoop` label. ```tla+ macro Fail(messageId) begin @@ -100,8 +101,8 @@ macro Fail(messageId) begin goto MainLoop; end macro; ``` - * Split `UpdateDbAndSend` lables back into two separate labels. - * Change specification in the `UpdateDb` and `Send` and `AckInMsg` labels to model the fact that there can be up to`MaxFailuers` for any given message. E.g: + * Split `UpdateDbAndSend` labels back into two separate labels. + * Change specification in the `UpdateDb`, `Send` and `AckInMsg` labels to model the fact that there can be up to`MaxFailuers` for any given message. E.g: ```tla+ UpdateDb: @@ -121,7 +122,7 @@ UpdateDb: ## Exercise 6 -Let's check that the handler returns a consistent output using following formula: +Let's check that the handler returns consistent output using the following formula: ``` tla+ ConsistentOutput == \A m1 \in queueOut: @@ -130,9 +131,9 @@ ConsistentOutput == \A m1 \in queueOut: * Add the `ConsistentOutput` formula definition to the specification. * Add `ConsistentOutput` to the `INVARIANTS` section. - * Parse and model check the specification. + * Parse and model-check the specification. * Analyze the failing trace. - * Change the `Send` label part to make sure that the output messages are sent are based on the DB state with consistent transaction id. + * Change the `Send` label part to ensure that the output messages sent are based on the DB state with a consistent transaction id. HINT: You can get the version of the DB change for given message id using `with` statement: @@ -144,14 +145,14 @@ end with; ## Exercise 7 -Let's make the model a bigger by changing model parameters +Let's increase the model size by changing model parameters: * Change model to allow for `3` failures per-message and start with `3` messages in the input queue. * Parse and check the specification. - * Capture number of states checked. + * Capture the number of states checked. The model is getting big so let's put it on a diet: * Merge the `Process` label with `Receive`. - * Instead of per-message failure, let's move to a single "total failures" counter. Tweak the `Fail` macro and `Fails` formula to depend on the global `failures` variable. + * Instead of using per-message failure counters, let's move to a single "total failures" counter. Tweak the `Fail` macro and `Fails` formula to depend on the global `failures` variable. ``` tla+ macro Fail() begin @@ -159,7 +160,7 @@ macro Fail() begin goto MainLoop; end macro; ``` - * Compare size of the model before and after the changes. + * Compare the size of the model before and after the changes. ## Exercise 8 From 16690445f21c9e1be79cac8ae7a8d7e2349ff23b Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 16:04:56 +0100 Subject: [PATCH 22/24] editorial --- NewExercises/Exercise-17/README.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/NewExercises/Exercise-17/README.md b/NewExercises/Exercise-17/README.md index d16ea49..5c01094 100644 --- a/NewExercises/Exercise-17/README.md +++ b/NewExercises/Exercise-17/README.md @@ -2,12 +2,14 @@ The solution looks really good but we can make it even better. Message serialization is a very important aspect of every distributed system. Sometimes this serialization is quite complex. -In the current solution messages are serialized in two ways. First, they are serialized as part of the `Order` aggregate using whatever mechanism the data store uses. Then, they are serialized for sending over the wire. This may cause problems, especially with more complex data types. +In the current solution, messages are serialized in two ways. First, they are serialized as part of the `Order` aggregate using whatever mechanism the data store uses. Then, they are serialized for sending over the wire. This may cause problems, especially with more complex data types. -In this exercise we are going to plug in deeper into the messaging framework in order to allow for storing wire-formatted messages in the outbox. +In this exercise we're going to dive deeper into the messaging framework and plug in a mechanism that allows storage of wire-formatted messages in the outbox. -- In the `OutboxState` replace the `Message` class with built in `NServiceBus.Transport.TransportOperation` and make it an array instead of a list. -- Replace the call to `next()` in line 38 of the `OutboxBehavior` with a call to `InvokeMessageHandler`. Assign the returned value to `order.OutgoingMessages[context.MessageId].OutgoingMessages` +- In the `OutboxState`, replace the `Message` class with the built in `NServiceBus.Transport.TransportOperation` class and make it an array instead of a list. +- Replace the call to `next()` on line 38 of the `OutboxBehavior` with a call to `InvokeMessageHandler`. Assign the returned value to `order.OutgoingMessages[context.MessageId].OutgoingMessages` - Replace the `forach` loop with a call to `await Dispatch`. - Replace the calls to `outboxState.OutgoingMessages.Add` in the `AddItemHandler` to `context.Publish`. Remember to `await` this call. -- You can remove the `outboxState` from the `AddItemHandler` now. That's quite an achievement! There is no more deduplication logic in the message handler! +- You can remove the `outboxState` from the `AddItemHandler` now. + +- That's quite an achievement! There is no more deduplication logic in the message handler! From 044245bd60962abf1ae22991d6a075d7eed2a959 Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 16:10:55 +0100 Subject: [PATCH 23/24] editorial --- NewExercises/Exercise-16/README.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/NewExercises/Exercise-16/README.md b/NewExercises/Exercise-16/README.md index df4a4ae..160c506 100644 --- a/NewExercises/Exercise-16/README.md +++ b/NewExercises/Exercise-16/README.md @@ -1,27 +1,29 @@ # Exercise 16: Generic outbox - part 2 -In this exercise we are going to refactor the solution to simplify the management of the deduplication data. We are going to replace the `ProcessedMessages` and `OutgoingMessages` collections with a single `OutgoingMessages` collection. +In this exercise, we're' going to refactor the solution to simplify the management of deduplication data. We are going to replace the `ProcessedMessages` and `OutgoingMessages` collections with a single `OutgoingMessages` collection. The goal is to implement the following logic: -**If the `OutgoingMessages` dictionary contains a non-null value for a message ID, that message has been processed but the resulting outgoing messages have not been dispatched. If it contains `null` then that message have been processed and resulting outgoing messages dispatched. If it does not contain that value, the message has not been processed.** +**If the `OutgoingMessages` dictionary contains a non-null value for a given message ID, the message has been processed but the resulting outgoing messages have not been dispatched. If it contains `null` then the message was processed and resulting outgoing messages were dispatched. If it does not contain that value, the message has not been processed.** -- Change the type for the `OutgoingMessages` property of the `Order` to `Dictionary`. The `OutboxState` type represents a collection of complete serialized messages (including the headers). This is a significant change since from now on the key is going to be the ID of the **incoming message** while the ID of the outgoing message is going to be part of the value +- Change the type for the `OutgoingMessages` property of the `Order` to `Dictionary`. The `OutboxState` type represents a collection of complete serialized messages (including the headers). This is a significant change since from now on the key is going to be the ID of the **incoming message** while the ID of the outgoing message is going to be part of the value. - Change the _has been processed_ condition `!order.ProcessedMessages.Contains(context.MessageId)` to use the `OutgoingMessages` property: `!order.OutgoingMessages.ContainsKey(context.MessageId)` - Change the _mark as processed_ statement (`order.ProcessedMessages.Add(context.MessageId);`) in the `AddItemHandler` to use the `OutgoingMessages` property: `var outboxState = new OutboxState(); order.OutgoingMessages.Add(context.MessageId, outboxState)` - Notice that this statement can be moved to the `OutboxBehavior` just prior to the call to `next()`. Do that. Then put the `outboxState` into the context by doing `context.Extensions.Set(outboxState)` - Retrieve the `outboxState` in the `AddItemHandler` via `var outboxState = context.Extensions.Get();` - Replace the calls to `order.OutgoingMessages.Add` for publishing messages with `outboxState.OutgoingMessages.Add(new Message(id, payload))` -- Change the condition for dispatching outgoing messages to take into account the new structure of `OutgoingMessages`. Change `if (order.OutgoingMessages.Any())` to `order.OutgoingMessages[context.MessageId] != null`. Notice that we don't need to check if the value for the key exists because the structure code guarantees that. -- Replace the `order.OutgoingMessages` with `order.OutgoingMessages[context.MessageId].OutgoingMessages` in the `foreach` -- Replace the reference to `Value` with `Payload` and `Key` with `Id`. +- Change the condition for dispatching outgoing messages to take into account the new structure of `OutgoingMessages`. Change `if (order.OutgoingMessages.Any())` to `order.OutgoingMessages[context.MessageId] != null`. Notice that we don't need to check if the value for the key exists because the code structure guarantees that. +- Replace the `order.OutgoingMessages` with `order.OutgoingMessages[context.MessageId].OutgoingMessages` in the `foreach` statement +- Replace the reference to `Value` with `Payload` and `Key` with `Id` - Replace the `order.OutgoingMessages.Clear()` with `order.OutgoingMessages[context.MessageId] = null` to prevent removing the information about all processed messages - You can now safely remove the `ProcessedMessages` property and all its references. You can comment out the `RemoveItemHandler`. You can deal with that one later as an extra exercise. -Run the solution to check if it works. Now it is time for the last final step -- store serialized messages with headers in the outbox. In order to do that we will take advantage of an extensibility point within NServiceBus that allows us to intercept outgoing messages from a behavior in the pipeline. We will use the `Dispatch` and `InvokeMessageHandler` helper methods. +Run the solution to check if it works. Now it is time for the last final step -- storing serialized messages with headers in the outbox. In order to do that, we will take advantage of an extensibility point within NServiceBus that allows us to intercept outgoing messages from a behavior in the pipeline. We will use the `Dispatch` and `InvokeMessageHandler` helper methods. -- In the `OutboxState` replace the `Message` class with built in `NServiceBus.Transport.TransportOperation` and make it an array instead of a list. +- In the `OutboxState`, replace the `Message` class with the built-in `NServiceBus.Transport.TransportOperation` and make it an array instead of a list. - Replace the call to `next()` with a call to `InvokeMessageHandler`. Assign the returned value to `order.OutgoingMessages[context.MessageId].OutgoingMessages` - Replace the `forach` loop with a call to `Dispatch`. - Replace the calls to `outboxState.OutgoingMessages.Add` in the `AddItemHandler` to `context.Publish`. Drop the ID. The framework will generate one for you. Remember to `await` this call. -- You can remove the `outboxState` from the `AddItemHandler` now. That's quite an achievement! There is no more deduplication logic in the message handler! +- You can remove the `outboxState` from the `AddItemHandler` now. + +- That's quite an achievement! There is no more deduplication logic in the message handler! From 1810257e9df78115c870f8bec9056720f54ed48f Mon Sep 17 00:00:00 2001 From: lailabougria Date: Tue, 22 Mar 2022 16:19:18 +0100 Subject: [PATCH 24/24] ed --- NewExercises/Exercise-18/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/NewExercises/Exercise-18/README.md b/NewExercises/Exercise-18/README.md index 860b480..d41e068 100644 --- a/NewExercises/Exercise-18/README.md +++ b/NewExercises/Exercise-18/README.md @@ -1,14 +1,14 @@ # Exercise 18: Inbox -In the previous exercise we have implemented a generic version of the Outbox pattern. We quickly discussed its downsides and agreed that we can do better. In this exercise we will introduce a concept of Inbox as a shared storage of deduplication information. +In the previous exercise, we have implemented a generic version of the Outbox pattern. We quickly discussed its downsides and agreed that we can do better. In this exercise we will introduce the concept of an Inbox as a shared storage of deduplication information. Fortunately for us, the changes won't be big. - First, notice that the `OutboxBehavior` class has an additional field -- the `inboxStore`. As discussed, messages that have been completely processed should have entries in that store. - Let's add the code that checks the inbox store. Where should we insert this code? To check the inbox call `HasBeenProcessed` method passing the message ID. -The whole middle section of the algorithm remains unchanged. Let's take a look at the bottom part. Where should we insert the `MarkProcessed` call to mark add an entry to the inbox? +The whole middle section of the algorithm remains unchanged. Let's take a look at the bottom part. Where should we insert the `MarkProcessed` call to add an entry to the inbox? -Now let's focus on the cleanup. The goal was to avoid holding on to too many data inside the entity. To achieve it we need to replace the `OutboxState[context.MessageId] = null` code with simple `OutboxState.Remove(context.MessageId)`. +Now let's focus on the cleanup. The goal was to avoid holding on to too much data inside the entity. To achieve it we need to replace the `OutboxState[context.MessageId] = null` code with simple `OutboxState.Remove(context.MessageId)`. Now take a look at the `outboxState != null` condition. Do we still need it?