Event Driven .NET

An event-driven microservices platform for .NET

Introduction

Microservices is a popular paradigm for building highly scalable and flexible distributed systems based on Cloud computing. While the what of microservices is fairly unambiguous, the how is replete with pitfalls and perils. Many teams embark on the road of good intentions only to find themselves mired in a quicksand of tightly coupled systems -- the distributed monolith -- with few of the benefits but most of the difficulties of a traditional monolith. Much of this stems from using synchronous REST calls as the primary means of inter-service communication, which can result in systems that are brittle and inflexible, teams that block one another, and services that must be deployed and scaled as a single unit.

The purpose of Event Driven .NET is to provide a platform where .NET developers can build loosely coupled distributed systems consisting of services that can be deployed and scaled independently, replacing point-to-point communication with an event bus abstraction that promotes asynchronous communication using a message broker.

UI Clients, other domains and B2B partners interact synchronously and asynchronously with the Domain Edge by means of REST, GraphQL or events. Within a Domain, read and write services interact with one another using an Event Bus, and they perform updates to data stores which are private to each service. The Event Bus supports idempotency with an event cache that allows filtering of duplicate messages, and it uses a schema registry to ensure that events for the same topic are compatible with registered schemas. Sagas use the Event Bus to orchestrate updates across services so that they all succeed or are rolled back by means of compensating actions. Data analytics and AI / ML ingest events to update their respective data stores.

The event bus provides a pub-sub abstraction for asynchronous inter-service communication. Dapr provides application-level building blocks for state management, pub/sub messaging, actors, etc. The message broker transports messages between publishers and subscribers. Examples of supported message brokers are Amazon SNS+SQS, Azure Service Bus or Apache Kafka.

While Dapr provides some capabilities of a service mesh, such as inter-service encryption, monitoring, observability and delivery retries, it operates at the application level, whereas a service mesh operates at the networking level and provides features such as routing and traffic splitting. Dapr can be used in combination with a service mesh.

Kubernetes is an open-source system for automating deployment, scaling and management of containerized applications by grouping containers into units that can be discovered and managed. It hosts plugins for storage, networking and scheduling, and it serves as a foundation for container-based microservices.

Along side the microservices infrastructure stack are a number of cross-cutting concerns, such as monitoring and observability, security and privacy, feature flags, testing and deployment.

Strategy

The mission of Event Driven .NET is to furnish a multi-layered platform upon which .NET developers can build distributed applications that deliver on the promise of microservices by enabling multiple teams working in parallel to deliver features to customers with greater speed and reliability.

Rather than building a high prescriptive framework that locks developers into one way of building microservices, Event Driven .NET includes a set of abstractions that can be implemented in any number of ways. For convenience some default implementations are supplied so that developers can get started right away building event-driven microservices. But rather than attempting to do everything for the developer, Event Driven .NET instead provides a set of reference architectures which teams can use as examples for building distributed systems which reflect their organization's business needs and operational maturity.

Layered Approach

Event Driven .NET is designed to allow you to wade gradually into the waters of event-driven microservices, so that your system architecture aligns with your organizational structure, as well as the skill set of your developers and the maturity of your DevOps processes.

Starting at the bottom, you adopt a Domain Driven Design (DDD) approach, then build on this foundation by adding Command Query Responsibility Segregation (CQRS). When you have the need for inter-service communication, you can move to the Event Bus layer, utilizing Dapr as a pub-sub abstraction over an underlying message broker.

If you have the need to maintain eventual data consistency with updates to multiple services spanning a logically atomic operation, you are ready to advance to the next layer: Sagas. When your organization has the need for capabilities such as built-in audit trail and the ability to atomically persist and publish events, you may wish to move to the Event Sourcing layer (to be implemented).

Lastly, if you wish to perform real-time data analysis and transformation, you may wish to explore Event Streams (to be implemented).

Abstractions

Each layer is supported by a set of abstraction distributed as NuGet packages.

Note: EventDriven.CQRS.Abstractions version 2.0 or later uses MediatR to enable a handler per command pattern with behaviors for cross-cutting concerns.

• Domain Driven Design
  • EventDriven.DDD.Abstractions: Abstractions for implementing Domain Driven Design.
• Command Query Responsibility Segregation
  • EventDriven.CQRS.Abstractions: Abstractions for implementing Command Query Responsibility Segregation.
• Dependency Injection
  • EventDriven.DependencyInjection: Helper methods for configuring services with dependency injection.
• Event Bus
  • EventDriven.EventBus.Abstractions: Generic event bus abstraction.
  • EventDriven.SchemaRegistry.Abstractions: Generic schema registry abstraction.
• Sagas
  • EventDriven.Sagas.Abstractions: Abstractions for sagas, steps, actions, commands, dispatchers, handlers and evaluators.
  • EventDriven.Sagas.Configuration.Abstractions: Abstractions for saga configurations.
  • EventDriven.Sagas.EventBus.Abstractions: Abstractions for handling integration events and dispatching command results.
  • EventDriven.Sagas.Persistence.Abstractions: Abstractions for persisting saga snapshots.

Libraries

Each layer is supported by a set of abstraction distributed as NuGet packages. Simply reference the appropriate package in your project to utilize its interfaces and classes.

• Command Query Responsibility Segregation
  • EventDriven.CQRS.Extensions: Extensions for implementing Command Query Responsibility Segregation.
• Dependency Injection
  • EventDriven.DependencyInjection.Mongo: Helper methods for configuring services for MongoDB with dependency injection.
• Event Bus
  • EventDriven.EventBus.Dapr: Event bus abstraction over Dapr pub/sub.
  • EventDriven.EventBus.EventCache.Mongo: MongoDB implementation of event caching.
  • EventDriven.EventBus.EventCache.Redis: Redis implementation of event caching.
  • EventDriven.SchemaValidator.Json: JSON implementation of a schema validator.
  • EventDriven.SchemaRegistry.Mongo: MongoDB state store for validating messages against schemas that are stored in a registry by topic name.
• Sagas
  • EventDriven.Sagas.Configuration.Mongo: MongoDB implementation for saga configuration repositories.
  • EventDriven.Sagas.DependencyInjection: AddSaga service collection extension methods.
  • EventDriven.Sagas.Persistence.Mongo: MongoDB implementation for persisting saga snapshots.

Reference Architectures

To aid you in implementing an event-driven microservices architecture, Event Driven .NET includes reference architectures, which utilize the abstractions listed above to provide a working example of how to structure a microservices solution based on event-driven architecture.

• EventDriven.ReferenceArchitecture: Reference architecture for using EventDriven abstractions and libraries for Domain Driven Design (DDD), Command-Query Responsibility Segregation (CQRS) and Event Driven Architecture (EDA) with Dapr Event Bus.

• EventDriven.Sagas: Abstractions and reference architecture for implementing the Saga pattern to orchestrate atomic operations which span multiple services.

Getting Started

To get started with Event Driven .NET, take the following steps to create a .NET Web API project that applies the principles of Domain Driven Design, Command Query Responsibility Segregation and Event Driven Architecture.

1. Using a process such as Event Storming, define a business domain model with separate bounded contexts and aggregate roots.
   • While it may be possible to scope microservices at the level of a bounded context, it is often preferable to create one microservice per aggregate root.
2. Create a .NET Web API project with the following packages:
   • MongoDB.Driver
   • URF.Core.Mongo
   • AutoMapper.Extensions.Microsoft.DependencyInjection
   • EventDriven.DependencyInjection.URF.Mongo
   • EventDriven.CQRS.Abstractions
   • EventDriven.EventBus.Dapr
   • EventDriven.EventBus.Dapr.EventCache.Mongo
3. Using Event Driven Reference Architecture as a guide, follow the Development Guide to add domain entities, commands and events.
   • Use CustomerService in the reference architecture as an example.
4. Implement ICommandProcessor and IEventApplier interfaces on the aggregate root entity to add Process and Apply methods.
   • The Process method returns one or more domain events.
   • The Apply method accepts a domain event and contains business logic for updating the entity state.
   • This pattern enables the addition of Event Sourcing at a later stage while minimizing the need for refactoring entities.
5. Define repository interfaces and implementations to handle persistence concerns.
   • The reference architecture utilizes MongoDB for persistence, but if you prefer feel free to use a SQL database and ORM, such as Postgres or SQL Server with Entity Framework Core.
6. Add a command handler to process commands, apply events and persist entities.
7. Add Data Transfer Objects (DTO's) and an AutoMapper profile, so that controllers can accept and return DTO's instead of entities.
8. Add query and command controllers.
   • Query controllers can accept a repository interface.
   • Command controllers can accept a command handler.
9. Create an Integration class library project with C# records that extend IntegrationEvent and include models.
   • Use the Common project as an example.
   • Add the following package: EventDriven.EventBus.Abstractions.
10. Update the command handler to accept an IEventBus and publish integration events so that subscribing projects may receive notifications of events.
    • Reference the Integration project from the Web API project.
11. Repeat the previous steps to create other services based on different aggregate roots.
    • Use OrderService in the reference architecture as an example.
    • Reference the Integration project to use integration events.
    • Add integration event handlers with classes that extend IntegrationEventHandler.
    • Update Program to map integration event handlers to event bus subscribers.

Roadmap

The roadmap for Event Driven .NET includes the following items:

• Event Sourcing: This will provide the ability to treat domain events as the source of truth for a system, so that services no longer need to treat persistence and event publishing atomically. It also provides built-in audit trail.
• Event Streams: This will support real-time data analysis and transformation by means of a durable, append-only message broker, such as Apache Kafka or Amazon Kinesis.

Contributing

Contributions from the community are welcome!

To ask a question, report a defect or propose an enhancement, please open an issue in the appropriate repository. If you are unsure of which repository is appropriate, or your issue is general in nature, please open an issue in the Event Driven .NET Home repository.

If you wish to contribute a change to any repository, please refer to the Contributing Guidelines. Before creating a pull request, please open an issue in the repository where you wish to propose a change.