Building Microservices: Using Node.js with DDD, CQRS, and Event Sourcing — Part 2 of 2

tl;dr: I aim to help you learn and apply CQRS and event sourcing using a modern approach.

Part 1:

Part 2:

The big picture

Building microservices requires thinking about the holistic view. It is important to understand the topologies that make up the microservice architecture, as well as the rules that are implied. What does that mean? One way to think of it is to understand all domains within a given application and subdivide and conquer by service boundaries.

Microservices should completely align with their bounded contexts that represent a given domain — and are to be loosely-coupled and highly cohesive.

In Part 1, we discussed the DDD, CQRS, and event sourcing patterns that are used in my example project. Hopefully, you had some time to check out the GitHub repository and clone it as well. In this part, we will cover the implementation in detail — I also recommend you go beyond the scope of this article and learn more about the concepts applied here as well as other related concepts.

Let us break down the project’s microservice architecture with a flow chart:

• the domain model implements an aggregate root and sits at the heart of a microservice —it also validates and persists events on behalf of the aggregate using command handlers
• events are atomically persisted to the event store before the handling of a given event

As an added benefit of using DDD, we can create easy to understand directory tree structures. Let us take a look at the project’s directory tree…

Note: Create a new file called .env in the root directory by copying the given .env.example file.

Node.js was picked as the technology of choice since Node.js follows a modularized approach to building applications — it fits well with the nature of microservices as well as the Unix philosophy:

Small is beautiful. Make each program do one thing well. (source)

Modularization is just one component of building ever-lasting applications that extend the curve of Moore’s law. Applying best practices and good design patterns is equally as important — from the perspective of a developer and the compiler, it should be noticable that care was taken.

Nest.js was picked as the framework of choice since the Nest.js framework libraries follow best practices and promote good design patterns such as dependency injection. These patterns improve code quality and maintainability — and allow for evolvability. Check out Nest’s documentation and learn about its fundamentals before continuing.

Moving on, let us jump through some code now!

Here is what the AppModule that imports the UsersModule looks like…

Note: The AppModule is used to register all modules used by the microservice. In my project, the modules that are applied include UsersModule and EventStoreModule.

Easy right? Now let’s see the UsersModule…

Note: The UsersModule is used to bootstrap all Users domain-specific logic.

User Requests

As of the time of writing this article, Fastify handles approx. 76,835 req/sec and Express handles approx. 50,933 req/sec. Fastify has been implemented in my project since we want the maximum throughput possible. You can easily change from Fastify to any other router framework of choice.

Living documentation is at the heart of the controllers implemented in the UsersModule! This is neat since you save a lot of time by streamlining the process of creating and maintaining microservice API documentation.

The user creation flow is the most involved in this project so let’s take a look at a quick flow diagram of the process of creating a new user…

Users trigger commands via user interfaces. Upon the UserCreatedEvent event being captured, a userCreated saga has been implemented to continue the transaction of welcoming the user as a way of onboarding the user.

Commands — implementation & handlers

All changes to an aggregate must be preceded by a command. Upon a command being dispatched, an associated command handler validates and resumes a given request — and if successful, attempts to persist new events to the event store.

Commands are created by the client application and then sent to the domain layer. Commands are messages that instruct a specific entity to perform a certain action. (source)

Note: A command implementation is used to construct a command. Implementations should be easy to understand — even to non-developers.

Note: A command handler is used to manage a command request. Notice that we need to commit in order to dispatch an event.

Aggregate root — domain models validate business

The domain model is where domain rules are defined. Users are to be described as an aggregate instead of a domain model — in DDD, an aggregate is a cluster of domain objects that can be treated as a single unit. Remember, domain model objects in our case are immutable and therefore aggregate.

The aggregate forms a tree or graph of object relations. The aggregate root is the “top” one, which speaks for the whole and may delegates down to the rest. It is important because it is the one that the rest of the world communicates with. (source)

Note: The User model simply fires relevant events.

Note: The Users repository uses the Users model to persist and handle events.

Storing events — optimistic locking & idempotency

Concurrency, in the context of event sourcing, is an important thing to be aware of. With distributed command handlers, concurrent access to or the modification of aggregates is not always a problem — you can implement some sort of conflict resolution if it is.

Optimistic locking was consciously chosen as the locking mechanism in my project — it is important to learn about the trade-offs of using optimistic vs. pessimistic locking mechanisms with event sourcing.

Event Store uses the following ports:

http://localhost:1113 (tcp)
http://localhost:2113 (http)

All HTTP requests made to Event Store are idempotent — unless the expected version is ignored.

Publishing events to the event store are made easy by the following code:

In the project’s event-store.ts file, it contains logic for the Event Store bridge which subscribes to any events in a given domain category stream. In this case, the category stream is the Users domain — it uses a category projection called $ce-users.

Event Store comes with an interface out-of-the-box. Anytime an event occurs, the interface will allow us to know of and interact with the event in real-time.

Event persistence and streaming are made atomic thanks to the design behind the event store. Try it out, make a request to create a new User and monitor Event Store and see what happens.

Events are implemented with handlers — similar to commands.

Note: An event implementation is used to construct an event.

Note: An event handler is used to manage an event.

An event handler should be responsible for writing to materialized state or cache for faster reads.

Finally, the heroic User saga continues the User creation transaction…

After the UserCreatedEvent event is stored, the system fires the UserWelcomedEvent event triggered by the userCreated saga — the user creation transaction is completed thereafter.

Reading materialized data

Summary of user commands

1. a command is an object sent by the user via a user interface
2. domain models define rules for applying changes to a domain aggregate
3. command handlers validate before dispatching events

Summary of user queries

1. materialized data in the database is created by the denormalization of events data
2. replaying of events to recreate data is possible using projections

Running the project

$ ./scripts/up.sh            ## to boot up
$ ./scripts/down.sh          ## to shut down

Conclusion

Events allow us to focus on the domain instead of database schemas, cross-team communication is made easier because of it.

We have just completed the overview of building a decent starter microservice while explaining the concepts of DDD, CQRS, and event sourcing. Clone the GitHub repository and play with the project yourself.

Remember that all patterns have pros and cons and the CQRS and event sourcing patterns are not exempt from that — by no means a silver bullet.

in the real world

• external applications can also react to the events of a microservice

some use cases of CQRS/ES include

• auditing systems
• stock trading platforms
• licensing platforms
• database systems

As I am always learning and experimenting, I do not consider my project to be complete. As a way of thanks to all of those who have made contributions to OSS (open source software) technologies, I wanted to share my journey of building this project.

Below, you will find a link to the project as well as my contact info.

• Project’s GitHub
• Qasim Soomro’s website
• Qasim Soomro’s LinkedIn
• Qasim Soomro’s AngelList
• Qasim Soomro’s Twitter

Resources

1. Martin Fowler — CQRS (article)
2. Greg Young — Event Sourcing (video)
3. MSDN — CQRS Journey (article)
4. Eric Evans — Domain-Driven Design, Tackling Complexity in the Heart of Software (book)
5. Christopher Richardson — Microservice Patterns (book)
6. PayPal — API Guidelines (website)
7. Kanasz Robert — Introduction to CQRS (source)
8. Werner Vogels — Eventually Consistent (source)
9. CQRS.nu — FAQ (source)

Going further

Fork the project and add your own custom denormalization logic — remember to share your fork in the comments section below! You can also try and replay the events to construct the view if you prefer.

Read related books and other material, learn about the different diagramming methods for designing microservices using the links below, create your own event store implementation, and learn to use an API gateway to protect your microservices.

Share your thoughts

Feel free to leave a question, comment, or share your fork below!

Ad