Free AI web copilot to create summaries, insights and extended knowledge, download it at here

2983

Abstract

uplication using the version and aggregate Id.</figcaption></figure><p id="4e45">What you do after you detect this situation will depend on your application but at least you have a way to identify it happened.</p><p id="a27e">Now how to choose the version?</p><h1 id="61ca">Using Timestamp</h1><p id="a8aa">A simple and apparently common approach is to set the version as a timestamp. This approach has its merits as two consecutive timestamps are different and enable both the duplicate and the out-of-order detection.</p><figure id="d9e1"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/0*iX4ESlaJY__V11R9"><figcaption>Figure 4. The timestamp is used as the version. Newer events will have higher timestamps than old ones.</figcaption></figure><p id="62f5">But this approach is not without its limitations, so it is important to know and decide if they are relevant to you or not.</p><p id="b00a"><b>- Second/Millisecond precision</b></p><p id="0f7e">Many timestamps expose values at the second or millisecond precision. This means that if the same entity evolves two or more times within the same precision interval, those different events would have the same version.</p> <figure id="a987"> <div> <div>

            <iframe class="gist-iframe" src="/gist/bicatu/821f1a49f354eca78cb8cb78a01acac1.js" allowfullscreen="" frameborder="0" height="undefined" width="undefined">
          </div>
        </div>
    </figure></iframe></div></div></figure><p id="2206">In the above example, if the same entity mutates within the same second, two different events would end up with the same value.</p><p id="471a"><b>- Daylight Savings/Time Adjustments</b></p><p id="3053">Many countries adjust the clock twice a year, such as when daylight saving time (DST) begins and ends.</p><p id="7a6c">During these time adjustments, and depending on the implementation, subsequent events could contain a timestamp that is smaller than the first. Or even, in a very bad luck situation, end up with two different events having the same version.</p><figure id="9d6c"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/0*RtsGe-POU5mmcNJY"><figcaption>Figure 5. Adjusting the clock can lead to new events having older versions associated with it.</figcaption></figure><p id="4271"><b>- Clock skews between servers</b></p><p id="a93d">Your application can be executed simultaneously from different servers, and each one may use its internal clock to determine the current date. Those clocks are not guaranteed to be in sync and a drift can happen, opening the door for the same issues described with the precision topic.</p><p id="cd58"><b>- Harder to detect missing events</b></p><p id="2abe">The timestamp-based version is limited when comes down to detect if an event was lost.</p><figure id="2a80"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/0*hkBul52vy40GSW02"><figcaption>Figure 6. Sin

Options

ce the versions are meeting the criteria newer events have bigger version numbers, no problems are detected.</figcaption></figure><p id="4b7a">As the example indicates, I lost event 2 but the consumer does not actually have a way of knowing as event 3 has a bigger version than event 1. It all seems fine.</p><p id="9849">While many applications may not be susceptible to the aforementioned limitations, let’s see a different approach.</p><h1 id="0ddd">Increase-by-One</h1><p id="f173">This approach is also simple and involves simply setting for each entity the initial version and adding 1 to it as the entity successfully mutates.</p><figure id="fd87"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/0*rcG9iy2MJO9NAHdt"><figcaption>Figure 7. When creating entities, assign a version to it.</figcaption></figure><p id="7839">Then when you need to update the entity, you load the current state and as you persist it the expected version is inc of 1</p><figure id="4115"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*XY2yyiEoIB67GGS0ELvapg.png"><figcaption>Figure 8. When making updates, load the current version prior to updating it with a +1.</figcaption></figure><p id="118b">One of the side benefits of this approach is that you can get an optimistic lock, preventing simultaneous changes from overriding each other.</p><p id="7d0b">This opens the possibility for us to detect missing events because of the predictable fashion the version will evolve.</p><figure id="cb60"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/0*UZyTeWGVX9kOU__g"><figcaption>Figure 9. If the consumer wants it can detect missing events by inspecting the received vs expected version.</figcaption></figure><h1 id="d930">Wrapping Up</h1><p id="5c44">Event versioning is another of the many decisions you will have to make when going for an event-driven approach.</p><p id="b840">So far the two choices I have seen, and used, are the timestamp or the increase-by-one. While both are valid, I see the timestamp-based to be somewhat prevalent, even though it has limitations that can often be overlooked.</p><p id="111e">Look at your application and pick the one that fits your needs.</p><p id="0515">If you choose the increase-by-one, may I also interest you in the <a href="https://readmedium.com/event-sourcing-part-i-understanding-what-it-is-core-components-and-common-misconceptions-5dec65f6d61?source=friends_link&sk=fd14584bedb148f680849c263be82f02">event sourcing pattern</a> and the <a href="https://readmedium.com/implementing-a-transactional-outbox-pattern-with-dynamodb-streams-to-avoid-2-phase-commits-ed0f91e69e9?source=friends_link&sk=f729b6814fc3bfe93e5d367720404cf0">transactional outbox</a>? They are definitely not mandatory, but they fit nicely with this approach as well.</p><p id="d48c">Share which approach you have been using in the comments section and if you have ever felt any issues while using it.</p></article></body>

Event Versioning: Pros and Cons of the Most Common Options

You have decided to adopt an event-driven architecture, modeled your events, and decided what style of event you will expose. One of the aspects that I recommended was for you to add to the events the version of the entity after it mutated.

Figure 1. Sample event with its data and metadata

While interacting with other developers, the most common questions I see are:

- Why is it really important to add the version?

- How to generate the version value to be added?

Let’s cover both in this article, with an emphasis on discussing two alternatives to the generation strategy.

Why it is Important to Add the Version to the Event

Event-driven architecture is a powerful approach that touts the scalability and decoupling of the systems involved. I have covered in the past that it is not without its perils but those claims are largely true and can provide you with the foundation your application needs to grow.

Common issues with events can be seen below:

Figure 2. Common issues when consuming events.

Losing a message — Even though the producer emitted the message it never arrives at the consumer side

Out-of-order messages — Events arrive in a different order than those produced

Duplicate messages — The consumer receives the same message more than one time

While a full discussion on these topics is the subject of another article, let’s focus on the last two with the version in context.

In the case of out-of-order or duplicate messages, you have in common that the events will arrive at the destination, but not in the way we would expect. So your consumer needs to have a way to identify that this is the case and handle it accordingly.

Having a version attached to the event can help in both cases:

Figure 3. Detecting duplication using the version and aggregate Id.

What you do after you detect this situation will depend on your application but at least you have a way to identify it happened.

Now how to choose the version?

Using Timestamp

A simple and apparently common approach is to set the version as a timestamp. This approach has its merits as two consecutive timestamps are different and enable both the duplicate and the out-of-order detection.

Figure 4. The timestamp is used as the version. Newer events will have higher timestamps than old ones.

But this approach is not without its limitations, so it is important to know and decide if they are relevant to you or not.

- Second/Millisecond precision

Many timestamps expose values at the second or millisecond precision. This means that if the same entity evolves two or more times within the same precision interval, those different events would have the same version.

In the above example, if the same entity mutates within the same second, two different events would end up with the same value.

- Daylight Savings/Time Adjustments

Many countries adjust the clock twice a year, such as when daylight saving time (DST) begins and ends.

During these time adjustments, and depending on the implementation, subsequent events could contain a timestamp that is smaller than the first. Or even, in a very bad luck situation, end up with two different events having the same version.

Figure 5. Adjusting the clock can lead to new events having older versions associated with it.

- Clock skews between servers

Your application can be executed simultaneously from different servers, and each one may use its internal clock to determine the current date. Those clocks are not guaranteed to be in sync and a drift can happen, opening the door for the same issues described with the precision topic.

- Harder to detect missing events

The timestamp-based version is limited when comes down to detect if an event was lost.

Figure 6. Since the versions are meeting the criteria newer events have bigger version numbers, no problems are detected.

As the example indicates, I lost event 2 but the consumer does not actually have a way of knowing as event 3 has a bigger version than event 1. It all seems fine.

While many applications may not be susceptible to the aforementioned limitations, let’s see a different approach.

Increase-by-One

This approach is also simple and involves simply setting for each entity the initial version and adding 1 to it as the entity successfully mutates.

Figure 7. When creating entities, assign a version to it.

Then when you need to update the entity, you load the current state and as you persist it the expected version is inc of 1

Figure 8. When making updates, load the current version prior to updating it with a +1.

One of the side benefits of this approach is that you can get an optimistic lock, preventing simultaneous changes from overriding each other.

This opens the possibility for us to detect missing events because of the predictable fashion the version will evolve.

Figure 9. If the consumer wants it can detect missing events by inspecting the received vs expected version.

Wrapping Up

Event versioning is another of the many decisions you will have to make when going for an event-driven approach.

So far the two choices I have seen, and used, are the timestamp or the increase-by-one. While both are valid, I see the timestamp-based to be somewhat prevalent, even though it has limitations that can often be overlooked.

Look at your application and pick the one that fits your needs.

If you choose the increase-by-one, may I also interest you in the event sourcing pattern and the transactional outbox? They are definitely not mandatory, but they fit nicely with this approach as well.

Share which approach you have been using in the comments section and if you have ever felt any issues while using it.