Write Robust For Loops With Go 1.22
Let’s Encrypt (and you) will never break the code again
For loops can be a source of bugs in Go. At least, they were.
Even smart guys, like “Let’s Encrypt” engineers, have been misled by loops in Go.
In this article, I want to write about how loops can sometimes be deceitful (even for super smart guys) and how Go 1.22 has improved 10x our development activities (and our stress level). Thanks, Go Devs! ❤
If you want to experiment with the problem and the solutions explained in this article, I have created a GitHub repository (the link is at the end of this article) that you can clone to play around with for loops in different versions of Go.
Loops have a problem!
One day, a Let’s Encrypt engineer decided to write these lines of code:
They look like innocent lines of code:
- Iterate map
m; - Get the key and value of each item and assign them to
kandv; - Call the
modelToAuthzPBfunction at every iteration and passvaddress.
Sometimes, the most straightforward things can be surprising. This is the case. Let us analyze deeply what is in the code.
In this snippet, k and v are “loop variables”. They are created, exist, and die in the loop's scope. Go assigns these variables a “per-loop” scope, meaning their values are updated at each iteration.
Nothing surprising: it is a loop variable, it should change!
Then, they pass a reference to v to the function modelToAuthzPB. That is, the function uses the address of a loop variable.
This should already ring a bell. They are passing the address of something that will change its value. But we know that Go has escape analysis, so it could work. Let’s go ahead.
It does not say much about a possible bug by itself. To tell if it is a bug, we have to check the code of the modelToAuthzPB function.
Let us suppose that another engineer implemented the function like this:
And they want to store a list of these structures for later use.
This code will break.
What happens
- The
IdentifierandRegistrationIDare pointers to a field ofv; - This variable, in turn, is a pointer to a value in the map;
- Since loop variables have a “per-loop” scope, the value addressed by
vwill change at every iteration.
The fields in the structure created inside the function are pointers to v . You remember that v will change its pointed value, right? After the loop, v will point to the last element of m in the loop ( m is a map, so it is not guaranteed to be traversed in the order of definition).
When the returned structure is used later in the code, it will inevitably point to the last (visited) one instead of pointing at its “corresponding” value in the map.
Instead of having N structures with different fields, they had N with the same Identifier and RegistrationID (the last visited one in the map).
Boom! 💥
How to fix it
Before Go 1.22
Do not worry! There is a simple solution even before Go 1.22.
In fact, one way of fixing this bug is to define a copy of the variable inside every iteration and then pass it to the function.
It is as simple as this:
Creating a copy of the variable v will change the scope of the passed argument from “per-loop” to “per-iteration”.
vCopy is a variable created inside every loop iteration and will not change. At least, it would be destroyed and recreated with a different value.
Thanks to this and escape analysis, Go keeps a copy of this variable in the heap instead of destroying it at the end of every iteration.
So, when your code accesses the structure after the loop, it will find the correct values for the Identifier and RegistrationID fields!
With Go 1.22
Go 1.22 under the hood solves this issue in the same way.
Long story short, you will no longer need to explicitly define a copy of the variable. By default, Go 1.22 loop variables have a “per-iteration” scope.
So, you can write:
And your code will work as you expected.
Cool! 😎
Conclusions
In this article, we saw how even very smart engineers could be tricked by a deceitful feature of Go: for loops.
This has motivated Go to improve itself and introduce a more robust way of managing loop variables in Go 1.22.
Have you ever encountered this problem with your code?
Play With Loops!
If you want to play with these issues, I have made for you an example inspired by the Let’s Encrypt bug. You can find it here:
You will need Docker installed or something equivalent to switch from a Go 1.21 environment to a Go 1.22 one. Compare the output of the same code in the different versions of Go.
Incredible, right?
This is one of many cool features that Go 1.22 has brought. Check out this other article in which I explain how to write REST APIs with the newest version of Go. 👇
Join the Discussion!
If you have read the story until now, thanks! It means a lot to me. ❤️
Your experiences, insights, and opinions are invaluable.
👇 Drop your comments below, and don’t forget to follow me for more insights.
Feel free to connect with me on LinkedIn for further discussions ✍️
