Concurrency Design Patterns in Golang

In this article, I will introduce 12 concurrency design patterns, including Select, For-Select-Done, Future, Barrier, Pipeline, Worker Pool, Fan-Out Fan-in, Producer-Consumer, Semaphore, Rate limit, Circuit Breaker, and Publish-Subscribe.

These patterns deal with the efficient and safe execution of multiple operations or tasks simultaneously. Golang has native support for concurrency through its Goroutines and Channels, making it a popular language for concurrent programming.

1. Select Pattern

The select pattern in Go is a control flow construct that allows a goroutine to wait on multiple channel operations and execute different logic depending on which channel is ready first. This pattern is often used to implement timeouts, to multiplex input from multiple channels onto a single channel, or to implement non-blocking channel operations. The select pattern is a powerful tool for managing concurrent operations in Go and is a key part of many concurrency patterns. For more details please click here

2. For-Select-Done Pattern

The For-Select-Done pattern is a common idiom in Go that combines the for loop with the select statement to continuously listen for events until a "done" signal is received. This pattern is frequently used to handle asynchronous tasks in a non-blocking manner. For more details please click here

3. Future Pattern

The future pattern in Golang involves starting a goroutine to compute a value that will be needed in the future, and immediately returning a channel on which the computed value will be sent when ready. This pattern allows for computation to proceed concurrently with other tasks. The returned channel is a promise that the value will be available in the future, hence the name “future pattern”. This pattern is useful in scenarios where you have a computationally expensive operation that you don’t need the result of immediately, and you don’t want to block the current goroutine. For more details please click here

4. Barrier Pattern

The barrier pattern is a synchronization pattern that can be used to make goroutines wait for each other. It is often used in parallel algorithms that involve splitting a problem into independent sub-problems that can be solved concurrently. When a goroutine finishes its sub-problem, it reaches the barrier and waits. When all goroutines have reached the barrier, they are all released and can proceed. This pattern is useful in scenarios where you need to ensure that a phase of computation is completed before proceeding to the next phase. For more details please click here

5. Pipeline Pattern

The pipeline pattern in Go is a powerful tool for creating a chain of processing elements arranged so that the output of one element is the input of the next. This pattern is often used in data processing to break up a task that operates on a sequence of data into a series of individual stages that can run concurrently. Each stage is a group of one or more goroutines that take input from and send output to channels. The pipeline pattern can increase efficiency and performance in programs that involve a series of operations that can be executed concurrently. For more details please click here

6. Worker Pool Pattern

The worker pool pattern involves a collection of goroutines (workers) that are all listening on a single task channel. This pattern is useful for limiting the number of concurrently executing goroutines and controlling the concurrency level. It’s often used in scenarios where you have a large number of tasks to be processed and you want to limit the number of tasks being processed at the same time. For more details please click here

7. Semaphore Pattern

The semaphore pattern is a synchronization primitive used in concurrent programming to control access to a shared resource. It’s a fundamental concept in operating systems and multithreaded programming. While Go doesn’t have a built-in semaphore type, it provides a Semaphore implementation using channels, which are first-class citizens in Go's concurrency model. For more details please click here

8. Producer-Consumer Pattern

The producer-consumer pattern is a classic concurrency pattern where one or more goroutines (producers) generate data and one or more goroutines (consumers) use the data. In Go, this is typically implemented using goroutines for the producers and consumers, and channels for passing data between them. This pattern is useful for decoupling the production of data from its consumption, which can help to balance the load and manage back-pressure in systems that deal with streams of data. For more details please click here

9. Fan-Out Fan-in Pattern

The fan-out pattern involves starting multiple goroutines to handle input from a single channel concurrently. This can increase throughput and allow for computational distribution. The fan-in pattern involves merging multiple results into a single channel. This can be useful when you have multiple goroutines performing similar work and you want to aggregate their results into a single stream of data. These patterns are often used together in Go to manage and coordinate multiple goroutines. For more details please click here

10. Rate Limit Pattern

Rate limit is a technique for controlling the rate of actions such as requests to a server, method calls, or packet transmission. In Go, this can be implemented using a ticker (a time.Ticker object) and a buffered channel. The ticker ticks at regular intervals, and each tick represents a “permit” to act. The channel buffers these permits up to a certain capacity, representing the maximum burst size. For more details please click here

11. Circuit Breaker Pattern

The circuit breaker pattern is a design pattern used in modern software development to detect failures and encapsulates the logic of preventing a failure from constantly recurring, during maintenance, temporary external system failure or unexpected system difficulties. It’s used to prevent an application from performing an operation that’s likely to fail. If the operation is likely to fail, the circuit breaker returns an error immediately, without attempting the operation. For more details please click here(Coming soon…)

12. Pub-Sub Pattern

The pub-sub pattern is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers. Instead, published messages are characterized into structs, without knowledge of what, if any, subscribers there may be. Similarly, subscribers express interest in one or more structs, and only receive messages that are of interest, without knowledge of what, if any, publishers there are. This pattern is often used in distributed systems to decouple the senders of messages from the receivers of messages, allowing for greater scalability and complexity. For more details please click here

Scenarios

Different patterns always used in different scenarios.

1. Coordinate multiple goroutines with the Worker Pool pattern and Semaphore pattern.
2. Create a series of processing stages with the Pipeline pattern.
3. Handle multiple channel operations with the Select pattern and For-Select-Done pattern.
4. Increase throughput with the Fan-Out Fan-in patterns.
5. Manage data production and consumption with the Producer-Consumer pattern.
6. Control the rate of actions with the Rate Limit pattern.
7. Prevent constant failure recurrence with the Circuit Breaker pattern.
8. Compute future values with the Future pattern.
9. Synchronise goroutines with the Barrier pattern.
10. Broadcast messages to interested parties with the Publish-Subscribe pattern.

Conclusion

Concurrency patterns in Go, such as the Select, For-Select-Done, Future, Barrier, Pipeline, Worker Pool, Fan-Out Fan-in, Producer-Consumer, Semaphore, Rate limit, Circuit Breaker, and Publish-Subscribe patterns, provide powerful tools for managing concurrent execution and data flow in Go programs.

Click here to go back to Design Patterns in Golang

To view Creational Design Patterns in Golang, please click here.

To View Structural Design Patterns in Golang, please click here.

To View Behavioural Design Patterns in Golang, please click here.

To View Concurrency Design Patterns in Golang, please click here.

To View Microservices Design Patterns in Golang, please click here.