avatarOrestis Meikopoulos

Summary

The provided content discusses the nuances of concurrency, parallelism, and asynchronous execution in C#, emphasizing the role of threads in optimizing application performance and resource utilization.

Abstract

The article "Unlocking the power of Threads: Concurrency, Parallelism, and Asynchronous Execution in C#" delves into the critical concepts of concurrency, parallelism, and asynchronous execution, which are essential for developing high-performance applications. It explains how concurrency allows multiple tasks to progress simultaneously, potentially on a single-threaded system through context-switching, while parallelism involves executing tasks simultaneously on multiple cores. Asynchronous execution is presented as a model that enables a single thread to manage multiple tasks efficiently, facilitating both concurrency and parallelism in a multi-threaded environment. The article further explores the concept of threads, their management by the operating system, and the sharing of resources within and across processes. Practical examples in C# illustrate thread creation, context switching, and the use of the thread pool to optimize thread management, highlighting the importance of understanding these concepts for writing efficient and scalable code.

Opinions

  • The author emphasizes the importance of understanding concurrency, parallelism, and asynchronous execution for writing efficient and scalable applications.
  • It is suggested that even with a single-core CPU, concurrent application execution is possible through context-switching.
  • The author conveys that asynchronous execution is a key programming model that helps achieve concurrency within a single thread by allowing it to start a task and then proceed with other tasks without waiting.
  • The article posits that parallelism is a subset of concurrency and is specifically related to the simultaneous execution of tasks on multiple processors.
  • The author opines that the Thread Scheduler plays a crucial role in managing thread execution and context switching, ensuring that active threads are allocated appropriate execution time on the CPU.
  • It is highlighted that threads share heap memory within the same process but have their own local stack memory, which is not accessible by other threads.
  • The author advocates for the use of the Thread Pool to reduce the overhead associated with creating and destroying threads, suggesting that reusing threads is a more efficient approach.
  • The article implies that developers should be mindful of the overheads and limitations when using threads, such as the time and memory costs of thread creation and the default memory allocation per thread in Windows.
  • The author provides a rationale for setting maximum thread limits in the Thread Pool, stressing the importance of controlling the number of active threads to prevent system throttling.
  • The article concludes with an encouragement for developers to continue learning about threading and synchronization to improve their coding practices and application performance.

Unlocking the power of Threads: Concurrency, Parallelism, and Asynchronous Execution in C#

In today’s article, we are going to explore the differences between concepts like Concurrency, Parallelism, and Asynchronous Execution. We will also examine what exactly Threads are, as well as various concepts related to them.

Concurrency vs. Parallelism vs. Asynchronous Execution

Concurrency, Parallelism, and Asynchronous Execution are three different but very similar and related terms in computing.

Concurrency basically means multiple pieces of work being done in overlapping time, which may or may not be parallel (e.g., multiple threads sharing the same processor core). It is the notion of programming as the composition of independently executing tasks.

In the context of an application, it means that an application is making progress on more than one task at the same time (concurrently). For example, let’s imagine that a web application starts processing one request on one thread. Then, another request comes in while our application is still processing the first one, so it starts processing the next one on another thread.

Concurrency

Concurrency means doing multiple things at one time but does not specifically refer to the use of multiple threads. JavaScript, for example, uses an event loop to implement concurrency using a single thread.

Now, I have a question for you. If we possessed a computer with only one CPU, is it possible to have a concurrent application even though it only has a single thread running inside it?

One thread running multiple tasks concurrently

The answer here is yes. But how? As we mentioned above, concurrency means executing multiple tasks at the same time but not necessarily in parallel. In a concurrent application, two tasks can start, run, and complete in overlapping time periods. For example, if we see the image above, this means that Task-2 can start even before Task-1 gets completed.

In the computer science world, the way how concurrency is achieved is different. In a single-core environment (your processor has a single core), concurrency is achieved via a process called context-switching. In this case, the CPU does not have to completely finish one task before it begins the next. Instead, it switches between the different tasks until the tasks are complete. If it’s a multi-core environment, concurrency can be achieved through parallelism.

Parallel specifically means multiple things are being done at the same point in time (e.g., threads in different processor cores running simultaneously). It is the notion of programming as the parallel execution of (possibly related) computations.

An application splits its tasks up into smaller subtasks which can be processed in parallel, for instance on multiple CPUs. To achieve true parallelism, an application must have more than one thread running, or at least be able to schedule tasks for execution in other threads, processes, CPUs, graphics cards, etc.

Parallelism

What about asynchronous execution now? Asynchronous means some operation is started out-of-sync in time, which will run on its own and notify us when completed. This may involve threads running in the same or different cores or even in different systems. Async describes how individual threads can be used:

  • Synchronous execution of code: Block the current thread and wait for the operation to complete.
  • Asynchronous execution of code: Do not block the current thread, instead delegate the work somewhere else, do something else in the meantime and come back when the job is complete to continue execution.

In general, asynchronous execution is a programming model. It helps us achieve concurrency by allowing a single thread to start one task and then do something else instead of waiting for the first task to finish. In a multi-threaded environment, it is also a way to achieve parallelism.

Let’s dive into the differences between synchronous and asynchronous execution in a single and a multi-threaded environment.

We have the following cases:

Synchronous

  • Single-Threaded: Each task gets executed one after another. That means it waits for its previous task to get executed before starting.
Synchronous & Single-Threaded
  • Multi-Threaded: Tasks get executed in different threads but wait for any other executing tasks on any other thread.
Synchronous & Multi-Threaded

Asynchronous

  • Single-Threaded: Tasks start executing without waiting for a different task to finish. At a given time, a single task gets executed.
Asynchronous & Multi-Threaded
  • Multi-Threaded: Tasks get executed in different threads without waiting for any tasks and independently finish off their executions.
Asynchronous & Multi-Threaded

Threads and related concepts

Let’s now turn our attention to the concept of Threads. What exactly is a thread? A thread is a basic unit of execution that is allocated processor time by the OS. It is a sequence of program instructions that can be managed independently by a scheduler, called the thread scheduler, which is part of the OS.

What is an application in Windows (e.g., Microsoft Word, Teams, etc.)? An application consists of one or more processes. A process is an executing program for an application. Every single process can have one or more threads running in the context of the process.

As we have mentioned above we have two different program execution models:

  • Single-Threaded: Only one thread has full access to the running process.
  • Multi-Threaded: Multiple threads coexist and run independently but share resources within the context of the same process. In a more generic way, it’s when you have more threads than CPU cores in a system, so that the OS has to rotate them and allocate CPU and other resources in turn.

Some of the most common threading scenarios include:

  • Thick client applications, where there is a UI thread that delegates CPU-bound code on a different thread.
  • Divide and conquer algorithms, which take advantage of multiprocessor computers.
  • Scalability (for web servers), which is the ability to handle a large volume of incoming HTTP requests.
Main thread delegating CPU-bound work to worker threads

Let’s see a quick example of a very simple threading use case in C#.

using System.Threading;
using System;

Thread.CurrentThread.Name = "Main Thread";

Console.WriteLine($"Main starts execution in {Thread.CurrentThread.Name}.");

// A very basic example of running a simple code on a new thread
DoWorkOnThread();

Console.ReadLine();

static void SlowMethod()
{
    Console.WriteLine($"{nameof(SlowMethod)} starts execution in {Thread.CurrentThread.Name}.");

    Thread.Sleep(1500);

    Console.WriteLine($"{nameof(SlowMethod)} Work completed in {Thread.CurrentThread.Name}.");
}

static void DoWorkOnThread()
{
    Console.WriteLine($"{nameof(DoWorkOnThread)} starts execution in {Thread.CurrentThread.Name}.");

    var thread = new Thread(SlowMethod)
    {
        Name = "Simple Worker Thread"
    };

    // Starts work on new thread
    thread.Start();

    // Continue working on current thread
    Console.WriteLine($"{nameof(DoWorkOnThread)} continues working in {Thread.CurrentThread.Name}.");
}

Here we are starting with the execution of the Main method inside the “Main Thread”. Every console application you create in C# has this notion of a single main thread. Then we are calling the DoWorkOnThread method, where at first we once again are inside the Main Thread, but then we create a new Thread and give it a name of “Simple Worker Thread”. We pass this new thread a function called SlowMethod, where we just simulate some long-running process with a Thread.Sleep command and we block the “Simple Worker Thread” for 1.5 seconds.

The output of the above code would be:

Threading use case code sample output

As we can see, after we call thread.Start() the new piece of work (SlowMethod) is being executed in a different thread (“Simple Worker Thread”). Using the thread.Start() command is the developer’s way of letting the Common Language Runtime (CLR) know that it needs to talk to the Thread Scheduler to spawn off a new thread. But the Main Thread will continue its execution so we will see the “DoWorkOnThread continues working in Main Thread” message in the console. The SlowMethod messages printed in the console are coming from the execution of the “Simple Worker Thread”.

After seeing this little and simple example, let’s now dive into trying to understand how threading works in C#. Threading is managed internally by an OS thread scheduler. The .NET CLR delegates the task of thread scheduling to the OS and works directly with the thread scheduler.

What happens to shared resources inside the context of the same process?

The CLR assigns each thread its own local memory space and a separate copy of local variables is created for each one of them to keep local variables separate. Static variables, on the other hand, are not separate. They are shared across the whole application. Threads from the same process also share the same heap memory, while different processes are completely isolated. Finally, system-level resources are shared across processes.

Sometimes, we may want to share these values / memory / system-level resources between multiple threads. More on this in a future story, so stay tuned.

Thread Scheduler

We mentioned above that the NET CLR “speaks” directly and delegates the task of thread scheduling to the OS Thread Scheduler. What exactly are the thread scheduler’s responsibilities?

First of all, it ensures that all active threads are allocated appropriate execution time on the CPU processor. In a Multi-Processor computer, this is achieved by using different threads to run code simultaneously on different processors. In a Single-Processor computer, this is achieved through a process called time-slicing, which means that the Thread Scheduler will rapidly switch execution between each of the active threads. On Windows, the slice of time is generally close to 10 milliseconds. The overhead of this context switching is in the order of microseconds.

Context switching is a technique by which the CPU time is divided between running threads. The current thread is switched with the next thread. This switching of threads requires saving the state of the thread in memory so that the execution can be resumed where it was left off previously. When a thread is suspended from execution (e.g., due to time-slicing), it’s called a “preempted” thread. The Thread (and the developer) has absolutely no control over which time exactly it will be preempted.

Let’s see context switching in action with a simple example in C#.

// This is a worker thread
var thread = new Thread(WriteUsingNewThread)
{
    Name = "Custom Worker Thread"
};

// Starts work on new thread
// Our way of letting CLR know that it needs to talk to the Thread Scheduler 
// to spawn off a new thread
thread.Start();

// Continue working on current thread
// This is the main thread
Thread.CurrentThread.Name = "Custom Main Thread";

for (var i = 0; i < 100; i++)
{
    Console.Write($" MT: {i} ");
}

Console.ReadLine();

static void WriteUsingNewThread()
{
    for (var i = 0; i < 100; i++)
    {
        Console.Write($" WT: {i} ");
    }
}

Here we are first creating a new thread and give it a method to run called WriteUsingNewThread. Then we name it as “Custom Worker Thread” and we are calling thread.Start() to spawn off a new thread and start its work, which is to execute the WriteUsingNewThread method. After thread.Start(), we continue working in the main thread and use a simple for loop to print values from 0 to 99. Let’s see the output of this program:

Context switching code sample output

As we can see, the Main Thread took precedence by the OS scheduler and printed values from 0 (MT: 0) to 9 (MT: 9). Then, context switching took place and the worker thread started its work by printing values from 0 (WT: 0) to 9 (WT: 9). Context switching happened once again and as a result, the Main thread continued its execution, where it left off, by printing values from 10 (MT: 10) to 11 (MT: 11). This process continues until the work in both threads is done, which means for the worker thread to print WT: 99 and the main thread to print MT: 99.

The very interesting thing to see is that if we run the same code again, then the output will probably be different, as we can see below:

Context switching code sample output

This proves what we have mentioned above about context switching and in particular that the thread and the developer have absolutely no control over which time exactly its thread will be preempted. This depends entirely on the OS and the Thread Scheduler.

Thread vs. Process

I have another question for you now. What do you believe is the difference between a thread and a process? Let’s try to clarify things a bit.

Threads:

  • Run in parallel within the context of a single process.
  • Have a limited degree of isolation.
  • Have local stack memory that cannot be accessed by other running threads of the same process.
  • Share heap memory with other threads running in the same process.

Process:

  • Run in parallel within the context of a computer.
  • Fully isolated from each other.
  • A process can have one or more threads running execution code paths within its own context.

Now let’s see a simple example to explain the notion of the local stack memory that its thread has and that it cannot be accessed by other running threads of the same process.

// Worker Thread
var workerThread = new Thread(PrintOneToOneHundred)
{
    Name = "WT"
};
workerThread.Start();

// Main Thread
Thread.CurrentThread.Name = "MT";
PrintOneToOneHundred();

Console.ReadLine();

static void PrintOneToOneHundred()
{
    // This variable i will be part of the local memory allocated for each thread
    for (var i = 0; i < 100; i++)
    {
        Console.Write($"{Thread.CurrentThread.Name}: {i + 1} ");
    }
}

We have created a new thread and called it “WT” and we have also named the Main thread as “MT”. We want both these threads to run the same function and in particular the one named as PrintOneToOneHundred, which just print into the console values from 1 to 100. As we can observe in the output of the code, the variable i declared inside the PrintOneToOneHundred method will be part of the local memory allocated for each thread, so even when context switching happens and the processor time is being switched between the main and worker thread, the values between the two threads do not interfere with each other, so every thread will eventually print 1 to 100.

Thread’s local memory stack code sample output

Are there any overheads in using threads in our code? The answer is yes and some of them include:

  • Every single thread creation has some overhead in time and memory.
  • Creating a new thread takes about a few hundred milliseconds, usually for the new stack allocation, allotting resources for spawning, etc.
  • Also uses around 1MB of memory per thread (default in Windows).

Thread Pool

Because of the above-mentioned thread overheads, most of the time, we want to re-use threads, by using a .NET managed notion, called the ThreadPool. Basically, what it does is:

  • Reduces the performance overhead by sharing and recycling threads.
  • Keeps a pool of worker threads. All of these threads are background threads (identical to foreground threads, except the managed execution environment is not kept running). If the main thread (or else foreground thread) ends its lifetime, all background threads will terminate at once. A call to Thread.CurrentThread.IsBackground method shows if the current thread is a background thread or not.
  • Limits the number of threads that can run in parallel because many active threads may throttle the OS.
  • New work is allocated to idle threads, and threads are returned to the pool once done with their work.
  • If there is more work than the maximum thread limit defined inside the machine that our code runs, then the work is being queued. A call to Thread.CurrentThread.IsThreadPoolThread method shows if the current thread execution is happening on a thread pool thread or not.

What are some ways we can use to ensure that we are going to do our work inside a ThreadPool thread?

  • Use the Task Parallel Library (TPL).
  • Create an asynchronous delegate.
  • Create a background worker.
  • Call ThreadPool.QueueUserWorkItem.

Below, we can see a small C# example of using the notion of a thread pool thread for running our code, instead of creating a new one ourselves.

using ThreadPoolDemo;

Console.WriteLine($"Is main thread a thread pool thread? {Thread.CurrentThread.IsThreadPoolThread}");

var employee = new Employee
{
    Name = "John Doe",
    CompanyName = "ACME"
};

ThreadPool.QueueUserWorkItem(new WaitCallback(DisplayEmployeeInfo!), employee);

// The below two examples are just a way you can use to be
// able to set the maximum threads. There is a bit of a caveat here. 
// That number could be anything. You can even say thousand, but it wouldn't really matter. 
// At one point of time, you'll only have a certain amount of threads that can do the processing. 
// Everything else will have to wait. But this is important because you want to set a max number of threads. 
// If you know what your usage is going to look like, you want to keep it in a very controlled manner. 
// Of course you have the min threads, which is equal to the number of different CPUs in the current machine and that's the minimum it gets. 
// But the fact that you have the power to set it, it makes it so much better because now, you have less overhead with dealing with threads.

// Set the max number of threads in the thread pool - 1st way
// Get the number of processors in the host machine 
//var processorsCount = Environment.ProcessorCount;

//ThreadPool.SetMaxThreads(processorsCount * 2, processorsCount * 2);

// Set the max number of threads in the thread pool - 2nd way
var workerThreads = 0;
var completionPortThreads = 0;
ThreadPool.GetMinThreads(out workerThreads, out completionPortThreads);
ThreadPool.SetMaxThreads(workerThreads * 2, completionPortThreads * 2);

Console.ReadLine();

// It does expect an object type and not an Employee type specifically that we pass
static void DisplayEmployeeInfo(object employee)
{
    // Is this indeed a thread pool thread?
    Console.WriteLine($"Is thread running this piece of code a thread pool thread? {Thread.CurrentThread.IsThreadPoolThread}.");

    var emp = employee as Employee;

    Console.WriteLine($"Person name is: {emp!.Name} and company name is: {emp.CompanyName}");
}

class Employee
{
    public string Name { get; init; } = default!;

    public string CompanyName { get; init; } = default!;
}

Here we are using the ThreadPool.QueueUserWorkItem method and pass it some function we want to spawn off to be run inside a thread pool thread. We are also seeing 2 different ways of how we can configure the max number of threads inside the thread pool.

The fact that you are able to do something like this comes with a bit of a caveat. That number could be anything. You can even say a thousand, but it wouldn’t really matter. At one point in time, you’ll only have a certain number of threads that can do the actual processing. Everything else will have to wait.

But this is important because you want to set a max number of threads. If you know what your usage is going to look like, you want to keep it in a very controlled manner. Of course, you have the min threads, which are equal to the number of different CPUs in the current machine, and that’s the minimum it gets. But the fact that you have the power to set it makes it so much better because now you have less overhead dealing with threads yourself.

You can find the above examples in the following places:

Summary

Understanding the concepts of concurrency, parallelism, and asynchronous execution is crucial for every programmer aiming to write efficient and scalable code. By grasping these fundamental principles, developers can design applications that make the most out of available resources and deliver better performance. We’ve explored the differences between these concepts, delved into the intricacies of threads, and examined real-world examples using C#. From understanding how threads operate within processes to leveraging thread pools for efficient resource management, we’ve covered a wide array of topics. Stay curious, keep learning, and happy coding!

Programming
Software Development
Concurrency
Csharp
Threads
Recommended from ReadMedium