Async is fast, except when it isn’t

04/12/2019

When I was becoming a developer, hammering away at online tutorials, picking a discipline and deciding on my bias (front-end, back-end…) I assumed somewhat optimistically, that the work I’d be doing would involve being given a spec for a thing and it being my job to go off and build said thing, any old way I chose. What I didn’t expect, and have since learned, was that most of my work would be on maintaining, extending and rebuilding systems that are already basically ‘done’. This has its pros and its cons. When I set to work in a team, I already have the conventions right there to work within which is great. Equally, when I set to work in a team, I already have the conventions right there to work within. So it’s really a double-edged sword.

When I started in my latest role, I’d see methods everywhere with ‘await’ or ‘async’ at the start and so when I wrote my methods I did the same. I asked about it, I did a quick google and I discovered that it was an architecture choice made to improve performance and so I deduced, incorrectly, that we used async code to speed things up. It was later, during a deeper dive into what exactly it was that I discovered why you might want to write async methods. And why it’s so important to know exactly is when you’re making your architecture choices.

How does it work?

The async programming model is organised differently to your garden variety code. You write your code as a series of statements, as always, but it executes in a different way; rather than executing the work a statement at a time, the compiler looks at external resource allocation and then decides in what order the work must be executed.

The Thread Pool

External resources could be referring to a number of things but it is most likely that the compiler will be looking at the number of remaining threads in the thread pool. When we are running code, we do so using a ‘thread’. Thread creation is expensive so the .NET framework ensures that we begin with a pool of 25 threads. Just enough so that we don’t need to create a new thread if we have multiple units of work happening at the same time. When the thread finishes its job, it returns to the pool, ready to begin another unit of work when it is called. When all the threads are in use at once, we end up with an error called a ‘deadlock’. Threadpool threads are used on background tasks. It is best suited to anything too trivial to warrant creating a new thread. 

Let’s get cooking

As is so often the case, asynchronous programming is best explained using an analogy and the analogy I wish to use is that of a Christmas dinner, to tie in with the festive theme of this article. Whether you’re a rank amateur or a seasoned pro, you’ll know that cooking often involves having a number of tasks running in parallel. If you’re frying an egg, you might want to start the toaster off while you do. You’d probably butter the toast while the eggs finish off in the pan too. In this way, you’d be executing your tasks asynchronously. This would only take one person to perform the tasks and in the same way, one thread would handle this too. 

The alternative? Well, you’d put bread in the toaster, wait for it to pop, butter the toast and place it on your plate then crack an egg and fry it, adding it to your now very cold toast when it’s done. 

So frying an egg is one thing but what if we made this example just a little more festive?

For the purposes of this article, and in keeping with the yuletide theme, I thought we’d make a Christmas dinner. Those of you who have tried your hand at your average Sunday roast can well imagine that the big day’s spread is like scaling a mountain. There are so many moving parts! Different cooking times, different temperatures and different cooking methods, all carefully managed so that everything arrives on the table in front of our hungry guests at the right time, cooked perfectly and with all the trimmings.

 

Each module of work executes in series, one after the other. Some items must be performed one after the other; you couldn’t roast a turkey before it’s been stuffed. Or you could, but it would be an altogether less satisfying turkey. In other places though, the programme doesn’t take advantage of the ability that we have to do things in parallel using multiple threads from the thread pool.  It doesn’t use asynchronous programming at all, using the one thread but giving that thread the option of jumping around from task to task as and when it is needed. The thread in use is blocked until the work is done. In culinary terms, we are served a mostly cold dinner. 

Now, in tech terms, imagine we have some logic rendering the UI that needs to execute but our threads are blocked while it does, the user won’t be able to interact with anything else on the screen while they’re waiting for the threads to become available again. It would make for a very disappointing experience. 

The one-thing-at-a-time approach can work beautifully in relatively simple scenarios. Applications with an if-this-then-that approach function well this way. Text driven adventure games and toast spring to mind. The series of tasks can play out one after another and each segment of logic can wait for the one before to finish executing before it starts without consequence. But that’s no we’re dealing with right here. 

How to make a better dinner

If we make use of await, we make sure we don’t block threads. By using the await keyword, we are ensuring that we do not slow down our application when we could be performing tasks in parallel. 

Sticking with the Christmas feast, here’s how the same dinner would look built efficiently.

 

As you can see, the code doesn’t look terribly different from the inefficient sample further up the page. The big difference is the use of the word ‘async’ and ‘await’ before the methods. This allows the compiler to action all the tasks at once. In this way, the tasks are not blocked. It’s good but it’s not great. The code still executes sequentially. The way this code is written, the processor has room to go do something else with its resources. You could start cooking a second Christmas dinner, for instance. That said, you’d still have to wait for the turkey to cook before you could start the gravy. 

One instance that this kind of architecture could be useful is one where we’re doing some background logic, but wish to keep a UI responsive to the user at the same time. With the await keyword, your compiler would be free to execute what was necessary to do just that, while executing this background logic. 

How do we ensure dinner isn’t cold?

We use the Task keyword. Using Task ensures that all tasks start immediately. Then, once the tasks are done, we can move on to other work. Our dinner gets ready in far less time, roughly as long as it takes to cook the ingredient that takes the longest. So in our case, the turkey. Below is an example of how that would look if we were still cooking programmatically.

In this example, we have used the keyword Task which allows all work assigned the keyword to start at the same time. You’ll notice that some of the work does not have the Task keyword and these are tasks that do depend on another finishing before it can begin. Overall though, all tasks that can be done concurrently are being done concurrently. That means that no time is wasted. While the thread working on the turkey gets going, another can start the gravy. Thus, ensuring that the food is served when it’s ready and not in stages. 

You might also notice that some of the variable names have changed. I’ve changed each name to include the word ‘task’ as it’s best practise to do so; it helps to make your code more readable too, for the next developer or for you, a few weeks into the future when you’ve perhaps forgotten what it was you were doing here.

It’s in the same way that an application can work efficiently too. The UI threads are working parallel to the background logic so the user is able to interact with the front end of the application without having to slow down to wait for other things to finish executing, but this time around, we also allow our background tasks to run parallel to each other. So it’s quicker! In a way…

Why async is fast (and why it isn’t)

Async can speed things up massively, but as we have seen, the way we architect the application is incredibly important. Using the asynchronous abstraction actually takes longer than if we were to run several simple tasks in series. The .NET compiler is designed to execute tasks sequentially and fast too. That’s it’s raison d’etre. It’s when you’re trying to handle complex logic in an application that needs to do things in the background without that affecting the user’s experience, that it really comes into its own. 

In the system I work with, we use async methods to set up users, to send emails, to log errors, the list goes on. In short, anything we wish to set up that doesn’t need to fire immediately is fair game. Say we have set up a user and wish to email them an activation code. Do we need to ensure this is done before we set up the next? From a compile perspective, no. So we set up an async method to do that. When we send the next message, it doesn’t depend on that first one being sent. And when we’re sending emails to an entire organisation, and there are a 100 people signed up, sending each email synchronously could really slow our user down. So we go with async. Async should speed up our code but it could also slow it right down. In order to make sure we get the best performance from our code, we need careful planning and design.

In short, when we consider the options and we’re wondering if writing async code is worth it, we can remember this rule: it’s worth it for a Christmas dinner but not if you’re only making a cheese sandwich. 

 

For more information (or just a better analogy!) check out the microsoft docs here

https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/async/

See also, Jon Skeet’s article on the thread pool at his blog here

https://jonskeet.uk/csharp/threads/threadpool.html

 

 

About the Author

Emma Burstow

@emaburst

Emma is a .NET developer with 4 good years of experience in both Umbraco and the .NET world. After embarking on a career in tech after leaving teaching, she feels passionate about creating pathways for new talent and nurturing environments for newcomers to the profession. Based in Surrey, Emma is a mother of two brilliant people and spends much of her time outside of work, writing about the tech world, reading murder mysteries and crocheting things that no one will wear.