What is the difference between parallelism and concurrency?

My favorite definitions of parallelism and concurrency come from Brian Goetz. They are not the traditional ones, which focus mostly on # of cores. In modern computing, we are sharing so many resources, parallelism and concurrency need to account for that. In this episode, we go over those definitions.

Transcript

Eric Normand: What is the difference between parallelism and concurrency?

In this episode, we're going to go right to the heart, and we're going to talk about my favorite definition that distinguishes between these two. My name is Eric Normand, and I help people thrive with functional programming.

This is an important topic. These are two words that are often confused. They're very closely related, very similar even. I and other people use them incorrectly in passing all the time.

We interchange them really. It's just because they are such closely related concepts, but they are different enough that they do warrant their own terms. If you really need to be precise, then they do have very precise definitions that we could use to gain a little bit of power over our systems.

What is the difference? I'm going to define both of them. Concurrency is the ability to share resources safely and efficiently. When it says resources, typically, back in the past, we used to think of concurrency as a CPU sharing system. How do we share the CPU safely and efficiently?

Overtime, we've realized that the CPU is only one resource that we might want to share. You could also share memory. You could also share a database. You could share a thread pool. You could share a socket pool, if you're doing something like an HTTP server. This is concurrency. How do you share it efficiently?

You have multiple things that want this limited resource. How do you make sure that everybody gets all those things that are wanting to access it? How do they get it fairly so no one gets starved? How do you do it so it doesn't take too much processing to switch between them? This is concurrency.

Parallelism is a much simpler idea. It's deceptively simple. It is simply adding more resources so that you can do more work faster. This is adding another core. This is adding a mirror database or adding more memory. How do you scale it so that you can do more work?

They work together. If you don't know how to share the cores, adding more cores might not help your problem. You need them to work together, but they are separate because you can have concurrency where you're sharing a single resource.

If you're in an apartment that has one bathroom and you've got four roommates, you can have a concurrent system where you get in line, or you have a lock or something to make sure that only one person uses the bathroom at a time. You have some system to keep it fair. That's concurrency.

Then, once you've got that system, "Wow, we can put in a new bathroom, and now twice as many people can use the bathroom at the same time," but you still need a system to share them. They work together.

I like this definition because I've seen others, but they miss out on what's important that it's not just about the CPU. I've seen it where concurrency is about sharing resources, like memory or the CPU, and parallelism is just adding more cores.

I think that doesn't really get to the heart of it. You can have multiple databases running in parallel and they're not even on the same machine. They're on different machines in the Cloud, but that will help your system go faster. That is parallel. You can run two queries in parallel. It's just a more general definition.

I think that's all I have to say on this. It's a very useful concept to be able to distinguish between concurrency and parallelism. It's confusing because we don't use them quite so properly all the time because they're so related.

It is something that, if you need to be precise about it, this is where the distinction should be made. Awesome. If you like this episode, if you want to get all the past episodes, you can go to lispcast.com/podcast.

There, you'll find audio, video, and text transcripts of all of the past episodes. You'll also find links to subscribe as RSS for your podcast subscription or if you want to subscribe on YouTube. It's all there.

You'll also find links to get in touch with me on social media, including email, Twitter, LinkedIn, etc. I love to get into discussions, so please hit me up. Don't be afraid.

Cool. My name is Eric Normand. This has been my thought on functional programming. Thank you for being there, and rock on.