The seventh factor in a 12-factor app is that the app should “scale out via the process model.” Concurrent processing helps us scale out our application by performing processing in parallel.
This is the eighth video in our 12-factor Application Modernisation series. In this video, Marc Firth (Managing Director at Firney) explains software engineering best practices for building scalable, reliable web services that are efficient to work with.
See our Application Modernisation with 12-Factor App series and the full App Modernisation playlist on YouTube.
A transcript of the video above is included below.
Use concurrent processes
Today we’re talking about how a 12-factor app scales its pr by using concurrent processes.
This is particularly useful when you need to handle web or background jobs in parallel, or when you have a large amount of data to process.
If you’ve never dealt with threads or parallelisation, it’s awesome to see just how much faster you can get your processes to run. So let’s dig into that.
(This video was posted on valentines day)
Hey, everyone, and great to have you here and happy Valentine’s Day. And if you’re lucky enough to be celebrating with somebody, I hope you’re doing something awesome today. And if not, after doing something awesome as a free agent and as I’m recording this well before Valentine’s Day, you can only hope I use this on the right day. But I digress.
12-factor app
So this is our series on the 12-factor app, where we aim to make applications more scalable, reliable and efficient to work with.
What is concurrency?
Now, one of the ways we can make applications more efficient and scalable is by using concurrency. Concurrency is the ability to run multiple processes at the same time.
So you can think of it as there are two ways to scale:
- You can either scale up by increasing the resources that your server has to run its processes on
- Or you can scale horizontally, in which case you are essentially setting up many, many servers to run the same job in tandem and therefore separate the workloads amongst many, many nodes.
Scaling horizontal is usually more effective
Now, in certain scenarios, it’s far better to scale horizontally than to scale up.
Scaling up, you can run into limitations in terms of processing and resources that are available to you. Scaling horizontally, you can scale as much as you can afford [laughs].
What it allows us to do is cater to a larger audience and scale up our processing.
Examples of concurrency in practice
So let’s give a few examples of concurrency and where we might use it in our applications.
Concurrency – Example 1
If you have a database or a queue, you could have a process that grabs an item from that database or queue, does some processing with it and then sends it elsewhere or returns it to the database.
So if we were to have one process doing all of that work, it could only do it as fast as it could get through the items. Whereas if we scale horizontally, we can get lots of processes working in tandem.
Concurrency – Example 2
Another example might be if you were doing log file processing or some form of text processing, you could have lots of workers handling lots of files or lots of lines in files.
Concurrency – Example 3
But the most obvious example is a web server.
With a web server, we have lots and lots of users who want to access our website at the same time, but we’re not going to make them wait until the previous person has accessed the site before serving that request. We want to serve all those requests in real-time. That’s where concurrency comes in and enables us to scale horizontally, either via processes, threads or actual servers and serve all of those requests in tandem; which is absolutely awesome, because then we can get infinitely faster in the way that we process our workloads.
How does concurrency work?
So how does it work? We have a few different ways of performing concurrent operations.
Threads
The first is threads. So you may have a single process, and you can divide that process into multiple threads that run at the same time.
Processes
Similarly, we can do this with processes, or we can use the async and await functions in JavaScript.
Servers / Nodes
We can spin up entirely new workers or nodes or servers, which is where something like Kubernetes comes in and will handle that scaling for us. We’ll go into Kubernetes in a bit more detail in our next video.
Things to watch out for with concurrency
There are a few gotchas when it comes to concurrent processing and things to look out for.
Work should be self-contained.
So all the work in your process should be self-contained in that process. And if it needs to communicate with another process, it should do so via some form of database or some form of port communication.
Processes may not complete in order
The second thing is that you need to be careful with log files or output because we have all these processes running at the same time and there’s no guarantee that one process will complete before another. So you may end up with disordered items in your results.
Now, one of the things you can do when it comes to logging is to make sure that you’re using a thread safe log output. When it comes to [logging on] Kubernetes, we’re going to be picking up those logs using STDOUT and some form of log aggregator to collect all of our log output. We’ll handle those logs within the log aggregator itself.
Who uses concurrency?
Now, this is the same methodology used by some of the biggest tech around, such as Hadoop and some of the Google Cloud products, such as BigQuery. There’s a whole paper on something called MapReduce, which is the process of taking a portion of work, distributing it amongst many nodes and getting them to work on that data in tandem before collecting the result at the end.
It’s okay to have multiple stages in that process. BigQuery, for instance, will take your query and break down the sections of your query and run those via MapReduce on each of those stages until it gets to an end result.
Working with queues and concurrency
If you’re working with queues, one of the things you should do is return your jobs to the queue.
If you have any errors in your processing [you should] have a failed jobs queue where you’re collecting all of those failed jobs.
This is where we want to make sure that our processes are disposable. But that’s the topic for the next video.
So I hope that helps you scale out your processing and get through your workloads faster.
Firney is a GCP Reseller
In other news, Firney is officially a GCP reseller now, and what that means is we can now offer invoice-based billing to our clients.
It’s another win in our GCP history. So Direct Message me on LinkedIn if you want more details.
Summary
I hope you enjoy the rest of your Valentine’s Day, whatever you’re doing. If you’re not spending it with anyone, there are a lot of videos on our YouTube channel you can go and watch if you like! [laughs]
Whatever you’re doing, don’t forget to like, subscribe and share, and I’ll see you in the next video.