Setting up a Self-Hosted GitHub Actions Runner on AWS Virtual Machines
This article is a step-by-step guide to configuring your Github Actions runners to work with “self-hosted” AWS EC2 VMs.
When it comes to CI/CD tools, GitHub Actions is the number one choice for developers and small to medium teams for its ease and integrations with various software development tools. Github Actions is nothing like other traditional tools such as Jenkins that need complex setup and maintenance as it offers a more straightforward approach. Developers love GitHub Actions because it works smoothly with Github and makes workflow automation simple. Github Actions has an event-driven approach. It runs tasks when specific events happen, like when code is pushed, an issue is opened, or pull requests are created.
Fun Fact 😉
Github Actions has a vast marketplace of over 15,000 pre-built actions which makes it easy to connect with various development tools and cloud services.
Small to medium teams especially benefit from GitHub Actions due to its generous free tier offering 2,000 minutes monthly for private repositories and unlimited usage for public ones.
This sounds cool until you want to run large actions that need more than 10 GB of cache memory 😂.
This offering is pretty cool for small tasks like automatic deployment or running tests for medium-sized software in the CI. But sometimes you may have heavy actions that require much more than the capacity provided by the free-tier GitHub actions.
My personal story 😂
I was working on benchmarking software, and I had set up these benchmarks to run automatically via GitHub Actions cron jobs. Everything seemed perfect in theory, but then my actions started failing.
I spent a couple of times looking at my workflow configurations because I was convinced I must have messed up something in the setup 😂, Until I discovered the real problem. The benchmarks that I was trying running in the CI were quite heavy and needed more than 10 GB of cache storage and disk space to properly run.
At this point, there are several ways to tackle the insufficient disk space issue in GitHub Actions. The simplest approach is using the maximize-build-space action.
It can free up to 32GB on Ubuntu runners by removing unnecessary preinstalled software like Android SDK, Haskell compiler, and Docker images. But this is just a workaround! This solution works for moderate space requirements, but it is still limited by the runner’s base disk capacity.
This is where AWS EC2 VMs come in!
For more demanding workflows like heavy benchmarking, a more robust solution is setting up self-hosted runners on AWS EC2 instances. In the next section of this tutorial, we will focus on setting up a self-hosted GitHub Actions runner on AWS EC2 for a permanent solution to the disk space limitations while offering greater control over the execution environment 😉.
Moving to Self-Hosted Runners.
Well, first things first. We need to setup the AWS EC2 VM infrastructure on AWS.
Infrastructure Setup and Runner Configuration
For this, we need to sign in to our AWS account and create a new EC2 instance.
- From your dashboard, search EC2 in the search bar
- And go to the dashboard page of your AWS EC2 service
- From there, you will find an action to launch a new instance
When launching the new EC2 instance, selecting the right instance type is crucial for optimal performance. For most self-hosted runners, the General Purpose (T3 or M5) instances provide an excellent balance of compute, memory, and networking capabilities.
Well, let’s walk through the setup configurations.
Give a name that makes sense to our EC2 instance, and you may give a tag if you want. For the operating system image, we select Amazon Linux for the purpose of this demonstration.
Well, as we mentioned earlier, the machine type will really depend on how heavy your Github Actions workflows are. You can select more capable machine types that can handle your workloads properly. You can use the following reference to find the best machine type that fits your needs.
For the purpose of this tutorial, we will select the free-tier eligible T2 micro instance 🥱
One last important thing in the configuration! We need to activate the SSH traffic in the network configuration to be able to connect to your EC2 from SSH. This will be crucial for the next steps.
That’s all for the configuration! Next, we need to launch the instance and connect to it via SSH. From the dashboard of the EC2 service, locate your instance, open its page, and click on the connect button. From there, get its ID and connect to it via your cloudshell. The command to run is
aws ec2-instance-connect ssh --instance-id i-8113YourInstanceIDhereOne very last thing 😉! We need to download and install our github actions runner on the EC2 VM.
At this point, you need to open your github repository. For this purpose, I have prepared a small action that runs to check spell errors in the README file. You can fork the repository at
When you have your repository setup, go to Settings > Actions > Runners.
When you are on the Runners page, click on the New Self-Hosted Runner button to open the following configuration page.
Copy all these commands and run them on the previously opened SSH-connected EC2 VM instance.
Well, in the process of running these commands, specifically the configuration script, you may encounter some issues. Fear not, it is a classic one. This thread is more than enough to fix it.
After you successfully ran those commands, you should have something similar to this.
Congratulations for making it up to this point. Let’s finalize it 🔥
Workflow Modification
We are almost done. The only thing left is to reconfigure our workflow to run on Self-Hosted Runner. This is pretty easy; we just need to update the Runs-On key on the YAML workflow file to self-hosted.
Workflow Trigger
Our time is now! We should now trigger our github action to run on our self-hosted and custom VM. Saving our action workflow after the update or pushing the update if you modified it locally would trigger the workflow, and this time, it would run properly on our EC2 VM instance.
Congratulations 🔥🎉. We have setup a custom and “self-hosted” Github Actions Runner on AWS using an EC2 VM instance.
If you encountered an issue at any step, you can leave a comment below. I would be happy to help 🎊.
Thanks for reading ❤️🔥✨
My name is Baimam Boukar. I’m a software engineer, and I enjoy sharing my knowledge through blog posts. I write about serverless on AWS, Flutter, Machine Learning and sometines on random topics like space technologies. Let’s stay connected
