Setup Details:

• Set up Karpenter v1.0.2 and EKS via Terraform for infrastructure provisioning. EKS cluster runs on Kubernetes v1.30 with a dedicated node group for essential add-ons. default-ng node group uses t3.xlarge instance types and taints to schedule only critical workloads.
• EKS nodes in private subnets communicate with external resources through a NAT Gateway.
• Karpenter provides autoscaling for optimizing cost and resource efficiency. Configured with Amazon Linux 2023 (AL2023), private subnets, m7a node families, on-demand instances, and a 5-minute consolidation delay.
• Set up GitHub Actions Runner Controller (ARC) and Runner Scale Set using Helm. Applied tolerations to the controller and listener to run on default-ng node group. Used a custom Docker image for runner pods with added tools.

Running Workflows:

• Used PostHog OSS as an example repo to demonstrate cost comparison over 960 jobs. Leveraged PostHog's Frontend CI workflow with continuous commit simulation script to trigger CI workflow every minute for 2 hours.

Results:

• In Frontend-CI workflow, ARC with varied node sizes had slower performance due to shared resources. ARC with 1 job per node was more performant but had higher job start delays.
• Cost comparison showed that ARC with 1 job per node is more cost-effective than varied node sizes. Total cost for ARC with varied node sizes is $45.88 and for 1 job per node is $42.60.

Conclusion:

• ARC is flexible and scalable for running GitHub Actions workflows but has operational overhead. It is 10 times cheaper than default Github Actions runners.
• WarpBuild provides similar flexibility with no operational complexity, is more cost-effective (~41% saving), and offers faster job start times, caching, and easy-to-use dashboards.