[Kubernetes]

The Ultimate Kubernetes Cost Guide: AWS vs GCP vs Azure vs Digital Ocean

Kubernetes costs can vary significantly across cloud providers and their respective managed Kubernetes services. Read on for an in-depth Kubernetes cost comparison across the top 4 cloud providers including AWS, GCP, Azure and Digital Ocean.

Hasham Haider

Hasham Haider

September 19, 2018

16 minute read

AWS, GCP, Microsoft Azure and Digital Ocean are in the top 6 environments, enterprises deploy Kubernetes workloads on. OpenStack and on-premise are the other two. AWS, GCP and Azure also have managed Kubernetes offerings, while Digital Ocean's is on the way. This coupled with the cloud first trend makes it safe to assume that most enterprise Kubernetes workloads are en-route to the cloud.

As is the case with most cloud deployments, the first question is also the most important one; Which cloud provider should I choose to deploy my Kubernetes clusters on and what will it cost me?

In this blog post we will compare Kubernetes costs for deployments on all four major public cloud providers. We will also compare the costs for workloads deployed directly onto virtual machine instances and ones leveraging managed Kubernetes services like AWS EKS, GCP GKE or Azure AKS.

TL;DR “The following table shows you yearly cloud provider costs for a 100 core, 400 GB Kubernetes cluster. Even though Digital Ocean doesn’t provide either a managed Kubernetes offering or discounts for reserved instances, it still has the lowest cost for running our cluster. GCP is a close second. When our cluster exclusively leverages on-demand instances, GCP’s cost is 37% lower than AWS and 27% lower than Azure. GKE cost, when deployed on on-demand instances is also 38% and 27% lower as compared to EKS and AKS, respectively.”

100 Core, 400 GB Kubernetes cluster

AWS

GCP

Azure

Digital Ocean

Direct Deployment (on-demand instances)

$50,882

$32,040

$43,730

$25,920

Direct Deployment (70% reserved instances)

$37,974

$29,883

$31,830

-

Managed Kubernetes (EKS,GKE, AKS - on-demand instances)

$50,064

$30,874

$42,048

-

Managed Kubernetes (EKS,GKE, AKS - 70% reserved instances)

$37,156

$28,718

$30,148

-

Reserving cloud virtual machines (VMs) is one of the most effective strategies for cloud cost control. Real world cloud workloads usually have a 70-30 split between reserved and on-demand instance usage. So we will also be comparing Kubernetes costs across cloud providers, when 70% of the worker nodes are reserved instances.

VM sizes differ across cloud providers, with unique combinations of compute, memory and throughput. As such it is difficult to find a workload that fits all cloud provider VM sizes perfectly. However, we have tried to keep the difference to a minimum and start off with a fleet of containers requiring a 100 core, 400 GB Kubernetes cluster.

Amazon Web Services: AWS

Deployment share of all Kubernetes workloads: 57%

There are two ways of deploying Kubernetes workloads on AWS; run Kubernetes directly on AWS EC2 instances using Kops or use the AWS EKS managed service. While running Kubernetes directly on EC2 instances, DevOps have to manage both master and worker nodes. Using AWS EKS, they only have to take care of the worker nodes and master nodes are managed by the system itself. Kubernetes costs for both services can differ slightly for the same workload.

1. Direct Deployment on EC2 using Kops

Let's first look into the costs of deploying Kubernetes directly on AWS EC2 instances. If we leverage an m4.large AWS instance, we will need 50 of them to fulfill the resource requirements of our cluster. In addition to the worker nodes, a master node will also be configured by Kube-up at cluster startup time. For a 50 node cluster this will be an m3.xlarge instance.

So the total Kubernetes costs for running a 50 instance cluster on AWS with one master node equal $50,882.

Deployment size

AWS 

(on-demand instances)

AWS 

(70% RI with no up-front payment)

AWS EKS 

(on-demand instances)

AWS EKS 

(70% RI with no up-front payment)

50 m4.large worker nodes + 1 m3.xlarge master node

$50,882

$37,974

$50,064

$37,156

With a workload that is operational all year round, we can also use AWS reserved instances to make significant cost cuts. Reserved instances have three billing models; no up-front, partial up-front and all up-front payments. They can also be reserved for a period of 1 or 3 years.

Reserving 70% of the worker instances for a period of 1 year, with no up-front payment will give us savings of 25% or a total yearly cost of $37,974.

2. Deployment on AWS EKS

Now let's switch over to AWS EKS and see what our cluster costs to run there. Since AWS EKS manages the control node, we can ignore the cost of the m3.xlarge instance. It does, however, charge $ 0.2/cluster/hr for managing the master node. Worker nodes are charged at the regular EC2 instance costs.

Discounting the master node cost and adding those for EKS control plane management, the total yearly costs for running our cluster on AWS EKS equal $50,064. With discounted RI pricing for 70% of our worker nodes, the total yearly Kubernetes costs equal $37,156.

Master node size depends on the number of Kubernetes nodes that make up a cluster. As the number increases so does the size of the AWS instance provisioned as the master node. Since AWS EKS charges a flat rate for master node management, leveraging AWS EKS for larger cluster sizes will result in progressively lower costs as compared to deploying on EC2 instances directly.

GCP: Google Cloud platform

Deployment share of all Kubernetes workloads: 33%

On GCP too, Kubernetes workloads can be deployed either directly onto GCP instances or using Google Kubernetes Engine (GKE). GKE also takes over management of master nodes.

1. Direct deployment on Google Compute Engine instances

We will require 53 n1-standard-2 GCP instances, to fulfill our cluster resource requirements. The recommended master node for our cluster size is n1-standard-4.

Deployment size

GCP

(regular instances)

GCP 

(70% instances 1 year commited discount)

GCP GKE

(regular instances)

GCP GKE

(70% instances 1 year commited discount)

53 n1-standard-2 worker nodes  + 1 n1-standard-4 master node

$32,040

$29,883

$30,874

$28,718

One cool feature from GCP is sustained use discounts. These discounts are applied automatically to instances that see sustained usage. Discounts grow progressively larger as instances run longer. Since we are assuming a relatively stable workload with sustained usage throughout the year, our workload will benefit from these discounts.

With a sustained use discount the total cost for our cluster with 53 n1-standard-2 and 1 n1-standard-4 instance equals $32,040/yr.

GCP also provides committed use discounts. Committed use discounts are comparable to AWS reserved instances. However one crucial difference between the two is the lack of any upfront payment in GCP.

Applying a 1 year commited use discount to 70% of our worker nodes brings the discounted cost to $ 29,883.

2. Deployment on GCP GKE

Now let's figure out the pricing for GKE. In contrast to AWS EKS, GKE does not charge for master node management, so that can be discounted altogether. Worker nodes are priced at regular machine instance prices.

So the total costs for running our cluster using GKE equals $30,874 (with sustained use discounts). When we apply the committed use discount for 70% of the worker nodes, the total Kubernetes cost equals $28,718.

Since larger clusters require bigger GCP instances to manage, the fact that management costs are discounted provides even more cost benefits. This gives GKE a definite edge in terms of cost as compared to AWS GKE.

Microsoft Azure

Deployment share of all Kubernetes workloads: 16%

Kubernetes clusters can be directly deployed on Azure using the Azure Container Service. Azure also provides a managed Kubernetes service called Azure Kubernetes service (AKS)

1. Direct Deployment on Azure VM's using Azure Container Service

For our cluster, we will leverage 50 Standard_D2_v3 azure virtual machines. Based on our cluster size the master node provisioned for our cluster size is Standard_D4_v3.

Total costs for running both the worker and master nodes for a period of 1 year equal $43,730.

Deployment size

Azure

(pay as you go)

Azure

(70% 1 year reserved instances)

Azure AKS

(pay as you go)

Azure AKS

(70% 1 year reserved instances)

50 Standard_D2_v3 worker nodes + 1 standard_D4_v3 master node

$43,730

$31,830

$42,048

$30,148

 Azure also provides discounted virtual machine rates for a 1 or 3 year commitment. In azure's case all virtual machine costs have to be paid upfront.

If we apply a discounted 1 year rate for 70% of the workload, the cost of our cluster is $31,830.

2. Deployment on Azure AKS

We can also leverage Azure AKS for our cluster. Similar to GCP, Azure AKS also doesn't charge for managing master nodes. This means that we can discount the cost for the standard_D4_v3 master node.

This results in an yearly cost of $42,048. If we also apply 1 year discounted rates for reserving 70% of the worker nodes the costs equal $30,148.

Digital Ocean

Deployment share of all Kubernetes workloads: 7%

In contrast to the three cloud providers covered above, Digital Ocean does not yet have a managed Kubernetes offering. It also does not provide any way to reserve droplets. However, it does have the fourth largest share of Kubernetes deployments for public cloud providers.

For our cluster, the most appropriate droplet size is one with 8 vCPUs and 32 GB RAM. We need 13 of these droplets.The yearly costs for having these droplets up and running equal $24,960. We will also need a master droplet with 6 vCPUs and 16 GB RAM, the yearly costs for which equal $960.

The total yearly costs for running our Kubernetes cluster on Digital Ocean equal $25,920.

Deployment size

Digital Ocean

(Regular droplets)

Digital Ocean

(70% 1 year reserved)

Digital Ocean - Managed Kubernetes

(Regular droplets)

Digital Ocean - Managed Kubernetes

(Regular droplets)

50 Standard_D2_v3 worker nodes + 1 standard_D4_v3 master node

$25,920

-

-

-

 

Conclusion

Digital Ocean has the lowest cost for running our Kubernetes cluster, with GCP a close second. Digital Ocean also has the cleanest and easiest pricing structure. DevOps teams get the best priced offer, without having to go down the rabbit hole of instance type, size, usage and billing permutations. However, the absence of a managed Digital Ocean Kubernetes service does leave it at a disadvantage (which they are correcting pretty soon). The ultimate decision will also consider the more extensive feature requirements of enterprise workloads, which could be better served by one of the bigger cloud providers.

GCP has a unique advantage over AWS and Azure with their sustained usage feature, which pays out discounts on par with reserved instance usage, without Ops having to click a single button.

GCP’s cost is 37% lower than AWS’s and 27% lower than Azure’s when our cluster is deployed directly on on-demand instances. Deployments on GKE that leverage on-demand instances are also 38% and 27% lower than EKS and AKS, respectively.

When we reserve 70% of our instances for 1 year, GCP’s Kubernetes cost is 23% lower than that for AWS. Our cluster also has a 21% lower price tag on GKE as compared to AWS EKS, when leveraging reserved instances. Compared to Azure these numbers are 6% and 5% respectively. What tips the balance in GCP’s favour even more is the fact that in Azure’s case all reserved instance costs have to be paid up-front.

Request a quick 20 minute demo to learn how best to leverage cloud VM’s for your Kubernetes clusters and make infrastructure cost savings of up to 30% using Replex.io.

Hasham Haider

Author

Hasham Haider

Fan of all things cloud, containers and micro-services!

Want to Learn How Cloud Infrastructure Drives your Kubernetes Costs and Identify Optimization Opportunities?

Request a quick 20 minute demo to learn how best to leverage cloud VM’s for your Kubernetes clusters and make infrastructure cost savings of up to 30% using Replex.io.

Schedule a Meeting