If you’re looking to deploy virtual machines for your next AI training or inference project, you may have noticed how the cloud GPU market is a wild place at the moment. 🤠
Both hyperscalers and new AI-focused startups compete fiercely to get access to the latest premium cloud GPUs. The cost of GPU instances and clusters varies greatly between different service providers. It’s not easy to get a clear view of your options. That’s why we’re here to help.
In this article, we go through some of the common options for deploying cloud GPU instances, including Google Cloud Platform, Amazon AWS, and Microsoft Azure. We also assess three of the best-known cloud computing providers—OVH, Scaleway, and Paperspace—and introduce you to DataCrunch as a new option to consider.
Cloud GPU Hyperscaler Pricing
Many AI developers look first to hyperscalers such as Google or Amazon when starting their cloud computing journey. This option makes sense because many people may have worked with a preferred cloud computing platform in the past and know how it works. It is also not uncommon for the big providers to offer large amounts of free credits for startups to use on AI-related projects. Congrats if you got your credits—here is how you can use them!
Google Cloud Platform GPU Pricing
The Google Cloud Platform offers virtual machines for machine learning use cases through the Compute Engine solution. Each GPU that GCP offers has an internal model name, such as a3-highgpu-8g. You also have multiple international data center locations to choose from.
GPU | Memory | Machine Type | Region | Price per hour |
|---|---|---|---|---|
H100 | 80GB | a3-highgpu-1g | us-central1 | Not available |
H100 x8 | 80 GB | a3-highgpu-8g | europe-west4 | $88.49 |
A100 | 80 GB | a2-ultragpu-1g | europe-west4 | $5.58 |
A100 x2 | 80 GB | a2-ultragpu-2g | europe-west4 | $11.12 |
A100 x4 | 80 GB | a2-ultragpu-4g | europe-west4 | $22.32 |
A100 x8 | 80GB | a2-ultragpu-8g | europe-west4 | $44.65 |
A100 | 40 GB | a2-highgpu-1g | europe-west4 | $3.78 |
A100 x2 | 40 GB | a2-highgpu-2g | europe-west4 | $7.49 |
A100 x4 | 40 GB | a2-highgpu-4g | europe-west4 | $14.99 |
A100 x8 | 40GB | a2-highgpu-8g | europe-west4 | $29.98 |
Currently, the NVIDIA H100 is only offered by GCP on an 8-GPU instance from the us-central1 region for $88.49 per hour. The Google Cloud Platform has many different options to choose from for both 80GB and 40GB versions of the A100.
Google also offers older cloud GPU options, including the NVIDIA V100 Tensor Core GPU at a cost of $2.48 per GPU for up to 8x V100 instances. In a direct comparison, GCP was a bit more affordable than Amazon AWS and Microsoft Azure.
Machine Type | GPU | Memory | Region | Price per Hour |
|---|---|---|---|---|
nvidia-tesla-v100-1g | V100 | 16 GB | europe-west4-b | $2.48 |
nvidia-tesla-v100-2g | V100 x2 | 32 GB | europe-west4-b | $4.96 |
nvidia-tesla-v100-4g | V100 x4 | 64 GB | europe-west4-b | $9.92 |
nvidia-tesla-v100-8g | V100 x8 | 128 GB | europe-west4-b | $19.84 |
For more information and the latest costs, see GCP Compute Pricing.
Amazon AWS Cloud GPU Pricing
Amazon offers a number of cloud GPUs through AWS EC2. Each virtual machine has its own Amazon model naming convention for instance types, e.g., “p5.48xlarge.” This AWS instance naming convention can take some time to decypher, so here is a quick cheat sheet of different options.
Model | GPU | GPU Memory | Region | Price per hour |
|---|---|---|---|---|
p5e.48xlarge | H200 x8 | 1128 GB | Not available | N/A |
p5.48xlarge | H100 x8 | 640 GB | $98.32 | |
p4d.24xlarge | A100 40GB x8 | 320 GB | $32.77 | |
p4de.24xlarge | A100 80GB x8 | 640 GB | Not available | N/A |
p3.2xlarge | V100 x1 | 16 GB | $3.06 | |
p3.8xlarge | V100 x4 | 64 GB | $12.24 | |
p3.16xlarge | V100 x8 | 128 GB | $24.48 |
The H100 is currently offered only in 8-GPU instances at a price of $98.32 per hour. AWS has only 40GB versions of the A100 available for a price of $32.77 for a 8x GPU instance.
Amazon has multiple options on offer for the V100 from 1x to 8x GPU instances across various AWS EC2 regions starting from $3.06 per GPU or $24.48 for an 8xV100 instance.
While they have instance models listed, the H200 and A100 80GB are not currently available through AWS.
Microsoft Azure Cloud GPU Pricing
Microsoft Azure offers a number of high-performance cloud GPUs across different international data center locations.
Instance Name | GPU | GPU Memory | Region | Price per hour |
|---|---|---|---|---|
NC40ads H100 v5 | H100 | 80GB | East US | $6.98 |
NC80adis H100 v5 | H100 x2 | 160 GB | East US | $13.96 |
NC24ads A100 v4 | A100 80GB | 80GB | East US | $3.67 |
NC48ads A100 v4 | A100 80GB x2 | 160 GB | East US | $7.35 |
NC96ads A100 v4 | A100 80GB x 4 | 320 GB | East US | $14.69 |
NC6s v3 | V100 | 32 GB | East US | $3.06 |
NC12s v3 | V100 x2 | 64 GB | East US | $6.12 |
NC24rs v3 | V100 x4 | 128 GB | East US | $12.24 |
Currently the H100 is offered at $6.98 per GPU instance and the A100 comes with a number of options ranging from $3.67 for a single GPU to $14.69 for 4xA100 instances.
For the V100 there are also many options to choose from ranging from $3.06 per hour to $12.24 for a 4xV100 instance.
In addition to Windows, you can run Ubuntu and other Linux OS on your Azure virtual machine instance. For up-to-date pricing information on Microsoft Azure, see Linux Virtual Machine Pricing.
GCP vs. Amazon AWS vs. Azure GPU Cost Comparison
It is not easy to exactly compare Amazon AWS, Google Cloud Platform, and Microsoft Azure cloud GPU prices because there is not much overlap in the GPU instance offerings. All three solution providers do currently offer the V100 at similar prices, which Amazon AWS and Azure offering the exact same $3.06 per hour rate.
What are your other options?
In addition to Google, Amazon and Microsoft there are other options to consider in the Hyperscaler front. Enterprise IT companies like Oracle, IBM and HP all offer some level of cloud computing capacity. To get an accurate understanding of pricing you may need to contact them directly.
If you’re simply looking for a cost-effective option, you’re not likely to find the best deal from Amazon, Google, Microsoft or other hyperscalers. ML-focused cloud GPU providers like DataCrunch offer the same high performance GPUs up to 8x cheaper. For example, the current cost for the V100 on the DataCrunch Cloud Platform is just $0.39 per hour and the H100 is available for $3.35/hour.
Cloud Computing Providers
If you’re looking beyond the hyperscalers, there are a number of established large cloud computing and hosting providers that provide GPU virtual machines. Three of the most common options to consider include OVH, Paperspace and Scaleway.
OVHcloud Cloud GPU Pricing
OVHcloud is Europe’s largest cloud computing provider. With headquarters in France OVH has customers across the globe with data centers across multiple international locations. In addition to various CPU VM instances OVHcloud has on-demand instances of popular cloud GPUs available.
Instance Name | GPU | GPU Memory | Price per hour |
|---|---|---|---|
h100-380 | H100 | 80 GB | $ 2.99 |
h100-760 | H100 x2 | 160 GB | $ 5.98 |
h100-1520 | H200 x4 | 320 GB | $ 11.97 |
a100-180 | A100 | 80 GB | $ 3.07 |
a100-360 | A100 x2 | 160 GB | $ 6.15 |
a100-720 | A100 x4 | 320 GB | $ 12.29 |
t1-45 | V100 | 16 GB | $1.97 |
l1-90 | V100 x2 | 32 GB | $ 3.94 |
t1-180 | V100 x4 | 64 GB | $ 7.89 |
l40s-90 | L40S | 48 GB | $1.80 |
l40s-180 | L40S x2 | 96 GB | $ 3.60 |
l40s-360 | L40S x4 | 192 GB | $ 7.20 |
OVHcloud offers high performance NVIDIA instances in a number of different configurations and options. It is the only cloud GPU provider to offer the H100 at a lower hourly price point than the A100.
OVHcloud also has multiple instances of the V100 and L40S available in addition to dedicated servers and data hosting services.
Up-to-date pricing from OVH can be found here.
Paperspace Cloud GPU Pricing
Paperspace is a major cloud computing platform focused on deploying machine learning models owned by Digital Ocean, the web hosting company.
Paperspace offers Multi-GPU instances of the H100, A100 and V100. In addition, they offer various RTX-based GPUs on demand, including the RTX A6000, RTX 5000 and RTX 4000.
The current hourly price for an H100 GPU instance from Paperspace is $5.95, but they do offer discounts on multi-year commitments.
GPU | GPU Memory | Price per hour |
|---|---|---|
H100 | 80 GB | $ 5.95 |
H100 x8 | 640 GB | $ 47.60 |
A100 x8 | 640 GB | $ 25.44 |
V100 | 32 GB | $ 2.30 |
V100 x2 | 64 GB | $ 4.60 |
V100 x4 | 128 GB | $ 9.20 |
RTX A6000 | 48 GB | $ 1.89 |
More information on Paperspace pricing can be found here.
Scaleway Cloud GPU Pricing
Scaleway is another large web hosting and cloud computing company with headquarters in France and a presence in many international locations. Like OVHcloud Scaleway offer a broad range of virtual machines and cloud storage options, including cloud GPUs instances.
Instance Name | GPU | GPU Memory | Price per hour |
|---|---|---|---|
H100-1-80G | H100 | 80 GB | $ 2.73 |
H100-2-80G | H100 x2 | 160 GB | $ 5.46 |
L40S-1-48G | L40S | 48 GB | $ 1.40 |
L40S-2-48G | L40S x2 | 96 GB | $ 2.80 |
L40S-4-48G | L40S x4 | 192 GB | $ 5.60 |
L40S-8-48G | L40S x8 | 384 GB | $ 11.20 |
Scaleway currently offers one of the lowest hourly prices for the H100 instance among large cloud GPU providers. In addition, it also has L40S and P100 instances on offer.
Scaleway also provides dedicated servers and bare metal solutions.
More information on latest Scaleway GPU pricing can be found here.
DataCrunch Cloud GPU Pricing
If you’re open to working with new cloud GPU providers, you should consider DataCrunch. We represent a new generation of AI-focused accelerated computing platforms built by AI engineers for AI engineers.
We only work with premium NVIDIA cloud GPUs and emphasise speed, value for money and efficiency in both our hardware and software solutions. In some cases, we’ve been able to reduce the cost of AI inference by 70% simply by getting more efficiency out of NVIDIA’s hardware stack.
In a direct cost comparison our fixed cost hourly GPU instance rates are competitive. Not only that, we are the first cloud GPU platform to offer dynamic pricing - a new way to reduce the cost of AI-focused computing on our less-used inventory.
This is how dynamic pricing works. When we have spare capacity on our GPU models, like the RTX 6000 ADA example below, we offer a discounted rate for hourly GPU instances. You can save up to 40% on the cost of GPU instances by picking dynamic pricing instead of a fixed price at off-peak times and GPUs with less market competition.
At DataCrunch, we don’t just stand out for our flexible and fair pricing. Our data centers utilize 100% renewable energy and enterprise-grade security. We’re often praised for the expert-level service we can provide customers.
The best way to see if DataCrunch is a good option for you is to spin up an instance on our intuitive and easy-to-use GPU Cloud Platform.
To get an even better deal you can reach out to us to request a quote on bare metal clusters or serverless inference solutions.
All of the pricing and cost information provided here is accurate as of September 2024. If you find any errors, you can reach out to us on Reddit or X.