Guide
Bare metal vs. cloud GPU instances
Dedicated bare-metal nodes and on-demand cloud instances suit different workloads. Compare control, commitment and cost.
By Tera Compute · GPU rental and colocation
A cloud GPU instance is a virtual machine that you start and stop by the hour. A bare-metal node is a whole physical server that is reserved for you for a contract term. Both can run the same GPUs. The differences are in commitment, control and the way the cost adds up.
What is a bare-metal GPU node?
Bare metal means that no virtualization layer sits between your software and the server. You get the full machine: all of its GPUs, CPUs, memory and local storage. No other customer shares the hardware.
When does an on-demand instance fit better?
On-demand instances fit short or irregular work: experiments, short evaluations and loads that change from day to day. You pay only for the hours that you run, and you can stop at any time. The hourly rate is usually higher than a committed rate, and the newest GPUs are not always available when you ask.
When does dedicated bare metal fit better?
Dedicated nodes fit steady work: training runs that last weeks, fine-tuning pipelines and inference that serves traffic all day. A commitment for a term usually brings a lower GPU-hour rate, and the capacity is yours for the term. In return you pay for all hours of the term, so utilization matters.
How do you compare the cost?
Estimate the GPU-hours that you expect to use each month and multiply by the on-demand rate. Then calculate the committed total for the same period: the number of GPUs, multiplied by the committed rate, multiplied by all hours. If your expected utilization is high, the committed rate often wins. If it is low or uncertain, the flexibility of on-demand can be worth the higher rate. The GPU rental cost calculator gives the committed total.
What else changes?
- Performance: your software runs directly on the hardware, and no other customer's work runs on the same server.
- Cluster networking: dedicated nodes can be connected with InfiniBand or RoCE for training across nodes.
- Operations: you manage more of the software yourself. Confirm what the provider supports.
- Lead time: dedicated capacity has a ready-for-service date, so plan ahead.
Tera rents dedicated bare-metal GPU nodes to AI startups. See the current offers, or read how dedicated GPU rental works.
About Tera Compute. Tera Compute provides GPU rentals to startups: dedicated bare-metal nodes for AI teams. Tera also finds colocation for the GPUs that a startup owns. The company is based in the United States and is backed by Alliance. It is built by an operating team with deep experience in industrial-scale compute and data centers. Learn more at tera.gw.