GPU rental and colocation glossary
The terms in a GPU rental offer and in a colocation offer, each in one or two sentences.
Rental terms
- Bare metal
- A whole physical server for one customer, with no virtualization layer between the software and the hardware.
- Dedicated node
- A server that is reserved for one customer for the contract term. No other customer uses it.
- Node
- One server with its GPUs, CPUs, memory, storage and network ports. The nodes that Tera rents have 8 GPUs each.
- Cluster
- A group of nodes that work together, usually with a fast connection between them.
- GPU-hour
- One GPU for one hour. The unit of the rental rate. A node of 8 GPUs uses 8 GPU-hours each hour.
- Contract term
- The period for which the nodes are reserved and paid for, counted in months.
- Prepayment
- The part of the contract total that is paid before the term starts, shown as a percentage.
- Reserved capacity
- Capacity that a customer commits to for a term. It is billed for all hours, used or not.
- On-demand
- Capacity that is started and stopped by the hour with no commitment, usually at a higher rate.
- Indicative offer
- A published price and term that shows what is possible. It is not a reservation. A written offer confirms it.
- Written offer
- The document that confirms configuration, availability, price and dates for one customer.
- Ready for service (RFS)
- The date on which the nodes are installed, tested and handed to the customer. The term usually starts on this date.
- Service level agreement (SLA)
- The part of a contract that states the service that the provider commits to, for example availability, and what the customer receives if the provider misses it.
- Utilization
- The share of paid GPU-hours that do useful work. Low utilization raises the true cost of each hour used.
Hardware and workloads
- GPU
- Graphics processing unit. A processor that does many calculations in parallel, used to train and to run AI models.
- GPU memory
- The memory on the GPU itself, also called VRAM. A model and its working data must fit in it.
- Training
- The work of making a model from data. It runs for hours to weeks and often uses many GPUs together.
- Fine-tuning
- Training that adapts an existing model to new data. It needs less compute than training from the start.
- Inference
- The work of running a trained model to answer requests.
- Checkpoint
- A saved copy of a model during training, so that a run can continue after a stop.
- NVMe
- A fast type of solid-state storage inside a server, used for datasets and checkpoints.
Networking
- InfiniBand
- A networking technology made for high-performance computing, with its own switches and adapters. It connects nodes with very low delay.
- RoCE
- RDMA over Converged Ethernet. It gives the low delay of RDMA on Ethernet equipment.
- RDMA
- Remote direct memory access. One server reads and writes the memory of another without the usual path through the operating system.
- Bandwidth
- The quantity of data that a connection carries in one second, for example 10 Gbps.
- Cross-connect
- A physical cable in a data center between the equipment of a customer and a carrier or another customer.
- Egress
- Data that leaves the provider toward the internet or another site. Some providers charge for it.
Data center and colocation
- Colocation
- A service in which a customer puts its own servers in the data center of another company, which supplies space, power, cooling and connectivity.
- Rack
- The standard frame that holds servers and switches in a data center.
- Rack unit (U)
- The unit of height in a rack: 1.75 inches. A full rack commonly has 42 U or more.
- Rack density
- The power that one rack draws, in kW. GPU racks have a much higher density than ordinary server racks.
- Kilowatt (kW)
- A unit of power: the rate at which equipment uses electricity. Colocation capacity is sold in kW.
- Kilowatt-hour (kWh)
- A unit of energy: one kilowatt for one hour. Electricity use is billed in kWh.
- Megawatt (MW)
- 1,000 kW. The usual unit for the capacity of a data-center hall or site.
- IT load
- The power that the servers and the network equipment draw, without cooling and other facility systems.
- PUE
- Power usage effectiveness. The total power of a facility divided by its IT load. A lower figure means less power for cooling and other systems.
- PDU
- Power distribution unit. The equipment in a rack that brings power to each server.
- Power feed
- One electrical supply to a rack. Two separate feeds, A and B, let a server stay on when one fails.
- N+1 and 2N
- Two levels of redundancy. N+1 has one spare unit more than the load needs. 2N has a complete second system.
- Air cooling
- Cooling that moves cold air through the servers and removes the hot air.
- Direct liquid cooling
- Cooling that brings a liquid to cold plates on the GPUs and CPUs. Some new GPU servers need it.
- Remote hands
- Work on a customer's equipment by the staff of the data center, for example to replace a part or a cable.
- Letter of intent (LOI)
- A document in which two parties state that they intend to make an agreement. It usually does not bind them to the full terms.