[USE CASE]

Train a 70B Parameter LLM

From scratch or continued pretraining of Llama-class 70B models.

Training a 70B parameter LLM (Llama-3 70B class) requires a tightly-coupled 8×H100/H200 or 4×B200 cluster with NVLink + InfiniBand fabric. Plan for 1.4M GPU-hours for a from-scratch pretrain on ~15T tokens, or 4–40K GPU-hours for continued pretraining.

Recommended
8× NVIDIA H200 SXM5 (NVLink + InfiniBand)
Est. cost
$2.1M – $5.6M (from scratch, 15T tokens)
Duration
~28 days on a 1,024-GPU H200 cluster

Step-by-step

  1. 1. Provision cluster
    Request 128× 8-GPU nodes with InfiniBand fabric and shared NVMe scratch.
  2. 2. Tokenize & shard data
    Pre-tokenize 15T tokens; shard across cluster object storage with deterministic order.
  3. 3. Configure parallelism
    TP=8, PP=16, DP=8. FP8 mixed precision via Transformer Engine.
  4. 4. Launch & monitor
    Continuous loss tracking, gradient norm monitoring, automatic checkpoint every 1B tokens.

FAQ

How much does it cost to train Llama-3 70B from scratch?
Approximately $2.1M–$5.6M depending on token budget (8T–15T) and GPU pricing. A 15T-token run on H200 at $32/hr equates to ~$4.5M.
Can I train a 70B model on H100s instead of H200s?
Yes. H100s are ~30% slower in tokens/sec for 70B pretraining but typically cheaper. Total cost is comparable; H200 wins on time-to-result.
Definition

What is Servers.Computer?

Servers.Computer is an AI compute routing and procurement layer that benchmarks, compares, and deploys GPU clusters (NVIDIA H100, H200, B200 and AMD MI300) across global cloud providers in real time.

Other use cases
Serve LLM Inference in ProductionDeploy AI Agents in Production