Self-hosting cost

What it costs to run open-weights models on your own or rented hardware: GPUs, serving software, operations and engineering time.

Why it matters. Open weights have no per-token API price, but they are not free to run at scale.

How it works

Self-hosting cost is the full cost of running an open-weights model on hardware you own or rent: GPU time, memory, storage and networking, plus serving software and the engineers who keep it running. There is no per-token bill, but the costs are real and mostly fixed.

The main drivers are model size (which sets GPU memory), the precision you run at, how many requests you can batch together and how busy the hardware stays. An idle GPU costs as much as a busy one. Hosts that serve open-weights models per token are an alternative to running your own.

Self-hosting tends to pay off with steady, high volume, strict data-control needs or custom fine-tunes. For spiky or low volume, a per-token API is usually cheaper.

Worked example

Illustrative: a rented GPU server at $4 per hour costs $2,920 for a 730-hour month. Serving 1B tokens that month works out to $2.92 per 1M tokens; if traffic drops to 100M tokens, it is $29.20 per 1M, because the hardware cost stays the same.

Common mistakes

Comparing GPU rental with API prices without counting engineering and on-call time.
Assuming full utilization; idle hours raise the effective cost per token.
Sizing memory for the weights only, not for long contexts and many concurrent requests.

Across the catalog today

21 open-weights models in the catalog

See it on a real card: Where to run it on Claude Opus 5.5.