⚡ LLM Throughput Calculator
Tokens / sec · TTFT · TPOT · total time · requests per minute
⚙️ Configuration
Model size
7B
13B
34B
70B
405B
Custom
B parameters
Hardware
A100 40GB
A100 80GB
H100
L4
T4
M1 Max
M2 Ultra
M3 Max
Quantization
FP16
INT8
INT4
Batch size
1
1
128
Average output tokens
256
1
4096
⟳ Calculate
Reset defaults
Estimates based on memory bandwidth & compute FLOPS. Real-world may vary.
📊 Estimated metrics
—
Tokens / sec
—
TTFT (ms)
—
TPOT (ms)
—
Total gen. (s)
—
Requests / minute
🔄 Hardware comparison
Hardware
Tokens/s
TTFT (ms)
TPOT (ms)