Coming soon

Routing economics

Compare frontier API costs vs self-hosted open models across traffic tiers

Users

Queries / user / day

Peak concurrency %

Simple queries60%

$12K

frontier API · Claude 3.5 Haiku

200 in / 100 out tokens per query

↻ configure

60%

Standard queries30%

$92K

frontier API · Claude 3.5 Sonnet

800 in / 400 out tokens per query

↻ configure

30%

Complex reasoning10%

$55K

frontier API · GPT-4o

2000 in / 1000 out tokens per query

↻ configure

10%

Annual savings

$72K

45% less

Self-hosted saves vs frontier API

Frontier API cost

$159K

100K queries/day × 365

Self-hosted cost

$87K

11 GPUs total · hardware amortization

Cost vs. volume

Frontier API Self-hostedBreakeven at ~55K queries/day

Model

Type

Tier

Annual cost

Cost / query

Claude 3.5 Haiku	API	Simple queries	$12.3K	$0.000560
Llama 3.1 8B	Self-hosted	Simple queries	$4.6K	$0.000210

Claude 3.5 Sonnet	API	Standard queries	$92.0K	$0.0084
Llama 3.1 70B	Self-hosted	Standard queries	$16.4K	$0.0015

GPT-4o	API	Complex reasoning	$54.8K	$0.015
Llama 3.1 405B	Self-hosted	Complex reasoning	$65.8K	$0.018

Total

API $159.0K

Self-hosted $86.8K