Interactive · Week 9 · Section 7

Where the money actually goes.

Set the workload — model, context length, output length, traffic. The calculator computes per-request and per-year cost using real 2026 hosted-API pricing, with toggles for prompt caching and the Batch API. The visceral lesson: small changes to model choice and caching strategy produce 10-50× cost differences. The team that ships at scale isn't the team with the best model — it's the team with the best routing. Use this to project your monthly bill before you build something at 1000 RPS only to discover it costs $5M/year.

Workload

Model —

Requests per second 10

10 req/s · 864k req/day · 26M req/month

Prompt caching Batch API −50%, 24h SLA

Per-request shape

Input tokens 4,000

system prompt + retrieved chunks + user query

Output tokens 200

model's generated response

Cache hit ratio 0%

fraction of input that hits the cache

Per request

—

input + output

Per month

—

at current RPS

Per year

—

the headline number

Savings vs no-opt

—

vs same model, no cache, no batch

Cost breakdown · per request

—

cached input

uncached input

output

Same workload, all six models

cheapest highlighted · current model has accent border

What's happening here

Pricing reflects hosted-API list rates verified August 2026. Cache economics differ by vendor: Anthropic (Opus 5, Sonnet 5, Haiku 4.5) charges 10% of input for cache hits with a 25% premium on writes; OpenAI (GPT-5.6 Sol, Terra, Luna) charges 50% of input for hits with no write premium; Google (Gemini 2.5 Pro, 3.6 Flash, 3.5 Flash-Lite) charges roughly 10% of input for hits with no write premium, plus an hourly cache storage fee this model omits. Long-context surcharges are also omitted — Gemini 2.5 Pro doubles above 200k input tokens and the GPT-5.6 family switches to long-context rates above 272k, so very large prompts cost more than shown. Batch API discount applies to both input and output for all three vendors with 24-hour SLA. Cache write premium is amortized across many requests and modeled as the steady-state cached-hit rate; first-request write penalty is small at production traffic. Source-of-truth lives as <InferenceCostCalculator/> in the curriculum codebase.