LLM API pricing is evolving from simple per-token charges to more nuanced structures: input/output separation, caching discounts, reserved capacity, and more. Meanwhile, cloud providers like Microsoft are actively promoting their own models as cheaper alternatives to OpenAI/Anthropic, making the choice involve not just capability but also cost structure, latency, and vendor lock-in.
Key players include OpenAI (GPT-4o series), Anthropic (Claude 3.5/4), Google (Gemini 2.0/2.5), xAI (Grok-2/Inkling), Meta (Llama 3.1/4 via cloud providers), and DeepSeek (V3/R1). Pricing models vary: OpenAI offers prompt caching and 50% discount on batch API; Anthropic charges per-token with a focus on long-context (200K tokens); Google provides 1-minute free quota. Self-hosting and third-party resellers (e.g., Together AI, Fireworks) offer more flexible cost control.
Getting started & choosing well
Start by listing your core requirements: peak throughput, acceptable latency, context length, and data privacy needs (self-hosting or private cloud). Then collect the latest pricing pages from each vendor and build a comparison table for input/output token price, caching discounts, and rate limits (RPM/TPM). Beware of hidden costs: system prompts occupying long contexts, or outputs being truncated due to length limits.
Core selection strategy: If load is stable and high, consider reserved capacity or batch API to save 40-50%. If requests often share common prompts (e.g., AI search, customer service), enable prompt caching to cut input costs significantly. For experimental projects, pick vendors with generous free tiers (like Google Gemini). Avoid early lock-in: use abstraction layers (LiteLLM, OpenRouter) to switch between providers.
Common pitfalls: 1) Only looking at input price but ignoring output price—many apps (code generation, writing) generate far more output tokens. 2) Overlooking rate limits—high concurrency may force you to upgrade tiers or self-host. 3) Ignoring tokenization differences—different models tokenize the same text differently, causing cost surprises. 4) Overlooking data usage terms—some providers may train on API inputs unless you opt out.
Frequently asked questions
Is per-token the only pricing model?
Not exactly. While most providers use per-token as base, they offer variations: separate prices for input/output, discounts for cached tokens, half-price batch processing, monthly reserved capacity, or subscription by time. Some also provide free tiers for light usage.
Should I choose cheap or powerful models?
Depends on task complexity. For simple classification or extraction, cheap models (e.g., GPT-4o mini, Llama 3.1 8B) offer high cost-efficiency. For complex reasoning or long-form generation, powerful models (GPT-4o, Claude 3.5 Opus) are necessary but costly. Consider using a cascade: try cheap model first, if confidence low, escalate to expensive one.
How to handle rate limits?
Assess your peak requests/minute (RPM) and tokens/minute (TPM), and pick a tier that meets them. If insufficient, you can: 1) upgrade tier (usually prepaid or usage-based); 2) load balance across multiple API keys; 3) self-host open-source models (e.g., Llama 4); 4) implement queuing and retries.
How much can prompt caching save?
If your app has repetitive system prompts or similar contexts (e.g., chatbot history), caching can reduce input costs by 50-90%. For example, OpenAI offers ~50% off cached input; Anthropic has 'write-then-read' caching for long contexts. Explicitly enable cache keys in API calls and monitor hit rates.
Self-hosting vs third-party API: which is cheaper?
Self-hosting is usually cheaper at high throughput (>1M tokens/day), but you must account for GPU rental/purchase and maintenance. For medium throughput, third-party resellers (Together AI, Anyscale) or cloud providers (AWS Bedrock, GCP Vertex AI) offer pay-as-you-go with easy switching. Low throughput: direct vendor API is simplest.
How to avoid vendor lock-in?
Use a unified SDK (LiteLLM, LangChain) to abstract APIs; maintain multiple endpoints and keys in config; regularly compare costs and plan migration. Keep an eye on open-source models (Llama 4, DeepSeek V3) that can be self-hosted or accessed via resellers, reducing dependency on a single vendor.