LLM Research & Benchmarks
Original research on LLM pricing trends, performance benchmarks, latency data, and value analysis across all major providers.
AI API Latency Comparison
Compare AI API latency across OpenAI, Anthropic, Google, and Mistral. Real first-token and total response time data.
Best LLM for Every Use Case (2026)
Data-driven recommendations for chatbots, code generation, RAG, summarization, translation, and more.
LLM Context Window Benchmark
Benchmark comparison of 30+ LLM context windows with real performance data at maximum context length.
LLM Pricing History (2023-2026)
Complete timeline of LLM API pricing changes. Track how GPT-4, Claude, Gemini, and Llama costs dropped 100x.
Where Published LLM Prices Disagree
Models whose published per-token prices two independent public sources did not agree on, checked 2026-08-24. The widest same model gap is 2x.
Gemini vs OpenAI vs Claude, tokens per second measured
96 streaming runs across 16 models on 6 September 2026. Median first token 2.0 s for GPT-5.4 Nano and Gemini 3.8 Flash, 389 tokens per second for Gemini 3.5 Flash Lite, and reasoning tokens explain most of the wait.
OpenAI Calculator Price Accuracy
Six ranking OpenAI cost calculators measured against two price sources on one day. Two displayed rates differing from both sources, up to 5x on GPT-5.6 Luna.
Claude Calculator Price Accuracy
Nine ranking Claude cost calculators measured against two price sources on one day. Four still print a cancelled Sonnet 5 price rise, and one shows Opus 5 at 3x the corroborated rate.
What a Telegram Bot Fleet Costs on Cloudflare Workers
A real cost sheet: Cloudflare's free-tier request and CPU limits, Telegram Stars payout math, and why a rule-based bot fleet's marginal cost stays near zero next to an LLM API bill.
LLM Speed Benchmark
Tokens per second across 30+ models. TTFT, streaming speed, and batch throughput for all major providers.
Context per Dollar Index 2026
Which models give you the most context window per dollar? Ranked index across 400+ live-priced models.
LLM Value Index 2026
40+ models ranked by quality per dollar. Sortable table with pricing, benchmarks, speed, and capabilities.
Open Source vs API: Break-Even Analysis
When does self-hosting Llama or Mistral beat Claude and GPT-4o APIs? Real GPU rental costs and TCO compared.
About This Research
Each research article provides original data-driven analysis using pricing, benchmarks, and performance data sourced from official provider documentation and independent testing. We cover models from OpenAI, Anthropic, Google, Meta, Mistral, DeepSeek, and Cohere. All data is updated regularly as providers announce changes. Use the KickLLM calculator for interactive cost modeling based on the data in these studies.
Built by Michael Lip. Research data updated regularly from official provider sources.