Rules we don’t bend
- Unknown stays unknown. An unpublished price shows as “—”, never as $0. Missing privacy metadata is not a privacy claim.
- Zero is not free. Zero-priced placeholders and :free routes are dropped, never shown as a free offering.
- Promos are labelled. Discounted prices always carry a promo badge, so a sale is never mistaken for the regular rate.
- Published rates stay raw. We normalize units to $/M but never edit a provider’s rate; estimates are computed on top.
- The provider’s own price wins. A direct provider API outranks aggregators and resellers for the same offering.
- Estimates, not invoices. Costs use your stated token mix; cache-storage charges and taxes are excluded.
How TokenWatch builds a comparable catalog
The current text catalog contains 1567 provider-specific offerings across 116 inference providers. Its source timestamp is .
An offering is a model-and-provider combination, not a unique model. Quantization and endpoint variants remain distinct. See the model directory for comparisons across providers.
Source order
Direct provider APIs are the first source because a provider is authoritative for its own prices. OpenRouter endpoint data comes next and is split into provider-specific rows. CSV and maintained fallback records fill gaps after live sources.
Inspect the text catalog, image catalog, and video catalog for published rates and each catalog's own generation timestamp. The text fetcher and fal.ai fetcher document the provider source URLs and fallback records. Image/video coverage also uses OpenRouter and fal.ai.
Price normalization
Text prices are stored as US dollars per million tokens. Sources quoted per token are multiplied by one million; Wafer cents-per-million values are divided by 100. Image prices keep their native flat-image, megapixel, or image-token units. Video prices are normalized to dollars per second.
Model identity and provider precedence
The deduplication key combines a canonical model ID with a normalized provider. Provider prefixes, dated aliases, and selected routing suffixes can normalize to one identity. Quantization suffixes remain part of the key, so FP8, NVFP4, and INT4 rows stay separate.
Cost calculations
Blended $/M and total cost use the same input, cached-input, and output percentages. A provider's explicit default cache-write billing rule takes precedence, even when its write tariff is lower than input; a missing explicit tariff makes fresh-input workloads unpriceable. Without that rule, fresh input uses a finite positive cache-write rate only when higher than normal input pricing; otherwise normal input applies. Cached reads use the published cache-read price or the original input rate. There is no separate write charge. Additional cache-storage charges are excluded, so this simplified estimate is not an exact bill.
Why the cheapest offering depends on the workload
These illustrative token mixes are not measured averages or quality recommendations. Retrieval-augmented generation (RAG) supplies retrieved context as input; caching that context can change the ranking. Each row below selects the lowest positive computable blended rate in this text snapshot, not necessarily a model capable of your task. Free/unpriced offerings are excluded. Promotions can affect the result.
| Example workload | Input / cached / output | Lowest-cost offering | Provider | Blended $/M |
|---|---|---|---|---|
| Cache-heavy agents | 2.5% / 97% / 0.5% | DeepSeek: DeepSeek V4.1 Flash | baidu | $0.0029 |
| Uncached retrieval/RAG | 80% / 0% / 20% | Mistral: Mistral Nemo | io-net | $0.018 |
| Balanced uncached | 50% / 0% / 50% | Mistral: Mistral Nemo | io-net | $0.019 |
| Output-heavy generation | 10% / 0% / 90% | ScaleDown: Summarize | scaledown | $0.0040 |
For one million total tokens, cost is input rate × input share + cache-read rate × cached share + output rate × output share, using the fresh-input rule above. Monthly Volume multiplies an entered daily volume by 30. Retries, tool calls, storage, tax, and other charges need separate consideration. Use the calculator to compare suitable models at your own mix; a 97% cached mix is inappropriate when you cannot reuse most input.
How recommendations work
The open model finder, the recommendation API, and the WebMCP recommend_model/recommend_provider tools share one engine. It answers “which open-weight model suits this workload, and which provider should run it?” for nine workload presets. It ranks candidates from this catalog snapshot; it is not a universal best-model score.
Workload presets
Each preset defines an assumed token mix, hard requirements, benchmark weights, provider weights, a quantization policy, and an absolute quality floor on its primary benchmark. The mixes price providers consistently; they are not measurements of your traffic. Weights and floors are documented judgments, calibrated against the 2026-10-04 catalog snapshot.
| Workload | Assumed input / cached / output | Requirements | Benchmark weights | Provider weights (price / TTFT / throughput / uptime) | Quality floor |
|---|---|---|---|---|---|
| Agentic coding | 2.5% / 97% / 0.5% | tool calling, ≥32K context | AA agentic 45%, AA coding 30%, AA intelligence 15%, LiveBench agentic coding 10% | 25% / 20% / 35% / 20% | AA coding ≥ 25 |
| Tool-using agents | 5% / 90% / 5% | tool calling, ≥16K context | AA agentic 40%, AA intelligence 25%, AA coding 20%, LiveBench instruction following 15% | 25% / 20% / 35% / 20% | AA agentic ≥ 10 |
| Long-context RAG | 8% / 87% / 5% | ≥128K context | AA intelligence 40%, AA agentic 20%, LiveBench reasoning 15%, LiveBench data analysis 15%, LiveBench language 10% | 35% / 20% / 20% / 25% | AA intelligence ≥ 15 |
| Structured extraction | 25% / 65% / 10% | structured output, ≥16K context | LiveBench instruction following 40%, LiveBench data analysis 25%, AA intelligence 25%, AA coding 10% | 35% / 20% / 25% / 20% | AA intelligence ≥ 15 |
| High-volume, low-cost | 70% / 20% / 10% | ≥8K context | AA intelligence 50%, LiveBench instruction following 30%, AA coding 20% | 55% / 10% / 20% / 15% | AA intelligence ≥ 10 |
| Chat assistant | 60% / 20% / 20% | ≥8K context | AA intelligence 55%, LiveBench language 25%, LiveBench instruction following 20% | 25% / 30% / 20% / 25% | AA intelligence ≥ 15 |
| Creative writing | 35% / 10% / 55% | ≥16K context | Arena Creative Writing 75%, AA intelligence 25% | 25% / 25% / 30% / 20% | Arena Creative Writing ≥ 1310 |
| Reasoning and math | 25% / 15% / 60% | ≥32K context | LiveBench math 50%, LiveBench reasoning 25%, AA intelligence 15%, AA agentic 10% | 25% / 15% / 35% / 25% | AA intelligence ≥ 15 |
| Frontend and UI development | 35% / 15% / 50% | tool calling, ≥32K context | Design Arena 40%, AA coding 35%, AA agentic 15%, LiveBench coding 10% | 25% / 20% / 35% / 20% | Design Arena ≥ 1120 |
Eligibility gates
By default only offerings with resolved open weights are considered. Open-weight status comes from reviewed overrides, then a strict majority of models.dev provider records, then known-closed creator priors; unknown status is excluded, not guessed. Subscription offerings are excluded, and :batch variants are excluded except for high-volume work. A provider must pass the workload’s tool-calling, structured-output, and minimum-context requirements, any ZDR or headquarters constraint the visitor sets, and known-issue checks (broken or unavailable blocks; degraded warns), and it must have a computable price at the assumed mix. An endpoint’s published parameter list is authoritative for capabilities; models.dev flags are used only when it is absent. Providers missing required capability or context metadata are listed as unverified and never enter the ranking. Low-bit quantizations (fp4, nvfp4, mxfp4, int4) are rejected for demanding workloads when a higher-precision provider qualifies, and the fallback is stated when none does.
Quality score and coverage rule
Each benchmark is converted to a percentile among eligible canonical models, and the available weighted percentiles are averaged on a 0–100 scale. The composite is then shrunk toward the cohort median in proportion to the benchmark weight that is missing. A model needs at least 50% of its workload’s benchmark weight observed to compete for best quality or best value; models below that coverage, or with no scores, are shown separately as not having enough benchmark data. Benchmark scores are relative within a workload; only the floor is absolute.
The three picks
- Best quality — the highest composite score among coverage-eligible models.
- Best value — chosen from the non-dominated frontier of quality versus cheapest blended price, as the point closest to the ideal after normalizing both axes. It is never a raw score-per-dollar ratio, which would let cheap weak models win.
- Cheapest good-enough — the cheapest priced model whose primary benchmark reaches the workload’s absolute floor. It is flagged when it costs more than the best-value pick.
Provider ranking and priority presets
Eligible providers for a model are scored on blended price at the assumed mix, time to first token (p50), throughput (p50), and uptime (the 30-minute window is preferred to the one-day window). Each metric is scaled against the other eligible providers; missing measurements are omitted and the remaining weights renormalized, so a missing metric is never treated as zero. Balanced uses the workload’s own provider weights; the other priority presets replace them:
| Priority | Price | TTFT | Throughput | Uptime |
|---|---|---|---|---|
| balanced | Workload’s own provider weights | |||
| cheapest | 70% | 10% | 10% | 10% |
| fastest | 15% | 35% | 40% | 10% |
| most-reliable | 10% | 10% | 15% | 65% |
Confidence
Every pick and provider rank reports its margin over the runner-up on a 0–100 scale: at least 10 points is stable, 3 to 10 points is moderately stable, and under 3 points is a close call. Best quality compares quality scores; best value compares distance to the Pareto ideal; cheapest good-enough compares relative blended cost. A close call should be read as a tie. A local sensitivity script perturbs one weight, floor, or mix at a time to check how often each default pick changes; it is a robustness screen, not a probability.
Arena preference ratings
Creative-writing recommendations use Arena Creative Writing ratings as their primary preference signal (75%), with Artificial Analysis intelligence as a secondary capability signal (25%). Chat recommendations keep benchmark capability as the pick ranking and expose Arena Text overall as a separate people’s preference ranking. Ratings come from the LMArena leaderboard dataset; Arena (LMArena), Leaderboard Dataset; filtered to the listed categories and matched to TokenWatch canonical IDs; source ratings are unchanged. Licensed under CC BY 4.0. Text overall ratings are dated 2026-10-15; Creative Writing ratings are dated 2026-10-15. Ratings are a snapshot of pairwise preference, not a universal measure of model quality.
Known limitations
Token mixes are assumed per workload. Benchmark and provider weights are judgment-based, and percentiles shift as the catalog changes. Some benchmark sets are thin (several LiveBench categories cover only a handful of open models), and Arena ratings match only part of the open-weight catalog. Provider headquarters, latency, throughput, uptime, and capability metadata are incomplete: excluding a country removes only providers whose headquarters is known, and unverified providers are not ranked. Recommendations exclude subscription plans, cache-storage charges, and taxes. The design is recorded in ADR 0012, with a dated comparison against outside leaderboards in the validation notes.
Privacy and policy data
ZDR tags come from endpoint-level OpenRouter data or reviewed provider metadata. Missing metadata does not become a positive or negative privacy claim. Provider pages link to reviewed policies when TokenWatch has them.
Benchmarks and performance
Quality indices and design-arena scores are sidecar enrichment, not prices. Variant matching is conservative to reduce false attribution. Throughput is displayed separately because speed, quality, context limits, and cost answer different questions.
Known limits
Provider catalogs can change between refreshes. Promotions may expire, regional prices may differ, and unpublished cache prices are unknown rather than zero. Treat TokenWatch as a comparison and estimation tool, then confirm a shortlisted provider's current terms.
Maintainer and corrections
TokenWatch is an open-source project by WyrdWerk, maintained by Yash Jain. Pricing tables rank by published rates and the stated workload, not by subscription badges. Report stale prices, incorrect model identities, or policy corrections through GitHub issues, with the affected offering, source URL, and observation date. Do not include API keys or private prompts.