Frequently asked questions

Pick one of nine workloads on the Choose page, such as agentic coding, long-context RAG, structured extraction, chat, creative writing, or UI work. Each workload preset sets an assumed token mix, hard requirements (tool calling, structured output, minimum context), benchmark weights, and an absolute quality floor. Models that pass those gates and have at least one confirmed, priced provider receive a benchmark composite score. TokenWatch then shows three picks: best quality (highest composite), best value (a balanced point on the quality-versus-price Pareto frontier, not a score-per-dollar ratio), and cheapest good-enough (the cheapest model whose primary benchmark clears the floor).

Open-weight models can be hosted by several competing inference providers, so the same model can be compared on price, speed, uptime, and data policy across hosts. Open-weight status comes from reviewed overrides, then a strict majority of models.dev provider records, then known-closed creator priors; a model with unknown status is excluded rather than guessed. Turn on “Include proprietary models”, or pass include_proprietary=true to the API, to widen the list.

Providers must first pass the workload gates: required capabilities, minimum context, any ZDR or headquarters constraint you set, known-issue checks, and a computable price at the assumed mix. Eligible providers are then scored on blended price, time to first token (p50), throughput (p50), and uptime, preferring the 30-minute uptime window. Missing measurements are left out and the remaining weights renormalized. Providers whose capability or context metadata is missing are listed as unverified, never ranked. The priority control (balanced, cheapest, fastest, most reliable) changes those provider weights.

Every pick and provider rank carries a confidence label from its margin over the runner-up on a 0–100 scale: 10 points or more is stable, 3 to 10 points is moderately stable, and under 3 points is a close call. Treat a close call as a tie and decide on price, speed, policy, or your own evaluation.

A model must have at least half of its workload’s benchmark weight observed to compete for best quality or best value. Scores from partial evidence are shrunk toward the cohort median, and models below that coverage, or with no scores at all, are listed separately. Missing benchmarks are unknown, not a sign the model is poor.

Creative-writing and chat preference ratings come from the official LMArena leaderboard dataset on Hugging Face (Text Arena overall and creative-writing categories), licensed CC BY 4.0 and refreshed weekly. Creative writing is preference-led: 75% Arena Creative Writing rating plus 25% Artificial Analysis intelligence. Chat picks stay capability-ranked, and the Arena Text favourite is shown separately as the people’s favourite. Ratings are unchanged and matched conservatively to TokenWatch model IDs.

No. Each workload uses an assumed input, cached-input, and output mix so that providers can be priced consistently; the mixes are listed on the methodology page and at /api/v1/use-cases. Weights and floors are documented judgments, not measured truths. Open the calculator with your own mix before committing spend, and verify provider prices and policies.

Switch the calculator to Budget → Tokens, enter $10, and choose a workload mix. TokenWatch shows how many tokens each priced offering can serve. A cheap rate does not guarantee the model has the quality, context window, or capabilities your task needs.

Search for the provider and model, enter your token volume, then set the input, cached-input, and output percentages. The result uses that provider offering’s current rates rather than a model-wide average.

Search for the model and compare every matching provider row. The cheapest provider can change with your token mix because input, output, cache-read, and cache-write rates differ.

The same underlying model can be hosted by several inference providers. Quantization, batch, fast, preview, and endpoint variants can also remain separate when they have different identities or prices.

Choose Monthly Volume, enter daily token usage, and select a workload preset or your own mix. TokenWatch multiplies the daily result by 30; retries, tool calls, and context growth still need to be represented in the volume you enter.

Blended $/M combines fresh-input, cached-input, and output prices using the selected workload percentages and the same cache-write assumption as total cost. It is a rate per million tokens, so it excludes token volume and the monthly multiplier.

A provider can charge a lower cache-read rate when repeated prompt content is reused. TokenWatch applies that rate only to the cached-input share you enter.

A missing cache-read price is not free usage. TokenWatch falls back to the provider’s normal input rate for the cached-input share unless the source publishes a separate cache price.

When a provider explicitly bills fresh input at the cache-write tariff by default, TokenWatch uses that rate even when it is lower than normal input pricing. Otherwise it assumes fresh input is written to cache when a finite positive cache-write rate is higher than input; if not, normal input applies. A missing explicit default tariff makes fresh-input workloads unpriceable. Cached reads use the published cache-read rate or original input price. There is no separate write charge, and additional cache-storage charges are excluded; this is a simplified estimate, not an exact bill.

ZDR means the provider says request content is not retained beyond processing. It is a data-retention property, not a general security or compliance certification; review the linked provider policy before sending sensitive data.

You can compare up to six offerings, copy a comparison card as an image, export the current result set as CSV, or share the URL hash that stores the calculator state.

The current catalog contains 1567 text offerings across 116 providers and records a generation timestamp. Direct-provider data takes precedence over OpenRouter, then maintained fallback sources. Promotions and provider pages can change between refreshes, so verify prices before a purchasing decision.

Choose Budget → Count, enter $10, and filter to a model or provider. Flat per-image offerings return a direct count; megapixel and image-token offerings need workload details before a final image count is meaningful.

Per-image pricing charges a flat amount for one generated image. Per-megapixel pricing scales with output area. Image-token pricing depends on the provider’s image-token calculation. The three units are not interchangeable.

A flat image rate, a megapixel rate, and an image-token rate measure different work. TokenWatch separates them rather than inventing assumptions about resolution or token use.

Providers may price sizes, quality tiers, aspect ratios, edit modes, or endpoint versions separately. The calculator keeps those variants visible so a cheaper option is not mistaken for an equivalent configuration.

Yes. Search or filter by model and provider, then compare rows that use the same billing unit and a comparable output configuration.

Resolution can raise cost directly under per-megapixel pricing and can select a more expensive variant under flat pricing. Use the variant filter and confirm the provider’s size limits.

Multiply the selected per-second rate by 30. The crawlable table shows that example for each model’s cheapest listed variant, while the calculator lets you change duration and filters.

Choose Budget → Seconds and enter $10. TokenWatch divides the budget by each selected variant’s per-second price.

Higher-resolution variants often have a higher per-second rate. TokenWatch shows the resolution attached to each price so a 720p rate is not presented as a 1080p or 4K rate.

Some endpoints include generated audio, some exclude it, and some do not state the audio mode. Use the audio filter and compare like-for-like variants.

A provider can publish separate prices for resolution, audio, generation mode, or endpoint variants. TokenWatch preserves those records and selects only the cheapest one for the crawlable summary table.

Yes. Filter by model, resolution, and audio, then compare the per-second and total-duration costs of the remaining rows.

A single 0-100 score from Artificial Analysis, an independent lab that runs the same evaluations on every model. It blends nine practical tests: real-world work tasks, terminal-based coding, tool use, scientific reasoning, knowledge, and long-context reasoning. Think of it as a general capability rating measured identically for everyone. A 60 is genuinely strong; solid workhorse models land in the mid-40s. It anchors the Reasoning & Knowledge and Chat & UI Quality tabs.

A 0-100 zoom-in on autonomous multi-step work: using tools, navigating a terminal, recovering from errors without a human in the loop. It is the primary metric of the Agentic Coding tab because it measures the exact skill bots and automation need. A model can chat beautifully and still score poorly here.

A 0-100 measure of writing and fixing real code, from AA's own coding evaluations. Distinct from the Agentic Index: coding is "can it produce correct code", agentic is "can it complete a whole task on its own". Shown together on the Agentic Coding tab because real coding bots need both.

A benchmark suite from an academic consortium that refreshes its questions every six months, so models cannot have memorized the answers from training data ("contamination") — a real problem for older benchmarks. Scores are objective right/wrong results, not opinions, and every model runs the same release. We publish the release date we use.

LiveBench's test of producing working code in JavaScript, TypeScript, and Python inside an agent harness — the model must iterate, run, and fix its own code. It anchors the Agentic Coding tab alongside the AA indices because it measures execution, not just code writing.

Theory-of-mind, spatial, logic-puzzle, and navigation-logic tasks — multi-step "think it through" problems with verifiable answers. Shown on the Reasoning & Knowledge tab; a good proxy for analysis and problem-solving workloads.

Competition and olympiad mathematics plus integral solving. It stresses precise symbolic reasoning where a near-miss is still wrong. Useful when your workload cannot tolerate plausible-but-wrong derivations (financial models, scientific code).

Joining tables, reformatting tabular data, and tracking events across records — the spreadsheet-and-database skills behind most knowledge-work automation. It anchors the Knowledge Work tab.

Whether the model does exactly what was asked — paraphrasing under constraints, simplifying without losing meaning, hitting story and summary requirements. The difference between "a good answer" and "the answer you specified"; critical for repeatable workflows.

Wordplay, connections, plot reconstruction, and typo detection — precision with language itself rather than world knowledge. Complements instruction following on the Knowledge Work tab.

A head-to-head vote: two models build a website or UI from the same prompt, humans pick the better result, and Elo scores accumulate like chess ratings. Around 1300 is decent; 1450+ is excellent. It anchors the Chat & UI Quality tab and appears on Agentic Coding because frontend output quality is part of shipping. The model detail view also shows each model's best category (website, 3D, dataviz, and so on).

The cheapest price for that model across tracked providers, as a blended rate per million tokens at the token mix from the Text calculator (cached-heavy by default). The benchmarks page recomputes this live from the mix you last used on the Text tab, since the cheapest provider can change with the mix.

Capability per dollar: the tab's primary score divided by the blended price, scaled so the best model in the current view equals 100. It answers "if I do not need the absolute best, what gives me the most capability per cent?" It is a relative ranking within the current tab and filters, not an absolute measure, and it is never compared across tabs.

No one has published that benchmark for that model yet, or the evaluation is newer than our data. New models typically receive scores within days to weeks of release. Only models purchasable through a tracked provider are listed — a score without a price cannot be comparison-shopped.

Pick the tab closest to your workload: agents or coding bots — Agentic Coding; analysis, research, or reasoning — Reasoning & Knowledge; documents, summaries, spreadsheets — Knowledge Work; anything visual — Chat & UI Quality. When two models are within a few points, treat them as tied; price and speed usually decide it.