LLM Context-Window Token Cost & Latency Sizer
Size an LLM request against its context window, estimate input/output token cost and prefill/decode latency, and estimate KV cache memory for concurrent sequences. Prices and throughput are user-supplied; this is not a provider quote or benchmark.
Result
Context-window status
Estimated cost per request
Estimated 30-day token cost
Estimated prefill + decode time
KV cache per sequence
KV cache for concurrent sequences
Context tokens remaining
Tokens over context window
How to use this tool
- Enter the full prompt token count (including system and tool context), generated tokens and model context window.
- Enter input/output prices per 1M tokens, requests per day and measured prefill/decode throughput.
- For KV cache, enter Transformer layers, KV heads, head dimension, bytes per element and concurrent sequences.
- Review context fit, estimated token spend, serial prefill-plus-decode time and KV cache size; compare with measured serving behavior.
Assumptions
- Prompt tokens include all input context, while output tokens are added to test context-window fit and KV cache length.
- The KV cache estimate assumes a conventional dense K/V cache with the same bytes per element at every layer. It excludes weights, allocator/page-table overhead, quantization metadata, prefix sharing, sliding-window layers and fragmentation.
- Latency is a simple sum of prefill and serial decode time from measured throughputs; it excludes queueing, batching gains, network time, TTFT overhead and speculative decoding.
- Token prices are fixed user inputs in GBP per 1M tokens; no provider pricing feed, taxes, caching discounts or tiered rates are included.
Technical details: what the fields mean
- Prompt tokens per request
promptTokens - All input tokens for an average request, including system prompts, retrieved context and tool descriptions.
- Generated tokens per request
outputTokens - Average generated tokens per request.
- Context window
contextWindowTokens - Maximum model context in tokens for the selected deployment and configuration.
- Requests per day
requestsPerDay - Average requests billed per day; monthly projection uses a 30-day month.
- Input price per 1M tokens
inputPricePerMillionTokensGbp - Your input-token price in GBP per 1M tokens. Enter the rate from the provider or internal cost model.
- Output price per 1M tokens
outputPricePerMillionTokensGbp - Your generated-token price in GBP per 1M tokens.
- Measured prefill throughput
prefillThroughputTokensPerSecond - Measured prompt prefill throughput for the model and deployment, not a vendor peak claim.
- Measured decode throughput
decodeThroughputTokensPerSecond - Measured decode throughput per sequence under the batch and serving setup being sized.
- Transformer layers
modelLayers - Transformer layer count used in the KV cache estimate.
- KV heads
kvHeads - Number of key/value attention heads, not query heads. Use the model configuration's KV-head count.
- Head dimension
headDimension - Dimension of each attention head in elements.
- KV cache bytes per element
bytesPerElement - Storage bytes per KV cache element, such as 2 bytes for FP16/BF16.
- Concurrent sequences
concurrentSequences - Simultaneously active sequences whose KV caches need to fit in memory.
Methodology
Request cost = prompt tokens × input price + output tokens × output price, divided by 1,000,000. Thirty-day cost = request cost × requests/day × 30. Estimated serial latency = prompt tokens ÷ measured prefill tokens/s + output tokens ÷ measured decode tokens/s. KV cache bytes per sequence = 2 (K and V) × layers × KV heads × head dimension × bytes/element × (prompt tokens + output tokens); total KV cache multiplies this by concurrent sequences. MiB/GiB use powers of 1024.
Worked example
8,192-token context, 5,000 requests/day and eight concurrent sequences
Inputs: Prompt tokens per request: 4096 tokens, Generated tokens per request: 1024 tokens, Context window: 8192 tokens, Requests per day: 5000 requests/day, Input price per 1M tokens: 2 GBP/1M tokens, Output price per 1M tokens: 8 GBP/1M tokens, Measured prefill throughput: 800 tokens/s, Measured decode throughput: 60 tokens/s, Transformer layers: 32 layers, KV heads: 8 heads, Head dimension: 128 elements, KV cache bytes per element: 2, Concurrent sequences: 8 sequences
Result: The request fits with 3,072 context tokens remaining. Estimated cost is £0.01638/request or £2,457.60 per 30 days; serial prefill + decode is 22.19 seconds. KV cache is 640 MiB per sequence and 5.00 GiB for eight sequences.
Practical guidance
- Use the exact model configuration: KV heads, head dimension, layers and cache data type can differ across model variants and quantization modes.
- Benchmark prefill/decode throughput and token prices for the actual serving batch, caching policy and provider before sizing production capacity.
Common mistakes
- Using query heads instead of KV heads for grouped-query attention.
- Counting only user-visible prompt text while omitting system, retrieval or tool tokens from the context window.
- Treating the KV cache estimate as total model VRAM; model weights and runtime overhead are excluded.
FAQs
Does this predict provider latency?
No. It sums user-entered measured prefill and decode rates and does not model queueing, batching, network or provider scheduling.
Does KV cache use the whole context window?
This estimate uses prompt plus generated tokens for each active sequence, not the unused remainder of the context window.
Last reviewed: 2026-10-18
A user-input estimate for one model configuration and average request. It does not retrieve model specifications, prices, provider limits or live serving telemetry.
This tool provides an indicative estimate, based on the assumptions stated above.
Theme and tool preferences, along with saved and recently used lists, are stored only in this browser.