Kubernetes GPU Autoscaler Replica Planner
Estimate the Kubernetes replica count and GPU capacity needed for a measured request load. This is a static capacity calculation, not an HPA configuration generator or a prediction of production behavior.
Result
Desired replicas
Additional replicas
Total GPU devices required
Additional GPU devices
Utilization at desired replicas
How to use this tool
- Enter the peak requests per second you want to serve and the throughput measured for one GPU-backed replica.
- Set the target utilization below the measured maximum to reserve capacity for bursts and variation.
- Enter the Kubernetes minimum, currently ready replicas, and GPU devices attached to each replica.
- Use the desired replica and additional GPU counts as a planning estimate; validate them with load tests and cluster capacity checks.
Assumptions
- All replicas are treated as identical and scale linearly with request load; queueing, batching, model variation, startup time, placement and GPU sharing are not modeled.
- The per-replica throughput must come from a representative benchmark of the same model, precision, request shape and hardware.
- The planner applies a minimum replica floor, but does not model maximumReplicas, HPA metric behavior, scaling policies, quotas or available cluster GPUs.
Technical details: what the fields mean
- Peak requests per second
peakRequestsPerSecond - The peak request rate the deployment must serve, in requests per second.
- Measured throughput per replica
throughputPerReplicaPerSecond - Sustained requests per second measured for one replica on the target GPU and workload.
- Target utilization
targetUtilizationPercent - Fraction of measured replica throughput to plan against. A lower target adds headroom.
- Kubernetes minimum replicas
minimumReplicas - The minimum replica floor configured for the workload. Desired replicas will not go below it.
- Currently ready replicas
readyReplicas - Replicas already ready to serve this workload; used only to estimate how many more to add.
- GPU devices per replica
gpuDevicesPerReplica - GPU devices requested by each replica, as configured for the workload.
Methodology
Capacity per replica at the target utilization = measured throughput per replica × target utilization. Desired replicas = max(Kubernetes minimum replicas, ceil(peak requests per second ÷ capacity per replica)). Additional replicas = max(0, desired − currently ready). GPU totals multiply replica counts by GPU devices per replica.
Worked example
Measured GPU deployment under a 1,200 requests/s peak
Inputs: Peak requests per second: 1200 requests/s, Measured throughput per replica: 90 requests/s, Target utilization: 75%, Kubernetes minimum replicas: 1 replicas, Currently ready replicas: 10 replicas, GPU devices per replica: 2 GPUs
Result: Capacity per replica at target utilization is 67.5 requests/s. Desired: 18 replicas; add 8 replicas (16 GPU devices). Total planned capacity: 36 GPU devices.
Practical guidance
- Benchmark throughput with the same model, precision, batch size and request shape expected in production; peak TOPS alone is not a per-replica throughput measurement.
- Use the desired count as an input to HPA or cluster-capacity planning, then validate scale-up time and GPU placement separately.
Common mistakes
- Using theoretical GPU TOPS as the replica throughput instead of a representative end-to-end benchmark.
- Setting target utilization to 100% when request bursts, latency targets or model variability require headroom.
FAQs
Does this generate a Kubernetes HPA manifest?
No. It estimates a replica count from user-supplied load and benchmark data; it does not configure HPA metrics or scaling behavior.
Why is the target utilization below 100%?
A lower target plans spare measured throughput for burstiness and variation. Choose a target from your own latency and load-testing requirements.
Last reviewed: 2026-10-18
A steady-state replica planner only. It does not model GPU memory, batching, queueing, autoscaler stabilization windows, node capacity, maxReplicas, quotas or scheduling constraints.
This tool provides an indicative estimate, based on the assumptions stated above.
Theme and tool preferences, along with saved and recently used lists, are stored only in this browser.