Memory-First Sizing: GPU count determined by total memory requirements (model weights + KV cache), not compute requirements.

KV Cache Calculation: Uses actual model architecture details (layers, hidden size, attention heads) with support for GQA and MoE models.

Memory-Bandwidth Bound: Throughput estimates assume memory bandwidth is the bottleneck, typical for autoregressive inference.

Real-World Efficiency: Throughput includes configurable efficiency modifier (default 70%) to account for batching and scheduling overhead.

Model Parameters: Extracted directly from HuggingFace model configs when available, with manual input fallback.

Precision Support: Supports 4-bit through 32-bit precision for both model weights and KV cache independently.

Model Instance Deployment: Small models use 1 instance per GPU. Models using >80% of GPU memory span 2 GPUs. Large models can span up to 4 servers per instance.

Server Configuration: Default 8 GPUs per server. Large models span multiple servers (2-4) per instance. Smaller models achieve better utilization with multiple instances per server.