InferGrid
Hardware
By
@aarivpatel
- 1 Devlogs
- 1 Total hours
Choose GPU models, batch size, and context length. KV-cache sizing and Roofline estimates compare memory bandwidth with compute capacity, producing per-device memory budgets and modeled token-routing bottlenecks.
