LLM Request Batching API System Design

## The Challenge Design an HTTP API for a Large Language Model (LLM) service. Users send single requests, but the system must group these requests into batches to run efficiently on GPUs. ### The Prob