View of A Goodput-Centric Benchmark for Large Language Model Inference under Realistic Online Load

Return to Article Details A Goodput-Centric Benchmark for Large Language Model Inference under Realistic Online Load Download Download PDF