Reproducible LLM Inference Needs Fixed Batch Geometry
vLLM's `VLLM_BATCH_INVARIANT=1` flag delivers reproducible LLM inference by making output "deterministic and independent of the batch size or the order of requests in a batch," per the project's own documentation. That one environment variable is the difference between a pipeline you
