Two vLLM servers on one GPU: the startup check that bites, and the fix

Someone asked on the vLLM tracker how to run a small embedding model next to a chat model on one GPU, so the card isn't half idle. I tested it on a rented RTX 3090 (24 GB) with vLLM 0.27.1: BAAI/bge-small-en-v1.5 for embeddings and Qwen2.5-1.5B-Instruct for chat, each in its own vllm serve. It works Embedding server at --gpu-memory-utilization... Weiterlesen: Two vLLM servers on one GPU: the startup check that bites, and the fix