Topic
vllm
4 articles
Running a big LLM across multiple GPUs with vLLM
A runbook for serving a model too big for one GPU: download to serving in seven steps, with every vLLM flag, startup log line and real error explained, plus tensor, pipeline and expert parallelism benchmarked head to head on a 235B model across four RTX PRO 6000 cards.
Running Qwen3.8-Flash-Next on a DGX Spark and RTX PRO 6000
Qwen dropped Qwen3.8-Flash-Next this week, and the first thing I saw on my timeline was somebody saying it will not fit on a single DGX Spark. The NVFP4…
The Local LLM Glossary: Every Term, Flag, and Number in Plain English
Plain-English definitions for every term you hit in local LLM posts: prefill and decode, tokens per second, FP8 and NVFP4, Q4_K_M, KV cache, YaRN, Gated DeltaNet, speculative decoding, and every vLLM, llama.cpp, and Ollama flag worth knowing.
Running Qwen3.8-27B on DGX Spark
Qwen3.8-27B on DGX Spark with llama.cpp, Ollama, vLLM, and SGLang: the recipes, the tokens per second I measured, MTP speculative decoding, and the sharp edges I hit along the way.