My local LLM server is a mini PC with no graphics card

The job pipeline I wrote about last month scored postings on a Windows gaming machine reached over the VPN, borrowed for the purpose and only available when it happened to be on. I wanted a dedicated box that is always on, serves models to every device I own, and costs nothing per token.

The default build for that is a tower with a used RTX 3090, because local LLM advice starts and ends with VRAM. I went another way: a Minisforum AI X1 Pro-470, a mini PC the size of a hardcover book, with 96GB of RAM and no discrete graphics card. It runs an 80-billion-parameter coding model at 20 tokens per second. It works because of arithmetic, and working through that arithmetic changed how I think about hardware for this.

Memory bandwidth sets the token speed

A language model generates one token at a time, and for each token the hardware has to read the model’s active weights from memory. So generation speed is roughly memory bandwidth divided by bytes read per token. Compute barely enters into it. The cores spend most of their time waiting for RAM.

That one formula explains every number in this post. This machine moves about 65 to 70 GB/s in practice. A dense 70B model quantized to 4 bits is about 40GB, and every token requires reading essentially all of it: 65 divided by 40 is about 1.5 tokens per second, which is unusable. No setting can fix that, because the model reads too many bytes per token for the memory feeding it.

Mixture-of-experts models change the bytes side. An MoE model stores many small expert networks and routes each token through only a few of them. Qwen3-Coder-Next has 80 billion parameters in total but only about 3 billion active per token, so it occupies 50GB of RAM while each token reads just a few gigabytes. The bandwidth is unchanged, but the bytes per token drop to a fraction, so the formula gives about 20 tokens per second.

This inverts the usual buying logic. A 16GB graphics card has five times my memory bandwidth and cannot fit the model at all. 96GB of ordinary laptop RAM fits it with room to spare, and the small active fraction keeps it fast enough to use. So the strategy is to hold as many total parameters as RAM allows while keeping the active count small, and the best open models of 2026 all happen to be built that way.

The machine

The X1 Pro-470 has a Ryzen AI 9 HX 470 (12 Zen 5 cores), a Radeon 890M integrated GPU, two DDR5-5600 SO-DIMMs running dual channel for 96GB, and a 2TB NVMe drive. It also has an NPU rated at 55 TOPS that I ignore entirely: Linux support for it is immature, and the integrated GPU through Vulkan is faster and far better supported. I’ll check back on it in 2027.

Two setup details did most of the work. First, the OS is Bazzite, an immutable Fedora variant built for gaming machines. Immutable means updates are transactional and roll back cleanly, which is what you want on a box running unattended jobs at 3am, and the gaming heritage is not wasted because this machine moonlights as one. Everything server-side lives in containers and the home directory, which is how immutable distros are meant to be used. One thing surprised me: the image shipped without Docker, so the whole stack runs on rootless podman under systemd quadlet units instead of docker-compose.

Second, the kernel by default caps how much system RAM an integrated GPU can address at half the total. Three kernel arguments raise that ceiling to 80GB:

rpm-ostree kargs \
  --append-if-missing=amdgpu.gttsize=81920 \
  --append-if-missing=ttm.pages_limit=20971520 \
  --append-if-missing=ttm.page_pool_size=20971520

That leaves about 16GB for the OS, the containers, and the pipeline. Without these arguments the 50GB coder model never loads, because half of 96 is 48.

One endpoint serves five models

llama.cpp with its Vulkan backend does the inference. ROCm support for this iGPU is shakier, and Vulkan is what the community benchmark numbers for these chips were measured on. In front of it sits llama-swap, a small proxy that makes the whole thing feel like a hosted API: every request names a model, and llama-swap starts the right llama.cpp instance on demand, evicts whatever was loaded before, and unloads anything idle past a timeout. The result is one OpenAI-compatible endpoint that serves five models with no manual switching.

The roster, chosen by the formula above:

When I benchmarked the roster, the big MoE models all landed between 18 and 21 tokens per second, which is what bandwidth divided by active bytes predicts. The NVMe makes swapping painless: the 80B coder loads in about 12 seconds. And because idle models unload on their own, the RAM comes back automatically when I want to play something.

The 122B model is a squeeze. At 77GB it sits near the ceiling of what the iGPU can address, so it runs with a reduced 32K context window. It exists for questions where I want a considered answer and don’t care that it arrives overnight.

What it does all day

From my Mac, opencode (a terminal coding agent) points at the coder model over Tailscale, so coding sessions run against my own hardware from anywhere. The job pipeline from the last post moved here and grew: a scraper pulls new postings from job boards and company career pages into SQLite twice a day, the 4B model scores each one against a written profile of what I’m looking for, and a static HTML report publishes to my private network. A Telegram assistant runs 24/7 on the box, with a cheap cloud model as its brain and the local models as fallback; the high-volume scoring work never leaves the machine. Open WebUI provides a phone-friendly chat interface to all of it. A small FastAPI control panel shows container state, loaded models, pipeline stats, and logs, with an agent chat whose shell commands execute only after I click approve.

Security works the same way as on the rest of my homelab. There are no forwarded ports or public endpoints, and everything is reachable only over Tailscale. The LLM endpoint has no authentication, which is an acceptable decision only because nothing on the internet can reach it.

Then I wiped the OS

A few days after everything was running, I wiped the OS by accident. The details are not instructive, so I’ll skip to the recovery.

The box was rebuilt the same day, and the reason is a habit from day one: every configuration file on the machine was committed to a private git repo as it was created, including the quadlet units, the llama-swap config, the pipeline scripts, the model download script, and notes on every deviation from the plan. The models are 130GB of re-downloadable weights, so they were never backed up; everything else was. The restore meant reinstalling the OS, cloning the repo and replaying the setup in order: kernel arguments, containers, timers. Then I started the downloads and went to do something else.

The rebuild also re-tested every early decision, and one of them had been wrong the whole time. The first install used Tailscale’s built-in SSH, which broke under SELinux on the reinstalled system, so access is now ordinary OpenSSH keys over the tailnet. I would not have found that until it locked me out at a worse moment.

Where it falls short

Prompt processing is the weak side of integrated graphics. Generation is fine at 20 tokens per second, but reading a long prompt happens at a few hundred tokens per second, so an agent session that sends 50K tokens of context pauses noticeably before the first word comes back. This is a machine for patient work: chat, background pipelines, overnight batch jobs. It does not feel like a frontier model, and I still use Claude for hard problems and anything high-stakes.

The planned fix is a discrete GPU over the box’s OCuLink port, which would take over prompt processing and the small models while the big MoE models stay in system RAM. That is not attached yet. If the measured numbers justify it, it gets its own post.