A personal always-on agent, with tool use and memory, running entirely on owned hardware: orchestrator on a 24/7 mini-PC, reasoning on a GPU workstation over a private mesh. The question was how capable a self-hosted agent can get on a single 24 GB GPU. The answer was in understanding where the latency was, not in making the model bigger.
This page describes a system running 24/7 on my own hardware. The repository publishes the architecture, the measurements and the trade-offs — the engineering argument, not the implementation. Every number below was measured on that machine.
A process, not a chatbot. It starts with the machine and stays running — no window to open, no session to remember. When a task needs reasoning, it hands the request to a GPU on the same network; the rest of the time it just executes.
It's a background service. You define once what it watches and how it should react, and it keeps doing that without a browser tab open.
Requests, context and outputs stay on hardware you own. There's no vendor to trust with what the agent sees — the trade-off is that the ceiling is whatever a single GPU can do.
The agentic core keeps tool use — browsing, fetching — so a task can go past producing text: check something, act on it, log the result.
The 27B model, "unusable" at 398 s, became snappy at 33 s once reasoning was capped and the KV cache fit into VRAM. The trick wasn't a bigger model.
| Problem | Finding | Fix |
|---|---|---|
| First response ~398 s (unusable) | The bottleneck wasn't the clock or the thinking (~17 s): it was thinking × agentic system prompt (tools + skills index) on every turn | reasoning_effort: low → 33 s. Caps reasoning per turn without turning it off; later turns hit the prompt cache and drop further |
| 64k of context didn't fit in 24 GB | The KV cache at full precision blew the VRAM budget | Quantized KV cache (q8_0) + flash attention → 64k running 100% on GPU |
| Dead weight in the toolset | computer_use is macOS-only → useless on a Linux host; image_gen/tts unused | Disabled; kept the agentic core + web and browser |
| Skills looked heavy (dozens of them) | Confirmed: they're progressive-disclosure — only a lightweight index sits in context, each skill loads on-demand | Kept all of them — they don't cost context |
| Two chats seemed to hang | Ollama runs one runner per model → concurrent requests queue and compete for the GPU | Diagnosed by inspecting processes; serialize the heavy calls |
The decision space was left open on purpose: a small model at higher reasoning effort vs. a large one at "low" is a real trade-off worth measuring, not assuming.
Supports tool-calling and thinking, quantized to run entirely on a single 24 GB GPU. q8_0 KV cache + flash attention is what makes the long context viable.
A more resource-hungry "personal agent" framework was evaluated and rejected for VRAM cost: it wanted 32B+ at long context, the most demanding case for a single 24 GB GPU. The lean orchestrator delivers the same promise within budget.
398 s → 33 s came from understanding, not from buying hardware.
low keeps thinking alive while making it usable.
KV cache quantization is what makes 64k of context viable in 24 GB.
A personal agent shouldn't depend on someone else's API to respond.