This is the complete hermes local model setup, start to finish: pick the right small model, wire it into Hermes, prove it on a real agent task, and leave the whole thing running around the clock — without sending a single token to the cloud or paying for a single message.
Most people bolt Hermes onto a cloud API because that is the default path. It works, but every message has a price, everything you type leaves your machine, and the agent stops the moment your connection does. A local model fixes all three problems at once — and the newest generation of small models has quietly removed most of the old trade-offs.
This walkthrough is the current local-model route for the Agent OS stack: Hermes as the agent, a free local model as the engine, your own machine as the infrastructure. Older guides on this site covered earlier models; this page reflects the newest model guidance.
📺 Watch: LFM2.5-2.6B: New FREE Local AI
Why run Hermes on a local model at all?
Five reasons, all practical rather than ideological:
- Private by default. Most decent Hermes APIs are cloud-based, which means your messages and memory lookups travel to someone else's servers. A local model sends nothing to the cloud. Everything stays on your machine.
- Free. No paid API charging you per message. Once the model is downloaded, a thousand agent tasks cost the same as one: nothing.
- Works offline. Plane mode on, Wi-Fi off — Hermes keeps answering, because the model lives on your disk.
- No token anxiety. You stop rationing your own agent. Long tasks and big memory pulls run without a meter ticking in the background.
- Still runs 24/7. Local does not mean part-time. Scheduled tasks keep firing on time, so your agent works nights and weekends on hardware you already own.
The old objection was quality: small models used to fumble tool calls. That has changed — but only if you pick the right model, which is where most setups go wrong.
Step zero: pick your model
Every verdict here comes from Goldie Bench, Julian Goldie's hands-on testing for exactly this job: real agent tasks and real tool calls on ordinary machines, not leaderboard trivia.
The standout for Hermes right now is LFM2.5-2.6B from Liquid AI — and the reason is unusual. It was literally trained with Hermes as a harness, so it behaves like a model that grew up inside the agent it now powers. It is fast enough that it never slows the setup down, and it is genuinely strong at tool calls and agent tasks. What it is not is a coding model — more on that below. The full breakdown is in the LFM2.5-2.6B review.
Two alternatives earn a place. Maple Preview is the fast, lightweight reasoning option — ternary weights, built with mobile-class hardware in mind. Gemma 4 remains a solid all-rounder if you want one model for a bit of everything.
And one classic trap: the big-name downloads. A Qwen-class 27B model looks impressive on paper, then runs painfully slowly on a typical machine. A slow agent is an agent you stop using.
| Model | Strength | The catch |
|---|---|---|
| LFM2.5-2.6B (Liquid AI) | Trained with Hermes as its harness; fast, excellent at tool calls and agent tasks | Not a coding model |
| Maple Preview | Fast, lightweight reasoning; ternary weights, mobile-focused | A reasoning specialist, and still a preview |
| Gemma 4 | Solid all-rounder for general local work | Good everywhere, Hermes-specialised nowhere |
| Qwen-class 27B models | Big-name pedigree on paper | Painfully slow on typical machines |
Want a longer shortlist? The round-up of the best free models for Hermes covers more options. If you just want the answer: start with LFM2.5-2.6B.
📺 Watch: Hermes: New FREE Local AI Model
The complete hermes local model setup, step by step
Four steps, and none of them is a terminal marathon. The whole thing amounts to a few commands and a couple of settings.
- Get the model weights. Two easy routes: download LFM2.5-2.6B from Hugging Face, or load it through LM Studio and let that handle the serving. If you are running one of the other models, such as Gemma 4, Ollama does the same job. Either way it is a few commands or a few clicks — nothing to compile, nothing exotic to configure.
- Point Hermes at the local model. In your Hermes profile, set the local model as the model. That single setting is the entire switch: from that moment, every message routes through your own machine instead of a cloud API.
- Test it with a small agent task. Do not open with a monster request. Give it one modest job — a quick memory lookup or a single tool call — and watch it complete cleanly. You are checking the wiring, not the model's IQ.
- Split your profiles. Since the Quicksilver release, one gateway routes to multiple profiles. That means you can give one profile the local model and keep another on a frontier model — local for the everyday volume, frontier for the rare heavy lift, both behind the same front door.
That is the whole setup. No subscription appears anywhere in the chain: Hermes is free, the model is free, and the Obsidian-based memory underneath it is free too.
📺 Watch: Maple AI: FREE Local Model on Your iPhone
Three ways to run it day to day
1. The local model as your main agent
The simplest pattern: the local model handles your day-to-day agent work. Tool calls, memory retrieval, scheduled routines — the steady drumbeat of small jobs. This is what it was trained for, and it is what most Hermes usage actually looks like.
2. The delegate pattern
Keep a frontier model as the brain and let it delegate the token-heavy grunt work to the local model. The expensive model does the thinking; the free model does the volume. Your cloud usage collapses while the output stays the same.
3. Local sub-agents
When a job means high-volume repetition — hundreds of small calls, long processing loops — hand it to sub-agents running on the local model. High-volume work is exactly where a free, fast, always-on model earns its keep.
📺 Watch: Hermes Agent OS: Building Local + Free AI Agents
What to expect (the honest bit)
Local models are agent models. They are built for tool calls, memory retrieval and routine work — and at that job, the current crop is genuinely capable. They are not frontier coders. Ask LFM2.5-2.6B to architect your app and you will be disappointed; that is what your frontier profile is for.
So test it on real agent tasks, the way Julian Goldie does. Two that tell you everything:
- The /learn skill. Point it at a guide and have it learn the guide into a skill. That exercises reading, structure and tool use in one pass.
- Memory tool-calls against the Obsidian vault. Ask something only your own notes can answer, and watch it pull a personalised answer straight from the vault. This is the everyday magic of a local agent: your knowledge, your machine, nobody else in the loop.
If it passes both, you have a working local agent — private, free and offline-capable — and you can start moving real workloads onto it.
If you want Hermes running free on local models and making you money, check out the AI Profit Boardroom — the full Agent OS build-out and the free-running local stack training are inside. → Get your local agent earning
FAQ
Do I need an expensive machine?
No — that is the point of the model choice. LFM2.5-2.6B is small and quick, and Maple Preview was designed with mobile-class hardware in mind. The mistake is downloading a Qwen-class 27B model because the name is famous, then wondering why every reply crawls.
Is the whole stack really free?
Yes. Hermes is free, the recommended models are free downloads, and Obsidian-based memory is free. There is no paid API per message anywhere in the setup.
Can I keep a frontier model for hard problems?
Yes, and you should. Since Quicksilver, one gateway routes to multiple profiles — local model on one, frontier model on another. Or run the delegate pattern and let the frontier brain hand the grunt work down automatically.
Will a local model write my code?
Not well. Local small models are for agent tasks — tool calls, memory, routine work. For serious coding, switch to your frontier profile.
Does it genuinely work offline?
Yes. Once the weights are on your disk, plane mode changes nothing. And because scheduled tasks run locally too, the agent keeps working 24/7 whether or not you are connected.
The Bottom Line
The recipe is short: choose LFM2.5-2.6B, the Goldie Bench standout that was trained with Hermes as its harness; pull it via Hugging Face or LM Studio; set it as the model on a Hermes profile; and prove it with one small agent task. Add a frontier profile through the Quicksilver gateway if you want a heavyweight on call.
What you end up with is the useful kind of AI stack: private by default, free to run, happy in plane mode, and still doing its scheduled rounds at 3am. That is the Agent OS idea in its purest form — your agent, your model, your machine.
