I don’t own any Apple hardware, but I think this should probably be quite usable without too much trouble:
Short version: yes, I think a 64 GB Mac mini is a reasonable machine for this model and this use case. I would worry much less about whether the model fits, and much more about whether the exact checkpoint + runtime + OpenCode combination remains reliable during a long tool-using coding session.
If I were setting this up, my default starting point would probably be:
- 64 GB unified memory
- Q5_K_M or Q6_K
- llama.cpp / Metal +
llama-server
- OpenCode through its OpenAI-compatible llama.cpp provider
- ~64K context initially, rather than immediately maximizing context
- leave reasoning/tool behavior close to the model/runtime defaults at first
- only change quantization/backend after verifying that the tool loop itself is stable
The six questions look roughly like this to me:
| Question |
My current guess |
| 1. 64 GB enough? |
Yes, comfortably enough for the useful quants. |
| 2. Quant? |
Q5_K_M as a conservative default; Q6_K if quality is the priority; Q4_K_M is also quite plausible. |
| 3. Backend? |
llama.cpp/Metal is a very sensible starting point. MLX/oMLX is also worth considering on Apple Silicon. |
| 4. OpenCode/tool use? |
Promising, but this is the part I would validate before treating the setup as production-stable. |
| 5. Speed? |
Expect tens of tokens/sec, not single-digit speeds, but context/prefill/backend matter a lot. |
| 6. Mac vs cloud? |
Mac makes sense for a single-user, always-available local agent; cloud wins for burst throughput, very large prefills, or parallel agents. |
Memory / quantization
The available GGUFs for this exact Huihui model are approximately:
| Quant |
File size |
| Q4_K_M |
21.2 GB |
| Q5_K_M |
24.7 GB |
| Q6_K |
28.5 GB |
| Q8_0 |
36.9 GB |
So 64 GB is not a “Q4 or it will not fit” situation.
I would probably start with Q5_K_M because it leaves a lot of headroom for macOS, context/cache state, OpenCode, an editor, test processes, etc. If coding quality is much more important than maximizing context, Q6_K is also very reasonable.
I would not automatically reject Q4 for agentic work, though. I did a small sanity test with the exact Q4_K_M Huihui checkpoint, and its basic tool protocol behaved cleanly in that test; more on that below. So I would not assume that a tool-calling problem seen with Q4 is necessarily caused by quantization.
Q8 also fits in 64 GB, but “the weights fit” is not the same as “I still have comfortable room for a long agent session”. I would only move to Q8 if an A/B test showed an actual coding-quality benefit worth the reduced memory margin.
Backend: llama.cpp first, MLX/oMLX as the other obvious option
For OpenCode specifically, llama.cpp is probably the easiest baseline because OpenCode documents llama-server as a local provider using an OpenAI-compatible endpoint.
A minimal starting point could look roughly like:
llama-server \
--hf-repo shennguyen/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-GGUF \
--hf-file huihui-qwen3.6-35b-a3b-claude-4.7-opus-abliterated-q5_k_m.gguf \
--alias huihui-qwen36 \
--jinja \
-c 65536 \
--host 127.0.0.1 \
--port 8080
Current llama.cpp builds have Metal enabled by default on macOS, and current llama-server also defaults GPU-layer selection to auto; the current server options are documented here.
Then something along these lines in OpenCode:
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"llama.cpp": {
"npm": "@ai-sdk/openai-compatible",
"name": "llama-server (local)",
"options": {
"baseURL": "http://127.0.0.1:8080/v1"
},
"models": {
"huihui-qwen36": {
"name": "Huihui Qwen3.6 35B-A3B",
"limit": {
"context": 65536,
"output": 32768
}
}
}
}
}
}
I would treat those context/output numbers as a starting configuration, not sacred values.
MLX/oMLX is the other backend I would keep on the shortlist. Apple Silicon is exactly the environment MLX is designed for, and there are already some very useful performance measurements for this exact checkpoint.
For example, oMLX reports this exact Huihui derivative on an M4 Pro 16-core GPU / 64 GB system at 8-bit:
- ~54.9 generation tok/s at 4K context
- ~36.2 GB peak memory
Benchmark
And on an M5 Pro 20-core GPU / 64 GB system at 8-bit:
- ~44.8 generation tok/s at 64K
- ~34.9 generation tok/s at 128K
- ~41.2 GB peak at 64K
- ~47.9 GB peak at 128K
Benchmark
Those are oMLX numbers, not llama.cpp/Metal numbers, so I would not use them as a direct llama.cpp performance prediction. But they are good evidence that the model itself is quite practical on a 64 GB Apple Silicon system.
Apple’s newly announced M5 Pro Mac mini supports up to 64 GB unified memory and 307 GB/s memory bandwidth, so if that is the configuration you are considering, the hardware side looks particularly plausible.
What I would expect for speed
I would mentally budget for “tens of tokens/sec”, rather than trying to predict one exact number.
There are really at least two different speeds that matter for a coding agent:
- prompt processing / prefill
- token generation
Generation can remain quite fast even when the agent experience starts feeling slow because the growing repository/tool history has to be processed.
The M5 Pro oMLX measurement above is a useful example: at 128K it still generates at ~34.9 tok/s, but the measured time-to-first-token for a full 128K prompt was about 158 seconds.
That is why I would not optimize the setup around maximum advertised context on day one.
For an agent, good prefix/cache reuse can matter as much as raw decode speed.
A practical sequence might be:
32K or 64K
↓
normal real repository work
↓
only increase context if you actually need it
rather than:
"the model says 262K"
↓
set 262K immediately
The base Qwen3.6-35B-A3B has a default 262,144-token context and Qwen recommends keeping at least 128K for its full long-context reasoning behavior.
But this derivative is not literally the base checkpoint. The reasoning-distilled parent was trained on 4096-token sequences, and its author describes 64K as usable at inference while also warning that hard problems may generate tens of thousands of reasoning tokens.
So I would treat extremely long-context quality on this derivative as something to validate, not something automatically inherited just because the underlying config permits it.
OpenCode / agentic coding is the main thing I would test
There is actually encouraging evidence here.
The base Qwen3.6-35B-A3B is explicitly evaluated as an agentic model. Its official model card includes SWE-bench-family results, Terminal-Bench, MCP-style evaluations, etc.
More directly relevant to your question: Qwen’s SkillsBench evaluation was run through OpenCode, on 78 self-contained tasks, averaged over five runs.
So I would not be worried that Qwen3.6-35B-A3B is fundamentally the wrong kind of model for OpenCode.
The uncertainty is narrower:
Does this exact reasoning-distilled -> abliterated -> quantized -> local runtime derivative preserve that behavior reliably over a long OpenCode trajectory?
That part is much less directly benchmarked.
The lineage is:
Qwen3.6-35B-A3B
↓
Claude Opus 4.7 reasoning distillation
↓
Huihui abliteration
↓
GGUF quantization
The reasoning-distilled model was trained on roughly 7,800 Claude Opus reasoning conversations with an attention-only LoRA.
Then the Huihui checkpoint applies abliteration. Huihui describes that implementation itself as a “crude, proof-of-concept implementation”.
That does not mean it will be bad at tools. It only means I would not directly copy the base model’s OpenCode benchmark onto the derivative and call the question settled.
A small sanity check on the exact checkpoint
I tried a deliberately small tool-protocol sanity test with the exact:
Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated
Q4_K_M
The purpose was not to benchmark a Mac, but simply to see whether the checkpoint itself could still produce the tool syntax defined by its GGUF chat template.
I tested:
- thinking enabled
- thinking disabled
- repeated calls after an explicit reset
- repeated calls reusing the same model context
- a case where no tool should be called
- a longer system prompt
- 20 irrelevant/distractor tools
- more than one tool call in one response
- a short multi-turn sequence where tool results were fed back to the model
In that small test:
- the required tool was selected consistently
- the no-tool case did not spuriously invoke one
- multiple tool calls were emitted correctly
- the returned XML was balanced and parseable
- thinking-on and thinking-off both worked
- the short multi-turn sequence completed correctly
So I would move this hypothesis quite far down the list:
“The Huihui Q4 checkpoint itself simply cannot do tool calling.”
At least in a short controlled test, it clearly can.
That still leaves a separate integration question, though.
Why a model can emit correct tools and OpenCode can still fail
A local coding agent has several layers:
model
↓
chat template
↓
raw tool-call syntax
↓
runtime parser
↓
OpenAI-compatible tool_calls JSON
↓
OpenCode / AI SDK
↓
tool execution
↓
tool-result history
↓
next turn
A failure in one layer can look like a failure in another.
Qwen3.6’s template can use an XML-like intermediate representation for tool calls. A serving runtime such as llama.cpp parses that representation and exposes it as structured OpenAI-style message.tool_calls.
So seeing literal:
<tool_call>
...
</tool_call>
in an agent UI does not necessarily mean “Qwen decided to use the wrong API format”. It can also mean the conversion step did not happen correctly.
There is a particularly relevant open OpenCode issue where Qwen3.6 + llama.cpp on macOS eventually exposes a naked tool call during a long coding session:
OpenCode #24316 — Qwen3.6 progress halts with naked tool call
The report shows the failure around 94.5K context during an actual long-running repository task.
Importantly, the reporter explicitly does not establish whether the root cause is Qwen, llama.cpp, or OpenCode.
That is a useful failure example, but not evidence that every Qwen3.6/OpenCode session fails.
There have also been llama.cpp-side parser edge cases. For example:
llama.cpp #26763
documents a Qwen3.6 case where a tiny formatting variation around </parameter> caused the parser to consume following tool-call text as part of an argument. That issue is now closed as not planned, so I would treat it as evidence about the type of boundary failure that can occur, not as an “open blocker”.
There is also a currently open issue involving this combination:
Qwen3 template
+ enable_thinking=false
+ tool_choice=required/named tool
where the forced tool constraint is not enforced:
llama.cpp #27767
That is one reason I would initially leave reasoning/tool behavior close to the normal template defaults instead of immediately forcing a special no-thinking configuration.
The important practical point is just:
tool reliability belongs to the model + template + runtime parser + client combination, not the model alone.
A very cheap diagnostic if OpenCode misbehaves
I would not build a giant evaluation harness first.
If a real OpenCode task fails, the highest-information first comparison is probably:
same model
same llama-server
same prompt/task
OpenCode
vs
direct /v1/chat/completions request
If the direct API produces a valid structured tool call but OpenCode fails, the search space gets much smaller.
If both fail, then test:
cache/reuse ON
vs
fresh/reset path
If the problem is still present, then:
base Qwen3.6
vs
Huihui derivative
Only after that would I spend much time on:
Q4 vs Q5 vs Q6
In other words, I would diagnose in roughly this order:
1. Client boundary
OpenCode vs direct llama-server
2. Runtime/state boundary
cache/reuse vs fresh state
3. Model/checkpoint boundary
base Qwen3.6 vs Huihui
4. Quantization
Q4 vs Q5 vs Q6
That order gives a lot of information without requiring a large benchmark.
One OpenCode-specific performance trap
If OpenCode feels much slower than calling the local model directly, do not immediately blame Apple Silicon or the model.
There is an open report where a local llama.cpp request took about 2.5 seconds directly, but about 22 seconds through OpenCode; supplying a fixed session title reduced the OpenCode path to roughly 5 seconds.
OpenCode #33126 — local llama.cpp models slow with session title generation
That does not mean you will reproduce those exact numbers. It is just a good example of why I would measure:
direct llama-server latency
vs
OpenCode end-to-end latency
before deciding the local hardware is too slow.
Mac mini vs cloud GPU
For one developer running one main coding agent, I think the Mac mini is a very defensible choice.
The Mac is attractive if you value:
- always-on local availability
- no per-hour inference bill
- privacy/local repositories
- quiet operation
- simple unified-memory access to a fairly large model
- long daily usage
Cloud is more attractive if you need:
- very fast cold-prefill of huge contexts
- multiple simultaneous agents
- very high throughput
- experimentation with much larger models
- occasional burst use rather than continuous use
- the ability to change GPU size on demand
The trade-off is therefore not really:
Mac = slow
cloud = fast
It is closer to:
Mac:
persistent, private, predictable, single-user
Cloud:
elastic, high-throughput, high-prefill, parallel
For the workload you described — one primary coding agent reading a repository, editing, testing, debugging, and iterating — I would not consider a 64 GB Mac mini an unreasonable compromise at all.
I would probably only prefer cloud by default if you already know that you routinely need huge contexts with fast first-token latency, several simultaneous agents, or models substantially larger than this one.
A little more context on why this 35B model is relatively practical
The base model is nominally a 35B model, but it is an MoE:
- 35B total parameters
- ~3B activated per token
- 40 layers
- hybrid Gated DeltaNet / attention architecture
So its runtime characteristics are quite different from a dense 35B model.
That is one reason generation throughput can be surprisingly good on memory-bandwidth-oriented machines such as Apple Silicon.
The same hybrid/recurrent architecture also means that long-session state/cache handling is more complicated than the familiar “ordinary Transformer KV cache” mental model.
That is another reason I would pay attention to prefix reuse and long-session behavior instead of using only a simple tok/s benchmark.
One non-performance caution
Because this is an abliterated checkpoint, its refusal/safety behavior has intentionally been reduced; the model card explicitly warns about this.
For ordinary chat that may simply be a model preference.
For an agent that can:
write files
run shell commands
delete/rename things
access credentials
it matters more.
I would therefore run it with the same basic precautions I would use for any autonomous local coding model:
- work on a git branch/worktree
- keep production credentials out of the environment
- review destructive shell actions
- give it only the filesystem permissions it needs
- keep tests/builds isolated when practical
That does not change whether the Mac is suitable; it just keeps “uncensored model” and “unrestricted machine access” as two separate decisions.
My default recommendation
If I were buying/configuring specifically for this use case, I would probably do:
64 GB Mac mini
↓
Q5_K_M first
↓
current llama.cpp / Metal
↓
llama-server OpenAI-compatible endpoint
↓
OpenCode
↓
64K context initially
↓
one real repository task with ~20–30 tool turns
If that is stable, I would simply use it.
Then optimize based on what is actually limiting you:
Need more quality?
→ try Q6
Need more memory/context headroom?
→ try Q4
Need more Apple-Silicon throughput?
→ compare oMLX/MLX
Tool calls break?
→ compare direct llama-server vs OpenCode
Long sessions degrade?
→ inspect cache/state/context behavior
Cold 128K+ prompts are too slow?
→ cloud GPU becomes more attractive
So overall: I think your plan is technically reasonable.
The part I would validate before making strong claims is not “can 64 GB run the weights?” — that part looks fairly comfortable.
It is:
Can this exact derivative remain a reliable OpenCode agent over the length and tool complexity of your real software-development sessions?
The base Qwen3.6 evidence is encouraging, the exact checkpoint fits comfortably, the available Apple Silicon performance data looks quite usable, and the exact Q4 checkpoint can at least produce clean short-horizon tool calls.
That is enough that I would proceed with the Mac idea rather than reject it because it is a “35B” model — I would just keep the model/runtime/OpenCode layers separate when diagnosing anything odd.