---
license: apache-2.0
base_model: openbmb/MiniCPM5-1B
pipeline_tag: text-generation
library_name: coreai
tags:
- coreai
- core-ai
- coreml
- apple
- on-device
- iphone
- metal
base_model_relation: quantized
---

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's `coreai-torch` (LLMs: `coreai.llm.export`) into `.aimodel` bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol ([apple-silicon-llm-bench](https://github.com/john-rocky/apple-silicon-llm-bench), macOS 27 beta 26A5353q, 2026-06-11).

<!-- gen-cards:devicemark begin (managed by scripts/gen-cards + tools/devicemark_row.py — edit cards.json, not this block) -->
This model has no row on [DeviceMark](https://devicemark.github.io/), the on-device LLM leaderboard.
<!-- gen-cards:devicemark end -->

# MiniCPM5-1B — Core AI (int8 block-32, runs on iPhone)

Apple **Core AI** (`.aimodel`) conversion of [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B) —
OpenBMB's 1.08B on-device LLM with **hybrid Think / No-Think reasoning** and **128K** context, reaching
1B-class open-source SOTA. Runs fully on-device on **iPhone** and Apple Silicon Macs (GPU, pipelined engine).

> **Revision note (2026-09-09).** This revision replaces the per-channel int8 bundle published as
> `5ad650f`. That bundle's LM head had **dead rows from vocab id ~65024 up** — every token there
> scored ~0 in the engine, `<|im_end|>` (130073) included — so a chat turn **never halted** (it ran
> to the token cap) and any answer needing a high-id token lost it. The card's "halts cleanly" line
> was wrong. Details and the measurements are under *Why per-block-32* below; pin this revision or
> newer.

Part of the community Core AI model zoo: **https://github.com/john-rocky/coreai-model-zoo**

<!-- gen-cards:use-it begin id=minicpm5-1b (managed by scripts/gen-cards — edit cards.json / QuickStart.swift, not this block) -->
## Use it

**New to Core AI? [Start with CoreAIKit 0.7.3](https://github.com/john-rocky/coreai-kit#readme).** Follow its requirements and first-run steps for `qwen3-0.6b`, then open the same release's [ChatDemo](https://github.com/john-rocky/coreai-kit/tree/0.7.3/Examples/ChatDemo). The README records the tested OS/SDK and download size; model and device coverage is stated per example.

⚡ **One line** — run the kit's task op on this model
(`import CoreAIOps`; no session, no model plumbing, downloads on first use):

```swift
let tldr = try await CoreAI.summarize(text, options: .model("minicpm5-1b"))
```

Every op, one shape — [Cookbook](https://github.com/john-rocky/coreai-kit/blob/0.7.3/docs/COOKBOOK.md).

▶️ **Run it (source)** — the [ChatDemo runner](https://github.com/john-rocky/coreai-kit/tree/0.7.3/Examples/ChatDemo)
(GUI + CLI, one app for every chat model in the catalog):

```bash
git clone --branch 0.7.3 --depth 1 https://github.com/john-rocky/coreai-kit
export DEVELOPER_DIR=/Applications/Xcode-27.0.0-RC.app/Contents/Developer
open -a /Applications/Xcode-27.0.0-RC.app coreai-kit/Examples/ChatDemo/ChatDemo.xcodeproj
# → Run, then pick "MiniCPM5 1B" in the model picker

# agents / headless (macOS):
cd coreai-kit/Examples/ChatDemo
swift run -c release chat-cli --model minicpm5-1b --prompt "What can you do, offline?"
```

Use Xcode build **27A266a** from the release's `.xcode-pin`; adjust the app path if your installation is named differently.

💻 **Build with it** — complete; the glue is kit API, copy-paste runs:

```swift
import CoreAIKit

let id = "minicpm5-1b"
let chat: ChatSession
if id == "qwen3-0.6b" {
    // Freeze the release starter; other selections retain the live catalog's
    // model-specific dispatch (including paired Gemma bundles).
    guard let model = ModelCatalog.builtin.entry(id: id)?.modelID else {
        throw CoreAIKitError.modelNotAvailableOnPlatform(id: id)
    }
    chat = try await ChatSession(model: model)
} else {
    chat = try await ChatSession(catalog: "minicpm5-1b")
}
let reply = try await chat.respond(to: prompt)
// reply: the answer, generated fully on-device
```

The take-home is [`Examples/ChatDemo/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/0.7.3/Examples/ChatDemo/Sources/QuickStart.swift)
— this exact code as one typed function, no UI; the CLI is an argument shell over it, and
the GUI drives the same `ChatSession` across turns for its transcript.
Multi-turn? Hold the `ChatSession` and call `respond(to:)` per turn — it keeps the
conversation history; `streamResponse(to:)` yields tokens as they decode.

**Integration checklist**

- SPM: `https://github.com/john-rocky/coreai-kit` (exact **0.7.3**) → product **CoreAIKit**
- Info.plist: none needed
- Entitlements: none needed
- First run downloads the model — ~1,159 MB (Mac) / ~1,159 MB (iPhone) — then it loads from the
  local cache (Application Support; progress via the `downloadProgress` callback)
- Measure in Release — Debug is ~3× slower on per-token host work
<!-- gen-cards:use-it end -->

## Neural Engine bundle (iOS static export, AOT h18p) — 2026-09-15

`ios-ane-h18p/` is Apple's stock `coreai.llm.export --platform iOS` static export of this checkpoint
(k-means **8-bit palettized, group 32** (`conversion/minicpm5_pal8_g32.yaml`); embeddings int8; static graphs `prompt_opt`/`extend` ×
contexts {256, 512, 1024, 2048, 4096} × query {8, 16, 64}), AOT-compiled with
`xcrun coreai-build compile --platform iOS --preferred-compute neural-engine --architecture h18p`
(31/31 ANE regions, 1.3 GB). It loads through Apple's `EngineFactory` → `StaticShapeEngine` unchanged
(iPhone 17-class devices). `ios-static/` is the same export before AOT — the portable IR; compile it
for another chip yourself. Apple's default 4-bit preset **fails** the same gate on this checkpoint (two margin-clear flips on a chat turn, unchanged with fp16 embeddings; `gate-minicpm5-1b-ane-device-4bit-FAIL.json`), so the 1B ships at 8 bits. 

Gated on an iPhone 17 Pro (iOS 27.0) against the fp32 HF oracle: teacher-forced single-step sweep 24/24 + 10/10 + 16/16 and free-run greedy token-exact including the stop — transcript in the zoo,
`models/minicpm5-1b/gate-minicpm5-1b-ane-device.json`. Speed on the same phone (Apple llm-benchmark
method, 512-token prompt, 1024 generated, 5 trials, two back-to-back runs): decode **69.6 / 58.6 tok/s**, prefill 2710 / 2364 tok/s; footprint 1.3 GB (p128 / g256: decode 62.3). The Neural Engine is the power-efficient lane; these are
not a GPU comparison (different protocol from the `int8/` row above).

Same-day interleaved A/B on the same phone (one app embedding both bundles, 128-token prompt, 256 generated, 5 trials, ANE-GPU-ANE-GPU): Neural Engine 8-bit **76.8 / 76.8 tok/s** decode (3899 / 3865 prefill) vs this repo's `int8/` GPU bundle **67.1 / 64.4** (2359 / 2270) — equal weight bytes per token, so +17 % decode and +70 % prefill on the ANE; GPU cold specialization 3.6 s vs ANE AOT load 0.2 s.

## Measured

| | decode | prefill | numerics | size |
|---|---:|---:|---|---:|
| **iPhone 17 Pro** (A19 Pro, `PipelinedBench`, Release) | **61.7 tok/s** | 65.6 tok/s | **24/24 token-exact** vs HF fp32 on the margin-clean alphabet prompt (min fp32 top-2 margin 0.841) **+ 6/6 including the stop** on the no-think turn `1+1=?` (`1+1=2` then `<|im_end|>`); engine ready 7.3 s cold | **1.1 GB** |
| **M4 Max** (macOS 27, `llm-benchmark`, 512p/1024g) | **246.6 tok/s** | 6649 tok/s | **16/16 token-exact** vs the fp32 oracle (margin-aware gate, min margin 0.913) + the same 6/6 stop gate | |

Halt check through the engine on the same phone-shaped bundle: the Think-mode turn `1+1=?` stops on
its own after 171 tokens (cap 400); the no-think turn after 6.

⚠️ **iPhone context cap: prompt + generated tokens must stay under 1024.** The bundle declares a 131072
dynamic KV, and the shipped `CoreAIPipelinedEngine` caps iOS growing-KV capacity at 1024 (its guard
against the iOS compiler miscompiling growing-KV specializations at seq ≥ 2048). Chunk or trim the
history on iOS; macOS has no cap.

## Why per-block-32 (what was wrong with the previous revision)

The previous bundle (`5ad650f`, int8 **per-channel** absmax) passed a 24-token greedy parity check and
still never ended a chat turn. Teacher-forcing it through the engine against the fp32 reference
located the defect in the LM head, by vocab id:

| token (id) | fp32 P | per-channel bundle | per-block-32 bundle (this revision) |
|---|---:|---:|---:|
| `.lineTo` (65023) | 0.978 | 0.993 | — |
| `粒子` (65039) | 0.977 | **0.0000** | 0.98 |
| ` chromosomal` (65528) | 0.383 | **0.0000** | 0.380 |
| ` OpenAI` (130051) | 0.929 | **0.0000** | 0.923 |
| `\n\n` after `</think>` (130063) | 0.9999 | **0.0000** | 0.9999 |
| `<|im_end|>` after `1+1=2` (130073) | 0.873 | **0.0000** (top-1 `.`) | 0.867 |

Every probed row at id ≥ 65039 is dead in the per-channel bundle and every probed row ≤ 65023 is
healthy (16 probes; the boundary lies in that 16-id window, which contains 65024 = 127 × 512). Rows
below it match fp32 to ~0.01 in probability, which is why free-running English prose looked fine.
A fresh per-channel export on the same toolchain (coreai-torch 0.4.1 / coreai-opt 0.2.1 /
coreai-core 1.0.0b2) reproduces the shipped bundle's logits to four decimals, so this is a property
of the per-channel int8 path for this 130560-row head, not a one-off; per-block-32 (this bundle),
the CLI's default int4 preset and fp16 (`--compression none`) are all clean on the same probes.
Which component owns the row cut-off (quantizer, converter or the runtime's per-channel int8 matmul)
is not established here.

The gate that catches it is now part of the zoo's `cli/coreai_verify.py`: `--chat no-think --prompt
"1+1=?"` makes the fp32 oracle's `<|im_end|>` a gated step — a bundle that runs past a stop the oracle
takes at margin 0.80 fails — and `--must-stop-within N` is the plain halt check. The old bundle is red
on both, this one green.

## Quantization

Weight-only **symmetric int8, per-block-32** (a scale per 32-wide block along the input dim; no
clipping), applied as a torch pre-export pass via `coreai-opt`; SDPA / RoPE / RMSNorm stay full
precision — the same YAML that ships the [2B](https://huggingface.co/mlboydaisuke/MiniCPM5-2B-CoreAI).

```bash
uv run coreai.llm.export openbmb/MiniCPM5-1B --experimental --compute-precision float16 \
  --compression-config minicpm5_int8sym_b32.yaml
# minicpm5_int8sym_b32.yaml: quantization_config → op_state_spec.weight = {dtype: int8,
#   qscheme: symmetric, granularity: {type: per_block, block_size: 32}}
```

## Conversion notes

- **`llama → mistral` remap.** MiniCPM5-1B's `model_type` is `llama`; the stock exporter has no
  `llama` graph family, but Mistral's builder is architecturally identical for this config (GQA,
  no qkv bias, no qk-norm, explicit `head_dim` honored). One-line remap in the model registry.
- **Chat EOS.** Base `eos_token` is `</s>`, but the chat template ends turns with `<|im_end|>`
  (id 130073). The bundle's tokenizer `eos_token` is set to `<|im_end|>` (as Qwen ships). Checked
  through the engine with the chat template applied: the no-think turn `1+1=?` answers `1+1=2` and
  stops at step 5, where the fp32 reference stops (margin 0.80).
- **Dynamic-shape bundle** → the Core AI pipelined engine (the iPhone path); a static iOS export
  routes to the static-shape engine instead, which this FM-format bundle doesn't target.
- **Thinking.** The model thinks by default (`<think>…</think>` before the answer); pass
  `enable_thinking=False` through the chat template for a direct answer. Give generation a
  generous budget (the kit caps at 4096) — the think trace alone can run a few hundred tokens.

## Run

```swift
// iOS / macOS, via Foundation Models
import FoundationModels
import CoreAILanguageModels
let model = try await CoreAILanguageModel(resourcesAt: modelURL)   // int8/ bundle
let session = LanguageModelSession(model: model)
print(try await session.respond(to: "Explain on-device AI in one sentence."))
```

## Reproduce

Exporter, gate, card and port notes live in the
[Core AI model zoo](https://github.com/john-rocky/coreai-model-zoo):
[`models/minicpm5-1b/`](https://github.com/john-rocky/coreai-model-zoo/tree/main/models/minicpm5-1b),
[`conversion/export_minicpm5.py`](https://github.com/john-rocky/coreai-model-zoo/blob/main/conversion/export_minicpm5.py),
[`knowledge/minicpm5-1b.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/minicpm5-1b.md).

```bash
python3 conversion/zoo_convert.py show minicpm5-1b
python3 conversion/zoo_convert.py run  minicpm5-1b
python3 cli/coreai_verify.py <bundle> --chat no-think --prompt "1+1=?" -n 16 --must-stop-within 16
```

## License

Apache-2.0 (upstream MiniCPM5 license). Model © OpenBMB — see
https://huggingface.co/openbmb/MiniCPM5-1B. Conversion: community.

<!-- funnel:v1 -->

---

**More models in this format:** [Core AI Model Zoo](https://huggingface.co/collections/mlboydaisuke/core-ai-model-zoo-6a7ff330f753e8dcae04671a) — 75 models, each with the recipe that produced it.

**Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->