Instructions to use rapid-mlx/GLM-5.3-Flash-MTP-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rapid-mlx/GLM-5.3-Flash-MTP-4bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("rapid-mlx/GLM-5.3-Flash-MTP-4bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use rapid-mlx/GLM-5.3-Flash-MTP-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "rapid-mlx/GLM-5.3-Flash-MTP-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "rapid-mlx/GLM-5.3-Flash-MTP-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use rapid-mlx/GLM-5.3-Flash-MTP-4bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "rapid-mlx/GLM-5.3-Flash-MTP-4bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "rapid-mlx/GLM-5.3-Flash-MTP-4bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "rapid-mlx/GLM-5.3-Flash-MTP-4bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use rapid-mlx/GLM-5.3-Flash-MTP-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "rapid-mlx/GLM-5.3-Flash-MTP-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default rapid-mlx/GLM-5.3-Flash-MTP-4bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use rapid-mlx/GLM-5.3-Flash-MTP-4bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "rapid-mlx/GLM-5.3-Flash-MTP-4bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "rapid-mlx/GLM-5.3-Flash-MTP-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Run the paired model on your Mac

This repository is a sidecar, not a standalone model. The command serves its paired target and uses this sidecar.

curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve glm5.3-flash-4bit
curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"default","messages":[{"role":"user","content":"Say hello"}]}'

Use it from OpenCode / Claude Code / Codex: rapid-mlx agents <name> --setup · Agent setup

Download the Desktop app · 192 GB unified memory for the paired target plus this 4.2 GB drafter. This repo alone cannot answer prompts.

GLM-5.3-Flash MLX Q4 MTP drafter

This repository contains the standalone, target-matched MTP drafter used by Rapid-MLX for Vontra/GLM-5.3-Flash-MLX-4bit-MTP.

It is a sidecar head, not a standalone language model. The target and drafter must be loaded together by a runtime that implements GLM-5.3's cache-owned MTP transaction.

Immutable pairing

Item	Value
Target	`Vontra/GLM-5.3-Flash-MLX-4bit-MTP`
Qualified target revision	`76add2a341a1cd90ad0e86bb69839ea9c35827c6`
Drafter architecture	`glm5_next_mtp`
Block size	2 (one proposed token plus target bonus)
Quantization	4-bit affine, group size 64
Tensor count	55
Weight size	4,183,323,401 bytes
Weight SHA-256	`369cf9c0f9cdf3ae5f1b9f72d3db8e65ad3026b8e416b8de5618e9117d3f00ca`

Do not pair this drafter with another target revision or quantization. Rapid-MLX checks the immutable target/drafter revisions, architecture, and block size before loading.

Rapid-MLX qualification

Measured on an Apple M3 Ultra with 256 GB unified memory, single-row greedy decoding, temperature 0, and the target revision above:

Six real tasks covered coding, knowledge, math, instruction following, creative writing, and long-context document retrieval.
Two consecutive runs passed 12/12 tasks.
Reasoning and final responses were byte-identical to the autoregressive control on all six tasks.
Per-task speedups were 1.405x, 1.377x, 1.306x, 1.390x, 1.285x, and 1.106x.
Paired median throughput improved 1.341x (+34.1%).

Sampling, penalties, logprobs, and batch decoding are not qualified for this artifact. Unsupported request shapes should use ordinary autoregressive decoding.

Provenance

The drafter was split from the native MTP layer preserved in the qualified target checkpoint and converted to the target's affine Q4 representation. The target is itself a quantization of zai-org/GLM-5.3-Flash. See the target model card for upstream provenance, architecture, intended use, and limitations.

License

MIT. The upstream copyright and license notice are included in LICENSE.

Downloads last month: 993

Safetensors

Model size

7B params

Tensor type

U32

BF16

F32

MLX

Hardware compatibility

4-bit

Model tree for rapid-mlx/GLM-5.3-Flash-MTP-4bit

Base model

zai-org/GLM-5.3-Flash

Quantized

TensorFold/GLM-5.3-Flash-MLX-4bit-MTP

Quantized

(1)

this model