README.md

---
tags:
- moe
- minimax
- bfloat16
- sglang
- gguf
license: mit
datasets:
- nick007x/github-code-2025
- tatsu-lab/alpaca
base_model:
- MiniMaxAI/MiniMax-M2
---
![Screenshot](https://huggingface.co/VibeStudio/MiniMax-M2-THRIFT/resolve/main/vibe_processed_by_imagy.png)

# VibeStudio/MiniMax-M2-THRIFT-55-v1

**Targeted Reduction for Inference and Fine-Tuning — ~55% Expert Pruned**

A lean, efficiency-first variant of MiniMax-M2 designed to maximize **latency, throughput, and VRAM savings** for local, on-prem, and edge deployments.

> **Note:** `MiniMax-M2-THRIFT-55` and `MiniMax-M2-THRIFT-55-v1` refer to the same model variant.

---

## Why it’s useful

* **Lower latency:** Fast, responsive interactions for interactive apps and tools.
* **Smaller memory footprint:** Fits tighter VRAM budgets and increases node density.
* **Higher throughput:** Serve more concurrent users on the same hardware.
* **Deployment-friendly:** Smooth drop-in via SGLang with OpenAI-compatible API.
* **Adaptable:** Plays well with light fine-tuning to match domain and style.

## Intended use

* Local/air-gapped assistants and dev tools
* Cost-sensitive batches and realtime services
* Edge and on-prem deployments prioritizing efficiency

---

## How Our Approach Works

> **Active research in progress** — we continue to iterate and expand ablations.

* **Teacher–student setup:** Start with **MiniMax-M2** as teacher and a copy as student.
* **Gradual expert pruning:** Remove **≈5% experts per stage** over **~11 stages** (≈**55% total**), guided by importance scores with a lightweight **Leave-One-Expert-Out** check to retain rare-but-important experts.
* **Distill after each prune:** Retrain the student to imitate the teacher on
  * **Outputs** (token probability distributions),
  * **Hidden states**, and
  * **Router behavior** over the **surviving experts**.

---

**Run AI Coding Agents Fully Locally (Mac Studio, DGX Spark, AMD AI Max)**
https://github.com/latent-variable/minimax-agent-guide

# Model Report — THRIFT-55-v1

**Evaluation windows:** Nov 7–9, 2025 & Nov 24–25, 2025  
**Last updated:** Nov 26, 2025  
**Eval status:** 6/8 benchmarks complete (**75%**) – WildBench & SWE-Bench pending.

---

## 📊 Results to date

### 1) Multiple Choice Q&A (lm-eval)

**MMLU (overall and bands)**

| Metric       |  Score |
| :----------- | -----: |
| MMLU Overall | 60.45% |
| Humanities   | 51.65% |
| STEM         | 59.44% |
| Social Sci.  | 71.66% |
| Other        | 63.69% |

**Selected Tasks (lm-eval)**

| Task                     |  Score |
| :----------------------- | -----: |
| arc_challenge (acc_norm) | 50.77% |
| arc_easy                 | 74.07% |
| boolq                    | 75.02% |
| hellaswag (acc_norm)     | 64.99% |
| mmlu                     | 60.45% |
| openbookqa (acc_norm)    | 38.20% |
| rte                      | 68.23% |
| winogrande               | 64.64% |
| **Average (8 tasks)**    | **62.05%** |

---

### 2) Code Generation (EvalPlus)

**MBPP (Python, 378 problems)**

| Metric  | Score  | Problems Solved |
| :------ | -----: | --------------: |
| MBPP    | 42.1%  | 159 / 378       |
| MBPP+   | 37.3%  | 141 / 378       |
| Average | 39.7%  |        –        |

**HumanEval (164 problems)**

| Metric     | Score  | Problems Solved |
| :--------- | -----: | --------------: |
| HumanEval  | 40.2%  | 66 / 164        |
| HumanEval+ | 39.6%  | 65 / 164        |
| Average    | 39.9%  |        –        |

---

### 3) LiveCodeBench (Live Coding)

| Metric   |      Value |
| :------- | ---------: |
| pass@1   |     16.48% |
| Problems |        182 |

Configuration: temperature **0.2** (greedy-ish decoding).

---

### 4) Math Reasoning

**GSM8K (Grade School Math, 1,319 problems)**

| Metric | Score   | Problems Solved |
|--------|--------:|----------------:|
| GSM8K  | 84.91% | 1,120 / 1,319   |

**MATH-500 (Competition Math)**

| Metric   | Score   |
|----------|--------:|
| Overall  | 90.8%   |
| Level 1  | 97.67%  |
| Level 2  | 95.56%  |
| Level 3  | 89.52%  |
| Level 4  | 90.62%  |
| Level 5  | 86.57%  |

---
```

---

## SGLang Deployment (Python)

Use a fresh virtual environment (e.g., `venv`, `conda`, or `uv`).

```shell
git clone -b v0.5.4.post1 https://github.com/sgl-project/sglang.git
cd sglang

pip install --upgrade pip
pip install -e "python"
```

**4-GPU launch**

```shell
python -m sglang.launch_server \
  --model-path VibeStudio/MiniMax-M2-THRIFT-55-v1 \
  --tp-size 4 \
  --tool-call-parser minimax-m2 \
  --reasoning-parser minimax-append-think \
  --host 0.0.0.0 \
  --trust-remote-code \
  --port 8000 \
  --mem-fraction-static 0.85
```

**8-GPU launch**

```shell
python -m sglang.launch_server \
  --model-path VibeStudio/MiniMax-M2-THRIFT-55-v1 \
  --tp-size 8 \
  --ep-size 8 \
  --tool-call-parser minimax-m2 \
  --reasoning-parser minimax-append-think \
  --host 0.0.0.0 \
  --trust-remote-code \
  --port 8000 \
  --mem-fraction-static 0.85
```

### Quick Test (OpenAI-compatible)

```shell
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "VibeStudio/MiniMax-M2-THRIFT-55-v1",
    "messages": [
      {"role":"system","content":[{"type":"text","text":"You are a helpful assistant."}]},
      {"role":"user","content":[{"type":"text","text":"Write a Python function to reverse a linked list."}]}
    ]
  }'
```

---

---

## License

Derived from MiniMax-M2 and distributed under the **MIT License** [http://github.com/MiniMax-AI/MiniMax-M2/blob/main/LICENSE](http://github.com/MiniMax-AI/MiniMax-M2/blob/main/LICENSE)

---

## Credits

Model conversion and Transformers glue by **@Qubitum** at **ModelCloud**.

## References (BibTeX)

```
@article{cai2025thinking,
  title        = {Thinking with DistilQwen: A Tale of Four Distilled Reasoning and Reward Model Series},
  author       = {Cai, Wenrui and Wang, Chengyu and Yan, Junbing and Huang, Jun and Fang, Xiangzhong},
  journal      = {arXiv preprint arXiv:2511.01354},
  year         = {2025},
  eprinttype   = {arXiv},
  eprint       = {2511.01354},
  primaryclass = {cs.CL}
}

@misc{lasby-reap,
  title       = {{REAP the Experts: Why Pruning Prevails for One-Shot MoE compression}},
  author      = {Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan},
  year        = {2025},
  publisher   = {arXiv},
  note        = {arXiv:2510.13999v1 [cs]},
  url         = {https://arxiv.org/abs/2510.13999v1}
}

@article{yang2025wanda++,
  title        = {Wanda++: Pruning Large Language Models via Regional Gradients},
  author       = {Yang, Yifan and Zhen, Kai and Ganesh, Bhavana and Galstyan, Aram and Huybrechts, Goeric and Müller, Markus and Kübler, Jonas M. and Swaminathan, Rupak Vignesh and Mouchtaris, Athanasios and Bodapati, Sravan Babu and Susanj, Nathan and Zhang, Zheng and FitzGerald, Jack and Kumar, Abhishek},
  journal      = {arXiv preprint arXiv:2503.04992},
  year         = {2025},
  eprinttype   = {arXiv},
  eprint       = {2503.04992},
  primaryclass = {cs.CL}
}

@article{li2025tyr,
  title        = {Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization},
  author       = {Li, G. and Xu, Yixing and Li, Zeping and Liu, Ji and Yin, Xuanwu and Li, Dong and Barsoum, Emad},
  journal      = {arXiv preprint arXiv:2503.09657},
  year         = {2025},
  eprinttype   = {arXiv},
  eprint       = {2503.09657},
  primaryclass = {cs.CL}
}

@article{xia2023sheared,
  title        = {Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning},
  author       = {Xia, Mengzhou and Gao, Tianyu and Zeng, Zhiyuan and Chen, Danqi},
  journal      = {arXiv preprint arXiv:2310.06694},
  year         = {2023},
  eprinttype   = {arXiv},
  eprint       = {2310.06694},
  primaryclass = {cs.CL}
}

@article{ma2023llmpruner,
  title        = {LLM-Pruner: On the Structural Pruning of Large Language Models},
  author       = {Ma, Xinyin and Fang, Gongfan and Wang, Xinchao},
  journal      = {arXiv preprint arXiv:2305.11627},
  year         = {2023},
  eprinttype   = {arXiv},
  eprint       = {2305.11627},
  primaryclass = {cs.CL}
}

@article{yang2023wanda,
  title        = {Wanda: Pruning by Weights and Activation-based Discriminant Analysis},
  author       = {Yang, Yifan and Ganesh, Bhavana and Galstyan, Aram and Huybrechts, Goeric and Müller, Markus and Kübler, Jonas M. and Swaminathan, Rupak Vignesh and Mouchtaris, Athanasios and Bodapati, Sravan Babu and Susanj, Nathan and Zhang, Zheng and FitzGerald, Jack and Kumar, Abhishek},
  journal      = {arXiv preprint arXiv:2306.11695},
  year         = {2023},
  eprinttype   = {arXiv},
  eprint       = {2306.11695},
  primaryclass = {cs.CL}
}

@article{frantar2023sparsegpt,
  title        = {SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot},
  author       = {Frantar, Elias and Alistarh, Dan},
  journal      = {arXiv preprint arXiv:2301.00774},
  year         = {2023},
  eprinttype   = {arXiv},
  eprint       = {2301.00774},
  primaryclass = {cs.CL}
}

@article{dettmers2023qlora,
  title        = {QLoRA: Efficient Finetuning of Quantized LLMs},
  author       = {Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
  journal      = {arXiv preprint arXiv:2307.02973},
  year         = {2023},
  eprinttype   = {arXiv},
  eprint       = {2307.02973},
  primaryclass = {cs.CL}
}
```