Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,29 @@
+# NON-COMMERCIAL LICENSE
+
+Non-commercial use permitted based on MIT-style terms; commercial use requires prior written authorization.
+
+Copyright (c) 2026 MiniMax
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software for non-commercial purposes, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or provide copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+2. If the Software (or any derivative works thereof) is used for any Commercial Use, you shall prominently display "Built with MiniMax M2.7" on a related website, user interface, blogpost, about page or product documentation.
+
+3. Any Commercial Use of the Software or any derivative work thereof is prohibited without obtaining a separate, prior written authorization from MiniMax. To request such authorization, please contact api@minimax.io with the subject line "M2.7 licensing".
+
+4. "Commercial Use" means any use of the Software or any derivative work thereof that is primarily intended for commercial advantage or monetary compensation, which includes, without limitation: (i) offering products or services to third parties for a fee, which utilize, incorporate, or rely on the Software or its derivatives, (ii) the commercial use of APIs provided by or for the Software or its derivatives, including to support or enable commercial products, services, or operations, whether in a cloud-based, hosted, or other similar environment, and (iii) the deployment or provision of the Software or its derivatives that have been subjected to post-training, fine-tuning, instruction-tuning, or any other form of modification, for any commercial purpose.
+
+5. Permitted Free Uses. The following uses are expressly permitted free of charge: (a) personal use, including self-hosted deployment for coding, development of applications, agents, tools, integrations, research, experimentation, or other personal purposes; (b) use by non-profit organizations, academic institutions, and researchers for non-commercial research or educational purposes; (c) modification of the Software solely for the uses described in (a) or (b) above.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+## Appendix: Prohibited Uses
+
+You agree you will not use, or allow others to use, the Software or any derivatives of the Software to:
+
+1. Generate or disseminate content prohibited by applicable laws or regulations.
+2. Assist with, engage in or otherwise support any military purpose.
+3. Exploit, harm, or attempt to exploit or harm minors.
+4. Generate or disseminate false or misleading information with the intent to cause harm.
+5. Promote discrimination, hate speech, or harmful behavior against individuals or groups based on race or ethnic origin, religion, disability, age, nationality and national origin, veteran status, sexual orientation, gender or gender identity, caste, immigration status, or any other characteristic that is associated with systemic discrimination or marginalization.

README.md

@@ -1,3 +1,229 @@
----
-license: apache-2.0
----
+---
+base_model: MiniMaxAI/MiniMax-M2.7
+base_model_relation: quantized
+library_name: transformers
+pipeline_tag: text-generation
+license: other
+license_name: minimax-m2.7-license
+license_link: LICENSE
+language:
+  - en
+tags:
+  - moe
+  - mixture-of-experts
+  - quantization
+  - nvfp4
+  - fp4
+  - fp8
+  - reap
+  - pruned
+  - minimax
+  - minimax-m2
+  - blackwell
+  - dgx-spark
+  - vllm
+---
+
+# MiniMax-M2.7-REAP-172B-A10B-NVFP4
+
+This is a my second attempt on this model, which I hope to use as a local coder for long range tasks on an NVIDIA Thor dev kit. I tried to find the best reference sources for calibrated weights and expert pruning and see a significant improvement in chat and coding from the first take. One remaining problem, that could be due to REAP as NVFP4 source seems high quality / has proper KV cache scale, is model getting confused with pathnames, like changing case, dropping path components or inserting spaces. I added the following instructions to Kilo code that fixed these issues:
+
+You are an AI coding assistant. Help user write high quality, modular, clean code. Pay special attention to pathnames - make sure you preserve case, do not drop path components or insert spaces and other extra characters.
+
+I will now try to use model for real life tasks and monitor for issues correctable or non correctable by prompts. If you prefer a little smaller un-REAPed model that is less likely to have regressions, you can try my [`catplusplus/Qwen3.5-122B-A10B-heretic-v2-NVFP4`](https://huggingface.co/catplusplus/Qwen3.5-122B-A10B-heretic-v2-NVFP4)
+
+
+A **REAP-pruned** variant of MiniMax-M2.7 with **NVFP4-quantized** expert
+weights and **FP8 KV-cache** scales.  The original 256-expert-per-layer MoE
+has been reduced to **192 experts per layer (25% compression)** using the same
+pruning mask as
+[saricles/MiniMax-M2.5-REAP-172B-A10B-NVFP4-GB10](https://huggingface.co/saricles/MiniMax-M2.5-REAP-172B-A10B-NVFP4-GB10).
+
+- **NVFP4 weights source**: [`NinjaBoffin/MiniMax-M2.7-NVFP4`](https://huggingface.co/NinjaBoffin/MiniMax-M2.7-NVFP4)
+- **Pruning reference**: [`saricles/MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10`](https://huggingface.co/saricles/MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10)
+- **Architecture**: 62 transformer layers, 192 experts/layer, top-8 routing,
+  hidden size 3072, 48 attention heads, 8 KV heads
+- **Total parameters**: ~172 B (A10B — ~10 B activated per token)
+
+## Quantization Details
+
+| Component | Format | Notes |
+|---|---|---|
+| Expert FFN weights (`w1`/`w2`/`w3`) | **NVFP4** | 4-bit float, group_size=16, per-group + global scales |
+| Attention projections (`q/k/v/o_proj`) | **bfloat16** | Excluded from quantization |
+| Gate weights (`block_sparse_moe.gate`) | **bfloat16** | Excluded from quantization |
+| KV cache | **FP8** | Per-layer `k_scale`/`v_scale` tensors |
+
+Each expert stores three scale tensors per weight matrix: `input_scale`
+(per-tensor activation scale), `weight_scale` (per-group weight scale), and
+`weight_scale_2` (global weight scale).  The FP8 KV cache uses
+`k_proj.k_scale` and `v_proj.v_scale` which vLLM remaps to its internal
+attention scale slots.
+
+## How This Was Made
+
+### Pruning Mask Extraction
+
+The REAP expert pruning removes 64 out of 256 experts per layer (different
+experts per layer — not a uniform pattern). We extracted the pruning mask by
+comparing router matrices between the original 256-expert NVFP4 model and the
+already-pruned 192-expert reference model:
+
+1. **Router comparison**: For each of the 62 layers, the 256-row original
+   gate weight matrix is matched against the 192-row pruned gate weight matrix
+   using the **Hungarian algorithm on cosine distance**.
+
+2. **Mask generation**: Experts in the original that don't match any row in
+   the pruned model are marked as deleted (64 per layer). The per-layer mask
+   is saved to [`extras/deleted_experts.json`](extras/deleted_experts.json).
+
+3. **Expert deletion**: The identified experts are removed from all NVFP4 weight
+   and scale tensors, gate weights are row-sliced, and remaining experts are
+   renumbered 0–191.  Asymmetric KV-cache zero-point tensors (`k_bias`/`v_bias`)
+   are stripped since vLLM does not use them; the symmetric `k_scale`/`v_scale`
+   tensors are preserved.
+
+The full script lives at [`extras/delete_experts.py`](extras/delete_experts.py)
+and can also apply a saved mask to any compatible model.
+
+## Usage with vLLM
+
+This model requires a vLLM build with NVFP4 and FP8 KV-cache support (Blackwell
+/ GB10 or later) and the `minimax_m2` model backend.
+
+```bash
+vllm serve /path/to/MiniMax-M2.7-REAP-172B-A10B-NVFP4 \
+    --served-model-name Nikola \
+    --port 9000 \
+    --enable-auto-tool-choice \
+    --tool-call-parser minimax_m2 \
+    --reasoning-parser minimax_m2_optthink \
+    --reasoning-parser-plugin extras/minimax_m2_optthink_reasoning_parser.py \
+    --chat-template extras/chat_template.jinja \
+    --enable-prefix-caching \
+    --attention-backend FLASHINFER \
+    --enable-chunked-prefill \
+    --gpu-memory-utilization 0.95 \
+    --max-num-seqs 4
+```
+
+See [`extras/inference_minimax.sh`](extras/inference_minimax.sh) for the full
+launch script used on DGX Spark / GB10.
+
+## `extras/` Directory
+
+### [`inference_minimax.sh`](extras/inference_minimax.sh)
+
+vLLM server launcher for DGX Spark / GB10.  Runs inside a custom
+`unglitched_vllm` container via `dockless`.  By default loads the parent model
+directory (where this script lives); pass a path to override:
+
+```bash
+./extras/inference_minimax.sh                    # serves ../MiniMax-M2.7-REAP-172B-A10B-NVFP4
+./extras/inference_minimax.sh /path/to/model     # custom path
+```
+
+Notable flags used:
+- `VLLM_USE_FLASHINFER_MOE_FP4=0` — disables FlashInfer MoE FP4 kernel (uses
+  the fallback NVFP4 GEMM path which is more stable on GB10)
+- `--async-scheduling --enable-chunked-prefill` — latency-oriented scheduling
+- `--cudagraph-capture-sizes 1 2 4` — captures CUDA graphs for typical batch
+  sizes
+
+### [`chat_template.jinja`](extras/chat_template.jinja)
+
+Custom Jinja2 chat template that supports **optional chain-of-thought
+reasoning**.  Pass `enable_thinking=False` in `chat_template_kwargs` to
+suppress `<think>` blocks:
+
+```python
+# OpenAI client — disable thinking
+client.chat.completions.create(
+    model="Nikola",
+    messages=[{"role": "user", "content": "..."}],
+    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
+)
+```
+
+When `enable_thinking=True` (the default), the model produces a `<think>…</think>`
+block before its answer which the reasoning parser extracts into the
+`reasoning_content` field of the response.
+
+### [`minimax_m2_optthink_reasoning_parser.py`](extras/minimax_m2_optthink_reasoning_parser.py)
+
+vLLM reasoning parser registered as `minimax_m2_optthink`.  Handles MiniMax
+M2's convention where the model emits only a closing `</think>` token (no
+opening tag) — all content before `</think>` is treated as reasoning and
+placed in `reasoning_content`; everything after is the assistant reply.
+
+When `enable_thinking=False` was passed in `chat_template_kwargs`, the parser
+skips extraction entirely so no thinking tokens appear in the output.
+
+Load at runtime via `--reasoning-parser-plugin` — **no installation needed**:
+
+```bash
+--reasoning-parser minimax_m2_optthink \
+--reasoning-parser-plugin /path/to/extras/minimax_m2_optthink_reasoning_parser.py
+```
+
+### [`deleted_experts.json`](extras/deleted_experts.json)
+
+Pre-computed pruning mask: a JSON mapping each of the 62 layer indices to the
+list of 64 expert indices (0–255) deleted from that layer.  Can be passed
+directly to `delete_experts.py` via `--deleted-experts-file` to skip the
+Hungarian-matching step.
+
+### [`delete_experts.py`](extras/delete_experts.py)
+
+Utility script to apply expert deletion to a compatible model.  Two modes:
+
+**Find pruning mask by comparison then apply:**
+```bash
+python extras/delete_experts.py \
+    /path/to/MiniMax-M2.7-NVFP4 \
+    /path/to/output \
+    --num-original-experts 256 \
+    --num-retained-experts 192 \
+    --compare-with /path/to/MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10 \
+    --save-deleted-experts extras/deleted_experts.json
+```
+
+**Apply a saved mask directly:**
+```bash
+python extras/delete_experts.py \
+    /path/to/MiniMax-M2.7-NVFP4 \
+    /path/to/output \
+    --num-original-experts 256 \
+    --num-retained-experts 192 \
+    --deleted-experts-file extras/deleted_experts.json
+```
+
+### [`unglitched_vllm`](extras/unglitched_vllm) & [`force_swap.cpp`](extras/force_swap.cpp)
+
+Custom vLLM build script and GPU memory swap utility for DGX Spark.
+`force_swap.cpp` implements a small helper that forces GPU pages to swap to
+system memory, useful for fitting the model on a single GB10 node.
+
+## License
+
+This model inherits the
+[MiniMax M2.7 Non-Commercial License](LICENSE).
+See [LICENSE](LICENSE) for full terms.
+
+## Citation
+
+```bibtex
+@misc{minimax-m2-7,
+  title={MiniMax-M2.7},
+  author={MiniMax},
+  url={https://huggingface.co/MiniMaxAI/MiniMax-M2.7}
+}
+
+@misc{minimax-m2-7-reap-nvfp4,
+  title={MiniMax-M2.7-REAP-172B-A10B-NVFP4},
+  author={Oleg K.},
+  note={NVFP4 weights from NinjaBoffin/MiniMax-M2.7-NVFP4; pruning mask
+        from saricles/MiniMax-M2.5-REAP-172B-A10B-NVFP4-GB10},
+  year={2026}
+}
+```

added_tokens.json

@@ -0,0 +1,56 @@
+{
+  "</minimax:tool_call>": 200053,
+  "</think>": 200051,
+  "<add_file>": 200036,
+  "<code_context>": 200043,
+  "<code_interpreter>": 200023,
+  "<commit_after>": 200018,
+  "<commit_before>": 200016,
+  "<commit_message>": 200040,
+  "<commit_msg>": 200017,
+  "<delete_file>": 200037,
+  "<edit_file>": 200039,
+  "<empty_output>": 200015,
+  "<empty_source_file>": 200041,
+  "<file_content>": 200044,
+  "<file_sep>": 200049,
+  "<filename>": 200006,
+  "<filepath>": 200048,
+  "<fim_middle>": 200002,
+  "<fim_pad>": 200004,
+  "<fim_prefix>": 200001,
+  "<fim_suffix>": 200003,
+  "<function_call>": 200022,
+  "<gh_stars>": 200007,
+  "<issue_closed>": 200010,
+  "<issue_comment>": 200009,
+  "<issue_start>": 200008,
+  "<jupyter_code>": 200013,
+  "<jupyter_error>": 200035,
+  "<jupyter_output>": 200014,
+  "<jupyter_start>": 200011,
+  "<jupyter_text>": 200012,
+  "<minimax:tool_call>": 200052,
+  "<pr_start>": 200046,
+  "<rename_file>": 200038,
+  "<repo_struct>": 200042,
+  "<reponame>": 200005,
+  "<review_comment>": 200047,
+  "<source_files>": 200045,
+  "<think>": 200050,
+  "[e~[": 200020,
+  "]!d~[": 200021,
+  "]!p~[": 200000,
+  "]<]end of image[>[": 200030,
+  "]<]end of speech[>[": 200028,
+  "]<]end of video[>[": 200032,
+  "]<]image[>[": 200025,
+  "]<]speech[>[": 200024,
+  "]<]start of image[>[": 200029,
+  "]<]start of speech[>[": 200027,
+  "]<]start of video[>[": 200031,
+  "]<]video[>[": 200026,
+  "]<]vision pad[>[": 200033,
+  "]~!b[": 200034,
+  "]~b]": 200019
+}

chat_template.jinja

@@ -0,0 +1,159 @@
+{# ----------‑‑‑ special token variables ‑‑‑---------- #}
+{%- set toolcall_begin_token   = '<minimax:tool_call>'         -%}
+{%- set toolcall_end_token     = '</minimax:tool_call>'        -%}
+{#- Tool Rendering Functions ============================================== -#}
+{%- macro render_tool_namespace(namespace_name, tool_list) -%}
+{%- for tool in tool_list -%}
+<tool>{{ tool.function | tojson(ensure_ascii=False) }}</tool>
+{% endfor -%}
+{%- endmacro -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{ content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{#- System Message Construction ============================================ -#}
+{%- macro build_system_message(system_message) -%}
+    {%- if system_message and system_message.content -%}
+        {{- visible_text(system_message.content) }}
+    {%- else -%}
+        {%- if model_identity is not defined -%}
+            {%- set model_identity = "You are a helpful assistant. Your name is MiniMax-M2.7 and is built by MiniMax." -%}
+        {%- endif -%}
+        {{- model_identity }}
+    {%- endif -%}
+    
+    {#- Handle current_date -#}
+    {%- if system_message and system_message.current_date -%}
+        {{- '\n' ~ 'Current date: ' + system_message.current_date }}
+    {%- endif -%}
+    {#- Handle current_location -#}
+    {%- if system_message and system_message.current_location -%}
+        {{- '\n' ~ 'Current location: ' + system_message.current_location }}
+    {%- endif -%}
+{%- endmacro -%}
+{#- Main Template Logic ================================================= -#}
+{#- Extract system message (only first message if it's system) -#}
+{%- set system_message = none -%}
+{%- set conversation_messages = messages -%}
+{%- if messages and messages[0].role == "system" -%}
+    {%- set system_message = messages[0] -%}
+    {%- set conversation_messages = messages[1:] -%}
+{%- endif -%}
+{#- Get the last user message turn, for interleved thinking -#}
+{%- set ns = namespace(last_user_index=-1) %}
+{% for m in conversation_messages %}
+    {%- if m.role == 'user' %}
+        {% set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{#- Render system message -#}
+{{- ']~!b[' ~ ']~b]system' ~ '\n' }}
+{{- build_system_message(system_message) }}
+{#- Render tools if available -#}
+{%- if tools -%}
+    {{- '\n\n' ~ '# Tools' ~ '\n' ~ 'You may call one or more tools to assist with the user query.\nHere are the tools available in JSONSchema format:' ~ '\n' }}
+    {{- '\n' ~ '<tools>' ~ '\n' }}
+    {{- render_tool_namespace("functions", tools) }}
+    {{- '</tools>' ~ '\n\n' }}
+{{- 'When making tool calls, use XML format to invoke tools and pass parameters:' ~ '\n' }}
+{{- '\n' ~ toolcall_begin_token }}
+<invoke name="tool-name-1">
+<parameter name="param-key-1">param-value-1</parameter>
+<parameter name="param-key-2">param-value-2</parameter>
+...
+</invoke>
+{{- '\n' ~ toolcall_end_token }}
+{%- endif -%}
+{{- '[e~[\n' }}
+
+{#- Render messages -#}
+{%- set last_tool_call = namespace(name=none) -%}
+{%- for message in conversation_messages -%}
+    {%- if message.role == 'assistant' -%}
+        {#- Only render reasoning_content if no user message follows -#}
+        {{- ']~b]ai' ~ '\n' }}
+
+        {%- set reasoning_content = '' %}
+        {%- set content = visible_text(message.content) %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].strip('\n').split('<think>')[-1].strip('\n') %}
+                {%- set content = content.split('</think>')[-1].strip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if reasoning_content and loop.index0 > ns.last_user_index -%}
+            {{- '<think>' ~ '\n' ~ reasoning_content ~ '\n' ~ '</think>' ~ '\n\n' }}
+        {%- endif -%}
+        {%- if content -%}
+            {{- content }}
+        {%- endif -%}
+        {%- if message.tool_calls -%}
+            {{- '\n' ~ toolcall_begin_token ~ '\n' }}
+
+            {%- for tool_call in message.tool_calls -%}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<invoke name="' + tool_call.name + '">' }}
+                {% set _args = tool_call.arguments %}
+                {%- for k, v in _args.items() %}
+                {{- '<parameter name="' + k + '">' }}
+                {{- v | tojson(ensure_ascii=False) if v is not string else v }}
+                {{- '</parameter>' }}
+                {% endfor %}
+                {{- '</invoke>' ~ '\n' }}
+            {%- endfor -%}
+            
+            {{- toolcall_end_token}}
+            {%- set last_tool_call.name = message.tool_calls[-1].name -%}
+        {%- else -%}
+            {%- set last_tool_call.name = none -%}
+        {%- endif -%}
+        {{- '[e~[' ~ '\n' }}
+        
+    {%- elif message.role == 'tool' -%}
+    {%- if last_tool_call.name is none -%}
+        {{- raise_exception("Message has tool role, but there was no previous assistant message with a tool call!") }}
+    {%- endif -%}
+    {%- if loop.first or (conversation_messages[loop.index0 - 1].role != 'tool') -%}
+        {{- ']~b]tool' }}
+    {%- endif -%}
+    {%- if message.content is string -%}
+        {{- '\n<response>' }}
+        {{- message.content }}
+        {{- '</response>' }}
+    {%- else -%}
+        {%- for tr in message.content -%}
+            {{- '\n<response>' }}
+            {{- tr.output if tr.output is defined else (tr.text if tr.type == 'text' and tr.text is defined else tr) }}
+            {{- '\n</response>' }}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- if loop.last or (conversation_messages[loop.index0 + 1].role != 'tool') -%}
+        {{- '[e~[\n' -}}
+    {%- endif -%}
+        
+    {%- elif message.role == 'user' -%}
+        {{- ']~b]user' ~ '\n' }}
+        {{- visible_text(message.content) }}
+        {{- '[e~[' ~ '\n' }}
+    {%- endif -%}
+{%- endfor -%}
+
+{#- Generation prompt -#}
+{%- if add_generation_prompt -%}
+{{- ']~b]ai' ~ '\n' ~ '<think>' ~ '\n' }}
+{%- endif -%}

config.json

@@ -0,0 +1,268 @@
+{
+  "architectures": [
+    "MiniMaxM2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "attn_type_list": [
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1
+  ],
+  "auto_map": {
+    "AutoConfig": "configuration_minimax_m2.MiniMaxM2Config",
+    "AutoModelForCausalLM": "modeling_minimax_m2.MiniMaxM2ForCausalLM"
+  },
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "max_position_embeddings": 196608,
+  "model_type": "minimax_m2",
+  "mtp_transformer_layers": 1,
+  "num_attention_heads": 48,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 62,
+  "num_key_value_heads": 8,
+  "num_local_experts": 192,
+  "num_mtp_modules": 3,
+  "output_router_logits": false,
+  "partial_rotary_factor": 0.5,
+  "qk_norm_type": "per_layer",
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "input_activations": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "weights": {
+          "dynamic": false,
+          "num_bits": 4,
+          "type": "float",
+          "group_size": 16
+        },
+        "targets": [
+          "Linear"
+        ]
+      }
+    },
+    "ignore": [
+      "lm_head",
+      "model.layers.0.block_sparse_moe.gate",
+      "model.layers.0.self_attn*",
+      "model.layers.1.block_sparse_moe.gate",
+      "model.layers.1.self_attn*",
+      "model.layers.10.block_sparse_moe.gate",
+      "model.layers.10.self_attn*",
+      "model.layers.11.block_sparse_moe.gate",
+      "model.layers.11.self_attn*",
+      "model.layers.12.block_sparse_moe.gate",
+      "model.layers.12.self_attn*",
+      "model.layers.13.block_sparse_moe.gate",
+      "model.layers.13.self_attn*",
+      "model.layers.14.block_sparse_moe.gate",
+      "model.layers.14.self_attn*",
+      "model.layers.15.block_sparse_moe.gate",
+      "model.layers.15.self_attn*",
+      "model.layers.16.block_sparse_moe.gate",
+      "model.layers.16.self_attn*",
+      "model.layers.17.block_sparse_moe.gate",
+      "model.layers.17.self_attn*",
+      "model.layers.18.block_sparse_moe.gate",
+      "model.layers.18.self_attn*",
+      "model.layers.19.block_sparse_moe.gate",
+      "model.layers.19.self_attn*",
+      "model.layers.2.block_sparse_moe.gate",
+      "model.layers.2.self_attn*",
+      "model.layers.20.block_sparse_moe.gate",
+      "model.layers.20.self_attn*",
+      "model.layers.21.block_sparse_moe.gate",
+      "model.layers.21.self_attn*",
+      "model.layers.22.block_sparse_moe.gate",
+      "model.layers.22.self_attn*",
+      "model.layers.23.block_sparse_moe.gate",
+      "model.layers.23.self_attn*",
+      "model.layers.24.block_sparse_moe.gate",
+      "model.layers.24.self_attn*",
+      "model.layers.25.block_sparse_moe.gate",
+      "model.layers.25.self_attn*",
+      "model.layers.26.block_sparse_moe.gate",
+      "model.layers.26.self_attn*",
+      "model.layers.27.block_sparse_moe.gate",
+      "model.layers.27.self_attn*",
+      "model.layers.28.block_sparse_moe.gate",
+      "model.layers.28.self_attn*",
+      "model.layers.29.block_sparse_moe.gate",
+      "model.layers.29.self_attn*",
+      "model.layers.3.block_sparse_moe.gate",
+      "model.layers.3.self_attn*",
+      "model.layers.30.block_sparse_moe.gate",
+      "model.layers.30.self_attn*",
+      "model.layers.31.block_sparse_moe.gate",
+      "model.layers.31.self_attn*",
+      "model.layers.32.block_sparse_moe.gate",
+      "model.layers.32.self_attn*",
+      "model.layers.33.block_sparse_moe.gate",
+      "model.layers.33.self_attn*",
+      "model.layers.34.block_sparse_moe.gate",
+      "model.layers.34.self_attn*",
+      "model.layers.35.block_sparse_moe.gate",
+      "model.layers.35.self_attn*",
+      "model.layers.36.block_sparse_moe.gate",
+      "model.layers.36.self_attn*",
+      "model.layers.37.block_sparse_moe.gate",
+      "model.layers.37.self_attn*",
+      "model.layers.38.block_sparse_moe.gate",
+      "model.layers.38.self_attn*",
+      "model.layers.39.block_sparse_moe.gate",
+      "model.layers.39.self_attn*",
+      "model.layers.4.block_sparse_moe.gate",
+      "model.layers.4.self_attn*",
+      "model.layers.40.block_sparse_moe.gate",
+      "model.layers.40.self_attn*",
+      "model.layers.41.block_sparse_moe.gate",
+      "model.layers.41.self_attn*",
+      "model.layers.42.block_sparse_moe.gate",
+      "model.layers.42.self_attn*",
+      "model.layers.43.block_sparse_moe.gate",
+      "model.layers.43.self_attn*",
+      "model.layers.44.block_sparse_moe.gate",
+      "model.layers.44.self_attn*",
+      "model.layers.45.block_sparse_moe.gate",
+      "model.layers.45.self_attn*",
+      "model.layers.46.block_sparse_moe.gate",
+      "model.layers.46.self_attn*",
+      "model.layers.47.block_sparse_moe.gate",
+      "model.layers.47.self_attn*",
+      "model.layers.48.block_sparse_moe.gate",
+      "model.layers.48.self_attn*",
+      "model.layers.49.block_sparse_moe.gate",
+      "model.layers.49.self_attn*",
+      "model.layers.5.block_sparse_moe.gate",
+      "model.layers.5.self_attn*",
+      "model.layers.50.block_sparse_moe.gate",
+      "model.layers.50.self_attn*",
+      "model.layers.51.block_sparse_moe.gate",
+      "model.layers.51.self_attn*",
+      "model.layers.52.block_sparse_moe.gate",
+      "model.layers.52.self_attn*",
+      "model.layers.53.block_sparse_moe.gate",
+      "model.layers.53.self_attn*",
+      "model.layers.54.block_sparse_moe.gate",
+      "model.layers.54.self_attn*",
+      "model.layers.55.block_sparse_moe.gate",
+      "model.layers.55.self_attn*",
+      "model.layers.56.block_sparse_moe.gate",
+      "model.layers.56.self_attn*",
+      "model.layers.57.block_sparse_moe.gate",
+      "model.layers.57.self_attn*",
+      "model.layers.58.block_sparse_moe.gate",
+      "model.layers.58.self_attn*",
+      "model.layers.59.block_sparse_moe.gate",
+      "model.layers.59.self_attn*",
+      "model.layers.6.block_sparse_moe.gate",
+      "model.layers.6.self_attn*",
+      "model.layers.60.block_sparse_moe.gate",
+      "model.layers.60.self_attn*",
+      "model.layers.61.block_sparse_moe.gate",
+      "model.layers.61.self_attn*",
+      "model.layers.7.block_sparse_moe.gate",
+      "model.layers.7.self_attn*",
+      "model.layers.8.block_sparse_moe.gate",
+      "model.layers.8.self_attn*",
+      "model.layers.9.block_sparse_moe.gate",
+      "model.layers.9.self_attn*"
+    ],
+    "quant_algo": "NVFP4",
+    "kv_cache_scheme": {
+      "dynamic": false,
+      "num_bits": 8,
+      "type": "float"
+    },
+    "producer": {
+      "name": "modelopt",
+      "version": "0.43.0rc2.dev105+g0b42c143d"
+    },
+    "quant_method": "modelopt"
+  },
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 5000000,
+  "rotary_dim": 64,
+  "router_aux_loss_coef": 0.001,
+  "router_jitter_noise": 0.0,
+  "scoring_func": "sigmoid",
+  "shared_intermediate_size": 0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.6",
+  "use_cache": true,
+  "use_mtp": true,
+  "use_qk_norm": true,
+  "use_routing_bias": true,
+  "vocab_size": 200064
+}

configuration_minimax_m2.py

@@ -0,0 +1,200 @@
+#                🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨
+#           This file was automatically generated from src/transformers/models/minimax_m2/modular_minimax_m2.py.
+#               Do NOT edit this file manually as any edits will be overwritten by the generation of
+#             the file from the modular. If any change should be done, please apply the change to the
+#                          modular_minimax_m2.py file directly. One of our CI enforces this.
+#                🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨
+# coding=utf-8
+# Copyright 2025 the HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+from transformers.configuration_utils import PretrainedConfig
+
+
+class MiniMaxM2Config(PretrainedConfig):
+    r"""
+    This is the configuration class to store the configuration of a [`MiniMaxM2Model`]. It is used to instantiate an
+    MiniMaxM2 model according to the specified arguments, defining the model architecture. Instantiating a configuration
+    with the defaults will yield a similar configuration to that of the MiniMaxM2-7B-v0.1 or MiniMaxM2-7B-Instruct-v0.1.
+
+    [minimax_m2ai/MiniMaxM2-8x7B](https://huggingface.co/minimax_m2ai/MiniMaxM2-8x7B)
+    [minimax_m2ai/MiniMaxM2-7B-Instruct-v0.1](https://huggingface.co/minimax_m2ai/MiniMaxM2-7B-Instruct-v0.1)
+
+    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
+    documentation from [`PretrainedConfig`] for more information.
+
+
+    Args:
+        vocab_size (`int`, *optional*, defaults to 32000):
+            Vocabulary size of the MiniMaxM2 model. Defines the number of different tokens that can be represented by the
+            `inputs_ids` passed when calling [`MiniMaxM2Model`]
+        hidden_size (`int`, *optional*, defaults to 4096):
+            Dimension of the hidden representations.
+        intermediate_size (`int`, *optional*, defaults to 14336):
+            Dimension of the MLP representations.
+        num_hidden_layers (`int`, *optional*, defaults to 32):
+            Number of hidden layers in the Transformer encoder.
+        num_attention_heads (`int`, *optional*, defaults to 32):
+            Number of attention heads for each attention layer in the Transformer encoder.
+        num_key_value_heads (`int`, *optional*, defaults to 8):
+            This is the number of key_value heads that should be used to implement Grouped Query Attention. If
+            `num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if
+            `num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When
+            converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed
+            by meanpooling all the original heads within that group. For more details, check out [this
+            paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `8`.
+        head_dim (`int`, *optional*, defaults to `hidden_size // num_attention_heads`):
+            The attention head dimension.
+        hidden_act (`str` or `function`, *optional*, defaults to `"silu"`):
+            The non-linear activation function (function or string) in the decoder.
+        max_position_embeddings (`int`, *optional*, defaults to `4096*32`):
+            The maximum sequence length that this model might ever be used with. MiniMaxM2's sliding window attention
+            allows sequence of up to 4096*32 tokens.
+        initializer_range (`float`, *optional*, defaults to 0.02):
+            The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
+        rms_norm_eps (`float`, *optional*, defaults to 1e-05):
+            The epsilon used by the rms normalization layers.
+        use_cache (`bool`, *optional*, defaults to `True`):
+            Whether or not the model should return the last key/values attentions (not used by all models). Only
+            relevant if `config.is_decoder=True`.
+        pad_token_id (`int`, *optional*):
+            The id of the padding token.
+        bos_token_id (`int`, *optional*, defaults to 1):
+            The id of the "beginning-of-sequence" token.
+        eos_token_id (`int`, *optional*, defaults to 2):
+            The id of the "end-of-sequence" token.
+        tie_word_embeddings (`bool`, *optional*, defaults to `False`):
+            Whether the model's input and output word embeddings should be tied.
+        rope_theta (`float`, *optional*, defaults to 1000000.0):
+            The base period of the RoPE embeddings.
+        sliding_window (`int`, *optional*):
+            Sliding window attention window size. If not specified, will default to `4096`.
+        attention_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout ratio for the attention probabilities.
+        num_experts_per_tok (`int`, *optional*, defaults to 2):
+            The number of experts to route per-token, can be also interpreted as the `top-k` routing
+            parameter
+        num_local_experts (`int`, *optional*, defaults to 8):
+            Number of experts per Sparse MLP layer.
+        output_router_logits (`bool`, *optional*, defaults to `False`):
+            Whether or not the router logits should be returned by the model. Enabling this will also
+            allow the model to output the auxiliary loss. See [here]() for more details
+        router_aux_loss_coef (`float`, *optional*, defaults to 0.001):
+            The aux loss factor for the total loss.
+        router_jitter_noise (`float`, *optional*, defaults to 0.0):
+            Amount of noise to add to the router.
+
+    ```python
+    >>> from transformers import MiniMaxM2Model, MiniMaxM2Config
+
+    >>> # Initializing a MiniMaxM2 7B style configuration
+    >>> configuration = MiniMaxM2Config()
+
+    >>> # Initializing a model from the MiniMaxM2 7B style configuration
+    >>> model = MiniMaxM2Model(configuration)
+
+    >>> # Accessing the model configuration
+    >>> configuration = model.config
+    ```"""
+
+    model_type = "minimax_m2"
+    keys_to_ignore_at_inference = ["past_key_values"]
+    base_model_tp_plan = {
+        "layers.*.self_attn.q_proj": "colwise",
+        "layers.*.self_attn.k_proj": "colwise",
+        "layers.*.self_attn.v_proj": "colwise",
+        "layers.*.self_attn.o_proj": "rowwise",
+        "layers.*.block_sparse_moe.gate": "colwise_rep",  # we need to replicate here to correctly route experts
+        "layers.*.block_sparse_moe.experts.*.w1": "colwise",
+        "layers.*.block_sparse_moe.experts.*.w2": "rowwise",
+        "layers.*.block_sparse_moe.experts.*.w3": "colwise",
+    }
+    base_model_pp_plan = {
+        "embed_tokens": (["input_ids"], ["inputs_embeds"]),
+        "layers": (["hidden_states", "attention_mask"], ["hidden_states"]),
+        "norm": (["hidden_states"], ["hidden_states"]),
+    }
+
+    def __init__(
+        self,
+        vocab_size=32000,
+        hidden_size=4096,
+        intermediate_size=14336,
+        num_hidden_layers=32,
+        num_attention_heads=32,
+        num_key_value_heads=8,
+        head_dim=None,
+        hidden_act="silu",
+        max_position_embeddings=4096 * 32,
+        initializer_range=0.02,
+        rms_norm_eps=1e-5,
+        use_cache=True,
+        pad_token_id=None,
+        bos_token_id=1,
+        eos_token_id=2,
+        tie_word_embeddings=False,
+        rope_theta=1e6,
+        sliding_window=None,
+        attention_dropout=0.0,
+        num_experts_per_tok=2,
+        num_local_experts=8,
+        output_router_logits=False,
+        router_aux_loss_coef=0.001,
+        router_jitter_noise=0.0,
+        **kwargs,
+    ):
+        self.vocab_size = vocab_size
+        self.max_position_embeddings = max_position_embeddings
+        self.hidden_size = hidden_size
+        self.intermediate_size = intermediate_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_attention_heads = num_attention_heads
+        self.sliding_window = sliding_window
+
+        # for backward compatibility
+        if num_key_value_heads is None:
+            num_key_value_heads = num_attention_heads
+
+        self.num_key_value_heads = num_key_value_heads
+        self.hidden_act = hidden_act
+        self.initializer_range = initializer_range
+        self.rms_norm_eps = rms_norm_eps
+        self.use_cache = use_cache
+        self.rope_theta = rope_theta
+        self.attention_dropout = attention_dropout
+        self.head_dim = head_dim
+
+        self.num_experts_per_tok = num_experts_per_tok
+        self.num_local_experts = num_local_experts
+        self.output_router_logits = output_router_logits
+        self.router_aux_loss_coef = router_aux_loss_coef
+        self.router_jitter_noise = router_jitter_noise
+
+        self.use_qk_norm = kwargs.pop("use_qk_norm", False)
+        self.rotary_dim = kwargs.pop("rotary_dim", self.head_dim)
+        self.partial_rotary_factor = kwargs.pop("partial_rotary_factor", 1)
+        if self.head_dim is not None:
+            self.partial_rotary_factor = self.rotary_dim / self.head_dim
+
+        super().__init__(
+            pad_token_id=pad_token_id,
+            bos_token_id=bos_token_id,
+            eos_token_id=eos_token_id,
+            tie_word_embeddings=tie_word_embeddings,
+            **kwargs,
+        )
+
+
+__all__ = ["MiniMaxM2Config"]

extras/chat_template.jinja

@@ -0,0 +1,163 @@
+{# ----------‑‑‑ special token variables ‑‑‑---------- #}
+{%- set toolcall_begin_token   = '<minimax:tool_call>'         -%}
+{%- set toolcall_end_token     = '</minimax:tool_call>'        -%}
+{#- Tool Rendering Functions ============================================== -#}
+{%- macro render_tool_namespace(namespace_name, tool_list) -%}
+{%- for tool in tool_list -%}
+<tool>{{ tool.function | tojson(ensure_ascii=False) }}</tool>
+{% endfor -%}
+{%- endmacro -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{ content }}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text }}
+            {%- elif item is string -%}
+                {{- item }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{#- System Message Construction ============================================ -#}
+{%- macro build_system_message(system_message) -%}
+    {%- if system_message and system_message.content -%}
+        {{- visible_text(system_message.content) }}
+    {%- else -%}
+        {%- if model_identity is not defined -%}
+            {%- set model_identity = "You are a helpful assistant. Your name is MiniMax-M2.5 and is built by MiniMax." -%}
+        {%- endif -%}
+        {{- model_identity }}
+    {%- endif -%}
+    
+    {#- Handle current_date -#}
+    {%- if system_message and system_message.current_date -%}
+        {{- '\n' ~ 'Current date: ' + system_message.current_date }}
+    {%- endif -%}
+    {#- Handle current_location -#}
+    {%- if system_message and system_message.current_location -%}
+        {{- '\n' ~ 'Current location: ' + system_message.current_location }}
+    {%- endif -%}
+{%- endmacro -%}
+{#- Main Template Logic ================================================= -#}
+{#- Extract system message (only first message if it's system) -#}
+{%- set system_message = none -%}
+{%- set conversation_messages = messages -%}
+{%- if messages and messages[0].role == "system" -%}
+    {%- set system_message = messages[0] -%}
+    {%- set conversation_messages = messages[1:] -%}
+{%- endif -%}
+{#- Get the last user message turn, for interleved thinking -#}
+{%- set ns = namespace(last_user_index=-1) %}
+{% for m in conversation_messages %}
+    {%- if m.role == 'user' %}
+        {% set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{#- Render system message -#}
+{{- ']~!b[' ~ ']~b]system' ~ '\n' }}
+{{- build_system_message(system_message) }}
+{#- Render tools if available -#}
+{%- if tools -%}
+    {{- '\n\n' ~ '# Tools' ~ '\n' ~ 'You may call one or more tools to assist with the user query.\nHere are the tools available in JSONSchema format:' ~ '\n' }}
+    {{- '\n' ~ '<tools>' ~ '\n' }}
+    {{- render_tool_namespace("functions", tools) }}
+    {{- '</tools>' ~ '\n\n' }}
+{{- 'When making tool calls, use XML format to invoke tools and pass parameters:' ~ '\n' }}
+{{- '\n' ~ toolcall_begin_token }}
+<invoke name="tool-name-1">
+<parameter name="param-key-1">param-value-1</parameter>
+<parameter name="param-key-2">param-value-2</parameter>
+...
+</invoke>
+{{- '\n' ~ toolcall_end_token }}
+{%- endif -%}
+{{- '[e~[\n' }}
+
+{#- Render messages -#}
+{%- set last_tool_call = namespace(name=none) -%}
+{%- for message in conversation_messages -%}
+    {%- if message.role == 'assistant' -%}
+        {#- Only render reasoning_content if no user message follows -#}
+        {{- ']~b]ai' ~ '\n' }}
+
+        {%- set reasoning_content = '' %}
+        {%- set content = visible_text(message.content) %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].strip('\n').split('<think>')[-1].strip('\n') %}
+                {%- set content = content.split('</think>')[-1].strip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if reasoning_content and loop.index0 > ns.last_user_index -%}
+            {{- '<think>' ~ '\n' ~ reasoning_content ~ '\n' ~ '</think>' ~ '\n\n' }}
+        {%- endif -%}
+        {%- if content -%}
+            {{- content }}
+        {%- endif -%}
+        {%- if message.tool_calls -%}
+            {{- '\n' ~ toolcall_begin_token ~ '\n' }}
+
+            {%- for tool_call in message.tool_calls -%}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<invoke name="' + tool_call.name + '">' }}
+                {% set _args = tool_call.arguments %}
+                {%- for k, v in _args.items() %}
+                {{- '<parameter name="' + k + '">' }}
+                {{- v | tojson(ensure_ascii=False) if v is not string else v }}
+                {{- '</parameter>' }}
+                {% endfor %}
+                {{- '</invoke>' ~ '\n' }}
+            {%- endfor -%}
+            
+            {{- toolcall_end_token}}
+            {%- set last_tool_call.name = message.tool_calls[-1].name -%}
+        {%- else -%}
+            {%- set last_tool_call.name = none -%}
+        {%- endif -%}
+        {{- '[e~[' ~ '\n' }}
+        
+    {%- elif message.role == 'tool' -%}
+    {%- if last_tool_call.name is none -%}
+        {{- raise_exception("Message has tool role, but there was no previous assistant message with a tool call!") }}
+    {%- endif -%}
+    {%- if loop.first or (conversation_messages[loop.index0 - 1].role != 'tool') -%}
+        {{- ']~b]tool' }}
+    {%- endif -%}
+    {%- if message.content is string -%}
+        {{- '\n<response>' }}
+        {{- message.content }}
+        {{- '</response>' }}
+    {%- else -%}
+        {%- for tr in message.content -%}
+            {{- '\n<response>' }}
+            {{- tr.output if tr.output is defined else (tr.text if tr.type == 'text' and tr.text is defined else tr) }}
+            {{- '\n</response>' }}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- if loop.last or (conversation_messages[loop.index0 + 1].role != 'tool') -%}
+        {{- '[e~[\n' -}}
+    {%- endif -%}
+        
+    {%- elif message.role == 'user' -%}
+        {{- ']~b]user' ~ '\n' }}
+        {{- visible_text(message.content) }}
+        {{- '[e~[' ~ '\n' }}
+    {%- endif -%}
+{%- endfor -%}
+
+{#- Generation prompt -#}
+{%- if add_generation_prompt -%}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- ']~b]ai' ~ '\n' ~ '<think>\n</think>\n\n' }}
+    {%- else %}
+        {{- ']~b]ai' ~ '\n' ~ '<think>' ~ '\n' }}
+    {%- endif %}
+{%- endif -%}

extras/delete_experts.py

@@ -0,0 +1,409 @@
+#!/usr/bin/env python3
+"""
+Script to delete experts from a MoE model layer by layer.
+Processes one safetensor file at a time.
+
+Can either:
+1. Use a pre-computed deleted_experts.json file
+2. Analyze original vs pruned model to find deleted experts first
+"""
+
+import argparse
+import gc
+import json
+import re
+import shutil
+from pathlib import Path
+from typing import Optional
+
+import numpy as np
+import torch
+from safetensors import safe_open
+from safetensors.torch import save_file
+from scipy.optimize import linear_sum_assignment
+
+
+def load_deleted_experts(deleted_file: Path) -> dict[int, list[int]]:
+    """Load the deleted experts mapping from JSON file."""
+    with open(deleted_file, "r") as f:
+        data = json.load(f)
+    return {int(k): v for k, v in data["deleted_experts_per_layer"].items()}
+
+
+def find_deleted_experts(
+    original_model_path: Path,
+    pruned_model_path: Path,
+    num_original_experts: int,
+    num_pruned_experts: int,
+    hidden_dim: int = 3072,
+) -> dict[int, list[int]]:
+    """
+    Find deleted experts by comparing router matrices between original and pruned models.
+    Uses Hungarian algorithm for optimal matching based on cosine distance.
+    
+    Returns a dict mapping layer_num -> list of deleted expert indices.
+    """
+    deleted_experts = {}
+    num_layers = 62  # MiniMax-M2.x models have 62 layers
+    
+    for layer_num in range(num_layers):
+        router_key = f"model.layers.{layer_num}.block_sparse_moe.gate.weight"
+        
+        # Load router from original model
+        orig_router = None
+        for sf_file in sorted(original_model_path.glob("model-*.safetensors")):
+            with safe_open(sf_file, framework="pt") as f:
+                if router_key in f.keys():
+                    orig_router = f.get_tensor(router_key)
+                    break
+        
+        # Load router from pruned model
+        prune_router = None
+        for sf_file in sorted(pruned_model_path.glob("model-*.safetensors")):
+            with safe_open(sf_file, framework="pt") as f:
+                if router_key in f.keys():
+                    prune_router = f.get_tensor(router_key)
+                    break
+        
+        if orig_router is None or prune_router is None:
+            print(f"  Layer {layer_num}: Router not found, skipping")
+            continue
+        
+        # Convert bfloat16 if needed
+        if orig_router.dtype == torch.bfloat16:
+            orig_router = orig_router.to(torch.float32)
+        if prune_router.dtype == torch.bfloat16:
+            prune_router = prune_router.to(torch.float32)
+        
+        # L2 normalize rows for cosine distance
+        orig_np = orig_router.numpy()
+        prune_np = prune_router.numpy()
+        
+        orig_norm = orig_np / (np.linalg.norm(orig_np, axis=1, keepdims=True) + 1e-10)
+        prune_norm = prune_np / (np.linalg.norm(prune_np, axis=1, keepdims=True) + 1e-10)
+        
+        # Cosine distance matrix
+        distance_matrix = 1 - np.dot(orig_norm, prune_norm.T)
+        
+        # Hungarian algorithm
+        row_ind, col_ind = linear_sum_assignment(distance_matrix)
+        
+        # Find unmatched (deleted) experts
+        matched_original = set(row_ind)
+        all_original = set(range(num_original_experts))
+        deleted = sorted(all_original - matched_original)
+        
+        deleted_experts[layer_num] = deleted
+        
+        if layer_num in [0, 1, 2, 10, 20, 30, 40, 50, 61]:
+            print(f"  Layer {layer_num}: {len(deleted)} deleted experts")
+    
+    return deleted_experts
+
+
+def get_retained_experts(num_original: int, deleted: list[int]) -> list[int]:
+    """Get the list of retained expert indices (sorted)."""
+    all_experts = set(range(num_original))
+    deleted_set = set(deleted)
+    return sorted(all_experts - deleted_set)
+
+
+def get_layer_for_tensor(tensor_name: str) -> Optional[int]:
+    """Get layer number for a tensor name."""
+    parts = tensor_name.split(".")
+    for i, part in enumerate(parts):
+        if part == "layers" and i + 1 < len(parts):
+            try:
+                return int(parts[i + 1])
+            except ValueError:
+                pass
+    return None
+
+
+def is_expert_tensor(key: str) -> bool:
+    """
+    Check if a tensor key belongs to a specific numbered expert.
+    Uses regex to match .experts.<N>. pattern robustly, covering NVFP4
+    sub-tensors like w1.weight, w1.weight_scale, w1.weight_scale_2,
+    w1.input_scale, etc.
+    """
+    return bool(re.search(r'\.experts\.\d+\.', key))
+
+
+# Keys to strip: asymmetric KV-cache quantization zero-points that vLLM
+# doesn't support (only scale factors are used).
+_STRIP_SUFFIXES = (".k_bias", ".v_bias")
+
+
+def should_strip_tensor(key: str) -> bool:
+    """Return True for tensors that should be omitted from the output model."""
+    return any(key.endswith(sfx) for sfx in _STRIP_SUFFIXES)
+
+
+def process_tensor(
+    key: str,
+    tensor: torch.Tensor,
+    deleted_experts: dict[int, list[int]],
+    num_original_experts: int,
+) -> tuple[str, torch.Tensor] | None:
+    """
+    Process a single tensor. Returns (new_key, new_tensor) or None to skip.
+    """
+    # Strip unsupported KV quantization zero-points.
+    if should_strip_tensor(key):
+        return None
+
+    layer_num = get_layer_for_tensor(key)
+
+    # Non-layer or non-MoE tensors pass through unchanged.
+    if layer_num is None or "block_sparse_moe" not in key:
+        return (key, tensor)
+
+    if is_expert_tensor(key):
+        # Expert weight/scale tensor – delete or renumber.
+        parts = key.split(".")
+        for i, part in enumerate(parts):
+            if part == "experts" and i + 1 < len(parts):
+                expert_idx = int(parts[i + 1])
+                deleted = deleted_experts.get(layer_num, [])
+                retained = get_retained_experts(num_original_experts, deleted)
+                if expert_idx not in retained:
+                    return None  # Deleted expert – drop tensor.
+                # Renumber to sequential 0-based index.
+                new_expert_idx = retained.index(expert_idx)
+                new_key_parts = parts.copy()
+                new_key_parts[i + 1] = str(new_expert_idx)
+                new_key = ".".join(new_key_parts)
+                return (new_key, tensor)
+        # Shouldn't happen, but fall through unchanged.
+        return (key, tensor)
+
+    elif "gate.weight" in key:
+        deleted = deleted_experts.get(layer_num, [])
+        retained = get_retained_experts(num_original_experts, deleted)
+        indices = torch.tensor(retained, dtype=torch.long)
+        new_tensor = tensor[indices].clone()
+        print(f"    gate.weight layer {layer_num}: {tuple(tensor.shape)} → {tuple(new_tensor.shape)}")
+        return (key, new_tensor)
+
+    elif "e_score_correction_bias" in key:
+        deleted = deleted_experts.get(layer_num, [])
+        retained = get_retained_experts(num_original_experts, deleted)
+        indices = torch.tensor(retained, dtype=torch.long)
+        return (key, tensor[indices].clone())
+
+    else:
+        return (key, tensor)
+
+
+def process_file(
+    input_path: Path,
+    output_path: Path,
+    deleted_experts: dict[int, list[int]],
+    num_original_experts: int,
+) -> tuple[int, int, int]:
+    """
+    Process one safetensor file.
+    Returns (kept, deleted, stripped) tensor counts.
+    """
+    tensors = {}
+    kept = deleted_count = stripped = 0
+
+    with safe_open(input_path, framework="pt") as f:
+        all_keys = list(f.keys())
+
+    # Report stripped keys upfront for this file.
+    stripped_keys = [k for k in all_keys if should_strip_tensor(k)]
+    if stripped_keys:
+        print(f"    Stripping {len(stripped_keys)} KV-bias tensors: {stripped_keys[:4]}{'…' if len(stripped_keys) > 4 else ''}")
+
+    with safe_open(input_path, framework="pt") as f:
+        for key in all_keys:
+            tensor = f.get_tensor(key)
+            result = process_tensor(key, tensor, deleted_experts, num_original_experts)
+            if result is not None:
+                new_key, new_tensor = result
+                tensors[new_key] = new_tensor
+                kept += 1
+            else:
+                if should_strip_tensor(key):
+                    stripped += 1
+                else:
+                    deleted_count += 1
+
+    save_file(tensors, output_path)
+    del tensors
+    gc.collect()
+
+    return kept, deleted_count, stripped
+
+
+def get_file_to_tensors(model_path: Path) -> dict[str, list[str]]:
+    """Get mapping from filename to list of tensor names."""
+    index_path = model_path / "model.safetensors.index.json"
+    with open(index_path, "r") as f:
+        index_data = json.load(f)
+    weight_map = index_data["weight_map"]
+    file_to_tensors: dict[str, list[str]] = {}
+    for key, file_name in weight_map.items():
+        if file_name not in file_to_tensors:
+            file_to_tensors[file_name] = []
+        file_to_tensors[file_name].append(key)
+    return file_to_tensors
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Delete experts from MoE model. "
+        "Either provide --deleted-experts-file or use --compare-with to find deleted experts."
+    )
+    parser.add_argument("input_model", type=Path, help="Input model path (256 experts)")
+    parser.add_argument("output_model", type=Path, help="Output model path")
+    parser.add_argument("--num-original-experts", type=int, default=256)
+    parser.add_argument("--num-retained-experts", type=int, default=192)
+    parser.add_argument("--deleted-experts-file", type=Path,
+                        help="JSON with pre-computed deleted experts (optional)")
+    parser.add_argument("--compare-with", type=Path,
+                        help="Pruned model path (192 experts) to compare and find deleted experts")
+    parser.add_argument("--save-deleted-experts", type=Path,
+                        help="Save found deleted experts to JSON file (for --compare-with mode)")
+
+    args = parser.parse_args()
+
+    # Either load from file or find via comparison
+    if args.deleted_experts_file:
+        print("Loading deleted experts from file...")
+        deleted_experts = load_deleted_experts(args.deleted_experts_file)
+        print(f"  {len(deleted_experts)} layers")
+        # Sanity-check: verify expected deletions per layer.
+        counts = {l: len(v) for l, v in deleted_experts.items()}
+        expected = args.num_original_experts - args.num_retained_experts
+        bad = {l: c for l, c in counts.items() if c != expected}
+        if bad:
+            print(f"  WARNING: {len(bad)} layers have unexpected deletion count "
+                  f"(expected {expected}): {dict(list(bad.items())[:5])}")
+    elif args.compare_with:
+        print("Finding deleted experts by comparing models...")
+        print(f"  Original: {args.input_model}")
+        print(f"  Pruned:   {args.compare_with}")
+        deleted_experts = find_deleted_experts(
+            args.input_model,
+            args.compare_with,
+            args.num_original_experts,
+            args.num_retained_experts,
+        )
+        print(f"  Found deleted experts for {len(deleted_experts)} layers")
+
+        if args.save_deleted_experts:
+            print(f"\nSaving deleted experts to {args.save_deleted_experts}...")
+            data = {"deleted_experts_per_layer": {str(k): v for k, v in deleted_experts.items()}}
+            with open(args.save_deleted_experts, "w") as f:
+                json.dump(data, f, indent=2)
+    else:
+        parser.error("Either --deleted-experts-file or --compare-with is required")
+
+    # Create output dir
+    args.output_model.mkdir(parents=True, exist_ok=True)
+
+    # Copy non-safetensor files
+    print("\nCopying config/tokenizer files...")
+    for fname in ["config.json", "configuration_minimax_m2.py", "tokenizer.json",
+                  "tokenizer_config.json", "vocab.json", "merges.txt",
+                  "special_tokens_map.json", "added_tokens.json",
+                  "generation_config.json", "chat_template.jinja", ".gitattributes",
+                  "hf_quant_config.json", "modeling_minimax_m2.py"]:
+        src = args.input_model / fname
+        if src.exists():
+            shutil.copy2(src, args.output_model / fname)
+            print(f"  {fname}")
+
+    # Update config: set num_local_experts to the retained count.
+    print("\nUpdating config...")
+    with open(args.input_model / "config.json") as f:
+        config = json.load(f)
+    config["num_local_experts"] = args.num_retained_experts
+    with open(args.output_model / "config.json", "w") as f:
+        json.dump(config, f, indent=2)
+    print(f"  num_local_experts = {args.num_retained_experts}")
+
+    # Process safetensors
+    print("\nProcessing safetensor files...")
+    file_to_tensors = get_file_to_tensors(args.input_model)
+
+    total_kept = total_deleted = total_stripped = 0
+    for file_name in sorted(file_to_tensors.keys()):
+        input_path = args.input_model / file_name
+        output_path = args.output_model / file_name
+        print(f"  {file_name}...")
+        kept, deleted_count, stripped = process_file(
+            input_path, output_path, deleted_experts, args.num_original_experts
+        )
+        print(f"    kept={kept}  deleted={deleted_count}  stripped={stripped}")
+        total_kept += kept
+        total_deleted += deleted_count
+        total_stripped += stripped
+        gc.collect()
+
+    print(f"\nTotal: kept={total_kept}  deleted={total_deleted}  stripped={total_stripped}")
+
+    # Update index.
+    # Bug fixes vs original:
+    #   1. Use `layer_num is not None` instead of truthy `layer_num` (layer 0 == 0 is falsy).
+    #   2. Use a `handled` flag so deleted experts are not re-added via the for-else clause.
+    #   3. Use is_expert_tensor() regex instead of ".w" heuristic.
+    print("\nUpdating model index...")
+    with open(args.input_model / "model.safetensors.index.json") as f:
+        index_data = json.load(f)
+
+    weight_map = index_data["weight_map"]
+    new_weight_map: dict[str, str] = {}
+    idx_skipped = idx_stripped = idx_renamed = 0
+
+    for key, file_name in weight_map.items():
+        # Drop stripped tensors (KV bias).
+        if should_strip_tensor(key):
+            idx_stripped += 1
+            continue
+
+        layer_num = get_layer_for_tensor(key)
+
+        # FIX 1: use `is not None` so layer 0 (== 0, falsy) is handled correctly.
+        if layer_num is not None and "block_sparse_moe" in key and is_expert_tensor(key):
+            parts = key.split(".")
+            handled = False
+            for i, part in enumerate(parts):
+                if part == "experts" and i + 1 < len(parts):
+                    expert_idx = int(parts[i + 1])
+                    deleted = deleted_experts.get(layer_num, [])
+                    retained = get_retained_experts(args.num_original_experts, deleted)
+                    if expert_idx not in retained:
+                        # FIX 2: mark as handled and break so we don't fall
+                        # through to the for-else and re-add the old key.
+                        idx_skipped += 1
+                        handled = True
+                        break
+                    new_expert_idx = retained.index(expert_idx)
+                    new_key_parts = parts.copy()
+                    new_key_parts[i + 1] = str(new_expert_idx)
+                    new_key = ".".join(new_key_parts)
+                    new_weight_map[new_key] = file_name
+                    idx_renamed += 1
+                    handled = True
+                    break
+            if not handled:
+                # No expert index found in key (shouldn't happen), keep as-is.
+                new_weight_map[key] = file_name
+        else:
+            new_weight_map[key] = file_name
+
+    index_data["weight_map"] = new_weight_map
+    with open(args.output_model / "model.safetensors.index.json", "w") as f:
+        json.dump(index_data, f, indent=2)
+
+    print(f"  {len(new_weight_map)} index entries  "
+          f"(renamed={idx_renamed}  skipped={idx_skipped}  stripped={idx_stripped})")
+    print("\n✓ Done!")
+
+
+if __name__ == "__main__":
+    main()

extras/deleted_experts.json

@@ -0,0 +1,4096 @@
+{
+  "deleted_experts_per_layer": {
+    "0": [
+      12,
+      16,
+      19,
+      20,
+      25,
+      30,
+      42,
+      48,
+      52,
+      55,
+      56,
+      57,
+      59,
+      62,
+      63,
+      64,
+      68,
+      69,
+      73,
+      79,
+      85,
+      87,
+      88,
+      91,
+      94,
+      101,
+      103,
+      110,
+      114,
+      115,
+      126,
+      127,
+      128,
+      131,
+      136,
+      140,
+      147,
+      149,
+      150,
+      153,
+      154,
+      182,
+      194,
+      195,
+      196,
+      198,
+      200,
+      204,
+      208,
+      213,
+      215,
+      217,
+      218,
+      222,
+      225,
+      226,
+      227,
+      231,
+      235,
+      238,
+      243,
+      247,
+      248,
+      254
+    ],
+    "1": [
+      7,
+      9,
+      11,
+      13,
+      14,
+      17,
+      24,
+      25,
+      26,
+      27,
+      31,
+      33,
+      36,
+      48,
+      49,
+      52,
+      56,
+      57,
+      60,
+      62,
+      72,
+      73,
+      75,
+      78,
+      82,
+      84,
+      85,
+      94,
+      97,
+      100,
+      105,
+      108,
+      109,
+      110,
+      119,
+      121,
+      131,
+      133,
+      140,
+      142,
+      143,
+      146,
+      149,
+      158,
+      165,
+      167,
+      175,
+      179,
+      182,
+      184,
+      187,
+      190,
+      192,
+      201,
+      206,
+      207,
+      217,
+      218,
+      227,
+      232,
+      233,
+      243,
+      245,
+      247
+    ],
+    "2": [
+      1,
+      2,
+      8,
+      13,
+      16,
+      17,
+      19,
+      25,
+      27,
+      31,
+      40,
+      41,
+      43,
+      47,
+      55,
+      59,
+      62,
+      66,
+      72,
+      77,
+      82,
+      84,
+      85,
+      88,
+      91,
+      94,
+      96,
+      97,
+      101,
+      102,
+      106,
+      113,
+      119,
+      122,
+      126,
+      137,
+      144,
+      146,
+      147,
+      148,
+      166,
+      168,
+      172,
+      173,
+      175,
+      183,
+      187,
+      196,
+      200,
+      206,
+      207,
+      212,
+      217,
+      221,
+      223,
+      225,
+      230,
+      236,
+      238,
+      239,
+      240,
+      248,
+      249,
+      255
+    ],
+    "3": [
+      2,
+      9,
+      10,
+      14,
+      21,
+      30,
+      32,
+      38,
+      48,
+      50,
+      53,
+      55,
+      59,
+      62,
+      69,
+      74,
+      84,
+      85,
+      88,
+      94,
+      96,
+      97,
+      99,
+      100,
+      104,
+      107,
+      110,
+      117,
+      122,
+      123,
+      140,
+      143,
+      146,
+      151,
+      161,
+      162,
+      163,
+      164,
+      168,
+      172,
+      173,
+      177,
+      179,
+      183,
+      186,
+      190,
+      193,
+      198,
+      201,
+      210,
+      214,
+      216,
+      218,
+      221,
+      229,
+      230,
+      231,
+      233,
+      236,
+      241,
+      243,
+      245,
+      249,
+      252
+    ],
+    "4": [
+      2,
+      3,
+      4,
+      6,
+      10,
+      16,
+      18,
+      21,
+      24,
+      25,
+      28,
+      33,
+      36,
+      37,
+      46,
+      48,
+      50,
+      52,
+      53,
+      57,
+      58,
+      59,
+      65,
+      72,
+      73,
+      75,
+      77,
+      87,
+      88,
+      90,
+      93,
+      94,
+      102,
+      107,
+      113,
+      115,
+      117,
+      121,
+      125,
+      132,
+      137,
+      147,
+      149,
+      163,
+      166,
+      170,
+      173,
+      187,
+      189,
+      193,
+      194,
+      203,
+      204,
+      209,
+      211,
+      212,
+      214,
+      215,
+      228,
+      235,
+      239,
+      246,
+      247,
+      254
+    ],
+    "5": [
+      6,
+      14,
+      19,
+      25,
+      26,
+      27,
+      29,
+      32,
+      33,
+      35,
+      38,
+      39,
+      47,
+      55,
+      57,
+      60,
+      67,
+      68,
+      70,
+      71,
+      73,
+      81,
+      84,
+      92,
+      95,
+      99,
+      106,
+      110,
+      114,
+      116,
+      117,
+      119,
+      120,
+      124,
+      129,
+      135,
+      136,
+      139,
+      142,
+      146,
+      150,
+      154,
+      156,
+      161,
+      165,
+      168,
+      172,
+      181,
+      185,
+      186,
+      190,
+      195,
+      196,
+      197,
+      213,
+      224,
+      226,
+      227,
+      230,
+      231,
+      236,
+      238,
+      239,
+      241
+    ],
+    "6": [
+      10,
+      11,
+      13,
+      14,
+      16,
+      18,
+      20,
+      22,
+      26,
+      27,
+      31,
+      32,
+      45,
+      57,
+      63,
+      70,
+      76,
+      77,
+      81,
+      88,
+      94,
+      98,
+      101,
+      109,
+      110,
+      116,
+      119,
+      120,
+      122,
+      132,
+      137,
+      142,
+      143,
+      144,
+      146,
+      149,
+      153,
+      154,
+      159,
+      161,
+      164,
+      179,
+      181,
+      182,
+      184,
+      193,
+      195,
+      196,
+      197,
+      200,
+      202,
+      205,
+      214,
+      221,
+      222,
+      225,
+      226,
+      227,
+      228,
+      231,
+      241,
+      245,
+      246,
+      255
+    ],
+    "7": [
+      5,
+      12,
+      14,
+      21,
+      22,
+      24,
+      25,
+      26,
+      37,
+      38,
+      39,
+      46,
+      54,
+      56,
+      66,
+      67,
+      73,
+      75,
+      77,
+      81,
+      87,
+      88,
+      93,
+      94,
+      103,
+      112,
+      116,
+      119,
+      121,
+      138,
+      144,
+      145,
+      146,
+      153,
+      156,
+      165,
+      172,
+      174,
+      178,
+      182,
+      187,
+      191,
+      192,
+      193,
+      200,
+      201,
+      214,
+      216,
+      218,
+      220,
+      226,
+      227,
+      231,
+      235,
+      237,
+      239,
+      241,
+      243,
+      246,
+      247,
+      248,
+      250,
+      254,
+      255
+    ],
+    "8": [
+      1,
+      2,
+      8,
+      14,
+      16,
+      22,
+      26,
+      29,
+      34,
+      35,
+      41,
+      43,
+      51,
+      54,
+      56,
+      59,
+      65,
+      67,
+      68,
+      69,
+      70,
+      74,
+      77,
+      99,
+      105,
+      111,
+      112,
+      120,
+      122,
+      125,
+      127,
+      134,
+      135,
+      138,
+      142,
+      144,
+      147,
+      150,
+      152,
+      155,
+      163,
+      164,
+      168,
+      185,
+      187,
+      197,
+      198,
+      202,
+      203,
+      213,
+      225,
+      228,
+      229,
+      233,
+      235,
+      236,
+      237,
+      243,
+      245,
+      247,
+      252,
+      253,
+      254,
+      255
+    ],
+    "9": [
+      3,
+      5,
+      19,
+      20,
+      23,
+      24,
+      25,
+      31,
+      34,
+      36,
+      38,
+      40,
+      45,
+      51,
+      54,
+      57,
+      62,
+      72,
+      82,
+      88,
+      101,
+      109,
+      110,
+      115,
+      116,
+      117,
+      118,
+      123,
+      124,
+      133,
+      137,
+      140,
+      151,
+      155,
+      160,
+      162,
+      163,
+      164,
+      165,
+      167,
+      180,
+      181,
+      182,
+      185,
+      195,
+      199,
+      202,
+      208,
+      213,
+      215,
+      217,
+      219,
+      220,
+      226,
+      227,
+      230,
+      233,
+      234,
+      235,
+      238,
+      242,
+      250,
+      251,
+      255
+    ],
+    "10": [
+      0,
+      1,
+      2,
+      4,
+      9,
+      11,
+      13,
+      16,
+      17,
+      19,
+      27,
+      29,
+      32,
+      34,
+      35,
+      42,
+      54,
+      67,
+      76,
+      77,
+      78,
+      79,
+      84,
+      91,
+      97,
+      99,
+      101,
+      104,
+      110,
+      112,
+      117,
+      128,
+      132,
+      139,
+      147,
+      149,
+      154,
+      155,
+      156,
+      158,
+      164,
+      165,
+      167,
+      172,
+      176,
+      184,
+      192,
+      194,
+      195,
+      197,
+      204,
+      207,
+      208,
+      210,
+      211,
+      213,
+      224,
+      227,
+      238,
+      242,
+      245,
+      248,
+      250,
+      252
+    ],
+    "11": [
+      1,
+      2,
+      5,
+      27,
+      29,
+      36,
+      40,
+      51,
+      62,
+      63,
+      64,
+      70,
+      71,
+      72,
+      77,
+      80,
+      83,
+      86,
+      96,
+      104,
+      111,
+      117,
+      119,
+      123,
+      125,
+      130,
+      138,
+      145,
+      149,
+      153,
+      155,
+      157,
+      160,
+      167,
+      172,
+      174,
+      175,
+      176,
+      177,
+      179,
+      184,
+      189,
+      194,
+      196,
+      205,
+      206,
+      210,
+      212,
+      213,
+      214,
+      215,
+      220,
+      225,
+      226,
+      229,
+      232,
+      234,
+      235,
+      236,
+      238,
+      245,
+      246,
+      253,
+      255
+    ],
+    "12": [
+      2,
+      6,
+      9,
+      17,
+      19,
+      22,
+      30,
+      33,
+      34,
+      36,
+      65,
+      69,
+      72,
+      77,
+      78,
+      82,
+      83,
+      84,
+      85,
+      86,
+      90,
+      92,
+      95,
+      98,
+      102,
+      106,
+      110,
+      113,
+      115,
+      117,
+      119,
+      120,
+      121,
+      122,
+      125,
+      129,
+      130,
+      131,
+      139,
+      144,
+      148,
+      154,
+      170,
+      176,
+      180,
+      188,
+      195,
+      196,
+      201,
+      206,
+      208,
+      210,
+      212,
+      213,
+      214,
+      216,
+      217,
+      229,
+      230,
+      231,
+      238,
+      243,
+      249,
+      250
+    ],
+    "13": [
+      0,
+      1,
+      6,
+      8,
+      9,
+      13,
+      16,
+      18,
+      19,
+      23,
+      25,
+      26,
+      33,
+      37,
+      43,
+      46,
+      49,
+      51,
+      55,
+      56,
+      60,
+      61,
+      63,
+      66,
+      67,
+      68,
+      81,
+      83,
+      88,
+      89,
+      92,
+      94,
+      95,
+      96,
+      103,
+      106,
+      110,
+      123,
+      124,
+      127,
+      138,
+      139,
+      148,
+      151,
+      153,
+      156,
+      160,
+      178,
+      180,
+      190,
+      192,
+      195,
+      196,
+      200,
+      211,
+      218,
+      220,
+      221,
+      224,
+      233,
+      236,
+      251,
+      253,
+      255
+    ],
+    "14": [
+      5,
+      7,
+      21,
+      28,
+      29,
+      42,
+      46,
+      47,
+      58,
+      61,
+      62,
+      65,
+      66,
+      67,
+      73,
+      74,
+      75,
+      80,
+      82,
+      88,
+      91,
+      92,
+      95,
+      96,
+      98,
+      101,
+      103,
+      105,
+      110,
+      111,
+      120,
+      121,
+      129,
+      131,
+      133,
+      138,
+      139,
+      142,
+      143,
+      144,
+      148,
+      152,
+      153,
+      161,
+      162,
+      166,
+      167,
+      181,
+      182,
+      189,
+      192,
+      198,
+      204,
+      205,
+      207,
+      210,
+      230,
+      231,
+      239,
+      240,
+      243,
+      251,
+      254,
+      255
+    ],
+    "15": [
+      8,
+      10,
+      14,
+      16,
+      20,
+      24,
+      26,
+      28,
+      30,
+      32,
+      34,
+      36,
+      40,
+      42,
+      43,
+      48,
+      54,
+      55,
+      58,
+      67,
+      70,
+      71,
+      77,
+      78,
+      79,
+      93,
+      94,
+      99,
+      105,
+      107,
+      111,
+      112,
+      114,
+      115,
+      116,
+      118,
+      123,
+      128,
+      133,
+      135,
+      142,
+      143,
+      150,
+      156,
+      157,
+      160,
+      163,
+      165,
+      169,
+      195,
+      197,
+      199,
+      201,
+      207,
+      208,
+      216,
+      229,
+      231,
+      237,
+      239,
+      249,
+      250,
+      252,
+      255
+    ],
+    "16": [
+      10,
+      15,
+      21,
+      23,
+      26,
+      27,
+      29,
+      32,
+      35,
+      38,
+      42,
+      43,
+      46,
+      53,
+      54,
+      55,
+      56,
+      61,
+      63,
+      64,
+      71,
+      73,
+      74,
+      75,
+      83,
+      86,
+      87,
+      91,
+      99,
+      103,
+      109,
+      116,
+      121,
+      123,
+      126,
+      129,
+      136,
+      139,
+      142,
+      143,
+      151,
+      157,
+      165,
+      170,
+      171,
+      178,
+      188,
+      197,
+      200,
+      202,
+      204,
+      207,
+      213,
+      214,
+      217,
+      223,
+      225,
+      230,
+      240,
+      241,
+      244,
+      248,
+      249,
+      252
+    ],
+    "17": [
+      0,
+      4,
+      5,
+      9,
+      11,
+      12,
+      14,
+      15,
+      16,
+      19,
+      21,
+      29,
+      31,
+      43,
+      44,
+      49,
+      50,
+      55,
+      68,
+      74,
+      75,
+      76,
+      79,
+      80,
+      82,
+      83,
+      85,
+      94,
+      101,
+      104,
+      110,
+      112,
+      120,
+      121,
+      122,
+      123,
+      125,
+      143,
+      148,
+      163,
+      166,
+      168,
+      174,
+      184,
+      185,
+      188,
+      196,
+      202,
+      203,
+      208,
+      212,
+      213,
+      216,
+      217,
+      220,
+      222,
+      224,
+      227,
+      230,
+      235,
+      246,
+      249,
+      252,
+      254
+    ],
+    "18": [
+      0,
+      10,
+      11,
+      12,
+      15,
+      22,
+      23,
+      45,
+      47,
+      49,
+      50,
+      51,
+      52,
+      58,
+      59,
+      62,
+      66,
+      70,
+      76,
+      77,
+      84,
+      85,
+      94,
+      95,
+      97,
+      100,
+      102,
+      104,
+      106,
+      108,
+      115,
+      117,
+      120,
+      126,
+      137,
+      139,
+      141,
+      147,
+      149,
+      159,
+      160,
+      166,
+      169,
+      172,
+      180,
+      181,
+      196,
+      199,
+      201,
+      209,
+      218,
+      219,
+      221,
+      222,
+      229,
+      230,
+      234,
+      236,
+      240,
+      244,
+      246,
+      249,
+      252,
+      253
+    ],
+    "19": [
+      3,
+      4,
+      5,
+      6,
+      8,
+      11,
+      13,
+      15,
+      20,
+      32,
+      33,
+      36,
+      39,
+      41,
+      43,
+      47,
+      49,
+      51,
+      55,
+      56,
+      59,
+      77,
+      80,
+      82,
+      86,
+      90,
+      105,
+      106,
+      110,
+      112,
+      113,
+      118,
+      120,
+      121,
+      128,
+      129,
+      142,
+      152,
+      155,
+      157,
+      165,
+      166,
+      167,
+      171,
+      173,
+      176,
+      177,
+      192,
+      193,
+      199,
+      201,
+      202,
+      203,
+      212,
+      224,
+      225,
+      233,
+      234,
+      235,
+      236,
+      243,
+      244,
+      251,
+      252
+    ],
+    "20": [
+      0,
+      4,
+      5,
+      9,
+      10,
+      12,
+      13,
+      15,
+      18,
+      25,
+      27,
+      29,
+      31,
+      32,
+      33,
+      34,
+      38,
+      40,
+      49,
+      50,
+      53,
+      54,
+      60,
+      68,
+      72,
+      78,
+      84,
+      86,
+      87,
+      90,
+      94,
+      96,
+      99,
+      104,
+      110,
+      111,
+      126,
+      127,
+      129,
+      132,
+      136,
+      137,
+      141,
+      145,
+      147,
+      148,
+      149,
+      150,
+      161,
+      163,
+      181,
+      183,
+      196,
+      197,
+      198,
+      212,
+      213,
+      218,
+      229,
+      235,
+      238,
+      244,
+      247,
+      248
+    ],
+    "21": [
+      0,
+      4,
+      6,
+      8,
+      10,
+      28,
+      29,
+      32,
+      39,
+      44,
+      45,
+      47,
+      48,
+      49,
+      56,
+      60,
+      61,
+      67,
+      72,
+      73,
+      81,
+      82,
+      87,
+      89,
+      92,
+      93,
+      104,
+      113,
+      114,
+      115,
+      119,
+      126,
+      131,
+      132,
+      142,
+      146,
+      148,
+      155,
+      156,
+      157,
+      158,
+      160,
+      161,
+      162,
+      163,
+      179,
+      188,
+      194,
+      196,
+      198,
+      200,
+      204,
+      206,
+      207,
+      211,
+      215,
+      224,
+      230,
+      233,
+      237,
+      239,
+      241,
+      243,
+      251
+    ],
+    "22": [
+      4,
+      9,
+      10,
+      12,
+      16,
+      24,
+      25,
+      26,
+      29,
+      39,
+      47,
+      52,
+      56,
+      57,
+      70,
+      73,
+      77,
+      78,
+      82,
+      84,
+      86,
+      95,
+      96,
+      102,
+      103,
+      106,
+      107,
+      110,
+      112,
+      115,
+      130,
+      134,
+      140,
+      141,
+      147,
+      156,
+      158,
+      161,
+      164,
+      166,
+      171,
+      177,
+      183,
+      187,
+      188,
+      190,
+      191,
+      193,
+      196,
+      197,
+      199,
+      203,
+      208,
+      213,
+      217,
+      218,
+      225,
+      227,
+      233,
+      245,
+      246,
+      247,
+      248,
+      255
+    ],
+    "23": [
+      2,
+      5,
+      11,
+      17,
+      19,
+      26,
+      29,
+      34,
+      40,
+      42,
+      54,
+      55,
+      56,
+      60,
+      62,
+      67,
+      70,
+      72,
+      74,
+      79,
+      82,
+      85,
+      90,
+      92,
+      94,
+      96,
+      97,
+      98,
+      101,
+      103,
+      122,
+      126,
+      133,
+      134,
+      139,
+      140,
+      141,
+      142,
+      146,
+      150,
+      154,
+      157,
+      162,
+      167,
+      177,
+      178,
+      179,
+      182,
+      194,
+      199,
+      205,
+      208,
+      210,
+      211,
+      212,
+      217,
+      219,
+      224,
+      231,
+      233,
+      240,
+      241,
+      249,
+      251
+    ],
+    "24": [
+      2,
+      14,
+      20,
+      23,
+      29,
+      34,
+      35,
+      39,
+      42,
+      44,
+      53,
+      55,
+      56,
+      58,
+      61,
+      64,
+      65,
+      66,
+      67,
+      71,
+      73,
+      74,
+      75,
+      80,
+      84,
+      89,
+      98,
+      102,
+      120,
+      124,
+      132,
+      136,
+      141,
+      144,
+      146,
+      148,
+      149,
+      154,
+      162,
+      165,
+      168,
+      170,
+      172,
+      177,
+      179,
+      182,
+      183,
+      189,
+      191,
+      195,
+      196,
+      198,
+      202,
+      206,
+      211,
+      213,
+      218,
+      224,
+      232,
+      235,
+      236,
+      237,
+      240,
+      254
+    ],
+    "25": [
+      1,
+      2,
+      8,
+      11,
+      13,
+      17,
+      29,
+      33,
+      35,
+      37,
+      42,
+      49,
+      52,
+      58,
+      59,
+      60,
+      66,
+      80,
+      82,
+      84,
+      86,
+      87,
+      95,
+      101,
+      105,
+      107,
+      108,
+      115,
+      117,
+      121,
+      122,
+      130,
+      135,
+      136,
+      142,
+      149,
+      150,
+      151,
+      153,
+      154,
+      158,
+      159,
+      160,
+      162,
+      164,
+      169,
+      172,
+      181,
+      183,
+      188,
+      192,
+      197,
+      198,
+      209,
+      225,
+      227,
+      233,
+      234,
+      237,
+      238,
+      243,
+      250,
+      251,
+      253
+    ],
+    "26": [
+      4,
+      8,
+      9,
+      12,
+      13,
+      16,
+      28,
+      29,
+      34,
+      37,
+      47,
+      51,
+      52,
+      53,
+      55,
+      56,
+      58,
+      66,
+      71,
+      72,
+      74,
+      76,
+      78,
+      79,
+      81,
+      84,
+      85,
+      86,
+      90,
+      94,
+      95,
+      96,
+      98,
+      119,
+      129,
+      132,
+      135,
+      137,
+      144,
+      145,
+      149,
+      158,
+      159,
+      160,
+      164,
+      165,
+      170,
+      180,
+      184,
+      194,
+      202,
+      204,
+      208,
+      210,
+      227,
+      228,
+      232,
+      233,
+      234,
+      235,
+      237,
+      242,
+      243,
+      244
+    ],
+    "27": [
+      1,
+      11,
+      14,
+      20,
+      25,
+      26,
+      30,
+      31,
+      36,
+      41,
+      46,
+      48,
+      50,
+      52,
+      54,
+      58,
+      61,
+      64,
+      68,
+      71,
+      72,
+      76,
+      77,
+      82,
+      89,
+      92,
+      100,
+      101,
+      104,
+      108,
+      109,
+      117,
+      120,
+      122,
+      134,
+      139,
+      141,
+      144,
+      158,
+      162,
+      163,
+      166,
+      168,
+      173,
+      174,
+      176,
+      180,
+      184,
+      191,
+      192,
+      196,
+      198,
+      211,
+      217,
+      219,
+      220,
+      222,
+      226,
+      233,
+      237,
+      239,
+      244,
+      249,
+      251
+    ],
+    "28": [
+      1,
+      3,
+      4,
+      13,
+      15,
+      26,
+      29,
+      30,
+      35,
+      39,
+      41,
+      43,
+      47,
+      54,
+      55,
+      57,
+      62,
+      64,
+      76,
+      78,
+      83,
+      85,
+      87,
+      89,
+      92,
+      93,
+      97,
+      109,
+      127,
+      133,
+      134,
+      136,
+      137,
+      143,
+      145,
+      151,
+      155,
+      164,
+      165,
+      166,
+      168,
+      172,
+      173,
+      175,
+      176,
+      195,
+      198,
+      204,
+      207,
+      208,
+      216,
+      218,
+      219,
+      223,
+      224,
+      225,
+      231,
+      234,
+      235,
+      237,
+      238,
+      239,
+      243,
+      244
+    ],
+    "29": [
+      2,
+      3,
+      6,
+      17,
+      21,
+      27,
+      32,
+      35,
+      36,
+      40,
+      41,
+      44,
+      47,
+      49,
+      53,
+      56,
+      61,
+      62,
+      63,
+      65,
+      68,
+      70,
+      71,
+      73,
+      76,
+      80,
+      85,
+      93,
+      94,
+      102,
+      115,
+      119,
+      120,
+      121,
+      134,
+      135,
+      138,
+      145,
+      148,
+      151,
+      156,
+      171,
+      174,
+      175,
+      180,
+      189,
+      192,
+      200,
+      203,
+      206,
+      211,
+      214,
+      217,
+      223,
+      227,
+      229,
+      230,
+      233,
+      235,
+      236,
+      240,
+      248,
+      249,
+      251
+    ],
+    "30": [
+      0,
+      1,
+      2,
+      10,
+      11,
+      12,
+      13,
+      18,
+      20,
+      28,
+      30,
+      38,
+      41,
+      48,
+      52,
+      53,
+      56,
+      64,
+      65,
+      66,
+      69,
+      70,
+      73,
+      85,
+      96,
+      99,
+      102,
+      104,
+      112,
+      117,
+      120,
+      127,
+      129,
+      132,
+      133,
+      137,
+      140,
+      144,
+      146,
+      152,
+      160,
+      165,
+      175,
+      177,
+      179,
+      184,
+      188,
+      189,
+      197,
+      207,
+      211,
+      212,
+      215,
+      223,
+      224,
+      226,
+      228,
+      230,
+      231,
+      238,
+      241,
+      251,
+      252,
+      254
+    ],
+    "31": [
+      2,
+      3,
+      5,
+      17,
+      25,
+      27,
+      29,
+      30,
+      32,
+      33,
+      34,
+      35,
+      39,
+      40,
+      41,
+      46,
+      49,
+      50,
+      51,
+      61,
+      67,
+      70,
+      76,
+      79,
+      82,
+      84,
+      87,
+      90,
+      91,
+      92,
+      106,
+      109,
+      115,
+      117,
+      124,
+      125,
+      130,
+      133,
+      140,
+      142,
+      145,
+      146,
+      150,
+      154,
+      165,
+      169,
+      173,
+      177,
+      181,
+      192,
+      197,
+      198,
+      199,
+      200,
+      209,
+      213,
+      223,
+      225,
+      229,
+      235,
+      238,
+      240,
+      244,
+      250
+    ],
+    "32": [
+      5,
+      6,
+      8,
+      9,
+      11,
+      12,
+      13,
+      14,
+      19,
+      28,
+      30,
+      31,
+      37,
+      41,
+      46,
+      51,
+      54,
+      55,
+      58,
+      60,
+      64,
+      70,
+      71,
+      78,
+      81,
+      93,
+      94,
+      100,
+      101,
+      103,
+      112,
+      114,
+      118,
+      120,
+      123,
+      129,
+      130,
+      132,
+      135,
+      137,
+      143,
+      144,
+      153,
+      155,
+      166,
+      170,
+      171,
+      184,
+      185,
+      187,
+      189,
+      191,
+      194,
+      195,
+      196,
+      198,
+      201,
+      210,
+      215,
+      229,
+      237,
+      248,
+      251,
+      252
+    ],
+    "33": [
+      0,
+      4,
+      6,
+      11,
+      13,
+      16,
+      22,
+      26,
+      30,
+      37,
+      41,
+      43,
+      46,
+      50,
+      51,
+      58,
+      61,
+      64,
+      65,
+      66,
+      69,
+      75,
+      77,
+      78,
+      82,
+      84,
+      88,
+      108,
+      110,
+      113,
+      115,
+      119,
+      123,
+      125,
+      127,
+      132,
+      133,
+      140,
+      143,
+      145,
+      149,
+      153,
+      155,
+      157,
+      159,
+      161,
+      162,
+      168,
+      169,
+      172,
+      178,
+      180,
+      190,
+      197,
+      206,
+      214,
+      216,
+      224,
+      227,
+      228,
+      232,
+      247,
+      254,
+      255
+    ],
+    "34": [
+      6,
+      8,
+      9,
+      14,
+      15,
+      28,
+      31,
+      33,
+      37,
+      38,
+      41,
+      47,
+      51,
+      60,
+      65,
+      67,
+      71,
+      72,
+      74,
+      81,
+      85,
+      88,
+      100,
+      101,
+      106,
+      116,
+      117,
+      124,
+      126,
+      127,
+      131,
+      133,
+      135,
+      137,
+      139,
+      140,
+      150,
+      152,
+      165,
+      167,
+      170,
+      174,
+      179,
+      183,
+      188,
+      189,
+      191,
+      197,
+      198,
+      202,
+      203,
+      204,
+      213,
+      216,
+      221,
+      231,
+      238,
+      240,
+      248,
+      249,
+      250,
+      252,
+      253,
+      254
+    ],
+    "35": [
+      3,
+      6,
+      12,
+      17,
+      20,
+      23,
+      24,
+      30,
+      31,
+      38,
+      40,
+      41,
+      45,
+      49,
+      50,
+      56,
+      63,
+      67,
+      69,
+      73,
+      74,
+      77,
+      79,
+      85,
+      86,
+      89,
+      91,
+      93,
+      95,
+      100,
+      102,
+      106,
+      108,
+      113,
+      115,
+      125,
+      127,
+      134,
+      136,
+      140,
+      143,
+      148,
+      153,
+      159,
+      161,
+      163,
+      164,
+      169,
+      180,
+      181,
+      195,
+      196,
+      210,
+      211,
+      223,
+      224,
+      227,
+      231,
+      238,
+      241,
+      245,
+      248,
+      251,
+      255
+    ],
+    "36": [
+      4,
+      6,
+      11,
+      19,
+      26,
+      33,
+      42,
+      48,
+      51,
+      56,
+      57,
+      59,
+      60,
+      61,
+      64,
+      70,
+      74,
+      77,
+      83,
+      88,
+      92,
+      95,
+      100,
+      103,
+      108,
+      109,
+      111,
+      123,
+      131,
+      132,
+      141,
+      145,
+      146,
+      148,
+      149,
+      151,
+      153,
+      154,
+      155,
+      158,
+      161,
+      168,
+      169,
+      170,
+      171,
+      177,
+      184,
+      186,
+      188,
+      190,
+      197,
+      201,
+      202,
+      203,
+      209,
+      211,
+      212,
+      214,
+      215,
+      216,
+      228,
+      249,
+      252,
+      254
+    ],
+    "37": [
+      3,
+      5,
+      8,
+      11,
+      13,
+      14,
+      22,
+      24,
+      33,
+      36,
+      46,
+      51,
+      59,
+      60,
+      67,
+      70,
+      79,
+      82,
+      83,
+      85,
+      93,
+      95,
+      105,
+      111,
+      113,
+      117,
+      118,
+      125,
+      128,
+      133,
+      136,
+      141,
+      142,
+      143,
+      145,
+      151,
+      153,
+      154,
+      155,
+      161,
+      162,
+      163,
+      166,
+      171,
+      172,
+      180,
+      181,
+      184,
+      188,
+      189,
+      192,
+      197,
+      200,
+      212,
+      219,
+      220,
+      225,
+      227,
+      232,
+      237,
+      239,
+      246,
+      252,
+      254
+    ],
+    "38": [
+      0,
+      2,
+      6,
+      10,
+      15,
+      17,
+      22,
+      24,
+      28,
+      30,
+      37,
+      39,
+      41,
+      42,
+      47,
+      50,
+      51,
+      53,
+      60,
+      67,
+      68,
+      76,
+      78,
+      89,
+      91,
+      97,
+      98,
+      102,
+      105,
+      107,
+      108,
+      121,
+      123,
+      132,
+      138,
+      145,
+      147,
+      152,
+      153,
+      157,
+      159,
+      162,
+      164,
+      168,
+      178,
+      181,
+      183,
+      185,
+      187,
+      189,
+      199,
+      204,
+      207,
+      210,
+      214,
+      223,
+      225,
+      227,
+      228,
+      235,
+      239,
+      244,
+      245,
+      249
+    ],
+    "39": [
+      1,
+      7,
+      9,
+      10,
+      12,
+      13,
+      14,
+      16,
+      17,
+      19,
+      21,
+      22,
+      26,
+      32,
+      33,
+      37,
+      46,
+      58,
+      61,
+      62,
+      70,
+      71,
+      80,
+      86,
+      88,
+      102,
+      103,
+      104,
+      112,
+      114,
+      123,
+      125,
+      126,
+      127,
+      139,
+      141,
+      142,
+      146,
+      148,
+      153,
+      155,
+      161,
+      162,
+      168,
+      169,
+      177,
+      183,
+      184,
+      190,
+      196,
+      198,
+      199,
+      203,
+      206,
+      210,
+      212,
+      214,
+      221,
+      225,
+      230,
+      244,
+      247,
+      250,
+      253
+    ],
+    "40": [
+      2,
+      8,
+      10,
+      11,
+      26,
+      27,
+      28,
+      30,
+      37,
+      39,
+      43,
+      47,
+      50,
+      53,
+      56,
+      57,
+      59,
+      68,
+      74,
+      75,
+      76,
+      77,
+      88,
+      89,
+      93,
+      97,
+      99,
+      101,
+      103,
+      106,
+      111,
+      112,
+      119,
+      122,
+      123,
+      127,
+      128,
+      133,
+      137,
+      143,
+      145,
+      157,
+      168,
+      169,
+      171,
+      175,
+      177,
+      184,
+      188,
+      189,
+      191,
+      192,
+      194,
+      196,
+      197,
+      205,
+      227,
+      230,
+      237,
+      240,
+      244,
+      245,
+      251,
+      254
+    ],
+    "41": [
+      2,
+      7,
+      9,
+      10,
+      11,
+      12,
+      13,
+      16,
+      17,
+      18,
+      19,
+      25,
+      27,
+      29,
+      34,
+      37,
+      41,
+      42,
+      44,
+      57,
+      66,
+      73,
+      74,
+      80,
+      85,
+      89,
+      90,
+      99,
+      110,
+      111,
+      113,
+      115,
+      120,
+      126,
+      129,
+      131,
+      135,
+      138,
+      139,
+      147,
+      150,
+      152,
+      171,
+      178,
+      179,
+      180,
+      181,
+      195,
+      204,
+      205,
+      210,
+      226,
+      227,
+      229,
+      235,
+      239,
+      240,
+      241,
+      244,
+      249,
+      250,
+      251,
+      254,
+      255
+    ],
+    "42": [
+      2,
+      5,
+      11,
+      14,
+      16,
+      18,
+      20,
+      22,
+      29,
+      37,
+      47,
+      53,
+      55,
+      56,
+      60,
+      63,
+      66,
+      68,
+      70,
+      71,
+      74,
+      79,
+      81,
+      89,
+      93,
+      94,
+      101,
+      102,
+      103,
+      107,
+      112,
+      114,
+      119,
+      121,
+      127,
+      129,
+      131,
+      138,
+      141,
+      147,
+      148,
+      151,
+      158,
+      160,
+      163,
+      170,
+      172,
+      179,
+      182,
+      183,
+      188,
+      189,
+      190,
+      191,
+      193,
+      195,
+      202,
+      211,
+      217,
+      218,
+      228,
+      233,
+      242,
+      255
+    ],
+    "43": [
+      4,
+      5,
+      15,
+      18,
+      19,
+      27,
+      28,
+      30,
+      32,
+      35,
+      36,
+      38,
+      39,
+      41,
+      48,
+      49,
+      50,
+      51,
+      61,
+      63,
+      67,
+      74,
+      79,
+      80,
+      95,
+      100,
+      104,
+      108,
+      111,
+      128,
+      132,
+      144,
+      145,
+      147,
+      154,
+      156,
+      158,
+      162,
+      163,
+      164,
+      165,
+      169,
+      171,
+      176,
+      182,
+      189,
+      190,
+      192,
+      194,
+      199,
+      201,
+      204,
+      205,
+      211,
+      212,
+      214,
+      221,
+      233,
+      235,
+      238,
+      242,
+      244,
+      249,
+      252
+    ],
+    "44": [
+      18,
+      20,
+      21,
+      23,
+      24,
+      27,
+      34,
+      40,
+      50,
+      52,
+      61,
+      63,
+      64,
+      65,
+      69,
+      70,
+      73,
+      78,
+      84,
+      85,
+      88,
+      90,
+      91,
+      92,
+      101,
+      103,
+      104,
+      108,
+      111,
+      115,
+      122,
+      127,
+      131,
+      133,
+      135,
+      138,
+      142,
+      144,
+      148,
+      150,
+      153,
+      157,
+      158,
+      160,
+      174,
+      178,
+      180,
+      181,
+      182,
+      185,
+      186,
+      198,
+      208,
+      209,
+      210,
+      212,
+      220,
+      223,
+      228,
+      231,
+      242,
+      248,
+      251,
+      254
+    ],
+    "45": [
+      3,
+      12,
+      13,
+      16,
+      22,
+      27,
+      28,
+      39,
+      41,
+      42,
+      43,
+      45,
+      48,
+      65,
+      80,
+      81,
+      85,
+      88,
+      89,
+      93,
+      95,
+      96,
+      103,
+      105,
+      118,
+      121,
+      124,
+      135,
+      136,
+      137,
+      138,
+      140,
+      143,
+      144,
+      145,
+      146,
+      148,
+      151,
+      159,
+      162,
+      167,
+      169,
+      172,
+      186,
+      198,
+      199,
+      204,
+      216,
+      217,
+      222,
+      223,
+      226,
+      227,
+      228,
+      238,
+      239,
+      242,
+      245,
+      246,
+      247,
+      248,
+      250,
+      253,
+      255
+    ],
+    "46": [
+      2,
+      4,
+      10,
+      11,
+      13,
+      16,
+      18,
+      25,
+      28,
+      32,
+      42,
+      47,
+      51,
+      55,
+      60,
+      61,
+      62,
+      66,
+      67,
+      68,
+      73,
+      75,
+      77,
+      79,
+      95,
+      102,
+      104,
+      111,
+      118,
+      121,
+      127,
+      131,
+      139,
+      140,
+      144,
+      148,
+      152,
+      153,
+      155,
+      156,
+      157,
+      174,
+      186,
+      187,
+      192,
+      199,
+      201,
+      203,
+      206,
+      207,
+      210,
+      222,
+      224,
+      225,
+      227,
+      228,
+      229,
+      230,
+      231,
+      238,
+      239,
+      240,
+      243,
+      251
+    ],
+    "47": [
+      1,
+      4,
+      6,
+      18,
+      22,
+      41,
+      43,
+      45,
+      47,
+      48,
+      52,
+      56,
+      61,
+      63,
+      64,
+      65,
+      67,
+      70,
+      73,
+      74,
+      75,
+      78,
+      91,
+      93,
+      96,
+      106,
+      112,
+      117,
+      120,
+      122,
+      128,
+      129,
+      133,
+      135,
+      141,
+      146,
+      151,
+      154,
+      157,
+      158,
+      172,
+      173,
+      182,
+      195,
+      197,
+      200,
+      202,
+      203,
+      204,
+      207,
+      208,
+      209,
+      214,
+      215,
+      216,
+      217,
+      223,
+      225,
+      228,
+      230,
+      236,
+      241,
+      245,
+      249
+    ],
+    "48": [
+      14,
+      15,
+      18,
+      19,
+      22,
+      25,
+      28,
+      30,
+      48,
+      54,
+      55,
+      57,
+      61,
+      65,
+      66,
+      71,
+      73,
+      75,
+      78,
+      81,
+      82,
+      86,
+      94,
+      100,
+      104,
+      107,
+      111,
+      112,
+      114,
+      116,
+      117,
+      118,
+      122,
+      124,
+      129,
+      137,
+      138,
+      139,
+      146,
+      149,
+      151,
+      153,
+      159,
+      161,
+      162,
+      165,
+      173,
+      174,
+      176,
+      187,
+      189,
+      193,
+      200,
+      205,
+      210,
+      212,
+      234,
+      235,
+      236,
+      237,
+      244,
+      245,
+      247,
+      248
+    ],
+    "49": [
+      11,
+      16,
+      21,
+      23,
+      43,
+      53,
+      55,
+      57,
+      59,
+      61,
+      70,
+      77,
+      78,
+      79,
+      82,
+      83,
+      86,
+      91,
+      92,
+      95,
+      98,
+      103,
+      106,
+      110,
+      118,
+      119,
+      122,
+      125,
+      127,
+      134,
+      139,
+      140,
+      141,
+      142,
+      143,
+      147,
+      149,
+      161,
+      162,
+      163,
+      165,
+      168,
+      180,
+      181,
+      184,
+      185,
+      187,
+      192,
+      197,
+      198,
+      206,
+      210,
+      211,
+      212,
+      215,
+      217,
+      220,
+      226,
+      227,
+      229,
+      230,
+      247,
+      248,
+      251
+    ],
+    "50": [
+      1,
+      2,
+      5,
+      7,
+      15,
+      16,
+      27,
+      28,
+      29,
+      30,
+      32,
+      35,
+      38,
+      42,
+      44,
+      60,
+      61,
+      63,
+      65,
+      66,
+      69,
+      73,
+      74,
+      81,
+      85,
+      87,
+      89,
+      95,
+      98,
+      106,
+      107,
+      112,
+      125,
+      127,
+      133,
+      141,
+      142,
+      143,
+      144,
+      145,
+      153,
+      155,
+      157,
+      161,
+      163,
+      168,
+      170,
+      171,
+      192,
+      194,
+      197,
+      200,
+      204,
+      206,
+      207,
+      216,
+      218,
+      221,
+      229,
+      231,
+      244,
+      246,
+      248,
+      251
+    ],
+    "51": [
+      1,
+      10,
+      11,
+      16,
+      21,
+      22,
+      26,
+      29,
+      42,
+      45,
+      54,
+      56,
+      63,
+      65,
+      67,
+      70,
+      71,
+      72,
+      79,
+      85,
+      88,
+      92,
+      99,
+      102,
+      103,
+      105,
+      109,
+      114,
+      124,
+      126,
+      129,
+      130,
+      132,
+      138,
+      140,
+      141,
+      146,
+      147,
+      151,
+      152,
+      157,
+      164,
+      167,
+      174,
+      182,
+      186,
+      188,
+      190,
+      191,
+      195,
+      203,
+      208,
+      209,
+      211,
+      213,
+      214,
+      223,
+      226,
+      236,
+      237,
+      238,
+      242,
+      247,
+      253
+    ],
+    "52": [
+      2,
+      11,
+      12,
+      13,
+      17,
+      25,
+      27,
+      36,
+      41,
+      43,
+      45,
+      47,
+      50,
+      51,
+      55,
+      56,
+      71,
+      72,
+      77,
+      84,
+      86,
+      95,
+      99,
+      100,
+      101,
+      104,
+      105,
+      106,
+      114,
+      120,
+      131,
+      146,
+      147,
+      148,
+      152,
+      153,
+      156,
+      159,
+      162,
+      163,
+      165,
+      168,
+      172,
+      174,
+      175,
+      179,
+      193,
+      199,
+      201,
+      205,
+      209,
+      214,
+      215,
+      225,
+      230,
+      232,
+      236,
+      237,
+      240,
+      241,
+      242,
+      250,
+      252,
+      253
+    ],
+    "53": [
+      5,
+      19,
+      24,
+      28,
+      33,
+      36,
+      37,
+      38,
+      42,
+      45,
+      47,
+      48,
+      54,
+      65,
+      72,
+      88,
+      90,
+      93,
+      97,
+      99,
+      109,
+      112,
+      113,
+      116,
+      118,
+      119,
+      125,
+      128,
+      129,
+      130,
+      138,
+      139,
+      140,
+      141,
+      143,
+      145,
+      147,
+      150,
+      154,
+      166,
+      171,
+      173,
+      176,
+      177,
+      178,
+      181,
+      183,
+      195,
+      196,
+      204,
+      205,
+      206,
+      211,
+      212,
+      215,
+      216,
+      220,
+      224,
+      225,
+      226,
+      227,
+      236,
+      243,
+      249
+    ],
+    "54": [
+      3,
+      9,
+      13,
+      17,
+      18,
+      28,
+      39,
+      53,
+      54,
+      55,
+      56,
+      57,
+      68,
+      69,
+      75,
+      94,
+      95,
+      97,
+      100,
+      106,
+      113,
+      114,
+      116,
+      121,
+      122,
+      125,
+      138,
+      145,
+      147,
+      152,
+      153,
+      155,
+      158,
+      159,
+      160,
+      162,
+      163,
+      165,
+      170,
+      171,
+      172,
+      173,
+      174,
+      178,
+      184,
+      185,
+      191,
+      196,
+      198,
+      202,
+      207,
+      212,
+      219,
+      226,
+      227,
+      228,
+      230,
+      231,
+      232,
+      241,
+      242,
+      249,
+      252,
+      255
+    ],
+    "55": [
+      1,
+      21,
+      24,
+      32,
+      35,
+      53,
+      55,
+      58,
+      59,
+      63,
+      69,
+      82,
+      85,
+      90,
+      91,
+      93,
+      97,
+      102,
+      103,
+      104,
+      106,
+      107,
+      114,
+      122,
+      124,
+      125,
+      129,
+      133,
+      134,
+      136,
+      138,
+      140,
+      145,
+      152,
+      153,
+      155,
+      157,
+      162,
+      164,
+      165,
+      166,
+      188,
+      191,
+      192,
+      193,
+      195,
+      196,
+      198,
+      202,
+      204,
+      206,
+      208,
+      209,
+      217,
+      220,
+      226,
+      232,
+      235,
+      236,
+      241,
+      245,
+      246,
+      251,
+      252
+    ],
+    "56": [
+      6,
+      10,
+      11,
+      14,
+      25,
+      27,
+      31,
+      34,
+      36,
+      37,
+      39,
+      42,
+      44,
+      58,
+      62,
+      66,
+      67,
+      69,
+      71,
+      78,
+      84,
+      94,
+      99,
+      102,
+      107,
+      109,
+      115,
+      121,
+      127,
+      130,
+      132,
+      137,
+      141,
+      144,
+      145,
+      147,
+      156,
+      159,
+      164,
+      169,
+      177,
+      178,
+      180,
+      184,
+      185,
+      187,
+      188,
+      194,
+      196,
+      201,
+      203,
+      211,
+      213,
+      214,
+      219,
+      221,
+      223,
+      226,
+      232,
+      235,
+      238,
+      239,
+      244,
+      249
+    ],
+    "57": [
+      0,
+      4,
+      5,
+      8,
+      9,
+      10,
+      25,
+      35,
+      40,
+      44,
+      53,
+      54,
+      60,
+      69,
+      70,
+      71,
+      75,
+      79,
+      80,
+      85,
+      97,
+      98,
+      99,
+      109,
+      111,
+      112,
+      118,
+      119,
+      124,
+      128,
+      130,
+      131,
+      145,
+      147,
+      148,
+      151,
+      154,
+      160,
+      165,
+      169,
+      170,
+      171,
+      181,
+      189,
+      200,
+      206,
+      207,
+      211,
+      212,
+      216,
+      218,
+      220,
+      221,
+      222,
+      227,
+      234,
+      235,
+      236,
+      239,
+      242,
+      248,
+      249,
+      250,
+      251
+    ],
+    "58": [
+      0,
+      1,
+      4,
+      8,
+      13,
+      16,
+      20,
+      21,
+      25,
+      34,
+      36,
+      39,
+      41,
+      45,
+      46,
+      47,
+      57,
+      59,
+      68,
+      74,
+      76,
+      80,
+      81,
+      82,
+      87,
+      88,
+      92,
+      93,
+      101,
+      104,
+      105,
+      113,
+      114,
+      122,
+      123,
+      134,
+      141,
+      147,
+      149,
+      150,
+      156,
+      158,
+      159,
+      173,
+      177,
+      179,
+      180,
+      181,
+      182,
+      190,
+      193,
+      207,
+      212,
+      214,
+      216,
+      218,
+      221,
+      225,
+      227,
+      228,
+      235,
+      240,
+      241,
+      250
+    ],
+    "59": [
+      1,
+      2,
+      3,
+      12,
+      14,
+      15,
+      17,
+      27,
+      28,
+      35,
+      36,
+      49,
+      60,
+      69,
+      71,
+      78,
+      83,
+      94,
+      95,
+      100,
+      104,
+      106,
+      107,
+      108,
+      110,
+      111,
+      113,
+      116,
+      119,
+      123,
+      125,
+      126,
+      128,
+      131,
+      134,
+      136,
+      150,
+      159,
+      162,
+      163,
+      166,
+      168,
+      172,
+      175,
+      178,
+      180,
+      181,
+      182,
+      194,
+      195,
+      206,
+      210,
+      214,
+      216,
+      218,
+      221,
+      224,
+      225,
+      226,
+      229,
+      236,
+      237,
+      248,
+      253
+    ],
+    "60": [
+      5,
+      17,
+      20,
+      25,
+      26,
+      27,
+      33,
+      35,
+      38,
+      50,
+      52,
+      58,
+      59,
+      63,
+      64,
+      71,
+      72,
+      73,
+      74,
+      81,
+      82,
+      87,
+      95,
+      97,
+      104,
+      105,
+      107,
+      109,
+      113,
+      114,
+      120,
+      121,
+      123,
+      125,
+      131,
+      132,
+      147,
+      154,
+      157,
+      158,
+      159,
+      160,
+      162,
+      163,
+      169,
+      170,
+      178,
+      186,
+      192,
+      194,
+      196,
+      197,
+      199,
+      205,
+      213,
+      215,
+      216,
+      218,
+      221,
+      237,
+      246,
+      247,
+      248,
+      254
+    ],
+    "61": [
+      5,
+      8,
+      11,
+      16,
+      18,
+      24,
+      29,
+      32,
+      33,
+      36,
+      42,
+      45,
+      48,
+      49,
+      55,
+      56,
+      75,
+      79,
+      81,
+      82,
+      90,
+      94,
+      99,
+      104,
+      112,
+      117,
+      118,
+      124,
+      126,
+      128,
+      130,
+      131,
+      132,
+      140,
+      141,
+      145,
+      149,
+      156,
+      159,
+      161,
+      169,
+      171,
+      172,
+      176,
+      179,
+      183,
+      184,
+      190,
+      196,
+      199,
+      201,
+      202,
+      207,
+      210,
+      225,
+      228,
+      230,
+      231,
+      239,
+      241,
+      242,
+      245,
+      252,
+      255
+    ]
+  }
+}

extras/force_swap.cpp

@@ -0,0 +1,57 @@
+#include <iostream>
+#include <string>
+#include <cstdlib>
+#include <sys/mman.h>
+#include <unistd.h>
+#include <cstring>
+
+int main(int argc, char** argv) {
+    if (argc != 2) {
+        std::cerr << "Usage: " << argv[0] << " <GB to allocate and lock>\n";
+        return 1;
+    }
+
+    double gb = std::stod(argv[1]);
+    size_t bytes = static_cast<size_t>(gb * 1024.0 * 1024.0 * 1024.0);
+
+    std::cout << "Allocating " << gb << " GB (" << bytes << " bytes) of RAM...\n";
+    
+    void* ptr = mmap(NULL, bytes, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+    if (ptr == MAP_FAILED) {
+        perror("mmap failed");
+        return 1;
+    }
+
+    std::cout << "Memory allocated. Faulting pages and pinning to RAM...\n";
+    
+    // Write to memory to map it to physical pages and prevent lazy allocation
+    size_t page_size = sysconf(_SC_PAGESIZE);
+    char* char_ptr = static_cast<char*>(ptr);
+    for (size_t i = 0; i < bytes; i += page_size) {
+        char_ptr[i] = 1;
+    }
+
+    // Mlock to pin it to RAM and prevent it from being swapped out itself
+    if (mlock(ptr, bytes) != 0) {
+        perror("mlock failed (you probably need to run with sudo)");
+    } else {
+        std::cout << "mlock successful.\n";
+    }
+
+    std::cout << "Memory is fully resident. Other inactive processes/caches should be pushed to swap.\n";
+    std::cout << "Clearing filesystem caches...\n";
+    
+    int ret = system("echo 3 | sudo tee /proc/sys/vm/drop_caches > /dev/null");
+    if (ret != 0) {
+        std::cerr << "Failed to clear caches. (Maybe sudo failed?)\n";
+    } else {
+        std::cout << "Caches cleared successfully.\n";
+    }
+
+    std::cout << "Unlocking and releasing memory...\n";
+    munlock(ptr, bytes);
+    munmap(ptr, bytes);
+
+    std::cout << "Done! Try starting your model now.\n";
+    return 0;
+}

extras/inference_minimax.sh

@@ -0,0 +1,29 @@
+#!/bin/sh -
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+MODEL_DIR="$(dirname "$SCRIPT_DIR")"
+. ~/venv/vllm/bin/activate
+
+dockless destroy seeker-inference
+sleep 5
+dockless run -d --name seeker-inference \
+  -e CUDA_HOME=/usr/local/cuda-13.0 \
+  -e C_INCLUDE_PATH=/usr/local/cuda-13.0/include \
+  -e LIBRARY_PATH=/usr/lib/aarch64-linux-gnu/nvidia \
+  -e FLASHINFER_NVCC=/usr/local/cuda-13.0/bin/nvcc \
+  -e VLLM_USE_FLASHINFER_MOE_FP4=0  "$SCRIPT_DIR/unglitched_vllm" \
+  --served-model-name Nikola \
+  --port 9000 \
+  --enable-auto-tool-choice \
+  --tool-call-parser minimax_m2 \
+  --reasoning-parser minimax_m2_optthink \
+  --reasoning-parser-plugin "$SCRIPT_DIR/minimax_m2_optthink_reasoning_parser.py" \
+  --enable-prefix-caching \
+  --max-num-seqs 4 --cudagraph-capture-sizes 1 2 4 --max-model-len auto \
+  --max_num_batched_tokens 8192 \
+  --gpu-memory-utilization 0.95 \
+  --attention-backend FLASHINFER \
+  --async-scheduling \
+  --enable-chunked-prefill \
+  --chat-template "$SCRIPT_DIR/chat_template.jinja" \
+  --model "${@:-$MODEL_DIR}"
+

extras/minimax_m2_optthink_reasoning_parser.py

@@ -0,0 +1,102 @@
+# minimax_m2_optthink_reasoning_parser.py
+# SPDX-License-Identifier: Apache-2.0
+
+from collections.abc import Sequence
+from typing import TYPE_CHECKING
+
+from vllm.entrypoints.openai.engine.protocol import DeltaMessage
+from vllm.logger import init_logger
+from vllm.reasoning.abs_reasoning_parsers import ReasoningParserManager
+from vllm.reasoning.basic_parsers import BaseThinkingReasoningParser
+from vllm.tokenizers import TokenizerLike
+
+if TYPE_CHECKING:
+    from vllm.entrypoints.openai.chat_completion.protocol import ChatCompletionRequest
+    from vllm.entrypoints.openai.responses.protocol import ResponsesRequest
+
+logger = init_logger(__name__)
+
+@ReasoningParserManager.register_module("minimax_m2_optthink")
+class MiniMaxM2OptthinkReasoningParser(BaseThinkingReasoningParser):
+    """
+    Reasoning parser for MiniMax M2 model that handles enable_thinking=False configs.
+
+    MiniMax M2 models don't generate <think> start token, only </think> end
+    token. All content before </think> is reasoning, content after is the
+    actual response. This checks chat_template_kwargs for thinking config.
+    """
+
+    def __init__(self, tokenizer: TokenizerLike, *args, **kwargs):
+        super().__init__(tokenizer, *args, **kwargs)
+        chat_kwargs = kwargs.get("chat_template_kwargs", {}) or {}
+        self.thinking_enabled = chat_kwargs.get("enable_thinking", True)
+
+    @property
+    def start_token(self) -> str:
+        """The token that starts reasoning content."""
+        return "<think>"
+
+    @property
+    def end_token(self) -> str:
+        """The token that ends reasoning content."""
+        return "</think>"
+
+    def extract_reasoning(
+        self, model_output: str, request: "ChatCompletionRequest | ResponsesRequest"
+    ) -> tuple[str | None, str | None]:
+        # Strip <think> if present in the generated output.
+        model_output_parts = model_output.partition(self.start_token)
+        model_output = (
+            model_output_parts[2] if model_output_parts[1] else model_output_parts[0]
+        )
+
+        if self.end_token not in model_output:
+            if not self.thinking_enabled:
+                # Thinking explicitly disabled \u2014 treat everything as content.
+                return None, model_output
+            # Thinking enabled but no </think>: output was truncated.
+            return model_output, None
+
+        # Extract reasoning content from the model output.
+        reasoning, _, content = model_output.partition(self.end_token)
+        final_content = content or None
+        return reasoning, final_content
+
+    def extract_reasoning_streaming(
+        self,
+        previous_text: str,
+        current_text: str,
+        delta_text: str,
+        previous_token_ids: Sequence[int],
+        current_token_ids: Sequence[int],
+        delta_token_ids: Sequence[int],
+    ) -> DeltaMessage | None:
+        """
+        Extract reasoning content from a delta message for streaming.
+        """
+        # Skip single special tokens
+        if len(delta_token_ids) == 1 and (
+            delta_token_ids[0] in [self.start_token_id, self.end_token_id]
+        ):
+            return None
+
+        # Check if end token has already appeared in previous tokens
+        if self.end_token_id in previous_token_ids:
+            # We're past the reasoning phase, this is content
+            return DeltaMessage(content=delta_text)
+
+        # Check if end token is in delta tokens
+        if self.end_token_id in delta_token_ids:
+            # End token in delta, split reasoning and content
+            end_index = delta_text.find(self.end_token)
+            if end_index >= 0:
+                reasoning = delta_text[:end_index]
+                content = delta_text[end_index + len(self.end_token) :]
+                return DeltaMessage(
+                    reasoning=reasoning if reasoning else None,
+                    content=content if content else None,
+                )
+            return None
+
+        # No end token yet, all content is reasoning
+        return DeltaMessage(reasoning=delta_text)

extras/unglitched_vllm

@@ -0,0 +1,3 @@
+#!/bin/sh -
+unglitch 119
+exec python -m vllm.entrypoints.openai.api_server "$@"

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "bos_token_id": 200019,
+  "do_sample": true,
+  "eos_token_id": 200020,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "top_k": 40,
+  "transformers_version": "4.46.1"
+}

hf_quant_config.json

@@ -0,0 +1,138 @@
+{
+    "producer": {
+        "name": "modelopt",
+        "version": "0.43.0rc2.dev105+g0b42c143d"
+    },
+    "quantization": {
+        "quant_algo": "NVFP4",
+        "kv_cache_quant_algo": "FP8",
+        "group_size": 16,
+        "exclude_modules": [
+            "lm_head",
+            "model.layers.0.block_sparse_moe.gate",
+            "model.layers.0.self_attn*",
+            "model.layers.1.block_sparse_moe.gate",
+            "model.layers.1.self_attn*",
+            "model.layers.10.block_sparse_moe.gate",
+            "model.layers.10.self_attn*",
+            "model.layers.11.block_sparse_moe.gate",
+            "model.layers.11.self_attn*",
+            "model.layers.12.block_sparse_moe.gate",
+            "model.layers.12.self_attn*",
+            "model.layers.13.block_sparse_moe.gate",
+            "model.layers.13.self_attn*",
+            "model.layers.14.block_sparse_moe.gate",
+            "model.layers.14.self_attn*",
+            "model.layers.15.block_sparse_moe.gate",
+            "model.layers.15.self_attn*",
+            "model.layers.16.block_sparse_moe.gate",
+            "model.layers.16.self_attn*",
+            "model.layers.17.block_sparse_moe.gate",
+            "model.layers.17.self_attn*",
+            "model.layers.18.block_sparse_moe.gate",
+            "model.layers.18.self_attn*",
+            "model.layers.19.block_sparse_moe.gate",
+            "model.layers.19.self_attn*",
+            "model.layers.2.block_sparse_moe.gate",
+            "model.layers.2.self_attn*",
+            "model.layers.20.block_sparse_moe.gate",
+            "model.layers.20.self_attn*",
+            "model.layers.21.block_sparse_moe.gate",
+            "model.layers.21.self_attn*",
+            "model.layers.22.block_sparse_moe.gate",
+            "model.layers.22.self_attn*",
+            "model.layers.23.block_sparse_moe.gate",
+            "model.layers.23.self_attn*",
+            "model.layers.24.block_sparse_moe.gate",
+            "model.layers.24.self_attn*",
+            "model.layers.25.block_sparse_moe.gate",
+            "model.layers.25.self_attn*",
+            "model.layers.26.block_sparse_moe.gate",
+            "model.layers.26.self_attn*",
+            "model.layers.27.block_sparse_moe.gate",
+            "model.layers.27.self_attn*",
+            "model.layers.28.block_sparse_moe.gate",
+            "model.layers.28.self_attn*",
+            "model.layers.29.block_sparse_moe.gate",
+            "model.layers.29.self_attn*",
+            "model.layers.3.block_sparse_moe.gate",
+            "model.layers.3.self_attn*",
+            "model.layers.30.block_sparse_moe.gate",
+            "model.layers.30.self_attn*",
+            "model.layers.31.block_sparse_moe.gate",
+            "model.layers.31.self_attn*",
+            "model.layers.32.block_sparse_moe.gate",
+            "model.layers.32.self_attn*",
+            "model.layers.33.block_sparse_moe.gate",
+            "model.layers.33.self_attn*",
+            "model.layers.34.block_sparse_moe.gate",
+            "model.layers.34.self_attn*",
+            "model.layers.35.block_sparse_moe.gate",
+            "model.layers.35.self_attn*",
+            "model.layers.36.block_sparse_moe.gate",
+            "model.layers.36.self_attn*",
+            "model.layers.37.block_sparse_moe.gate",
+            "model.layers.37.self_attn*",
+            "model.layers.38.block_sparse_moe.gate",
+            "model.layers.38.self_attn*",
+            "model.layers.39.block_sparse_moe.gate",
+            "model.layers.39.self_attn*",
+            "model.layers.4.block_sparse_moe.gate",
+            "model.layers.4.self_attn*",
+            "model.layers.40.block_sparse_moe.gate",
+            "model.layers.40.self_attn*",
+            "model.layers.41.block_sparse_moe.gate",
+            "model.layers.41.self_attn*",
+            "model.layers.42.block_sparse_moe.gate",
+            "model.layers.42.self_attn*",
+            "model.layers.43.block_sparse_moe.gate",
+            "model.layers.43.self_attn*",
+            "model.layers.44.block_sparse_moe.gate",
+            "model.layers.44.self_attn*",
+            "model.layers.45.block_sparse_moe.gate",
+            "model.layers.45.self_attn*",
+            "model.layers.46.block_sparse_moe.gate",
+            "model.layers.46.self_attn*",
+            "model.layers.47.block_sparse_moe.gate",
+            "model.layers.47.self_attn*",
+            "model.layers.48.block_sparse_moe.gate",
+            "model.layers.48.self_attn*",
+            "model.layers.49.block_sparse_moe.gate",
+            "model.layers.49.self_attn*",
+            "model.layers.5.block_sparse_moe.gate",
+            "model.layers.5.self_attn*",
+            "model.layers.50.block_sparse_moe.gate",
+            "model.layers.50.self_attn*",
+            "model.layers.51.block_sparse_moe.gate",
+            "model.layers.51.self_attn*",
+            "model.layers.52.block_sparse_moe.gate",
+            "model.layers.52.self_attn*",
+            "model.layers.53.block_sparse_moe.gate",
+            "model.layers.53.self_attn*",
+            "model.layers.54.block_sparse_moe.gate",
+            "model.layers.54.self_attn*",
+            "model.layers.55.block_sparse_moe.gate",
+            "model.layers.55.self_attn*",
+            "model.layers.56.block_sparse_moe.gate",
+            "model.layers.56.self_attn*",
+            "model.layers.57.block_sparse_moe.gate",
+            "model.layers.57.self_attn*",
+            "model.layers.58.block_sparse_moe.gate",
+            "model.layers.58.self_attn*",
+            "model.layers.59.block_sparse_moe.gate",
+            "model.layers.59.self_attn*",
+            "model.layers.6.block_sparse_moe.gate",
+            "model.layers.6.self_attn*",
+            "model.layers.60.block_sparse_moe.gate",
+            "model.layers.60.self_attn*",
+            "model.layers.61.block_sparse_moe.gate",
+            "model.layers.61.self_attn*",
+            "model.layers.7.block_sparse_moe.gate",
+            "model.layers.7.self_attn*",
+            "model.layers.8.block_sparse_moe.gate",
+            "model.layers.8.self_attn*",
+            "model.layers.9.block_sparse_moe.gate",
+            "model.layers.9.self_attn*"
+        ]
+    }
+}

model-00001-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:005d3c3fa71564476658cac6169fe165f86b616a7a183142b239589ca7ffaee2
+size 7983477840

model-00002-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e8d753b20328e591d7473fcca8e43bceaa65ff7f1860e63660afa9c2477965c
+size 7736049080

model-00003-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a6df42a11c868576d109a2716cc09b9a122e26837be4392e0e015f39e46c113
+size 7654258000

model-00004-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84b7c59d56500b27b78277b15b349dc3acc52febf921e9736c58607f9a6e9950
+size 7698893312

model-00005-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:087db31546dc220410b2bf4613c726363b4ef1e7a54b7f5b5e993992dc7e94f9
+size 7699398176

model-00006-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75f6c08f376e44e25f25c4ab5092cd879eecb72ca6db012756ad363ce57964cf
+size 7688262080

model-00007-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db7b38244fe7cf73f2e5575ef037b1672021a7eca9fccb4a7dcd41a5fcad3818
+size 7670196272

model-00008-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6ca577b76b4ac12b440dbc5a382881eb85b91e5a74395b0c3aa862a46c5a9f8
+size 7680297864

model-00009-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce84cc4ff6612f95e1c948ddf4de8c35614406c752366b1a365245ac7b5f9eeb
+size 7678160424

model-00010-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdb26e874b3e6f0a2506b2b70a60bb08de6b1b5ec3981f31ed2e00c09859899d
+size 7720118944

model-00011-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9ffec0eefcd499d0d21702660f71d057f17d0f4ca214e5f5a751fe7849066f0
+size 7638339656

model-00012-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0cee4e7c25124f84065b4595b694f126953a323466a8deebafe06280b1650017
+size 7751975800

model-00013-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46107bc18db6fd66f0e09e0d7397b6ecb9c8e67fe7e3f1dde9be72bb484ebd13
+size 7606483032

model-00014-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:629d67634d0bd610aae31fb20f993173a4b1f70dac730f70b3267490e078561f
+size 6825988456

model-00015-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a68c3eb1afc80eca0909878fda4cb51750ff5edf3f6d4bda1ec059cd8f8fc76
+size 1229193312

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb2c946cc1efede3be704f7762265295192c7a869ea3d0086a96d7a63125323d
+size 14636608

modeling_minimax_m2.py

@@ -0,0 +1,706 @@
+#                🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨
+#           This file was automatically generated from src/transformers/models/minimax_m2/modular_minimax_m2.py.
+#               Do NOT edit this file manually as any edits will be overwritten by the generation of
+#             the file from the modular. If any change should be done, please apply the change to the
+#                          modular_minimax_m2.py file directly. One of our CI enforces this.
+#                🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨
+# coding=utf-8
+# Copyright 2025 the HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+from collections.abc import Callable
+from typing import Optional, Union, Unpack
+
+import torch
+from torch import nn
+
+from transformers.activations import ACT2FN
+from transformers.cache_utils import Cache, DynamicCache
+from transformers.generation import GenerationMixin
+from transformers.integrations import use_kernel_forward_from_hub
+from transformers.masking_utils import create_causal_mask, create_sliding_window_causal_mask
+from transformers.modeling_flash_attention_utils import FlashAttentionKwargs
+from transformers.modeling_layers import (
+    GenericForQuestionAnswering,
+    GenericForSequenceClassification,
+    GenericForTokenClassification,
+    GradientCheckpointingLayer,
+)
+from transformers.modeling_outputs import MoeCausalLMOutputWithPast, MoeModelOutputWithPast
+from transformers.modeling_rope_utils import ROPE_INIT_FUNCTIONS, dynamic_rope_update
+from transformers.modeling_utils import ALL_ATTENTION_FUNCTIONS, PreTrainedModel
+from transformers.utils import TransformersKwargs, auto_docstring, can_return_tuple
+from transformers.utils.deprecation import deprecate_kwarg
+from transformers.utils.generic import OutputRecorder, check_model_inputs
+from .configuration_minimax_m2 import MiniMaxM2Config
+
+
+class MiniMaxM2MLP(nn.Module):
+    def __init__(self, config: MiniMaxM2Config):
+        super().__init__()
+        self.ffn_dim = config.intermediate_size
+        self.hidden_dim = config.hidden_size
+
+        self.w1 = nn.Linear(self.hidden_dim, self.ffn_dim, bias=False)
+        self.w2 = nn.Linear(self.ffn_dim, self.hidden_dim, bias=False)
+        self.w3 = nn.Linear(self.hidden_dim, self.ffn_dim, bias=False)
+
+        self.act_fn = ACT2FN[config.hidden_act]
+
+    def forward(self, hidden_states):
+        current_hidden_states = self.act_fn(self.w1(hidden_states)) * self.w3(hidden_states)
+        current_hidden_states = self.w2(current_hidden_states)
+        return current_hidden_states
+
+
+class MiniMaxM2Experts(nn.ModuleList):
+    """
+    ModuleList of experts.
+    """
+
+    def __init__(self, config: MiniMaxM2Config):
+        super().__init__()
+        self.top_k = config.num_experts_per_tok
+        self.num_experts = config.num_local_experts
+        for _ in range(self.num_experts):
+            self.append(MiniMaxM2MLP(config))
+
+    def forward(
+        self, hidden_states: torch.Tensor, top_k_index: torch.Tensor, top_k_weights: torch.Tensor
+    ) -> torch.Tensor:
+        """
+        Args:
+            hidden_states: (batch_size * sequence_length, hidden_dim)
+            selected_experts: (batch_size * sequence_length, top_k)
+            routing_weights: (batch_size * sequence_length, top_k)
+        Returns:
+            (batch_size * sequence_length, hidden_dim)
+        """
+        final_hidden_states = torch.zeros_like(hidden_states)
+        expert_mask = torch.nn.functional.one_hot(top_k_index, num_classes=self.num_experts).permute(2, 1, 0)
+
+        expert_hit = torch.greater(expert_mask.sum(dim=(-1, -2)), 0).nonzero()
+        for expert_idx in expert_hit:
+            idx, top_x = torch.where(expert_mask[expert_idx].squeeze(0))
+            current_state = hidden_states[None, top_x].reshape(-1, hidden_states.shape[-1])
+            current_hidden_states = self[expert_idx](current_state) * top_k_weights[top_x, idx, None]
+            final_hidden_states.index_add_(0, top_x, current_hidden_states.to(hidden_states.dtype))
+        return final_hidden_states
+
+
+class MiniMaxM2SparseMoeBlock(nn.Module):
+    def __init__(self, config):
+        super().__init__()
+        self.top_k = config.num_experts_per_tok
+        self.jitter_noise = config.router_jitter_noise
+        self.gate = nn.Linear(config.hidden_size, config.num_local_experts, bias=False)
+        self.experts = MiniMaxM2Experts(config)
+        self.register_buffer("e_score_correction_bias", torch.zeros(config.num_local_experts))
+
+    def route_tokens_to_experts(self, router_logits):
+        routing_weights = torch.nn.functional.sigmoid(router_logits.float())
+        scores_for_choice = routing_weights + self.e_score_correction_bias
+        _, top_k_index = torch.topk(scores_for_choice, self.top_k, dim=-1, sorted=False)
+        top_k_weights = routing_weights.gather(1, top_k_index)
+        top_k_weights /= top_k_weights.sum(dim=-1, keepdim=True)
+        return top_k_index, top_k_weights.to(router_logits.dtype)
+
+    def forward(self, hidden_states: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
+        batch_size, sequence_length, hidden_dim = hidden_states.shape
+        if self.training and self.jitter_noise > 0:
+            hidden_states *= torch.empty_like(hidden_states).uniform_(1.0 - self.jitter_noise, 1.0 + self.jitter_noise)
+        hidden_states = hidden_states.view(-1, hidden_states.shape[-1])
+        router_logits = self.gate(hidden_states)
+        top_k_index, top_k_weights = self.route_tokens_to_experts(router_logits)
+        hidden_states = self.experts(hidden_states, top_k_index, top_k_weights.to(hidden_states.dtype))
+        hidden_states = hidden_states.reshape(batch_size, sequence_length, hidden_dim)
+        return hidden_states, router_logits
+
+
+@use_kernel_forward_from_hub("RMSNorm")
+class MiniMaxM2RMSNorm(nn.Module):
+    def __init__(self, hidden_size, eps=1e-6):
+        """
+        MiniMaxM2RMSNorm is equivalent to T5LayerNorm
+        """
+        super().__init__()
+        self.weight = nn.Parameter(torch.ones(hidden_size))
+        self.variance_epsilon = eps
+
+    def forward(self, hidden_states):
+        input_dtype = hidden_states.dtype
+        hidden_states = hidden_states.to(torch.float32)
+        variance = hidden_states.pow(2).mean(-1, keepdim=True)
+        hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon)
+        return self.weight * hidden_states.to(input_dtype)
+
+    def extra_repr(self):
+        return f"{tuple(self.weight.shape)}, eps={self.variance_epsilon}"
+
+
+def repeat_kv(hidden_states: torch.Tensor, n_rep: int) -> torch.Tensor:
+    """
+    This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch,
+    num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim)
+    """
+    batch, num_key_value_heads, slen, head_dim = hidden_states.shape
+    if n_rep == 1:
+        return hidden_states
+    hidden_states = hidden_states[:, :, None, :, :].expand(batch, num_key_value_heads, n_rep, slen, head_dim)
+    return hidden_states.reshape(batch, num_key_value_heads * n_rep, slen, head_dim)
+
+
+def eager_attention_forward(
+    module: nn.Module,
+    query: torch.Tensor,
+    key: torch.Tensor,
+    value: torch.Tensor,
+    attention_mask: Optional[torch.Tensor],
+    scaling: float,
+    dropout: float = 0.0,
+    **kwargs: Unpack[TransformersKwargs],
+):
+    key_states = repeat_kv(key, module.num_key_value_groups)
+    value_states = repeat_kv(value, module.num_key_value_groups)
+
+    attn_weights = torch.matmul(query, key_states.transpose(2, 3)) * scaling
+    if attention_mask is not None:
+        causal_mask = attention_mask[:, :, :, : key_states.shape[-2]]
+        attn_weights = attn_weights + causal_mask
+
+    attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query.dtype)
+    attn_weights = nn.functional.dropout(attn_weights, p=dropout, training=module.training)
+    attn_output = torch.matmul(attn_weights, value_states)
+    attn_output = attn_output.transpose(1, 2).contiguous()
+
+    return attn_output, attn_weights
+
+
+def rotate_half(x):
+    """Rotates half the hidden dims of the input."""
+    x1 = x[..., : x.shape[-1] // 2]
+    x2 = x[..., x.shape[-1] // 2 :]
+    return torch.cat((-x2, x1), dim=-1)
+
+
+def apply_rotary_pos_emb(q, k, cos, sin, position_ids=None, unsqueeze_dim=1):
+    """Applies Rotary Position Embedding to the query and key tensors.
+
+    Args:
+        q (`torch.Tensor`): The query tensor.
+        k (`torch.Tensor`): The key tensor.
+        cos (`torch.Tensor`): The cosine part of the rotary embedding.
+        sin (`torch.Tensor`): The sine part of the rotary embedding.
+        position_ids (`torch.Tensor`, *optional*):
+            Deprecated and unused.
+        unsqueeze_dim (`int`, *optional*, defaults to 1):
+            The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and
+            sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note
+            that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and
+            k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes
+            cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have
+            the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2.
+    Returns:
+        `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding.
+    """
+    cos = cos.unsqueeze(unsqueeze_dim)
+    sin = sin.unsqueeze(unsqueeze_dim)
+
+    # Keep half or full tensor for later concatenation
+    rotary_dim = cos.shape[-1]
+    q_rot, q_pass = q[..., :rotary_dim], q[..., rotary_dim:]
+    k_rot, k_pass = k[..., :rotary_dim], k[..., rotary_dim:]
+
+    # Apply rotary embeddings on the first half or full tensor
+    q_embed = (q_rot * cos) + (rotate_half(q_rot) * sin)
+    k_embed = (k_rot * cos) + (rotate_half(k_rot) * sin)
+
+    # Concatenate back to full shape
+    q_embed = torch.cat([q_embed, q_pass], dim=-1)
+    k_embed = torch.cat([k_embed, k_pass], dim=-1)
+    return q_embed, k_embed
+
+
+class MiniMaxM2Attention(nn.Module):
+    """Multi-headed attention from 'Attention Is All You Need' paper"""
+
+    def __init__(self, config: MiniMaxM2Config, layer_idx: int):
+        super().__init__()
+        self.config = config
+        self.layer_idx = layer_idx
+        self.head_dim = getattr(config, "head_dim", None) or config.hidden_size // config.num_attention_heads
+        self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
+        self.scaling = self.head_dim**-0.5
+        self.attention_dropout = config.attention_dropout
+        self.is_causal = True
+        self.q_proj = nn.Linear(config.hidden_size, config.num_attention_heads * self.head_dim, bias=False)
+        self.k_proj = nn.Linear(config.hidden_size, config.num_key_value_heads * self.head_dim, bias=False)
+        self.v_proj = nn.Linear(config.hidden_size, config.num_key_value_heads * self.head_dim, bias=False)
+        self.o_proj = nn.Linear(config.num_attention_heads * self.head_dim, config.hidden_size, bias=False)
+
+        self.use_qk_norm = config.use_qk_norm
+        if self.use_qk_norm:
+            self.q_norm = MiniMaxM2RMSNorm(self.head_dim * config.num_attention_heads, eps=config.rms_norm_eps)
+            self.k_norm = MiniMaxM2RMSNorm(self.head_dim * config.num_key_value_heads, eps=config.rms_norm_eps)
+
+    @deprecate_kwarg("past_key_value", new_name="past_key_values", version="4.58")
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        position_embeddings: tuple[torch.Tensor, torch.Tensor],
+        attention_mask: Optional[torch.Tensor],
+        past_key_values: Optional[Cache] = None,
+        cache_position: Optional[torch.LongTensor] = None,
+        **kwargs: Unpack[FlashAttentionKwargs],
+    ) -> tuple[torch.Tensor, Optional[torch.Tensor]]:
+        input_shape = hidden_states.shape[:-1]
+        hidden_shape = (*input_shape, -1, self.head_dim)
+
+        query_states = self.q_proj(hidden_states)
+        key_states = self.k_proj(hidden_states)
+        value_states = self.v_proj(hidden_states)
+
+        if self.use_qk_norm:  # main diff from Llama
+            query_states = self.q_norm(query_states)
+            key_states = self.k_norm(key_states)
+
+        key_states = key_states.view(hidden_shape)
+        query_states = query_states.view(hidden_shape)
+        value_states = value_states.view(hidden_shape)
+
+        query_states = query_states.transpose(1, 2)
+        key_states = key_states.transpose(1, 2)
+        value_states = value_states.transpose(1, 2)
+
+        cos, sin = position_embeddings
+        query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin)
+
+        if past_key_values is not None:
+            # sin and cos are specific to RoPE models; position_ids needed for the static cache
+            cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position}
+            key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_kwargs)
+
+        attention_interface: Callable = eager_attention_forward
+        if self.config._attn_implementation != "eager":
+            attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation]
+
+        attn_output, attn_weights = attention_interface(
+            self,
+            query_states,
+            key_states,
+            value_states,
+            attention_mask,
+            dropout=0.0 if not self.training else self.attention_dropout,
+            scaling=self.scaling,
+            **kwargs,
+        )
+
+        attn_output = attn_output.reshape(*input_shape, -1).contiguous()
+        attn_output = self.o_proj(attn_output)
+        return attn_output, attn_weights
+
+
+class MiniMaxM2DecoderLayer(GradientCheckpointingLayer):
+    def __init__(self, config: MiniMaxM2Config, layer_idx: int):
+        super().__init__()
+        self.hidden_size = config.hidden_size
+
+        self.self_attn = MiniMaxM2Attention(config, layer_idx)
+
+        self.block_sparse_moe = MiniMaxM2SparseMoeBlock(config)
+        self.input_layernorm = MiniMaxM2RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.post_attention_layernorm = MiniMaxM2RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+
+    @deprecate_kwarg("past_key_value", new_name="past_key_values", version="4.58")
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        position_embeddings: tuple[torch.Tensor, torch.Tensor],
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[Cache] = None,
+        cache_position: Optional[torch.LongTensor] = None,
+        **kwargs: Unpack[TransformersKwargs],
+    ) -> torch.FloatTensor:
+        residual = hidden_states
+
+        hidden_states = self.input_layernorm(hidden_states)
+
+        # Self Attention
+        hidden_states, _ = self.self_attn(
+            hidden_states=hidden_states,
+            position_embeddings=position_embeddings,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            cache_position=cache_position,
+            **kwargs,
+        )
+        hidden_states = residual + hidden_states
+
+        # Fully Connected
+        residual = hidden_states
+        hidden_states = self.post_attention_layernorm(hidden_states)
+        hidden_states, _ = self.block_sparse_moe(hidden_states)
+        hidden_states = residual + hidden_states
+
+        return hidden_states
+
+
+class MiniMaxM2RotaryEmbedding(nn.Module):
+    inv_freq: torch.Tensor  # fix linting for `register_buffer`
+
+    def __init__(self, config: MiniMaxM2Config, device=None):
+        super().__init__()
+        # BC: "rope_type" was originally "type"
+        if hasattr(config, "rope_scaling") and isinstance(config.rope_scaling, dict):
+            self.rope_type = config.rope_scaling.get("rope_type", config.rope_scaling.get("type"))
+        else:
+            self.rope_type = "default"
+        self.max_seq_len_cached = config.max_position_embeddings
+        self.original_max_seq_len = config.max_position_embeddings
+
+        self.config = config
+        self.rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
+
+        inv_freq, self.attention_scaling = self.rope_init_fn(self.config, device)
+        self.register_buffer("inv_freq", inv_freq, persistent=False)
+        self.original_inv_freq = self.inv_freq
+
+    @torch.no_grad()
+    @dynamic_rope_update  # power user: used with advanced RoPE types (e.g. dynamic rope)
+    def forward(self, x, position_ids):
+        inv_freq_expanded = self.inv_freq[None, :, None].float().expand(position_ids.shape[0], -1, 1).to(x.device)
+        position_ids_expanded = position_ids[:, None, :].float()
+
+        device_type = x.device.type if isinstance(x.device.type, str) and x.device.type != "mps" else "cpu"
+        with torch.autocast(device_type=device_type, enabled=False):  # Force float32
+            freqs = (inv_freq_expanded.float() @ position_ids_expanded.float()).transpose(1, 2)
+            emb = torch.cat((freqs, freqs), dim=-1)
+            cos = emb.cos() * self.attention_scaling
+            sin = emb.sin() * self.attention_scaling
+
+        return cos.to(dtype=x.dtype), sin.to(dtype=x.dtype)
+
+
+@auto_docstring
+class MiniMaxM2PreTrainedModel(PreTrainedModel):
+    config: MiniMaxM2Config
+    base_model_prefix = "model"
+    supports_gradient_checkpointing = True
+    _no_split_modules = ["MiniMaxM2DecoderLayer"]
+    _skip_keys_device_placement = ["past_key_values"]
+    _supports_flash_attn = True
+    _supports_sdpa = True
+    _supports_flex_attn = True
+    _can_compile_fullgraph = False  # MoE models don't work with torch.compile (`torch.where(condition)` not supported)
+    _supports_attention_backend = True
+    _can_record_outputs = {
+        "router_logits": OutputRecorder(MiniMaxM2SparseMoeBlock, index=1),
+        "hidden_states": MiniMaxM2DecoderLayer,
+        "attentions": MiniMaxM2Attention,
+    }
+
+
+@auto_docstring
+class MiniMaxM2Model(MiniMaxM2PreTrainedModel):
+    def __init__(self, config: MiniMaxM2Config):
+        super().__init__(config)
+        self.padding_idx = config.pad_token_id
+        self.vocab_size = config.vocab_size
+
+        self.embed_tokens = nn.Embedding(config.vocab_size, config.hidden_size, self.padding_idx)
+        self.layers = nn.ModuleList(
+            [MiniMaxM2DecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)]
+        )
+        self.norm = MiniMaxM2RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.rotary_emb = MiniMaxM2RotaryEmbedding(config=config)
+        self.gradient_checkpointing = False
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @check_model_inputs
+    @auto_docstring
+    def forward(
+        self,
+        input_ids: Optional[torch.LongTensor] = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[Cache] = None,
+        inputs_embeds: Optional[torch.FloatTensor] = None,
+        use_cache: Optional[bool] = None,
+        cache_position: Optional[torch.LongTensor] = None,
+        **kwargs: Unpack[TransformersKwargs],
+    ) -> MoeModelOutputWithPast:
+        if (input_ids is None) ^ (inputs_embeds is not None):
+            raise ValueError("You must specify exactly one of input_ids or inputs_embeds")
+
+        if use_cache and past_key_values is None:
+            past_key_values = DynamicCache(config=self.config)
+
+        if inputs_embeds is None:
+            inputs_embeds = self.embed_tokens(input_ids)
+
+        if cache_position is None:
+            past_seen_tokens = past_key_values.get_seq_length() if past_key_values is not None else 0
+            cache_position = torch.arange(
+                past_seen_tokens, past_seen_tokens + inputs_embeds.shape[1], device=inputs_embeds.device
+            )
+        if position_ids is None:
+            position_ids = cache_position.unsqueeze(0)
+
+        mask_function = create_causal_mask if self.config.sliding_window is None else create_sliding_window_causal_mask
+        causal_mask = mask_function(
+            config=self.config,
+            input_embeds=inputs_embeds,
+            attention_mask=attention_mask,
+            cache_position=cache_position,
+            past_key_values=past_key_values,
+            position_ids=position_ids,
+        )
+
+        hidden_states = inputs_embeds
+
+        # create position embeddings to be shared across the decoder layers
+        position_embeddings = self.rotary_emb(hidden_states, position_ids)
+
+        for decoder_layer in self.layers[: self.config.num_hidden_layers]:
+            hidden_states = decoder_layer(
+                hidden_states,
+                position_embeddings=position_embeddings,
+                attention_mask=causal_mask,
+                position_ids=position_ids,
+                past_key_values=past_key_values,
+                use_cache=use_cache,
+                cache_position=cache_position,
+                **kwargs,
+            )
+
+        hidden_states = self.norm(hidden_states)
+
+        return MoeModelOutputWithPast(  # only diff with Mistral is the output type, we need MoE
+            last_hidden_state=hidden_states,
+            past_key_values=past_key_values,
+        )
+
+
+def load_balancing_loss_func(
+    gate_logits: Union[torch.Tensor, tuple[torch.Tensor], None],
+    num_experts: Optional[int] = None,
+    top_k=2,
+    attention_mask: Optional[torch.Tensor] = None,
+) -> Union[torch.Tensor, int]:
+    r"""
+    Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch.
+
+    See Switch Transformer (https://huggingface.co/papers/2101.03961) for more details. This function implements the loss
+    function presented in equations (4) - (6) of the paper. It aims at penalizing cases where the routing between
+    experts is too unbalanced.
+
+    Args:
+        gate_logits:
+            Logits from the `gate`, should be a tuple of model.config.num_hidden_layers tensors of
+            shape [batch_size X sequence_length, num_experts].
+        num_experts:
+            Number of experts
+        top_k:
+            The number of experts to route per-token, can be also interpreted as the `top-k` routing
+            parameter.
+        attention_mask (`torch.Tensor`, *optional*):
+            The attention_mask used in forward function
+            shape [batch_size X sequence_length] if not None.
+
+    Returns:
+        The auxiliary loss.
+    """
+    if gate_logits is None or not isinstance(gate_logits, tuple):
+        return 0
+
+    if isinstance(gate_logits, tuple):
+        compute_device = gate_logits[0].device
+        concatenated_gate_logits = torch.cat([layer_gate.to(compute_device) for layer_gate in gate_logits], dim=0)
+
+    routing_weights = torch.nn.functional.softmax(concatenated_gate_logits, dim=-1)
+
+    _, selected_experts = torch.topk(routing_weights, top_k, dim=-1)
+
+    expert_mask = torch.nn.functional.one_hot(selected_experts, num_experts)
+
+    if attention_mask is None:
+        # Compute the percentage of tokens routed to each experts
+        tokens_per_expert = torch.mean(expert_mask.float(), dim=0)
+
+        # Compute the average probability of routing to these experts
+        router_prob_per_expert = torch.mean(routing_weights, dim=0)
+    else:
+        batch_size, sequence_length = attention_mask.shape
+        num_hidden_layers = concatenated_gate_logits.shape[0] // (batch_size * sequence_length)
+
+        # Compute the mask that masks all padding tokens as 0 with the same shape of expert_mask
+        expert_attention_mask = (
+            attention_mask[None, :, :, None, None]
+            .expand((num_hidden_layers, batch_size, sequence_length, top_k, num_experts))
+            .reshape(-1, top_k, num_experts)
+            .to(compute_device)
+        )
+
+        # Compute the percentage of tokens routed to each experts
+        tokens_per_expert = torch.sum(expert_mask.float() * expert_attention_mask, dim=0) / torch.sum(
+            expert_attention_mask, dim=0
+        )
+
+        # Compute the mask that masks all padding tokens as 0 with the same shape of tokens_per_expert
+        router_per_expert_attention_mask = (
+            attention_mask[None, :, :, None]
+            .expand((num_hidden_layers, batch_size, sequence_length, num_experts))
+            .reshape(-1, num_experts)
+            .to(compute_device)
+        )
+
+        # Compute the average probability of routing to these experts
+        router_prob_per_expert = torch.sum(routing_weights * router_per_expert_attention_mask, dim=0) / torch.sum(
+            router_per_expert_attention_mask, dim=0
+        )
+
+    overall_loss = torch.sum(tokens_per_expert * router_prob_per_expert.unsqueeze(0))
+    return overall_loss * num_experts
+
+
+@auto_docstring
+class MiniMaxM2ForCausalLM(MiniMaxM2PreTrainedModel, GenerationMixin):
+    _tied_weights_keys = ["lm_head.weight"]
+    _tp_plan = {"lm_head": "colwise_rep"}
+    _pp_plan = {"lm_head": (["hidden_states"], ["logits"])}
+
+    def __init__(self, config):
+        super().__init__(config)
+        self.model = MiniMaxM2Model(config)
+        self.vocab_size = config.vocab_size
+        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+        self.router_aux_loss_coef = config.router_aux_loss_coef
+        self.num_experts = config.num_local_experts
+        self.num_experts_per_tok = config.num_experts_per_tok
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @can_return_tuple
+    @auto_docstring
+    def forward(
+        self,
+        input_ids: Optional[torch.LongTensor] = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[Cache] = None,
+        inputs_embeds: Optional[torch.FloatTensor] = None,
+        labels: Optional[torch.LongTensor] = None,
+        use_cache: Optional[bool] = None,
+        output_router_logits: Optional[bool] = None,
+        cache_position: Optional[torch.LongTensor] = None,
+        logits_to_keep: Union[int, torch.Tensor] = 0,
+        **kwargs: Unpack[TransformersKwargs],
+    ) -> MoeCausalLMOutputWithPast:
+        r"""
+        labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
+            Labels for computing the masked language modeling loss. Indices should either be in `[0, ...,
+            config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored
+            (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`.
+
+        Example:
+
+        ```python
+        >>> from transformers import AutoTokenizer, MiniMaxM2ForCausalLM
+
+        >>> model = MiniMaxM2ForCausalLM.from_pretrained("mistralai/MiniMaxM2-8x7B-v0.1")
+        >>> tokenizer = AutoTokenizer.from_pretrained("mistralai/MiniMaxM2-8x7B-v0.1")
+
+        >>> prompt = "Hey, are you conscious? Can you talk to me?"
+        >>> inputs = tokenizer(prompt, return_tensors="pt")
+
+        >>> # Generate
+        >>> generate_ids = model.generate(inputs.input_ids, max_length=30)
+        >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
+        "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you."
+        ```"""
+
+        output_router_logits = (
+            output_router_logits if output_router_logits is not None else self.config.output_router_logits
+        )
+
+        # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn)
+        outputs: MoeModelOutputWithPast = self.model(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            inputs_embeds=inputs_embeds,
+            use_cache=use_cache,
+            output_router_logits=output_router_logits,
+            cache_position=cache_position,
+            **kwargs,
+        )
+
+        hidden_states = outputs.last_hidden_state
+        # Only compute necessary logits, and do not upcast them to float if we are not computing the loss
+        slice_indices = slice(-logits_to_keep, None) if isinstance(logits_to_keep, int) else logits_to_keep
+        logits = self.lm_head(hidden_states[:, slice_indices, :])
+
+        loss = None
+        if labels is not None:
+            loss = self.loss_function(logits, labels, self.vocab_size, **kwargs)
+
+        aux_loss = None
+        if output_router_logits:
+            aux_loss = load_balancing_loss_func(
+                outputs.router_logits,
+                self.num_experts,
+                self.num_experts_per_tok,
+                attention_mask,
+            )
+            if labels is not None:
+                loss += self.router_aux_loss_coef * aux_loss.to(loss.device)  # make sure to reside in the same device
+
+        return MoeCausalLMOutputWithPast(
+            loss=loss,
+            aux_loss=aux_loss,
+            logits=logits,
+            past_key_values=outputs.past_key_values,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+            router_logits=outputs.router_logits,
+        )
+
+
+class MiniMaxM2ForSequenceClassification(GenericForSequenceClassification, MiniMaxM2PreTrainedModel):
+    pass
+
+
+class MiniMaxM2ForTokenClassification(GenericForTokenClassification, MiniMaxM2PreTrainedModel):
+    pass
+
+
+class MiniMaxM2ForQuestionAnswering(GenericForQuestionAnswering, MiniMaxM2PreTrainedModel):
+    pass
+
+
+__all__ = [
+    "MiniMaxM2ForCausalLM",
+    "MiniMaxM2ForQuestionAnswering",
+    "MiniMaxM2Model",
+    "MiniMaxM2PreTrainedModel",
+    "MiniMaxM2ForSequenceClassification",
+    "MiniMaxM2ForTokenClassification",
+]

special_tokens_map.json

@@ -0,0 +1,76 @@
+{
+  "additional_special_tokens": [
+    "<code_interpreter>",
+    "<commit_after>",
+    "<commit_before>",
+    "<commit_msg>",
+    "<empty_output>",
+    "<filename>",
+    "<fim_middle>",
+    "<fim_pad>",
+    "<fim_prefix>",
+    "<fim_suffix>",
+    "<function_call>",
+    "<gh_stars>",
+    "]<]speech[>[",
+    "]<]image[>[",
+    "]<]video[>[",
+    "]<]start of speech[>[",
+    "]<]end of speech[>[",
+    "]<]start of image[>[",
+    "]<]end of image[>[",
+    "]<]start of video[>[",
+    "]<]end of video[>[",
+    "]<]vision pad[>[",
+    "]~!b[",
+    "<issue_closed>",
+    "<issue_comment>",
+    "<issue_start>",
+    "<jupyter_code>",
+    "<jupyter_output>",
+    "<jupyter_start>",
+    "<jupyter_text>",
+    "<reponame>",
+    "[e~[",
+    "]!d~[",
+    "]!p~[",
+    "]~b]",
+    "<jupyter_error>",
+    "<add_file>",
+    "<delete_file>",
+    "<rename_file>",
+    "<edit_file>",
+    "<commit_message>",
+    "<empty_source_file>",
+    "<repo_struct>",
+    "<code_context>",
+    "<file_content>",
+    "<source_files>",
+    "<pr_start>",
+    "<review_comment>",
+    "<filepath>",
+    "<file_sep>"
+  ],
+  "bos_token": {
+    "content": "]~!b[",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[e~[",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "[e~[",
+  "unk_token": {
+    "content": "]!d~[",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,495 @@
+{
+  "added_tokens_decoder": {
+  "200000": {
+      "content": "]!p~[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200001": {
+      "content": "<fim_prefix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200002": {
+      "content": "<fim_middle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200003": {
+      "content": "<fim_suffix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200004": {
+      "content": "<fim_pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200005": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200006": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200007": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200008": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200009": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200010": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200011": {
+    "content": "<jupyter_start>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false,
+    "special": true
+  },
+  "200012": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200013": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200014": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200015": {
+    "content": "<empty_output>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false,
+    "special": true
+  },
+  "200016": {
+      "content": "<commit_before>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200017": {
+      "content": "<commit_msg>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200018": {
+      "content": "<commit_after>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200019": {
+      "content": "]~b]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200020": {
+      "content": "[e~[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200021": {
+      "content": "]!d~[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200022": {
+      "content": "<function_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200023": {
+      "content": "<code_interpreter>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200024": {
+      "content": "]<]speech[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200025": {
+      "content": "]<]image[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200026": {
+      "content": "]<]video[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200027": {
+      "content": "]<]start of speech[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200028": {
+      "content": "]<]end of speech[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200029": {
+      "content": "]<]start of image[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200030": {
+      "content": "]<]end of image[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200031": {
+      "content": "]<]start of video[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200032": {
+      "content": "]<]end of video[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200033": {
+      "content": "]<]vision pad[>[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200034": {
+      "content": "]~!b[",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200035": {
+      "content": "<jupyter_error>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200036": {
+      "content": "<add_file>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+  },
+  "200037": {
+      "content": "<delete_file>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200038": {
+      "content": "<rename_file>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200039": {
+      "content": "<edit_file>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200040": {
+      "content": "<commit_message>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200041": {
+      "content": "<empty_source_file>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200042": {
+      "content": "<repo_struct>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+  },
+  "200043": {
+    "content": "<code_context>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": true
+  },
+  "200044": {
+    "content": "<file_content>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": true
+  },
+  "200045": {
+    "content": "<source_files>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": true
+  },
+  "200046": {
+    "content": "<pr_start>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": true
+  },
+  "200047": {
+    "content": "<review_comment>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": true
+  },
+  "200048": {
+    "content": "<filepath>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": true
+  },
+  "200049": {
+    "content": "<file_sep>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": true
+  },
+  "200050": {
+    "content": "<think>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": false
+  },
+  "200051": {
+    "content": "</think>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": false
+  },
+  "200052": {
+    "content": "<minimax:tool_call>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": false
+  },  
+  "200053": {
+    "content": "</minimax:tool_call>",
+    "single_word": false,
+    "lstrip": false,
+    "rstrip": false,
+    "normalized": false,
+    "special": false
+  }
+  },
+  "additional_special_tokens": [
+        "<code_interpreter>",
+        "<commit_after>",
+        "<commit_before>",
+        "<commit_msg>",
+        "<empty_output>",
+        "<filename>",
+        "<fim_middle>",
+        "<fim_pad>",
+        "<fim_prefix>",
+        "<fim_suffix>",
+        "<function_call>",
+        "<gh_stars>",
+        "]<]speech[>[",
+        "]<]image[>[",
+        "]<]video[>[",
+        "]<]start of speech[>[",
+        "]<]end of speech[>[",
+        "]<]start of image[>[",
+        "]<]end of image[>[",
+        "]<]start of video[>[",
+        "]<]end of video[>[",
+        "]<]vision pad[>[",
+        "]~!b[",
+        "<issue_closed>",
+        "<issue_comment>",
+        "<issue_start>",
+        "<jupyter_code>",
+        "<jupyter_output>",
+        "<jupyter_start>",
+        "<jupyter_text>",
+        "<reponame>",
+        "[e~[",
+        "]!d~[",
+        "]!p~[",
+        "]~b]",
+        "<jupyter_error>",
+        "<add_file>",
+        "<delete_file>",
+        "<rename_file>",
+        "<edit_file>",
+        "<commit_message>",
+        "<empty_source_file>",
+        "<repo_struct>",
+        "<code_context>",
+        "<file_content>",
+        "<source_files>",
+        "<pr_start>",
+        "<review_comment>",
+        "<filepath>",
+        "<file_sep>"
+    ],
+  "add_prefix_space": false,
+  "bos_token": "]~!b[",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "[e~[",
+  "model_max_length": 40960000,
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "]!d~["
+}