coder-lfm3-1.7b-qwen4-max

LFM3-1.7B fine-tuned on the Coder profile of the Qwen4 Max reasoning corpus: 70 code/CS/stats prompts answered by qwen/qwen4-max (OpenRouter, canonical qwen/qwen4-max-20261115).

The teacher's traces mix reasoning and fenced implementations. This adapter moves LFM3 toward that shape for code and CS prompts.

Training

base LiquidAI/LFM3-1.7B
data AJ-Gonzalez/qwen4-max-reasoning-250x, coder profile; 70 rows → 340 train / 45 valid chunks
chunking 2048 tokens, 256-token replay, one example per chunk, no prompt masking
LoRA rank 16, alpha 32, all attention/MLP/short-conv linears, 16.47M trainable (0.9436%)
precision 4-bit NF4 QLoRA, fp16 compute (T4 has no bf16)
batch / seq per-device 1, grad-accum 8, seq 2048; effective batch 16 on Kaggle's 2-GPU accounting
optimizer paged AdamW 8-bit, lr 1e-4, 10 warmup steps
schedule 3 epochs, 66 optimizer steps, 50.2 min on one T4
losses train 3.9531; eval 4.056 → 3.861 → 3.827

The run completed with zero failures. The adapter was merged into the bf16 base, then converted to MLX 4-bit (4.503 bpw) and GGUF Q4_K_M (5.05 bpw).

Benchmark

HumanEval+ through the same T4/fp16 harness used for the earlier LFM3 evaluations: one user message, enable_thinking=False, greedy, 768 new tokens, evalplus sanitize + evaluate. Both models go through the identical loop.

model HumanEval+ base pass@1 HumanEval+ plus pass@1
LiquidAI/LFM3-1.7B stock 0.695 0.652
coder-lfm3-1.7b-qwen4-max 0.732 0.683

Deltas: +3.7 points base, +3.1 points plus (164 tasks, evalplus 0.3.1). Raw results: bench/bench_results.json.

SWE-bench prompt NLL (20 questions, 512-token chunks, same tokenizer and same T4/fp16 harness):

model NLL (nats/token) ppl
stock, T4 fp16 2.5127 12.34
this model, T4 fp16 2.2310 9.31

On the final MLX 4-bit artifacts the same 20-question NLL is 2.6936 (stock) → 2.3156 (this model), so the 4-bit conversion retained the fine-tune delta (the adapter alone measured 2.3138 on the T4).

Files

path contents
model.safetensors + root files MLX 4-bit, 946 MB, 4.503 bpw
bf16/ merged HF bf16 checkpoint, 3.3 GB
adapter/ the PEFT LoRA adapter, 63 MB
Q4_K_M.gguf GGUF Q4_K_M, 1.1 GB, 5.05 bpw
LICENSE LFM Open License v1.0

SHA-256:

file SHA-256
model.safetensors (MLX 4-bit) e6e1fac09c70d78794e47ab708d98be9ceebac15a8f98488fec4acf089203f4d
bf16/model.safetensors b05b53e7976b12446364c865672df3e2441e6b92c8aa37ee68c924e5d79ba50c
Q4_K_M.gguf b4675e1003ad035e19f70b36c18bf3d4d0e6ff3bbd8e603279a9ab67aa2eea1a

Usage

MLX:

mlx_lm.generate --model AJ-Gonzalez/coder-lfm3-1.7b-qwen4-max \
    --prompt "Write a Python function that returns the first 12 prime numbers." \
    --max-tokens 3000

Transformers (bf16):

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("AJ-Gonzalez/coder-lfm3-1.7b-qwen4-max", subfolder="bf16",
                                    trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "AJ-Gonzalez/coder-lfm3-1.7b-qwen4-max", subfolder="bf16",
    trust_remote_code=True, dtype="float16", device_map="auto")

GGUF:

llama-cli -m Q4_K_M.gguf \
    -p "Write a Python function that returns the first 12 prime numbers." -n 96

The LFM3 chat template is applied from the tokenizer. On an M1 8 GB the MLX 4-bit model runs at ~55 tok/s (64-token smoke; not a benchmark) with ~1.15 GB peak.

How the data was made

  1. openrouter_reasoning_corpus.py regenerated the 250 MiMo prompts with qwen/qwen4-max at max_tokens=32768; 3 rows were retried at 65536. All 250 finished stop.
  2. build_profile_dataset.py coder --source corpus.jsonl rendered the LFM3 chat template and chunked to 2048/256, producing 340 train / 45 valid chunks (642,753 / 86,227 tokens).
  3. QLoRA SFT on Kaggle T4, merge, MLX 4-bit conversion, GGUF Q4_K_M.

The full corpus, per-category costs and generation settings are in the dataset card.

Caveats

  • The benchmark merges the adapter into the fp16 base in-kernel; it does not load the uploaded bf16/ file, and it is not the Q4_K_M GGUF. Quantization can move code quality, especially at 4 bits.
  • The training run used Kaggle's 2-GPU batch accounting; the model was pinned to cuda:0, so the effective global batch was 16 rather than 8.
  • The 4-bit base was quantized NF4 for training; the benchmark reloads the merged bf16 base in fp16, so train-time and eval-time quantization differ.
  • Model outputs are not verified; code may be wrong or unsafe.

Licence

Base model: LFM Open License v1.0 (commercial use licensed below the $10M annual-revenue threshold; see LICENSE). The training traces derive from the MiMo/TeichAI prompt set (license: other) and Qwen4 Max outputs via OpenRouter; the dataset card records those terms. Review both before commercial use. No new licence is asserted over the model-generated traces.

Downloads last month
7,008
Safetensors
Model size
2B params
Tensor type
U32
·
BF16
·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AJ-Gonzalez/coder-lfm3-1.7b-qwen4-max

Adapter
(36)
this model

Dataset used to train AJ-Gonzalez/coder-lfm3-1.7b-qwen4-max