Olmo-3.1-32B-Instruct-exl3 / README.md

Update README.md

110fbf3 verified about 2 months ago

5.68 kB

license: apache-2.0
base_model: allenai/Olmo-3.1-32B-Instruct
base_model_relation: quantized
quantized_by: UnstableLlama
tags:
  - exl3

>> allenai/Olmo-3.1-32B-Instruct

[QUANTS BY :: UNSTABLELLAMA]

// REPO

EXL3 quantizations of allenai's Olmo-3.1-32B-Instruct.

Quantized with exllamav3 0.0.18 ⚠️ dev branch required!

Uploads in progress...

[BRANCH]	[GiB]	[K/L_DIV]	[PPL]
2.00bpw	8.6	0.56968149	9.7437922
3.00bpw	12.23	0.19111139	8.65133396
4.00bpw	15.86	0.06036107	8.49095044
5.00bpw	19.5	0.02379244	8.50298723
6.00bpw	23.13	0.01238659	8.47749867
bf16	60.04	0	8.4801074

Use HF-CLI to pull specific branches to your local machine:

huggingface-cli download UnstableLlama/Olmo-3.1-32B-Instruct-exl3 --revision "3.00bpw" --local-dir ./