alexgusevski/CapybaraHermes-2.5-Mistral-7B-mlx-3Bit

The Model alexgusevski/CapybaraHermes-2.5-Mistral-7B-mlx-3Bit was converted to MLX format from argilla/CapybaraHermes-2.5-Mistral-7B using mlx-lm version 0.29.1.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("alexgusevski/CapybaraHermes-2.5-Mistral-7B-mlx-3Bit")

prompt="hello"

if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)

Downloads last month: 15

Safetensors

Model size

0.9B params

Tensor type

F16

U32

MLX

Hardware compatibility

3-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for alexgusevski/CapybaraHermes-2.5-Mistral-7B-mlx-3Bit

Base model

mistralai/Mistral-7B-v0.1

Finetuned

teknium/OpenHermes-2.5-Mistral-7B

Finetuned

argilla/CapybaraHermes-2.5-Mistral-7B

Quantized

(13)

this model

Dataset used to train alexgusevski/CapybaraHermes-2.5-Mistral-7B-mlx-3Bit

Evaluation results

normalized accuracy on AI2 Reasoning Challenge (25-Shot)
test set Open LLM Leaderboard

65.780
normalized accuracy on HellaSwag (10-Shot)
validation set Open LLM Leaderboard

85.450
accuracy on MMLU (5-Shot)
test set Open LLM Leaderboard

63.130
mc2 on TruthfulQA (0-shot)
validation set Open LLM Leaderboard

56.910
accuracy on Winogrande (5-shot)
validation set Open LLM Leaderboard

78.300
accuracy on GSM8k (5-shot)
test set Open LLM Leaderboard

59.290