Smol-reason
Collection
My first ever usage of GRPO fine tuning techniques, information learned from this model will be used on future Andy models. • 6 items • Updated
A newer version of this model is available: Sweaterdog/Smol-reason2.1
🧠 Smol-reason, a 3B model test for future models 🧠
Why?
When making the Andy series of models, I have been using PPO techniques to train models.
But as the bleeding edge of small models is becoming clear, reasoning models are the winners.
So, in order to learn the nuances of training models, I decided to train a small 3B model using GRPO techniques instead of PPO.
What do I do with the LoRA?
This LoRA was made so others can train off of this GPRO model, and can be used for any purpose.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support