Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models
Paper • 2609.13680 • Published • 13
MaLA-LM
community
AI & ML interests
NLP & LLM
Recent Activity
View all activity
yileitu
authored a
paper 11 days ago
Zihao-Li
authored a
paper 11 days ago
yileitu
authored a
paper 11 days ago
Post
29
Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models
Standard SFT usually ask:
How can we SFT an instruct model without behavioral drift?
We reverse the question:
Given a fixed drift budget, how should we SFT?
This predicts that changing *where* the model can move—not the supervision—can reverse fine-tuning failure.
On Qwen3-8B and Qwen3-14B, it does.
code: https://github.com/CONE-MT/DCO/tree/main/dco
models: https://huggingface.co/collections/LLaMAX/dco
Standard SFT usually ask:
How can we SFT an instruct model without behavioral drift?
We reverse the question:
Given a fixed drift budget, how should we SFT?
This predicts that changing *where* the model can move—not the supervision—can reverse fine-tuning failure.
On Qwen3-8B and Qwen3-14B, it does.
code: https://github.com/CONE-MT/DCO/tree/main/dco
models: https://huggingface.co/collections/LLaMAX/dco
FeYuan
submitted 2
papers to Daily Papers 11 days ago
FeYuan
updated a
model 14 days ago
FeYuan
published a
model 14 days ago
pinzhenchen
authored a
paper 22 days ago
FeYuan
updated a
model 28 days ago
FeYuan
published a
model 28 days ago
jisx
submitted a
paper to Daily Papers about 1 month ago
FeYuan
updated 7
models about 1 month ago
MaLA-LM/FT-FineOPUS-Filtered-Stage5-sample5k
8B • Updated • 7
MaLA-LM/FT-FineOPUS-Filtered-Stage4-sample5k
8B • Updated • 6
MaLA-LM/FT-FineOPUS-Filtered-Stage3-sample5k
8B • Updated • 8
MaLA-LM/FT-FineOPUS-Filtered-Stage2-sample5k
8B • Updated • 7
MaLA-LM/FT-FineOPUS-Filtered-Stage1-sample5k
8B • Updated • 10
MaLA-LM/FT-FineOPUS-ReLID-sample5k
8B • Updated • 13
MaLA-LM/FT-FineOPUS-Deduplicated-sample5k
8B • Updated • 10 • 1
FeYuan
published a
model about 1 month ago