aripos1/gorani-lora-3b

🦙 Gorani LoRA 3B (Llama 3.2-3B 기반)

🔹 Model Details

• Base Model: unsloth/Llama-3.2-3B-Instruct-bnb-4bit
• LoRA Adapter: QLoRA 적용 (bnb-4bit)
• Fine-tuned on: Custom parallel corpus (Korean-English)
• Quantization: 4-bit (bnb-4bit)
• Language: Korean & English
• Training Method: Supervised Fine-tuning (SFT) + LoRA
• Optimizer: AdamW (betas=(0.9, 0.95), weight_decay=0.01)

🔹 Hyperparameters

Hyperparameter	Value
Learning Rate	2e-4
Batch Size	16
Epochs	3
Warmup Steps	500
Gradient Accumulation	4

🔹 Evaluation Results

모델 평가를 위해 Comet Score 및 BERT Score를 사용했음.

Model Version	Comet Score ↑	BERT Score ↑
gorani-lora-v1	0.78	0.85
gorani-lora-v2	0.82	0.88
gorani-lora-v3	0.85	0.90

🔹 How to Use

from transformers import AutoModel, AutoTokenizer
from peft import PeftModel

base_model = AutoModel.from_pretrained("unsloth/Llama-3.2-3B-Instruct-bnb-4bit")
adapter_model = PeftModel.from_pretrained(base_model, "aripos1/gorani-lora-3b")

tokenizer = AutoTokenizer.from_pretrained("unsloth/Llama-3.2-3B-Instruct-bnb-4bit")

text = "안녕하세요, 오늘의 날씨는?"
inputs = tokenizer(text, return_tensors="pt")
outputs = adapter_model.generate(**inputs)
print(tokenizer.decode(outputs[0]))