mylesgoose
/

Llama-3.1-Minitron-4B-Llava-Nvidia-siglip-ov

Model card Files Files and versions

mylesgoose commited on Sep 26, 2024

Commit

72021da

·

verified ·

1 Parent(s): d90abc7

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -189,7 +189,7 @@
   "model_type": "llava_onevision",
   "num_attention_heads": 32,
   "num_hidden_layers": 32,
-  "num_key_value_heads": 4,
   "pad_token_id": 128004,
   "pos_skipping_range": 4096,
   "pretraining_tp": 1,

   "model_type": "llava_onevision",
   "num_attention_heads": 32,
   "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
   "pad_token_id": 128004,
   "pos_skipping_range": 4096,
   "pretraining_tp": 1,