ilsp
/

Meltemi-7B-Instruct-v1-AWQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions

LVouk commited on Apr 8, 2024

Commit

155eed6

·

verified ·

1 Parent(s): 4c75f09

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -47,7 +47,7 @@ from transformers import AutoTokenizer
 device = "cuda" # the device to load the model onto
-model = AutoAWQModelForCausalLM.from_quantized(
   "ilsp/Meltemi-7B-Instruct-v1-AWQ",
   fuse_layers=True,
   trust_remote_code=False,

 device = "cuda" # the device to load the model onto
+model = AutoAWQForCausalLM.from_quantized(
   "ilsp/Meltemi-7B-Instruct-v1-AWQ",
   fuse_layers=True,
   trust_remote_code=False,