Radiantloom
/

radintloom-mistral-7b-fusion

Text Generation

Mistral 7B Instruct

text-generation-inference

Model card Files Files and versions

aigeek0x0 commited on Feb 20, 2024

Commit

d56a4fd

·

verified ·

1 Parent(s): 3d4eb13

Update README.md

Files changed (1) hide show

README.md +1 -2

README.md CHANGED Viewed

@@ -58,8 +58,7 @@ bnb_config = BitsAndBytesConfig(
 # load the model and tokenizer
 model = AutoModelForCausalLM.from_pretrained(
-    model_name,
-    load_in_4bit=True,
     quantization_config=bnb_config,
     device_map="auto",
 )

 # load the model and tokenizer
 model = AutoModelForCausalLM.from_pretrained(
+    model_name,
     quantization_config=bnb_config,
     device_map="auto",
 )