Upload 10 files

.gitattributes

@@ -33,3 +33,12 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-fp16-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-fp16-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-iq4_nl.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-typhoon2-3b-instruct-q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,42 @@
+---
+license: llama3.2
+pipeline_tag: text-generation
+base_model: scb10x/llama3.2-typhoon2-3b-instruct
+tags:
+- llama-cpp
+- gguf-my-repo
+---
+# Float16-cloud/llama3.2-typhoon2-3b-instruct-gguf
+This model was converted to GGUF format from [`scb10x/llama3.2-typhoon2-3b-instruct`](https://huggingface.co/scb10x/llama3.2-typhoon2-3b-instruct) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
+Refer to the [original model card](https://huggingface.co/scb10x/llama3.2-typhoon2-3b-instruct) for more details on the model.
+## Use with llama.cpp
+Install llama.cpp through brew (works on Mac and Linux)
+```bash
+brew install llama.cpp
+```
+Invoke the llama.cpp server or the CLI.
+### CLI:
+```bash
+llama-cli --hf-repo Float16-cloud/llama3.2-typhoon2-3b-instruct-gguf --hf-file llama3.2-typhoon2-3b-instruct-iq4_nl.gguf -p "The meaning to life and the universe is"
+```
+### Server:
+```bash
+llama-server --hf-repo Float16-cloud/llama3.2-typhoon2-3b-instruct-gguf --hf-file llama3.2-typhoon2-3b-instruct-iq4_nl.gguf -c 2048
+```
+Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
+Step 1: Clone llama.cpp from GitHub.
+```
+git clone https://github.com/ggerganov/llama.cpp
+```
+Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
+```
+cd llama.cpp && LLAMA_CURL=1 make
+```
+Step 3: Run inference through the main binary.
+```
+./llama-cli --hf-repo Float16-cloud/llama3.2-typhoon2-3b-instruct-gguf --hf-file llama3.2-typhoon2-3b-instruct-iq4_nl.gguf -p "The meaning to life and the universe is"
+```
+or 
+```
+./llama-server --hf-repo Float16-cloud/llama3.2-typhoon2-3b-instruct-gguf --hf-file llama3.2-typhoon2-3b-instruct-iq4_nl.gguf -c 2048
+```

llama3.2-typhoon2-3b-instruct-fp16-00001-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c07f737fbace5047720361977224cf928d8f093052d1a26cd72dae2e70a62cf
+size 3967109376

llama3.2-typhoon2-3b-instruct-fp16-00002-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ba65449c155137324941c18d9465cf8479cb5aa05cc10b3240d26f49678fe97
+size 2466576928

llama3.2-typhoon2-3b-instruct-iq4_nl.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5657d33d8e7816bcbf158228d7928be9bc154cad7608ba1a61d3eb078b47fca8
+size 1926625888

llama3.2-typhoon2-3b-instruct-q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:179350eee25024620731f5b8d407d8ec664f6f36fb073444dc72e2a231c8a204
+size 1917188704

llama3.2-typhoon2-3b-instruct-q4_k_m.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54b4e575db46f11533e8e8bea0873c56c6bfd186be1ed6b682a0912162a79d89
+size 2019375712

llama3.2-typhoon2-3b-instruct-q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce6f3899960e5b30fa070653e190c35accbde1321bb69d163dad8c342b217ffe
+size 2269510240

llama3.2-typhoon2-3b-instruct-q5_k_m.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41a2a5897fbd964c864cec0721005495205a03121a647d738dba38c3b86e8acc
+size 2322152032

llama3.2-typhoon2-3b-instruct-q6_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a52491275b94c3beaa2493b2b6494d90ba9cca6c4896facfd24a1a3c2df9e6dc
+size 2643851872

llama3.2-typhoon2-3b-instruct-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7f4327c69870337aa6e66e6a073832772c327648bb9c3f63cc39ab489f0339a
+size 3421897312