Commit
·
0e3b644
1
Parent(s):
9f076ce
Add ggml file.
Browse files- README.md +22 -0
- openbuddy-openllama-7b-v5-q4_0.bin +3 -0
README.md
CHANGED
|
@@ -1,3 +1,25 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
|
| 5 |
+
## How to reproduce
|
| 6 |
+
```bash
|
| 7 |
+
# Prerequisites
|
| 8 |
+
apt update -y
|
| 9 |
+
apt install -y git git-lfs python3 python3-pip curl pkg-config libssl-dev
|
| 10 |
+
python3 -m pip install numpy==1.25.0 sentencepiece==0.1.99
|
| 11 |
+
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh && source "$HOME/.cargo/env"
|
| 12 |
+
|
| 13 |
+
# Clone repositories
|
| 14 |
+
git clone https://huggingface.co/OpenBuddy/openbuddy-openllama-7b-v5-fp16 # Commit hash 1fedac68b34952eecec849a5938b778d6004d632
|
| 15 |
+
git clone https://github.com/ggerganov/llama.cpp # Commit hash 16b9cd193965769089881bb8ec012fccca7b37b6
|
| 16 |
+
git clone --recurse-submodules https://github.com/rustformers/llm.git # Commit hash 3becd728c0d6eeb2d649f86158c7018d5aaaba40
|
| 17 |
+
|
| 18 |
+
# Build ggml model
|
| 19 |
+
cd llama.cpp/
|
| 20 |
+
python3 convert.py ../openbuddy-openllama-7b-v5-fp16/
|
| 21 |
+
cd ../llm/
|
| 22 |
+
cargo build --release
|
| 23 |
+
cargo run --release llama quantize ../openbuddy-openllama-7b-v5-fp16/ggml-model-f16.bin ../openbuddy-openllama-7b-v5-fp16/openbuddy-openllama-7b-v5-q4_0.bin q4_0
|
| 24 |
+
```
|
| 25 |
+
(The commit hashes are confirmed at the time of 2023/06/19)
|
openbuddy-openllama-7b-v5-q4_0.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:97f7e76b331478ea78f74813ce77b98c58367b513f8a8bec2c857f3202c5a9f8
|
| 3 |
+
size 3821537184
|