Instructions to use OpenNMT/codet5p-770m-ct2-int8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenNMT/codet5p-770m-ct2-int8 with Transformers:
# Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("OpenNMT/codet5p-770m-ct2-int8") model = AutoModelForSeq2SeqLM.from_pretrained("OpenNMT/codet5p-770m-ct2-int8") - Notebooks
- Google Colab
- Kaggle
Commit ·
eea668b
1
Parent(s): a19a70f
Upload Salesforce/codet5p-770m ctranslate2 weights
Browse files
README.md
CHANGED
|
@@ -14,26 +14,6 @@ quantized version of [Salesforce/codet5p-770m](https://huggingface.co/Salesforce
|
|
| 14 |
pip install ctranslate2
|
| 15 |
```
|
| 16 |
|
| 17 |
-
```python
|
| 18 |
-
# from transformers import AutoTokenizer
|
| 19 |
-
model_name = "OpenNMT/codet5p-770m-ct2-int8"
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
from hf_hub_ctranslate2 import TranslatorCT2fromHfHub
|
| 23 |
-
model = TranslatorCT2fromHfHub(
|
| 24 |
-
# load in int8 on CUDA
|
| 25 |
-
model_name_or_path=model_name,
|
| 26 |
-
device="cuda",
|
| 27 |
-
compute_type="int8_float16",
|
| 28 |
-
# tokenizer=AutoTokenizer.from_pretrained("{ORG}/{NAME}")
|
| 29 |
-
)
|
| 30 |
-
outputs = model.generate(
|
| 31 |
-
text=["def fibonnaci(", "User: How are you doing? Bot:"],
|
| 32 |
-
max_length=64,
|
| 33 |
-
)
|
| 34 |
-
print(outputs)
|
| 35 |
-
```
|
| 36 |
-
|
| 37 |
Checkpoint compatible to [ctranslate2>=3.22.0](https://github.com/OpenNMT/CTranslate2)
|
| 38 |
- `compute_type=int8_float16` for `device="cuda"`
|
| 39 |
- `compute_type=int8` for `device="cpu"`
|
|
|
|
| 14 |
pip install ctranslate2
|
| 15 |
```
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
Checkpoint compatible to [ctranslate2>=3.22.0](https://github.com/OpenNMT/CTranslate2)
|
| 18 |
- `compute_type=int8_float16` for `device="cuda"`
|
| 19 |
- `compute_type=int8` for `device="cpu"`
|