# Is there llama3 api for hugging face to use?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%3E%5C
**Category:** Beginners\
**Created:** [September 5, 2024, 8:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411 "2024-09-05T08:53:03Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [September 8, 2024, 1:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%2F6 "2024-09-08T13:15:41Z")

</div>

Hi,

HF provides the [serverless Inference API](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%2F6) to do just that. It comes with OpenAI-compatible APIs.

Usage is as follows (add your HF token):

```python
# instead of `from openai import OpenAI`
from huggingface_hub import InferenceClient

# instead of `client = OpenAI(...)`
client = InferenceClient(
    "meta-llama/Meta-Llama-3.1-8B-Instruct",
    token=<your-hf-token>,
)

output = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Count to 10"},
    ],
    stream=True,
    max_tokens=1024,
)

for chunk in output:
    print(chunk.choices[0].delta.content)

```

By getting a [PRO subscription](https://huggingface.co/subscribe/pro), you get higher rate limits.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411)._
