# Is there llama3 api for hugging face to use?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%3E%5C
**Category:** Beginners\
**Created:** [September 5, 2024, 8:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411 "2024-09-05T08:53:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![alice86](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@alice86](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falice86)%5C
**Post date:** [September 5, 2024, 8:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%2F1 "2024-09-05T08:53:03Z")

</div>

I don’t have any gpu on my PC.  
So I want to the api to call. Just like openai、cohere、…  
I want to the llama3 api

---

<div class="post-metadata">

**Author:** ![NatureSon](https://avatars.discourse-cdn.com/v4/letter/n/54ee81/32.png) [@NatureSon](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FNatureSon)%5C
**Post date:** [September 5, 2024, 10:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%2F2 "2024-09-05T10:47:35Z")

</div>

not sure but you can check this. It provides free API inference: [https://console.groq.com/playground](https://console.groq.com/playground)

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [September 5, 2024, 2:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%2F4 "2024-09-05T14:10:51Z")

</div>

> **[HF's Missing Inference Widget - a Hugging Face Space by featherless-ai](https://huggingface.co/spaces/featherless-ai/try-this-model)**
>
> Discover amazing ML apps made by the community

If you’re looking for free, this space might be useful.  
On a $10/month subs, you could also use the 70B Llama3 model for example. (I use it)

---

<div class="post-metadata">

**Author:** ![rmayormartins](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rmayormartins/32/29916_2.png) [@rmayormartins](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frmayormartins)%5C
**Post date:** [September 8, 2024, 12:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%2F5 "2024-09-08T00:18:04Z")

</div>

Like NatureSon said, I use the groq api playground

> **[My-Llama3-Groq - a Hugging Face Space by rmayormartins](https://huggingface.co/spaces/rmayormartins/my-llama3-groq)**
>
> Discover amazing ML apps made by the community

> **[My-Llama3.1-Groq - a Hugging Face Space by rmayormartins](https://huggingface.co/spaces/rmayormartins/my-llama3.1-groq)**
>
> Discover amazing ML apps made by the community

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [September 8, 2024, 1:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%2F6 "2024-09-08T13:15:41Z")

</div>

Hi,

HF provides the [serverless Inference API](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-there-llama3-api-for-hugging-face-to-use%2F105411%2F6) to do just that. It comes with OpenAI-compatible APIs.

Usage is as follows (add your HF token):

```python
# instead of `from openai import OpenAI`
from huggingface_hub import InferenceClient

# instead of `client = OpenAI(...)`
client = InferenceClient(
    "meta-llama/Meta-Llama-3.1-8B-Instruct",
    token=<your-hf-token>,
)

output = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Count to 10"},
    ],
    stream=True,
    max_tokens=1024,
)

for chunk in output:
    print(chunk.choices[0].delta.content)

```

By getting a [PRO subscription](https://huggingface.co/subscribe/pro), you get higher rate limits.
