Constant 503 error for several days when running LLAMA 3.1

Inference Endpoints on the Hub
1

Hi all. Earlier this month I wrote a script incorporating LLAMA 3.1, and it worked well (with timeouts, etc), but for the past week whenever I try to access LLAMA 3.1 at all I get a 503 error (including for short simple requests, like HF’s sample code).

Any idea what the problem may be? I do indeed have Pro.

1 Like
Inference API returns 504 error for Llama-3.2-3B-Instruct & google/gemma-2-2b-it
2

Did you find a solution to this problem? I am experiencing the same problem on and off for a couple of days now.

1 Like
3

There were at least three reports of the same situation on Discord, so I think it’s probably some kind of error. @michellehbn

4

Am still getting a 503 when running Llama-3.2-3B-Instruct. Any known solution?
huggingface_hub.errors.HfHubHTTPError: 503 Server Error: Service Temporarily Unavailable for url: /static-proxy?url=https%3A%2F%2Frouter.huggingface.co%2Fhf-inference%2Fmodels%2Fmeta-llama%2FLlama-3.2-3B-Instruct%3C%2Fcode%3E%3C%2Fp%3E

1 Like
5

Depending on the model, it seems that this can be avoided by the following methods.

1 Like
6

Thanks. Using providers works like magic. Here is the documentation: HF Inference
Sample

client = InferenceClient(
    provider="hf-inference", 
    token=os.environ["HF_TOKEN"])
2 Likes