# Deepseek Inference so slow

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepseek-inference-so-slow%2F162753%3E%5C
**Category:** Beginners\
**Created:** [July 12, 2025, 1:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepseek-inference-so-slow%2F162753 "2025-07-12T13:29:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![serpastorg](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/serpastorg/32/50994_2.png) [@serpastorg](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fserpastorg)%5C
**Post date:** [July 12, 2025, 1:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepseek-inference-so-slow%2F162753%2F1 "2025-07-12T13:29:46Z")

</div>

I’m a pure newbie in this AI topic. I need some help.  
My hardware is Ryzen 5 7600, 16 GB RAM, and NVidia RTX 3050 8GB graphics card, with Linux 6.11.0-1016-lowlatency x86\_64, NVidia’s open driver 570.158.01 and CUDA 12.4.127 in the virtual environment running in Ubuntu 24.04.2 LTS.  
It’s not a really powerful computer, but it’s not that bad to work that slow.  
Deepseek’s inference is completely slow working with transformers interface. I recently installed BitsAndBytes.  
I’d like to know which are the recommended parammeters to start CLI chatbot interface, or IPython’s one. That’s the reason to write this post.  
Please, have into account I’m just a newbie. I’ve not kept in touch with any AI’s model before. My model is open-R1 from HuggingFace.  
Thank you very much.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [July 12, 2025, 1:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepseek-inference-so-slow%2F162753%2F2 "2025-07-12T13:50:09Z")

</div>

With those GPU specs, I think you’d be better off using [Ollama](https://huggingface.co/docs/hub/ollama) or [vLLM](https://github.com/vllm-project/vllm) to easily leverage the performance. In particular, [the GGUF format](https://huggingface.co/posts/ngxson/723185341195650) used with Ollama isn’t particularly fast, but it’s an excellent format for offloading when VRAM is limited.  
Generally speaking, Transformers offer advanced customization options, but if you don’t need that level of customization, other backends are easier to use.

---

<div class="post-metadata">

**Author:** ![serpastorg](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/serpastorg/32/50994_2.png) [@serpastorg](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fserpastorg)%5C
**Post date:** [July 12, 2025, 2:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepseek-inference-so-slow%2F162753%2F3 "2025-07-12T14:18:16Z")

</div>

How can I use this things?  
I have just installed ollama.  
Thank you very much. Excuse me, I’m a newbie.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [July 12, 2025, 2:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepseek-inference-so-slow%2F162753%2F4 "2025-07-12T14:28:49Z")

</div>

Ollama is the quickest option.

Once you download and install Ollama, you should be able to launch it directly from the command line.  
Ollama runs as a local server and can also be used as an API server compatible with the OpenAI API, so if you want to create your own chatbot GUI, you can access it via that API. It also handles load balancing.

Incidentally, you can achieve the same thing with Transformers using [TGI](https://huggingface.co/docs/text-generation-inference/index). It’s also possible with vLLM.

> **[Ollama](https://ollama.com/)**
>
> Get up and running with large language models.

> **[deepseek-r1](https://ollama.com/library/deepseek-r1)**
>
> DeepSeek-R1 is a family of open reasoning models with performance approaching that of leading models, such as O3 and Gemini 2.5 Pro.

> **[OpenAI compatibility · Ollama Blog](https://ollama.com/blog/openai-compatibility)**
>
> Ollama now has initial compatibility with the OpenAI Chat Completions API, making it possible to use existing tooling built for OpenAI with local models via Ollama.
