Instructions to use openbmb/MiniCPM-V-4_5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openbmb/MiniCPM-V-4_5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="openbmb/MiniCPM-V-4_5", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("openbmb/MiniCPM-V-4_5", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use openbmb/MiniCPM-V-4_5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openbmb/MiniCPM-V-4_5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-V-4_5",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/openbmb/MiniCPM-V-4_5

SGLang

How to use openbmb/MiniCPM-V-4_5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openbmb/MiniCPM-V-4_5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-V-4_5",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openbmb/MiniCPM-V-4_5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-V-4_5",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use openbmb/MiniCPM-V-4_5 with Docker Model Runner:
```
docker model run hf.co/openbmb/MiniCPM-V-4_5
```

Multiple claims that this model is "... not compatible ..."

#14

by dakerholdings - opened Sep 3, 2025

Discussion

dakerholdings

Sep 3, 2025

I was initially hoping to convert this to mlx, but apparently it took days to download it (the download would slow to a crawl after some gigs had been downloaded & I'd have to ctrl-c & restart it again the next night) before refusing to convert it!?

ds@Ds-MacBook-Air ~ % mlx_lm.convert --hf-path openbmb/MiniCPM-V-4_5 -q --quant-predicate [... omitted ...]
[INFO] Loading ... Fetching 21 files: ... 100%| ERROR:root:Model type minicpmv not supported.

So, after reading your 'model card' which claims that llama.cpp supports it, I tried that:

ds@Ds-MacBook-Air bin % ./llama-cli -hf openbmb/MiniCPM-V-4_5
error from HF API, response code: 400, data: {"error":"Repository is not GGUF or is not compatible with llama.cpp"}
ds@Ds-MacBook-Air bin % ./llama-cli --version
version: 6318 (81017865)
built with Apple clang version 15.0.0 (clang-1500.3.9.4) for arm64-apple-darwin23.6.0

[ I also tried ollama: ]
ds@Ds-MacBook-Air ~ % ollama run hf.co/openbmb/MiniCPM-V-4_5
pulling manifest
Error: pull model manifest: 400: {"error":"Repository is not GGUF or is not compatible with llama.cpp"}

I'm wondering if I'm now required to convert to gguf, or? I hadn't downloaded that one 'cause mlx_lm didn't appear to support!?

tc-mb

OpenBMB org Sep 4, 2025

https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/llama.cpp/minicpm-v4_5_llamacpp.md
https://github.com/OpenSQZ/MiniCPM-V-CookBook/blob/main/deployment/ollama/minicpm-v4_5_ollama.md

I received your issue, but after seeing the usage commands you sent, I have to say that these usages are not correct.
I sent you llama.cpp and ollama's usage documentation, you can refer to them, I hope this can help you.

dakerholdings

Sep 4, 2025

Both of these 'usage documentation' examples appear to refer to 'gguf' format files, which you apparently have available as a different model download (though the 'model card' for both claims to support ollama+).

tc-mb

OpenBMB org Sep 4, 2025

Refer to the documentation for llama.cpp.
You can convert the gguf model yourself or download it directly from our converted repository.
https://huggingface.co/openbmb/MiniCPM-V-4_5-gguf

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment