Instructions to use QwenPilot/FIPO_32B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use QwenPilot/FIPO_32B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="QwenPilot/FIPO_32B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("QwenPilot/FIPO_32B")
model = AutoModelForCausalLM.from_pretrained("QwenPilot/FIPO_32B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use QwenPilot/FIPO_32B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "QwenPilot/FIPO_32B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QwenPilot/FIPO_32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/QwenPilot/FIPO_32B

SGLang

How to use QwenPilot/FIPO_32B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "QwenPilot/FIPO_32B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QwenPilot/FIPO_32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "QwenPilot/FIPO_32B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QwenPilot/FIPO_32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use QwenPilot/FIPO_32B with Docker Model Runner:
```
docker model run hf.co/QwenPilot/FIPO_32B
```

QwenPilot commited on Apr 12

Commit

fe5755b

verified ·

1 Parent(s): 8b3575d

Upload folder using huggingface_hub

Browse files

Files changed (15) hide show

model-00001-of-00014.safetensors +2 -2
model-00002-of-00014.safetensors +2 -2
model-00003-of-00014.safetensors +2 -2
model-00004-of-00014.safetensors +2 -2
model-00005-of-00014.safetensors +2 -2
model-00006-of-00014.safetensors +2 -2
model-00007-of-00014.safetensors +2 -2
model-00008-of-00014.safetensors +2 -2
model-00009-of-00014.safetensors +2 -2
model-00010-of-00014.safetensors +2 -2
model-00011-of-00014.safetensors +2 -2
model-00012-of-00014.safetensors +2 -2
model-00013-of-00014.safetensors +2 -2
model-00014-of-00014.safetensors +2 -2
model.safetensors.index.json +0 -0

model-00001-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4dad74623df9ef3c1d2f8558b51a89fcf4645a23fc49ba33c11f8ca515ad7b8d
-size 4897043920

 version https://git-lfs.github.com/spec/v1
+oid sha256:efc57ea4d7ce8e9bb3b62850ee8202b7a9bf640c5ded5740267c8bb10474a63f
+size 4991381672

model-00002-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ddd875b1edc08dc006a995d40323dabecf320adb2bf0b4a3b361d5a85a7b0902
-size 4907418368

 version https://git-lfs.github.com/spec/v1
+oid sha256:3dfd50d9258716c35ea55fd5ad5e270f49e14d19ddecabc0b22099833dd4a2c4
+size 4917923224

model-00003-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8170cb671a9adb1f26ae2e5af0ed06a1b0241ceede51914bfffb7f22bf474be7
-size 4792152840

 version https://git-lfs.github.com/spec/v1
+oid sha256:7fb09f1385428bbbcda33d55d41002b06d4d190fe7f8d946e8494e81989a2b6b
+size 4572017744

model-00004-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:262c6cf8869ed1bc916dd0c32c3f415e8433ab7e31dfe90babfbbca9f95ceded
-size 4970482864

 version https://git-lfs.github.com/spec/v1
+oid sha256:d943f94d078ca2023d702d7391b688e5044de82de3b1b34d8cdbab6b40fb9280
+size 4744886088

model-00005-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cbeea6420a034f93791c1ee1c1b7dfc3f7002665ee6ff45eb1e619234a42d895
-size 4320359728

 version https://git-lfs.github.com/spec/v1
+oid sha256:89c86f9c4972d0844ca339b8e362d5b4b22154071a449e2823f4cdd0dd4166da
+size 4771117824

model-00006-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:98462fe93128431562585bc260ca12e8c288df92ad255c042728bf6c4c5236d7
-size 4797398328

 version https://git-lfs.github.com/spec/v1
+oid sha256:28cc5e6c6ce7fb993d55b815026c3f7add9e15cd4d3a7bc33f88ed6323efda8a
+size 4839341832

model-00007-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1d464ceb0909ad6b71d68843f63fd3c20cfeb82371e2a6b96858b314d0f4c722
-size 4844681208

 version https://git-lfs.github.com/spec/v1
+oid sha256:a5298acbe5ba54d4643124e643316937cb1da24b0311e15a9126b7804b789d1d
+size 4960004200

model-00008-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b4e06ca2d02c6cd38861686b14b68edb31df412390db5eb6d7f9e02441d3c719
-size 4938927192

 version https://git-lfs.github.com/spec/v1
+oid sha256:ef286120d278f5325a4e86f2abde7cc437d1aeb1be09e1c4b56fefc35fb8b732
+size 4970426792

model-00009-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:15b462bb9af7f5fd310c3ceac1aeaf9bda2247df524b490ec97400b625b943a5
-size 4750214368

 version https://git-lfs.github.com/spec/v1
+oid sha256:dc41571deb28a5c7db3935bfbd1cedff7c88f92361db23fce3e0fa811f9ebbd1
+size 4991510416

model-00010-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b528d990bdc6b7748534be1bdfff7feb1aeca66dd99582ce40d59c65a23ad372
-size 4970429440

 version https://git-lfs.github.com/spec/v1
+oid sha256:ecb2ba8049551c8e78de4381cf72a56e5642900bb1007f8382e57b3fcd0c2363
+size 4750255088

model-00011-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cdacd43f6679bd452333c8962f02d33a50d377f0423676d43bc82be10f2fba46
-size 4855137704

 version https://git-lfs.github.com/spec/v1
+oid sha256:3a8d803cf8e6d1bfa00841276abc239faeff5b9107c53ce7068a542cbac933ba
+size 4729227280

model-00012-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fa0b82f686eafbf3520d7222c44fd8f42dce3c56fce15ff72f98aecb0181f8d2
-size 4734438248

 version https://git-lfs.github.com/spec/v1
+oid sha256:a1f0be67d21eeaf431366e2784c9019ffea8208b7ea9f5dd8dbacfa08e860e8c
+size 4844587096

model-00013-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2a08e97df0fdaa6de71bc1d5bf1f0d3ae933973c320fc64a8132358272915280
-size 4729221224

 version https://git-lfs.github.com/spec/v1
+oid sha256:8b71de7aaf2b3c30aa99de88e5e2631b05b244536c5efecf61c20b3a52c17196
+size 4781671040

model-00014-of-00014.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:081be130cf4071ce8826c23034abcdb4361d25ec18c199236462cd003d5bef03
-size 3019936152

 version https://git-lfs.github.com/spec/v1
+oid sha256:f105e23a09c244ba63727dedc392b241b4a81b4d364e16c2333c925c4c948c27
+size 2663491240

model.safetensors.index.json CHANGED Viewed

The diff for this file is too large to render. See raw diff