Instructions to use Qwen/Qwen3.8-Flash-Next with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Qwen/Qwen3.8-Flash-Next with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Qwen/Qwen3.8-Flash-Next") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Qwen/Qwen3.8-Flash-Next") model = AutoModelForMultimodalLM.from_pretrained("Qwen/Qwen3.8-Flash-Next", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Qwen/Qwen3.8-Flash-Next with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Qwen/Qwen3.8-Flash-Next" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen3.8-Flash-Next", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Qwen/Qwen3.8-Flash-Next
- SGLang
How to use Qwen/Qwen3.8-Flash-Next with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Qwen/Qwen3.8-Flash-Next" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen3.8-Flash-Next", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Qwen/Qwen3.8-Flash-Next" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen3.8-Flash-Next", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use Qwen/Qwen3.8-Flash-Next with Docker Model Runner:
docker model run hf.co/Qwen/Qwen3.8-Flash-Next
Accept consecutive leading system messages in chat template
Fixes #75: the template rejects consecutive leading system messages with System message must be at the beginning.
Change
chat_template.jinja, with the chat_template string in tokenizer_config.json updated to the identical text:
- The template now collects the contiguous run of system messages at the start of
messages. Each one is trimmed as before, empty ones are skipped, and the rest are joined with"\n\n". - Both the tools and no-tools branches render that text in the single system block where
messages[0]'s content used to go. - A system message is rejected only when it appears after a non-system message. Before this change, any system message after index 0 was rejected.
Unchanged
- Requests with no system message or one system message render byte-identically, with or without
tools. - The tool-definition block and the reasoning-effort preamble do not change.
- A system message after a user, assistant or tool message is still rejected. This PR does not decide that question.
- The rest of
tokenizer_config.jsonis unchanged (the diff is one line).
Verification
repro_leading_system.py(in #75):[system, system, user]rendered with the original template raisesTemplateError. With the fixed template it renders one system block containing both texts, with and without tools. Six previously handled conversation shapes (no system, one system, empty system, list content, a tool-call turn, and a system message after a user message), each with and without tools, render identically before and after.- transformers 5.20.1
apply_chat_templatereturns the same result. The original raises, and the fixed template renders exactly one<|im_start|>system.
๐ค Generated with Claude Code
Independent offline check against base de4b8e4 and PR head 4381e064, using Jinja 3.1.6:
| Check | Configuration cases | Result |
|---|---|---|
| Previously accepted inputs, byte-identical output | 448 | Pass |
| Multiple leading system messages, exact match to a single merged-system control | 640 | Pass |
| Late system messages remain rejected | 96 | Pass |
| Existing invalid inputs retain their rejection | 12 | Pass |
The matrix crosses empty/null/text-list/Unicode system content, multi-turn and tool-result histories, reasoning controls, preserved thinking, generation headers and vision numbering, with and without tools. These are 1,196 configuration cases, rather than distinct failure modes. I also checked that both standalone templates exactly match their embedded tokenizer-config strings and that the other tokenizer-config values are unchanged.
No rendering regression found in that coverage. This check used only offline Jinja rendering; it does not validate inference-server or application behavior.
Reproduction script, pinned source manifest and coverage notes. The published runner also passes in CI.