Instructions to use open-r1/OlympicCoder-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use open-r1/OlympicCoder-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="open-r1/OlympicCoder-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("open-r1/OlympicCoder-7B")
model = AutoModelForCausalLM.from_pretrained("open-r1/OlympicCoder-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use open-r1/OlympicCoder-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "open-r1/OlympicCoder-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-r1/OlympicCoder-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/open-r1/OlympicCoder-7B

SGLang

How to use open-r1/OlympicCoder-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "open-r1/OlympicCoder-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-r1/OlympicCoder-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "open-r1/OlympicCoder-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "open-r1/OlympicCoder-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use open-r1/OlympicCoder-7B with Docker Model Runner:
```
docker model run hf.co/open-r1/OlympicCoder-7B
```

Omitted <think> at the start and almost 10k tokens to debug 2 JS functions

by operationdarkside - opened Mar 13, 2025

Discussion

operationdarkside

Mar 13, 2025

Hello,
I recently downloaded this model as a Q4_K_M GGUF in LMStudio and tested it on a little error finding problem in 2 JS functions.
As mentioned in the title, the model took almost 10k tokens to think and forgot the "" at the start.
2 questions:

Is this amount of thinking tokens intended?
Did the model also omit the "" for anyone else?

Thank you for your work!

urtuuuu

Mar 13, 2025

Same, i just deleted it for this reason.

Biggbran

Mar 13, 2025

•

edited Mar 13, 2025

I said "Hi" and it's been responding for like 5 minutes because I forgot the <think>!

lewtun

Open R1 org Mar 17, 2025

Hi everyone, we explicitly prefill the chat template with <think> to ensure it always generates the long chain-of-thought: https://huggingface.co/open-r1/OlympicCoder-7B/blob/a093c195a14f190b8228b12cf6cd180c21bfbeec/tokenizer_config.json#L198

In our experience, using sampling with temperature=0.6 and top_p=0.95 gives pretty coherent answers that don't get stuck in the infinite loop!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment