# Llama 3.2 Prompting with pipeline - system message

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-3-2-prompting-with-pipeline-system-message%2F168814%3E%5C
**Category:** Beginners\
**Created:** [September 30, 2025, 7:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-3-2-prompting-with-pipeline-system-message%2F168814 "2025-09-30T19:03:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![aboodmm](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@aboodmm](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faboodmm)%5C
**Post date:** [September 30, 2025, 7:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-3-2-prompting-with-pipeline-system-message%2F168814%2F1 "2025-09-30T19:03:53Z")

</div>

I am applying a transformers pipeline to prompt Llama 3.2 with several dozens of prompts.

My question is, is it correct to zip a system prompt in along with each prompt into the pipeline, or simply have the system prompt as the first message in the pipeline

Which approach would perform better based on your experience?

Here’s my code.

```auto
for cur in alert_messages:

```

```auto
    messages = [system_message, cur]

```

```auto
    outputs = pipe(

```

```auto
        messages,

```

```auto
    )

```

```auto
    results.append(outputs[0]["generated_text"])

```

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [September 30, 2025, 9:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-3-2-prompting-with-pipeline-system-message%2F168814%2F2 "2025-09-30T21:25:04Z")

</div>

Regarding the standard Pipeline class in Transoformers, since it has no memory, you must pass it every time.

* * *

Use a `system` message as the **first message in every chat you send to the pipeline**. Pipelines are stateless, so they won’t remember a prior system prompt. Chat templates also assume the system message sits at the front. ([Hugging Face](https://huggingface.co/docs/transformers/en/main_classes/pipelines "Pipelines"))

### What to do

- Keep: `messages = [system_message, cur]` for each prompt.
- Prefer batching to cut overhead.
- If you need more speed, use a server that **prefix-caches** identical system prompts.

### Minimal fixes

**Batched calls in Transformers**

```python
# Docs: https://huggingface.co/docs/transformers/en/main_classes/pipelines
# Chat templates: https://huggingface.co/docs/transformers/en/chat_templating
chats = [[system_message, cur] for cur in alert_messages]
outs = pipe(chats, max_new_tokens=256) # batch many chats at once
results = [o["generated_text"] for o in outs]

```

([Hugging Face](https://huggingface.co/docs/transformers/en/main_classes/pipelines "Pipelines"))

**If you ever build prompts manually, still put system first**

```python
# Docs: https://huggingface.co/docs/transformers/en/chat_templating
inputs = tokenizer.apply_chat_template(
    [
      {"role": "system", "content": SYSTEM_TXT},
      {"role": "user", "content": USER_TXT},
    ],
    add_generation_prompt=True,
    return_tensors="pt",
)

```

([Hugging Face](https://huggingface.co/docs/transformers/en/chat_templating "Chat templates"))

### Performance realities

- Sending the system prompt per chat is required. There is no “once globally” in a plain pipeline. ([Hugging Face](https://huggingface.co/docs/transformers/en/main_classes/pipelines "Pipelines"))

- For throughput, run the same chats on a backend with **prefix caching** so the shared system prefix is reused:

### Restated

- Correct: include the system prompt as the first message in each call.
- Better performance: batch in Transformers; or use a runtime that reuses the identical system prefix across requests. ([Hugging Face](https://huggingface.co/docs/transformers/en/main_classes/pipelines "Pipelines"))

**Short references**

- Transformers pipelines and batching. ([Hugging Face](https://huggingface.co/docs/transformers/en/main_classes/pipelines "Pipelines"))
- Chat templates and system-first placement. ([Hugging Face](https://huggingface.co/docs/transformers/en/chat_templating "Chat templates"))
- vLLM prefix caching overview. ([VLLM](https://docs.vllm.ai/en/v0.9.2/features/automatic_prefix_caching.html "Automatic Prefix Caching - vLLM"))
- llama.cpp prompt cache usage. ([app.semanticdiff.com](https://app.semanticdiff.com/gh/ggerganov/llama.cpp/pull/7745/overview "server: update cache\_prompt documentation [no ci] #7745"))
