Accept consecutive leading system messages in chat template

#76
by jlapenna - opened

Fixes #75: the template rejects consecutive leading system messages with System message must be at the beginning.

Change

chat_template.jinja, with the chat_template string in tokenizer_config.json updated to the identical text:

  • The template now collects the contiguous run of system messages at the start of messages. Each one is trimmed as before, empty ones are skipped, and the rest are joined with "\n\n".
  • Both the tools and no-tools branches render that text in the single system block where messages[0]'s content used to go.
  • A system message is rejected only when it appears after a non-system message. Before this change, any system message after index 0 was rejected.

Unchanged

  • Requests with no system message or one system message render byte-identically, with or without tools.
  • The tool-definition block and the reasoning-effort preamble do not change.
  • A system message after a user, assistant or tool message is still rejected. This PR does not decide that question.
  • The rest of tokenizer_config.json is unchanged (the diff is one line).

Verification

  • repro_leading_system.py (in #75): [system, system, user] rendered with the original template raises TemplateError. With the fixed template it renders one system block containing both texts, with and without tools. Six previously handled conversation shapes (no system, one system, empty system, list content, a tool-call turn, and a system message after a user message), each with and without tools, render identically before and after.
  • transformers 5.20.1 apply_chat_template returns the same result. The original raises, and the fixed template renders exactly one <|im_start|>system.

๐Ÿค– Generated with Claude Code

https://claude.ai/code/session_01Knj2s56UENx8AitLLRDr6T

Independent offline check against base de4b8e4 and PR head 4381e064, using Jinja 3.1.6:

Check Configuration cases Result
Previously accepted inputs, byte-identical output 448 Pass
Multiple leading system messages, exact match to a single merged-system control 640 Pass
Late system messages remain rejected 96 Pass
Existing invalid inputs retain their rejection 12 Pass

The matrix crosses empty/null/text-list/Unicode system content, multi-turn and tool-result histories, reasoning controls, preserved thinking, generation headers and vision numbering, with and without tools. These are 1,196 configuration cases, rather than distinct failure modes. I also checked that both standalone templates exactly match their embedded tokenizer-config strings and that the other tokenizer-config values are unchanged.

No rendering regression found in that coverage. This check used only offline Jinja rendering; it does not validate inference-server or application behavior.

Reproduction script, pinned source manifest and coverage notes. The published runner also passes in CI.

Cannot merge
This branch has merge conflicts in the following files:
  • tokenizer_config.json

Sign up or log in to comment