arxiv:2608.06113

DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds

Published on Aug 6

· Submitted by

Boyuan Chen on Aug 10

Centre for Software Excellence

Upvote

Authors:

Kishanthan Thangarajah ,

Boyuan Chen ,

Abstract

Decoupling CLI agent scaffolding enables cross-scaffold evaluation and shows that learning planning improves generalization across different agent environments.

Generated by thinkingmachines/Inkling-Small

CLI-based software-engineering agents have matured rapidly, yet the open ecosystem has converged on a single training environment: trajectory datasets used to fine-tune open models are collected almost exclusively under OpenHands. Models fine-tuned on this data score well under OpenHands but degrade substantially when deployed under any non-training scaffold. Untrained base models do not show this divergence, indicating the gap is fine-tuning-induced and tied to the conventions of the training scaffold. We argue that a load-bearing scaffold-specific behavior is planning structure, in two senses this paper distinguishes: explicit planning, a pre-execution plan produced as a first-class artifact, and implicit planning, the structural conventions that shape execution throughout the agent loop. Under this hypothesis, closing the gap requires moving planning from a fixed scaffold artifact to a learned model capability. We introduce Decoupling CLI Agent Scaffolding (DCAS), a backend-substitution interception layer that routes API traffic between any CLI scaffold and any backend model without modifying the scaffold, enabling cross-scaffold evaluation and planning-aware trajectory collection. Using DCAS, a controlled plan-source intervention confirms planning quality is a high-leverage component, with gains exceeding the cross-scaffold drops we observe. A model fine-tuned on a small set of DCAS-collected planning-aware trajectories under a single scaffold gains consistently across non-training scaffolds, and the two senses of planning are empirically separable in training data.

View arXiv page View PDF Add to collection

Community

nemocbb

Paper author Paper submitter about 22 hours ago

SWE-Lego, Nebius, CoderForge, SWE-Gym. Every major open trajectory dataset for CLI coding agents is collected primarily under the same scaffold, OpenHands, but practitioners pick their scaffold on cost, licensing, latency and data privacy.

We measured what happens when the training scaffold and the deployment scaffold are not the same. Across four top open models at the 30B to 32B scale on SWE-bench Verified, every fine-tuned model degraded on at least one scaffold it was not trained on, one losing 29 points under mini-swe-agent and another falling to 8.4%. The untrained base models did not show this spread, so the divergence is fine-tuning-induced.

We argue the highest-leverage behavior here is planning structure, both the plan a model produces as a first-class artifact before acting and the structural conventions that shape execution throughout the agent loop. To test that we built DCAS, a backend-substitution interception layer that routes API traffic between any CLI scaffold and any backend model without modifying the scaffold. Changing only who wrote the plan moves Qwen3-Coder-30B-A3B-Instruct from 42.8% to 57.8% Pass@1 with the executor and scaffold held fixed, and full-parameter SFT on just 576 planning-aware trajectories collected through DCAS reaches 55.8% under self-plan, then 57.2% on a Claude Code release published after collection, 49.4% on OpenCode and 42.4% on mini-swe-agent, none of which the model saw during training.