Abstract
Large language models gain reasoning improvements from truncated trajectory endpoints rather than full reasoning traces, reducing redundancy while benefiting supervised fine-tuning and reinforcement learning.
Large language models (LLMs) are often post-trained on pre-collected reasoning trajectories to improve their reasoning capability. Such trajectories tend to be long due to complex, interwoven paths, which often include detours on the path toward the answer. However, it has been underexplored whether LLMs indeed benefit from learning complete trajectories in post-training, such as supervised fine-tuning (SFT). Starting from our pilot study, we find that full trajectories provide only limited benefit, while partial trajectories are effective even under heavy truncation. We analyze redundancy in reasoning trajectories through attention-based analyses and controlled token-removal studies, both of which show that intermediate tokens contribute minimally to final reasoning quality. This suggests that avoiding redundant information may allow LLMs to internally infer coherent alternatives by inferring missing steps from their internal knowledge, given known trajectory endpoints. Furthermore, we show that training LLMs using endpoints leads to consistent changes in reasoning behavior, and that it also benefits post-training methods based on reinforcement learning or on-policy distillation, highlighting the need to revisit complete reasoning traces. Code is available at https://github.com/naver-ai/revisiting-trace.
Community
We study whether LLMs indeed benefit from learning complete reasoning traces. Interestingly, full reasoning traces provide only limited benefits, while partial traces remain effective even under heavy truncation. Through attention-based analyses and controlled token-removal studies, we show that intermediate parts of reasoning traces are often redundant. As a result, endpoint-based SFT (E-SFT), using only the beginning and ending parts, retains or even improves SFT performance. We further find that endpoint-based training changes reasoning behavior and benefits both reinforcement learning (e.g., GRPO) and distillation (e.g., OPD).
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Extremely Sparse Supervision Incentivizes Reasoning Ability (2026)
- Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape CoT Calibration (2026)
- Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning (2026)
- ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning (2026)
- MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation (2026)
- Is Next-Chunk Reasoning RL Really Better than SFT? Revisiting Training Strategies under no-CoT Data (2026)
- ChainPrune: Evaluating and Reducing Redundancy in Long Chain-of-Thought Reasoning (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper