arxiv:2608.11994

Claim-Level Reliability Assessment for Efficient Test-Time Reasoning

Published on Aug 12

· Submitted by

Sen Xu on Aug 17

WeiboAI

Upvote

Authors:

Sen Xu ,

Jixin Min ,

Zhibin Yin ,

Abstract

Claim-Level Reliability Assessment improves reasoning accuracy by verifying critical claims instead of sampling more solutions, reducing token use while boosting performance.

Generated by thinkingmachines/Inkling-Small

We propose claim-level falsification as a principle for test-time scaling and instantiate it through Claim-Level Reliability Assessment (CLR), a training-free framework that reallocates test-time compute from additional solution sampling to targeted verification. Since whole-trace evaluation often obscures decisive errors due to signal dilution from routine tokens, CLR condenses each reasoning trace into a compact set of decision-critical claims, thereby isolating its logical anchors. Furthermore, recognizing the inherent difficulty of generating entirely correct solutions under fixed model capabilities, CLR shifts the focus to semantic falsification. This approach exploits a fundamental asymmetry between solution construction and claim refutation. Constructing a valid solution requires a flawless reasoning path, whereas refuting an incorrect claim requires identifying only a single decisive flaw. This targeted search for negative evidence systematically compresses the survival space of high-confidence incorrect traces, effectively suppressing erroneous consensus via nonlinear reliability scoring. Across four LLMs and four reasoning benchmarks under matched budgets, CLR generally improves upon pass@1 and self-consistency. On GPT-OSS-20B/CMIMC25, for instance, CLR exceeds pass@1 by 27.15 percentage-points and raises self-consistency accuracy from 77.50\% to 82.19\% with 37.0\% fewer tokens.

View arXiv page View PDF GitHub 2 Add to collection

Community

SenXu1123

Paper author Paper submitter 1 day ago

We introduce CLR (Claim-Level Reliability Assessment), a training-free test-time scaling method previously used in VibeThinker-3B. CLR is built on a simple idea that exploits the asymmetry between solving and falsification to improve reasoning reliability.

Falsification exploits asymmetric capability requirements.
With the same model parameters, establishing correctness through forward search is harder than falsifying a decisive claim. CLR exploits this asymmetry without requiring a stronger model.
Claim-level verification improves the signal-to-noise ratio.
By focusing on decision-critical claims, CLR reduces the influence of erroneous or irrelevant tokens in a full reasoning trace, making decisive failure signals easier to identify.
Falsification compresses the survival space of incorrect reasoning.
Rather than explicitly proving which trace is correct, CLR suppresses traces with decisive flaws, allowing more reliable reasoning to naturally emerge in the final consensus.

Empirically, this translates into substantial recovery of failed consensus. On GPT-OSS-20B, when at least one correct trace is already present but standard self-consistency still fails, CLR rescues ~37% of such cases on average. On CMIMC25, CLR outperforms standard self-consistency under matched model-call budgets, achieving 82.19% vs. 77.50% while using 37.0% fewer tokens, and delivers a +27.15 pp gain over Pass@1.

📄 Paper: https://arxiv.org/abs/2608.11994
💻 Code: https://github.com/WeiboAI/CLR