2026-01-19

Title: LLMs for Game Theory: Entropy-Guided In-Context Learning and Adaptive CoT Reasoning

• Authors: Tommaso Felice Banfi, Sashenka Gamage
• Subjects: cs.CL, cs.GT, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.10775
• Pdf URL: https://arxiv.org/pdf/2601.10775
• Copy Paste: [[2601.10775]] LLMs for Game Theory: Entropy-Guided In-Context Learning and Adaptive CoT Reasoning(https://arxiv.org/abs/2601.10775)
• Keywords: llm, chain-of-thought
• Abstract: We propose a novel LLM-based framework for reasoning in discrete, game-theoretic tasks, illustrated with \emph{Tic-Tac-Toe}. The method integrates in-context learning with entropy-guided chain-of-thought (CoT) reasoning and adaptive context retrieval. The model dynamically adjusts both the number of retrieved examples and reasoning paths according to token-level uncertainty: concise reasoning with minimal context is used when uncertainty is low, whereas higher uncertainty triggers expanded multi-path CoT exploration. Experimental evaluation against a sub-optimal algorithmic opponent shows that entropy-aware adaptive reasoning substantially improves decision quality, increasing the average game outcome from \(-11.6\%\) with the baseline LLM to \(+9.5\%\) with entropy-guided adaptive reasoning over 100 games (win = +1, tie = 0, loss = -1), while maintaining a relatively low number of LLM queries per game. Statistical validation confirms that the improvement is significant, and correlation analysis reveals a negative association between token-level entropy and move optimality. These findings demonstrate that uncertainty-guided adaptive reasoning effectively enhances LLM performance in sequential decision-making environments.
• 摘要：我们提出了一种基于 LLM 的新颖框架，用于离散、博弈论任务的推理，如 \emph{Tic-Tac-Toe} 所示。该方法将上下文学习与熵引导的思想链 (CoT) 推理和自适应上下文检索相结合。该模型根据 token 级别的不确定性动态调整检索示例的数量和推理路径：当不确定性较低时，使用最小上下文的简洁推理，而较高的不确定性会触发扩展的多路径 CoT 探索。针对次优算法对手的实验评估表明，熵感知自适应推理大大提高了决策质量，将平均游戏结果从基线 LLM 的 \(-11.6\%\) 提高到熵引导自适应推理超过 100 场游戏的 \(+9.5\%\)（获胜 = +1，平局 = 0，损失 = -1），同时保持每场比赛相对较低的 LLM 查询数量。统计验证证实了改进是显着的，相关性分析揭示了令牌级熵和移动最优性之间的负相关性。这些发现表明，不确定性引导的自适应推理有效地提高了法学硕士在顺序决策环境中的表现。

Title: BYOL: Bring Your Own Language Into LLMs

• Authors: Syed Waqas Zamir, Wassim Hamidouche, Boulbaba Ben Amor, Luana Marotti, Inbal Becker-Reshef, Juan Lavista Ferres
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.10804
• Pdf URL: https://arxiv.org/pdf/2601.10804
• Copy Paste: [[2601.10804]] BYOL: Bring Your Own Language Into LLMs(https://arxiv.org/abs/2601.10804)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) exhibit strong multilingual capabilities, yet remain fundamentally constrained by the severe imbalance in global language resources. While over 7,000 languages are spoken worldwide, only a small subset (fewer than 100) has sufficient digital presence to meaningfully influence modern LLM training. This disparity leads to systematic underperformance, cultural misalignment, and limited accessibility for speakers of low-resource and extreme-low-resource languages. To address this gap, we introduce Bring Your Own Language (BYOL), a unified framework for scalable, language-aware LLM development tailored to each language's digital footprint. BYOL begins with a language resource classification that maps languages into four tiers (Extreme-Low, Low, Mid, High) using curated web-scale corpora, and uses this classification to select the appropriate integration pathway. For low-resource languages, we propose a full-stack data refinement and expansion pipeline that combines corpus cleaning, synthetic text generation, continual pretraining, and supervised finetuning. Applied to Chichewa and Maori, this pipeline yields language-specific LLMs that achieve approximately 12 percent average improvement over strong multilingual baselines across 12 benchmarks, while preserving English and multilingual capabilities via weight-space model merging. For extreme-low-resource languages, we introduce a translation-mediated inclusion pathway, and show on Inuktitut that a tailored machine translation system improves over a commercial baseline by 4 BLEU, enabling high-accuracy LLM access when direct language modeling is infeasible. Finally, we release human-translated versions of the Global MMLU-Lite benchmark in Chichewa, Maori, and Inuktitut, and make our codebase and models publicly available at this https URL .
• 摘要：大型语言模型（LLM）表现出强大的多语言能力，但仍然受到全球语言资源严重不平衡的根本限制。虽然全世界使用 7,000 多种语言，但只有一小部分（少于 100 种）拥有足够的数字化存在，能够对现代法学硕士培训产生有意义的影响。这种差异导致系统性的表现不佳、文化错位以及资源匮乏和资源极少的语言使用者的可及性有限。为了弥补这一差距，我们引入了自带语言 (BYOL)，这是一个针对每种语言的数字足迹量身定制的可扩展、语言感知的法学硕士开发的统一框架。 BYOL 从语言资源分类开始，使用精心策划的网络规模语料库将语言映射到四个级别（极低、低、中、高），并使用此分类来选择适当的集成路径。对于资源匮乏的语言，我们提出了一个全栈数据细化和扩展管道，结合了语料库清理、合成文本生成、持续预训练和监督微调。该管道应用于奇切瓦语和毛利语，产生特定语言的法学硕士，在 12 个基准中比强大的多语言基线平均提高约 12%，同时通过权重空间模型合并保留英语和多语言能力。对于资源极少的语言，我们引入了一种以翻译为中介的包容途径，并在 Inuktitut 上展示了定制的机器翻译系统比商业基线提高了 4 BLEU，从而在直接语言建模不可行时实现高精度 LLM 访问。最后，我们发布了奇切瓦语、毛利语和因纽特语的 Global MMLU-Lite 基准的人工翻译版本，并在此 https URL 上公开提供我们的代码库和模型。

Title: A Concise Agent is Less Expert: Revealing Side Effects of Using Style Features on Conversational Agents

• Authors: Young-Min Cho, Yuan Yuan, Sharath Chandra Guntuku, Lyle Ungar
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.10809
• Pdf URL: https://arxiv.org/pdf/2601.10809
• Copy Paste: [[2601.10809]] A Concise Agent is Less Expert: Revealing Side Effects of Using Style Features on Conversational Agents(https://arxiv.org/abs/2601.10809)
• Keywords: language model, llm, prompt, agent
• Abstract: Style features such as friendly, helpful, or concise are widely used in prompts to steer the behavior of Large Language Model (LLM) conversational agents, yet their unintended side effects remain poorly understood. In this work, we present the first systematic study of cross-feature stylistic side effects. We conduct a comprehensive survey of 127 conversational agent papers from ACL Anthology and identify 12 frequently used style features. Using controlled, synthetic dialogues across task-oriented and open domain settings, we quantify how prompting for one style feature causally affects others via a pairwise LLM as a Judge evaluation framework. Our results reveal consistent and structured side effects, such as prompting for conciseness significantly reduces perceived expertise. They demonstrate that style features are deeply entangled rather than orthogonal. To support future research, we introduce CASSE (Conversational Agent Stylistic Side Effects), a dataset capturing these complex interactions. We further evaluate prompt based and activation steering based mitigation strategies and find that while they can partially restore suppressed traits, they often degrade the primary intended style. These findings challenge the assumption of faithful style control in LLMs and highlight the need for multi-objective and more principled approaches to safe, targeted stylistic steering in conversational agents.
• 摘要：友好、乐于助人或简洁等风格特征被广泛用于提示中，以引导大型语言模型 (LLM) 对话代理的行为，但它们的意外副作用仍然知之甚少。在这项工作中，我们首次提出了跨特征风格副作用的系统研究。我们对 ACL Anthology 中的 127 篇会话代理论文进行了全面调查，并确定了 12 种常用的风格特征。使用跨任务导向和开放领域设置的受控综合对话，我们通过成对的法学硕士作为法官评估框架，量化对一种风格特征的提示如何对其他风格特征产生因果影响。我们的结果揭示了一致且结构化的副作用，例如提示简洁性会显着降低感知的专业知识。它们表明风格特征是深深纠缠在一起的，而不是正交的。为了支持未来的研究，我们引入了 CASSE（会话代理文体副作用），这是一个捕获这些复杂交互的数据集。我们进一步评估基于提示和基于激活转向的缓解策略，发现虽然它们可以部分恢复被抑制的特征，但它们通常会降低主要预期风格。这些发现挑战了法学硕士中忠实风格控制的假设，并强调需要采用多目标和更具原则性的方法来在会话代理中进行安全、有针对性的风格控制。

Title: Reasoning Models Generate Societies of Thought

• Authors: Junsol Kim, Shiyang Lai, Nino Scherrer, Blaise Agüera y Arcas, James Evans
• Subjects: cs.CL, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.10825
• Pdf URL: https://arxiv.org/pdf/2601.10825
• Copy Paste: [[2601.10825]] Reasoning Models Generate Societies of Thought(https://arxiv.org/abs/2601.10825)
• Keywords: language model, agent
• Abstract: Large language models have achieved remarkable capabilities across domains, yet mechanisms underlying sophisticated reasoning remain elusive. Recent reasoning models outperform comparable instruction-tuned models on complex cognitive tasks, attributed to extended computation through longer chains of thought. Here we show that enhanced reasoning emerges not from extended computation alone, but from simulating multi-agent-like interactions -- a society of thought -- which enables diversification and debate among internal cognitive perspectives characterized by distinct personality traits and domain expertise. Through quantitative analysis and mechanistic interpretability methods applied to reasoning traces, we find that reasoning models like DeepSeek-R1 and QwQ-32B exhibit much greater perspective diversity than instruction-tuned models, activating broader conflict between heterogeneous personality- and expertise-related features during reasoning. This multi-agent structure manifests in conversational behaviors, including question-answering, perspective shifts, and the reconciliation of conflicting views, and in socio-emotional roles that characterize sharp back-and-forth conversations, together accounting for the accuracy advantage in reasoning tasks. Controlled reinforcement learning experiments reveal that base models increase conversational behaviors when rewarded solely for reasoning accuracy, and fine-tuning models with conversational scaffolding accelerates reasoning improvement over base models. These findings indicate that the social organization of thought enables effective exploration of solution spaces. We suggest that reasoning models establish a computational parallel to collective intelligence in human groups, where diversity enables superior problem-solving when systematically structured, which suggests new opportunities for agent organization to harness the wisdom of crowds.
• 摘要：大型语言模型已经在跨领域取得了非凡的能力，但复杂推理背后的机制仍然难以捉摸。最近的推理模型在复杂的认知任务上优于类似的指令调整模型，这归因于通过更长的思维链进行扩展计算。在这里，我们表明，增强推理不仅仅来自扩展计算，而是来自模拟类似多智能体的交互（一个思想社会），这使得以独特的人格特质和领域专业知识为特征的内部认知观点之间的多样化和辩论成为可能。通过应用于推理轨迹的定量分析和机械解释方法，我们发现像 DeepSeek-R1 和 QwQ-32B 这样的推理模型比指令调整模型表现出更大的视角多样性，在推理过程中引发了异质人格和专业知识相关特征之间更广泛的冲突。这种多智能体结构体现在对话行为中，包括回答问题、视角转换和冲突观点的调和，以及以尖锐的来回对话为特征的社会情感角色，共同解释了推理任务中的准确性优势。受控强化学习实验表明，当仅因推理准确性而获得奖励时，基本模型会增加会话行为，并且使用会话支架微调模型可加速基本模型的推理改进。这些发现表明，思想的社会组织能够有效地探索解决方案空间。我们建议推理模型在人类群体中建立与集体智慧并行的计算，其中多样性在系统化构建时能够实现卓越的问题解决，这为代理组织利用群体智慧提供了新的机会。

Title: EncodeRec: An Embedding Backbone for Recommendation Systems

• Authors: Guy Hadad, Neomi Rabaev, Bracha Shapira
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2601.10837
• Pdf URL: https://arxiv.org/pdf/2601.10837
• Copy Paste: [[2601.10837]] EncodeRec: An Embedding Backbone for Recommendation Systems(https://arxiv.org/abs/2601.10837)
• Keywords: language model
• Abstract: Recent recommender systems increasingly leverage embeddings from large pre-trained language models (PLMs). However, such embeddings exhibit two key limitations: (1) PLMs are not explicitly optimized to produce structured and discriminative embedding spaces, and (2) their representations remain overly generic, often failing to capture the domain-specific semantics crucial for recommendation tasks. We present EncodeRec, an approach designed to align textual representations with recommendation objectives while learning compact, informative embeddings directly from item descriptions. EncodeRec keeps the language model parameters frozen during recommender system training, making it computationally efficient without sacrificing semantic fidelity. Experiments across core recommendation benchmarks demonstrate its effectiveness both as a backbone for sequential recommendation models and for semantic ID tokenization, showing substantial gains over PLM-based and embedding model baselines. These results underscore the pivotal role of embedding adaptation in bridging the gap between general-purpose language models and practical recommender systems.
• 摘要：最近的推荐系统越来越多地利用大型预训练语言模型 (PLM) 的嵌入。然而，此类嵌入存在两个关键限制：(1) PLM 没有明确优化以生成结构化和有区别的嵌入空间，(2) 它们的表示仍然过于通用，通常无法捕获对推荐任务至关重要的特定领域语义。我们提出了 EncodeRec，一种旨在将文本表示与推荐目标结合起来的方法，同时直接从项目描述中学习紧凑、信息丰富的嵌入。 EncodeRec 在推荐系统训练期间保持语言模型参数冻结，使其在不牺牲语义保真度的情况下提高计算效率。跨核心推荐基准的实验证明了其作为顺序推荐模型和语义 ID 标记化骨干的有效性，显示出相对于基于 PLM 和嵌入模型基线的巨大收益。这些结果强调了嵌入适应在弥合通用语言模型和实用推荐系统之间差距方面的关键作用。

Title: DialDefer: A Framework for Detecting and Mitigating LLM Dialogic Deference

• Authors: Parisa Rabbani, Priyam Sahoo, Ruben Mathew, Aishee Mondal, Harshita Ketharaman, Nimet Beyza Bozdag, Dilek Hakkani-Tür
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.10896
• Pdf URL: https://arxiv.org/pdf/2601.10896
• Copy Paste: [[2601.10896]] DialDefer: A Framework for Detecting and Mitigating LLM Dialogic Deference(https://arxiv.org/abs/2601.10896)
• Keywords: llm
• Abstract: LLMs are increasingly used as third-party judges, yet their reliability when evaluating speakers in dialogue remains poorly understood. We show that LLMs judge identical claims differently depending on framing: the same content elicits different verdicts when presented as a statement to verify ("Is this statement correct?") versus attributed to a speaker ("Is this speaker correct?"). We call this dialogic deference and introduce DialDefer, a framework for detecting and mitigating these framing-induced judgment shifts. Our Dialogic Deference Score (DDS) captures directional shifts that aggregate accuracy obscures. Across nine domains, 3k+ instances, and four models, conversational framing induces large shifts (|DDS| up to 87pp, p < .0001) while accuracy remains stable (<2pp), with effects amplifying 2-4x on naturalistic Reddit conversations. Models can shift toward agreement (deference) or disagreement (skepticism) depending on domain -- the same model ranges from DDS = -53 on graduate-level science to +58 on social judgment. Ablations reveal that human-vs-LLM attribution drives the largest shifts (17.7pp swing), suggesting models treat disagreement with humans as more costly than with AI. Mitigation attempts reduce deference but can over-correct into skepticism, framing this as a calibration problem beyond accuracy optimization.
• 摘要：法学硕士越来越多地被用作第三方评委，但他们在评估对话中发言者时的可靠性仍然知之甚少。我们表明，法学硕士根据框架对相同的主张做出不同的判断：相同的内容在作为验证声明（“这个声明正确吗？”）与归因于演讲者（“这个演讲者正确吗？”）时会引起不同的判断。我们将这种现象称为对话尊重，并引入 DialDefer，这是一个用于检测和减轻这些框架引起的判断转变的框架。我们的对话遵从评分 (DDS) 捕捉到了聚合准确度所掩盖的方向变化。在 9 个域、3k+ 个实例和 4 个模型中，对话框架会引起较大的变化（|DDS| 高达 87pp，p < .0001），而准确性保持稳定（<2pp），自然 Reddit 对话的效果放大 2-4 倍。根据领域的不同，模型可以转向同意（尊重）或不同意（怀疑）——同一模型的范围从研究生水平科学的 DDS = -53 到社会判断的 +58。消融显示，人类与法学硕士的归因推动了最大的转变（17.7pp 波动），这表明模型认为与人类的分歧比与人工智能的分歧成本更高。缓解措施尝试减少尊重，但可能会过度纠正怀疑，将其视为超出精度优化的校准问题。

Title: Massively Multilingual Joint Segmentation and Glossing

• Authors: Michael Ginn, Lindia Tjuatja, Enora Rice, Ali Marashian, Maria Valentini, Jasmine Xu, Graham Neubig, Alexis Palmer
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.10925
• Pdf URL: https://arxiv.org/pdf/2601.10925
• Copy Paste: [[2601.10925]] Massively Multilingual Joint Segmentation and Glossing(https://arxiv.org/abs/2601.10925)
• Keywords: llm
• Abstract: Automated interlinear gloss prediction with neural networks is a promising approach to accelerate language documentation efforts. However, while state-of-the-art models like GlossLM achieve high scores on glossing benchmarks, user studies with linguists have found critical barriers to the usefulness of such models in real-world scenarios. In particular, existing models typically generate morpheme-level glosses but assign them to whole words without predicting the actual morpheme boundaries, making the predictions less interpretable and thus untrustworthy to human annotators. We conduct the first study on neural models that jointly predict interlinear glosses and the corresponding morphological segmentation from raw text. We run experiments to determine the optimal way to train models that balance segmentation and glossing accuracy, as well as the alignment between the two tasks. We extend the training corpus of GlossLM and pretrain PolyGloss, a family of seq2seq multilingual models for joint segmentation and glossing that outperforms GlossLM on glossing and beats various open-source LLMs on segmentation, glossing, and alignment. In addition, we demonstrate that PolyGloss can be quickly adapted to a new dataset via low-rank adaptation.
• 摘要：使用神经网络进行自动行间光泽度预测是加速语言文档工作的一种有前途的方法。然而，虽然像 GlossLM 这样最先进的模型在注释基准上取得了高分，但与语言学家的用户研究发现，此类模型在现实场景中的实用性存在关键障碍。特别是，现有模型通常会生成语素级注释，但将它们分配给整个单词，而不预测实际的语素边界，从而使预测难以解释，因此对人类注释者来说不可信。我们对联合预测行间注释和原始文本相应的形态分割的神经模型进行了首次研究。我们进行实验来确定训练模型的最佳方法，以平衡分割和注释准确性以及两个任务之间的一致性。我们扩展了 GlossLM 的训练语料库并预训练了 PolyGloss，这是一系列用于联合分割和修饰的 seq2seq 多语言模型，其在修饰方面优于 GlossLM，并在分割、修饰和对齐方面击败了各种开源 LLM。此外，我们还证明了 PolyGloss 可以通过低秩适应快速适应新的数据集。

Title: Selecting Language Models for Social Science: Start Small, Start Open, and Validate

• Authors: Dustin S. Stoltz, Marshall A. Taylor, Sanuj Kumar
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.10926
• Pdf URL: https://arxiv.org/pdf/2601.10926
• Copy Paste: [[2601.10926]] Selecting Language Models for Social Science: Start Small, Start Open, and Validate(https://arxiv.org/abs/2601.10926)
• Keywords: language model, llm
• Abstract: Currently, there are thousands of large pretrained language models (LLMs) available to social scientists. How do we select among them? Using validity, reliability, reproducibility, and replicability as guides, we explore the significance of: (1) model openness, (2) model footprint, (3) training data, and (4) model architectures and fine-tuning. While ex-ante tests of validity (i.e., benchmarks) are often privileged in these discussions, we argue that social scientists cannot altogether avoid validating computational measures (ex-post). Replicability, in particular, is a more pressing guide for selecting language models. Being able to reliably replicate a particular finding that entails the use of a language model necessitates reliably reproducing a task. To this end, we propose starting with smaller, open models, and constructing delimited benchmarks to demonstrate the validity of the entire computational pipeline.
• 摘要：目前，社会科学家可以使用数千个大型预训练语言模型 (LLM)。我们如何在其中进行选择？以有效性、可靠性、再现性和可复制性为指导，我们探讨了以下方面的重要性：(1) 模型开放性，(2) 模型足迹，(3) 训练数据，以及 (4) 模型架构和微调。虽然事前有效性测试（即基准）在这些讨论中通常受到重视，但我们认为社会科学家不能完全避免验证计算措施（事后）。尤其是可复制性，是选择语言模型的一个更紧迫的指南。能够可靠地复制需要使用语言模型的特定发现就需要可靠地复制任务。为此，我们建议从更小的开放模型开始，并构建界定的基准来证明整个计算管道的有效性。

Title: Multi-Stage Patient Role-Playing Framework for Realistic Clinical Interactions

• Authors: Shijie Jiang, Zefan Zhang, Kehua Zhu, Tian Bai, Ruihong Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.10951
• Pdf URL: https://arxiv.org/pdf/2601.10951
• Copy Paste: [[2601.10951]] Multi-Stage Patient Role-Playing Framework for Realistic Clinical Interactions(https://arxiv.org/abs/2601.10951)
• Keywords: language model, llm
• Abstract: The simulation of realistic clinical interactions plays a pivotal role in advancing clinical Large Language Models (LLMs) and supporting medical diagnostic education. Existing approaches and benchmarks rely on generic or LLM-generated dialogue data, which limits the authenticity and diversity of doctor-patient interactions. In this work, we propose the first Chinese patient simulation dataset (Ch-PatientSim), constructed from realistic clinical interaction scenarios to comprehensively evaluate the performance of models in emulating patient behavior. Patients are simulated based on a five-dimensional persona structure. To address issues of the persona class imbalance, a portion of the dataset is augmented using few-shot generation, followed by manual verification. We evaluate various state-of-the-art LLMs and find that most produce overly formal responses that lack individual personality. To address this limitation, we propose a training-free Multi-Stage Patient Role-Playing (MSPRP) framework, which decomposes interactions into three stages to ensure both personalization and realism in model responses. Experimental results demonstrate that our approach significantly improves model performance across multiple dimensions of patient simulation.
• 摘要：真实临床交互的模拟在推进临床大语言模型 (LLM) 和支持医学诊断教育方面发挥着关键作用。现有的方法和基准依赖于通用或法学硕士生成的对话数据，这限制了医患互动的真实性和多样性。在这项工作中，我们提出了第一个中国患者模拟数据集（Ch-PatientSim），该数据集根据真实的临床交互场景构建，以全面评估模型在模拟患者行为方面的性能。基于五维角色结构模拟患者。为了解决角色类别不平衡的问题，使用少数镜头生成来增强数据集的一部分，然后进行手动验证。我们评估了各种最先进的法学硕士，发现大多数都会产生过于正式的反应，缺乏个性。为了解决这一限制，我们提出了一种免训练的多阶段患者角色扮演（MSPRP）框架，它将交互分解为三个阶段，以确保模型响应的个性化和真实性。实验结果表明，我们的方法显着提高了患者模拟多个维度的模型性能。

Title: Steering Language Models Before They Speak: Logit-Level Interventions

• Authors: Hyeseon An, Shinwoo Park, Hyundong Jin, Yo-Sub Han
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.10960
• Pdf URL: https://arxiv.org/pdf/2601.10960
• Copy Paste: [[2601.10960]] Steering Language Models Before They Speak: Logit-Level Interventions(https://arxiv.org/abs/2601.10960)
• Keywords: language model, llm, prompt
• Abstract: Steering LLMs is essential for specialized applications such as style-sensitive text rewriting, user-adaptive communication, and toxicity mitigation. Current steering methods, such as prompting-based and activation-based approaches, are widely used to guide model behavior. However, activation-based techniques require deep access to internal layers, while prompting-based steering often fails to provide consistent or fine-grained control. In order to address these limitations, we propose a training-free inference-time logit intervention for controllable generation. Our approach utilizes a statistical token score table derived from z-normalized log-odds of labeled corpora to shift the decoding distribution. Empirical evaluations across three diverse datasets focusing on writing complexity, formality, and toxicity demonstrate that our method effectively steers output characteristics, confirming its broad applicability and task-agnostic nature. Our results show that statistically grounded logit steering can achieve large, consistent, and multi-task control gains: up to +47%p accuracy and 50x f1 improvement.
• 摘要：指导法学硕士对于风格敏感的文本重写、用户自适应通信和毒性缓解等专业应用至关重要。当前的引导方法，例如基于提示和基于激活的方法，被广泛用于指导模型行为。然而，基于激活的技术需要深入访问内部层，而基于提示的转向通常无法提供一致或细粒度的控制。为了解决这些限制，我们提出了一种用于可控生成的免训练推理时间 log​​it 干预。我们的方法利用从标记语料库的 z 归一化对数几率导出的统计标记分数表来改变解码分布。对三个不同数据集的实证评估侧重于写作复杂性、形式性和毒性，表明我们的方法有效地引导输出特征，证实了其广泛的适用性和与任务无关的性质。我们的结果表明，基于统计的 Logit 转向可以实现大的、一致的、多任务的控制增益：高达 +47%p 的精度和 50 倍的 f1 改进。

Title: ZPD Detector: Data Selection via Capability-Difficulty Alignment for Large Language Models

• Authors: Bo Yang, Yunkui Chen, Lanfei Feng, Yu Zhang, Shijian Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.10986
• Pdf URL: https://arxiv.org/pdf/2601.10986
• Copy Paste: [[2601.10986]] ZPD Detector: Data Selection via Capability-Difficulty Alignment for Large Language Models(https://arxiv.org/abs/2601.10986)
• Keywords: language model
• Abstract: As the cost of training large language models continues to increase and high-quality training data become increasingly scarce, selecting high-value samples or synthesizing effective training data under limited data budgets has emerged as a critical research problem. Most existing data selection methods rely on static criteria, such as difficulty, uncertainty, or heuristics, and fail to model the evolving relationship between the model and the data. Inspired by the educational theory of the Zone of Proximal Development (ZPD), we propose ZPD Detector, a data selection framework that adopts a bidirectional perspective between models and data by explicitly modeling the alignment between sample difficulty and the model's current capability. ZPD Detector integrates difficulty calibration, model capability estimation based on Item Response Theory (IRT), and a capability-difficulty matching score to dynamically identify the most informative samples at each learning stage, improving data utilization efficiency; moreover, this dynamic matching strategy provides new insights into training strategy design. All code and data will be released after our work be accepted to support reproducible researc
• 摘要：随着训练大型语言模型的成本不断增加和高质量训练数据变得越来越稀缺，在有限的数据预算下选择高价值样本或合成有效的训练数据已成为一个关键的研究问题。大多数现有的数据选择方法依赖于静态标准，例如难度、不确定性或启发式，并且无法对模型和数据之间不断变化的关系进行建模。受最近发展区（ZPD）教育理论的启发，我们提出了 ZPD Detector，这是一种数据选择框架，通过显式建模样本难度与模型当前能力之间的一致性，在模型和数据之间采用双向视角。 ZPD Detector集成难度校准、基于项目响应理论（IRT）的模型能力估计以及能力-难度匹配评分，动态识别每个学习阶段信息量最大的样本，提高数据利用效率；此外，这种动态匹配策略为训练策略设计提供了新的见解。所有代码和数据将在我们的工作被接受后发布，以支持可重复的研究

Title: When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs

• Authors: Zhongxiang Sun, Yi Zhan, Chenglei Shen, Weijie Yu, Xiao Zhang, Ming He, Jun Xu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.11000
• Pdf URL: https://arxiv.org/pdf/2601.11000
• Copy Paste: [[2601.11000]] When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs(https://arxiv.org/abs/2601.11000)
• Keywords: language model, llm, hallucination
• Abstract: Personalized large language models (LLMs) adapt model behavior to individual users to enhance user satisfaction, yet personalization can inadvertently distort factual reasoning. We show that when personalized LLMs face factual queries, there exists a phenomenon where the model generates answers aligned with a user's prior history rather than the objective truth, resulting in personalization-induced hallucinations that degrade factual reliability and may propagate incorrect beliefs, due to representational entanglement between personalization and factual representations. To address this issue, we propose Factuality-Preserving Personalized Steering (FPPS), a lightweight inference-time approach that mitigates personalization-induced factual distortions while preserving personalized behavior. We further introduce PFQABench, the first benchmark designed to jointly evaluate factual and personalized question answering under personalization. Experiments across multiple LLM backbones and personalization methods show that FPPS substantially improves factual accuracy while maintaining personalized performance.
• 摘要：个性化大语言模型 (LLM) 使模型行为适应各个用户，以提高用户满意度，但个性化可能会无意中扭曲事实推理。我们表明，当个性化法学硕士面临事实查询时，存在一种现象，即模型生成的答案与用户的先前历史而不是客观事实一致，从而导致个性化引起的幻觉，降低事实可靠性，并可能传播错误的信念，因为个性化和事实表征之间的表征纠缠。为了解决这个问题，我们提出了保留事实的个性化指导（FPPS），这是一种轻量级的推理时间方法，可以减轻个性化引起的事实扭曲，同时保留个性化行为。我们进一步介绍了PFQABench，这是第一个旨在联合评估个性化下的事实性和个性化问答的基准。跨多个 LLM 主干和个性化方法的实验表明，FPPS 显着提高了事实准确性，同时保持个性化性能。

Title: Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies

• Authors: Qianen Zhang, Zeyu Yang, Satoshi Nakamura
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11002
• Pdf URL: https://arxiv.org/pdf/2601.11002
• Copy Paste: [[2601.11002]] Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies(https://arxiv.org/abs/2601.11002)
• Keywords: language model, llm, prompt
• Abstract: Simultaneous Machine Translation (SiMT) requires high-quality translations under strict real-time constraints, which traditional policies with only READ/WRITE actions cannot fully address. We extend the action space of SiMT with four adaptive actions: Sentence_Cut, Drop, Partial_Summarization and Pronominalization, which enable real-time restructuring, omission, and simplification while preserving semantic fidelity. We adapt these actions in a large language model (LLM) framework and construct training references through action-aware prompting. To evaluate both quality and word-level monotonicity, we further develop a latency-aware TTS pipeline that maps textual outputs to speech with realistic timing. Experiments on the ACL60/60 English-Chinese, English-German and English-Japanese benchmarks show that our framework consistently improves semantic metrics and achieves lower delay compared to reference translations and salami-based baselines. Notably, combining Drop and Sentence_Cut leads to consistent improvements in the balance between fluency and latency. These results demonstrate that enriching the action space of LLM-based SiMT provides a promising direction for bridging the gap between human and machine interpretation.
• 摘要：同步机器翻译 (SiMT) 需要在严格的实时约束下实现高质量翻译，而仅具有读/写操作的传统策略无法完全解决这一问题。我们通过四种自适应动作扩展了 SiMT 的动作空间：Sentence_Cut、Drop、Partial_Summarization 和 Pronominalization，这些动作可以实现实时重组、省略和简化，同时保持语义保真度。我们在大型语言模型（LLM）框架中调整这些动作，并通过动作感知提示构建培训参考。为了评估质量和字级单调性，我们进一步开发了一个延迟感知的 TTS 管道，可以将文本输出映射到具有真实时序的语音。 ACL60/60 英汉、英德和英日基准的实验表明，与参考翻译和基于萨拉米的基准相比，我们的框架持续改进了语义指标并实现了更低的延迟。值得注意的是，结合 Drop 和 Sentence_Cut 可以持续改善流畅性和延迟之间的平衡。这些结果表明，丰富基于 LLM 的 SiMT 的动作空间为弥合人类和机器解释之间的差距提供了一个有希望的方向。

Title: NAACL: Noise-AwAre Verbal Confidence Calibration for LLMs in RAG Systems

• Authors: Jiayu Liu, Rui Wang, Qing Zong, Qingcheng Zeng, Tianshi Zheng, Haochen Shi, Dadi Guo, Baixuan Xu, Chunyang Li, Yangqiu Song
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11004
• Pdf URL: https://arxiv.org/pdf/2601.11004
• Copy Paste: [[2601.11004]] NAACL: Noise-AwAre Verbal Confidence Calibration for LLMs in RAG Systems(https://arxiv.org/abs/2601.11004)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Accurately assessing model confidence is essential for deploying large language models (LLMs) in mission-critical factual domains. While retrieval-augmented generation (RAG) is widely adopted to improve grounding, confidence calibration in RAG settings remains poorly understood. We conduct a systematic study across four benchmarks, revealing that LLMs exhibit poor calibration performance due to noisy retrieved contexts. Specifically, contradictory or irrelevant evidence tends to inflate the model's false certainty, leading to severe overconfidence. To address this, we propose NAACL Rules (Noise-AwAre Confidence CaLibration Rules) to provide a principled foundation for resolving overconfidence under noise. We further design NAACL, a noise-aware calibration framework that synthesizes supervision from about 2K HotpotQA examples guided by these rules. By performing supervised fine-tuning (SFT) with this data, NAACL equips models with intrinsic noise awareness without relying on stronger teacher models. Empirical results show that NAACL yields substantial gains, improving ECE scores by 10.9% in-domain and 8.0% out-of-domain. By bridging the gap between retrieval noise and verbal calibration, NAACL paves the way for both accurate and epistemically reliable LLMs.
• 摘要：准确评估模型置信度对于在关键任务事实领域部署大型语言模型 (LLM) 至关重要。虽然检索增强生成 (RAG) 被广泛采用来改善基础，但对 RAG 设置中的置信度校准仍然知之甚少。我们对四个基准进行了系统研究，发现法学硕士由于检索到的上下文有噪音而表现出较差的校准性能。具体来说，矛盾或不相关的证据往往会夸大模型的错误确定性，导致严重的过度自信。为了解决这个问题，我们提出 NAACL 规则（Noise-AwAre Confidence CalLibration Rules），为解决噪声下的过度自信问题提供原则基础。我们进一步设计了 NAACL，这是一个噪声感知校准框架，它综合了这些规则指导下的约 2K 个 HotpotQA 示例的监督。通过使用这些数据执行监督微调 (SFT)，NAACL 为模型配备了内在的噪声感知能力，而无需依赖更强大的教师模型。实证结果表明，NAACL 取得了显着的成果，将 ECE 分数在域内提高了 10.9%，在域外提高了 8.0%。通过弥合检索噪声和言语校准之间的差距，NAACL 为准确且认知上可靠的法学硕士铺平了道路。

Title: Finding the Translation Switch: Discovering and Exploiting the Task-Initiation Features in LLMs

• Authors: Xinwei Wu, Heng Liu, Xiaohu Zhao, Yuqi Ren, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo, Kaifu Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.11019
• Pdf URL: https://arxiv.org/pdf/2601.11019
• Copy Paste: [[2601.11019]] Finding the Translation Switch: Discovering and Exploiting the Task-Initiation Features in LLMs(https://arxiv.org/abs/2601.11019)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) frequently exhibit strong translation abilities, even without task-specific fine-tuning. However, the internal mechanisms governing this innate capability remain largely opaque. To demystify this process, we leverage Sparse Autoencoders (SAEs) and introduce a novel framework for identifying task-specific features. Our method first recalls features that are frequently co-activated on translation inputs and then filters them for functional coherence using a PCA-based consistency metric. This framework successfully isolates a small set of **translation initiation** features. Causal interventions demonstrate that amplifying these features steers the model towards correct translation, while ablating them induces hallucinations and off-task outputs, confirming they represent a core component of the model's innate translation competency. Moving from analysis to application, we leverage this mechanistic insight to propose a new data selection strategy for efficient fine-tuning. Specifically, we prioritize training on **mechanistically hard** samples-those that fail to naturally activate the translation initiation features. Experiments show this approach significantly improves data efficiency and suppresses hallucinations. Furthermore, we find these mechanisms are transferable to larger models of the same family. Our work not only decodes a core component of the translation mechanism in LLMs but also provides a blueprint for using internal model mechanism to create more robust and efficient models. The codes are available at this https URL.
• 摘要：即使没有针对特定任务的微调，大型语言模型 (LLM) 也经常表现出强大的翻译能力。然而，控制这种先天能力的内部机制在很大程度上仍然不透明。为了揭开这个过程的神秘面纱，我们利用稀疏自动编码器（SAE）并引入了一种新颖的框架来识别特定于任务的特征。我们的方法首先回顾翻译输入中经常共同激活的特征，然后使用基于 PCA 的一致性指标过滤它们以实现功能一致性。该框架成功地隔离了一小组**翻译启动**功能。因果干预表明，放大这些特征可以引导模型走向正确的翻译，而消除它们会引起幻觉和脱任务输出，从而确认它们代表了模型固有翻译能力的核心组成部分。从分析到应用，我们利用这种机械洞察力提出了一种新的数据选择策略，以进行有效的微调。具体来说，我们优先对**机械困难**样本进行训练——那些无法自然激活翻译启动功能的样本。实验表明，这种方法显着提高了数据效率并抑制了幻觉。此外，我们发现这些机制可以转移到同一系列的更大模型上。我们的工作不仅解码了法学硕士翻译机制的核心组成部分，而且还为使用内部模型机制创建更强大和更高效的模型提供了蓝图。这些代码可从此 https URL 获取。

Title: From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models

• Authors: Youmi Ma, Naoaki Okazaki
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11020
• Pdf URL: https://arxiv.org/pdf/2601.11020
• Copy Paste: [[2601.11020]] From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models(https://arxiv.org/abs/2601.11020)
• Keywords: language model, llm
• Abstract: Advances in mechanistic interpretability have identified special attention heads, known as retrieval heads, that are responsible for retrieving information from the context. However, the role of these retrieval heads in improving model performance remains unexplored. This work investigates whether retrieval heads can be leveraged to enhance the long-context capabilities of LLMs. Specifically, we propose RetMask, a method that generates training signals by contrasting normal model outputs with those from an ablated variant in which the retrieval heads are masked. This mechanism-based approach achieves substantial improvements: +2.28 points on HELMET at 128K for Llama-3.1, with +70% gains on generation with citation and +32% on passage re-ranking, while preserving performance on general tasks. Experiments across three model families reveal that the effectiveness depends on retrieval head organization: models with concentrated patterns of retrieval heads respond strongly, while those with distributed patterns show limited gains. This mechanistic relationship validates the function of retrieval heads and demonstrates that mechanistic insights can be transformed into performance enhancements.
• 摘要：机械可解释性的进步已经确定了特殊的注意头，称为检索头，负责从上下文中检索信息。然而，这些检索头在提高模型性能方面的作用仍有待探索。这项工作研究了是否可以利用检索头来增强法学硕士的长上下文能力。具体来说，我们提出了 RetMask，一种通过将正常模型输出与检索头被屏蔽的消融变体的输出进行对比来生成训练信号的方法。这种基于机制的方法实现了重大改进：Llama-3.1 在 128K 时的 HELMET 上提高了 2.28 分，引用生成提高了 70%，段落重新排名提高了 32%，同时保留了一般任务的性能。三个模型系列的实验表明，有效性取决于检索头组织：具有集中检索头模式的模型响应强烈，而具有分布式模式的模型则表现出有限的增益。这种机械关系验证了检索头的功能，并表明机械见解可以转化为性能增强。

Title: Budget-Aware Anytime Reasoning with LLM-Synthesized Preference Data

• Authors: Xuanming Zhang, Shwan Ashrafi, Aziza Mirsaidova, Amir Rezaeian, Miguel Ballesteros, Lydia B. Chilton, Zhou Yu, Dan Roth
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11038
• Pdf URL: https://arxiv.org/pdf/2601.11038
• Copy Paste: [[2601.11038]] Budget-Aware Anytime Reasoning with LLM-Synthesized Preference Data(https://arxiv.org/abs/2601.11038)
• Keywords: language model, gpt, llm
• Abstract: We study the reasoning behavior of large language models (LLMs) under limited computation budgets. In such settings, producing useful partial solutions quickly is often more practical than exhaustive reasoning, which incurs high inference costs. Many real-world tasks, such as trip planning, require models to deliver the best possible output within a fixed reasoning budget. We introduce an anytime reasoning framework and the Anytime Index, a metric that quantifies how effectively solution quality improves as reasoning tokens increase. To further enhance efficiency, we propose an inference-time self-improvement method using LLM-synthesized preference data, where models learn from their own reasoning comparisons to produce better intermediate solutions. Experiments on NaturalPlan (Trip), AIME, and GPQA datasets show consistent gains across Grok-3, GPT-oss, GPT-4.1/4o, and LLaMA models, improving both reasoning quality and efficiency under budget constraints.
• 摘要：我们研究有限计算预算下大型语言模型（LLM）的推理行为。在这种情况下，快速生成有用的部分解决方案通常比穷举推理更实用，因为穷举推理会带来高昂的推理成本。许多现实世界的任务，例如旅行计划，需要模型在固定的推理预算内提供最佳的输出。我们引入了随时推理框架和随时指数，该指标量化了随着推理代币的增加，解决方案质量如何有效提高。为了进一步提高效率，我们提出了一种使用 LLM 合成的偏好数据的推理时间自我改进方法，其中模型从自己的推理比较中学习以产生更好的中间解决方案。 NaturalPlan (Trip)、AIME 和 GPQA 数据集上的实验表明，Grok-3、GPT-oss、GPT-4.1/4o 和 LLaMA 模型都有一致的增益，在预算限制下提高了推理质量和效率。

Title: Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapse

• Authors: Chi Zhang, Mengqi Zhang, Xiaotian Ye, Runxi Cheng, Zisheng Zhou, Ying Zhou, Pengjie Ren, Zhumin Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.11042
• Pdf URL: https://arxiv.org/pdf/2601.11042
• Copy Paste: [[2601.11042]] Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapse(https://arxiv.org/abs/2601.11042)
• Keywords: language model
• Abstract: Sequential knowledge editing in large language models often causes catastrophic collapse of the model's general abilities, especially for parameter-modifying methods. Existing approaches mitigate this issue through heuristic constraints on parameter updates, yet the mechanisms underlying such degradation remain insufficiently understood. In this work, we present a spectral analysis of sequential knowledge editing and show that a model's general abilities are closely associated with dominant singular directions of pretrained weight matrices. These directions are highly sensitive to perturbations and are progressively disrupted by repeated edits, closely tracking the collapse in both editing efficacy and general performance. Building on this insight, we propose REVIVE, a plug-and-play framework that stabilizes sequential editing by explicitly preserving the dominant singular subspace. REVIVE represents parameter updates in the spectral basis of the original weights and filters components that would interfere with the protected region. Extensive experiments across multiple models and benchmarks show that REVIVE consistently improves editing efficacy while substantially preserving general abilities under long-horizon sequential editing, including extreme settings with up to 20,000 edits.
• 摘要：大型语言模型中的顺序知识编辑通常会导致模型一般能力的灾难性崩溃，特别是对于参数修改方法。现有的方法通过对参数更新的启发式约束来缓解这个问题，但这种退化背后的机制仍然没有得到充分的了解。在这项工作中，我们提出了顺序知识编辑的谱分析，并表明模型的一般能力与预训练权重矩阵的主导奇异方向密切相关。这些方向对扰动高度敏感，并且会因重复编辑而逐渐受到干扰，密切跟踪编辑效率和总体性能的崩溃。基于这一见解，我们提出了 REVIVE，这是一种即插即用框架，通过显式保留主要奇异子空间来稳定顺序编辑。 REVIVE 表示原始权重的频谱基础上的参数更新，并过滤掉会干扰受保护区域的组件。跨多个模型和基准的广泛实验表明，REVIVE 持续提高编辑效率，同时大幅保留长视野顺序编辑下的一般能力，包括高达 20,000 次编辑的极端设置。

Title: CoG: Controllable Graph Reasoning via Relational Blueprints and Failure-Aware Refinement over Knowledge Graphs

• Authors: Yuanxiang Liu, Songze Li, Xiaoke Guo, Zhaoyan Gong, Qifei Zhang, Huajun Chen, Wen Zhang
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.11047
• Pdf URL: https://arxiv.org/pdf/2601.11047
• Copy Paste: [[2601.11047]] CoG: Controllable Graph Reasoning via Relational Blueprints and Failure-Aware Refinement over Knowledge Graphs(https://arxiv.org/abs/2601.11047)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities but often grapple with reliability challenges like hallucinations. While Knowledge Graphs (KGs) offer explicit grounding, existing paradigms of KG-augmented LLMs typically exhibit cognitive rigidity--applying homogeneous search strategies that render them vulnerable to instability under neighborhood noise and structural misalignment leading to reasoning stagnation. To address these challenges, we propose CoG, a training-free framework inspired by Dual-Process Theory that mimics the interplay between intuition and deliberation. First, functioning as the fast, intuitive process, the Relational Blueprint Guidance module leverages relational blueprints as interpretable soft structural constraints to rapidly stabilize the search direction against noise. Second, functioning as the prudent, analytical process, the Failure-Aware Refinement module intervenes upon encountering reasoning impasses. It triggers evidence-conditioned reflection and executes controlled backtracking to overcome reasoning stagnation. Experimental results on three benchmarks demonstrate that CoG significantly outperforms state-of-the-art approaches in both accuracy and efficiency.
• 摘要：大型语言模型 (LLM) 已展现出卓越的推理能力，但经常面临诸如幻觉之类的可靠性挑战。虽然知识图（KG）提供了明确的基础，但知识图谱增强的法学硕士的现有范式通常表现出认知僵化——应用同质搜索策略，这使得它们在邻里噪音和结构失调的情况下容易受到不稳定的影响，从而导致推理停滞。为了应对这些挑战，我们提出了 CoG，这是一种受双过程理论启发的免训练框架，模仿直觉和深思熟虑之间的相互作用。首先，作为快速、直观的过程，关系蓝图指导模块利用关系蓝图作为可解释的软结构约束，以快速稳定搜索方向以对抗噪声。其次，作为审慎的分析过程，故障感知细化模块会在遇到推理僵局时进行干预。它触发证据条件反射并执行受控回溯以克服推理停滞。三个基准的实验结果表明，CoG 在准确性和效率方面都显着优于最先进的方法。

Title: Integrity Shield A System for Ethical AI Use & Authorship Transparency in Assessments

• Authors: Ashish Raj Shekhar, Shiven Agarwal, Priyanuj Bordoloi, Yash Shah, Tejas Anvekar, Vivek Gupta
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11093
• Pdf URL: https://arxiv.org/pdf/2601.11093
• Copy Paste: [[2601.11093]] Integrity Shield A System for Ethical AI Use & Authorship Transparency in Assessments(https://arxiv.org/abs/2601.11093)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) can now solve entire exams directly from uploaded PDF assessments, raising urgent concerns about academic integrity and the reliability of grades and credentials. Existing watermarking techniques either operate at the token level or assume control over the model's decoding process, making them ineffective when students query proprietary black-box systems with instructor-provided documents. We present Integrity Shield, a document-layer watermarking system that embeds schema-aware, item-level watermarks into assessment PDFs while keeping their human-visible appearance unchanged. These watermarks consistently prevent MLLMs from answering shielded exam PDFs and encode stable, item-level signatures that can be reliably recovered from model or student responses. Across 30 exams spanning STEM, humanities, and medical reasoning, Integrity Shield achieves exceptionally high prevention (91-94% exam-level blocking) and strong detection reliability (89-93% signature retrieval) across four commercial MLLMs. Our demo showcases an interactive interface where instructors upload an exam, preview watermark behavior, and inspect pre/post AI performance & authorship evidence.
• 摘要：大型语言模型 (LLM) 现在可以直接从上传的 PDF 评估中解决整个考试，这引发了人们对学术诚信以及成绩和证书可靠性的迫切担忧。现有的水印技术要么在令牌级别运行，要么承担对模型解码过程的控制，当学生使用教师提供的文档查询专有黑匣子系统时，它们会变得无效。我们推出了 Integrity Shield，这是一种文档层水印系统，可将模式感知、项目级水印嵌入到评估 PDF 中，同时保持其人类可见的外观不变。这些水印始终阻止 MLLM 回答受屏蔽的考试 PDF，并对稳定的项目级签名进行编码，这些签名可以从模型或学生的回答中可靠地恢复。在涵盖 STEM、人文学科和医学推理的 30 项考试中，Integrity Shield 在四个商业 MLLM 中实现了极高的预防性（91-94% 考试级别阻止）和强大的检测可靠性（89-93% 签名检索）。我们的演示展示了一个交互式界面，教师可以在其中上传考试、预览水印行为以及检查前/后人工智能性能和作者身份证据。

Title: T$^\star$: Progressive Block Scaling for MDM Through Trajectory Aware RL

• Authors: Hanchen Xia, Baoyou Chen, Yutang Ge, Guojiang Zhao, Siyu Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11214
• Pdf URL: https://arxiv.org/pdf/2601.11214
• Copy Paste: [[2601.11214]] T$^\star$: Progressive Block Scaling for MDM Through Trajectory Aware RL(https://arxiv.org/abs/2601.11214)
• Keywords: language model
• Abstract: We present T$^\star$, a simple \textsc{TraceRL}-based training curriculum for progressive block-size scaling in masked diffusion language models (MDMs). Starting from an AR-initialized small-block MDM, T$^\star$~transitions smoothly to larger blocks, enabling higher-parallelism decoding with minimal performance degradation on math reasoning benchmarks. Moreover, further analysis suggests that T$^\star$~can converge to an alternative decoding schedule $\hat{\rm S}$ that achieves comparable performance.
• 摘要：我们提出了 T$^\star$，一个简单的基于 \textsc{TraceRL} 的培训课程，用于在掩码扩散语言模型（MDM）中渐进式块大小缩放。从 AR 初始化的小块 MDM 开始，T$^\star$~ 平滑过渡到更大的块，从而实现更高并行度的解码，同时在数学推理基准上将性能下降降至最低。此外，进一步的分析表明 T$^\star$~ 可以收敛到替代解码方案 $\hat{\rm S}$，从而实现可比的性能。

Title: MultiCaption: Detecting disinformation using multilingual visual claims

• Authors: Rafael Martins Frade, Rrubaa Panchendrarajan, Arkaitz Zubiaga
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11220
• Pdf URL: https://arxiv.org/pdf/2601.11220
• Copy Paste: [[2601.11220]] MultiCaption: Detecting disinformation using multilingual visual claims(https://arxiv.org/abs/2601.11220)
• Keywords: language model
• Abstract: Online disinformation poses an escalating threat to society, driven increasingly by the rapid spread of misleading content across both multimedia and multilingual platforms. While automated fact-checking methods have advanced in recent years, their effectiveness remains constrained by the scarcity of datasets that reflect these real-world complexities. To address this gap, we first present MultiCaption, a new dataset specifically designed for detecting contradictions in visual claims. Pairs of claims referring to the same image or video were labeled through multiple strategies to determine whether they contradict each other. The resulting dataset comprises 11,088 visual claims in 64 languages, offering a unique resource for building and evaluating misinformation-detection systems in truly multimodal and multilingual environments. We then provide comprehensive experiments using transformer-based architectures, natural language inference models, and large language models, establishing strong baselines for future research. The results show that MultiCaption is more challenging than standard NLI tasks, requiring task-specific finetuning for strong performance. Moreover, the gains from multilingual training and testing highlight the dataset's potential for building effective multilingual fact-checking pipelines without relying on machine translation.
• 摘要：由于误导性内容在多媒体和多语言平台上的快速传播，在线虚假信息对社会构成了不断升级的威胁。尽管近年来自动事实核查方法取得了进步，但其有效性仍然受到反映现实世界复杂性的数据集稀缺的限制。为了解决这一差距，我们首先提出 MultiCaption，这是一个专门为检测视觉声明中的矛盾而设计的新数据集。通过多种策略对引用相同图像或视频的声明对进行标记，以确定它们是否相互矛盾。生成的数据集包含 64 种语言的 11,088 个视觉声明，为在真正的多模式和多语言环境中构建和评估错误信息检测系统提供了独特的资源。然后，我们使用基于 Transformer 的架构、自然语言推理模型和大型语言模型进行全面的实验，为未来的研究建立强有力的基线。结果表明，MultiCaption 比标准 NLI 任务更具挑战性，需要针对特定​​任务进行微调才能获得强大的性能。此外，多语言训练和测试的收益凸显了该数据集在不依赖机器翻译的情况下构建有效的多语言事实检查管道的潜力。

Title: Language of Thought Shapes Output Diversity in Large Language Models

• Authors: Shaoyang Xu, Wenxuan Zhang
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2601.11227
• Pdf URL: https://arxiv.org/pdf/2601.11227
• Copy Paste: [[2601.11227]] Language of Thought Shapes Output Diversity in Large Language Models(https://arxiv.org/abs/2601.11227)
• Keywords: language model, llm
• Abstract: Output diversity is crucial for Large Language Models as it underpins pluralism and creativity. In this work, we reveal that controlling the language used during model thinking-the language of thought-provides a novel and structural source of output diversity. Our preliminary study shows that different thinking languages occupy distinct regions in a model's thinking space. Based on this observation, we study two repeated sampling strategies under multilingual thinking-Single-Language Sampling and Mixed-Language Sampling-and conduct diversity evaluation on outputs that are controlled to be in English, regardless of the thinking language used. Across extensive experiments, we demonstrate that switching the thinking language from English to non-English languages consistently increases output diversity, with a clear and consistent positive correlation such that languages farther from English in the thinking space yield larger gains. We further show that aggregating samples across multiple thinking languages yields additional improvements through compositional effects, and that scaling sampling with linguistic heterogeneity expands the model's diversity ceiling. Finally, we show that these findings translate into practical benefits in pluralistic alignment scenarios, leading to broader coverage of cultural knowledge and value orientations in LLM outputs. Our code is publicly available at this https URL.
• 摘要：输出多样性对于大型语言模型至关重要，因为它支撑着多元化和创造力。在这项工作中，我们揭示了控制模型思维过程中使用的语言（思维语言）提供了输出多样性的新颖的结构性来源。我们的初步研究表明，不同的思维语言在模型的思维空间中占据不同的区域。基于这一观察，我们研究了多语言思维下的两种重复抽样策略——单一语言抽样和混合语言抽样——并对控制为英语的输出进行多样性评估，无论使用何种思维语言。通过大量的实验，我们证明，将思维语言从英语切换到非英语语言会持续增加输出多样性，并具有明显且一致的正相关性，使得思维空间中远离英语的语言会产生更大的收益。我们进一步表明，跨多种思维语言聚合样本可以通过组合效应产生额外的改进，并且具有语言异质性的缩放采样扩大了模型的多样性上限。最后，我们表明这些发现可以转化为多元联盟场景中的实际效益，从而导致法学硕士输出中文化知识和价值取向的更广泛覆盖。我们的代码可通过此 https URL 公开获取。

Title: FactCorrector: A Graph-Inspired Approach to Long-Form Factuality Correction of Large Language Models

• Authors: Javier Carnerero-Cano, Massimiliano Pronesti, Radu Marinescu, Tigran Tchrakian, James Barry, Jasmina Gajcin, Yufang Hou, Alessandra Pascale, Elizabeth Daly
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.11232
• Pdf URL: https://arxiv.org/pdf/2601.11232
• Copy Paste: [[2601.11232]] FactCorrector: A Graph-Inspired Approach to Long-Form Factuality Correction of Large Language Models(https://arxiv.org/abs/2601.11232)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are widely used in knowledge-intensive applications but often generate factually incorrect responses. A promising approach to rectify these flaws is correcting LLMs using feedback. Therefore, in this paper, we introduce FactCorrector, a new post-hoc correction method that adapts across domains without retraining and leverages structured feedback about the factuality of the original response to generate a correction. To support rigorous evaluations of factuality correction methods, we also develop the VELI5 benchmark, a novel dataset containing systematically injected factual errors and ground-truth corrections. Experiments on VELI5 and several popular long-form factuality datasets show that the FactCorrector approach significantly improves factual precision while preserving relevance, outperforming strong baselines. We release our code at this https URL.
• 摘要：大型语言模型 (LLM) 广泛用于知识密集型应用程序，但经常生成事实上不正确的响应。纠正这些缺陷的一个有希望的方法是使用反馈来纠正法学硕士。因此，在本文中，我们引入了 FactCorrector，这是一种新的事后校正方法，无需重新训练即可跨领域适应，并利用有关原始响应真实性的结构化反馈来生成校正。为了支持对事实校正方法的严格评估，我们还开发了 VELI5 基准，这是一个包含系统注入的事实错误和真实校正的新颖数据集。在 VELI5 和几个流行的长格式事实数据集上进行的实验表明，FactCorrector 方法在保持相关性的同时显着提高了事实精度，优于强大的基线。我们在此 https URL 发布我们的代码。

Title: How DDAIR you? Disambiguated Data Augmentation for Intent Recognition

• Authors: Galo Castillo-López, Alexis Lombard, Nasredine Semmar, Gaël de Chalendar
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.11234
• Pdf URL: https://arxiv.org/pdf/2601.11234
• Copy Paste: [[2601.11234]] How DDAIR you? Disambiguated Data Augmentation for Intent Recognition(https://arxiv.org/abs/2601.11234)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are effective for data augmentation in classification tasks like intent detection. In some cases, they inadvertently produce examples that are ambiguous with regard to untargeted classes. We present DDAIR (Disambiguated Data Augmentation for Intent Recognition) to mitigate this problem. We use Sentence Transformers to detect ambiguous class-guided augmented examples generated by LLMs for intent recognition in low-resource scenarios. We identify synthetic examples that are semantically more similar to another intent than to their target one. We also provide an iterative re-generation method to mitigate such ambiguities. Our findings show that sentence embeddings effectively help to (re)generate less ambiguous examples, and suggest promising potential to improve classification performance in scenarios where intents are loosely or broadly defined.
• 摘要：大型语言模型 (LLM) 对于意图检测等分类任务中的数据增强非常有效。在某些情况下，他们会无意中生成对于非目标类来说不明确的示例。我们提出了 DDAIR（用于意图识别的消歧数据增强）来缓解这个问题。我们使用句子转换器来检测由法学硕士生成的模糊类引导增强示例，以在资源匮乏的情况下进行意图识别。我们识别出在语义上与另一种意图比与目标意图更相似的合成示例。我们还提供了一种迭代重新生成方法来减轻这种歧义。我们的研究结果表明，句子嵌入有效地帮助（重新）生成不太模糊的示例，并表明在意图松散或广泛定义的场景中提高分类性能的潜力。

Title: Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering

• Authors: Yuling Shi, Maolin Sun, Zijun Liu, Mo Yang, Yixiong Fang, Tianran Sun, Xiaodong Gu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.11255
• Pdf URL: https://arxiv.org/pdf/2601.11255
• Copy Paste: [[2601.11255]] Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering(https://arxiv.org/abs/2601.11255)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) has demonstrated significant effectiveness in enhancing large language models (LLMs) for complex multi-hop question answering (QA). For multi-hop QA tasks, current iterative approaches predominantly rely on LLMs to self-guide and plan multi-step exploration paths during retrieval, leading to substantial challenges in maintaining reasoning coherence across steps from inaccurate query decomposition and error propagation. To address these issues, we introduce Reasoning Tree Guided RAG (RT-RAG), a novel hierarchical framework for complex multi-hop QA. RT-RAG systematically decomposes multi-hop questions into explicit reasoning trees, minimizing inaccurate decomposition through structured entity analysis and consensus-based tree selection that clearly separates core queries, known entities, and unknown entities. Subsequently, a bottom-up traversal strategy employs iterative query rewriting and refinement to collect high-quality evidence, thereby mitigating error propagation. Comprehensive experiments show that RT-RAG substantially outperforms state-of-the-art methods by 7.0% F1 and 6.0% EM, demonstrating the effectiveness of RT-RAG in complex multi-hop QA.
• 摘要：检索增强生成 (RAG) 在增强复杂多跳问答 (QA) 的大型语言模型 (LLM) 方面已表现出显着的有效性。对于多跳 QA 任务，当前的迭代方法主要依靠 LLM 在检索过程中自我指导和规划多步骤探索路径，从而导致在保持不准确的查询分解和错误传播的步骤之间的推理一致性方面面临巨大挑战。为了解决这些问题，我们引入了推理树引导 RAG (RT-RAG)，这是一种用于复杂多跳 QA 的新型分层框架。 RT-RAG 将多跳问题系统地分解为显式推理树，通过结构化实体分析和基于共识的树选择来最大限度地减少不准确的分解，从而明确区分核心查询、已知实体和未知实体。随后，自下而上的遍历策略采用迭代查询重写和细化来收集高质量的证据，从而减少错误传播。综合实验表明，RT-RAG 的 F1 性能和 EM 性能分别比最先进的方法高出 7.0% 和 6.0%，从而证明了 RT-RAG 在复杂多跳 QA 中的有效性。

Title: One LLM to Train Them All: Multi-Task Learning Framework for Fact-Checking

• Authors: Malin Astrid Larsson, Harald Fosen Grunnaleite, Vinay Setty
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11293
• Pdf URL: https://arxiv.org/pdf/2601.11293
• Copy Paste: [[2601.11293]] One LLM to Train Them All: Multi-Task Learning Framework for Fact-Checking(https://arxiv.org/abs/2601.11293)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are reshaping automated fact-checking (AFC) by enabling unified, end-to-end verification pipelines rather than isolated components. While large proprietary models achieve strong performance, their closed weights, complexity, and high costs limit sustainability. Fine-tuning smaller open weight models for individual AFC tasks can help but requires multiple specialized models resulting in high costs. We propose \textbf{multi-task learning (MTL)} as a more efficient alternative that fine-tunes a single model to perform claim detection, evidence ranking, and stance detection jointly. Using small decoder-only LLMs (e.g., Qwen3-4b), we explore three MTL strategies: classification heads, causal language modeling heads, and instruction-tuning, and evaluate them across model sizes, task orders, and standard non-LLM baselines. While multitask models do not universally surpass single-task baselines, they yield substantial improvements, achieving up to \textbf{44\%}, \textbf{54\%}, and \textbf{31\%} relative gains for claim detection, evidence re-ranking, and stance detection, respectively, over zero-/few-shot settings. Finally, we also provide practical, empirically grounded guidelines to help practitioners apply MTL with LLMs for automated fact-checking.
• 摘要：大型语言模型 (LLM) 正在通过启用统一的端到端验证管道而不是孤立的组件来重塑自动事实检查 (AFC)。虽然大型专有模型具有强大的性能，但其封闭的重量、复杂性和高成本限制了可持续性。针对单个 AFC 任务微调较小的开放权重模型可能会有所帮助，但需要多个专用模型，从而导致成本高昂。我们提出 \textbf{多任务学习（MTL）} 作为一种更有效的替代方案，可以微调单个模型以联合执行主张检测、证据排名和立场检测。使用小型仅解码器的 LLM（例如 Qwen3-4b），我们探索了三种 MTL 策略：分类头、因果语言建模头和指令调整，并跨模型大小、任务顺序和标准非 LLM 基线对其进行评估。虽然多任务模型并没有普遍超越单任务基线，但它们产生了实质性的改进，在零/少样本设置下，在主张检测、证据重新排序和立场检测方面分别实现了 \textbf{44\%}、\textbf{54\%} 和 \textbf{31\%} 相对增益。最后，我们还提供实用的、以经验为基础的指南，帮助从业者将 MTL 与法学硕士一起应用于自动事实核查。

Title: Membership Inference on LLMs in the Wild

• Authors: Jiatong Yi, Yanyang Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11314
• Pdf URL: https://arxiv.org/pdf/2601.11314
• Copy Paste: [[2601.11314]] Membership Inference on LLMs in the Wild(https://arxiv.org/abs/2601.11314)
• Keywords: language model, llm
• Abstract: Membership Inference Attacks (MIAs) act as a crucial auditing tool for the opaque training data of Large Language Models (LLMs). However, existing techniques predominantly rely on inaccessible model internals (e.g., logits) or suffer from poor generalization across domains in strict black-box settings where only generated text is available. In this work, we propose SimMIA, a robust MIA framework tailored for this text-only regime by leveraging an advanced sampling strategy and scoring mechanism. Furthermore, we present WikiMIA-25, a new benchmark curated to evaluate MIA performance on modern proprietary LLMs. Experiments demonstrate that SimMIA achieves state-of-the-art results in the black-box setting, rivaling baselines that exploit internal model information.
• 摘要：成员推理攻击 (MIA) 是大型语言模型 (LLM) 不透明训练数据的重要审计工具。然而，现有技术主要依赖于不可访问的模型内部结构（例如，logits），或者在严格的黑盒设置中跨域泛化能力差，其中仅生成文本可用。在这项工作中，我们提出了 SimMIA，这是一个通过利用先进的采样策略和评分机制，为这种纯文本机制量身定制的强大的 MIA 框架。此外，我们还推出了 WikiMIA-25，这是一个新的基准，旨在评估现代专有法学硕士的 MIA 性能。实验表明，SimMIA 在黑盒设置中实现了最先进的结果，可与利用内部模型信息的基线相媲美。

Title: F-Actor: Controllable Conversational Behaviour in Full-Duplex Models

• Authors: Maike Züfle, Ondrej Klejch, Nicholas Sanders, Jan Niehues, Alexandra Birch, Tsz Kin Lam
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11329
• Pdf URL: https://arxiv.org/pdf/2601.11329
• Copy Paste: [[2601.11329]] F-Actor: Controllable Conversational Behaviour in Full-Duplex Models(https://arxiv.org/abs/2601.11329)
• Keywords: language model
• Abstract: Spoken conversational systems require more than accurate speech generation to have human-like conversations: to feel natural and engaging, they must produce conversational behaviour that adapts dynamically to the context. Current spoken conversational systems, however, rarely allow such customization, limiting their naturalness and usability. In this work, we present the first open, instruction-following full-duplex conversational speech model that can be trained efficiently under typical academic resource constraints. By keeping the audio encoder frozen and finetuning only the language model, our model requires just 2,000 hours of data, without relying on large-scale pretraining or multi-stage optimization. The model can follow explicit instructions to control speaker voice, conversation topic, conversational behaviour (e.g., backchanneling and interruptions), and dialogue initiation. We propose a single-stage training protocol and systematically analyze design choices. Both the model and training code will be released to enable reproducible research on controllable full-duplex speech systems.
• 摘要：口语会话系统需要的不仅仅是准确的语音生成才能进行类似人类的对话：为了感觉自然且引人入胜，它们必须产生动态适应上下文的会话行为。然而，当前的口语会话系统很少允许这种定制，限制了它们的自然性和可用性。在这项工作中，我们提出了第一个开放的、遵循指令的全双工会话语音模型，该模型可以在典型的学术资源限制下进行有效的训练。通过保持音频编码器冻结并仅微调语言模型，我们的模型仅需要 2,000 小时的数据，而不依赖于大规模预训练或多阶段优化。该模型可以遵循明确的指令来控制说话者的声音、对话主题、对话行为（例如，反向渠道和中断）和对话发起。我们提出了一个单阶段训练方案并系统地分析了设计选择。模型和训练代码都将发布，以便能够对可控全双工语音系统进行可重复的研究。

Title: Idea First, Code Later: Disentangling Problem Solving from Code Generation in Evaluating LLMs for Competitive Programming

• Authors: Sama Hadhoud, Alaa Elsetohy, Frederikus Hudi, Jan Christian Blaise Cruz, Steven Halim, Alham Fikri Aji
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11332
• Pdf URL: https://arxiv.org/pdf/2601.11332
• Copy Paste: [[2601.11332]] Idea First, Code Later: Disentangling Problem Solving from Code Generation in Evaluating LLMs for Competitive Programming(https://arxiv.org/abs/2601.11332)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) increasingly succeed on competitive programming problems, yet existing evaluations conflate algorithmic reasoning with code-level implementation. We argue that competitive programming is fundamentally a problem-solving task and propose centering natural-language editorials in both solution generation and evaluation. Generating an editorial prior to code improves solve rates for some LLMs, with substantially larger gains when using expertly written gold editorials. However, even with gold editorials, models continue to struggle with implementation, while the gap between generated and gold editorials reveals a persistent problem-solving bottleneck in specifying correct and complete algorithms. Beyond pass/fail metrics, we diagnose reasoning errors by comparing model-generated editorials to gold standards using expert annotations and validate an LLM-as-a-judge protocol for scalable evaluation. We introduce a dataset of 83 ICPC-style problems with gold editorials and full test suites, and evaluate 19 LLMs, arguing that future benchmarks should explicitly separate problem solving from implementation.
• 摘要：大型语言模型 (LLM) 在竞争性编程问题上越来越成功，但现有的评估将算法推理与代码级实现混为一谈。我们认为，竞争性编程从根本上来说是一项解决问题的任务，并建议将自然语言社论集中在解决方案生成和评估中。在编码之前生成社论可以提高一些法学硕士的解决率，当使用专业撰写的黄金社论时，可以获得更大的收益。然而，即使有了黄金社论，模型仍然难以实现，而生成的社论和黄金社论之间的差距揭示了在指定正确和完整的算法方面持续存在的解决问题的瓶颈。除了通过/失败指标之外，我们还通过使用专家注释将模型生成的社论与黄金标准进行比较来诊断推理错误，并验证法学硕士作为法官的协议以进行可扩展的评估。我们引入了包含 83 个 ICPC 风格问题的数据集，其中包含黄金社论和完整的测试套件，并评估了 19 个法学硕士，认为未来的基准测试应该明确地将问题解决与实施分开。

Title: Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models

• Authors: Guoming Ling, Zhongzhan Huang, Yupei Lin, Junxin Li, Shanshan Zhong, Hefeng Wu, Liang Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11340
• Pdf URL: https://arxiv.org/pdf/2601.11340
• Copy Paste: [[2601.11340]] Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models(https://arxiv.org/abs/2601.11340)
• Keywords: language model, chain-of-thought
• Abstract: Chain-of-Thought reasoning has significantly enhanced the problem-solving capabilities of Large Language Models. Unfortunately, current models generate reasoning steps sequentially without foresight, often becoming trapped in suboptimal reasoning paths with redundant steps. In contrast, we introduce Neural Chain-of-Thought Search (NCoTS), a framework that reformulates reasoning as a dynamic search for the optimal thinking strategy. By quantitatively characterizing the solution space, we reveal the existence of sparse superior reasoning paths that are simultaneously more accurate and concise than standard outputs. Our method actively navigates towards these paths by evaluating candidate reasoning operators using a dual-factor heuristic that optimizes for both correctness and computational cost. Consequently, NCoTS achieves a Pareto improvement across diverse reasoning benchmarks, boosting accuracy by over 3.5% while reducing generation length by over 22%. Our code and data are available at this https URL.
• 摘要：思想链推理显着增强了大型语言模型解决问题的能力。不幸的是，当前的模型在没有预见性的情况下按顺序生成推理步骤，常常陷入具有冗余步骤的次优推理路径中。相比之下，我们引入了神经思维链搜索（NCoTS），这是一个将推理重新表述为对最佳思维策略的动态搜索的框架。通过定量表征解空间，我们揭示了稀疏优越推理路径的存在，该路径同时比标准输出更准确和简洁。我们的方法通过使用双因素启发式评估候选推理运算符来主动导航到这些路径，该双因素启发式优化了正确性和计算成本。因此，NCoTS 在各种推理基准上实现了帕累托改进，将准确性提高了 3.5% 以上，同时将生成长度缩短了 22% 以上。我们的代码和数据可在此 https URL 中获取。

Title: How Much Would a Clinician Edit This Draft? Evaluating LLM Alignment for Patient Message Response Drafting

• Authors: Parker Seegmiller, Joseph Gatto, Sarah E. Greer, Ganza Belise Isingizwe, Rohan Ray, Timothy E. Burdick, Sarah Masud Preum
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.11344
• Pdf URL: https://arxiv.org/pdf/2601.11344
• Copy Paste: [[2601.11344]] How Much Would a Clinician Edit This Draft? Evaluating LLM Alignment for Patient Message Response Drafting(https://arxiv.org/abs/2601.11344)
• Keywords: language model, llm, prompt, retrieval-augmented generation
• Abstract: Large language models (LLMs) show promise in drafting responses to patient portal messages, yet their integration into clinical workflows raises various concerns, including whether they would actually save clinicians time and effort in their portal workload. We investigate LLM alignment with individual clinicians through a comprehensive evaluation of the patient message response drafting task. We develop a novel taxonomy of thematic elements in clinician responses and propose a novel evaluation framework for assessing clinician editing load of LLM-drafted responses at both content and theme levels. We release an expert-annotated dataset and conduct large-scale evaluations of local and commercial LLMs using various adaptation techniques including thematic prompting, retrieval-augmented generation, supervised fine-tuning, and direct preference optimization. Our results reveal substantial epistemic uncertainty in aligning LLM drafts with clinician responses. While LLMs demonstrate capability in drafting certain thematic elements, they struggle with clinician-aligned generation in other themes, particularly question asking to elicit further information from patients. Theme-driven adaptation strategies yield improvements across most themes. Our findings underscore the necessity of adapting LLMs to individual clinician preferences to enable reliable and responsible use in patient-clinician communication workflows.
• 摘要：大型语言模型 (LLM) 在起草对患者门户消息的响应方面表现出了良好的前景，但它们与临床工作流程的集成引起了各种担忧，包括它们是否真的会节省临床医生在门户工作负载中的时间和精力。我们通过对患者信息回复起草任务的综合评估来调查法学硕士与个体临床医生的一致性。我们开发了一种新颖的临床医生回复中主题元素的分类法，并提出了一种新颖的评估框架，用于评估临床医生在内容和主题层面对法学硕士起草的回复的编辑负担。我们发布了专家注释的数据集，并使用各种适应技术对本地和商业法学硕士进行大规模评估，包括主题提示、检索增强生成、监督微调和直接偏好优化。我们的结果揭示了法学硕士草案与临床医生反应之间存在很大的认知不确定性。虽然法学硕士在起草某些主题元素方面表现出了能力，但他们在其他主题上很难与临床医生保持一致，特别是通过提问来从患者那里获取更多信息。主题驱动的适应策略可以在大多数主题上带来改进。我们的研究结果强调了使法学硕士适应临床医生个人偏好的必要性，以便在患者与临床医生的沟通工作流程中实现可靠和负责任的使用。

Title: Reward Modeling for Scientific Writing Evaluation

• Authors: Furkan Şahinuç, Subhabrata Dutta, Iryna Gurevych
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11374
• Pdf URL: https://arxiv.org/pdf/2601.11374
• Copy Paste: [[2601.11374]] Reward Modeling for Scientific Writing Evaluation(https://arxiv.org/abs/2601.11374)
• Keywords: llm
• Abstract: Scientific writing is an expert-domain task that demands deep domain knowledge, task-specific requirements and reasoning capabilities that leverage the domain knowledge to satisfy the task specifications. While scientific text generation has been widely studied, its evaluation remains a challenging and open problem. It is critical to develop models that can be reliably deployed for evaluating diverse open-ended scientific writing tasks while adhering to their distinct requirements. However, existing LLM-based judges and reward models are primarily optimized for general-purpose benchmarks with fixed scoring rubrics and evaluation criteria. Consequently, they often fail to reason over sparse knowledge of scientific domains when interpreting task-dependent and multi-faceted criteria. Moreover, fine-tuning for each individual task is costly and impractical for low-resource settings. To bridge these gaps, we propose cost-efficient, open-source reward models tailored for scientific writing evaluation. We introduce a two-stage training framework that initially optimizes scientific evaluation preferences and then refines reasoning capabilities. Our multi-aspect evaluation design and joint training across diverse tasks enable fine-grained assessment and robustness to dynamic criteria and scoring rubrics. Experimental analysis shows that our training regime strongly improves LLM-based scientific writing evaluation. Our models generalize effectively across tasks and to previously unseen scientific writing evaluation settings, allowing a single trained evaluator to be reused without task-specific retraining.
• 摘要：科学写作是一项专家领域任务，需要深厚的领域知识、特定于任务的要求以及利用领域知识来满足任务规范的推理能力。尽管科学文本生成已被广泛研究，但其评估仍然是一个具有挑战性和开放性的问题。开发可以可靠部署的模型来评估各种开放式科学写作任务，同时遵守其独特的要求，这一点至关重要。然而，现有的基于法学硕士的法官和奖励模型主要针对具有固定评分标准和评估标准的通用基准进行优化。因此，在解释任务相关和多方面的标准时，他们常常无法对科学领域知识稀疏的情况进行推理。此外，对于资源匮乏的环境来说，对每个单独的任务进行微调成本高昂且不切实际。为了弥补这些差距，我们提出了专为科学写作评估量身定制的经济高效的开源奖励模型。我们引入了一个两阶段的训练框架，首先优化科学评估偏好，然后细化推理能力。我们的多方面评估设计和跨不同任务的联合培训能够实现细粒度的评估以及动态标准和评分标准的稳健性。实验分析表明，我们的培训制度极大地提高了基于法学硕士的科学写作评估。我们的模型可以有效地跨任务和以前未见过的科学写作评估设置进行推广，允许重复使用单个训练有素的评估者，而无需针对特定任务进行再培训。

Title: Evaluating LLM Behavior in Hiring: Implicit Weights, Fairness Across Groups, and Alignment with Human Preferences

• Authors: Morgane Hoffmann, Emma Jouffroy, Warren Jouanneau, Marc Palyart, Charles Pebereau
• Subjects: cs.CL, cs.AI, cs.CY, cs.SI
• Abstract URL: https://arxiv.org/abs/2601.11379
• Pdf URL: https://arxiv.org/pdf/2601.11379
• Copy Paste: [[2601.11379]] Evaluating LLM Behavior in Hiring: Implicit Weights, Fairness Across Groups, and Alignment with Human Preferences(https://arxiv.org/abs/2601.11379)
• Keywords: language model, llm
• Abstract: General-purpose Large Language Models (LLMs) show significant potential in recruitment applications, where decisions require reasoning over unstructured text, balancing multiple criteria, and inferring fit and competence from indirect productivity signals. Yet, it is still uncertain how LLMs assign importance to each attribute and whether such assignments are in line with economic principles, recruiter preferences or broader societal norms. We propose a framework to evaluate an LLM's decision logic in recruitment, by drawing on established economic methodologies for analyzing human hiring behavior. We build synthetic datasets from real freelancer profiles and project descriptions from a major European online freelance marketplace and apply a full factorial design to estimate how a LLM weighs different match-relevant criteria when evaluating freelancer-project fit. We identify which attributes the LLM prioritizes and analyze how these weights vary across project contexts and demographic subgroups. Finally, we explain how a comparable experimental setup could be implemented with human recruiters to assess alignment between model and human decisions. Our findings reveal that the LLM weighs core productivity signals, such as skills and experience, but interprets certain features beyond their explicit matching value. While showing minimal average discrimination against minority groups, intersectional effects reveal that productivity signals carry different weights between demographic groups.
• 摘要：通用大型语言模型 (LLM) 在招聘应用中显示出巨大的潜力，其中决策需要对非结构化文本进行推理、平衡多个标准，并从间接生产力信号中推断适合性和能力。然而，目前仍不确定法学硕士如何分配每个属性的重要性，以及此类分配是否符合经济原则、招聘人员偏好或更广泛的社会规范。我们提出了一个框架，通过利用现有的经济方法来分析人类招聘行为，来评估法学硕士在招聘中的决策逻辑。我们根据来自欧洲主要在线自由职业市场的真实自由职业者资料和项目描述构建综合数据集，并应用完整的因子设计来估计法学硕士在评估自由职业者项目契合度时如何权衡不同的匹配相关标准。我们确定法学硕士优先考虑的属性，并分析这些权重在项目背景和人口亚组之间的变化。最后，我们解释了如何与人类招聘人员一起实施可比较的实验设置，以评估模型和人类决策之间的一致性。我们的研究结果表明，法学硕士会权衡核心生产力信号，例如技能和经验，但对某些特征的解释超出了其明确的匹配价值。虽然对少数群体的平均歧视程度最低，但交叉效应表明，生产力信号在人口群体之间具有不同的权重。

Title: Relational Linearity is a Predictor of Hallucinations

• Authors: Yuetian Lu, Yihong Liu, Hinrich Schütze
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.11429
• Pdf URL: https://arxiv.org/pdf/2601.11429
• Copy Paste: [[2601.11429]] Relational Linearity is a Predictor of Hallucinations(https://arxiv.org/abs/2601.11429)
• Keywords: language model, llm, hallucination
• Abstract: Hallucination is a central failure mode in large language models (LLMs). We focus on hallucinations of answers to questions like: "Which instrument did Glenn Gould play?", but we ask these questions for synthetic entities that are unknown to the model. Surprisingly, we find that medium-size models like Gemma-7B-IT frequently hallucinate, i.e., they have difficulty recognizing that the hallucinated fact is not part of their knowledge. We hypothesize that an important factor in causing these hallucinations is the linearity of the relation: linear relations tend to be stored more abstractly, making it difficult for the LLM to assess its knowledge; the facts of nonlinear relations tend to be stored more directly, making knowledge assessment easier. To investigate this hypothesis, we create SyntHal, a dataset of 6000 synthetic entities for six relations. In our experiments with four models, we determine, for each relation, the hallucination rate on SyntHal and also measure its linearity, using $\Delta\cos$. We find a strong correlation ($r \in [.78,.82]$) between relational linearity and hallucination rate, providing evidence for our hypothesis that the underlying storage of triples of a relation is a factor in how well a model can self-assess its knowledge. This finding has implications for how to manage hallucination behavior and suggests new research directions for improving the representation of factual knowledge in LLMs.
• 摘要：幻觉是大型语言模型 (LLM) 中的主要失败模式。我们专注于对诸如“格伦·古尔德演奏哪种乐器？”之类的问题的答案的幻觉，但我们针对模型未知的合成实体提出这些问题。令人惊讶的是，我们发现像 Gemma-7B-IT 这样的中型模型经常产生幻觉，也就是说，它们很难认识到幻觉的事实不是他们知识的一部分。我们假设引起这些幻觉的一个重要因素是关系的线性：线性关系往往存储得更加抽象，使得法学硕士很难评估其知识；非线性关系的事实往往更直接地存储，使得知识评估更容易。为了研究这个假设，我们创建了 SyntHal，这是一个包含六种关系的 6000 个合成实体的数据集。在我们使用四个模型的实验中，我们为每个关系确定 SyntHal 上的幻觉率，并使用 $\Delta\cos$ 测量其线性度。我们发现关系线性和幻觉率之间存在很强的相关性（$r \in [.78,.82]$），这为我们的假设提供了证据，即关系三元组的基础存储是模型自我评估其知识程度的一个因素。这一发现对于如何管理幻觉行为具有重要意义，并为改善法学硕士事实知识的表达提出了新的研究方向。

Title: The unreasonable effectiveness of pattern matching

• Authors: Gary Lupyan, Blaise Agüera y Arcas
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11432
• Pdf URL: https://arxiv.org/pdf/2601.11432
• Copy Paste: [[2601.11432]] The unreasonable effectiveness of pattern matching(https://arxiv.org/abs/2601.11432)
• Keywords: language model, llm
• Abstract: We report on an astonishing ability of large language models (LLMs) to make sense of "Jabberwocky" language in which most or all content words have been randomly replaced by nonsense strings, e.g., translating "He dwushed a ghanc zawk" to "He dragged a spare chair". This result addresses ongoing controversies regarding how to best think of what LLMs are doing: are they a language mimic, a database, a blurry version of the Web? The ability of LLMs to recover meaning from structural patterns speaks to the unreasonable effectiveness of pattern-matching. Pattern-matching is not an alternative to "real" intelligence, but rather a key ingredient.
• 摘要：我们报告了大型语言模型（LLM）理解“Jabberwocky”语言的惊人能力，其中大多数或所有内容词都被无意义的字符串随机替换，例如，将“He dwushed a ghanc zawk”翻译为“He Dragged a Spare chair”。这一结果解决了关于如何最好地思考法学硕士正在做的事情的持续争议：它们是语言模仿、数据库还是网络的模糊版本？法学硕士从结构模式中恢复意义的能力说明了模式匹配的不合理有效性。模式匹配并不是“真正”智能的替代品，而是一个关键要素。

Title: Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models

• Authors: Xiaojie Gu, Guangxu Chen, Yuheng Yang, Jingxin Han, Andi Zhang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.11441
• Pdf URL: https://arxiv.org/pdf/2601.11441
• Copy Paste: [[2601.11441]] Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models(https://arxiv.org/abs/2601.11441)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) exhibit exceptional performance across various domains, yet they face critical safety concerns. Model editing has emerged as an effective approach to mitigate these issues. Existing model editing methods often focus on optimizing an information matrix that blends new and old knowledge. While effective, these approaches can be computationally expensive and may cause conflicts. In contrast, we shift our attention to Hierarchical Orthogonal Residual SprEad of the information matrix, which reduces noisy gradients and enables more stable edits from a different perspective. We demonstrate the effectiveness of our method HORSE through a clear theoretical comparison with several popular methods and extensive experiments conducted on two datasets across multiple LLMs. The results show that HORSE maintains precise massive editing across diverse scenarios. The code is available at this https URL
• 摘要：大型语言模型 (LLM) 在各个领域都表现出卓越的性能，但它们面临着严重的安全问题。模型编辑已成为缓解这些问题的有效方法。现有的模型编辑方法通常侧重于优化融合新旧知识的信息矩阵。虽然有效，但这些方法的计算成本可能很高，并且可能会导致冲突。相反，我们将注意力转移到信息矩阵的分层正交残差 SprEad 上，它减少了噪声梯度，并从不同的角度实现了更稳定的编辑。我们通过与几种流行方法进行清晰的理论比较以及在多个法学硕士的两个数据集上进行的广泛实验来证明我们的方法 HORSE 的有效性。结果表明，HORSE 在不同场景下都能保持精确的大规模编辑。该代码可在此 https URL 获取

Title: Predict the Retrieval! Test time adaptation for Retrieval Augmented Generation

• Authors: Xin Sun, Zhongqi Chen, Qiang Liu, Shu Wu, Bowen Song, Weiqiang Wang, Zilei Wang, Liang Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11443
• Pdf URL: https://arxiv.org/pdf/2601.11443
• Copy Paste: [[2601.11443]] Predict the Retrieval! Test time adaptation for Retrieval Augmented Generation(https://arxiv.org/abs/2601.11443)
• Keywords: language model, retrieval augmented generation, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) has emerged as a powerful approach for enhancing large language models' question-answering capabilities through the integration of external knowledge. However, when adapting RAG systems to specialized domains, challenges arise from distribution shifts, resulting in suboptimal generalization performance. In this work, we propose TTARAG, a test-time adaptation method that dynamically updates the language model's parameters during inference to improve RAG system performance in specialized domains. Our method introduces a simple yet effective approach where the model learns to predict retrieved content, enabling automatic parameter adjustment to the target domain. Through extensive experiments across six specialized domains, we demonstrate that TTARAG achieves substantial performance improvements over baseline RAG systems. Code available at this https URL.
• 摘要：检索增强生成（RAG）已成为一种通过集成外部知识来增强大型语言模型的问答能力的强大方法。然而，当将 RAG 系统应用于专门领域时，分布变化会带来挑战，导致泛化性能不佳。在这项工作中，我们提出了 TTARAG，一种测试时适应方法，可在推理过程中动态更新语言模型的参数，以提高 RAG 系统在专门领域的性能。我们的方法引入了一种简单而有效的方法，模型学习预测检索的内容，从而实现对目标域的自动参数调整。通过跨六个专业领域的广泛实验，我们证明 TTARAG 比基准 RAG 系统实现了显着的性能改进。代码可在此 https URL 获取。

Title: CTest-Metric: A Unified Framework to Assess Clinical Validity of Metrics for CT Report Generation

• Authors: Vanshali Sharma, Andrea Mia Bejar, Gorkem Durak, Ulas Bagci
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2601.11488
• Pdf URL: https://arxiv.org/pdf/2601.11488
• Copy Paste: [[2601.11488]] CTest-Metric: A Unified Framework to Assess Clinical Validity of Metrics for CT Report Generation(https://arxiv.org/abs/2601.11488)
• Keywords: llm
• Abstract: In the generative AI era, where even critical medical tasks are increasingly automated, radiology report generation (RRG) continues to rely on suboptimal metrics for quality assessment. Developing domain-specific metrics has therefore been an active area of research, yet it remains challenging due to the lack of a unified, well-defined framework to assess their robustness and applicability in clinical contexts. To address this, we present CTest-Metric, a first unified metric assessment framework with three modules determining the clinical feasibility of metrics for CT RRG. The modules test: (i) Writing Style Generalizability (WSG) via LLM-based rephrasing; (ii) Synthetic Error Injection (SEI) at graded severities; and (iii) Metrics-vs-Expert correlation (MvE) using clinician ratings on 175 "disagreement" cases. Eight widely used metrics (BLEU, ROUGE, METEOR, BERTScore-F1, F1-RadGraph, RaTEScore, GREEN Score, CRG) are studied across seven LLMs built on a CT-CLIP encoder. Using our novel framework, we found that lexical NLG metrics are highly sensitive to stylistic variations; GREEN Score aligns best with expert judgments (Spearman~0.70), while CRG shows negative correlation; and BERTScore-F1 is least sensitive to factual error injection. We will release the framework, code, and allowable portion of the anonymized evaluation data (rephrased/error-injected CT reports), to facilitate reproducible benchmarking and future metric development.
• 摘要：在生成式人工智能时代，即使是关键的医疗任务也日益自动化，放射学报告生成 (RRG) 继续依赖次优指标进行质量评估。因此，开发特定领域的指标一直是一个活跃的研究领域，但由于缺乏统一、明确的框架来评估其在临床环境中的稳健性和适用性，因此仍然具有挑战性。为了解决这个问题，我们提出了 CTest-Metric，这是第一个统一的指标评估框架，具有三个模块，确定 CT RRG 指标的临床可行性。这些模块测试：(i) 通过基于 LLM 的改写进行写作风格概括性 (WSG)； (ii) 分级严重性的综合错误注入（SEI）； (iii) 使用临床医生对 175 个“不一致”病例进行评级的指标与专家相关性 (MvE)。在基于 CT-CLIP 编码器的七个法学硕士中研究了八个广泛使用的指标（BLEU、ROUGE、METEOR、BERTScore-F1、F1-RadGraph、RaTEScore、GREEN Score、CRG）。使用我们的新颖框架，我们发现词汇 NLG 指标对文体变化高度敏感； GREEN Score 与专家判断最吻合（Spearman~0.70），而 CRG 呈现负相关； BERTScore-F1 对事实错误注入最不敏感。我们将发布框架、代码和匿名评估数据的允许部分（改写/注入错误的 CT 报告），以促进可重复的基准测试和未来的指标开发。

Title: How Long Is a Piece of String? A Brief Empirical Analysis of Tokenizers

• Authors: Jonathan Roberts, Kai Han, Samuel Albanie
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.11518
• Pdf URL: https://arxiv.org/pdf/2601.11518
• Copy Paste: [[2601.11518]] How Long Is a Piece of String? A Brief Empirical Analysis of Tokenizers(https://arxiv.org/abs/2601.11518)
• Keywords: llm
• Abstract: Frontier LLMs are increasingly utilised across academia, society and industry. A commonly used unit for comparing models, their inputs and outputs, and estimating inference pricing is the token. In general, tokens are used as a stable currency, assumed to be broadly consistent across tokenizers and contexts, enabling direct comparisons. However, tokenization varies significantly across models and domains of text, making naive interpretation of token counts problematic. We quantify this variation by providing a comprehensive empirical analysis of tokenization, exploring the compression of sequences to tokens across different distributions of textual data. Our analysis challenges commonly held heuristics about token lengths, finding them to be overly simplistic. We hope the insights of our study add clarity and intuition toward tokenization in contemporary LLMs.
• 摘要：前沿法学硕士越来越多地在学术界、社会和工业界得到利用。用于比较模型、模型的输入和输出以及估计推理定价的常用单位是代币。一般来说，代币被用作稳定货币，假设在代币化器和上下文之间广泛一致，从而可以进行直接比较。然而，标记化在不同的模型和文本域之间存在显着差异，这使得对标记计数的简单解释产生了问题。我们通过提供对标记化的全面实证分析来量化这种变化，探索不同文本数据分布中序列到标记的压缩。我们的分析挑战了关于令牌长度的常见启发式方法，发现它们过于简单化。我们希望我们的研究见解能够为当代法学硕士的代币化增添清晰度和直觉。