2026-03-02

Title: Toward General Semantic Chunking: A Discriminative Framework for Ultra-Long Documents

• Authors: Kaifeng Wu, Junyan Wu, Qiang Liu, Jiarui Zhang, Wen Xu
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2602.23370
• Pdf URL: https://arxiv.org/pdf/2602.23370
• Copy Paste: [[2602.23370]] Toward General Semantic Chunking: A Discriminative Framework for Ultra-Long Documents(https://arxiv.org/abs/2602.23370)
• Keywords: language model
• Abstract: Long-document topic segmentation plays an important role in information retrieval and document understanding, yet existing methods still show clear shortcomings in ultra-long text settings. Traditional discriminative models are constrained by fixed windows and cannot model document-level semantics; generative large language models can output paragraph boundaries, but inference is expensive and long inputs are difficult to support. To address these issues, we propose a discriminative segmentation model based on Qwen3-0.6B. On top of the backbone network, we add a cross-window context fusion layer and a boundary classification head, and combine them with an overlapping sliding-window strategy. Our model supports single-pass inputs of up to 13k tokens and can be extended to ultra-long documents for paragraph boundary detection. To further enhance downstream retrieval efficiency, we derive a vector fusion method with scalar correction, which compresses the representation of ultra-long segments into a single vector without semantic loss. Experiments on the Wikipedia long-document topic segmentation dataset WIKI-727K show that, compared with three generative models based on Qwen2-0.5B released by Jina, our method achieves a better macro-averaged F1 and delivers two orders of magnitude faster inference, substantially improving the practicality and scalability of long-document processing.
• 摘要：长文档主题分割在信息检索和文档理解中发挥着重要作用，但现有方法在超长文本设置中仍然表现出明显的缺陷。传统判别模型受限于固定窗口，无法建模文档级语义；生成式大型语言模型可以输出段落边界，但推理成本高昂且难以支持长输入。为了解决这些问题，我们提出了一种基于 Qwen3-0.6B 的判别分割模型。在主干网络之上，我们添加了跨窗口上下文融合层和边界分类头，并将它们与重叠滑动窗口策略结合起来。我们的模型支持最多 13k 个标记的单遍输入，并且可以扩展到超长文档以进行段落边界检测。为了进一步提高下游检索效率，我们推导了一种带有标量校正的向量融合方法，它将超长片段的表示压缩为单个向量，而不会造成语义损失。在维基百科长文档主题分割数据集WIKI-727K上的实验表明，与Jina发布的基于Qwen2-0.5B的三种生成模型相比，我们的方法实现了更好的宏观平均F1，推理速度提高了两个数量级，大幅提高了长文档处理的实用性和可扩展性。

Title: Truncated Step-Level Sampling with Process Rewards for Retrieval-Augmented Reasoning

• Authors: Chris Samarinas, Haw-Shiuan Chang, Hamed Zamani
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2602.23440
• Pdf URL: https://arxiv.org/pdf/2602.23440
• Copy Paste: [[2602.23440]] Truncated Step-Level Sampling with Process Rewards for Retrieval-Augmented Reasoning(https://arxiv.org/abs/2602.23440)
• Keywords: language model, llm
• Abstract: Training large language models to reason with search engines via reinforcement learning is hindered by a fundamental credit assignment problem: existing methods such as Search-R1 provide only a sparse outcome reward after an entire multi-step trajectory, making it infeasible to attribute success or failure to individual reasoning and retrieval decisions. Process-reward methods like StepSearch alleviate this by introducing step-level supervision, but rely on heuristic rewards such as TF-IDF overlap with gold documents, and still sample k complete trajectories per example, retaining high gradient variance. We propose SLATE, a framework built on two complementary ideas: (1) truncated step-level sampling, which generates k trajectories that share a common prefix and differ only at the next step, and (2) dense LLM-as-judge rewards, which replace heuristic scoring with a capable LLM evaluator that assesses the quality of each reasoning step, search query, and answer, providing richer and more reliable supervision. We theoretically prove that under the same dense reward structure, truncated sampling reduces the variance of advantage estimates by up to a factor of T compared to full-trajectory sampling for T-step trajectories, yielding lower-variance, better-targeted policy gradients. Experiments on seven QA benchmarks confirm that SLATE consistently outperforms both sparse-reward and process-reward baselines, with the largest gains on harder multi-hop tasks and smaller models.
• 摘要：通过强化学习训练大型语言模型以使用搜索引擎进行推理受到基本信用分配问题的阻碍：Search-R1 等现有方法在整个多步轨迹之后仅提供稀疏的结果奖励，从而无法将成功或失败归因于个人推理和检索决策。像 StepSearch 这样的过程奖励方法通过引入步骤级监督来缓解这一问题，但依赖于启发式奖励，例如 TF-IDF 与黄金文档的重叠，并且仍然对每个示例的 k 个完整轨迹进行采样，保留高梯度方差。我们提出了 SLATE，一个建立在两个互补思想之上的框架：(1) 截断的步骤级采样，它生成 k 个共享共同前缀且仅在下一步有所不同的轨迹，以及 (2) 密集的 LLM 作为法官奖励，它用一个有能力的 LLM 评估器取代启发式评分，评估每个推理步骤、搜索查询和答案的质量，提供更丰富、更可靠的监督。我们从理论上证明，在相同的密集奖励结构下，与 T 步轨迹的全轨迹采样相比，截断采样将优势估计的方差减少了 T 倍，从而产生方差更低、针对性更强的策略梯度。对七个 QA 基准的实验证实，SLATE 始终优于稀疏奖励和过程奖励基线，在较难的多跳任务和较小的模型上收益最大。

Title: CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era

• Authors: Zhengqing Yuan, Kaiwen Shi, Zheyuan Zhang, Lichao Sun, Nitesh V. Chawla, Yanfang Ye
• Subjects: cs.CL, cs.DL
• Abstract URL: https://arxiv.org/abs/2602.23452
• Pdf URL: https://arxiv.org/pdf/2602.23452
• Copy Paste: [[2602.23452]] CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era(https://arxiv.org/abs/2602.23452)
• Keywords: language model, llm, agent
• Abstract: Scientific research relies on accurate citation for attribution and integrity, yet large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications. Such hallucinated citations have already been observed in submissions and accepted papers at major machine learning venues, exposing vulnerabilities in peer review. Meanwhile, rapidly growing reference lists make manual verification impractical, and existing automated tools remain fragile to noisy and heterogeneous citation formats and lack standardized evaluation. We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing. Our multi-agent verification pipeline decomposes citation checking into claim extraction, evidence retrieval, passage matching, reasoning, and calibrated judgment to assess whether a cited source truly supports its claim. We construct a large-scale human-validated dataset across domains and define unified metrics for citation faithfulness and evidence alignment. Experiments with state-of-the-art LLMs reveal substantial citation errors and show that our framework significantly outperforms prior methods in both accuracy and interpretability. This work provides the first scalable infrastructure for auditing citations in the LLM era and practical tools to improve the trustworthiness of scientific references.
• 摘要：科学研究依赖于准确引用的归属和完整性，但大型语言模型 (LLM) 引入了新的风险：伪造的参考文献看似合理，但与真实的出版物不相符。这种幻觉引用已经在主要机器学习场所的提交和接受的论文中观察到，暴露了同行评审中的漏洞。与此同时，快速增长的参考文献列表使得手动验证变得不切实际，现有的自动化工具仍然容易受到噪音和异构引文格式的影响，并且缺乏标准化的评估。我们提出了第一个针对科学写作中的幻觉引用的综合基准和检测框架。我们的多代理验证管道将引文检查分解为主张提取、证据检索、段落匹配、推理和校准判断，以评估引用来源是否真正支持其主张。我们构建了跨领域的大规模人工验证数据集，并为引文忠实度和证据对齐定义了统一的指标。使用最先进的法学硕士进行的实验揭示了严重的引用错误，并表明我们的框架在准确性和可解释性方面都显着优于先前的方法。这项工作为法学硕士时代的引文审计提供了第一个可扩展的基础设施，并提供了提高科学参考文献可信度的实用工具。

Title: FHIRPath-QA: Executable Question Answering over FHIR Electronic Health Records

• Authors: Michael Frew, Nishit Bheda, Bryan Tripp
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23479
• Pdf URL: https://arxiv.org/pdf/2602.23479
• Copy Paste: [[2602.23479]] FHIRPath-QA: Executable Question Answering over FHIR Electronic Health Records(https://arxiv.org/abs/2602.23479)
• Keywords: language model, llm, hallucination
• Abstract: Though patients are increasingly granted digital access to their electronic health records (EHRs), existing interfaces may not support precise, trustworthy answers to patient-specific questions. Large language models (LLM) show promise in clinical question answering (QA), but retrieval-based approaches are computationally inefficient, prone to hallucination, and difficult to deploy over real-life EHRs. In this work, we introduce FHIRPath-QA, the first open dataset and benchmark for patient-specific QA that includes open-standard FHIRPath queries over real-world clinical data. We propose a text-to-FHIRPath QA paradigm that shifts reasoning from free-text generation to FHIRPath query synthesis, significantly reducing LLM usage. Built on MIMIC-IV on FHIR Demo, the dataset pairs over 14k natural language questions in patient and clinician phrasing with validated FHIRPath queries and answers. Further, we demonstrate that state-of-the-art LLMs struggle to deal with ambiguity in patient language and perform poorly in FHIRPath query synthesis. However, they benefit strongly from supervised fine-tuning. Our results highlight that text-to-FHIRPath synthesis has the potential to serve as a practical foundation for safe, efficient, and interoperable consumer health applications, and our dataset and benchmark serve as a starting point for future research on the topic. The full dataset and generation code is available at: this https URL.
• 摘要：尽管患者越来越多地获得对其电子健康记录 (EHR) 的数字访问权限，但现有界面可能不支持针对患者特定问题提供精确、可信的答案。大型语言模型 (LLM) 在临床问答 (QA) 方面显示出前景，但基于检索的方法计算效率低下，容易产生幻觉，并且难以在现实生活中的 EHR 上部署。在这项工作中，我们介绍了 FHIRPath-QA，这是第一个针对患者特定 QA 的开放数据集和基准，其中包括对真实世界临床数据的开放标准 FHIRPath 查询。我们提出了一种文本到 FHIRPath 的 QA 范式，将推理从自由文本生成转变为 FHIRPath 查询合成，从而显着减少了 LLM 的使用。该数据集基于 FHIR 演示上的 MIMIC-IV 构建，将患者和临床医生措辞中的超过 14k 个自然语言问题与经过验证的 FHIRPath 查询和答案配对。此外，我们证明最先进的法学硕士很难处理患者语言中的歧义，并且在 FHIRPath 查询合成中表现不佳。然而，他们从监督微调中受益匪浅。我们的结果强调，文本到 FHIRPath 的合成有潜力成为安全、高效和可互操作的消费者健康应用程序的实践基础，并且我们的数据集和基准可以作为该主题未来研究的起点。完整的数据集和生成代码可在以下位置获得：此 https URL。

Title: IDP Accelerator: Agentic Document Intelligence from Extraction to Compliance Validation

• Authors: Md Mofijul Islam, Md Sirajus Salekin, Joe King, Priyashree Roy, Vamsi Thilak Gudi, Spencer Romo, Akhil Nooney, Boyi Xie, Bob Strahan, Diego A. Socolinsky
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23481
• Pdf URL: https://arxiv.org/pdf/2602.23481
• Copy Paste: [[2602.23481]] IDP Accelerator: Agentic Document Intelligence from Extraction to Compliance Validation(https://arxiv.org/abs/2602.23481)
• Keywords: language model, llm, agent
• Abstract: Understanding and extracting structured insights from unstructured documents remains a foundational challenge in industrial NLP. While Large Language Models (LLMs) enable zero-shot extraction, traditional pipelines often fail to handle multi-document packets, complex reasoning, and strict compliance requirements. We present IDP (Intelligent Document Processing) Accelerator, a framework enabling agentic AI for end-to-end document intelligence with four key components: (1) DocSplit, a novel benchmark dataset and multimodal classifier using BIO tagging to segment complex document packets; (2) configurable Extraction Module leveraging multimodal LLMs to transform unstructured content into structured data; (3) Agentic Analytics Module, compliant with the Model Context Protocol (MCP) providing data access through secure, sandboxed code execution; and (4) Rule Validation Module replacing deterministic engines with LLM-driven logic for complex compliance checks. The interactive demonstration enables users to upload document packets, visualize classification results, and explore extracted data through an intuitive web interface. We demonstrate effectiveness across industries, highlighting a production deployment at a leading healthcare provider achieving 98% classification accuracy, 80% reduced processing latency, and 77% lower operational costs over legacy baselines. IDP Accelerator is open-sourced with a live demonstration available to the community.
• 摘要：从非结构化文档中理解并提取结构化见解仍然是工业 NLP 的基本挑战。虽然大型语言模型 (LLM) 可实现零样本提取，但传统管道通常无法处理多文档数据包、复杂推理和严格的合规性要求。我们推出了 IDP（智能文档处理）加速器，这是一个支持代理 AI 实现端到端文档智能的框架，具有四个关键组件：（1）DocSplit，一种新颖的基准数据集和多模态分类器，使用 BIO 标记来分割复杂的文档数据包； (2) 可配置的提取模块利用多模式法学硕士将非结构化内容转换为结构化数据； (3) 代理分析模块，符合模型上下文协议 (MCP)，通过安全的沙盒代码执行提供数据访问； (4) 规则验证模块用 LLM 驱动的逻辑替换确定性引擎，以进行复杂的合规性检查。交互式演示使用户能够上传文档包、可视化分类结果并通过直观的 Web 界面探索提取的数据。我们展示了跨行业的有效性，重点介绍了领先医疗保健提供商的生产部署，与传统基线相比，实现了 98% 的分类准确率、80% 的处理延迟减少以及 77% 的运营成本降低。 IDP Accelerator 是开源的，并向社区提供现场演示。

Title: Humans and LLMs Diverge on Probabilistic Inferences

• Authors: Gaurav Kamath, Sreenath Madathil, Sebastian Schuster, Marie-Catherine de Marneffe, Siva Reddy
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.23546
• Pdf URL: https://arxiv.org/pdf/2602.23546
• Copy Paste: [[2602.23546]] Humans and LLMs Diverge on Probabilistic Inferences(https://arxiv.org/abs/2602.23546)
• Keywords: llm
• Abstract: Human reasoning often involves working over limited information to arrive at probabilistic conclusions. In its simplest form, this involves making an inference that is not strictly entailed by a premise, but rather only likely given the premise. While reasoning LLMs have demonstrated strong performance on logical and mathematical tasks, their behavior on such open-ended, non-deterministic inferences remains largely unexplored. We introduce ProbCOPA, a dataset of 210 handcrafted probabilistic inferences in English, each annotated for inference likelihood by 25--30 human participants. We find that human responses are graded and varied, revealing probabilistic judgments of the inferences in our dataset. Comparing these judgments with responses from eight state-of-the-art reasoning LLMs, we show that models consistently fail to produce human-like distributions. Finally, analyzing LLM reasoning chains, we find evidence of a common reasoning pattern used to evaluate such inferences. Our findings reveal persistent differences between humans and LLMs, and underscore the need to evaluate reasoning beyond deterministic settings.
• 摘要：人类推理通常涉及处理有限的信息以得出概率结论。在最简单的形式中，这涉及做出一个并非严格由前提所蕴含的推论，而是仅可能在给定前提的情况下进行推论。虽然推理法学硕士在逻辑和数学任务上表现出了很强的表现，但他们在这种开放式、非确定性推论上的行为在很大程度上仍未得到探索。我们引入了 ProbCOPA，这是一个包含 210 个手工制作的英语概率推理的数据集，每个推理都由 25--30 名人类参与者注释了推理可能性。我们发现人类的反应是分级和变化的，揭示了对我们数据集中的推论的概率判断。将这些判断与八位最先进的推理法学硕士的回答进行比较，我们发现模型始终无法产生类似人类的分布。最后，通过分析法学硕士推理链，我们找到了用于评估此类推论的常见推理模式的证据。我们的研究结果揭示了人类和法学硕士之间持续存在的差异，并强调需要评估确定性环境之外的推理。

Title: France or Spain or Germany or France: A Neural Account of Non-Redundant Redundant Disjunctions

• Authors: Sasha Boguraev, Qing Yao, Kyle Mahowald
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23547
• Pdf URL: https://arxiv.org/pdf/2602.23547
• Copy Paste: [[2602.23547]] France or Spain or Germany or France: A Neural Account of Non-Redundant Redundant Disjunctions(https://arxiv.org/abs/2602.23547)
• Keywords: language model
• Abstract: Sentences like "She will go to France or Spain, or perhaps to Germany or France." appear formally redundant, yet become acceptable in contexts such as "Mary will go to a philosophy program in France or Spain, or a mathematics program in Germany or France." While this phenomenon has typically been analyzed using symbolic formal representations, we aim to provide a complementary account grounded in artificial neural mechanisms. We first present new behavioral evidence from humans and large language models demonstrating the robustness of this apparent non-redundancy across contexts. We then show that, in language models, redundancy avoidance arises from two interacting mechanisms: models learn to bind contextually relevant information to repeated lexical items, and Transformer induction heads selectively attend to these context-licensed representations. We argue that this neural explanation sheds light on the mechanisms underlying context-sensitive semantic interpretation, and that it complements existing symbolic analyses.
• 摘要：像“她会去法国或西班牙，或者也许去德国或法国”这样的句子。看起来形式上是多余的，但在诸如“玛丽将去法国或西班牙参加哲学课程，或者德国或法国的数学课程”之类的情况下却变得可以接受。虽然这种现象通常是使用符号形式表示来分析的，但我们的目标是提供基于人工神经机制的补充说明。 We first present new behavioral evidence from humans and large language models demonstrating the robustness of this apparent non-redundancy across contexts. We then show that, in language models, redundancy avoidance arises from two interacting mechanisms: models learn to bind contextually relevant information to repeated lexical items, and Transformer induction heads selectively attend to these context-licensed representations. We argue that this neural explanation sheds light on the mechanisms underlying context-sensitive semantic interpretation, and that it complements existing symbolic analyses.

Title: Multi-Agent Causal Reasoning for Suicide Ideation Detection Through Online Conversations

• Authors: Jun Li, Xiangmeng Wang, Haoyang Li, Yifei Yan, Shijie Zhang, Hong Va Leong, Ling Feng, Nancy Xiaonan Yu, Qing Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23577
• Pdf URL: https://arxiv.org/pdf/2602.23577
• Copy Paste: [[2602.23577]] Multi-Agent Causal Reasoning for Suicide Ideation Detection Through Online Conversations(https://arxiv.org/abs/2602.23577)
• Keywords: agent
• Abstract: Suicide remains a pressing global public health concern. While social media platforms offer opportunities for early risk detection through online conversation trees, existing approaches face two major limitations: (1) They rely on predefined rules (e.g., quotes or relies) to log conversations that capture only a narrow spectrum of user interactions, and (2) They overlook hidden influences such as user conformity and suicide copycat behavior, which can significantly affect suicidal expression and propagation in online communities. To address these limitations, we propose a Multi-Agent Causal Reasoning (MACR) framework that collaboratively employs a Reasoning Agent to scale user interactions and a Bias-aware Decision-Making Agent to mitigate harmful biases arising from hidden influences. The Reasoning Agent integrates cognitive appraisal theory to generate counterfactual user reactions to posts, thereby scaling user interactions. It analyses these reactions through structured dimensions, i.e., cognitive, emotional, and behavioral patterns, with a dedicated sub-agent responsible for each dimension. The Bias-aware Decision-Making Agent mitigates hidden biases through a front-door adjustment strategy, leveraging the counterfactual user reactions produced by the Reasoning Agent. Through the collaboration of reasoning and bias-aware decision making, the proposed MACR framework not only alleviates hidden biases, but also enriches contextual information of user interactions with counterfactual knowledge. Extensive experiments on real-world conversational datasets demonstrate the effectiveness and robustness of MACR in identifying suicide risk.
• 摘要：自杀仍然是一个紧迫的全球公共卫生问题。虽然社交媒体平台提供了通过在线对话树进行早期风险检测的机会，但现有方法面临两个主要局限性：（1）它们依赖于预定义的规则（例如引用或依赖）来记录仅捕获一小部分用户交互的对话，以及（2）它们忽略了隐藏的影响，例如用户一致性和自杀模仿行为，这些影响可能会严重影响在线社区中的自杀表达和传播。为了解决这些限制，我们提出了一个多代理因果推理（MACR）框架，该框架协作使用推理代理来扩展用户交互，并使用偏差感知决策代理来减轻因隐藏影响而产生的有害偏差。推理代理集成了认知评估理论，以生成对帖子的反事实用户反应，从而扩展用户交互。它通过结构化维度（即认知、情感和行为模式）分析这些反应，并由专门的子代理负责每个维度。偏见感知决策代理通过前门调整策略减轻隐藏偏见，利用推理代理产生的反事实用户反应。通过推理和偏见意识决策的协作，所提出的 MACR 框架不仅减轻了隐藏的偏见，而且还丰富了用户与反事实知识交互的上下文信息。对现实世界对话数据集的大量实验证明了 MACR 在识别自杀风险方面的有效性和稳健性。

Title: BRIDGE the Gap: Mitigating Bias Amplification in Automated Scoring of English Language Learners via Inter-group Data Augmentation

• Authors: Yun Wang, Xuansheng Wu, Jingyuan Huang, Lei Liu, Xiaoming Zhai, Ninghao Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.23580
• Pdf URL: https://arxiv.org/pdf/2602.23580
• Copy Paste: [[2602.23580]] BRIDGE the Gap: Mitigating Bias Amplification in Automated Scoring of English Language Learners via Inter-group Data Augmentation(https://arxiv.org/abs/2602.23580)
• Keywords: language model, llm
• Abstract: In the field of educational assessment, automated scoring systems increasingly rely on deep learning and large language models (LLMs). However, these systems face significant risks of bias amplification, where model prediction gaps between student groups become larger than those observed in training data. This issue is especially severe for underrepresented groups such as English Language Learners (ELLs), as models may inherit and further magnify existing disparities in the data. We identify that this issue is closely tied to representation bias: the scarcity of minority (high-scoring ELL) samples makes models trained with empirical risk minimization favor majority (non-ELL) linguistic patterns. Consequently, models tend to under-predict ELL students who even demonstrate comparable domain knowledge but use different linguistic patterns, thereby undermining the fairness of automated scoring outcomes. To mitigate this, we propose BRIDGE, a Bias-Reducing Inter-group Data GEneration framework designed for low-resource assessment settings. Instead of relying on the limited minority samples, BRIDGE synthesizes high-scoring ELL samples by "pasting" construct-relevant (i.e., rubric-aligned knowledge and evidence) content from abundant high-scoring non-ELL samples into authentic ELL linguistic patterns. We further introduce a discriminator model to ensure the quality of synthetic samples. Experiments on California Science Test (CAST) datasets demonstrate that BRIDGE effectively reduces prediction bias for high-scoring ELL students while maintaining overall scoring performance. Notably, our method achieves fairness gains comparable to using additional real human data, offering a cost-effective solution for ensuring equitable scoring in large-scale assessments.
• 摘要：在教育评估领域，自动评分系统越来越依赖深度学习和大型语言模型（LLM）。然而，这些系统面临着偏差放大的重大风险，学生群体之间的模型预测差距变得比训练数据中观察到的差距更大。对于英语学习者 (ELL) 等代表性不足的群体来说，这个问题尤其严重，因为模型可能会继承并进一步放大数据中现有的差异。我们发现这个问题与表征偏差密切相关：少数群体（高分 ELL）样本的稀缺使得经过经验风险最小化训练的模型偏向多数群体（非 ELL）语言模式。因此，模型往往会低估 ELL 学生的预测，这些学生甚至表现出可比较的领域知识，但使用不同的语言模式，从而破坏了自动评分结果的公平性。为了缓解这一问题，我们提出了 BRIDGE，这是一种专为低资源评估环境而设计的减少偏差的组间数据生成框架。 BRIDGE 不依赖有限的少数样本，而是通过将丰富的高分非 ELL 样本中的构造相关（即，与标题对齐的知识和证据）内容“粘贴”到真实的 ELL 语言模式中来合成高分 ELL 样本。我们进一步引入鉴别器模型来确保合成样本的质量。在加州科学考试 (CAST) 数据集上进行的实验表明，BRIDGE 可以有效减少高分 ELL 学生的预测偏差，同时保持整体评分表现。值得注意的是，我们的方法实现了与使用额外的真实人类数据相当的公平性收益，为确保大规模评估中的公平评分提供了一种经济有效的解决方案。

Title: LFQA-HP-1M: A Large-Scale Human Preference Dataset for Long-Form Question Answering

• Authors: Rafid Ishrak Jahan, Fahmid Shahriar Iqbal, Sagnik Ray Choudhury
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2602.23603
• Pdf URL: https://arxiv.org/pdf/2602.23603
• Copy Paste: [[2602.23603]] LFQA-HP-1M: A Large-Scale Human Preference Dataset for Long-Form Question Answering(https://arxiv.org/abs/2602.23603)
• Keywords: llm
• Abstract: Long-form question answering (LFQA) demands nuanced evaluation of multi-sentence explanatory responses, yet existing metrics often fail to reflect human judgment. We present LFQA-HP-1M, a large-scale dataset comprising 1.3M human pairwise preference annotations for LFQA. We propose nine rubrics for answer quality evaluation, and show that simple linear models based on these features perform comparably to state-of-the-art LLM evaluators. We further examine transitivity consistency, positional bias, and verbosity biases in LLM evaluators and demonstrate their vulnerability to adversarial perturbations. Overall, this work provides one of the largest public LFQA preference datasets and a rubric-driven framework for transparent and reliable evaluation.
• 摘要：长篇问答 (LFQA) 需要对多句解释性响应进行细致入微的评估，但现有指标通常无法反映人类判断。我们提出了 LFQA-HP-1M，这是一个包含 130 万个 LFQA 人类成对偏好注释的大型数据集。我们提出了九个用于答案质量评估的标准，并表明基于这些特征的简单线性模型的性能与最先进的 LLM 评估器相当。我们进一步检查了 LLM 评估者的传递性一致性、位置偏差和冗长偏差，并证明了他们对对抗性扰动的脆弱性。总体而言，这项工作提供了最大的公共 LFQA 偏好数据集之一，以及用于透明和可靠评估的标题驱动框架。

Title: LLM-Driven Multi-Turn Task-Oriented Dialogue Synthesis for Realistic Reasoning

• Authors: Yu Zhu, Kai Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.23610
• Pdf URL: https://arxiv.org/pdf/2602.23610
• Copy Paste: [[2602.23610]] LLM-Driven Multi-Turn Task-Oriented Dialogue Synthesis for Realistic Reasoning(https://arxiv.org/abs/2602.23610)
• Keywords: language model, llm
• Abstract: The reasoning capability of large language models (LLMs), defined as their ability to analyze, infer, and make decisions based on input information, is essential for building intelligent task-oriented dialogue systems. However, existing benchmarks do not sufficiently reflect the complexity of real-world scenarios, which limits their effectiveness in evaluating and enhancing LLM reasoning in practical contexts. Many current reasoning datasets are overly simplistic and abstract, often disconnected from realistic task flows, domain constraints, and operational rules, making it difficult to effectively evaluate LLMs' logical reasoning ability. In addition, data contamination from pretraining corpora undermines the reliability of evaluation results, and traditional crowdsourcing methods for dataset construction are labor-intensive and difficult to scale. To address these challenges, we propose a LLM-driven framework for synthesizing multi-turn, task-oriented dialogues grounded in realistic reasoning scenarios, leveraging trilevel optimization to enhance dialogue quality. Our method generates dialogues grounded in authentic task scenarios, enriched with real-world information, and exhibiting strong contextual coherence. Corresponding reasoning tasks are carefully designed around these dialogues and iteratively refined to continuously improve the tasks' quality and challenge. The resulting dataset serves as a valuable benchmark for assessing and advancing the realistic logical reasoning capabilities of LLMs. Experimental results show that our synthetic data-based reasoning tasks introduce non-trivial reasoning challenges and provide meaningful support for improving the reasoning capabilities of LLMs.
• 摘要：大语言模型（LLM）的推理能力，定义为根据输入信息进行分析、推理和决策的能力，对于构建面向任务的智能对话系统至关重要。然而，现有的基准并不能充分反映现实场景的复杂性，这限制了它们在实际环境中评估和增强 LLM 推理的有效性。目前许多推理数据集过于简单化和抽象，往往与现实任务流程、领域约束和操作规则脱节，难以有效评估法学硕士的逻辑推理能力。此外，预训练语料库的数据污染损害了评估结果的可靠性，而传统的数据集构建众包方法是劳动密集型的且难以规模化。为了应对这些挑战，我们提出了一个法学硕士驱动的框架，用于综合基于现实推理场景的多轮、面向任务的对话，利用三级优化来提高对话质量。我们的方法生成基于真实任务场景的对话，丰富了现实世界的信息，并表现出很强的上下文连贯性。围绕这些对话精心设计相应的推理任务，并迭代完善，不断提高任务的质量和挑战性。所得数据集可作为评估和提高法学硕士现实逻辑推理能力的宝贵基准。实验结果表明，我们基于合成数据的推理任务引入了不平凡的推理挑战，并为提高法学硕士的推理能力提供了有意义的支持。

Title: TRIZ-RAGNER: A Retrieval-Augmented Large Language Model for TRIZ-Aware Named Entity Recognition in Patent-Based Contradiction Mining

• Authors: Zitong Xu, Yuqing Wu, Yue Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.23656
• Pdf URL: https://arxiv.org/pdf/2602.23656
• Copy Paste: [[2602.23656]] TRIZ-RAGNER: A Retrieval-Augmented Large Language Model for TRIZ-Aware Named Entity Recognition in Patent-Based Contradiction Mining(https://arxiv.org/abs/2602.23656)
• Keywords: language model, gpt, llm, hallucination, prompt
• Abstract: TRIZ-based contradiction mining is a fundamental task in patent analysis and systematic innovation, as it enables the identification of improving and worsening technical parameters that drive inventive problem solving. However, existing approaches largely rely on rule-based systems or traditional machine learning models, which struggle with semantic ambiguity, domain dependency, and limited generalization when processing complex patent language. Recently, large language models (LLMs) have shown strong semantic understanding capabilities, yet their direct application to TRIZ parameter extraction remains challenging due to hallucination and insufficient grounding in structured TRIZ knowledge. To address these limitations, this paper proposes TRIZ-RAGNER, a retrieval-augmented large language model framework for TRIZ-aware named entity recognition in patent-based contradiction mining. TRIZ-RAGNER reformulates contradiction mining as a semantic-level NER task and integrates dense retrieval over a TRIZ knowledge base, cross-encoder reranking for context refinement, and structured LLM prompting to extract improving and worsening parameters from patent sentences. By injecting domain-specific TRIZ knowledge into the LLM reasoning process, the proposed framework effectively reduces semantic noise and improves extraction consistency. Experiments on the PaTRIZ dataset demonstrate that TRIZ-RAGNER consistently outperforms traditional sequence labeling models and LLM-based baselines. The proposed framework achieves a precision of 85.6%, a recall of 82.9%, and an F1-score of 84.2% in TRIZ contradiction pair identification. Compared with the strongest baseline using prompt-enhanced GPT, TRIZ-RAGNER yields an absolute F1-score improvement of 7.3 percentage points, confirming the effectiveness of retrieval-augmented TRIZ knowledge grounding for robust and accurate patent-based contradiction mining.
• 摘要：基于 TRIZ 的矛盾挖掘是专利分析和系统创新的一项基本任务，因为它能够识别推动创造性问题解决的改进和恶化的技术参数。然而，现有的方法很大程度上依赖于基于规则的系统或传统的机器学习模型，在处理复杂的专利语言时，它们会遇到语义模糊、领域依赖和有限泛化的问题。最近，大型语言模型（LLM）表现出了强大的语义理解能力，但由于幻觉和结构化 TRIZ 知识基础不足，将其直接应用于 TRIZ 参数提取仍然具有挑战性。为了解决这些局限性，本文提出了 TRIZ-RAGNER，这是一种检索增强型大型语言模型框架，用于基于专利的矛盾挖掘中的 TRIZ 感知命名实体识别。 TRIZ-RAGNER 将矛盾挖掘重新表述为语义级 NER 任务，并集成了 TRIZ 知识库上的密集检索、用于上下文细化的跨编码器重新排名以及提示从专利句子中提取改进和恶化参数的结构化 LLM。通过将特定领域的 TRIZ 知识注入 LLM 推理过程，所提出的框架有效地减少了语义噪声并提高了提取的一致性。 PaTRIZ 数据集上的实验表明，TRIZ-RAGNER 始终优于传统的序列标记模型和基于 LLM 的基线。所提出的框架在 TRIZ 矛盾对识别中实现了 85.6% 的准确率、82.9% 的召回率和 84.2% 的 F1 分数。与使用即时增强 GPT 的最强基线相比，TRIZ-RAGNER 的绝对 F1 分数提高了 7.3 个百分点，证实了检索增强的 TRIZ 知识基础对于稳健且准确的基于专利的矛盾挖掘的有效性。

Title: From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning

• Authors: Seungdong Yoa, Sanghyu Yoon, Suhee Yoon, Dongmin Kim, Ye Seul Sim, Junhyun Lee, Woohyung Lim
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.23729
• Pdf URL: https://arxiv.org/pdf/2602.23729
• Copy Paste: [[2602.23729]] From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning(https://arxiv.org/abs/2602.23729)
• Keywords: language model, llm, prompt, agent
• Abstract: The evaluation of large language models (LLMs) has predominantly relied on static datasets, which offer limited scalability and fail to capture the evolving reasoning capabilities of recent models. To overcome these limitations, we propose an agent-centric benchmarking paradigm that moves beyond static datasets by introducing a dynamic protocol in which autonomous agents iteratively generate, validate, and solve problems. Within this protocol, a teacher agent generates candidate problems, an orchestrator agent rigorously verifies their validity and guards against adversarial attacks, and a student agent attempts to solve the validated problems. An invalid problem is revised by the teacher agent until it passes validation. If the student correctly solves the problem, the orchestrator prompts the teacher to generate more challenging variants. Consequently, the benchmark scales in difficulty automatically as more capable agents are substituted into any role, enabling progressive evaluation of large language models without manually curated datasets. Adopting text anomaly detection as our primary evaluation format, which demands cross-sentence logical inference and resists pattern-matching shortcuts, we demonstrate that this protocol systematically exposes corner-case reasoning errors that conventional benchmarks fail to reveal. We further advocate evaluating systems along several complementary axes including cross-model pairwise performance and progress between the initial and orchestrator-finalized problems. By shifting the focus from fixed datasets to dynamic protocols, our approach offers a sustainable direction for evaluating ever-evolving language models and introduces a research agenda centered on the co-evolution of agent-centric benchmarks.
• 摘要：大型语言模型（LLM）的评估主要依赖于静态数据集，其可扩展性有限，并且无法捕获最新模型不断发展的推理能力。为了克服这些限制，我们提出了一种以代理为中心的基准测试范式，该范式超越了静态数据集，引入了动态协议，其中自主代理迭代地生成、验证和解决问题。在该协议中，教师代理生成候选问题，编排代理严格验证其有效性并防范对抗性攻击，学生代理尝试解决已验证的问题。无效问题由教师代理进行修改，直到通过验证。如果学生正确解决了问题，协调器就会提示教师生成更具挑战性的变体。因此，随着能力更强的代理被替换为任何角色，基准测试的难度会自动扩展，从而无需手动管理数据集即可逐步评估大型语言模型。采用文本异常检测作为我们的主要评估格式，需要跨句子逻辑推理并抵制模式匹配捷径，我们证明该协议系统地暴露了传统基准无法揭示的极端情况推理错误。我们进一步提倡沿着几个互补的轴评估系统，包括跨模型成对性能以及初始问题和协调器最终问题之间的进展。通过将重点从固定数据集转移到动态协议，我们的方法为评估不断发展的语言模型提供了可持续的方向，并引入了以以代理为中心的基准的共同进化为中心的研究议程。

Title: Structured Prompt Optimization for Few-Shot Text Classification via Semantic Alignment in Latent Space

• Authors: Jiasen Zheng, Zijun Zhou, Huajun Zhang, Junjiang Lin, Jingyun Jia, Qi Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23753
• Pdf URL: https://arxiv.org/pdf/2602.23753
• Copy Paste: [[2602.23753]] Structured Prompt Optimization for Few-Shot Text Classification via Semantic Alignment in Latent Space(https://arxiv.org/abs/2602.23753)
• Keywords: language model, prompt
• Abstract: This study addresses the issues of semantic entanglement, unclear label structure, and insufficient feature representation in few-shot text classification, and proposes an optimization framework based on structured prompts to enhance semantic understanding and task adaptation under low-resource conditions. The framework first uses a pretrained language model to encode the input text and obtain basic semantic representations. It then introduces structured prompts composed of multi-dimensional semantic factors and integrates them with text features through a learnable combination mechanism, which forms task-related representations with clear boundaries in the latent space. To further strengthen the consistency between text representations and label semantics, the method constructs a structured label embedding matrix and employs a cross-space alignment mechanism to ensure stable matching between textual features and label attributes. In addition, the model applies prompt orthogonality constraints and a joint optimization objective to maintain independence across different semantic factors in the prompts, allowing the structured prompts to provide transparent and controllable guidance for classification decisions. Three types of sensitivity experiments, including learning rate sensitivity, prompt length sensitivity, and data scale sensitivity, are designed to evaluate the stability and robustness of the framework under different conditions. Experimental results show that the proposed structured prompt optimization framework effectively alleviates semantic conflicts and label ambiguity in few-shot text classification. It significantly improves performance on accuracy, precision, recall, and AUC, and demonstrates strong cross-task applicability.
• 摘要：该研究解决了少样本文本分类中的语义纠缠、标签结构不清晰和特征表示不足的问题，并提出了一种基于结构化提示的优化框架，以增强低资源条件下的语义理解和任务适应。该框架首先使用预训练的语言模型对输入文本进行编码并获得基本的语义表示。然后引入由多维语义因素组成的结构化提示，并通过可学习的组合机制将其与文本特征集成，在潜在空间中形成边界清晰的任务相关表示。为了进一步加强文本表示和标签语义之间的一致性，该方法构建了结构化标签嵌入矩阵，并采用跨空间对齐机制来确保文本特征和标签属性之间的稳定匹配。此外，该模型应用提示正交性约束和联合优化目标来保持提示中不同语义因素的独立性，从而允许结构化提示为分类决策提供透明且可控的指导。设计了学习率敏感性、提示长度敏感性和数据规模敏感性三种类型的敏感性实验，以评估框架在不同条件下的稳定性和鲁棒性。实验结果表明，所提出的结构化提示优化框架有效地缓解了少镜头文本分类中的语义冲突和标签歧义。它显着提高了准确性、精确性、召回率和 AUC 的性能，并表现出强大的跨任务适用性。

Title: Divide and Conquer: Accelerating Diffusion-Based Large Language Models via Adaptive Parallel Decoding

• Authors: Xiangzhong Luo, Yilin An, Zhicheng Yu, Weichen Liu, Xu Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23792
• Pdf URL: https://arxiv.org/pdf/2602.23792
• Copy Paste: [[2602.23792]] Divide and Conquer: Accelerating Diffusion-Based Large Language Models via Adaptive Parallel Decoding(https://arxiv.org/abs/2602.23792)
• Keywords: language model, llm
• Abstract: Diffusion-based large language models (dLLMs) have shown promising performance across various reasoning tasks, establishing themselves as an alternative to autoregressive large language models (LLMs). Unlike autoregressive LLMs that generate one token per step based on all previous tokens, dLLMs theoretically enable parallel generation of multiple tokens at each decoding step. However, recent dLLMs still favor one-token-per-step generation in practice, as directly decoding multiple masked tokens often leads to degraded generation quality and stability. This reveals a substantial gap between the theoretical parallelism and practical performance of dLLMs. To bridge this gap, we introduce an adaptive parallel decoding approach, namely DiCo, which features a three-phase divide-and-conquer paradigm to unleash the inherent parallelism of dLLMs. During the Divide phase, DiCo first explores the input masked sequence and identifies masked tokens as seed tokens, which are then expanded to construct a set of local clusters. During the Conquer phase, DiCo performs parallel decoding across different local clusters constructed in the Divide phase. The divide-and-conquer process repeatedly alternates between the Divide and Conquer phases until convergence. During the Finalize phase, DiCo decodes the remaining few masked tokens using an effective fine-grained compound decoding scheme to finalize the generation. Extensive experiments demonstrate that DiCo can achieve significant inference speedups while maintaining competitive generation quality.
• 摘要：基于扩散的大语言模型 (dLLM) 在各种推理任务中表现出了良好的性能，成为自回归大语言模型 (LLM) 的替代方案。与基于所有先前标记每一步生成一个标记的自回归 LLM 不同，dLLM 理论上可以在每个解码步骤并行生成多个标记。然而，最近的 dLLM 在实践中仍然倾向于每步生成一个令牌，因为直接解码多个屏蔽令牌通常会导致生成质量和稳定性下降。这揭示了 dLLM 的理论并行性和实际性能之间存在巨大差距。为了弥补这一差距，我们引入了一种自适应并行解码方法，即 DiCo，它采用三相分治范式来释放 dLLM 固有的并行性。在划分阶段，DiCo 首先探索输入屏蔽序列并将屏蔽标记识别为种子标记，然后将其扩展以构造一组本地集群。在征服阶段，DiCo 在划分阶段构建的不同本地集群之间执行并行解码。分而治之的过程在分而治之的阶段之间反复交替，直到收敛。在 Finalize 阶段，DiCo 使用有效的细粒度复合解码方案对剩余的几个屏蔽令牌进行解码，以完成生成。大量实验表明，DiCo 可以显着提高推理速度，同时保持有竞争力的生成质量。

Title: GLUScope: A Tool for Analyzing GLU Neurons in Transformer Language Models

• Authors: Sebastian Gerstner, Hinrich Schütze
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.23826
• Pdf URL: https://arxiv.org/pdf/2602.23826
• Copy Paste: [[2602.23826]] GLUScope: A Tool for Analyzing GLU Neurons in Transformer Language Models(https://arxiv.org/abs/2602.23826)
• Keywords: language model
• Abstract: We present GLUScope, an open-source tool for analyzing neurons in Transformer-based language models, intended for interpretability researchers. We focus on more recent models than previous tools do; specifically we consider gated activation functions such as SwiGLU. This introduces a new challenge: understanding positive activations is not enough. Instead, both the gate and the in activation of a neuron can be positive or negative, leading to four different possible sign combinations that in some cases have quite different functionalities. Accordingly, for any neuron, our tool shows text examples for each of the four sign combinations, and indicates how often each combination occurs. We describe examples of how our tool can lead to novel insights. A demo is available at https: //sjgerstner.this http URL.
• 摘要：我们推出了 GLUScope，这是一种开源工具，用于分析基于 Transformer 的语言模型中的神经元，供可解释性研究人员使用。我们专注于比以前的工具更新的模型；具体来说，我们考虑门控激活函数，例如 SwiGLU。这带来了一个新的挑战：理解正向激活是不够的。相反，神经元的门和激活都可以是正的或负的，从而导致四种不同的可能的符号组合，在某些情况下具有完全不同的功能。因此，对于任何神经元，我们的工具都会显示四种符号组合中每一种的文本示例，并指示每种组合出现的频率。我们描述了我们的工具如何带来新颖见解的示例。演示可在 https://sjgerstner.this http URL 获取。

Title: CLFEC: A New Task for Unified Linguistic and Factual Error Correction in paragraph-level Chinese Professional Writing

• Authors: Jian Kai, Zidong Zhang, Jiwen Chen, Zhengxiang Wu, Songtao Sun, Fuyang Li, Yang Cao, Qiang Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23845
• Pdf URL: https://arxiv.org/pdf/2602.23845
• Copy Paste: [[2602.23845]] CLFEC: A New Task for Unified Linguistic and Factual Error Correction in paragraph-level Chinese Professional Writing(https://arxiv.org/abs/2602.23845)
• Keywords: llm, prompt, retrieval-augmented generation, agent
• Abstract: Chinese text correction has traditionally focused on spelling and grammar, while factual error correction is usually treated separately. However, in paragraph-level Chinese professional writing, linguistic (word/grammar/punctuation) and factual errors frequently co-occur and interact, making unified correction both necessary and challenging. This paper introduces CLFEC (Chinese Linguistic & Factual Error Correction), a new task for joint linguistic and factual correction. We construct a mixed, multi-domain Chinese professional writing dataset spanning current affairs, finance, law, and medicine. We then conduct a systematic study of LLM-based correction paradigms, from prompting to retrieval-augmented generation (RAG) and agentic workflows. The analysis reveals practical challenges, including limited generalization of specialized correction models, the need for evidence grounding for factual repair, the difficulty of mixed-error paragraphs, and over-correction on clean inputs. Results further show that handling linguistic and factual Error within the same context outperform decoupled processes, and that agentic workflows can be effective with suitable backbone models. Overall, our dataset and empirical findings provide guidance for building reliable, fully automatic proofreading systems in industrial settings.
• 摘要：中文文本纠正传统上侧重于拼写和语法，而事实错误纠正通常是分开处理的。然而，在段落级中文专业写作中，语言（单词/语法/标点符号）和事实错误经常同时出现和相互作用，使得统一纠正既必要又具有挑战性。本文介绍了 CLFEC（中文语言与事实错误纠正），这是一种联合语言与事实纠错的新任务。我们构建了一个混合、多领域的中文专业写作数据集，涵盖时事、金融、法律和医学。然后，我们对基于 LLM 的校正范式进行系统研究，从提示到检索增强生成 (RAG) 和代理工作流程。分析揭示了实际挑战，包括专业校正模型的有限泛化、事实修复的证据基础的需要、混合错误段落的困难以及对干净输入的过度校正。结果进一步表明，在同一上下文中处理语言和事实错误优于解耦流程，并且代理工作流程可以通过合适的骨干模型发挥作用。总的来说，我们的数据集和实证研究结果为在工业环境中构建可靠的全自动校对系统提供了指导。

Title: The Astonishing Ability of Large Language Models to Parse Jabberwockified Language

• Authors: Gary Lupyan, Senyi Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23928
• Pdf URL: https://arxiv.org/pdf/2602.23928
• Copy Paste: [[2602.23928]] The Astonishing Ability of Large Language Models to Parse Jabberwockified Language(https://arxiv.org/abs/2602.23928)
• Keywords: language model, llm
• Abstract: We show that large language models (LLMs) have an astonishing ability to recover meaning from severely degraded English texts. Texts in which content words have been randomly substituted by nonsense strings, e.g., "At the ghybe of the swuint, we are haiveed to Wourge Phrear-gwurr, who sproles into an ghitch flount with his crurp", can be translated to conventional English that is, in many cases, close to the original text, e.g., "At the start of the story, we meet a man, Chow, who moves into an apartment building with his wife." These results show that structural cues (e.g., morphosyntax, closed-class words) constrain lexical meaning to a much larger degree than imagined. Although the abilities of LLMs to make sense of "Jabberwockified" English are clearly superhuman, they are highly relevant to understanding linguistic structure and suggest that efficient language processing either in biological or artificial systems likely benefits from very tight integration between syntax, lexical semantics, and general world knowledge.
• 摘要：我们证明大型语言模型（LLM）具有从严重退化的英语文本中恢复含义的惊人能力。内容词被随机替换为无意义字符串的文本，例如，“At the ghybe of the swuint, we are haied to Wourge Phrear-gwurr, who sproles into an ghitch flount with his crurp”，可以翻译成传统英语，即在许多情况下接近原始文本，例如，“在故事的开头，我们遇到了一个男人，Chow，他搬进了公寓。”和他的妻子一起建设。”这些结果表明，结构线索（例如形态句法、封闭类词）对词汇意义的限制程度比想象的要大得多。尽管法学硕士理解“Jabberwockified”英语的能力显然是超人的，但它们与理解语言结构高度相关，并表明生物或人工系统中的有效语言处理可能受益于句法、词汇语义和一般世界知识之间的紧密集成。

Title: MemEmo: Evaluating Emotion in Memory Systems of Agents

• Authors: Peng Liu, Zhen Tao, Jihao Zhao, Ding Chen, Yansong Zhang, Cuiping Li, Zhiyu Li, Hong Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23944
• Pdf URL: https://arxiv.org/pdf/2602.23944
• Copy Paste: [[2602.23944]] MemEmo: Evaluating Emotion in Memory Systems of Agents(https://arxiv.org/abs/2602.23944)
• Keywords: language model, agent
• Abstract: Memory systems address the challenge of context loss in Large Language Model during prolonged interactions. However, compared to human cognition, the efficacy of these systems in processing emotion-related information remains inconclusive. To address this gap, we propose an emotion-enhanced memory evaluation benchmark to assess the performance of mainstream and state-of-the-art memory systems in handling affective information. We developed the \textbf{H}uman-\textbf{L}ike \textbf{M}emory \textbf{E}motion (\textbf{HLME}) dataset, which evaluates memory systems across three dimensions: emotional information extraction, emotional memory updating, and emotional memory question answering. Experimental results indicate that none of the evaluated systems achieve robust performance across all three tasks. Our findings provide an objective perspective on the current deficiencies of memory systems in processing emotional memories and suggest a new trajectory for future research and system optimization.
• 摘要：记忆系统解决了大型语言模型在长时间交互过程中上下文丢失的挑战。然而，与人类认知相比，这些系统在处理情绪相关信息方面的功效仍然没有定论。为了解决这一差距，我们提出了一种情感增强记忆评估基准，以评估主流和最先进的记忆系统在处理情感信息方面的表现。我们开发了 \textbf{H}uman-\textbf{L}ike \textbf{M}emory \textbf{E}motion (\textbf{HLME}) 数据集，该数据集跨三个维度评估记忆系统：情绪信息提取、情绪记忆更新和情绪记忆问答。实验结果表明，所评估的系统都没有在所有三项任务中实现稳健的性能。我们的研究结果为当前记忆系统在处理情感记忆方面的缺陷提供了客观的视角，并为未来的研究和系统优化提出了新的轨迹。

Title: The GRADIEND Python Package: An End-to-End System for Gradient-Based Feature Learning

• Authors: Jonathan Drechsel, Steffen Herbold
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23993
• Pdf URL: https://arxiv.org/pdf/2602.23993
• Copy Paste: [[2602.23993]] The GRADIEND Python Package: An End-to-End System for Gradient-Based Feature Learning(https://arxiv.org/abs/2602.23993)
• Keywords: language model
• Abstract: We present gradiend, an open-source Python package that operationalizes the GRADIEND method for learning feature directions from factual-counterfactual MLM and CLM gradients in language models. The package provides a unified workflow for feature-related data creation, training, evaluation, visualization, persistent model rewriting via controlled weight updates, and multi-feature comparison. We demonstrate GRADIEND on an English pronoun paradigm and on a large-scale feature comparison that reproduces prior use cases.
• 摘要：我们提出了 gradiend，一个开源 Python 包，它可操作 GRADIEND 方法，用于从语言模型中的事实-反事实 MLM 和 CLM 梯度中学习特征方向。该软件包提供了一个统一的工作流程，用于与特征相关的数据创建、训练、评估、可视化、通过受控权重更新进行持久模型重写以及多特征比较。我们在英语代词范式和重现先前用例的大规模特征比较上演示了 GRADIEND。

Title: Task Complexity Matters: An Empirical Study of Reasoning in LLMs for Sentiment Analysis

• Authors: Donghao Huang, Zhaoxia Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.24060
• Pdf URL: https://arxiv.org/pdf/2602.24060
• Copy Paste: [[2602.24060]] Task Complexity Matters: An Empirical Study of Reasoning in LLMs for Sentiment Analysis(https://arxiv.org/abs/2602.24060)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) with reasoning capabilities have fueled a compelling narrative that reasoning universally improves performance across language tasks. We test this claim through a comprehensive evaluation of 504 configurations across seven model families--including adaptive, conditional, and reinforcement learning-based reasoning architectures--on sentiment analysis datasets of varying granularity (binary, five-class, and 27-class emotion). Our findings reveal that reasoning effectiveness is strongly task-dependent, challenging prevailing assumptions: (1) Reasoning shows task-complexity dependence--binary classification degrades up to -19.9 F1 percentage points (pp), while 27-class emotion recognition gains up to +16.0pp; (2) Distilled reasoning variants underperform base models by 3-18 pp on simpler tasks, though few-shot prompting enables partial recovery; (3) Few-shot learning improves over zero-shot in most cases regardless of model type, with gains varying by architecture and task complexity; (4) Pareto frontier analysis shows base models dominate efficiency-performance trade-offs, with reasoning justified only for complex emotion recognition despite 2.1x-54x computational overhead. We complement these quantitative findings with qualitative error analysis revealing that reasoning degrades simpler tasks through systematic over-deliberation, offering mechanistic insight beyond the high-level overthinking hypothesis.
• 摘要：具有推理能力的大型语言模型（LLM）催生了一种令人信服的说法，即推理普遍可以提高跨语言任务的性能。我们通过在不同粒度（二元、五类和 27 类情感）的情感分析数据集上对七个模型系列（包括自适应、条件和基于强化学习的推理架构）的 504 个配置进行综合评估来测试这一说法。我们的研究结果表明，推理有效性强烈依赖于任务，这对普遍的假设提出了挑战：(1) 推理显示出任务复杂性依赖性——二元分类下降高达 -19.9 F1 个百分点 (pp)，而 27 类情感识别则增益高达 +16.0 个百分点； (2) 尽管少量提示可以实现部分恢复，但在较简单的任务上，精炼推理变体的表现比基本模型低 3-18 个百分点； (3) 无论模型类型如何，在大多数情况下，小样本学习都比零样本学习有所改进，增益因架构和任务复杂性而异； (4) Pareto 前沿分析显示，基本模型主导着效率与性能的权衡，尽管计算开销为 2.1x-54x，但推理仅适用于复杂的情感识别。我们通过定性误差分析来补充这些定量发现，揭示推理通过系统性的过度深思熟虑降低了更简单的任务的质量，提供了超越高级过度思考假设的机械见解。

Title: Preference Packing: Efficient Preference Optimization for Large Language Models

• Authors: Jaekyung Cho
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.24082
• Pdf URL: https://arxiv.org/pdf/2602.24082
• Copy Paste: [[2602.24082]] Preference Packing: Efficient Preference Optimization for Large Language Models(https://arxiv.org/abs/2602.24082)
• Keywords: language model, llm, prompt
• Abstract: Resource-efficient training optimization techniques are becoming increasingly important as the size of large language models (LLMs) continues to grow. In particular, batch packing is commonly used in pre-training and supervised fine-tuning to achieve resource-efficient training. We propose preference packing, a method to enhance resource efficiency in training techniques that use data with different responses for the same input prompt, such as reward models or Direct Preference Optimization (DPO). Preference packing improves resource efficiency by reducing the attention operations for duplicate input prompts and decreasing KV cache memory usage. We conducted experiments on text-only datasets and image-included datasets and achieved at least 37% reduction in training time. Notably, this method can be applied alongside existing optimization techniques such as batch sorting, resulting in a 3.22x speedup.
• 摘要：随着大型语言模型 (LLM) 规模的不断增长，资源高效的训练优化技术变得越来越重要。特别是，批量打包通常用于预训练和监督微调，以实现资源高效的训练。我们提出了偏好打包，这是一种提高训练技术资源效率的方法，该方法使用对同一输入提示具有不同响应的数据，例如奖励模型或直接偏好优化（DPO）。偏好打包通过减少重复输入提示的注意力操作并减少 KV 缓存内存使用来提高资源效率。我们对纯文本数据集和包含图像的数据集进行了实验，训练时间减少了至少 37%。值得注意的是，该方法可以与批量排序等现有优化技术一起应用，从而实现 3.22 倍的加速。

Title: ARGUS: Seeing the Influence of Narrative Features on Persuasion in Argumentative Texts

• Authors: Sara Nabhani, Federico Pianzola, Khalid Al-Khatib, Malvina Nissim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.24109
• Pdf URL: https://arxiv.org/pdf/2602.24109
• Copy Paste: [[2602.24109]] ARGUS: Seeing the Influence of Narrative Features on Persuasion in Argumentative Texts(https://arxiv.org/abs/2602.24109)
• Keywords: language model, llm
• Abstract: Can narratives make arguments more persuasive? And to this end, which narrative features matter most? Although stories are often seen as powerful tools for persuasion, their specific role in online, unstructured argumentation remains underexplored. To address this gap, we present ARGUS, a framework for studying the impact of narration on persuasion in argumentative discourse. ARGUS introduces a new ChangeMyView corpus annotated for story presence and six key narrative features, integrating insights from two established theoretical frameworks that capture both textual narrative features and their effects on recipients. Leveraging both encoder-based classifiers and zero-shot large language models (LLMs), ARGUS identifies stories and narrative features and applies them at scale to examine how different narrative dimensions influence persuasion success in online argumentation.
• 摘要：叙述可以使论点更有说服力吗？为此，哪些叙事特征最重要？尽管故事通常被视为强大的说服工具，但它们在在线非结构化论证中的具体作用仍未得到充分探索。为了解决这一差距，我们提出了 ARGUS，一个用于研究叙事对论证性话语中说服的影响的框架。 ARGUS 推出了一个新的 ChangeMyView 语料库，该语料库针对故事呈现和六个关键叙事特征进行了注释，整合了两个已建立的理论框架的见解，这些理论框架捕获了文本叙事特征及其对接收者的影响。利用基于编码器的分类器和零样本大型语言模型 (LLM)，ARGUS 识别故事和叙事特征，并大规模应用它们来检查不同的叙事维度如何影响在线论证中的说服成功。

Title: Terminology Rarity Predicts Catastrophic Failure in LLM Translation of Low-Resource Ancient Languages: Evidence from Ancient Greek

• Authors: James L. Zainaldin, Cameron Pattison, Manuela Marai, Jacob Wu, Mark J. Schiefsky
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.24119
• Pdf URL: https://arxiv.org/pdf/2602.24119
• Copy Paste: [[2602.24119]] Terminology Rarity Predicts Catastrophic Failure in LLM Translation of Low-Resource Ancient Languages: Evidence from Ancient Greek(https://arxiv.org/abs/2602.24119)
• Keywords: language model, gpt, llm, chat
• Abstract: This study presents the first systematic, reference-free human evaluation of large language model (LLM) machine translation (MT) for Ancient Greek (AG) technical prose. We evaluate translations by three commercial LLMs (Claude, Gemini, ChatGPT) of twenty paragraph-length passages from two works by the Greek physician Galen of Pergamum (ca. 129-216 CE): On Mixtures, which has two published English translations, and On the Composition of Drugs according to Kinds, which has never been fully translated into English. We assess translation quality using both standard automated evaluation metrics (BLEU, chrF++, METEOR, ROUGE-L, BERTScore, COMET, BLEURT) and expert human evaluation via a modified Multidimensional Quality Metrics (MQM) framework applied to all 60 translations by a team of domain specialists. On the previously translated expository text, LLMs achieved high translation quality (mean MQM score 95.2/100), with performance approaching expert level. On the untranslated pharmacological text, aggregate quality was lower (79.9/100) but with high variance driven by two passages presenting extreme terminological density; excluding these, scores converged to within 4 points of the translated text. Terminology rarity, operationalized via corpus frequency in the literary Diorisis Ancient Greek Corpus, emerged as a strong predictor of translation failure (r = -.97 for passage-level quality on the untranslated text). Automated metrics showed moderate correlation with human judgment overall on the text with a wide quality spread (Composition), but no metric discriminated among high-quality translations. We discuss implications for the use of LLMs in Classical scholarship and for the design of automated evaluation pipelines for low-resource ancient languages.
• 摘要：这项研究首次对古希腊语 (AG) 技术散文的大语言模型 (LLM) 机器翻译 (MT) 进行了系统的、无参考的人类评估。我们评估了三位商业法学硕士（Claude、Gemini、ChatGPT）对希腊医生盖伦（Galen of Pergamum，约公元 129-216 年）两部著作中 20 个段落的翻译：《论混合物》（已出版两本英文译本）和《论药物的成分根据种类》（从未完全翻译成英文）。我们使用标准自动评估指标（BLEU、chrF++、METEOR、ROUGE-L、BERTScore、COMET、BLEURT）和专家人工评估（通过由领域专家团队将修改后的多维质量指标 (MQM) 框架应用于所有 60 个翻译）来评估翻译质量。在之前翻译的说明性文本中，法学硕士取得了很高的翻译质量（平均 MQM 分数 95.2/100），表现接近专家水平。在未翻译的药理学文本中，总体质量较低（79.9/100），但由于两段呈现极端术语密度的段落而导致方差较高；排除这些，分数收敛到翻译文本的 4 分以内。术语稀有性通过文学 Diorisis 古希腊语料库中的语料库频率进行操作，成为翻译失败的有力预测因素（未翻译文本的段落级别质量 r = -.97）。自动化指标显示出与人类对质量分布广泛（构图）的文本的总体判断具有中等相关性，但没有指标能够区分高质量翻译。我们讨论了法学硕士在古典学术中的使用以及对资源匮乏的古代语言的自动评估流程设计的影响。

Title: CoME: Empowering Channel-of-Mobile-Experts with Informative Hybrid-Capabilities Reasoning

• Authors: Yuxuan Liu, Weikai Xu, Kun Huang, Changyu Chen, Jiankun Zhao, Pengzhi Gao, Wei Liu, Jian Luan, Shuo Shang, Bo Du, Ji-Rong Wen, Rui Yan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.24142
• Pdf URL: https://arxiv.org/pdf/2602.24142
• Copy Paste: [[2602.24142]] CoME: Empowering Channel-of-Mobile-Experts with Informative Hybrid-Capabilities Reasoning(https://arxiv.org/abs/2602.24142)
• Keywords: agent
• Abstract: Mobile Agents can autonomously execute user instructions, which requires hybrid-capabilities reasoning, including screen summary, subtask planning, action decision and action function. However, existing agents struggle to achieve both decoupled enhancement and balanced integration of these capabilities. To address these challenges, we propose Channel-of-Mobile-Experts (CoME), a novel agent architecture consisting of four distinct experts, each aligned with a specific reasoning stage, CoME activates the corresponding expert to generate output tokens in each reasoning stage via output-oriented activation. To empower CoME with hybrid-capabilities reasoning, we introduce a progressive training strategy: Expert-FT enables decoupling and enhancement of different experts' capability; Router-FT aligns expert activation with the different reasoning stage; CoT-FT facilitates seamless collaboration and balanced optimization across multiple capabilities. To mitigate error propagation in hybrid-capabilities reasoning, we propose InfoGain-Driven DPO (Info-DPO), which uses information gain to evaluate the contribution of each intermediate step, thereby guiding CoME toward more informative reasoning. Comprehensive experiments show that CoME outperforms dense mobile agents and MoE methods on both AITZ and AMEX datasets.
• 摘要：移动代理可以自主执行用户指令，这需要混合能力推理，包括屏幕摘要、子任务规划、动作决策和动作函数。然而，现有的代理很难实现这些功能的解耦增强和平衡集成。为了应对这些挑战，我们提出了移动专家通道（CoME），这是一种新颖的代理架构，由四个不同的专家组成，每个专家都与特定的推理阶段对齐，CoME 通过面向输出的激活激活相应的专家在每个推理阶段生成输出令牌。为了赋予 CoME 混合能力推理能力，我们引入了渐进式训练策略：Expert-FT 能够解耦并增强不同专家的能力； Router-FT 将专家激活与不同的推理阶段结合起来； CoT-FT 促进跨多种功能的无缝协作和平衡优化。为了减轻混合能力推理中的错误传播，我们提出了 InfoGain-Driven DPO (Info-DPO)，它使用信息增益来评估每个中间步骤的贡献，从而指导 CoME 进行更多信息推理。综合实验表明，CoME 在 AITZ 和 AMEX 数据集上均优于密集移动代理和 MoE 方法。

Title: ArgLLM-App: An Interactive System for Argumentative Reasoning with Large Language Models

• Authors: Adam Dejl, Deniz Gorur, Francesca Toni
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.24172
• Pdf URL: https://arxiv.org/pdf/2602.24172
• Copy Paste: [[2602.24172]] ArgLLM-App: An Interactive System for Argumentative Reasoning with Large Language Models(https://arxiv.org/abs/2602.24172)
• Keywords: language model, llm, agent
• Abstract: Argumentative LLMs (ArgLLMs) are an existing approach leveraging Large Language Models (LLMs) and computational argumentation for decision-making, with the aim of making the resulting decisions faithfully explainable to and contestable by humans. Here we propose a web-based system implementing ArgLLM-empowered agents for binary tasks. ArgLLM-App supports visualisation of the produced explanations and interaction with human users, allowing them to identify and contest any mistakes in the system's reasoning. It is highly modular and enables drawing information from trusted external sources. ArgLLM-App is publicly available at this https URL, with a video demonstration at this https URL.
• 摘要：论证性 LLM (ArgLLM) 是一种利用大型语言模型 (LLM) 和计算论证进行决策的现有方法，旨在使最终的决策能够忠实地向人类解释并可供人类质疑。在这里，我们提出了一个基于 Web 的系统，实现了 ArgLLM 授权的二进制任务代理。 ArgLLM-App 支持生成的解释的可视化以及与人类用户的交互，使他们能够识别和质疑系统推理中的任何错误。它是高度模块化的，可以从可信的外部来源获取信息。 ArgLLM-App 可在此 https URL 公开获取，并在此 https URL 提供视频演示。

Title: Task-Centric Acceleration of Small-Language Models

• Authors: Dor Tsur, Sharon Adar, Ran Levy
• Subjects: cs.CL, cs.AI, cs.IT
• Abstract URL: https://arxiv.org/abs/2602.24174
• Pdf URL: https://arxiv.org/pdf/2602.24174
• Copy Paste: [[2602.24174]] Task-Centric Acceleration of Small-Language Models(https://arxiv.org/abs/2602.24174)
• Keywords: language model
• Abstract: Small language models (SLMs) have emerged as efficient alternatives to large language models for task-specific applications. However, they are often employed in high-volume, low-latency settings, where efficiency is crucial. We propose TASC, Task-Adaptive Sequence Compression, a framework for SLM acceleration comprising two use-cases: When performing SLM fine-tuning, we propose TASC-ft, which iteratively enriches the tokenizer vocabulary with high-frequency output n-grams and then fine-tunes the model to utilize the expanded vocabulary. Next, we propose an inference-time method, termed TASC-spec. TASC-spec is a lightweight, training-free speculative decoding method that constructs an n-gram draft model from the task's output corpus, mixing task and context n-gram this http URL-spec avoids any additional training, while bypassing draft-target vocabulary alignment constraints. We demonstrate the effectiveness of both methods across multiple low output-variability generation tasks. Our methods show consistent improvements in inference efficiency while maintaining task performance.
• 摘要：小语言模型 (SLM) 已成为针对特定任务应用程序的大语言模型的有效替代方案。然而，它们通常用于高容量、低延迟的环境中，在这些环境中，效率至关重要。我们提出了 TASC（任务自适应序列压缩），这是一个 SLM 加速框架，包含两个用例：在执行 SLM 微调时，我们提出了 TASC-ft，它通过高频输出 n-gram 迭代丰富分词器词汇，然后微调模型以利用扩展的词汇。接下来，我们提出了一种推理时间方法，称为 TASC-spec。 TASC-spec 是一种轻量级、免训练的推测性解码方法，它从任务的输出语料库构建 n-gram 草稿模型，混合任务和上下文 n-gram，此 http URL-spec 避免了任何额外的训练，同时绕过草稿目标词汇对齐约束。我们证明了这两种方法在多个低输出变异性生成任务中的有效性。我们的方法在保持任务性能的同时显示了推理效率的持续改进。

Title: MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Games

• Authors: Jacob Eisenstein, Fantine Huot, Adam Fisch, Jonathan Berant, Mirella Lapata
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.24188
• Pdf URL: https://arxiv.org/pdf/2602.24188
• Copy Paste: [[2602.24188]] MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Games(https://arxiv.org/abs/2602.24188)
• Keywords: language model, agent
• Abstract: We present a scalable methodology for evaluating language models in multi-turn interactions, using a suite of collaborative games that require effective communication about private information. This enables an interactive scaling analysis, in which a fixed token budget is divided over a variable number of turns. We find that in many cases, language models are unable to use interactive collaboration to improve over the non-interactive baseline scenario in which one agent attempts to summarize its information and the other agent immediately acts -- despite substantial headroom. This suggests that state-of-the-art models still suffer from significant weaknesses in planning and executing multi-turn collaborative conversations. We analyze the linguistic features of these dialogues, assessing the roles of sycophancy, information density, and discourse coherence. While there is no single linguistic explanation for the collaborative weaknesses of contemporary language models, we note that humans achieve comparable task success at superior token efficiency by producing dialogues that are more coherent than those produced by most language models. The proactive management of private information is a defining feature of real-world communication, and we hope that MT-PingEval will drive further work towards improving this capability.
• 摘要：我们提出了一种可扩展的方法，用于评估多轮交互中的语言模型，使用一套需要有效沟通私人信息的协作游戏。这使得交互式扩展分析成为可能，其中固定的代币预算被划分为可变的轮数。我们发现，在许多情况下，语言模型无法使用交互式协作来改进非交互式基线场景，在非交互式基线场景中，一个智能体试图总结其信息，而另一个智能体立即采取行动——尽管有很大的空间。这表明最先进的模型在规划和执行多轮协作对话方面仍然存在明显的弱点。我们分析这些对话的语言特征，评估阿谀奉承、信息密度和话语连贯性的作用。虽然对于当代语言模型的协作弱点没有单一的语言学解释，但我们注意到，人类通过产生比大多数语言模型产生的对话更连贯的对话，以卓越的符号效率实现了类似的任务成功。主动管理私人信息是现实世界通信的一个决定性特征，我们希望 MT-PingEval 能够推动进一步提高这一能力。

Title: Controllable Reasoning Models Are Private Thinkers

• Authors: Haritz Puerto, Haonan Li, Xudong Han, Timothy Baldwin, Iryna Gurevych
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.24210
• Pdf URL: https://arxiv.org/pdf/2602.24210
• Copy Paste: [[2602.24210]] Controllable Reasoning Models Are Private Thinkers(https://arxiv.org/abs/2602.24210)
• Keywords: agent
• Abstract: AI agents powered by reasoning models require access to sensitive user data. However, their reasoning traces are difficult to control, which can result in the unintended leakage of private information to external parties. We propose training models to follow instructions not only in the final answer, but also in reasoning traces, potentially under different constraints. We hypothesize that improving their instruction following abilities in the reasoning traces can improve their privacy-preservation skills. To demonstrate this, we fine-tune models on a new instruction-following dataset with explicit restrictions on reasoning traces. We further introduce a generation strategy that decouples reasoning and answer generation using separate LoRA adapters. We evaluate our approach on six models from two model families, ranging from 1.7B to 14B parameters, across two instruction-following benchmarks and two privacy benchmarks. Our method yields substantial improvements, achieving gains of up to 20.9 points in instruction-following performance and up to 51.9 percentage points on privacy benchmarks. These improvements, however, can come at the cost of task utility, due to the trade-off between reasoning performance and instruction-following abilities. Overall, our results show that improving instruction-following behavior in reasoning models can significantly enhance privacy, suggesting a promising direction for the development of future privacy-aware agents. Our code and data are available at this https URL
• 摘要：由推理模型支持的人工智能代理需要访问敏感的用户数据。然而，它们的推理痕迹难以控制，这可能导致私人信息无意中泄露给外部各方。我们建议训练模型不仅遵循最终答案中的指令，而且还遵循可能在不同约束下的推理轨迹。我们假设提高他们在推理轨迹中遵循指令的能力可以提高他们的隐私保护技能。为了证明这一点，我们在新的指令跟踪数据集上微调模型，并对推理轨迹进行明确的限制。我们进一步介绍了一种生成策略，该策略使用单独的 LoRA 适配器将推理和答案生成解耦。我们在两个模型系列的六个模型上评估我们的方法，参数范围从 1.7B 到 14B，跨越两个指令跟踪基准和两个隐私基准。我们的方法取得了显着的改进，在指令跟踪性能方面实现了高达 20.9 个百分点的提升，在隐私基准方面实现了高达 51.9 个百分点的提升。然而，由于推理性能和指令跟踪能力之间的权衡，这些改进可能会以任务效用为代价。总的来说，我们的结果表明，改善推理模型中的指令遵循行为可以显着增强隐私，这为未来隐私感知代理的发展提供了一个有希望的方向。我们的代码和数据可在此 https URL 获取

Title: Do LLMs Benefit From Their Own Words?

• Authors: Jenny Y. Huang, Leshem Choshen, Ramon Astudillo, Tamara Broderick, Jacob Andreas
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.24287
• Pdf URL: https://arxiv.org/pdf/2602.24287
• Copy Paste: [[2602.24287]] Do LLMs Benefit From Their Own Words?(https://arxiv.org/abs/2602.24287)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Multi-turn interactions with large language models typically retain the assistant's own past responses in the conversation history. In this work, we revisit this design choice by asking whether large language models benefit from conditioning on their own prior responses. Using in-the-wild, multi-turn conversations, we compare standard (full-context) prompting with a user-turn-only prompting approach that omits all previous assistant responses, across three open reasoning models and one state-of-the-art model. To our surprise, we find that removing prior assistant responses does not affect response quality on a large fraction of turns. Omitting assistant-side history can reduce cumulative context lengths by up to 10x. To explain this result, we find that multi-turn conversations consist of a substantial proportion (36.4%) of self-contained prompts, and that many follow-up prompts provide sufficient instruction to be answered using only the current user turn and prior user turns. When analyzing cases where user-turn-only prompting substantially outperforms full context, we identify instances of context pollution, in which models over-condition on their previous responses, introducing errors, hallucinations, or stylistic artifacts that propagate across turns. Motivated by these findings, we design a context-filtering approach that selectively omits assistant-side context. Our findings suggest that selectively omitting assistant history can improve response quality while reducing memory consumption.
• 摘要：与大型语言模型的多轮交互通常会在对话历史记录中保留助手自己过去的响应。在这项工作中，我们通过询问大型语言模型是否受益于它们自己先前的响应来重新审视这种设计选择。使用野外多回合对话，我们将标准（全上下文）提示与用户仅回合提示方法进行比较，该方法忽略了所有先前的助理响应，跨越三种开放推理模型和一种最先进的模型。令我们惊讶的是，我们发现删除先前的助理响应并不会影响大部分转弯的响应质量。省略助手端历史记录可以将累积上下文长度减少多达 10 倍。为了解释这个结果，我们发现多轮对话包含很大比例（36.4％）的独立提示，并且许多后续提示提供了足够的指令，仅使用当前用户轮次和先前用户轮次即可回答。在分析仅用户轮次提示明显优于完整上下文的情况时，我们识别了上下文污染的实例，其中模型对其先前的响应进行了过度调节，引入了错误、幻觉或跨轮次传播的风格伪像。受这些发现的启发，我们设计了一种上下文过滤方法，有选择地忽略助手端上下文。我们的研究结果表明，有选择地省略助理历史记录可以提高响应质量，同时减少内存消耗。