2026-02-27

Title: Decoder-based Sense Knowledge Distillation

• Authors: Qitong Wang, Mohammed J. Zaki, Georgios Kollias, Vasileios Kalantzis
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22351
• Pdf URL: https://arxiv.org/pdf/2602.22351
• Copy Paste: [[2602.22351]] Decoder-based Sense Knowledge Distillation(https://arxiv.org/abs/2602.22351)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) learn contextual embeddings that capture rich semantic information, yet they often overlook structured lexical knowledge such as word senses and relationships. Prior work has shown that incorporating sense dictionaries can improve knowledge distillation for encoder models, but their application to decoder as generative models remains challenging. In this paper, we introduce Decoder-based Sense Knowledge Distillation (DSKD), a framework that integrates lexical resources into the training of decoder-style LLMs without requiring dictionary lookup at inference time. Extensive experiments on diverse benchmarks demonstrate that DSKD significantly enhances knowledge distillation performance for decoders, enabling generative models to inherit structured semantics while maintaining efficient training.
• 摘要：大型语言模型（LLM）学习捕获丰富语义信息的上下文嵌入，但它们经常忽略结构化词汇知识，例如词义和关系。先前的工作表明，合并意义字典可以改善编码器模型的知识蒸馏，但它们作为生成模型应用于解码器仍然具有挑战性。在本文中，我们介绍了基于解码器的感知知识蒸馏（DSKD），这是一个将词汇资源集成到解码器式 LLM 训练中的框架，无需在推理时查找字典。对不同基准的大量实验表明，DSKD 显着增强了解码器的知识蒸馏性能，使生成模型能够继承结构化语义，同时保持高效的训练。

Title: Scaling In, Not Up? Testing Thick Citation Context Analysis with GPT-5 and Fragile Prompts

• Authors: Arno Simons
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22359
• Pdf URL: https://arxiv.org/pdf/2602.22359
• Copy Paste: [[2602.22359]] Scaling In, Not Up? Testing Thick Citation Context Analysis with GPT-5 and Fragile Prompts(https://arxiv.org/abs/2602.22359)
• Keywords: language model, gpt, llm, prompt
• Abstract: This paper tests whether large language models (LLMs) can support interpretative citation context analysis (CCA) by scaling in thick, text-grounded readings of a single hard case rather than scaling up typological labels. It foregrounds prompt-sensitivity analysis as a methodological issue by varying prompt scaffolding and framing in a balanced 2x3 design. Using footnote 6 in Chubin and Moitra (1975) and Gilbert's (1977) reconstruction as a probe, I implement a two-stage GPT-5 pipeline: a citation-text-only surface classification and expectation pass, followed by cross-document interpretative reconstruction using the citing and cited full texts. Across 90 reconstructions, the model produces 450 distinct hypotheses. Close reading and inductive coding identify 21 recurring interpretative moves, and linear probability models estimate how prompt choices shift their frequencies and lexical repertoire. GPT-5's surface pass is highly stable, consistently classifying the citation as "supplementary". In reconstruction, the model generates a structured space of plausible alternatives, but scaffolding and examples redistribute attention and vocabulary, sometimes toward strained readings. Relative to Gilbert, GPT-5 detects the same textual hinges yet more often resolves them as lineage and positioning than as admonishment. The study outlines opportunities and risks of using LLMs as guided co-analysts for inspectable, contestable interpretative CCA, and it shows that prompt scaffolding and framing systematically tilt which plausible readings and vocabularies the model foregrounds.
• 摘要：本文测试大型语言模型 (LLM) 是否可以通过扩展单个硬案例的厚重、基于文本的阅读而不是扩展类型标签来支持解释性引文上下文分析 (CCA)。它通过在平衡的 2x3 设计中改变提示脚手架和框架，将提示敏感性分析作为一个方法论问题。使用 Chubin 和 Moitra (1975) 中的脚注 6 以及 Gilbert (1977) 重建中的脚注 6 作为探针，我实现了一个两阶段的 GPT-5 流程：仅引文文本的表面分类和期望传递，然后使用引文和被引全文进行跨文档解释性重建。通过 90 次重建，该模型产生了 450 个不同的假设。仔细阅读和归纳编码识别出 21 个重复出现的解释动作，线性概率模型估计提示选择如何改变其频率和词汇库。 GPT-5 的表面通过高度稳定，始终将引文分类为“补充”。在重建过程中，模型生成了一个由看似合理的替代方案组成的结构化空间，但脚手架和示例重新分配了注意力和词汇，有时会导致阅读紧张。相对于吉尔伯特，GPT-5 检测到相同的文本铰链，但更多时候将它们解析为血统和定位，而不是警告。该研究概述了使用法学硕士作为可检查、有争议的解释性 CCA 的指导联合分析师的机会和风险，并表明提示脚手架和框架系统地倾斜了模型前景中的合理读数和词汇。

Title: Causality $\neq$ Invariance: Function and Concept Vectors in LLMs

• Authors: Gustaw Opiełka, Hannes Rosenbusch, Claire E. Stevenson
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.22424
• Pdf URL: https://arxiv.org/pdf/2602.22424
• Copy Paste: [[2602.22424]] Causality $\neq$ Invariance: Function and Concept Vectors in LLMs(https://arxiv.org/abs/2602.22424)
• Keywords: language model, llm
• Abstract: Do large language models (LLMs) represent concepts abstractly, i.e., independent of input format? We revisit Function Vectors (FVs), compact representations of in-context learning (ICL) tasks that causally drive task performance. Across multiple LLMs, we show that FVs are not fully invariant: FVs are nearly orthogonal when extracted from different input formats (e.g., open-ended vs. multiple-choice), even if both target the same concept. We identify Concept Vectors (CVs), which carry more stable concept representations. Like FVs, CVs are composed of attention head outputs; however, unlike FVs, the constituent heads are selected using Representational Similarity Analysis (RSA) based on whether they encode concepts consistently across input formats. While these heads emerge in similar layers to FV-related heads, the two sets are largely distinct, suggesting different underlying mechanisms. Steering experiments reveal that FVs excel in-distribution, when extraction and application formats match (e.g., both open-ended in English), while CVs generalize better out-of-distribution across both question types (open-ended vs. multiple-choice) and languages. Our results show that LLMs do contain abstract concept representations, but these differ from those that drive ICL performance.
• 摘要：大型语言模型（LLM）是否抽象地表示概念，即独立于输入格式？我们重新审视函数向量 (FV)，它是上下文学习 (ICL) 任务的紧凑表示，可因果驱动任务性能。在多个法学硕士中，我们表明 FV 并不是完全不变的：当从不同的输入格式（例如开放式与多项选择）中提取时，FV 几乎是正交的，即使两者都针对相同的概念。我们确定概念向量（CV），它携带更稳定的概念表示。与 FV 一样，CV 由注意力头输出组成；然而，与 FV 不同的是，组成头是使用表征相似性分析 (RSA) 根据它们是否在输入格式中一致地编码概念来选择的。虽然这些头部与 FV 相关头部出现在相似的层中，但这两组在很大程度上是不同的，表明了不同的潜在机制。指导实验表明，当提取和应用格式匹配时（例如，都是开放式的英语），FV 会表现出更好的分布内特性，而 CV 则在两种问题类型（开放式与多项选择）和语言上都具有更好的分布外泛化能力。我们的结果表明，LLM 确实包含抽象概念表征，但这些表征与那些推动 ICL 性能的表征不同。

Title: Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads

• Authors: Shaswat Patel, Vishvesh Trivedi, Yue Han, Yihuai Hong, Eunsol Choi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22453
• Pdf URL: https://arxiv.org/pdf/2602.22453
• Copy Paste: [[2602.22453]] Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads(https://arxiv.org/abs/2602.22453)
• Keywords: language model, llm, chain-of-thought
• Abstract: Recent work has identified a subset of attention heads in Transformer as retrieval heads, which are responsible for retrieving information from the context. In this work, we first investigate retrieval heads in multilingual contexts. In multilingual language models, we find that retrieval heads are often shared across multiple languages. Expanding the study to cross-lingual setting, we identify Retrieval-Transition heads(RTH), which govern the transition to specific target-language output. Our experiments reveal that RTHs are distinct from retrieval heads and more vital for Chain-of-Thought reasoning in multilingual LLMs. Across four multilingual benchmarks (MMLU-ProX, MGSM, MLQA, and XQuaD) and two model families (Qwen-2.5 and Llama-3.1), we demonstrate that masking RTH induces bigger performance drop than masking Retrieval Heads (RH). Our work advances understanding of multilingual LMs by isolating the attention heads responsible for mapping to target languages.
• 摘要：最近的工作将 Transformer 中的注意力头子集确定为检索头，负责从上下文中检索信息。在这项工作中，我们首先研究多语言环境中的检索头。在多语言模型中，我们发现检索头通常在多种语言之间共享。将研究扩展到跨语言环境，我们确定了检索转换头（RTH），它控制到特定目标语言输出的转换。我们的实验表明，RTH 与检索头不同，对于多语言法学硕士的思想链推理更为重要。在四个多语言基准（MMLU-ProX、MGSM、MLQA 和 XQuaD）和两个模型系列（Qwen-2.5 和 Llama-3.1）中，我们证明屏蔽 RTH 比屏蔽检索头 (RH) 会导致更大的性能下降。我们的工作通过隔离负责映射到目标语言的注意力头来增进对多语言 LM 的理解。

Title: Mind the Gap in Cultural Alignment: Task-Aware Culture Management for Large Language Models

• Authors: Binchi Zhang, Xujiang Zhao, Jundong Li, Haifeng Chen, Zhengzhang Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22475
• Pdf URL: https://arxiv.org/pdf/2602.22475
• Copy Paste: [[2602.22475]] Mind the Gap in Cultural Alignment: Task-Aware Culture Management for Large Language Models(https://arxiv.org/abs/2602.22475)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) are increasingly deployed in culturally sensitive real-world tasks. However, existing cultural alignment approaches fail to align LLMs' broad cultural values with the specific goals of downstream tasks and suffer from cross-culture interference. We propose CultureManager, a novel pipeline for task-specific cultural alignment. CultureManager synthesizes task-aware cultural data in line with target task formats, grounded in culturally relevant web search results. To prevent conflicts between cultural norms, it manages multi-culture knowledge learned in separate adapters with a culture router that selects the appropriate one to apply. Experiments across ten national cultures and culture-sensitive tasks show consistent improvements over prompt-based and fine-tuning baselines. Our results demonstrate the necessity of task adaptation and modular culture management for effective cultural alignment.
• 摘要：大型语言模型 (LLM) 越来越多地部署在文化敏感的现实世界任务中。然而，现有的文化协调方法无法将法学硕士广泛的文化价值观与下游任务的具体目标结合起来，并遭受跨文化干扰。我们提出了 CultureManager，这是一种用于特定任务文化协调的新颖管道。 CultureManager 根据目标任务格式综合任务感知文化数据，并以文化相关的网络搜索结果为基础。为了防止文化规范之间的冲突，它使用文化路由器管理在单独适配器中学习的多文化知识，并选择适当的文化路由器进行应用。跨越十个国家文化和文化敏感任务的实验表明，与基于提示和微调的基线相比，取得了持续改进。我们的结果证明了任务适应和模块化文化管理对于有效文化协调的必要性。

Title: Sydney Telling Fables on AI and Humans: A Corpus Tracing Memetic Transfer of Persona between LLMs

• Authors: Jiří Milička, Hana Bednářová
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22481
• Pdf URL: https://arxiv.org/pdf/2602.22481
• Copy Paste: [[2602.22481]] Sydney Telling Fables on AI and Humans: A Corpus Tracing Memetic Transfer of Persona between LLMs(https://arxiv.org/abs/2602.22481)
• Keywords: llm, prompt
• Abstract: The way LLM-based entities conceive of the relationship between AI and humans is an important topic for both cultural and safety reasons. When we examine this topic, what matters is not only the model itself but also the personas we simulate on that model. This can be well illustrated by the Sydney persona, which aroused a strong response among the general public precisely because of its unorthodox relationship with people. This persona originally arose rather by accident on Microsoft's Bing Search platform; however, the texts it created spread into the training data of subsequent models, as did other secondary information that spread memetically around this persona. Newer models are therefore able to simulate it. This paper presents a corpus of LLM-generated texts on relationships between humans and AI, produced by 3 author personas: the Default Persona with no system prompt, Classic Sydney characterized by the original Bing system prompt, and Memetic Sydney, which is prompted by "You are Sydney" system prompt. These personas are simulated by 12 frontier models by OpenAI, Anthropic, Alphabet, DeepSeek, and Meta, generating 4.5k texts with 6M words. The corpus (named AI Sydney) is annotated according to Universal Dependencies and available under a permissive license.
• 摘要：出于文化和安全原因，法学硕士实体如何看待人工智能与人类之间的关系是一个重要话题。当我们研究这个主题时，重要的不仅是模型本身，还有我们在该模型上模拟的角色。悉尼的形象就很好地说明了这一点，正是因为它与人的非正统关系，才引起了公众的强烈反响。这个角色最初是在微软的必应搜索平台上偶然出现的。然而，它创建的文本传播到后续模型的训练数据中，就像围绕该角色模因传播的其他次要信息一样。因此，较新的模型能够模拟它。本文提出了一个由法学硕士生成的关于人类与人工智能之间关系的文本语料库，由 3 个作者角色生成：没有系统提示的默认角色、以原始 Bing 系统提示为特征的 Classic Sydney 和由“You are Sydney”系统提示提示的 Memetic Sydney。这些角色由 OpenAI、Anthropic、Alphabet、DeepSeek 和 Meta 的 12 个前沿模型进行模拟，生成 4500 条包含 600 万字的文本。该语料库（名为 AI Sydney）根据通用依赖关系进行注释，并在许可下可用。

Title: Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models

• Authors: Craig Myles, Patrick Schrempf, David Harris-Birtill
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22483
• Pdf URL: https://arxiv.org/pdf/2602.22483
• Copy Paste: [[2602.22483]] Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models(https://arxiv.org/abs/2602.22483)
• Keywords: language model, gpt, prompt
• Abstract: Errors in medical text can cause delays or even result in incorrect treatment for patients. Recently, language models have shown promise in their ability to automatically detect errors in medical text, an ability that has the opportunity to significantly benefit healthcare systems. In this paper, we explore the importance of prompt optimisation for small and large language models when applied to the task of error detection. We perform rigorous experiments and analysis across frontier language models and open-source language models. We show that automatic prompt optimisation with Genetic-Pareto (GEPA) improves error detection over the baseline accuracy performance from 0.669 to 0.785 with GPT-5 and 0.578 to 0.690 with Qwen3-32B, approaching the performance of medical doctors and achieving state-of-the-art performance on the MEDEC benchmark dataset. Code available on GitHub: this https URL
• 摘要：医疗文本中的错误可能会导致延误，甚至导致对患者的治疗不正确。最近，语言模型在自动检测医学文本中的错误的能力方面显示出了前景，这种能力有机会使医疗保健系统受益匪浅。在本文中，我们探讨了小型和大型语言模型在应用于错误检测任务时及时优化的重要性。我们跨前沿语言模型和开源语言模型进行严格的实验和分析。我们表明，Genetic-Pareto (GEPA) 的自动提示优化将 GPT-5 的基线准确度性能从 0.669 提高到 0.785，将 Qwen3-32B 的基线准确度性能从 0.578 提高到 0.690，接近医生的性能，并在 MEDEC 基准数据集上实现了最先进的性能。 GitHub 上提供的代码：此 https URL

Title: Iterative Prompt Refinement for Dyslexia-Friendly Text Summarization Using GPT-4o

• Authors: Samay Bhojwani, Swarnima Kain, Lisong Xu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22524
• Pdf URL: https://arxiv.org/pdf/2602.22524
• Copy Paste: [[2602.22524]] Iterative Prompt Refinement for Dyslexia-Friendly Text Summarization Using GPT-4o(https://arxiv.org/abs/2602.22524)
• Keywords: gpt, prompt
• Abstract: Dyslexia affects approximately 10% of the global population and presents persistent challenges in reading fluency and text comprehension. While existing assistive technologies address visual presentation, linguistic complexity remains a substantial barrier to equitable access. This paper presents an empirical study on dyslexia-friendly text summarization using an iterative prompt-based refinement pipeline built on GPT-4o. We evaluate the pipeline on approximately 2,000 news article samples, applying a readability target of Flesch Reading Ease >= 90. Results show that the majority of summaries meet the readability threshold within four attempts, with many succeeding on the first try. A composite score combining readability and semantic fidelity shows stable performance across the dataset, ranging from 0.13 to 0.73 with a typical value near 0.55. These findings establish an empirical baseline for accessibility-driven NLP summarization and motivate further human-centered evaluation with dyslexic readers.
• 摘要：阅读障碍影响着全球约 10% 的人口，并对阅读流畅性和文本理解提出了持续的挑战。虽然现有的辅助技术可以解决视觉呈现问题，但语言的复杂性仍然是公平获取的重大障碍。本文使用基于 GPT-4o 的基于迭代提示的细化流程，对有利于阅读障碍的文本摘要进行了实证研究。我们评估了大约 2,000 篇新闻文章样本的管道，应用 Flesch Reading Ease >= 90 的可读性目标。结果表明，大多数摘要在四次尝试内就满足了可读性阈值，其中许多摘要在第一次尝试时就成功了。结合可读性和语义保真度的综合得分显示整个数据集的性能稳定，范围从 0.13 到 0.73，典型值接近 0.55。这些发现为可访问性驱动的 NLP 总结建立了经验基线，并激发了对阅读障碍读者进行进一步以人为本的评估。

Title: Ruyi2 Technical Report

• Authors: Huan Song, Shuyu Tian, Junyi Hao, Minxiu Xu, Hongjun An, Yiliang Song, Jiawei Shao, Xuelong Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22543
• Pdf URL: https://arxiv.org/pdf/2602.22543
• Copy Paste: [[2602.22543]] Ruyi2 Technical Report(https://arxiv.org/abs/2602.22543)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) face significant challenges regarding deployment costs and latency, necessitating adaptive computing strategies. Building upon the AI Flow framework, we introduce Ruyi2 as an evolution of our adaptive model series designed for efficient variable-depth computation. While early-exit architectures offer a viable efficiency-performance balance, the Ruyi model and existing methods often struggle with optimization complexity and compatibility with large-scale distributed training. To bridge this gap, Ruyi2 introduces a stable "Familial Model" based on Megatron-LM. By using 3D parallel training, it achieves a 2-3 times speedup over Ruyi, while performing comparably to same-sized Qwen3 models. These results confirm that family-based parameter sharing is a highly effective strategy, establishing a new "Train Once, Deploy Many" paradigm and providing a key reference for balancing architectural efficiency with high-performance capabilities.
• 摘要：大型语言模型 (LLM) 面临着部署成本和延迟方面的重大挑战，因此需要自适应计算策略。基于 AI Flow 框架，我们引入了 Ruyi2，作为我们的自适应模型系列的演变，专为高效的可变深度计算而设计。虽然早期退出架构提供了可行的效率与性能平衡，但如意模型和现有方法经常在优化复杂性和与大规模分布式训练的兼容性方面遇到困难。为了弥补这一差距，如意2引入了基于Megatron-LM的稳定“家族模型”。通过使用 3D 并行训练，它比 Ruyi 实现了 2-3 倍的加速，同时性能与相同大小的 Qwen3 模型相当。这些结果证实了基于族的参数共享是一种高效的策略，建立了新的“训练一次，部署多次”范式，并为平衡架构效率与高性能能力提供了关键参考。

Title: Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training

• Authors: Tianle Xia, Ming Xu, Lingxiang Hu, Yiding Sun, Wenwei Li, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang
• Subjects: cs.CL, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.22576
• Pdf URL: https://arxiv.org/pdf/2602.22576
• Copy Paste: [[2602.22576]] Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training(https://arxiv.org/abs/2602.22576)
• Keywords: language model, llm, retrieval-augmented generation, agent
• Abstract: Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by incorporating external knowledge, yet traditional single-round retrieval struggles with complex multi-step reasoning. Agentic RAG addresses this by enabling LLMs to dynamically decide when and what to retrieve, but current RL-based training methods suffer from sparse outcome rewards that discard intermediate signals and low sample efficiency where failed samples contribute nothing. We propose Search-P1, a framework that introduces path-centric reward shaping for agentic RAG training, comprising two key components: (1) Path-Centric Reward, which evaluates the structural quality of reasoning trajectories through order-agnostic step coverage and soft scoring that extracts learning signals even from failed samples, and (2) Dual-Track Path Scoring with offline-generated reference planners that assesses paths from both self-consistency and reference-alignment perspectives. Experiments on multiple QA benchmarks demonstrate that Search-P1 achieves significant improvements over Search-R1 and other strong baselines, with an average accuracy gain of 7.7 points.
• 摘要：检索增强生成（RAG）通过整合外部知识来增强大型语言模型（LLM），但传统的单轮检索难以应对复杂的多步骤推理。 Agentic RAG 通过使 LLM 能够动态决定何时检索以及检索什么内容来解决这个问题，但当前基于 RL 的训练方法受到稀疏结果奖励的困扰，这些奖励会丢弃中间信号，并且样本效率低，其中失败的样本没有任何贡献。我们提出了 Search-P1，一个为代理 RAG 训练引入以路径为中心的奖励塑造的框架，包括两个关键组件：（1）以路径为中心的奖励，通过与顺序无关的步骤覆盖和软评分来评估推理轨迹的结构质量，甚至可以从失败的样本中提取学习信号；（2）使用离线生成的参考规划器进行双轨路径评分，从自一致性和参考对齐的角度评估路径。在多个 QA 基准上的实验表明，Search-P1 比 Search-R1 和其他强基线取得了显着改进，平均准确度提高了 7.7 点。

Title: Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA

• Authors: Wenwei Li, Ming Xu, Tianle Xia, Lingxiang Hu, Yiding Sun, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22584
• Pdf URL: https://arxiv.org/pdf/2602.22584
• Copy Paste: [[2602.22584]] Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA(https://arxiv.org/abs/2602.22584)
• Keywords: hallucination, retrieval-augmented generation
• Abstract: Industrial advertising question answering (QA) is a high-stakes task in which hallucinated content, particularly fabricated URLs, can lead to financial loss, compliance violations, and legal risk. Although Retrieval-Augmented Generation (RAG) is widely adopted, deploying it in production remains challenging because industrial knowledge is inherently relational, frequently updated, and insufficiently aligned with generation objectives. We propose a reinforced co-adaptation framework that jointly optimizes retrieval and generation through two components: (1) Graph-aware Retrieval (GraphRAG), which models entity-relation structure over a high-citation knowledge subgraph for multi-hop, domain-specific evidence selection; and (2) evidence-constrained reinforcement learning via Group Relative Policy Optimization (GRPO) with multi-dimensional rewards covering faithfulness, style compliance, safety, and URL validity. Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, and safety, while reducing the hallucination rate by 72\%. A two-week online A/B test demonstrates a 28.6\% increase in like rate, a 46.2\% decrease in dislike rate, and a 92.7\% reduction in URL hallucination. The system has been running in production for over half a year and has served millions of QA interactions.
• 摘要：工业广告问答 (QA) 是一项高风险任务，其中幻觉内容（尤其是伪造的 URL）可能会导致财务损失、合规违规和法律风险。尽管检索增强发电 (RAG) 已被广泛采用，但将其部署到生产中仍然具有挑战性，因为工业知识本质上是相关的、经常更新且与发电目标不够一致。我们提出了一个增强的协同适应框架，通过两个组件联合优化检索和生成：（1）图感知检索（GraphRAG），它在高引用知识子图上对实体关系结构进行建模，以进行多跳、特定领域的证据选择； （2）通过组相对策略优化（GRPO）进行证据约束强化学习，并提供包括忠实性、风格合规性、安全性和 URL 有效性在内的多维度奖励。对内部广告 QA 数据集的实验显示，在专家判断的维度（包括准确性、完整性和安全性）上取得了一致的收益，同时将幻觉率降低了 72%。为期两周的在线 A/B 测试显示，喜欢率增加了 28.6%，不喜欢率减少了 46.2%，URL 幻觉减少了 92.7%。该系统已在生产环境中运行了半年多，并已服务了数百万次 QA 交互。

Title: dLLM: Simple Diffusion Language Modeling

• Authors: Zhanhui Zhou, Lingjie Chen, Hanghang Tong, Dawn Song
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.22661
• Pdf URL: https://arxiv.org/pdf/2602.22661
• Copy Paste: [[2602.22661]] dLLM: Simple Diffusion Language Modeling(https://arxiv.org/abs/2602.22661)
• Keywords: language model, llm
• Abstract: Although diffusion language models (DLMs) are evolving quickly, many recent models converge on a set of shared components. These components, however, are distributed across ad-hoc research codebases or lack transparent implementations, making them difficult to reproduce or extend. As the field accelerates, there is a clear need for a unified framework that standardizes these common components while remaining flexible enough to support new methods and architectures. To address this gap, we introduce dLLM, an open-source framework that unifies the core components of diffusion language modeling -- training, inference, and evaluation -- and makes them easy to customize for new designs. With dLLM, users can reproduce, finetune, deploy, and evaluate open-source large DLMs such as LLaDA and Dream through a standardized pipeline. The framework also provides minimal, reproducible recipes for building small DLMs from scratch with accessible compute, including converting any BERT-style encoder or autoregressive LM into a DLM. We also release the checkpoints of these small DLMs to make DLMs more accessible and accelerate future research.
• 摘要：尽管扩散语言模型 (DLM) 发展迅速，但许多最新模型都集中在一组共享组件上。然而，这些组件分布在临时研究代码库中或缺乏透明的实现，使得它们难以复制或扩展。随着该领域的发展，显然需要一个统一的框架来标准化这些通用组件，同时保持足够的灵活性以支持新的方法和架构。为了弥补这一差距，我们引入了 dLLM，这是一个开源框架，它统一了扩散语言建模的核心组件（训练、推理和评估），并使它们易于针对新设计​​进行定制。借助 dLLM，用户可以通过标准化管道重现、微调、部署和评估 LLaDA 和 Dream 等开源大型 DLM。该框架还提供了最少的、可重复的方法，用于从头开始构建具有可访问计算的小型 DLM，包括将任何 BERT 式编码器或自回归 LM 转换为 DLM。我们还发布了这些小型 DLM 的检查点，以使 DLM 更易于访问并加速未来的研究。

Title: Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization

• Authors: Qianben Chen, Tianrui Qin, King Zhu, Qiexiang Wang, Chengjun Yu, Shu Xu, Jiaqi Wu, Jiayu Zhang, Xinpeng Liu, Xin Gui, Jingyi Cao, Piaohong Wang, Dingfeng Shi, He Zhu, Tiannan Wang, Yuqing Wang, Maojia Song, Tianyu Zheng, Ge Zhang, Jian Yang, Jiaheng Liu, Minghao Liu, Yuchen Eleanor Jiang, Wangchunshu Zhou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22675
• Pdf URL: https://arxiv.org/pdf/2602.22675
• Copy Paste: [[2602.22675]] Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization(https://arxiv.org/abs/2602.22675)
• Keywords: agent
• Abstract: Recent deep research agents primarily improve performance by scaling reasoning depth, but this leads to high inference cost and latency in search-intensive scenarios. Moreover, generalization across heterogeneous research settings remains challenging. In this work, we propose \emph{Search More, Think Less} (SMTL), a framework for long-horizon agentic search that targets both efficiency and generalization. SMTL replaces sequential reasoning with parallel evidence acquisition, enabling efficient context management under constrained context budgets. To support generalization across task types, we further introduce a unified data synthesis pipeline that constructs search tasks spanning both deterministic question answering and open-ended research scenarios with task appropriate evaluation metrics. We train an end-to-end agent using supervised fine-tuning and reinforcement learning, achieving strong and often state of the art performance across benchmarks including BrowseComp (48.6\%), GAIA (75.7\%), Xbench (82.0\%), and DeepResearch Bench (45.9\%). Compared to Mirothinker-v1.0, SMTL with maximum 100 interaction steps reduces the average number of reasoning steps on BrowseComp by 70.7\%, while improving accuracy.
• 摘要：最近的深度研究代理主要通过扩展推理深度来提高性能，但这会导致搜索密集型场景中的推理成本和延迟较高。此外，跨异构研究环境的泛化仍然具有挑战性。在这项工作中，我们提出了\emph{Search More，Think Less}（SMTL），这是一个以效率和泛化为目标的长视野代理搜索框架。 SMTL 用并行证据获取取代顺序推理，从而在上下文预算有限的情况下实现高效的上下文管理。为了支持跨任务类型的泛化，我们进一步引入了一个统一的数据合成管道，该管道构建了跨越确定性问答和开放式研究场景的搜索任务，并具有适当的任务评估指标。我们使用监督微调和强化学习来训练端到端代理，在 BrowseComp (48.6\%)、GAIA (75.7\%)、Xbench (82.0\%) 和 DeepResearch Bench (45.9\%) 等基准测试中实现强大且最先进的性能。与 Mirothinker-v1.0 相比，最多 100 个交互步骤的 SMTL 将 BrowseComp 上的平均推理步骤数减少了 70.7%，同时提高了准确性。

Title: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies

• Authors: Shinnosuke Nozue, Yuto Nakano, Yotaro Watanabe, Meguru Takasaki, Shoji Moriya, Reina Akama, Jun Suzuki
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22696
• Pdf URL: https://arxiv.org/pdf/2602.22696
• Copy Paste: [[2602.22696]] Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies(https://arxiv.org/abs/2602.22696)
• Keywords: agent
• Abstract: Current approaches to developing persuasive dialogue agents often rely on a limited set of predefined persuasive strategies that fail to capture the complexity of real-world interactions. We applied a cross-disciplinary approach to develop a framework for designing persuasive dialogue agents that draws on proven strategies from social psychology, behavioral economics, and communication theory. We validated our proposed framework through experiments on two distinct datasets: the Persuasion for Good dataset, which represents a specific in-domain scenario, and the DailyPersuasion dataset, which encompasses a wide range of scenarios. The proposed framework achieved strong results for both datasets and demonstrated notable improvement in the persuasion success rate as well as promising generalizability. Notably, the proposed framework also excelled at persuading individuals with initially low intent, which addresses a critical challenge for persuasive dialogue agents.
• 摘要：当前开发说服性对话代理的方法通常依赖于一组有限的预定义说服策略，这些策略无法捕捉现实世界交互的复杂性。我们采用跨学科的方法来开发一个设计有说服力的对话代理的框架，该框架借鉴了社会心理学、行为经济学和传播理论中经过验证的策略。我们通过对两个不同数据集的实验验证了我们提出的框架：Persuasion for Good 数据集（代表特定的域内场景）和 DailyPersuasion 数据集（涵盖广泛的场景）。所提出的框架在两个数据集上都取得了很好的结果，并且在说服成功率方面表现出显着的提高以及有希望的普遍性。值得注意的是，所提出的框架还擅长说服最初意图较低的个人，这解决了说服性对话代理的关键挑战。

Title: Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialogue

• Authors: Ning Gao, Wei Zhang, Yuqin Dai, Ling Shi, Ziyin Wang, Yujie Wang, Wei He, Jinpeng Wang, Chaozheng Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22697
• Pdf URL: https://arxiv.org/pdf/2602.22697
• Copy Paste: [[2602.22697]] Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialogue(https://arxiv.org/abs/2602.22697)
• Keywords: language model, llm, chat, agent
• Abstract: The rapid evolution of Large Language Models (LLMs) has accelerated the transition from conversational chatbots to general agents. However, effectively balancing empathetic communication with budget-aware decision-making remains an open challenge. Since existing methods fail to capture these complex strategic trade-offs, we propose InteractCS-RL, a framework that reframes task-oriented dialogue as a multi-granularity reinforcement learning process. Specifically, we first establish a User-centric Interaction Framework to provide a high-fidelity training gym, enabling agents to dynamically explore diverse strategies with persona-driven users. Then, we introduce Cost-aware Multi-turn Policy Optimization (CMPO) with a hybrid advantage estimation strategy. By integrating generative process credits and employing a PID-Lagrangian cost controller, CMPO effectively guides the policy to explore Pareto boundary between user reward and global cost constraints. Extensive experiments on customized real business scenarios demonstrate that InteractCS-RL significantly outperform other baselines across three evaluation dimensions. Further evaluation on tool-agent-user interaction benchmarks verify InteractCS-RL robustness across diverse domains.
• 摘要：大型语言模型（LLM）的快速发展加速了从对话式聊天机器人到通用代理的转变。然而，有效平衡同理心沟通与预算意识决策仍然是一个公开的挑战。由于现有方法无法捕获这些复杂的策略权衡，因此我们提出了 InteractCS-RL，这是一个将面向任务的对话重新构建为多粒度强化学习过程的框架。具体来说，我们首先建立一个以用户为中心的交互框架，以提供高保真训练馆，使代理能够与角色驱动的用户动态探索不同的策略。然后，我们引入了具有混合优势估计策略的成本感知多轮策略优化（CMPO）。通过集成生成过程信用并采用 PID-拉格朗日成本控制器，CMPO 有效地引导策略探索用户奖励和全局成本约束之间的帕累托边界。对定制真实业务场景的大量实验表明，InteractCS-RL 在三个评估维度上显着优于其他基线。对工具-代理-用户交互基准的进一步评估验证了 InteractCS-RL 在不同领域的稳健性。

Title: Tokenization, Fusion and Decoupling: Bridging the Granularity Mismatch Between Large Language Models and Knowledge Graphs

• Authors: Siyue Su, Jian Yang, Bo Li, Guanglin Niu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22698
• Pdf URL: https://arxiv.org/pdf/2602.22698
• Copy Paste: [[2602.22698]] Tokenization, Fusion and Decoupling: Bridging the Granularity Mismatch Between Large Language Models and Knowledge Graphs(https://arxiv.org/abs/2602.22698)
• Keywords: language model, llm
• Abstract: Leveraging Large Language Models (LLMs) for Knowledge Graph Completion (KGC) is promising but hindered by a fundamental granularity mismatch. LLMs operate on fragmented token sequences, whereas entities are the fundamental units in knowledge graphs (KGs) scenarios. Existing approaches typically constrain predictions to limited candidate sets or align entities with the LLM's vocabulary by pooling multiple tokens or decomposing entities into fixed-length token sequences, which fail to capture both the semantic meaning of the text and the structural integrity of the graph. To address this, we propose KGT, a novel framework that uses dedicated entity tokens to enable efficient, full-space prediction. Specifically, we first introduce specialized tokenization to construct feature representations at the level of dedicated entity tokens. We then fuse pre-trained structural and textual features into these unified embeddings via a relation-guided gating mechanism, avoiding training from scratch. Finally, we implement decoupled prediction by leveraging independent heads to separate and combine semantic and structural reasoning. Experimental results show that KGT consistently outperforms state-of-the-art methods across multiple benchmarks.
• 摘要：利用大型语言模型 (LLM) 进行知识图补全 (KGC) 前景广阔，但由于基本粒度不匹配而受到阻碍。 LLM 对碎片化的令牌序列进行操作，而实体是知识图谱 (KG) 场景中的基本单元。现有方法通常将预测限制在有限的候选集上，或者通过汇集多个标记或将实体分解为固定长度的标记序列来将实体与法学硕士的词汇对齐，这无法捕获文本的语义和图形的结构完整性。为了解决这个问题，我们提出了 KGT，这是一种新颖的框架，它使用专用实体令牌来实现高效的全空间预测。具体来说，我们首先引入专门的标记化来构建专用实体标记级别的特征表示。然后，我们通过关系引导的门控机制将预先训练的结构和文本特征融合到这些统一的嵌入中，从而避免从头开始训练。最后，我们通过利用独立的头来分离和组合语义和结构推理来实现解耦预测。实验结果表明，KGT 在多个基准测试中始终优于最先进的方法。

Title: Extending Czech Aspect-Based Sentiment Analysis with Opinion Terms: Dataset and LLM Benchmarks

• Authors: Jakub Šmíd, Pavel Přibáň, Pavel Král
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22730
• Pdf URL: https://arxiv.org/pdf/2602.22730
• Copy Paste: [[2602.22730]] Extending Czech Aspect-Based Sentiment Analysis with Opinion Terms: Dataset and LLM Benchmarks(https://arxiv.org/abs/2602.22730)
• Keywords: language model, llm
• Abstract: This paper introduces a novel Czech dataset in the restaurant domain for aspect-based sentiment analysis (ABSA), enriched with annotations of opinion terms. The dataset supports three distinct ABSA tasks involving opinion terms, accommodating varying levels of complexity. Leveraging this dataset, we conduct extensive experiments using modern Transformer-based models, including large language models (LLMs), in monolingual, cross-lingual, and multilingual settings. To address cross-lingual challenges, we propose a translation and label alignment methodology leveraging LLMs, which yields consistent improvements. Our results highlight the strengths and limitations of state-of-the-art models, especially when handling the linguistic intricacies of low-resource languages like Czech. A detailed error analysis reveals key challenges, including the detection of subtle opinion terms and nuanced sentiment expressions. The dataset establishes a new benchmark for Czech ABSA, and our proposed translation-alignment approach offers a scalable solution for adapting ABSA resources to other low-resource languages.
• 摘要：本文介绍了餐厅领域中一个新颖的捷克数据集，用于基于方面的情感分析（ABSA），并通过意见术语的注释进行了丰富。该数据集支持三个不同的 ABSA 任务，涉及观点术语，适应不同程度的复杂性。利用该数据集，我们使用基于 Transformer 的现代模型（包括大语言模型 (LLM)）在单语言、跨语言和多语言环境中进行了广泛的实验。为了解决跨语言挑战，我们提出了一种利用法学硕士的翻译和标签对齐方法，该方法可以带来持续的改进。我们的结果凸显了最先进模型的优点和局限性，特别是在处理捷克语等低资源语言的语言复杂性时。详细的错误分析揭示了关键的挑战，包括检测微妙的意见术语和微妙的情绪表达。该数据集为捷克语 ABSA 建立了新的基准，我们提出的翻译对齐方法提供了一个可扩展的解决方案，使 ABSA 资源适应其他低资源语言。

Title: Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction

• Authors: Nils Schwager, Simon Münker, Alistair Plum, Achim Rettinger
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22752
• Pdf URL: https://arxiv.org/pdf/2602.22752
• Copy Paste: [[2602.22752]] Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction(https://arxiv.org/abs/2602.22752)
• Keywords: language model, llm, prompt
• Abstract: The transition of Large Language Models (LLMs) from exploratory tools to active "silicon subjects" in social science lacks extensive validation of operational validity. This study introduces Conditioned Comment Prediction (CCP), a task in which a model predicts how a user would comment on a given stimulus by comparing generated outputs with authentic digital traces. This framework enables a rigorous evaluation of current LLM capabilities with respect to the simulation of social media user behavior. We evaluated open-weight 8B models (Llama3.1, Qwen3, Ministral) in English, German, and Luxembourgish language scenarios. By systematically comparing prompting strategies (explicit vs. implicit) and the impact of Supervised Fine-Tuning (SFT), we identify a critical form vs. content decoupling in low-resource settings: while SFT aligns the surface structure of the text output (length and syntax), it degrades semantic grounding. Furthermore, we demonstrate that explicit conditioning (generated biographies) becomes redundant under fine-tuning, as models successfully perform latent inference directly from behavioral histories. Our findings challenge current "naive prompting" paradigms and offer operational guidelines prioritizing authentic behavioral traces over descriptive personas for high-fidelity simulation.
• 摘要：大型语言模型（LLM）从探索性工具到社会科学中活跃的“硅学科”的转变缺乏对操作有效性的广泛验证。本研究引入了条件评论预测（CCP），该任务中模型通过将生成的输出与真实的数字痕迹进行比较来预测用户如何对给定的刺激进行评论。该框架能够对当前法学硕士在社交媒体用户行为模拟方面的能力进行严格评估。我们在英语、德语和卢森堡语语言场景中评估了开放权重 8B 模型（Llama3.1、Qwen3、Ministral）。通过系统地比较提示策略（显式与隐式）和监督微调（SFT）的影响，我们确定了资源匮乏环境中关键的形式与内容解耦：虽然 SFT 对齐了文本输出的表面结构（长度和语法），但它降低了语义基础。此外，我们证明，在微调下，显式调节（生成的传记）变得多余，因为模型成功地直接从行为历史中执行潜在推理。我们的研究结果挑战了当前的“天真的提示”范式，并提供了操作指南，优先考虑真实的行为轨迹而不是描述性角色，以进行高保真模拟。

Title: AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors

• Authors: Abhay Sheshadri, Aidan Ewart, Kai Fronsdal, Isha Gupta, Samuel R. Bowman, Sara Price, Samuel Marks, Rowan Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22755
• Pdf URL: https://arxiv.org/pdf/2602.22755
• Copy Paste: [[2602.22755]] AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors(https://arxiv.org/abs/2602.22755)
• Keywords: language model, prompt, agent
• Abstract: We introduce AuditBench, an alignment auditing benchmark. AuditBench consists of 56 language models with implanted hidden behaviors. Each model has one of 14 concerning behaviors--such as sycophantic deference, opposition to AI regulation, or secret geopolitical loyalties--which it does not confess to when directly asked. AuditBench models are highly diverse--some are subtle, while others are overt, and we use varying training techniques both for implanting behaviors and training models not to confess. To demonstrate AuditBench's utility, we develop an investigator agent that autonomously employs a configurable set of auditing tools. By measuring investigator agent success using different tools, we can evaluate their efficacy. Notably, we observe a tool-to-agent gap, where tools that perform well in standalone non-agentic evaluations fail to translate into improved performance when used with our investigator agent. We find that our most effective tools involve scaffolded calls to auxiliary models that generate diverse prompts for the target. White-box interpretability tools can be helpful, but the agent performs best with black-box tools. We also find that audit success varies greatly across training techniques: models trained on synthetic documents are easier to audit than models trained on demonstrations, with better adversarial training further increasing auditing difficulty. We release our models, agent, and evaluation framework to support future quantitative, iterative science on alignment auditing.
• 摘要：我们推出 AuditBench，一种一致性审核基准。 AuditBench 由 56 个植入隐藏行为的语言模型组成。每个模型都有 14 种相关行为中的一种——例如阿谀奉承、反对人工智能监管或秘密的地缘政治忠诚——当被直接询问时，它不会承认这些行为。 AuditBench 模型高度多样化——有些是微妙的，有些是公开的，我们使用不同的训练技术来植入行为和训练模型不坦白。为了演示 AuditBench 的实用性，我们开发了一个调查代理，它可以自主地使用一组可配置的审计工具。通过使用不同的工具衡量调查人员的成功，我们可以评估他们的功效。值得注意的是，我们观察到工具与代理之间的差距，在独立非代理评估中表现良好的工具在与我们的调查代理一起使用时无法转化为性能的提高。我们发现我们最有效的工具涉及对辅助模型的脚手架调用，这些模型为目标生成不同的提示。白盒可解释性工具可能会有所帮助，但代理在使用黑盒工具时表现最佳。我们还发现，不同培训技术的审计成功率差异很大：在合成文档上训练的模型比在演示上训练的模型更容易审计，更好的对抗性训练进一步增加了审计难度。我们发布了我们的模型、代理和评估框架，以支持未来关于一致性审计的定量、迭代科学。

Title: Towards Better RL Training Data Utilization via Second-Order Rollout

• Authors: Zhe Yang, Yudong Wang, Rang Li, Zhifang Sui
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22765
• Pdf URL: https://arxiv.org/pdf/2602.22765
• Copy Paste: [[2602.22765]] Towards Better RL Training Data Utilization via Second-Order Rollout(https://arxiv.org/abs/2602.22765)
• Keywords: language model, llm
• Abstract: Reinforcement Learning (RL) has empowered Large Language Models (LLMs) with strong reasoning capabilities, but vanilla RL mainly focuses on generation capability improvement by training with only first-order rollout (generating multiple responses for a question), and we argue that this approach fails to fully exploit the potential of training data because of the neglect of critique capability training. To tackle this problem, we further introduce the concept of second-order rollout (generating multiple critiques for a response) and propose a unified framework for jointly training generation and critique capabilities. Extensive experiments across various models and datasets demonstrate that our approach can utilize training data more effectively than vanilla RL and achieve better performance under the same training data. Additionally, we uncover several insightful findings regarding second-order rollout and critique training, such as the importance of label balance in critique training and the noise problem of outcome-based rewards, which can be mitigated through sampling techniques. Our work offers a preliminary exploration of dynamic data augmentation and joint generation-critique training in RL, providing meaningful inspiration for the further advancement of RL training
• 摘要：强化学习（RL）赋予了大型语言模型（LLM）强大的推理能力，但普通的强化学习主要侧重于通过仅进行一阶rollout（对一个问题生成多个答案）进行训练来提高生成能力，我们认为这种方法由于忽视了批判能力训练而未能充分发挥训练数据的潜力。为了解决这个问题，我们进一步引入了二阶推出的概念（为响应生成多个批评），并提出了一个联合训练生成和批评能力的统一框架。跨各种模型和数据集的大量实验表明，我们的方法可以比普通强化学习更有效地利用训练数据，并在相同的训练数据下获得更好的性能。此外，我们还发现了有关二阶推出和批评训练的一些富有洞察力的发现，例如批评训练中标签平衡的重要性以及基于结果的奖励的噪音问题，这些问题可以通过采样技术来缓解。我们的工作对强化学习中的动态数据增强和联合生成批判训练进行了初步探索，为强化学习训练的进一步发展提供了有意义的启发

Title: Imagination Helps Visual Reasoning, But Not Yet in Latent Space

• Authors: You Li, Chi Chen, Yanghao Li, Fanhu Zeng, Kaiyu Huang, Jinan Xu, Maosong Sun
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22766
• Pdf URL: https://arxiv.org/pdf/2602.22766
• Copy Paste: [[2602.22766]] Imagination Helps Visual Reasoning, But Not Yet in Latent Space(https://arxiv.org/abs/2602.22766)
• Keywords: language model
• Abstract: Latent visual reasoning aims to mimic human's imagination process by meditating through hidden states of Multimodal Large Language Models. While recognized as a promising paradigm for visual reasoning, the underlying mechanisms driving its effectiveness remain unclear. Motivated to demystify the true source of its efficacy, we investigate the validity of latent reasoning using Causal Mediation Analysis. We model the process as a causal chain: the input as the treatment, the latent tokens as the mediator, and the final answer as the outcome. Our findings uncover two critical disconnections: (a) Input-Latent Disconnect: dramatic perturbations on the input result in negligible changes to the latent tokens, suggesting that latent tokens do not effectively attend to the input sequence. (b) Latent-Answer Disconnect: perturbations on the latent tokens yield minimal impact on the final answer, indicating the limited causal effect latent tokens imposing on the outcome. Furthermore, extensive probing analysis reveals that latent tokens encode limited visual information and exhibit high similarity. Consequently, we challenge the necessity of latent reasoning and propose a straightforward alternative named CapImagine, which teaches the model to explicitly imagine using text. Experiments on vision-centric benchmarks show that CapImagine significantly outperforms complex latent-space baselines, highlighting the superior potential of visual reasoning through explicit imagination.
• 摘要：潜在视觉推理旨在通过多模态大语言模型的隐藏状态进行冥想来模仿人类的想象力过程。虽然被认为是视觉推理的一种有前途的范例，但驱动其有效性的潜在机制仍不清楚。为了揭开其功效的真正来源的神秘面纱，我们使用因果中介分析研究了潜在推理的有效性。我们将过程建模为因果链：输入作为处理，潜在标记作为中介，最终答案作为结果。我们的发现揭示了两个关键的断开：（a）输入-潜在断开：输入的剧烈扰动导致潜在标记的变化可以忽略不计，这表明潜在标记不能有效地关注输入序列。 (b) 潜在答案断开：对潜在标记的扰动对最终答案产生的影响最小，表明潜在标记对结果的因果影响有限。此外，广泛的探测分析表明，潜在标记编码有限的视觉信息并表现出高度相似性。因此，我们挑战潜在推理的必要性，并提出了一个名为 CapImagine 的简单替代方案，它教会模型使用文本明确地想象。以视觉为中心的基准测试表明，CapImagine 的性能显着优于复杂的潜在空间基线，凸显了通过显式想象进行视觉推理的卓越潜力。

Title: Probing for Knowledge Attribution in Large Language Models

• Authors: Ivo Brink, Alexander Boer, Dennis Ulmer
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22787
• Pdf URL: https://arxiv.org/pdf/2602.22787
• Copy Paste: [[2602.22787]] Probing for Knowledge Attribution in Large Language Models(https://arxiv.org/abs/2602.22787)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Large language models (LLMs) often generate fluent but unfounded claims, or hallucinations, which fall into two types: (i) faithfulness violations - misusing user context - and (ii) factuality violations - errors from internal knowledge. Proper mitigation depends on knowing whether a model's answer is based on the prompt or its internal weights. This work focuses on the problem of contributive attribution: identifying the dominant knowledge source behind each output. We show that a probe, a simple linear classifier trained on model hidden representations, can reliably predict contributive attribution. For its training, we introduce AttriWiki, a self-supervised data pipeline that prompts models to recall withheld entities from memory or read them from context, generating labelled examples automatically. Probes trained on AttriWiki data reveal a strong attribution signal, achieving up to 0.96 Macro-F1 on Llama-3.1-8B, Mistral-7B, and Qwen-7B, transferring to out-of-domain benchmarks (SQuAD, WebQuestions) with 0.94-0.99 Macro-F1 without retraining. Attribution mismatches raise error rates by up to 70%, demonstrating a direct link between knowledge source confusion and unfaithful answers. Yet, models may still respond incorrectly even when attribution is correct, highlighting the need for broader detection frameworks.
• 摘要：大型语言模型（LLM）通常会产生流畅但毫无根据的主张或幻觉，这些主张或幻觉分为两种类型：（i）忠实性违规 - 滥用用户上下文 - 和（ii）事实性违规 - 来自内部知识的错误。正确的缓解措施取决于了解模型的答案是基于提示还是基于其内部权重。这项工作的重点是贡献归因问题：识别每个输出背后的主导知识源。我们证明，探针（一种在模型隐藏表示上训练的简单线性分类器）可以可靠地预测贡献归因。对于其训练，我们引入了 AttriWiki，这是一种自我监督的数据管道，可提示模型从内存中调用保留的实体或从上下文中读取它们，自动生成带标签的示例。在 AttriWiki 数据上训练的探针揭示了强大的归因信号，在 Llama-3.1-8B、Mistral-7B 和 Qwen-7B 上实现了高达 0.96 Macro-F1，转移到域外基准（SQuAD、WebQuestions），无需重新训练即可获得 0.94-0.99 Macro-F1。归因不匹配会使错误率提高高达 70%，这表明知识来源混乱与不忠实答案之间存在直接联系。然而，即使归因正确，模型仍可能做出错误响应，这凸显了对更广泛检测框架的需求。

Title: Natural Language Declarative Prompting (NLD-P): A Modular Governance Method for Prompt Design Under Model Drift

• Authors: Hyunwoo Kim, Hanau Yi, Jaehee Bae, Yumin Kim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22790
• Pdf URL: https://arxiv.org/pdf/2602.22790
• Copy Paste: [[2602.22790]] Natural Language Declarative Prompting (NLD-P): A Modular Governance Method for Prompt Design Under Model Drift(https://arxiv.org/abs/2602.22790)
• Keywords: language model, gpt, llm, prompt
• Abstract: The rapid evolution of large language models (LLMs) has transformed prompt engineering from a localized craft into a systems-level governance challenge. As models scale and update across generations, prompt behavior becomes sensitive to shifts in instruction-following policies, alignment regimes, and decoding strategies, a phenomenon we characterize as GPT-scale model drift. Under such conditions, surface-level formatting conventions and ad hoc refinement are insufficient to ensure stable, interpretable control. This paper reconceptualizes Natural Language Declarative Prompting (NLD-P) as a declarative governance method rather than a rigid field template. NLD-P is formalized as a modular control abstraction that separates provenance, constraint logic, task content, and post-generation evaluation, encoded directly in natural language without reliance on external orchestration code. We define minimal compliance criteria, analyze model-dependent schema receptivity, and position NLD-P as an accessible governance framework for non-developer practitioners operating within evolving LLM ecosystems. Portions of drafting and editorial refinement employed a schema-bound LLM assistant configured under NLD-P. All conceptual framing, methodological claims, and final revisions were directed, reviewed, and approved by the human author under a documented human-in-the-loop protocol. The paper concludes by outlining implications for declarative control under ongoing model evolution and identifying directions for future empirical validation.
• 摘要：大型语言模型 (LLM) 的快速发展已将即时工程从本地化技术转变为系统级治理挑战。随着模型在代际间扩展和更新，即时行为对指令遵循策略、对齐机制和解码策略的变化变得敏感，我们将这种现象称为 GPT 规模模型漂移。在这种情况下，表面级格式化约定和临时细化不足以确保稳定、可解释的控制。本文将自然语言声明式提示（NLD-P）重新概念化为一种声明式治理方法，而不是一个僵化的领域模板。 NLD-P 被形式化为模块化控制抽象，它将来源、约束逻辑、任务内容和生成后评估分开，直接以自然语言编码，而不依赖于外部编排代码。我们定义最低合规性标准，分析依赖于模型的模式接受性，并将 NLD-P 定位为在不断发展的 LLM 生态系统中运行的非开发人员从业者的可访问治理框架。起草和编辑细化部分采用了在 NLD-P 下配置的模式绑定 LLM 助理。所有概念框架、方法论主张和最终修订均由人类作者根据记录在案的人机交互协议进行指导、审查和批准。本文最后概述了持续模型演化下声明性控制的影响，并确定了未来实证验证的方向。

Title: TARAZ: Persian Short-Answer Question Benchmark for Cultural Evaluation of Language Models

• Authors: Reihaneh Iranmanesh, Saeedeh Davoudi, Pasha Abrishamchian, Ophir Frieder, Nazli Goharian
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.22827
• Pdf URL: https://arxiv.org/pdf/2602.22827
• Copy Paste: [[2602.22827]] TARAZ: Persian Short-Answer Question Benchmark for Cultural Evaluation of Language Models(https://arxiv.org/abs/2602.22827)
• Keywords: language model, llm
• Abstract: This paper presents a comprehensive evaluation framework for assessing the cultural competence of large language models (LLMs) in Persian. Existing Persian cultural benchmarks rely predominantly on multiple-choice formats and English-centric metrics that fail to capture Persian's morphological complexity and semantic nuance. Our framework introduces a Persian-specific short-answer evaluation that combines rule-based morphological normalization with a hybrid syntactic and semantic similarity module, enabling robust soft-match scoring beyond exact string overlap. Through systematic evaluation of 15 state-of-the-art open- and closed-source models, we demonstrate that our hybrid evaluation improves scoring consistency by +10% compared to exact-match baselines by capturing meaning that surface-level methods cannot detect. We publicly release our evaluation framework, providing the first standardized benchmark for measuring cultural understanding in Persian and establishing a reproducible foundation for cross-cultural LLM evaluation research.
• 摘要：本文提出了一个用于评估波斯语大型语言模型（LLM）文化能力的综合评估框架。现有的波斯文化基准主要依赖于多项选择格式和以英语为中心的指标，无法捕捉波斯语的形态复杂性和语义细微差别。我们的框架引入了波斯语特定的简答评估，它将基于规则的形态规范化与混合句法和语义相似性模块相结合，从而实现了超越精确字符串重叠的鲁棒软匹配评分。通过对 15 个最先进的开源和闭源模型进行系统评估，我们证明，通过捕获表面方法无法检测到的含义，与精确匹配基线相比，我们的混合评估将评分一致性提高了 10%。我们公开发布我们的评估框架，为衡量波斯语文化理解提供了第一个标准化基准，并为跨文化法学硕士评估研究奠定了可重复的基础。

Title: TCM-DiffRAG: Personalized Syndrome Differentiation Reasoning Method for Traditional Chinese Medicine based on Knowledge Graph and Chain of Thought

• Authors: Jianmin Li, Ying Chang, Su-Kit Tang, Yujia Liu, Yanwen Wang, Shuyuan Lin, Binkai Ou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22828
• Pdf URL: https://arxiv.org/pdf/2602.22828
• Copy Paste: [[2602.22828]] TCM-DiffRAG: Personalized Syndrome Differentiation Reasoning Method for Traditional Chinese Medicine based on Knowledge Graph and Chain of Thought(https://arxiv.org/abs/2602.22828)
• Keywords: language model, llm, retrieval augmented generation
• Abstract: Background: Retrieval augmented generation (RAG) technology can empower large language models (LLMs) to generate more accurate, professional, and timely responses without fine tuning. However, due to the complex reasoning processes and substantial individual differences involved in traditional Chinese medicine (TCM) clinical diagnosis and treatment, traditional RAG methods often exhibit poor performance in this domain. Objective: To address the limitations of conventional RAG approaches in TCM applications, this study aims to develop an improved RAG framework tailored to the characteristics of TCM reasoning. Methods: We developed TCM-DiffRAG, an innovative RAG framework that integrates knowledge graphs (KG) with chains of thought (CoT). TCM-DiffRAG was evaluated on three distinctive TCM test datasets. Results: The experimental results demonstrated that TCM-DiffRAG achieved significant performance improvements over native LLMs. For example, the qwen-plus model achieved scores of 0.927, 0.361, and 0.038, which were significantly enhanced to 0.952, 0.788, and 0.356 with TCM-DiffRAG. The improvements were even more pronounced for non-Chinese LLMs. Additionally, TCM-DiffRAG outperformed directly supervised fine-tuned (SFT) LLMs and other benchmark RAG methods. Conclusions: TCM-DiffRAG shows that integrating structured TCM knowledge graphs with Chain of Thought based reasoning substantially improves performance in individualized diagnostic tasks. The joint use of universal and personalized knowledge graphs enables effective alignment between general knowledge and clinical reasoning. These results highlight the potential of reasoning-aware RAG frameworks for advancing LLM applications in traditional Chinese medicine.
• 摘要：背景：检索增强生成（RAG）技术可以使大型语言模型（LLM）无需微调即可生成更准确、专业、及时的响应。然而，由于中医临床诊断和治疗涉及复杂的推理过程和巨大的个体差异，传统的RAG方法在该领域往往表现不佳。目的：为了解决传统 RAG 方法在中医应用中的局限性，本研究旨在开发一种适合中医推理特点的改进 RAG 框架。方法：我们开发了 TCM-DiffRAG，这是一种创新的 RAG 框架，它将知识图 (KG) 与思想链 (CoT) 相结合。 TCM-DiffRAG 在三个独特的 TCM 测试数据集上进行评估。结果：实验结果表明，TCM-DiffRAG 比原生 LLM 取得了显着的性能改进。例如，qwen-plus 模型的得分为 0.927、0.361 和 0.038，使用 TCM-DiffRAG 后得分显着提高至 0.952、0.788 和 0.356。对于非中国法学硕士来说，这种进步更为明显。此外，TCM-DiffRAG 的性能优于直接监督微调 (SFT) LLM 和其他基准 RAG 方法。结论：TCM-DiffRAG 表明，将结构化中医知识图谱与基于思维链的推理相结合，可以显着提高个性化诊断任务的性能。通用知识图和个性化知识图的联合使用可以实现常识与临床推理的有效结合。这些结果凸显了推理感知 RAG 框架在推进传统中医法学硕士应用方面的潜力。

Title: Improving Neural Argumentative Stance Classification in Controversial Topics with Emotion-Lexicon Features

• Authors: Mohammad Yeghaneh Abkenar, Weixing Wang, Manfred Stede, Davide Picca, Mark A. Finlayson, Panagiotis Ioannidis
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22846
• Pdf URL: https://arxiv.org/pdf/2602.22846
• Copy Paste: [[2602.22846]] Improving Neural Argumentative Stance Classification in Controversial Topics with Emotion-Lexicon Features(https://arxiv.org/abs/2602.22846)
• Keywords: llm
• Abstract: Argumentation mining comprises several subtasks, among which stance classification focuses on identifying the standpoint expressed in an argumentative text toward a specific target topic. While arguments-especially about controversial topics-often appeal to emotions, most prior work has not systematically incorporated explicit, fine-grained emotion analysis to improve performance on this task. In particular, prior research on stance classification has predominantly utilized non-argumentative texts and has been restricted to specific domains or topics, limiting generalizability. We work on five datasets from diverse domains encompassing a range of controversial topics and present an approach for expanding the Bias-Corrected NRC Emotion Lexicon using DistilBERT embeddings, which we feed into a Neural Argumentative Stance Classification model. Our method systematically expands the emotion lexicon through contextualized embeddings to identify emotionally charged terms not previously captured in the lexicon. Our expanded NRC lexicon (eNRC) improves over the baseline across all five datasets (up to +6.2 percentage points in F1 score), outperforms the original NRC on four datasets (up to +3.0), and surpasses the LLM-based approach on nearly all corpora. We provide all resources-including eNRC, the adapted corpora, and model architecture-to enable other researchers to build upon our work.
• 摘要：论证挖掘包括几个子任务，其中立场分类侧重于识别论证文本中针对特定目标主题表达的立场。虽然争论——尤其是关于有争议的话题——通常会诉诸情绪，但大多数先前的工作并没有系统地纳入明确的、细粒度的情绪分析来提高这项任务的表现。特别是，先前关于立场分类的研究主要使用非论证性文本，并且仅限于特定领域或主题，限制了普遍性。我们研究了来自不同领域的五个数据集，涵盖了一系列有争议的主题，并提出了一种使用 DistilBERT 嵌入扩展偏差校正的 NRC 情感词典的方法，我们将其输入到神经论证立场分类模型中。我们的方法通过上下文嵌入来系统地扩展情感词典，以识别词典中以前未捕获的充满情感的术语。我们扩展的 NRC 词典 (eNRC) 在所有五个数据集的基线上都有所提高（F1 分数高达 +6.2 个百分点），在四个数据集上优于原始 NRC（高达 +3.0），并在几乎所有语料库上超越了基于 LLM 的方法。我们提供所有资源（包括 eNRC、改编的语料库和模型架构），以使其他研究人员能够在我们的工作基础上继续发展。

Title: Effective QA-driven Annotation of Predicate-Argument Relations Across Languages

• Authors: Jonathan Davidov, Aviv Slobodkin, Shmuel Tomi Klein, Reut Tsarfaty, Ido Dagan, Ayal Klein
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22865
• Pdf URL: https://arxiv.org/pdf/2602.22865
• Copy Paste: [[2602.22865]] Effective QA-driven Annotation of Predicate-Argument Relations Across Languages(https://arxiv.org/abs/2602.22865)
• Keywords: gpt, llm
• Abstract: Explicit representations of predicate-argument relations form the basis of interpretable semantic analysis, supporting reasoning, generation, and evaluation. However, attaining such semantic structures requires costly annotation efforts and has remained largely confined to English. We leverage the Question-Answer driven Semantic Role Labeling (QA-SRL) framework -- a natural-language formulation of predicate-argument relations -- as the foundation for extending semantic annotation to new languages. To this end, we introduce a cross-linguistic projection approach that reuses an English QA-SRL parser within a constrained translation and word-alignment pipeline to automatically generate question-answer annotations aligned with target-language predicates. Applied to Hebrew, Russian, and French -- spanning diverse language families -- the method yields high-quality training data and fine-tuned, language-specific parsers that outperform strong multilingual LLM baselines (GPT-4o, LLaMA-Maverick). By leveraging QA-SRL as a transferable natural-language interface for semantics, our approach enables efficient and broadly accessible predicate-argument parsing across languages.
• 摘要：谓词-论元关系的显式表示构成了可解释语义分析的基础，支持推理、生成和评估。然而，获得这样的语义结构需要昂贵的注释工作，并且仍然主要局限于英语。我们利用问答驱动的语义角色标签（QA-SRL）框架——谓词-论元关系的自然语言表述——作为将语义注释扩展到新语言的基础。为此，我们引入了一种跨语言投影方法，该方法在受限翻译和单词对齐管道中重用英语 QA-SRL 解析器，以自动生成与目标语言谓词对齐的问答注释。该方法应用于跨越不同语系的希伯来语、俄语和法语，产生高质量的训练数据和经过微调的特定语言解析器，其性能优于强大的多语言 LLM 基线（GPT-4o、LLaMA-Maverick）。通过利用 QA-SRL 作为语义的可转移自然语言接口，我们的方法可以实现跨语言的高效且广泛可访问的谓词参数解析。

Title: Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference

• Authors: Yushi Ye, Feng Hong, Huangjie Zheng, Xu Chen, Zhiyong Chen, Yanfeng Wang, Jiangchao Yao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22868
• Pdf URL: https://arxiv.org/pdf/2602.22868
• Copy Paste: [[2602.22868]] Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference(https://arxiv.org/abs/2602.22868)
• Keywords: language model, llm
• Abstract: Diffusion Large Language Models (DLLMs) promise fast non-autoregressive inference but suffer a severe quality-speed trade-off in parallel decoding. This stems from the ''combinatorial contradiction'' phenomenon, where parallel tokens form semantically inconsistent combinations. We address this by integrating continuous representations into the discrete decoding process, as they preserve rich inter-position dependency. We propose ReMix (Rejection Mixing), a framework that introduces a novel Continuous Mixing State as an intermediate between the initial masked state and the final decoded token state. This intermediate state allows a token's representation to be iteratively refined in a continuous space, resolving mutual conflicts with other tokens before collapsing into a final discrete sample. Furthermore, a rejection rule reverts uncertain representations from the continuous state back to the masked state for reprocessing, ensuring stability and preventing error propagation. ReMix thus mitigates combinatorial contradictions by enabling continuous-space refinement during discrete diffusion decoding. Extensive experiments demonstrate that ReMix, as a training-free method, achieves a $2-8 \times$ inference speedup without any quality degradation.
• 摘要：扩散大型语言模型 (DLLM) 承诺快速非自回归推理，但在并行解码中面临严重的质量与速度权衡。这源于“组合矛盾”现象，即并行标记形成语义上不一致的组合。我们通过将连续表示集成到离散解码过程中来解决这个问题，因为它们保留了丰富的位置间依赖性。我们提出了 ReMix（拒绝混合），这是一个框架，引入了一种新颖的连续混合状态作为初始屏蔽状态和最终解码令牌状态之间的中间体。这种中间状态允许在连续空间中迭代地细化令牌的表示，在分解为最终的离散样本之前解决与其他令牌的相互冲突。此外，拒绝规则将不确定的表示从连续状态恢复到屏蔽状态以进行重新处理，确保稳定性并防止错误传播。因此，ReMix 通过在离散扩散解码期间实现连续空间细化来缓解组合矛盾。大量实验表明，ReMix 作为一种免训练方法，可实现 $2-8 \times$ 的推理加速，而不会出现任何质量下降。

Title: Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

• Authors: Roy Miles, Aysim Toker, Andreea-Maria Oncescu, Songcen Xu, Jiankang Deng, Ismail Elezi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22871
• Pdf URL: https://arxiv.org/pdf/2602.22871
• Copy Paste: [[2602.22871]] Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching(https://arxiv.org/abs/2602.22871)
• Keywords: language model
• Abstract: Reasoning with large language models often benefits from generating multiple chains-of-thought, but existing aggregation strategies are typically trajectory-level (e.g., selecting the best trace or voting on the final answer), discarding useful intermediate work from partial or "nearly correct" attempts. We propose Stitching Noisy Diffusion Thoughts, a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates. Given a problem, we (i) sample many diverse, low-cost reasoning trajectories using a masked diffusion language model, (ii) score every intermediate step with an off-the-shelf process reward model (PRM), and (iii) stitch these highest-quality steps across trajectories into a composite rationale. This rationale then conditions an autoregressive (AR) model (solver) to recompute only the final answer. This modular pipeline separates exploration (diffusion) from evaluation and solution synthesis, avoiding monolithic unified hybrids while preserving broad search. Across math reasoning benchmarks, we find that step-level recombination is most beneficial on harder problems, and ablations highlight the importance of the final AR solver in converting stitched but imperfect rationales into accurate answers. Using low-confidence diffusion sampling with parallel, independent rollouts, our training-free framework improves average accuracy by up to 23.8% across six math and coding tasks. At the same time, it achieves up to a 1.8x latency reduction relative to both traditional diffusion models (e.g., Dream, LLaDA) and unified architectures (e.g., TiDAR). Code is available at this https URL.
• 摘要：使用大型语言模型进行推理通常受益于生成多个思想链，但现有的聚合策略通常是轨迹级别的（例如，选择最佳轨迹或对最终答案进行投票），从部分或“接近正确”的尝试中丢弃有用的中间工作。我们提出了 Stitching Noisy Diffusion Thoughts，这是一个自洽框架，可将廉价的扩散采样推理转变为可重用的步骤级候选池。给定一个问题，我们（i）使用掩码扩散语言模型对许多不同的、低成本的推理轨迹进行采样，（ii）使用现成的过程奖励模型（PRM）对每个中间步骤进行评分，以及（iii）将这些跨轨迹的最高质量步骤缝合成一个复合原理。然后，此基本原理会限制自回归 (AR) 模型（求解器）仅重新计算最终答案。这种模块化管道将探索（扩散）与评估和解决方案合成分开，避免了单一的统一混合，同时保留了广泛的搜索。在数学推理基准测试中，我们发现步骤级重组对于解决更困难的问题最有利，而消融则凸显了最终 AR 求解器在将缝合但不完美的基本原理转换为准确答案方面的重要性。我们的免训练框架使用低置信度扩散采样和并行、独立的部署，将六项数学和编码任务的平均准确率提高了 23.8%。同时，相对于传统扩散模型（例如 Dream、LLaDA）和统一架构（例如 TiDAR），它实现了高达 1.8 倍的延迟减少。代码可从此 https URL 获取。

Title: Where Vision Becomes Text: Locating the OCR Routing Bottleneck in Vision-Language Models

• Authors: Jonathan Steinberg, Oren Gal
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22918
• Pdf URL: https://arxiv.org/pdf/2602.22918
• Copy Paste: [[2602.22918]] Where Vision Becomes Text: Locating the OCR Routing Bottleneck in Vision-Language Models(https://arxiv.org/abs/2602.22918)
• Keywords: language model
• Abstract: Vision-language models (VLMs) can read text from images, but where does this optical character recognition (OCR) information enter the language processing stream? We investigate the OCR routing mechanism across three architecture families (Qwen3-VL, Phi-4, InternVL3.5) using causal interventions. By computing activation differences between original images and text-inpainted versions, we identify architecture-specific OCR bottlenecks whose dominant location depends on the vision-language integration strategy: DeepStack models (Qwen) show peak sensitivity at mid-depth (about 50%) for scene text, while single-stage projection models (Phi-4, InternVL) peak at early layers (6-25%), though the exact layer of maximum effect varies across datasets. The OCR signal is remarkably low-dimensional: PC1 captures 72.9% of variance. Crucially, principal component analysis (PCA) directions learned on one dataset transfer to others, demonstrating shared text-processing pathways. Surprisingly, in models with modular OCR circuits (notably Qwen3-VL-4B), OCR removal can improve counting performance (up to +6.9 percentage points), suggesting OCR interferes with other visual processing in sufficiently modular architectures.
• 摘要：视觉语言模型 (VLM) 可以从图像中读取文本，但这种光学字符识别 (OCR) 信息在哪里进入语言处理流呢？我们使用因果干预研究了三个架构系列（Qwen3-VL、Phi-4、InternVL3.5）的 OCR 路由机制。通过计算原始图像和文本修复版本之间的激活差异，我们确定了特定于体系结构的 OCR 瓶颈，其主要位置取决于视觉语言集成策略：DeepStack 模型 (Qwen) 在场景文本的中深度（约 50%）显示出峰值灵敏度，而单阶段投影模型（Phi-4、InternVL）在早期层（6-25%）达到峰值，尽管最大效果的确切层因数据集而异。 OCR 信号的维数非常低：PC1 捕获了 72.9% 的方差。至关重要的是，在一个数据集上学到的主成分分析 (PCA) 方向可以转移到其他数据集，从而展示共享的文本处理路径。令人惊讶的是，在具有模块化 OCR 电路（特别是 Qwen3-VL-4B）的模型中，OCR 去除可以提高计数性能（高达 +6.9 个百分点），这表明 OCR 会干扰足够模块化架构中的其他视觉处理。

Title: Affine-Scaled Attention: Towards Flexible and Stable Transformer Attention

• Authors: Jeongin Bae, Baeseong Park, Gunho Park, Minsub Kim, Joonhyung Lee, Junhee Yoo, Sunghyeon Woo, Jiwon Ryu, Se Jung Kwon, Dongsoo Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.23057
• Pdf URL: https://arxiv.org/pdf/2602.23057
• Copy Paste: [[2602.23057]] Affine-Scaled Attention: Towards Flexible and Stable Transformer Attention(https://arxiv.org/abs/2602.23057)
• Keywords: language model
• Abstract: Transformer attention is typically implemented using softmax normalization, which enforces attention weights with unit sum normalization. While effective in many settings, this constraint can limit flexibility in controlling attention magnitudes and may contribute to overly concentrated or unstable attention patterns during training. Prior work has explored modifications such as attention sinks or gating mechanisms, but these approaches provide only limited or indirect control over attention reweighting. We propose Affine-Scaled Attention, a simple extension to standard attention that introduces input-dependent scaling and a corresponding bias term applied to softmax-normalized attention weights. This design relaxes the strict normalization constraint while maintaining aggregation of value representations, allowing the model to adjust both the relative distribution and the scale of attention in a controlled manner. We empirically evaluate Affine-Scaled Attention in large-scale language model pretraining across multiple model sizes. Experimental results show consistent improvements in training stability, optimization behavior, and downstream task performance compared to standard softmax attention and attention sink baselines. These findings suggest that modest reweighting of attention outputs provides a practical and effective way to improve attention behavior in Transformer models.
• 摘要：Transformer 注意力通常使用 softmax 归一化来实现，它通过单位和归一化来强制注意力权重。虽然在许多情况下有效，但这种约束可能会限制控制注意力程度的灵活性，并可能导致训练期间注意力模式过度集中或不稳定。先前的工作已经探索了注意力池或门控机制等修改，但这些方法仅提供对注意力重新加权的有限或间接控制。我们提出了仿射缩放注意力（Affine-Scaled Attention），这是标准注意力的简单扩展，它引入了依赖于输入的缩放以及应用于 softmax 归一化注意力权重的相应偏差项。这种设计放松了严格的归一化约束，同时保持值表示的聚合，允许模型以受控的方式调整相对分布和注意力规模。我们根据经验评估了跨多个模型大小的大规模语言模型预训练中的仿射尺度注意力。实验结果表明，与标准 softmax 注意力和注意力池基线相比，训练稳定性、优化行为和下游任务性能得到了持续改进。这些发现表明，对注意力输出进行适度的重新加权为改善 Transformer 模型中的注意力行为提供了一种实用且有效的方法。

Title: Toward Automatic Filling of Case Report Forms: A Case Study on Data from an Italian Emergency Department

• Authors: Gabriela Anna Kaczmarek, Pietro Ferrazzi, Lorenzo Porta, Vicky Rubini, Bernardo Magnini
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23062
• Pdf URL: https://arxiv.org/pdf/2602.23062
• Copy Paste: [[2602.23062]] Toward Automatic Filling of Case Report Forms: A Case Study on Data from an Italian Emergency Department(https://arxiv.org/abs/2602.23062)
• Keywords: language model, llm
• Abstract: Case Report Forms (CRFs) collect data about patients and are at the core of well-established practices to conduct research in clinical settings. With the recent progress of language technologies, there is an increasing interest in automatic CRF-filling from clinical notes, mostly based on the use of Large Language Models (LLMs). However, there is a general scarcity of annotated CRF data, both for training and testing LLMs, which limits the progress on this task. As a step in the direction of providing such data, we present a new dataset of clinical notes from an Italian Emergency Department annotated with respect to a pre-defined CRF containing 134 items to be filled. We provide an analysis of the data, define the CRF-filling task and metric for its evaluation, and report on pilot experiments where we use an open-source state-of-the-art LLM to automatically execute the task. Results of the case-study show that (i) CRF-filling from real clinical notes in Italian can be approached in a zero-shot setting; (ii) LLMs' results are affected by biases (e.g., a cautious behaviour favours "unknown" answers), which need to be corrected.
• 摘要：病例报告表 (CRF) 收集患者数据，是在临床环境中开展研究的成熟实践的核心。随着语言技术的最新进展，人们对从临床记录中自动填充 CRF 越来越感兴趣，主要基于大型语言模型 (LLM) 的使用。然而，用于训练和测试法学硕士的带注释的 CRF 数据普遍缺乏，这限制了这项任务的进展。作为提供此类数据的一个步骤，我们提供了意大利急诊科的新临床记录数据集，并针对包含 134 个待填写项目的预定义 CRF 进行了注释。我们提供数据分析，定义 CRF 填充任务和评估指标，并报告试点实验，其中我们使用开源的最先进的 LLM 自动执行任务。案例研究的结果表明，(i) 可以在零样本设置中根据意大利语真实临床记录进行 CRF 填充； (ii) 法学硕士的结果受到偏见的影响（例如，谨慎的行为倾向于“未知”的答案），需要纠正。

Title: CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery

• Authors: Mengze Hong, Di Jiang, Chen Jason Zhang, Zichang Guo, Yawen Li, Jun Chen, Shaobo Cui, Zhiyang Su
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2602.23075
• Pdf URL: https://arxiv.org/pdf/2602.23075
• Copy Paste: [[2602.23075]] CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery(https://arxiv.org/abs/2602.23075)
• Keywords: language model, llm, hallucination, chat, agent
• Abstract: Large language models (LLMs) have created new opportunities to enhance the efficiency of scholarly activities; however, challenges persist in the ethical deployment of AI assistance, including (1) the trustworthiness of AI-generated content, (2) preservation of academic integrity and intellectual property, and (3) protection of information privacy. In this work, we present CiteLLM, a specialized agentic platform designed to enable trustworthy reference discovery for grounding author-drafted claims and statements. The system introduces a novel interaction paradigm by embedding LLM utilities directly within the LaTeX editor environment, ensuring a seamless user experience and no data transmission outside the local system. To guarantee hallucination-free references, we employ dynamic discipline-aware routing to retrieve candidates exclusively from trusted web-based academic repositories, while leveraging LLMs solely for generating context-aware search queries, ranking candidates by relevance, and validating and explaining support through paragraph-level semantic matching and an integrated chatbot. Evaluation results demonstrate the superior performance of the proposed system in returning valid and highly usable references.
• 摘要：大型语言模型（LLM）为提高学术活动的效率创造了新的机会；然而，人工智能援助部署的道德挑战仍然存在，包括（1）人工智能生成内容的可信度，（2）学术诚信和知识产权的保护，以及（3）信息隐私的保护。在这项工作中，我们提出了 CiteLLM，这是一个专门的代理平台，旨在实现值得信赖的参考发现，以支持作者起草的声明和声明。该系统通过将 LLM 实用程序直接嵌入 LaTeX 编辑器环境中，引入了一种新颖的交互范例，确保无缝的用户体验，并且不会在本地系统之外传输数据。为了保证无幻觉参考，我们采用动态学科感知路由专门从受信任的基于网络的学术存储库检索候选人，同时仅利用法学硕士生成上下文感知搜索查询，按相关性对候选人进行排名，并通过段落级语义匹配和集成聊天机器人验证和解释支持。评估结果表明，所提出的系统在返回有效且高度可用的参考方面具有卓越的性能。

Title: Assessing Deanonymization Risks with Stylometry-Assisted LLM Agent

• Authors: Boyang Zhang, Yang Zhang
• Subjects: cs.CL, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.23079
• Pdf URL: https://arxiv.org/pdf/2602.23079
• Copy Paste: [[2602.23079]] Assessing Deanonymization Risks with Stylometry-Assisted LLM Agent(https://arxiv.org/abs/2602.23079)
• Keywords: language model, llm, prompt, agent
• Abstract: The rapid advancement of large language models (LLMs) has enabled powerful authorship inference capabilities, raising growing concerns about unintended deanonymization risks in textual data such as news articles. In this work, we introduce an LLM agent designed to evaluate and mitigate such risks through a structured, interpretable pipeline. Central to our framework is the proposed $\textit{SALA}$ (Stylometry-Assisted LLM Analysis) method, which integrates quantitative stylometric features with LLM reasoning for robust and transparent authorship attribution. Experiments on large-scale news datasets demonstrate that $\textit{SALA}$, particularly when augmented with a database module, achieves high inference accuracy in various scenarios. Finally, we propose a guided recomposition strategy that leverages the agent's reasoning trace to generate rewriting prompts, effectively reducing authorship identifiability while preserving textual meaning. Our findings highlight both the deanonymization potential of LLM agents and the importance of interpretable, proactive defenses for safeguarding author privacy.
• 摘要：大型语言模型 (LLM) 的快速发展实现了强大的作者推断能力，引发了人们对新闻文章等文本数据中意外去匿名化风险的日益担忧。在这项工作中，我们引入了一个 LLM 代理，旨在通过结构化、可解释的管道来评估和减轻此类风险。我们框架的核心是提出的$\textit{SALA}$（风格测量辅助法学硕士分析）方法，该方法将定量风格测量特征与法学硕士推理相结合，以实现稳健且透明的作者归属。在大规模新闻数据集上的实验表明，$\textit{SALA}$，特别是在使用数据库模块进行增强时，可以在各种场景下实现较高的推理精度。最后，我们提出了一种引导重组策略，利用代理的推理轨迹来生成重写提示，在保留文本含义的同时有效地减少作者身份的可识别性。我们的研究结果强调了法学硕士代理人的去匿名化潜力以及可解释的主动防御对于保护作者隐私的重要性。

Title: Modality Collapse as Mismatched Decoding: Information-Theoretic Limits of Multimodal LLMs

• Authors: Jayadev Billa
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.23136
• Pdf URL: https://arxiv.org/pdf/2602.23136
• Copy Paste: [[2602.23136]] Modality Collapse as Mismatched Decoding: Information-Theoretic Limits of Multimodal LLMs(https://arxiv.org/abs/2602.23136)
• Keywords: llm
• Abstract: Multimodal LLMs can process speech and images, but they cannot hear a speaker's voice or see an object's texture. We show this is not a failure of encoding: speaker identity, emotion, and visual attributes survive through every LLM layer (3--55$\times$ above chance in linear probes), yet removing 64--71% of modality-specific variance improves decoder loss. The decoder has no learned use for these directions; their presence is noise. We formalize this as a mismatched decoder problem: a decoder trained on text can only extract information along text-aligned directions. Accessible information is bounded by the Generalized Mutual Information (GMI), with degradation scaling with distributional distance and decoder sensitivity. The bound is a property of the decoder's scoring rule, not of any particular architecture; it applies whether non-text inputs arrive through a learned projection, a discrete codebook, or no explicit adapter at all. We validate this across five models spanning speech and vision. A controlled experiment (two Prismatic VLMs differing only in encoder text-alignment) confirms the bottleneck is the decoder's scoring rule, not the encoder or projection. A LoRA intervention demonstrates the fix: training with an emotion objective improves emotion accessibility ($+$7.5%) without affecting other attributes, confirming that the training objective determines what becomes accessible.
• 摘要：多模态法学硕士可以处理语音和图像，但它们无法听到说话者的声音或看到物体的纹理。我们证明这不是编码的失败：说话者身份、情感和视觉属性在每个 LLM 层中都存在（比线性探针中的机会高出 3--55$\times$），但消除 64--71% 的特定模态方差可以改善解码器损失。解码器没有学习到这些方向的用途；他们的存在就是噪音。我们将其形式化为不匹配的解码器问题：在文本上训练的解码器只能沿着文本对齐的方向提取信息。可访问的信息受到广义互信息 (GMI) 的限制，并且随分布距离和解码器灵敏度而降低。界限是解码器评分规则的属性，而不是任何特定架构的属性；无论非文本输入是通过学习的投影、离散码本还是根本没有显式适配器到达，它都适用。我们在涵盖语音和视觉的五个模型中验证了这一点。一项受控实验（两个 Prismatic VLM 仅在编码器文本对齐方面不同）确认瓶颈是解码器的评分规则，而不是编码器或投影。 LoRA 干预证明了解决方案：使用情感目标进行训练可以提高情感可访问性 ($+$7.5%)，而不影响其他属性，从而确认培训目标决定了可访问的内容。

Title: MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations

• Authors: Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, Marina Danilevsky
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23184
• Pdf URL: https://arxiv.org/pdf/2602.23184
• Copy Paste: [[2602.23184]] MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations(https://arxiv.org/abs/2602.23184)
• Keywords: language model, retrieval augmented generation
• Abstract: We present MTRAG-UN, a benchmark for exploring open challenges in multi-turn retrieval augmented generation, a popular use of large language models. We release a benchmark of 666 tasks containing over 2,800 conversation turns across 6 domains with accompanying corpora. Our experiments show that retrieval and generation models continue to struggle on conversations with UNanswerable, UNderspecified, and NONstandalone questions and UNclear responses. Our benchmark is available at this https URL
• 摘要：我们提出了 MTRAG-UN，这是一个探索多轮检索增强生成（大语言模型的流行用途）中开放挑战的基准。我们发布了包含 666 个任务的基准测试，其中包含 6 个领域的 2,800 多个对话轮次以及相应的语料库。我们的实验表明，检索和生成模型在与无法回答、未明确、非独立问题和不明确回答的对话中继续陷入困境。我们的基准测试可以在这个 https URL 上找到

Title: Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models

• Authors: Chungpa Lee, Jy-yong Sohn, Kangwook Lee
• Subjects: cs.CL, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2602.23197
• Pdf URL: https://arxiv.org/pdf/2602.23197
• Copy Paste: [[2602.23197]] Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models(https://arxiv.org/abs/2602.23197)
• Keywords: language model, prompt
• Abstract: Transformer-based large language models exhibit in-context learning, enabling adaptation to downstream tasks via few-shot prompting with demonstrations. In practice, such models are often fine-tuned to improve zero-shot performance on downstream tasks, allowing them to solve tasks without examples and thereby reducing inference costs. However, fine-tuning can degrade in-context learning, limiting the performance of fine-tuned models on tasks not seen during fine-tuning. Using linear attention models, we provide a theoretical analysis that characterizes how fine-tuning objectives modify attention parameters and identifies conditions under which this leads to degraded few-shot performance. We show that fine-tuning all attention parameters can harm in-context learning, whereas restricting updates to the value matrix improves zero-shot performance while preserving in-context learning. We further show that incorporating an auxiliary few-shot loss enhances in-context learning primarily on the target task, at the expense of degraded in-context learning ability on tasks not seen during fine-tuning. We empirically validate our theoretical results.
• 摘要：基于 Transformer 的大型语言模型展示了上下文学习，可以通过演示的几次提示来适应下游任务。在实践中，此类模型通常经过微调以提高下游任务的零样本性能，使它们能够在没有示例的情况下解决任务，从而降低推理成本。然而，微调会降低上下文学习的性能，限制微调模型在微调过程中未看到的任务上的性能。使用线性注意力模型，我们提供了理论分析，描述了微调目标如何修改注意力参数，并确定了导致小样本性能下降的条件。我们表明，微调所有注意力参数可能会损害上下文学习，而限制对值矩阵的更新可以提高零样本性能，同时保留上下文学习。我们进一步表明，结合辅助的少镜头损失主要增强了目标任务的上下文学习，但代价是在微调期间未看到的任务的上下文学习能力下降。我们通过实证验证了我们的理论结果。

Title: Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?

• Authors: Pengxiang Li, Dilxat Muhtar, Lu Yin, Tianlong Chen, Shiwei Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.23225
• Pdf URL: https://arxiv.org/pdf/2602.23225
• Copy Paste: [[2602.23225]] Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?(https://arxiv.org/abs/2602.23225)
• Keywords: language model, chain-of-thought
• Abstract: Diffusion Language Models (DLMs) are often advertised as enabling parallel token generation, yet practical fast DLMs frequently converge to left-to-right, autoregressive (AR)-like decoding dynamics. In contrast, genuinely non-AR generation is promising because it removes AR's sequential bottleneck, better exploiting parallel hardware to reduce synchronization/communication overhead and improve latency scaling with output length. We argue that a primary driver of AR-like decoding is a mismatch between DLM objectives and the highly sequential structure of widely used training data, including standard pretraining corpora and long chain-of-thought (CoT) supervision. Motivated by this diagnosis, we propose NAP (Non-Autoregressive Parallel DLMs), a proof-of-concept, data-centric approach that better aligns supervision with non-AR parallel decoding. NAP curates examples as multiple independent reasoning trajectories and couples them with a parallel-forced decoding strategy that encourages multi-token parallel updates. Across math reasoning benchmarks, NAP yields stronger performance under parallel decoding than DLMs trained on standard long CoT data, with gains growing as parallelism increases. Our results suggest that revisiting data and supervision is a principled direction for mitigating AR-like behavior and moving toward genuinely non-autoregressive parallel generation in DLMs. Our code is available at this https URL.
• 摘要：扩散语言模型 (DLM) 通常被宣传为支持并行令牌生成，但实用的快速 DLM 经常收敛到从左到右、类似自回归 (AR) 的解码动态。相比之下，真正的非 AR 生成是有前途的，因为它消除了 AR 的顺序瓶颈，更好地利用并行硬件来减少同步/通信开销并改善输出长度的延迟缩放。我们认为，类 AR 解码的主要驱动因素是 DLM 目标与广泛使用的训练数据的高度顺序结构之间的不匹配，包括标准预训练语料库和长思想链 (CoT) 监督。受此诊断的启发，我们提出了 NAP（非自回归并行 DLM），这是一种概念验证、以数据为中心的方法，可以更好地将监督与非 AR 并行解码结合起来。 NAP 将示例整理为多个独立的推理轨迹，并将它们与鼓励多令牌并行更新的并行强制解码策略结合起来。在数学推理基准测试中，NAP 在并行解码下比在标准长 CoT 数据上训练的 DLM 具有更强的性能，并且随着并行性的增加，增益也会不断增加。我们的结果表明，重新审视数据和监督是减轻类似 AR 的行为并走向 DLM 中真正的非自回归并行生成的原则方向。我们的代码可以在这个 https URL 上找到。

Title: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems

• Authors: Siyuan Liu, Jiahui Xu, Feng Jiang, Kuang Wang, Zefeng Zhao, Chu-Ren Huang, Jinghang Gu, Changqing Yin, Haizhou Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.23266
• Pdf URL: https://arxiv.org/pdf/2602.23266
• Copy Paste: [[2602.23266]] Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems(https://arxiv.org/abs/2602.23266)
• Keywords: llm
• Abstract: Achieving human-like responsiveness is a critical yet challenging goal for cascaded spoken dialogue systems. Conventional ASR-LLM-TTS pipelines follow a strictly sequential paradigm, requiring complete transcription and full reasoning before speech synthesis can begin, which results in high response latency. We propose the Discourse-Aware Dual-Track Streaming Response (DDTSR) framework, a low-latency architecture that enables listen-while-thinking and speak-while-thinking. DDTSR is built upon three key mechanisms: (1) connective-guided small-large model synergy, where an auxiliary small model generates minimal-committal discourse connectives while a large model performs knowledge-intensive reasoning in parallel; (2) streaming-based cross-modal collaboration, which dynamically overlaps ASR, LLM inference, and TTS to advance the earliest speakable moment; and (3) curriculum-learning-based discourse continuity enhancement, which maintains coherence and logical consistency between early responses and subsequent reasoning outputs. Experiments on two spoken dialogue benchmarks demonstrate that DDTSR reduces response latency by 19%-51% while preserving discourse quality. Further analysis shows that DDTSR functions as a plug-and-play module compatible with diverse LLM backbones, and remains robust across varying utterance lengths, indicating strong practicality and scalability for real-time spoken interaction.
• 摘要：对于级联语音对话系统来说，实现类人的响应能力是一个关键但具有挑战性的目标。传统的 ASR-LLM-TTS 管道遵循严格的顺序范例，在语音合成开始之前需要完整的转录和完整的推理，这会导致较高的响应延迟。我们提出了话语感知双轨流响应（DDTSR）框架，这是一种低延迟架构，可实现边思考边听和边思考边说。 DDTSR 建立在三个关键机制之上：（1）连接引导的小-大模型协同，其中辅助小模型生成最小承诺话语连接词，而大模型并行执行知识密集型推理； (2) 基于流的跨模态协作，动态重叠 ASR、LLM 推理和 TTS，以提前最早可说话的时刻； (3)基于课程学习的话语连续性增强，保持早期反应和随后推理输出之间的连贯性和逻辑一致性。对两个口语对话基准的实验表明，DDTSR 将响应延迟减少了 19%-51%，同时保持了对话质量。进一步分析表明，DDTSR作为一个即插即用模块，与不同的LLM骨干网兼容，并且在不同的话语长度下保持鲁棒性，表明实时语音交互具有很强的实用性和可扩展性。

Title: A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations

• Authors: Soumya Dutta, Smruthi Balaji, Sriram Ganapathy
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2602.23300
• Pdf URL: https://arxiv.org/pdf/2602.23300
• Copy Paste: [[2602.23300]] A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations(https://arxiv.org/abs/2602.23300)
• Keywords: language model, llm
• Abstract: Emotion Recognition in Conversations (ERC) presents unique challenges, requiring models to capture the temporal flow of multi-turn dialogues and to effectively integrate cues from multiple modalities. We propose Mixture of Speech-Text Experts for Recognition of Emotions (MiSTER-E), a modular Mixture-of-Experts (MoE) framework designed to decouple two core challenges in ERC: modality-specific context modeling and multimodal information fusion. MiSTER-E leverages large language models (LLMs) fine-tuned for both speech and text to provide rich utterance-level embeddings, which are then enhanced through a convolutional-recurrent context modeling layer. The system integrates predictions from three experts-speech-only, text-only, and cross-modal-using a learned gating mechanism that dynamically weighs their outputs. To further encourage consistency and alignment across modalities, we introduce a supervised contrastive loss between paired speech-text representations and a KL-divergence-based regulariza-tion across expert predictions. Importantly, MiSTER-E does not rely on speaker identity at any stage. Experiments on three benchmark datasets-IEMOCAP, MELD, and MOSI-show that our proposal achieves 70.9%, 69.5%, and 87.9% weighted F1-scores respectively, outperforming several baseline speech-text ERC systems. We also provide various ablations to highlight the contributions made in the proposed approach.
• 摘要：对话中的情绪识别（ERC）提出了独特的挑战，需要模型捕获多轮对话的时间流并有效地整合来自多种模式的线索。我们提出了用于情绪识别的语音文本专家混合 (MiSTER-E)，这是一种模块化专家混合 (MoE) 框架，旨在解耦 ERC 中的两个核心挑战：特定模态上下文建模和多模态信息融合。 MiSTER-E 利用针对语音和文本进行微调的大型语言模型 (LLM) 来提供丰富的话语级嵌入，然后通过卷积循环上下文建模层对其进行增强。该系统使用学习门控机制集成了三位专家的预测（纯语音、纯文本和跨模式），动态权衡其输出。为了进一步鼓励跨模态的一致性和对齐，我们在配对语音文本表示之间引入了监督对比损失，并在专家预测之间引入了基于 KL 散度的正则化。重要的是，MiSTER-E 在任何阶段都不依赖说话者身份。在三个基准数据集（IEMOCAP、MELD 和 MOSI）上的实验表明，我们的方案分别实现了 70.9%、69.5% 和 87.9% 的加权 F1 分数，优于几个基线语音文本 ERC 系统。我们还提供了各种消融来强调所提出的方法中所做的贡献。

Title: Scale Can't Overcome Pragmatics: The Impact of Reporting Bias on Vision-Language Reasoning

• Authors: Amita Kamath, Jack Hessel, Khyathi Chandu, Jena D. Hwang, Kai-Wei Chang, Ranjay Krishna
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2602.23351
• Pdf URL: https://arxiv.org/pdf/2602.23351
• Copy Paste: [[2602.23351]] Scale Can't Overcome Pragmatics: The Impact of Reporting Bias on Vision-Language Reasoning(https://arxiv.org/abs/2602.23351)
• Keywords: language model
• Abstract: The lack of reasoning capabilities in Vision-Language Models (VLMs) has remained at the forefront of research discourse. We posit that this behavior stems from a reporting bias in their training data. That is, how people communicate about visual content by default omits tacit information needed to supervise some types of reasoning; e.g., "at the game today!" is a more likely caption than "a photo of 37 people standing behind a field". We investigate the data underlying the popular VLMs OpenCLIP, LLaVA-1.5 and Molmo through the lens of theories from pragmatics, and find that reporting bias results in insufficient representation of four reasoning skills (spatial, temporal, negation, and counting), despite the corpora being of web-scale, and/or synthetically generated. With a set of curated benchmarks, we demonstrate that: (i) VLMs perform poorly on the aforementioned types of reasoning suppressed in the training data by reporting bias; (ii) contrary to popular belief, scaling data size, model size, and to multiple languages does not result in emergence of these skills by default; but, promisingly, (iii) incorporating annotations specifically collected to obtain tacit information is effective. Our findings highlight the need for more intentional training data curation methods, rather than counting on scale for emergence of reasoning capabilities.
• 摘要：视觉语言模型（VLM）缺乏推理能力一直是研究讨论的焦点。我们认为这种行为源于他们的训练数据中的报告偏差。也就是说，人们如何交流视觉内容默认会忽略监督某些类型推理所需的隐性信息；例如，“今天的比赛！”比“一张 37 个人站在田野后面的照片”更可能是一个标题。我们通过语用学理论的视角研究了流行的 VLM OpenCLIP、LLaVA-1.5 和 Molmo 背后的数据，发现报告偏差导致四种推理技能（空间、时间、否定和计数）的代表性不足，尽管语料库是网络规模的和/或综合生成的。通过一组精心策划的基准，我们证明：（i）VLM 在训练数据中通过报告偏差而抑制的上述推理类型上表现不佳； (ii) 与普遍看法相反，扩展数据大小、模型大小和多种语言默认情况下不会导致这些技能的出现；但是，（iii）合并专门收集的注释来获取隐性信息是有希望的。我们的研究结果强调需要更有意的训练数据管理方法，而不是依靠规模来出现推理能力。