2026-03-11

Title: One Language, Two Scripts: Probing Script-Invariance in LLM Concept Representations

• Authors: Sripad Karne
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.08869
• Pdf URL: https://arxiv.org/pdf/2603.08869
• Copy Paste: [[2603.08869]] One Language, Two Scripts: Probing Script-Invariance in LLM Concept Representations(https://arxiv.org/abs/2603.08869)
• Keywords: llm
• Abstract: Do the features learned by Sparse Autoencoders (SAEs) represent abstract meaning, or are they tied to how text is written? We investigate this question using Serbian digraphia as a controlled testbed: Serbian is written interchangeably in Latin and Cyrillic scripts with a near-perfect character mapping between them, enabling us to vary orthography while holding meaning exactly constant. Crucially, these scripts are tokenized completely differently, sharing no tokens whatsoever. Analyzing SAE feature activations across the Gemma model family (270M-27B parameters), we find that identical sentences in different Serbian scripts activate highly overlapping features, far exceeding random baselines. Strikingly, changing script causes less representational divergence than paraphrasing within the same script, suggesting SAE features prioritize meaning over orthographic form. Cross-script cross-paraphrase comparisons provide evidence against memorization, as these combinations rarely co-occur in training data yet still exhibit substantial feature overlap. This script invariance strengthens with model scale. Taken together, our findings suggest that SAE features can capture semantics at a level of abstraction above surface tokenization, and we propose Serbian digraphia as a general evaluation paradigm for probing the abstractness of learned representations.
• 摘要：稀疏自编码器 (SAE) 学习的特征是否代表抽象含义，或者它们与文本的编写方式相关吗？我们使用塞尔维亚双写法作为受控测试平台来研究这个问题：塞尔维亚语是用拉丁文和西里尔文交替书写的，它们之间有近乎完美的字符映射，使我们能够改变正字法，同时保持含义完全不变。至关重要的是，这些脚本的标记化方式完全不同，不共享任何标记。分析 Gemma 模型系列（270M-27B 参数）中的 SAE 特征激活，我们发现不同塞尔维亚文字中的相同句子激活高度重叠的特征，远远超过随机基线。引人注目的是，与同一脚本中的释义相比，更改脚本导致的表征分歧较小，这表明 SAE 特征优先考虑含义而不是拼写形式。跨脚本跨释义比较提供了反对记忆的证据，因为这些组合很少在训练数据中同时出现，但仍然表现出大量的特征重叠。这种脚本不变性随着模型规模的增加而增强。总而言之，我们的研究结果表明，SAE 特征可以在表面标记化之上的抽象级别捕获语义，并且我们提出塞尔维亚双写法作为探索学习表示的抽象性的通用评估范式。

Title: ConFu: Contemplate the Future for Better Speculative Sampling

• Authors: Zongyue Qin, Raghavv Goel, Mukul Gagrani, Risheek Garrepalli, Mingu Lee, Yizhou Sun
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.08899
• Pdf URL: https://arxiv.org/pdf/2603.08899
• Copy Paste: [[2603.08899]] ConFu: Contemplate the Future for Better Speculative Sampling(https://arxiv.org/abs/2603.08899)
• Keywords: language model, llm, prompt
• Abstract: Speculative decoding has emerged as a powerful approach to accelerate large language model (LLM) inference by employing lightweight draft models to propose candidate tokens that are subsequently verified by the target model. The effectiveness of this paradigm critically depends on the quality of the draft model. While recent advances such as the EAGLE series achieve state-of-the-art speedup, existing draft models remain limited by error accumulation: they condition only on the current prefix, causing their predictions to drift from the target model over steps. In this work, we propose \textbf{ConFu} (Contemplate the Future), a novel speculative decoding framework that enables draft models to anticipate the future direction of generation. ConFu introduces (i) contemplate tokens and soft prompts that allow the draft model to leverage future-oriented signals from the target model at negligible cost, (ii) a dynamic contemplate token mechanism with MoE to enable context-aware future prediction, and (iii) a training framework with anchor token sampling and future prediction replication that learns robust future prediction. Experiments demonstrate that ConFu improves token acceptance rates and generation speed over EAGLE-3 by 8--11% across various downstream tasks with Llama-3 3B and 8B models. We believe our work is the first to bridge speculative decoding with continuous reasoning tokens, offering a new direction for accelerating LLM inference.
• 摘要：推测性解码已成为一种强大的方法，通过采用轻量级草稿模型来提出随后由目标模型验证的候选标记，从而加速大型语言模型 (LLM) 推理。这种范式的有效性关键取决于模型草案的质量。虽然 EAGLE 系列等最新进展实现了最先进的加速，但现有的草稿模型仍然受到错误累积的限制：它们仅以当前前缀为条件，导致其预测随着步骤的推移而偏离目标模型。在这项工作中，我们提出了 \textbf{ConFu} （思考未来），这是一种新颖的推测性解码框架，使草稿模型能够预测未来的生成方向。 ConFu 引入了 (i) 考虑令牌和软提示，允许草稿模型以可忽略不计的成本利用来自目标模型的面向未来的信号，(ii) 与 MoE 一起的动态考虑令牌机制，以实现上下文感知的未来预测，以及 (iii) 具有锚令牌采样和未来预测复制的训练框架，用于学习稳健的未来预测。实验表明，在使用 Llama-3 3B 和 8B 模型的各种下游任务中，ConFu 将 EAGLE-3 的代币接受率和生成速度提高了 8--11%。我们相信我们的工作是第一个将推测性解码与连续推理标记结合起来的工作，为加速 LLM 推理提供了新的方向。

Title: SciTaRC: Benchmarking QA on Scientific Tabular Data that Requires Language Reasoning and Complex Computation

• Authors: Hexuan Wang, Yaxuan Ren, Srikar Bommireddypalli, Shuxian Chen, Adarsh Prabhudesai, Rongkun Zhou, Elina Baral, Philipp Koehn
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.08910
• Pdf URL: https://arxiv.org/pdf/2603.08910
• Copy Paste: [[2603.08910]] SciTaRC: Benchmarking QA on Scientific Tabular Data that Requires Language Reasoning and Complex Computation(https://arxiv.org/abs/2603.08910)
• Keywords: language model
• Abstract: We introduce SciTaRC, an expert-authored benchmark of questions about tabular data in scientific papers requiring both deep language reasoning and complex computation. We show that current state-of-the-art AI models fail on at least 23% of these questions, a gap that remains significant even for highly capable open-weight models like Llama-3.3-70B-Instruct, which fails on 65.5% of the tasks. Our analysis reveals a universal "execution bottleneck": both code and language models struggle to faithfully execute plans, even when provided with correct strategies. Specifically, code-based methods prove brittle on raw scientific tables, while natural language reasoning primarily fails due to initial comprehension issues and calculation errors.
• 摘要：我们介绍 SciTaRC，这是一个由专家编写的基准，针对需要深度语言推理和复杂计算的科学论文中的表格数据问题。我们表明，当前最先进的 AI 模型至少在 23% 的问题上失败了，即使对于像 Llama-3.3-70B-Instruct 这样功能强大的开放权重模型（在 65.5% 的任务上失败）来说，这一差距仍然很大。我们的分析揭示了一个普遍的“执行瓶颈”：即使提供了正确的策略，代码和语言模型都难以忠实地执行计划。具体来说，基于代码的方法在原始科学表格上被证明是脆弱的，而自然语言推理主要由于初始理解问题和计算错误而失败。

Title: Automated Thematic Analysis for Clinical Qualitative Data: Iterative Codebook Refinement with Full Provenance

• Authors: Seungjun Yi, Joakim Nguyen, Huimin Xu, Terence Lim, Joseph Skrovan, Mehak Beri, Hitakshi Modi, Andrew Well, Carlos M. Mery, Yan Zhang, Mia K. Markey, Ying Ding
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.08989
• Pdf URL: https://arxiv.org/pdf/2603.08989
• Copy Paste: [[2603.08989]] Automated Thematic Analysis for Clinical Qualitative Data: Iterative Codebook Refinement with Full Provenance(https://arxiv.org/abs/2603.08989)
• Keywords: llm
• Abstract: Thematic analysis (TA) is widely used in health research to extract patterns from patient interviews, yet manual TA faces challenges in scalability and reproducibility. LLM-based automation can help, but existing approaches produce codebooks with limited generalizability and lack analytic auditability. We present an automated TA framework combining iterative codebook refinement with full provenance tracking. Evaluated on five corpora spanning clinical interviews, social media, and public transcripts, the framework achieves the highest composite quality score on four of five datasets compared to six baselines. Iterative refinement yields statistically significant improvements on four datasets with large effect sizes, driven by gains in code reusability and distributional consistency while preserving descriptive quality. On two clinical corpora (pediatric cardiology), generated themes align with expert-annotated themes.
• 摘要：主题分析 (TA) 广泛应用于健康研究中，从患者访谈中提取模式，但手动 TA 面临可扩展性和可重复性方面的挑战。基于法学硕士的自动化可以提供帮助，但现有方法生成的密码本的通用性有限，并且缺乏分析审核性。我们提出了一个自动化的 TA 框架，将迭代码本细化与完整的来源跟踪相结合。通过对涵盖临床访谈、社交媒体和公共记录的五个语料库进行评估，与六个基线相比，该框架在五个数据集中的四个上获得了最高的综合质量得分。迭代细化在四个具有较大效应量的数据集上产生了统计上显着的改进，这是由于代码可重用性和分布一致性的提高，同时保持描述质量。在两个临床语料库（儿科心脏病学）上，生成的主题与专家注释的主题一致。

Title: Learning When to Sample: Confidence-Aware Self-Consistency for Efficient LLM Chain-of-Thought Reasoning

• Authors: Juming Xiong, Kevin Guo, Congning Ni, Chao Yan, Katherine Brown, Avinash Baidya, Xiang Gao, Bradley Marlin, Zhijun Yin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.08999
• Pdf URL: https://arxiv.org/pdf/2603.08999
• Copy Paste: [[2603.08999]] Learning When to Sample: Confidence-Aware Self-Consistency for Efficient LLM Chain-of-Thought Reasoning(https://arxiv.org/abs/2603.08999)
• Keywords: language model, llm, chain-of-thought
• Abstract: Large language models (LLMs) achieve strong reasoning performance through chain-of-thought (CoT) reasoning, yet often generate unnecessarily long reasoning paths that incur high inference cost. Recent self-consistency-based approaches further improve accuracy but require sampling and aggregating multiple reasoning trajectories, leading to substantial additional computational overhead. This paper introduces a confidence-aware decision framework that analyzes a single completed reasoning trajectory to adaptively select between single-path and multi-path reasoning. The framework is trained using sentence-level numeric and linguistic features extracted from intermediate reasoning states in the MedQA dataset and generalizes effectively to MathQA, MedMCQA, and MMLU without additional fine-tuning. Experimental results show that the proposed method maintains accuracy comparable to multi-path baselines while using up to 80\% fewer tokens. These findings demonstrate that reasoning trajectories contain rich signals for uncertainty estimation, enabling a simple, transferable mechanism to balance accuracy and efficiency in LLM reasoning.
• 摘要：大型语言模型（LLM）通过思想链（CoT）推理实现强大的推理性能，但通常会生成不必要的长推理路径，从而导致较高的推理成本。最近基于自我一致性的方法进一步提高了准确性，但需要采样和聚合多个推理轨迹，从而导致大量额外的计算开销。本文介绍了一种置信感知决策框架，该框架分析单个完整的推理轨迹，以在单路径和多路径推理之间进行自适应选择。该框架使用从 MedQA 数据集中的中间推理状态提取的句子级数字和语言特征进行训练，并有效地推广到 MathQA、MedMCQA 和 MMLU，而无需额外的微调。实验结果表明，所提出的方法保持了与多路径基线相当的精度，同时使用的令牌数量减少了 80%。这些发现表明，推理轨迹包含丰富的不确定性估计信号，从而实现了一种简单、可转移的机制来平衡法学硕士推理的准确性和效率。

Title: Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs

• Authors: Kaiser Sun, Xiaochuang Yuan, Hongjun Liu, Chen Zhao, Cheng Zhang, Mark Dredze, Fan Bai
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2603.09095
• Pdf URL: https://arxiv.org/pdf/2603.09095
• Copy Paste: [[2603.09095]] Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs(https://arxiv.org/abs/2603.09095)
• Keywords: language model, llm, chain-of-thought
• Abstract: Multimodal large language models (MLLMs) can process text presented as images, yet they often perform worse than when the same content is provided as textual tokens. We systematically diagnose this "modality gap" by evaluating seven MLLMs across seven benchmarks in five input modes, spanning both synthetically rendered text and realistic document images from arXiv PDFs to Wikipedia pages. We find that the modality gap is task- and data-dependent. For example, math tasks degrade by over 60 points on synthetic renderings, while natural document images often match or exceed text-mode performance. Rendering choices such as font and resolution are strong confounds, with font alone swinging accuracy by up to 47 percentage points. To understand this, we conduct a grounded-theory error analysis of over 4,000 examples, revealing that image mode selectively amplifies reading errors (calculation and formatting failures) while leaving knowledge and reasoning errors largely unchanged, and that some models exhibit a chain-of-thought reasoning collapse under visual input. Motivated by these findings, we propose a self-distillation method that trains the model on its own pure text reasoning traces paired with image inputs, raising image-mode accuracy on GSM8K from 30.71% to 92.72% and transferring to unseen benchmarks without catastrophic forgetting. Overall, our study provides a systematic understanding of the modality gap and suggests a practical path toward improving visual text understanding in multimodal language models.
• 摘要：多模态大语言模型 (MLLM) 可以处理以图像形式呈现的文本，但它们的性能通常比以文本标记形式提供相同内容时要差。我们通过在五种输入模式下的七个基准测试中评估七个 MLLM，涵盖从 arXiv PDF 到维基百科页面的合成渲染文本和真实文档图像，系统地诊断了这种“模态差距”。我们发现模态差距取决于任务和数据。例如，数学任务在合成渲染上的性能下降了 60 多点，而自然文档图像的性能通常与文本模式性能相当或超过。字体和分辨率等渲染选择是一个很大的混杂因素，仅字体一项就会使准确度波动高达 47 个百分点。为了理解这一点，我们对 4000 多个示例进行了扎根理论错误分析，揭示了图像模式选择性地放大了阅读错误（计算和格式化失败），而知识和推理错误基本保持不变，并且一些模型在视觉输入下表现出思维链推理崩溃。受这些发现的启发，我们提出了一种自蒸馏方法，该方法根据与图像输入配对的纯文本推理轨迹来训练模型，将 GSM8K 上的图像模式准确率从 30.71% 提高到 92.72%，并转移到看不见的基准，而不会发生灾难性遗忘。总的来说，我们的研究提供了对模态差距的系统理解，并提出了提高多模态语言模型中视觉文本理解的实用路径。

Title: Bioalignment: Measuring and Improving LLM Disposition Toward Biological Systems for AI Safety

• Authors: Trent R Northen, Mingxun Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09154
• Pdf URL: https://arxiv.org/pdf/2603.09154
• Copy Paste: [[2603.09154]] Bioalignment: Measuring and Improving LLM Disposition Toward Biological Systems for AI Safety(https://arxiv.org/abs/2603.09154)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) trained on internet-scale corpora can exhibit systematic biases that increase the probability of unwanted behavior. In this study, we examined potential biases towards synthetic vs. biological technological solutions across four domains (materials, energy, manufacturing, and algorithms). A sample of 5 frontier and 5 open-weight models were measured using 50 curated Bioalignment prompts with a Kelly criterion-inspired evaluation framework. According to this metric, most models were not bioaligned in that they exhibit biases in favor of synthetic (non-biological) solutions. We next examined if fine-tuning could increase the preferences of two open-weight models, Llama 3.2-3B-Instruct and Qwen2.5-3B-Instruct, for biological-based approaches. A curated corpus of ~22M tokens from 6,636 PMC articles emphasizing biological problem-solving was used first to fine-tune Llama 3B with a mixed corpus of continued training and instruction-formatted. This was then extended to Qwen 3B using instruction-formatted only. We found that QLoRA fine-tuning significantly increased the scoring of biological solutions for both models without degrading general capabilities (Holm-Bonferroni-corrected p < 0.001 and p < 0.01, respectively). This suggests that even a small amount of fine-tuning can change how models weigh the relative value of biological and bioinspired vs. synthetic approaches. Although this work focused on small open-weight LLMs, it may be extensible to much larger models and could be used to develop models that favor bio-based approaches. We release the benchmark, corpus, code, and adapter weights.
• 摘要：在互联网规模的语料库上训练的大型语言模型（LLM）可能会表现出系统性偏差，从而增加不良行为的可能性。在这项研究中，我们研究了四个领域（材料、能源、制造和算法）对合成与生物技术解决方案的潜在偏见。使用 50 个精心策划的 Bioalignment 提示和凯利准则启发的评估框架来测量 5 个前沿模型和 5 个开放重量模型的样本。根据这个指标，大多数模型都不是生物对齐的，因为它们表现出有利于合成（非生物）解决方案的偏见。接下来我们研究了微调是否可以增加两种开放权重模型（Llama 3.2-3B-Instruct 和 Qwen2.5-3B-Instruct）对基于生物学的方法的偏好。首先使用来自 6,636 篇强调生物学问题解决的 PMC 文章的约 2200 万个标记的精选语料库，通过持续训练和指令格式的混合语料库对 Llama 3B 进行微调。然后仅使用指令格式将其扩展到 Qwen 3B。我们发现 QLoRA 微调显着提高了两种模型的生物解决方案评分，而不会降低一般能力（Holm-Bonferroni 校正分别为 p < 0.001 和 p < 0.01）。这表明，即使是少量的微调也可以改变模型衡量生物方法和生物启发方法与合成方法的相对价值的方式。尽管这项工作主要针对小型开放权重法学硕士，但它可能可以扩展到更大的模型，并且可以用于开发有利于基于生物的方法的模型。我们发布了基准、语料库、代码和适配器权重。

Title: DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimization

• Authors: Jianing Yang, Yusuke Fujita, Yui Sudo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.09180
• Pdf URL: https://arxiv.org/pdf/2603.09180
• Copy Paste: [[2603.09180]] DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimization(https://arxiv.org/abs/2603.09180)
• Keywords: llm
• Abstract: Spoken dialog systems with cascaded ASR-LLM-TTS modules retain strong LLM intelligence, but VAD segmentation often forces half-duplex turns and brittle control. On the other hand, VAD-free end-to-end model support full-duplex interaction but is hard to maintain conversational intelligence. In this paper, we present DuplexCascade, a VAD-free cascaded streaming pipeline for full-duplex speech-to-speech dialogue. Our key idea is to convert conventional utterance-wise long turns into chunk-wise micro-turn interactions, enabling rapid bidirectional exchange while preserving the strengths of a capable text LLM. To reliably coordinate turn-taking and response timing, we introduce a set of conversational special control tokens that steer the LLM's behavior under streaming constraints. On Full-DuplexBench and VoiceBench, DuplexCascade delivers state-of-the-art full-duplex turn-taking and strong conversational intelligence among open-source speech-to-speech dialogue systems.
• 摘要：具有级联 ASR-LLM-TTS 模块的语音对话系统保留了强大的 LLM 智能，但 VAD 分段通常会强制半双工转弯和脆弱的控制。另一方面，无VAD的端到端模型支持全双工交互，但难以保持会话智能。在本文中，我们提出了 DuplexCascade，一种用于全双工语音到语音对话的无 VAD 级联流管道。我们的关键想法是将传统的话语式长轮转转换为块式微轮交互，实现快速双向交换，同时保留有能力的文本法学硕士的优势。为了可靠地协调轮流和响应时间，我们引入了一组会话特殊控制令牌，用于在流约束下引导 LLM 的行为。在 Full-DuplexBench 和 VoiceBench 上，DuplexCascade 在开源语音到语音对话系统中提供最先进的全双工轮流和强大的对话智能。

Title: DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrieval

• Authors: Taegyeong Lee, Jiwon Park, Seunghyun Hwang, JooYoung Jang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09185
• Pdf URL: https://arxiv.org/pdf/2603.09185
• Copy Paste: [[2603.09185]] DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrieval(https://arxiv.org/abs/2603.09185)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Recent advances in Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) have enabled diverse retrieval methods. However, existing retrieval methods often fail to accurately retrieve results for negation and exclusion queries. To address this limitation, prior approaches rely on embedding adaptation or fine-tuning, which introduce additional computational cost and deployment complexity. We propose Direct Embedding Optimization (DEO), a training-free method for negation-aware text and multimodal retrieval. DEO decomposes queries into positive and negative components and optimizes the query embedding with a contrastive objective. Without additional training data or model updates, DEO outperforms baselines on NegConstraint, with gains of +0.0738 nDCG@10 and +0.1028 MAP@100, while improving Recall@5 by +6\% over OpenAI CLIP in multimodal retrieval. These results demonstrate the practicality of DEO for negation- and exclusion-aware retrieval in real-world settings.
• 摘要：大型语言模型 (LLM) 和检索增强生成 (RAG) 的最新进展使得多样化的检索方法成为可能。然而，现有的检索方法常常无法准确地检索否定查询和排除查询的结果。为了解决这一限制，现有方法依赖于嵌入自适应或微调，这会带来额外的计算成本和部署复杂性。我们提出了直接嵌入优化（DEO），这是一种用于否定感知文本和多模态检索的免训练方法。 DEO 将查询分解为正向和负向组件，并以对比目标优化查询嵌入。在没有额外的训练数据或模型更新的情况下，DEO 在 NegConstraint 上的表现优于基线，增益为 +0.0738 nDCG@10 和 +0.1028 MAP@100，同时在多模态检索中将 Recall@5 比 OpenAI CLIP 提高了 +6%。这些结果证明了 DEO 在现实环境中进行否定和排除感知检索的实用性。

Title: Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing

• Authors: Benjamin Reichman, Adar Avasian, Samuel Webster, Larry Heck
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.09205
• Pdf URL: https://arxiv.org/pdf/2603.09205
• Copy Paste: [[2603.09205]] Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing(https://arxiv.org/abs/2603.09205)
• Keywords: language model, llm
• Abstract: Large language models are routinely deployed on text that varies widely in emotional tone, yet their reasoning behavior is typically evaluated without accounting for emotion as a source of representational variation. Prior work has largely treated emotion as a prediction target, for example in sentiment analysis or emotion classification. In contrast, we study emotion as a latent factor that shapes how models attend to and reason over text. We analyze how emotional tone systematically alters attention geometry in transformer models, showing that metrics such as locality, center-of-mass distance, and entropy vary across emotions and correlate with downstream question-answering performance. To facilitate controlled study of these effects, we introduce Affect-Uniform ReAding QA (AURA-QA), a question-answering dataset with emotionally balanced, human-authored context passages. Finally, an emotional regularization framework is proposed that constrains emotion-conditioned representational drift during training. Experiments across multiple QA benchmarks demonstrate that this approach improves reading comprehension in both emotionally-varying and non-emotionally varying datasets, yielding consistent gains under distribution shift and in-domain improvements on several benchmarks.
• 摘要：大型语言模型通常部署在情感基调差异很大的文本上，但它们的推理行为通常在评估时不考虑情感作为表征变化的来源。先前的工作主要将情绪视为预测目标，例如在情绪分析或情绪分类中。相比之下，我们将情感作为一种潜在因素来研究，它决定了模型如何关注和推理文本。我们分析了情绪基调如何系统地改变 Transformer 模型中的注意力几何，表明局部性、质心距离和熵等指标因情绪而异，并与下游问答表现相关。为了促进对这些影响的对照研究，我们引入了 Affect-Uniform ReAding QA (AURA-QA)，这是一个问答数据集，其中包含情感平衡的、人类创作的上下文段落。最后，提出了一种情绪正则化框架，可以限制训练期间情绪条件下的表征漂移。跨多个 QA 基准的实验表明，这种方法提高了情绪变化和非情绪变化数据集的阅读理解能力，在分布变化和多个基准的域内改进下产生了一致的增益。

Title: SPAR-K: Scheduled Periodic Alternating Early Exit for Spoken Language Models

• Authors: Hsiao-Ying Huang, Cheng-Han Chiang, Hung-yi Lee
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2603.09215
• Pdf URL: https://arxiv.org/pdf/2603.09215
• Copy Paste: [[2603.09215]] SPAR-K: Scheduled Periodic Alternating Early Exit for Spoken Language Models(https://arxiv.org/abs/2603.09215)
• Keywords: language model, llm
• Abstract: Interleaved spoken language models (SLMs) alternately generate text and speech tokens, but decoding at full transformer depth for every step becomes costly, especially due to long speech sequences. We propose SPAR-K, a modality-aware early exit framework designed to accelerate interleaved SLM inference while preserving perceptual quality. SPAR-K introduces a speech alternating-depth schedule: most speech positions exit at a fixed intermediate layer, while periodic full-depth "refresh" steps mitigate distribution shift due to early exit. We evaluate our framework using Step-Audio-2-mini and GLM-4-Voice across four datasets spanning reasoning, factual QA, and dialogue tasks, measuring performance in terms of ASR transcription accuracy and perceptual quality. Experimental results demonstrate that SPAR-K largely preserves question-answering accuracy with a maximum accuracy drop of 0.82\% while reducing average speech decoding depth by up to 11\% on Step-Audio-2-mini and 5\% on GLM-4-Voice, both with negligible changes in MOS and WER and no auxiliary computation overhead. We further demonstrate that confidence-based early exit strategies, widely used in text LLMs, are suboptimal for SLMs, highlighting that the unique statistical nature of speech tokens necessitates a specialized early exit design.
• 摘要：交错口语模型 (SLM) 交替生成文本和语音标记，但在每个步骤的完整转换器深度上进行解码成本高昂，尤其是由于语音序列较长。我们提出了 SPAR-K，一种模态感知的早期退出框架，旨在加速交错 SLM 推理，同时保持感知质量。 SPAR-K 引入了语音交替深度调度：大多数语音位置在固定的中间层退出，而周期性全深度“刷新”步骤减轻了由于提前退出而导致的分布偏移。我们使用 Step-Audio-2-mini 和 GLM-4-Voice 跨推理、事实 QA 和对话任务的四个数据集评估我们的框架，衡量 ASR 转录准确性和感知质量方面的性能。实验结果表明，SPAR-K 在很大程度上保留了问答精度，最大精度下降了 0.82%，同时在 Step-Audio-2-mini 上将平均语音解码深度降低了 11%，在 GLM-4-Voice 上降低了 5%，MOS 和 WER 的变化可以忽略不计，并且没有辅助计算开销。我们进一步证明，在文本法学硕士中广泛使用的基于置信度的早期退出策略对于 SLM 来说并不是最优的，这强调了语音标记独特的统计性质需要专门的早期退出设计。

Title: LooComp: Leverage Leave-One-Out Strategy to Encoder-only Transformer for Efficient Query-aware Context Compression

• Authors: Thao Do, Dinh Phu Tran, An Vo, Seon Kwon Kim, Daeyoung Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09222
• Pdf URL: https://arxiv.org/pdf/2603.09222
• Copy Paste: [[2603.09222]] LooComp: Leverage Leave-One-Out Strategy to Encoder-only Transformer for Efficient Query-aware Context Compression(https://arxiv.org/abs/2603.09222)
• Keywords: llm, retrieval augmented generation
• Abstract: Efficient context compression is crucial for improving the accuracy and scalability of question answering. For the efficiency of Retrieval Augmented Generation, context should be delivered fast, compact, and precise to ensure clue sufficiency and budget-friendly LLM reader cost. We propose a margin-based framework for query-driven context pruning, which identifies sentences that are critical for answering a query by measuring changes in clue richness when they are omitted. The model is trained with a composite ranking loss that enforces large margins for critical sentences while keeping non-critical ones near neutral. Built on a lightweight encoder-only Transformer, our approach generally achieves strong exact-match and F1 scores with high-throughput inference and lower memory requirements than those of major baselines. In addition to efficiency, our method yields effective compression ratios without degrading answering performance, demonstrating its potential as a lightweight and practical alternative for retrieval-augmented tasks.
• 摘要：高效的上下文压缩对于提高问答的准确性和可扩展性至关重要。为了提高检索增强生成的效率，应快速、紧凑和精确地提供上下文，以确保线索充足和预算友好的 LLM 读者成本。我们提出了一种基于边距的查询驱动上下文修剪框架，该框架通过测量省略线索丰富度的变化来识别对于回答查询至关重要的句子。该模型使用复合排名损失进行训练，该损失对关键句子施加较大的余量，同时使非关键句子保持接近中性。我们的方法建立在仅包含编码器的轻量级 Transformer 之上，通常可以通过高吞吐量推理实现强大的精确匹配和 F1 分数，并且比主要基线的内存要求更低。除了效率之外，我们的方法还可以在不降低应答性能的情况下产生有效的压缩比，这证明了其作为检索增强任务的轻量级实用替代方案的潜力。

Title: TaSR-RAG: Taxonomy-guided Structured Reasoning for Retrieval-Augmented Generation

• Authors: Jiashuo Sun, Yixuan Xie, Jimeng Shi, Shaowen Wang, Jiawei Han
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.09341
• Pdf URL: https://arxiv.org/pdf/2603.09341
• Copy Paste: [[2603.09341]] TaSR-RAG: Taxonomy-guided Structured Reasoning for Retrieval-Augmented Generation(https://arxiv.org/abs/2603.09341)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) helps large language models (LLMs) answer knowledge-intensive and time-sensitive questions by conditioning generation on external evidence. However, most RAG systems still retrieve unstructured chunks and rely on one-shot generation, which often yields redundant context, low information density, and brittle multi-hop reasoning. While structured RAG pipelines can improve grounding, they typically require costly and error-prone graph construction or impose rigid entity-centric structures that do not align with the query's reasoning chain. We propose \textsc{TaSR-RAG}, a taxonomy-guided structured reasoning framework for evidence selection. We represent both queries and documents as relational triples, and constrain entity semantics with a lightweight two-level taxonomy to balance generalization and precision. Given a complex question, \textsc{TaSR-RAG} decomposes it into an ordered sequence of triple sub-queries with explicit latent variables, then performs step-wise evidence selection via hybrid triple matching that combines semantic similarity over raw triples with structural consistency over typed triples. By maintaining an explicit entity binding table across steps, \textsc{TaSR-RAG} resolves intermediate variables and reduces entity conflation without explicit graph construction or exhaustive search. Experiments on multiple multi-hop question answering benchmarks show that \textsc{TaSR-RAG} consistently outperforms strong RAG and structured-RAG baselines by up to 14\%, while producing clearer evidence attribution and more faithful reasoning traces.
• 摘要：检索增强生成 (RAG) 通过根据外部证据调节生成，帮助大型语言模型 (LLM) 回答知识密集型和时间敏感的问题。然而，大多数 RAG 系统仍然检索非结构化块并依赖于一次性生成，这通常会产生冗余上下文、低信息密度和脆弱的多跳推理。虽然结构化 RAG 管道可以改善基础，但它们通常需要昂贵且容易出错的图形构建，或者强加严格的以实体为中心的结构，这些结构与查询的推理链不相符。我们提出 \textsc{TaSR-RAG}，一种用于证据选择的分类引导的结构化推理框架。我们将查询和文档表示为关系三元组，并使用轻量级两级分类法来约束实体语义，以平衡泛化和精度。给定一个复杂的问题， \textsc{TaSR-RAG} 将其分解为具有显式潜在变量的三元组子查询的有序序列，然后通过混合三元组匹配执行逐步证据选择，该混合三元组匹配将原始三元组的语义相似性与类型三元组的结构一致性相结合。通过跨步骤维护显式实体绑定表，\textsc{TaSR-RAG} 可以解析中间变量并减少实体合并，而无需显式图构造或穷举搜索。对多个多跳问答基准的实验表明，\textsc{TaSR-RAG} 始终优于强 RAG 和结构化 RAG 基线高达 14\%，同时产生更清晰的证据归因和更忠实的推理痕迹。

Title: Quantifying and extending the coverage of spatial categorization data sets

• Authors: Wanchun Li, Alexandra Carstensen, Yang Xu, Terry Regier, Charles Kemp
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09373
• Pdf URL: https://arxiv.org/pdf/2603.09373
• Copy Paste: [[2603.09373]] Quantifying and extending the coverage of spatial categorization data sets(https://arxiv.org/abs/2603.09373)
• Keywords: language model, llm
• Abstract: Variation in spatial categorization across languages is often studied by eliciting human labels for the relations depicted in a set of scenes known as the Topological Relations Picture Series (TRPS). We demonstrate that labels generated by large language models (LLMs) align relatively well with human labels, and show how LLM-generated labels can help to decide which scenes and languages to add to existing spatial data sets. To illustrate our approach we extend the TRPS by adding 42 new scenes, and show that this extension achieves better coverage of the space of possible scenes than two previous extensions of the TRPS. Our results provide a foundation for scaling towards spatial data sets with dozens of languages and hundreds of scenes.
• 摘要：通常通过为一组称为拓扑关系图系列 (TRPS) 的场景中所描述的关系引出人类标签来研究跨语言的空间分类变化。我们证明了大型语言模型 (LLM) 生成的标签与人类标签的吻合程度相对较好，并展示了 LLM 生成的标签如何帮助决定将哪些场景和语言添加到现有空间数据集中。为了说明我们的方法，我们通过添加 42 个新场景来扩展 TRPS，并表明此扩展比 TRPS 的两个先前扩展实现了对可能场景空间的更好覆盖。我们的结果为扩展到具有数十种语言和数百个场景的空间数据集提供了基础。

Title: Reward Prediction with Factorized World States

• Authors: Yijun Shen, Delong Chen, Xianming Hu, Jiaming Mi, Hongbo Zhao, Kai Zhang, Pascale Fung
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09400
• Pdf URL: https://arxiv.org/pdf/2603.09400
• Copy Paste: [[2603.09400]] Reward Prediction with Factorized World States(https://arxiv.org/abs/2603.09400)
• Keywords: language model, llm, agent
• Abstract: Agents must infer action outcomes and select actions that maximize a reward signal indicating how close the goal is to being reached. Supervised learning of reward models could introduce biases inherent to training data, limiting generalization to novel goals and environments. In this paper, we investigate whether well-defined world state representations alone can enable accurate reward prediction across domains. To address this, we introduce StateFactory, a factorized representation method that transforms unstructured observations into a hierarchical object-attribute structure using language models. This structured representation allows rewards to be estimated naturally as the semantic similarity between the current state and the goal state under hierarchical constraint. Overall, the compact representation structure induced by StateFactory enables strong reward generalization capabilities. We evaluate on RewardPrediction, a new benchmark dataset spanning five diverse domains and comprising 2,454 unique action-observation trajectories with step-wise ground-truth rewards. Our method shows promising zero-shot results against both VLWM-critic and LLM-as-a-Judge reward models, achieving 60% and 8% lower EPIC distance, respectively. Furthermore, this superior reward quality successfully translates into improved agent planning performance, yielding success rate gains of +21.64% on AlfWorld and +12.40% on ScienceWorld over reactive system-1 policies and enhancing system-2 agent planning. Project Page: this https URL
• 摘要：智能体必须推断行动结果并选择能够最大化奖励信号的行动，该奖励信号表明距离实现目标有多近。奖励模型的监督学习可能会引入训练数据固有的偏差，限制对新目标和环境的泛化。在本文中，我们研究了仅明确定义的世界状态表示是否可以实现跨领域的准确奖励预测。为了解决这个问题，我们引入了 StateFactory，这是一种分解表示方法，可以使用语言模型将非结构化观察结果转换为分层对象属性结构。这种结构化表示允许在层次约束下将奖励自然地估计为当前状态和目标状态之间的语义相似性。总体而言，StateFactory 引入的紧凑表示结构具有强大的奖励泛化能力。我们对 RewardPrediction 进行评估，这是一个跨越五个不同领域的新基准数据集，包含 2,454 个独特的行动观察轨迹，并具有逐步的真实奖励。我们的方法在 VLWM-critic 和 LLM-as-a-Judge 奖励模型上显示出有希望的零样本结果，分别实现了 60% 和 8% 的 EPIC 距离降低。此外，这种卓越的奖励质量成功地转化为改进的智能体规划绩效，与反应性系统 1 策略相比，在 AlfWorld 上获得了 +21.64% 的成功率，在 ScienceWorld 上获得了 +12.40% 的成功率，并增强了系统 2 智能体规划。项目页面：此 https URL

Title: LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation

• Authors: Lukáš Eigler, Jindřich Libovický, David Hurych
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09403
• Pdf URL: https://arxiv.org/pdf/2603.09403
• Copy Paste: [[2603.09403]] LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation(https://arxiv.org/abs/2603.09403)
• Keywords: llm
• Abstract: Validating evaluation metrics for NLG typically relies on expensive and time-consuming human annotations, which predominantly exist only for English datasets. We propose \textit{LLM as a Meta-Judge}, a scalable framework that utilizes LLMs to generate synthetic evaluation datasets via controlled semantic degradation of real data, replacing human judgment. We validate our approach using \textit{meta-correlation}, measuring the alignment between metric rankings derived from synthetic data and those from standard human benchmarks. Experiments across Machine Translation, Question Answering, and Summarization demonstrate that synthetic validation serves as a reliable proxy for human judgment, achieving meta-correlations exceeding 0.9 in multilingual QA and proves to be a viable alternative where human judgments are unavailable or too expensive to obtain. Our code and data will become publicly available upon paper acceptance.
• 摘要：验证 NLG 的评估指标通常依赖于昂贵且耗时的人工注释，这些注释主要仅适用于英语数据集。我们提出 \textit{LLM 作为元法官}，这是一个可扩展的框架，利用 LLM 通过实际数据的受控语义退化来生成综合评估数据集，取代人类判断。我们使用 \textit{meta-correlation} 验证我们的方法，测量从合成数据得出的指标排名与来自标准人类基准的指标排名之间的一致性。机器翻译、问答和摘要方面的实验表明，综合验证可以作为人类判断的可靠代理，在多语言 QA 中实现超过 0.9 的元相关性，并被证明是在人类判断不可用或获取成本太高的情况下的可行替代方案。我们的代码和数据将在论文被接受后公开。

Title: Investigating Gender Stereotypes in Large Language Models via Social Determinants of Health

• Authors: Trung Hieu Ngo, Adrien Bazoge, Solen Quiniou, Pierre-Antoine Gourraud, Emmanuel Morin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.09416
• Pdf URL: https://arxiv.org/pdf/2603.09416
• Copy Paste: [[2603.09416]] Investigating Gender Stereotypes in Large Language Models via Social Determinants of Health(https://arxiv.org/abs/2603.09416)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) excel in Natural Language Processing (NLP) tasks, but they often propagate biases embedded in their training data, which is potentially impactful in sensitive domains like healthcare. While existing benchmarks evaluate biases related to individual social determinants of health (SDoH) such as gender or ethnicity, they often overlook interactions between these factors and lack context-specific assessments. This study investigates bias in LLMs by probing the relationships between gender and other SDoH in French patient records. Through a series of experiments, we found that embedded stereotypes can be probed using SDoH input and that LLMs rely on embedded stereotypes to make gendered decisions, suggesting that evaluating interactions among SDoH factors could usefully complement existing approaches to assessing LLM performance and bias.
• 摘要：大型语言模型 (LLM) 在自然语言处理 (NLP) 任务中表现出色，但它们经常传播训练数据中嵌入的偏差，这可能对医疗保健等敏感领域产生影响。虽然现有基准评估与个人健康社会决定因素（SDoH）相关的偏见，例如性别或种族，但它们往往忽视这些因素之间的相互作用，并且缺乏针对具体情况的评估。本研究通过探讨法国患者记录中性别与其他 SDoH 之间的关系来调查法学硕士的偏见。通过一系列实验，我们发现可以使用 SDoH 输入来探究嵌入的刻板印象，并且法学硕士依赖嵌入的刻板印象来做出性别决定，这表明评估 SDoH 因素之间的相互作用可以有效地补充评估法学硕士表现和偏见的现有方法。

Title: Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs

• Authors: Saugata Purkayastha, Pranav Kushare, Pragya Paramita Pal, Sukannya Purkayastha
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.09434
• Pdf URL: https://arxiv.org/pdf/2603.09434
• Copy Paste: [[2603.09434]] Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs(https://arxiv.org/abs/2603.09434)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are increasingly deployed across diverse real-world applications and user communities. As such, it is crucial that these models remain both morally grounded and knowledge-aware. In this work, we uncover a critical limitation of current LLMs -- their tendency to prioritize moral reasoning over commonsense understanding. To investigate this phenomenon, we introduce CoMoral, a novel benchmark dataset containing commonsense contradictions embedded within moral dilemmas. Through extensive evaluation of ten LLMs across different model sizes, we find that existing models consistently struggle to identify such contradictions without prior signal. Furthermore, we observe a pervasive narrative focus bias, wherein LLMs more readily detect commonsense contradictions when they are attributed to a secondary character rather than the primary (narrator) character. Our comprehensive analysis underscores the need for enhanced reasoning-aware training to improve the commonsense robustness of large language models.
• 摘要：大型语言模型 (LLM) 越来越多地部署在不同的现实世界应用程序和用户社区中。因此，这些模型必须保持道德基础和知识意识，这一点至关重要。在这项工作中，我们发现了当前法学硕士的一个关键局限性——他们倾向于优先考虑道德推理而不是常识理解。为了研究这一现象，我们引入了 CoMoral，这是一个新颖的基准数据集，其中包含嵌入道德困境的常识矛盾。通过对不同模型规模的十个法学硕士进行广泛评估，我们发现现有模型始终难以在没有事先信号的情况下识别此类矛盾。此外，我们观察到普遍存在的叙事焦点偏差，其中当法学硕士归因于次要角色而不是主要（叙述者）角色时，法学硕士更容易发现常识性矛盾。我们的综合分析强调需要加强推理意识训练，以提高大型语言模型的常识鲁棒性。

Title: You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases

• Authors: Isaia Gisler (1), Zhonghao He (2), Tianyi Qiu (3) ((1) ETH Zürich, (2) University of Cambridge, (3) Peking University)
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.09517
• Pdf URL: https://arxiv.org/pdf/2603.09517
• Copy Paste: [[2603.09517]] You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases(https://arxiv.org/abs/2603.09517)
• Keywords: language model, prompt, chain-of-thought
• Abstract: When language models are trained on synthetic data, they (student model) can covertly acquire behavioral traits from the data-generating model (teacher model). Subliminal learning refers to the transmission of traits from a teacher to a student model via training on data unrelated to those traits. Prior work demonstrated this in the training domains of number sequences, code, and math Chain-of-Thought traces including transmission of misaligned behaviors. We investigate whether transmission occurs through natural language paraphrases with fixed semantic content, and whether content explicitly contradicting the teacher's preference can block it. We find that training on paraphrases from a teacher system-prompted to love a particular animal increases a student's preference for that animal by up to 19 percentage points. This occurs when paraphrased content is semantically unrelated to the animal, or even when it explicitly expresses dislike. The transmission succeeds despite aggressive filtering to ensure paraphrase fidelity. This raises concerns for pipelines where models generate their own training data: content-based inspection cannot detect such transmission, and even preference-contradicting content fails to prevent it.
• 摘要：当语言模型接受合成数据的训练时，它们（学生模型）可以从数据生成模型（教师模型）中秘密获取行为特征。潜意识学习是指通过对与这些特征无关的数据进行训练，将特征从教师传递给学生模型。先前的工作在数字序列、代码和数学思维链跟踪（包括不一致行为的传输）的训练领域证明了这一点。我们研究传输是否通过具有固定语义内容的自然语言释义发生，以及与教师偏好明显矛盾的内容是否可以阻止它。我们发现，对教师系统提示喜爱某种特定动物的释义进行训练，可以使学生对该动物的偏好提高高达 19 个百分点。当释义内容在语义上与动物无关，或者甚至明确表达不喜欢时，就会发生这种情况。尽管进行了严格的过滤以确保释义保真度，但传输仍然成功。这引起了人们对模型生成自己的训练数据的管道的担忧：基于内容的检查无法检测到这种传输，甚至偏好矛盾的内容也无法阻止这种传输。

Title: ALARM: Audio-Language Alignment for Reasoning Models

• Authors: Petr Grinberg, Hassan Shahmohammadi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09556
• Pdf URL: https://arxiv.org/pdf/2603.09556
• Copy Paste: [[2603.09556]] ALARM: Audio-Language Alignment for Reasoning Models(https://arxiv.org/abs/2603.09556)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Large audio language models (ALMs) extend LLMs with auditory understanding. A common approach freezes the LLM and trains only an adapter on self-generated targets. However, this fails for reasoning LLMs (RLMs) whose built-in chain-of-thought traces expose the textual surrogate input, yielding unnatural responses. We propose self-rephrasing, converting self-generated responses into audio-understanding variants compatible with RLMs while preserving distributional alignment. We further fuse and compress multiple audio encoders for stronger representations. For training, we construct a 6M-instance multi-task corpus (2.5M unique prompts) spanning 19K hours of speech, music, and sound. Our 4B-parameter ALM outperforms similarly sized models and surpasses most larger ALMs on related audio-reasoning benchmarks, while preserving textual capabilities with a low training cost. Notably, we achieve the best open-source result on the MMAU-speech and MMSU benchmarks and rank third among all the models.
• 摘要：大型音频语言模型 (ALM) 扩展了法学硕士的听觉理解能力。一种常见的方法是冻结 LLM 并仅在自行生成的目标上训练适配器。然而，这对于推理法学硕士（RLM）来说是失败的，其内置的思维链轨迹暴露了文本代理输入，产生不自然的响应。我们提出自我改写，将自我生成的响应转换为与 RLM 兼容的音频理解变体，同时保持分布对齐。我们进一步融合和压缩多个音频编码器以获得更强的表示。为了进行训练，我们构建了一个 600 万个实例的多任务语料库（250 万个独特的提示），涵盖 19000 小时的语音、音乐和声音。我们的 4B 参数 ALM 优于类似大小的模型，并在相关音频推理基准上超越大多数较大的 ALM，同时以较低的训练成本保留文本功能。值得注意的是，我们在 MMAU 语音和 MMSU 基准测试中取得了最好的开源结果，并在所有模型中排名第三。

Title: Surgical Repair of Collapsed Attention Heads in ALiBi Transformers

• Authors: Palmer Schallon
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09616
• Pdf URL: https://arxiv.org/pdf/2603.09616
• Copy Paste: [[2603.09616]] Surgical Repair of Collapsed Attention Heads in ALiBi Transformers(https://arxiv.org/abs/2603.09616)
• Keywords: language model
• Abstract: We identify a systematic attention collapse pathology in the BLOOM family of transformer language models, where ALiBi positional encoding causes 31-44% of attention heads to attend almost entirely to the beginning-of-sequence token. The collapse follows a predictable pattern across four model scales (560M to 7.1B parameters), concentrating in head indices where ALiBi's slope schedule imposes the steepest distance penalties. We introduce surgical reinitialization: targeted Q/K/V reinitialization with zeroed output projections and gradient-masked freezing of all non-surgical parameters. Applied to BLOOM-1b7 on a single consumer GPU, the technique recovers 98.7% operational head capacity (242 to 379 of 384 heads) in two passes. A controlled comparison with C4 training data confirms that reinitialization -- not corpus content -- drives recovery, and reveals two distinct post-surgical phenomena: early global functional redistribution that improves the model, and late local degradation that accumulates under noisy training signal. An extended experiment reinitializing mostly-healthy heads alongside collapsed ones produces a model that transiently outperforms stock BLOOM-1b7 by 25% on training perplexity (12.70 vs. 16.99), suggesting that pretrained attention configurations are suboptimal local minima. Code, checkpoints, and diagnostic tools are released as open-source software.
• 摘要：我们在 Transformer 语言模型的 BLOOM 系列中发现了系统性注意力崩溃病理学，其中 ALiBi 位置编码导致 31-44% 的注意力头几乎完全关注序列开头标记。崩溃遵循四个模型尺度（560M 到 7.1B 参数）的可预测模式，集中在 ALiBi 坡度表施加最陡距离惩罚的头部指数中。我们引入手术重新初始化：有针对性的 Q/K/V 重新初始化，具有归零输出投影和所有非手术参数的梯度屏蔽冻结。该技术应用于单个消费级 GPU 上的 BLOOM-1b7 时，可在两次传递中恢复 98.7% 的操作磁头容量（384 个磁头中的 242 至 379 个）。与 C4 训练数据的受控比较证实，重新初始化（而不是语料库内容）驱动恢复，并揭示了两种不同的术后现象：改进模型的早期全局功能重新分配，以及在噪声训练信号下累积的后期局部退化。一项扩展实验将大部分健康的头部与崩溃的头部一起重新初始化，产生的模型在训练困惑度方面暂时优于库存 BLOOM-1b7 25%（12.70 与 16.99），这表明预训练的注意力配置是次优的局部最小值。代码、检查点和诊断工具作为开源软件发布。

Title: Tracking Cancer Through Text: Longitudinal Extraction From Radiology Reports Using Open-Source Large Language Models

• Authors: Luc Builtjes, Alessa Hering
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09638
• Pdf URL: https://arxiv.org/pdf/2603.09638
• Copy Paste: [[2603.09638]] Tracking Cancer Through Text: Longitudinal Extraction From Radiology Reports Using Open-Source Large Language Models(https://arxiv.org/abs/2603.09638)
• Keywords: language model, llm
• Abstract: Radiology reports capture crucial longitudinal information on tumor burden, treatment response, and disease progression, yet their unstructured narrative format complicates automated analysis. While large language models (LLMs) have advanced clinical text processing, most state-of-the-art systems remain proprietary, limiting their applicability in privacy-sensitive healthcare environments. We present a fully open-source, locally deployable pipeline for longitudinal information extraction from radiology reports, implemented using the \texttt{llm\_extractinator} framework. The system applies the \texttt{qwen2.5-72b} model to extract and link target, non-target, and new lesion data across time points in accordance with RECIST criteria. Evaluation on 50 Dutch CT Thorax/Abdomen report pairs yielded high extraction performance, with attribute-level accuracies of 93.7\% for target lesions, 94.9\% for non-target lesions, and 94.0\% for new lesions. The approach demonstrates that open-source LLMs can achieve clinically meaningful performance in multi-timepoint oncology tasks while ensuring data privacy and reproducibility. These results highlight the potential of locally deployable LLMs for scalable extraction of structured longitudinal data from routine clinical text.
• 摘要：放射学报告捕获了有关肿瘤负荷、治疗反应和疾病进展的重要纵向信息，但其非结构化叙述格式使自动分析变得复杂。虽然大型语言模型 (LLM) 具有先进的临床文本处理功能，但大多数最先进的系统仍然是专有的，限制了它们在隐私敏感的医疗保健环境中的适用性。我们提出了一个完全开源、可本地部署的管道，用于从放射学报告中提取纵向信息，并使用 \texttt{llm\_extractinator} 框架实现。该系统应用 \texttt{qwen2.5-72b} 模型根据 RECIST 标准提取和链接跨时间点的目标、非目标和新病变数据。对 50 个荷兰 CT 胸部/腹部报告对的评估产生了很高的提取性能，目标病灶的属性级别准确度为 93.7%，非目标病灶的属性级别准确度为 94.9%，新病灶的属性级别准确度为 94.0%。该方法表明，开源法学硕士可以在多时间点肿瘤学任务中实现具有临床意义的表现，同时确保数据隐私和可重复性。这些结果凸显了本地可部署的法学硕士在从常规临床文本中可扩展地提取结构化纵向数据方面的潜力。

Title: Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge: A keynote at ECIR 2025

• Authors: Isabelle Augenstein
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2603.09654
• Pdf URL: https://arxiv.org/pdf/2603.09654
• Copy Paste: [[2603.09654]] Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge: A keynote at ECIR 2025(https://arxiv.org/abs/2603.09654)
• Keywords: language model, llm
• Abstract: Language Models (LMs) acquire parametric knowledge from their training process, embedding it within their weights. The increasing scalability of LMs, however, poses significant challenges for understanding a model's inner workings and further for updating or correcting this embedded knowledge without the significant cost of retraining. Moreover, when using these language models for knowledge-intensive language understanding tasks, LMs have to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated knowledge. Nevertheless, studies indicate that LMs often ignore the provided context as it can be in conflict with the pre-existing LM's memory learned during pre-training. Conflicting knowledge can also already be present in the LM's parameters, termed intra-memory conflict. This underscores the importance of understanding the interplay between how a language model uses its parametric knowledge and the retrieved contextual knowledge. In this talk, I will aim to shed light on this important issue by presenting our research on evaluating the knowledge present in LMs, diagnostic tests that can reveal knowledge conflicts, as well as on understanding the characteristics of successfully used contextual knowledge.
• 摘要：语言模型 (LM) 从训练过程中获取参数知识，并将其嵌入到权重中。然而，LM 不断增强的可扩展性给理解模型的内部工作原理以及进一步更新或纠正这种嵌入式知识带来了重大挑战，而无需花费大量的再训练成本。此外，当使用这些语言模型进行知识密集型语言理解任务时，语言模型必须整合相关上下文，减轻其固有的弱点，例如不完整或过时的知识。然而，研究表明 LM 经常忽​​略所提供的上下文，因为它可能与预训练期间学到的预先存在的 LM 记忆相冲突。冲突的知识也可能已经存在于 LM 的参数中，称为内存内冲突。这强调了理解语言模型如何使用其参数知识和检索到的上下文知识之间相互作用的重要性。在本次演讲中，我的目标是通过介绍我们对评估 LM 中存在的知识、可以揭示知识冲突的诊断测试以及了解成功使用的上下文知识的特征的研究来阐明这一重要问题。

Title: Automatic Cardiac Risk Management Classification using large-context Electronic Patients Health Records

• Authors: Jacopo Vitale, David Della Morte, Luca Bacco, Mario Merone, Mark de Groot, Saskia Haitjema, Leandro Pecchia, Bram van Es
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2603.09685
• Pdf URL: https://arxiv.org/pdf/2603.09685
• Copy Paste: [[2603.09685]] Automatic Cardiac Risk Management Classification using large-context Electronic Patients Health Records(https://arxiv.org/abs/2603.09685)
• Keywords: language model, llm
• Abstract: To overcome the limitations of manual administrative coding in geriatric Cardiovascular Risk Management, this study introduces an automated classification framework leveraging unstructured Electronic Health Records (EHRs). Using a dataset of 3,482 patients, we benchmarked three distinct modeling paradigms on longitudinal Dutch clinical narratives: classical machine learning baselines, specialized deep learning architectures optimized for large-context sequences, and general-purpose generative Large Language Models (LLMs) in a zero-shot setting. Additionally, we evaluated a late fusion strategy to integrate unstructured text with structured medication embeddings and anthropometric data. Our analysis reveals that the custom Transformer architecture outperforms both traditional methods and generative \acs{llm}s, achieving the highest F1-scores and Matthews Correlation Coefficients. These findings underscore the critical role of specialized hierarchical attention mechanisms in capturing long-range dependencies within medical texts, presenting a robust, automated alternative to manual workflows for clinical risk stratification.
• 摘要：为了克服老年心血管风险管理中手动管理编码的局限性，本研究引入了利用非结构化电子健康记录 (EHR) 的自动分类框架。使用 3,482 名患者的数据集，我们对纵向荷兰临床叙述的三种不同建模范例进行了基准测试：经典机器学习基线、针对大上下文序列优化的专门深度学习架构以及零样本设置中的通用生成大语言模型 (LLM)。此外，我们评估了后期融合策略，将非结构化文本与结构化药物嵌入和人体测量数据相集成。我们的分析表明，定制的 Transformer 架构优于传统方法和生成式 \acs{llm}，实现了最高的 F1 分数和马修斯相关系数。这些发现强调了专门的分层注意力机制在捕获医学文本中的长期依赖性方面的关键作用，为临床风险分层的手动工作流程提供了一种强大的、自动化的替代方案。

Title: ESAinsTOD: A Unified End-to-End Schema-Aware Instruction-Tuning Framework for Task-Oriented Dialog Modeling

• Authors: Dechuan Teng, Chunlin Lu, Libo Qin, Wanxiang Che
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.09691
• Pdf URL: https://arxiv.org/pdf/2603.09691
• Copy Paste: [[2603.09691]] ESAinsTOD: A Unified End-to-End Schema-Aware Instruction-Tuning Framework for Task-Oriented Dialog Modeling(https://arxiv.org/abs/2603.09691)
• Keywords: language model, llm
• Abstract: Existing end-to-end modeling methods for modular task-oriented dialog systems are typically tailored to specific datasets, making it challenging to adapt to new dialog scenarios. In this work, we propose ESAinsTOD, a unified End-to-end Schema-Aware Instruction-tuning framework for general Task-Oriented Dialog modeling. This framework introduces a structured methodology to go beyond simply fine-tuning Large Language Models (LLMs), enabling flexible adaptation to various dialogue task flows and schemas. Specifically, we leverage full-parameter fine-tuning of LLMs and introduce two alignment mechanisms to make the resulting system both instruction-aware and schema-aware: (i) instruction alignment, which ensures that the system faithfully follows task instructions to complete various task flows from heterogeneous TOD datasets; and (ii) schema alignment, which encourages the system to make predictions adhering to the specified schema. In addition, we employ session-level end-to-end modeling, which allows the system to access the results of previously executed task flows within the dialogue history, to bridge the gap between the instruction-tuning paradigm and the real-world application of TOD systems. Empirical results show that while a fine-tuned LLM serves as a strong baseline, our structured approach provides significant additional benefits. In particular, our findings indicate that: (i) ESAinsTOD outperforms state-of-the-art models by a significant margin on end-to-end task-oriented dialog modeling benchmarks: CamRest676, In-Car and MultiWOZ; (ii) more importantly, it exhibits superior generalization capabilities across various low-resource settings, with the proposed alignment mechanisms significantly enhancing zero-shot performance; and (iii) our instruction-tuning paradigm substantially improves the model's robustness against data noise and cascading errors.
• 摘要：面向模块化任务的对话系统的现有端到端建模方法通常是针对特定数据集定制的，这使得适应新的对话场景具有挑战性。在这项工作中，我们提出了 ESAinsTOD，一个用于通用面向任务的对话建模的统一的端到端模式感知指令调整框架。该框架引入了一种结构化方法，超越了简单地微调大型语言模型（LLM）的范围，能够灵活适应各种对话任务流和模式。具体来说，我们利用LLM的全参数微调，并引入两种对齐机制，使所得系统既具有指令感知能力又具有模式感知能力：（i）指令对齐，确保系统忠实地遵循任务指令，以完成来自异构TOD数据集的各种任务流； (ii) 模式对齐，鼓励系统按照指定的模式进行预测。此外，我们采用会话级端到端建模，允许系统访问对话历史记录中先前执行的任务流的结果，以弥合指令调整范式与 TOD 系统的实际应用之间的差距。实证结果表明，虽然经过微调的法学硕士可以作为强有力的基线，但我们的结构化方法提供了显着的额外好处。特别是，我们的研究结果表明：(i) ESAinsTOD 在端到端面向任务的对话建模基准上显着优于最先进的模型：CamRest676、In-Car 和 MultiWOZ； (ii) 更重要的是，它在各种低资源环境中表现出卓越的泛化能力，所提出的对齐机制显着增强了零样本性能； (iii)我们的指令调整范例大大提高了模型针对数据噪声和级联错误的鲁棒性。

Title: Evaluation of LLMs in retrieving food and nutritional context for RAG systems

• Authors: Maks Požarnik Vavken, Matevž Ogrinc, Tome Eftimov, Barbara Koroušić Seljak
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09704
• Pdf URL: https://arxiv.org/pdf/2603.09704
• Copy Paste: [[2603.09704]] Evaluation of LLMs in retrieving food and nutritional context for RAG systems(https://arxiv.org/abs/2603.09704)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: In this article, we evaluate four Large Language Models (LLMs) and their effectiveness at retrieving data within a specialized Retrieval-Augmented Generation (RAG) system, using a comprehensive food composition database. Our method is focused on the LLMs ability to translate natural language queries into structured metadata filters, enabling efficient retrieval via a Chroma vector database. By achieving high accuracy in this critical retrieval step, we demonstrate that LLMs can serve as an accessible, high-performance tool, drastically reducing the manual effort and technical expertise previously required for domain experts, such as food compilers and nutritionists, to leverage complex food and nutrition data. However, despite the high performance on easy and moderately complex queries, our analysis of difficult questions reveals that reliable retrieval remains challenging when queries involve non-expressible constraints. These findings demonstrate that LLM-driven metadata filtering excels when constraints can be explicitly expressed, but struggles when queries exceed the representational scope of the metadata format.
• 摘要：在本文中，我们使用综合食品成分数据库评估了四种大型语言模型 (LLM) 及其在专门的检索增强生成 (RAG) 系统中检索数据的有效性。我们的方法侧重于法学硕士将自然语言查询转换为结构化元数据过滤器的能力，从而能够通过 Chroma 向量数据库进行高效检索。通过在这一关键检索步骤中实现高精度，我们证明法学硕士可以作为一种易于使用的高性能工具，大大减少以前食品编译者和营养学家等领域专家利用复杂食品和营养数据所需的人工工作和技术专业知识。然而，尽管在简单和中等复杂的查询上具有高性能，但我们对困难问题的分析表明，当查询涉及不可表达的约束时，可靠的检索仍然具有挑战性。这些发现表明，当约束可以明确表达时，LLM 驱动的元数据过滤表现出色，但当查询超出元数据格式的表示范围时，就会陷入困境。

Title: RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation

• Authors: Sihong Wu, Yiling Ma, Yilun Zhao, Tiansheng Hu, Owen Jiang, Manasi Patwardhan, Arman Cohan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.09723
• Pdf URL: https://arxiv.org/pdf/2603.09723
• Copy Paste: [[2603.09723]] RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation(https://arxiv.org/abs/2603.09723)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are increasingly used across the scientific workflow, including to draft peer-review reports. However, many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses. We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning. Rebuttals show which reviewer comments led to concrete revisions or specific plans, and which were only defended. Building on this insight, we leverage rebuttal as implicit supervision to directly optimize a feedback generator for actionability. To support this objective, we propose a new task called perspective-conditioned segment-level review feedback generation, in which the model is required to produce a single focused comment based on the complete paper and a specified perspective such as experiments and writing. We also build a large dataset named RMR-75K that maps review segments to the rebuttal segments that address them, with perspective labels and impact categories that order author uptake. We then train the Llama-3.1-8B-Instruct model with supervised fine-tuning on review segments followed by preference optimization using rebuttal derived pairs. Experiments with human experts and LLM-as-a-judge show consistent gains in actionability and specificity over strong baselines while maintaining grounding and relevance.
• 摘要：大语言模型 (LLM) 越来越多地在科学工作流程中使用，包括起草同行评审报告。然而，许多人工智能生成的评论都是肤浅的且缺乏可操作性，使作者缺乏具体的、可实施的指导，并激发了这项工作所解决的差距。我们提出 RbtAct，其目标是生成可操作的评审反馈，并将现有的同行评审反驳置于学习的中心。反驳表明哪些审稿人的意见导致了具体的修改或具体计划，哪些只是进行了辩护。基于这一见解，我们利用反驳作为隐式监督来直接优化反馈生成器的可操作性。为了支持这一目标，我们提出了一项称为视角条件分段级评论反馈生成的新任务，其中模型需要根据完整的论文和特定的视角（例如实验和写作）生成单一的重点评论。我们还构建了一个名为 RMR-75K 的大型数据集，它将评论片段映射到解决这些问题的反驳片段，并带有观点标签和影响类别，以排序作者的吸收。然后，我们通过对评论片段进行监督微调来训练 Llama-3.1-8B-Instruct 模型，然后使用反驳派生对进行偏好优化。人类专家和法学硕士作为法官进行的实验表明，在保持基础性和相关性的同时，在强大的基线上，可操作性和特异性不断提高。

Title: Beyond Fine-Tuning: Robust Food Entity Linking under Ontology Drift with FoodOntoRAG

• Authors: Jan Drole, Ana Gjorgjevikj, Barbara Korouši'c Seljak, Tome Eftimov
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09758
• Pdf URL: https://arxiv.org/pdf/2603.09758
• Copy Paste: [[2603.09758]] Beyond Fine-Tuning: Robust Food Entity Linking under Ontology Drift with FoodOntoRAG(https://arxiv.org/abs/2603.09758)
• Keywords: language model, llm, agent
• Abstract: Standardizing food terms from product labels and menus into ontology concepts is a prerequisite for trustworthy dietary assessment and safety reporting. The dominant approach to Named Entity Linking (NEL) in the food and nutrition domains fine-tunes Large Language Models (LLMs) on task-specific corpora. Although effective, fine-tuning incurs substantial computational cost, ties models to a particular ontology snapshot (i.e., version), and degrades under ontology drift. This paper presents FoodOntoRAG, a model- and ontology-agnostic pipeline that performs few-shot NEL by retrieving candidate entities from domain ontologies and conditioning an LLM on structured evidence (food labels, synonyms, definitions, and relations). A hybrid lexical--semantic retriever enumerates candidates; a selector agent chooses a best match with rationale; a separate scorer agent calibrates confidence; and, when confidence falls below a threshold, a synonym generator agent proposes reformulations to re-enter the loop. The pipeline approaches state-of-the-art accuracy while revealing gaps and inconsistencies in existing annotations. The design avoids fine-tuning, improves robustness to ontology evolution, and yields interpretable decisions through grounded justifications.
• 摘要：将产品标签和菜单中的食品术语标准化为本体概念是可靠饮食评估和安全报告的先决条件。食品和营养领域中命名实体链接 (NEL) 的主要方法是在特定任务的语料库上微调大型语言模型 (LLM)。尽管有效，但微调会产生大量的计算成本，将模型与特定的本体快照（即版本）联系起来，并在本体漂移的情况下降低性能。本文介绍了 FoodOntoRAG，这是一种与模型和本体无关的管道，通过从领域本体中检索候选实体并根据结构化证据（食品标签、同义词、定义和关系）调节法学硕士来执行少量 NEL。混合词汇语义检索器枚举候选者；选择代理选择与基本原理最匹配的；一个单独的记分代理来校准信心；并且，当置信度低于阈值时，同义词生成器代理会建议重新表述以重新进入循环。该管道达到了最先进的准确性，同时揭示了现有注释中的差距和不一致之处。该设计避免了微调，提高了本体进化的鲁棒性，并通过有根据的理由产生可解释的决策。

Title: EPIC-EuroParl-UdS: Information-Theoretic Perspectives on Translation and Interpreting

• Authors: Maria Kunilovskaya, Christina Pollkläsener
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09785
• Pdf URL: https://arxiv.org/pdf/2603.09785
• Copy Paste: [[2603.09785]] EPIC-EuroParl-UdS: Information-Theoretic Perspectives on Translation and Interpreting(https://arxiv.org/abs/2603.09785)
• Keywords: gpt
• Abstract: This paper introduces an updated and combined version of the bidirectional English-German EPIC-UdS (spoken) and EuroParl-UdS (written) corpora containing original European Parliament speeches as well as their translations and interpretations. The new version corrects metadata and text errors identified through previous use, refines the content, updates linguistic annotations, and adds new layers, including word alignment and word-level surprisal indices. The combined resource is designed to support research using information-theoretic approaches to language variation, particularly studies comparing written and spoken modes, and examining disfluencies in speech, as well as traditional translationese studies, including parallel (source vs. target) and comparable (original vs. translated) analyses. The paper outlines the updates introduced in this release, summarises previous results based on the corpus, and presents a new illustrative study. The study validates the integrity of the rebuilt spoken data and evaluates probabilistic measures derived from base and fine-tuned GPT-2 and machine translation models on the task of filler particles prediction in interpreting.
• 摘要：本文介绍了双向英德 EPIC-UdS（口语）和 EuroParl-UdS（书面）语料库的更新和组合版本，其中包含原始欧洲议会演讲及其翻译和解释。新版本纠正了之前使用中发现的元数据和文本错误，细化了内容，更新了语言注释，并添加了新的图层，包括单词对齐和单词级惊喜索引。合并的资源旨在支持使用信息论方法来研究语言变异的研究，特别是比较书面和口语模式的研究，以及检查言语不流畅的研究，以及传统的翻译研究，包括平行（源与目标）和可比（原文与翻译）分析。本文概述了此版本中引入的更新，总结了之前基于语料库的结果，并提出了一项新的说明性研究。该研究验证了重建口语数据的完整性，并评估了从基本和微调的 GPT-2 以及机器翻译模型得出的概率测量，以完成口译中填充颗粒预测的任务。

Title: One-Eval: An Agentic System for Automated and Traceable LLM Evaluation

• Authors: Chengyu Shen, Yanheng Hou, Minghui Pan, Runming He, Zhen Hao Wong, Meiyi Qiang, Zhou Liu, Hao Liang, Peichao Lai, Zeang Sheng, Wentao Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09821
• Pdf URL: https://arxiv.org/pdf/2603.09821
• Copy Paste: [[2603.09821]] One-Eval: An Agentic System for Automated and Traceable LLM Evaluation(https://arxiv.org/abs/2603.09821)
• Keywords: language model, llm, agent
• Abstract: Reliable evaluation is essential for developing and deploying large language models, yet in practice it often requires substantial manual effort: practitioners must identify appropriate benchmarks, reproduce heterogeneous evaluation codebases, configure dataset schema mappings, and interpret aggregated metrics. To address these challenges, we present One-Eval, an agentic evaluation system that converts natural-language evaluation requests into executable, traceable, and customizable evaluation workflows. One-Eval integrates (i) NL2Bench for intent structuring and personalized benchmark planning, (ii) BenchResolve for benchmark resolution, automatic dataset acquisition, and schema normalization to ensure executability, and (iii) Metrics \& Reporting for task-aware metric selection and decision-oriented reporting beyond scalar scores. The system further incorporates human-in-the-loop checkpoints for review, editing, and rollback, while preserving sample evidence trails for debugging and auditability. Experiments show that One-Eval can execute end-to-end evaluations from diverse natural-language requests with minimal user effort, supporting more efficient and reproducible evaluation in industrial settings. Our framework is publicly available at this https URL.
• 摘要：可靠的评估对于开发和部署大型语言模型至关重要，但在实践中，它通常需要大量的手动工作：从业者必须确定适当的基准、重现异构评估代码库、配置数据集模式映射并解释聚合指标。为了应对这些挑战，我们推出了 One-Eval，这是一种代理评估系统，可将自然语言评估请求转换为可执行、可追踪和可定制的评估工作流程。 One-Eval 集成了 (i) 用于意图构建和个性化基准规划的 NL2Bench，(ii) 用于基准解析、自动数据集采集和模式标准化以确保可执行性的 BenchResolve，以及 (iii) 用于任务感知指标选择和超越标量分数的面向决策的报告的 Metrics \& Reporting。该系统还包含用于审查、编辑和回滚的人机循环检查点，同时保留样本证据跟踪以进行调试和可审计。实验表明，One-Eval 可以以最少的用户工作量根据不同的自然语言请求执行端到端评估，支持工业环境中更高效、可重复的评估。我们的框架可通过此 https URL 公开获取。

Title: Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents

• Authors: Naman Gupta, Vaibhav Singh, Arun Iyer, Kirankumar Shiragur, Pratham Grover, Ramakrishna B. Bairi, Ritabrata Maiti, Sankarshan Damle, Shachee Mishra Gupta, Rishikesh Maurya, Vageesh D. C
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09835
• Pdf URL: https://arxiv.org/pdf/2603.09835
• Copy Paste: [[2603.09835]] Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents(https://arxiv.org/abs/2603.09835)
• Keywords: llm, long context, agent
• Abstract: Sequential multi-agent reasoning frameworks such as Chain-of-Agents (CoA) handle long-context queries by decomposing inputs into chunks and processing them sequentially using LLM-based worker agents that read from and update a bounded shared memory. From a probabilistic perspective, CoA aims to approximate the conditional distribution corresponding to a model capable of jointly reasoning over the entire long context. CoA achieves this through a latent-state factorization in which only bounded summaries of previously processed evidence are passed between agents. The resulting bounded-memory approximation introduces a lossy information bottleneck, making the final evidence state inherently dependent on the order in which chunks are processed. In this work, we study the problem of chunk ordering for long-context reasoning. We use the well-known Chow-Liu trees to learn a dependency structure that prioritizes strongly related chunks. Empirically, we show that a breadth-first traversal of the resulting tree yields chunk orderings that reduce information loss across agents and consistently outperform both default document-chunk ordering and semantic score-based ordering in answer relevance and exact-match accuracy across three long-context benchmarks.
• 摘要：顺序多代理推理框架（例如代理链 (CoA)）通过将输入分解为块并使用基于 LLM 的工作代理（读取和更新有界共享内存）顺序处理它们来处理长上下文查询。从概率的角度来看，CoA 的目标是近似与能够在整个长上下文上联合推理的模型相对应的条件分布。 CoA 通过潜在状态分解来实现这一点，其中仅在代理之间传递先前处理的证据的有限摘要。由此产生的有界内存近似引入了有损信息瓶颈，使得最终证据状态本质上依赖于块的处理顺序。在这项工作中，我们研究了长上下文推理的块排序问题。我们使用著名的 Chow-Liu 树来学习优先考虑强相关块的依赖结构。根据经验，我们表明，对结果树进行广度优先遍历产生的块排序可以减少代理之间的信息丢失，并且在三个长上下文基准的答案相关性和精确匹配精度方面始终优于默认文档块排序和基于语义分数的排序。

Title: N-gram-like Language Models Predict Reading Time Best

• Authors: James A. Michaelov, Roger P. Levy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09872
• Pdf URL: https://arxiv.org/pdf/2603.09872
• Copy Paste: [[2603.09872]] N-gram-like Language Models Predict Reading Time Best(https://arxiv.org/abs/2603.09872)
• Keywords: language model
• Abstract: Recent work has found that contemporary language models such as transformers can become so good at next-word prediction that the probabilities they calculate become worse for predicting reading time. In this paper, we propose that this can be explained by reading time being sensitive to simple n-gram statistics rather than the more complex statistics learned by state-of-the-art transformer language models. We demonstrate that the neural language models whose predictions are most correlated with n-gram probability are also those that calculate probabilities that are the most correlated with eye-tracking-based metrics of reading time on naturalistic text.
• 摘要：最近的研究发现，诸如 Transformer 之类的当代语言模型在下一个单词预测方面变得如此出色，以至于它们计算出的概率在预测阅读时间方面变得更差。在本文中，我们提出这可以通过阅读时间对简单的 n 元语法统计数据敏感而不是通过最先进的 Transformer 语言模型学习到的更复杂的统计数据来解释。我们证明，预测与 n 元语法概率最相关的神经语言模型也是那些计算与基于眼动追踪的自然文本阅读时间指标最相关的概率的神经语言模型。

Title: Do What I Say: A Spoken Prompt Dataset for Instruction-Following

• Authors: Maike Züfle, Sara Papi, Fabian Retkowski, Szymon Mazurek, Marek Kasztelnik, Alexander Waibel, Luisa Bentivogli, Jan Niehues
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09881
• Pdf URL: https://arxiv.org/pdf/2603.09881
• Copy Paste: [[2603.09881]] Do What I Say: A Spoken Prompt Dataset for Instruction-Following(https://arxiv.org/abs/2603.09881)
• Keywords: language model, llm, prompt
• Abstract: Speech Large Language Models (SLLMs) have rapidly expanded, supporting a wide range of tasks. These models are typically evaluated using text prompts, which may not reflect real-world scenarios where users interact with speech. To address this gap, we introduce DoWhatISay (DOWIS), a multilingual dataset of human-recorded spoken and written prompts designed to pair with any existing benchmark for realistic evaluation of SLLMs under spoken instruction conditions. Spanning 9 tasks and 11 languages, it provides 10 prompt variants per task-language pair, across five styles. Using DOWIS, we benchmark state-of-the-art SLLMs, analyzing the interplay between prompt modality, style, language, and task type. Results show that text prompts consistently outperform spoken prompts, particularly for low-resource and cross-lingual settings. Only for tasks with speech output, spoken prompts do close the gap, highlighting the need for speech-based prompting in SLLM evaluation.
• 摘要：语音大型语言模型 (SLLM) 迅速扩展，支持广泛的任务。这些模型通常使用文本提示进行评估，这可能无法反映用户与语音交互的现实场景。为了解决这一差距，我们引入了 DoWhatISay (DOWIS)，这是一个由人工录制的口语和书面提示组成的多语言数据集，旨在与任何现有基准配对，以便在口语教学条件下对 SLLM 进行实际评估。它涵盖 9 种任务和 11 种语言，为每个任务语言对提供 10 种提示变体，涵盖 5 种风格。使用 DOWIS，我们对最先进的 SLLM 进行基准测试，分析提示模式、风格、语言和任务类型之间的相互作用。结果表明，文本提示始终优于语音提示，特别是在资源匮乏和跨语言的环境中。仅对于具有语音输出的任务，语音提示确实可以缩小差距，这凸显了 SLLM 评估中基于语音的提示的必要性。

Title: Benchmarking Political Persuasion Risks Across Frontier Large Language Models

• Authors: Zhongren Chen, Joshua Kalla, Quan Le
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2603.09884
• Pdf URL: https://arxiv.org/pdf/2603.09884
• Copy Paste: [[2603.09884]] Benchmarking Political Persuasion Risks Across Frontier Large Language Models(https://arxiv.org/abs/2603.09884)
• Keywords: language model, gpt, llm, prompt
• Abstract: Concerns persist regarding the capacity of Large Language Models (LLMs) to sway political views. Although prior research has claimed that LLMs are not more persuasive than standard political campaign practices, the recent rise of frontier models warrants further study. In two survey experiments (N=19,145) across bipartisan issues and stances, we evaluate seven state-of-the-art LLMs developed by Anthropic, OpenAI, Google, and xAI. We find that LLMs outperform standard campaign advertisements, with heterogeneity in performance across models. Specifically, Claude models exhibit the highest persuasiveness, while Grok exhibits the lowest. The results are robust across issues and stances. Moreover, in contrast to the findings in Hackenburg et al. (2025b) and Lin et al. (2025) that information-based prompts boost persuasiveness, we find that the effectiveness of information-based prompts is model-dependent: they increase the persuasiveness of Claude and Grok while substantially reducing that of GPT. We introduce a data-driven and strategy-agnostic LLM-assisted conversation analysis approach to identify and assess underlying persuasive strategies. Our work benchmarks the persuasive risks of frontier models and provides a framework for cross-model comparative risk assessment.
• 摘要：人们对于大型语言模型 (LLM) 影响政治观点的能力仍然存在担忧。尽管之前的研究声称法学硕士并不比标准的政治竞选实践更有说服力，但最近兴起的前沿模型值得进一步研究。在两项针对两党问题和立场的调查实验 (N=19,145) 中，我们评估了由 Anthropic、OpenAI、Google 和 xAI 开发的七个最先进的法学硕士。我们发现法学硕士的表现优于标准竞选广告，不同模型的表现存在异质性。具体来说，Claude 模型表现出最高的说服力，而 Grok 表现出最低的说服力。各个问题和立场的结果都是稳健的。此外，与 Hackenburg 等人的研究结果相反。 (2025b) 和 Lin 等人。 （2025）认为基于信息的提示可以提高说服力，我们发现基于信息的提示的有效性取决于模型：它们增加了 Claude 和 Grok 的说服力，同时大大降低了 GPT 的说服力。我们引入了一种数据驱动且与策略无关的法学硕士辅助对话分析方法来识别和评估潜在的说服策略。我们的工作对前沿模型的说服性风险进行了基准测试，并为跨模型比较风险评估提供了框架。

Title: Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs

• Authors: Zorik Gekhman, Roee Aharoni, Eran Ofek, Mor Geva, Roi Reichart, Jonathan Herzig
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09906
• Pdf URL: https://arxiv.org/pdf/2603.09906
• Copy Paste: [[2603.09906]] Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs(https://arxiv.org/abs/2603.09906)
• Keywords: llm, hallucination
• Abstract: While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not require step-by-step logical decomposition, making the utility of reasoning highly counterintuitive. Nevertheless, we find that enabling reasoning substantially expands the capability boundary of the model's parametric knowledge recall, unlocking correct answers that are otherwise effectively unreachable. Why does reasoning aid parametric knowledge recall when there are no complex reasoning steps to be done? To answer this, we design a series of hypothesis-driven controlled experiments, and identify two key driving mechanisms: (1) a computational buffer effect, where the model uses the generated reasoning tokens to perform latent computation independent of their semantic content; and (2) factual priming, where generating topically related facts acts as a semantic bridge that facilitates correct answer retrieval. Importantly, this latter generative self-retrieval mechanism carries inherent risks: we demonstrate that hallucinating intermediate facts during reasoning increases the likelihood of hallucinations in the final answer. Finally, we show that our insights can be harnessed to directly improve model accuracy by prioritizing reasoning trajectories that contain hallucination-free factual statements.
• 摘要：虽然法学硕士的推理在数学、代码生成和多跳事实问题中发挥着自然的作用，但它对简单、单跳事实问题的影响仍不清楚。此类问题不需要逐步的逻辑分解，使得推理的效用非常违反直觉。尽管如此，我们发现启用推理极大地扩展了模型参数化知识回忆的能力边界，解锁了否则实际上无法获得的正确答案。当没有复杂的推理步骤需要完成时，为什么推理有助于参数知识的回忆？为了回答这个问题，我们设计了一系列假设驱动的受控实验，并确定了两个关键的驱动机制：（1）计算缓冲效应，模型使用生成的推理标记来执行独立于其语义内容的潜在计算； (2)事实启动，生成主题相关的事实充当语义桥梁，促进正确答案检索。重要的是，后一种生成式自我检索机制具有固有的风险：我们证明，在推理过程中产生幻觉中间事实会增加最终答案中产生幻觉的可能性。最后，我们表明，通过优先考虑包含无幻觉事实陈述的推理轨迹，可以利用我们的见解来直接提高模型的准确性。

Title: Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions

• Authors: Mingyang Song, Mao Zheng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09938
• Pdf URL: https://arxiv.org/pdf/2603.09938
• Copy Paste: [[2603.09938]] Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions(https://arxiv.org/abs/2603.09938)
• Keywords: language model, llm
• Abstract: Model merging has emerged as a transformative paradigm for combining the capabilities of multiple neural networks into a single unified model without additional training. With the rapid proliferation of fine-tuned large language models~(LLMs), merging techniques offer a computationally efficient alternative to ensembles and full retraining, enabling practitioners to compose specialized capabilities at minimal cost. This survey presents a comprehensive and structured examination of model merging in the LLM era through the \textbf{FUSE} taxonomy, a four-dimensional framework organized along \textbf{F}oundations, \textbf{U}nification Strategies, \textbf{S}cenarios, and \textbf{E}cosystem. We first establish the theoretical underpinnings of merging, including loss landscape geometry, mode connectivity, and the linear mode connectivity hypothesis. We then systematically review the algorithmic landscape, spanning weight averaging, task vector arithmetic, sparsification-enhanced methods, mixture-of-experts architectures, and evolutionary optimization approaches. For each method family, we analyze the core formulation, highlight representative works, and discuss practical trade-offs. We further examine downstream applications across multi-task learning, safety alignment, domain specialization, multilingual transfer, and federated learning. Finally, we survey the supporting ecosystem of open-source tools, community platforms, and evaluation benchmarks, and identify key open challenges including theoretical gaps, scalability barriers, and standardization needs. This survey aims to equip researchers and practitioners with a structured foundation for advancing model merging.
• 摘要：模型合并已成为一种变革范式，无需额外训练即可将多个神经网络的功能组合成一个统一的模型。随着微调大型语言模型（LLM）的快速普及，合并技术为集成和完全再训练提供了计算高效的替代方案，使从业者能够以最低的成本组合专业功能。这项调查通过 \textbf{FUSE} 分类法对法学硕士时代的模型合并进行了全面和结构化的检查，这是一个沿着 \textbf{F}基础、\textbf{U}化策略、\textbf{S}场景和 \textbf{E}生态系统组织的四维框架。我们首先建立合并的理论基础，包括损失景观几何、模式连通性和线性模式连通性假设。然后，我们系统地回顾了算法景观、跨越权重平均、任务向量算术、稀疏增强方法、专家混合架构和进化优化方法。对于每个方法族，我们分析核心公式，突出代表性作品，并讨论实际的权衡。我们进一步研究跨多任务学习、安全调整、领域专业化、多语言迁移和联邦学习的下游应用程序。最后，我们调查了开源工具、社区平台和评估基准的支持生态系统，并确定了关键的开放挑战，包括理论差距、可扩展性障碍和标准化需求。这项调查旨在为研究人员和从业者提供推进模型合并的结构化基础。

Title: CREATE: Testing LLMs for Associative Creativity

• Authors: Manya Wadhwa, Tiasa Singha Roy, Harvey Lederman, Junyi Jessy Li, Greg Durrett
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.09970
• Pdf URL: https://arxiv.org/pdf/2603.09970
• Copy Paste: [[2603.09970]] CREATE: Testing LLMs for Associative Creativity(https://arxiv.org/abs/2603.09970)
• Keywords: llm, prompt
• Abstract: A key component of creativity is associative reasoning: the ability to draw novel yet meaningful connections between concepts. We introduce CREATE, a benchmark designed to evaluate models' capacity for creative associative reasoning. CREATE requires models to generate sets of paths connecting concepts in a model's parametric knowledge. Paths should have high specificity (distinctiveness and closeness of the concept connection) and high diversity (dissimilarity from other paths), and models are scored more highly if they produce a larger set of strong, diverse paths. This task shares demands of real creativity tasks like hypothesis generation, including an extremely large search space, but enables collection of a sizable benchmark with objective answer grading. Evaluation of frontier models shows that the strongest models achieve higher creative utility than others, with the high multiplicity of answers and complexity of the search making benchmark saturation difficult to achieve. Furthermore, our results illustrate that thinking models are not always more effective on our task, even with high token budgets. Recent approaches for creative prompting give some but limited additional improvement. CREATE provides a sandbox for developing new methods to improve models' capacity for associative creativity.
• 摘要：创造力的一个关键组成部分是联想推理：在概念之间建立新颖而有意义的联系的能力。我们引入了 CREATE，这是一个旨在评估模型创造性联想推理能力的基准。 CREATE 要求模型生成连接模型参数知识中的概念的路径集。路径应该具有高特异性（概念联系的独特性和紧密性）和高多样性（与其他路径的不同性），如果模型产生更大的一组强大的、多样化的路径，则模型的得分更高。该任务与假设生成等真正的创造力任务有共同的需求，包括极大的搜索空间，但可以收集具有客观答案评分的相当大的基准。对前沿模型的评估表明，最强的模型比其他模型具有更高的创造性效用，答案的多样性和搜索的复杂性使得基准饱和难以实现。此外，我们的结果表明，即使代币预算很高，思维模型也并不总是对我们的任务更有效。最近的创造性提示方法提供了一些但有限的额外改进。 CREATE 提供了一个沙箱，用于开发新方法来提高模型的联想创造力能力。