2025-09-29

Title: A Novel Differential Feature Learning for Effective Hallucination Detection and Classification

• Authors: Wenkai Wang, Vincent Lee, Yizhen Zheng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21357
• Pdf URL: https://arxiv.org/pdf/2509.21357
• Copy Paste: [[2509.21357]] A Novel Differential Feature Learning for Effective Hallucination Detection and Classification(https://arxiv.org/abs/2509.21357)
• Keywords: language model, hallucination
• Abstract: Large language model hallucination represents a critical challenge where outputs deviate from factual accuracy due to distributional biases in training data. While recent investigations establish that specific hidden layers exhibit differences between hallucinatory and factual content, the precise localization of hallucination signals within layers remains unclear, limiting the development of efficient detection methods. We propose a dual-model architecture integrating a Projected Fusion (PF) block for adaptive inter-layer feature weighting and a Differential Feature Learning (DFL) mechanism that identifies discriminative features by computing differences between parallel encoders learning complementary representations from identical inputs. Through systematic experiments across HaluEval's question answering, dialogue, and summarization datasets, we demonstrate that hallucination signals concentrate in highly sparse feature subsets, achieving significant accuracy improvements on question answering and dialogue tasks. Notably, our analysis reveals a hierarchical "funnel pattern" where shallow layers exhibit high feature diversity while deep layers demonstrate concentrated usage, enabling detection performance to be maintained with minimal degradation using only 1\% of feature dimensions. These findings suggest that hallucination signals are more concentrated than previously assumed, offering a pathway toward computationally efficient detection systems that could reduce inference costs while maintaining accuracy.
• 摘要：大型语言模型幻觉是一个关键的挑战，即由于培训数据中的分配偏见，产出偏离了事实准确性。尽管最近的研究表明，特定的隐藏层在幻觉和事实含量之间表现出差异，但层中幻觉信号的精确定位仍不清楚，从而限制了有效的检测方法的发展。我们提出了一种双模型体系结构，该体系结构集成了用于自适应间层间特征加权和差异特征学习（DFL）机制的投影融合（PF）块，该机制通过计算平行编码者之间的差异差异来识别歧视性特征，从而从相同的输入中计算学习互补表示。通过在Halueval的问题答案，对话和摘要数据集中进行的系统实验，我们证明了幻觉信号集中在高度稀疏的特征子集中，从而在问答和对话任务上取得了重大准确的改进。值得注意的是，我们的分析揭示了层次的“漏斗模式”，其中浅层表现出较高的特征多样性，而深层则表现出浓缩用法，从而使检测性能仅使用1 \％的特征维度来维持最小的降解。这些发现表明，幻觉信号比以前假设的更集中，为计算有效的检测系统提供了一种可以降低推理成本同时保持准确性的途径。

Title: Influence Guided Context Selection for Effective Retrieval-Augmented Generation

• Authors: Jiale Deng, Yanyan Shen, Ziyuan Pei, Youmin Chen, Linpeng Huang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21359
• Pdf URL: https://arxiv.org/pdf/2509.21359
• Copy Paste: [[2509.21359]] Influence Guided Context Selection for Effective Retrieval-Augmented Generation(https://arxiv.org/abs/2509.21359)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) addresses large language model (LLM) hallucinations by grounding responses in external knowledge, but its effectiveness is compromised by poor-quality retrieved contexts containing irrelevant or noisy information. While existing approaches attempt to improve performance through context selection based on predefined context quality assessment metrics, they show limited gains over standard RAG. We attribute this limitation to their failure in holistically utilizing available information (query, context list, and generator) for comprehensive quality assessment. Inspired by recent advances in data selection, we reconceptualize context quality assessment as an inference-time data valuation problem and introduce the Contextual Influence Value (CI value). This novel metric quantifies context quality by measuring the performance degradation when removing each context from the list, effectively integrating query-aware relevance, list-aware uniqueness, and generator-aware alignment. Moreover, CI value eliminates complex selection hyperparameter tuning by simply retaining contexts with positive CI values. To address practical challenges of label dependency and computational overhead, we develop a parameterized surrogate model for CI value prediction during inference. The model employs a hierarchical architecture that captures both local query-context relevance and global inter-context interactions, trained through oracle CI value supervision and end-to-end generator feedback. Extensive experiments across 8 NLP tasks and multiple LLMs demonstrate that our context selection method significantly outperforms state-of-the-art baselines, effectively filtering poor-quality contexts while preserving critical information. Code is available at this https URL.
• 摘要：检索增强的生成（RAG）通过基于外部知识的响应来解决大语言模型（LLM）幻觉，但其有效性因含有无关或嘈杂信息的质量质量较差而损害了其有效性。尽管现有的方法试图通过基于预定义的上下文质量评估指标来通过上下文选择来提高性能，但它们显示出比标准抹布有限的收益。我们将这种局限性归因于它们在整体上使用可用信息（查询，上下文列表和生成器）进行全面质量评估的失败。受数据选择最新进展的启发，我们将上下文质量评估重新概念化为推理时间数据评估问题，并引入上下文影响值（CI值）。这种新颖的度量标准通过测量从列表中删除每个上下文时的性能降低来量化上下文质量，从而有效整合了查询意识到的相关性，列表意识到的唯一性和发电机感知的对齐。此外，CI值通过简单地保留具有正CI值的上下文来消除复杂选择超参数调整。为了解决标签依赖性和计算开销的实际挑战，我们为推断期间CI值预测开发了一个参数化的替代模型。该模型采用了层次结构，该体系结构既捕获本地查询语言相关性和全局互相互作用，又通过Oracle CI值监督和端到端的生成器反馈进行了训练。跨8个NLP任务和多个LLM的广泛实验表明，我们的上下文选择方法显着胜过最先进的基线，有效地过滤了质量不佳的环境，同时保留关键信息。代码可在此HTTPS URL上找到。

Title: Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs

• Authors: Norman Paulsen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21361
• Pdf URL: https://arxiv.org/pdf/2509.21361
• Copy Paste: [[2509.21361]] Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs(https://arxiv.org/abs/2509.21361)
• Keywords: language model, llm, hallucination
• Abstract: Large language model (LLM) providers boast big numbers for maximum context window sizes. To test the real world use of context windows, we 1) define a concept of maximum effective context window, 2) formulate a testing method of a context window's effectiveness over various sizes and problem types, and 3) create a standardized way to compare model efficacy for increasingly larger context window sizes to find the point of failure. We collected hundreds of thousands of data points across several models and found significant differences between reported Maximum Context Window (MCW) size and Maximum Effective Context Window (MECW) size. Our findings show that the MECW is, not only, drastically different from the MCW but also shifts based on the problem type. A few top of the line models in our test group failed with as little as 100 tokens in context; most had severe degradation in accuracy by 1000 tokens in context. All models fell far short of their Maximum Context Window by as much as 99 percent. Our data reveals the Maximum Effective Context Window shifts based on the type of problem provided, offering clear and actionable insights into how to improve model accuracy and decrease model hallucination rates.
• 摘要：大型语言模型（LLM）提供商拥有最大上下文窗口大小的大数字。为了测试上下文窗口的现实世界使用，我们1）定义一个最大有效上下文窗口的概念，2）制定上下文窗口对各种尺寸和问题类型的有效性的测试方法，以及3）创建一种标准化方法，比较越来越大的上下文窗口尺寸的模型功效，以找到失败点。我们在几个模型上收集了数十万个数据点，发现报告的最大上下文窗口（MCW）大小和最大有效上下文窗口（MECW）大小之间存在显着差异。我们的发现表明，MECW不仅与MCW截然不同，而且还基于问题类型而变化。我们的测试组中的一些线模型在上下文中失败了100个令牌。在上下文中，大多数人的准确性严重降解也会被1000个令牌降解。所有模型都远远远远超过其最大上下文窗口的99％。我们的数据揭示了根据所提供的问题类型揭示最大有效上下文窗口的变化，从而为如何提高模型准确性和降低模型幻觉率提供了清晰可行的见解。

Title: How Large Language Models Need Symbolism

• Authors: Xiaotie Deng, Hanyu Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21404
• Pdf URL: https://arxiv.org/pdf/2509.21404
• Copy Paste: [[2509.21404]] How Large Language Models Need Symbolism(https://arxiv.org/abs/2509.21404)
• Keywords: language model
• Abstract: We argue that AI's future requires more than scaling. To unlock genuine discovery, large language models need a compass: human-crafted symbols to guide their powerful but blind intuition.
• 摘要：我们认为，AI的未来不仅需要扩展。为了解锁真正的发现，大型语言模型需要一个指南针：人力制作的符号来指导其强大但盲目的直觉。

Title: One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning

• Authors: Sualeha Farid, Jayden Lin, Zean Chen, Shivani Kumar, David Jurgens
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21443
• Pdf URL: https://arxiv.org/pdf/2509.21443
• Copy Paste: [[2509.21443]] One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning(https://arxiv.org/abs/2509.21443)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are increasingly deployed in multilingual and multicultural environments where moral reasoning is essential for generating ethically appropriate responses. Yet, the dominant pretraining of LLMs on English-language data raises critical concerns about their ability to generalize judgments across diverse linguistic and cultural contexts. In this work, we systematically investigate how language mediates moral decision-making in LLMs. We translate two established moral reasoning benchmarks into five culturally and typologically diverse languages, enabling multilingual zero-shot evaluation. Our analysis reveals significant inconsistencies in LLMs' moral judgments across languages, often reflecting cultural misalignment. Through a combination of carefully constructed research questions, we uncover the underlying drivers of these disparities, ranging from disagreements to reasoning strategies employed by LLMs. Finally, through a case study, we link the role of pretraining data in shaping an LLM's moral compass. Through this work, we distill our insights into a structured typology of moral reasoning errors that calls for more culturally-aware AI.
• 摘要：大型语言模型（LLM）越来越多地部署在多语言和多文化环境中，在道德上，道德推理对于产生道德上适当的回应至关重要。然而，在英语数据上，LLM的主要审计训练引起了人们对它们在各种语言和文化背景之间概括判断的能力的关键问题。在这项工作中，我们系统地研究语言如何介导LLM中的道德决策。我们将两个既定的道德推理基准转化为五种文化和类型上多样的语言，从而实现了多种语言的零摄影评估。我们的分析揭示了LLMS跨语言的道德判断的明显不一致，通常反映了文化错位。通过精心构建的研究问题的结合，我们发现了这些差异的基本驱动因素，从分歧到LLMS采用的推理策略。最后，通过案例研究，我们将预处理数据在塑造LLM的道德指南针中的作用联系起来。通过这项工作，我们将洞察力提炼成道德推理错误的结构化类型，该错误需要更具文化意识的AI。

Title: LLM-Based Support for Diabetes Diagnosis: Opportunities, Scenarios, and Challenges with GPT-5

• Authors: Gaurav Kumar Gupta, Nirajan Acharya, Pranal Pande
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21450
• Pdf URL: https://arxiv.org/pdf/2509.21450
• Copy Paste: [[2509.21450]] LLM-Based Support for Diabetes Diagnosis: Opportunities, Scenarios, and Challenges with GPT-5(https://arxiv.org/abs/2509.21450)
• Keywords: language model, gpt, llm
• Abstract: Diabetes mellitus is a major global health challenge, affecting over half a billion adults worldwide with prevalence projected to rise. Although the American Diabetes Association (ADA) provides clear diagnostic thresholds, early recognition remains difficult due to vague symptoms, borderline laboratory values, gestational complexity, and the demands of long-term monitoring. Advances in large language models (LLMs) offer opportunities to enhance decision support through structured, interpretable, and patient-friendly outputs. This study evaluates GPT-5, the latest generative pre-trained transformer, using a simulation framework built entirely on synthetic cases aligned with ADA Standards of Care 2025 and inspired by public datasets including NHANES, Pima Indians, EyePACS, and MIMIC-IV. Five representative scenarios were tested: symptom recognition, laboratory interpretation, gestational diabetes screening, remote monitoring, and multimodal complication detection. For each, GPT-5 classified cases, generated clinical rationales, produced patient explanations, and output structured JSON summaries. Results showed strong alignment with ADA-defined criteria, suggesting GPT-5 may function as a dual-purpose tool for clinicians and patients, while underscoring the importance of reproducible evaluation frameworks for responsibly assessing LLMs in healthcare.
• 摘要：糖尿病是全球重大的健康挑战，影响了全球超过十亿成年人，预计患病率会上升。尽管美国糖尿病协会（ADA）提供了明确的诊断阈值，但由于模糊的症状，边界实验室价值，妊娠复杂性以及长期监测的需求，早期识别仍然很困难。大型语言模型（LLMS）的进步提供了通过结构化，可解释和患者友好的产出来增强决策支持的机会。这项研究评估了GPT-5，这是最新的生成预训练的变压器，它使用完全基于与ADA Care 2025的合成案例建立的仿真框架，并受到包括NHANES，PIMA Intians，Eyepacs和Mimic-IV在内的公共数据集的启发。测试了五种代表性的情况：症状识别，实验室解释，妊娠糖尿病筛查，远程监测和多模式并发症检测。对于每种，GPT-5分类病例，产生的临床原理，产生了患者解释和输出结构化的JSON摘要。结果表明，与ADA定义的标准有很强的一致性，这表明GPT-5可能是临床医生和患者的双重用途工具，同时强调了可重复评估框架以负责任地评估医疗保健中LLM的重要性。

Title: Diagnosing the Performance Trade-off in Moral Alignment: A Case Study on Gender Stereotypes

• Authors: Guangliang Liu, Bocheng Chen, Xitong Zhang, Kristen Marie Johnson
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21456
• Pdf URL: https://arxiv.org/pdf/2509.21456
• Copy Paste: [[2509.21456]] Diagnosing the Performance Trade-off in Moral Alignment: A Case Study on Gender Stereotypes(https://arxiv.org/abs/2509.21456)
• Keywords: language model
• Abstract: Moral alignment has emerged as a widely adopted approach for regulating the behavior of pretrained language models (PLMs), typically through fine-tuning or model editing on curated datasets. However, this process often comes at the cost of degraded downstream task performance. Prior studies commonly aim to achieve a performance trade-off by encouraging PLMs to selectively forget stereotypical knowledge through carefully designed fairness objectives, while preserving their helpfulness. In this short paper, we investigate the underlying mechanisms of the performance trade-off in the context of mitigating gender stereotypes, through the lens of forgetting and the fairness objective. Our analysis reveals the limitations of current fairness objective in achieving trade-off by demonstrating that: (1) downstream task performance is primarily driven by the overall forgetting level; (2) selective forgetting of stereotypes tends to increase overall forgetting; and (3) general solutions for mitigating forgetting are ineffective at reducing overall forgetting and fail to improve downstream task performance.
• 摘要：道德一致性已成为一种广泛采用的方法，用于调节预审计的语言模型（PLM）的行为，通常是通过策划数据集上的微调或模型编辑。但是，这个过程通常以下游任务性能退化为代价。先前的研究通常旨在通过鼓励PLM通过精心设计的公平目标来选择性地忘记刻板知识，同时保持其帮助，从而实现绩效权衡。在这篇简短的论文中，我们通过遗忘和公平目标的镜头调查了在减轻性别刻板印象的背景下，在减轻性别刻板印象的背景下研究了绩效权衡的基本机制。我们的分析揭示了当前公平目标在实现折衷方面的局限性：（1）下游任务绩效主要由整体遗忘水平驱动； （2）选择性忘记刻板印象往往会增加整体遗忘； （3）用于减轻遗忘的一般解决方案无效地减少整体遗忘和无法改善下游任务绩效。

Title: A State-of-the-Art SQL Reasoning Model using RLVR

• Authors: Alnur Ali, Ashutosh Baheti, Jonathan Chang, Ta-Chung Chi, Brandon Cui, Andrew Drozdov, Jonathan Frankle, Abhay Gupta, Pallavi Koppol, Sean Kulinski, Jonathan Li, Dipendra Misra, Krista Opsahl-Ong, Jose Javier Gonzalez Ortiz, Matei Zaharia, Yue Zhang
• Subjects: cs.CL, cs.AI, cs.DB, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21459
• Pdf URL: https://arxiv.org/pdf/2509.21459
• Copy Paste: [[2509.21459]] A State-of-the-Art SQL Reasoning Model using RLVR(https://arxiv.org/abs/2509.21459)
• Keywords: prompt, agent
• Abstract: Developing custom reasoning models via Reinforcement Learning (RL) that can incorporate organization-specific knowledge has great potential to address problems faced by enterprise customers. In many of these problems, the reward function is verifiable, a setting termed RL with Verifiable Rewards (RLVR). We apply RLVR to a popular data science benchmark called BIRD that measures the ability of an AI agent to convert a natural language query for a database to SQL executions. We apply a simple and general-purpose training recipe involving careful prompt and model selection, a warm-up stage using our offline RL approach called TAO, followed by rigorous online RLVR training. With no additional training data beyond the BIRD training set and no use of proprietary models, our very first submission to the BIRD leaderboard reached state-of-the-art accuracy on the private test set: 73.56% without self-consistency and 75.68% with self-consistency. In the latter case, our model also required fewer generations than the second-best approach. While BIRD is only a proxy task, the simplicity of our framework makes it broadly applicable to enterprise domains such as business intelligence, data science, and coding.
• 摘要：通过加强学习（RL）开发自定义推理模型，可以合并特定于组织的知识，具有解决企业客户面临的问题的巨大潜力。在许多问题中，奖励函数都是可验证的，该设置称为具有可验证奖励（RLVR）的RL。我们将RLVR应用于称为Bird的流行数据科学基准，该基准测量了AI代理将数据库自然语言查询转换为SQL执行的能力。我们应用了一个简单而通用的培训配方，涉及仔细提示和模型选择，这是使用我们的离线RL方法称为TAO的热身阶段，然后进行了严格的在线RLVR培训。除了鸟类训练外，没有其他训练数据，也没有使用专有模型，我们对鸟类排行榜的首次提交在私人测试集上达到了最先进的准确性：73.56％而没有自我持续性，而75.68％的人具有自愿性。在后一种情况下，我们的模型也需要少于第二好的方法。尽管Bird只是一项代理任务，但我们框架的简单性使其广泛适用于诸如商业智能，数据科学和编码之类的企业领域。

Title: Learning to Reason with Mixture of Tokens

• Authors: Adit Jain, Brendan Rappazzo
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21482
• Pdf URL: https://arxiv.org/pdf/2509.21482
• Copy Paste: [[2509.21482]] Learning to Reason with Mixture of Tokens(https://arxiv.org/abs/2509.21482)
• Keywords: language model, llm, chain-of-thought
• Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a leading approach for improving large language model (LLM) reasoning capabilities. Most current methods follow variants of Group Relative Policy Optimization, which samples multiple reasoning completions, scores them relative to each other, and adjusts the policy accordingly. However, these approaches invariably sample discrete tokens at each reasoning step, discarding the rich distributional information in the model's probability distribution over candidate tokens. While preserving and utilizing this distributional information has proven beneficial in non-RL settings, current RLVR methods seem to be unnecessarily constraining the reasoning search space by not using this information. To address this limitation, we investigate mixture-of-token generation (MoT-G) in RLVR. We present a unified framework that generalizes existing MoT-G approaches, including existing training-free methods that construct mixture embeddings as weighted sums over token embeddings, and extend RLVR to operate directly in this continuous mixture space for generating chain-of-thought. Evaluating two MoT-G variants on Reasoning-Gym, a suite of reasoning-intensive language tasks, we find that MoT--G methods achieve substantial improvements (5--35 \% gains on 7 out of 10 tasks) compared to standard decoding with the Qwen2.5-1.5B model, while reaching comparable accuracy with half the number of trajectories, suggesting improved training efficiency. Through comprehensive hidden-state and token-level analyses, we provide evidence that MoT--G's benefits may stem from its ability to maintain higher hidden-state entropy throughout the reasoning process and promote exploration in token space.
• 摘要：具有可验证奖励（RLVR）的强化学习已成为改善大语模型（LLM）推理能力的领先方法。大多数当前方法遵循小组相对策略优化的变体，该变体对多个推理完成，相对于彼此进行评分，并相应地调整策略。但是，这些方法总是在每个推理步骤中示例离散令牌，从而丢弃了模型对候选令牌的概率分布中的丰富分布信息。尽管保留和利用这些分布信息已被证明在非RL设置中有益，但当前的RLVR方法似乎不必要地通过不使用此信息来限制推理搜索空间。为了解决这一限制，我们研究了RLVR中的混合物（MOT-G）。我们提出了一个统一的框架，该框架概括了现有的MOT-G方法，包括现有的无训练方法，这些方法将混合物嵌入为加权总和上，并扩展RLVR以直接在这种连续的混合空间中运行，以生成思想链。与QWEN2.5-1.5B的标准解码相比，评估MOT-G方法的两个MOT-G变体，这是一系列推理密集型语言任务，这是一系列推理密集型语言任务（在10个任务中的7个任务中，有7个任务中的7个任务中的7个任务中的5--35 \％增长），同时可以与一半的准确准确的培训培训效率相比，提高了一半的准确性。通过全面的隐藏状态和令牌级别的分析，我们提供了证据，表明MOT-G的好处可能源于其在整个推理过程中保持更高的隐藏状态熵并促进令牌空间中的探索的能力。

Title: Dual-Head Reasoning Distillation: Improving Classifier Accuracy with Train-Time-Only Reasoning

• Authors: Jillian Xu, Dylan Zhou, Vinay Shukla, Yang Yang, Junrui Ruan, Shuhuai Lin, Wenfei Zou, Yinxiao Liu, Karthik Lakshmanan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21487
• Pdf URL: https://arxiv.org/pdf/2509.21487
• Copy Paste: [[2509.21487]] Dual-Head Reasoning Distillation: Improving Classifier Accuracy with Train-Time-Only Reasoning(https://arxiv.org/abs/2509.21487)
• Keywords: language model, prompt, chain-of-thought
• Abstract: Chain-of-Thought (CoT) prompting often improves classification accuracy, but it introduces a significant throughput penalty with rationale generation (Wei et al., 2022; Cheng and Van Durme, 2024). To resolve this trade-off, we introduce Dual-Head Reasoning Distillation (DHRD), a simple training method for decoder-only language models (LMs) that adds (i) a pooled classification head used during training and inference and (ii) a reasoning head supervised by teacher rationales used only in training. We train with a loss function that is a weighted sum of label cross-entropy and token-level LM loss over input-plus-rationale sequences. On seven SuperGLUE tasks, DHRD yields relative gains of 0.65-5.47% over pooled baselines, with notably larger gains on entailment/causal tasks. Since we disable the reasoning head at test time, inference throughput matches pooled classifiers and exceeds CoT decoding on the same backbones by 96-142 times in QPS.
• 摘要：经过思考链（COT）的提示通常会提高分类精度，但它引入了基本原理产生的重大吞吐量（Wei等，2022； Cheng和Van Durme，2024）。为了解决这一权衡，我们引入了双头推理蒸馏（DHRD），这是一种仅解码器语言模型（LMS）的简单培训方法，添加了（i）在培训和推理过程中使用的汇总分类头以及（ii）仅在培训中使用的教师理性的推理负责人。我们具有损失函数的训练，该损失函数是标签跨熵和代币级别的LM损失的加权总和。在七个超级劳动任务中，DHRD比合并基线的相对增长率为0.65-5.47％，在累积/因果任务上的增长幅度明显更大。由于我们在测试时间禁用推理头，因此推理吞吐量匹配汇总分类器，并在QPS中超过96-142次在同一骨架上解码的COT。

Title: On Code-Induced Reasoning in LLMs

• Authors: Abdul Waheed, Zhen Wu, Carolyn Rosé, Daphne Ippolito
• Subjects: cs.CL, cs.PL
• Abstract URL: https://arxiv.org/abs/2509.21499
• Pdf URL: https://arxiv.org/pdf/2509.21499
• Copy Paste: [[2509.21499]] On Code-Induced Reasoning in LLMs(https://arxiv.org/abs/2509.21499)
• Keywords: language model, llm
• Abstract: Code data has been shown to enhance the reasoning capabilities of large language models (LLMs), but it remains unclear which aspects of code are most responsible. We investigate this question with a systematic, data-centric framework. We construct parallel instruction datasets in ten programming languages and apply controlled perturbations that selectively disrupt structural or semantic properties of code. We then finetune LLMs from five model families and eight scales on each variant and evaluate their performance on natural language, math, and code tasks. Across 3,331 experiments, our results show that LLMs are more vulnerable to structural perturbations than semantic ones, particularly on math and code tasks. Appropriate abstractions like pseudocode and flowcharts can be as effective as code, while encoding the same information with fewer tokens without adhering to original syntax can often retain or even improve performance. Remarkably, even corrupted code with misleading signals remains competitive when surface-level regularities persist. Finally, syntactic styles also shape task-specific gains with Python favoring natural language reasoning and lower-level languages such as Java and Rust favoring math. Through our systematic framework, we aim to provide insight into how different properties of code influence reasoning and inform the design of training data for enhancing LLM reasoning capabilities.
• 摘要：已显示代码数据可以增强大语言模型（LLMS）的推理能力，但尚不清楚代码的哪些方面最负责。我们使用系统的，以数据为中心的框架调查了这个问题。我们用十种编程语言构建并行指令数据集，并应用受控的扰动，以选择性破坏代码的结构或语义属性。然后，我们从五个模型系列和每个变体中的八个量表中进行了Finetune llms，并评估其在自然语言，数学和代码任务上的性能。在3,331个实验中，我们的结果表明，LLM比语义扰动更容易受到结构扰动的影响，尤其是在数学和代码任务上。诸如伪代码和流程图之类的适当抽象可以像代码一样有效，同时使用较少的令牌编码相同的信息而不遵守原始语法通常可以保留甚至可以提高性能。值得注意的是，即使表面规律持续存在，甚至具有误导性信号的损坏的代码仍然具有竞争力。最后，句法样式还塑造了特定于任务的收益，其中有利于自然语言推理和低级语言，例如Java和Rust偏爱数学。通过我们的系统框架，我们旨在深入了解代码的不同属性如何影响推理，并为培训数据设计以增强LLM推理能力。

Title: Agribot: agriculture-specific question answer system

• Authors: Naman Jain, Pranjali Jain, Pratik Kayal, Jayakrishna Sahit, Soham Pachpande, Jayesh Choudhari
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21535
• Pdf URL: https://arxiv.org/pdf/2509.21535
• Copy Paste: [[2509.21535]] Agribot: agriculture-specific question answer system(https://arxiv.org/abs/2509.21535)
• Keywords: chat
• Abstract: India is an agro-based economy and proper information about agricultural practices is the key to optimal agricultural growth and output. In order to answer the queries of the farmer, we have build an agricultural chatbot based on the dataset from Kisan Call Center. This system is robust enough to answer queries related to weather, market rates, plant protection and government schemes. This system is available 24* 7, can be accessed through any electronic device and the information is delivered with the ease of understanding. The system is based on a sentence embedding model which gives an accuracy of 56%. After eliminating synonyms and incorporating entity extraction, the accuracy jumps to 86%. With such a system, farmers can progress towards easier information about farming related practices and hence a better agricultural output. The job of the Call Center workforce would be made easier and the hard work of various such workers can be redirected to a better goal.
• 摘要：印度是一种基于农业的经济，有关农业实践的适当信息是最佳农业增长和产出的关键。为了回答农民的疑问，我们根据Kisan呼叫中心的数据集建立了一个农业聊天机器人。该系统足够强大，可以回答与天气，市场价格，植物保护和政府计划有关的查询。该系统可在24* 7中获得，可以通过任何电子设备访问，并且信息的易于理解。该系统基于一个句子嵌入模型，其精度为56％。消除同义词并结合实体提取后，精度会跳至86％。通过这样的系统，农民可以朝着有关与农业相关的实践的更轻松信息迈进，从而更好地提供农业产出。呼叫中心劳动力的工作将变得更加容易，并且可以将各种工人的辛勤工作重定向到一个更好的目标。

Title: Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attribution

• Authors: Yash Saxena, Raviteja Bommireddy, Ankur Padia, Manas Gaur
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21557
• Pdf URL: https://arxiv.org/pdf/2509.21557
• Copy Paste: [[2509.21557]] Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attribution(https://arxiv.org/abs/2509.21557)
• Keywords: language model, llm
• Abstract: Trustworthy Large Language Models (LLMs) must cite human-verifiable sources in high-stakes domains such as healthcare, law, academia, and finance, where even small errors can have severe consequences. Practitioners and researchers face a choice: let models generate citations during decoding, or let models draft answers first and then attach appropriate citations. To clarify this choice, we introduce two paradigms: Generation-Time Citation (G-Cite), which produces the answer and citations in one pass, and Post-hoc Citation (P-Cite), which adds or verifies citations after drafting. We conduct a comprehensive evaluation from zero-shot to advanced retrieval-augmented methods across four popular attribution datasets and provide evidence-based recommendations that weigh trade-offs across use cases. Our results show a consistent trade-off between coverage and citation correctness, with retrieval as the main driver of attribution quality in both paradigms. P-Cite methods achieve high coverage with competitive correctness and moderate latency, whereas G-Cite methods prioritize precision at the cost of coverage and speed. We recommend a retrieval-centric, P-Cite-first approach for high-stakes applications, reserving G-Cite for precision-critical settings such as strict claim verification. Our codes and human evaluation results are available at this https URL
• 摘要：值得信赖的大语言模型（LLM）必须在医疗保健，法律，学术界和金融等高风险领域中引用人类验证的来源，即使很小的错误也会带来严重的后果。从业人员和研究人员面对一个选择：让模型在解码过程中产生引用，或者让模型首先起草答案，然后附加适当的引用。为了澄清这一选择，我们介绍了两个范式：生成时间引用（G-cite），它在一次通过时产生答案和引用，并在事后引用（P-Cite）（p-cite），在起草后增加或验证引用。我们对四个流行归因数据集的高级检索方法进行了全面的评估，并提供了基于证据的建议，这些建议可以权衡跨用例的权衡权衡。我们的结果表明，覆盖范围和引文正确性之间的权衡是一致的，并且取回了两个范式归因质量的主要驱动力。 P-cite方法以竞争性正确和中等延迟获得高覆盖范围，而G-Cite方法则以覆盖和速度为代价确定精度。我们建议针对高风险应用程序采用以检索为中心的P-CITE-CITE-CITE-CITE-CORTERT方法，为严格的索赔验证（例如严格的索赔验证）保留G-Cite。我们的代码和人类评估结果可在此HTTPS URL上获得

Title: Vision Language Models Cannot Plan, but Can They Formalize?

• Authors: Muyu He, Yuxi Zheng, Yuchen Liu, Zijian An, Bill Cai, Jiani Huang, Lifeng Zhou, Feng Liu, Ziyang Li, Li Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21576
• Pdf URL: https://arxiv.org/pdf/2509.21576
• Copy Paste: [[2509.21576]] Vision Language Models Cannot Plan, but Can They Formalize?(https://arxiv.org/abs/2509.21576)
• Keywords: language model, llm, agent
• Abstract: The advancement of vision language models (VLMs) has empowered embodied agents to accomplish simple multimodal planning tasks, but not long-horizon ones requiring long sequences of actions. In text-only simulations, long-horizon planning has seen significant improvement brought by repositioning the role of LLMs. Instead of directly generating action sequences, LLMs translate the planning domain and problem into a formal planning language like the Planning Domain Definition Language (PDDL), which can call a formal solver to derive the plan in a verifiable manner. In multimodal environments, research on VLM-as-formalizer remains scarce, usually involving gross simplifications such as predefined object vocabulary or overly similar few-shot examples. In this work, we present a suite of five VLM-as-formalizer pipelines that tackle one-shot, open-vocabulary, and multimodal PDDL formalization. We evaluate those on an existing benchmark while presenting another two that for the first time account for planning with authentic, multi-view, and low-quality images. We conclude that VLM-as-formalizer greatly outperforms end-to-end plan generation. We reveal the bottleneck to be vision rather than language, as VLMs often fail to capture an exhaustive set of necessary object relations. While generating intermediate, textual representations such as captions or scene graphs partially compensate for the performance, their inconsistent gain leaves headroom for future research directions on multimodal planning formalization.
• 摘要：视觉语言模型（VLM）的进步已授权具有体现的代理人完成简单的多模式计划任务，但不需要长的胜利者需要长时间的操作。在纯文本模拟中，长途计划通过重新定位LLM的作用带来了重大改善。 LLM并没有直接生成动作序列，而是将计划域和问题转化为诸如计划域定义语言（PDDL）之类的正式计划语言，该语言可以调用正式求解器以可验证的方式得出计划。在多模式环境中，对VLM-As-As-formalizer的研究仍然稀缺，通常涉及总体简化，例如预定义的对象词汇或过度相似的少数示例。在这项工作中，我们提供了一套由五个VLM-As-As-formalizer管道，它们可以解决一声，开放量和多模式PDDL形式化。我们在现有基准测试的同时评估了这些评估，同时提出了另外两个，这是首次使用真实，多视图和低质量图像进行计划的帐户。我们得出的结论是，VLM-As-As-As-As-As-As-As-As-As-As-As-As-As-formalizer的端到端计划生成的表现极大。我们揭示瓶颈是视觉而不是语言，因为VLMS通常无法捕获一组必要的对象关系。在产生中间的文本表示（例如字幕或场景图）的同时，部分补偿了性能，但它们不一致的收益为关于多模式计划形式化的未来研究方向留下了余地。

Title: "Be My Cheese?": Assessing Cultural Nuance in Multilingual LLM Translations

• Authors: Madison Van Doren, Cory Holland
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21577
• Pdf URL: https://arxiv.org/pdf/2509.21577
• Copy Paste: [[2509.21577]] "Be My Cheese?": Assessing Cultural Nuance in Multilingual LLM Translations(https://arxiv.org/abs/2509.21577)
• Keywords: llm
• Abstract: This pilot study explores the localisation capabilities of state-of-the-art multilingual AI models when translating figurative language, such as idioms and puns, from English into a diverse range of global languages. It expands on existing LLM translation research and industry benchmarks, which emphasise grammatical accuracy and token-level correctness, by focusing on cultural appropriateness and overall localisation quality - critical factors for real-world applications like marketing and e-commerce. To investigate these challenges, this project evaluated a sample of 87 LLM-generated translations of e-commerce marketing emails across 24 regional dialects of 20 languages. Human reviewers fluent in each target language provided quantitative ratings and qualitative feedback on faithfulness to the original's tone, meaning, and intended audience. Findings suggest that, while leading models generally produce grammatically correct translations, culturally nuanced language remains a clear area for improvement, often requiring substantial human refinement. Notably, even high-resource global languages, despite topping industry benchmark leaderboards, frequently mistranslated figurative expressions and wordplay. This work challenges the assumption that data volume is the most reliable predictor of machine translation quality and introduces cultural appropriateness as a key determinant of multilingual LLM performance - an area currently underexplored in existing academic and industry benchmarks. As a proof of concept, this pilot highlights limitations of current multilingual AI systems for real-world localisation use cases. Results of this pilot support the opportunity for expanded research at greater scale to deliver generalisable insights and inform deployment of reliable machine translation workflows in culturally diverse contexts.
• 摘要：这项试点研究探讨了最先进的多语言AI模型的本地化功能，例如，将象征性语言（例如成语和双关语）从英语翻译成各种各样的全球语言。它扩展了现有的LLM翻译研究和行业基准，这些基准强调语法准确性和令牌级别的正确性，它通过专注于文化适当性和整体本地化质量 - 对现实世界应用和电子商务等现实世界应用的关键因素。为了调查这些挑战，该项目评估了在24种20种语言的区域方言中的87个LLM生成的电子商务营销电子邮件翻译样本。每个目标语言的人类审阅者都提供了定量评级和定性反馈，内容涉及原始语调，含义和预期受众的忠诚。调查结果表明，尽管领先的模型通常会产生语法正确的翻译，但文化上细微的语言仍然是一个明确的改进领域，通常需要大量的人类精致。值得注意的是，即使是高资源的全球语言，尽管行业基准排行榜顶峰，但经常误译了象征性的表情和文字游戏。这项工作挑战了以下假设：数据量是机器翻译质量的最可靠预测指标，并将文化适当性作为多语言LLM绩效的关键决定因素 - 该领域目前在现有的学术和行业基准中尚未忽视。作为概念的证明，该试验强调了当前用于现实世界本地化用例的多语言AI系统的局限性。该飞行员的结果支持扩大更大规模研究的机会，以提供可普遍的见解，并为在文化上不同的环境中提供可靠的机器翻译工作流程的部署。

Title: Multi-Objective Reinforcement Learning for Large Language Model Optimization: Visionary Perspective

• Authors: Lingxiao Kong, Cong Yang, Oya Deniz Beyan, Zeyd Boukhers
• Subjects: cs.CL, cs.AI, cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2509.21613
• Pdf URL: https://arxiv.org/pdf/2509.21613
• Copy Paste: [[2509.21613]] Multi-Objective Reinforcement Learning for Large Language Model Optimization: Visionary Perspective(https://arxiv.org/abs/2509.21613)
• Keywords: language model, llm
• Abstract: Multi-Objective Reinforcement Learning (MORL) presents significant challenges and opportunities for optimizing multiple objectives in Large Language Models (LLMs). We introduce a MORL taxonomy and examine the advantages and limitations of various MORL methods when applied to LLM optimization, identifying the need for efficient and flexible approaches that accommodate personalization functionality and inherent complexities in LLMs and RL. We propose a vision for a MORL benchmarking framework that addresses the effects of different methods on diverse objective relationships. As future research directions, we focus on meta-policy MORL development that can improve efficiency and flexibility through its bi-level learning paradigm, highlighting key research questions and potential solutions for improving LLM performance.
• 摘要：多目标增强学习（MORL）提出了在大型语言模型（LLMS）中优化多个目标的重大挑战和机会。我们引入了MORL分类法，并在应用于LLM优化的各种MORL方法的优势和局限性，确定了适应LLMS和RL中个性化功能和固有复杂性的有效和灵活方法的需求。我们提出了MORL基准测试框架的愿景，该框架解决了不同方法对各种客观关系的影响。作为未来的研究方向，我们专注于元式MORL开发，可以通过其双层学习范式来提高效率和灵活性，从而突出了关键的研究问题和潜在的解决方案，以改善LLM的性能。

Title: OjaKV: Context-Aware Online Low-Rank KV Cache Compression with Oja's Rule

• Authors: Yuxuan Zhu, David H. Yang, Mohammad Mohammadi Amiri, Keerthiram Murugesan, Tejaswini Pedapati, Pin-Yu Chen
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21623
• Pdf URL: https://arxiv.org/pdf/2509.21623
• Copy Paste: [[2509.21623]] OjaKV: Context-Aware Online Low-Rank KV Cache Compression with Oja's Rule(https://arxiv.org/abs/2509.21623)
• Keywords: language model, prompt
• Abstract: The expanding long-context capabilities of large language models are constrained by a significant memory bottleneck: the key-value (KV) cache required for autoregressive generation. This bottleneck is substantial; for instance, a Llama-3.1-8B model processing a 32K-token prompt at a batch size of 4 requires approximately 16GB for its KV cache, a size exceeding the model's weights. While KV-cache compression via low-rank projection is a promising direction, existing methods rely on a static, offline-learned subspace that performs poorly under data distribution shifts. To overcome these limitations, we introduce OjaKV, a novel framework that integrates a strategic hybrid storage policy with online subspace adaptation. First, OjaKV recognizes that not all tokens are equally important for compression; it preserves the crucial first and most recent tokens in full-rank, maintaining high-fidelity anchors for attention. Second, for the vast majority of intermediate tokens, it applies low-rank compression by incrementally adapting the projection basis using Oja's algorithm for online principal component analysis. This adaptation involves a comprehensive update during prompt prefilling and lightweight periodic updates during decoding, ensuring the subspace remains aligned with the evolving context. Crucially, our framework is fully compatible with modern attention modules like FlashAttention. Experiments demonstrate that OjaKV maintains or even improves zero-shot accuracy at high compression ratios. In particular, OjaKV achieves its strongest gains on very long-context benchmarks that require complex reasoning, highlighting the importance of online subspace adaptation in dynamically tracking context shifts. These results establish our hybrid framework as a practical, plug-and-play solution for memory-efficient long-context inference without requiring model fine-tuning.
• 摘要：大型语言模型的扩展长篇小说能力受到重要的内存瓶颈的约束：自动回归产生所需的键值（KV）缓存。这个瓶颈很重要。例如，Llama-3.1-8B模型处理32K token的提示，其批次大小为4，其KV高速缓存大约需要16GB，大小超过了模型的权重。虽然通过低级别投影的KV-CACHE压缩是一个有希望的方向，但现有方法依赖于在数据分布变化下静态的，离线学习的子空间的性能较差。为了克服这些局限性，我们介绍了Ojakv，这是一个新颖的框架，将战略性混合存储政策与在线子空间适应相结合。首先，Ojakv认识到并非所有令牌对于压缩都同样重要。它保留了全级关键的第一个也是最近的代币，并保持了高保真锚点以供注意。其次，对于绝大多数中间令牌，它通过使用OJA的算法进行在线主体组件分析来逐步适应投影基础，以逐步适应投影基础。这种适应性涉及在解码过程中迅速预填充和轻巧的周期性更新期间的全面更新，以确保子空间与不断发展的环境保持一致。至关重要的是，我们的框架与诸如闪存的现代注意模块完全兼容。实验表明，在高压比下，Ojakv保持甚至提高了零照片的精度。尤其是，Ojakv在需要复杂推理的非常长的基准上实现了最大的收益，突出了在动态跟踪上下文转移中在线子空间适应的重要性。这些结果将我们的混合动力框架作为一种实用的，即插即用的解决方案，用于内存有效的长篇小说推断，而无需模型进行微调。

Title: Towards Transparent AI: A Survey on Explainable Language Models

• Authors: Avash Palikhe, Zichong Wang, Zhipeng Yin, Rui Guo, Qiang Duan, Jie Yang, Wenbin Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21631
• Pdf URL: https://arxiv.org/pdf/2509.21631
• Copy Paste: [[2509.21631]] Towards Transparent AI: A Survey on Explainable Language Models(https://arxiv.org/abs/2509.21631)
• Keywords: language model
• Abstract: Language Models (LMs) have significantly advanced natural language processing and enabled remarkable progress across diverse domains, yet their black-box nature raises critical concerns about the interpretability of their internal mechanisms and decision-making processes. This lack of transparency is particularly problematic for adoption in high-stakes domains, where stakeholders need to understand the rationale behind model outputs to ensure accountability. On the other hand, while explainable artificial intelligence (XAI) methods have been well studied for non-LMs, they face many limitations when applied to LMs due to their complex architectures, considerable training corpora, and broad generalization abilities. Although various surveys have examined XAI in the context of LMs, they often fail to capture the distinct challenges arising from the architectural diversity and evolving capabilities of these models. To bridge this gap, this survey presents a comprehensive review of XAI techniques with a particular emphasis on LMs, organizing them according to their underlying transformer architectures: encoder-only, decoder-only, and encoder-decoder, and analyzing how methods are adapted to each while assessing their respective strengths and limitations. Furthermore, we evaluate these techniques through the dual lenses of plausibility and faithfulness, offering a structured perspective on their effectiveness. Finally, we identify open research challenges and outline promising future directions, aiming to guide ongoing efforts toward the development of robust, transparent, and interpretable XAI methods for LMs.
• 摘要：语言模型（LMS）具有显着高级的自然语言处理，并在各种领域都取得了显着的进步，但是他们的黑盒本质引起了人们对其内部机制和决策过程的解释性的关键关注。缺乏透明度对于在高风险领域中采用的尤其有问题，在这些领域中，利益相关者需要了解模型输出背后的基本原理以确保问责制。另一方面，尽管对非LMS的可解释人工智能（XAI）方法进行了很好的研究，但由于其复杂的架构，相当多的培训语料库和广泛的概括能力，它们在LMS应用于LMS时面临许多局限性。尽管各种调查在LMS的背景下检查了XAI，但它们通常未能捕获这些模型的建筑多样性和不断发展的能力所带来的独特挑战。为了弥合这一差距，这项调查对XAI技术进行了全面的审查，并特别强调LMS，根据其潜在的变压器体系结构组织：仅编码，仅解码器，而编码器编码器，并分析如何对每个方法进行调整，同时评估其相应的强度和局限性。此外，我们通过合理性和忠诚的双镜头评估这些技术，从而为它们的有效性提供了结构化的观点。最后，我们确定了开放的研究挑战，并概述了有希望的未来方向，旨在指导为LMS开发强大，透明和可解释的XAI方法的持续努力。

Title: ReviewScore: Misinformed Peer Review Detection with Large Language Models

• Authors: Hyun Ryu, Doohyuk Jang, Hyemin S. Lee, Joonhyun Jeong, Gyeongman Kim, Donghyeon Cho, Gyouk Chu, Minyeong Hwang, Hyeongwon Jang, Changhun Kim, Haechan Kim, Jina Kim, Joowon Kim, Yoonjeon Kim, Kwanhyung Lee, Chanjae Park, Heecheol Yun, Gregor Betz, Eunho Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21679
• Pdf URL: https://arxiv.org/pdf/2509.21679
• Copy Paste: [[2509.21679]] ReviewScore: Misinformed Peer Review Detection with Large Language Models(https://arxiv.org/abs/2509.21679)
• Keywords: language model, llm
• Abstract: Peer review serves as a backbone of academic research, but in most AI conferences, the review quality is degrading as the number of submissions explodes. To reliably detect low-quality reviews, we define misinformed review points as either "weaknesses" in a review that contain incorrect premises, or "questions" in a review that can be already answered by the paper. We verify that 15.2% of weaknesses and 26.4% of questions are misinformed and introduce ReviewScore indicating if a review point is misinformed. To evaluate the factuality of each premise of weaknesses, we propose an automated engine that reconstructs every explicit and implicit premise from a weakness. We build a human expert-annotated ReviewScore dataset to check the ability of LLMs to automate ReviewScore evaluation. Then, we measure human-model agreements on ReviewScore using eight current state-of-the-art LLMs and verify moderate agreements. We also prove that evaluating premise-level factuality shows significantly higher agreements than evaluating weakness-level factuality. A thorough disagreement analysis further supports a potential of fully automated ReviewScore evaluation.
• 摘要：同行评审是学术研究的骨干，但是在大多数AI会议中，随着提交的数量爆炸，审查质量正在降低。为了可靠地检测到低质量的评论，我们将错误信息的审查点定义为包含不正确前提的评论中的“弱点”，或者在评论中可以通过论文回答的评论中的“问题”。我们验证是否有15.2％的弱点和26.4％的问题被误导，并介绍了审查，指示审查点是否误导了。为了评估弱点每个前提的事实，我们提出了一个自动化引擎，该引擎从弱点中重建每个明确和隐性的前提。我们构建了一个人类专家注册的评论Score数据集，以检查LLMS自动化评论评估的能力。然后，我们使用八个最新的LLMS衡量ReviewScore的人类模式协议，并验证中等协议。我们还证明，评估前提级别的事实比评估弱点级别的事实表现出明显更高的协议。彻底的分歧分析进一步支持了全自动评论评估的潜力。

Title: Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval

• Authors: Xiaojun Wu, Cehao Yang, Xueyuan Lin, Chengjin Xu, Xuhui Jiang, Yuanliang Sun, Hui Xiong, Jia Li, Jian Guo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21710
• Pdf URL: https://arxiv.org/pdf/2509.21710
• Copy Paste: [[2509.21710]] Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval(https://arxiv.org/abs/2509.21710)
• Keywords: language model, llm, retrieval-augmented generation, agent
• Abstract: Retrieval-Augmented Generation (RAG) and Graph-based RAG has become the important paradigm for enhancing Large Language Models (LLMs) with external knowledge. However, existing approaches face a fundamental trade-off. While graph-based methods are inherently dependent on high-quality graph structures, they face significant practical constraints: manually constructed knowledge graphs are prohibitively expensive to scale, while automatically extracted graphs from corpora are limited by the performance of the underlying LLM extractors, especially when using smaller, local-deployed models. This paper presents Think-on-Graph 3.0 (ToG-3), a novel framework that introduces Multi-Agent Context Evolution and Retrieval (MACER) mechanism to overcome these limitations. Our core innovation is the dynamic construction and refinement of a Chunk-Triplets-Community heterogeneous graph index, which pioneeringly incorporates a dual-evolution mechanism of Evolving Query and Evolving Sub-Graph for precise evidence retrieval. This approach addresses a critical limitation of prior Graph-based RAG methods, which typically construct a static graph index in a single pass without adapting to the actual query. A multi-agent system, comprising Constructor, Retriever, Reflector, and Responser agents, collaboratively engages in an iterative process of evidence retrieval, answer generation, sufficiency reflection, and, crucially, evolving query and subgraph. This dual-evolving multi-agent system allows ToG-3 to adaptively build a targeted graph index during reasoning, mitigating the inherent drawbacks of static, one-time graph construction and enabling deep, precise reasoning even with lightweight LLMs. Extensive experiments demonstrate that ToG-3 outperforms compared baselines on both deep and broad reasoning benchmarks, and ablation studies confirm the efficacy of the components of MACER framework.
• 摘要：检索增强的生成（RAG）和基于图的抹布已成为增强具有外部知识的大型语言模型（LLM）的重要范式。但是，现有的方法面临着基本的权衡。尽管基于图的方法固有地取决于高质量的图形结构，但它们面临着重要的实践约束：手动构造的知识图的扩展性昂贵，而从Corpora中自动提取的图形则受到基础LLM提取器的性能的限制，尤其是在使用较小的本地剥离模型时。本文介绍了思维3.0（TOG-3），这是一个新颖的框架，介绍了多代理上下文的演变和检索（MACER）机制，以克服这些局限性。我们的核心创新是块 - 三个 - 共同性异质图指数的动态构建和完善，该索引率先融合了不断发展的查询和不断发展的子图的双进化机制，以精确证据检索。该方法解决了先前基于图的抹布方法的关键局限性，该方法通常在单个通过中构造静态图索引而不适应实际查询。一个由构造函数，检索器，反射器和响应剂组成的多代理系统协作进行了迭代的证据检索过程，答案产生，充足反射，并且至关重要的是，不断发展的查询和子图。这个双重发展的多代理系统允许TOW-3在推理过程中自适应地构建目标图索引，从而减轻静态，一次性图形构造的固有缺点，并在轻巧的LLMS中启用深层，精确的推理。广泛的实验表明，TOG-3的表现优于深层和广泛推理基准的基准，而消融研究证实了Macer框架组件的功效。

Title: ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulation

• Authors: Jiho Kim, Junseong Choi, Woosog Chay, Daeun Kyung, Yeonsu Kwon, Yohan Jo, Edward Choi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21730
• Pdf URL: https://arxiv.org/pdf/2509.21730
• Copy Paste: [[2509.21730]] ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulation(https://arxiv.org/abs/2509.21730)
• Keywords: language model, llm, agent
• Abstract: As large language models (LLMs) become increasingly integrated into daily life, there is growing demand for AI assistants that are not only reactive but also proactive and personalized. While recent advances have pushed forward proactivity and personalization individually, their combination remains underexplored. To bridge this gap, we introduce ProPerSim, a new task and simulation framework for developing assistants capable of making timely, personalized recommendations in realistic home scenarios. In our simulation environment, a user agent with a rich persona interacts with the assistant, providing ratings on how well each suggestion aligns with its preferences and context. The assistant's goal is to use these ratings to learn and adapt to achieve higher scores over time. Built on ProPerSim, we propose ProPerAssistant, a retrieval-augmented, preference-aligned assistant that continually learns and adapts through user feedback. Experiments across 32 diverse personas show that ProPerAssistant adapts its strategy and steadily improves user satisfaction, highlighting the promise of uniting proactivity and personalization.
• 摘要：随着大型语言模型（LLMS）越来越多地融入日常生活中，对不仅反应性而且积极主动和个性化的AI助手的需求不断增长。尽管最近的进步已经分别推动了前进的积极性和个性化，但它们的组合仍然没有得到充实。为了弥合这一差距，我们介绍了ProperSim，这是一个新的任务和模拟框架，用于开发能够在现实的家庭场景中及时，个性化建议的助手。在我们的仿真环境中，具有丰富角色的用户代理与助手互动，对每个建议与其偏好和环境的一致程度都很好。助手的目标是使用这些评分来学习和适应以随着时间的推移获得更高的分数。基于ProperSim，我们提出了ProperAssistant，这是一种取回，偏爱的助手，它不断通过用户反馈来学习和适应。在32种不同角色的实验表明，适当的服用者适应其策略并稳步提高用户满意度，突出了团结起积极性和个性化的希望。

Title: How Accurate Are LLMs at Multi-Question Answering on Conversational Transcripts?

• Authors: Xiliang Zhu, Shi Zong, David Rossouw
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21732
• Pdf URL: https://arxiv.org/pdf/2509.21732
• Copy Paste: [[2509.21732]] How Accurate Are LLMs at Multi-Question Answering on Conversational Transcripts?(https://arxiv.org/abs/2509.21732)
• Keywords: language model, gpt, llm
• Abstract: Deploying Large Language Models (LLMs) for question answering (QA) over lengthy contexts is a significant challenge. In industrial settings, this process is often hindered by high computational costs and latency, especially when multiple questions must be answered based on the same context. In this work, we explore the capabilities of LLMs to answer multiple questions based on the same conversational context. We conduct extensive experiments and benchmark a range of both proprietary and public models on this challenging task. Our findings highlight that while strong proprietary LLMs like GPT-4o achieve the best overall performance, fine-tuned public LLMs with up to 8 billion parameters can surpass GPT-4o in accuracy, which demonstrates their potential for transparent and cost-effective deployment in real-world applications.
• 摘要：在冗长的上下文中部署大型语言模型（LLMS）进行问答（QA）是一个重大挑战。在工业环境中，这一过程通常受到高计算成本和延迟的阻碍，尤其是当必须基于相同的上下文回答多个问题时。在这项工作中，我们探讨了LLMS基于相同的对话环境回答多个问题的功能。我们进行了广泛的实验，并在这项具有挑战性的任务上进行了一系列专有和公共模型。我们的发现强调，尽管像GPT-4O这样的强大专有LLM达到了最佳的整体性能，但具有多达80亿个参数的微调公共LLM可以超过GPT-4O的准确性，这表明了它们在现实应用应用程序中的透明和成本效益的潜力。

Title: Self-Speculative Biased Decoding for Faster Live Translation

• Authors: Linxiao Zeng, Haoyun Deng, Kangyuan Shu, Shizhen Wang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21740
• Pdf URL: https://arxiv.org/pdf/2509.21740
• Copy Paste: [[2509.21740]] Self-Speculative Biased Decoding for Faster Live Translation(https://arxiv.org/abs/2509.21740)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have recently demonstrated impressive capabilities in various text generation tasks. However, it remains challenging to use them off-the-shelf in streaming applications (such as live translation), where the output must continually update as the input context expands, while still maintaining a reasonable computational cost to meet the latency requirement. In this work, we reexamine the re-translation approach to simultaneous translation and propose Self-Speculative Biased Decoding, a novel inference paradigm designed to avoid repeatedly generating output from scratch for a consistently growing input stream. We propose using the most recent output as a draft for the current growing input context. During the verification stage, the output will be biased towards the draft token for a higher draft acceptance rate. This strategy not only minimizes flickering that might distract users but also leads to higher speedups. Conventional decoding may take charge from the point of divergence after draft verification and continue until the end condition is met. Unlike existing speculative decoding strategies, our approach eliminates the need for draft computations, making it a model-agnostic and plug-and-play solution for accelerating latency-sensitive streaming applications. Experimental results on simultaneous text-to-text re-translation demonstrate that our approach achieves up to 1.7x speedup compared to conventional auto-regressive re-translation without compromising quality. Additionally, it significantly reduces flickering by 80% by incorporating the display-only mask-k technique.
• 摘要：大型语言模型（LLMS）最近在各种文本生成任务中表现出了令人印象深刻的功能。但是，在流媒体应用程序（例如实时翻译）中使用它们的现成仍然是一项挑战，在该应用程序中，随着输入上下文的扩展，输出必须不断更新，同时仍保持合理的计算成本以满足延迟需求。在这项工作中，我们重新审查了同时翻译的重新翻译方法，并提出了自我指示的偏置解码，这是一种新颖的推理范式，旨在避免反复产生scratch的输出，以始终如一地增长输入流。我们建议将最新的输出作为当前增长输入环境的草稿。在验证阶段，以更高的草稿接受率，输出将偏向草稿令牌。这种策略不仅可以最大程度地减少可能分散用户注意力的闪烁，而且会导致更高的加速。常规解码可以从草稿验证后的分歧点起负责，并继续直至满足最终条件。与现有的投机解码策略不同，我们的方法消除了对草稿计算的需求，这使其成为一种模型不合时宜的和插件的解决方案，用于加速潜伏期敏感的流媒体应用程序。同时进行文本对文本重新翻译的实验结果表明，与传统的自动回归重新翻译相比，我们的方法达到了1.7倍的速度，而不会损害质量。此外，通过融合仅显示显示器-K技术，它可以将闪烁的闪烁显着降低80％。

Title: Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning in Large Audio-Language Models

• Authors: Zhen Xiong, Yujun Cai, Zhecheng Li, Junsong Yuan, Yiwei Wang
• Subjects: cs.CL, cs.SD
• Abstract URL: https://arxiv.org/abs/2509.21749
• Pdf URL: https://arxiv.org/pdf/2509.21749
• Copy Paste: [[2509.21749]] Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning in Large Audio-Language Models(https://arxiv.org/abs/2509.21749)
• Keywords: language model, chain-of-thought
• Abstract: Recent Large Audio-Language Models (LALMs) have shown strong performance on various audio understanding tasks such as speech translation and Audio Q\&A. However, they exhibit significant limitations on challenging audio reasoning tasks in complex acoustic scenarios. These situations would greatly benefit from the use of acoustic tools like noise suppression, source separation, and precise temporal alignment, but current LALMs lack access to such tools. To address this limitation, we introduce Thinking-with-Sound (TwS), a framework that equips LALMs with Audio CoT by combining linguistic reasoning with on-the-fly audio-domain analysis. Unlike existing approaches that treat audio as static input, TwS enables models to actively think with audio signals, performing numerical analysis and digital manipulation through multimodal reasoning. To evaluate this approach, we construct MELD-Hard1k, a new robustness benchmark created by introducing various acoustic perturbations. Experiments reveal that state-of-the-art LALMs suffer dramatic performance degradation on MELD-Hard1k, with accuracy dropping by more than $50\%$ compared to clean audio. TwS achieves substantial improvements in robustness, demonstrating both effectiveness and scalability: small models gain $24.73\%$ absolute accuracy, with improvements scaling consistently up to $36.61\%$ for larger models. Our findings demonstrate that Audio CoT can significantly enhance robustness without retraining, opening new directions for developing more robust audio understanding systems.
• 摘要：最近的大型音频语言模型（LALMS）在各种音频理解任务（例如语音翻译和音频Q \＆a）上表现出了很强的表现。但是，它们在复杂的声学场景中对挑战的音频推理任务显示出重大局限性。这些情况将受益于使用声学工具，例如抑制噪声，源分离和精确的时间对齐，但是当前的LALMS无法使用此类工具。为了解决此限制，我们介绍了与之相关的思维（TWS），该框架通过将语言推理与在现行的音频域分析相结合，从而使LALMS与音频COT相比。与将音频视为静态输入的现有方法不同，TWS使模型可以通过音频信号积极思考，通过多模式推理执行数值分析和数字操作。为了评估这种方法，我们构建了Meld-Hard1k，这是一种通过引入各种声学扰动而创建的新的鲁棒性基准。实验表明，最新的LALMS在MELD-HARD1K上遭受了巨大的性能降解，与清洁音频相比，准确性下降了50美元以上。 TWS在鲁棒性方面取得了重大改善，证明了有效性和可伸缩性：小型型号获得了$ 24.73 \％$绝对精度，而改进的扩展始终为较大型号的$ 36.61 \％$。我们的发现表明，音频婴儿床可以显着增强鲁棒性，而无需重新训练，开辟了新的方向，以开发更强大的音频理解系统。

Title: Navigating the Impact of Structured Output Format on Large Language Models through the Compass of Causal Inference

• Authors: Han Yuan, Yue Zhao, Li Zhang, Wuqiong Luo, Zheng Ma
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21791
• Pdf URL: https://arxiv.org/pdf/2509.21791
• Copy Paste: [[2509.21791]] Navigating the Impact of Structured Output Format on Large Language Models through the Compass of Causal Inference(https://arxiv.org/abs/2509.21791)
• Keywords: language model, gpt, llm
• Abstract: Structured output from large language models (LLMs) has enhanced efficiency in processing generated information and is increasingly adopted in industrial applications. Prior studies have investigated the impact of structured output on LLMs' generation quality, often presenting one-way findings. Some suggest that structured format enhances completeness and factual accuracy, while others argue that it restricts the reasoning capacity of LLMs and leads to reductions in standard evaluation metrics. Potential limitations of these assessments include restricted testing scenarios, weakly controlled comparative settings, and reliance on coarse metrics. In this work, we present a refined analysis using causal inference. Based on one assumed and two guaranteed constraints, we derive five potential causal structures characterizing the influence of structured output on LLMs' generation: (1) collider without m-bias, (2) collider with m-bias, (3) single cause from instruction, (4) single cause from output format, and (5) independence. Across seven public and one developed reasoning tasks, we find that coarse metrics report positive, negative, or neutral effects of structured output on GPT-4o's generation. However, causal inference reveals no causal impact in 43 out of 48 scenarios. In the remaining 5, 3 involve multifaceted causal structures influenced by concrete instructions.
• 摘要：大型语言模型（LLM）的结构化输出在处理生成的信息方面具有提高效率，并且在工业应用中越来越多地采用。先前的研究研究了结构化产量对LLMS发电质量的影响，通常会提出单向发现。有些人认为结构化格式提高了完整性和事实准确性，而另一些人则认为它限制了LLM的推理能力，并导致标准评估指标的降低。这些评估的潜在局限性包括受限的测试方案，弱控制的比较设置以及对粗指标的依赖。在这项工作中，我们提出了使用因果推断的精致分析。基于一个假定的和两个保证的约束，我们得出了五个潜在的因果结构，这些因果结构表征了结构化输出对LLMS生成的影响：（1）没有M偏置的对撞机，（2）具有M偏置的对撞机，（3）指导的单一原因，（4）（4）来自输出格式的单一原因，以及（5）独立。在七个公众和一项开发的推理任务中，我们发现粗制指标报告了结构化输出对GPT-4O代的积极，负或中性效应。然而，因果推断在48个情况中有43个没有因果影响。在其余的5中，3涉及由具体指令影响的多方面因果结构。

Title: Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment

• Authors: Hongbin Zhang, Kehai Chen, Xuefeng Bai, Yang Xiang, Min Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21798
• Pdf URL: https://arxiv.org/pdf/2509.21798
• Copy Paste: [[2509.21798]] Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment(https://arxiv.org/abs/2509.21798)
• Keywords: language model, llm
• Abstract: Reward models (RMs) are crucial for aligning large language models (LLMs) with diverse cultures. Consequently, evaluating their cultural awareness is essential for further advancing global alignment of LLMs. However, existing RM evaluations fall short in assessing cultural awareness due to the scarcity of culturally relevant evaluation datasets. To fill this gap, we propose Cultural Awareness Reward modeling Benchmark (CARB), covering 10 distinct cultures across 4 cultural domains. Our extensive evaluation of state-of-the-art RMs reveals their deficiencies in modeling cultural awareness and demonstrates a positive correlation between performance on CARB and downstream multilingual cultural alignment tasks. Further analysis identifies the spurious correlations within culture-aware reward modeling, wherein RM's scoring relies predominantly on surface-level features rather than authentic cultural nuance understanding. To address these, we propose Think-as-Locals to elicit deeper culturally grounded reasoning from generative RMs via reinforcement learning from verifiable rewards (RLVR) and employ well-designed rewards to ensure accurate preference judgments and high-quality structured evaluation criteria generation. Experimental results validate its efficacy in mitigating spurious features interference and advancing culture-aware reward modeling.
• 摘要：奖励模型（RMS）对于将大型语言模型（LLM）与不同文化保持一致至关重要。因此，评估其文化意识对于进一步推进LLM的全球一致性至关重要。但是，由于文化相关的评估数据集缺乏，现有的RM评估在评估文化意识方面缺乏。为了填补这一空白，我们提出了文化意识奖励建模基准（CARB），涵盖了4个文化领域的10种不同文化。我们对最先进的RMS的广泛评估揭示了它们在建模文化意识时的缺陷，并证明了碳水化合物和下游多语言文化一致性任务之间的性能之间存在正相关。进一步的分析确定了文化吸引人的奖励建模中的虚假相关性，其中RM的评分主要依赖于表面水平的特征，而不是真实的文化细微差别理解。为了解决这些问题，我们提出思想局部性，以通过从可验证的奖励（RLVR）学习从生成RMS（RLVR）学习，并采用精心设计的奖励来确保准确的偏好判断和高质量结构化的评估标准，从而引起了更深入的文化基础。实验结果证明了其在减轻虚假特征干扰和推进文化意识奖励建模方面的功效。

Title: Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies

• Authors: Qianen Zhang, Satoshi Nakamura
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21801
• Pdf URL: https://arxiv.org/pdf/2509.21801
• Copy Paste: [[2509.21801]] Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies(https://arxiv.org/abs/2509.21801)
• Keywords: language model, llm, prompt
• Abstract: Simultaneous Machine Translation (SiMT) requires high-quality translations under strict real-time constraints, which traditional encoder-decoder policies with only READ/WRITE actions cannot fully address. We extend the action space of SiMT with four adaptive actions: SENTENCE_CUT, DROP, PARTIAL_SUMMARIZATION and PRONOMINALIZATION, which enable real-time restructuring, omission, and simplification while preserving semantic fidelity. We implement these actions in a decoder-only large language model (LLM) framework and construct training references through action-aware prompting. To evaluate both quality and latency, we further develop a latency-aware TTS pipeline that maps textual outputs to speech with realistic timing. Experiments on the ACL60/60 English-Chinese and English-German benchmarks show that our framework consistently improves semantic metrics (e.g., COMET-KIWI) and achieves lower delay (measured by Average Lagging) compared to reference translations and salami-based baselines. Notably, combining DROP and SENTENCE_CUT yields the best overall balance between fluency and latency. These results demonstrate that enriching the action space of LLM-based SiMT provides a promising direction for bridging the gap between human and machine interpretation.
• 摘要：同时的机器翻译（SIMT）需要在严格的实时约束下进行高质量的翻译，而传统的编码器编码器策略只有读/写操作无法完全解决。我们通过四个自适应动作扩展了SIMT的动作空间：句子_cut，drop，partial_summarization和代词化，可以实时重组，遗漏和简化，同时保留语义忠诚度。我们以仅解码器的大型语言模型（LLM）框架实施这些动作，并通过动作感知提示来构建培训参考。为了评估质量和延迟，我们进一步开发了一种延迟感知的TTS管道，该管道将文本输出映射到具有现实时机的语音。 ACL60/60英语 - 中国和英国 - 德国基准的实验表明，与参考翻译和基于Salami的基线相比，我们的框架始终改善语义指标（例如Comet-KIWI），并实现较低的延迟（按平均滞后）。值得注意的是，滴滴和句子的结合可以在流利度和延迟之间产生最佳的总体平衡。这些结果表明，丰富基于LLM的SIMT的动作空间为弥合人与机器解释之间的差距提供了一个有希望的方向。

Title: Can LLMs Solve and Generate Linguistic Olympiad Puzzles?

• Authors: Neh Majmudar, Elena Filatova
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21820
• Pdf URL: https://arxiv.org/pdf/2509.21820
• Copy Paste: [[2509.21820]] Can LLMs Solve and Generate Linguistic Olympiad Puzzles?(https://arxiv.org/abs/2509.21820)
• Keywords: language model, llm
• Abstract: In this paper, we introduce a combination of novel and exciting tasks: the solution and generation of linguistic puzzles. We focus on puzzles used in Linguistic Olympiads for high school students. We first extend the existing benchmark for the task of solving linguistic puzzles. We explore the use of Large Language Models (LLMs), including recent state-of-the-art models such as OpenAI's o1, for solving linguistic puzzles, analyzing their performance across various linguistic topics. We demonstrate that LLMs outperform humans on most puzzles types, except for those centered on writing systems, and for the understudied languages. We use the insights from puzzle-solving experiments to direct the novel task of puzzle generation. We believe that automating puzzle generation, even for relatively simple puzzles, holds promise for expanding interest in linguistics and introducing the field to a broader audience. This finding highlights the importance of linguistic puzzle generation as a research task: such puzzles can not only promote linguistics but also support the dissemination of knowledge about rare and understudied languages.
• 摘要：在本文中，我们介绍了新颖而令人兴奋的任务的结合：语言难题的解决方案和产生。我们专注于高中生语言奥林匹克运动会中使用的难题。我们首先将现有基准扩展到解决语言难题的任务。我们探讨了大型语言模型（LLM）的使用，包括最近最新的模型，例如OpenAI的O1来解决语言难题，分析了它们在各种语言主题中的性能。我们证明，LLM在大多数难题类型上的表现都要优于人类，除了以写作系统和研究的语言为中心。我们使用来自拼图实验的见解来指导拼图生成的新任务。我们认为，即使对于相对简单的难题，自动化的拼图产生也有望扩大对语言学的兴趣并向更广泛的受众介绍该领域的兴趣。这一发现突出了语言拼图作为研究任务的重要性：这种难题不仅可以促进语言学，而且还支持传播有关稀有和研究的语言的知识。

Title: ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Models

• Authors: Zihan Lin, Xiaohan Wang, Jie Cao, Jiajun Chai, Guojun Yin, Wei Lin, Ran He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21826
• Pdf URL: https://arxiv.org/pdf/2509.21826
• Copy Paste: [[2509.21826]] ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Models(https://arxiv.org/abs/2509.21826)
• Keywords: language model, gpt, llm, agent
• Abstract: Large language models (LLMs) transcend passive generation and act as goal-directed agents by invoking external tools. Reinforcement learning (RL) offers a principled framework for optimizing these emergent tool-use policies, yet the prevailing paradigm relies exclusively on sparse outcome rewards and lacks consideration of the particularity of tool-use tasks, inflating policy-gradient variance and resulting in inefficient training. To better understand and address these challenges, we first establish a theoretical link between policy entropy and training stability of tool-use tasks, which reveals that structured, low-entropy tokens are primary determinants of rewards. Motivated by this insight, we propose \textbf{Res}haped \textbf{T}oken-level policy gradients (\textbf{ResT}) for tool-use tasks. ResT reshapes the policy gradient through entropy-informed token reweighting, progressively upweighting reasoning tokens as training proceeds. This entropy-aware scheme enables a smooth shift from structural correctness to semantic reasoning and stabilizes convergence in multi-turn tool-use tasks. Evaluation on BFCL and API-Bank shows that ResT achieves state-of-the-art results, outperforming prior methods by up to $8.76\%$. When fine-tuned on a 4B base LLM, ResT further surpasses GPT-4o by $4.11\%$ on single-turn tasks and $1.50\%$ on multi-turn base tasks.
• 摘要：大型语言模型（LLMS）通过调用外部工具来超越被动生成并充当目标定向的代理。强化学习（RL）提供了一个原则上的框架来优化这些紧急的工具使用政策，但是，当前的范式仅依赖于稀疏结果奖励，并且缺乏对工具使用任务的特殊性的考虑，从而使政策促进差异膨胀并导致效率低下的培训。为了更好地理解和应对这些挑战，我们首先建立了策略熵与工具使用任务的训练稳定性之间的理论联系，这表明结构化的低渗透令牌是奖励的主要决定因素。在此洞察力的激励下，我们建议\ textbf {res} haped \ textbf {t} oken-level策略渐变（\ textbf {rest}）用于工具使用任务。 REST通过熵的令牌重量重量重塑政策梯度，随着培训的收益，逐渐上升推理令牌。这种熵感知的方案使从结构正确性转向语义推理，并稳定多转弯工具使用任务中的收敛性。对BFCL和API-BANK的评估表明，REST可以实现最新的结果，优于先前的方法高达$ 8.76 \％$。当在4B基础LLM上进行微调时，REST在单转任务上进一步超过了GPT-4O $ 4.11 \％$，而多转弯基础任务的$ 1.50 \％$ $。

Title: Semantic Agreement Enables Efficient Open-Ended LLM Cascades

• Authors: Duncan Soiffer, Steven Kolawole, Virginia Smith
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21837
• Pdf URL: https://arxiv.org/pdf/2509.21837
• Copy Paste: [[2509.21837]] Semantic Agreement Enables Efficient Open-Ended LLM Cascades(https://arxiv.org/abs/2509.21837)
• Keywords: llm
• Abstract: Cascade systems route computational requests to smaller models when possible and defer to larger models only when necessary, offering a promising approach to balance cost and quality in LLM deployment. However, they face a fundamental challenge in open-ended text generation: determining output reliability when generation quality lies on a continuous spectrum, often with multiple valid responses. To address this, we propose semantic agreement -- meaning-level consensus between ensemble outputs -- as a training-free signal for reliable deferral. We show that when diverse model outputs agree semantically, their consensus is a stronger reliability signal than token-level confidence. Evaluated from 500M to 70B-parameter models, we find that semantic cascades match or surpass target-model quality at 40% of the cost and reduce latency by up to 60%. Our method requires no model internals, works across black-box APIs, and remains robust to model updates, making it a practical baseline for real-world LLM deployment.
• 摘要：级联系统在可能的情况下将计算请求路由到较小的型号，并仅在必要时才延迟到较大的模型，从而提供有前途的方法来平衡LLM部署的成本和质量。但是，他们在开放式文本生成中面临着一个根本的挑战：当发电质量在连续频谱上，通常具有多个有效响应时，确定输出可靠性。为了解决这个问题，我们提出了语义协议 - 集合输出之间的含义级别的共识 - 作为可靠延期的无训练信号。我们表明，当各种模型输出以语义同意时，它们的共识比令牌级别的信心更强。从500m到70B参数型号进行了评估，我们发现语义级联级别以40％的成本匹配或超过目标模型的质量，并将潜伏期降低高达60％。我们的方法不需要模型内部设备，可以跨Black-Box API工作，并且保持强大的模型以建模更新，从而使其成为现实世界中LLM部署的实用基线。

Title: Following the TRACE: A Structured Path to Empathetic Response Generation with Multi-Agent Models

• Authors: Ziqi Liu, Ziyang Zhou, Yilin Li, Haiyang Zhang, Yangbin Chen
• Subjects: cs.CL, cs.MA
• Abstract URL: https://arxiv.org/abs/2509.21849
• Pdf URL: https://arxiv.org/pdf/2509.21849
• Copy Paste: [[2509.21849]] Following the TRACE: A Structured Path to Empathetic Response Generation with Multi-Agent Models(https://arxiv.org/abs/2509.21849)
• Keywords: language model, llm, agent
• Abstract: Empathetic response generation is a crucial task for creating more human-like and supportive conversational agents. However, existing methods face a core trade-off between the analytical depth of specialized models and the generative fluency of Large Language Models (LLMs). To address this, we propose TRACE, Task-decomposed Reasoning for Affective Communication and Empathy, a novel framework that models empathy as a structured cognitive process by decomposing the task into a pipeline for analysis and synthesis. By building a comprehensive understanding before generation, TRACE unites deep analysis with expressive generation. Experimental results show that our framework significantly outperforms strong baselines in both automatic and LLM-based evaluations, confirming that our structured decomposition is a promising paradigm for creating more capable and interpretable empathetic agents. Our code is available at this https URL.
• 摘要：善解人意的响应产生是创建更类似人类和支持的对话剂的关键任务。但是，现有方法面临着专业模型的分析深度与大语言模型（LLMS）的生成流利度之间的核心权衡。为了解决这个问题，我们提出了痕迹，任务分解的情感交流和移情推理，这是一个新颖的框架，通过将任务分解为分析和综合的管道，将同理心作为结构化认知过程建模为结构化的认知过程。通过在世代前建立全面的理解，可以将深入的分析与表现力的产生相关联。实验结果表明，我们的框架在基于自动和LLM的评估中的表现明显优于强大的基线，证实我们的结构化分解是创造更有能力和可解释的促进剂的有希望的范式。我们的代码可在此HTTPS URL上找到。

Title: KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues

• Authors: Junhao Chen, Yu Huang, Siyuan Li, Rui Yao, Hanqian Li, Hanyu Zhang, Jungang Li, Jian Chen, Bowen Wang, Xuming Hu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21856
• Pdf URL: https://arxiv.org/pdf/2509.21856
• Copy Paste: [[2509.21856]] KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues(https://arxiv.org/abs/2509.21856)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Multi-Turn Long-Form Question Answering (MT-LFQA) is a key application paradigm of Large Language Models (LLMs) in knowledge-intensive domains. However, existing benchmarks are limited to single-turn dialogue, while multi-turn dialogue benchmarks typically assess other orthogonal capabilities rather than knowledge-intensive factuality. To bridge this critical gap, we introduce \textbf{KnowMT-Bench}, the \textit{first-ever} benchmark designed to systematically evaluate MT-LFQA for LLMs across knowledge-intensive fields, including medicine, finance, and law. To faithfully assess the model's real-world performance, KnowMT-Bench employs a dynamic evaluation setting where models generate their own multi-turn dialogue histories given logically progressive question sequences. The factual capability and information delivery efficiency of the \textit{final-turn} answer are then evaluated using a human-validated automated pipeline. Our experiments reveal that multi-turn contexts degrade performance: factual capability declines due to the contextual noise from self-generated histories, while information efficiency drops as models become more verbose with increasing dialogue length. We then investigate mitigation strategies, demonstrating that retrieval-augmented generation (RAG) can effectively alleviate and even reverse this factual degradation. These findings underscore the importance of our benchmark in evaluating and enhancing the conversational factual capabilities of LLMs in real-world knowledge-intensive applications. Code is available at \href{this https URL}{\textcolor{cyan}{\texttt{KnowMT-Bench}}}.
• 摘要：多转变的长格式答案（MT-LFQA）是知识密集型域中大语言模型（LLM）的关键应用程序范式。但是，现有的基准限于单转向对话，而多转向对话基准通常评估其他正交功能，而不是知识密集的事实。为了弥合这个关键的差距，我们介绍\ textbf {knowmt-bench}，\ textit {first-ever}基准，旨在系统地评估跨知识密集型领域的LLM，包括医学，财务和法律。为了忠实评估模型的现实世界绩效，知识基础采用了动态评估设置，其中模型在逻辑上渐进式的问题序列中生成了自己的多转向对话历史。然后，使用人类验证的自动化管道评估\ textit {Final-turn}答案的事实能力和信息传递效率。我们的实验表明，多转向环境降低了性能：由于自我生成的历史的上下文噪音，事实能力下降，而随着模型随着对话长度的增加，信息效率下降。然后，我们调查缓解策略，表明检索增强发电（RAG）可以有效地减轻甚至扭转这种事实的退化。这些发现强调了我们的基准测试在评估和增强现实世界知识密集型应用中LLM的对话事实能力方面的重要性。代码可在\ href {此https url} {\ textColor {cyan} {\ texttt {knowmt-bench}}}}}中获得。

Title: Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations

• Authors: Guanzhi Deng, Mingyang Liu, Dapeng Wu, Yinqiao Li, Linqi Song
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21870
• Pdf URL: https://arxiv.org/pdf/2509.21870
• Copy Paste: [[2509.21870]] Enhancing Low-Rank Adaptation with Structured Nonlinear Transformations(https://arxiv.org/abs/2509.21870)
• Keywords: language model
• Abstract: Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient fine-tuning method for large language models. However, its linear nature limits expressiveness. We propose LoRAN, a non-linear extension of LoRA that applies lightweight transformations to the low-rank updates. We further introduce Sinter, a sine-based activation that adds structured perturbations without increasing parameter count. Experiments across summarization and classification tasks show that LoRAN consistently improves over QLoRA. Ablation studies reveal that Sinter outperforms standard activations such as Sigmoid, ReLU, and Tanh, highlighting the importance of activation design in lowrank tuning.
• 摘要：低级适应性（LORA）是大语模型的广泛采用的参数有效的微调方法。但是，其线性性质限制了表现力。我们提出了洛兰（Loran），这是洛拉（Lora）的非线性扩展，该扩展将轻量级转换应用于低级更新。我们进一步介绍了Sinter，这是一种基于正弦的激活，可在不增加参数计数的情况下增加结构化的扰动。跨摘要和分类任务的实验表明，洛兰一贯改善Qlora。消融研究表明，烧结的表现优于诸如Sigmoid，Relu和Tanh之类的标准激活，强调了LowRank调整中激活设计的重要性。

Title: LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals

• Authors: Min-Hsuan Yeh, Yixuan Li, Tanwi Mallick
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21875
• Pdf URL: https://arxiv.org/pdf/2509.21875
• Copy Paste: [[2509.21875]] LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals(https://arxiv.org/abs/2509.21875)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) aims to mitigate hallucinations in large language models (LLMs) by grounding responses in retrieved documents. Yet, RAG-based LLMs still hallucinate even when provided with correct and sufficient context. A growing line of work suggests that this stems from an imbalance between how models use external context and their internal knowledge, and several approaches have attempted to quantify these signals for hallucination detection. However, existing methods require extensive hyperparameter tuning, limiting their generalizability. We propose LUMINA, a novel framework that detects hallucinations in RAG systems through context-knowledge signals: external context utilization is quantified via distributional distance, while internal knowledge utilization is measured by tracking how predicted tokens evolve across transformer layers. We further introduce a framework for statistically validating these measurements. Experiments on common RAG hallucination benchmarks and four open-source LLMs show that LUMINA achieves consistently high AUROC and AUPRC scores, outperforming prior utilization-based methods by up to +13% AUROC on HalluRAG. Moreover, LUMINA remains robust under relaxed assumptions about retrieval quality and model matching, offering both effectiveness and practicality.
• 摘要：检索增强的一代（RAG）旨在通过在检索的文档中扎根响应来减轻大语言模型（LLMS）的幻觉。但是，即使在提供正确且充分的环境时，基于抹布的LLM仍然会幻觉。越来越多的工作表明，这源于模型如何使用外部环境及其内部知识之间的失衡，而几种方法试图量化这些信号以进行幻觉检测。但是，现有方法需要广泛的超参数调整，从而限制了它们的普遍性。我们提出了Lumina，这是一个新颖的框架，该框架通过上下文知识信号检测抹布系统中的幻觉：外部上下文利用是通过分布距离量化的，而内部知识利用是通过跟踪如何在变压器层跨变压器层进化的图形来衡量的。我们进一步引入了一个框架，用于统计验证这些测量结果。对常见的RAG幻觉基准和四个开源LLMS的实验表明，Lumina始终达到高的AUROC和AUPRC分数，在Hallurag上的实验表现优于先前基于利用率的方法，在Hallurag上的AUROC最高为+13％。此外，Lumina在对检索质量和模型匹配的放松假设下仍然坚固，提供有效性和实用性。

Title: No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping

• Authors: Thanh-Long V. Le, Myeongho Jeon, Kim Vu, Viet Lai, Eunho Yang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21880
• Pdf URL: https://arxiv.org/pdf/2509.21880
• Copy Paste: [[2509.21880]] No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping(https://arxiv.org/abs/2509.21880)
• Keywords: language model, llm, prompt
• Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is a powerful framework for improving the reasoning abilities of Large Language Models (LLMs). However, current methods such as GRPO rely only on problems where the model responses to the same input differ in correctness, while ignoring those where all responses receive the same reward - so-called zero-variance prompts. In this work, we argue that such prompts are not useless but can, in fact, provide meaningful feedback for policy optimization. To this end, we introduce RL with Zero-Variance Prompts (RL-ZVP), a novel algorithm that extract learning signals from zero-variance prompts. RL-ZVP directly rewards correctness and penalizes errors even without contrasting responses, modulating feedback with token-level characteristics to preserve informative, nuanced signals. Across six math reasoning benchmarks, RL-ZVP achieves significant improvements of up to 8.61 points in accuracy and 7.77 points in pass rate over GRPO, while consistently outperforming other baselines that filter out zero-variance prompts. These results highlight the untapped potential of learning from zero-variance prompts in RLVR.
• 摘要：通过可验证的奖励（RLVR）的增强学习是提高大语言模型（LLMS）推理能力的有力框架。但是，诸如GRPO之类的当前方法仅依赖于模型对相同输入的响应在正确性方面有所不同的问题，而忽略所有响应都会获得相同奖励的那些 - 所谓的零变量提示。在这项工作中，我们认为这样的提示不是没有用的，但实际上可以为政策优化提供有意义的反馈。为此，我们使用零变化提示（RL-ZVP）引入RL，这是一种新型算法，从零变量提示提取学习信号。 RL-ZVP即使没有对比响应，也可以直接奖励正确性并惩罚错误，从而将反馈与令牌级别的特征调节，以保留信息丰富的细微信号。在六个数学推理基准中，RL-ZVP的准确性高达8.61点，超过GRPO的高度提高了7.77点，同时始终优于其他过滤零变化提示的基线。这些结果突出了RLVR中零变化提示中学习的尚未开发的潜力。

Title: A Large-Scale Dataset and Citation Intent Classification in Turkish with LLMs

• Authors: Kemal Sami Karaca, Bahaeddin Eravcı
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21907
• Pdf URL: https://arxiv.org/pdf/2509.21907
• Copy Paste: [[2509.21907]] A Large-Scale Dataset and Citation Intent Classification in Turkish with LLMs(https://arxiv.org/abs/2509.21907)
• Keywords: language model, llm, prompt
• Abstract: Understanding the qualitative intent of citations is essential for a comprehensive assessment of academic research, a task that poses unique challenges for agglutinative languages like Turkish. This paper introduces a systematic methodology and a foundational dataset to address this problem. We first present a new, publicly available dataset of Turkish citation intents, created with a purpose-built annotation tool. We then evaluate the performance of standard In-Context Learning (ICL) with Large Language Models (LLMs), demonstrating that its effectiveness is limited by inconsistent results caused by manually designed prompts. To address this core limitation, we introduce a programmable classification pipeline built on the DSPy framework, which automates prompt optimization systematically. For final classification, we employ a stacked generalization ensemble to aggregate outputs from multiple optimized models, ensuring stable and reliable predictions. This ensemble, with an XGBoost meta-model, achieves a state-of-the-art accuracy of 91.3\%. Ultimately, this study provides the Turkish NLP community and the broader academic circles with a foundational dataset and a robust classification framework paving the way for future qualitative citation studies.
• 摘要：了解引用的定性意图对于对学术研究的全面评估至关重要，该任务对像土耳其这样的凝集性语言构成了独特的挑战。本文介绍了一种系统的方法和一个基础数据集来解决此问题。我们首先提出了一个新的公开可用的土耳其引文意图数据集，该数据集是使用专门建造的注释工具创建的。然后，我们通过大型语言模型（LLMS）评估标准内在学习（ICL）的性能，表明其有效性受到手动设计的提示引起的不一致的结果的限制。为了解决此核心限制，我们引入了基于DSPY框架的可编程分类管道，该管道会系统地自动化提示。对于最终分类，我们采用堆叠的概括集合来汇总来自多个优化模型的输出，从而确保稳定且可靠的预测。这个合奏具有XGBoost元模型，达到了91.3 \％的最先进精度。最终，这项研究为土耳其NLP社区和更广泛的学术界提供了基础数据集和强大的分类框架为未来的定性引文研究铺平了道路。

Title: AutoSCORE: Enhancing Automated Scoring with Multi-Agent Large Language Models via Structured Component Recognition

• Authors: Yun Wang, Zhaojun Ding, Xuansheng Wu, Siyue Sun, Ninghao Liu, Xiaoming Zhai
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21910
• Pdf URL: https://arxiv.org/pdf/2509.21910
• Copy Paste: [[2509.21910]] AutoSCORE: Enhancing Automated Scoring with Multi-Agent Large Language Models via Structured Component Recognition(https://arxiv.org/abs/2509.21910)
• Keywords: language model, gpt, llm, prompt, agent
• Abstract: Automated scoring plays a crucial role in education by reducing the reliance on human raters, offering scalable and immediate evaluation of student work. While large language models (LLMs) have shown strong potential in this task, their use as end-to-end raters faces challenges such as low accuracy, prompt sensitivity, limited interpretability, and rubric misalignment. These issues hinder the implementation of LLM-based automated scoring in assessment practice. To address the limitations, we propose AutoSCORE, a multi-agent LLM framework enhancing automated scoring via rubric-aligned Structured COmponent REcognition. With two agents, AutoSCORE first extracts rubric-relevant components from student responses and encodes them into a structured representation (i.e., Scoring Rubric Component Extraction Agent), which is then used to assign final scores (i.e., Scoring Agent). This design ensures that model reasoning follows a human-like grading process, enhancing interpretability and robustness. We evaluate AutoSCORE on four benchmark datasets from the ASAP benchmark, using both proprietary and open-source LLMs (GPT-4o, LLaMA-3.1-8B, and LLaMA-3.1-70B). Across diverse tasks and rubrics, AutoSCORE consistently improves scoring accuracy, human-machine agreement (QWK, correlations), and error metrics (MAE, RMSE) compared to single-agent baselines, with particularly strong benefits on complex, multi-dimensional rubrics, and especially large relative gains on smaller LLMs. These results demonstrate that structured component recognition combined with multi-agent design offers a scalable, reliable, and interpretable solution for automated scoring.
• 摘要：自动评分通过减少对人类评估者的依赖，提供可扩展和立即评估学生工作的依赖，从而在教育中起着至关重要的作用。尽管大型语言模型（LLMS）在这项任务中表现出强大的潜力，但他们用作端到端评估者的使用却面临着诸如较低精度，迅速灵敏度，有限的可解释性和划界未对准之类的挑战。这些问题阻碍了在评估实践中实施基于LLM的自动评分。为了解决局限性，我们提出了Autoscore，Autoscore是通过标准的结构化组件识别来增强自动评分的多代理LLM框架。使用两种代理，Autoscore首先从学生响应中提取了相关的组件，并将其编码为结构化表示形式（即，评分标题分数分量提取剂），然后用于分配最终分数（即评分代理）。该设计确保模型推理遵循类似人类的分级过程，从而增强了解释性和鲁棒性。我们使用专有和开源LLM（GPT-4O，Llama-3.1-8B和Llama-3.1-70B）在ASAP基准测试中评估了从ASAP基准测试的四个基准数据集评估Autoscore。在各种任务和专栏之间，与单个基本基线相比，Autoscore始终提高评分准确性，人机协议（QWK，相关性）和错误指标（MAE，RMSE），对复杂，多维标题，尤其是较小的LLM的相对相对较大的相对增长，对复杂的，多维的标题具有尤其强大的好处。这些结果表明，结构化组件识别与多代理设计相结合提供了可扩展，可靠且可解释的自动评分解决方案。

Title: SimulSense: Sense-Driven Interpreting for Efficient Simultaneous Speech Translation

• Authors: Haotian Tan, Hiroki Ouchi, Sakriani Sakti
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21932
• Pdf URL: https://arxiv.org/pdf/2509.21932
• Copy Paste: [[2509.21932]] SimulSense: Sense-Driven Interpreting for Efficient Simultaneous Speech Translation(https://arxiv.org/abs/2509.21932)
• Keywords: language model, llm
• Abstract: How to make human-interpreter-like read/write decisions for simultaneous speech translation (SimulST) systems? Current state-of-the-art systems formulate SimulST as a multi-turn dialogue task, requiring specialized interleaved training data and relying on computationally expensive large language model (LLM) inference for decision-making. In this paper, we propose SimulSense, a novel framework for SimulST that mimics human interpreters by continuously reading input speech and triggering write decisions to produce translation when a new sense unit is perceived. Experiments against two state-of-the-art baseline systems demonstrate that our proposed method achieves a superior quality-latency tradeoff and substantially improved real-time efficiency, where its decision-making is up to 9.6x faster than the baselines.
• 摘要：如何制作类似人互动的读/写决策，以同时进行语音翻译（Simulst）系统？当前的最新系统将SIMULST作为多转向对话任务，需要专门的交织培训数据，并依靠计算昂贵的大语言模型（LLM）推断进行决策。在本文中，我们提出了Simulsense，这是模拟的新型框架，即通过不断阅读输入语音并触发写作决策来模仿人类口译员，以在感知新的感觉单元时产生翻译。针对两个最先进的基线系统的实验表明，我们提出的方法实现了卓越的质量延迟权衡，并实现了实时效率，其决策比基线快9.6倍。

Title: Why Chain of Thought Fails in Clinical Text Understanding

• Authors: Jiageng Wu, Kevin Xie, Bowen Gu, Nils Krüger, Kueiyu Joshua Lin, Jie Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21933
• Pdf URL: https://arxiv.org/pdf/2509.21933
• Copy Paste: [[2509.21933]] Why Chain of Thought Fails in Clinical Text Understanding(https://arxiv.org/abs/2509.21933)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Large language models (LLMs) are increasingly being applied to clinical care, a domain where both accuracy and transparent reasoning are critical for safe and trustworthy deployment. Chain-of-thought (CoT) prompting, which elicits step-by-step reasoning, has demonstrated improvements in performance and interpretability across a wide range of tasks. However, its effectiveness in clinical contexts remains largely unexplored, particularly in the context of electronic health records (EHRs), the primary source of clinical documentation, which are often lengthy, fragmented, and noisy. In this work, we present the first large-scale systematic study of CoT for clinical text understanding. We assess 95 advanced LLMs on 87 real-world clinical text tasks, covering 9 languages and 8 task types. Contrary to prior findings in other domains, we observe that 86.3\% of models suffer consistent performance degradation in the CoT setting. More capable models remain relatively robust, while weaker ones suffer substantial declines. To better characterize these effects, we perform fine-grained analyses of reasoning length, medical concept alignment, and error profiles, leveraging both LLM-as-a-judge evaluation and clinical expert evaluation. Our results uncover systematic patterns in when and why CoT fails in clinical contexts, which highlight a critical paradox: CoT enhances interpretability but may undermine reliability in clinical text tasks. This work provides an empirical basis for clinical reasoning strategies of LLMs, highlighting the need for transparent and trustworthy approaches.
• 摘要：大型语言模型（LLM）越来越多地应用于临床护理，一个领域，精度和透明推理对于安全且值得信赖的部署至关重要。逐步推理的想法（COT）提示（COT）提示已证明了各种任务的性能和可解释性的提高。但是，其在临床环境中的有效性在很大程度上尚未开发，尤其是在电子健康记录（EHRS）的背景下，这是临床文献的主要来源，通常是冗长，分散且嘈杂的。在这项工作中，我们介绍了有关COT的首次大规模系统研究，以了解临床文本理解。我们评估了87个现实世界临床文本任务的95个高级LLM，涵盖了9种语言和8种任务类型。与其他域中的先前发现相反，我们观察到86.3％的模型在COT设置中持续稳定的性能降解。更有能力的模型仍然相对强大，而较弱的模型却大幅下降。为了更好地表征这些效果，我们对推理长度，医学概念一致性和错误概况进行细粒度分析，利用LLM-AS-A-A-A-A-A-Gudge评估和临床专家评估。我们的结果发现何时何时以及为什么在临床环境中失败的系统模式，这突出了一个关键的悖论：COT可增强可解释性，但可能会破坏临床文本任务中的可靠性。这项工作为LLM的临床推理策略提供了经验基础，强调了对透明和值得信赖的方法的需求。

Title: Debiasing Large Language Models in Thai Political Stance Detection via Counterfactual Calibration

• Authors: Kasidit Sermsri, Teerapong Panboonyuen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21946
• Pdf URL: https://arxiv.org/pdf/2509.21946
• Copy Paste: [[2509.21946]] Debiasing Large Language Models in Thai Political Stance Detection via Counterfactual Calibration(https://arxiv.org/abs/2509.21946)
• Keywords: language model, llm
• Abstract: Political stance detection in low-resource and culturally complex settings poses a critical challenge for large language models (LLMs). In the Thai political landscape - marked by indirect language, polarized figures, and entangled sentiment and stance - LLMs often display systematic biases such as sentiment leakage and favoritism toward entities. These biases undermine fairness and reliability. We present ThaiFACTUAL, a lightweight, model-agnostic calibration framework that mitigates political bias without requiring fine-tuning. ThaiFACTUAL uses counterfactual data augmentation and rationale-based supervision to disentangle sentiment from stance and reduce bias. We also release the first high-quality Thai political stance dataset, annotated with stance, sentiment, rationales, and bias markers across diverse entities and events. Experimental results show that ThaiFACTUAL significantly reduces spurious correlations, enhances zero-shot generalization, and improves fairness across multiple LLMs. This work highlights the importance of culturally grounded debiasing techniques for underrepresented languages.
• 摘要：低资源和文化复杂环境中的政治立场检测对大型语言模型（LLM）提出了关键挑战。在泰国政治景观中 - 以间接语言，两极分化的数字以及纠缠的情绪和立场为标志 - LLM经常表现出系统的偏见，例如对实体的情感泄漏和偏爱。这些偏见破坏了公平和可靠性。我们提出了Thaifactual，这是一个轻巧的模型静态校准框架，可以减轻政治偏见而无需进行微调。 ThaifActual使用反事实数据增强和基于基本原理的监督，以使情绪与立场的情感相关并减少偏见。我们还释放了第一个高质量的泰国政治立场数据集，并以各种实体和事件的姿态，情感，理由和偏见标记注释。实验结果表明，thaifactual显着降低了虚假相关性，增强了零拍的概括并提高了多个LLM的公平性。这项工作强调了文化扎根的伪造技术对于代表性不足的语言的重要性。

Title: MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation

• Authors: Xinping Lei, Tong Zhou, Yubo Chen, Kang Liu, Jun Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21978
• Pdf URL: https://arxiv.org/pdf/2509.21978
• Copy Paste: [[2509.21978]] MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation(https://arxiv.org/abs/2509.21978)
• Keywords: language model, llm, agent
• Abstract: Large Language Models (LLMs) hold substantial potential for accelerating academic ideation but face critical challenges in grounding ideas and mitigating confirmation bias for further refinement. We propose integrating motivational knowledge graphs and socratic dialogue to address these limitations in enhanced LLM ideation (MotivGraph-SoIQ). This novel framework provides essential grounding and practical idea improvement steps for LLM ideation by integrating a Motivational Knowledge Graph (MotivGraph) with a Q-Driven Socratic Ideator. The MotivGraph structurally stores three key node types(problem, challenge and solution) to offer motivation grounding for the LLM ideation process. The Ideator is a dual-agent system utilizing Socratic questioning, which facilitates a rigorous refinement process that mitigates confirmation bias and improves idea quality across novelty, experimental rigor, and motivational rationality dimensions. On the ICLR25 paper topics dataset, MotivGraph-SoIQ exhibits clear advantages over existing state-of-the-art approaches across LLM-based scoring, ELO ranking, and human evaluation metrics.
• 摘要：大型语言模型（LLM）具有加速学术意见的巨大潜力，但在基础思想和减轻确认偏见方面面临着挑战，以进一步完善。我们提出了整合动机知识图和苏格拉底对话，以在增强的LLM Iseation（Mithgraph-Soiq）中解决这些局限性。这个新颖的框架通过将动机知识图（Motivgraph）与Q驱动的苏格拉底式构想者整合在一起，为LLM构思提供了基本的基础和实用的思想改进步骤。动机在结构上存储了三种关键节点类型（问题，挑战和解决方案），以为LLM构思过程提供动机基础。 Iteator是一种利用苏格拉底询问的双重机构系统，它促进了严格的完善过程，可以减轻确认偏见并提高新颖性，实验性严格和动机理性维度的思想质量。在ICLR25纸质主题数据集上，Motivgraph-Soiq在基于LLM的评分，ELO排名和人类评估指标的现有最新方法上具有明显的优势。

Title: Black-Box Hallucination Detection via Consistency Under the Uncertain Expression

• Authors: Seongho Joo, Kyungmin Min, Jahyun Koo, Kyomin Jung
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21999
• Pdf URL: https://arxiv.org/pdf/2509.21999
• Copy Paste: [[2509.21999]] Black-Box Hallucination Detection via Consistency Under the Uncertain Expression(https://arxiv.org/abs/2509.21999)
• Keywords: language model, gpt, llm, hallucination
• Abstract: Despite the great advancement of Language modeling in recent days, Large Language Models (LLMs) such as GPT3 are notorious for generating non-factual responses, so-called "hallucination" problems. Existing methods for detecting and alleviating this hallucination problem require external resources or the internal state of LLMs, such as the output probability of each token. Given the LLM's restricted external API availability and the limited scope of external resources, there is an urgent demand to establish the Black-Box approach as the cornerstone for effective hallucination detection. In this work, we propose a simple black-box hallucination detection metric after the investigation of the behavior of LLMs under expression of uncertainty. Our comprehensive analysis reveals that LLMs generate consistent responses when they present factual responses while non-consistent responses vice versa. Based on the analysis, we propose an efficient black-box hallucination detection metric with the expression of uncertainty. The experiment demonstrates that our metric is more predictive of the factuality in model responses than baselines that use internal knowledge of LLMs.
• 摘要：尽管最近几天的语言建模取得了巨大的进步，但GPT3等大型语言模型（LLM）还是臭名昭著的，因为它会产生非事实响应，即所谓的“幻觉”问题。现有的检测和减轻此幻觉问题的方法需要外部资源或LLM的内部状态，例如每个令牌的输出概率。鉴于LLM受限的外部API可用性和外部资源范围有限，因此迫切需要建立黑框方法作为有效幻觉检测的基石。在这项工作中，我们提出了一个简单的黑盒幻觉检测指标，在研究不确定性表达下LLM的行为后。我们的全面分析表明，LLM在提出事实反应时会产生一致的响应，反之亦然。基于分析，我们提出了具有不确定性表达的有效的黑盒幻觉检测度量。该实验表明，与使用LLMS内部知识的基准相比，我们的指标更能预测模型响应中的事实。

Title: GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation

• Authors: Cehao Yang, Xiaojun Wu, Xueyuan Lin, Chengjin Xu, Xuhui Jiang, Yuanliang Sun, Jia Li, Hui Xiong, Jian Guo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22009
• Pdf URL: https://arxiv.org/pdf/2509.22009
• Copy Paste: [[2509.22009]] GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation(https://arxiv.org/abs/2509.22009)
• Keywords: llm, retrieval-augmented generation, agent
• Abstract: Graph Retrieval-Augmented Generation (GraphRAG) enhances factual reasoning in LLMs by structurally modeling knowledge through graph-based representations. However, existing GraphRAG approaches face two core limitations: shallow retrieval that fails to surface all critical evidence, and inefficient utilization of pre-constructed structural graph data, which hinders effective reasoning from complex queries. To address these challenges, we propose \textsc{GraphSearch}, a novel agentic deep searching workflow with dual-channel retrieval for GraphRAG. \textsc{GraphSearch} organizes the retrieval process into a modular framework comprising six modules, enabling multi-turn interactions and iterative reasoning. Furthermore, \textsc{GraphSearch} adopts a dual-channel retrieval strategy that issues semantic queries over chunk-based text data and relational queries over structural graph data, enabling comprehensive utilization of both modalities and their complementary strengths. Experimental results across six multi-hop RAG benchmarks demonstrate that \textsc{GraphSearch} consistently improves answer accuracy and generation quality over the traditional strategy, confirming \textsc{GraphSearch} as a promising direction for advancing graph retrieval-augmented generation.
• 摘要：图检索 - 调格生成（GraphRag）通过通过基于图形的表示来对知识进行结构建模，从而增强了LLMS中的事实推理。但是，现有的GraphRag方法面临两个核心局限性：未能浮出所有关键证据的浅检索，并且对预构建的结构图数据的利用效率低下，这阻碍了复杂查询的有效推理。为了应对这些挑战，我们提出\ textsc {GraphSearch}，这是一种新颖的代理深度搜索工作流，并使用双通道检索GraphRag。 \ textsc {GraphSearch}将检索过程组织成一个模块化框架，该框架包括六个模块，实现了多转交互和迭代推理。此外，\ textsc {GraphSearch}采用了双通道检索策略，该策略对基于块的基于块的文本数据和关系图数据的语义查询发出了语义查询，从而可以全面利用模态及其互补优势。六个多跳抹布基准的实验结果表明，\ textsc {GraphSearch}始终提高答案的准确性和生成质量，而不是传统策略，证实了\ textsc {GraphSearch}，这是推进图形检验式登记生成的有前途的方向。

Title: From Outliers to Topics in Language Models: Anticipating Trends in News Corpora

• Authors: Evangelia Zve, Benjamin Icard, Alice Breton, Lila Sainero, Gauvain Bourgne, Jean-Gabriel Ganascia
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22030
• Pdf URL: https://arxiv.org/pdf/2509.22030
• Copy Paste: [[2509.22030]] From Outliers to Topics in Language Models: Anticipating Trends in News Corpora(https://arxiv.org/abs/2509.22030)
• Keywords: language model
• Abstract: This paper examines how outliers, often dismissed as noise in topic modeling, can act as weak signals of emerging topics in dynamic news corpora. Using vector embeddings from state-of-the-art language models and a cumulative clustering approach, we track their evolution over time in French and English news datasets focused on corporate social responsibility and climate change. The results reveal a consistent pattern: outliers tend to evolve into coherent topics over time across both models and languages.
• 摘要：本文探讨了在主题建模中经常被视为噪音的离群值如何充当动态新闻库中新兴主题的弱信号。使用最先进的语言模型的向量嵌入和累积聚类方法，我们在法语和英语新闻数据集中跟踪它们的演变，该数据集以企业社会责任和气候变化为重点。结果揭示了一个一致的模式：在模型和语言中，离群值倾向于演变为一致的主题。

Title: Taxonomy of Comprehensive Safety for Clinical Agents

• Authors: Jean Seo, Hyunkyung Lee, Gibaeg Kim, Wooseok Han, Jaehyo Yoo, Seungseop Lim, Kihun Shin, Eunho Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22041
• Pdf URL: https://arxiv.org/pdf/2509.22041
• Copy Paste: [[2509.22041]] Taxonomy of Comprehensive Safety for Clinical Agents(https://arxiv.org/abs/2509.22041)
• Keywords: chat, agent
• Abstract: Safety is a paramount concern in clinical chatbot applications, where inaccurate or harmful responses can lead to serious consequences. Existing methods--such as guardrails and tool calling--often fall short in addressing the nuanced demands of the clinical domain. In this paper, we introduce TACOS (TAxonomy of COmprehensive Safety for Clinical Agents), a fine-grained, 21-class taxonomy that integrates safety filtering and tool selection into a single user intent classification step. TACOS is a taxonomy that can cover a wide spectrum of clinical and non-clinical queries, explicitly modeling varying safety thresholds and external tool dependencies. To validate our framework, we curate a TACOS-annotated dataset and perform extensive experiments. Our results demonstrate the value of a new taxonomy specialized for clinical agent settings, and reveal useful insights about train data distribution and pretrained knowledge of base models.
• 摘要：安全是临床聊天机器人应用中的最重要问题，在临床聊天机器人应用中，不准确或有害的响应可能导致严重后果。现有的方法（例如护栏和工具调用）通常在满足临床领域的细微需求方面差点不足。在本文中，我们介绍了炸玉米饼（临床代理的综合安全性分类法），这是一种精细的21级分类法，将安全过滤和工具选择集成到单个用户意图分类步骤中。炸玉米饼是一种分类法，可以涵盖各种临床和非临床查询，明确对改变安全阈值以及外部工具依赖性进行建模。为了验证我们的框架，我们策划了炸玉米饼的注销数据集并执行广泛的实验。我们的结果证明了专门用于临床代理设置的新分类法的价值，并揭示了有关火车数据分布和验证基础模型知识的有用见解。

Title: Fuzzy Reasoning Chain (FRC): An Innovative Reasoning Framework from Fuzziness to Clarity

• Authors: Ping Chen, Xiang Liu, Zhaoxiang Liu, Zezhou Chen, Xingpeng Zhang, Huan Hu, Zipeng Wang, Kai Wang, Shuming Shi, Shiguo Lian
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22054
• Pdf URL: https://arxiv.org/pdf/2509.22054
• Copy Paste: [[2509.22054]] Fuzzy Reasoning Chain (FRC): An Innovative Reasoning Framework from Fuzziness to Clarity(https://arxiv.org/abs/2509.22054)
• Keywords: language model, llm
• Abstract: With the rapid advancement of large language models (LLMs), natural language processing (NLP) has achieved remarkable progress. Nonetheless, significant challenges remain in handling texts with ambiguity, polysemy, or uncertainty. We introduce the Fuzzy Reasoning Chain (FRC) framework, which integrates LLM semantic priors with continuous fuzzy membership degrees, creating an explicit interaction between probability-based reasoning and fuzzy membership reasoning. This transition allows ambiguous inputs to be gradually transformed into clear and interpretable decisions while capturing conflicting or uncertain signals that traditional probability-based methods cannot. We validate FRC on sentiment analysis tasks, where both theoretical analysis and empirical results show that it ensures stable reasoning and facilitates knowledge transfer across different model scales. These findings indicate that FRC provides a general mechanism for managing subtle and ambiguous expressions with improved interpretability and robustness.
• 摘要：随着大语言模型（LLM）的快速发展，自然语言处理（NLP）取得了显着的进步。但是，在处理歧义，多义或不确定性的文本中仍然存在重大挑战。我们介绍了模糊的推理链（FRC）框架，该框架将LLM语义先验与连续模糊的成员资格学位集成在一起，从而在基于概率的推理和模糊成员资格推理之间建立了明确的相互作用。这种过渡允许模棱两可的输入逐渐转化为明确，可解释的决策，同时捕获冲突或不确定传统基于概率的方法不能。我们在情感分析任务上验证了FRC，其中理论分析和经验结果都表明，它确保了稳定的推理并促进了跨不同模型量表的知识转移。这些发现表明，FRC提供了一种通用机制，可以利用可解释性和鲁棒性来管理微妙和模棱两可的表达。

Title: RedNote-Vibe: A Dataset for Capturing Temporal Dynamics of AI-Generated Text in Social Media

• Authors: Yudong Li, Yufei Sun, Yuhan Yao, Peiru Yang, Wanyue Li, Jiajun Zou, Yongfeng Huang, Linlin Shen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22055
• Pdf URL: https://arxiv.org/pdf/2509.22055
• Copy Paste: [[2509.22055]] RedNote-Vibe: A Dataset for Capturing Temporal Dynamics of AI-Generated Text in Social Media(https://arxiv.org/abs/2509.22055)
• Keywords: language model, llm
• Abstract: The proliferation of Large Language Models (LLMs) has led to widespread AI-Generated Text (AIGT) on social media platforms, creating unique challenges where content dynamics are driven by user engagement and evolve over time. However, existing datasets mainly depict static AIGT detection. In this work, we introduce RedNote-Vibe, the first longitudinal (5-years) dataset for social media AIGT analysis. This dataset is sourced from Xiaohongshu platform, containing user engagement metrics (e.g., likes, comments) and timestamps spanning from the pre-LLM period to July 2025, which enables research into the temporal dynamics and user interaction patterns of AIGT. Furthermore, to detect AIGT in the context of social media, we propose PsychoLinguistic AIGT Detection Framework (PLAD), an interpretable approach that leverages psycholinguistic features. Our experiments show that PLAD achieves superior detection performance and provides insights into the signatures distinguishing human and AI-generated content. More importantly, it reveals the complex relationship between these linguistic features and social media engagement. The dataset is available at this https URL.
• 摘要：大型语言模型（LLM）的扩散导致社交媒体平台上的广泛AI生成的文本（AIGT），从而在内容动态受到用户参与并随着时间的推移而发展而产生了独特的挑战。但是，现有数据集主要描述静态AIGT检测。在这项工作中，我们介绍了Rednote-Vibe，这是第一个用于社交媒体AIGT分析的纵向（5年）数据集。该数据集来自Xiaohongshu平台，其中包含用户参与度量标准（例如，喜欢，注释）和时间戳，跨越前LLM期间到2025年7月，这可以研究AIGT的时间动态和用户交互模式。此外，为了在社交媒体的背景下检测AIGT，我们提出了心理语言AIGT检测框架（PLAD），这是一种利用心理语言特征的可解释方法。我们的实验表明，PLAD实现了卓越的检测性能，并提供了与人类和AI生成含量区分的特征的见解。更重要的是，它揭示了这些语言特征与社交媒体参与之间的复杂关系。该数据集可在此HTTPS URL上找到。

Title: Fine-tuning Done Right in Model Editing

• Authors: Wanli Yang, Fei Sun, Rui Tang, Hongyu Zang, Du Su, Qi Cao, Jingang Wang, Huawei Shen, Xueqi Cheng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22072
• Pdf URL: https://arxiv.org/pdf/2509.22072
• Copy Paste: [[2509.22072]] Fine-tuning Done Right in Model Editing(https://arxiv.org/abs/2509.22072)
• Keywords: language model, llm
• Abstract: Fine-tuning, a foundational method for adapting large language models, has long been considered ineffective for model editing. Here, we challenge this belief, arguing that the reported failure arises not from the inherent limitation of fine-tuning itself, but from adapting it to the sequential nature of the editing task, a single-pass depth-first pipeline that optimizes each sample to convergence before moving on. While intuitive, this depth-first pipeline coupled with sample-wise updating over-optimizes each edit and induces interference across edits. Our controlled experiments reveal that simply restoring fine-tuning to the standard breadth-first (i.e., epoch-based) pipeline with mini-batch optimization substantially improves its effectiveness for model editing. Moreover, fine-tuning in editing also suffers from suboptimal tuning parameter locations inherited from prior methods. Through systematic analysis of tuning locations, we derive LocFT-BF, a simple and effective localized editing method built on the restored fine-tuning framework. Extensive experiments across diverse LLMs and datasets demonstrate that LocFT-BF outperforms state-of-the-art methods by large margins. Notably, to our knowledge, it is the first to sustain 100K edits and 72B-parameter models,10 x beyond prior practice, without sacrificing general capabilities. By clarifying a long-standing misconception and introducing a principled localized tuning strategy, we advance fine-tuning from an underestimated baseline to a leading method for model editing, establishing a solid foundation for future research.
• 摘要：通过调整大型语言模型的基础方法微调一直被认为是模型编辑的无效。在这里，我们挑战了这种信念，认为报告的失败不是源于微调本身的固有限制，而是由于将其调整为编辑任务的顺序性质，这是一种单通行的深度优先管道，可在继续前进之前优化每个样本以收敛到收敛。虽然直观，但该深度优先的管道以及样品的更新更新过度优化，每种编辑都会诱导跨编辑的干扰。我们的受控实验表明，仅将微调恢复到具有小批量优化的标准广度优先（即基于时期的）管道，从而大大提高了其模型编辑的有效性。此外，编辑中的微调还遭受了从先前方法继承的次优调参数位置。通过对调整位置的系统分析，我们得出了LoCFT-BF，这是一种基于恢复的微调框架构建的简单有效的局部编辑方法。跨不同LLM和数据集进行的广泛实验表明，LOCFT-BF以大幅度的优于最先进的方法。值得注意的是，据我们所知，它是第一个维持100K编辑和72B参数模型，即超越先前实践的10 x，而无需牺牲一般能力。通过澄清长期存在的误解并引入了原则性的本地调整策略，我们将微调从低估的基线到了模型编辑的领先方法，为未来的研究建立了坚实的基础。

Title: COSPADI: Compressing LLMs via Calibration-Guided Sparse Dictionary Learning

• Authors: Dmitriy Shopkhoev, Denis Makhov, Magauiya Zhussip, Ammar Ali, Stamatios Lefkimmiatis
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22075
• Pdf URL: https://arxiv.org/pdf/2509.22075
• Copy Paste: [[2509.22075]] COSPADI: Compressing LLMs via Calibration-Guided Sparse Dictionary Learning(https://arxiv.org/abs/2509.22075)
• Keywords: language model, llm
• Abstract: Post-training compression of large language models (LLMs) largely relies on low-rank weight approximation, which represents each column of a weight matrix in a shared low-dimensional subspace. While this is a computationally efficient strategy, the imposed structural constraint is rigid and can lead to a noticeable model accuracy drop. In this work, we propose CoSpaDi (Compression via Sparse Dictionary Learning), a novel training-free compression framework that replaces low-rank decomposition with a more flexible structured sparse factorization in which each weight matrix is represented with a dense dictionary and a column-sparse coefficient matrix. This formulation enables a union-of-subspaces representation: different columns of the original weight matrix are approximated in distinct subspaces spanned by adaptively selected dictionary atoms, offering greater expressiveness than a single invariant basis. Crucially, CoSpaDi leverages a small calibration dataset to optimize the factorization such that the output activations of compressed projection layers closely match those of the original ones, thereby minimizing functional reconstruction error rather than mere weight approximation. This data-aware strategy preserves better model fidelity without any fine-tuning under reasonable compression ratios. Moreover, the resulting structured sparsity allows efficient sparse-dense matrix multiplication and is compatible with post-training quantization for further memory and latency gains. We evaluate CoSpaDi across multiple Llama and Qwen models under per-layer and per-group settings at 20-50\% compression ratios, demonstrating consistent superiority over state-of-the-art data-aware low-rank methods both in accuracy and perplexity. Our results establish structured sparse dictionary learning as a powerful alternative to conventional low-rank approaches for efficient LLM deployment.
• 摘要：大语言模型（LLMS）的训练后压缩在很大程度上取决于低级别的重量近似，这代表了共享低维度子空间中权重矩阵的每一列。尽管这是一种计算高效的策略，但施加的结构约束是刚性的，可以导致明显的模型精度下降。在这项工作中，我们提出了COSPADI（通过稀疏字典学习压缩），这是一个新型的无训练压缩框架，将低级分解替换为更灵活的结构稀疏分解，在该稀疏分解中，每个重量矩阵都用密集的词典和列式词曲板和柱状系数矩阵表示。该公式可以实现subspace的表示：原始权重矩阵的不同列以由自适应选择的词典原子跨越不同的子空间近似，提供了比单个不变的基础更高的表现力。至关重要的是，Cospadi利用一个小的校准数据集来优化分解化，以使压缩投影层的输出激活与原始层的输出激活非常匹配，从而最大程度地减少了功能重建误差，而不是单纯的重量近似。这种数据感知策略可以保留更好的模型保真度，而无需在合理的压缩比下进行任何微调。此外，所得的结构化稀疏性允许有效的稀疏密度矩阵乘法，并且与训练后量化兼容，以获得进一步的记忆和延迟增长。我们以20-50 \％的压缩比评估了每层和每组设置下的多个美洲驼和QWEN模型的Cospadi，这表明了准确性和困惑性的最先进的数据感知低级别方法一致。我们的结果建立结构化的稀疏字典学习，作为用于有效LLM部署的常规低级方法的有力替代方法。

Title: S2J: Bridging the Gap Between Solving and Judging Ability in Generative Reward Models

• Authors: Shaoning Sun, Jiachen Yu, Zongqi Wang, Xuewei Yang, Tianle Gu, Yujiu Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22099
• Pdf URL: https://arxiv.org/pdf/2509.22099
• Copy Paste: [[2509.22099]] S2J: Bridging the Gap Between Solving and Judging Ability in Generative Reward Models(https://arxiv.org/abs/2509.22099)
• Keywords: language model, llm
• Abstract: With the rapid development of large language models (LLMs), generative reward models (GRMs) have been widely adopted for reward modeling and evaluation. Previous studies have primarily focused on training specialized GRMs by optimizing them on preference datasets with the judgment correctness as supervision. While it's widely accepted that GRMs with stronger problem-solving capabilities typically exhibit superior judgment abilities, we first identify a significant solve-to-judge gap when examining individual queries. Specifically, the solve-to-judge gap refers to the phenomenon where GRMs struggle to make correct judgments on some queries (14%-37%), despite being fully capable of solving them. In this paper, we propose the Solve-to-Judge (S2J) approach to address this problem. Specifically, S2J simultaneously leverages both the solving and judging capabilities on a single GRM's output for supervision, explicitly linking the GRM's problem-solving and evaluation abilities during model optimization, thereby narrowing the gap. Our comprehensive experiments demonstrate that S2J effectively reduces the solve-to-judge gap by 16.2%, thereby enhancing the model's judgment performance by 5.8%. Notably, S2J achieves state-of-the-art (SOTA) performance among GRMs built on the same base model while utilizing a significantly smaller training dataset. Moreover, S2J accomplishes this through self-evolution without relying on more powerful external models for distillation.
• 摘要：随着大语言模型（LLM）的快速发展，已广泛采用生成奖励模型（GRM）进行奖励建模和评估。先前的研究主要集中在培训专业的GRM上，通过以判断正确性为监督，在偏好数据集上进行优化。虽然人们广泛认为，具有更强问题的能力的GRM通常表现出卓越的判断力，但我们首先在检查单个查询时发现了一个重大的解决方案差距。具体而言，解决法官的差距是指GRMS难以在某些查询（14％-37％）上做出正确判断的现象，尽管完全有能力解决它们。在本文中，我们提出了解决这个问题的解决方案（S2J）方法。具体而言，S2J同时利用单个GRM的输出的求解和判断功能进行监督，从而在模型优化过程中明确链接GRM的问题解决和评估能力，从而缩小了差距。我们的全面实验表明，S2J有效地将解决方案与法官差距降低了16.2％，从而将模型的判断力提高了5.8％。值得注意的是，S2J在使用同一基本模型的GRM中实现了最先进的（SOTA）性能，同时使用了明显较小的培训数据集。此外，S2J通过自我进化实现了这一点，而无需依靠更强大的外部模型进行蒸馏。

Title: Think Right, Not More: Test-Time Scaling for Numerical Claim Verification

• Authors: Primakov Chungkham, V Venktesh, Vinay Setty, Avishek Anand
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22101
• Pdf URL: https://arxiv.org/pdf/2509.22101
• Copy Paste: [[2509.22101]] Think Right, Not More: Test-Time Scaling for Numerical Claim Verification(https://arxiv.org/abs/2509.22101)
• Keywords: language model, llm
• Abstract: Fact-checking real-world claims, particularly numerical claims, is inherently complex that require multistep reasoning and numerical reasoning for verifying diverse aspects of the claim. Although large language models (LLMs) including reasoning models have made tremendous advances, they still fall short on fact-checking real-world claims that require a combination of compositional and numerical reasoning. They are unable to understand nuance of numerical aspects, and are also susceptible to the reasoning drift issue, where the model is unable to contextualize diverse information resulting in misinterpretation and backtracking of reasoning process. In this work, we systematically explore scaling test-time compute (TTS) for LLMs on the task of fact-checking complex numerical claims, which entails eliciting multiple reasoning paths from an LLM. We train a verifier model (VERIFIERFC) to navigate this space of possible reasoning paths and select one that could lead to the correct verdict. We observe that TTS helps mitigate the reasoning drift issue, leading to significant performance gains for fact-checking numerical claims. To improve compute efficiency in TTS, we introduce an adaptive mechanism that performs TTS selectively based on the perceived complexity of the claim. This approach achieves 1.8x higher efficiency than standard TTS, while delivering a notable 18.8% performance improvement over single-shot claim verification methods. Our code and data can be found at this https URL
• 摘要：事实核对现实世界的主张，尤其是数值主张，是固有的复杂，需要多步推理和数值推理来验证索赔的各个方面。尽管包括推理模型在内的大型语言模型（LLM）取得了巨大的进步，但它们仍然缺乏事实核对现实世界的主张，这些主张需要组合和数值推理。他们无法理解数值方面的细微差别，并且也容易受到推理漂移问题的影响，在这种情况下，该模型无法将各种信息上下文化，从而导致对推理过程的误解和回溯。在这项工作中，我们系统地探索了LLMS的扩展测试时间计算（TTS），以检查事实检查复杂的数值主张的任务，这需要从LLM引起多个推理路径。我们训练一个验证器模型（VerifierFC），以导航这一可能的推理路径，并选择可能导致正确判决的一个。我们观察到TTS有助于减轻推理漂移问题，从而为事实检查数值主张带来了显着的绩效提高。为了提高TTS的计算效率，我们引入了一种自适应机制，该机制根据声明的感知复杂性选择性地执行TT。这种方法的效率比标准TT提高1.8倍，同时在单次索赔验证方法上提供了显着的18.8％的性能提高。我们的代码和数据可以在此HTTPS URL上找到

Title: Universal Legal Article Prediction via Tight Collaboration between Supervised Classification Model and LLM

• Authors: Xiao Chi, Wenlin Zhong, Yiquan Wu, Wei Wang, Kun Kuang, Fei Wu, Minghui Xiong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22119
• Pdf URL: https://arxiv.org/pdf/2509.22119
• Copy Paste: [[2509.22119]] Universal Legal Article Prediction via Tight Collaboration between Supervised Classification Model and LLM(https://arxiv.org/abs/2509.22119)
• Keywords: language model, llm
• Abstract: Legal Article Prediction (LAP) is a critical task in legal text classification, leveraging natural language processing (NLP) techniques to automatically predict relevant legal articles based on the fact descriptions of cases. As a foundational step in legal decision-making, LAP plays a pivotal role in determining subsequent judgments, such as charges and penalties. Despite its importance, existing methods face significant challenges in addressing the complexities of LAP. Supervised classification models (SCMs), such as CNN and BERT, struggle to fully capture intricate fact patterns due to their inherent limitations. Conversely, large language models (LLMs), while excelling in generative tasks, perform suboptimally in predictive scenarios due to the abstract and ID-based nature of legal articles. Furthermore, the diversity of legal systems across jurisdictions exacerbates the issue, as most approaches are tailored to specific countries and lack broader applicability. To address these limitations, we propose Uni-LAP, a universal framework for legal article prediction that integrates the strengths of SCMs and LLMs through tight collaboration. Specifically, in Uni-LAP, the SCM is enhanced with a novel Top-K loss function to generate accurate candidate articles, while the LLM employs syllogism-inspired reasoning to refine the final predictions. We evaluated Uni-LAP on datasets from multiple jurisdictions, and empirical results demonstrate that our approach consistently outperforms existing baselines, showcasing its effectiveness and generalizability.
• 摘要：法律文章预测（LAP）是法律文本分类中的一项关键任务，利用自然语言处理（NLP）技术根据案件的事实描述自动预测相关的法律文章。作为法律决策的基本步骤，圈圈在确定随后的判决（例如指控和惩罚）方面起着关键作用。尽管它很重要，但现有的方法在解决圈圈的复杂性方面面临重大挑战。有监督的分类模型（SCM），例如CNN和BERT，由于其固有的局限性而难以完全捕获复杂的事实模式。相反，大型语言模型（LLMS）虽然在生成任务中出色，但由于法律文章的抽象和基于ID的性质，在预测方案中表现出色。此外，跨司法管辖区的法律体系的多样性加剧了这个问题，因为大多数方法是针对特定国家量身定制的，并且缺乏更广泛的适用性。为了解决这些限制，我们提出了Uni-Lap，这是一个法律文章预测的通用框架，通过紧密的协作整合了SCM和LLM的优势。具体而言，在Uni-lap中，SCM通过新颖的TOP-K损失函数增强，以生成准确的候选文章，而LLM采用了三段论的启发性推理来完善最终预测。我们评估了来自多个司法管辖区的数据集对数据集的评估，经验结果表明，我们的方法始终优于现有基准，展示了其有效性和普遍性。

Title: Multilingual Vision-Language Models, A Survey

• Authors: Andrei-Alexandru Manea, Jindřich Libovický
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22123
• Pdf URL: https://arxiv.org/pdf/2509.22123
• Copy Paste: [[2509.22123]] Multilingual Vision-Language Models, A Survey(https://arxiv.org/abs/2509.22123)
• Keywords: language model
• Abstract: This survey examines multilingual vision-language models that process text and images across languages. We review 31 models and 21 benchmarks, spanning encoder-only and generative architectures, and identify a key tension between language neutrality (consistent cross-lingual representations) and cultural awareness (adaptation to cultural contexts). Current training methods favor neutrality through contrastive learning, while cultural awareness depends on diverse data. Two-thirds of evaluation benchmarks use translation-based approaches prioritizing semantic consistency, though recent work incorporates culturally grounded content. We find discrepancies in cross-lingual capabilities and gaps between training objectives and evaluation goals.
• 摘要：这项调查研究了多种语言的视觉模型，这些模型可以处理跨语言的文本和图像。我们审查了31个模型和21个基准，涵盖了仅编码和生成的体系结构，并确定语言中立性（一致的跨语性表示）和文化意识（适应文化背景）之间的关键张力。当前的培训方法通过对比学习有利于中立，而文化意识则取决于不同的数据。三分之二的评估基准测试使用基于翻译的方法优先考虑语义一致性，尽管最近的工作包含了文化上的内容。我们发现跨语言能力和培训目标和评估目标之间的差距差异。

Title: FoodSEM: Large Language Model Specialized in Food Named-Entity Linking

• Authors: Ana Gjorgjevikj, Matej Martinc, Gjorgjina Cenikj, Sašo Džeroski, Barbara Koroušić Seljak, Tome Eftimov
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.22125
• Pdf URL: https://arxiv.org/pdf/2509.22125
• Copy Paste: [[2509.22125]] FoodSEM: Large Language Model Specialized in Food Named-Entity Linking(https://arxiv.org/abs/2509.22125)
• Keywords: language model, llm, prompt
• Abstract: This paper introduces FoodSEM, a state-of-the-art fine-tuned open-source large language model (LLM) for named-entity linking (NEL) to food-related ontologies. To the best of our knowledge, food NEL is a task that cannot be accurately solved by state-of-the-art general-purpose (large) language models or custom domain-specific models/systems. Through an instruction-response (IR) scenario, FoodSEM links food-related entities mentioned in a text to several ontologies, including FoodOn, SNOMED-CT, and the Hansard taxonomy. The FoodSEM model achieves state-of-the-art performance compared to related models/systems, with F1 scores even reaching 98% on some ontologies and datasets. The presented comparative analyses against zero-shot, one-shot, and few-shot LLM prompting baselines further highlight FoodSEM's superior performance over its non-fine-tuned version. By making FoodSEM and its related resources publicly available, the main contributions of this article include (1) publishing a food-annotated corpora into an IR format suitable for LLM fine-tuning/evaluation, (2) publishing a robust model to advance the semantic understanding of text in the food domain, and (3) providing a strong baseline on food NEL for future benchmarking.
• 摘要：本文介绍了FoodSem，这是一种最先进的微调开源大语模型（LLM），用于命名实体链接（NEL）与食品相关的本体学。据我们所知，食品NEL是一项无法通过最先进的通用（大型）语言模型或自定义域特异性模型/系统来准确解决的任务。通过指示反应（IR）的情况，FoodSem将文本中与食品相关的实体链接到几个本体论，包括Foodon，Snomed-CT和Hansard分类法。与相关模型/系统相比，FoodSem模型可实现最先进的性能，其中一些本体和数据集的F1得分甚至达到98％。提出的对零拍，单发和少量LLM的比较分析促使基线进一步突出了食品比其非预先调整版本的优越性能。通过将食品及其相关资源公开提供，本文的主要贡献包括（1）将食品注释的语料库发布到适合LLM微调/评估的IR格式中，（2）发表强大的模型，以提高对食品领域文本的语义理解，以及（3）在Food Nel上为Future Nel提供强有力的基线，以实现未来的bench Markermarks。

Title: R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning

• Authors: Hongyu Shan, Mingyang Song, Chang Dai, Di Liang, Han Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22131
• Pdf URL: https://arxiv.org/pdf/2509.22131
• Copy Paste: [[2509.22131]] R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning(https://arxiv.org/abs/2509.22131)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Chain-of-Thought (CoT) prompting helps Large Language Models (LLMs) tackle complex reasoning by eliciting explicit step-by-step rationales. However, CoT's verbosity increases latency and memory usage and may propagate early errors across long chains. We propose the Reasoning Capsule (R-Capsule), a framework that aims to combine the efficiency of latent reasoning with the transparency of explicit CoT. The core idea is to compress the high-level plan into a small set of learned latent tokens (a Reasoning Capsule) while keeping execution steps lightweight or explicit. This hybrid approach is inspired by the Information Bottleneck (IB) principle, where we encourage the capsule to be approximately minimal yet sufficient for the task. Minimality is encouraged via a low-capacity bottleneck, which helps improve efficiency. Sufficiency is encouraged via a dual objective: a primary task loss for answer accuracy and an auxiliary plan-reconstruction loss that encourages the capsule to faithfully represent the original textual plan. The reconstruction objective helps ground the latent space, thereby improving interpretability and reducing the use of uninformative shortcuts. Our framework strikes a balance between efficiency, accuracy, and interpretability, thereby reducing the visible token footprint of reasoning while maintaining or improving accuracy on complex benchmarks. Our codes are available at: this https URL
• 摘要：通过思考链（COT）提示，通过引发明确的逐步理解来帮助大型语言模型（LLMS）解决复杂的推理。但是，COT的详细性增加了潜伏期和记忆使用量，并可能在长链上传播早期错误。我们提出了推理胶囊（R-Capsule），该框架旨在将潜在推理的效率与显式COT的透明度相结合。核心想法是将高级计划压缩为一小群学习的潜在令牌（推理胶囊），同时使执行步骤轻巧或显式。这种混合方法的灵感来自信息瓶颈（IB）原理，我们鼓励胶囊大约很小，但足以完成任务。通过低容量的瓶颈鼓励最小化，这有助于提高效率。通过双重目标鼓励足够的能力：回答准确性的主要任务损失和辅助计划重建损失，鼓励胶囊忠实地代表原始的文本计划。重建目标有助于扎根潜在的空间，从而改善了解释性并减少了不明智的快捷方式的使用。我们的框架在效率，准确性和可解释性之间取得了平衡，从而减少了可见的代币占地推理的占地面积，同时保持复杂基准的准确性或提高准确性。我们的代码可用：此HTTPS URL

Title: Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding

• Authors: Shijing Hu, Jingyang Li, Zhihui Lu, Pan Zhou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22134
• Pdf URL: https://arxiv.org/pdf/2509.22134
• Copy Paste: [[2509.22134]] Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding(https://arxiv.org/abs/2509.22134)
• Keywords: language model, llm
• Abstract: Speculative decoding accelerates large language model (LLM) inference by letting a lightweight draft model propose multiple tokens that the target model verifies in parallel. Yet existing training objectives optimize only a single greedy draft path, while decoding follows a tree policy that re-ranks and verifies multiple branches. This draft policy misalignment limits achievable speedups. We introduce Group Tree Optimization (GTO), which aligns training with the decoding-time tree policy through two components: (i) Draft Tree Reward, a sampling-free objective equal to the expected acceptance length of the draft tree under the target model, directly measuring decoding performance; (ii) Group-based Draft Policy Training, a stable optimization scheme that contrasts trees from the current and a frozen reference draft model, forming debiased group-standardized advantages and applying a PPO-style surrogate along the longest accepted sequence for robust updates. We further prove that increasing our Draft Tree Reward provably improves acceptance length and speedup. Across dialogue (MT-Bench), code (HumanEval), and math (GSM8K), and multiple LLMs (e.g., LLaMA-3.1-8B, LLaMA-3.3-70B, Vicuna-1.3-13B, DeepSeek-R1-Distill-LLaMA-8B), GTO increases acceptance length by 7.4% and yields an additional 7.7% speedup over prior state-of-the-art EAGLE-3. By bridging draft policy misalignment, GTO offers a practical, general solution for efficient LLM inference.
• 摘要：投机解码通过让轻量级草稿模型提出了目标模型并行验证的多个令牌，从而加速了大语言模型（LLM）推断。然而，现有的培训目标仅优化单个贪婪的草稿路径，而解码则遵循重新排列和验证多个分支的树策略。该政策错位草案限制了可实现的加速。我们介绍了小组树优化（GTO），该优化将培训与解码时间树策略保持一致：（i）树木奖励草案，一个无抽样的物镜，等于目标模型下草稿树的预期接受长度，直接测量解码性能； （ii）基于小组的政策培训草案，这是一种稳定的优化方案，将当前和冷冻参考草案模型的树木与树木形成鲜明的群体标准化优势，并在强大的更新中使用最长的公认序列应用PPO风格的代理。我们进一步证明，增加树木奖励可以证明可以提高接受时间和加速。 Across dialogue (MT-Bench), code (HumanEval), and math (GSM8K), and multiple LLMs (e.g., LLaMA-3.1-8B, LLaMA-3.3-70B, Vicuna-1.3-13B, DeepSeek-R1-Distill-LLaMA-8B), GTO increases acceptance length by 7.4% and yields an additional 7.7% speedup over prior state-of-the-art Eagle-3。通过弥合政策未对准草案，GTO为有效的LLM推论提供了实用的一般解决方案。

Title: From Long to Lean: Performance-aware and Adaptive Chain-of-Thought Compression via Multi-round Refinement

• Authors: Jianzhi Yan, Le Liu, Youcheng Pan, Shiwei Chen, Zike Yuan, Yang Xiang, Buzhou Tang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22144
• Pdf URL: https://arxiv.org/pdf/2509.22144
• Copy Paste: [[2509.22144]] From Long to Lean: Performance-aware and Adaptive Chain-of-Thought Compression via Multi-round Refinement(https://arxiv.org/abs/2509.22144)
• Keywords: chain-of-thought
• Abstract: Chain-of-Thought (CoT) reasoning improves performance on complex tasks but introduces significant inference latency due to verbosity. We propose Multiround Adaptive Chain-of-Thought Compression (MACC), a framework that leverages the token elasticity phenomenon--where overly small token budgets can paradoxically increase output length--to progressively compress CoTs via multiround refinement. This adaptive strategy allows MACC to determine the optimal compression depth for each input. Our method achieves an average accuracy improvement of 5.6 percent over state-of-the-art baselines, while also reducing CoT length by an average of 47 tokens and significantly lowering latency. Furthermore, we show that test-time performance--accuracy and token length--can be reliably predicted using interpretable features like perplexity and compression rate on the training set. Evaluated across different models, our method enables efficient model selection and forecasting without repeated fine-tuning, demonstrating that CoT compression is both effective and predictable. Our code will be released in this https URL.
• 摘要：经营链（COT）推理提高了复杂任务的性能，但由于冗长而引入了明显的推理潜伏期。我们提出了多个自适应链的压缩链（MACC），该框架利用令牌弹性现象 - 在这种框架中，过度的标记预算可以自相矛盾地增加产出长度 - 通过多型精炼逐步逐步逐步压缩COTS。这种自适应策略使MACC能够确定每个输入的最佳压缩深度。我们的方法的平均准确性比最先进的基线提高了5.6％，同时还将COT长度降低了47个令牌，并显着降低了延迟。此外，我们表明测试时间性能 - 准确性和令牌长度 - 可以可靠地使用可解释的功能（如训练集中的困惑和压缩率）进行可靠的预测。我们的方法在不同模型中进行了评估，可以实现有效的模型选择和预测，而无需重复进行微调，这表明COT压缩既有效又可预测。我们的代码将在此HTTPS URL中发布。

Title: Mixture of Detectors: A Compact View of Machine-Generated Text Detection

• Authors: Sai Teja Lekkala, Yadagiri Annepaka, Arun Kumar Challa, Samatha Reddy Machireddy, Partha Pakray, Chukhu Chunka
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22147
• Pdf URL: https://arxiv.org/pdf/2509.22147
• Copy Paste: [[2509.22147]] Mixture of Detectors: A Compact View of Machine-Generated Text Detection(https://arxiv.org/abs/2509.22147)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are gearing up to surpass human creativity. The veracity of the statement needs careful consideration. In recent developments, critical questions arise regarding the authenticity of human work and the preservation of their creativity and innovative abilities. This paper investigates such issues. This paper addresses machine-generated text detection across several scenarios, including document-level binary and multiclass classification or generator attribution, sentence-level segmentation to differentiate between human-AI collaborative text, and adversarial attacks aimed at reducing the detectability of machine-generated text. We introduce a new work called BMAS English: an English language dataset for binary classification of human and machine text, for multiclass classification, which not only identifies machine-generated text but can also try to determine its generator, and Adversarial attack addressing where it is a common act for the mitigation of detection, and Sentence-level segmentation, for predicting the boundaries between human and machine-generated text. We believe that this paper will address previous work in Machine-Generated Text Detection (MGTD) in a more meaningful way.
• 摘要：大型语言模型（LLM）正在准备超越人类的创造力。声明的真实性需要仔细考虑。在最近的发展中，关于人类工作的真实性以及保护其创造力和创新能力的关键问题。本文调查了此类问题。本文介绍了几种情况下的机器生成的文本检测，包括文档级二进制二进制和多类分类或生成器归因，句子级分段以区分人类AI协作文本，以及旨在降低机器生成文本检测性的对抗性攻击。我们介绍了一项名为BMAS English的新作品：用于人类和机器文本的二进制分类的英语数据集，用于多类别的分类，不仅可以确定机器生成的文本，而且还可以尝试确定其发电机和对抗性攻击，以解决它是在何处进行检测，以及对人类和机器之间的限制和机器之间的限制和句子的构成和机器的文字。我们认为，本文将以更有意义的方式解决机器生成的文本检测（MGTD）的先前工作。

Title: Context Parametrization with Compositional Adapters

• Authors: Josip Jukić, Martin Tutek, Jan Šnajder
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22158
• Pdf URL: https://arxiv.org/pdf/2509.22158
• Copy Paste: [[2509.22158]] Context Parametrization with Compositional Adapters(https://arxiv.org/abs/2509.22158)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) often seamlessly adapt to new tasks through in-context learning (ICL) or supervised fine-tuning (SFT). However, both of these approaches face key limitations: ICL is inefficient when handling many demonstrations, and SFT incurs training overhead while sacrificing flexibility. Mapping instructions or demonstrations from context directly into adapter parameters offers an appealing alternative. While prior work explored generating adapters based on a single input context, it has overlooked the need to integrate multiple chunks of information. To address this gap, we introduce CompAs, a meta-learning framework that translates context into adapter parameters with a compositional structure. Adapters generated this way can be merged algebraically, enabling instructions, demonstrations, or retrieved passages to be seamlessly combined without reprocessing long prompts. Critically, this approach yields three benefits: lower inference cost, robustness to long-context instability, and establishes a principled solution when input exceeds the model's context window. Furthermore, CompAs encodes information into adapter parameters in a reversible manner, enabling recovery of input context through a decoder, facilitating safety and security. Empirical results on diverse multiple-choice and extractive question answering tasks show that CompAs outperforms ICL and prior generator-based methods, especially when scaling to more inputs. Our work establishes composable adapter generation as a practical and efficient alternative for scaling LLM deployment.
• 摘要：大型语言模型（LLMS）通常通过内在学习（ICL）或监督微调（SFT）无缝地适应新任务。但是，这两种方法都面临着关键的局限性：在处理许多示范时，ICL效率低下，而SFT会在牺牲灵活性的同时造成训练开销。将上下文的映射说明或演示直接映射到适配器参数中提供了一种吸引人的选择。在先前的工作探索基于单个输入上下文的生成适配器的同时，它忽略了整合多个信息的必要性。为了解决此差距，我们引入了Compas，Compas是一种元学习框架，将上下文转化为具有组成结构的适配器参数。可以通过代数合并以这种方式生成的适配器，启用说明，演示或检索到的段落可以无缝合并而无需重新处理长时间。至关重要的是，这种方法产生了三个好处：较低的推理成本，对长篇文化不稳定性的稳健性，并在输入超过模型的上下文窗口时建立了原则的解决方案。此外，Compas以可逆的方式将信息编码为适配器参数，从而通过解码器恢复输入上下文，从而促进安全性和安全性。关于多种选择和提取性问题答案任务的经验结果表明，Compas的表现优于ICL和先前的基于发生器的方法，尤其是在扩展到更多输入时。我们的工作确立了可复合的适配器生成，作为扩展LLM部署的实用和有效替代方案。

Title: When Does Reasoning Matter? A Controlled Study of Reasoning's Contribution to Model Performance

• Authors: Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Kevin El-Haddad, Céline Hudelot, Pierre Colombo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22193
• Pdf URL: https://arxiv.org/pdf/2509.22193
• Copy Paste: [[2509.22193]] When Does Reasoning Matter? A Controlled Study of Reasoning's Contribution to Model Performance(https://arxiv.org/abs/2509.22193)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) with reasoning capabilities have achieved state-of-the-art performance on a wide range of tasks. Despite its empirical success, the tasks and model scales at which reasoning becomes effective, as well as its training and inference costs, remain underexplored. In this work, we rely on a synthetic data distillation framework to conduct a large-scale supervised study. We compare Instruction Fine-Tuning (IFT) and reasoning models of varying sizes, on a wide range of math-centric and general-purpose tasks, evaluating both multiple-choice and open-ended formats. Our analysis reveals that reasoning consistently improves model performance, often matching or surpassing significantly larger IFT systems. Notably, while IFT remains Pareto-optimal in training and inference costs, reasoning models become increasingly valuable as model size scales, overcoming IFT performance limits on reasoning-intensive and open-ended tasks.
• 摘要：具有推理能力的大型语言模型（LLM）在各种任务上都达到了最先进的表现。尽管取得了经验的成功，但推理变得有效的任务和模型量表以及其培训和推理成本，仍然没有得到充实的态度。在这项工作中，我们依靠合成数据蒸馏框架进行大规模监督研究。我们在各种以数学为中心和通用的任务上比较了指令微调（IFT）和不同尺寸的推理模型，评估了多项选择和开放式格式。我们的分析表明，推理一致地改善了模型性能，通常匹配或超过更大的IFT系统。值得注意的是，尽管IFT在培训和推理成本方面仍然是帕累托最佳选择，但随着模型尺寸尺度，推理模型变得越来越有价值，克服了IFT性能限制了推理密集型和开放式任务。

Title: The Outputs of Large Language Models are Meaningless

• Authors: Anandi Hattiangadi, Anders J. Schoubye
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22206
• Pdf URL: https://arxiv.org/pdf/2509.22206
• Copy Paste: [[2509.22206]] The Outputs of Large Language Models are Meaningless(https://arxiv.org/abs/2509.22206)
• Keywords: language model, llm
• Abstract: In this paper, we offer a simple argument for the conclusion that the outputs of large language models (LLMs) are meaningless. Our argument is based on two key premises: (a) that certain kinds of intentions are needed in order for LLMs' outputs to have literal meanings, and (b) that LLMs cannot plausibly have the right kinds of intentions. We defend this argument from various types of responses, for example, the semantic externalist argument that deference can be assumed to take the place of intentions and the semantic internalist argument that meanings can be defined purely in terms of intrinsic relations between concepts, such as conceptual roles. We conclude the paper by discussing why, even if our argument is sound, the outputs of LLMs nevertheless seem meaningful and can be used to acquire true beliefs and even knowledge.
• 摘要：在本文中，我们为结论提供了一个简单的论点，即大语言模型（LLMS）的输出毫无意义。我们的论点基于两个关键前提：（a）为了使LLMS的输出具有字面意义，需要某些类型的意图，并且（b）LLMS不能合理地具有正确的意图。我们从各种类型的响应中辩护这种论点，例如，可以假定尊重的语义外部主义论点，即可以假定意图的代替和语义内部主义论点，即可以纯粹根据概念之间的内在关系（例如概念角色）来定义含义。我们通过讨论为什么即使我们的论点是正确的，LLM的输出似乎有意义，可以用来获得真正的信念甚至知识，从而结束了本文。

Title: Question-Driven Analysis and Synthesis: Building Interpretable Thematic Trees with LLMs for Text Clustering and Controllable Generation

• Authors: Tiago Fernandes Tavares
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22211
• Pdf URL: https://arxiv.org/pdf/2509.22211
• Copy Paste: [[2509.22211]] Question-Driven Analysis and Synthesis: Building Interpretable Thematic Trees with LLMs for Text Clustering and Controllable Generation(https://arxiv.org/abs/2509.22211)
• Keywords: language model, llm, prompt
• Abstract: Unsupervised analysis of text corpora is challenging, especially in data-scarce domains where traditional topic models struggle. While these models offer a solution, they typically describe clusters with lists of keywords that require significant manual effort to interpret and often lack semantic coherence. To address this critical interpretability gap, we introduce Recursive Thematic Partitioning (RTP), a novel framework that leverages Large Language Models (LLMs) to interactively build a binary tree. Each node in the tree is a natural language question that semantically partitions the data, resulting in a fully interpretable taxonomy where the logic of each cluster is explicit. Our experiments demonstrate that RTP's question-driven hierarchy is more interpretable than the keyword-based topics from a strong baseline like BERTopic. Furthermore, we establish the quantitative utility of these clusters by showing they serve as powerful features in downstream classification tasks, particularly when the data's underlying themes correlate with the task labels. RTP introduces a new paradigm for data exploration, shifting the focus from statistical pattern discovery to knowledge-driven thematic analysis. Furthermore, we demonstrate that the thematic paths from the RTP tree can serve as structured, controllable prompts for generative models. This transforms our analytical framework into a powerful tool for synthesis, enabling the consistent imitation of specific characteristics discovered in the source corpus.
• 摘要：对文本语料库的无监督分析具有挑战性，尤其是在传统主题模型挣扎的数据筛选领域。尽管这些模型提供了一种解决方案，但它们通常用重要的手动努力来解释并经常缺乏语义连贯性的关键字列表来描述群集。为了解决这个关键的可解释性差距，我们介绍了递归主题分区（RTP），这是一个利用大型语言模型（LLMS）进行交互构建二进制树的新颖框架。树上的每个节点都是一个自然的语言问题，可以将数据划分数据，从而使每个群集的逻辑明确地进行了完全可解释的分类法。我们的实验表明，RTP的问题驱动的层次结构比Bertopic这样的强基线的基于关键字的主题更容易解释。此外，我们通过证明它们是下游分类任务中的强大功能来建立这些群集的定量实用性，尤其是当数据基础主题与任务标签相关时。 RTP引入了用于数据探索的新范式，将重点从统计模式发现转移到知识驱动的主题分析。此外，我们证明了RTP树的主题路径可以用作生成模型的结构化，可控制的提示。这将我们的分析框架转变为一个有力的综合工具，从而使能够对源语料库中发现的特定特征的一致模仿。

Title: StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs

• Authors: Yuhan Song, Linhao Zhang, Chuhan Wu, Aiwei Liu, Wei Jia, Houfeng Wang, Xiao Zhou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22220
• Pdf URL: https://arxiv.org/pdf/2509.22220
• Copy Paste: [[2509.22220]] StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs(https://arxiv.org/abs/2509.22220)
• Keywords: llm
• Abstract: Prevalent semantic speech tokenizers, designed to capture linguistic content, are surprisingly fragile. We find they are not robust to meaning-irrelevant acoustic perturbations; even at high Signal-to-Noise Ratios (SNRs) where speech is perfectly intelligible, their output token sequences can change drastically, increasing the learning burden for downstream LLMs. This instability stems from two flaws: a brittle single-path quantization architecture and a distant training signal indifferent to intermediate token stability. To address this, we introduce StableToken, a tokenizer that achieves stability through a consensus-driven mechanism. Its multi-branch architecture processes audio in parallel, and these representations are merged via a powerful bit-wise voting mechanism to form a single, stable token sequence. StableToken sets a new state-of-the-art in token stability, drastically reducing Unit Edit Distance (UED) under diverse noise conditions. This foundational stability translates directly to downstream benefits, significantly improving the robustness of SpeechLLMs on a variety of tasks.
• 摘要：旨在捕获语言内容的普遍语义语音引导者令人惊讶地脆弱。我们发现它们对意义上的声学扰动并不强大。即使在语音完全可理解的高信噪比（SNR）的高信噪比（SNR）中，它们的输出令牌序列也会巨大变化，从而增加了下游LLM的学习负担。这种不稳定性源于两个缺陷：一个脆弱的单路量化结构和遥远的训练信号对中间令牌稳定性无关。为了解决这个问题，我们介绍了stabletoken，这是一种通过共识驱动的机制实现稳定性的令牌。它的多分支体系结构在并行的音频过程中，这些表示形式通过强大的位投票机制合并，以形成单个稳定的令牌序列。 StableToken在多种噪声条件下设定了一个新的象征稳定性，大幅度降低了单元编辑距离（UED）。这种基本稳定性直接转化为下游福利，从而显着改善了各种任务上语音的鲁棒性。

Title: Thinking in Many Modes: How Composite Reasoning Elevates Large Language Model Performance with Limited Data

• Authors: Zishan Ahmad, Saisubramaniam Gopalakrishnan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22224
• Pdf URL: https://arxiv.org/pdf/2509.22224
• Copy Paste: [[2509.22224]] Thinking in Many Modes: How Composite Reasoning Elevates Large Language Model Performance with Limited Data(https://arxiv.org/abs/2509.22224)
• Keywords: language model, llm, chain-of-thought
• Abstract: Large Language Models (LLMs), despite their remarkable capabilities, rely on singular, pre-dominant reasoning paradigms, hindering their performance on intricate problems that demand diverse cognitive strategies. To address this, we introduce Composite Reasoning (CR), a novel reasoning approach empowering LLMs to dynamically explore and combine multiple reasoning styles like deductive, inductive, and abductive for more nuanced problem-solving. Evaluated on scientific and medical question-answering benchmarks, our approach outperforms existing baselines like Chain-of-Thought (CoT) and also surpasses the accuracy of DeepSeek-R1 style reasoning (SR) capabilities, while demonstrating superior sample efficiency and adequate token usage. Notably, CR adaptively emphasizes domain-appropriate reasoning styles. It prioritizes abductive and deductive reasoning for medical question answering, but shifts to causal, deductive, and inductive methods for scientific reasoning. Our findings highlight that by cultivating internal reasoning style diversity, LLMs acquire more robust, adaptive, and efficient problem-solving abilities.
• 摘要：大型语言模型（LLMS）尽管具有显着的功能，但仍依赖于单一的，先前的推理范式，阻碍了他们在需要多种认知策略的复杂问题上的表现。为了解决这个问题，我们引入了复合推理（CR），这是一种新颖的推理方法，赋予LLMS能够动态探索和结合多种推理风格，例如演绎，归纳和绑架，以更细微的问题解决问题。在评估科学和医学提问的基准测试方面，我们的方法的表现优于现有基准（COT）（COT）（COT），并且超过了DeepSeek-R1样式推理（SR）功能的准确性，同时证明了出色的样本效率和足够的标记使用。值得注意的是，CR自适应地强调了适合领域的推理方式。它优先考虑绑架和演绎推理的医学问题回答，但转向了科学推理的因果，演绎和归纳方法。我们的发现强调，通过培养内部推理风格的多样性，LLMS可以获得更健壮，适应性和有效的解决问题的能力。

Title: In Their Own Words: Reasoning Traces Tailored for Small Models Make Them Better Reasoners

• Authors: Jaehoon Kim, Kwangwook Seo, Dongha Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22230
• Pdf URL: https://arxiv.org/pdf/2509.22230
• Copy Paste: [[2509.22230]] In Their Own Words: Reasoning Traces Tailored for Small Models Make Them Better Reasoners(https://arxiv.org/abs/2509.22230)
• Keywords: language model
• Abstract: Transferring reasoning capabilities from larger language models to smaller ones through supervised fine-tuning often fails counterintuitively, with performance degrading despite access to high-quality teacher demonstrations. We identify that this failure stems from distributional misalignment: reasoning traces from larger models contain tokens that are low probability under the student's distribution, exceeding the internal representation capacity of smaller architectures and creating learning barriers rather than helpful guidance. We propose Reverse Speculative Decoding (RSD), a mechanism for generating student-friendly reasoning traces in which the teacher model proposes candidate tokens but the student model determines acceptance based on its own probability distributions, filtering low probability tokens. When applied to Qwen3-0.6B, direct distillation of s1K-1.1 reasoning trace data degrades average performance across major reasoning benchmarks by 20.5\%, while the same model trained on RSD-generated reasoning traces achieves meaningful improvements of 4.9\%. Our analysis reveals that low probability tokens constitute the critical bottleneck in reasoning ability transfer. However, cross-model experiments demonstrate that RSD traces are model-specific rather than universally applicable, indicating that distributional alignment must be tailored for each student architecture's unique internal representation.
• 摘要：通过监督的微调将推理能力从较大的语言模型转移到较小的语言模型通常会违反直觉，尽管可以访问高质量的老师演示，但表现会退化。我们确定这种故障源于分配未对准：​​来自较大模型的推理迹线包含在学生分布下概率较低的令牌，超过了较小的体系结构的内部表示能力，并创建学习障碍而不是有用的指导。我们提出了反向投机解码（RSD），这是一种生成对学生友好的推理痕迹的机制，其中教师模型提出了候选代币，但学生模型根据其自身的概率分布来确定接受，从而过滤较低的概率令牌。当应用于QWEN3-0.6B时，S1K-1.1的直接蒸馏推理痕量数据将主要推理基准的平均性能降低了20.5 \％，而对RSD生成的推理痕迹进行培训的相同模型可实现4.9 \％的有意义的改进。我们的分析表明，低概率令牌构成了推理能力传递中的关键瓶颈。但是，跨模型实验表明，RSD痕迹是特定于模型的，而不是普遍适用的，这表明必须针对每个学生架构的独特内部表示。

Title: FeatBench: Evaluating Coding Agents on Feature Implementation for Vibe Coding

• Authors: Haorui Chen, Chengze Li, Jia Li
• Subjects: cs.CL, cs.AI, cs.SE
• Abstract URL: https://arxiv.org/abs/2509.22237
• Pdf URL: https://arxiv.org/pdf/2509.22237
• Copy Paste: [[2509.22237]] FeatBench: Evaluating Coding Agents on Feature Implementation for Vibe Coding(https://arxiv.org/abs/2509.22237)
• Keywords: language model, llm, prompt, agent
• Abstract: The rapid advancement of Large Language Models (LLMs) has given rise to a novel software development paradigm known as "vibe coding," where users interact with coding agents through high-level natural language. However, existing evaluation benchmarks for code generation inadequately assess an agent's vibe coding capabilities. Existing benchmarks are misaligned, as they either require code-level specifications or focus narrowly on issue-solving, neglecting the critical scenario of feature implementation within the vibe coding paradiam. To address this gap, we propose FeatBench, a novel benchmark for vibe coding that focuses on feature implementation. Our benchmark is distinguished by several key features: 1. Pure Natural Language Prompts. Task inputs consist solely of abstract natural language descriptions, devoid of any code or structural hints. 2. A Rigorous & Evolving Data Collection Process. FeatBench is built on a multi-level filtering pipeline to ensure quality and a fully automated pipeline to evolve the benchmark, mitigating data contamination. 3. Comprehensive Test Cases. Each task includes Fail-to-Pass (F2P) and Pass-to-Pass (P2P) tests to verify correctness and prevent regressions. 4. Diverse Application Domains. The benchmark includes repositories from diverse domains to ensure it reflects real-world scenarios. We evaluate two state-of-the-art agent frameworks with four leading LLMs on FeatBench. Our evaluation reveals that feature implementation within the vibe coding paradigm is a significant challenge, with the highest success rate of only 29.94%. Our analysis also reveals a tendency for "aggressive implementation," a strategy that paradoxically leads to both critical failures and superior software design. We release FeatBench, our automated collection pipeline, and all experimental results to facilitate further community research.
• 摘要：大型语言模型（LLMS）的快速发展引起了一种新型的软件开发范式，称为“ Vibe编码”，在该范式中，用户通过高级自然语言与编码代理进行交互。但是，代码生成的现有评估基准不足地评估了代理的氛围编码功能。现有的基准是未对准的，因为它们要么需要代码级规格，要么将重点放在解决问题上，从而忽略了Vibe编码Paradiam中特征实现的关键方案。为了解决这一差距，我们提出了Feat Bench，这是一种用于Vibe编码的新基准，重点是功能实现。我们的基准分为几个关键特征：1。纯自然语言提示。任务输入仅由抽象的自然语言描述组成，没有任何代码或结构提示。 2。严格而不断发展的数据收集过程。 Feat Bench建立在多层过滤管道上，以确保质量和全自动管道，以发展基准测试，从而减轻数据污染。 3。全面的测试用例。每个任务都包括失败通道（F2P）和传递通道（P2P）测试，以验证正确性并防止回归。 4。各种应用领域。基准包括来自不同领域的存储库，以确保它反映现实世界的情况。我们在Feat Bench上使用四个领先的LLMS评估了两个最先进的代理框架。我们的评估表明，在Vibe编码范式中的特征实现是一个重大挑战，最高成功率仅为29.94％。我们的分析还揭示了“积极实施”的趋势，这种策略矛盾地导致关键故障和出色的软件设计。我们发布了Feat Bench，自动化收集管道以及所有实验结果，以促进进一步的社区研究。

Title: FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction

• Authors: Yuan Ge, Saihan Chen, Jingqi Xiao, Xiaoqian Liu, Tong Xiao, Yan Xiang, Zhengtao Yu, Jingbo Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22243
• Pdf URL: https://arxiv.org/pdf/2509.22243
• Copy Paste: [[2509.22243]] FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction(https://arxiv.org/abs/2509.22243)
• Keywords: language model, llm
• Abstract: Full-Duplex Speech-to-Speech Large Language Models (LLMs) are foundational to natural human-computer interaction, enabling real-time spoken dialogue systems. However, benchmarking and modeling these models remains a fundamental challenge. We introduce FLEXI, the first benchmark for full-duplex LLM-human spoken interaction that explicitly incorporates model interruption in emergency scenarios. FLEXI systematically evaluates the latency, quality, and conversational effectiveness of real-time dialogue through six diverse human-LLM interaction scenarios, revealing significant gaps between open source and commercial models in emergency awareness, turn terminating, and interaction latency. Finally, we suggest that next token-pair prediction offers a promising path toward achieving truly seamless and human-like full-duplex interaction.
• 摘要：完整的语音到语音大语模型（LLM）是自然人类计算机互动的基础，实现了实时口语对话系统。但是，基准和建模这些模型仍然是一个基本挑战。我们介绍了FlexI，这是全双工LLM-Human口语互动的第一个基准，该基准明确将模型中断纳入紧急情况。 FlexI系统地评估了通过六种不同的人类交互场景实时对话的潜伏，质量和对话效果，从而揭示了紧急意识，转弯终止和交互潜伏期中开源和商业模型之间的显着差距。最后，我们建议下一步的标记预测为实现真正无缝和类似人类的全双工互动提供了有前途的途径。

Title: Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliance

• Authors: Wenbin Hu, Huihao Jing, Haochen Shi, Haoran Li, Yangqiu Song
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22250
• Pdf URL: https://arxiv.org/pdf/2509.22250
• Copy Paste: [[2509.22250]] Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliance(https://arxiv.org/abs/2509.22250)
• Keywords: language model, llm
• Abstract: The proliferation of Large Language Models (LLMs) has demonstrated remarkable capabilities, elevating the critical importance of LLM safety. However, existing safety methods rely on ad-hoc taxonomy and lack a rigorous, systematic protection, failing to ensure safety for the nuanced and complex behaviors of modern LLM systems. To address this problem, we solve LLM safety from legal compliance perspectives, named safety compliance. In this work, we posit relevant established legal frameworks as safety standards for defining and measuring safety compliance, including the EU AI Act and GDPR, which serve as core legal frameworks for AI safety and data security in Europe. To bridge the gap between LLM safety and legal compliance, we first develop a new benchmark for safety compliance by generating realistic LLM safety scenarios seeded with legal statutes. Subsequently, we align Qwen3-8B using Group Policy Optimization (GRPO) to construct a safety reasoner, Compliance Reasoner, which effectively aligns LLMs with legal standards to mitigate safety risks. Our comprehensive experiments demonstrate that the Compliance Reasoner achieves superior performance on the new benchmark, with average improvements of +10.45% for the EU AI Act and +11.85% for GDPR.
• 摘要：大型语言模型（LLM）的扩散表现出了显着的功能，从而提高了LLM安全的重要性。但是，现有的安全方法依赖于临时分类法，并且缺乏严格的系统保护，无法确保对现代LLM系统的细微差别和复杂行为的安全。为了解决这个问题，我们从法律合规角度解决了LLM安全性，名为“安全合规”。在这项工作中，我们认为相关的法律框架是定义和衡量安全合规性的安全标准，包括《欧盟AI法案》和GDPR，它们是欧洲AI安全和数据安全的核心法律框架。为了弥合LLM安全性与法律合规性之间的差距，我们首先通过产生与法律法规播种的现实LLM安全方案，为安全合规性开发新的基准。随后，我们使用小组策略优化（GRPO）来对齐QWEN3-8B来构建安全推理者合规性推理者，该问题有效地使LLM与法律标准相一致，以减轻安全风险。我们的全面实验表明，合规性推理者在新的基准上取得了出色的性能，欧盟AI法案的平均提高 +10.45％，而GDPR的平均提高了 +11.85％。

Title: Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs

• Authors: Yifang Zhang, Pengfei Duan, Yiwen Yang, Shengwu Xiong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22251
• Pdf URL: https://arxiv.org/pdf/2509.22251
• Copy Paste: [[2509.22251]] Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs(https://arxiv.org/abs/2509.22251)
• Keywords: language model, llm, hallucination
• Abstract: Currently, the main approach for Large Language Models (LLMs) to tackle the hallucination issue is incorporating Knowledge Graphs(KGs).However, LLMs typically treat KGs as plain text, extracting only semantic information and limiting their use of the crucial structural aspects of KGs. Another challenge is the gap between the embedding spaces of KGs encoders and LLMs text embeddings, which hinders the effective integration of structured knowledge. To overcome these obstacles, we put forward the SSKG-LLM, an innovative model architecture that is designed to efficiently integrate both the Structural and Semantic information of KGs into the reasoning processes of LLMs. SSKG-LLM incorporates the Knowledge Graph Retrieval (KGR) module and the Knowledge Graph Encoding (KGE) module to preserve semantics while utilizing structure. Then, the Knowledge Graph Adaptation (KGA) module is incorporated to enable LLMs to understand KGs embeddings. We conduct extensive experiments and provide a detailed analysis to explore how incorporating the structural information of KGs can enhance the factual reasoning abilities of LLMs. Our code are available at this https URL.
• 摘要：当前，解决幻觉问题的大型语言模型（LLM）的主要方法是合并知识图（kgs）。另一个挑战是KGS编码器的嵌入空间与LLMS文本嵌入之间的差距，这阻碍了结构化知识的有效整合。为了克服这些障碍，我们提出了SSKG-LLM，这是一种创新的模型体系结构，旨在将KGS的结构和语义信息有效地整合到LLMS的推理过程中。 SSKG-LLM结合了知识图检索（KGR）模块和知识图编码（KGE）模块，以在利用结构时保留语义。然后，合并了知识图适应（KGA）模块，以使LLMS能够理解KGS嵌入。我们进行了广泛的实验，并提供了详细的分析，以探索合并KGS结构信息如何增强LLM的事实推理能力。我们的代码可在此HTTPS URL上找到。

Title: Advancing Natural Language Formalization to First Order Logic with Fine-tuned LLMs

• Authors: Felix Vossel, Till Mossakowski, Björn Gehrke
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22338
• Pdf URL: https://arxiv.org/pdf/2509.22338
• Copy Paste: [[2509.22338]] Advancing Natural Language Formalization to First Order Logic with Fine-tuned LLMs(https://arxiv.org/abs/2509.22338)
• Keywords: gpt, llm
• Abstract: Automating the translation of natural language to first-order logic (FOL) is crucial for knowledge representation and formal methods, yet remains challenging. We present a systematic evaluation of fine-tuned LLMs for this task, comparing architectures (encoder-decoder vs. decoder-only) and training strategies. Using the MALLS and Willow datasets, we explore techniques like vocabulary extension, predicate conditioning, and multilingual training, introducing metrics for exact match, logical equivalence, and predicate alignment. Our fine-tuned Flan-T5-XXL achieves 70% accuracy with predicate lists, outperforming GPT-4o and even the DeepSeek-R1-0528 model with CoT reasoning ability as well as symbolic systems like ccg2lambda. Key findings show: (1) predicate availability boosts performance by 15-20%, (2) T5 models surpass larger decoder-only LLMs, and (3) models generalize to unseen logical arguments (FOLIO dataset) without specific training. While structural logic translation proves robust, predicate extraction emerges as the main bottleneck.
• 摘要：自动化自然语言为一阶逻辑（FOL）对知识表示和形式方法至关重要，但仍然具有挑战性。我们对此任务进行了微调LLM的系统评估，比较体系结构（编码器与仅解码器）和培训策略。使用购物中心和柳树数据集，我们探索了词汇扩展，谓词调节和多语言训练等技术，引入了确切匹配，逻辑等效性和谓词比对的指标。我们的微调Flan-T5-XXL具有谓词列表，超过GPT-4O甚至具有CCG2LAMBDA（例如CCG2LAMBDA）的符号系统的DeepSeek-R1-0528模型，达到70％的精度。关键发现显示：（1）谓词可用性使性能提高了15-20％，（2）T5模型超过较大的解码器llms，（3）模型概括了未看到的逻辑参数（Folio DataSet），而无需进行特定的培训。虽然结构逻辑翻译证明了鲁棒，但谓词提取作为主要瓶颈出现。

Title: Transformers Can Learn Connectivity in Some Graphs but Not Others

• Authors: Amit Roy, Abulhair Saparov
• Subjects: cs.CL, cs.AI, cs.LG, cs.LO
• Abstract URL: https://arxiv.org/abs/2509.22343
• Pdf URL: https://arxiv.org/pdf/2509.22343
• Copy Paste: [[2509.22343]] Transformers Can Learn Connectivity in Some Graphs but Not Others(https://arxiv.org/abs/2509.22343)
• Keywords: language model, llm, prompt
• Abstract: Reasoning capability is essential to ensure the factual correctness of the responses of transformer-based Large Language Models (LLMs), and robust reasoning about transitive relations is instrumental in many settings, such as causal inference. Hence, it is essential to investigate the capability of transformers in the task of inferring transitive relations (e.g., knowing A causes B and B causes C, then A causes C). The task of inferring transitive relations is equivalent to the task of connectivity in directed graphs (e.g., knowing there is a path from A to B, and there is a path from B to C, then there is a path from A to C). Past research focused on whether transformers can learn to infer transitivity from in-context examples provided in the input prompt. However, transformers' capability to infer transitive relations from training examples and how scaling affects the ability is unexplored. In this study, we seek to answer this question by generating directed graphs to train transformer models of varying sizes and evaluate their ability to infer transitive relations for various graph sizes. Our findings suggest that transformers are capable of learning connectivity on "grid-like'' directed graphs where each node can be embedded in a low-dimensional subspace, and connectivity is easily inferable from the embeddings of the nodes. We find that the dimensionality of the underlying grid graph is a strong predictor of transformers' ability to learn the connectivity task, where higher-dimensional grid graphs pose a greater challenge than low-dimensional grid graphs. In addition, we observe that increasing the model scale leads to increasingly better generalization to infer connectivity over grid graphs. However, if the graph is not a grid graph and contains many disconnected components, transformers struggle to learn the connectivity task, especially when the number of components is large.
• 摘要：推理能力对于确保基于变形金刚的大型语言模型（LLM）的响应的事实正确性以及对传递关系的强大推理至关重要。因此，必须研究变压器在推断传递关系任务中的能力（例如，知道a会导致b和b会导致c，然后a会导致c）。推断传递关系的任务等于有向图中的连接性任务（例如，知道从a到b的路径，并且有从b到c的路径，然后有一个从a到c的路径）。过去的研究重点是变形金刚是否可以从输入提示中提供的范围内的示例中学习推断出传递性。但是，变形金刚从训练示例中推断出传递关系的能力以及缩放如何影响能力的能力。在这项研究中，我们试图通过生成有向图的图形来训练各种大小的变压器模型，并评估其推断各种图形大小的传递关系的能力来回答这个问题。 Our findings suggest that transformers are capable of learning connectivity on "grid-like'' directed graphs where each node can be embedded in a low-dimensional subspace, and connectivity is easily inferable from the embeddings of the nodes. We find that the dimensionality of the underlying grid graph is a strong predictor of transformers' ability to learn the connectivity task, where higher-dimensional grid graphs pose a greater challenge than low-dimensional网格图。此外，我们观察到，增加模型量表会导致越来越多的概括，以推断网格图上的连接性。

Title: The InviTE Corpus: Annotating Invectives in Tudor English Texts for Computational Modeling

• Authors: Sophie Spliethoff, Sanne Hoeken, Silke Schwandt, Sina Zarrieß, Özge Alaçam
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22345
• Pdf URL: https://arxiv.org/pdf/2509.22345
• Copy Paste: [[2509.22345]] The InviTE Corpus: Annotating Invectives in Tudor English Texts for Computational Modeling(https://arxiv.org/abs/2509.22345)
• Keywords: language model, llm, prompt
• Abstract: In this paper, we aim at the application of Natural Language Processing (NLP) techniques to historical research endeavors, particularly addressing the study of religious invectives in the context of the Protestant Reformation in Tudor England. We outline a workflow spanning from raw data, through pre-processing and data selection, to an iterative annotation process. As a result, we introduce the InviTE corpus -- a corpus of almost 2000 Early Modern English (EModE) sentences, which are enriched with expert annotations regarding invective language throughout 16th-century England. Subsequently, we assess and compare the performance of fine-tuned BERT-based models and zero-shot prompted instruction-tuned large language models (LLMs), which highlights the superiority of models pre-trained on historical data and fine-tuned to invective detection.
• 摘要：在本文中，我们旨在将自然语言处理（NLP）技术应用于历史研究努力，尤其是在英格兰都铎·英格兰的新教改革的背景下解决宗教投资的研究。我们概述了从原始数据，通过预处理和数据选择到迭代注释过程的工作流程。结果，我们介绍了邀请语料库 - 近2000个现代英语（EMODE）句子的语料库，这些句子丰富了有关整个16世纪英格兰的邀请语言的专家注释。随后，我们评估和比较了基于BERT的模型的性能和零射击促使指导调节的大语模型（LLMS），该模型（LLMS）突出了对历史数据进行预训练的模型的优越性，并进行了精细调整以进行启发性检测。

Title: CHRONOBERG: Capturing Language Evolution and Temporal Awareness in Foundation Models

• Authors: Niharika Hegde, Subarnaduti Paul, Lars Joel-Frey, Manuel Brack, Kristian Kersting, Martin Mundt, Patrick Schramowski
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22360
• Pdf URL: https://arxiv.org/pdf/2509.22360
• Copy Paste: [[2509.22360]] CHRONOBERG: Capturing Language Evolution and Temporal Awareness in Foundation Models(https://arxiv.org/abs/2509.22360)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) excel at operating at scale by leveraging social media and various data crawled from the web. Whereas existing corpora are diverse, their frequent lack of long-term temporal structure may however limit an LLM's ability to contextualize semantic and normative evolution of language and to capture diachronic variation. To support analysis and training for the latter, we introduce CHRONOBERG, a temporally structured corpus of English book texts spanning 250 years, curated from Project Gutenberg and enriched with a variety of temporal annotations. First, the edited nature of books enables us to quantify lexical semantic change through time-sensitive Valence-Arousal-Dominance (VAD) analysis and to construct historically calibrated affective lexicons to support temporally grounded interpretation. With the lexicons at hand, we demonstrate a need for modern LLM-based tools to better situate their detection of discriminatory language and contextualization of sentiment across various time-periods. In fact, we show how language models trained sequentially on CHRONOBERG struggle to encode diachronic shifts in meaning, emphasizing the need for temporally aware training and evaluation pipelines, and positioning CHRONOBERG as a scalable resource for the study of linguistic change and temporal generalization. Disclaimer: This paper includes language and display of samples that could be offensive to readers. Open Access: Chronoberg is available publicly on HuggingFace at ( this https URL). Code is available at (this https URL).
• 摘要：大型语言模型（LLMS）通过利用社交媒体和从网络上拖走的各种数据来大规模运行。尽管现有的语料库是多种多样的，但它们经常缺乏长期时间结构可能会限制LLM在语义和规范性语言演变上的能力，并捕获惯性变化。为了支持后者的分析和培训，我们介绍了Chronoberg，Chronoberg是一个临时结构化的英语书籍文本语料库，涵盖了250年，策划了Gutenberg Project，并充满了各种时间注释。首先，书籍的编辑性质使我们能够通过时间敏感的价值占主导地位（VAD）分析来量化词汇语义变化，并构建历史上校准的情感词典以支持时间基础的解释。借助手头的词典，我们表明了需要现代化LLM的工具，以更好地将他们对歧视性语言的发现和对各个时间周期的情感化的发现。实际上，我们展示了如何在Chronoberg依次训练的语言模型，以编码意义上的惯性变化，强调需要进行时间意识的培训和评估管道，并将Chronoberg定位为研究语言变化和时间概括的可扩展资源。免责声明：本文包括可能对读者冒犯的样本的语言和显示。开放访问：Chronoberg可在（此HTTPS URL）上的HuggingFace上公开使用。代码可在（此HTTPS URL）上找到。

Title: Exploratory Semantic Reliability Analysis of Wind Turbine Maintenance Logs using Large Language Models

• Authors: Max Malyi, Jonathan Shek, Andre Biscaya
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22366
• Pdf URL: https://arxiv.org/pdf/2509.22366
• Copy Paste: [[2509.22366]] Exploratory Semantic Reliability Analysis of Wind Turbine Maintenance Logs using Large Language Models(https://arxiv.org/abs/2509.22366)
• Keywords: language model, llm
• Abstract: A wealth of operational intelligence is locked within the unstructured free-text of wind turbine maintenance logs, a resource largely inaccessible to traditional quantitative reliability analysis. While machine learning has been applied to this data, existing approaches typically stop at classification, categorising text into predefined labels. This paper addresses the gap in leveraging modern large language models (LLMs) for more complex reasoning tasks. We introduce an exploratory framework that uses LLMs to move beyond classification and perform deep semantic analysis. We apply this framework to a large industrial dataset to execute four analytical workflows: failure mode identification, causal chain inference, comparative site analysis, and data quality auditing. The results demonstrate that LLMs can function as powerful "reliability co-pilots," moving beyond labelling to synthesise textual information and generate actionable, expert-level hypotheses. This work contributes a novel and reproducible methodology for using LLMs as a reasoning tool, offering a new pathway to enhance operational intelligence in the wind energy sector by unlocking insights previously obscured in unstructured data.
• 摘要：大量的操作智能被锁定在风力涡轮机维护日志的非结构化自由文本中，这是传统的定量可靠性分析无法访问的资源。在将机器学习应用于此数据的同时，现有方法通常在分类时停止，将文本分类为预定义的标签。本文解决了利用现代大型语言模型（LLM）进行更复杂的推理任务的差距。我们介绍了一个探索性框架，该探索框架使用LLM超越分类并进行深度语义分析。我们将此框架应用于大型工业数据集，以执行四个分析工作流程：故障模式识别，因果链推断，比较站点分析和数据质量审核。结果表明，LLM可以充当强大的“可靠性副驾驶”，而不是标记，以综合文本信息并生成可行的专家级别的假设。这项工作为使用LLM作为推理工具提供了一种新颖而可重复的方法，提供了一种新的途径来通过解锁以前在非结构化数据中被掩盖的见解来增强风能领域的运行智能。

Title: What Is The Political Content in LLMs' Pre- and Post-Training Data?

• Authors: Tanise Ceron, Dmitry Nikolaev, Dominik Stammbach, Debora Nozza
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2509.22367
• Pdf URL: https://arxiv.org/pdf/2509.22367
• Copy Paste: [[2509.22367]] What Is The Political Content in LLMs' Pre- and Post-Training Data?(https://arxiv.org/abs/2509.22367)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are known to generate politically biased text, yet how such biases arise remains unclear. A crucial step toward answering this question is the analysis of training data, whose political content remains largely underexplored in current LLM research. To address this gap, we present in this paper an analysis of the pre- and post-training corpora of OLMO2, the largest fully open-source model released together with its complete dataset. From these corpora, we draw large random samples, automatically annotate documents for political orientation, and analyze their source domains and content. We then assess how political content in the training data correlates with models' stance on specific policy issues. Our analysis shows that left-leaning documents predominate across datasets, with pre-training corpora containing significantly more politically engaged content than post-training data. We also find that left- and right-leaning documents frame similar topics through distinct values and sources of legitimacy. Finally, the predominant stance in the training data strongly correlates with models' political biases when evaluated on policy issues. These findings underscore the need to integrate political content analysis into future data curation pipelines as well as in-depth documentation of filtering strategies for transparency.
• 摘要：众所周知，大型语言模型（LLM）会产生政治上有偏见的文本，但是这种偏见的产生尚不清楚。回答这个问题的关键步骤是对培训数据的分析，其政治内容在当前的LLM研究中仍未得到充分影响。为了解决这一差距，我们在本文中介绍了OLMO2的培训前和培训后Corpora，这是最大的完全开源模型及其完整数据集。从这些语料库中，我们绘制大量随机样本，自动注释文档以进行政治取向，并分析其来源领域和内容。然后，我们评估培训数据中的政治内容如何与模型对特定政策问题的立场相关。我们的分析表明，左倾文档遍及整个数据集，而培训前的Corpora包含比培训后数据更多的政治参与内容。我们还发现，左派和右倾文档通过合法性的不同价值和来源构成了相似的主题。最后，在对政策问题进行评估时，培训数据中的主要立场与模型的政治偏见密切相关。这些发现强调了将政治内容分析整合到未来数据策划管道中的必要性以及对透明度过滤策略的深入文档。

Title: Chimera: Diagnosing Shortcut Learning in Visual-Language Understanding

• Authors: Ziheng Chi, Yifan Hou, Chenxi Pang, Shaobo Cui, Mubashara Akhtar, Mrinmaya Sachan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22437
• Pdf URL: https://arxiv.org/pdf/2509.22437
• Copy Paste: [[2509.22437]] Chimera: Diagnosing Shortcut Learning in Visual-Language Understanding(https://arxiv.org/abs/2509.22437)
• Keywords: language model
• Abstract: Diagrams convey symbolic information in a visual format rather than a linear stream of words, making them especially challenging for AI models to process. While recent evaluations suggest that vision-language models (VLMs) perform well on diagram-related benchmarks, their reliance on knowledge, reasoning, or modality shortcuts raises concerns about whether they genuinely understand and reason over diagrams. To address this gap, we introduce Chimera, a comprehensive test suite comprising 7,500 high-quality diagrams sourced from Wikipedia; each diagram is annotated with its symbolic content represented by semantic triples along with multi-level questions designed to assess four fundamental aspects of diagram comprehension: entity recognition, relation understanding, knowledge grounding, and visual reasoning. We use Chimera to measure the presence of three types of shortcuts in visual question answering: (1) the visual-memorization shortcut, where VLMs rely on memorized visual patterns; (2) the knowledge-recall shortcut, where models leverage memorized factual knowledge instead of interpreting the diagram; and (3) the Clever-Hans shortcut, where models exploit superficial language patterns or priors without true comprehension. We evaluate 15 open-source VLMs from 7 model families on Chimera and find that their seemingly strong performance largely stems from shortcut behaviors: visual-memorization shortcuts have slight impact, knowledge-recall shortcuts play a moderate role, and Clever-Hans shortcuts contribute significantly. These findings expose critical limitations in current VLMs and underscore the need for more robust evaluation protocols that benchmark genuine comprehension of complex visual inputs (e.g., diagrams) rather than question-answering shortcuts.
• 摘要：图表以视觉格式传达符号信息，而不是线性单词流，这使其对AI模型进行处理特别具有挑战性。尽管最近的评估表明，视觉模型（VLM）在与图相关的基准测试方面表现良好，但它们对知识，推理或模态捷径的依赖引起了人们对它们是否真正理解和理解图表的关注。为了解决这一差距，我们引入了Chimera，这是一个综合测试套件，其中包括7,500个来自Wikipedia的高质量图；每个图都用其符号内容注释，其语义三元组以及旨在评估图表理解的四个基本方面的多层次问题：实体识别，关系理解，知识基础和视觉推理。我们使用嵌合体来测量视觉问题回答中三种类型的快捷方式的存在：（1）视觉效率快捷方式，其中VLMS依赖于记忆的视觉模式； （2）知识回顾快捷方式，其中模型利用记忆的事实知识而不是解释图表； （3）聪明的hans捷径，模型在没有真正理解的情况下利用肤浅的语言模式或先验。我们评估了来自嵌合体上7个模型家族的15个开源VLM，发现它们看似强大的性能很大程度上是源自捷径的行为：视觉效率快捷方式具有轻微的影响，知识记录的快捷方式起着适度的作用，并且聪明的Hans捷径可显着贡献。这些发现暴露了当前VLM中的临界局限性，并强调了对更强大的评估协议的需求，这些协议基于对复杂的视觉输入（例如图表）的真实理解，而不是提出问题的快捷方式。

Title: Detecting (Un)answerability in Large Language Models with Linear Directions

• Authors: Maor Juliet Lavi, Tova Milo, Mor Geva
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22449
• Pdf URL: https://arxiv.org/pdf/2509.22449
• Copy Paste: [[2509.22449]] Detecting (Un)answerability in Large Language Models with Linear Directions(https://arxiv.org/abs/2509.22449)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) often respond confidently to questions even when they lack the necessary information, leading to hallucinated answers. In this work, we study the problem of (un)answerability detection, focusing on extractive question answering (QA) where the model should determine if a passage contains sufficient information to answer a given question. We propose a simple approach for identifying a direction in the model's activation space that captures unanswerability and uses it for classification. This direction is selected by applying activation additions during inference and measuring their impact on the model's abstention behavior. We show that projecting hidden activations onto this direction yields a reliable score for (un)answerability classification. Experiments on two open-weight LLMs and four extractive QA benchmarks show that our method effectively detects unanswerable questions and generalizes better across datasets than existing prompt-based and classifier-based approaches. Moreover, the obtained directions extend beyond extractive QA to unanswerability that stems from factors, such as lack of scientific consensus and subjectivity. Last, causal interventions show that adding or ablating the directions effectively controls the abstention behavior of the model.
• 摘要：大型语言模型（LLMS）即使缺乏必要的信息，也经常对问题做出自信的回答，从而导致幻觉答案。在这项工作中，我们研究了（联合国）回答性检测的问题，重点介绍了提取问题答案（QA），模型应确定段落是否包含足够的信息来回答给定问题。我们提出了一种简单的方法，用于识别模型激活空间中的方向，该方向捕获了无法选择性并将其用于分类。通过在推理过程中应用激活添加并测量其对模型的弃权行为的影响来选择该方向。我们表明，将隐藏的激活投射到这个方向上可以得出（联合国）答案性分类的可靠分数。在两个开放量LLM和四个提取性质量检查基准测试的实验表明，与现有的基于基于及时的及格和基于分类器的方法相比，我们的方法有效地检测到无法回答的问题并概括了数据集。此外，所获得的方向超出了提取性质量检查，到源于因素，例如缺乏科学共识和主观性。最后，因果干预表明，添加或消融方向有效地控制模型的弃戒行为。

Title: Evaluating the Limits of Large Language Models in Multilingual Legal Reasoning

• Authors: Antreas Ioannou, Andreas Shiamishis, Nora Hollenstein, Nezihe Merve Gürel
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.22472
• Pdf URL: https://arxiv.org/pdf/2509.22472
• Copy Paste: [[2509.22472]] Evaluating the Limits of Large Language Models in Multilingual Legal Reasoning(https://arxiv.org/abs/2509.22472)
• Keywords: language model, gpt, llm, prompt, chat
• Abstract: In an era dominated by Large Language Models (LLMs), understanding their capabilities and limitations, especially in high-stakes fields like law, is crucial. While LLMs such as Meta's LLaMA, OpenAI's ChatGPT, Google's Gemini, DeepSeek, and other emerging models are increasingly integrated into legal workflows, their performance in multilingual, jurisdictionally diverse, and adversarial contexts remains insufficiently explored. This work evaluates LLaMA and Gemini on multilingual legal and non-legal benchmarks, and assesses their adversarial robustness in legal tasks through character and word-level perturbations. We use an LLM-as-a-Judge approach for human-aligned evaluation. We moreover present an open-source, modular evaluation pipeline designed to support multilingual, task-diverse benchmarking of any combination of LLMs and datasets, with a particular focus on legal tasks, including classification, summarization, open questions, and general reasoning. Our findings confirm that legal tasks pose significant challenges for LLMs with accuracies often below 50% on legal reasoning benchmarks such as LEXam, compared to over 70% on general-purpose tasks like XNLI. In addition, while English generally yields more stable results, it does not always lead to higher accuracy. Prompt sensitivity and adversarial vulnerability is also shown to persist across languages. Finally, a correlation is found between the performance of a language and its syntactic similarity to English. We also observe that LLaMA is weaker than Gemini, with the latter showing an average advantage of about 24 percentage points across the same task. Despite improvements in newer LLMs, challenges remain in deploying them reliably for critical, multilingual legal applications.
• 摘要：在一个以大型语言模型（LLM）为主导的时代，了解它们的能力和局限性，尤其是在法律等高风险领域，至关重要。虽然Meta的Llama，OpenAi的Chatgpt，Google的双子座，DeepSeek和其他新兴模型等LLM越来越多地整合到法律工作流程中，但它们在多语言，管辖范围内多样化的表现，并且对抗性环境仍未得到充分的探索。这项工作评估了Llama和Gemini的多语言法律和非法律基准，并通过性格和单词级别的扰动评估其在法律任务中的对抗性鲁棒性。我们使用LLM-AS-A-A-Gudge方法进行人类对准评估。此外，我们提供了一个开源的模块化评估管道，旨在支持LLM和数据集组合的多语言，任务多样性的基准测试，并特别关注法律任务，包括分类，摘要，开放问题和一般推理。我们的研究结果证实，对于法律推理基准（例如Lexam），法律任务对LLM构成了巨大的挑战，而在诸如Lexam之类的法律推理基准中，诸如XNLI等通用任务的70％以上。此外，尽管英语通常会产生更稳定的结果，但并不总是会带来更高的准确性。迅速的灵敏度和对抗性脆弱性也显示出跨语言持续存在。最后，在语言的表现与其与英语的句法相似性之间发现了相关性。我们还观察到，美洲驼比双子座弱，后者在同一任务中的平均优势约为24个百分点。尽管有新的LLMS有所改善，但仍在为关键的多语言法律应用程序可靠地部署它们。

Title: NeLLCom-Lex: A Neural-agent Framework to Study the Interplay between Lexical Systems and Language Use

• Authors: Yuqing Zhang, Ecesu Ürker, Tessa Verhoef, Gemma Boleda, Arianna Bisazza
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22479
• Pdf URL: https://arxiv.org/pdf/2509.22479
• Copy Paste: [[2509.22479]] NeLLCom-Lex: A Neural-agent Framework to Study the Interplay between Lexical Systems and Language Use(https://arxiv.org/abs/2509.22479)
• Keywords: agent
• Abstract: Lexical semantic change has primarily been investigated with observational and experimental methods; however, observational methods (corpus analysis, distributional semantic modeling) cannot get at causal mechanisms, and experimental paradigms with humans are hard to apply to semantic change due to the extended diachronic processes involved. This work introduces NeLLCom-Lex, a neural-agent framework designed to simulate semantic change by first grounding agents in a real lexical system (e.g. English) and then systematically manipulating their communicative needs. Using a well-established color naming task, we simulate the evolution of a lexical system within a single generation, and study which factors lead agents to: (i) develop human-like naming behavior and lexicons, and (ii) change their behavior and lexicons according to their communicative needs. Our experiments with different supervised and reinforcement learning pipelines show that neural agents trained to 'speak' an existing language can reproduce human-like patterns in color naming to a remarkable extent, supporting the further use of NeLLCom-Lex to elucidate the mechanisms of semantic change.
• 摘要：词汇语义变化主要是通过观察和实验方法研究的。但是，观察方法（语料库分析，分布语义建模）无法在因果机制上获得，并且由于涉及扩展的直觉过程，与人类的实验范式很难应用于语义变化。这项工作介绍了Nellcom-Lex，这是一种神经代理框架，旨在通过真正的词汇系统（例如英语）中的第一基接地代理模拟语义变化，然后系统地操纵其交流需求。使用完善的色彩命名任务，我们模拟了单代词汇系统的演变，并研究了导致代理的因素导致：（i）发展类似人类的命名行为和词典，以及（ii）根据其交流需求改变其行为和词典。我们使用不同的监督和强化学习管道进行的实验表明，经过训练的“说”现有语言的神经代理可以在很大程度上重现颜色命名的类似人类模式，从而支持Nellcom-Lex的进一步使用来阐明语义变化的机制。

Title: Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving

• Authors: Hang Li, Kaiqi Yang, Yucheng Chu, Hui Liu, Jiliang Tang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22480
• Pdf URL: https://arxiv.org/pdf/2509.22480
• Copy Paste: [[2509.22480]] Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving(https://arxiv.org/abs/2509.22480)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) have been widely used for problem-solving tasks. Most recent work improves their performance through supervised fine-tuning (SFT) with labeled data or reinforcement learning (RL) from task feedback. In this paper, we study a new perspective: the divergence in solutions generated by LLMs for a single problem. We show that higher solution divergence is positively related to better problem-solving abilities across various models. Based on this finding, we propose solution divergence as a novel metric that can support both SFT and RL strategies. We test this idea on three representative problem domains and find that using solution divergence consistently improves success rates. These results suggest that solution divergence is a simple but effective tool for advancing LLM training and evaluation.
• 摘要：大型语言模型（LLM）已被广泛用于解决问题的任务。最新的工作通过受到监督的微调（SFT）的标记数据或增强学习（RL）来提高其性能。在本文中，我们研究了一种新的观点：LLMS为单个问题产生的解决方案的差异。我们表明，较高的解决方案差异与各种模型的更好解决问题的能力呈正相关。基于这一发现，我们提出解决方案差异是一种可以支持SFT和RL策略的新型指标。我们在三个代表性问题域上测试了这一想法，发现使用解决方案差异始终提高成功率。这些结果表明，解决方案差异是一种简单但有效的工具，用于推进LLM培训和评估。

Title: JGU Mainz's Submission to the WMT25 Shared Task on LLMs with Limited Resources for Slavic Languages: MT and QA

• Authors: Hossain Shaikh Saadi, Minh Duc Bui, Mario Sanz-Guerrero, Katharina von der Wense
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22490
• Pdf URL: https://arxiv.org/pdf/2509.22490
• Copy Paste: [[2509.22490]] JGU Mainz's Submission to the WMT25 Shared Task on LLMs with Limited Resources for Slavic Languages: MT and QA(https://arxiv.org/abs/2509.22490)
• Keywords: llm, retrieval-augmented generation
• Abstract: This paper presents the JGU Mainz submission to the WMT25 Shared Task on LLMs with Limited Resources for Slavic Languages: Machine Translation and Question Answering, focusing on Ukrainian, Upper Sorbian, and Lower Sorbian. For each language, we jointly fine-tune a Qwen2.5-3B-Instruct model for both tasks with parameter-efficient finetuning. Our pipeline integrates additional translation and multiple-choice question answering (QA) data. For Ukrainian QA, we further use retrieval-augmented generation. We also apply ensembling for QA in Upper and Lower Sorbian. Experiments show that our models outperform the baseline on both tasks.
• 摘要：本文介绍了JGU MAINZ提交给WMT25在LLMS上共享的任务，而斯​​拉夫语言的资源有限：机器翻译和问题答案，重点关注乌克兰，上索尔比安和下索尔比安。对于每种语言，我们共同对两个任务的QWEN2.5-3B教学模型进行了微调，并通过参数有效的芬太尼进行了调整。我们的管道集成了其他翻译和多项选择问题答案（QA）数据。对于乌克兰质量保证，我们进一步使用了检索演出的一代。我们还将连接用于上和下索尔比安的质量检查。实验表明，我们的模型在这两个任务上的基线优于基线。

Title: Representing LLMs in Prompt Semantic Task Space

• Authors: Idan Kashani, Avi Mendelson, Yaniv Nemcovsky
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.22506
• Pdf URL: https://arxiv.org/pdf/2509.22506
• Copy Paste: [[2509.22506]] Representing LLMs in Prompt Semantic Task Space(https://arxiv.org/abs/2509.22506)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) achieve impressive results over various tasks, and ever-expanding public repositories contain an abundance of pre-trained models. Therefore, identifying the best-performing LLM for a given task is a significant challenge. Previous works have suggested learning LLM representations to address this. However, these approaches present limited scalability and require costly retraining to encompass additional models and datasets. Moreover, the produced representation utilizes distinct spaces that cannot be easily interpreted. This work presents an efficient, training-free approach to representing LLMs as linear operators within the prompts' semantic task space, thus providing a highly interpretable representation of the models' application. Our method utilizes closed-form computation of geometrical properties and ensures exceptional scalability and real-time adaptability to dynamically expanding repositories. We demonstrate our approach on success prediction and model selection tasks, achieving competitive or state-of-the-art results with notable performance in out-of-sample scenarios.
• 摘要：大型语言模型（LLMS）在各种任务上取得了令人印象深刻的结果，并且不断扩展的公共存储库包含大量的预训练模型。因此，确定给定任务的表现最佳的LLM是一个重大挑战。先前的工作建议学习LLM表示以解决这一问题。但是，这些方法具有有限的可扩展性，需要昂贵的重新培训才能涵盖其他模型和数据集。此外，产生的表示形式利用了无法轻易解释的不同空间。这项工作为提示的语义任务空间中的线性操作员提供了一种高效，无训练的方法，从而提供了对模型应用程序的高度可解释的表示。我们的方法利用了几何特性的封闭式计算，并确保了对动态扩展存储库的出色可伸缩性和实时适应性。我们展示了我们在成功预测和模型选择任务上的方法，并在样本外场景中取得了竞争性或最先进的结果。

Title: We Think, Therefore We Align LLMs to Helpful, Harmless and Honest Before They Go Wrong

• Authors: Gautam Siddharth Kashyap, Mark Dras, Usman Naseem
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22510
• Pdf URL: https://arxiv.org/pdf/2509.22510
• Copy Paste: [[2509.22510]] We Think, Therefore We Align LLMs to Helpful, Harmless and Honest Before They Go Wrong(https://arxiv.org/abs/2509.22510)
• Keywords: language model, llm
• Abstract: Alignment of Large Language Models (LLMs) along multiple objectives-helpfulness, harmlessness, and honesty (HHH)-is critical for safe and reliable deployment. Prior work has used steering vector-small control signals injected into hidden states-to guide LLM outputs, typically via one-to-one (1-to-1) Transformer decoders. In this setting, optimizing a single alignment objective can inadvertently overwrite representations learned for other objectives, leading to catastrophic forgetting. More recent approaches extend steering vectors via one-to-many (1-to-N) Transformer decoders. While this alleviates catastrophic forgetting, naive multi-branch designs optimize each objective independently, which can cause inference fragmentation-outputs across HHH objectives may become inconsistent. We propose Adaptive Multi-Branch Steering (AMBS), a two-stage 1-to-N framework for unified and efficient multi-objective alignment. In Stage I, post-attention hidden states of the Transformer layer are computed once to form a shared representation. In Stage II, this representation is cloned into parallel branches and steered via a policy-reference mechanism, enabling objective-specific control while maintaining cross-objective consistency. Empirical evaluations on Alpaca, BeaverTails, and TruthfulQA show that AMBS consistently improves HHH alignment across multiple 7B LLM backbones. For example, on DeepSeek-7B, AMBS improves average alignment scores by +32.4% and reduces unsafe outputs by 11.0% compared to a naive 1-to-N baseline, while remaining competitive with state-of-the-art methods.
• 摘要：大型语言模型（LLM）沿多个目标，无害和诚实（HHH）的一致性对安全可靠的部署至关重要。先前的工作使用了注入隐藏状态的转向矢量 - 小型控制信号，通常是通过一对一（1-1）变压器解码器。在这种情况下，优化单个对齐目标可以无意中覆盖针对其他目标的表示形式，从而导致灾难性遗忘。最新的方法通过一对多（1至N）变压器解码器扩展了转向向量。尽管这减轻了灾难性的遗忘，但幼稚的多分支设计可独立地优化每个目标，这可能会导致HHH目标的推理碎片输出可能会变得不一致。我们提出了自适应多分支转向（AMB），这是一个两阶段的1到N框架，用于统一和高效的多目标对准。在第I阶段，计算出注意力后的隐藏状态一次形成共享表示形式。在第二阶段中，将此表示形式克隆到平行分支中，并通过政策引用机制进行转向，从而实现了特定于特定的控制，同时保持了交叉目标的一致性。对羊驼，海狸和真实性的经验评估表明，AMB始终改善了多个7B LLM骨架的HHH对准。例如，在DeepSeek-7B上，与天真的1至N基线相比，AMBS将平均比对分数提高 +32.4％，并将不安全的输出降低11.0％，同时与先进方法保持竞争力。

Title: InfiR2: A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models

• Authors: Wenjun Wang, Shuo Cai, Congkai Xie, Mingfa Feng, Yiming Zhang, Zhen Li, Kejing Yang, Ming Li, Jiannong Cao, Yuan Xie, Hongxia Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22536
• Pdf URL: https://arxiv.org/pdf/2509.22536
• Copy Paste: [[2509.22536]] InfiR2: A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models(https://arxiv.org/abs/2509.22536)
• Keywords: language model, llm
• Abstract: The immense computational cost of training Large Language Models (LLMs) presents a major barrier to innovation. While FP8 training offers a promising solution with significant theoretical efficiency gains, its widespread adoption has been hindered by the lack of a comprehensive, open-source training recipe. To bridge this gap, we introduce an end-to-end FP8 training recipe that seamlessly integrates continual pre-training and supervised fine-tuning. Our methodology employs a fine-grained, hybrid-granularity quantization strategy to maintain numerical fidelity while maximizing computational efficiency. Through extensive experiments, including the continue pre-training of models on a 160B-token corpus, we demonstrate that our recipe is not only remarkably stable but also essentially lossless, achieving performance on par with the BF16 baseline across a suite of reasoning benchmarks. Crucially, this is achieved with substantial efficiency improvements, including up to a 22% reduction in training time, a 14% decrease in peak memory usage, and a 19% increase in throughput. Our results establish FP8 as a practical and robust alternative to BF16, and we will release the accompanying code to further democratize large-scale model training.
• 摘要：培训大语模型（LLMS）的巨大计算成本构成了创新的主要障碍。尽管FP8培训提供了一种有希望的解决方案，并具有显着的理论效率提高，但由于缺乏全面的开源培训食谱而阻碍了其广泛的采用。为了弥合这一差距，我们引入了端到端的FP8培训配方，该配方无缝整合持续的预训练和监督微调。我们的方法采用了细粒度的杂种量化策略来维持数值保真度，同时最大程度地提高了计算效率。通过广泛的实验，包括在160b token语料库上继续预先培训模型，我们证明了我们的配方不仅非常稳定，而且基本上是无损的，可以在一系列推理基准套件中与BF16基线相同。至关重要的是，这是通过大量效率提高来实现的，包括训练时间减少22％，峰值记忆使用率下降14％，吞吐量增加了19％。我们的结果将FP8确定为BF16的实用且可靠的替代方案，我们将发布随附的代码，以进一步使大规模模型培训民主化。

Title: Think Socially via Cognitive Reasoning

• Authors: Jinfeng Zhou, Zheyu Chen, Shuai Wang, Quanyu Dai, Zhenhua Dong, Hongning Wang, Minlie Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22546
• Pdf URL: https://arxiv.org/pdf/2509.22546
• Copy Paste: [[2509.22546]] Think Socially via Cognitive Reasoning(https://arxiv.org/abs/2509.22546)
• Keywords: llm
• Abstract: LLMs trained for logical reasoning excel at step-by-step deduction to reach verifiable answers. However, this paradigm is ill-suited for navigating social situations, which induce an interpretive process of analyzing ambiguous cues that rarely yield a definitive outcome. To bridge this gap, we introduce Cognitive Reasoning, a paradigm modeled on human social cognition. It formulates the interpretive process into a structured cognitive flow of interconnected cognitive units (e.g., observation or attribution), which combine adaptively to enable effective social thinking and responses. We then propose CogFlow, a complete framework that instills this capability in LLMs. CogFlow first curates a dataset of cognitive flows by simulating the associative and progressive nature of human thought via tree-structured planning. After instilling the basic cognitive reasoning capability via supervised fine-tuning, CogFlow adopts reinforcement learning to enable the model to improve itself via trial and error, guided by a multi-objective reward that optimizes both cognitive flow and response quality. Extensive experiments show that CogFlow effectively enhances the social cognitive capabilities of LLMs, and even humans, leading to more effective social decision-making.
• 摘要：在分步扣除方面，训练有逻辑推理的LLM擅长于获得可验证的答案。但是，这种范式不适合浏览社交情况，这引起了解释性的过程，即分析歧义的提示很少产生明确的结果。为了弥合这一差距，我们介绍了认知推理，这是一种以人类社会认知为基础的范式。它将解释过程制定为互连的认知单元（例如观察或归因）的结构化认知流，该过程适应性地结合了有效的社会思维和反应。然后，我们提出了COGFLOW，这是一个完整的框架，将此功能灌输在LLMS中。 Cogflow首先通过树结构计划模拟人类思想的关联和进步性质来策划认知流的数据集。在通过监督的微调灌输基本的认知推理能力之后，CogFlow采用了强化学习，以使模型能够通过反复试验改善自身，并在多目标奖励的指导下，以优化认知流量和响应质量。广泛的实验表明，CogFlow有效地增强了LLMS甚至人类的社会认知能力，从而导致更有效的社会决策。

Title: Retrieval-Augmented Guardrails for AI-Drafted Patient-Portal Messages: Error Taxonomy Construction and Large-Scale Evaluation

• Authors: Wenyuan Chen, Fateme Nateghi Haredasht, Kameron C. Black, Francois Grolleau, Emily Alsentzer, Jonathan H. Chen, Stephen P. Ma
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.22565
• Pdf URL: https://arxiv.org/pdf/2509.22565
• Copy Paste: [[2509.22565]] Retrieval-Augmented Guardrails for AI-Drafted Patient-Portal Messages: Error Taxonomy Construction and Large-Scale Evaluation(https://arxiv.org/abs/2509.22565)
• Keywords: language model, llm, prompt
• Abstract: Asynchronous patient-clinician messaging via EHR portals is a growing source of clinician workload, prompting interest in large language models (LLMs) to assist with draft responses. However, LLM outputs may contain clinical inaccuracies, omissions, or tone mismatches, making robust evaluation essential. Our contributions are threefold: (1) we introduce a clinically grounded error ontology comprising 5 domains and 59 granular error codes, developed through inductive coding and expert adjudication; (2) we develop a retrieval-augmented evaluation pipeline (RAEC) that leverages semantically similar historical message-response pairs to improve judgment quality; and (3) we provide a two-stage prompting architecture using DSPy to enable scalable, interpretable, and hierarchical error detection. Our approach assesses the quality of drafts both in isolation and with reference to similar past message-response pairs retrieved from institutional archives. Using a two-stage DSPy pipeline, we compared baseline and reference-enhanced evaluations on over 1,500 patient messages. Retrieval context improved error identification in domains such as clinical completeness and workflow appropriateness. Human validation on 100 messages demonstrated superior agreement (concordance = 50% vs. 33%) and performance (F1 = 0.500 vs. 0.256) of context-enhanced labels vs. baseline, supporting the use of our RAEC pipeline as AI guardrails for patient messaging.
• 摘要：异步通过EHR门户网站的异步患者访问消息传递是临床医生工作量的越来越多的来源，促使人们对大语言模型（LLMS）的兴趣以协助草稿响应。但是，LLM输出可能包含临床上的不准确性，遗漏或音调不匹配，从而使稳健的评估必不可少。我们的贡献是三重的：（1）我们引入了一个临床上扎根的误差本体，其中包括5个域和59个颗粒误差代码，这些误差代码是通过电感编码和专家裁决开发的； （2）我们开发了一个检索提示的评估管道（RAEC），该管道利用语义上相似的历史信息响应对来提高判断质量； （3）我们使用DSPY提供了两阶段的提示架构，以启用可扩展，可解释和分层错误检测。我们的方法评估了孤立的草稿质量，并参考了从机构档案中检索到的类似的过去消息响应对。使用两阶段的DSPY管道，我们比较了1,500多个患者消息的基线和参考增强评估。检索环境改善了临床完整性和工作流程适当性等领域的错误识别。人类对100条消息的验证表明，在上下文增强标签与基线的绩效（F1 = 0.500 vs. 0.256）表现出了卓越的一致性（一致性= 50％对33％），并且支持我们的RAEC Pipeline用作患者信息的AI Pupeleline。

Title: Fine-Grained Detection of Context-Grounded Hallucinations Using LLMs

• Authors: Yehonatan Pesiakhovsky, Zorik Gekhman, Yosi Mass, Liat Ein-Dor, Roi Reichart
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22582
• Pdf URL: https://arxiv.org/pdf/2509.22582
• Copy Paste: [[2509.22582]] Fine-Grained Detection of Context-Grounded Hallucinations Using LLMs(https://arxiv.org/abs/2509.22582)
• Keywords: llm, hallucination, prompt
• Abstract: Context-grounded hallucinations are cases where model outputs contain information not verifiable against the source text. We study the applicability of LLMs for localizing such hallucinations, as a more practical alternative to existing complex evaluation pipelines. In the absence of established benchmarks for meta-evaluation of hallucinations localization, we construct one tailored to LLMs, involving a challenging human annotation of over 1,000 examples. We complement the benchmark with an LLM-based evaluation protocol, verifying its quality in a human evaluation. Since existing representations of hallucinations limit the types of errors that can be expressed, we propose a new representation based on free-form textual descriptions, capturing the full range of possible errors. We conduct a comprehensive study, evaluating four large-scale LLMs, which highlights the benchmark's difficulty, as the best model achieves an F1 score of only 0.67. Through careful analysis, we offer insights into optimal prompting strategies for the task and identify the main factors that make it challenging for LLMs: (1) a tendency to incorrectly flag missing details as inconsistent, despite being instructed to check only facts in the output; and (2) difficulty with outputs containing factually correct information absent from the source - and thus not verifiable - due to alignment with the model's parametric knowledge.
• 摘要：上下文接地的幻觉是模型输出包含无法对源文本进行验证的信息的情况。我们研究了LLM在本地化此类幻觉中的适用性，这是现有复杂评估管道的更实际的替代方法。在没有建立的基准来对幻觉定位进行元评估的基准下，我们构建了一个针对LLM的量身定制的，涉及一个挑战性的人类注释，对1000多个例子进行了挑战。我们通过基于LLM的评估协议对基准进行补充，并在人类评估中验证其质量。由于幻觉的现有表示形式限制了可以表达的错误类型，因此我们根据自由形式的文本描述提出了一个新表示形式，从而捕获了所有可能的错误范围。我们进行了一项全面的研究，评估了四个大型LLM，这突出了基准的难度，因为最佳模型的F1得分仅为0.67。通过仔细的分析，我们可以洞悉最佳的任务提示策略，并确定对LLMS具有挑战性的主要因素：（1）尽管被指示仅检查输出中的事实，但仍倾向于错误地标记丢失细节的趋势； （2）由于与模型的参数知识对齐，因此在包含源缺失的事实正确信息的输出方面难度。

Title: ArabJobs: A Multinational Corpus of Arabic Job Ads

• Authors: Mo El-Haj
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22589
• Pdf URL: https://arxiv.org/pdf/2509.22589
• Copy Paste: [[2509.22589]] ArabJobs: A Multinational Corpus of Arabic Job Ads(https://arxiv.org/abs/2509.22589)
• Keywords: language model
• Abstract: ArabJobs is a publicly available corpus of Arabic job advertisements collected from Egypt, Jordan, Saudi Arabia, and the United Arab Emirates. Comprising over 8,500 postings and more than 550,000 words, the dataset captures linguistic, regional, and socio-economic variation in the Arab labour market. We present analyses of gender representation and occupational structure, and highlight dialectal variation across ads, which offers opportunities for future research. We also demonstrate applications such as salary estimation and job category normalisation using large language models, alongside benchmark tasks for gender bias detection and profession classification. The findings show the utility of ArabJobs for fairness-aware Arabic NLP and labour market research. The dataset is publicly available on GitHub: this https URL.
• 摘要：阿拉伯人是从埃及，约旦，沙特阿拉伯和阿拉伯联合酋长国收集的阿拉伯招聘广告的公开语料库。该数据集包含超过8,500个帖子和超过550,000个单词，在阿拉伯劳动力市场中捕获了语言，区域和社会经济差异。我们介绍了性别代表和职业结构的分析，并突出了广告之间的方言变化，这为将来的研究提供了机会。我们还展示了使用大语言模型的薪资估算和工作类别标准化等应用，以及用于性别偏见检测和专业分类的基准任务。调查结果表明，阿拉伯人对公平意识的阿拉伯语NLP和劳动力市场研究的实用性。该数据集可在GitHub上公开可用：此HTTPS URL。

Title: Capturing Opinion Shifts in Deliberative Discourse through Frequency-based Quantum deep learning methods

• Authors: Rakesh Thakur, Harsh Chaturvedi, Ruqayya Shah, Janvi Chauhan, Ayush Sharma
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.22603
• Pdf URL: https://arxiv.org/pdf/2509.22603
• Copy Paste: [[2509.22603]] Capturing Opinion Shifts in Deliberative Discourse through Frequency-based Quantum deep learning methods(https://arxiv.org/abs/2509.22603)
• Keywords: prompt
• Abstract: Deliberation plays a crucial role in shaping outcomes by weighing diverse perspectives before reaching decisions. With recent advancements in Natural Language Processing, it has become possible to computationally model deliberation by analyzing opinion shifts and predicting potential outcomes under varying scenarios. In this study, we present a comparative analysis of multiple NLP techniques to evaluate how effectively models interpret deliberative discourse and produce meaningful insights. Opinions from individuals of varied backgrounds were collected to construct a self-sourced dataset that reflects diverse viewpoints. Deliberation was simulated using product presentations enriched with striking facts, which often prompted measurable shifts in audience opinions. We have given comparative analysis between two models namely Frequency-Based Discourse Modulation and Quantum-Deliberation Framework which outperform the existing state of art models. The findings highlight practical applications in public policy-making, debate evaluation, decision-support frameworks, and large-scale social media opinion mining.
• 摘要：审议在制定决定之前通过权衡各种观点来塑造结果至关重要。随着自然语言处理的最新进展，通过分析观点变化并预测不同情况下的潜在结果，可以在计算上进行审议。在这项研究中，我们介绍了多种NLP技术的比较分析，以评估如何有效模型解释审议性话语并产生有意义的见解。收集了不同背景的个人的意见，以构建一个反映不同观点的自源数据集。使用富含惊人事实的产品演示文稿模拟了审议，这通常会引起观众意见的可衡量转变。我们已经在两个模型之间进行了比较分析，即基于频率的话语调制和量子限制框架，这些框架的表现优于现有模型的现有状态。这些发现突出了公共政策制定，辩论评估，决策支持框架和大规模社交媒体意见挖掘中的实际应用。

Title: StateX: Enhancing RNN Recall via Post-training State Expansion

• Authors: Xingyu Shen, Yingfa Chen, Zhen Leng Thai, Xu Han, Zhiyuan Liu, Maosong Sun
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.22630
• Pdf URL: https://arxiv.org/pdf/2509.22630
• Copy Paste: [[2509.22630]] StateX: Enhancing RNN Recall via Post-training State Expansion(https://arxiv.org/abs/2509.22630)
• Keywords: language model, long context
• Abstract: While Transformer-based models have demonstrated remarkable language modeling performance, their high complexities result in high costs when processing long contexts. In contrast, recurrent neural networks (RNNs) such as linear attention and state space models have gained popularity due to their constant per-token complexities. However, these recurrent models struggle with tasks that require accurate recall of contextual information from long contexts, because all contextual information is compressed into a constant-size recurrent state. Previous works have shown that recall ability is positively correlated with the recurrent state size, yet directly training RNNs with larger recurrent states results in high training costs. In this paper, we introduce StateX, a training pipeline for efficiently expanding the states of pre-trained RNNs through post-training. For two popular classes of RNNs, linear attention and state space models, we design post-training architectural modifications to scale up the state size with no or negligible increase in model parameters. Experiments on models up to 1.3B parameters demonstrate that StateX efficiently enhances the recall and in-context learning ability of RNNs without incurring high post-training costs or compromising other capabilities.
• 摘要：尽管基于变压器的模型表现出了非凡的语言建模性能，但它们的高复杂性在处理较长的上下文时会导致高成本。相比之下，由于其持续不断的综合性，诸如线性注意力和状态空间模型之类的复发神经网络（RNN）已获得了普及。但是，这些经常发生的模型与需要从长上下文中准确召回上下文信息的任务努力，因为所有上下文信息都被压缩到恒定大小的经常性状态。先前的工作表明，召回能力与经常性状态大小呈正相关，但直接培训具有较大复发状态的RNN会导致高训练成本。在本文中，我们介绍了Statex，这是一种培训管道，用于通过训练后有效地扩大预训练的RNN状态。对于两种流行的RNN类别，线性注意力和状态空间模型，我们设计了训练后的体系结构修改，以扩大状态大小，而模型参数的增加或不可忽略的增加。在高达1.3B参数的模型上进行的实验表明，Statex有效地增强了RNN的回忆和在不产生高训练后成本或损害其他功能的情况下的召回和内在学习能力。

Title: Variational Reasoning for Language Models

• Authors: Xiangxin Zhou, Zichen Liu, Haonan Wang, Chao Du, Min Lin, Chongxuan Li, Liang Wang, Tianyu Pang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.22637
• Pdf URL: https://arxiv.org/pdf/2509.22637
• Copy Paste: [[2509.22637]] Variational Reasoning for Language Models(https://arxiv.org/abs/2509.22637)
• Keywords: language model
• Abstract: We introduce a variational reasoning framework for language models that treats thinking traces as latent variables and optimizes them through variational inference. Starting from the evidence lower bound (ELBO), we extend it to a multi-trace objective for tighter bounds and propose a forward-KL formulation that stabilizes the training of the variational posterior. We further show that rejection sampling finetuning and binary-reward RL, including GRPO, can be interpreted as local forward-KL objectives, where an implicit weighting by model accuracy naturally arises from the derivation and reveals a previously unnoticed bias toward easier questions. We empirically validate our method on the Qwen 2.5 and Qwen 3 model families across a wide range of reasoning tasks. Overall, our work provides a principled probabilistic perspective that unifies variational inference with RL-style methods and yields stable objectives for improving the reasoning ability of language models. Our code is available at this https URL.
• 摘要：我们为语言模型介绍了一个变性推理框架，该框架将思维痕迹视为潜在变量，并通过变异推理优化它们。从证据下限（ELBO）开始，我们将其扩展到一个多条件物镜，以进行更紧密的界限，并提出了一种稳定变异后部训练的正向kl公式。我们进一步表明，拒绝采样列出和二进制奖励RL（包括GRPO）可以解释为局部前向kl目标，其中通过模型准确性的隐式加权自然来自派生，并揭示了先前未知的偏见对更简单的问题。我们从经验上验证了QWEN 2.5和QWEN 3模型家族的方法。总体而言，我们的工作提供了一种有原则的概率观点，该观点将变异推断统一使用RL式方法，并产生稳定的目标，以提高语言模型的推理能力。我们的代码可在此HTTPS URL上找到。

Title: Language Models Can Learn from Verbal Feedback Without Scalar Rewards

• Authors: Renjie Luo, Zichen Liu, Xiangyan Liu, Chao Du, Min Lin, Wenhu Chen, Wei Lu, Tianyu Pang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.22638
• Pdf URL: https://arxiv.org/pdf/2509.22638
• Copy Paste: [[2509.22638]] Language Models Can Learn from Verbal Feedback Without Scalar Rewards(https://arxiv.org/abs/2509.22638)
• Keywords: language model, llm, prompt
• Abstract: LLMs are often trained with RL from human or AI feedback, yet such methods typically compress nuanced feedback into scalar rewards, discarding much of their richness and inducing scale imbalance. We propose treating verbal feedback as a conditioning signal. Inspired by language priors in text-to-image generation, which enable novel outputs from unseen prompts, we introduce the feedback-conditional policy (FCP). FCP learns directly from response-feedback pairs, approximating the feedback-conditional posterior through maximum likelihood training on offline data. We further develop an online bootstrapping stage where the policy generates under positive conditions and receives fresh feedback to refine itself. This reframes feedback-driven learning as conditional generation rather than reward optimization, offering a more expressive way for LLMs to directly learn from verbal feedback. Our code is available at this https URL.
• 摘要：LLM经常接受来自人类或AI反馈的RL训练，但是这些方法通常会压缩细微的反馈到标量奖励中，丢弃了它们的大部分丰富性并引起了规模不平衡。我们建议将口头反馈视为条件信号。受文本到图像生成中语言先验的启发，这使得从看不见的提示中获得了新的输出，我们介绍了反馈条件策略（FCP）。 FCP直接从响应反馈对中学习，通过在离线数据上的最大似然训练来近似反馈条件后部。我们进一步开发了一个在线启动阶段，该阶段在积极条件下产生了策略，并收到新的反馈以完善自己。这将反馈驱动的学习作为有条件的生成而不是奖励优化，为LLMS提供了一种直接从口头反馈中学习的方式。我们的代码可在此HTTPS URL上找到。

Title: Death of the Novel(ty): Beyond n-Gram Novelty as a Metric for Textual Creativity

• Authors: Arkadiy Saakyan, Najoung Kim, Smaranda Muresan, Tuhin Chakrabarty
• Subjects: cs.CL, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2509.22641
• Pdf URL: https://arxiv.org/pdf/2509.22641
• Copy Paste: [[2509.22641]] Death of the Novel(ty): Beyond n-Gram Novelty as a Metric for Textual Creativity(https://arxiv.org/abs/2509.22641)
• Keywords: language model, llm
• Abstract: N-gram novelty is widely used to evaluate language models' ability to generate text outside of their training data. More recently, it has also been adopted as a metric for measuring textual creativity. However, theoretical work on creativity suggests that this approach may be inadequate, as it does not account for creativity's dual nature: novelty (how original the text is) and appropriateness (how sensical and pragmatic it is). We investigate the relationship between this notion of creativity and n-gram novelty through 7542 expert writer annotations (n=26) of novelty, pragmaticality, and sensicality via close reading of human and AI-generated text. We find that while n-gram novelty is positively associated with expert writer-judged creativity, ~91% of top-quartile expressions by n-gram novelty are not judged as creative, cautioning against relying on n-gram novelty alone. Furthermore, unlike human-written text, higher n-gram novelty in open-source LLMs correlates with lower pragmaticality. In an exploratory study with frontier close-source models, we additionally confirm that they are less likely to produce creative expressions than humans. Using our dataset, we test whether zero-shot, few-shot, and finetuned models are able to identify creative expressions (a positive aspect of writing) and non-pragmatic ones (a negative aspect). Overall, frontier LLMs exhibit performance much higher than random but leave room for improvement, especially struggling to identify non-pragmatic expressions. We further find that LLM-as-a-Judge novelty scores from the best-performing model were predictive of expert writer preferences.
• 摘要：N-gram新颖性被广泛用于评估语言模型在其培训数据之外生成文本的能力。最近，它也被用作测量文本创造力的指标。但是，关于创造力的理论工作表明，这种方法可能不足，因为它不能解释创造力的双重性质：新颖性（本文的原始程度）和适当性（它是多么的感官和务实）。我们通过7542个专家作家注释（n = 26）的新颖性，陈旧性和感官性通过仔细阅读人类和AI生成的文本之间的新颖性注释（n = 26），研究了这种创造力和N-Gram新颖性的概念。我们发现，尽管N-gram新颖性与专家作家判断的创造力呈正相关，但〜91％的N-gram新奇表达式中有91％的表情并没有被判断为创造性，因此警告不要仅依靠N-Gram单独的新颖性。此外，与人写的文本不同，开源llms中的较高的n-gram新颖性与较低的务实性相关。在Frontier封闭式模型的探索性研究中，我们还确认它们比人类不太可能产生创意表达。使用我们的数据集，我们测试零射，很少的射击和填充模型是否能够识别创意表达式（写作的积极方面）和非刺激性表达式（负面方面）。总体而言，Frontier LLM的表现远高于随机性，但留出了改进的空间，尤其是努力识别非弹性表达式。 We further find that LLM-as-a-Judge novelty scores from the best-performing model were predictive of expert writer preferences.

Title: WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning

• Authors: Zimu Lu, Houxing Ren, Yunqiao Yang, Ke Wang, Zhuofan Zong, Junting Pan, Mingjie Zhan, Hongsheng Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.22644
• Pdf URL: https://arxiv.org/pdf/2509.22644
• Copy Paste: [[2509.22644]] WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning(https://arxiv.org/abs/2509.22644)
• Keywords: language model, llm, agent
• Abstract: Agent systems powered by large language models (LLMs) have demonstrated impressive performance on repository-level code-generation tasks. However, for tasks such as website codebase generation, which depend heavily on visual effects and user-interaction feedback, current code agents rely only on simple code execution for feedback and verification. This approach fails to capture the actual quality of the generated code. In this paper, we propose WebGen-Agent, a novel website-generation agent that leverages comprehensive and multi-level visual feedback to iteratively generate and refine the website codebase. Detailed and expressive text descriptions and suggestions regarding the screenshots and GUI-agent testing of the websites are generated by a visual language model (VLM), together with scores that quantify their quality. The screenshot and GUI-agent scores are further integrated with a backtracking and select-best mechanism, enhancing the performance of the agent. Utilizing the accurate visual scores inherent in the WebGen-Agent workflow, we further introduce \textit{Step-GRPO with Screenshot and GUI-agent Feedback} to improve the ability of LLMs to act as the reasoning engine of WebGen-Agent. By using the screenshot and GUI-agent scores at each step as the reward in Step-GRPO, we provide a dense and reliable process supervision signal, which effectively improves the model's website-generation ability. On the WebGen-Bench dataset, WebGen-Agent increases the accuracy of Claude-3.5-Sonnet from 26.4% to 51.9% and its appearance score from 3.0 to 3.9, outperforming the previous state-of-the-art agent system. Additionally, our Step-GRPO training approach increases the accuracy of Qwen2.5-Coder-7B-Instruct from 38.9% to 45.4% and raises the appearance score from 3.4 to 3.7.
• 摘要：由大语言模型（LLM）提供动力的代理系统在存储库级代码生成任务上表现出了令人印象深刻的性能。但是，对于很大程度上取决于视觉效果和用户交互反馈的网站代码库生成等任务，当前代码代理仅依靠简单的代码执行来进行反馈和验证。这种方法无法捕获生成代码的实际质量。在本文中，我们提出了一个新型的网站生成代理Webgen-Agent，它利用全面和多层次的视觉反馈来迭代生成和完善网站代码库。有关网站的屏幕截图和GUI-AGENT测试的详细和表现力的文本描述和建议是由视觉语言模型（VLM）生成的，以及量化其质量的分数。屏幕截图和Gui-Agent分数将与回溯和精选最佳机制进一步集成，从而增强了代理的性能。利用网络代理工作流中固有的准确视觉分数，我们进一步介绍了\ textit {带有屏幕截图和Gui-agent Feffercable的steptIt {step-grpo}，以提高LLMS作为WebGen-Agent推理引擎的能力。通过在每个步骤中使用屏幕截图和Gui-Agent分数作为Step-Grpo中的奖励，我们提供了一个密集可靠的过程监督信号，从而有效地提高了该模型的网站生成能力。在WebGEN BENCH数据集上，WebGen代理将Claude-3.5-Sonnet的准确性从26.4％提高到51.9％，其外观得分从3.0增加到3.9，表现优于先前的最新代理系统。此外，我们的继GRPO训练方法还将QWEN2.5代码-7B教学的准确性从38.9％提高到45.4％，并将外观得分从3.4提高到3.7。

Title: VoiceAssistant-Eval: Benchmarking AI Assistants across Listening, Speaking, and Viewing

• Authors: Ke Wang, Houxing Ren, Zimu Lu, Mingjie Zhan, Hongsheng Li
• Subjects: cs.CL, cs.AI, cs.CV, cs.HC, cs.SD
• Abstract URL: https://arxiv.org/abs/2509.22651
• Pdf URL: https://arxiv.org/pdf/2509.22651
• Copy Paste: [[2509.22651]] VoiceAssistant-Eval: Benchmarking AI Assistants across Listening, Speaking, and Viewing(https://arxiv.org/abs/2509.22651)
• Keywords: language model, gpt
• Abstract: The growing capabilities of large language models and multimodal systems have spurred interest in voice-first AI assistants, yet existing benchmarks are inadequate for evaluating the full range of these systems' capabilities. We introduce VoiceAssistant-Eval, a comprehensive benchmark designed to assess AI assistants across listening, speaking, and viewing. VoiceAssistant-Eval comprises 10,497 curated examples spanning 13 task categories. These tasks include natural sounds, music, and spoken dialogue for listening; multi-turn dialogue, role-play imitation, and various scenarios for speaking; and highly heterogeneous images for viewing. To demonstrate its utility, we evaluate 21 open-source models and GPT-4o-Audio, measuring the quality of the response content and speech, as well as their consistency. The results reveal three key findings: (1) proprietary models do not universally outperform open-source models; (2) most models excel at speaking tasks but lag in audio understanding; and (3) well-designed smaller models can rival much larger ones. Notably, the mid-sized Step-Audio-2-mini (7B) achieves more than double the listening accuracy of LLaMA-Omni2-32B-Bilingual. However, challenges remain: multimodal (audio plus visual) input and role-play voice imitation tasks are difficult for current models, and significant gaps persist in robustness and safety alignment. VoiceAssistant-Eval identifies these gaps and establishes a rigorous framework for evaluating and guiding the development of next-generation AI assistants. Code and data will be released at this https URL .
• 摘要：大型语言模型和多模式系统的越来越多的功能激发了人们对语音优先的AI助手的兴趣，但是现有的基准不足以评估这些系统功能的全部范围。我们介绍了VoiceAssistant-Eval，这是一个综合基准，旨在评估聆听，讲话和观看的AI助手。 VoiceAssistant-Eval包括10,497个策划的示例，涵盖13个任务类别。这些任务包括自然的声音，音乐和聆听的对话。多转向对话，角色扮演模仿和各种讲话的场景；和高度异质图像可供观看。为了展示其效用，我们评估了21种开源模型和GPT-4O-Audio，测量了响应内容和语音的质量及其一致性。结果揭示了三个关键发现：（1）专有模型并不能普遍胜过开源模型； （2）大多数模型在说话任务方面表现出色，但在音频理解方面却滞后； （3）精心设计的较小型号可以与较大的型号匹敌。值得注意的是，中型的Step-Audio-2-Mini（7b）取得了比Llama-omni2-32b blingual的听力精度的两倍以上。但是，仍然存在挑战：对于当前模型而言，很难进行多模式（音频加视觉）输入和角色扮演的语音模仿任务，并且稳健性和安全性持续存在巨大差距。 VoiceAssistant-Eval确定了这些差距，并建立了一个严格的框架，用于评估和指导下一代AI助手的发展。代码和数据将在此HTTPS URL上发布。