2025-09-26

Title: Interpreting Public Sentiment in Diplomacy Events: A Counterfactual Analysis Framework Using Large Language Models

• Authors: Leyi Ouyang
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2509.20367
• Pdf URL: https://arxiv.org/pdf/2509.20367
• Copy Paste: [[2509.20367]] Interpreting Public Sentiment in Diplomacy Events: A Counterfactual Analysis Framework Using Large Language Models(https://arxiv.org/abs/2509.20367)
• Keywords: language model, prompt
• Abstract: Diplomatic events consistently prompt widespread public discussion and debate. Public sentiment plays a critical role in diplomacy, as a good sentiment provides vital support for policy implementation, helps resolve international issues, and shapes a nation's international image. Traditional methods for gauging public sentiment, such as large-scale surveys or manual content analysis of media, are typically time-consuming, labor-intensive, and lack the capacity for forward-looking analysis. We propose a novel framework that identifies specific modifications for diplomatic event narratives to shift public sentiment from negative to neutral or positive. First, we train a language model to predict public reaction towards diplomatic events. To this end, we construct a dataset comprising descriptions of diplomatic events and their associated public discussions. Second, guided by communication theories and in collaboration with domain experts, we predetermined several textual features for modification, ensuring that any alterations changed the event's narrative framing while preserving its core this http URL develop a counterfactual generation algorithm that employs a large language model to systematically produce modified versions of an original text. The results show that this framework successfully shifted public sentiment to a more favorable state with a 70\% success rate. This framework can therefore serve as a practical tool for diplomats, policymakers, and communication specialists, offering data-driven insights on how to frame diplomatic initiatives or report on events to foster a more desirable public sentiment.
• 摘要：外交活动始终促使广泛的公众讨论和辩论。公众情绪在外交中起着至关重要的作用，因为良好的情绪为政策实施提供了重要的支持，有助于解决国际问题，并塑造一个国家的国际形象。衡量公共情绪的传统方法，例如大规模调查或媒体的手动内容分析，通常是耗时，劳动力密集的，并且缺乏前瞻性分析的能力。我们提出了一个新颖的框架，该框架确定了外交事件叙事的特定修改，以将公共情感从负面或积极转变。首先，我们训练一种语言模型，以预测公众对外交事件的反应。为此，我们构建了一个数据集，其中包括外交事件的描述及其相关的公众讨论。其次，在交流理论和与域专家合作的指导下，我们预定了几个文本特征进行修改，以确保任何更改都改变了事件的叙述性框架，同时保留其核心此HTTP URL会开发出一种反事实算法，该算法采用了大型语言模型，以系统地生成原始文本的修改版本。结果表明，该框架成功地将公共情绪转移到了一个更有利的状态，成功率为70 \％。因此，该框架可以作为外交官，政策制定者和传播专家的实用工具，提供有关如何制定外交计划的数据驱动的见解，或报告有关事件的报道，以培养更理想的公众情绪。

Title: CFD-LLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics

• Authors: Nithin Somasekharan, Ling Yue, Yadi Cao, Weichao Li, Patrick Emami, Pochinapeddi Sai Bhargav, Anurag Acharya, Xingyu Xie, Shaowu Pan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20374
• Pdf URL: https://arxiv.org/pdf/2509.20374
• Copy Paste: [[2509.20374]] CFD-LLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics(https://arxiv.org/abs/2509.20374)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have demonstrated strong performance across general NLP tasks, but their utility in automating numerical experiments of complex physical system -- a critical and labor-intensive component -- remains underexplored. As the major workhorse of computational science over the past decades, Computational Fluid Dynamics (CFD) offers a uniquely challenging testbed for evaluating the scientific capabilities of LLMs. We introduce CFDLLMBench, a benchmark suite comprising three complementary components -- CFDQuery, CFDCodeBench, and FoamBench -- designed to holistically evaluate LLM performance across three key competencies: graduate-level CFD knowledge, numerical and physical reasoning of CFD, and context-dependent implementation of CFD workflows. Grounded in real-world CFD practices, our benchmark combines a detailed task taxonomy with a rigorous evaluation framework to deliver reproducible results and quantify LLM performance across code executability, solution accuracy, and numerical convergence behavior. CFDLLMBench establishes a solid foundation for the development and evaluation of LLM-driven automation of numerical experiments for complex physical systems. Code and data are available at this https URL.
• 摘要：大型语言模型（LLM）在一般的NLP任务中表现出了很强的性能，但是它们在自动化复杂物理系统的数值实验方面的实用性（一种关键且劳动力密集的组件）仍然没有得到充实。作为过去几十年来计算科学的主要主力，计算流体动力学（CFD）为评估LLMS的科学能力提供了独特的挑战性测试床。我们介绍了Cfdllmbench，这是一个基准套件，其中包括三个互补组件-CFDQuery，CFDCodeBench和FoAmbench-旨在在三个关键能力上整体评估LLM的性能：研究生水平的CFD知识，数值，数值和cfd和上下文依赖性依赖性依赖性依赖性依赖性工作的工作。基于实际CFD实践，我们的基准将详细的任务分类法与严格的评估框架相结合，以提供可重复的结果并量化代码可执行性，解决方案准确性和数值收敛行为的LLM性能。 CFDLLMBENCH为复杂物理系统的数值实验的开发和评估建立了坚实的基础。代码和数据可在此HTTPS URL上找到。

Title: Assessing Classical Machine Learning and Transformer-based Approaches for Detecting AI-Generated Research Text

• Authors: Sharanya Parimanoharan, Ruwan D. Nawarathna
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20375
• Pdf URL: https://arxiv.org/pdf/2509.20375
• Copy Paste: [[2509.20375]] Assessing Classical Machine Learning and Transformer-based Approaches for Detecting AI-Generated Research Text(https://arxiv.org/abs/2509.20375)
• Keywords: language model, gpt, llm, chat
• Abstract: The rapid adoption of large language models (LLMs) such as ChatGPT has blurred the line between human and AI-generated texts, raising urgent questions about academic integrity, intellectual property, and the spread of misinformation. Thus, reliable AI-text detection is needed for fair assessment to safeguard human authenticity and cultivate trust in digital communication. In this study, we investigate how well current machine learning (ML) approaches can distinguish ChatGPT-3.5-generated texts from human-written texts employing a labeled data set of 250 pairs of abstracts from a wide range of research topics. We test and compare both classical (Logistic Regression armed with classical Bag-of-Words, POS, and TF-IDF features) and transformer-based (BERT augmented with N-grams, DistilBERT, BERT with a lightweight custom classifier, and LSTM-based N-gram models) ML detection techniques. As we aim to assess each model's performance in detecting AI-generated research texts, we also aim to test whether an ensemble of these models can outperform any single detector. Results show DistilBERT achieves the overall best performance, while Logistic Regression and BERT-Custom offer solid, balanced alternatives; LSTM- and BERT-N-gram approaches lag. The max voting ensemble of the three best models fails to surpass DistilBERT itself, highlighting the primacy of a single transformer-based representation over mere model diversity. By comprehensively assessing the strengths and weaknesses of these AI-text detection approaches, this work lays a foundation for more robust transformer frameworks with larger, richer datasets to keep pace with ever-improving generative AI models.
• 摘要：诸如Chatgpt之类的大型语言模型（LLM）的迅速采用使人与AI生成的文本之间的界限模糊了，提出了有关学术完整性，知识产权和错误信息传播的紧急问题。因此，公平评估需要可靠的AI文本检测来保护人类的真实性并培养对数字通信的信任。在这项研究中，我们研究了当前的机器学习（ML）方法如何将Chatgpt-3.5生成的文本与人撰写的文本区分开，并采用标有250对摘要的标记数据集与广泛的研究主题的标记数据集。我们测试和比较古典（带有古典词袋，POS和TF-IDF功能的逻辑回归）和基于变压器的（用N-Grams增强，Distilbert，Distilbert，bert，带有轻质自定义分类器以及基于LSTM的N-Gram型号））ML检测技术。当我们旨在评估每个模型在检测AI生成的研究文本中的性能时，我们还旨在测试这些模型的集合是否可以胜过任何单个检测器。结果表明，Distilbert取得了总体最佳性能，而Logistic回归和Bert-Custom提供了稳固，平衡的替代方案。 LSTM和BERT-N-GRAM接近滞后。这三种最佳模型的最大投票合奏无法超越大型曲目本身，强调了基于单一的变压器代表在单纯的模型多样性上的首要地位。通过全面评估这些AI文本检测方法的优势和劣势，这项工作为具有更大，更丰富的数据集的更强大的变压器框架奠定了基础，以与不断提高的生成AI模型保持同步。

Title: ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Models

• Authors: Haoxuan Li, Zhen Wen, Qiqi Jiang, Chenxiao Li, Yuwei Wu, Yuchen Yang, Yiyao Wang, Xiuqi Huang, Minfeng Zhu, Wei Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20376
• Pdf URL: https://arxiv.org/pdf/2509.20376
• Copy Paste: [[2509.20376]] ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Models(https://arxiv.org/abs/2509.20376)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks. Understanding how LLMs internally represent knowledge remains a significant challenge. Despite Sparse Autoencoders (SAEs) have emerged as a promising technique for extracting interpretable features from LLMs, SAE features do not inherently align with human-understandable concepts, making their interpretation cumbersome and labor-intensive. To bridge the gap between SAE features and human concepts, we present ConceptViz, a visual analytics system designed for exploring concepts in LLMs. ConceptViz implements a novel dentification => Interpretation => Validation pipeline, enabling users to query SAEs using concepts of interest, interactively explore concept-to-feature alignments, and validate the correspondences through model behavior verification. We demonstrate the effectiveness of ConceptViz through two usage scenarios and a user study. Our results show that ConceptViz enhances interpretability research by streamlining the discovery and validation of meaningful concept representations in LLMs, ultimately aiding researchers in building more accurate mental models of LLM features. Our code and user guide are publicly available at this https URL.
• 摘要：大型语言模型（LLM）在各种自然语言任务中取得了出色的表现。了解LLM在内部表示知识仍然是一个重大挑战。尽管稀疏的自动编码器（SAE）已经成为一种有前途的技术，可以从LLM中提取可解释的功能，但SAE功能并不固有地与人类可行的概念保持一致，这使得它们的解释繁琐而劳动力密集。为了弥合SAE特征和人类概念之间的差距，我们提出了概念，这是一种视觉分析系统，旨在探索LLMS中的概念。 ConceptViz实现了新颖的牙齿化=>解释=>验证管道，使用户能够使用感兴趣的概念查询SAE，交互式探索概念对功能对齐，并通过模型行为验证验证对应关系。我们通过两种用法方案和一个用户研究来证明概念维斯的有效性。我们的结果表明，ConceptViz通过简化LLM中有意义的概念表示的发现和验证来增强可解释性研究，最终帮助研究人员构建LLM功能的更准确的心理模型。我们的代码和用户指南可在此HTTPS URL上公开获得。

Title: SKILL-RAG: Self-Knowledge Induced Learning and Filtering for Retrieval-Augmented Generation

• Authors: Tomoaki Isoda
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20377
• Pdf URL: https://arxiv.org/pdf/2509.20377
• Copy Paste: [[2509.20377]] SKILL-RAG: Self-Knowledge Induced Learning and Filtering for Retrieval-Augmented Generation(https://arxiv.org/abs/2509.20377)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) has significantly improved the performance of large language models (LLMs) on knowledge-intensive tasks in recent years. However, since retrieval systems may return irrelevant content, incorporating such information into the model often leads to hallucinations. Thus, identifying and filtering out unhelpful retrieved content is a key challenge for improving RAG this http URL better integrate the internal knowledge of the model with external knowledge from retrieval, it is essential to understand what the model "knows" and "does not know" (which is also called "self-knowledge"). Based on this insight, we propose SKILL-RAG (Self-Knowledge Induced Learning and Filtering for RAG), a novel method that leverages the model's self-knowledge to determine which retrieved documents are beneficial for answering a given query. We design a reinforcement learning-based training framework to explicitly elicit self-knowledge from the model and employs sentence-level granularity to filter out irrelevant content while preserving useful this http URL evaluate SKILL-RAG using Llama2-7B and Qwen3-8B on several question answering benchmarks. Experimental results demonstrate that SKILL-RAG not only improves generation quality but also significantly reduces the number of input documents, validating the importance of self-knowledge in guiding the selection of high-quality retrievals.
• 摘要：近年来，检索增强的生成（RAG）已大大提高了大语言模型（LLM）的性能。但是，由于检索系统可能会返回无关的内容，因此将这些信息纳入模型通常会导致幻觉。因此，识别和滤除无用的检索内容是改善抹布的关键挑战。此HTTP URL可以通过从检索中的外部知识更好地整合模型的内部知识，因此必须了解模型“知道”和“不知道”（也称为“自我知识”）至关重要。基于这种见解，我们提出了技能剥离（自我知识诱导的学习和抹布过滤），这是一种利用模型的自我知识来确定哪些检索文档有益于回答给定查询的新方法。我们设计了一个基于增强学习的培训框架，以明确地从模型中引起自我知识，并采用句子级粒度来过滤无关的内容，同时保留了使用Llama2-7B和Qwen3-8b的其他问题答案基准的HTTP URL评估Skill-rag。实验结果表明，技能障碍不仅可以提高发电质量，而且还大大减少了输入文档的数量，从而验证了自我知识在指导选择高质量检索中的重要性。

Title: Beyond Global Emotion: Fine-Grained Emotional Speech Synthesis with Dynamic Word-Level Modulation

• Authors: Sirui Wang, Andong Chen, Tiejun Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20378
• Pdf URL: https://arxiv.org/pdf/2509.20378
• Copy Paste: [[2509.20378]] Beyond Global Emotion: Fine-Grained Emotional Speech Synthesis with Dynamic Word-Level Modulation(https://arxiv.org/abs/2509.20378)
• Keywords: llm, prompt
• Abstract: Emotional text-to-speech (E-TTS) is central to creating natural and trustworthy human-computer interaction. Existing systems typically rely on sentence-level control through predefined labels, reference audio, or natural language prompts. While effective for global emotion expression, these approaches fail to capture dynamic shifts within a sentence. To address this limitation, we introduce Emo-FiLM, a fine-grained emotion modeling framework for LLM-based TTS. Emo-FiLM aligns frame-level features from emotion2vec to words to obtain word-level emotion annotations, and maps them through a Feature-wise Linear Modulation (FiLM) layer, enabling word-level emotion control by directly modulating text embeddings. To support evaluation, we construct the Fine-grained Emotion Dynamics Dataset (FEDD) with detailed annotations of emotional transitions. Experiments show that Emo-FiLM outperforms existing approaches on both global and fine-grained tasks, demonstrating its effectiveness and generality for expressive speech synthesis.
• 摘要：情感文本到语音（E-TTS）对于建立自然而值得信赖的人类计算机互动至关重要。现有系统通常通过预定义的标签，参考音频或自然语言提示依赖句子级控制。尽管对于全球情绪表达有效，但这些方法无法捕获句子中的动态变化。为了解决此限制，我们介绍了Emo-Film，这是一个基于LLM的TTS的细粒度情感建模框架。 Emo-Film将框架级特征从Emotion2Vec调整为单词，以获取单词级的情感注释，并通过功能的线性调制（膜）层绘制它们，从而通过直接调节文本嵌入来启用单词级别的情感控制。为了支持评估，我们使用情感过渡的详细注释来构建细粒度的情绪动态数据集（FedD）。实验表明，Emo-FILM在全球和细粒度的任务上都胜过现有的方法，证明了其有效性和对表达性语音综合的有效性和一般性。

Title: USB-Rec: An Effective Framework for Improving Conversational Recommendation Capability of Large Language Model

• Authors: Jianyu Wen, Jingyun Wang, Cilin Yan, Jiayin Cai, Xiaolong Jiang, Ying Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20381
• Pdf URL: https://arxiv.org/pdf/2509.20381
• Copy Paste: [[2509.20381]] USB-Rec: An Effective Framework for Improving Conversational Recommendation Capability of Large Language Model(https://arxiv.org/abs/2509.20381)
• Keywords: language model, llm
• Abstract: Recently, Large Language Models (LLMs) have been widely employed in Conversational Recommender Systems (CRSs). Unlike traditional language model approaches that focus on training, all existing LLMs-based approaches are mainly centered around how to leverage the summarization and analysis capabilities of LLMs while ignoring the issue of training. Therefore, in this work, we propose an integrated training-inference framework, User-Simulator-Based framework (USB-Rec), for improving the performance of LLMs in conversational recommendation at the model level. Firstly, we design a LLM-based Preference Optimization (PO) dataset construction strategy for RL training, which helps the LLMs understand the strategies and methods in conversational recommendation. Secondly, we propose a Self-Enhancement Strategy (SES) at the inference stage to further exploit the conversational recommendation potential obtained from RL training. Extensive experiments on various datasets demonstrate that our method consistently outperforms previous state-of-the-art methods.
• 摘要：最近，大型语言模型（LLM）已被广泛用于会话推荐系统（CRS）。与关注培训的传统语言模型方法不同，所有现有的基于LLMS的方法主要围绕如何利用LLM的摘要和分析功能，同时忽略培训问题。因此，在这项工作中，我们提出了一个集成的培训框架，基于用户模拟器的框架（USB-REC），以在模型级别上提高LLM的性能。首先，我们为RL培训设计了基于LLM的偏好优化（PO）数据集构建策略，这有助于LLMS了解会话建议中的策略和方法。其次，我们在推理阶段提出了一种自我增强策略（SES），以进一步利用从RL培训获得的对话推荐潜力。各种数据集上的广泛实验表明，我们的方法始终优于先前的最新方法。

Title: Document Summarization with Conformal Importance Guarantees

• Authors: Bruce Kuwahara, Chen-Yuan Lin, Xiao Shi Huang, Kin Kwan Leung, Jullian Arta Yapeter, Ilya Stanevich, Felipe Perez, Jesse C. Cresswell
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.20461
• Pdf URL: https://arxiv.org/pdf/2509.20461
• Copy Paste: [[2509.20461]] Document Summarization with Conformal Importance Guarantees(https://arxiv.org/abs/2509.20461)
• Keywords: language model, llm
• Abstract: Automatic summarization systems have advanced rapidly with large language models (LLMs), yet they still lack reliable guarantees on inclusion of critical content in high-stakes domains like healthcare, law, and finance. In this work, we introduce Conformal Importance Summarization, the first framework for importance-preserving summary generation which uses conformal prediction to provide rigorous, distribution-free coverage guarantees. By calibrating thresholds on sentence-level importance scores, we enable extractive document summarization with user-specified coverage and recall rates over critical content. Our method is model-agnostic, requires only a small calibration set, and seamlessly integrates with existing black-box LLMs. Experiments on established summarization benchmarks demonstrate that Conformal Importance Summarization achieves the theoretically assured information coverage rate. Our work suggests that Conformal Importance Summarization can be combined with existing techniques to achieve reliable, controllable automatic summarization, paving the way for safer deployment of AI summarization tools in critical applications. Code is available at this https URL.
• 摘要：自动摘要系统已通过大型语言模型（LLM）迅速发展，但由于将关键内容纳入医疗保健，法律和金融等高风险领域，它们仍然缺乏可靠的保证。在这项工作中，我们介绍了相互构想的重要性摘要，这是“重要性更重要的摘要生成”的第一个框架，它使用保形预测提供了严格的，无分配的覆盖范围保证。通过对句子级别的重要性分数校准阈值，我们可以通过用户指定的覆盖范围和召回率对关键内容进行提取文档摘要。我们的方法是模型 - 不合时宜的，仅需要一个小的校准集，并且与现有的Black-Box LLM无缝集成。对既定的摘要基准的实验表明，共形重要性摘要达到了理论上确保的信息覆盖率。我们的工作表明，可以将共形重要性摘要与现有技术结合使用，以实现可靠，可控的自动摘要，从而为在关键应用程序中更安全的AI摘要工具铺平了道路。代码可在此HTTPS URL上找到。

Title: MARS: toward more efficient multi-agent collaboration for LLM reasoning

• Authors: Xiao Wang, Jia Wang, Yijie Wang, Pengtao Dang, Sha Cao, Chi Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20502
• Pdf URL: https://arxiv.org/pdf/2509.20502
• Copy Paste: [[2509.20502]] MARS: toward more efficient multi-agent collaboration for LLM reasoning(https://arxiv.org/abs/2509.20502)
• Keywords: language model, llm, agent
• Abstract: Large language models (LLMs) have achieved impressive results in natural language understanding, yet their reasoning capabilities remain limited when operating as single agents. Multi-Agent Debate (MAD) has been proposed to address this limitation by enabling collaborative reasoning among multiple models in a round-table debate manner. While effective, MAD introduces substantial computational overhead due to the number of agents involved and the frequent communication required. In this paper, we propose MARS (Multi-Agent Review System), a role-based collaboration framework inspired by the review process. In MARS, an author agent generates an initial solution, reviewer agents provide decisions and comments independently, and a meta-reviewer integrates the feedback to make the final decision and guide further revision. This design enhances reasoning quality while avoiding costly reviewer-to-reviewer interactions, thereby controlling token consumption and inference time. We compared MARS with both MAD and other state-of-the-art reasoning strategies across multiple benchmarks. Extensive experiments with different LLMs show that MARS matches the accuracy of MAD while reducing both token usage and inference time by approximately 50\%. Code is available at this https URL.
• 摘要：大型语言模型（LLMS）在自然语言理解方面取得了令人印象深刻的结果，但是当作为单个代理运行时，其推理能力仍然有限。已经提出了多代理辩论（MAD）来解决这一限制，通过以圆桌辩论的方式启用多个模型之间的协作推理。由于涉及的代理数量和所需的频繁通信，MAD虽然有效，但引入了大量的计算间接费用。在本文中，我们提出了MARS（多代理审查系统），这是一个受审查过程启发的基于角色的协作框架。在火星中，作者代理人生成了初始解决方案，审阅者代理人独立提供决策和评论，而元审视者则整合了反馈以做出最终决定并指导进一步修订。该设计提高了推理质量，同时避免了昂贵的审阅者到评审器的互动，从而控制了令牌消费和推理时间。我们将火星与多个基准的MAD和其他最先进的推理策略进行了比较。使用不同LLM的广泛实验表明，MARS与MAD的准确性相匹配，同时将令牌使用和推理时间降低了约50 \％。代码可在此HTTPS URL上找到。

Title: SwasthLLM: a Unified Cross-Lingual, Multi-Task, and Meta-Learning Zero-Shot Framework for Medical Diagnosis Using Contrastive Representations

• Authors: Ayan Sar, Pranav Singh Puri, Sumit Aich, Tanupriya Choudhury, Abhijit Kumar
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.20567
• Pdf URL: https://arxiv.org/pdf/2509.20567
• Copy Paste: [[2509.20567]] SwasthLLM: a Unified Cross-Lingual, Multi-Task, and Meta-Learning Zero-Shot Framework for Medical Diagnosis Using Contrastive Representations(https://arxiv.org/abs/2509.20567)
• Keywords: llm
• Abstract: In multilingual healthcare environments, automatic disease diagnosis from clinical text remains a challenging task due to the scarcity of annotated medical data in low-resource languages and the linguistic variability across populations. This paper proposes SwasthLLM, a unified, zero-shot, cross-lingual, and multi-task learning framework for medical diagnosis that operates effectively across English, Hindi, and Bengali without requiring language-specific fine-tuning. At its core, SwasthLLM leverages the multilingual XLM-RoBERTa encoder augmented with a language-aware attention mechanism and a disease classification head, enabling the model to extract medically relevant information regardless of the language structure. To align semantic representations across languages, a Siamese contrastive learning module is introduced, ensuring that equivalent medical texts in different languages produce similar embeddings. Further, a translation consistency module and a contrastive projection head reinforce language-invariant representation learning. SwasthLLM is trained using a multi-task learning strategy, jointly optimizing disease classification, translation alignment, and contrastive learning objectives. Additionally, we employ Model-Agnostic Meta-Learning (MAML) to equip the model with rapid adaptation capabilities for unseen languages or tasks with minimal data. Our phased training pipeline emphasizes robust representation alignment before task-specific fine-tuning. Extensive evaluation shows that SwasthLLM achieves high diagnostic performance, with a test accuracy of 97.22% and an F1-score of 97.17% in supervised settings. Crucially, in zero-shot scenarios, it attains 92.78% accuracy on Hindi and 73.33% accuracy on Bengali medical text, demonstrating strong generalization in low-resource contexts.
• 摘要：在多语言医疗保健环境中，由于低资源语言中注释的医学数据缺乏，临床文本的自动疾病诊断仍然是一项具有挑战性的任务。本文提出了Swasthllm，这是一种用于医学诊断的统一，零射，跨语言和多任务学习框架，该框架在英语，印地语和孟加拉语之间有效运行，而无需进行语言特定的精心调整。 Swasthllm以语言意识到的注意机制和疾病分类的头脑增强了多语言XLM-Roberta编码器的核心，使该模型能够提取医学上相关的信息，而无论语言结构如何。为了使语言跨语言保持一致，引入了暹罗对比度学习模块，以确保使用不同语言的等效医学文本产生相似的嵌入。此外，翻译一致性模块和对比的投影头增强语言不变的表示。 SWASTHLLM是使用多任务学习策略训练的，共同优化了疾病分类，翻译对准和对比度学习目标。此外，我们采用模型不足的元学习（MAML）为模型提供了对看不见的语言或任务的快速适应能力，具有最少的数据。我们的分阶段培训管道强调在特定于任务的微调之前，强大的表示形式对齐。广泛的评估表明，SWASTHLLM的诊断性能高，测试准确性为97.22％，在监督环境中的F1分数为97.17％。至关重要的是，在零拍摄的情况下，它在印地语方面的准确性为92.78％，孟加拉医学文本的准确性为73.33％，在低资源环境中表现出强烈的概括。

Title: Dynamic Reasoning Chains through Depth-Specialized Mixture-of-Experts in Transformer Architectures

• Authors: Sampurna Roy, Ayan Sar, Anurag Kaushish, Kanav Gupta, Tanupriya Choudhury, Abhijit Kumar
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.20577
• Pdf URL: https://arxiv.org/pdf/2509.20577
• Copy Paste: [[2509.20577]] Dynamic Reasoning Chains through Depth-Specialized Mixture-of-Experts in Transformer Architectures(https://arxiv.org/abs/2509.20577)
• Keywords: language model
• Abstract: Contemporary transformer architectures apply identical processing depth to all inputs, creating inefficiencies and limiting reasoning quality. Simple factual queries are subjected to the same multilayered computation as complex logical problems, wasting resources while constraining deep inference. To overcome this, we came up with a concept of Dynamic Reasoning Chains through Depth Specialised Mixture of Experts (DS-MoE), a modular framework that extends the Mixture of Experts paradigm from width-based to depth specialised computation. DS-MoE introduces expert modules optimised for distinct reasoning depths, shallow pattern recognition, compositional reasoning, logical inference, memory integration, and meta-cognitive supervision. A learned routing network dynamically assembles custom reasoning chains, activating only the necessary experts to match input complexity. The dataset on which we trained and evaluated DS-MoE is on The Pile, an 800GB corpus covering diverse domains such as scientific papers, legal texts, programming code, and web content, enabling systematic assessment across reasoning depths. Experimental results demonstrate that DS-MoE achieves up to 16 per cent computational savings and 35 per cent faster inference compared to uniform-depth transformers, while delivering 2.8 per cent higher accuracy on complex multi-step reasoning benchmarks. Furthermore, routing decisions yield interpretable reasoning chains, enhancing transparency and scalability. These findings establish DS-MoE as a significant advancement in adaptive neural architectures, demonstrating that depth-specialised modular processing can simultaneously improve efficiency, reasoning quality, and interpretability in large-scale language models.
• 摘要：当代变压器体系结构将相同的处理深度应用于所有投入，从而造成效率低下并限制推理质量。简单的事实查询与复杂的逻辑问题进行了相同的多层计算，浪费资源，同时限制了深度推断。为了克服这一点，我们通过深度专家（DS-MOE）提出了一个动态推理链的概念，这是一个模块化的框架，将专家范式的混合物从基于宽度的范式扩展到了基于宽度的深度专用计算。 DS-MOE介绍了针对不同的推理深度，浅模式识别，组成推理，逻辑推断，记忆集成和元认知监督的优化的专家模块。学习的路由网络动态组装自定义推理链，仅激活必要的专家以匹配输入复杂性。我们培训和评估DS-MOE的数据集是在堆上，一个800GB的语料库涵盖了诸如科学论文，法律文本，编程代码和Web内容之类的各种领域，从而跨推理深度启用系统评估。实验结果表明，与统一的深度变压器相比，DS-MOE可实现多达16％的计算节省和35％的推断，同时在复杂的多步骤推理基准上提供了2.8％的精度。此外，路由决策产生可解释的推理链，提高透明度和可扩展性。这些发现确立了DS-MOE作为自适应神经体系结构的重大进步，表明深度专用的模块化处理可以同时提高大规模语言模型中的效率，推理质量和可解释性。

Title: Hierarchical Resolution Transformers: A Wavelet-Inspired Architecture for Multi-Scale Language Understanding

• Authors: Ayan Sar, Sampurna Roy, Kanav Gupta, Anurag Kaushish, Tanupriya Choudhury, Abhijit Kumar
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.20581
• Pdf URL: https://arxiv.org/pdf/2509.20581
• Copy Paste: [[2509.20581]] Hierarchical Resolution Transformers: A Wavelet-Inspired Architecture for Multi-Scale Language Understanding(https://arxiv.org/abs/2509.20581)
• Keywords: gpt
• Abstract: Transformer architectures have achieved state-of-the-art performance across natural language tasks, yet they fundamentally misrepresent the hierarchical nature of human language by processing text as flat token sequences. This results in quadratic computational cost, weak computational cost, weak compositional generalization, and inadequate discourse-level modeling. We propose Hierarchical Resolution Transformer (HRT), a novel wavelet-inspired neural architecture that processes language simultaneously across multiple resolutions, from characters to discourse-level units. HRT constructs a multi-resolution attention, enabling bottom-up composition and top-down contextualization. By employing exponential sequence reduction across scales, HRT achieves O(nlogn) complexity, offering significant efficiency improvements over standard transformers. We evaluated HRT on a diverse suite of benchmarks, including GLUE, SuperGLUE, Long Range Arena, and WikiText-103, and results demonstrated that HRT outperforms standard transformer baselines by an average of +3.8% on GLUE, +4.5% on SuperGLUE, and +6.1% on Long Range Arena, while reducing memory usage by 42% and inference latency by 37% compared to BERT and GPT style models of similar parameter count. Ablation studies confirm the effectiveness of cross-resolution attention and scale-specialized modules, showing that each contributes independently to both efficiency and accuracy. Our findings establish HRT as the first architecture to align computational structure with the hierarchical organization of human language, demonstrating that multi-scale, wavelet-inspired processing yields both theoretical efficiency gains and practical improvements in language understanding.
• 摘要：变形金刚在自然语言任务中实现了最新的表现，但是它们从根本上歪曲了人类语言的等级性质，通过将文本作为平坦的令牌序列处理。这会导致二次计算成本，计算成本弱，组成概括弱和话语级建模不足。我们提出了层次分辨率变压器（HRT），这是一种新颖的小波启发的神经体系结构，从字符到话语级别单位，同时处理语言跨多个分辨率。 HRT构建了多分辨率的关注，从而实现自下而上的组成和自上而下的上下文化。通过在范围内采用指数序列降低，HRT达到O（NLOGN）的复杂性，从而比标准变压器具有显着效率的提高。 We evaluated HRT on a diverse suite of benchmarks, including GLUE, SuperGLUE, Long Range Arena, and WikiText-103, and results demonstrated that HRT outperforms standard transformer baselines by an average of +3.8% on GLUE, +4.5% on SuperGLUE, and +6.1% on Long Range Arena, while reducing memory usage by 42% and inference latency by 37% compared to BERT and GPT样式类似参数计数的模型。消融研究证实了跨分辨率注意力和规模特异化模块的有效性，表明每种模块都对效率和准确性有独立的贡献。我们的发现将HRT与人类语言的分层组织相结合为第一个将计算结构结合起来的架构，表明多尺度，小波启发的处理可带来理论上的效率提高和语言理解的实际改进。

Title: FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models

• Authors: Amin Karimi Monsefi, Nikhil Bhendawade, Manuel Rafael Ciosici, Dominic Culver, Yizhe Zhang, Irina Belousova
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.20624
• Pdf URL: https://arxiv.org/pdf/2509.20624
• Copy Paste: [[2509.20624]] FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models(https://arxiv.org/abs/2509.20624)
• Keywords: language model
• Abstract: Autoregressive language models (ARMs) deliver strong likelihoods, but are inherently serial: they generate one token per forward pass, which limits throughput and inflates latency for long sequences. Diffusion Language Models (DLMs) parallelize across positions and thus appear promising for language generation, yet standard discrete diffusion typically needs hundreds to thousands of model evaluations to reach high quality, trading serial depth for iterative breadth. We introduce FS-DFM, Few-Step Discrete Flow-Matching. A discrete flow-matching model designed for speed without sacrificing quality. The core idea is simple: make the number of sampling steps an explicit parameter and train the model to be consistent across step budgets, so one big move lands where many small moves would. We pair this with a reliable update rule that moves probability in the right direction without overshooting, and with strong teacher guidance distilled from long-run trajectories. Together, these choices make few-step sampling stable, accurate, and easy to control. On language modeling benchmarks, FS-DFM with 8 sampling steps achieves perplexity parity with a 1,024-step discrete-flow baseline for generating 1,024 tokens using a similar-size model, delivering up to 128 times faster sampling and corresponding latency/throughput gains.
• 摘要：自回归语言模型（ARM）具有很强的可能性，但本质上是串行的：它们每次向前传球产生一个令牌，这限制了吞吐量并膨胀长序列的潜伏期。扩散语言模型（DLMS）在跨位置并行化，因此对于语言产生而言似乎很有希望，但是标准离散扩散通常需要数百到数千个模型评估才能达到高质量，以交易序列深度，以进行迭代宽度。我们介绍了FS-DFM，几步离散流程匹配。一个用于速度的离散流匹配模型而不牺牲质量。核心想法很简单：将采样步骤的数量变为明确的参数，并训练模型在步骤预算中保持一致，因此，许多小动作都可以在一个大型移动的土地上进行。我们将其与可靠的更新规则配对，该规则将概率朝着正确的方向移动而不过度旋转，并与长期轨迹蒸馏出强大的教师指导。这些选择共同使稳定，准确且易于控制的步骤采样。在语言建模基准测试基准上，具有8个采样步骤的FS-DFM具有1,024步的离散基线基线，可使用类似尺寸的型号生成1,024个令牌，可提供多达128倍的快速采样和相应的延迟/透视/透视率。

Title: Look Before you Leap: Estimating LLM Benchmark Scores from Descriptions

• Authors: Jungsoo Park, Ethan Mendes, Gabriel Stanovsky, Alan Ritter
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.20645
• Pdf URL: https://arxiv.org/pdf/2509.20645
• Copy Paste: [[2509.20645]] Look Before you Leap: Estimating LLM Benchmark Scores from Descriptions(https://arxiv.org/abs/2509.20645)
• Keywords: language model, gpt, llm
• Abstract: Progress in large language models is constrained by an evaluation bottleneck: build a benchmark, evaluate models and settings, then iterate. We therefore ask a simple question: can we forecast outcomes before running any experiments? We study text-only performance forecasting: estimating a model's score from a redacted task description and intended configuration, with no access to dataset instances. To support systematic study, we curate PRECOG, a corpus of redacted description-performance pairs spanning diverse tasks, domains, and metrics. Experiments show the task is challenging but feasible: models equipped with a retrieval module that excludes source papers achieve moderate prediction performance with well-calibrated uncertainty, reaching mean absolute error as low as 8.7 on the Accuracy subset at high-confidence thresholds. Our analysis indicates that stronger reasoning models engage in diverse, iterative querying, whereas current open-source models lag and often skip retrieval or gather evidence with limited diversity. We further test a zero-leakage setting, forecasting on newly released datasets or experiments before their papers are indexed, where GPT-5 with built-in web search still attains nontrivial prediction accuracy. Overall, our corpus and analyses offer an initial step toward open-ended anticipatory evaluation, supporting difficulty estimation and smarter experiment prioritization.
• 摘要：大语言模型的进度受到评估瓶颈的约束：建立基准测试，评估模型和设置，然后迭代。因此，我们提出一个简单的问题：我们可以在进行任何实验之前预测结果吗？我们研究纯文本性能预测：从编辑的任务描述和预期配置中估算模型的分数，而无法访问数据集实例。为了支持系统的研究，我们策划了Predog，这是一个涵盖各种任务，域和指标的编辑描述 - 性能对。实验表明该任务具有挑战性但可行：配备了检索模块的模型，该模块排除了源论文的预测性能中等的预测性能，并且在高度保存阈值下的准确性子集中达到了8.7的平均绝对误差。我们的分析表明，更强大的推理模型参与了各种各样的迭代查询，而当前的开源模型滞后，通常跳过检索或收集具有有限多样性的证据。我们进一步测试了零裂口设置，在索引之前对新发布的数据集或实验进行了预测，其中带有内置Web搜索的GPT-5仍然具有非平凡的预测准确性。总体而言，我们的语料库和分析为开放式预期评估提供了第一步，支持难度估计和更智能的实验优先级。

Title: Enhancing Molecular Property Prediction with Knowledge from Large Language Models

• Authors: Peng Zhou, Lai Hou Tim, Zhixiang Cheng, Kun Xie, Chaoyi Li, Wei Liu, Xiangxiang Zeng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20664
• Pdf URL: https://arxiv.org/pdf/2509.20664
• Copy Paste: [[2509.20664]] Enhancing Molecular Property Prediction with Knowledge from Large Language Models(https://arxiv.org/abs/2509.20664)
• Keywords: language model, gpt, llm, hallucination, prompt
• Abstract: Predicting molecular properties is a critical component of drug discovery. Recent advances in deep learning, particularly Graph Neural Networks (GNNs), have enabled end-to-end learning from molecular structures, reducing reliance on manual feature engineering. However, while GNNs and self-supervised learning approaches have advanced molecular property prediction (MPP), the integration of human prior knowledge remains indispensable, as evidenced by recent methods that leverage large language models (LLMs) for knowledge extraction. Despite their strengths, LLMs are constrained by knowledge gaps and hallucinations, particularly for less-studied molecular properties. In this work, we propose a novel framework that, for the first time, integrates knowledge extracted from LLMs with structural features derived from pre-trained molecular models to enhance MPP. Our approach prompts LLMs to generate both domain-relevant knowledge and executable code for molecular vectorization, producing knowledge-based features that are subsequently fused with structural representations. We employ three state-of-the-art LLMs, GPT-4o, GPT-4.1, and DeepSeek-R1, for knowledge extraction. Extensive experiments demonstrate that our integrated method outperforms existing approaches, confirming that the combination of LLM-derived knowledge and structural information provides a robust and effective solution for MPP.
• 摘要：预测分子特性是药物发现的关键组成部分。深度学习的最新进展，尤其是图形神经网络（GNNS），已使端到端学习从分子结构中进行了学习，从而减少了对手动功能工程的依赖。但是，尽管GNN和自我监管的学习方法具有先进的分子财产预测（MPP），但人类先验知识的整合仍然是必不可少的，这是通过利用大型语言模型（LLMS）进行知识提取的最新方法所证明的。尽管具有优势，但LLM仍受到知识差距和幻觉的限制，尤其是针对较少的分子特性。在这项工作中，我们提出了一个新颖的框架，该框架首次将从LLM提取的知识与预先训练的分子模型得出的结构特征相结合以增强MPP。我们的方法促使LLMS生成与域相关的知识和用于分子矢量化的可执行代码，从而产生基于知识的特征，这些特征随后与结构表示。我们采用三个最先进的LLM，GPT-4O，GPT-4.1和DeepSeek-R1，用于提取知识。广泛的实验表明，我们的综合方法优于现有方法，证实了LLM衍生的知识和结构信息的组合为MPP提供了强大而有效的解决方案。

Title: Overcoming Black-box Attack Inefficiency with Hybrid and Dynamic Select Algorithms

• Authors: Abhinay Shankar Belde, Rohit Ramkumar, Jonathan Rusert
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20699
• Pdf URL: https://arxiv.org/pdf/2509.20699
• Copy Paste: [[2509.20699]] Overcoming Black-box Attack Inefficiency with Hybrid and Dynamic Select Algorithms(https://arxiv.org/abs/2509.20699)
• Keywords: llm
• Abstract: Adversarial text attack research plays a crucial role in evaluating the robustness of NLP models. However, the increasing complexity of transformer-based architectures has dramatically raised the computational cost of attack testing, especially for researchers with limited resources (e.g., GPUs). Existing popular black-box attack methods often require a large number of queries, which can make them inefficient and impractical for researchers. To address these challenges, we propose two new attack selection strategies called Hybrid and Dynamic Select, which better combine the strengths of previous selection algorithms. Hybrid Select merges generalized BinarySelect techniques with GreedySelect by introducing a size threshold to decide which selection algorithm to use. Dynamic Select provides an alternative approach of combining the generalized Binary and GreedySelect by learning which lengths of texts each selection method should be applied to. This greatly reduces the number of queries needed while maintaining attack effectiveness (a limitation of BinarySelect). Across 4 datasets and 6 target models, our best method(sentence-level Hybrid Select) is able to reduce the number of required queries per attack up 25.82\% on average against both encoder models and LLMs, without losing the effectiveness of the attack.
• 摘要：对抗文本攻击研究在评估NLP模型的鲁棒性中起着至关重要的作用。但是，基于变压器的架构的复杂性日益严重，大大提高了攻击测试的计算成本，尤其是对于资源有限的研究人员（例如GPU）。现有流行的黑盒攻击方法通常需要大量查询，这可能会使它们效率低下且对研究人员不切实际。为了应对这些挑战，我们提出了两种称为混合和动态选择的新攻击选择策略，它们更好地结合了先前选择算法的优势。混合选择通过引入尺寸阈值来确定要使用哪种选择算法，将广义二进制选择技术与GreedySelect合并。动态选择提供了一种替代方法，可以通过学习应将每个选择方法的文本长度结合到合并广义的二进制和贪婪选择。这大大减少了在保持攻击效率（二进制选择的限制）的同时所需的查询数量。在4个数据集和6个目标模型中，我们的最佳方法（句子级混合选择）能够平均每次攻击所需的查询数量为25.82 \％，而对于Encoder模型和LLM，而不会失去攻击的有效性。

Title: MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with Closed-Source Large-Audio Language Model

• Authors: Hsiao-Ying Huang, Yi-Cheng Lin, Hung-yi Lee
• Subjects: cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2509.20706
• Pdf URL: https://arxiv.org/pdf/2509.20706
• Copy Paste: [[2509.20706]] MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with Closed-Source Large-Audio Language Model(https://arxiv.org/abs/2509.20706)
• Keywords: language model
• Abstract: Large audio-language models (LALMs) show strong zero-shot ability on speech tasks, suggesting promise for speech emotion recognition (SER). However, SER in real-world deployments often fails under domain mismatch, where source data are unavailable and powerful LALMs are accessible only through an API. We ask: given only unlabeled target-domain audio and an API-only LALM, can a student model be adapted to outperform the LALM in the target domain? To this end, we propose MI-Fuse, a denoised label fusion framework that supplements the LALM with a source-domain trained SER classifier as an auxiliary teacher. The framework draws multiple stochastic predictions from both teachers, weights their mean distributions by mutual-information-based uncertainty, and stabilizes training with an exponential moving average teacher. Experiments across three public emotion datasets and six cross-domain transfers show consistent gains, with the student surpassing the LALM and outperforming the strongest baseline by 3.9%. This approach strengthens emotion-aware speech systems without sharing source data, enabling realistic adaptation.
• 摘要：大型音频语言模型（LALMS）在语音任务上表现出强大的零拍能，这表明了语音情绪识别（SER）的希望。但是，现实世界部署中的SER通​​常在域不匹配下失败，在域不匹配下，源数据不可用，并且仅通过API才能访问强大的LALMS。我们问：只有未标记的目标域音频和仅API-LALM，学生模型是否可以适应以优于目标域中的LALM？为此，我们提出了Mi-Fuse，这是一个DeNOCIED的标签融合框架，该框架将LALM用源头训练的Ser分类器作为辅助老师补充了LALM。该框架从两位教师那里得出了多个随机预测，通过基于共同信息的不确定性来加权其平均分布，并通过指数移动的平均教师稳定培训。在三个公共情绪数据集和六个跨域转移的实验表现出一致的收益，学生超过了LALM，并以最强的基准优于3.9％。这种方法可以增强情绪感知的语音系统，而无需共享源数据，从而实现了现实的适应。

Title: SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs

• Authors: Jiacheng Lin, Zhongruo Wang, Kun Qian, Tian Wang, Arvind Srinivasan, Hansi Zeng, Ruochen Jiao, Xie Zhou, Jiri Gesi, Dakuo Wang, Yufan Guo, Kai Zhong, Weiqi Zhang, Sujay Sanghavi, Changyou Chen, Hyokun Yun, Lihong Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20758
• Pdf URL: https://arxiv.org/pdf/2509.20758
• Copy Paste: [[2509.20758]] SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs(https://arxiv.org/abs/2509.20758)
• Keywords: language model, llm
• Abstract: Supervised Fine-Tuning (SFT) on domain-specific datasets is a common approach to adapt Large Language Models (LLMs) to specialized tasks but is often believed to degrade their general capabilities. In this work, we revisit this trade-off and present both empirical and theoretical insights. First, we show that SFT does not always hurt: using a smaller learning rate can substantially mitigate general performance degradation while preserving comparable target-domain performance. We then provide a theoretical analysis that explains these phenomena and further motivates a new method, Token-Adaptive Loss Reweighting (TALR). Building on this, and recognizing that smaller learning rates alone do not fully eliminate general-performance degradation in all cases, we evaluate a range of strategies for reducing general capability loss, including L2 regularization, LoRA, model averaging, FLOW, and our proposed TALR. Experimental results demonstrate that while no method completely eliminates the trade-off, TALR consistently outperforms these baselines in balancing domain-specific gains and general capabilities. Finally, we distill our findings into practical guidelines for adapting LLMs to new domains: (i) using a small learning rate to achieve a favorable trade-off, and (ii) when a stronger balance is further desired, adopt TALR as an effective strategy.
• 摘要：在特定领域的数据集上有监督的微调（SFT）是将大型语言模型（LLMS）适应专用任务的常见方法，但通常被认为会降低其一般能力。在这项工作中，我们重新审视了这一权衡，并介绍了经验和理论见解。首先，我们表明SFT并不总是受到伤害：使用较小的学习率可以大大减轻一般绩效降解，同时保留可比的目标域性能。然后，我们提供了理论分析，该分析解释了这些现象，并进一步激发了一种新的方法，即使自适应损失重新加权（TALR）。在此基础上，我们认识到单独的学习率并不能在所有情况下都无法完全消除一般性绩效降解，因此我们评估了一系列降低一般能力损失的策略，包括L2正则化，LORA，LORA，模型平均，流程，流动以及我们拟议的TALR。实验结果表明，尽管没有任何方法完全消除了权衡，但塔尔在平衡域特异性增长和一般能力方面始终优于这些基准。最后，我们将发现提炼为将LLM适应新领域的实用指南：（i）使用较小的学习率来实现有利的权衡，（ii）当需要更强大的平衡，采用TALR作为有效的策略。

Title: Towards Atoms of Large Language Models

• Authors: Chenhui Hu, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20784
• Pdf URL: https://arxiv.org/pdf/2509.20784
• Copy Paste: [[2509.20784]] Towards Atoms of Large Language Models(https://arxiv.org/abs/2509.20784)
• Keywords: language model, llm
• Abstract: The fundamental units of internal representations in large language models (LLMs) remain undefined, limiting further understanding of their mechanisms. Neurons or features are often regarded as such units, yet neurons suffer from polysemy, while features face concerns of unreliable reconstruction and instability. To address this issue, we propose the Atoms Theory, which defines such units as atoms. We introduce the atomic inner product (AIP) to correct representation shifting, formally define atoms, and prove the conditions that atoms satisfy the Restricted Isometry Property (RIP), ensuring stable sparse representations over atom set and linking to compressed sensing. Under stronger conditions, we further establish the uniqueness and exact $\ell_1$ recoverability of the sparse representations, and provide guarantees that single-layer sparse autoencoders (SAEs) with threshold activations can reliably identify the atoms. To validate the Atoms Theory, we train threshold-activated SAEs on Gemma2-2B, Gemma2-9B, and Llama3.1-8B, achieving 99.9% sparse reconstruction across layers on average, and more than 99.8% of atoms satisfy the uniqueness condition, compared to 0.5% for neurons and 68.2% for features, showing that atoms more faithfully capture intrinsic representations of LLMs. Scaling experiments further reveal the link between SAEs size and recovery capacity. Overall, this work systematically introduces and validates Atoms Theory of LLMs, providing a theoretical framework for understanding internal representations and a foundation for mechanistic interpretability. Code available at this https URL.
• 摘要：大语言模型（LLM）中内部表示的基本单位仍然不确定，从而限制了对其机制的进一步理解。神经元或特征通常被视为这样的单位，但神经元患有多义，而特征则面临着不可靠的重建和不稳定性的关注。为了解决这个问题，我们提出了原子理论，该理论定义了原子等单元。我们介绍原子内部产品（AIP）以纠正表示的变化，正式定义原子，并证明原子满足受限的等轴测特性（RIP）的条件，确保原子集的稳定稀疏表示并链接到压缩感应。在更强的条件下，我们进一步建立了稀疏表示形式的唯一性和确切的$ \ ell_1 $可恢复性，并保证具有阈值激活的单层稀疏自动编码器（SAE）可以可靠地识别原子。为了验证原子理论，我们在GEMMA2-2B，GEMMA2-9B和LLAMA3.1-8B上训练阈值激活的SAE，在平均层次上实现了99.9％的稀疏重建，与Neurons和68.2％相比，占据了0.5％的独特状态，超过99.8％的原子满足了独特性的范围。扩展实验进一步揭示了SAES大小与恢复能力之间的联系。总体而言，这项工作系统地引入并验证了LLM的原子理论，为理解内部表示形式和机械解释性的基础提供了理论框架。可在此HTTPS URL上找到代码。

Title: Few-Shot and Training-Free Review Generation via Conversational Prompting

• Authors: Genki Kusano
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.20805
• Pdf URL: https://arxiv.org/pdf/2509.20805
• Copy Paste: [[2509.20805]] Few-Shot and Training-Free Review Generation via Conversational Prompting(https://arxiv.org/abs/2509.20805)
• Keywords: language model, llm, prompt
• Abstract: Personalized review generation helps businesses understand user preferences, yet most existing approaches assume extensive review histories of the target user or require additional model training. Real-world applications often face few-shot and training-free situations, where only a few user reviews are available and fine-tuning is infeasible. It is well known that large language models (LLMs) can address such low-resource settings, but their effectiveness depends on prompt engineering. In this paper, we propose Conversational Prompting, a lightweight method that reformulates user reviews as multi-turn conversations. Its simple variant, Simple Conversational Prompting (SCP), relies solely on the user's own reviews, while the contrastive variant, Contrastive Conversational Prompting (CCP), inserts reviews from other users or LLMs as incorrect replies and then asks the model to correct them, encouraging the model to produce text in the user's style. Experiments on eight product domains and five LLMs showed that the conventional non-conversational prompt often produced reviews similar to those written by random users, based on text-based metrics such as ROUGE-L and BERTScore, and application-oriented tasks like user identity matching and sentiment analysis. In contrast, both SCP and CCP produced reviews much closer to those of the target user, even when each user had only two reviews. CCP brings further improvements when high-quality negative examples are available, whereas SCP remains competitive when such data cannot be collected. These results suggest that conversational prompting offers a practical solution for review generation under few-shot and training-free constraints.
• 摘要：个性化的审核生成可帮助企业了解用户的偏好，但是大多数现有方法都会对目标用户进行广泛的审查历史，或者需要其他模型培训。现实世界中的应用程序通常面临很少的无镜头和无培训的情况，在那里只有几个用户评论，并且微调是不可行的。众所周知，大型语言模型（LLM）可以解决这种低资源设置，但它们的有效性取决于迅速的工程。在本文中，我们提出了对话提示，这是一种轻巧的方法，将用户评论作为多转向对话进行了重新制定。它简单的变体，简单的对话提示（SCP）仅依赖用户自己的评论，而对比度变体的对比度对话提示（CCP），将其他用户或LLMS的评论插入不正确的答案，然后要求型号纠正模型，以纠正模型，鼓励模型在用户的样式中产生文本。对八个产品域和五个LLM的实验表明，基于基于文本的指标，例如Rouge-L和BertScore，以及以应用程序为导向的任务，例如用户身份匹配和情感分析，通常会产生与随机用户所写的评论相似的评论。相比之下，即使每个用户只有两个评论，SCP和CCP都会产生与目标用户的评论。当有高质量的负示例可用时，CCP会带来进一步的改进，而当无法收集此类数据时，SCP仍然具有竞争力。这些结果表明，对话提示为在几次无训练和无训练的约束下的审查生成提供了实用的解决方案。

Title: Enrich-on-Graph: Query-Graph Alignment for Complex Reasoning with LLM Enriching

• Authors: Songze Li, Zhiqiang Liu, Zhengke Gui, Huajun Chen, Wen Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20810
• Pdf URL: https://arxiv.org/pdf/2509.20810
• Copy Paste: [[2509.20810]] Enrich-on-Graph: Query-Graph Alignment for Complex Reasoning with LLM Enriching(https://arxiv.org/abs/2509.20810)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) exhibit strong reasoning capabilities in complex tasks. However, they still struggle with hallucinations and factual errors in knowledge-intensive scenarios like knowledge graph question answering (KGQA). We attribute this to the semantic gap between structured knowledge graphs (KGs) and unstructured queries, caused by inherent differences in their focuses and structures. Existing methods usually employ resource-intensive, non-scalable workflows reasoning on vanilla KGs, but overlook this gap. To address this challenge, we propose a flexible framework, Enrich-on-Graph (EoG), which leverages LLMs' prior knowledge to enrich KGs, bridge the semantic gap between graphs and queries. EoG enables efficient evidence extraction from KGs for precise and robust reasoning, while ensuring low computational costs, scalability, and adaptability across different methods. Furthermore, we propose three graph quality evaluation metrics to analyze query-graph alignment in KGQA task, supported by theoretical validation of our optimization objectives. Extensive experiments on two KGQA benchmark datasets indicate that EoG can effectively generate high-quality KGs and achieve the state-of-the-art performance. Our code and data are available at this https URL.
• 摘要：大型语言模型（LLMS）在复杂的任务中具有强大的推理能力。但是，他们仍然在知识图形问题（KGQA）（KGQA）（KGQA）（KGQA）（KGQA）等知识密集情况下的幻觉和事实错误中挣扎。我们将其归因于结构化知识图（kg）和非结构化查询之间的语义差距，这是由于其焦点和结构固有的差异引起的。现有的方法通常在香草千克中采用资源密集型，不可估计的工作流程推理，但忽略了这一差距。为了应对这一挑战，我们提出了一个灵活的框架，即富集式（EOG），该框架利用LLMS的先验知识丰富了KG，弥合了图和查询之间的语义差距。 EOG可以从KGS中提取有效的证据，以确保跨不同方法的计算成本，可伸缩性和适应性较低。此外，我们提出了三个图形质量评估指标，以分析KGQA任务中的查询图形对齐，并由我们优化目标的理论验证支持。在两个KGQA基准数据集上进行的广泛实验表明，EOG可以有效地产生高质量的kg并实现最先进的性能。我们的代码和数据可在此HTTPS URL上找到。

Title: Leveraging What's Overfixed: Post-Correction via LLM Grammatical Error Overcorrection

• Authors: Taehee Park, Heejin Do, Gary Geunbae Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20811
• Pdf URL: https://arxiv.org/pdf/2509.20811
• Copy Paste: [[2509.20811]] Leveraging What's Overfixed: Post-Correction via LLM Grammatical Error Overcorrection(https://arxiv.org/abs/2509.20811)
• Keywords: language model, llm
• Abstract: Robust supervised fine-tuned small Language Models (sLMs) often show high reliability but tend to undercorrect. They achieve high precision at the cost of low recall. Conversely, Large Language Models (LLMs) often show the opposite tendency, making excessive overcorrection, leading to low precision. To effectively harness the strengths of LLMs to address the recall challenges in sLMs, we propose Post-Correction via Overcorrection (PoCO), a novel approach that strategically balances recall and precision. PoCO first intentionally triggers overcorrection via LLM to maximize recall by allowing comprehensive revisions, then applies a targeted post-correction step via fine-tuning smaller models to identify and refine erroneous outputs. We aim to harmonize both aspects by leveraging the generative power of LLMs while preserving the reliability of smaller supervised models. Our extensive experiments demonstrate that PoCO effectively balances GEC performance by increasing recall with competitive precision, ultimately improving the overall quality of grammatical error correction.
• 摘要：强大的监督微调的小语言模型（SLM）通常显示出很高的可靠性，但往往不正确。他们以低召回成本获得了高精度。相反，大型语言模型（LLMS）通常显示出相反的趋势，使过度校正过度校正，导致精确度较低。为了有效利用LLM的优势应对SLM中的召回挑战，我们通过过度校正（POCO）提出了校正后，这种新颖方法在战略上平衡了回忆和精度。 POCO首先通过LLM故意触发过度纠正，以通过允许全面修订来最大化召回，然后通过微调较小的模型应用目标后校正步骤，以识别和完善错误的输出。我们的目标是通过利用LLM的生成能力，同时保留较小的监督模型的可靠性来协调这两个方面。我们的广泛实验表明，POCO通过以竞争精度提高回忆来有效地平衡GEC的性能，最终提高语法误差校正的总体质量。

Title: Distilling Many-Shot In-Context Learning into a Cheat Sheet

• Authors: Ukyo Honda, Soichiro Murakami, Peinan Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20820
• Pdf URL: https://arxiv.org/pdf/2509.20820
• Copy Paste: [[2509.20820]] Distilling Many-Shot In-Context Learning into a Cheat Sheet(https://arxiv.org/abs/2509.20820)
• Keywords: language model, llm
• Abstract: Recent advances in large language models (LLMs) enable effective in-context learning (ICL) with many-shot examples, but at the cost of high computational demand due to longer input tokens. To address this, we propose cheat-sheet ICL, which distills the information from many-shot ICL into a concise textual summary (cheat sheet) used as the context at inference time. Experiments on challenging reasoning tasks show that cheat-sheet ICL achieves comparable or better performance than many-shot ICL with far fewer tokens, and matches retrieval-based ICL without requiring test-time retrieval. These findings demonstrate that cheat-sheet ICL is a practical alternative for leveraging LLMs in downstream tasks.
• 摘要：大型语言模型（LLMS）的最新进展使有效的内在学习（ICL）具有许多示例，但由于更长的输入令牌而导致高计算需求的代价。为了解决这个问题，我们提出了作弊表ICL，将信息从许多弹药ICL提炼成简洁的文本摘要（备忘单），用作推理时间的上下文。关于挑战性推理任务的实验表明，作弊表ICL比许多具有更少令牌的ICL的ICL具有可比性或更好的性能，并且与基于检索的ICL匹配而无需测试时间检索。这些发现表明，作弊表ICL是利用LLM在下游任务中的实用替代方法。

Title: Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Search

• Authors: Shuo Huang, Xingliang Yuan, Gholamreza Haffari, Lizhen Qu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20838
• Pdf URL: https://arxiv.org/pdf/2509.20838
• Copy Paste: [[2509.20838]] Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Search(https://arxiv.org/abs/2509.20838)
• Keywords: language model, llm
• Abstract: The increasing adoption of large language models (LLMs) in cloud-based services has raised significant privacy concerns, as user inputs may inadvertently expose sensitive information. Existing text anonymization and de-identification techniques, such as rule-based redaction and scrubbing, often struggle to balance privacy preservation with text naturalness and utility. In this work, we propose a zero-shot, tree-search-based iterative sentence rewriting algorithm that systematically obfuscates or deletes private information while preserving coherence, relevance, and naturalness. Our method incrementally rewrites privacy-sensitive segments through a structured search guided by a reward model, enabling dynamic exploration of the rewriting space. Experiments on privacy-sensitive datasets show that our approach significantly outperforms existing baselines, achieving a superior balance between privacy protection and utility preservation.
• 摘要：在基于云的服务中，大型语言模型（LLM）的采用越来越多，引起了严重的隐私问题，因为用户输入可能无意间暴露了敏感信息。现有的文本匿名和去识别技术，例如基于规则的修订和擦洗，通常很难平衡隐私保护与文本自然和实用性。在这项工作中，我们提出了一个基于零射门的，基于树搜索的迭代句子重写算法，该算法会系统地混淆或删除私人信息，同时保持连贯，相关性和自然性。我们的方法通过以奖励模型为指导的结构化搜索来逐渐重写对隐私敏感的段，从而使重写空间的动态探索能够进行动态探索。关于隐私敏感数据集的实验表明，我们的方法极大地胜过现有的基准，在隐私保护和公用事业保护之间取得了卓越的平衡。

Title: Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generation

• Authors: Guo Chen, Qiuyuan Li, Qiuxian Li, Hongliang Dai, Xiang Chen, Piji Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20859
• Pdf URL: https://arxiv.org/pdf/2509.20859
• Copy Paste: [[2509.20859]] Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generation(https://arxiv.org/abs/2509.20859)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: In retrieval-augmented generation (RAG) question answering systems, generating citations for large language model (LLM) outputs enhances verifiability and helps users identify potential hallucinations. However, we observe two problems in the citations produced by existing attribution methods. First, the citations are typically provided at the sentence or even paragraph level. Long sentences or paragraphs may include a substantial amount of irrelevant content. Second, sentence-level citations may omit information that is essential for verifying the output, forcing users to read the surrounding context. In this paper, we propose generating sub-sentence citations that are both concise and sufficient, thereby reducing the effort required by users to confirm the correctness of the generated output. To this end, we first develop annotation guidelines for such citations and construct a corresponding dataset. Then, we propose an attribution framework for generating citations that adhere to our standards. This framework leverages LLMs to automatically generate fine-tuning data for our task and employs a credit model to filter out low-quality examples. Our experiments on the constructed dataset demonstrate that the propose approach can generate high-quality and more readable citations.
• 摘要：在检索提升的生成（RAG）问答系统中，为大语言模型（LLM）产生引用可增强可验证性，并帮助用户识别潜在的幻觉。但是，我们观察到现有归因方法产生的引用中有两个问题。首先，引用通常是在句子甚至段落级别提供的。长句或段落可能包括大量无关的内容。其次，句子级引用可能会忽略对于验证输出至关重要的信息，迫使用户阅读周围环境。在本文中，我们建议生成既简洁又充分的次句引用，从而减少用户确认生成的输出的正确性所需的努力。为此，我们首先制定了此类引用的注释准则，并构建相应的数据集。然后，我们提出了一个归因框架，以生成遵守我们标准的引用。该框架利用LLMS自动为我们的任务生成微调数据，并采用信用模型来过滤低质量的示例。我们对构造数据集的实验表明，提出的方法可以产生高质量和更可读的引用。

Title: WeFT: Weighted Entropy-driven Fine-Tuning for dLLMs

• Authors: Guowei Xu, Wenxin Xu, Jiawang Zhao, Kaisheng Ma
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20863
• Pdf URL: https://arxiv.org/pdf/2509.20863
• Copy Paste: [[2509.20863]] WeFT: Weighted Entropy-driven Fine-Tuning for dLLMs(https://arxiv.org/abs/2509.20863)
• Keywords: language model, llm
• Abstract: Diffusion models have recently shown strong potential in language modeling, offering faster generation compared to traditional autoregressive approaches. However, applying supervised fine-tuning (SFT) to diffusion models remains challenging, as they lack precise probability estimates at each denoising step. While the diffusion mechanism enables the model to reason over entire sequences, it also makes the generation process less predictable and often inconsistent. This highlights the importance of controlling key tokens that guide the direction of generation. To address this issue, we propose WeFT, a weighted SFT method for diffusion language models, where tokens are assigned different weights based on their entropy. Derived from diffusion theory, WeFT delivers substantial gains: training on s1K, s1K-1.1, and 3k samples from open-r1, it achieves relative improvements of 39%, 64%, and 83% over standard SFT on four widely used reasoning benchmarks (Sudoku, Countdown, GSM8K, and MATH-500). The code and models will be made publicly available.
• 摘要：扩散模型最近在语言建模方面表现出强大的潜力，与传统的自回归方法相比，具有更快的生成。但是，将监督的微调（SFT）应用于扩散模型仍然具有挑战性，因为它们在每个降级步骤中都缺乏精确的概率估计。尽管扩散机制使模型能够在整个序列上进行推理，但它也使生成过程降低了可预测的，并且通常不一致。这突出了控制指导生成方向的关键令牌的重要性。为了解决这个问题，我们提出了Weft，这是一种用于扩散语言模型的加权SFT方法，在该方法中，令牌是根据其熵分配不同权重的。 Weft源自扩散理论，可带来可观的增长：S1K，S1K-1.1和3K样本的训练，从Open-R1中获得了39％，64％和83％的相对改善，比标准SFT在四个广泛使用的推理基准（Sudoku，Countoku，Countdown，Countdown，countdown，gsm8k和Math-500）上实现了相对的SFT。代码和模型将公开可用。

Title: Single Answer is Not Enough: On Generating Ranked Lists with Medical Reasoning Models

• Authors: Pittawat Taveekitworachai, Natpatchara Pongjirapat, Krittaphas Chaisutyakorn, Piyalitt Ittichaiwong, Tossaporn Saengja, Kunat Pipatanakul
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.20866
• Pdf URL: https://arxiv.org/pdf/2509.20866
• Copy Paste: [[2509.20866]] Single Answer is Not Enough: On Generating Ranked Lists with Medical Reasoning Models(https://arxiv.org/abs/2509.20866)
• Keywords: prompt
• Abstract: This paper presents a systematic study on enabling medical reasoning models (MRMs) to generate ranked lists of answers for open-ended questions. Clinical decision-making rarely relies on a single answer but instead considers multiple options, reducing the risks of narrow perspectives. Yet current MRMs are typically trained to produce only one answer, even in open-ended settings. We propose an alternative format: ranked lists and investigate two approaches: prompting and fine-tuning. While prompting is a cost-effective way to steer an MRM's response, not all MRMs generalize well across different answer formats: choice, short text, and list answers. Based on our prompting findings, we train and evaluate MRMs using supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT). SFT teaches a model to imitate annotated responses, and RFT incentivizes exploration through the responses that maximize a reward. We propose new reward functions targeted at ranked-list answer formats, and conduct ablation studies for RFT. Our results show that while some SFT models generalize to certain answer formats, models trained with RFT are more robust across multiple formats. We also present a case study on a modified MedQA with multiple valid answers, finding that although MRMs might fail to select the benchmark's preferred ground truth, they can recognize valid answers. To the best of our knowledge, this is the first systematic investigation of approaches for enabling MRMs to generate answers as ranked lists. We hope this work provides a first step toward developing alternative answer formats that are beneficial beyond single answers in medical domains.
• 摘要：本文介绍了一项有关使医学推理模型（MRM）的系统研究，以生成对开放式问题的答案的排名列表。临床决策很少依赖一个答案，而是考虑了多种选择，从而降低了狭窄的观点的风险。但是，即使在开放式设置中，当前的MRM通常也经过培训，即使在开放式设置中也只能产生一个答案。我们提出了一种替代格式：排名列表并研究两种方法：提示和微调。虽然提示是指导MRM响应的一种经济有效的方法，但并非所有MRM都跨越了不同的答案格式：选择，短文和列表答案。根据我们的促进发现，我们使用监督的微调（SFT）和增强微调（RFT）培训和评估MRM。 SFT教授一个模型来模仿带注释的响应，RFT通过最大化奖励的响应来激发探索。我们提出针对排名列表答案格式的新奖励功能，并为RFT进行消融研究。我们的结果表明，虽然某些SFT模型推广到某些答案格式，但在多种格式中，经过RFT训练的模型更强大。我们还提出了一个具有多个有效答案的修改后的MEDQA的案例研究，发现尽管MRM可能无法选择基准的首选地面真相，但他们可以识别有效的答案。据我们所知，这是对使MRMS能够作为排名列表生成答案的方法的首次系统调查。我们希望这项工作为开发替代答案格式提供了第一步，这些格式超出了医疗领域的单个答案。

Title: Learning to Summarize by Learning to Quiz: Adversarial Agentic Collaboration for Long Document Summarization

• Authors: Weixuan Wang, Minghao Wu, Barry Haddow, Alexandra Birch
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20900
• Pdf URL: https://arxiv.org/pdf/2509.20900
• Copy Paste: [[2509.20900]] Learning to Summarize by Learning to Quiz: Adversarial Agentic Collaboration for Long Document Summarization(https://arxiv.org/abs/2509.20900)
• Keywords: language model, llm, agent
• Abstract: Long document summarization remains a significant challenge for current large language models (LLMs), as existing approaches commonly struggle with information loss, factual inconsistencies, and coherence issues when processing excessively long documents. We propose SummQ, a novel adversarial multi-agent framework that addresses these limitations through collaborative intelligence between specialized agents operating in two complementary domains: summarization and quizzing. Our approach employs summary generators and reviewers that work collaboratively to create and evaluate comprehensive summaries, while quiz generators and reviewers create comprehension questions that serve as continuous quality checks for the summarization process. This adversarial dynamic, enhanced by an examinee agent that validates whether the generated summary contains the information needed to answer the quiz questions, enables iterative refinement through multifaceted feedback mechanisms. We evaluate SummQ on three widely used long document summarization benchmarks. Experimental results demonstrate that our framework significantly outperforms existing state-of-the-art methods across ROUGE and BERTScore metrics, as well as in LLM-as-a-Judge and human evaluations. Our comprehensive analyses reveal the effectiveness of the multi-agent collaboration dynamics, the influence of different agent configurations, and the impact of the quizzing mechanism. This work establishes a new approach for long document summarization that uses adversarial agentic collaboration to improve summarization quality.
• 摘要：长期的文档摘要仍然是当前大语模型（LLM）的重大挑战，因为现有方法通常在处理过长的文档时与信息丢失，事实不一致和连贯性问题困难。我们提出了Summq，这是一种新颖的对抗性多代理框架，通过在两个互补域中运行的专门智力之间的协作智能来解决这些局限性：摘要和测验。我们的方法采用了摘要生成器和审阅者，可以协作创建和评估全面的摘要，而测验生成器和审阅者创建了理解问题，这些问题是摘要过程的持续质量检查。这种对抗性动态，由考生代理增强，该动态验证生成的摘要是否包含回答测验问题所需的信息，可以通过多方面的反馈机制进行迭代的完善。我们评估了三个广泛使用的长文档摘要基准测试的Summq。实验结果表明，我们的框架明显胜过盘旋和Bertscore指标的现有最新方法，以及在LLM-AS-A-A-Gudge和人类评估中。我们的全面分析揭示了多代理协作动力学，不同代理配置的影响以及测试机制的影响的有效性。这项工作为长期文档摘要建立了一种新的方法，该方法使用对抗性代理协作来提高摘要质量。

Title: MemLens: Uncovering Memorization in LLMs with Activation Trajectories

• Authors: Zirui He, Haiyan Zhao, Ali Payani, Mengnan du
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20909
• Pdf URL: https://arxiv.org/pdf/2509.20909
• Copy Paste: [[2509.20909]] MemLens: Uncovering Memorization in LLMs with Activation Trajectories(https://arxiv.org/abs/2509.20909)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are commonly evaluated on challenging benchmarks such as AIME and Math500, which are susceptible to contamination and risk of being memorized. Existing detection methods, which primarily rely on surface-level lexical overlap and perplexity, demonstrate low generalization and degrade significantly when encountering implicitly contaminated data. In this paper, we propose MemLens (An Activation Lens for Memorization Detection) to detect memorization by analyzing the probability trajectories of numeric tokens during generation. Our method reveals that contaminated samples exhibit ``shortcut'' behaviors, locking onto an answer with high confidence in the model's early layers, whereas clean samples show more gradual evidence accumulation across the model's full depth. We observe that contaminated and clean samples exhibit distinct and well-separated reasoning trajectories. To further validate this, we inject carefully designed samples into the model through LoRA fine-tuning and observe the same trajectory patterns as in naturally contaminated data. These results provide strong evidence that MemLens captures genuine signals of memorization rather than spurious correlations.
• 摘要：大型语言模型（LLMS）通常在诸如AIME和MATH500之类的具有挑战性的基准上进行评估，这些基准容易受到污染和记忆的风险。现有的检测方法主要依赖于表面水平的词汇重叠和困惑，在遇到隐式污染的数据时表现出较低的概括和降解。在本文中，我们提出了Memlens（用于记忆检测的激活透镜），以通过分析生成过程中数字令牌的概率轨迹来检测记忆。我们的方法表明，受污染的样品表现出``快捷方式''的行为，对模型的早期层有很高的信心锁定答案，而干净的样品在模型的完整深度中显示出更多的逐渐证据。我们观察到，受污染和干净的样品表现出明显且分离的推理轨迹。为了进一步验证这一点，我们通过Lora微调注入了精心设计的样品，并观察到与自然污染数据相同的轨迹模式。这些结果提供了有力的证据，表明Memlens捕获了记忆的真正信号，而不是虚假的相关性。

Title: Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning

• Authors: Asim Ersoy, Enes Altinisik, Husrev Taha Sencar, Kareem Darwish
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.20957
• Pdf URL: https://arxiv.org/pdf/2509.20957
• Copy Paste: [[2509.20957]] Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning(https://arxiv.org/abs/2509.20957)
• Keywords: language model, llm, agent
• Abstract: Tool calling is a critical capability that allows Large Language Models (LLMs) to interact with external systems, significantly expanding their utility. However, research and resources for tool calling are predominantly English-centric, leaving a gap in our understanding of how to enable this functionality for other languages, such as Arabic. This paper investigates three key research questions: (1) the necessity of in-language (Arabic) tool-calling data versus relying on cross-lingual transfer, (2) the effect of general-purpose instruction tuning on tool-calling performance, and (3) the value of fine-tuning on specific, high-priority tools. To address these questions, we conduct extensive experiments using base and post-trained variants of an open-weight Arabic LLM. To enable this study, we bridge the resource gap by translating and adapting two open-source tool-calling datasets into Arabic. Our findings provide crucial insights into the optimal strategies for developing robust tool-augmented agents for Arabic.
• 摘要：工具调用是一个关键的功能，允许大型语言模型（LLMS）与外部系统进行交互，从而大大扩展其实用性。但是，工具调用的研究和资源主要以英语为中心，在我们对如何为其他语言（例如阿拉伯语）启用此功能的理解留下了差距。本文研究了三个关键的研究问题：（1）需要语言中的（阿拉伯语）工具称呼数据与依赖跨语性转移的必要性，（2）通用说明对工具呼叫性能的影响，以及（3）微调对特定高优势工具的价值。为了解决这些问题，我们使用开放式阿拉伯语LLM的基础和训练后变体进行了广泛的实验。为了启用这项研究，我们通过将两个开源工具称数据集转化为阿拉伯语来弥合资源差距。我们的发现提供了对为阿拉伯语开发强大的工具增强剂的最佳策略的重要见解。

Title: Analysis of instruction-based LLMs' capabilities to score and judge text-input problems in an academic setting

• Authors: Valeria Ramirez-Garcia, David de-Fitero-Dominguez, Antonio Garcia-Cabot, Eva Garcia-Lopez
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.20982
• Pdf URL: https://arxiv.org/pdf/2509.20982
• Copy Paste: [[2509.20982]] Analysis of instruction-based LLMs' capabilities to score and judge text-input problems in an academic setting(https://arxiv.org/abs/2509.20982)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) can act as evaluators, a role studied by methods like LLM-as-a-Judge and fine-tuned judging LLMs. In the field of education, LLMs have been studied as assistant tools for students and teachers. Our research investigates LLM-driven automatic evaluation systems for academic Text-Input Problems using rubrics. We propose five evaluation systems that have been tested on a custom dataset of 110 answers about computer science from higher education students with three models: JudgeLM, Llama-3.1-8B and DeepSeek-R1-Distill-Llama-8B. The evaluation systems include: The JudgeLM evaluation, which uses the model's single answer prompt to obtain a score; Reference Aided Evaluation, which uses a correct answer as a guide aside from the original context of the question; No Reference Evaluation, which ommits the reference answer; Additive Evaluation, which uses atomic criteria; and Adaptive Evaluation, which is an evaluation done with generated criteria fitted to each question. All evaluation methods have been compared with the results of a human evaluator. Results show that the best method to automatically evaluate and score Text-Input Problems using LLMs is Reference Aided Evaluation. With the lowest median absolute deviation (0.945) and the lowest root mean square deviation (1.214) when compared to human evaluation, Reference Aided Evaluation offers fair scoring as well as insightful and complete evaluations. Other methods such as Additive and Adaptive Evaluation fail to provide good results in concise answers, No Reference Evaluation lacks information needed to correctly assess questions and JudgeLM Evaluations have not provided good results due to the model's limitations. As a result, we conclude that Artificial Intelligence-driven automatic evaluation systems, aided with proper methodologies, show potential to work as complementary tools to other academic resources.
• 摘要：大型语言模型（LLMS）可以充当评估者，这是通过LLM-AS-A-A-Gudge和微调评审LLM等方法研究的角色。在教育领域，LLM已被研究为学生和老师的助理工具。我们的研究调查了LLM驱动的自动评估系统，用于使用专栏的学术文本输入问题。我们提出了五个评估系统，这些评估系统已在具有三种模型的高等教育学生的110个关于计算机科学的答案的自定义数据集上进行了测试：Judgelm，Llama-3.1-8B和DeepSeek-R1-Distill-distill-lllama-8B。评估系统包括：使用模型的单个答案提示来获得分数；参考辅助评估，除了问题的原始上下文外，它使用正确的答案作为指南；没有参考答案的参考评估；使用原子标准的添加剂评估；和自适应评估，这是针对每个问题的生成标准进行的评估。所有评估方法均已与人类评估者的结果进行了比较。结果表明，使用LLMS自动评估和评分文本输入问题的最佳方法是参考辅助评估。与人类评估相比，具有最低的绝对偏差（0.945）和最低的均方根偏差（1.214），参考辅助评估提供了公平的评分以及有见地和完整的评估。其他方法（例如加性和自适应评估）无法在简洁的答案中提供良好的结果，没有参考评估缺乏正确评估问题所需的信息，并且由于模型的局限性，判断评估并未提供良好的结果。结果，我们得出的结论是，以适当的方法为帮助的人工智能驱动的自动评估系统显示出作为其他学术资源的补充工具的潜力。

Title: Generative AI for FFRDCs

• Authors: Arun S. Maiya
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21040
• Pdf URL: https://arxiv.org/pdf/2509.21040
• Copy Paste: [[2509.21040]] Generative AI for FFRDCs(https://arxiv.org/abs/2509.21040)
• Keywords: language model, llm
• Abstract: Federally funded research and development centers (FFRDCs) face text-heavy workloads, from policy documents to scientific and engineering papers, that are slow to analyze manually. We show how large language models can accelerate summarization, classification, extraction, and sense-making with only a few input-output examples. To enable use in sensitive government contexts, we apply OnPrem$.$LLM, an open-source framework for secure and flexible application of generative AI. Case studies on defense policy documents and scientific corpora, including the National Defense Authorization Act (NDAA) and National Science Foundation (NSF) Awards, demonstrate how this approach enhances oversight and strategic analysis while maintaining auditability and data sovereignty.
• 摘要：由联邦资助的研发中心（FFRDC）面临文本繁重的工作负载，从政策文件到科学和工程论文，这些工作速度很慢。我们仅使用几个输入输出示例来展示大型语言模型如何加速摘要，分类，提取和感知。为了在敏感的政府环境中使用，我们应用Onprem $。$ LLM，这是一个开源框架，用于安全，灵活地应用生成AI。关于国防政策文件和科学语料库的案例研究，包括《国防授权法》（NDAA）和国家科学基金会（NSF）奖，证明了这种方法如何增强监督和战略分析，同时保持可审核性和数据主权。

Title: Behind RoPE: How Does Causal Mask Encode Positional Information?

• Authors: Junu Kim, Xiao Liu, Zhenghao Lin, Lei Ji, Yeyun Gong, Edward Choi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21042
• Pdf URL: https://arxiv.org/pdf/2509.21042
• Copy Paste: [[2509.21042]] Behind RoPE: How Does Causal Mask Encode Positional Information?(https://arxiv.org/abs/2509.21042)
• Keywords: language model
• Abstract: While explicit positional encodings such as RoPE are a primary source of positional information in Transformer decoders, the causal mask also provides positional information. In this work, we prove that the causal mask can induce position-dependent patterns in attention scores, even without parameters or causal dependency in the input. Our theoretical analysis indicates that the induced attention pattern tends to favor nearby query-key pairs, mirroring the behavior of common positional encodings. Empirical analysis confirms that trained models exhibit the same behavior, with learned parameters further amplifying these patterns. Notably, we found that the interaction of causal mask and RoPE distorts RoPE's relative attention score patterns into non-relative ones. We consistently observed this effect in modern large language models, suggesting the importance of considering the causal mask as a source of positional information alongside explicit positional encodings.
• 摘要：尽管绳索等显式位置编码是变压器解码器中位置信息的主要来源，但因果面具还提供位置信息。在这项工作中，我们证明了因果面具可以在注意力评分中诱导位置依赖性模式，即使没有参数或输入中的因果关系。我们的理论分析表明，诱导的注意模式倾向于偏爱附近的查询键对，反映了共同位置编码的行为。经验分析证实，受过训练的模型表现出相同的行为，学习的参数进一步扩大了这些模式。值得注意的是，我们发现因果面具和绳索的相互作用将绳索的相对注意力评分模式扭曲为非相关性。我们在现代大型语言模型中一直观察到这种影响，这表明将因果面具作为位置信息的来源以及显式位置编码的重要性。

Title: When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Following

• Authors: Keno Harada, Yudai Yamazaki, Masachika Taniguchi, Edison Marrese-Taylor, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21051
• Pdf URL: https://arxiv.org/pdf/2509.21051
• Copy Paste: [[2509.21051]] When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Following(https://arxiv.org/abs/2509.21051)
• Keywords: language model, llm
• Abstract: As large language models (LLMs) are increasingly applied to real-world scenarios, it becomes crucial to understand their ability to follow multiple instructions simultaneously. To systematically evaluate these capabilities, we introduce two specialized benchmarks for fundamental domains where multiple instructions following is important: Many Instruction-Following Eval (ManyIFEval) for text generation with up to ten instructions, and Style-aware Mostly Basic Programming Problems (StyleMBPP) for code generation with up to six instructions. Our experiments with the created benchmarks across ten LLMs reveal that performance consistently degrades as the number of instructions increases. Furthermore, given the fact that evaluating all the possible combinations of multiple instructions is computationally impractical in actual use cases, we developed three types of regression models that can estimate performance on both unseen instruction combinations and different numbers of instructions which are not used during training. We demonstrate that a logistic regression model using instruction count as an explanatory variable can predict performance of following multiple instructions with approximately 10% error, even for unseen instruction combinations. We show that relatively modest sample sizes (500 for ManyIFEval and 300 for StyleMBPP) are sufficient for performance estimation, enabling efficient evaluation of LLMs under various instruction combinations.
• 摘要：随着大型语言模型（LLM）越来越多地应用于现实世界情景，因此了解他们同时遵循多个指令的能力至关重要。为了系统地评估这些功能，我们为基本领域介绍了两个专门的基准测试，其中以下多个说明很重要：许多指令遵循的评估（ManyifeVal），用于具有多达十个说明的文本生成，并且最多有六个说明的代码为基本的编程问题（Stylembpp）。我们对十个LLM的创建基准测试的实验表明，随着指令数量的增加，性能始终如一地降低。此外，鉴于评估多个指令的所有可能组合的事实在实际用例中是计算上不切实际的，我们开发了三种类型的回归模型，可以估计在看不见的说明组合和训练过程中未使用的不同数量的指令中的性能。我们证明，使用指令计数作为解释变量的逻辑回归模型也可以预测以下多个指令的性能，即使对于看不见的指令组合，也可以预测误差约10％的误差。我们表明，相对适量的样本量（ManyifeVal的500，而对于Stylembpp的300）足以进行性能估算，从而在各种指令组合下对LLM有效评估。

Title: SoM-1K: A Thousand-Problem Benchmark Dataset for Strength of Materials

• Authors: Qixin Wan, Zilong Wang, Jingwen Zhou, Wanting Wang, Ziheng Geng, Jiachen Liu, Ran Cao, Minghui Cheng, Lu Cheng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21079
• Pdf URL: https://arxiv.org/pdf/2509.21079
• Copy Paste: [[2509.21079]] SoM-1K: A Thousand-Problem Benchmark Dataset for Strength of Materials(https://arxiv.org/abs/2509.21079)
• Keywords: language model, llm, prompt
• Abstract: Foundation models have shown remarkable capabilities in various domains, but their performance on complex, multimodal engineering problems remains largely unexplored. We introduce SoM-1K, the first large-scale multimodal benchmark dataset dedicated to evaluating foundation models on problems in the strength of materials (SoM). The dataset, which contains 1,065 annotated SoM problems, mirrors real-world engineering tasks by including both textual problem statements and schematic diagrams. Due to the limited capabilities of current foundation models in understanding complicated visual information, we propose a novel prompting strategy called Descriptions of Images (DoI), which provides rigorous expert-generated text descriptions of the visual diagrams as the context. We evaluate eight representative foundation models, including both large language models (LLMs) and vision language models (VLMs). Our results show that current foundation models struggle significantly with these engineering problems, with the best-performing model achieving only 56.6% accuracy. Interestingly, we found that LLMs, when provided with DoI, often outperform VLMs provided with visual diagrams. A detailed error analysis reveals that DoI plays a crucial role in mitigating visual misinterpretation errors, suggesting that accurate text-based descriptions can be more effective than direct image input for current foundation models. This work establishes a rigorous benchmark for engineering AI and highlights a critical need for developing more robust multimodal reasoning capabilities in foundation models, particularly in scientific and engineering contexts.
• 摘要：基础模型在各个领域都显示出了显着的功能，但是它们在复杂的多模式工程问题上的性能仍然在很大程度上没有探索。我们介绍了SOM-1K，这是第一个大规模的多模式基准数据集，该数据集致力于评估材料强度（SOM）问题的基础模型。包含1,065个注释的SOM问题的数据集通过包含文本问题语句和示意图图表来反映现实世界工程任务。由于当前基础模型在理解复杂的视觉信息方面的功能有限，我们提出了一种新颖的提示策略，称为图像描述（DOI），该策略提供了严格的专家生成的文本描述，以视觉图作为上下文。我们评估了八种代表性基础模型，包括大语言模型（LLM）和视觉语言模型（VLMS）。我们的结果表明，当前的基础模型在这些工程问题上大为挣扎，表现最佳的模型仅达到56.6％的精度。有趣的是，我们发现LLMS在提供DOI时，通常比提供视觉图的VLM。详细的错误分析表明，DOI在减轻视觉误解错误中起着至关重要的作用，这表明准确的基于文本的描述比当前基础模型的直接图像输入更有效。这项工作为工程AI建立了严格的基准，并突出了在基础模型中，尤其是在科学和工程环境中开发更强大的多模式推理能力的关键需求。

Title: Which Cultural Lens Do Models Adopt? On Cultural Positioning Bias and Agentic Mitigation in LLMs

• Authors: Yixin Wan, Xingrun Chen, Kai-Wei Chang
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2509.21080
• Pdf URL: https://arxiv.org/pdf/2509.21080
• Copy Paste: [[2509.21080]] Which Cultural Lens Do Models Adopt? On Cultural Positioning Bias and Agentic Mitigation in LLMs(https://arxiv.org/abs/2509.21080)
• Keywords: language model, llm, prompt, agent
• Abstract: Large language models (LLMs) have unlocked a wide range of downstream generative applications. However, we found that they also risk perpetuating subtle fairness issues tied to culture, positioning their generations from the perspectives of the mainstream US culture while demonstrating salient externality towards non-mainstream ones. In this work, we identify and systematically investigate this novel culture positioning bias, in which an LLM's default generative stance aligns with a mainstream view and treats other cultures as outsiders. We propose the CultureLens benchmark with 4000 generation prompts and 3 evaluation metrics for quantifying this bias through the lens of a culturally situated interview script generation task, in which an LLM is positioned as an onsite reporter interviewing local people across 10 diverse cultures. Empirical evaluation on 5 state-of-the-art LLMs reveals a stark pattern: while models adopt insider tones in over 88 percent of US-contexted scripts on average, they disproportionately adopt mainly outsider stances for less dominant cultures. To resolve these biases, we propose 2 inference-time mitigation methods: a baseline prompt-based Fairness Intervention Pillars (FIP) method, and a structured Mitigation via Fairness Agents (MFA) framework consisting of 2 pipelines: (1) MFA-SA (Single-Agent) introduces a self-reflection and rewriting loop based on fairness guidelines. (2) MFA-MA (Multi-Agent) structures the process into a hierarchy of specialized agents: a Planner Agent(initial script generation), a Critique Agent (evaluates initial script against fairness pillars), and a Refinement Agent (incorporates feedback to produce a polished, unbiased script). Empirical results showcase the effectiveness of agent-based methods as a promising direction for mitigating biases in generative LLMs.
• 摘要：大型语言模型（LLM）已解锁了广泛的下游生成应用。但是，我们发现它们还冒着与文化相关的微妙公平问题的延续，从美国主流文化的角度定位了世代，同时表现出对非主流人的显着外部性。在这项工作中，我们确定并系统地研究了这种新颖的文化定位偏见，其中LLM的默认生成姿态与主流视图一致，并将其他文化视为局外人。我们提出了具有4000代提示和3个评估指标的文化基准，以通过文化定位的访谈脚本生成任务来量化这种偏见，其中LLM被定位为现场记者，采访了10种不同文化的当地人。对5个最先进的LLM的经验评估揭示了一种鲜明的模式：虽然模型平均采用了超过88％的美国文化脚本的内幕色调，但它们主要采用主要是局外人立场来实现较低的统治文化。为了解决这些偏见，我们提出了两种推理时间缓解方法：一种基线及时的公平干预支柱（FIP）方法，以及通过公平剂（MFA）框架（MFA）框架的结构化缓解措施，该框架由2种管道组成：（1）MFA-SA（单一代理）引入了基于自我反射和重新处理公平的指南。 （2）MFA-MA（多代理）将过程结构为专业代理的层次结构：计划者代理（初始脚本生成），批评代理（评估针对公平支柱的初始脚本）和改进剂（将反馈结合起来以产生抛光，无偏见的脚本）。经验结果将基于代理的方法作为缓解生成LLM中偏见的有希望的方向的有效性。

Title: PerHalluEval: Persian Hallucination Evaluation Benchmark for Large Language Models

• Authors: Mohammad Hosseini, Kimia Hosseini, Shayan Bali, Zahra Zanjani, Saeedeh Momtazi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21104
• Pdf URL: https://arxiv.org/pdf/2509.21104
• Copy Paste: [[2509.21104]] PerHalluEval: Persian Hallucination Evaluation Benchmark for Large Language Models(https://arxiv.org/abs/2509.21104)
• Keywords: language model, llm, hallucination
• Abstract: Hallucination is a persistent issue affecting all large language Models (LLMs), particularly within low-resource languages such as Persian. PerHalluEval (Persian Hallucination Evaluation) is the first dynamic hallucination evaluation benchmark tailored for the Persian language. Our benchmark leverages a three-stage LLM-driven pipeline, augmented with human validation, to generate plausible answers and summaries regarding QA and summarization tasks, focusing on detecting extrinsic and intrinsic hallucinations. Moreover, we used the log probabilities of generated tokens to select the most believable hallucinated instances. In addition, we engaged human annotators to highlight Persian-specific contexts in the QA dataset in order to evaluate LLMs' performance on content specifically related to Persian culture. Our evaluation of 12 LLMs, including open- and closed-source models using PerHalluEval, revealed that the models generally struggle in detecting hallucinated Persian text. We showed that providing external knowledge, i.e., the original document for the summarization task, could mitigate hallucination partially. Furthermore, there was no significant difference in terms of hallucination when comparing LLMs specifically trained for Persian with others.
• 摘要：幻觉是影响所有大型语言模型（LLM）的持续问题，尤其是在波斯语等低资源语言中。 Perhallueval（波斯幻觉评估）是为波斯语量身定制的第一个动态幻觉评估基准。我们的基准测试利用了三阶段LLM驱动的管道，并通过人为验证增强，以产生有关质量检查和摘要任务的合理答案和摘要，重点是检测外部和内在的幻觉。此外，我们使用了生成代币的日志概率来选择最令人信服的幻觉实例。此外，我们介入了人类注释者，以突出质量检查数据集中的波斯特定环境，以评估LLMS在与波斯文化特别相关的内容上的性能。我们对12个LLM的评估，包括使用Perhallueval的开放式和封闭式模型，表明这些模型通常在检测幻觉的波斯文本方面难度。我们表明，提供外部知识，即摘要任务的原始文档可以部分减轻幻觉。此外，在比较专门为波斯人训练的LLM与他人的LLM时，幻觉没有显着差异。

Title: BESPOKE: Benchmark for Search-Augmented Large Language Model Personalization via Diagnostic Feedback

• Authors: Hyunseo Kim, Sangam Lee, Kwangwook Seo, Dongha Lee
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.21106
• Pdf URL: https://arxiv.org/pdf/2509.21106
• Copy Paste: [[2509.21106]] BESPOKE: Benchmark for Search-Augmented Large Language Model Personalization via Diagnostic Feedback(https://arxiv.org/abs/2509.21106)
• Keywords: language model, gpt, llm, chat
• Abstract: Search-augmented large language models (LLMs) have advanced information-seeking tasks by integrating retrieval into generation, reducing users' cognitive burden compared to traditional search systems. Yet they remain insufficient for fully addressing diverse user needs, which requires recognizing how the same query can reflect different intents across users and delivering information in preferred forms. While recent systems such as ChatGPT and Gemini attempt personalization by leveraging user histories, systematic evaluation of such personalization is under-explored. To address this gap, we propose BESPOKE, the realistic benchmark for evaluating personalization in search-augmented LLMs. BESPOKE is designed to be both realistic, by collecting authentic chat and search histories directly from humans, and diagnostic, by pairing responses with fine-grained preference scores and feedback. The benchmark is constructed through long-term, deeply engaged human annotation, where human annotators contributed their own histories, authored queries with detailed information needs, and evaluated responses with scores and diagnostic feedback. Leveraging BESPOKE, we conduct systematic analyses that reveal key requirements for effective personalization in information-seeking tasks, providing a foundation for fine-grained evaluation of personalized search-augmented LLMs. Our code and data are available at this https URL.
• 摘要：通过将检索整合到一代中，与传统搜索系统相比，搜索型大型语言模型（LLMS）通过将检索纳入生成，可以减少用户的认知负担，从而具有高级信息寻求任务。然而，它们仍然不足以完全满足不同的用户需求，这需要认识到相同的查询如何反映用户之间的不同意图并以优选形式传递信息。尽管最近通过利用用户历史记录来尝试个性化的最新系统，但对这种个性化的系统评估却没有探索。为了解决这一差距，我们提出了定制的定制，这是评估搜索授权LLM中个性化的现实基准。定制的旨在通过直接从人类那里收集真实的聊天和搜索历史，并通过将响应与细粒度的偏好分数和反馈配对来收集真实的聊天和诊断。基准是通过长期，深入参与的人类注释来构建的，人类注释者贡献了自己的历史，撰写了有关信息需求的查询，并通过分数和诊断反馈进行了评估的响应。利用定制的定制，我们进行了系统的分析，揭示了在寻求信息的任务中有效个性化的关键要求，从而为个性化搜索型LLMS进行了精细评估的基础。我们的代码和数据可在此HTTPS URL上找到。

Title: VoiceBBQ: Investigating Effect of Content and Acoustics in Social Bias of Spoken Language Model

• Authors: Junhyuk Choi, Ro-hoon Oh, Jihwan Seol, Bugeun Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21108
• Pdf URL: https://arxiv.org/pdf/2509.21108
• Copy Paste: [[2509.21108]] VoiceBBQ: Investigating Effect of Content and Acoustics in Social Bias of Spoken Language Model(https://arxiv.org/abs/2509.21108)
• Keywords: language model
• Abstract: We introduce VoiceBBQ, a spoken extension of the BBQ (Bias Benchmark for Question Answering) - a dataset that measures social bias by presenting ambiguous or disambiguated contexts followed by questions that may elicit stereotypical responses. Due to the nature of speech, social bias in Spoken Language Models (SLMs) can emerge from two distinct sources: 1) content aspect and 2) acoustic aspect. The dataset converts every BBQ context into controlled voice conditions, enabling per-axis accuracy, bias, and consistency scores that remain comparable to the original text benchmark. Using VoiceBBQ, we evaluate two SLMs - LLaMA-Omni and Qwen2-Audio - and observe architectural contrasts: LLaMA-Omni resists acoustic bias while amplifying gender and accent bias, whereas Qwen2-Audio substantially dampens these cues while preserving content fidelity. VoiceBBQ thus provides a compact, drop-in testbed for jointly diagnosing content and acoustic bias across spoken language models.
• 摘要：我们介绍了VoiceBBQ，这是烧烤的口语扩展（提出问题回答的偏置基准） - 一个数据集，该数据集通过提出模棱两可或歧义的上下文来衡量社会偏见，然后是可能引起刻板印象回答的问题。由于语音的性质，可以从两个不同的来源出现口语模型（SLM）中的社会偏见：1）内容方面和2）声学方面。该数据集将每个烧烤上下文转换为受控的语音条件，从而使每轴精度，偏差和一致性得分保持与原始文本基准相当。使用VoiceBBQ，我们评估了两个SLM-Llama-omni和Qwen2-Audio-并观察到建筑对比：Llama-Omni抵抗声学偏见，同时放大性别和口音偏见，而QWEN2-AUDIO实质上会抑制这些线索，同时使这些线索降低了这些提示，同时证明了内容的富裕性。因此，VoiceBBQ提供了一个紧凑的，插入测试床，以共同诊断口语模型的内容和声学偏见。

Title: Acoustic-based Gender Differentiation in Speech-aware Language Models

• Authors: Junhyuk Choi, Jihwan Seol, Nayeon Kim, Chanhee Cho, EunBin Cho, Bugeun Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21125
• Pdf URL: https://arxiv.org/pdf/2509.21125
• Copy Paste: [[2509.21125]] Acoustic-based Gender Differentiation in Speech-aware Language Models(https://arxiv.org/abs/2509.21125)
• Keywords: language model, llm
• Abstract: Speech-aware Language Models (SpeechLMs) have fundamentally transformed human-AI interaction by enabling voice-based communication, yet they may exhibit acoustic-based gender differentiation where identical questions lead to different responses based on the speaker's gender. This paper propose a new dataset that enables systematic analysis of this phenomenon, containing 9,208 speech samples across three categories: Gender-Independent, Gender-Stereotypical, and Gender-Dependent. We further evaluated LLaMA-Omni series and discovered a paradoxical pattern; while overall responses seems identical regardless of gender, the pattern is far from unbiased responses. Specifically, in Gender-Stereotypical questions, all models consistently exhibited male-oriented responses; meanwhile, in Gender-Dependent questions where gender differentiation would be contextually appropriate, models exhibited responses independent to gender instead. We also confirm that this pattern does not result from neutral options nor perceived gender of a voice. When we allow neutral response, models tends to respond neutrally also in Gender-Dependent questions. The paradoxical pattern yet retains when we applied gender neutralization methods on speech. Through comparison between SpeechLMs with corresponding backbone LLMs, we confirmed that these paradoxical patterns primarily stem from Whisper speech encoders, which generates male-oriented acoustic tokens. These findings reveal that current SpeechLMs may not successfully remove gender biases though they prioritized general fairness principles over contextual appropriateness, highlighting the need for more sophisticated techniques to utilize gender information properly in speech technology.
• 摘要：语音感知语言模型（语音LMS）通过实现语音沟通从根本上改变了人类的互动，但是它们可能表现出基于声学的性别差异化，在这种性别差异上，相同的问题会根据说话者的性别产生不同的回答。本文提出了一个新的数据集，可以对该现象进行系统分析，其中包含跨三个类别的9,208个语音样本：独立性别，性别 - 性型和性别依赖性。我们进一步评估了Llama-omni系列，并发现了一种矛盾的模式。尽管无论性别如何，总体反应似乎相同，但这种模式远非公正的反应。具体而言，在性别概念性问题中，所有模型都始终表现出男性为导向的反应。同时，在与性别差异化的性别依赖性问题中，在上下文中是适当的，模型表现出与性别无关的回答。我们还确认，这种模式不是由中性选项引起的，也不是语音的性别。当我们允许中性反应时，模型也倾向于在性别依赖性问题中响应中立。当我们将性别中和方法应用于语音时，矛盾的模式仍保留。通过与相应的主链LLM之间的语音LM之间的比较，我们证实了这些矛盾的模式主要源自耳语的语音编码器，这会产生以男性为导向的声音令牌。这些发现表明，当前的语音LS可能无法成功消除性别偏见，尽管它们优先考虑一般公平原则而不是上下文适当性，从而强调了需要更复杂的技术在语音技术中正确利用性别信息。

Title: Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction

• Authors: Lei Hei, Tingjing Liao, Yingxin Pei, Yiyang Qi, Jiaqi Wang, Ruiting Li, Feiliang Ren
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.21151
• Pdf URL: https://arxiv.org/pdf/2509.21151
• Copy Paste: [[2509.21151]] Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction(https://arxiv.org/abs/2509.21151)
• Keywords: language model
• Abstract: Relation extraction (RE) aims to identify semantic relations between entities in unstructured text. Although recent work extends traditional RE to multimodal scenarios, most approaches still adopt classification-based paradigms with fused multimodal features, representing relations as discrete labels. This paradigm has two significant limitations: (1) it overlooks structural constraints like entity types and positional cues, and (2) it lacks semantic expressiveness for fine-grained relation understanding. We propose \underline{R}etrieval \underline{O}ver \underline{C}lassification (ROC), a novel framework that reformulates multimodal RE as a retrieval task driven by relation semantics. ROC integrates entity type and positional information through a multimodal encoder, expands relation labels into natural language descriptions using a large language model, and aligns entity-relation pairs via semantic similarity-based contrastive learning. Experiments show that our method achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability.
• 摘要：关系提取（RE）旨在确定非结构化文本中实体之间的语义关系。尽管最近的工作将传统RE扩展到多模式方案，但大多数方法仍然采用具有融合多模式特征的基于分类的范式，代表关系为离散标签。该范式有两个重要的局限性：（1）它忽略了实体类型和位置提示等结构性约束，并且（2）它缺乏语义表达性，无法获得细粒度的关系理解。我们提出\下划线{r} etrieval \下划线{o} ver \ undusline {c}套索（roc），这是一个新颖的框架，将多模式RE重新定义为由关系语义驱动的检索任务。 ROC通过多模式编码器整合实体类型和位置信息，使用大语言模型将关系标签扩展到自然语言描述中，并通过基于语义相似性的对比度学习来使实体关系对结盟。实验表明，我们的方法在基准数据集MNRE上实现了最先进的性能，并且具有更强的鲁棒性和解释性。

Title: Learning the Wrong Lessons: Syntactic-Domain Spurious Correlations in Language Models

• Authors: Chantal Shaib, Vinith M. Suriyakumar, Levent Sagun, Byron C. Wallace, Marzyeh Ghassemi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21155
• Pdf URL: https://arxiv.org/pdf/2509.21155
• Copy Paste: [[2509.21155]] Learning the Wrong Lessons: Syntactic-Domain Spurious Correlations in Language Models(https://arxiv.org/abs/2509.21155)
• Keywords: language model, gpt, llm, prompt
• Abstract: For an LLM to correctly respond to an instruction it must understand both the semantics and the domain (i.e., subject area) of a given task-instruction pair. However, syntax can also convey implicit information Recent work shows that syntactic templates--frequent sequences of Part-of-Speech (PoS) tags--are prevalent in training data and often appear in model outputs. In this work we characterize syntactic templates, domain, and semantics in task-instruction pairs. We identify cases of spurious correlations between syntax and domain, where models learn to associate a domain with syntax during training; this can sometimes override prompt semantics. Using a synthetic training dataset, we find that the syntactic-domain correlation can lower performance (mean 0.51 +/- 0.06) on entity knowledge tasks in OLMo-2 models (1B-13B). We introduce an evaluation framework to detect this phenomenon in trained models, and show that it occurs on a subset of the FlanV2 dataset in open (OLMo-2-7B; Llama-4-Maverick), and closed (GPT-4o) models. Finally, we present a case study on the implications for safety finetuning, showing that unintended syntactic-domain correlations can be used to bypass refusals in OLMo-2-7B Instruct and GPT-4o. Our findings highlight two needs: (1) to explicitly test for syntactic-domain correlations, and (2) to ensure syntactic diversity in training data, specifically within domains, to prevent such spurious correlations.
• 摘要：为了使LLM正确响应指令，必须同时了解给定的任务实施对的语义和域（即主题区域）。但是，语法还可以传达隐式信息最近的工作表明，句法模板 - 频繁的词性词曲（POS）标签的频率 - 在训练数据中很普遍，并且经常出现在模型输出中。在这项工作中，我们表征了任务指导对中的句法模板，域和语义。我们确定了语法与域之间虚假相关性的案例，其中模型学会在训练过程中将域与语法联系起来；这有时会覆盖迅速的语义。使用合成训练数据集，我们发现句法域相关性可以降低OLMO-2模型（1B-13B）实体知识任务的性能（平均0.51 +/- 0.06）。我们介绍了一个评估框架，以检测训练有素的模型中的这种现象，并表明它发生在开放式（OLMO-2-7B; Llama-4-Maverick）中的Flanv2数据集的子集中，并进行了封闭（GPT-4O）模型。最后，我们提出了一项案例研究，涉及对安全卫生的影响，表明意想不到的句法域相关性可用于绕过OLMO-2-7B指导和GPT-4O中的拒绝。我们的发现突出了两个需求：（1）明确测试句法域相关性，（2）确保训练数据中的句法多样性，特别是在域内，以防止这种虚假相关性。

Title: Who's Laughing Now? An Overview of Computational Humour Generation and Explanation

• Authors: Tyler Loakman, William Thorne, Chenghua Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21175
• Pdf URL: https://arxiv.org/pdf/2509.21175
• Copy Paste: [[2509.21175]] Who's Laughing Now? An Overview of Computational Humour Generation and Explanation(https://arxiv.org/abs/2509.21175)
• Keywords: language model, llm
• Abstract: The creation and perception of humour is a fundamental human trait, positioning its computational understanding as one of the most challenging tasks in natural language processing (NLP). As an abstract, creative, and frequently context-dependent construct, humour requires extensive reasoning to understand and create, making it a pertinent task for assessing the common-sense knowledge and reasoning abilities of modern large language models (LLMs). In this work, we survey the landscape of computational humour as it pertains to the generative tasks of creation and explanation. We observe that, despite the task of understanding humour bearing all the hallmarks of a foundational NLP task, work on generating and explaining humour beyond puns remains sparse, while state-of-the-art models continue to fall short of human capabilities. We bookend our literature survey by motivating the importance of computational humour processing as a subdiscipline of NLP and presenting an extensive discussion of future directions for research in the area that takes into account the subjective and ethically ambiguous nature of humour.
• 摘要：幽默的创造和感知是人类的基本特征，将其计算理解定位为自然语言处理（NLP）中最具挑战性的任务之一。作为一种抽象，创造性且经常与上下文相关的结构，幽默需要广泛的理解和创造的理解，这使其成为评估现代大型语言模型（LLMS）的常识性知识和推理能力的重要任务。在这项工作中，我们调查了计算幽默的景观，因为它与创建和解释的生成任务有关。我们观察到，尽管理解幽默的任务具有基础NLP任务的所有标志，但仍致力于产生和解释《超越双关语》的幽默仍然很少，而最先进的模型仍未达到人类能力。我们通过激励计算幽默处理作为NLP的子学科的重要性来预订文献调查，并对该领域的未来研究方向进行了广泛的讨论，这些方向考虑了幽默的主观和道德上歧义的本质。

Title: GEP: A GCG-Based method for extracting personally identifiable information from chatbots built on small language models

• Authors: Jieli Zhu, Vi Ngoc-Nha Tran
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21192
• Pdf URL: https://arxiv.org/pdf/2509.21192
• Copy Paste: [[2509.21192]] GEP: A GCG-Based method for extracting personally identifiable information from chatbots built on small language models(https://arxiv.org/abs/2509.21192)
• Keywords: language model, gpt, llm, chat
• Abstract: Small language models (SLMs) become unprecedentedly appealing due to their approximately equivalent performance compared to large language models (LLMs) in certain fields with less energy and time consumption during training and inference. However, the personally identifiable information (PII) leakage of SLMs for downstream tasks has yet to be explored. In this study, we investigate the PII leakage of the chatbot based on SLM. We first finetune a new chatbot, i.e., ChatBioGPT based on the backbone of BioGPT using medical datasets Alpaca and HealthCareMagic. It shows a matchable performance in BERTscore compared with previous studies of ChatDoctor and ChatGPT. Based on this model, we prove that the previous template-based PII attacking methods cannot effectively extract the PII in the dataset for leakage detection under the SLM condition. We then propose GEP, which is a greedy coordinate gradient-based (GCG) method specifically designed for PII extraction. We conduct experimental studies of GEP and the results show an increment of up to 60$\times$ more leakage compared with the previous template-based methods. We further expand the capability of GEP in the case of a more complicated and realistic situation by conducting free-style insertion where the inserted PII in the dataset is in the form of various syntactic expressions instead of fixed templates, and GEP is still able to reveal a PII leakage rate of up to 4.53%.
• 摘要：小语言模型（SLM）由于与大型语言模型（LLM）在某些领域的表现近似，因此在训练和推理期间能量和时间消耗较少的情况下，它们的表现大致相同。但是，尚未探索SLM的个人身份信息（PII）泄漏，尚未探索。在这项研究中，我们研究了基于SLM的聊天机器人的PII泄漏。我们首先使用新的聊天机器人，即使用Michical Dataset ashpaca和Healthcaremagic的Biogpt的骨干基于Biogpt的骨干。与以前对Chatdoctor和Chatgpt的研究相比，它显示出Bertscore的表现可匹配。基于此模型，我们证明了先前的基于模板的PII攻击方法无法有效地在数据集中提取PII，以便在SLM条件下泄漏检测。然后，我们提出GEP，这是一种专门为PII提取设计的贪婪坐标梯度（GCG）方法。我们对GEP进行了实验研究，结果表明，与以前的基于模板的方法相比，高达60美元$ \ times $泄漏的增加。在更复杂和现实的情况下，我们通过进行自由式插入来进一步扩展了GEP的能力，在数据集中插入的PII的形式是各种句法表达式而不是固定模板的形式，并且GEP仍然能够显示出最高4.53％的PII泄漏率。

Title: Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning

• Authors: Xiangru Tang, Wanghan Xu, Yujie Wang, Zijie Guo, Daniel Shao, Jiapeng Chen, Cixuan Zhang, Ziyi Wang, Lixin Zhang, Guancheng Wan, Wenlong Zhang, Lei Bai, Zhenfei Yin, Philip Torr, Hanrui Wang, Di Jin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21193
• Pdf URL: https://arxiv.org/pdf/2509.21193
• Copy Paste: [[2509.21193]] Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning(https://arxiv.org/abs/2509.21193)
• Keywords: language model, llm, agent
• Abstract: Large language models (LLMs) have recently shown strong progress on scientific reasoning, yet two major bottlenecks remain. First, explicit retrieval fragments reasoning, imposing a hidden "tool tax" of extra tokens and steps. Second, multi-agent pipelines often dilute strong solutions by averaging across all candidates. We address these challenges with a unified framework that combines implicit retrieval and structured collaboration. At its foundation, a Monitor-based retrieval module operates at the token level, integrating external knowledge with minimal disruption to reasoning. On top of this substrate, Hierarchical Solution Refinement (HSR) iteratively designates each candidate as an anchor to be repaired by its peers, while Quality-Aware Iterative Reasoning (QAIR) adapts refinement to solution quality. On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3\% accuracy -- the highest reported to date, surpassing the strongest agent baseline by 13.4 points and leading frontier LLMs by up to 18.1 points, while simultaneously reducing token usage by 53.5\% and agent steps by 43.7\%. Results on SuperGPQA and TRQA confirm robustness across domains. Error analysis shows that reasoning failures and knowledge gaps co-occur in over 85\% of cases, while diversity analysis reveals a clear dichotomy: retrieval tasks benefit from solution variety, whereas reasoning tasks favor consensus. Together, these findings demonstrate how implicit augmentation and structured refinement overcome the inefficiencies of explicit tool use and uniform aggregation. Code is available at: this https URL.
• 摘要：大型语言模型（LLMS）最近在科学推理上表现出了很大的进步，但仍有两个主要的瓶颈。首先，明确检索片段推理，施加了额外的令牌和步骤的隐藏“工具税”。其次，多代理管道通常通过在所有候选人中取平均值来稀释强解决方案。我们通过统一的框架来解决这些挑战，该框架结合了隐性检索和结构化的合作。在其基础上，基于监视器的检索模块在令牌级别运行，将外部知识集成到最小的推理中断。除此基板外，分层溶液的细化（HSR）迭代地将每个候选者指定为要通过同龄人修复的锚点，而质量意识的迭代推理（QAIR）则适应了溶液质量的细化。在人类的最后考试（HLE）Bio/Chem Gold上，我们的框架获得了48.3 \％的准确性 - 迄今为止报告的最高框架，超过了最强的代理基线13.4分，并领先Frontier LLMS最高18.1分，而同时将代币的用法同时减少了53.5 \％\％\％\％\％\％\％。 SuperGPQA和TRQA的结果证实了跨域的鲁棒性。错误分析表明，在超过85％的情况下，推理失败和知识差距共发生，而多样性分析揭示了明确的二分法：检索任务受益于解决方案的多样性，而推理任务有利于共识。这些发现共同证明了隐式增强和结构化的改进如何克服显式工具使用和均匀聚集的效率低下。代码可用：此HTTPS URL。

Title: CLaw: Benchmarking Chinese Legal Knowledge in Large Language Models - A Fine-grained Corpus and Reasoning Analysis

• Authors: Xinzhe Xu, Liang Zhao, Hongshen Xu, Chen Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21208
• Pdf URL: https://arxiv.org/pdf/2509.21208
• Copy Paste: [[2509.21208]] CLaw: Benchmarking Chinese Legal Knowledge in Large Language Models - A Fine-grained Corpus and Reasoning Analysis(https://arxiv.org/abs/2509.21208)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Large Language Models (LLMs) are increasingly tasked with analyzing legal texts and citing relevant statutes, yet their reliability is often compromised by general pre-training that ingests legal texts without specialized focus, obscuring the true depth of their legal knowledge. This paper introduces CLaw, a novel benchmark specifically engineered to meticulously evaluate LLMs on Chinese legal knowledge and its application in reasoning. CLaw comprises two key components: (1) a comprehensive, fine-grained corpus of all 306 Chinese national statutes, segmented to the subparagraph level and incorporating precise historical revision timesteps for rigorous recall evaluation (64,849 entries), and (2) a challenging set of 254 case-based reasoning instances derived from China Supreme Court curated materials to assess the practical application of legal knowledge. Our empirical evaluation reveals that most contemporary LLMs significantly struggle to faithfully reproduce legal provisions. As accurate retrieval and citation of legal provisions form the basis of legal reasoning, this deficiency critically undermines the reliability of their responses. We contend that achieving trustworthy legal reasoning in LLMs requires a robust synergy of accurate knowledge retrieval--potentially enhanced through supervised fine-tuning (SFT) or retrieval-augmented generation (RAG)--and strong general reasoning capabilities. This work provides an essential benchmark and critical insights for advancing domain-specific LLM reasoning, particularly within the complex legal sphere.
• 摘要：大型语言模型（LLMS）越来越多地负责分析法律文本并引用相关法规，但其可靠性经常受到一般预培训的损害，即摄入法律文本而没有专门的重点，从而掩盖了其法律知识的真实深度。本文介绍了Claw，这是一种专门设计的新型基准，该基准是针对中国法律知识及其在推理中的应用而精心评估的。 Claw包括两个关键组成部分：（1）所有306个中国国家法规中的全面，细粒度的语料库，分为部分级别，并结合了严格的召回评估（64,849个条目）（64,849个条目）的精确历史修订时间段，（2）基于案例的基于案例的推理实例的挑战性，从中国最高法院进行了挑战，以评估法定的材料，以评估法定的概述。我们的经验评估表明，大多数当代LLM都在忠实地繁殖法律规定。由于法律规定的准确检索和引用构成了法律推理的基础，因此这种缺陷严重破坏了其回应的可靠性。我们认为，在LLMS中实现值得信赖的法律推理需要具有良好的准确知识检索的协同作用 - 通过监督的微调（SFT）或检索成绩（RAG）和强大的一般推理能力来逐步增强。这项工作为推进特定领域的LLM推理，尤其是在复杂的法律领域中提供了必不可少的基准和关键见解。

Title: SGMem: Sentence Graph Memory for Long-Term Conversational Agents

• Authors: Yaxiong Wu, Yongyue Zhang, Sheng Liang, Yong Liu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.21212
• Pdf URL: https://arxiv.org/pdf/2509.21212
• Copy Paste: [[2509.21212]] SGMem: Sentence Graph Memory for Long-Term Conversational Agents(https://arxiv.org/abs/2509.21212)
• Keywords: language model, llm, agent
• Abstract: Long-term conversational agents require effective memory management to handle dialogue histories that exceed the context window of large language models (LLMs). Existing methods based on fact extraction or summarization reduce redundancy but struggle to organize and retrieve relevant information across different granularities of dialogue and generated memory. We introduce SGMem (Sentence Graph Memory), which represents dialogue as sentence-level graphs within chunked units, capturing associations across turn-, round-, and session-level contexts. By combining retrieved raw dialogue with generated memory such as summaries, facts and insights, SGMem supplies LLMs with coherent and relevant context for response generation. Experiments on LongMemEval and LoCoMo show that SGMem consistently improves accuracy and outperforms strong baselines in long-term conversational question answering.
• 摘要：长期对话代理需要有效的内存管理来处理超过大语模型（LLM）上下文窗口的对话历史。基于事实提取或摘要的现有方法降低了冗余，但努力组织和检索对话和产生记忆的不同粒度。我们介绍了sgmem（句子图内存），该sgmem表示对话为句子级的图形，并在块状单元内捕获了跨转交，圆形和会话级上下文的关联。通过将检索到的原始对话与生成的记忆（例如摘要，事实和见解）相结合，SGMEM为LLM提供了一个连贯且相关的上下文，以供响应生成。关于Longmemeval和Lonoo的实验表明，SGMEM始终提高准确性，并且在长期的对话问题回答中胜过强大的基准。

Title: Query-Centric Graph Retrieval Augmented Generation

• Authors: Yaxiong Wu, Jianyuan Bo, Yongyue Zhang, Sheng Liang, Yong Liu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2509.21237
• Pdf URL: https://arxiv.org/pdf/2509.21237
• Copy Paste: [[2509.21237]] Query-Centric Graph Retrieval Augmented Generation(https://arxiv.org/abs/2509.21237)
• Keywords: language model, llm, retrieval augmented generation, retrieval-augmented generation
• Abstract: Graph-based retrieval-augmented generation (RAG) enriches large language models (LLMs) with external knowledge for long-context understanding and multi-hop reasoning, but existing methods face a granularity dilemma: fine-grained entity-level graphs incur high token costs and lose context, while coarse document-level graphs fail to capture nuanced relations. We introduce QCG-RAG, a query-centric graph RAG framework that enables query-granular indexing and multi-hop chunk retrieval. Our query-centric approach leverages Doc2Query and Doc2Query{-}{-} to construct query-centric graphs with controllable granularity, improving graph quality and interpretability. A tailored multi-hop retrieval mechanism then selects relevant chunks via the generated queries. Experiments on LiHuaWorld and MultiHop-RAG show that QCG-RAG consistently outperforms prior chunk-based and graph-based RAG methods in question answering accuracy, establishing a new paradigm for multi-hop reasoning.
• 摘要：基于图形的检索效果生成（RAG）丰富了具有外部知识的大语言模型（LLM），具有长期的理解和多跳的推理，但是现有的方法面临着粒度的困境：良好的实体级别的图形会产生高代价和损失上下文，而粗糙的文档级别的图形则无法捕获nucection nucative的关系。我们介绍了QCG-rag，这是一种以查询为中心的图形抹布框架，可实现查询粒度索引和多跳块检索。我们以查询为中心的方法利用doc2query和doc2query { - } { - }来构建具有可控粒度的以查询为中心的图形，从而提高了图形质量和解释性。然后，量身定制的多跳检索机制通过生成的查询选择相关的块。关于Lihuaworld和Multihop-rag的实验表明，QCG-rag始终胜过基于块的基于块和基于图的抹布方法，以回答准确性，建立了用于多跳的推理的新范式。

Title: Un-Doubling Diffusion: LLM-guided Disambiguation of Homonym Duplication

• Authors: Evgeny Kaskov, Elizaveta Petrova, Petr Surovtsev, Anna Kostikova, Ilya Mistiurin, Alexander Kapitanov, Alexander Nagaev
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21262
• Pdf URL: https://arxiv.org/pdf/2509.21262
• Copy Paste: [[2509.21262]] Un-Doubling Diffusion: LLM-guided Disambiguation of Homonym Duplication(https://arxiv.org/abs/2509.21262)
• Keywords: language model, llm, prompt
• Abstract: Homonyms are words with identical spelling but distinct meanings, which pose challenges for many generative models. When a homonym appears in a prompt, diffusion models may generate multiple senses of the word simultaneously, which is known as homonym duplication. This issue is further complicated by an Anglocentric bias, which includes an additional translation step before the text-to-image model pipeline. As a result, even words that are not homonymous in the original language may become homonyms and lose their meaning after translation into English. In this paper, we introduce a method for measuring duplication rates and conduct evaluations of different diffusion models using both automatic evaluation utilizing Vision-Language Models (VLM) and human evaluation. Additionally, we investigate methods to mitigate the homonym duplication problem through prompt expansion, demonstrating that this approach also effectively reduces duplication related to Anglocentric bias. The code for the automatic evaluation pipeline is publicly available.
• 摘要：同音词是具有相同拼写但含义不同的单词，这对许多生成模型构成了挑战。当同音词出现在提示中时，扩散模型可能会同时产生单词的多种感官，这称为同名重复。这个问题更加复杂，以中心为中心的偏见，其中包括在文本到图像模型管道之前的额外翻译步骤。结果，即使在原始语言中不是同义词的单词也可能变成谐调，并在翻译成英语后失去其含义。在本文中，我们介绍了一种使用视觉模型（VLM）和人类评估的自动评估来测量重复率并进行不同扩散模型的评估方法。此外，我们研究了通过迅速扩展来减轻同义词重复问题的方法，这表明该方法还有效地减少了与同性心偏见相关的重复。自动评估管道的代码公开可用。

Title: LLM Output Homogenization is Task Dependent

• Authors: Shomik Jain, Jack Lanchantin, Maximilian Nickel, Karen Ullrich, Ashia Wilson, Jamelle Watson-Daniels
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2509.21267
• Pdf URL: https://arxiv.org/pdf/2509.21267
• Copy Paste: [[2509.21267]] LLM Output Homogenization is Task Dependent(https://arxiv.org/abs/2509.21267)
• Keywords: language model, llm
• Abstract: A large language model can be less helpful if it exhibits output response homogenization. But whether two responses are considered homogeneous, and whether such homogenization is problematic, both depend on the task category. For instance, in objective math tasks, we often expect no variation in the final answer but anticipate variation in the problem-solving strategy. Whereas, for creative writing tasks, we may expect variation in key narrative components (e.g. plot, genre, setting, etc), beyond the vocabulary or embedding diversity produced by temperature-sampling. Previous work addressing output homogenization often fails to conceptualize diversity in a task-dependent way. We address this gap in the literature directly by making the following contributions. (1) We present a task taxonomy comprised of eight task categories that each have distinct conceptualizations of output homogenization. (2) We introduce task-anchored functional diversity to better evaluate output homogenization. (3) We propose a task-anchored sampling technique that increases functional diversity for task categories where homogenization is undesired, while preserving homogenization where it is desired. (4) We challenge the perceived existence of a diversity-quality trade-off by increasing functional diversity while maintaining response quality. Overall, we demonstrate how task dependence improves the evaluation and mitigation of output homogenization.
• 摘要：如果大型语言模型表现出输出响应均匀化，则可能会较小。但是，两个响应是否被认为是均匀的，以及这种均质化是否有问题，都取决于任务类别。例如，在客观的数学任务中，我们通常期望最终答案没有变化，而是预计解决问题的策略会有所不同。尽管对于创意写作任务，我们可能会期望关键叙事组件（例如情节，流派，设置等）的变化，而不是温度采样产生的词汇或嵌入多样性。以前解决产出同质化的工作通常无法以任务依赖的方式概念化多样性。我们通过做出以下贡献直接解决文献中的这一差距。 （1）我们提出了一个由八个任务类别组成的任务分类法，每个任务类别都有不同的输出同质化概念化。 （2）我们介绍任务锚定的功能多样性，以更好地评估输出均匀化。 （3）我们提出了一种任务锚定的采样技术，该技术增加了不希望的均质化的任务类别的功能多样性，同时保留了需要的均质化。 （4）我们通过在保持响应质量的同时提高功能多样性来挑战多样性质量的权衡。总体而言，我们证明了任务依赖性如何改善输出均质化的评估和缓解。

Title: LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text

• Authors: Irina Tolstykh, Aleksandra Tsybina, Sergey Yakubson, Maksim Kuprashevich
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21269
• Pdf URL: https://arxiv.org/pdf/2509.21269
• Copy Paste: [[2509.21269]] LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text(https://arxiv.org/abs/2509.21269)
• Keywords: language model, llm
• Abstract: The widespread use of human-like text from Large Language Models (LLMs) necessitates the development of robust detection systems. However, progress is limited by a critical lack of suitable training data; existing datasets are often generated with outdated models, are predominantly in English, and fail to address the increasingly common scenario of mixed human-AI authorship. Crucially, while some datasets address mixed authorship, none provide the character-level annotations required for the precise localization of AI-generated segments within a text. To address these gaps, we introduce LLMTrace, a new large-scale, bilingual (English and Russian) corpus for AI-generated text detection. Constructed using a diverse range of modern proprietary and open-source LLMs, our dataset is designed to support two key tasks: traditional full-text binary classification (human vs. AI) and the novel task of AI-generated interval detection, facilitated by character-level annotations. We believe LLMTrace will serve as a vital resource for training and evaluating the next generation of more nuanced and practical AI detection models. The project page is available at \href{this https URL}{iitolstykh/LLMTrace}.
• 摘要：大型语言模型（LLM）的类似人类文本的广泛使用需要开发可靠的检测系统。但是，进展受到严重缺乏合适的培训数据的限制。现有的数据集通常是用过时的模型生成的，主要是英语，并且无法解决越来越常见的人类AI作者身份的情况。至关重要的是，虽然某些数据集解决了混合作者身份，但没有一个提供字符级别的注释，即在文本中精确定位AI生成的片段所需的字符级注​​释。为了解决这些差距，我们介绍了llmtrace，这是一种新的大型，双语（英语和俄语）语料库，以进行AI生成的文本检测。我们的数据集使用各种各样的现代专有和开源LLMS构建，旨在支持两个关键任务：传统的全文本二进制分类（Human vs. AI）以及AI生成的间隔检测的新任务，由角色级别的注释促进。我们认为，LLMtrace将成为培训和评估下一代更细微和实用的AI检测模型的重要资源。该项目页面可在\ href {this HTTPS url} {iitolstykh/llmtrace}上获得。

Title: Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyond

• Authors: Dingzirui Wang, Xuanliang Zhang, Keyan Xu, Qingfu Zhu, Wanxiang Che, Yang Deng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21284
• Pdf URL: https://arxiv.org/pdf/2509.21284
• Copy Paste: [[2509.21284]] Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyond(https://arxiv.org/abs/2509.21284)
• Keywords: prompt, chain-of-thought
• Abstract: Existing research indicates that the output of Chain-of-Thought (CoT) is significantly affected by input perturbations. Although many methods aim to mitigate such impact by optimizing prompts, a theoretical explanation of how these perturbations influence CoT outputs remains an open area of research. This gap limits our in-depth understanding of how input perturbations propagate during the reasoning process and hinders further improvements in prompt optimization methods. Therefore, in this paper, we theoretically analyze the effect of input perturbations on the fluctuation of CoT outputs. We first derive an upper bound for input perturbations under the condition that the output fluctuation is within an acceptable range, based on which we prove that: (i) This upper bound is positively correlated with the number of reasoning steps in the CoT; (ii) Even an infinitely long reasoning process cannot eliminate the impact of input perturbations. We then apply these conclusions to the Linear Self-Attention (LSA) model, which can be viewed as a simplified version of the Transformer. For the LSA model, we prove that the upper bound for input perturbation is negatively correlated with the norms of the input embedding and hidden state vectors. To validate this theoretical analysis, we conduct experiments on three mainstream datasets and four mainstream models. The experimental results align with our theoretical analysis, empirically demonstrating the correctness of our findings.
• 摘要：现有的研究表明，投入扰动的思想链（COT）的产出显着影响。尽管许多方法旨在通过优化提示来减轻这种影响，但对这些扰动如何影响COT输出的理论解释仍然是一个开放的研究领域。该差距限制了我们对在推理过程中输入扰动如何传播的深入理解，并阻碍了迅速优化方法的进一步改进。因此，在本文中，我们理论上分析了输入扰动对COT输出波动的影响。我们首先在输出波动在可接受的范围内的条件下，为输入扰动提供了上限，我们证明：（i）此上限与COT中的推理步骤的数量正相关； （ii）即使是无限长的推理过程，也无法消除输入扰动的影响。然后，我们将这些结论应用于线性自我注意（LSA）模型，可以将其视为变压器的简化版本。对于LSA模型，我们证明输入扰动的上限与输入嵌入和隐藏状态向量的规范负相关。为了验证这一理论分析，我们在三个主流数据集和四个主流模型上进行实验。实验结果与我们的理论分析保持一致，从经验上证明了我们发现的正确性。

Title: DisCoCLIP: A Distributional Compositional Tensor Network Encoder for Vision-Language Understanding

• Authors: Kin Ian Lo, Hala Hawashin, Mina Abbaszadeh, Tilen Limback-Stokin, Hadi Wazni, Mehrnoosh Sadrzadeh
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.21287
• Pdf URL: https://arxiv.org/pdf/2509.21287
• Copy Paste: [[2509.21287]] DisCoCLIP: A Distributional Compositional Tensor Network Encoder for Vision-Language Understanding(https://arxiv.org/abs/2509.21287)
• Keywords: language model
• Abstract: Recent vision-language models excel at large-scale image-text alignment but often neglect the compositional structure of language, leading to failures on tasks that hinge on word order and predicate-argument structure. We introduce DisCoCLIP, a multimodal encoder that combines a frozen CLIP vision transformer with a novel tensor network text encoder that explicitly encodes syntactic structure. Sentences are parsed with a Combinatory Categorial Grammar parser to yield distributional word tensors whose contractions mirror the sentence's grammatical derivation. To keep the model efficient, high-order tensors are factorized with tensor decompositions, reducing parameter count from tens of millions to under one million. Trained end-to-end with a self-supervised contrastive loss, DisCoCLIP markedly improves sensitivity to verb semantics and word order: it raises CLIP's SVO-Probes verb accuracy from 77.6% to 82.4%, boosts ARO attribution and relation scores by over 9% and 4%, and achieves 93.7% on a newly introduced SVO-Swap benchmark. These results demonstrate that embedding explicit linguistic structure via tensor networks yields interpretable, parameter-efficient representations that substantially improve compositional reasoning in vision-language tasks.
• 摘要：最近的视觉语言模型在大规模图像文本对齐中表现出色，但经常忽略语言的组成结构，从而导致私有的任务失败，这些任务取决于单词顺序和谓词题目结构。我们介绍了Discoplip，这是一种多模式编码器，将冷冻夹视觉变压器与新型张量网络文本编码器结合在一起，该编码器明确编码了句法结构。句子与组合性分类语法解析器解析，以产生分布单词张量，其收缩反映了该句子的语法派生。为了保持模型效率，通过张量分解将高阶张量分解，将参数计数从数千万减少到一百万不到一百万。经过训练的端到端训练有素的对比度损失，椎间盘显着提高了对动词语义和单词顺序的敏感性：它将Clip的SVO-Probes Verb精度从77.6％提高到82.4％，将ARO属性和关系片段提高到9％和4％，并在93. 73. 73. 73.7％上提高了SVO，并将其提高。这些结果表明，通过张量网络嵌入明确的语言结构会产生可解释的，有效的参数表示形式，从而在视觉任务中实质上改善了组成推理。

Title: The role of synthetic data in Multilingual, Multi-cultural AI systems: Lessons from Indic Languages

• Authors: Pranjal A. Chitale, Varun Gumma, Sanchit Ahuja, Prashant Kodali, Manan Uppadhyay, Deepthi Sudharsan, Sunayana Sitaram
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21294
• Pdf URL: https://arxiv.org/pdf/2509.21294
• Copy Paste: [[2509.21294]] The role of synthetic data in Multilingual, Multi-cultural AI systems: Lessons from Indic Languages(https://arxiv.org/abs/2509.21294)
• Keywords: llm, prompt
• Abstract: Developing AI systems that operate effectively across languages while remaining culturally grounded is a long-standing challenge, particularly in low-resource settings. Synthetic data provides a promising avenue, yet its effectiveness in multilingual and multicultural contexts remains underexplored. We investigate the creation and impact of synthetic, culturally contextualized datasets for Indian languages through a bottom-up generation strategy that prompts large open-source LLMs (>= 235B parameters) to ground data generation in language-specific Wikipedia content. This approach complements the dominant top-down paradigm of translating synthetic datasets from high-resource languages such as English. We introduce Updesh, a high-quality large-scale synthetic instruction-following dataset comprising 9.5M data points across 13 Indian languages, encompassing diverse reasoning and generative tasks with an emphasis on long-context, multi-turn capabilities, and alignment with Indian cultural contexts. A comprehensive evaluation incorporating both automated metrics and human annotation across 10k assessments indicates that generated data is high quality; though, human evaluation highlights areas for further improvement. Additionally, we perform downstream evaluations by fine-tuning models on our dataset and assessing the performance across 15 diverse multilingual datasets. Models trained on Updesh consistently achieve significant gains on generative tasks and remain competitive on multiple-choice style NLU tasks. Notably, relative improvements are most pronounced in low and medium-resource languages, narrowing their gap with high-resource languages. These findings provide empirical evidence that effective multilingual AI requires multi-faceted data curation and generation strategies that incorporate context-aware, culturally grounded methodologies.
• 摘要：开发跨语言有效运行同时保持文化扎根的AI系统是一个长期的挑战，尤其是在低资源环境中。合成数据提供了有希望的途径，但其在多语言和多元文化环境中的有效性仍未得到充实。我们通过自下而上的生成策略调查了综合，文化上下文化数据集对印度语言的创建和影响，该策略促使大型开源LLM（> = 235b参数）以特定语言的Wikipedia内容中的数据生成。这种方法补充了从高资源语言（例如英语）翻译合成数据集的主要自上而下的范式。我们介绍了Updesh，这是一项高质量的大规模合成指令，遵循数据集，其中包括13种印度语言的950万个数据点，其中包括多种推理和生成型任务，重点是长期文化，多转化功能，以及与印度文化背景的一致性。一项全面的评估，其中包含10K评估的自动指标和人类注释，表明生成的数据是高质量的；但是，人类评估突出了进一步改进的领域。此外，我们通过在数据集中的微调模型进行微调模型进行下游评估，并评估15种多种语言数据集的性能。在Updesh上训练的模型始终在生成任务上取得了重大收益，并在多项选择样式的NLU任务上保持竞争力。值得注意的是，相对改进在低资源语言中最为明显，用高资源语言缩小了差距。这些发现提供了经验证据，表明有效的多语言AI需要多方面的数据策略和生成策略，以结合上下文感知的文化扎根方法。

Title: Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs

• Authors: Daniel Vennemeyer, Phan Anh Duong, Tiffany Zhan, Tianyu Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21305
• Pdf URL: https://arxiv.org/pdf/2509.21305
• Copy Paste: [[2509.21305]] Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs(https://arxiv.org/abs/2509.21305)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) often exhibit sycophantic behaviors -- such as excessive agreement with or flattery of the user -- but it is unclear whether these behaviors arise from a single mechanism or multiple distinct processes. We decompose sycophancy into sycophantic agreement and sycophantic praise, contrasting both with genuine agreement. Using difference-in-means directions, activation additions, and subspace geometry across multiple models and datasets, we show that: (1) the three behaviors are encoded along distinct linear directions in latent space; (2) each behavior can be independently amplified or suppressed without affecting the others; and (3) their representational structure is consistent across model families and scales. These results suggest that sycophantic behaviors correspond to distinct, independently steerable representations.
• 摘要：大型语言模型（LLMS）经常表现出相关行为 - 例如与用户的过度同意或夸张的行为 - 但尚不清楚这些行为是源于单一机制还是多个不同的过程。我们将sopophancy分解为Sycophantic的一致性和忠实的赞美，与真正的同意形成鲜明对比。使用跨多个模型和数据集的差异方向，激活添加和子空间几何形状，我们表明：（1）三种行为是按照潜在空间中的不同线性方向编码的； （2）每个行为可以独立放大或抑制而不会影响其他行为； （3）它们的代表性结构在模型家族和量表之间是一致的。这些结果表明，相关行为对应于独立的独立表示表示。

Title: RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards

• Authors: Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Ellie Evans, Daniel Egert, Hoo-Chang Shin, Felipe Soares, Yi Dong, Oleksii Kuchaiev
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.21319
• Pdf URL: https://arxiv.org/pdf/2509.21319
• Copy Paste: [[2509.21319]] RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards(https://arxiv.org/abs/2509.21319)
• Keywords: llm
• Abstract: Reinforcement Learning with Human Feedback (RLHF) and Reinforcement Learning with Verifiable Rewards (RLVR) are the main RL paradigms used in LLM post-training, each offering distinct advantages. However, RLHF struggles with interpretability and reward hacking because it relies on human judgments that usually lack explicit criteria, whereas RLVR is limited in scope by its focus on correctness-based verifiers. We propose Reinforcement Learning with Binary Flexible Feedback (RLBFF), which combines the versatility of human-driven preferences with the precision of rule-based verification, enabling reward models to capture nuanced aspects of response quality beyond mere correctness. RLBFF extracts principles that can be answered in a binary fashion (e.g. accuracy of information: yes, or code readability: no) from natural language feedback. Such principles can then be used to ground Reward Model training as an entailment task (response satisfies or does not satisfy an arbitrary principle). We show that Reward Models trained in this manner can outperform Bradley-Terry models when matched for data and achieve top performance on RM-Bench (86.2%) and JudgeBench (81.4%, #1 on leaderboard as of September 24, 2025). Additionally, users can specify principles of interest at inference time to customize the focus of our reward models, in contrast to Bradley-Terry models. Finally, we present a fully open source recipe (including data) to align Qwen3-32B using RLBFF and our Reward Model, to match or exceed the performance of o3-mini and DeepSeek R1 on general alignment benchmarks of MT-Bench, WildBench, and Arena Hard v2 (at <5% of the inference cost).
• 摘要：通过人类反馈（RLHF）和可验证的奖励（RLVR）的增强学习是LLM后培训中使用的主要RL范式，每个范式都提供了明显的优势。但是，RLHF在可解释性和奖励黑客方面挣扎，因为它依赖于通常缺乏明确标准的人类判断，而RLVR的范围限制了其对基于正确性的验证者的范围。我们建议使用二进制灵活反馈（RLBFF）进行强化学习，该反馈将人类驱动的偏好的多功能性与基于规则的验证的精确性相结合，使奖励模型能够捕获响应质量的细微差别，而不是仅仅是正确性。 RLBFF提取了可以从自然语言反馈中以二进制方式回答的原则（例如，信息的准确性：是的，或代码可读性：否）。然后，这些原则可以用来将奖励模型培训作为一项任务（响应满足或不满足任意原则）。我们表明，以这种方式培训的奖励模型可以胜过Bradley-terry模型，并在RM Bench（86.2％）和法官（截至2025年9月24日）上获得RM BENCH（86.2％）和法官（81.4％，＃1，排名第一）。此外，与Bradley-Terry模型相比，用户可以在推理时间指定关注原则，以自定义奖励模型的焦点。最后，我们提出了一个完全开源的配方（包括数据），以使用RLBFF和我们的奖励模型对齐QWEN3-32B，以匹配或超过O3-Mini和DeepSeek R1在MT Bench，Wildbench和Arena Hard V2的一般对齐基准上的性能（以<5％的优势成本的5％）。

Title: SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines

• Authors: Yizhou Wang, Chen Tang, Han Deng, Jiabei Xiao, Jiaqi Liu, Jianyu Wu, Jun Yao, Pengze Li, Encheng Su, Lintao Wang, Guohang Zhuang, Yuchen Ren, Ben Fei, Ming Hu, Xin Chen, Dongzhan Zhou, Junjun He, Xiangyu Yue, Zhenfei Yin, Jiamin Wu, Qihao Zheng, Yuhao Zhou, Huihui Xu, Chenglong Ma, Yan Lu, Wenlong Zhang, Chunfeng Song, Philip Torr, Shixiang Tang, Xinzhu Ma, Wanli Ouyang, Lei Bai
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.21320
• Pdf URL: https://arxiv.org/pdf/2509.21320
• Copy Paste: [[2509.21320]] SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines(https://arxiv.org/abs/2509.21320)
• Keywords: chain-of-thought
• Abstract: We present a scientific reasoning foundation model that aligns natural language with heterogeneous scientific representations. The model is pretrained on a 206B-token corpus spanning scientific text, pure sequences, and sequence-text pairs, then aligned via SFT on 40M instructions, annealed cold-start bootstrapping to elicit long-form chain-of-thought, and reinforcement learning with task-specific reward shaping, which instills deliberate scientific reasoning. It supports four capability families, covering up to 103 tasks across workflows: (i) faithful translation between text and scientific formats, (ii) text/knowledge extraction, (iii) property prediction, (iv) property classification, (v) unconditional and conditional sequence generation and design. Compared with specialist systems, our approach broadens instruction coverage, improves cross-domain generalization, and enhances fidelity. We detail data curation and training and show that cross-discipline learning strengthens transfer and downstream reliability. The model, instruct tuning datasets and the evaluation code are open-sourced at this https URL and this https URL.
• 摘要：我们提出了一个科学推理基础模型，该模型将自然语言与异质的科学表述保持一致。该模型是在跨越科学文本，纯序列和序列文本对的206b token语体中预估计的，然后通过SFT在40m的说明上对齐，退火冷启动的引导启动以引发长期的思想链，并通过特定于任务的奖励成型，以灌输有意的科学理性。它支持四个能力系列，涵盖了跨工作流程多达103个任务：（i）文本和科学格式之间的忠实翻译，（ii）文本/知识提取，（iii）财产预测，（iv）属性分类，（v）无条件和条件的序列序列产生和设计。与专业系统相比，我们的方法扩大了指导覆盖范围，改善了跨域的概括并提高忠诚度。我们详细说明数据策展和培训，并表明跨学科学习可以增强转移和下游的可靠性。模型，指示调谐数据集和评估代码在此HTTPS URL和此HTTPS URL上开源。