2025-09-12

Title: Noise or Nuance: An Investigation Into Useful Information and Filtering For LLM Driven AKBC

• Authors: Alex Clay, Ernesto Jiménez-Ruiz, Pranava Madhyastha
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.08903
• Pdf URL: https://arxiv.org/pdf/2509.08903
• Copy Paste: [[2509.08903]] Noise or Nuance: An Investigation Into Useful Information and Filtering For LLM Driven AKBC(https://arxiv.org/abs/2509.08903)
• Keywords: llm
• Abstract: RAG and fine-tuning are prevalent strategies for improving the quality of LLM outputs. However, in constrained situations, such as that of the 2025 LM-KBC challenge, such techniques are restricted. In this work we investigate three facets of the triple completion task: generation, quality assurance, and LLM response parsing. Our work finds that in this constrained setting: additional information improves generation quality, LLMs can be effective at filtering poor quality triples, and the tradeoff between flexibility and consistency with LLM response parsing is setting dependent.
• 摘要：抹布和微调是提高LLM产出质量的普遍策略。但是，在有限的情况下，例如2025 LM-KBC挑战的情况下，这种技术受到限制。在这项工作中，我们研究了三重完成任务的三个方面：生成，质量保证和LLM响应解析。我们的工作发现，在这种限制的设置中：其他信息可以提高发电质量，LLM可以有效地过滤质量差的三元组，并且灵活性与LLM响应解析的一致性之间的权衡取决于设置。

Title: Automated Evidence Extraction and Scoring for Corporate Climate Policy Engagement: A Multilingual RAG Approach

• Authors: Imene Kolli, Ario Saeid Vaghefi, Chiara Colesanti Senni, Shantam Raj, Markus Leippold
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.08907
• Pdf URL: https://arxiv.org/pdf/2509.08907
• Copy Paste: [[2509.08907]] Automated Evidence Extraction and Scoring for Corporate Climate Policy Engagement: A Multilingual RAG Approach(https://arxiv.org/abs/2509.08907)
• Keywords: prompt, retrieval-augmented generation
• Abstract: InfluenceMap's LobbyMap Platform monitors the climate policy engagement of over 500 companies and 250 industry associations, assessing each entity's support or opposition to science-based policy pathways for achieving the Paris Agreement's goal of limiting global warming to 1.5°C. Although InfluenceMap has made progress with automating key elements of the analytical workflow, a significant portion of the assessment remains manual, making it time- and labor-intensive and susceptible to human error. We propose an AI-assisted framework to accelerate the monitoring of corporate climate policy engagement by leveraging Retrieval-Augmented Generation to automate the most time-intensive extraction of relevant evidence from large-scale textual data. Our evaluation shows that a combination of layout-aware parsing, the Nomic embedding model, and few-shot prompting strategies yields the best performance in extracting and classifying evidence from multilingual corporate documents. We conclude that while the automated RAG system effectively accelerates evidence extraction, the nuanced nature of the analysis necessitates a human-in-the-loop approach where the technology augments, rather than replaces, expert judgment to ensure accuracy.
• 摘要：Implasemap的LobbyMap平台监视了500多家公司和250个行业协会的气候政策参与，评估了每个实体对基于科学的政策途径的支持或反对，以实现《巴黎协定》将全球变暖限制为1.5°C的目标。尽管Imporcemap在分析工作流程的关键要素上取得了进展，但评估的很大一部分仍是手动的，使其具有时间和劳动力密集型，并且容易受到人为错误的影响。我们提出了一个AI辅助框架，以通过利用检索功能的生成来自动化最耗时的相关证据，从大规模的文本数据中自动化相关证据，从而加速对公司气候政策参与的监控。我们的评估表明，布局意识解析，提名嵌入模型以及少量弹性策略的结合在从多语言公司文档中提取和分类证据方面产生了最佳性能。我们得出的结论是，尽管自动化的抹布系统有效地加速了证据提取，但分析的细微差别是必须采取一种人类的方法，即技术增强而不是取代专家判断以确保准确性。

Title: Documents Are People and Words Are Items: A Psychometric Approach to Textual Data with Contextual Embeddings

• Authors: Jinsong Chen
• Subjects: cs.CL, stat.AP, stat.ME
• Abstract URL: https://arxiv.org/abs/2509.08920
• Pdf URL: https://arxiv.org/pdf/2509.08920
• Copy Paste: [[2509.08920]] Documents Are People and Words Are Items: A Psychometric Approach to Textual Data with Contextual Embeddings(https://arxiv.org/abs/2509.08920)
• Keywords: language model
• Abstract: This research introduces a novel psychometric method for analyzing textual data using large language models. By leveraging contextual embeddings to create contextual scores, we transform textual data into response data suitable for psychometric analysis. Treating documents as individuals and words as items, this approach provides a natural psychometric interpretation under the assumption that certain keywords, whose contextual meanings vary significantly across documents, can effectively differentiate documents within a corpus. The modeling process comprises two stages: obtaining contextual scores and performing psychometric analysis. In the first stage, we utilize natural language processing techniques and encoder based transformer models to identify common keywords and generate contextual scores. In the second stage, we employ various types of factor analysis, including exploratory and bifactor models, to extract and define latent factors, determine factor correlations, and identify the most significant words associated with each factor. Applied to the Wiki STEM corpus, our experimental results demonstrate the method's potential to uncover latent knowledge dimensions and patterns within textual data. This approach not only enhances the psychometric analysis of textual data but also holds promise for applications in fields rich in textual information, such as education, psychology, and law.
• 摘要：这项研究介绍了一种新型的心理测量方法，用于使用大语言模型分析文本数据。通过利用上下文嵌入来创建上下文分数，我们将文本数据转换为适合心理测量分析的响应数据。将文档视为个人和单词作为项目，这种方法提供了一种自然的心理计量学解释，假设某些关键字（其上下文含义在文档之间都有很大差异，可以有效地区分语料库中的文档。建模过程包括两个阶段：获得上下文分数并执行心理测量分析。在第一阶段，我们利用自然语言处理技术和基于编码器的变压器模型来识别通用关键字并生成上下文分数。在第二阶段，我们采用各种类型的因子分析，包括探索性和双向分子模型来提取和定义潜在因素，确定因子相关性并确定与每个因素相关的最重要单词。我们的实验结果应用于Wiki Stem语料库，证明了该方法在文本数据中发现潜在知识维度和模式的潜力。这种方法不仅增强了文本数据的心理测量分析，而且还对富含文本信息的领域的应用（例如教育，心理学和法律）持希望。

Title: BRoverbs -- Measuring how much LLMs understand Portuguese proverbs

• Authors: Thales Sales Almeida, Giovana Kerche Bonás, João Guilherme Alves Santos
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.08960
• Pdf URL: https://arxiv.org/pdf/2509.08960
• Copy Paste: [[2509.08960]] BRoverbs -- Measuring how much LLMs understand Portuguese proverbs(https://arxiv.org/abs/2509.08960)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) exhibit significant performance variations depending on the linguistic and cultural context in which they are applied. This disparity signals the necessity of mature evaluation frameworks that can assess their capabilities in specific regional settings. In the case of Portuguese, existing evaluations remain limited, often relying on translated datasets that may not fully capture linguistic nuances or cultural references. Meanwhile, native Portuguese-language datasets predominantly focus on structured national exams or sentiment analysis of social media interactions, leaving gaps in evaluating broader linguistic understanding. To address this limitation, we introduce BRoverbs, a dataset specifically designed to assess LLM performance through Brazilian proverbs. Proverbs serve as a rich linguistic resource, encapsulating cultural wisdom, figurative expressions, and complex syntactic structures that challenge the model comprehension of regional expressions. BRoverbs aims to provide a new evaluation tool for Portuguese-language LLMs, contributing to advancing regionally informed benchmarking. The benchmark is available at this https URL.
• 摘要：大型语言模型（LLMS）表现出显着的性能差异，具体取决于应用程序的语言和文化背景。这种差异标志着可以评估其在特定区域环境中的能力的成熟评估框架的必要性。在葡萄牙语的情况下，现有的评估仍然有限，通常依赖于可能无法完全捕获语言细微差别或文化参考的翻译数据集。同时，本地葡萄牙语数据集主要关注结构化的国家考试或社交媒体互动的情感分析，从而在评估更广泛的语言理解方面留下了差距。为了解决这一限制，我们介绍了Broverbs，这是一个专门旨在通过巴西谚语评估LLM性能的数据集。谚语是一种丰富的语言资源，封装了文化智慧，形象表达和复杂的句法结构，这些结构挑战了区域表达的模型理解。 Broverbs旨在为葡萄牙语LLM提供一种新的评估工具，从而有助于推进区域知情的基准测试。该基准标准可在此HTTPS URL上找到。

Title: Can Vision-Language Models Solve Visual Math Equations?

• Authors: Monjoy Narayan Choudhury, Junling Wang, Yifan Hou, Mrinmaya Sachan
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2509.09013
• Pdf URL: https://arxiv.org/pdf/2509.09013
• Copy Paste: [[2509.09013]] Can Vision-Language Models Solve Visual Math Equations?(https://arxiv.org/abs/2509.09013)
• Keywords: language model
• Abstract: Despite strong performance in visual understanding and language-based reasoning, Vision-Language Models (VLMs) struggle with tasks requiring integrated perception and symbolic computation. We study this limitation through visual equation solving, where mathematical equations are embedded in images, variables are represented by object icons, and coefficients must be inferred by counting. While VLMs perform well on textual equations, they fail on visually grounded counterparts. To understand this gap, we decompose the task into coefficient counting and variable recognition, and find that counting is the primary bottleneck, even when recognition is accurate. We also observe that composing recognition and reasoning introduces additional errors, highlighting challenges in multi-step visual reasoning. Finally, as equation complexity increases, symbolic reasoning itself becomes a limiting factor. These findings reveal key weaknesses in current VLMs and point toward future improvements in visually grounded mathematical reasoning.
• 摘要：尽管在视觉理解和基于语言的推理中表现出色，但视觉语言模型（VLM）与需要集成感知和符号计算的任务斗争。我们通过可视方程求解研究了这种限制，其中数学方程嵌入了图像中，变量由对象图标表示，并且必须通过计数来推断系数。当VLMS在文本方程式上表现良好时，它们在视觉扎根的对应方面失败。要了解这一差距，我们将任务分解为系数计数和可变识别，并发现计数是主要的瓶颈，即使识别准确。我们还观察到，组成识别和推理会引入其他错误，突出了多步视觉推理中的挑战。最后，随着方程复杂性的增加，符号推理本身成为一个限制因素。这些发现揭示了当前VLM中的关键弱点，并指出了视觉扎根的数学推理的未来改善。

Title: Stated Preference for Interaction and Continued Engagement (SPICE): Evaluating an LLM's Willingness to Re-engage in Conversation

• Authors: Thomas Manuel Rost, Martina Figlia, Bernd Wallraff
• Subjects: cs.CL, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2509.09043
• Pdf URL: https://arxiv.org/pdf/2509.09043
• Copy Paste: [[2509.09043]] Stated Preference for Interaction and Continued Engagement (SPICE): Evaluating an LLM's Willingness to Re-engage in Conversation(https://arxiv.org/abs/2509.09043)
• Keywords: language model, llm, chat
• Abstract: We introduce and evaluate Stated Preference for Interaction and Continued Engagement (SPICE), a simple diagnostic signal elicited by asking a Large Language Model a YES or NO question about its willingness to re-engage with a user's behavior after reviewing a short transcript. In a study using a 3-tone (friendly, unclear, abusive) by 10-interaction stimulus set, we tested four open-weight chat models across four framing conditions, resulting in 480 trials. Our findings show that SPICE sharply discriminates by user tone. Friendly interactions yielded a near-unanimous preference to continue (97.5% YES), while abusive interactions yielded a strong preference to discontinue (17.9% YES), with unclear interactions falling in between (60.4% YES). This core association remains decisive under multiple dependence-aware statistical tests, including Rao-Scott adjustment and cluster permutation tests. Furthermore, we demonstrate that SPICE provides a distinct signal from abuse classification. In trials where a model failed to identify abuse, it still overwhelmingly stated a preference not to continue the interaction (81% of the time). An exploratory analysis also reveals a significant interaction effect: a preamble describing the study context significantly impacts SPICE under ambiguity, but only when transcripts are presented as a single block of text rather than a multi-turn chat. The results validate SPICE as a robust, low-overhead, and reproducible tool for auditing model dispositions, complementing existing metrics by offering a direct, relational signal of a model's state. All stimuli, code, and analysis scripts are released to support replication.
• 摘要：我们介绍和评估对互动和持续参与度（SPICE）的偏爱，这是一个简单的诊断信号，通过向大型语言模型询问是或否定了其愿意在审查简短笔录后愿意与用户行为重新参与的疑问。在一项使用10相互作用刺激集使用3色调（友好，不清楚，虐待）的研究中，我们在四个框架条件下测试了四个开放式聊天模型，从而进行了480次试验。我们的发现表明，香料可以通过用户音调明显地区分。友好的相互作用产生了几乎一致的偏爱（97.5％是），而虐待相互作用则具有强烈的偏爱（17.9％是），而不清楚的相互作用介于介于介于60.4％之间（60.4％）。在多种依赖感知的统计检验下，这种核心关联仍然是决定性的，包括Rao-Scott调整和群集置换测试。此外，我们证明了香料从滥用分类中提供了明显的信号。在模型未能识别滥用的试验中，它仍然压倒性地表示不继续进行互动（81％的时间）。探索性分析还揭示了一种显着的互动效果：描述研究背景​​的序言在模棱两可的情况下显着影响香料，但仅当成绩单作为单个文本块而不是多转弯聊天时。结果将香料验证为用于审核模型处置的稳健，低空和可重复的工具，通过提供模型状态的直接关系信号来补充现有指标。所有刺激，代码和分析脚本都会发布以支持复制。

Title: Improving LLM Safety and Helpfulness using SFT and DPO: A Study on OPT-350M

• Authors: Piyush Pant
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.09055
• Pdf URL: https://arxiv.org/pdf/2509.09055
• Copy Paste: [[2509.09055]] Improving LLM Safety and Helpfulness using SFT and DPO: A Study on OPT-350M(https://arxiv.org/abs/2509.09055)
• Keywords: language model, llm
• Abstract: This research investigates the effectiveness of alignment techniques, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and a combined SFT+DPO approach on improving the safety and helpfulness of the OPT-350M language model. Utilizing the Anthropic Helpful-Harmless RLHF dataset, we train and evaluate four models: the base OPT350M, an SFT model, a DPO model, and a model trained with both SFT and DPO. We introduce three key evaluation metrics: Harmlessness Rate (HmR), Helpfulness Rate (HpR), and a Combined Alignment Score (CAS), all derived from reward model outputs. The results show that while SFT outperforms DPO, The combined SFT+DPO model outperforms all others across all metrics, demonstrating the complementary nature of these techniques. Our findings also highlight challenges posed by noisy data, limited GPU resources, and training constraints. This study offers a comprehensive view of how fine-tuning strategies affect model alignment and provides a foundation for more robust alignment pipelines in future work.
• 摘要：这项研究调查了对齐技术的有效性，受监督的微调（SFT），直接偏好优化（DPO）以及SFT+DPO合并的方法，以提高Opt-350m语言模型的安全性和帮助性。利用人类有用的无障碍RLHF数据集，我们训练和评估四个型号：基本Opt350m，SFT模型，DPO模型以及经过SFT和DPO训练的模型。我们介绍了三个关键的评估指标：无害性率（HMR），有用度率（HPR）和组合对齐评分（CAS），这些评分均来自奖励模型输出。结果表明，尽管SFT胜过DPO，但组合的SFT+DPO模型在所有指标中的表现都优于所有其他指标，这表明了这些技术的互补性质。我们的发现还强调了嘈杂的数据，有限的GPU资源和培训限制所带来的挑战。这项研究提供了有关微调策略如何影响模型对齐方式的全面看法，并为未来工作中更健壮的一致性管道提供了基础。

Title: MR-UIE: Multi-Perspective Reasoning with Reinforcement Learning for Universal Information Extraction

• Authors: Zhongqiu Li, Shiquan Wang, Ruiyu Fang, Mengjiao Bao, Zhenhe Wu, Shuangyong Song, Yongxiang Li, Zhongjiang He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09082
• Pdf URL: https://arxiv.org/pdf/2509.09082
• Copy Paste: [[2509.09082]] MR-UIE: Multi-Perspective Reasoning with Reinforcement Learning for Universal Information Extraction(https://arxiv.org/abs/2509.09082)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) demonstrate robust capabilities across diverse research domains. However, their performance in universal information extraction (UIE) remains insufficient, especially when tackling structured output scenarios that involve complex schema descriptions and require multi-step reasoning. While existing approaches enhance the performance of LLMs through in-context learning and instruction tuning, significant limitations nonetheless persist. To enhance the model's generalization ability, we propose integrating reinforcement learning (RL) with multi-perspective reasoning for information extraction (IE) tasks. Our work transitions LLMs from passive extractors to active reasoners, enabling them to understand not only what to extract but also how to reason. Experiments conducted on multiple IE benchmarks demonstrate that MR-UIE consistently elevates extraction accuracy across domains and surpasses state-of-the-art methods on several datasets. Furthermore, incorporating multi-perspective reasoning into RL notably enhances generalization in complex IE tasks, underscoring the critical role of reasoning in challenging scenarios.
• 摘要：大型语言模型（LLMS）展示了各种研究领域的强大功能。但是，它们在通用信息提取（UIE）中的性能仍然不足，尤其是在解决涉及复杂模式描述并需要多步推理的结构化输出方案时。尽管现有方法通过在文化学习和教学调整中增强了LLM的性能，但仍存在重大限制。为了增强模型的概括能力，我们建议将增强学习（RL）与信息提取（IE）任务的多方面推理集成。我们的工作将LLM从被动提取器转变为活跃的推理者，使他们不仅可以理解要提取的内容，还可以理解如何推理。在多个IE基准上进行的实验表明，MR-UIE始终提高范围内的提取精度，并超过了几个数据集上的最新方法。此外，将多镜头推理纳入RL中，显着增强了复杂的IE任务中的概括，强调了推理在挑战的情况下的关键作用。

Title: TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla

• Authors: Nishat Raihan, Antonios Anastasopoulos, Marcos Zampieri
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09101
• Pdf URL: https://arxiv.org/pdf/2509.09101
• Copy Paste: [[2509.09101]] TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla(https://arxiv.org/abs/2509.09101)
• Keywords: language model, llm
• Abstract: Despite being the 5th most spoken language, Bangla remains underrepresented in Large Language Models (LLMs), particularly for code generation. This primarily stems from the scarcity of high-quality data to pre-train and/or finetune such models. Hence, we introduce the first dedicated family of Code LLMs for Bangla (1B & 9B). We offer three major contributions: (1) a comprehensive Bangla code instruction datasets for programming domain adaptation; (2) MBPP-Bangla, an evaluation benchmark for Bangla code generation; and (3) the TigerCoder-family of Code LLMs, achieving significant ~11-18% performance gains at Pass@1 over existing multilingual and general-purpose Bangla LLMs. Our findings show that curated, high-quality datasets can overcome limitations of smaller models for low-resource languages. We open-source all resources to advance further Bangla LLM research.
• 摘要：尽管是口语中排名第五的语言，但孟加拉国在大型语言模型（LLMS）中的代表性不足，特别是对于代码生成。这主要源于高质量数据的稀缺性，以预先培训和/或此类模型。因此，我们介绍了孟加拉第一个专门的代码LLM家族（1B和9B）。我们提供三个主要贡献：（1）用于编程域适应的全面的孟加拉代码指令数据集； （2）MBPP-Bangla，Bangla代码生成的评估基准； （3）Code LLMS的tigercoder家庭，在PASS@1上获得了〜11-18％的性能增长，超过了现有的多语言和通用Bangla LLMS。我们的发现表明，精心策划的高质量数据集可以克服低资源语言的较小模型的局限性。我们开放所有资源，以进一步推进Bangla LLM研究。

Title: Compass-v3: Scaling Domain-Specific LLMs for Multilingual E-Commerce in Southeast Asia

• Authors: Sophia Maria
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09121
• Pdf URL: https://arxiv.org/pdf/2509.09121
• Copy Paste: [[2509.09121]] Compass-v3: Scaling Domain-Specific LLMs for Multilingual E-Commerce in Southeast Asia(https://arxiv.org/abs/2509.09121)
• Keywords: language model, gpt, llm
• Abstract: Large language models (LLMs) excel in general-domain applications, yet their performance often degrades in specialized tasks requiring domain-specific knowledge. E-commerce is particularly challenging, as its data are noisy, heterogeneous, multilingual, and highly dynamic. We present Compass-v3, a vertical-domain Mixture-of-Experts (MoE) model with 245B total parameters and 71B active per token, designed for Southeast Asian e-commerce. Compass-v3 adopts fewer but larger experts, combined with hardware-efficient optimizations-such as intra-node expert parallelism and a customized memcpy operator-to maximize GPU utilization. The model is trained on 12T tokens of curated multilingual corpora and large-scale synthetic e-commerce instructions using a mixed-training strategy. To enhance alignment, we propose Optimal-Transport Direct Preference Optimization (OTPO), which captures token-level distinctions and improves instruction adherence in commerce-specific scenarios. Extensive evaluations demonstrate that Compass-v3 delivers state-of-the-art e-commerce performance, surpassing DeepSeek-V3.1, GPT-4 series, and Qwen3-235B. Moreover, Compass-v3 demonstrates strong multilingual capability across low-resource Southeast Asian languages (Indonesian, Thai, Filipino, Vietnamese, Malay, Taglog) and Portuguese while sustaining competitive performance on general benchmarks. It has already been widely applied in Shopee's industrial-scale e-commerce platform and is gradually replacing OpenAI's traffic, now accounting for over 70\% of total LLM usage, highlighting its dual strengths in specialized commerce expertise and broad linguistic competence.
• 摘要：大型语言模型（LLM）在通用域应用中表现出色，但是它们的性能经常在需要特定领域知识的专用任务中降低。电子商务特别具有挑战性，因为它的数据是嘈杂，异构，多语言和高度动态的。我们提出Compass-V3，这是一种垂直域的混合物（MOE）模型，具有245B总参数和71B的每个令牌，为东南亚电子商务设计。 Compass-V3采用的较少但更大的专家，再加上硬件有效的优化，例如节点专家并行性和定制的Memcpy操作员，以最大程度地利用GPU利用率。该模型通过使用混合培训策略的12t代币进行了12t代币和大规模合成电子商务指令的培训。为了增强对齐方式，我们提出了最佳 - 传输直接偏好优化（OTPO），该优化（OTPO）捕获令牌级别的区别并改善了特定于商业方案的指导依从性。广泛的评估表明，Compass-V3提供了最先进的电子商务表现，超过了DeepSeek-V3.1，GPT-4系列和Qwen3-235b。此外，Compass-V3在低资源的东南亚语言（印尼，泰国，菲律宾，越南，马来语，Taglog）和葡萄牙语中表现出强大的多语言能力，同时在一般基准上保持了竞争性能。它已经被广泛应用于Shopee的工业规模的电子商务平台，并且正在逐渐取代OpenAI的流量，现在占LLM总使用量的70％以上，突出了其专业商业专业知识和广泛语言能力的双重优势。

Title: Automated Classification of Tutors' Dialogue Acts Using Generative AI: A Case Study Using the CIMA Corpus

• Authors: Liqun He, Jiaqi Xu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.09125
• Pdf URL: https://arxiv.org/pdf/2509.09125
• Copy Paste: [[2509.09125]] Automated Classification of Tutors' Dialogue Acts Using Generative AI: A Case Study Using the CIMA Corpus(https://arxiv.org/abs/2509.09125)
• Keywords: gpt, prompt
• Abstract: This study explores the use of generative AI for automating the classification of tutors' Dialogue Acts (DAs), aiming to reduce the time and effort required by traditional manual coding. This case study uses the open-source CIMA corpus, in which tutors' responses are pre-annotated into four DA categories. Both GPT-3.5-turbo and GPT-4 models were tested using tailored prompts. Results show that GPT-4 achieved 80% accuracy, a weighted F1-score of 0.81, and a Cohen's Kappa of 0.74, surpassing baseline performance and indicating substantial agreement with human annotations. These findings suggest that generative AI has strong potential to provide an efficient and accessible approach to DA classification, with meaningful implications for educational dialogue analysis. The study also highlights the importance of task-specific label definitions and contextual information in enhancing the quality of automated annotation. Finally, it underscores the ethical considerations associated with the use of generative AI and the need for responsible and transparent research practices. The script of this research is publicly available at this https URL.
• 摘要：这项研究探讨了生成AI的使用来自动化导师对话行为（DAS）的分类，以减少传统手动编码所需的时间和精力。该案例研究使用开源CIMA语料库，其中导师的反应预先注销了四个DA类别。 GPT-3.5-Turbo和GPT-4模型均使用量身定制的提示进行了测试。结果表明，GPT-4的精度达到了80％，加权F1分数为0.81，Cohen的Kappa为0.74，超过了基线性能，并表明与人类注释相当一致。这些发现表明，生成的AI具有强大的潜力，可以为DA分类提供有效且易于使用的方法，对教育对话分析产生了有意义的影响。该研究还强调了特定于任务标签定义和上下文信息在增强自动注释质量方面的重要性。最后，它强调了与生成AI的使用以及对负责任和透明研究实践的需求相关的道德考虑。这项研究的脚本在此HTTPS URL上公开可用。

Title: Target-oriented Multimodal Sentiment Classification with Counterfactual-enhanced Debiasing

• Authors: Zhiyue Liu, Fanrong Ma, Xin Ling
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.09160
• Pdf URL: https://arxiv.org/pdf/2509.09160
• Copy Paste: [[2509.09160]] Target-oriented Multimodal Sentiment Classification with Counterfactual-enhanced Debiasing(https://arxiv.org/abs/2509.09160)
• Keywords: prompt
• Abstract: Target-oriented multimodal sentiment classification seeks to predict sentiment polarity for specific targets from image-text pairs. While existing works achieve competitive performance, they often over-rely on textual content and fail to consider dataset biases, in particular word-level contextual biases. This leads to spurious correlations between text features and output labels, impairing classification accuracy. In this paper, we introduce a novel counterfactual-enhanced debiasing framework to reduce such spurious correlations. Our framework incorporates a counterfactual data augmentation strategy that minimally alters sentiment-related causal features, generating detail-matched image-text samples to guide the model's attention toward content tied to sentiment. Furthermore, for learning robust features from counterfactual data and prompting model decisions, we introduce an adaptive debiasing contrastive learning mechanism, which effectively mitigates the influence of biased words. Experimental results on several benchmark datasets show that our proposed method outperforms state-of-the-art baselines.
• 摘要：面向目标的多模式分类旨在预测图像文本对的特定目标的情感极性。尽管现有作品达到了竞争性能，但它们通常在文本内容上过度汇总，并且无法考虑数据集偏见，特别是单词级别的上下文偏见。这导致文本特征和输出标签之间的虚假相关性，从而损害了分类精度。在本文中，我们介绍了一个新颖的反事实增强式伪造框架，以减少这种虚假的相关性。我们的框架结合了反事实数据增强策略，该策略最小化了与情感相关的因果特征，生成了详细的匹配匹配的图像文本样本，以指导该模型注意与情感相关的内容的关注。此外，为了从反事实数据中学习强大的特征并提示模型决策，我们引入了一种适应性的对比度学习机制，从而有效地减轻了有偏见的单词的影响。几个基准数据集的实验结果表明，我们提出的方法的表现优于最先进的基准。

Title: EchoX: Towards Mitigating Acoustic-Semantic Gap via Echo Training for Speech-to-Speech LLMs

• Authors: Yuhao Zhang, Yuhao Du, Zhanchen Dai, Xiangnan Ma, Kaiqi Kou, Benyou Wang, Haizhou Li
• Subjects: cs.CL, cs.AI, cs.SD
• Abstract URL: https://arxiv.org/abs/2509.09174
• Pdf URL: https://arxiv.org/pdf/2509.09174
• Copy Paste: [[2509.09174]] EchoX: Towards Mitigating Acoustic-Semantic Gap via Echo Training for Speech-to-Speech LLMs(https://arxiv.org/abs/2509.09174)
• Keywords: language model, llm
• Abstract: Speech-to-speech large language models (SLLMs) are attracting increasing attention. Derived from text-based large language models (LLMs), SLLMs often exhibit degradation in knowledge and reasoning capabilities. We hypothesize that this limitation arises because current training paradigms for SLLMs fail to bridge the acoustic-semantic gap in the feature representation space. To address this issue, we propose EchoX, which leverages semantic representations and dynamically generates speech training targets. This approach integrates both acoustic and semantic learning, enabling EchoX to preserve strong reasoning abilities as a speech LLM. Experimental results demonstrate that EchoX, with about six thousand hours of training data, achieves advanced performance on multiple knowledge-based question-answering benchmarks. The project is available at this https URL.
• 摘要：语音到语音大语模型（SLLM）正在吸引越来越多的关注。 SLLM源自基于文本的大语言模型（LLM），经常在知识和推理能力中表现出降级。我们假设出现了这种限制，因为SLLMS的当前训练范例无法弥合特征表示空间中的声音 - 语义差距。为了解决这个问题，我们提出了ECHOX，该ECHOX利用语义表示并动态生成语音训练目标。这种方法同时整合了声学和语义学习，从而使Echox能够将强大的推理能力保留为语音LLM。实验结果表明，ECHOX在大约六千小时的培训数据中，可以在多个基于知识的问题的基准基准上实现高级绩效。该项目可在此HTTPS URL上找到。

Title: GmSLM : Generative Marmoset Spoken Language Modeling

• Authors: Talia Sternberg, Michael London, David Omer, Yossi Adi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09198
• Pdf URL: https://arxiv.org/pdf/2509.09198
• Copy Paste: [[2509.09198]] GmSLM : Generative Marmoset Spoken Language Modeling(https://arxiv.org/abs/2509.09198)
• Keywords: language model, llm
• Abstract: Marmoset monkeys exhibit complex vocal communication, challenging the view that nonhuman primates vocal communication is entirely innate, and show similar features of human speech, such as vocal labeling of others and turn-taking. Studying their vocal communication offers a unique opportunity to link it with brain activity-especially given the difficulty of accessing the human brain in speech and language research. Since Marmosets communicate primarily through vocalizations, applying standard LLM approaches is not straightforward. We introduce Generative Marmoset Spoken Language Modeling (GmSLM), an optimized spoken language model pipeline for Marmoset vocal communication. We designed a novel zero-shot evaluation metrics using unsupervised in-the-wild data, alongside weakly labeled conversational data, to assess GmSLM and demonstrate its advantage over a basic human-speech-based baseline. GmSLM generated vocalizations closely matched real resynthesized samples acoustically and performed well on downstream tasks. Despite being fully unsupervised, GmSLM effectively distinguish real from artificial conversations and may support further investigations of the neural basis of vocal communication and provides a practical framework linking vocalization and brain activity. We believe GmSLM stands to benefit future work in neuroscience, bioacoustics, and evolutionary biology. Samples are provided under: this http URL.
• 摘要：Marmoset Monkeys表现出复杂的声音交流，挑战了非人类灵长类动物的声音交流完全是天生的，并且显示了人类言语的类似特征，例如对他人的人声标记和转弯。研究他们的声音交流提供了一个独特的机会，可以将其与大脑活动联系起来，尤其是考虑到在语音和语言研究中访问人脑的困难。由于Marmosets主要通过发声进行交流，因此应用标准LLM方法并不简单。我们介绍了生成的Marmoset口语建模（GMSLM），这是一种针对Marmoset声乐交流的优化口头语言模型管道。我们使用无监督的内部数据以及弱标记的对话数据来评估GMSLM并证明其优势比基于人类语言基本的基线，设计了一个新颖的零摄像评估指标。 GMSLM生成的发声与真实的重新合成样品近距离匹配，并在下游任务上表现良好。尽管完全无监督，但GMSLM有效地将真实与人工对话区分开，并可能支持对声音交流神经基础的进一步研究，并提供了一个实用的框架，将声音和大脑活动联系起来。我们认为，GMSLM将有利于神经科学，生物声学和进化生物学的未来工作。提供样本下方：此HTTP URL。

Title: CCF: A Context Compression Framework for Efficient Long-Sequence Language Modeling

• Authors: Wenhao Li, Bangcheng Sun, Weihao Ye, Tianyi Zhang, Daohai Yu, Fei Chao, Rongrong Ji
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09199
• Pdf URL: https://arxiv.org/pdf/2509.09199
• Copy Paste: [[2509.09199]] CCF: A Context Compression Framework for Efficient Long-Sequence Language Modeling(https://arxiv.org/abs/2509.09199)
• Keywords: language model
• Abstract: Scaling language models to longer contexts is essential for capturing rich dependencies across extended discourse. However, naïve context extension imposes significant computational and memory burdens, often resulting in inefficiencies during both training and inference. In this work, we propose CCF, a novel context compression framework designed to enable efficient long-context modeling by learning hierarchical latent representations that preserve global semantics while aggressively reducing input redundancy. CCF integrates segment-wise semantic aggregation with key-value memory encoding, forming compact representations that support accurate reconstruction and long-range understanding. To further enhance scalability, we introduce a training-efficient optimization strategy that couples incremental segment decoding with sparse reservoir sampling, substantially reducing memory overhead without degrading performance. Empirical results on multiple long-context language modeling benchmarks demonstrate that CCF achieves competitive perplexity under high compression ratios, and significantly improves throughput and memory efficiency compared to existing approaches. These findings highlight the potential of structured compression for scalable and effective long-context language modeling.
• 摘要：将语言模型缩放到更长的上下文对于在扩展话语中捕获丰富的依赖性至关重要。但是，幼稚的环境扩展会施加巨大的计算和记忆负担，通常导致训练和推理期间效率低下。在这项工作中，我们提出了CCF，这是一种新型上下文压缩框架，旨在通过学习层次的潜在表示，以保留全局语义，同时积极降低输入冗余。 CCF将段的语义聚合与键值内存编码集成，形成紧凑的表示，以支持准确的重建和远程理解。为了进一步提高可扩展性，我们引入了一种训练有效的优化策略，该策略将逐段解码与稀疏的储层抽样相结合，从而大大降低了内存开销而不会降低性能。多种长篇文章建模基准的经验结果表明，CCF在高压缩比下实现了竞争性的困惑，并且与现有方法相比，CCF显着提高了吞吐量和记忆效率。这些发现突出了结构化压缩的潜力，以进行可扩展有效的长篇文章建模。

Title: Reading Between the Lines: Classifying Resume Seniority with Large Language Models

• Authors: Matan Cohen, Shira Shani, Eden Menahem, Yehudit Aperstein, Alexander Apartsin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09229
• Pdf URL: https://arxiv.org/pdf/2509.09229
• Copy Paste: [[2509.09229]] Reading Between the Lines: Classifying Resume Seniority with Large Language Models(https://arxiv.org/abs/2509.09229)
• Keywords: language model, llm
• Abstract: Accurately assessing candidate seniority from resumes is a critical yet challenging task, complicated by the prevalence of overstated experience and ambiguous self-presentation. In this study, we investigate the effectiveness of large language models (LLMs), including fine-tuned BERT architectures, for automating seniority classification in resumes. To rigorously evaluate model performance, we introduce a hybrid dataset comprising both real-world resumes and synthetically generated hard examples designed to simulate exaggerated qualifications and understated seniority. Using the dataset, we evaluate the performance of Large Language Models in detecting subtle linguistic cues associated with seniority inflation and implicit expertise. Our findings highlight promising directions for enhancing AI-driven candidate evaluation systems and mitigating bias introduced by self-promotional language. The dataset is available for the research community at this https URL
• 摘要：准确地从简历中评估候选人资历是一项至关重要但充满挑战的任务，这是由于经验夸大的经验和模棱两可的自我表现而变得复杂。在这项研究中，我们研究了大语言模型（LLMS）的有效性，包括精细的BERT体系结构，以自动化简历中的资历分类。为了严格评估模型性能，我们介绍了一个混合数据集，其中包括现实世界简历和合成生成的硬例子，旨在模拟夸张的资格和低调的资历。使用数据集，我们评估了大语言模型的性能，以检测与资历通货膨胀和隐性专业知识相关的微妙语言提示。我们的发现突出了有望增强AI驱动的候选评估系统并减轻自我促进语言引入的偏见的有希望的方向。该数据集可用于此HTTPS URL的研究社区

Title: Agentic LLMs for Question Answering over Tabular Data

• Authors: Rishit Tyagi, Mohit Gupta, Rahul Bouri
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09234
• Pdf URL: https://arxiv.org/pdf/2509.09234
• Copy Paste: [[2509.09234]] Agentic LLMs for Question Answering over Tabular Data(https://arxiv.org/abs/2509.09234)
• Keywords: language model, gpt, llm, agent
• Abstract: Question Answering over Tabular Data (Table QA) presents unique challenges due to the diverse structure, size, and data types of real-world tables. The SemEval 2025 Task 8 (DataBench) introduced a benchmark composed of large-scale, domain-diverse datasets to evaluate the ability of models to accurately answer structured queries. We propose a Natural Language to SQL (NL-to-SQL) approach leveraging large language models (LLMs) such as GPT-4o, GPT-4o-mini, and DeepSeek v2:16b to generate SQL queries dynamically. Our system follows a multi-stage pipeline involving example selection, SQL query generation, answer extraction, verification, and iterative refinement. Experiments demonstrate the effectiveness of our approach, achieving 70.5\% accuracy on DataBench QA and 71.6\% on DataBench Lite QA, significantly surpassing baseline scores of 26\% and 27\% respectively. This paper details our methodology, experimental results, and alternative approaches, providing insights into the strengths and limitations of LLM-driven Table QA.
• 摘要：对表格数据（表QA）的问题回答由于现实表的各种结构，大小和数据类型而提出了独特的挑战。 Semeval 2025 Task 8（Databench）引入了一个由大规模的，域多样性数据集组成的基准，以评估模型准确回答结构化查询的能力。我们向SQL（NL-TO-SQL）提出了一种自然语言，以利用大型语言模型（LLMS），例如GPT-4O，GPT-4O-Mini和DeepSeek V2：16B动态生成SQL查询。我们的系统遵循涉及示例选择，SQL查询生成，回答提取，验证和迭代精炼的多阶段管道。实验证明了我们方法的有效性，在数据级质量固定质量质量固定方面达到了70.5％的准确性和71.6 \％的数据，分别超过了基线得分26 \％和27 \％。本文详细介绍了我们的方法论，实验结果和替代方法，从而提供了有关LLM驱动的表质量质量质量QA的优势和局限性的见解。

Title: From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models

• Authors: Grazia Sveva Ascione, Nicolò Tamagnone
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09303
• Pdf URL: https://arxiv.org/pdf/2509.09303
• Copy Paste: [[2509.09303]] From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models(https://arxiv.org/abs/2509.09303)
• Keywords: language model, llm
• Abstract: Classifying patents by their relevance to the UN Sustainable Development Goals (SDGs) is crucial for tracking how innovation addresses global challenges. However, the absence of a large, labeled dataset limits the use of supervised learning. Existing methods, such as keyword searches, transfer learning, and citation-based heuristics, lack scalability and generalizability. This paper frames patent-to-SDG classification as a weak supervision problem, using citations from patents to SDG-tagged scientific publications (NPL citations) as a noisy initial signal. To address its sparsity and noise, we develop a composite labeling function (LF) that uses large language models (LLMs) to extract structured concepts, namely functions, solutions, and applications, from patents and SDG papers based on a patent ontology. Cross-domain similarity scores are computed and combined using a rank-based retrieval approach. The LF is calibrated via a custom positive-only loss that aligns with known NPL-SDG links without penalizing discovery of new SDG associations. The result is a silver-standard, soft multi-label dataset mapping patents to SDGs, enabling the training of effective multi-label regression models. We validate our approach through two complementary strategies: (1) internal validation against held-out NPL-based labels, where our method outperforms several baselines including transformer-based models, and zero-shot LLM; and (2) external validation using network modularity in patent citation, co-inventor, and co-applicant graphs, where our labels reveal greater thematic, cognitive, and organizational coherence than traditional technological classifications. These results show that weak supervision and semantic alignment can enhance SDG classification at scale.
• 摘要：通过与联合国可持续发展目标（SDG）相关的专利对专利进行分类对于跟踪创新如何应对全球挑战至关重要。但是，缺乏大型，标签的数据集限制了监督学习的使用。现有方法，例如关键字搜索，转移学习和基于引用的启发式方法，缺乏可伸缩性和可推广性。本文使用从专利到SDG标记的科学出版物（NPL引用）作为嘈杂的初始信号的引用，将专利至SDG的分类视为弱监督问题。为了解决其稀疏性和噪音，我们开发了一种复合标签函数（LF），该功能（LF）使用基于专利本体的专利和可持续发展论文提取结构化概念，即功能，解决方案和应用。使用基于等级的检索方法计算和合并跨域相似性得分。 LF通过自定义的正差损失进行校准，该损失与已知的NPL-SDG链接一致，而不会惩罚发现新的SDG关联。结果是银色标准，软标签的数据集映射专利，以实现有效的多标签回归模型的培训。我们通过两种互补策略来验证我们的方法：（1）针对基于NPL的标签的内部验证，我们的方法在其中优于包括基于变压器的模型在内的几个基准和零摄影LLM； （2）使用网络模块化在专利引用，共同发动机和共同应用图中的外部验证，我们的标签比传统的技术分类揭示了更大的主题，认知和组织连贯性。这些结果表明，弱的监督和语义一致性可以大规模增强可持续发展目标的分类。

Title: MetaRAG: Metamorphic Testing for Hallucination Detection in RAG Systems

• Authors: Channdeth Sok, David Luz, Yacine Haddam
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09360
• Pdf URL: https://arxiv.org/pdf/2509.09360
• Copy Paste: [[2509.09360]] MetaRAG: Metamorphic Testing for Hallucination Detection in RAG Systems(https://arxiv.org/abs/2509.09360)
• Keywords: language model, gpt, llm, hallucination, retrieval-augmented generation, agent
• Abstract: Large Language Models (LLMs) are increasingly deployed in enterprise applications, yet their reliability remains limited by hallucinations, i.e., confident but factually incorrect information. Existing detection approaches, such as SelfCheckGPT and MetaQA, primarily target standalone LLMs and do not address the unique challenges of Retrieval-Augmented Generation (RAG) systems, where responses must be consistent with retrieved evidence. We therefore present MetaRAG, a metamorphic testing framework for hallucination detection in Retrieval-Augmented Generation (RAG) systems. MetaRAG operates in a real-time, unsupervised, black-box setting, requiring neither ground-truth references nor access to model internals, making it suitable for proprietary and high-stakes domains. The framework proceeds in four stages: (1) decompose answers into atomic factoids, (2) generate controlled mutations of each factoid using synonym and antonym substitutions, (3) verify each variant against the retrieved context (synonyms are expected to be entailed and antonyms contradicted), and (4) aggregate penalties for inconsistencies into a response-level hallucination score. Crucially for identity-aware AI, MetaRAG localizes unsupported claims at the factoid span where they occur (e.g., pregnancy-specific precautions, LGBTQ+ refugee rights, or labor eligibility), allowing users to see flagged spans and enabling system designers to configure thresholds and guardrails for identity-sensitive queries. Experiments on a proprietary enterprise dataset illustrate the effectiveness of MetaRAG for detecting hallucinations and enabling trustworthy deployment of RAG-based conversational agents. We also outline a topic-based deployment design that translates MetaRAG's span-level scores into identity-aware safeguards; this design is discussed but not evaluated in our experiments.
• 摘要：大型语言模型（LLM）越来越多地部署在企业应用程序中，但其可靠性仍然受幻觉的限制，即自信但事实上不正确的信息。现有的检测方法，例如自我检查和元数据，主要针对独立的LLM，并且没有应对检索效果生成（RAG）系统的独特挑战，在这种挑战中，响应必须与检索的证据一致。因此，我们介绍了Metarag，这是一种用于检索功能（RAG）系统中幻觉检测的变质测试框架。 MetArag以实时的，无监督的黑盒设置进行操作，不需要地面真相引用，也不需要对模型内部设备的访问，使其适合专有和高风险域。该框架在四个阶段进行：（1）将答案分解为原子事实，（2）使用同义词和反义词替代品生成每个事实的受控突变，（3）验证每个变体针对检索到检索的上下文（预计同义词，预计会导致矛盾和反矛盾），以及（4）共同的惩罚范围，以使响应响应构成响应的范围。 Crucially for identity-aware AI, MetaRAG localizes unsupported claims at the factoid span where they occur (e.g., pregnancy-specific precautions, LGBTQ+ refugee rights, or labor eligibility), allowing users to see flagged spans and enabling system designers to configure thresholds and guardrails for identity-sensitive queries.对专有企业数据集进行的实验说明了Metarag在检测幻觉和实现基于抹布的对话剂的可信赖部署方面的有效性。我们还概述了基于主题的部署设计，该设计将Metarag的跨度分数转化为身份感知的保障措施。在我们的实验中讨论了这种设计，但未评估。

Title: GrACE: A Generative Approach to Better Confidence Elicitation in Large Language Models

• Authors: Zhaohan Zhang, Ziquan Liu, Ioannis Patras
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09438
• Pdf URL: https://arxiv.org/pdf/2509.09438
• Copy Paste: [[2509.09438]] GrACE: A Generative Approach to Better Confidence Elicitation in Large Language Models(https://arxiv.org/abs/2509.09438)
• Keywords: language model, llm
• Abstract: Assessing the reliability of Large Language Models (LLMs) by confidence elicitation is a prominent approach to AI safety in high-stakes applications, such as healthcare and finance. Existing methods either require expensive computational overhead or suffer from poor calibration, making them impractical and unreliable for real-world deployment. In this work, we propose GrACE, a Generative Approach to Confidence Elicitation that enables scalable and reliable confidence elicitation for LLMs. GrACE adopts a novel mechanism in which the model expresses confidence by the similarity between the last hidden state and the embedding of a special token appended to the vocabulary, in real-time. We fine-tune the model for calibrating the confidence with calibration targets associated with accuracy. Experiments with three LLMs and two benchmark datasets show that the confidence produced by GrACE achieves the best discriminative capacity and calibration on open-ended generation tasks, outperforming six competing methods without resorting to additional sampling or an auxiliary model. Moreover, we propose two strategies for improving test-time scaling based on confidence induced by GrACE. Experimental results show that using GrACE not only improves the accuracy of the final decision but also significantly reduces the number of required samples in the test-time scaling scheme, indicating the potential of GrACE as a practical solution for deploying LLMs with scalable, reliable, and real-time confidence estimation.
• 摘要：通过信心启发评估大语言模型（LLM）的可靠性是在高风险应用中（例如医疗保健和金融）中对AI安全的重要方法。现有的方法要么需要昂贵的计算开销，要么需要校准较差，这使得它们不切实际，并且对现实部署不可靠。在这项工作中，我们提出了Grace，这是一种产生的信心启发方法，可以为LLMS提供可扩展可靠的信心启发。恩典采用了一种新颖的机制，在这种机制中，模型通过最后一个隐藏状态与嵌入词汇的特殊令牌的嵌入之间的相似性来表达信心。我们将模型微调以使用与精度相关的校准目标校准置信度。使用三个LLM和两个基准数据集进行的实验表明，GRACE产生的置信度实现了开放式生成任务的最佳判别能力和校准，优于六种竞争方法，而无需诉诸其他采样或辅助模型。此外，我们提出了两种策略，以根据恩典引起的置信度提高测试时间缩放。实验结果表明，使用GRACE不仅提高了最终决定的准确性，而且还可以显着减少测试时间缩放方案中所需样本的数量，这表明宽限期作为用可扩展，可靠和实时置信度估计的LLM的实用解决方案的潜力。

Title: DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning

• Authors: Daniil Ignatev, Nan Li, Hugh Mee Wong, Anh Dang, Shane Kaszefski Yaschuk
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.09524
• Pdf URL: https://arxiv.org/pdf/2509.09524
• Copy Paste: [[2509.09524]] DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning(https://arxiv.org/abs/2509.09524)
• Keywords: language model
• Abstract: This system paper presents the DeMeVa team's approaches to the third edition of the Learning with Disagreements shared task (LeWiDi 2025; Leonardelli et al., 2025). We explore two directions: in-context learning (ICL) with large language models, where we compare example sampling strategies; and label distribution learning (LDL) methods with RoBERTa (Liu et al., 2019b), where we evaluate several fine-tuning methods. Our contributions are twofold: (1) we show that ICL can effectively predict annotator-specific annotations (perspectivist annotations), and that aggregating these predictions into soft labels yields competitive performance; and (2) we argue that LDL methods are promising for soft label predictions and merit further exploration by the perspectivist community.
• 摘要：该系统论文介绍了Demeva团队对分歧共享任务的第三版学习方法（Lewidi 2025； Leonardelli等，2025）。我们探讨了两个方向：具有大语言模型的文章学习（ICL），我们在其中比较了示例抽样策略；与Roberta（Liu等，2019b）的标签分布学习（LDL）方法，我们评估了几种微调方法。我们的贡献是双重的：（1）我们表明，ICL可以有效预测注释特定的注释（透视主义注释），并且将这些预测汇总到软标签中会产生竞争性能； （2）我们认为，LDL方法有望获得软标签的预测，并值得通过视角社区进一步探索。

Title: Prompting the Market? A Large-Scale Meta-Analysis of GenAI in Finance NLP (2022-2025)

• Authors: Paolo Pedinotti, Peter Baumann, Nathan Jessurun, Leslie Barrett, Enrico Santus
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09544
• Pdf URL: https://arxiv.org/pdf/2509.09544
• Copy Paste: [[2509.09544]] Prompting the Market? A Large-Scale Meta-Analysis of GenAI in Finance NLP (2022-2025)(https://arxiv.org/abs/2509.09544)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) have rapidly reshaped financial NLP, enabling new tasks and driving a proliferation of datasets and diversification of data sources. Yet, this transformation has outpaced traditional surveys. In this paper, we present MetaGraph, a generalizable methodology for extracting knowledge graphs from scientific literature and analyzing them to obtain a structured, queryable view of research trends. We define an ontology for financial NLP research and apply an LLM-based extraction pipeline to 681 papers (2022-2025), enabling large-scale, data-driven analysis. MetaGraph reveals three key phases: early LLM adoption and task/dataset innovation; critical reflection on LLM limitations; and growing integration of peripheral techniques into modular systems. This structured view offers both practitioners and researchers a clear understanding of how financial NLP has evolved - highlighting emerging trends, shifting priorities, and methodological shifts-while also demonstrating a reusable approach for mapping scientific progress in other domains.
• 摘要：大型语言模型（LLM）具有快速重塑的财务NLP，实现了新任务并推动数据集的扩散和数据源多样化。然而，这种转变超过了传统调查。在本文中，我们介绍了Metagraph，这是一种从科学文献中提取知识图的可推广方法，并分析它们以获得研究趋势的结构化，可查询的观点。我们为金融NLP研究定义了一个本体，并将基于LLM的提取管道应用于681篇论文（2022-2025），从而实现了大规模的数据驱动分析。 Metagraph揭示了三个关键阶段：早期采用和任务/数据集创新；对LLM限制的批判性反映；并将外围技术整合到模块化系统中。这种结构化的观点使从业者和研究人员都清楚地了解了金融NLP的发展方式 - 突出了新兴趋势，转移的优先级和方法论上的转变 - 同时也证明了可重复使用的方法来映射其他领域的科学进步。

Title: Personality-Enhanced Social Recommendations in SAMI: Exploring the Role of Personality Detection in Matchmaking

• Authors: Brittany Harbison, Samuel Taubman, Travis Taylor, Ashok. K. Goel
• Subjects: cs.CL, cs.CY, cs.HC, cs.LG, cs.SI
• Abstract URL: https://arxiv.org/abs/2509.09583
• Pdf URL: https://arxiv.org/pdf/2509.09583
• Copy Paste: [[2509.09583]] Personality-Enhanced Social Recommendations in SAMI: Exploring the Role of Personality Detection in Matchmaking(https://arxiv.org/abs/2509.09583)
• Keywords: gpt
• Abstract: Social connection is a vital part of learning, yet online course environments present barriers to the organic formation of social groups. SAMI offers one solution by facilitating student connections, but its effectiveness is constrained by an incomplete Theory of Mind, limiting its ability to create an effective mental model of a student. One facet of this is its inability to intuit personality, which may influence the relevance of its recommendations. To explore this, we propose a personality detection model utilizing GPTs zero-shot capability to infer Big-Five personality traits from forum introduction posts, often encouraged in online courses. We benchmark its performance against established models, demonstrating its efficacy in this task. Furthermore, we integrate this model into SAMIs entity-based matchmaking system, enabling personality-informed social recommendations. Initial integration suggests personality traits can complement existing matching factors, though additional evaluation is required to determine their full impact on student engagement and match quality.
• 摘要：社会联系是学习的重要组成部分，但是在线课程环境为社会群体的有机形成带来了障碍。萨米（Sami）通过促进学生的联系提供了一种解决方案，但其有效性受到不完整的心理理论的限制，从而限制了其创建有效的学生心理模型的能力。其中一个方面是它无法直觉性格，这可能会影响其建议的相关性。为了探讨这一点，我们提出了一种使用GPT零射击能力的人格检测模型，从论坛介绍帖子中推断出五大的个性特征，这通常是在在线课程中受到鼓励的。我们针对既定模型进行了基准测试，并证明了其在这项任务中的功效。此外，我们将该模型集成到基于SAMIS实体的对接系统中，从而实现了个性化的社交建议。最初的整合表明人格特质可以补充现有的匹配因素，尽管需要进行其他评估以确定其对学生参与度和匹配质量的全部影响。

Title: Fluent but Unfeeling: The Emotional Blind Spots of Language Models

• Authors: Bangzhao Shu, Isha Joshi, Melissa Karnaze, Anh C. Pham, Ishita Kakkar, Sindhu Kothe, Arpine Hovasapian, Mai ElSherief
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2509.09593
• Pdf URL: https://arxiv.org/pdf/2509.09593
• Copy Paste: [[2509.09593]] Fluent but Unfeeling: The Emotional Blind Spots of Language Models(https://arxiv.org/abs/2509.09593)
• Keywords: language model, llm, prompt
• Abstract: The versatility of Large Language Models (LLMs) in natural language understanding has made them increasingly popular in mental health research. While many studies explore LLMs' capabilities in emotion recognition, a critical gap remains in evaluating whether LLMs align with human emotions at a fine-grained level. Existing research typically focuses on classifying emotions into predefined, limited categories, overlooking more nuanced expressions. To address this gap, we introduce EXPRESS, a benchmark dataset curated from Reddit communities featuring 251 fine-grained, self-disclosed emotion labels. Our comprehensive evaluation framework examines predicted emotion terms and decomposes them into eight basic emotions using established emotion theories, enabling a fine-grained comparison. Systematic testing of prevalent LLMs under various prompt settings reveals that accurately predicting emotions that align with human self-disclosed emotions remains challenging. Qualitative analysis further shows that while certain LLMs generate emotion terms consistent with established emotion theories and definitions, they sometimes fail to capture contextual cues as effectively as human self-disclosures. These findings highlight the limitations of LLMs in fine-grained emotion alignment and offer insights for future research aimed at enhancing their contextual understanding.
• 摘要：大型语言模型（LLM）在自然语言理解中的多功能性使他们在心理健康研究中越来越受欢迎。尽管许多研究探讨了LLMS在情感识别方面的能力，但在评估LLMS是否与人类情绪保持在细粒度水平上的关键差距仍然存在。现有的研究通常着重于将情绪分类为预定义的有限类别，忽略了更细微的表情。为了解决这一差距，我们介绍了Express，这是一个由Reddit社区策划的基准数据集，该数据集具有251个精细的，自我披露的情感标签。我们的全面评估框架检查了预测的情绪术语，并使用既定的情感理论将它们分解为八种基本情绪，从而实现了细粒度的比较。在各种及时设置下对普遍的LLM进行系统的测试表明，准确预测与人类自我披露的情绪保持一致的情绪仍然具有挑战性。定性分析进一步表明，尽管某些LLM会产生与已建立的情感理论和定义一致的情感术语，但它们有时无法像人类的自我挑战一样有效地捕获上下文提示。这些发现突出了LLM在细粒度的情感结盟中的局限性，并为未来的研究提供了旨在增强其上下文理解的见解。

Title: LAVA: Language Model Assisted Verbal Autopsy for Cause-of-Death Determination

• Authors: Yiqun T. Chen, Tyler H. McCormick, Li Liu, Abhirup Datta
• Subjects: cs.CL, stat.AP
• Abstract URL: https://arxiv.org/abs/2509.09602
• Pdf URL: https://arxiv.org/pdf/2509.09602
• Copy Paste: [[2509.09602]] LAVA: Language Model Assisted Verbal Autopsy for Cause-of-Death Determination(https://arxiv.org/abs/2509.09602)
• Keywords: language model, gpt, llm
• Abstract: Verbal autopsy (VA) is a critical tool for estimating causes of death in resource-limited settings where medical certification is unavailable. This study presents LA-VA, a proof-of-concept pipeline that combines Large Language Models (LLMs) with traditional algorithmic approaches and embedding-based classification for improved cause-of-death prediction. Using the Population Health Metrics Research Consortium (PHMRC) dataset across three age categories (Adult: 7,580; Child: 1,960; Neonate: 2,438), we evaluate multiple approaches: GPT-5 predictions, LCVA baseline, text embeddings, and meta-learner ensembles. Our results demonstrate that GPT-5 achieves the highest individual performance with average test site accuracies of 48.6% (Adult), 50.5% (Child), and 53.5% (Neonate), outperforming traditional statistical machine learning baselines by 5-10%. Our findings suggest that simple off-the-shelf LLM-assisted approaches could substantially improve verbal autopsy accuracy, with important implications for global health surveillance in low-resource settings.
• 摘要：言语尸检（VA）是在无法获得医疗认证的资源有限设置中估算死亡原因的关键工具。这项研究提出了LA-VA，这是一种概念验证管道，将大型语言模型（LLMS）与传统算法方法和基于嵌入的分类结合在一起，以改善死亡原因预测。我们使用三个年龄类别（成人：7,580;儿童：1,960; Neonate：2,438）的人口健康指标研究联盟（PHMRC）数据集，我们评估了多种方法：GPT-5预测，LCVA基线，文本嵌入和Meta-Learner-Learner emembles。我们的结果表明，GPT-5的平均测试现场精确度为48.6％（成人），50.5％（儿童）和53.5％（新生儿），表现最高，表现最高，表现优于传统的统计机器学习基准5-10％。我们的发现表明，简单的现成LLM辅助方法可以大大提高言语尸检准确性，这对在低资源环境中的全球健康监视具有重要意义。

Title: Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems

• Authors: Minghang Zhu, Zhengliang Shi, Zhiwei Xu, Shiguang Wu, Lingjie Wang, Pengjie Ren, Zhaochun Ren, Zhumin Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09629
• Pdf URL: https://arxiv.org/pdf/2509.09629
• Copy Paste: [[2509.09629]] Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems(https://arxiv.org/abs/2509.09629)
• Keywords: language model, llm, agent
• Abstract: The advancement of large language models (LLMs) has enabled the construction of multi-agent systems to solve complex tasks by dividing responsibilities among specialized agents, such as a planning agent for subgoal generation and a grounding agent for executing tool-use actions. Most existing methods typically fine-tune these agents independently, leading to capability gaps among them with poor coordination. To address this, we propose MOAT, a Multi-Agent Joint Alignment Tuning framework that improves agents collaboration through iterative alignment. MOAT alternates between two key stages: (1) Planning Agent Alignment, which optimizes the planning agent to generate subgoal sequences that better guide the grounding agent; and (2) Grounding Agent Improving, which fine-tunes the grounding agent using diverse subgoal-action pairs generated by the agent itself to enhance its generalization capablity. Theoretical analysis proves that MOAT ensures a non-decreasing and progressively convergent training process. Experiments across six benchmarks demonstrate that MOAT outperforms state-of-the-art baselines, achieving average improvements of 3.1% on held-in tasks and 4.4% on held-out tasks.
• 摘要：大型语言模型（LLMS）的进步已使多机构系统的构建通过在专门的代理之间划分责任，例如用于子目标生成的计划代理和执行工具使用操作的基础代理。大多数现有的方法通常独立微调这些试剂，从而导致协调不良的能力差距。为了解决这个问题，我们提出了Moat，这是一个多代理的联合对齐调整框架，可通过迭代对齐来改善代理协作。护城河在两个关键阶段之间进行交替：（1）计划代理对齐，该计划优化计划代理以生成更好指导地面代理的子目标序列； （2）接地剂改进，它使用代理本身生成的各种亚距离行动对微调接地剂，以增强其泛化能力。理论分析证明，护城河可确保不稳定和逐步收敛的训练过程。六个基准的实验表明，护城河的表现要优于最先进的基线，在持有任务上的平均提高为3.1％，而持有任务的平均提高为4.4％。

Title: All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens

• Authors: Siddarth Mamidanna, Daking Rai, Ziyu Yao, Yilun Zhou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2509.09650
• Pdf URL: https://arxiv.org/pdf/2509.09650
• Copy Paste: [[2509.09650]] All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens(https://arxiv.org/abs/2509.09650)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) demonstrate proficiency across numerous computational tasks, yet their inner workings remain unclear. In theory, the combination of causal self-attention and multilayer perceptron layers allows every token to access and compute information based on all preceding tokens. In practice, to what extent are such operations present? In this paper, on mental math tasks (i.e., direct math calculation via next-token prediction without explicit reasoning), we investigate this question in three steps: inhibiting input-specific token computations in the initial layers, restricting the routes of information transfer across token positions in the next few layers, and forcing all computation to happen at the last token in the remaining layers. With two proposed techniques, Context-Aware Mean Ablation (CAMA) and Attention-Based Peeking (ABP), we identify an All-for-One subgraph (AF1) with high accuracy on a wide variety of mental math tasks, where meaningful computation occurs very late (in terms of layer depth) and only at the last token, which receives information of other tokens in few specific middle layers. Experiments on a variety of models and arithmetic expressions show that this subgraph is sufficient and necessary for high model performance, transfers across different models, and works on a variety of input styles. Ablations on different CAMA and ABP alternatives reveal their unique advantages over other methods, which may be of independent interest.
• 摘要：大型语言模型（LLMS）表明了众多计算任务的熟练程度，但它们的内部运作尚不清楚。从理论上讲，因果自我注意力和多层感知层的结合使每个令牌都可以根据所有上述令牌访问和计算信息。实际上，此类操作在多大程度上存在？在本文中，关于心理数学任务（即通过下一步的预测直接数学计算而无需明确推理），我们分三个步骤调查了这个问题：抑制初始层中特定的特定输入令牌计算，限制了在下几层中跨令牌位置的信息传输途径，并迫使所有计算在所有层次上都发生在造成的the Repeken she Repeken inseveken中。通过两种提出的技术，我们在各种各样的心理数学任务上以高准确的态度确定了一个全面的子图（AF1），而有意义的计算很晚（就图层深度的层面深度而言），并且仅在最后一个doken中，我们才能在其他一些层中获得其他特定的特定层的信息。在各种模型和算术表达式上进行的实验表明，该子图足以且需要进行高模型性能，跨不同模型转移，并使用各种输入样式。在不同的CAMA和ABP替代方案上进行消融揭示了它们与其他方法的独特优势，这可能具有独立的兴趣。

Title: Steering MoE LLMs via Expert (De)Activation

• Authors: Mohsen Fayyaz, Ali Modarressi, Hanieh Deilamsalehy, Franck Dernoncourt, Ryan Rossi, Trung Bui, Hinrich Schütze, Nanyun Peng
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.09660
• Pdf URL: https://arxiv.org/pdf/2509.09660
• Copy Paste: [[2509.09660]] Steering MoE LLMs via Expert (De)Activation(https://arxiv.org/abs/2509.09660)
• Keywords: language model, llm
• Abstract: Mixture-of-Experts (MoE) in Large Language Models (LLMs) routes each token through a subset of specialized Feed-Forward Networks (FFN), known as experts. We present SteerMoE, a framework for steering MoE models by detecting and controlling behavior-linked experts. Our detection method identifies experts with distinct activation patterns across paired inputs exhibiting contrasting behaviors. By selectively (de)activating such experts during inference, we control behaviors like faithfulness and safety without retraining or modifying weights. Across 11 benchmarks and 6 LLMs, our steering raises safety by up to +20% and faithfulness by +27%. In adversarial attack mode, it drops safety by -41% alone, and -100% when combined with existing jailbreak methods, bypassing all safety guardrails and exposing a new dimension of alignment faking hidden within experts.
• 摘要：大语模型（LLMS）中的Experts（MOE）的混合物通过专用馈送前馈网络（FFN）的子集（称为专家）。我们提出Steermoe，这是一个通过检测和控制与行为链接的专家来指导MOE模型的框架。我们的检测方法确定了具有对比度行为的成对输入的不同激活模式的专家。通过（DE）在推断期间有选择地激活此类专家，我们可以控制忠实和安全等行为，而无需重新训练或修改权重。在11个基准和6个LLM中，我们的转向提高了安全性高达20％，忠诚提高了 +27％。在对抗攻击模式下，它仅将安全性下降到-41％，而与现有的越狱方法相结合时，它会使安全性下降，绕过所有安全护栏，并揭示了隐藏在专家中的对齐的新维度。

Title: CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models

• Authors: Runpeng Dai, Linfeng Song, Haolin Liu, Zhenwen Liang, Dian Yu, Haitao Mi, Zhaopeng Tu, Rui Liu, Tong Zheng, Hongtu Zhu, Dong Yu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2509.09675
• Pdf URL: https://arxiv.org/pdf/2509.09675
• Copy Paste: [[2509.09675]] CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models(https://arxiv.org/abs/2509.09675)
• Keywords: language model, llm
• Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is a powerful paradigm for enhancing the reasoning ability of Large Language Models (LLMs). Yet current RLVR methods often explore poorly, leading to premature convergence and entropy collapse. To address this challenge, we introduce Curiosity-Driven Exploration (CDE), a framework that leverages the model's own intrinsic sense of curiosity to guide exploration. We formalize curiosity with signals from both the actor and the critic: for the actor, we use perplexity over its generated response, and for the critic, we use the variance of value estimates from a multi-head architecture. Both signals serve as an exploration bonus within the RLVR framework to guide the model. Our theoretical analysis shows that the actor-wise bonus inherently penalizes overconfident errors and promotes diversity among correct responses; moreover, we connect the critic-wise bonus to the well-established count-based exploration bonus in RL. Empirically, our method achieves an approximate +3 point improvement over standard RLVR using GRPO/PPO on AIME benchmarks. Further analysis identifies a calibration collapse mechanism within RLVR, shedding light on common LLM failure modes.
• 摘要：通过可验证的奖励（RLVR）的增强学习是增强大语模型（LLMS）的推理能力的强大范式。然而，当前的RLVR方法通常探索很差，导致过早收敛和熵崩溃。为了应对这一挑战，我们引入了好奇心驱动的探索（CDE），该框架利用模型自身的好奇心来指导探索。我们对演员和评论家的信号正式使好奇心形式化：对于演员而言，我们对其产生的响应进行了困惑，对于评论家，我们使用了来自多头体系结构的价值估计的差异。这两个信号都是RLVR框架中的探索奖金，以指导模型。我们的理论分析表明，演员的奖励固有地惩罚过度自信的错误，并促进了正确的回答之间的多样性。此外，我们将评论家的奖励与RL中基于计数的探索奖金联系起来。从经验上讲，我们的方法在AIME基准上使用GRPO/PPO实现了与标准RLVR相比的近似+3点。进一步分析确定了RLVR内的校准塌陷机制，从而阐明了公共LLM失败模式。