2025-11-11

Title: Ming-UniAudio: Speech LLM for Joint Understanding, Generation and Editing with Unified Representation

• Authors: Canxiang Yan, Chunxiang Jin, Dawei Huang, Haibing Yu, Han Peng, Hui Zhan, Jie Gao, Jing Peng, Jingdong Chen, Jun Zhou, Kaimeng Ren, Ming Yang, Mingxue Yang, Qiang Xu, Qin Zhao, Ruijie Xiong, Shaoxiong Lin, Xuezhi Wang, Yi Yuan, Yifei Wu, Yongjie Lyu, Zhengyu He, Zhihao Qiu, Zhiqiang Fang, Ziyuan Huang
• Subjects: cs.CL, cs.AI, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2511.05516
• Pdf URL: https://arxiv.org/pdf/2511.05516
• Copy Paste: [[2511.05516]] Ming-UniAudio: Speech LLM for Joint Understanding, Generation and Editing with Unified Representation(https://arxiv.org/abs/2511.05516)
• Keywords: language model, llm
• Abstract: Existing speech models suffer from competing requirements on token representations by understanding and generation tasks. This discrepancy in representation prevents speech language models from performing instruction-based free-form editing. To solve this challenge, we introduce a novel framework that unifies speech understanding, generation, and editing. The core of our unified model is a unified continuous speech tokenizer MingTok-Audio, the first continuous tokenizer to effectively integrate semantic and acoustic features, which makes it suitable for both understanding and generation tasks. Based on this unified continuous audio tokenizer, we developed the speech language model Ming-UniAudio, which achieved a balance between generation and understanding capabilities. Ming-UniAudio sets new state-of-the-art (SOTA) records on 8 out of 12 metrics on the ContextASR benchmark. Notably, for Chinese voice cloning, it achieves a highly competitive Seed-TTS-WER of 0.95. Leveraging this foundational model, we further trained a dedicated speech editing model Ming-UniAudio-Edit, the first speech language model that enables universal, free-form speech editing guided solely by natural language instructions, handling both semantic and acoustic modifications without timestamp condition. To rigorously assess the editing capability and establish a foundation for future research, we introduce Ming-Freeform-Audio-Edit, the first comprehensive benchmark tailored for instruction-based free-form speech editing, featuring diverse scenarios and evaluation dimensions spanning semantic correctness, acoustic quality, and instruction alignment. We open-sourced the continuous audio tokenizer, the unified foundational model, and the free-form instruction-based editing model to facilitate the development of unified audio understanding, generation, and manipulation.
• 摘要：现有的语音模型面临着理解和生成任务对令牌表示的竞争要求。这种表示上的差异阻止了语音语言模型执行基于指令的自由形式编辑。为了解决这一挑战，我们引入了一个统一语音理解、生成和编辑的新颖框架。我们统一模型的核心是统一的连续语音标记器 MingTok-Audio，这是第一个有效集成语义和声学特征的连续标记器，这使得它适用于理解和生成任务。基于这个统一的连续音频分词器，我们开发了语音语言模型 Ming-Uni​​Audio，实现了生成能力和理解能力之间的平衡。 Ming-Uni​​Audio 在 ContextASR 基准测试的 12 个指标中的 8 个指标上创下了新的最先进 (SOTA) 记录。值得注意的是，对于中文语音克隆，它实现了极具竞争力的Seed-TTS-WER 0.95。利用这个基础模型，我们进一步训练了专用语音编辑模型 Ming-Uni​​Audio-Edit，这是第一个语音语言模型，能够仅由自然语言指令引导进行通用、自由形式的语音编辑，无需时间戳条件即可处理语义和声学修改。为了严格评估编辑能力并为未来的研究奠定基础，我们推出了Ming-Freeform-Audio-Edit，这是第一个为基于指令的自由格式语音编辑量身定制的综合基准测试，具有涵盖语义正确性、音质和指令对齐等多种场景和评估维度。我们开源了连续音频分词器、统一的基础模型和自由格式的基于指令的编辑模型，以促进统一音频理解、生成和操作的开发。

Title: Retracing the Past: LLMs Emit Training Data When They Get Lost

• Authors: Myeongseob Ko, Nikhil Reddy Billa, Adam Nguyen, Charles Fleming, Ming Jin, Ruoxi Jia
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.05518
• Pdf URL: https://arxiv.org/pdf/2511.05518
• Copy Paste: [[2511.05518]] Retracing the Past: LLMs Emit Training Data When They Get Lost(https://arxiv.org/abs/2511.05518)
• Keywords: language model, llm
• Abstract: The memorization of training data in large language models (LLMs) poses significant privacy and copyright concerns. Existing data extraction methods, particularly heuristic-based divergence attacks, often exhibit limited success and offer limited insight into the fundamental drivers of memorization leakage. This paper introduces Confusion-Inducing Attacks (CIA), a principled framework for extracting memorized data by systematically maximizing model uncertainty. We empirically demonstrate that the emission of memorized text during divergence is preceded by a sustained spike in token-level prediction entropy. CIA leverages this insight by optimizing input snippets to deliberately induce this consecutive high-entropy state. For aligned LLMs, we further propose Mismatched Supervised Fine-tuning (SFT) to simultaneously weaken their alignment and induce targeted confusion, thereby increasing susceptibility to our attacks. Experiments on various unaligned and aligned LLMs demonstrate that our proposed attacks outperform existing baselines in extracting verbatim and near-verbatim training data without requiring prior knowledge of the training data. Our findings highlight persistent memorization risks across various LLMs and offer a more systematic method for assessing these vulnerabilities.
• 摘要：大语言模型 (LLM) 中训练数据的记忆会带来严重的隐私和版权问题。现有的数据提取方法，特别是基于启发式的发散攻击，通常表现出有限的成功，并且对记忆泄漏的基本驱动因素的洞察力有限。本文介绍了混淆诱导攻击（CIA），这是一种通过系统地最大化模型不确定性来提取记忆数据的原则框架。我们凭经验证明，在发散期间记忆文本的发射之前是令牌级预测熵的持续峰值。 CIA 通过优化输入片段来利用这种洞察力，故意诱导这种连续的高熵状态。对于对齐的法学硕士，我们进一步提出不匹配监督微调（SFT），以同时削弱它们的对齐并引起有针对性的混乱，从而增加对我们攻击的敏感性。对各种未对齐和对齐 LLM 的实验表明，我们提出的攻击在提取逐字和近逐字训练数据方面优于现有基线，而无需事先了解训练数据。我们的研究结果强调了各种法学硕士的持续记忆风险，并提供了一种更系统的方法来评估这些漏洞。

Title: Beyond One-Size-Fits-All: Personalized Harmful Content Detection with In-Context Learning

• Authors: Rufan Zhang, Lin Zhang, Xianghang Mi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.05532
• Pdf URL: https://arxiv.org/pdf/2511.05532
• Copy Paste: [[2511.05532]] Beyond One-Size-Fits-All: Personalized Harmful Content Detection with In-Context Learning(https://arxiv.org/abs/2511.05532)
• Keywords: prompt
• Abstract: The proliferation of harmful online content--e.g., toxicity, spam, and negative sentiment--demands robust and adaptable moderation systems. However, prevailing moderation systems are centralized and task-specific, offering limited transparency and neglecting diverse user preferences--an approach ill-suited for privacy-sensitive or decentralized environments. We propose a novel framework that leverages in-context learning (ICL) with foundation models to unify the detection of toxicity, spam, and negative sentiment across binary, multi-class, and multi-label settings. Crucially, our approach enables lightweight personalization, allowing users to easily block new categories, unblock existing ones, or extend detection to semantic variations through simple prompt-based interventions--all without model retraining. Extensive experiments on public benchmarks (TextDetox, UCI SMS, SST2) and a new, annotated Mastodon dataset reveal that: (i) foundation models achieve strong cross-task generalization, often matching or surpassing task-specific fine-tuned models; (ii) effective personalization is achievable with as few as one user-provided example or definition; and (iii) augmenting prompts with label definitions or rationales significantly enhances robustness to noisy, real-world data. Our work demonstrates a definitive shift beyond one-size-fits-all moderation, establishing ICL as a practical, privacy-preserving, and highly adaptable pathway for the next generation of user-centric content safety systems. To foster reproducibility and facilitate future research, we publicly release our code on GitHub and the annotated Mastodon dataset on Hugging Face.
• 摘要：有害在线内容（例如有毒内容、垃圾邮件和负面情绪）的扩散需要强大且适应性强的审核系统。然而，流行的审核系统是中心化的、特定于任务的，提供的透明度有限，并且忽略了不同的用户偏好——这种方法不适合隐私敏感或去中心化的环境。我们提出了一种新颖的框架，利用上下文学习（ICL）和基础模型来统一跨二进制、多类和多标签设置的毒性、垃圾邮件和负面情绪的检测。至关重要的是，我们的方法实现了轻量级个性化，允许用户轻松阻止新类别、解锁现有类别，或通过简单的基于提示的干预将检测扩展到语义变化 - 所有这些都无需模型重新训练。对公共基准（TextDetox、UCI SMS、SST2）和新的带注释 Mastodon 数据集的广泛实验表明：（i）基础模型实现了强大的跨任务泛化，通常匹配或超越特定于任务的微调模型； (ii) 只需用户提供一个示例或定义即可实现有效的个性化； (iii) 通过标签定义或基本原理来增强提示，可以显着增强对嘈杂的现实世界数据的鲁棒性。我们的工作展示了超越一刀切的审核的明确转变，将 ICL 确立为下一代以用户为中心的内容安全系统的实用、隐私保护和高度适应性的途径。为了提高可重复性并促进未来的研究，我们在 GitHub 上公开发布我们的代码，并在 Hugging Face 上公开发布带注释的 Mastodon 数据集。

Title: MCP4IFC: IFC-Based Building Design Using Large Language Models

• Authors: Bharathi Kannan Nithyanantham, Tobias Sesterhenn, Ashwin Nedungadi, Sergio Peral Garijo, Janis Zenkner, Christian Bartelt, Stefan Lüdtke
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.05533
• Pdf URL: https://arxiv.org/pdf/2511.05533
• Copy Paste: [[2511.05533]] MCP4IFC: IFC-Based Building Design Using Large Language Models(https://arxiv.org/abs/2511.05533)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Bringing generative AI into the architecture, engineering and construction (AEC) field requires systems that can translate natural language instructions into actions on standardized data models. We present MCP4IFC, a comprehensive open-source framework that enables Large Language Models (LLMs) to directly manipulate Industry Foundation Classes (IFC) data through the Model Context Protocol (MCP). The framework provides a set of BIM tools, including scene querying tools for information retrieval, predefined functions for creating and modifying common building elements, and a dynamic code-generation system that combines in-context learning with retrieval-augmented generation (RAG) to handle tasks beyond the predefined toolset. Experiments demonstrate that an LLM using our framework can successfully perform complex tasks, from building a simple house to querying and editing existing IFC data. Our framework is released as open-source to encourage research in LLM-driven BIM design and provide a foundation for AI-assisted modeling workflows. Our code is available at this https URL.
• 摘要：将生成式人工智能引入建筑、工程和施工 (AEC) 领域需要能够将自然语言指令转换为标准化数据模型上的操作的系统。我们推出了 MCP4IFC，这是一个全面的开源框架，使大型语言模型 (LLM) 能够通过模型上下文协议 (MCP) 直接操作行业基础类 (IFC) 数据。该框架提供了一组 BIM 工具，包括用于信息检索的场景查询工具、用于创建和修改常见建筑元素的预定义函数，以及将上下文学习与检索增强生成 (RAG) 相结合以处理预定义工具集之外的任务的动态代码生成系统。实验表明，使用我们框架的法学硕士可以成功执行复杂的任务，从建造简单的房屋到查询和编辑现有的 IFC 数据。我们的框架以开源方式发布，以鼓励法学硕士驱动的 BIM 设计研究，并为人工智能辅助建模工作流程提供基础。我们的代码可以在这个 https URL 上找到。

Title: FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference

• Authors: Kunxi Li, Yufan Xiong, Zhonghua Jiang, Yiyun Zhou, Zhaode Wang, Chengfei Lv, Shengyu Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.05534
• Pdf URL: https://arxiv.org/pdf/2511.05534
• Copy Paste: [[2511.05534]] FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference(https://arxiv.org/abs/2511.05534)
• Keywords: llm, hallucination
• Abstract: Traditional KV cache eviction strategies, which discard less critical KV-pairs based on attention scores, often degrade generation quality, causing context loss or hallucinations. Recent efforts shift toward KV merging, merging eviction tokens with retention tokens based on similarity. However, in multimodal scenarios, distributional biases across modality tokens and attentional biases in cross-modal interactions limit its effectiveness. This work introduces FlowMM, an adaptive framework for cross-modal information flow-guided multimodal KV cache merging. FlowMM leverages cross-modal information flow to dynamically apply layer-specific merging strategies, capturing modality-specific patterns while preserving contextual integrity. Furthermore, we introduce a sensitivity-adaptive token matching mechanism that jointly evaluates token similarity and task-critical sensitivity, merging low-risk tokens while safeguarding high-sensitivity ones. Extensive experiments across diverse leading MLLMs show that FlowMM reduces KV cache memory by 80% to 95% and decoding latency by 1.3-1.8x, while maintaining competitive task performance.
• 摘要：传统的 KV 缓存驱逐策略根据注意力分数丢弃不太关键的 KV 对，通常会降低生成质量，导致上下文丢失或幻觉。最近的工作转向 KV 合并，即基于相似性将驱逐令牌与保留令牌合并。然而，在多模态场景中，模态标记之间的分布偏差和跨模态交互中的注意力偏差限制了其有效性。这项工作介绍了 FlowMM，这是一种跨模态信息流引导的多模态 KV 缓存合并的自适应框架。 FlowMM 利用跨模态信息流动态应用特定于层的合并策略，捕获特定于模态的模式，同时保持上下文完整性。此外，我们引入了一种敏感性自适应令牌匹配机制，该机制联合评估令牌相似性和关键任务敏感性，合并低风险令牌，同时保护高敏感性令牌。跨不同领先 MLLM 的大量实验表明，FlowMM 将 KV 缓存内存减少了 80% 至 95%，并将解码延迟减少了 1.3-1.8 倍，同时保持了有竞争力的任务性能。

Title: Future of AI Models: A Computational perspective on Model collapse

• Authors: Trivikram Satharasi (1), S Sitharama Iyengar (2) ((1) University of Florida, Gainesville, FL, (2) Florida International University, Miami. FL)
• Subjects: cs.CL, cs.DB, cs.IT
• Abstract URL: https://arxiv.org/abs/2511.05535
• Pdf URL: https://arxiv.org/pdf/2511.05535
• Copy Paste: [[2511.05535]] Future of AI Models: A Computational perspective on Model collapse(https://arxiv.org/abs/2511.05535)
• Keywords: language model, llm
• Abstract: Artificial Intelligence, especially Large Language Models (LLMs), has transformed domains such as software engineering, journalism, creative writing, academia, and media (Naveed et al. 2025; arXiv:2307.06435). Diffusion models like Stable Diffusion generate high-quality images and videos from text. Evidence shows rapid expansion: 74.2% of newly published webpages now contain AI-generated material (Ryan Law 2025), 30-40% of the active web corpus is synthetic (Spennemann 2025; arXiv:2504.08755), 52% of U.S. adults use LLMs for writing, coding, or research (Staff 2025), and audits find AI involvement in 18% of financial complaints and 24% of press releases (Liang et al. 2025). The underlying neural architectures, including Transformers (Vaswani et al. 2023; arXiv:1706.03762), RNNs, LSTMs, GANs, and diffusion networks, depend on large, diverse, human-authored datasets (Shi & Iyengar 2019). As synthetic content dominates, recursive training risks eroding linguistic and semantic diversity, producing Model Collapse (Shumailov et al. 2024; arXiv:2307.15043; Dohmatob et al. 2024; arXiv:2402.07712). This study quantifies and forecasts collapse onset by examining year-wise semantic similarity in English-language Wikipedia (filtered Common Crawl) from 2013 to 2025 using Transformer embeddings and cosine similarity metrics. Results reveal a steady rise in similarity before public LLM adoption, likely driven by early RNN/LSTM translation and text-normalization pipelines, though modest due to a smaller scale. Observed fluctuations reflect irreducible linguistic diversity, variable corpus size across years, finite sampling error, and an exponential rise in similarity after the public adoption of LLM models. These findings provide a data-driven estimate of when recursive AI contamination may significantly threaten data richness and model generalization.
• 摘要：人工智能，尤其是大型语言模型 (LLM)，已经改变了软件工程、新闻、创意写作、学术界和媒体等领域（Naveed 等人，2025 年；arXiv：2307.06435）。稳定扩散等扩散模型可以从文本生成高质量的图像和视频。有证据表明，这一扩张正在迅速扩大：74.2% 的新发布网页现在包含人工智能生成的材料 (Ryan Law 2025)，30-40% 的活跃网络语料库是合成的 (Spennemann 2025；arXiv:2504.08755)，52% 的美国成年人使用法学硕士进行写作、编码或研究 (Staff 2025)，审计发现人工智能参与了以下领域： 18% 的财务投诉和 24% 的新闻稿（Liang et al. 2025）。底层神经架构，包括 Transformer（Vaswani 等人，2023；arXiv：1706.03762）、RNN、LSTM、GAN 和扩散网络，依赖于大型、多样化的人类创作数据集（Shi & Iyengar 2019）。由于合成内容占主导地位，递归训练可能会侵蚀语言和语义多样性，从而导致模型崩溃（Shumailov et al. 2024；arXiv:2307.15043；Dohmatob et al. 2024；arXiv:2402.07712）。本研究通过使用 Transformer 嵌入和余弦相似性指标检查 2013 年至 2025 年英语维基百科（经过过滤的 Common Crawl）中逐年的语义相似性来量化和预测崩溃的发生。结果显示，在公共法学硕士采用之前，相似性稳步上升，这可能是由早期的 RNN/LSTM 翻译和文本规范化管道推动的，尽管由于规模较小，相似性不大。观察到的波动反映了不可减少的语言多样性、多年来可变的语料库大小、有限的抽样误差以及公众采用法学硕士模型后相似性的指数上升。这些发现为递归人工智能污染何时可能严重威胁数据丰富性和模型泛化提供了数据驱动的估计。

Title: Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretability

• Authors: Usha Bhalla, Alex Oesterling, Claudio Mayrink Verdun, Himabindu Lakkaraju, Flavio P. Calmon
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.05541
• Pdf URL: https://arxiv.org/pdf/2511.05541
• Copy Paste: [[2511.05541]] Temporal Sparse Autoencoders: Leveraging the Sequential Nature of Language for Interpretability(https://arxiv.org/abs/2511.05541)
• Keywords: language model, llm
• Abstract: Translating the internal representations and computations of models into concepts that humans can understand is a key goal of interpretability. While recent dictionary learning methods such as Sparse Autoencoders (SAEs) provide a promising route to discover human-interpretable features, they suffer from a variety of problems, including a systematic failure to capture the rich conceptual information that drives linguistic understanding. Instead, they exhibit a bias towards shallow, token-specific, or noisy features, such as "the phrase 'The' at the start of sentences". In this work, we propose that this is due to a fundamental issue with how dictionary learning methods for LLMs are trained. Language itself has a rich, well-studied structure spanning syntax, semantics, and pragmatics; however, current unsupervised methods largely ignore this linguistic knowledge, leading to poor feature discovery that favors superficial patterns over meaningful concepts. We focus on a simple but important aspect of language: semantic content has long-range dependencies and tends to be smooth over a sequence, whereas syntactic information is much more local. Building on this insight, we introduce Temporal Sparse Autoencoders (T-SAEs), which incorporate a novel contrastive loss encouraging consistent activations of high-level features over adjacent tokens. This simple yet powerful modification enables SAEs to disentangle semantic from syntactic features in a self-supervised manner. Across multiple datasets and models, T-SAEs recover smoother, more coherent semantic concepts without sacrificing reconstruction quality. Strikingly, they exhibit clear semantic structure despite being trained without explicit semantic signal, offering a new pathway for unsupervised interpretability in language models.
• 摘要：将模型的内部表示和计算转化为人类可以理解的概念是可解释性的一个关键目标。虽然稀疏自动编码器（SAE）等最近的字典学习方法为发现人类可解释的特征提供了一条有前途的途径，但它们遇到了各种问题，包括无法捕获驱动语言理解的丰富概念信息的系统性失败。相反，它们表现出对浅薄、特定于标记或嘈杂特征的偏见，例如“句子开头的短语‘The’”。在这项工作中，我们认为这是由于法学硕士的字典学习方法如何训练的一个基本问题造成的。语言本身具有丰富的、经过深入研究的结构，涵盖语法、语义和语用学；然而，当前的无监督方法在很大程度上忽略了这种语言知识，导致特征发现不佳，倾向于肤浅的模式而不是有意义的概念。我们关注语言的一个简单但重要的方面：语义内容具有长期依赖性，并且在序列上往往是平滑的，而句法信息则更加本地化。基于这一见解，我们引入了时间稀疏自动编码器（T-SAE），它结合了一种新颖的对比损失，鼓励对相邻标记的高级特征进行一致激活。这种简单而强大的修改使 SAE 能够以自我监督的方式将语义与句法特征分开。在多个数据集和模型中，T-SAE 可以在不牺牲重建质量的情况下恢复更平滑、更连贯的语义概念。引人注目的是，尽管它们在没有明确语义信号的情况下进行训练，但仍表现出清晰的语义结构，为语言模型中的无监督可解释性提供了一条新途径。

Title: Sample-Efficient Language Modeling with Linear Attention and Lightweight Enhancements

• Authors: Patrick Haller, Jonas Golde, Alan Akbik
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.05560
• Pdf URL: https://arxiv.org/pdf/2511.05560
• Copy Paste: [[2511.05560]] Sample-Efficient Language Modeling with Linear Attention and Lightweight Enhancements(https://arxiv.org/abs/2511.05560)
• Keywords: language model
• Abstract: We study architectural and optimization tech- niques for sample-efficient language modeling under the constraints of the BabyLM 2025 shared task. Our model, BLaLM, replaces self-attention with a linear-time mLSTM to- ken mixer and explores lightweight enhance- ments, including short convolutions, sliding window attention with dynamic modulation, and Hedgehog feature maps. To support train- ing in low-resource settings, we curate a high- quality corpus emphasizing readability and ped- agogical structure. Experiments across both STRICT and STRICT-SMALL tracks show that (1) linear attention combined with sliding win- dow attention consistently improves zero-shot performance, and (2) the Muon optimizer stabi- lizes convergence and reduces perplexity over AdamW. These results highlight effective strate- gies for efficient language modeling without relying on scale.
• 摘要：我们研究了 BabyLM 2025 共享任务约束下样本高效语言建模的架构和优化技术。我们的模型 BLaLM 用线性时间 mLSTM 令牌混合器取代了自注意力，并探索了轻量级增强功能，包括短卷积、动态调制的滑动窗口注意力和 Hedgehog 特征图。为了支持资源匮乏环境中的培训，我们策划了一个高质量的语料库，强调可读性和教学结构。 STRICT 和 STRICT-SMALL 轨道上的实验表明，（1）线性注意力与滑动窗口注意力相结合持续提高了零样本性能，（2）Muon 优化器稳定了收敛并减少了 AdamW 的困惑。这些结果强调了不依赖规模的高效语言建模的有效策略。

Title: UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8

• Authors: Preston Firestone, Shubham Ugare, Gagandeep Singh, Sasa Misailovic
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.05578
• Pdf URL: https://arxiv.org/pdf/2511.05578
• Copy Paste: [[2511.05578]] UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8(https://arxiv.org/abs/2511.05578)
• Keywords: language model, llm
• Abstract: Subword tokenization segments input text according to a pre-defined vocabulary to feed it into a language model; the language model, in turn, generates a sequence made from this same vocabulary. The members of the vocabulary can be built of code points or bytes. Using code points means that all members of the vocabulary are valid UTF-8 characters. However, it also requires thousands of initial members to achieve acceptable coverage of inputs. Beginning with bytes, on the contrary, avoids out-of-vocabulary errors with only 256 initial members of the vocabulary, but the members of the vocabulary and sequences of them are not guaranteed to be valid UTF-8. Sequences that are not valid UTF-8 break code that assumes its input to be valid UTF-8. Applications of language models must account for the breakage thereby introduced. In this paper, we formalize tokenization using monoid theory and prove that tokenizers whose vocabularies contain tokens that are ill-formed UTF-8 can always produce sequences that are ill-formed UTF-8. We demonstrate formally that attempting to incrementally convert tokens back to a string and interpret the results as UTF-8 gives different results than converting the whole sequence of tokens at once. This formal result predicts real-world bugs: we evaluate mitigations for the problem identified and provide case studies of major foundation models, serving engines, and constrained generation systems.
• 摘要：子词标记化根据预定义的词汇对输入文本进行分段，以将其输入到语言模型中；反过来，语言模型会生成由相同词汇组成的序列。词汇表的成员可以由代码点或字节构建。使用代码点意味着词汇表的所有成员都是有效的 UTF-8 字符。然而，它还需要数千名初始成员才能实现可接受的投入覆盖范围。相反，以字节开头，仅使用 256 个词汇表初始成员即可避免词汇表外错误，但不能保证词汇表成员及其序列是有效的 UTF-8。不是有效 UTF-8 的序列会破坏假定其输入是有效 UTF-8 的代码。语言模型的应用必须考虑由此带来的破坏。在本文中，我们使用幺半群理论对标记化进行形式化，并证明词汇表包含格式错误的 UTF-8 标记的标记生成器始终可以生成格式错误的 UTF-8 序列。我们正式证明，尝试逐步将令牌转换回字符串并将结果解释为 UTF-8 会得到与立即转换整个令牌序列不同的结果。这个正式结果预测了现实世界的错误：我们评估已识别问题的缓解措施，并提供主要基础模型、服务引擎和约束生成系统的案例研究。

Title: Optimizing Diversity and Quality through Base-Aligned Model Collaboration

• Authors: Yichen Wang, Chenghao Yang, Tenghao Huang, Muhao Chen, Jonathan May, Mina Lee
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.05650
• Pdf URL: https://arxiv.org/pdf/2511.05650
• Copy Paste: [[2511.05650]] Optimizing Diversity and Quality through Base-Aligned Model Collaboration(https://arxiv.org/abs/2511.05650)
• Keywords: language model, llm, prompt
• Abstract: Alignment has greatly improved large language models (LLMs)' output quality at the cost of diversity, yielding highly similar outputs across generations. We propose Base-Aligned Model Collaboration (BACo), an inference-time token-level model collaboration framework that dynamically combines a base LLM with its aligned counterpart to optimize diversity and quality. Inspired by prior work (Fei et al., 2025), BACo employs routing strategies that determine, at each token, from which model to decode based on next-token prediction uncertainty and predicted contents' semantic role. Prior diversity-promoting methods, such as retraining, prompt engineering, and multi-sampling methods, improve diversity but often degrade quality or require costly decoding or post-training. In contrast, BACo achieves both high diversity and quality post hoc within a single pass, while offering strong controllability. We explore a family of routing strategies, across three open-ended generation tasks and 13 metrics covering diversity and quality, BACo consistently surpasses state-of-the-art inference-time baselines. With our best router, BACo achieves a 21.3% joint improvement in diversity and quality. Human evaluations also mirror these improvements. The results suggest that collaboration between base and aligned models can optimize and control diversity and quality.
• 摘要：对齐极大地提高了大型语言模型 (LLM) 的输出质量，但以多样性为代价，在各代之间产生高度相似的输出。我们提出了基础对齐模型协作（BACo），这是一种推理时间令牌级模型协作框架，它动态地将基础 LLM 与其对齐对应项相结合，以优化多样性和质量。受先前工作（Fei 等人，2025）的启发，BACo 采用路由策略，根据下一个令牌预测的不确定性和预测内容的语义角色，在每个令牌上确定从哪个模型进行解码。先前的多样性促进方法，例如再训练、即时工程和多重采样方法，可以提高多样性，但通常会降低质量或需要昂贵的解码或后期训练。相比之下，BACo 在一次传递中实现了高度多样性和事后质量，同时提供了强大的可控性。我们探索了一系列路由策略，涵盖三个开放式生成任务和涵盖多样性和质量的 13 个指标，BACo 始终超越最先进的推理时间基线。凭借我们最好的路由器，BACo 在多样性和质量方面实现了 21.3% 的联合改进。人类评估也反映了这些改进。结果表明，基础模型和对齐模型之间的协作可以优化和控制多样性和质量。

Title: OckBench: Measuring the Efficiency of LLM Reasoning

• Authors: Zheng Du, Hao Kang, Song Han, Tushar Krishna, Ligeng Zhu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.05722
• Pdf URL: https://arxiv.org/pdf/2511.05722
• Copy Paste: [[2511.05722]] OckBench: Measuring the Efficiency of LLM Reasoning(https://arxiv.org/abs/2511.05722)
• Keywords: language model, gpt, llm
• Abstract: Large language models such as GPT-4, Claude 3, and the Gemini series have improved automated reasoning and code generation. However, existing benchmarks mainly focus on accuracy and output quality, and they ignore an important factor: decoding token efficiency. In real systems, generating 10,000 tokens versus 100,000 tokens leads to large differences in latency, cost, and energy. In this work, we introduce OckBench, a model-agnostic and hardware-agnostic benchmark that evaluates both accuracy and token count for reasoning and coding tasks. Through experiments comparing multiple open- and closed-source models, we uncover that many models with comparable accuracy differ wildly in token consumption, revealing that efficiency variance is a neglected but significant axis of differentiation. We further demonstrate Pareto frontiers over the accuracy-efficiency plane and argue for an evaluation paradigm shift: we should no longer treat tokens as "free" to multiply. OckBench provides a unified platform for measuring, comparing, and guiding research in token-efficient reasoning. Our benchmarks are available at this https URL .
• 摘要：GPT-4、Claude 3、Gemini 系列等大型语言模型改进了自动推理和代码生成。然而，现有的基准测试主要关注准确性和输出质量，而忽略了一个重要因素：解码令牌效率。在实际系统中，生成 10,000 个代币与生成 100,000 个代币会导致延迟、成本和能源方面存在巨大差异。在这项工作中，我们引入了 OckBench，这是一种与模型无关和硬件无关的基准测试，可评估推理和编码任务的准确性和标记计数。通过比较多个开源和闭源模型的实验，我们发现许多具有相当精度的模型在代币消耗方面存在很大差异，这表明效率方差是一个被忽视但重要的差异轴。我们进一步证明了准确性-效率平面上的帕累托边界，并主张评估范式的转变：我们不应该再将代币视为“自由”繁殖。 OckBench 提供了一个统一的平台，用于测量、比较和指导令牌效率推理的研究。我们的基准测试可以在这个 https URL 上找到。

Title: Multi-Scale Feature Fusion and Graph Neural Network Integration for Text Classification with Large Language Models

• Authors: Xiangchen Song, Yulin Huang, Jinxu Guo, Yuchen Liu, Yaxuan Luan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.05752
• Pdf URL: https://arxiv.org/pdf/2511.05752
• Copy Paste: [[2511.05752]] Multi-Scale Feature Fusion and Graph Neural Network Integration for Text Classification with Large Language Models(https://arxiv.org/abs/2511.05752)
• Keywords: language model
• Abstract: This study investigates a hybrid method for text classification that integrates deep feature extraction from large language models, multi-scale fusion through feature pyramids, and structured modeling with graph neural networks to enhance performance in complex semantic contexts. First, the large language model captures contextual dependencies and deep semantic representations of the input text, providing a rich feature foundation for subsequent modeling. Then, based on multi-level feature representations, the feature pyramid mechanism effectively integrates semantic features of different scales, balancing global information and local details to construct hierarchical semantic expressions. Furthermore, the fused features are transformed into graph representations, and graph neural networks are employed to capture latent semantic relations and logical dependencies in the text, enabling comprehensive modeling of complex interactions among semantic units. On this basis, the readout and classification modules generate the final category predictions. The proposed method demonstrates significant advantages in robustness alignment experiments, outperforming existing models on ACC, F1-Score, AUC, and Precision, which verifies the effectiveness and stability of the framework. This study not only constructs an integrated framework that balances global and local information as well as semantics and structure, but also provides a new perspective for multi-scale feature fusion and structured semantic modeling in text classification tasks.
• 摘要：本研究研究了一种文本分类的混合方法，该方法集成了从大型语言模型中提取深度特征、通过特征金字塔进行多尺度融合以及使用图神经网络进行结构化建模，以提高复杂语义上下文中的性能。首先，大语言模型捕获输入文本的上下文依赖关系和深层语义表示，为后续建模提供丰富的特征基础。然后，基于多级特征表示，特征金字塔机制有效地整合不同尺度的语义特征，平衡全局信息和局部细节，构建层次语义表达。此外，融合的特征被转换为图表示，并采用图神经网络来捕获文本中潜在的语义关系和逻辑依赖关系，从而能够对语义单元之间的复杂交互进行全面建模。在此基础上，读出和分类模块生成最终的类别预测。该方法在鲁棒性对齐实验中表现出显着的优势，在ACC、F1-Score、AUC和Precision上优于现有模型，验证了该框架的有效性和稳定性。这项研究不仅构建了一个平衡全局和局部信息以及语义和结构的集成框架，而且为文本分类任务中的多尺度特征融合和结构化语义建模提供了新的视角。

Title: Language Generation: Complexity Barriers and Implications for Learning

• Authors: Marcelo Arenas, Pablo Barceló, Luis Cofré, Alexander Kozachinskiy
• Subjects: cs.CL, cs.AI, cs.FL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.05759
• Pdf URL: https://arxiv.org/pdf/2511.05759
• Copy Paste: [[2511.05759]] Language Generation: Complexity Barriers and Implications for Learning(https://arxiv.org/abs/2511.05759)
• Keywords: language model
• Abstract: Kleinberg and Mullainathan showed that, in principle, language generation is always possible: with sufficiently many positive examples, a learner can eventually produce sentences indistinguishable from those of a target language. However, the existence of such a guarantee does not speak to its practical feasibility. In this work, we show that even for simple and well-studied language families -- such as regular and context-free languages -- the number of examples required for successful generation can be extraordinarily large, and in some cases not bounded by any computable function. These results reveal a substantial gap between theoretical possibility and efficient learnability. They suggest that explaining the empirical success of modern language models requires a refined perspective -- one that takes into account structural properties of natural language that make effective generation possible in practice.
• 摘要：克莱因伯格和穆莱纳森表明，原则上，语言生成总是可能的：通过足够多的正面例子，学习者最终可以生成与目标语言无法区分的句子。然而，这种保证的存在并不能说明其实际可行性。在这项工作中，我们表明，即使对于简单且经过充分研究的语言系列（例如常规语言和上下文无关语言），成功生成所需的示例数量也可能非常大，并且在某些情况下不受任何可计算函数的限制。这些结果揭示了理论可能性和有效可学习性之间的巨大差距。他们认为，解释现代语言模型的经验成功需要一种精致的视角——一种考虑到自然语言的结构特性的观点，这些结构特性使得在实践中有效的生成成为可能。

Title: DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning

• Authors: Yaxuan Wang, Chris Yuhao Liu, Quan Liu, Jinglong Pang, Wei Wei, Yujia Bao, Yang Liu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.05784
• Pdf URL: https://arxiv.org/pdf/2511.05784
• Copy Paste: [[2511.05784]] DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning(https://arxiv.org/abs/2511.05784)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Unlearning in Large Language Models (LLMs) is crucial for protecting private data and removing harmful knowledge. Most existing approaches rely on fine-tuning to balance unlearning efficiency with general language capabilities. However, these methods typically require training or access to retain data, which is often unavailable in real world scenarios. Although these methods can perform well when both forget and retain data are available, few works have demonstrated equivalent capability in more practical, data-limited scenarios. To overcome these limitations, we propose Detect-Reasoning Augmented GeneratiON (DRAGON), a systematic, reasoning-based framework that utilizes in-context chain-of-thought (CoT) instructions to guard deployed LLMs before inference. Instead of modifying the base model, DRAGON leverages the inherent instruction-following ability of LLMs and introduces a lightweight detection module to identify forget-worthy prompts without any retain data. These are then routed through a dedicated CoT guard model to enforce safe and accurate in-context intervention. To robustly evaluate unlearning performance, we introduce novel metrics for unlearning performance and the continual unlearning setting. Extensive experiments across three representative unlearning tasks validate the effectiveness of DRAGON, demonstrating its strong unlearning capability, scalability, and applicability in practical scenarios.
• 摘要：忘记大型语言模型 (LLM) 中的学习对于保护私人数据和消除有害知识至关重要。大多数现有方法依赖于微调来平衡忘却效率与一般语言能力。然而，这些方法通常需要训练或访问来保留数据，而这在现实世界场景中通常是不可用的。尽管当忘记和保留数据都可用时，这些方法可以表现良好，但很少有工作在更实际、数据有限的场景中展示出同等的功能。为了克服这些限制，我们提出了检测推理增强生成（DRAGON），这是一种基于推理的系统框架，利用上下文思想链（CoT）指令在推理之前保护部署的 LLM。 DRAGON 没有修改基本模型，而是利用了法学硕士固有的指令跟踪能力，并引入了一个轻量级检测模块来识别值得遗忘的提示，而无需任何保留数据。然后，这些信息将通过专用的 CoT 防护模型进行路由，以实施安全、准确的上下文干预。为了稳健地评估忘却表现，我们引入了新的衡量忘却表现和持续忘却环境的指标。三个具有代表性的遗忘任务的大量实验验证了DRAGON的有效性，展示了其强大的遗忘能力、可扩展性和在实际场景中的适用性。

Title: Quantifying Edits Decay in Fine-tuned LLMs

• Authors: Yinjie Cheng, Paul Youssef, Christin Seifert, Jörg Schlötterer, Zhixue Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.05852
• Pdf URL: https://arxiv.org/pdf/2511.05852
• Copy Paste: [[2511.05852]] Quantifying Edits Decay in Fine-tuned LLMs(https://arxiv.org/abs/2511.05852)
• Keywords: language model, llm
• Abstract: Knowledge editing has emerged as a lightweight alternative to retraining for correcting or injecting specific facts in large language models (LLMs). Meanwhile, fine-tuning remains the default operation for adapting LLMs to new domains and tasks. Despite their widespread adoption, these two post-training interventions have been studied in isolation, leaving open a crucial question: if we fine-tune an edited model, do the edits survive? This question is motivated by two practical scenarios: removing covert or malicious edits, and preserving beneficial edits. If fine-tuning impairs edits as shown in Figure 1, current KE methods become less useful, as every fine-tuned model would require re-editing, which significantly increases the cost; if edits persist, fine-tuned models risk propagating hidden malicious edits, raising serious safety concerns. To this end, we systematically quantify edits decay after fine-tuning, investigating how fine-tuning affects knowledge editing. We evaluate two state-of-the-art editing methods (MEMIT, AlphaEdit) and three fine-tuning approaches (full-parameter, LoRA, DoRA) across five LLMs and three datasets, yielding 232 experimental configurations. Our results show that edits decay after fine-tuning, with survival varying across configurations, e.g., AlphaEdit edits decay more than MEMIT edits. Further, we propose selective-layer fine-tuning and find that fine-tuning edited layers only can effectively remove edits, though at a slight cost to downstream performance. Surprisingly, fine-tuning non-edited layers impairs more edits than full fine-tuning. Overall, our study establishes empirical baselines and actionable strategies for integrating knowledge editing with fine-tuning, and underscores that evaluating model editing requires considering the full LLM application pipeline.
• 摘要：知识编辑已成为重新训练的轻量级替代方案，用于在大型语言模型 (LLM) 中纠正或注入特定事实。与此同时，微调仍然是法学硕士适应新领域和任务的默认操作。尽管这两种训练后干预措施被广泛采用，但它们仍然被单独研究，留下了一个关键问题：如果我们对编辑后的模型进行微调，编辑后的内容还能保留下来吗？这个问题是由两个实际场景引起的：删除隐蔽或恶意编辑，以及保留有益的编辑。如果如图 1 所示微调会损害编辑，那么当前的 KE 方法就会变得不太有用，因为每个微调模型都需要重新编辑，这会显着增加成本；如果编辑持续存在，微调后的模型可能会传播隐藏的恶意编辑，从而引发严重的安全问题。为此，我们系统地量化微调后的编辑衰减，研究微调如何影响知识编辑。我们在五个法学硕士和三个数据集上评估了两种最先进的编辑方法（MEMIT、AlphaEdit）和三种微调方法（全参数、LoRA、DoRA），产生了 232 个实验配置。我们的结果表明，微调后编辑会衰减，不同配置的生存率有所不同，例如，AlphaEdit 编辑的衰减比 MEMIT 编辑的衰减更多。此外，我们提出选择性层微调，并发现微调编辑层只能有效地删除编辑，尽管对下游性能造成了轻微的损失。令人惊讶的是，微调未编辑的图层比完全微调会损害更多的编辑。总体而言，我们的研究为将知识编辑与微调相结合建立了经验基线和可行的策略，并强调评估模型编辑需要考虑完整的法学硕士申请流程。

Title: Retrieval-Augmented Generation in Medicine: A Scoping Review of Technical Implementations, Clinical Applications, and Ethical Considerations

• Authors: Rui Yang, Matthew Yu Heng Wong, Huitao Li, Xin Li, Wentao Zhu, Jingchi Liao, Kunyu Yu, Jonathan Chong Kai Liew, Weihao Xuan, Yingjian Chen, Yuhe Ke, Jasmine Chiat Ling Ong, Douglas Teodoro, Chuan Hong, Daniel Shi Wei Ting, Nan Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.05901
• Pdf URL: https://arxiv.org/pdf/2511.05901
• Copy Paste: [[2511.05901]] Retrieval-Augmented Generation in Medicine: A Scoping Review of Technical Implementations, Clinical Applications, and Ethical Considerations(https://arxiv.org/abs/2511.05901)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: The rapid growth of medical knowledge and increasing complexity of clinical practice pose challenges. In this context, large language models (LLMs) have demonstrated value; however, inherent limitations remain. Retrieval-augmented generation (RAG) technologies show potential to enhance their clinical applicability. This study reviewed RAG applications in medicine. We found that research primarily relied on publicly available data, with limited application in private data. For retrieval, approaches commonly relied on English-centric embedding models, while LLMs were mostly generic, with limited use of medical-specific LLMs. For evaluation, automated metrics evaluated generation quality and task performance, whereas human evaluation focused on accuracy, completeness, relevance, and fluency, with insufficient attention to bias and safety. RAG applications were concentrated on question answering, report generation, text summarization, and information extraction. Overall, medical RAG remains at an early stage, requiring advances in clinical validation, cross-linguistic adaptation, and support for low-resource settings to enable trustworthy and responsible global use.
• 摘要：医学知识的快速增长和临床实践日益复杂性带来了挑战。在这种背景下，大型语言模型（LLM）已经证明了其价值；然而，固有的局限性仍然存在。检索增强生成（RAG）技术显示出增强其临床适用性的潜力。本研究回顾了 RAG 在医学中的应用。我们发现研究主要依赖于公开数据，私人数据的应用有限。对于检索，方法通常依赖于以英语为中心的嵌入模型，而法学硕士大多是通用的，医学特定法学硕士的使用有限。在评估方面，自动化指标评估生成质量和任务绩效，而人工评估侧重于准确性、完整性、相关性和流畅性，而对偏差和安全性关注不够。 RAG 应用集中于问答、报告生成、文本摘要和信息提取。总体而言，医学 RAG 仍处于早期阶段，需要在临床验证、跨语言适应和对资源匮乏环境的支持方面取得进展，以实现值得信赖和负责任的全球使用。

Title: NILC: Discovering New Intents with LLM-assisted Clustering

• Authors: Hongtao Wang, Renchi Yang, Wenqing Lin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.05913
• Pdf URL: https://arxiv.org/pdf/2511.05913
• Copy Paste: [[2511.05913]] NILC: Discovering New Intents with LLM-assisted Clustering(https://arxiv.org/abs/2511.05913)
• Keywords: language model, llm
• Abstract: New intent discovery (NID) seeks to recognize both new and known intents from unlabeled user utterances, which finds prevalent use in practical dialogue systems. Existing works towards NID mainly adopt a cascaded architecture, wherein the first stage focuses on encoding the utterances into informative text embeddings beforehand, while the latter is to group similar embeddings into clusters (i.e., intents), typically by K-Means. However, such a cascaded pipeline fails to leverage the feedback from both steps for mutual refinement, and, meanwhile, the embedding-only clustering overlooks nuanced textual semantics, leading to suboptimal performance. To bridge this gap, this paper proposes NILC, a novel clustering framework specially catered for effective NID. Particularly, NILC follows an iterative workflow, in which clustering assignments are judiciously updated by carefully refining cluster centroids and text embeddings of uncertain utterances with the aid of large language models (LLMs). Specifically, NILC first taps into LLMs to create additional semantic centroids for clusters, thereby enriching the contextual semantics of the Euclidean centroids of embeddings. Moreover, LLMs are then harnessed to augment hard samples (ambiguous or terse utterances) identified from clusters via rewriting for subsequent cluster correction. Further, we inject supervision signals through non-trivial techniques seeding and soft must links for more accurate NID in the semi-supervised setting. Extensive experiments comparing NILC against multiple recent baselines under both unsupervised and semi-supervised settings showcase that NILC can achieve significant performance improvements over six benchmark datasets of diverse domains consistently.
• 摘要：新意图发现（NID）旨在从未标记的用户话语中识别新的和已知的意图，这在实际对话系统中得到了普遍使用。现有的 NID 工作主要采用级联架构，其中第一阶段侧重于预先将话语编码为信息文本嵌入，而后者通常通过 K-Means 将类似的嵌入分组为簇（即意图）。然而，这样的级联管道无法利用两个步骤的反馈来进行相互细化，同时，仅嵌入的聚类忽略了细微的文本语义，导致性能不佳。为了弥补这一差距，本文提出了 NILC，这是一种专门针对有效 NID 的新型聚类框架。特别是，NILC 遵循迭代工作流程，其中借助大型语言模型 (LLM) 仔细细化聚类质心和不确定话语的文本嵌入，从而明智地更新聚类分配。具体来说，NILC 首先利用 LLM 为集群创建额外的语义质心，从而丰富嵌入的欧几里得质心的上下文语义。此外，LLM 还可以通过重写来增强从聚类中识别出的硬样本（模糊或简洁的话语），以进行后续的聚类校正。此外，我们通过重要的技术播种和软必须链接注入监督信号，以便在半监督设置中获得更准确的 NID。在无监督和半监督设置下将 NILC 与多个最近的基线进行比较的大量实验表明，NILC 可以在不同领域的六个基准数据集上一致地实现显着的性能改进。

Title: IDALC: A Semi-Supervised Framework for Intent Detection and Active Learning based Correction

• Authors: Ankan Mullick, Sukannya Purkayastha, Saransh Sharma, Pawan Goyal, Niloy Ganguly
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.05921
• Pdf URL: https://arxiv.org/pdf/2511.05921
• Copy Paste: [[2511.05921]] IDALC: A Semi-Supervised Framework for Intent Detection and Active Learning based Correction(https://arxiv.org/abs/2511.05921)
• Keywords: agent
• Abstract: Voice-controlled dialog systems have become immensely popular due to their ability to perform a wide range of actions in response to diverse user queries. These agents possess a predefined set of skills or intents to fulfill specific user tasks. But every system has its own limitations. There are instances where, even for known intents, if any model exhibits low confidence, it results in rejection of utterances that necessitate manual annotation. Additionally, as time progresses, there may be a need to retrain these agents with new intents from the system-rejected queries to carry out additional tasks. Labeling all these emerging intents and rejected utterances over time is impractical, thus calling for an efficient mechanism to reduce annotation costs. In this paper, we introduce IDALC (Intent Detection and Active Learning based Correction), a semi-supervised framework designed to detect user intents and rectify system-rejected utterances while minimizing the need for human annotation. Empirical findings on various benchmark datasets demonstrate that our system surpasses baseline methods, achieving a 5-10% higher accuracy and a 4-8% improvement in macro-F1. Remarkably, we maintain the overall annotation cost at just 6-10% of the unlabelled data available to the system. The overall framework of IDALC is shown in Fig. 1
• 摘要：语音控制对话系统因其能够执行各种操作来响应不同的用户查询而变得非常受欢迎。这些代理拥有一组预定义的技能或意图来完成特定的用户任务。但每个系统都有其自身的局限性。在某些情况下，即使对于已知的意图，如果任何模型表现出低置信度，也会导致需要手动注释的话语被拒绝。此外，随着时间的推移，可能需要使用系统拒绝的查询中的新意图来重新训练这些代理，以执行其他任务。随着时间的推移，标记所有这些新出现的意图和被拒绝的话语是不切实际的，因此需要一种有效的机制来降低注释成本。在本文中，我们介绍了 IDALC（基于意图检测和主动学习的纠正），这是一种半监督框架，旨在检测用户意图并纠正系统拒绝的话语，同时最大限度地减少人工注释的需求。对各种基准数据集的实证结果表明，我们的系统超越了基线方法，实现了 5-10% 的准确度提高和宏观 F1 的 4-8% 的改进。值得注意的是，我们将总体注释成本维持在系统可用未标记数据的 6-10%。 IDALC的总体框架如图1所示

Title: Reinforcement Learning Improves Traversal of Hierarchical Knowledge in LLMs

• Authors: Renfei Zhang, Manasa Kaniselvan, Niloofar Mireshghallah
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.05933
• Pdf URL: https://arxiv.org/pdf/2511.05933
• Copy Paste: [[2511.05933]] Reinforcement Learning Improves Traversal of Hierarchical Knowledge in LLMs(https://arxiv.org/abs/2511.05933)
• Keywords: language model, llm, prompt
• Abstract: Reinforcement learning (RL) is often credited with improving language model reasoning and generalization at the expense of degrading memorized knowledge. We challenge this narrative by observing that RL-enhanced models consistently outperform their base and supervised fine-tuned (SFT) counterparts on pure knowledge recall tasks, particularly those requiring traversal of hierarchical, structured knowledge (e.g., medical codes). We hypothesize these gains stem not from newly acquired data, but from improved procedural skills in navigating and searching existing knowledge hierarchies within the model parameters. To support this hypothesis, we show that structured prompting, which explicitly guides SFTed models through hierarchical traversal, recovers most of the performance gap (reducing 24pp to 7pp on MedConceptsQA for DeepSeek-V3/R1). We further find that while prompting improves final-answer accuracy, RL-enhanced models retain superior ability to recall correct procedural paths on deep-retrieval tasks. Finally our layer-wise internal activation analysis reveals that while factual representations (e.g., activations for the statement "code 57.95 refers to urinary infection") maintain high cosine similarity between SFT and RL models, query representations (e.g., "what is code 57.95") diverge noticeably, indicating that RL primarily transforms how models traverse knowledge rather than the knowledge representation itself.
• 摘要：强化学习（RL）通常被认为可以改善语言模型推理和泛化，但代价是降低记忆知识的质量。我们通过观察来挑战这种说法，观察到强化学习模型在纯知识回忆任务上始终优于其基础模型和监督微调（SFT）模型，特别是那些需要遍历分层、结构化知识（例如医疗代码）的任务。我们假设这些收益不是来自新获取的数据，而是来自在模型参数内导航和搜索现有知识层次结构时改进的程序技能。为了支持这一假设，我们证明了结构化提示（通过分层遍历明确指导 SFT 模型）弥补了大部分性能差距（在 DeepSeek-V3/R1 的 MedConceptsQA 上将 24pp 减少到 7pp）。我们进一步发现，在提示提高最终答案准确性的同时，强化学习模型保留了在深度检索任务中回忆正确程序路径的卓越能力。最后，我们的逐层内部激活分析表明，虽然事实表示（例如，语句“代码 57.95 指的是泌尿道感染”的激活）在 SFT 和 RL 模型之间保持了较高的余弦相似性，但查询表示（例如“代码 57.95 是什么”）存在明显分歧，这表明 RL 主要改变模型遍历知识的方式，而不是知识表示本身。

Title: Revisiting Entropy in Reinforcement Learning for Large Reasoning Models

• Authors: Renren Jin, Pengzhi Gao, Yuqi Ren, Zhuowen Han, Tongxuan Zhang, Wuwei Huang, Wei Liu, Jian Luan, Deyi Xiong
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.05993
• Pdf URL: https://arxiv.org/pdf/2511.05993
• Copy Paste: [[2511.05993]] Revisiting Entropy in Reinforcement Learning for Large Reasoning Models(https://arxiv.org/abs/2511.05993)
• Keywords: language model, llm
• Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a predominant approach for enhancing the reasoning capabilities of large language models (LLMs). However, the entropy of LLMs usually collapses during RLVR training, causing premature convergence to suboptimal local minima and hinder further performance improvement. Although various approaches have been proposed to mitigate entropy collapse, a comprehensive study of entropy in RLVR remains lacking. To address this gap, we conduct extensive experiments to investigate the entropy dynamics of LLMs trained with RLVR and analyze how model entropy correlates with response diversity, calibration, and performance across various benchmarks. Our findings reveal that the number of off-policy updates, the diversity of training data, and the clipping thresholds in the optimization objective are critical factors influencing the entropy of LLMs trained with RLVR. Moreover, we theoretically and empirically demonstrate that tokens with positive advantages are the primary contributors to entropy collapse, and that model entropy can be effectively regulated by adjusting the relative loss weights of tokens with positive and negative advantages during training.
• 摘要：具有可验证奖励的强化学习（RLVR）已成为增强大型语言模型（LLM）推理能力的主要方法。然而，LLM 的熵通常在 RLVR 训练期间崩溃，导致过早收敛到次优的局部最小值，并阻碍进一步的性能改进。尽管已经提出了各种方法来减轻熵崩溃，但仍然缺乏对 RLVR 中熵的全面研究。为了解决这一差距，我们进行了广泛的实验来研究使用 RLVR 训练的法学硕士的熵动态，并分析模型熵如何与各种基准的响应多样性、校准和性能相关。我们的研究结果表明，离策略更新的数量、训练数据的多样性以及优化目标中的裁剪阈值是影响使用 RLVR 训练的 LLM 熵的关键因素。此外，我们从理论上和实证上证明，具有正优势的令牌是熵崩溃的主要贡献者，并且可以通过在训练过程中调整具有正优势和负优势的令牌的相对损失权重来有效调节模型熵。

Title: LLMs Do Not See Age: Assessing Demographic Bias in Automated Systematic Review Synthesis

• Authors: Favour Yahdii Aghaebe, Tanefa Apekey, Elizabeth Williams, Nafise Sadat Moosavi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06000
• Pdf URL: https://arxiv.org/pdf/2511.06000
• Copy Paste: [[2511.06000]] LLMs Do Not See Age: Assessing Demographic Bias in Automated Systematic Review Synthesis(https://arxiv.org/abs/2511.06000)
• Keywords: language model, gpt, llm, hallucination
• Abstract: Clinical interventions often hinge on age: medications and procedures safe for adults may be harmful to children or ineffective for older adults. However, as language models are increasingly integrated into biomedical evidence synthesis workflows, it remains uncertain whether these systems preserve such crucial demographic distinctions. To address this gap, we evaluate how well state-of-the-art language models retain age-related information when generating abstractive summaries of biomedical studies. We construct DemogSummary, a novel age-stratified dataset of systematic review primary studies, covering child, adult, and older adult populations. We evaluate three prominent summarisation-capable LLMs, Qwen (open-source), Longformer (open-source) and GPT-4.1 Nano (proprietary), using both standard metrics and a newly proposed Demographic Salience Score (DSS), which quantifies age-related entity retention and hallucination. Our results reveal systematic disparities across models and age groups: demographic fidelity is lowest for adult-focused summaries, and under-represented populations are more prone to hallucinations. These findings highlight the limitations of current LLMs in faithful and bias-free summarisation and point to the need for fairness-aware evaluation frameworks and summarisation pipelines in biomedical NLP.
• 摘要：临床干预措施通常取决于年龄：对成人安全的药物和手术可能对儿童有害或对老年人无效。然而，随着语言模型越来越多地集成到生物医学证据合成工作流程中，仍然不确定这些系统是否保留了如此重要的人口统计差异。为了解决这一差距，我们评估了最先进的语言模型在生成生物医学研究的抽象摘要时保留年龄相关信息的情况。我们构建了 DemogSummary，这是一个新颖的按年龄分层的系统回顾性初步研究数据集，涵盖儿童、成人和老年人群。我们使用标准指标和新提出的人口统计学显着性评分（DSS）（量化与年龄相关的实体保留和幻觉）来评估三个具有摘要能力的著名法学硕士：Qwen（开源）、Longformer（开源）和 GPT-4.1 Nano（专有）。我们的结果揭示了不同模型和年龄组之间的系统性差异：以成人为中心的摘要的人口统计保真度最低，而代表性不足的人群更容易产生幻觉。这些发现凸显了当前法学硕士在忠实和无偏见总结方面的局限性，并指出生物医学 NLP 中需要具有公平意识的评估框架和总结流程。

Title: Multi-Reward GRPO Fine-Tuning for De-biasing Large Language Models: A Study Based on Chinese-Context Discrimination Data

• Authors: Deng Yixuan, Ji Xiaoqiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06023
• Pdf URL: https://arxiv.org/pdf/2511.06023
• Copy Paste: [[2511.06023]] Multi-Reward GRPO Fine-Tuning for De-biasing Large Language Models: A Study Based on Chinese-Context Discrimination Data(https://arxiv.org/abs/2511.06023)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) often exhibit implicit biases and discriminatory tendencies that reflect underlying social stereotypes. While recent alignment techniques such as RLHF and DPO have mitigated some of these issues, they remain limited in addressing culturally specific and multi-dimensional forms of discrimination. This paper proposes a Multi-Reward Group Relative Policy Optimization (GRPO) framework to fine-tune LLMs toward ethical and bias-free behavior. Our approach constructs a synthetic English-language dataset derived from Chinese-context discrimination categories, including regional, ethnic, and occupational biases. Each instance is paired with both neutral and biased responses to train a reward model based on DeBERTa-v3, which provides multi-dimensional reward signals capturing fairness, neutrality, and linguistic quality. The trained reward model then guides GRPO fine-tuning to optimize model outputs along these ethical dimensions. Experimental results demonstrate significant reductions in bias intensity and improved alignment with non-discriminatory standards without compromising fluency or informativeness. This study highlights the effectiveness of GRPO-based multi-reward optimization for de-biasing LLMs and offers a replicable framework for cultural-contextual ethical alignment.
• 摘要：大型语言模型（LLM）经常表现出隐含的偏见和歧视倾向，反映了潜在的社会刻板印象。虽然 RLHF 和 DPO 等最近的协调技术已经缓解了其中一些问题，但它们在解决特定文化和多维形式的歧视方面仍然有限。本文提出了一种多奖励群体相对政策优化（GRPO）框架，以微调法学硕士的道德和无偏见行为。我们的方法构建了一个综合英语数据集，该数据集源自中国背景的歧视类别，包括地区、种族和职业偏见。每个实例都与中立和有偏见的响应配对，以训练基于 DeBERTa-v3 的奖励模型，该模型提供捕捉公平性、中立性和语言质量的多维奖励信号。然后，经过训练的奖励模型会指导 GRPO 微调，以沿着这些道德维度优化模型输出。实验结果表明，偏见强度显着降低，并且在不影响流畅性或信息量的情况下提高了与非歧视性标准的一致性。这项研究强调了基于 GRPO 的多重奖励优化对于消除法学硕士偏见的有效性，并为文化背景道德一致性提供了一个可复制的框架。

Title: Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts

• Authors: Xinyuan Yan, Shusen Liu, Kowshik Thopalli, Bei Wang
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.06048
• Pdf URL: https://arxiv.org/pdf/2511.06048
• Copy Paste: [[2511.06048]] Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts(https://arxiv.org/abs/2511.06048)
• Keywords: language model, llm
• Abstract: Sparse autoencoders (SAEs) have emerged as a powerful tool for uncovering interpretable features in large language models (LLMs) through the sparse directions they learn. However, the sheer number of extracted directions makes comprehensive exploration intractable. While conventional embedding techniques such as UMAP can reveal global structure, they suffer from limitations including high-dimensional compression artifacts, overplotting, and misleading neighborhood distortions. In this work, we propose a focused exploration framework that prioritizes curated concepts and their corresponding SAE features over attempts to visualize all available features simultaneously. We present an interactive visualization system that combines topology-based visual encoding with dimensionality reduction to faithfully represent both local and global relationships among selected features. This hybrid approach enables users to investigate SAE behavior through targeted, interpretable subsets, facilitating deeper and more nuanced analysis of concept representation in latent space.
• 摘要：稀疏自动编码器 (SAE) 已成为一种强大的工具，可通过其学习的稀疏方向来揭示大型语言模型 (LLM) 中的可解释特征。然而，提取的方向数量庞大，使得综合探索变得困难。虽然 UMAP 等传统嵌入技术可以揭示全局结构，但它们存在一些局限性，包括高维压缩伪影、过度绘制和误导性邻域扭曲。在这项工作中，我们提出了一个重点探索框架，该框架优先考虑策划的概念及其相应的 SAE 功能，而不是尝试同时可视化所有可用功能。我们提出了一种交互式可视化系统，它将基于拓扑的视觉编码与降维相结合，以忠实地表示所选特征之间的局部和全局关系。这种混合方法使用户能够通过有针对性的、可解释的子集来研究 SAE 行为，从而促进对潜在空间中的概念表示进行更深入、更细致的分析。

Title: Efficient Hate Speech Detection: A Three-Layer LoRA-Tuned BERTweet Framework

• Authors: Mahmoud El-Bahnasawi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06051
• Pdf URL: https://arxiv.org/pdf/2511.06051
• Copy Paste: [[2511.06051]] Efficient Hate Speech Detection: A Three-Layer LoRA-Tuned BERTweet Framework(https://arxiv.org/abs/2511.06051)
• Keywords: language model
• Abstract: This paper addresses the critical challenge of developing computationally efficient hate speech detection systems that maintain competitive performance while being practical for real-time deployment. We propose a novel three-layer framework that combines rule-based pre-filtering with a parameter-efficient LoRA-tuned BERTweet model and continuous learning capabilities. Our approach achieves 0.85 macro F1 score - representing 94% of the performance of state-of-the-art large language models like SafePhi (Phi-4 based) while using a base model that is 100x smaller (134M vs 14B parameters). Compared to traditional BERT-based approaches with similar computational requirements, our method demonstrates superior performance through strategic dataset unification and optimized fine-tuning. The system requires only 1.87M trainable parameters (1.37% of full fine-tuning) and trains in approximately 2 hours on a single T4 GPU, making robust hate speech detection accessible in resource-constrained environments while maintaining competitive accuracy for real-world deployment.
• 摘要：本文解决了开发计算效率高的仇恨语音检测系统的关键挑战，该系统在保持竞争性能的同时又可用于实时部署。我们提出了一种新颖的三层框架，它将基于规则的预过滤与参数高效的 LoRA 调整的 BERTweet 模型和持续学习功能相结合。我们的方法实现了 0.85 宏 F1 分数 - 相当于 SafePhi（基于 Phi-4）等最先进的大型语言模型性能的 94%，同时使用小 100 倍的基本模型（134M 与 14B 参数）。与具有相似计算要求的传统的基于 BERT 的方法相比，我们的方法通过战略数据集统一和优化微调展示了卓越的性能。该系统仅需要 187 万个可训练参数（完全微调的 1.37%），并在单个 T4 GPU 上训练大约 2 小时，从而在资源有限的环境中实现强大的仇恨语音检测，同时保持现实世界部署的竞争准确性。

Title: Stemming Hallucination in Language Models Using a Licensing Oracle

• Authors: Simeon Emanuilov, Richard Ackermann
• Subjects: cs.CL, cs.AI, cs.LG, cs.LO
• Abstract URL: https://arxiv.org/abs/2511.06073
• Pdf URL: https://arxiv.org/pdf/2511.06073
• Copy Paste: [[2511.06073]] Stemming Hallucination in Language Models Using a Licensing Oracle(https://arxiv.org/abs/2511.06073)
• Keywords: language model, hallucination, retrieval-augmented generation
• Abstract: Language models exhibit remarkable natural language generation capabilities but remain prone to hallucinations, generating factually incorrect information despite producing syntactically coherent responses. This study introduces the Licensing Oracle, an architectural solution designed to stem hallucinations in LMs by enforcing truth constraints through formal validation against structured knowledge graphs. Unlike statistical approaches that rely on data scaling or fine-tuning, the Licensing Oracle embeds a deterministic validation step into the model's generative process, ensuring that only factually accurate claims are made. We evaluated the effectiveness of the Licensing Oracle through experiments comparing it with several state-of-the-art methods, including baseline language model generation, fine-tuning for factual recall, fine-tuning for abstention behavior, and retrieval-augmented generation (RAG). Our results demonstrate that although RAG and fine-tuning improve performance, they fail to eliminate hallucinations. In contrast, the Licensing Oracle achieved perfect abstention precision (AP = 1.0) and zero false answers (FAR-NE = 0.0), ensuring that only valid claims were generated with 89.1% accuracy in factual responses. This work shows that architectural innovations, such as the Licensing Oracle, offer a necessary and sufficient solution for hallucinations in domains with structured knowledge representations, offering guarantees that statistical methods cannot match. Although the Licensing Oracle is specifically designed to address hallucinations in fact-based domains, its framework lays the groundwork for truth-constrained generation in future AI systems, providing a new path toward reliable, epistemically grounded models.
• 摘要：语言模型表现出卓越的自然语言生成能力，但仍然容易产生幻觉，尽管产生语法上连贯的响应，但仍会生成实际上不正确的信息。本研究介绍了 Licensing Oracle，这是一种架构解决方案，旨在通过针对结构化知识图的形式验证来强制真理约束，从而阻止 LM 中的幻觉。与依赖数据缩放或微调的统计方法不同，许可预言机将确定性验证步骤嵌入到模型的生成过程中，确保仅做出事实准确的声明。我们通过实验将许可预言机与几种最先进的方法进行比较，包括基线语言模型生成、事实回忆微调、弃权行为微调和检索增强生成（RAG），评估了许可预言机的有效性。我们的结果表明，尽管 RAG 和微调提高了性能，但它们无法消除幻觉。相比之下，Licensing Oracle 实现了完美的弃权精度 (AP = 1.0) 和零错误答案 (FAR-NE = 0.0)，确保仅生成有效的声明，事实响应的准确度为 89.1%。这项工作表明，诸如许可预言机之类的架构创新为具有结构化知识表示的领域中的幻觉提供了必要且充分的解决方案，并提供了统计方法无法匹配的保证。尽管许可预言机是专门为解决基于事实的领域中的幻觉而设计的，但其框架为未来人工智能系统中受事实约束的生成奠定了基础，为实现可靠的、基于认知的模型提供了一条新途径。

Title: MuonAll: Muon Variant for Efficient Finetuning of Large Language Models

• Authors: Saurabh Page, Advait Joshi, S. S. Sonawane
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.06086
• Pdf URL: https://arxiv.org/pdf/2511.06086
• Copy Paste: [[2511.06086]] MuonAll: Muon Variant for Efficient Finetuning of Large Language Models(https://arxiv.org/abs/2511.06086)
• Keywords: language model
• Abstract: Muon optimizer has demonstrated robust results in pretraining of language models but its performance in finetuning of existing public pretrained models is not yet explored. Currently, Muon is used along with AdamW introducing a scope of improvement for adopting all parameters inside Muon. We introduce MuonAll, which incorporates all the parameters inside Muon by transforming into 2D matrices. We conduct extensive finetuning experiments across publicly available language models with model sizes upto half billion parameters. Muon and MuonAll perform at par with AdamW across major benchmarks, highlighting their effectiveness as alternative optimizers. We open-source the distributed implementations of Muon and MuonAll, available at this https URL
• 摘要：Muon 优化器在语言模型的预训练方面表现出了稳健的结果，但其在现有公共预训练模型的微调方面的性能尚未得到探索。目前，Muon 与 AdamW 一起使用，引入了一系列改进，以采用 Muon 内部的所有参数。我们引入 MuonAll，它通过转换为 2D 矩阵来合并 Muon 内部的所有参数。我们对公开可用的语言模型进行了广泛的微调实验，模型大小高达 5 亿个参数。 Muon 和 MuonAll 在主要基准测试中的表现与 AdamW 相当，凸显了它们作为替代优化器的有效性。我们开源了 Muon 和 MuonAll 的分布式实现，可通过此 https URL 获取

Title: Evaluation of retrieval-based QA on QUEST-LOFT

• Authors: Nathan Scales, Nathanael Schärli, Olivier Bousquet
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2511.06125
• Pdf URL: https://arxiv.org/pdf/2511.06125
• Copy Paste: [[2511.06125]] Evaluation of retrieval-based QA on QUEST-LOFT(https://arxiv.org/abs/2511.06125)
• Keywords: language model, retrieval-augmented generation
• Abstract: Despite the popularity of retrieval-augmented generation (RAG) as a solution for grounded QA in both academia and industry, current RAG methods struggle with questions where the necessary information is distributed across many documents or where retrieval needs to be combined with complex reasoning. Recently, the LOFT study has shown that this limitation also applies to approaches based on long-context language models, with the QUEST benchmark exhibiting particularly large headroom. In this paper, we provide an in-depth analysis of the factors contributing to the poor performance on QUEST-LOFT, publish updated numbers based on a thorough human evaluation, and demonstrate that RAG can be optimized to significantly outperform long-context approaches when combined with a structured output format containing reasoning and evidence, optionally followed by answer re-verification.
• 摘要：尽管检索增强生成（RAG）作为基础 QA 的解决方案在学术界和工业界都很受欢迎，但当前的 RAG 方法很难解决必要信息分布在许多文档中或检索需要与复杂推理相结合的问题。最近，LOFT 研究表明，这种限制也适用于基于长上下文语言模型的方法，其中 QUEST 基准表现出特别大的空间。在本文中，我们对导致 QUEST-LOFT 性能不佳的因素进行了深入分析，根据全面的人类评估发布了更新的数据，并证明当与包含推理和证据的结构化输出格式相结合时，RAG 可以进行优化，以显着优于长上下文方法，并可选择随后进行答案重新验证。

Title: Referring Expressions as a Lens into Spatial Language Grounding in Vision-Language Models

• Authors: Akshar Tumu, Varad Shinde, Parisa Kordjamshidi
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.06146
• Pdf URL: https://arxiv.org/pdf/2511.06146
• Copy Paste: [[2511.06146]] Referring Expressions as a Lens into Spatial Language Grounding in Vision-Language Models(https://arxiv.org/abs/2511.06146)
• Keywords: language model
• Abstract: Spatial Reasoning is an important component of human cognition and is an area in which the latest Vision-language models (VLMs) show signs of difficulty. The current analysis works use image captioning tasks and visual question answering. In this work, we propose using the Referring Expression Comprehension task instead as a platform for the evaluation of spatial reasoning by VLMs. This platform provides the opportunity for a deeper analysis of spatial comprehension and grounding abilities when there is 1) ambiguity in object detection, 2) complex spatial expressions with a longer sentence structure and multiple spatial relations, and 3) expressions with negation ('not'). In our analysis, we use task-specific architectures as well as large VLMs and highlight their strengths and weaknesses in dealing with these specific situations. While all these models face challenges with the task at hand, the relative behaviors depend on the underlying models and the specific categories of spatial semantics (topological, directional, proximal, etc.). Our results highlight these challenges and behaviors and provide insight into research gaps and future directions.
• 摘要：空间推理是人类认知的重要组成部分，也是最新视觉语言模型（VLM）表现出困难迹象的领域。当前的分析工作使用图像字幕任务和视觉问答。在这项工作中，我们建议使用引用表达理解任务作为评估 VLM 空间推理的平台。当存在以下情况时，该平台提供了对空间理解和基础能力进行更深入分析的机会：1）对象检测中存在歧义，2）具有较长句子结构和多种空间关系的复杂空间表达，以及3）带有否定（“不”）的表达。在我们的分析中，我们使用特定于任务的架构以及大型 VLM，并强调它们在处理这些特定情况时的优点和缺点。虽然所有这些模型都面临着手头任务的挑战，但相关行为取决于底层模型和空间语义的特定类别（拓扑、方向、邻近等）。我们的结果强调了这些挑战和行为，并提供了对研究差距和未来方向的见解。

Title: BookAsSumQA: An Evaluation Framework for Aspect-Based Book Summarization via Question Answering

• Authors: Ryuhei Miyazato, Ting-Ruen Wei, Xuyang Wu, Hsin-Tai Wu, Kei Harada
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06183
• Pdf URL: https://arxiv.org/pdf/2511.06183
• Copy Paste: [[2511.06183]] BookAsSumQA: An Evaluation Framework for Aspect-Based Book Summarization via Question Answering(https://arxiv.org/abs/2511.06183)
• Keywords: llm
• Abstract: Aspect-based summarization aims to generate summaries that highlight specific aspects of a text, enabling more personalized and targeted summaries. However, its application to books remains unexplored due to the difficulty of constructing reference summaries for long text. To address this challenge, we propose BookAsSumQA, a QA-based evaluation framework for aspect-based book summarization. BookAsSumQA automatically generates aspect-specific QA pairs from a narrative knowledge graph to evaluate summary quality based on its question-answering performance. Our experiments using BookAsSumQA revealed that while LLM-based approaches showed higher accuracy on shorter texts, RAG-based methods become more effective as document length increases, making them more efficient and practical for aspect-based book summarization.
• 摘要：基于方面的摘要旨在生成突出显示文本特定方面的摘要，从而实现更加个性化和有针对性的摘要。然而，由于构建长文本参考摘要的困难，它在书籍中的应用仍未得到探索。为了应对这一挑战，我们提出了 BookAsSumQA，这是一种基于 QA 的评估框架，用于基于方面的书籍摘要。 BookAsSumQA 自动从叙述知识图生成特定方面的 QA 对，以根据其问答性能评估摘要质量。我们使用 BookAsSumQA 进行的实验表明，虽然基于 LLM 的方法在较短的文本上显示出更高的准确性，但随着文档长度的增加，基于 RAG 的方法变得更加有效，这使得它们对于基于方面的书籍摘要更加高效和实用。

Title: Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning

• Authors: Sangmook Lee, Dohyung Kim, Hyukhun Koh, Nakyeong Yang, Kyomin Jung
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06190
• Pdf URL: https://arxiv.org/pdf/2511.06190
• Copy Paste: [[2511.06190]] Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning(https://arxiv.org/abs/2511.06190)
• Keywords: language model, llm
• Abstract: Recent advances in Large Language Models (LLMs) - particularly model scaling and test-time techniques - have greatly enhanced the reasoning capabilities of language models at the expense of higher inference costs. To lower inference costs, prior works train router models or deferral mechanisms that allocate easy queries to a small, efficient model, while forwarding harder queries to larger, more expensive models. However, these trained router models often lack robustness under domain shifts and require expensive data synthesis techniques such as Monte Carlo rollouts to obtain sufficient ground-truth routing labels for training. In this work, we propose Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (STEER), a domain-agnostic framework that performs fine-grained, step-level routing between smaller and larger LLMs without utilizing external models. STEER leverages confidence scores from the smaller model's logits prior to generating a reasoning step, so that the large model is invoked only when necessary. Extensive evaluations using different LLMs on a diverse set of challenging benchmarks across multiple domains such as Mathematical Reasoning, Multi-Hop QA, and Planning tasks indicate that STEER achieves competitive or enhanced accuracy while reducing inference costs (up to +20% accuracy with 48% less FLOPs compared to solely using the larger model on AIME), outperforming baselines that rely on trained external modules. Our results establish model-internal confidence as a robust, domain-agnostic signal for model routing, offering a scalable pathway for efficient LLM deployment.
• 摘要：大型语言模型（LLM）的最新进展——特别是模型扩展和测试时技术——极大地增强了语言模型的推理能力，但代价是更高的推理成本。为了降低推理成本，先前的工作训练路由器模型或延迟机制，将简单的查询分配给小型、高效的模型，同时将较难的查询转发给更大、更昂贵的模型。然而，这些经过训练的路由器模型通常缺乏域转移下的鲁棒性，并且需要昂贵的数据合成技术（例如蒙特卡洛推出）来获得足够的地面实况路由标签用于训练。在这项工作中，我们提出了用于成本效益推理的置信引导逐步模型路由（STEER），这是一种与领域无关的框架，可以在较小和较大的 LLM 之间执行细粒度、步骤级路由，而无需利用外部模型。 STEER 在生成推理步骤之前利用较小模型的 logits 的置信度分数，以便仅在必要时调用大型模型。使用不同的法学硕士对跨多个领域（例如数学推理、多跳 QA 和规划任务）的各种具有挑战性的基准进行的广泛评估表明，STEER 实现了有竞争力或更高的准确度，同时降低了推理成本（与仅在 AIME 上使用较大模型相比，准确度高达 +20%，失败次数减少 48%），优于依赖经过训练的外部模块的基线。我们的结果将模型内部置信度建立为模型路由的稳健、与领域无关的信号，为高效的 LLM 部署提供可扩展的途径。

Title: Explicit Knowledge-Guided In-Context Learning for Early Detection of Alzheimer's Disease

• Authors: Puzhen Su, Yongzhu Miao, Chunxi Guo, Jintao Tang, Shasha Li, Ting Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06215
• Pdf URL: https://arxiv.org/pdf/2511.06215
• Copy Paste: [[2511.06215]] Explicit Knowledge-Guided In-Context Learning for Early Detection of Alzheimer's Disease(https://arxiv.org/abs/2511.06215)
• Keywords: language model, llm
• Abstract: Detecting Alzheimer's Disease (AD) from narrative transcripts remains a challenging task for large language models (LLMs), particularly under out-of-distribution (OOD) and data-scarce conditions. While in-context learning (ICL) provides a parameter-efficient alternative to fine-tuning, existing ICL approaches often suffer from task recognition failure, suboptimal demonstration selection, and misalignment between label words and task objectives, issues that are amplified in clinical domains like AD detection. We propose Explicit Knowledge In-Context Learners (EK-ICL), a novel framework that integrates structured explicit knowledge to enhance reasoning stability and task alignment in ICL. EK-ICL incorporates three knowledge components: confidence scores derived from small language models (SLMs) to ground predictions in task-relevant patterns, parsing feature scores to capture structural differences and improve demo selection, and label word replacement to resolve semantic misalignment with LLM priors. In addition, EK-ICL employs a parsing-based retrieval strategy and ensemble prediction to mitigate the effects of semantic homogeneity in AD transcripts. Extensive experiments across three AD datasets demonstrate that EK-ICL significantly outperforms state-of-the-art fine-tuning and ICL baselines. Further analysis reveals that ICL performance in AD detection is highly sensitive to the alignment of label semantics and task-specific context, underscoring the importance of explicit knowledge in clinical reasoning under low-resource conditions.
• 摘要：对于大型语言模型 (LLM) 来说，从叙述转录中检测阿尔茨海默病 (AD) 仍然是一项具有挑战性的任务，特别是在分布外 (OOD) 和数据稀缺的情况下。虽然上下文学习 (ICL) 提供了一种参数高效的微调替代方案，但现有的 ICL 方法经常遇到任务识别失败、演示选择不理想以及标签词与任务目标之间不一致的问题，这些问题在 AD 检测等临床领域中会被放大。我们提出了显性知识情境学习者（EK-ICL），这是一种集成结构化显性知识以增强 ICL 中推理稳定性和任务一致性的新颖框架。 EK-ICL 包含三个知识组件：从小语言模型 (SLM) 派生的置信度得分，以任务相关模式中的地面预测为基础，解析特征得分以捕获结构差异并改进演示选择，以及标签单词替换以解决与 LLM 先验的语义不一致问题。此外，EK-ICL 采用基于解析的检索策略和集成预测来减轻 AD 转录本中语义同质性的影响。三个 AD 数据集的大量实验表明，EK-ICL 的性能显着优于最先进的微调和 ICL 基线。进一步的分析表明，ICL 在 AD 检测中的性能对标签语义和特定任务上下文的对齐高度敏感，强调了显性知识在资源匮乏条件下临床推理中的重要性。

Title: SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization

• Authors: Yue Huang, Xiangqi Wang, Xiangliang Zhang
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2511.06222
• Pdf URL: https://arxiv.org/pdf/2511.06222
• Copy Paste: [[2511.06222]] SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization(https://arxiv.org/abs/2511.06222)
• Keywords: llm
• Abstract: In high-stakes scenarios-such as self-harm, legal, or medical queries-LLMs must be both trustworthy and helpful. However, these goals often conflict. We propose priority alignment, a new alignment paradigm that enforces a strict "trustworthy-before-helpful" ordering: optimization of helpfulness is conditioned on first meeting trustworthy thresholds (e.g., harmlessness or honesty). To realize this, we introduce Self-Priority Alignment (SPA)-a fully unsupervised framework that generates diverse responses, self-evaluates them and refines them by the model itself, and applies dual-criterion denoising to remove inconsistency and control variance. From this, SPA constructs lexicographically ordered preference pairs and fine-tunes the model using an uncertainty-weighted alignment loss that emphasizes high-confidence, high-gap decisions. Experiments across multiple benchmarks show that SPA improves helpfulness without compromising safety, outperforming strong baselines while preserving general capabilities. Our results demonstrate that SPA provides a scalable and interpretable alignment strategy for critical LLM applications.
• 摘要：在自残、法律或医疗查询等高风险场景中，法学硕士必须既值得信赖又乐于助人。然而，这些目标经常发生冲突。我们提出优先对齐，这是一种新的对齐范式，强制执行严格的“可信先于帮助”排序：帮助的优化取决于首先满足可信阈值（例如，无害或诚实）。为了实现这一点，我们引入了自优先对齐（SPA）——一个完全无监督的框架，它生成不同的响应，对它们进行自我评估并通过模型本身对其进行细化，并应用双标准去噪来消除不一致和控制方差。由此，SPA 构建按字典顺序排序的偏好对，并使用强调高置信度、高差距决策的不确定性加权对齐损失来微调模型。跨多个基准的实验表明，SPA 在不影响安全性的情况下提高了帮助性，在保持一般功能的同时超越了强大的基线。我们的结果表明，SPA 为关键的 LLM 应用程序提供了可扩展且可解释的对齐策略。

Title: Overview of CHIP 2025 Shared Task 2: Discharge Medication Recommendation for Metabolic Diseases Based on Chinese Electronic Health Records

• Authors: Juntao Li, Haobin Yuan, Ling Luo, Tengxiao Lv, Yan Jiang, Fan Wang, Ping Zhang, Huiyi Lv, Jian Wang, Yuanyuan Sun, Hongfei Lin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06230
• Pdf URL: https://arxiv.org/pdf/2511.06230
• Copy Paste: [[2511.06230]] Overview of CHIP 2025 Shared Task 2: Discharge Medication Recommendation for Metabolic Diseases Based on Chinese Electronic Health Records(https://arxiv.org/abs/2511.06230)
• Keywords: language model, llm
• Abstract: Discharge medication recommendation plays a critical role in ensuring treatment continuity, preventing readmission, and improving long-term management for patients with chronic metabolic diseases. This paper present an overview of the CHIP 2025 Shared Task 2 competition, which aimed to develop state-of-the-art approaches for automatically recommending appro-priate discharge medications using real-world Chinese EHR data. For this task, we constructed CDrugRed, a high-quality dataset consisting of 5,894 de-identified hospitalization records from 3,190 patients in China. This task is challenging due to multi-label nature of medication recommendation, het-erogeneous clinical text, and patient-specific variability in treatment plans. A total of 526 teams registered, with 167 and 95 teams submitting valid results to the Phase A and Phase B leaderboards, respectively. The top-performing team achieved the highest overall performance on the final test set, with a Jaccard score of 0.5102, F1 score of 0.6267, demonstrating the potential of advanced large language model (LLM)-based ensemble systems. These re-sults highlight both the promise and remaining challenges of applying LLMs to medication recommendation in Chinese EHRs. The post-evaluation phase remains open at this https URL.
• 摘要：出院药物推荐对于确保治疗连续性、防止再入院和改善慢性代谢性疾病患者的长期管理发挥着关键作用。本文概述了 CHIP 2025 共享任务 2 竞赛，该竞赛旨在开发最先进的方法，利用真实的中国 EHR 数据自动推荐适当的出院药物。对于这项任务，我们构建了 CDrugRed，这是一个高质量的数据集，包含来自中国 3,190 名患者的 5,894 条去识别化的住院记录。由于药物推荐的多标签性质、不同的临床文本以及治疗计划中患者特定的可变性，这项任务具有挑战性。共有 526 支队伍报名，其中 A 期和 B 期排行榜分别有 167 支和 95 支队伍提交了有效成绩。表现最好的团队在最终测试集上取得了最高的整体性能，Jaccard 得分为 0.5102，F1 得分为 0.6267，展示了基于高级大语言模型 (LLM) 的集成系统的潜力。这些结果凸显了将法学硕士应用于中国电子病历中的药物推荐的前景和仍然存在的挑战。后评估阶段在此 https URL 保持开放。

Title: TimeSense:Making Large Language Models Proficient in Time-Series Analysis

• Authors: Zhirui Zhang, Changhua Pei, Tianyi Gao, Zhe Xie, Yibo Hao, Zhaoyang Yu, Longlong Xu, Tong Xiao, Jing Han, Dan Pei
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06344
• Pdf URL: https://arxiv.org/pdf/2511.06344
• Copy Paste: [[2511.06344]] TimeSense:Making Large Language Models Proficient in Time-Series Analysis(https://arxiv.org/abs/2511.06344)
• Keywords: language model, llm
• Abstract: In the time-series domain, an increasing number of works combine text with temporal data to leverage the reasoning capabilities of large language models (LLMs) for various downstream time-series understanding tasks. This enables a single model to flexibly perform tasks that previously required specialized models for each domain. However, these methods typically rely on text labels for supervision during training, biasing the model toward textual cues while potentially neglecting the full temporal features. Such a bias can lead to outputs that contradict the underlying time-series context. To address this issue, we construct the EvalTS benchmark, comprising 10 tasks across three difficulty levels, from fundamental temporal pattern recognition to complex real-world reasoning, to evaluate models under more challenging and realistic scenarios. We also propose TimeSense, a multimodal framework that makes LLMs proficient in time-series analysis by balancing textual reasoning with a preserved temporal sense. TimeSense incorporates a Temporal Sense module that reconstructs the input time-series within the model's context, ensuring that textual reasoning is grounded in the time-series dynamics. Moreover, to enhance spatial understanding of time-series data, we explicitly incorporate coordinate-based positional embeddings, which provide each time point with spatial context and enable the model to capture structural dependencies more effectively. Experimental results demonstrate that TimeSense achieves state-of-the-art performance across multiple tasks, and it particularly outperforms existing methods on complex multi-dimensional time-series reasoning tasks.
• 摘要：在时间序列领域，越来越多的作品将文本与时态数据结合起来，利用大型语言模型（LLM）的推理能力来完成各种下游时间序列理解任务。这使得单个模型能够灵活地执行以前需要针对每个领域使用专门模型的任务。然而，这些方法通常依赖文本标签在训练期间进行监督，使模型偏向文本线索，同时可能忽略完整的时间特征。这种偏差可能会导致输出与潜在的时间序列背景相矛盾。为了解决这个问题，我们构建了 EvalTS 基准，包括三个难度级别的 10 项任务，从基本的时间模式识别到复杂的现实世界推理，以在更具挑战性和现实的场景下评估模型。我们还提出了 TimeSense，这是一个多模式框架，通过平衡文本推理与保留的时间感，使法学硕士能够精通时间序列分析。 TimeSense 包含一个 Temporal Sense 模块，可在模型上下文中重建输入时间序列，确保文本推理基于时间序列动态。此外，为了增强对时间序列数据的空间理解，我们显式地合并了基于坐标的位置嵌入，它为每个时间点提供了空间上下文，并使模型能够更有效地捕获结构依赖性。实验结果表明，TimeSense 在多个任务中实现了最先进的性能，尤其是在复杂的多维时间序列推理任务上优于现有方法。

Title: HatePrototypes: Interpretable and Transferable Representations for Implicit and Explicit Hate Speech Detection

• Authors: Irina Proskurina, Marc-Antoine Carpentier, Julien Velcin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06391
• Pdf URL: https://arxiv.org/pdf/2511.06391
• Copy Paste: [[2511.06391]] HatePrototypes: Interpretable and Transferable Representations for Implicit and Explicit Hate Speech Detection(https://arxiv.org/abs/2511.06391)
• Keywords: language model
• Abstract: Optimization of offensive content moderation models for different types of hateful messages is typically achieved through continued pre-training or fine-tuning on new hate speech benchmarks. However, existing benchmarks mainly address explicit hate toward protected groups and often overlook implicit or indirect hate, such as demeaning comparisons, calls for exclusion or violence, and subtle discriminatory language that still causes harm. While explicit hate can often be captured through surface features, implicit hate requires deeper, full-model semantic processing. In this work, we question the need for repeated fine-tuning and analyze the role of HatePrototypes, class-level vector representations derived from language models optimized for hate speech detection and safety moderation. We find that these prototypes, built from as few as 50 examples per class, enable cross-task transfer between explicit and implicit hate, with interchangeable prototypes across benchmarks. Moreover, we show that parameter-free early exiting with prototypes is effective for both hate types. We release the code, prototype resources, and evaluation scripts to support future research on efficient and transferable hate speech detection.
• 摘要：针对不同类型的仇恨消息的攻击性内容审核模型的优化通常是通过持续的预训练或对新的仇恨言论基准进行微调来实现的。然而，现有基准主要解决对受保护群体的明确仇恨，而往往忽视隐性或间接仇恨，例如贬低性比较、呼吁排斥或暴力，以及仍然造成伤害的微妙歧视性语言。虽然显性仇恨通常可以通过表面特征来捕获，但隐性仇恨需要更深入的全模型语义处理。在这项工作中，我们质疑重复微调的必要性，并分析了 HatePrototypes 的作用，HatePrototypes 是从针对仇恨语音检测和安全审核优化的语言模型派生的类级向量表示。我们发现，这些原型由每个类仅 50 个示例构建而成，可以实现显式仇恨和隐式仇恨之间的跨任务转移，并具有跨基准的可互换原型。此外，我们表明，无参数早期退出原型对于两种仇恨类型都是有效的。我们发布代码、原型资源和评估脚本，以支持未来高效且可转移的仇恨言论检测的研究。

Title: SugarTextNet: A Transformer-Based Framework for Detecting Sugar Dating-Related Content on Social Media with Context-Aware Focal Loss

• Authors: Lionel Z. Wang, Shihan Ben, Yulu Huang, Simeng Qing
• Subjects: cs.CL, cs.CY, cs.SI
• Abstract URL: https://arxiv.org/abs/2511.06402
• Pdf URL: https://arxiv.org/pdf/2511.06402
• Copy Paste: [[2511.06402]] SugarTextNet: A Transformer-Based Framework for Detecting Sugar Dating-Related Content on Social Media with Context-Aware Focal Loss(https://arxiv.org/abs/2511.06402)
• Keywords: language model
• Abstract: Sugar dating-related content has rapidly proliferated on mainstream social media platforms, giving rise to serious societal and regulatory concerns, including commercialization of intimate relationships and the normalization of transactional relationships.~Detecting such content is highly challenging due to the prevalence of subtle euphemisms, ambiguous linguistic cues, and extreme class imbalance in real-world data.~In this work, we present SugarTextNet, a novel transformer-based framework specifically designed to identify sugar dating-related posts on social media.~SugarTextNet integrates a pretrained transformer encoder, an attention-based cue extractor, and a contextual phrase encoder to capture both salient and nuanced features in user-generated text.~To address class imbalance and enhance minority-class detection, we introduce Context-Aware Focal Loss, a tailored loss function that combines focal loss scaling with contextual weighting.~We evaluate SugarTextNet on a newly curated, manually annotated dataset of 3,067 Chinese social media posts from Sina Weibo, demonstrating that our approach substantially outperforms traditional machine learning models, deep learning baselines, and large language models across multiple metrics.~Comprehensive ablation studies confirm the indispensable role of each component.~Our findings highlight the importance of domain-specific, context-aware modeling for sensitive content detection, and provide a robust solution for content moderation in complex, real-world scenarios.
• 摘要：与糖约会相关的内容在主流社交媒体平台上迅速扩散，引起了严重的社会和监管问题，包括亲密关系的商业化和交易关系的正常化。~由于现实世界数据中微妙的委婉语、模糊的语言线索和极端的阶级不平衡的普遍存在，检测此类内容非常具有挑战性。~在这项工作中，我们提出了 SugarTextNet，这是一种基于 Transformer 的新颖框架，专门用于识别社交媒体上与糖约会相关的帖子。~SugarTextNet 集成一个预训练的 Transformer 编码器、一个基于注意力的提示提取器和一个上下文短语编码器，用于捕获用户生成文本中的显着特征和细微差别特征。〜为了解决类别不平衡问题并增强少数类别检测，我们引入了上下文感知焦点损失，这是一种将焦点损失缩放与上下文加权相结合的定制损失函数。〜我们在新策划的、手动注释的新浪微博 3,067 个中文社交媒体帖子数据集上评估 SugarTextNet，证明我们的方法在很大程度上在多个指标上优于传统的机器学习模型、深度学习基线和大型语言模型。〜全面的消融研究证实了每个组件不可或缺的作用。〜我们的研究结果强调了特定领域的上下文感知建模对于敏感内容检测的重要性，并为复杂的现实场景中的内容审核提供了强大的解决方案。

Title: How Well Do LLMs Understand Drug Mechanisms? A Knowledge + Reasoning Evaluation Dataset

• Authors: Sunil Mohan, Theofanis Karaletsos
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06418
• Pdf URL: https://arxiv.org/pdf/2511.06418
• Copy Paste: [[2511.06418]] How Well Do LLMs Understand Drug Mechanisms? A Knowledge + Reasoning Evaluation Dataset(https://arxiv.org/abs/2511.06418)
• Keywords: language model, llm, prompt
• Abstract: Two scientific fields showing increasing interest in pre-trained large language models (LLMs) are drug development / repurposing, and personalized medicine. For both, LLMs have to demonstrate factual knowledge as well as a deep understanding of drug mechanisms, so they can recall and reason about relevant knowledge in novel situations. Drug mechanisms of action are described as a series of interactions between biomedical entities, which interlink into one or more chains directed from the drug to the targeted disease. Composing the effects of the interactions in a candidate chain leads to an inference about whether the drug might be useful or not for that disease. We introduce a dataset that evaluates LLMs on both factual knowledge of known mechanisms, and their ability to reason about them under novel situations, presented as counterfactuals that the models are unlikely to have seen during training. Using this dataset, we show that o4-mini outperforms the 4o, o3, and o3-mini models from OpenAI, and the recent small Qwen3-4B-thinking model closely matches o4-mini's performance, even outperforming it in some cases. We demonstrate that the open world setting for reasoning tasks, which requires the model to recall relevant knowledge, is more challenging than the closed world setting where the needed factual knowledge is provided. We also show that counterfactuals affecting internal links in the reasoning chain present a much harder task than those affecting a link from the drug mentioned in the prompt.
• 摘要：对预训练大语言模型 (LLM) 越来越感兴趣的两个科学领域是药物开发/再利用和个性化医疗。对于这两者，法学硕士必须展示事实知识以及对药物机制的深刻理解，以便他们能够在新情况下回忆和推理相关知识。药物作用机制被描述为生物医学实体之间的一系列相互作用，这些相互作用连接成一条或多条从药物指向目标疾病的链。将相互作用的影响组合到候选链中可以推断出该药物是否对该疾病有用。我们引入了一个数据集，用于评估法学硕士对已知机制的事实知识，以及他们在新情况下推理这些机制的能力，这些数据以模型在训练期间不太可能看到的反事实的形式呈现。使用此数据集，我们表明 o4-mini 优于 OpenAI 的 4o、o3 和 o3-mini 模型，并且最近的小型 Qwen3-4B 思维模型与 o4-mini 的性能非常匹配，甚至在某些情况下优于 o4-mini。我们证明，推理任务的开放世界环境需要模型回忆相关知识，这比提供所需事实知识的封闭世界环境更具挑战性。我们还表明，影响推理链中内部链接的反事实比影响提示中提到的药物链接的反事实提出的任务要困难得多。

Title: Dutch Metaphor Extraction from Cancer Patients' Interviews and Forum Data using LLMs and Human in the Loop

• Authors: Lifeng Han, David Lindevelt, Sander Puts, Erik van Mulligen, Suzan Verberne
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2511.06427
• Pdf URL: https://arxiv.org/pdf/2511.06427
• Copy Paste: [[2511.06427]] Dutch Metaphor Extraction from Cancer Patients' Interviews and Forum Data using LLMs and Human in the Loop(https://arxiv.org/abs/2511.06427)
• Keywords: language model, llm, prompt
• Abstract: Metaphors and metaphorical language (MLs) play an important role in healthcare communication between clinicians, patients, and patients' family members. In this work, we focus on Dutch language data from cancer patients. We extract metaphors used by patients using two data sources: (1) cancer patient storytelling interview data and (2) online forum data, including patients' posts, comments, and questions to professionals. We investigate how current state-of-the-art large language models (LLMs) perform on this task by exploring different prompting strategies such as chain of thought reasoning, few-shot learning, and self-prompting. With a human-in-the-loop setup, we verify the extracted metaphors and compile the outputs into a corpus named this http URL. We believe the extracted metaphors can support better patient care, for example shared decision making, improved communication between patients and clinicians, and enhanced patient health literacy. They can also inform the design of personalized care pathways. We share prompts and related resources at this https URL
• 摘要：隐喻和隐喻语言 (ML) 在临床医生、患者和患者家属之间的医疗保健沟通中发挥着重要作用。在这项工作中，我们重点关注癌症患者的荷兰语数据。我们使用两个数据源提取患者使用的隐喻：(1) 癌症患者讲故事的访谈数据和 (2) 在线论坛数据，包括患者的帖子、评论和向专业人士提出的问题。我们通过探索不同的提示策略（例如思想链推理、小样本学习和自我提示）来研究当前最先进的大型语言模型（LLM）如何执行此任务。通过人机交互设置，我们验证提取的隐喻并将输出编译到名为此 http URL 的语料库中。我们相信提取的隐喻可以支持更好的患者护理，例如共同决策、改善患者和临床医生之间的沟通以及提高患者的健康素养。它们还可以为个性化护理途径的设计提供信息。我们在此 https URL 共享提示和相关资源

Title: Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models

• Authors: Mayank Saini, Arit Kumar Bishwas
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.06441
• Pdf URL: https://arxiv.org/pdf/2511.06441
• Copy Paste: [[2511.06441]] Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models(https://arxiv.org/abs/2511.06441)
• Keywords: language model, llm, agent
• Abstract: As AI moves beyond text, large language models (LLMs) increasingly power vision, audio, and document understanding; however, their high inference costs hinder real-time, scalable deployment. Conversely, smaller open-source models offer cost advantages but struggle with complex or multimodal queries. We introduce a unified, modular framework that intelligently routes each query - textual, multimodal, or complex - to the most fitting expert model, using a learned routing network that balances cost and quality. For vision tasks, we employ a two-stage open-source pipeline optimized for efficiency and reviving efficient classical vision components where they remain SOTA for sub-tasks. On benchmarks such as Massive Multitask Language Understanding (MMLU) and Visual Question Answering (VQA), we match or exceed the performance of always-premium LLM (monolithic systems with one model serving all query types) performance, yet reduce the reliance on costly models by over 67%. With its extensible, multi-agent orchestration, we deliver high-quality, resource-efficient AI at scale.
• 摘要：随着人工智能超越文本，大型语言模型 (LLM) 日益增强视觉、音频和文档理解能力；然而，它们的高推理成本阻碍了实时、可扩展的部署。相反，较小的开源模型提供成本优势，但难以应对复杂或多模式查询。我们引入了一个统一的模块化框架，使用平衡成本和质量的学习路由网络，将每个查询（文本、多模式或复杂）智能路由到最合适的专家模型。对于视觉任务，我们采用了两级开源管道，该管道针对效率进行了优化，并恢复了高效的经典视觉组件，使它们在子任务中保持 SOTA。在大规模多任务语言理解 (MMLU) 和视觉问答 (VQA) 等基准测试中，我们匹配或超过了始终优质的 LLM（单一模型服务于所有查询类型的整体系统）性能，同时将对昂贵模型的依赖减少了 67% 以上。凭借其可扩展的多代理编排，我们大规模提供高质量、资源高效的人工智能。

Title: SR-KI: Scalable and Real-Time Knowledge Integration into LLMs via Supervised Attention

• Authors: Bohan Yu, Wei Huang, Kang Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06446
• Pdf URL: https://arxiv.org/pdf/2511.06446
• Copy Paste: [[2511.06446]] SR-KI: Scalable and Real-Time Knowledge Integration into LLMs via Supervised Attention(https://arxiv.org/abs/2511.06446)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: This paper proposes SR-KI, a novel approach for integrating real-time and large-scale structured knowledge bases (KBs) into large language models (LLMs). SR-KI begins by encoding KBs into key-value pairs using a pretrained encoder, and injects them into LLMs' KV cache. Building on this representation, we employ a two-stage training paradigm: first locating a dedicated retrieval layer within the LLM, and then applying an attention-based loss at this layer to explicitly supervise attention toward relevant KB entries. Unlike traditional retrieval-augmented generation methods that rely heavily on the performance of external retrievers and multi-stage pipelines, SR-KI supports end-to-end inference by performing retrieval entirely within the models latent space. This design enables efficient compression of injected knowledge and facilitates dynamic knowledge updates. Comprehensive experiments demonstrate that SR-KI enables the integration of up to 40K KBs into a 7B LLM on a single A100 40GB GPU, and achieves strong retrieval performance, maintaining over 98% Recall@10 on the best-performing task and exceeding 88% on average across all tasks. Task performance on question answering and KB ID generation also demonstrates that SR-KI maintains strong performance while achieving up to 99.75% compression of the injected KBs.
• 摘要：本文提出了 SR-KI，这是一种将实时大规模结构化知识库 (KB) 集成到大型语言模型 (LLM) 中的新方法。 SR-KI 首先使用预训练的编码器将 KB 编码为键值对，并将它们注入 LLM 的 KV 缓存中。在此表示的基础上，我们采用了两阶段训练范例：首先在 LLM 中定位专用检索层，然后在该层应用基于注意力的损失来明确监督对相关知识库条目的注意力。与严重依赖外部检索器和多级管道性能的传统检索增强生成方法不同，SR-KI 通过完全在模型潜在空间内执行检索来支持端到端推理。这种设计能够有效压缩注入的知识并促进动态知识更新。综合实验表明，SR-KI 能够在单个 A100 40GB GPU 上将多达 40K KB 集成到 7B LLM 中，并实现强大的检索性能，在表现最佳的任务上保持超过 98% 的 Recall@10，在所有任务上平均超过 88%。问答和 KB ID 生成的任务性能也表明 SR-KI 保持了强大的性能，同时实现了高达 99.75% 的注入 KB 压缩。

Title: Rethinking what Matters: Effective and Robust Multilingual Realignment for Low-Resource Languages

• Authors: Quang Phuoc Nguyen, David Anugraha, Felix Gaschi, Jun Bin Cheng, En-Shiun Annie Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06497
• Pdf URL: https://arxiv.org/pdf/2511.06497
• Copy Paste: [[2511.06497]] Rethinking what Matters: Effective and Robust Multilingual Realignment for Low-Resource Languages(https://arxiv.org/abs/2511.06497)
• Keywords: language model
• Abstract: Realignment is a promising strategy to improve cross-lingual transfer in multilingual language models. However, empirical results are mixed and often unreliable, particularly for typologically distant or low-resource languages (LRLs) compared to English. Moreover, word realignment tools often rely on high-quality parallel data, which can be scarce or noisy for many LRLs. In this work, we conduct an extensive empirical study to investigate whether realignment truly benefits from using all available languages, or if strategically selected subsets can offer comparable or even improved cross-lingual transfer, and study the impact on LRLs. Our controlled experiments show that realignment can be particularly effective for LRLs and that using carefully selected, linguistically diverse subsets can match full multilingual alignment, and even outperform it for unseen LRLs. This indicates that effective realignment does not require exhaustive language coverage and can reduce data collection overhead, while remaining both efficient and robust when guided by informed language selection.
• 摘要：重新调整是改善多语言语言模型中跨语言迁移的一种有前景的策略。然而，实证结果好坏参半，而且往往不可靠，特别是对于与英语相比类型较远或资源匮乏的语言 (LRL)。此外，字重新对齐工具通常依赖于高质量的并行数据，而这些数据对于许多 LRL 来说可能是稀缺的或嘈杂的。在这项工作中，我们进行了广泛的实证研究，以调查重新调整是否真正受益于使用所有可用语言，或者策略性选择的子集是否可以提供可比较甚至改进的跨语言迁移，并研究对 LRL 的影响。我们的对照实验表明，重新对齐对于 LRL 特别有效，并且使用精心选择的语言多样化子集可以匹配完整的多语言对齐，甚至在未见过的 LRL 上表现优于它。这表明有效的重新调整不需要详尽的语言覆盖，并且可以减少数据收集开销，同时在明智的语言选择指导下保持高效和稳健。

Title: You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

• Authors: Amit LeVi, Raz Lapid, Rom Himelstein, Yaniv Nemcovsky, Ravid Shwartz Ziv, Avi Mendelson
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06516
• Pdf URL: https://arxiv.org/pdf/2511.06516
• Copy Paste: [[2511.06516]] You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations(https://arxiv.org/abs/2511.06516)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) excel across diverse tasks, yet many applications require only limited capabilities, making large variants inefficient in memory and latency. Existing approaches often combine distillation and quantization, but most post-training quantization (PTQ) methods are task-agnostic, ignoring how task-specific signals are distributed across layers. In this work, we propose to use hidden representations that encode task-salient signals as a guideline for quantization. In order to fully utilize our innovative idea, this paper compares two new task-aware PTQ methods: Task-Aware Quantization (TAQ), which allocates bitwidths using task-conditioned statistics from hidden activations, and TAQO, which allocates precision based on direct layer sensitivity tests. From a small calibration set, these approaches identify task-relevant layers, preserving their precision while aggressively quantizing the rest. This yields stable task sensitivity profiles and efficient task-specialized models. Across models, TAQ and TAQO outperform the baselines; TAQ leads on Phi-4, while TAQO leads on Llama-3.1, Qwen3, and Qwen2.5. For instances, on Phi-4 it achieves 42.33 EM / 50.81 F1, far surpassing Activation-aware Weight Quantization (AWQ) (2.25 / 7.07), while remaining within < 1.0% of the original accuracy at lower average precision.
• 摘要：大型语言模型 (LLM) 在各种任务中表现出色，但许多应用程序仅需要有限的功能，导致大型变体在内存和延迟方面效率低下。现有的方法通常将蒸馏和量化结合起来，但大多数训练后量化（PTQ）方法与任务无关，忽略了特定于任务的信号如何跨层分布。在这项工作中，我们建议使用对任务显着信号进行编码的隐藏表示作为量化指南。为了充分利用我们的创新思想，本文比较了两种新的任务感知 PTQ 方法：任务感知量化（TAQ）和 TAQO，前者使用来自隐藏激活的任务条件统计来分配位宽，后者基于直接层灵敏度测试来分配精度。这些方法从一个小的校准集中识别与任务相关的层，在保持其精度的同时积极量化其余层。这会产生稳定的任务敏感性配置文件和高效的任务专用模型。在各个模型中，TAQ 和 TAQO ​​的表现均优于基线； TAQ 在 Phi-4 上领先，而 TAQO ​​在 Llama-3.1、Qwen3 和 Qwen2.5 上领先。例如，在 Phi-4 上，它实现了 42.33 EM / 50.81 F1，远远超过激活感知权重量化 (AWQ) (2.25 / 7.07)，同时在较低的平均精度下保持在原始精度的 < 1.0% 范围内。

Title: Better Datasets Start From RefineLab: Automatic Optimization for High-Quality Dataset Refinement

• Authors: Xiaonan Luo, Yue Huang, Ping He, Xiangliang Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06530
• Pdf URL: https://arxiv.org/pdf/2511.06530
• Copy Paste: [[2511.06530]] Better Datasets Start From RefineLab: Automatic Optimization for High-Quality Dataset Refinement(https://arxiv.org/abs/2511.06530)
• Keywords: language model, llm
• Abstract: High-quality Question-Answer (QA) datasets are foundational for reliable Large Language Model (LLM) evaluation, yet even expert-crafted datasets exhibit persistent gaps in domain coverage, misaligned difficulty distributions, and factual inconsistencies. The recent surge in generative model-powered datasets has compounded these quality challenges. In this work, we introduce RefineLab, the first LLM-driven framework that automatically refines raw QA textual data into high-quality datasets under a controllable token-budget constraint. RefineLab takes a set of target quality attributes (such as coverage and difficulty balance) as refinement objectives, and performs selective edits within a predefined token budget to ensure practicality and efficiency. In essence, RefineLab addresses a constrained optimization problem: improving the quality of QA samples as much as possible while respecting resource limitations. With a set of available refinement operations (e.g., rephrasing, distractor replacement), RefineLab takes as input the original dataset, a specified set of target quality dimensions, and a token budget, and determines which refinement operations should be applied to each QA sample. This process is guided by an assignment module that selects optimal refinement strategies to maximize overall dataset quality while adhering to the budget constraint. Experiments demonstrate that RefineLab consistently narrows divergence from expert datasets across coverage, difficulty alignment, factual fidelity, and distractor quality. RefineLab pioneers a scalable, customizable path to reproducible dataset design, with broad implications for LLM evaluation.
• 摘要：高质量的问答 (QA) 数据集是可靠的大型语言模型 (LLM) 评估的基础，但即使是专家制作的数据集，在领域覆盖范围、难度分布错位和事实不一致方面也存在持续的差距。最近生成模型驱动的数据集的激增加剧了这些质量挑战。在这项工作中，我们介绍了 RefineLab，这是第一个 LLM 驱动的框架，可以在可控的令牌预算约束下自动将原始 QA 文本数据细化为高质量的数据集。 RefineLab 以一组目标质量属性（例如覆盖率和难度平衡）作为细化目标，并在预定义的代币预算内进行选择性编辑，以确保实用性和效率。从本质上讲，RefineLab 解决的是受限优化问题：在尊重资源限制的同时尽可能提高 QA 样本的质量。通过一组可用的细化操作（例如，改写、干扰项替换），RefineLab 将原始数据集、一组指定的目标质量维度和令牌预算作为输入，并确定应将哪些细化操作应用于每个 QA 样本。此过程由分配模块指导，该模块选择最佳细化策略以最大限度地提高总体数据集质量，同时遵守预算约束。实验表明，RefineLab 在覆盖范围、难度调整、事实保真度和干扰质量方面始终缩小与专家数据集的差异。 RefineLab 开创了一种可扩展、可定制的可重复数据集设计途径，对 LLM 评估具有广泛的影响。

Title: Ibom NLP: A Step Toward Inclusive Natural Language Processing for Nigeria's Minority Languages

• Authors: Oluwadara Kalejaiye, Luel Hagos Beyene, David Ifeoluwa Adelani, Mmekut-Mfon Gabriel Edet, Aniefon Daniel Akpan, Eno-Abasi Urua, Anietie Andy
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06531
• Pdf URL: https://arxiv.org/pdf/2511.06531
• Copy Paste: [[2511.06531]] Ibom NLP: A Step Toward Inclusive Natural Language Processing for Nigeria's Minority Languages(https://arxiv.org/abs/2511.06531)
• Keywords: llm
• Abstract: Nigeria is the most populous country in Africa with a population of more than 200 million people. More than 500 languages are spoken in Nigeria and it is one of the most linguistically diverse countries in the world. Despite this, natural language processing (NLP) research has mostly focused on the following four languages: Hausa, Igbo, Nigerian-Pidgin, and Yoruba (i.e <1% of the languages spoken in Nigeria). This is in part due to the unavailability of textual data in these languages to train and apply NLP algorithms. In this work, we introduce ibom -- a dataset for machine translation and topic classification in four Coastal Nigerian languages from the Akwa Ibom State region: Anaang, Efik, Ibibio, and Oro. These languages are not represented in Google Translate or in major benchmarks such as Flores-200 or SIB-200. We focus on extending Flores-200 benchmark to these languages, and further align the translated texts with topic labels based on SIB-200 classification dataset. Our evaluation shows that current LLMs perform poorly on machine translation for these languages in both zero-and-few shot settings. However, we find the few-shot samples to steadily improve topic classification with more shots.
• 摘要：尼日利亚是非洲人口最多的国家，人口超过2亿。尼日利亚使用 500 多种语言，是世界上语言最多样化的国家之一。尽管如此，自然语言处理 (NLP) 研究主要集中在以下四种语言：豪萨语、伊博语、尼日利亚洋泾浜语和约鲁巴语（即尼日利亚使用的语言的 1% 以下）。部分原因是这些语言中无法使用文本数据来训练和应用 NLP 算法。在这项工作中，我们介绍了 ibom——一个用于阿夸伊博姆州地区四种尼日利亚沿海语言的机器翻译和主题分类的数据集：Anaang、Efik、Ibibio 和 Oro。这些语言未出现在 Google 翻译或主要基准（例如 Flores-200 或 SIB-200）中。我们专注于将 Flores-200 基准扩展到这些语言，并根据 SIB-200 分类数据集进一步将翻译文本与主题标签对齐。我们的评估表明，当前的法学硕士在零镜头和少镜头设置中对这些语言的机器翻译表现不佳。然而，我们发现少镜头样本可以通过更多镜头稳步提高主题分类。

Title: Rep2Text: Decoding Full Text from a Single LLM Token Representation

• Authors: Haiyan Zhao, Zirui He, Fan Yang, Ali Payani, Mengnan Du
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.06571
• Pdf URL: https://arxiv.org/pdf/2511.06571
• Copy Paste: [[2511.06571]] Rep2Text: Decoding Full Text from a Single LLM Token Representation(https://arxiv.org/abs/2511.06571)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) have achieved remarkable progress across diverse tasks, yet their internal mechanisms remain largely opaque. In this work, we address a fundamental question: to what extent can the original input text be recovered from a single last-token representation within an LLM? We propose Rep2Text, a novel framework for decoding full text from last-token representations. Rep2Text employs a trainable adapter that projects a target model's internal representations into the embedding space of a decoding language model, which then autoregressively reconstructs the input text. Experiments on various model combinations (Llama-3.1-8B, Gemma-7B, Mistral-7B-v0.1, Llama-3.2-3B) demonstrate that, on average, over half of the information in 16-token sequences can be recovered from this compressed representation while maintaining strong semantic integrity and coherence. Furthermore, our analysis reveals an information bottleneck effect: longer sequences exhibit decreased token-level recovery while preserving strong semantic integrity. Besides, our framework also demonstrates robust generalization to out-of-distribution medical data.
• 摘要：大型语言模型（LLM）在不同的任务上取得了显着的进展，但其内部机制在很大程度上仍然不透明。在这项工作中，我们解决了一个基本问题：在多大程度上可以从 LLM 中的单个最后一个令牌表示中恢复原始输入文本？我们提出了 Rep2Text，这是一种用于从最后一个令牌表示解码全文的新颖框架。 Rep2Text 采用可训练适配器，将目标模型的内部表示投影到解码语言模型的嵌入空间中，然后自回归重建输入文本。对各种模型组合（Llama-3.1-8B、Gemma-7B、Mistral-7B-v0.1、Llama-3.2-3B）的实验表明，平均而言，16 个令牌序列中超过一半的信息可以从这种压缩表示中恢复，同时保持强大的语义完整性和连贯性。此外，我们的分析揭示了信息瓶颈效应：较长的序列表现出降低的标记级恢复，同时保持强大的语义完整性。此外，我们的框架还展示了对分布外医疗数据的强大泛化能力。

Title: TabRAG: Tabular Document Retrieval via Structured Language Representations

• Authors: Jacob Si, Mike Qu, Michelle Lee, Yingzhen Li
• Subjects: cs.CL, cs.AI, cs.CV, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.06582
• Pdf URL: https://arxiv.org/pdf/2511.06582
• Copy Paste: [[2511.06582]] TabRAG: Tabular Document Retrieval via Structured Language Representations(https://arxiv.org/abs/2511.06582)
• Keywords: retrieval-augmented generation
• Abstract: Ingesting data for Retrieval-Augmented Generation (RAG) involves either fine-tuning the embedding model directly on the target corpus or parsing documents for embedding model encoding. The former, while accurate, incurs high computational hardware requirements, while the latter suffers from suboptimal performance when extracting tabular data. In this work, we address the latter by presenting TabRAG, a parsing-based RAG pipeline designed to tackle table-heavy documents via structured language representations. TabRAG outperforms existing popular parsing-based methods for generation and retrieval. Code is available at this https URL.
• 摘要：提取用于检索增强生成（RAG）的数据涉及直接在目标语料库上微调嵌入模型或解析文档以进行嵌入模型编码。前者虽然准确，但对计算硬件要求很高，而后者在提取表格数据时性能不佳。在这项工作中，我们通过提出 TabRAG 来解决后者，这是一种基于解析的 RAG 管道，旨在通过结构化语言表示来处理表密集型文档。 TabRAG 的性能优于现有流行的基于解析的生成和检索方法。代码可从此 https URL 获取。

Title: MedVoiceBias: A Controlled Study of Audio LLM Behavior in Clinical Decision-Making

• Authors: Zhi Rui Tam, Yun-Nung Chen
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2511.06592
• Pdf URL: https://arxiv.org/pdf/2511.06592
• Copy Paste: [[2511.06592]] MedVoiceBias: A Controlled Study of Audio LLM Behavior in Clinical Decision-Making(https://arxiv.org/abs/2511.06592)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: As large language models transition from text-based interfaces to audio interactions in clinical settings, they might introduce new vulnerabilities through paralinguistic cues in audio. We evaluated these models on 170 clinical cases, each synthesized into speech from 36 distinct voice profiles spanning variations in age, gender, and emotion. Our findings reveal a severe modality bias: surgical recommendations for audio inputs varied by as much as 35% compared to identical text-based inputs, with one model providing 80% fewer recommendations. Further analysis uncovered age disparities of up to 12% between young and elderly voices, which persisted in most models despite chain-of-thought prompting. While explicit reasoning successfully eliminated gender bias, the impact of emotion was not detected due to poor recognition performance. These results demonstrate that audio LLMs are susceptible to making clinical decisions based on a patient's voice characteristics rather than medical evidence, a flaw that risks perpetuating healthcare disparities. We conclude that bias-aware architectures are essential and urgently needed before the clinical deployment of these models.
• 摘要：随着大型语言模型从基于文本的界面过渡到临床环境中的音频交互，它们可能会通过音频中的副语言提示引入新的漏洞。我们在 170 个临床案例中评估了这些模型，每个案例都根据 36 种不同的声音特征合成了语音，涵盖了年龄、性别和情绪的变化。我们的研究结果揭示了严重的模态偏差：与相同的基于文本的输入相比，音频输入的手术建议差异高达 35%，其中一种模型提供的建议少了 80%。进一步的分析发现，年轻人和老年人的声音之间存在高达 12% 的年龄差异，尽管有思维链提示，但这种差异在大多数模型中仍然存在。虽然显式推理成功消除了性别偏见，但由于识别性能不佳，没有检测到情绪的影响。这些结果表明，音频法学硕士很容易根据患者的声音特征而不是医学证据做出临床决策，这一缺陷可能会导致医疗保健差异长期存在。我们的结论是，在这些模型的临床部署之前，偏差感知架构是必要且迫切需要的。

Title: Steering LLMs toward Korean Local Speech: Iterative Refinement Framework for Faithful Dialect Translation

• Authors: Keunhyeung Park, Seunguk Yu, Youngbin Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06680
• Pdf URL: https://arxiv.org/pdf/2511.06680
• Copy Paste: [[2511.06680]] Steering LLMs toward Korean Local Speech: Iterative Refinement Framework for Faithful Dialect Translation(https://arxiv.org/abs/2511.06680)
• Keywords: language model, llm
• Abstract: Standard-to-dialect machine translation remains challenging due to a persistent dialect gap in large language models and evaluation distortions inherent in n-gram metrics, which favor source copying over authentic dialect translation. In this paper, we propose the dialect refinement (DIA-REFINE) framework, which guides LLMs toward faithful target dialect outputs through an iterative loop of translation, verification, and feedback using external dialect classifiers. To address the limitations of n-gram-based metrics, we introduce the dialect fidelity score (DFS) to quantify linguistic shift and the target dialect ratio (TDR) to measure the success of dialect translation. Experiments on Korean dialects across zero-shot and in-context learning baselines demonstrate that DIA-REFINE consistently enhances dialect fidelity. The proposed metrics distinguish between False Success cases, where high n-gram scores obscure failures in dialectal translation, and True Attempt cases, where genuine attempts at dialectal translation yield low n-gram scores. We also observed that models exhibit varying degrees of responsiveness to the framework, and that integrating in-context examples further improves the translation of dialectal expressions. Our work establishes a robust framework for goal-directed, inclusive dialect translation, providing both rigorous evaluation and critical insights into model performance.
• 摘要：标准到方言的机器翻译仍然具有挑战性，因为大型语言模型中持续存在的方言差距以及 n-gram 指标固有的评估扭曲，这些指标有利于源代码复制而不是真实的方言翻译。在本文中，我们提出了方言细化（DIA-REFINE）框架，该框架通过使用外部方言分类器的翻译、验证和反馈的迭代循环，引导法学硕士获得忠实的目标方言输出。为了解决基于 n-gram 的指标的局限性，我们引入了方言保真度得分 (DFS) 来量化语言转变，并引入目标方言比率 (TDR) 来衡量方言翻译的成功程度。在零样本和上下文学习基线上对韩语方言进行的实验表明，DIA-REFINE 始终如一地增强了方言保真度。所提出的指标区分了“错误成功”案例（其中高 n 元语法分数掩盖了方言翻译中的失败）和“真实尝试”案例（其中真正的方言翻译尝试产生了低 n 元语法分数）。我们还观察到模型对框架表现出不同程度的响应能力，并且集成上下文示例进一步改善了方言表达的翻译。我们的工作为目标导向、包容性的方言翻译建立了一个强大的框架，为模型性能提供严格的评估和关键的见解。

Title: Textual Self-attention Network: Test-Time Preference Optimization through Textual Gradient-based Attention

• Authors: Shibing Mo, Haoyang Ruan, Kai Wu, Jing Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06682
• Pdf URL: https://arxiv.org/pdf/2511.06682
• Copy Paste: [[2511.06682]] Textual Self-attention Network: Test-Time Preference Optimization through Textual Gradient-based Attention(https://arxiv.org/abs/2511.06682)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have demonstrated remarkable generalization capabilities, but aligning their outputs with human preferences typically requires expensive supervised fine-tuning. Recent test-time methods leverage textual feedback to overcome this, but they often critique and revise a single candidate response, lacking a principled mechanism to systematically analyze, weigh, and synthesize the strengths of multiple promising candidates. Such a mechanism is crucial because different responses may excel in distinct aspects (e.g., clarity, factual accuracy, or tone), and combining their best elements may produce a far superior outcome. This paper proposes the Textual Self-Attention Network (TSAN), a new paradigm for test-time preference optimization that requires no parameter updates. TSAN emulates self-attention entirely in natural language to overcome this gap: it analyzes multiple candidates by formatting them into textual keys and values, weighs their relevance using an LLM-based attention module, and synthesizes their strengths into a new, preference-aligned response under the guidance of the learned textual attention. This entire process operates in a textual gradient space, enabling iterative and interpretable optimization. Empirical evaluations demonstrate that with just three test-time iterations on a base SFT model, TSAN outperforms supervised models like Llama-3.1-70B-Instruct and surpasses the current state-of-the-art test-time alignment method by effectively leveraging multiple candidate solutions.
• 摘要：大型语言模型 (LLM) 已表现出卓越的泛化能力，但将其输出与人类偏好保持一致通常需要昂贵的监督微调。最近的测试时方法利用文本反馈来克服这个问题，但它们经常批评和修改单个候选人的回答，缺乏系统分析、权衡和综合多个有前途候选人的优势的原则机制。这种机制至关重要，因为不同的反应可能在不同的方面（例如清晰度、事实准确性或语气）表现出色，并且将它们的最佳元素结合起来可能会产生更好的结果。本文提出了文本自注意力网络（TSAN），这是一种不需要参数更新的测试时偏好优化的新范式。 TSAN 完全用自然语言模拟自我注意力来克服这一差距：它通过将多个候选者格式化为文本键和值来分析它们，使用基于 LLM 的注意力模块权衡它们的相关性，并在学习到的文本注意力的指导下将它们的优势综合成一个新的、与偏好一致的响应。整个过程在文本梯度空间中运行，从而实现迭代和可解释的优化。实证评估表明，只需在基本 SFT 模型上进行 3 次测试时间迭代，TSAN 就优于 Llama-3.1-70B-Instruct 等监督模型，并通过有效利用多个候选解决方案超越了当前最先进的测试时间对齐方法。

Title: Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights

• Authors: Hyunjae Kim, Jiwoong Sohn, Aidan Gilson, Nicholas Cochran-Caggiano, Serina Applebaum, Heeju Jin, Seihee Park, Yujin Park, Jiyeong Park, Seoyoung Choi, Brittany Alexandra Herrera Contreras, Thomas Huang, Jaehoon Yun, Ethan F. Wei, Roy Jiang, Leah Colucci, Eric Lai, Amisha Dave, Tuo Guo, Maxwell B. Singer, Yonghoe Koo, Ron A. Adelman, James Zou, Andrew Taylor, Arman Cohan, Hua Xu, Qingyu Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06738
• Pdf URL: https://arxiv.org/pdf/2511.06738
• Copy Paste: [[2511.06738]] Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights(https://arxiv.org/abs/2511.06738)
• Keywords: language model, gpt, llm, retrieval-augmented generation
• Abstract: Large language models (LLMs) are transforming the landscape of medicine, yet two fundamental challenges persist: keeping up with rapidly evolving medical knowledge and providing verifiable, evidence-grounded reasoning. Retrieval-augmented generation (RAG) has been widely adopted to address these limitations by supplementing model outputs with retrieved evidence. However, whether RAG reliably achieves these goals remains unclear. Here, we present the most comprehensive expert evaluation of RAG in medicine to date. Eighteen medical experts contributed a total of 80,502 annotations, assessing 800 model outputs generated by GPT-4o and Llama-3.1-8B across 200 real-world patient and USMLE-style queries. We systematically decomposed the RAG pipeline into three components: (i) evidence retrieval (relevance of retrieved passages), (ii) evidence selection (accuracy of evidence usage), and (iii) response generation (factuality and completeness of outputs). Contrary to expectation, standard RAG often degraded performance: only 22% of top-16 passages were relevant, evidence selection remained weak (precision 41-43%, recall 27-49%), and factuality and completeness dropped by up to 6% and 5%, respectively, compared with non-RAG variants. Retrieval and evidence selection remain key failure points for the model, contributing to the overall performance drop. We further show that simple yet effective strategies, including evidence filtering and query reformulation, substantially mitigate these issues, improving performance on MedMCQA and MedXpertQA by up to 12% and 8.2%, respectively. These findings call for re-examining RAG's role in medicine and highlight the importance of stage-aware evaluation and deliberate system design for reliable medical LLM applications.
• 摘要：大语言模型 (LLM) 正在改变医学格局，但仍然存在两个基本挑战：跟上快速发展的医学知识并提供可验证的、基于证据的推理。检索增强生成（RAG）已被广泛采用，通过用检索到的证据补充模型输出来解决这些限制。然而，RAG 能否可靠地实现这些目标仍不清楚。在此，我们提出迄今为止最全面的 RAG 医学专家评估。 18 位医学专家总共贡献了 80,502 个注释，评估了 GPT-4o 和 Llama-3.1-8B 在 200 个真实患者和 USMLE 式查询中生成的 800 个模型输出。我们系统地将 RAG 管道分解为三个部分：(i) 证据检索（检索到的段落的相关性）、(ii) 证据选择（证据使用的准确性）和 (iii) 响应生成（输出的事实性和完整性）。与预期相反，标准 RAG 常常会降低性能：与非 RAG 变体相比，前 16 个段落中只有 22% 是相关的，证据选择仍然很弱（精确度 41-43%，召回率 27-49%），真实性和完整性分别下降高达 6% 和 5%。检索和证据选择仍然是模型的关键故障点，导致整体性能下降。我们进一步表明，简单而有效的策略，包括证据过滤和查询重新制定，可以大大缓解这些问题，将 MedMCQA 和 MedXpertQA 的性能分别提高高达 12% 和 8.2%。这些发现要求重新审视 RAG 在医学中的作用，并强调阶段感知评估和深思熟虑的系统设计对于可靠的医学法学硕士申请的重要性。

Title: Sensitivity of Small Language Models to Fine-tuning Data Contamination

• Authors: Nicy Scaria, Silvester John Joseph Kennedy, Deepak Subramani
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06763
• Pdf URL: https://arxiv.org/pdf/2511.06763
• Copy Paste: [[2511.06763]] Sensitivity of Small Language Models to Fine-tuning Data Contamination(https://arxiv.org/abs/2511.06763)
• Keywords: language model
• Abstract: Small Language Models (SLMs) are increasingly being deployed in resource-constrained environments, yet their behavioral robustness to data contamination during instruction tuning remains poorly understood. We systematically investigate the contamination sensitivity of 23 SLMs (270M to 4B parameters) across multiple model families by measuring susceptibility to syntactic and semantic transformation types during instruction tuning: syntactic transformations (character and word reversal) and semantic transformations (irrelevant and counterfactual responses), each applied at contamination levels of 25\%, 50\%, 75\%, and 100\%. Our results reveal fundamental asymmetries in vulnerability patterns: syntactic transformations cause catastrophic performance degradation, with character reversal producing near-complete failure across all models regardless of size or family, while semantic transformations demonstrate distinct threshold behaviors and greater resilience in core linguistic capabilities. Critically, we discover a ``\textit{capability curse}" where larger, more capable models become more susceptible to learning semantic corruptions, effectively following harmful instructions more readily, while our analysis of base versus instruction-tuned variants reveals that alignment provides inconsistent robustness benefits, sometimes even reducing resilience. Our work establishes three core contributions: (1) empirical evidence of SLMs' disproportionate vulnerability to syntactic pattern contamination, (2) identification of asymmetric sensitivity patterns between syntactic and semantic transformations, and (3) systematic evaluation protocols for contamination robustness assessment. These findings have immediate deployment implications, suggesting that current robustness assumptions may not hold for smaller models and highlighting the need for contamination-aware training protocols.
• 摘要：小语言模型 (SLM) 越来越多地部署在资源受限的环境中，但它们在指令调整期间对数据污染的行为鲁棒性仍然知之甚少。我们通过测量指令调整期间对句法和语义转换类型的敏感性，系统地研究了跨多个模型系列的 23 个 SLM（270M 到 4B 参数）的污染敏感性：句法转换（字符和单词反转）和语义转换（不相关和反事实响应），每种转换都应用于 25\%、50\%、75\% 和 100\% 的污染水平。我们的结果揭示了漏洞模式中的根本不对称性：句法转换会导致灾难性的性能下降，字符反转会导致所有模型（无论大小或系列）几乎完全失败，而语义转换则表现出不同的阈值行为和核心语言能力的更大弹性。重要的是，我们发现了一个“\textit{能力诅咒}”，其中更大、能力更强的模型更容易学习语义损坏，更容易有效地遵循有害指令，而我们对基础变量与指令调整变量的分析表明，对齐提供了不一致的鲁棒性优势，有时甚至降低了弹性。我们的工作建立了三个核心贡献：（1）SLM对句法模式污染的脆弱性的经验证据，（2）识别句法和语义转换之间的不对称敏感性模式，以及（3）系统性这些发现具有直接的部署意义，表明当前的稳健性假设可能不适用于较小的模型，并强调了污染感知培训协议的必要性。

Title: SAFENLIDB: A Privacy-Preserving Safety Alignment Framework for LLM-based Natural Language Database Interfaces

• Authors: Ruiheng Liu, XiaoBing Chen, Jinyu Zhang, Qiongwen Zhang, Yu Zhang, Bailong Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06778
• Pdf URL: https://arxiv.org/pdf/2511.06778
• Copy Paste: [[2511.06778]] SAFENLIDB: A Privacy-Preserving Safety Alignment Framework for LLM-based Natural Language Database Interfaces(https://arxiv.org/abs/2511.06778)
• Keywords: language model, llm, chain-of-thought, agent
• Abstract: The rapid advancement of Large Language Models (LLMs) has driven significant progress in Natural Language Interface to Database (NLIDB). However, the widespread adoption of LLMs has raised critical privacy and security concerns. During interactions, LLMs may unintentionally expose confidential database contents or be manipulated by attackers to exfiltrate data through seemingly benign queries. While current efforts typically rely on rule-based heuristics or LLM agents to mitigate this leakage risk, these methods still struggle with complex inference-based attacks, suffer from high false positive rates, and often compromise the reliability of SQL queries. To address these challenges, we propose \textsc{SafeNlidb}, a novel privacy-security alignment framework for LLM-based NLIDB. The framework features an automated pipeline that generates hybrid chain-of-thought interaction data from scratch, seamlessly combining implicit security reasoning with SQL generation. Additionally, we introduce reasoning warm-up and alternating preference optimization to overcome the multi-preference oscillations of Direct Preference Optimization (DPO), enabling LLMs to produce security-aware SQL through fine-grained reasoning without the need for human-annotated preference data. Extensive experiments demonstrate that our method outperforms both larger-scale LLMs and ideal-setting baselines, achieving significant security improvements while preserving high this http URL: This work may contain content that is offensive and harmful!
• 摘要：大型语言模型 (LLM) 的快速发展推动了自然语言数据库接口 (NLIDB) 的重大进展。然而，法学硕士的广泛采用引发了严重的隐私和安全问题。在交互过程中，LLM 可能会无意中暴露机密数据库内容，或者被攻击者操纵，通过看似良性的查询来窃取数据。虽然当前的工作通常依靠基于规则的启发式方法或 LLM 代理来减轻这种泄漏风险，但这些方法仍然难以应对复杂的基于推理的攻击，误报率很高，并且经常损害 SQL 查询的可靠性。为了应对这些挑战，我们提出了 \textsc{SafeNlidb}，这是一种基于 LLM 的 NLIDB 的新型隐私安全对齐框架。该框架具有自动管道，可从头开始生成混合思想链交互数据，将隐式安全推理与 SQL 生成无缝结合。此外，我们引入推理预热和交替偏好优化来克服直接偏好优化（DPO）的多偏好振荡，使法学硕士能够通过细粒度推理生成安全感知的 SQL，而不需要人工注释的偏好数据。大量实验表明，我们的方法优于更大规模的法学硕士和理想设置基线，在保持高此 http URL 的同时实现了显着的安全性改进：这项工作可能包含攻击性和有害的内容！

Title: Learning to Focus: Focal Attention for Selective and Scalable Transformers

• Authors: Dhananjay Ram, Wei Xia, Stefano Soatto
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.06818
• Pdf URL: https://arxiv.org/pdf/2511.06818
• Copy Paste: [[2511.06818]] Learning to Focus: Focal Attention for Selective and Scalable Transformers(https://arxiv.org/abs/2511.06818)
• Keywords: long context
• Abstract: Attention is a core component of transformer architecture, whether encoder-only, decoder-only, or encoder-decoder model. However, the standard softmax attention often produces noisy probability distribution, which can impair effective feature selection at every layer of these models, particularly for long contexts. We propose Focal Attention, a simple yet effective modification that sharpens the attention distribution by controlling the softmax temperature, either as a fixed hyperparameter or as a learnable parameter during training. This sharpening enables the model to concentrate on the most relevant tokens while suppressing irrelevant ones. Empirically, Focal Attention scales more favorably than standard transformer with respect to model size, training data, and context length. Across diverse benchmarks, it achieves the same accuracy with up to 42% fewer parameters or 33% less training data. On long-context tasks, it delivers substantial relative improvements ranging from 17% to 82%, demonstrating its effectiveness in real world applications.
• 摘要：无论是仅编码器模型、仅解码器模型还是编码器-解码器模型，注意力都是 Transformer 架构的核心组成部分。然而，标准的 softmax 注意力通常会产生噪声概率分布，这可能会损害这些模型每一层的有效特征选择，特别是对于长上下文。我们提出了焦点注意力（Focal Attention），这是一种简单而有效的修改，通过控制 softmax 温度（作为固定超参数或作为训练期间的可学习参数）来锐化注意力分布。这种锐化使模型能够专注于最相关的标记，同时抑制不相关的标记。根据经验，焦点注意力在模型大小、训练数据和上下文长度方面比标准转换器更有利。在不同的基准测试中，它实现了相同的准确度，但参数减少了 42%，训练数据减少了 33%。在长上下文任务中，它提供了 17% 到 82% 的实质性相对改进，证明了其在现实世界应用中的有效性。

Title: Beyond Plain Demos: A Demo-centric Anchoring Paradigm for In-Context Learning in Alzheimer's Disease Detection

• Authors: Puzhen Su, Haoran Yin, Yongzhu Miao, Jintao Tang, Shasha Li, Ting Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06826
• Pdf URL: https://arxiv.org/pdf/2511.06826
• Copy Paste: [[2511.06826]] Beyond Plain Demos: A Demo-centric Anchoring Paradigm for In-Context Learning in Alzheimer's Disease Detection(https://arxiv.org/abs/2511.06826)
• Keywords: language model, llm
• Abstract: Detecting Alzheimer's disease (AD) from narrative transcripts challenges large language models (LLMs): pre-training rarely covers this out-of-distribution task, and all transcript demos describe the same scene, producing highly homogeneous contexts. These factors cripple both the model's built-in task knowledge (\textbf{task cognition}) and its ability to surface subtle, class-discriminative cues (\textbf{contextual perception}). Because cognition is fixed after pre-training, improving in-context learning (ICL) for AD detection hinges on enriching perception through better demonstration (demo) sets. We demonstrate that standard ICL quickly saturates, its demos lack diversity (context width) and fail to convey fine-grained signals (context depth), and that recent task vector (TV) approaches improve broad task adaptation by injecting TV into the LLMs' hidden states (HSs), they are ill-suited for AD detection due to the mismatch of injection granularity, strength and position. To address these bottlenecks, we introduce \textbf{DA4ICL}, a demo-centric anchoring framework that jointly expands context width via \emph{\textbf{Diverse and Contrastive Retrieval}} (DCR) and deepens each demo's signal via \emph{\textbf{Projected Vector Anchoring}} (PVA) at every Transformer layer. Across three AD benchmarks, DA4ICL achieves large, stable gains over both ICL and TV baselines, charting a new paradigm for fine-grained, OOD and low-resource LLM adaptation.
• 摘要：从叙述文字记录中检测阿尔茨海默病 (AD) 对大型语言模型 (LLM) 提出了挑战：预训练很少涵盖这种分布外的任务，并且所有文字记录演示都描述相同的场景，产生高度同质的上下文。这些因素削弱了模型的内置任务知识（\textbf{任务认知}）及其表面微妙的、类别区分线索（\textbf{上下文感知}）的能力。由于认知在预训练后是固定的，因此改善 AD 检测的上下文学习 (ICL) 取决于通过更好的演示 (demo) 集来丰富感知。我们证明标准 ICL 很快饱和，其演示缺乏多样性（上下文宽度）并且无法传达细粒度信号（上下文深度），并且最近的任务向量（TV）方法通过将 TV 注入 LLM 的隐藏状态（HS）来改善广泛的任务适应性，由于注入粒度、强度和位置不匹配，它们不适合 AD 检测。为了解决这些瓶颈，我们引入了 \textbf{DA4ICL}，一个以演示为中心的锚定框架，它通过 \emph{\textbf{Diverse and Contrastive Retrieval}} (DCR) 联合扩展上下文宽度，并通过每个 Transformer 层的 \emph{\textbf{Projected Vector Anchoring}} (PVA) 加深每个演示的信号。在三个 AD 基准测试中，DA4ICL 比 ICL 和 TV 基准获得了巨大、稳定的收益，为细粒度、OOD 和低资源 LLM 适应绘制了新范例。

Title: EduGuardBench: A Holistic Benchmark for Evaluating the Pedagogical Fidelity and Adversarial Safety of LLMs as Simulated Teachers

• Authors: Yilin Jiang, Mingzi Zhang, Xuanyu Yin, Sheng Jin, Suyu Lu, Zuocan Ying, Zengyi Yu, Xiangjie Kong
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.06890
• Pdf URL: https://arxiv.org/pdf/2511.06890
• Copy Paste: [[2511.06890]] EduGuardBench: A Holistic Benchmark for Evaluating the Pedagogical Fidelity and Adversarial Safety of LLMs as Simulated Teachers(https://arxiv.org/abs/2511.06890)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models for Simulating Professions (SP-LLMs), particularly as teachers, are pivotal for personalized education. However, ensuring their professional competence and ethical safety is a critical challenge, as existing benchmarks fail to measure role-playing fidelity or address the unique teaching harms inherent in educational scenarios. To address this, we propose EduGuardBench, a dual-component benchmark. It assesses professional fidelity using a Role-playing Fidelity Score (RFS) while diagnosing harms specific to the teaching profession. It also probes safety vulnerabilities using persona-based adversarial prompts targeting both general harms and, particularly, academic misconduct, evaluated with metrics including Attack Success Rate (ASR) and a three-tier Refusal Quality assessment. Our extensive experiments on 14 leading models reveal a stark polarization in performance. While reasoning-oriented models generally show superior fidelity, incompetence remains the dominant failure mode across most models. The adversarial tests uncovered a counterintuitive scaling paradox, where mid-sized models can be the most vulnerable, challenging monotonic safety assumptions. Critically, we identified a powerful Educational Transformation Effect: the safest models excel at converting harmful requests into teachable moments by providing ideal Educational Refusals. This capacity is strongly negatively correlated with ASR, revealing a new dimension of advanced AI safety. EduGuardBench thus provides a reproducible framework that moves beyond siloed knowledge tests toward a holistic assessment of professional, ethical, and pedagogical alignment, uncovering complex dynamics essential for deploying trustworthy AI in education. See this https URL for Materials.
• 摘要：用于模拟职业的大型语言模型 (SP-LLM)，特别是作为教师，对于个性化教育至关重要。然而，确保他们的专业能力和道德安全是一项严峻的挑战，因为现有的基准无法衡量角色扮演的忠诚度或解决教育场景中固有的独特教学危害。为了解决这个问题，我们提出了 EduGuardBench，一个双组件基准测试。它使用角色扮演忠诚度分数 (RFS) 评估职业忠诚度，同时诊断特定于教师职业的危害。它还使用基于角色的对抗性提示来探测安全漏洞，针对一般危害，特别是学术不端行为，并使用攻击成功率 (ASR) 和三层拒绝质量评估等指标进行评估。我们对 14 个领先模型进行的广泛实验揭示了性能的明显两极分化。虽然面向推理的模型通常表现出卓越的保真度，但无能仍然是大多数模型的主要失败模式。对抗性测试揭示了一个违反直觉的缩放悖论，其中中型模型可能是最脆弱的，最具挑战性的单调安全假设。至关重要的是，我们发现了强大的教育转型效应：最安全的模型擅长通过提供理想的教育拒绝，将有害的请求转化为可教的时刻。这种能力与 ASR 呈强烈负相关，揭示了先进人工智能安全的新维度。因此，EduGuardBench 提供了一个可重复的框架，超越了孤立的知识测试，转向对专业、道德和教学一致性的整体评估，揭示了在教育中部署值得信赖的人工智能所必需的复杂动态。请参阅此 https URL 以获取材料。

Title: RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation

• Authors: Haofeng Wang, Yu Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.06899
• Pdf URL: https://arxiv.org/pdf/2511.06899
• Copy Paste: [[2511.06899]] RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation(https://arxiv.org/abs/2511.06899)
• Keywords: language model, gpt
• Abstract: Large Vision-Language Models (LVLMs) excel in multimodal reasoning and have shown impressive performance on various multimodal benchmarks. However, most of these benchmarks evaluate models primarily through multiple-choice or short-answer formats, which do not take the reasoning process into account. Although some benchmarks assess the reasoning process, their methods are often overly simplistic and only examine reasoning when answers are incorrect. This approach overlooks scenarios where flawed reasoning leads to correct answers. In addition, these benchmarks do not consider the impact of intermodal relationships on reasoning. To address this issue, we propose the Reasoning Process Tree Score (RPTS), a tree structure-based metric to assess reasoning processes. Specifically, we organize the reasoning steps into a reasoning tree and leverage its hierarchical information to assign weighted faithfulness scores to each reasoning step. By dynamically adjusting these weights, RPTS not only evaluates the overall correctness of the reasoning, but also pinpoints where the model fails in the reasoning. To validate RPTS in real-world multimodal scenarios, we construct a new benchmark, RPTS-Eval, comprising 374 images and 390 reasoning instances. Each instance includes reliable visual-textual clues that serve as leaf nodes of the reasoning tree. Furthermore, we define three types of intermodal relationships to investigate how intermodal interactions influence the reasoning process. We evaluated representative LVLMs (e.g., GPT4o, Llava-Next), uncovering their limitations in multimodal reasoning and highlighting the differences between open-source and closed-source commercial LVLMs. We believe that this benchmark will contribute to the advancement of research in the field of multimodal reasoning.
• 摘要：大型视觉语言模型 (LVLM) 擅长多模态推理，并在各种多模态基准测试中表现出令人印象深刻的性能。然而，大多数这些基准测试主要通过多项选择或简答题形式来评估模型，而没有考虑推理过程。尽管一些基准评估推理过程，但它们的方法往往过于简单化，仅在答案不正确时才检查推理。这种方法忽略了有缺陷的推理导致正确答案的情况。此外，这些基准没有考虑多式联运关系对推理的影响。为了解决这个问题，我们提出了推理过程树得分（RPTS），这是一种基于树结构的指标，用于评估推理过程。具体来说，我们将推理步骤组织成推理树，并利用其层次信息为每个推理步骤分配加权忠实度分数。通过动态调整这些权重，RPTS不仅可以评估推理的整体正确性，还可以精确定位模型在推理中失败的地方。为了在现实多模态场景中验证 RPTS，我们构建了一个新的基准 RPTS-Eval，其中包含 374 个图像和 390 个推理实例。每个实例都包含可靠的视觉文本线索，用作推理树的叶节点。此外，我们定义了三种类型的多式联运关系，以研究多式联运交互如何影响推理过程。我们评估了代表性的 LVLM（例如 GPT4o、Llava-Next），揭示了它们在多模态推理方面的局限性，并强调了开源和闭源商业 LVLM 之间的差异。我们相信该基准将有助于多模态推理领域研究的进步。

Title: HLPD: Aligning LLMs to Human Language Preference for Machine-Revised Text Detection

• Authors: Fangqi Dai, Xingjian Jiang, Zizhuang Deng
• Subjects: cs.CL, cs.CR
• Abstract URL: https://arxiv.org/abs/2511.06942
• Pdf URL: https://arxiv.org/pdf/2511.06942
• Copy Paste: [[2511.06942]] HLPD: Aligning LLMs to Human Language Preference for Machine-Revised Text Detection(https://arxiv.org/abs/2511.06942)
• Keywords: gpt, llm, prompt
• Abstract: To prevent misinformation and social issues arising from trustworthy-looking content generated by LLMs, it is crucial to develop efficient and reliable methods for identifying the source of texts. Previous approaches have demonstrated exceptional performance in detecting texts fully generated by LLMs. However, these methods struggle when confronting more advanced LLM output or text with adversarial multi-task machine revision, especially in the black-box setting, where the generating model is unknown. To address this challenge, grounded in the hypothesis that human writing possesses distinctive stylistic patterns, we propose Human Language Preference Detection (HLPD). HLPD employs a reward-based alignment process, Human Language Preference Optimization (HLPO), to shift the scoring model's token distribution toward human-like writing, making the model more sensitive to human writing, therefore enhancing the identification of machine-revised text. We test HLPD in an adversarial multi-task evaluation framework that leverages a five-dimensional prompt generator and multiple advanced LLMs to create diverse revision scenarios. When detecting texts revised by GPT-series models, HLPD achieves a 15.11% relative improvement in AUROC over ImBD, surpassing Fast-DetectGPT by 45.56%. When evaluated on texts generated by advanced LLMs, HLPD achieves the highest average AUROC, exceeding ImBD by 5.53% and Fast-DetectGPT by 34.14%. Code will be made available at this https URL.
• 摘要：为了防止法学硕士生成的看似可信的内容引起错误信息和社会问题，开发有效且可靠的方法来识别文本来源至关重要。以前的方法在检测完全由法学硕士生成的文本方面表现出了卓越的性能。然而，当面对更高级的 LLM 输出或具有对抗性多任务机器修订的文本时，这些方法会遇到困难，特别是在生成模型未知的黑盒设置中。为了应对这一挑战，基于人类写作具有独特文体模式的假设，我们提出了人类语言偏好检测（HLPD）。 HLPD 采用基于奖励的对齐过程，即人类语言偏好优化 (HLPO)，将评分模型的标记分布转向类人书写，使模型对人类书写更加敏感，从而增强机器修改文本的识别。我们在对抗性多任务评估框架中测试 HLPD，该框架利用五维提示生成器和多个高级 LLM 来创建不同的修订场景。在检测 GPT 系列模型修改的文本时，HLPD 的 AUROC 相对 ImBD 提高了 15.11%，超过 Fast-DetectGPT 45.56%。当对高级法学硕士生成的文本进行评估时，HLPD 实现了最高的平均 AUROC，超过 ImBD 5.53%，超过 Fast-DetectGPT 34.14%。代码将在此 https URL 中提供。

Title: SCOPE: Intrinsic Semantic Space Control for Mitigating Copyright Infringement in LLMs

• Authors: Zhenliang Zhang, Xinyu Hu, Xiaojun Wan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07001
• Pdf URL: https://arxiv.org/pdf/2511.07001
• Copy Paste: [[2511.07001]] SCOPE: Intrinsic Semantic Space Control for Mitigating Copyright Infringement in LLMs(https://arxiv.org/abs/2511.07001)
• Keywords: language model, llm
• Abstract: Large language models sometimes inadvertently reproduce passages that are copyrighted, exposing downstream applications to legal risk. Most existing studies for inference-time defences focus on surface-level token matching and rely on external blocklists or filters, which add deployment complexity and may overlook semantically paraphrased leakage. In this work, we reframe copyright infringement mitigation as intrinsic semantic-space control and introduce SCOPE, an inference-time method that requires no parameter updates or auxiliary filters. Specifically, the sparse autoencoder (SAE) projects hidden states into a high-dimensional, near-monosemantic space; benefiting from this representation, we identify a copyright-sensitive subspace and clamp its activations during decoding. Experiments on widely recognized benchmarks show that SCOPE mitigates copyright infringement without degrading general utility. Further interpretability analyses confirm that the isolated subspace captures high-level semantics.
• 摘要：大型语言模型有时会无意中复制受版权保护的段落，使下游应用程序面临法律风险。大多数现有的推理时间防御研究都集中在表面级别的令牌匹配上，并依赖于外部阻止列表或过滤器，这增加了部署的复杂性，并且可能会忽略语义释义的泄漏。在这项工作中，我们将版权侵权缓解重新定义为内在语义空间控制，并引入 SCOPE，一种不需要参数更新或辅助过滤器的推理时间方法。具体来说，稀疏自动编码器（SAE）将隐藏状态投影到高维、近单语义空间中；受益于这种表示，我们识别了版权敏感的子空间并在解码期间限制其激活。在广泛认可的基准上进行的实验表明，SCOPE 在不降低通用性的情况下减轻了版权侵权。进一步的可解释性分析证实，孤立的子空间捕获了高级语义。

Title: Automated Circuit Interpretation via Probe Prompting

• Authors: Giuseppe Birardi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07002
• Pdf URL: https://arxiv.org/pdf/2511.07002
• Copy Paste: [[2511.07002]] Automated Circuit Interpretation via Probe Prompting(https://arxiv.org/abs/2511.07002)
• Keywords: prompt
• Abstract: Mechanistic interpretability aims to understand neural networks by identifying which learned features mediate specific behaviors. Attribution graphs reveal these feature pathways, but interpreting them requires extensive manual analysis -- a single prompt can take approximately 2 hours for an experienced circuit tracer. We present probe prompting, an automated pipeline that transforms attribution graphs into compact, interpretable subgraphs built from concept-aligned supernodes. Starting from a seed prompt and target logit, we select high-influence features, generate concept-targeted yet context-varying probes, and group features by cross-prompt activation signatures into Semantic, Relationship, and Say-X categories using transparent decision rules. Across five prompts including classic "capitals" circuits, probe-prompted subgraphs preserve high explanatory coverage while compressing complexity (Completeness 0.83, mean across circuits; Replacement 0.54). Compared to geometric clustering baselines, concept-aligned groups exhibit higher behavioral coherence: 2.3x higher peak-token consistency (0.425 vs 0.183) and 5.8x higher activation-pattern similarity (0.762 vs 0.130), despite lower geometric compactness. Entity-swap tests reveal a layerwise hierarchy: early-layer features transfer robustly (64% transfer rate, mean layer 6.3), while late-layer Say-X features specialize for output promotion (mean layer 16.4), supporting a backbone-and-specialization view of transformer computation. We release code (this https URL), an interactive demo (this https URL), and minimal artifacts enabling immediate reproduction and community adoption.
• 摘要：机械可解释性旨在通过识别哪些学习特征介导特定行为来理解神经网络。归因图揭示了这些特征路径，但解释它们需要大量的手动分析——对于经验丰富的电路追踪者来说，一次提示可能需要大约 2 小时。我们提出了探针提示，这是一种自动化管道，可将归因图转换为从概念一致的超级节点构建的紧凑、可解释的子图。从种子提示和目标 logit 开始，我们选择高影响力的特征，生成以概念为目标但上下文变化的探针，并使用透明的决策规则通过交叉提示激活签名将特征分组为语义、关系和 Say-X 类别。在包括经典“首都”电路在内的五个提示中，探针提示的子图在压缩复杂性的同时保留了高解释性覆盖范围（完整性 0.83，电路间平均值；替换 0.54）。与几何聚类基线相比，概念对齐组表现出更高的行为一致性：尽管几何紧凑性较低，但峰值标记一致性高出 2.3 倍（0.425 vs 0.183），激活模式相似性高出 5.8 倍（0.762 vs 0.130）。实体交换测试揭示了分层层次结构：早期层特征传输稳健（64% 传输率，平均层 6.3），而后期层 Say-X 特征专门用于输出提升（平均层 16.4），支持变压器计算的骨干和专业化视图。我们发布代码（此 https URL）、交互式演示（此 https URL）以及能够立即复制和社区采用的最小工件。

Title: Beyond English: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs

• Authors: Yingfeng Luo, Ziqiang Xu, Yuxuan Ouyang, Murun Yang, Dingyang Lin, Kaiyan Chang, Tong Zheng, Bei Li, Peinan Feng, Quan Du, Tong Xiao, Jingbo Zhu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07003
• Pdf URL: https://arxiv.org/pdf/2511.07003
• Copy Paste: [[2511.07003]] Beyond English: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs(https://arxiv.org/abs/2511.07003)
• Keywords: language model, llm, prompt
• Abstract: Large language models have significantly advanced Multilingual Machine Translation (MMT), yet the broad language coverage, consistent translation quality, and English-centric bias remain open challenges. To address these challenges, we introduce \textbf{LMT}, a suite of \textbf{L}arge-scale \textbf{M}ultilingual \textbf{T}ranslation models centered on both Chinese and English, covering 60 languages and 234 translation directions. During development, we identify a previously overlooked phenomenon of \textbf{directional degeneration}, where symmetric multi-way fine-tuning data overemphasize reverse directions (X $\to$ En/Zh), leading to excessive many-to-one mappings and degraded translation quality. We propose \textbf{Strategic Downsampling}, a simple yet effective method to mitigate this degeneration. In addition, we design \textbf{Parallel Multilingual Prompting (PMP)}, which leverages typologically related auxiliary languages to enhance cross-lingual transfer. Through rigorous data curation and refined adaptation strategies, LMT achieves SOTA performance among models of comparable language coverage, with our 4B model (LMT-60-4B) surpassing the much larger Aya-101-13B and NLLB-54B models by a substantial margin. We release LMT in four sizes (0.6B/1.7B/4B/8B) to catalyze future research and provide strong baselines for inclusive, scalable, and high-quality MMT \footnote{\href{this https URL}{this https URL}}.
• 摘要：大型语言模型显着推进了多语言机器翻译 (MMT)，但广泛的语言覆盖范围、一致的翻译质量和以英语为中心的偏见仍然是开放的挑战。为了应对这些挑战，我们引入了\textbf{LMT}，这是一套以中英文为中心的大规模\textbf{M}多语言\textbf{T}翻译模型，涵盖60种语言和234个翻译方向。在开发过程中，我们发现了一种先前被忽视的\textbf{方向退化}现象，即对称多路微调数据过分强调反向方向（X $\to$ En/Zh），导致过多的多对一映射和翻译质量下降。我们提出\textbf{策略下采样}，这是一种简单而有效的方法来减轻这种退化。此外，我们设计了\textbf{并行多语言提示（PMP）}，它利用类型相关的辅助语言来增强跨语言迁移。通过严格的数据管理和精细的适应策略，LMT 在可比语言覆盖范围的模型中实现了 SOTA 性能，我们的 4B 模型 (LMT-60-4B) 大幅超越了更大的 Aya-101-13B 和 NLLB-54B 模型。我们发布了四种尺寸 (0.6B/1.7B/4B/8B) 的 LMT，以促进未来的研究，并为包容性、可扩展性和高质量的 MMT \footnote{\href{this https URL}{this https URL}} 提供强有力的基线。

Title: A Picture is Worth a Thousand (Correct) Captions: A Vision-Guided Judge-Corrector System for Multimodal Machine Translation

• Authors: Siddharth Betala, Kushan Raj, Vipul Betala, Rohan Saswade
• Subjects: cs.CL, cs.CV, cs.HC
• Abstract URL: https://arxiv.org/abs/2511.07010
• Pdf URL: https://arxiv.org/pdf/2511.07010
• Copy Paste: [[2511.07010]] A Picture is Worth a Thousand (Correct) Captions: A Vision-Guided Judge-Corrector System for Multimodal Machine Translation(https://arxiv.org/abs/2511.07010)
• Keywords: language model, gpt
• Abstract: In this paper, we describe our system under the team name BLEU Monday for the English-to-Indic Multimodal Translation Task at WAT 2025. We participate in the text-only translation tasks for English-Hindi, English-Bengali, English-Malayalam, and English-Odia language pairs. We present a two-stage approach that addresses quality issues in the training data through automated error detection and correction, followed by parameter-efficient model fine-tuning. Our methodology introduces a vision-augmented judge-corrector pipeline that leverages multimodal language models to systematically identify and correct translation errors in the training data. The judge component classifies translations into three categories: correct, visually ambiguous (requiring image context), or mistranslated (poor translation quality). Identified errors are routed to specialized correctors: GPT-4o-mini regenerates captions requiring visual disambiguation, while IndicTrans2 retranslates cases with pure translation quality issues. This automated pipeline processes 28,928 training examples across four languages, correcting an average of 17.1% of captions per language. We then apply Low-Rank Adaptation (LoRA) to fine-tune the IndicTrans2 en-indic 200M distilled model on both original and corrected datasets. Training on corrected data yields consistent improvements, with BLEU score gains of +1.30 for English-Bengali on the evaluation set (42.00 -> 43.30) and +0.70 on the challenge set (44.90 -> 45.60), +0.60 for English-Odia on the evaluation set (41.00 -> 41.60), and +0.10 for English-Hindi on the challenge set (53.90 -> 54.00).
• 摘要：在本文中，我们以 BLEU Monday 团队名称描述了我们的系统，用于 WAT 2025 的英语到印度语多模态翻译任务。我们参与了英语-印地语、英语-孟加拉语、英语-马拉雅拉姆语和英语-奥迪亚语语言对的纯文本翻译任务。我们提出了一种两阶段方法，通过自动错误检测和纠正，然后进行参数高效的模型微调来解决训练数据中的质量问题。我们的方法引入了视觉增强的判断-纠正器管道，利用多模态语言模型来系统地识别和纠正训练数据中的翻译错误。判断组件将翻译分为三类：正确、视觉上模糊（需要图像上下文）或误译（翻译质量差）。识别出的错误将被发送至专门的校正器：GPT-4o-mini 重新生成需要视觉消歧的字幕，而 IndicTrans2 则重新翻译存在纯粹翻译质量问题的案例。 This automated pipeline processes 28,928 training examples across four languages, correcting an average of 17.1% of captions per language.然后，我们应用低秩适应 (LoRA) 在原始数据集和校正数据集上微调 IndicTrans2 en-indic 200M 蒸馏模型。对校正数据的训练产生了一致的改进，英语-孟加拉语在评估集 (42.00 -> 43.30) 上的 BLEU 分数增益为 +1.30，在挑战集 (44.90 -> 45.60) 上的 BLEU 分数增益为 +0.70，英语-奥迪亚语在评估集上的分数增益为 +0.60 (41.00 -> 41.60)，英语-印地语在评估集上的分数增益为 +0.10。挑战集（53.90 -> 54.00）。

Title: Multilingual Lexical Feature Analysis of Spoken Language for Predicting Major Depression Symptom Severity

• Authors: Anastasiia Tokareva, Judith Dineley, Zoe Firth, Pauline Conde, Faith Matcham, Sara Siddi, Femke Lamers, Ewan Carr, Carolin Oetzmann, Daniel Leightley, Yuezhou Zhang, Amos A. Folarin, Josep Maria Haro, Brenda W.J.H. Penninx, Raquel Bailon, Srinivasan Vairavan, Til Wykes, Richard J.B. Dobson, Vaibhav A. Narayan, Matthew Hotopf, Nicholas Cummins, The RADAR-CNS Consortium
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.07011
• Pdf URL: https://arxiv.org/pdf/2511.07011
• Copy Paste: [[2511.07011]] Multilingual Lexical Feature Analysis of Spoken Language for Predicting Major Depression Symptom Severity(https://arxiv.org/abs/2511.07011)
• Keywords: prompt
• Abstract: Background: Captured between clinical appointments using mobile devices, spoken language has potential for objective, more regular assessment of symptom severity and earlier detection of relapse in major depressive disorder. However, research to date has largely been in non-clinical cross-sectional samples of written language using complex machine learning (ML) approaches with limited interpretability. Methods: We describe an initial exploratory analysis of longitudinal speech data and PHQ-8 assessments from 5,836 recordings of 586 participants in the UK, Netherlands, and Spain, collected in the RADAR-MDD study. We sought to identify interpretable lexical features associated with MDD symptom severity with linear mixed-effects modelling. Interpretable features and high-dimensional vector embeddings were also used to test the prediction performance of four regressor ML models. Results: In English data, MDD symptom severity was associated with 7 features including lexical diversity measures and absolutist language. In Dutch, associations were observed with words per sentence and positive word frequency; no associations were observed in recordings collected in Spain. The predictive power of lexical features and vector embeddings was near chance level across all languages. Limitations: Smaller samples in non-English speech and methodological choices, such as the elicitation prompt, may have also limited the effect sizes observable. A lack of NLP tools in languages other than English restricted our feature choice. Conclusion: To understand the value of lexical markers in clinical research and practice, further research is needed in larger samples across several languages using improved protocols, and ML models that account for within- and between-individual variations in language.
• 摘要：背景：使用移动设备在临床预约之间捕获口语，有可能对症状严重程度进行客观、更定期的评估，并更早地发现重度抑郁症的复发。然而，迄今为止的研究主要是使用可解释性有限的复杂机器学习 (ML) 方法对书面语言的非临床横截面样本进行研究。方法：我们描述了对 RADAR-MDD 研究中收集的英国、荷兰和西班牙 586 名参与者的 5,836 份录音的纵向语音数据和 PHQ-8 评估的初步探索性分析。我们试图通过线性混合效应模型来识别与 MDD 症状严重程度相关的可解释的词汇特征。可解释的特征和高维向量嵌入也被用来测试四个回归器 ML 模型的预测性能。结果：在英语数据中，MDD 症状严重程度与 7 个特征相关，包括词汇多样性测量和绝对主义语言。在荷兰语中，观察到每个句子的单词和积极词频之间的关联；在西班牙收集的录音中没有观察到任何关联。在所有语言中，词汇特征和向量嵌入的预测能力都接近机会水平。局限性：非英语语音和方法选择（例如启发提示）中的较小样本也可能限制了可观察到的效应大小。缺乏英语以外的语言 NLP 工具限制了我们的特征选择。结论：为了了解词汇标记在临床研究和实践中的价值，需要使用改进的协议和解释个体内部和个体之间语言差异的 ML 模型，在多种语言的更大样本中进行进一步研究。

Title: Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks

• Authors: Yauhen Babakhin, Radek Osmulski, Ronay Ak, Gabriel Moreira, Mengyao Xu, Benedikt Schifferer, Bo Liu, Even Oldridge
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2511.07025
• Pdf URL: https://arxiv.org/pdf/2511.07025
• Copy Paste: [[2511.07025]] Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks(https://arxiv.org/abs/2511.07025)
• Keywords: llm
• Abstract: We introduce llama-embed-nemotron-8b, an open-weights text embedding model that achieves state-of-the-art performance on the Multilingual Massive Text Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent models show strong performance, their training data or methodologies are often not fully disclosed. We aim to address this by developing a fully open-source model, publicly releasing its weights and detailed ablation studies, and planning to share the curated training datasets. Our model demonstrates superior performance across all major embedding tasks -- including retrieval, classification and semantic textual similarity (STS) -- and excels in challenging multilingual scenarios, such as low-resource languages and cross-lingual setups. This state-of-the-art performance is driven by a novel data mix of 16.1 million query-document pairs, split between 7.7 million samples from public datasets and 8.4 million synthetically generated examples from various open-weight LLMs. One of our key contributions is a detailed ablation study analyzing core design choices, including a comparison of contrastive loss implementations, an evaluation of synthetic data generation (SDG) strategies, and the impact of model merging. The llama-embed-nemotron-8b is an instruction-aware model, supporting user-defined instructions to enhance performance for specific use-cases. This combination of top-tier performance, broad applicability, and user-driven flexibility enables it to serve as a universal text embedding solution.
• 摘要：我们推出了 llama-embed-nemotron-8b，这是一种开放权重文本嵌入模型，截至 2025 年 10 月 21 日，该模型在多语言大规模文本嵌入基准 (MMTEB) 排行榜上实现了最先进的性能。虽然最近的模型表现出强劲的性能，但其训练数据或方法通常没有完全公开。我们的目标是通过开发一个完全开源的模型、公开发布其权重和详细的消融研究并计划共享精选的训练数据集来解决这个问题。我们的模型在所有主要嵌入任务中都表现出了卓越的性能，包括检索、分类和语义文本相似性 (STS)，并且在具有挑战性的多语言场景（例如资源匮乏的语言和跨语言设置）中表现出色。这种最先进的性能是由 1610 万个查询文档对组成的新颖数据组合驱动的，这些数据组合分为来自公共数据集的 770 万个样本和来自各种开放权重法学硕士综合生成的 840 万个示例。我们的主要贡献之一是进行详细的消融研究，分析核心设计选择，包括对比损失实现的比较、合成数据生成（SDG）策略的评估以及模型合并的影响。 llama-embed-nemotron-8b 是一个指令感知模型，支持用户定义的指令以增强特定用例的性能。顶级性能、广泛适用性和用户驱动的灵活性的结合使其能够成为通用文本嵌入解决方案。

Title: Evaluating LLMs for Anxiety, Depression, and Stress Detection Evaluating Large Language Models for Anxiety, Depression, and Stress Detection: Insights into Prompting Strategies and Synthetic Data

• Authors: Mihael Arcan, David-Paul Niland
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07044
• Pdf URL: https://arxiv.org/pdf/2511.07044
• Copy Paste: [[2511.07044]] Evaluating LLMs for Anxiety, Depression, and Stress Detection Evaluating Large Language Models for Anxiety, Depression, and Stress Detection: Insights into Prompting Strategies and Synthetic Data(https://arxiv.org/abs/2511.07044)
• Keywords: language model, gpt, llm, prompt
• Abstract: Mental health disorders affect over one-fifth of adults globally, yet detecting such conditions from text remains challenging due to the subtle and varied nature of symptom expression. This study evaluates multiple approaches for mental health detection, comparing Large Language Models (LLMs) such as Llama and GPT with classical machine learning and transformer-based architectures including BERT, XLNet, and Distil-RoBERTa. Using the DAIC-WOZ dataset of clinical interviews, we fine-tuned models for anxiety, depression, and stress classification and applied synthetic data generation to mitigate class imbalance. Results show that Distil-RoBERTa achieved the highest F1 score (0.883) for GAD-2, while XLNet outperformed others on PHQ tasks (F1 up to 0.891). For stress detection, a zero-shot synthetic approach (SD+Zero-Shot-Basic) reached an F1 of 0.884 and ROC AUC of 0.886. Findings demonstrate the effectiveness of transformer-based models and highlight the value of synthetic data in improving recall and generalization. However, careful calibration is required to prevent precision loss. Overall, this work emphasizes the potential of combining advanced language models and data augmentation to enhance automated mental health assessment from text.
• 摘要：心理健康障碍影响着全球五分之一以上的成年人，但由于症状表达的微妙性和多样性，从文本中检测此类疾病仍然具有挑战性。这项研究评估了多种心理健康检测方法，将 Llama 和 GPT 等大型语言模型 (LLM) 与经典机器学习和基于 Transformer 的架构（包括 BERT、XLNet 和 Distil-RoBERTa）进行了比较。使用临床访谈的 DAIC-WOZ 数据集，我们对焦虑、抑郁和压力分类模型进行了微调，并应用合成数据生成来缓解类别不平衡。结果显示，Distil-RoBERTa 在 GAD-2 上取得了最高的 F1 分数（0.883），而 XLNet 在 PHQ 任务上表现优于其他（F1 高达 0.891）。对于压力检测，零样本合成方法 (SD+Zero-Shot-Basic) 的 F1 为 0.884，ROC AUC 为 0.886。研究结果证明了基于 Transformer 的模型的有效性，并强调了合成数据在提高召回和泛化方面的价值。然而，需要仔细校准以防止精度损失。总体而言，这项工作强调了将高级语言模型和数据增强相结合以增强文本自动化心理健康评估的潜力。

Title: Importance-Aware Data Selection for Efficient LLM Instruction Tuning

• Authors: Tingyu Jiang, Shen Li, Yiyao Song, Lan Zhang, Hualei Zhu, Yuan Zhao, Xiaohang Xu, Kenjiro Taura, Hao Henry Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07074
• Pdf URL: https://arxiv.org/pdf/2511.07074
• Copy Paste: [[2511.07074]] Importance-Aware Data Selection for Efficient LLM Instruction Tuning(https://arxiv.org/abs/2511.07074)
• Keywords: language model, llm
• Abstract: Instruction tuning plays a critical role in enhancing the performance and efficiency of Large Language Models (LLMs). Its success depends not only on the quality of the instruction data but also on the inherent capabilities of the LLM itself. Some studies suggest that even a small amount of high-quality data can achieve instruction fine-tuning results that are on par with, or even exceed, those from using a full-scale dataset. However, rather than focusing solely on calculating data quality scores to evaluate instruction data, there is a growing need to select high-quality data that maximally enhances the performance of instruction tuning for a given LLM. In this paper, we propose the Model Instruction Weakness Value (MIWV) as a novel metric to quantify the importance of instruction data in enhancing model's capabilities. The MIWV metric is derived from the discrepancies in the model's responses when using In-Context Learning (ICL), helping identify the most beneficial data for enhancing instruction tuning performance. Our experimental results demonstrate that selecting only the top 1\% of data based on MIWV can outperform training on the full dataset. Furthermore, this approach extends beyond existing research that focuses on data quality scoring for data selection, offering strong empirical evidence supporting the effectiveness of our proposed method.
• 摘要：指令调优在提高大型语言模型 (LLM) 的性能和效率方面发挥着至关重要的作用。它的成功不仅取决于教学数据的质量，还取决于法学硕士本身的固有能力。一些研究表明，即使是少量的高质量数据也可以实现与使用完整数据集相同甚至超过的指令微调结果。然而，越来越需要选择高质量数据来最大程度地增强给定法学硕士的指令调优性能，而不是仅仅关注计算数据质量分数来评估指令数据。在本文中，我们提出模型指令弱点值（MIWV）作为一种新颖的指标来量化指令数据在增强模型能力方面的重要性。 MIWV 指标源自使用上下文学习 (ICL) 时模型响应的差异，有助于识别对增强指令调优性能最有益的数据。我们的实验结果表明，仅选择基于 MIWV 的前 1% 的数据可以优于对整个数据集的训练。此外，这种方法超越了现有的专注于数据选择的数据质量评分的研究，提供了强有力的经验证据支持我们提出的方法的有效性。

Title: EmoBang: Detecting Emotion From Bengali Texts

• Authors: Abdullah Al Maruf, Aditi Golder, Zakaria Masud Jiyad, Abdullah Al Numan, Tarannum Shaila Zaman
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07077
• Pdf URL: https://arxiv.org/pdf/2511.07077
• Copy Paste: [[2511.07077]] EmoBang: Detecting Emotion From Bengali Texts(https://arxiv.org/abs/2511.07077)
• Keywords: language model, llm
• Abstract: Emotion detection from text seeks to identify an individual's emotional or mental state - positive, negative, or neutral - based on linguistic cues. While significant progress has been made for English and other high-resource languages, Bengali remains underexplored despite being the world's fourth most spoken language. The lack of large, standardized datasets classifies Bengali as a low-resource language for emotion detection. Existing studies mainly employ classical machine learning models with traditional feature engineering, yielding limited performance. In this paper, we introduce a new Bengali emotion dataset annotated across eight emotion categories and propose two models for automatic emotion detection: (i) a hybrid Convolutional Recurrent Neural Network (CRNN) model (EmoBangHybrid) and (ii) an AdaBoost-Bidirectional Encoder Representations from Transformers (BERT) ensemble model (EmoBangEnsemble). Additionally, we evaluate six baseline models with five feature engineering techniques and assess zero-shot and few-shot large language models (LLMs) on the dataset. To the best of our knowledge, this is the first comprehensive benchmark for Bengali emotion detection. Experimental results show that EmoBangH and EmoBangE achieve accuracies of 92.86% and 93.69%, respectively, outperforming existing methods and establishing strong baselines for future research.
• 摘要：文本情绪检测旨在根据语言线索识别个人的情绪或精神状态（积极、消极或中性）。尽管英语和其他资源丰富的语言已经取得了重大进展，但孟加拉语尽管是世界第四大语言，但仍未得到充分开发。由于缺乏大型标准化数据集，孟加拉语被归类为情感检测的低资源语言。现有的研究主要采用经典的机器学习模型和传统的特征工程，但性能有限。在本文中，我们介绍了一个新的孟加拉语情感数据集，该数据集注释了八种情感类别，并提出了两种自动情感检测模型：（i）混合卷积循环神经网络（CRNN）模型（EmoBangHybrid）和（ii）AdaBoost-来自变形金刚的双向编码器表示（BERT）集成模型（EmoBangEnsemble）。此外，我们使用五种特征工程技术评估了六个基线模型，并评估了数据集上的零样本和少样本大型语言模型（LLM）。据我们所知，这是孟加拉语情绪检测的第一个综合基准。实验结果表明，EmoBangH和EmoBangE的准确率分别达到92.86%和93.69%，优于现有方法，为未来的研究奠定了坚实的基础。

Title: Wasm: A Pipeline for Constructing Structured Arabic Interleaved Multimodal Corpora

• Authors: Khalil Hennara, Ahmad Bastati, Muhammad Hreden, Mohamed Motasim Hamed, Zeina Aldallal, Sara Chrouf, Safwan AlModhayan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07080
• Pdf URL: https://arxiv.org/pdf/2511.07080
• Copy Paste: [[2511.07080]] Wasm: A Pipeline for Constructing Structured Arabic Interleaved Multimodal Corpora(https://arxiv.org/abs/2511.07080)
• Keywords: language model, llm
• Abstract: The performance of large language models (LLMs) and large multimodal models (LMMs) depends heavily on the quality and scale of their pre-training datasets. Recent research shows that large multimodal models trained on natural documents where images and text are interleaved outperform those trained only on image-text pairs across a wide range of benchmarks, leveraging advanced pre- trained models to enforce semantic alignment, image-sequence consistency, and textual coherence. For Arabic, however, the lack of high-quality multimodal datasets that preserve document structure has limited progress. In this paper, we present our pipeline Wasm for processing the Common Crawl dataset to create a new Arabic multimodal dataset that uniquely provides markdown output. Unlike existing Arabic corpora that focus solely on text extraction, our approach preserves the structural integrity of web content while maintaining flexibility for both text-only and multimodal pre-training scenarios. We provide a comprehensive comparative analysis of our data processing pipeline against those used for major existing datasets, highlighting the convergences in filtering strategies and justifying our specific design choices. To support future research, we publicly release a representative dataset dump along with the multimodal processing pipeline for Arabic.
• 摘要：大型语言模型 (LLM) 和大型多模态模型 (LMM) 的性能在很大程度上取决于其预训练数据集的质量和规模。最近的研究表明，在图像和文本交错的自然文档上训练的大型多模态模型在各种基准上优于仅在图像文本对上训练的模型，利用先进的预训练模型来强制语义对齐、图像序列一致性和文本连贯性。然而，对于阿拉伯语来说，缺乏保留文档结构的高质量多模式数据集限制了进展。在本文中，我们介绍了用于处理 Common Crawl 数据集的 Wasm 管道，以创建一个新的阿拉伯多模式数据集，该数据集独特地提供 Markdown 输出。与仅关注文本提取的现有阿拉伯语语料库不同，我们的方法保留了网络内容的结构完整性，同时保持纯文本和多模式预训练场景的灵活性。我们对我们的数据处理管道与用于主要现有数据集的数据处理管道进行了全面的比较分析，突出了过滤策略的收敛性并证明了我们特定设计选择的合理性。为了支持未来的研究，我们公开发布了具有代表性的数据集转储以及阿拉伯语的多模式处理管道。

Title: More Agents Helps but Adversarial Robustness Gap Persists

• Authors: Khashayar Alavi, Zhastay Yeltay, Lucie Flek, Akbar Karimi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07112
• Pdf URL: https://arxiv.org/pdf/2511.07112
• Copy Paste: [[2511.07112]] More Agents Helps but Adversarial Robustness Gap Persists(https://arxiv.org/abs/2511.07112)
• Keywords: llm, agent
• Abstract: When LLM agents work together, they seem to be more powerful than a single LLM in mathematical question answering. However, are they also more robust to adversarial inputs? We investigate this question using adversarially perturbed math questions. These perturbations include punctuation noise with three intensities (10, 30, and 50 percent), plus real-world and human-like typos (WikiTypo, R2ATA). Using a unified sampling-and-voting framework (Agent Forest), we evaluate six open-source models (Qwen3-4B/14B, Llama3.1-8B, Mistral-7B, Gemma3-4B/12B) across four benchmarks (GSM8K, MATH, MMLU-Math, MultiArith), with various numbers of agents n from one to 25 (1, 2, 5, 10, 15, 20, 25). Our findings show that (1) Noise type matters: punctuation noise harm scales with its severity, and the human typos remain the dominant bottleneck, yielding the largest gaps to Clean accuracy and the highest ASR even with a large number of agents. And (2) Collaboration reliably improves accuracy as the number of agents, n, increases, with the largest gains from one to five agents and diminishing returns beyond 10 agents. However, the adversarial robustness gap persists regardless of the agent count.
• 摘要：当LLM代理一起工作时，他们在数学问题回答方面似乎比单个LLM更强大。然而，它们对于对抗性输入是否也更稳健？我们使用对抗性扰动的数学问题来研究这个问题。这些扰动包括三种强度（10%、30% 和 50%）的标点符号噪声，以及现实世界和类似人类的拼写错误（WikiTypo、R2ATA）。使用统一的采样和投票框架（Agent Forest），我们跨四个基准（GSM8K、MATH、MMLU-Math、MultiArith）评估了六种开源模型（Qwen3-4B/14B、Llama3.1-8B、Mistral-7B、Gemma3-4B/12B），代理数量 n 从 1 到 25（1、2、5、10、15、 20、25）。我们的研究结果表明，(1) 噪音类型很重要：标点符号噪音危害与其严重程度成正比，而人为拼写错误仍然是主要瓶颈，即使有大量代理，也会在 Clean 准确性和最高 ASR 方面产生最大差距。 (2) 随着代理数量 n 的增加，协作可靠地提高了准确性，1 到 5 个代理的收益最大，超过 10 个代理的收益递减。然而，无论代理数量如何，对抗鲁棒性差距仍然存在。

Title: Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought

• Authors: Zhikang Chen, Sen Cui, Deheng Ye, Yu Zhang, Yatao Bian, Tingting Zhu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.07124
• Pdf URL: https://arxiv.org/pdf/2511.07124
• Copy Paste: [[2511.07124]] Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought(https://arxiv.org/abs/2511.07124)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Large Language Models (LLMs) have demonstrated strong reasoning capabilities through \emph{Chain-of-Thought} (CoT) prompting, which enables step-by-step intermediate reasoning. However, explicit CoT methods rely on discrete token-level reasoning processes that are prone to error propagation and limited by vocabulary expressiveness, often resulting in rigid and inconsistent reasoning trajectories. Recent research has explored implicit or continuous reasoning in latent spaces, allowing models to perform internal reasoning before generating explicit output. Although such approaches alleviate some limitations of discrete CoT, they generally lack explicit mechanisms to enforce consistency among reasoning steps, leading to divergent reasoning paths and unstable outcomes. To address this issue, we propose EBM-CoT, an Energy-Based Chain-of-Thought Calibration framework that refines latent thought representations through an energy-based model (EBM). Our method dynamically adjusts latent reasoning trajectories toward lower-energy, high-consistency regions in the embedding space, improving both reasoning accuracy and consistency without modifying the base language model. Extensive experiments across mathematical, commonsense, and symbolic reasoning benchmarks demonstrate that the proposed framework significantly enhances the consistency and efficiency of multi-step reasoning in LLMs.
• 摘要：大型语言模型 (LLM) 通过 \emph{Chain-of-Thought} (CoT) 提示展示了强大的推理能力，可实现逐步中间推理。然而，显式 CoT 方法依赖于离散的标记级推理过程，这些过程容易出现错误传播，并且受到词汇表达能力的限制，通常会导致僵化且不一致的推理轨迹。最近的研究探索了潜在空间中的隐式或连续推理，允许模型在生成显式输出之前执行内部推理。尽管此类方法缓解了离散 CoT 的一些限制，但它们通常缺乏明确的机制来强制推理步骤之间的一致性，从而导致推理路径发散和结果不稳定。为了解决这个问题，我们提出了 EBM-CoT，一种基于能量的思想链校准框架，它通过基于能量的模型 (EBM) 细化潜在思想表征。我们的方法动态地将潜在推理轨迹调整到嵌入空间中的低能量、高一致性区域，从而在不修改基本语言模型的情况下提高推理准确性和一致性。数学、常识和符号推理基准的广泛实验表明，所提出的框架显着提高了法学硕士多步推理的一致性和效率。

Title: TCM-Eval: An Expert-Level Dynamic and Extensible Benchmark for Traditional Chinese Medicine

• Authors: Zihao Cheng, Yuheng Lu, Huaiqian Ye, Zeming Liu, Minqi Wang, Jingjing Liu, Zihan Li, Wei Fan, Yuanfang Guo, Ruiji Fu, Shifeng She, Gang Wang, Yunhong Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07148
• Pdf URL: https://arxiv.org/pdf/2511.07148
• Copy Paste: [[2511.07148]] TCM-Eval: An Expert-Level Dynamic and Extensible Benchmark for Traditional Chinese Medicine(https://arxiv.org/abs/2511.07148)
• Keywords: language model, llm, chain-of-thought
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in modern medicine, yet their application in Traditional Chinese Medicine (TCM) remains severely limited by the absence of standardized benchmarks and the scarcity of high-quality training data. To address these challenges, we introduce TCM-Eval, the first dynamic and extensible benchmark for TCM, meticulously curated from national medical licensing examinations and validated by TCM experts. Furthermore, we construct a large-scale training corpus and propose Self-Iterative Chain-of-Thought Enhancement (SI-CoTE) to autonomously enrich question-answer pairs with validated reasoning chains through rejection sampling, establishing a virtuous cycle of data and model co-evolution. Using this enriched training data, we develop ZhiMingTang (ZMT), a state-of-the-art LLM specifically designed for TCM, which significantly exceeds the passing threshold for human practitioners. To encourage future research and development, we release a public leaderboard, fostering community engagement and continuous improvement.
• 摘要：大语言模型（LLM）在现代医学中表现出了卓越的能力，但由于缺乏标准化基准和高质量训练数据的缺乏，其在中医（TCM）中的应用仍然受到严重限制。为了应对这些挑战，我们推出了 TCM-Eval，这是第一个动态且可扩展的中医基准，由国家医疗许可考试精心策划并由中医专家验证。此外，我们构建了一个大规模的训练语料库，并提出了自迭代思维链增强（SI-CoTE），通过拒绝采样，用经过验证的推理链自主丰富问答对，建立数据和模型共同进化的良性循环。利用这些丰富的训练数据，我们开发了智明堂（ZMT），这是一种专为中医设计的最先进的法学硕士，其显着超过了人类从业者的通过门槛。为了鼓励未来的研究和开发，我们发布了公共排行榜，促进社区参与和持续改进。

Title: Categorical Emotions or Appraisals - Which Emotion Model Explains Argument Convincingness Better?

• Authors: Lynn Greschner, Meike Bauer, Sabine Weber, Roman Klinger
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07162
• Pdf URL: https://arxiv.org/pdf/2511.07162
• Copy Paste: [[2511.07162]] Categorical Emotions or Appraisals - Which Emotion Model Explains Argument Convincingness Better?(https://arxiv.org/abs/2511.07162)
• Keywords: prompt
• Abstract: The convincingness of an argument does not only depend on its structure (logos), the person who makes the argument (ethos), but also on the emotion that it causes in the recipient (pathos). While the overall intensity and categorical values of emotions in arguments have received considerable attention in the research community, we argue that the emotion an argument evokes in a recipient is subjective. It depends on the recipient's goals, standards, prior knowledge, and stance. Appraisal theories lend themselves as a link between the subjective cognitive assessment of events and emotions. They have been used in event-centric emotion analysis, but their suitability for assessing argument convincingness remains unexplored. In this paper, we evaluate whether appraisal theories are suitable for emotion analysis in arguments by considering subjective cognitive evaluations of the importance and impact of an argument on its receiver. Based on the annotations in the recently published ContArgA corpus, we perform zero-shot prompting experiments to evaluate the importance of gold-annotated and predicted emotions and appraisals for the assessment of the subjective convincingness labels. We find that, while categorical emotion information does improve convincingness prediction, the improvement is more pronounced with appraisals. This work presents the first systematic comparison between emotion models for convincingness prediction, demonstrating the advantage of appraisals, providing insights for theoretical and practical applications in computational argumentation.
• 摘要：论证的说服力不仅取决于其结构（逻各斯）、提出论证的人（精神），还取决于它在接受者中引起的情感（悲情）。虽然争论中情绪的整体强度和分类值在研究界受到了相当大的关注，但我们认为争论在接受者中引起的情绪是主观的。这取决于接收者的目标、标准、先验知识和立场。评估理论可以作为事件的主观认知评估和情绪之间的联系。它们已被用于以事件为中心的情感分析，但它们评估论证说服力的适用性仍有待探索。在本文中，我们通过考虑对论点对其接受者的重要性和影响的主观认知评估来评估评价理论是否适合论点中的情感分析。基于最近发布的 ContArgA 语料库中的注释，我们进行了零样本提示实验，以评估黄金注释和预测情绪以及评估对于主观说服力标签评估的重要性。我们发现，虽然分类情感信息确实提高了预测的可信度，但评估的改善更为明显。这项工作首次系统地比较了用于可信度预测的情感模型，展示了评估的优势，为计算论证中的理论和实际应用提供了见解。

Title: AdaRec: Adaptive Recommendation with LLMs via Narrative Profiling and Dual-Channel Reasoning

• Authors: Meiyun Wang, Charin Polpanumas
• Subjects: cs.CL, cs.AI, cs.CE
• Abstract URL: https://arxiv.org/abs/2511.07166
• Pdf URL: https://arxiv.org/pdf/2511.07166
• Copy Paste: [[2511.07166]] AdaRec: Adaptive Recommendation with LLMs via Narrative Profiling and Dual-Channel Reasoning(https://arxiv.org/abs/2511.07166)
• Keywords: language model, llm
• Abstract: We propose AdaRec, a few-shot in-context learning framework that leverages large language models for an adaptive personalized recommendation. AdaRec introduces narrative profiling, transforming user-item interactions into natural language representations to enable unified task handling and enhance human readability. Centered on a bivariate reasoning paradigm, AdaRec employs a dual-channel architecture that integrates horizontal behavioral alignment, discovering peer-driven patterns, with vertical causal attribution, highlighting decisive factors behind user preferences. Unlike existing LLM-based approaches, AdaRec eliminates manual feature engineering through semantic representations and supports rapid cross-task adaptation with minimal supervision. Experiments on real ecommerce datasets demonstrate that AdaRec outperforms both machine learning models and LLM-based baselines by up to eight percent in few-shot settings. In zero-shot scenarios, it achieves up to a nineteen percent improvement over expert-crafted profiling, showing effectiveness for long-tail personalization with minimal interaction data. Furthermore, lightweight fine-tuning on synthetic data generated by AdaRec matches the performance of fully fine-tuned models, highlighting its efficiency and generalization across diverse tasks.
• 摘要：我们提出了 AdaRec，这是一个少量的上下文学习框架，它利用大型语言模型进行自适应个性化推荐。 AdaRec 引入了叙事分析，将用户-项目交互转换为自然语言表示，以实现统一的任务处理并增强人类可读性。 AdaRec 以双变量推理范式为中心，采用双通道架构，集成了水平行为对齐、发现同伴驱动模式以及垂直因果归因，突出了用户偏好背后的决定性因素。与现有的基于 LLM 的方法不同，AdaRec 通过语义表示消除了手动特征工程，并支持在最少监督的情况下快速跨任务适应。对真实电子商务数据集的实验表明，AdaRec 在少量样本设置中的性能优于机器学习模型和基于 LLM 的基线高达 8%。在零样本场景中，与专家制作的分析相比，它的性能提高了 19%，显示出以最少的交互数据实现长尾个性化的有效性。此外，对 AdaRec 生成的合成数据进行轻量级微调，与完全微调模型的性能相匹配，突出了其在不同任务中的效率和泛化性。

Title: EMODIS: A Benchmark for Context-Dependent Emoji Disambiguation in Large Language Models

• Authors: Jiacheng Huang, Ning Yu, Xiaoyin Yi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07193
• Pdf URL: https://arxiv.org/pdf/2511.07193
• Copy Paste: [[2511.07193]] EMODIS: A Benchmark for Context-Dependent Emoji Disambiguation in Large Language Models(https://arxiv.org/abs/2511.07193)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are increasingly deployed in real-world communication settings, yet their ability to resolve context-dependent ambiguity remains underexplored. In this work, we present EMODIS, a new benchmark for evaluating LLMs' capacity to interpret ambiguous emoji expressions under minimal but contrastive textual contexts. Each instance in EMODIS comprises an ambiguous sentence containing an emoji, two distinct disambiguating contexts that lead to divergent interpretations, and a specific question that requires contextual reasoning. We evaluate both open-source and API-based LLMs, and find that even the strongest models frequently fail to distinguish meanings when only subtle contextual cues are present. Further analysis reveals systematic biases toward dominant interpretations and limited sensitivity to pragmatic contrast. EMODIS provides a rigorous testbed for assessing contextual disambiguation, and highlights the gap in semantic reasoning between humans and LLMs.
• 摘要：大型语言模型 (LLM) 越来越多地部署在现实世界的通信环境中，但它们解决上下文相关歧义的能力仍未得到充分开发。在这项工作中，我们提出了 EMODIS，这是一个新的基准，用于评估法学硕士在最少但对比鲜明的文本上下文中解释模糊表情符号表达的能力。 EMODIS 中的每个实例都包含一个包含表情符号的歧义句子、两个导致不同解释的不同消歧上下文以及一个需要上下文推理的特定问题。我们评估了开源法学硕士和基于 API 的法学硕士，发现即使是最强大的模型，当仅存在微妙的上下文线索时，也经常无法区分含义。进一步的分析揭示了对主导解释的系统偏见和对实用对比的敏感性有限。 EMODIS 提供了一个严格的测试平台来评估上下文消歧，并强调了人类和法学硕士之间在语义推理方面的差距。

Title: Discourse Graph Guided Document Translation with Large Language Models

• Authors: Viet-Thanh Pham, Minghan Wang, Hao-Han Liao, Thuy-Trang Vu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07230
• Pdf URL: https://arxiv.org/pdf/2511.07230
• Copy Paste: [[2511.07230]] Discourse Graph Guided Document Translation with Large Language Models(https://arxiv.org/abs/2511.07230)
• Keywords: language model, agent
• Abstract: Adapting large language models to full document translation remains challenging due to the difficulty of capturing long-range dependencies and preserving discourse coherence throughout extended texts. While recent agentic machine translation systems mitigate context window constraints through multi-agent orchestration and persistent memory, they require substantial computational resources and are sensitive to memory retrieval strategies. We introduce TransGraph, a discourse-guided framework that explicitly models inter-chunk relationships through structured discourse graphs and selectively conditions each translation segment on relevant graph neighbourhoods rather than relying on sequential or exhaustive context. Across three document-level MT benchmarks spanning six languages and diverse domains, TransGraph consistently surpasses strong baselines in translation quality and terminology consistency while incurring significantly lower token overhead.
• 摘要：由于难以捕获长程依赖关系并在整个扩展文本中保持话语连贯性，因此使大型语言模型适应完整文档翻译仍然具有挑战性。虽然最近的代理机器翻译系统通过多代理编排和持久内存减轻了上下文窗口限制，但它们需要大量的计算资源并且对内存检索策略敏感。我们引入了 TransGraph，这是一种话语引导框架，它通过结构化话语图显式地建模块间关系，并选择性地在相关图邻域上调节每个翻译片段，而不是依赖于顺序或详尽的上下文。在跨越六种语言和不同领域的三个文档级机器翻译基准中，TransGraph 始终超越翻译质量和术语一致性方面的强大基线，同时显着降低标记开销。

Title: Who Is the Story About? Protagonist Entity Recognition in News

• Authors: Jorge Gabín, M. Eduardo Ares, Javier Parapar
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07296
• Pdf URL: https://arxiv.org/pdf/2511.07296
• Copy Paste: [[2511.07296]] Who Is the Story About? Protagonist Entity Recognition in News(https://arxiv.org/abs/2511.07296)
• Keywords: language model, llm, prompt
• Abstract: News articles often reference numerous organizations, but traditional Named Entity Recognition (NER) treats all mentions equally, obscuring which entities genuinely drive the narrative. This limits downstream tasks that rely on understanding event salience, influence, or narrative focus. We introduce Protagonist Entity Recognition (PER), a task that identifies the organizations that anchor a news story and shape its main developments. To validate PER, we compare he predictions of Large Language Models (LLMs) against annotations from four expert annotators over a gold corpus, establishing both inter-annotator consistency and human-LLM agreement. Leveraging these findings, we use state-of-the-art LLMs to automatically label large-scale news collections through NER-guided prompting, generating scalable, high-quality supervision. We then evaluate whether other LLMs, given reduced context and without explicit candidate guidance, can still infer the correct protagonists. Our results demonstrate that PER is a feasible and meaningful extension to narrative-centered information extraction, and that guided LLMs can approximate human judgments of narrative importance at scale.
• 摘要：新闻文章通常会引用众多组织，但传统的命名实体识别 (NER) 对所有提及的内容一视同仁，从而模糊了哪些实体真正推动了叙述。这限制了依赖于理解事件显着性、影响力或叙述焦点的下游任务。我们引入主角实体识别（PER），这项任务可以识别锚定新闻报道并塑造其主要发展的组织。为了验证 PER，我们将大型语言模型 (LLM) 的预测与四位专家注释者在黄金语料库上的注释进行比较，建立注释者间的一致性和人类与 LLM 的一致性。利用这些发现，我们使用最先进的法学硕士通过 NER 引导的提示自动标记大规模新闻集合，从而生成可扩展的高质量监督。然后，我们评估其他法学硕士在减少背景且没有明确候选人指导的情况下是否仍然可以推断出正确的主角。我们的结果表明，PER 是对以叙事为中心的信息提取的可行且有意义的扩展，并且引导法学硕士可以大规模地近似人类对叙事重要性的判断。

Title: ACE-ICD: Acronym Expansion As Data Augmentation For Automated ICD Coding

• Authors: Tuan-Dung Le, Shohreh Haddadan, Thanh Q. Thieu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07311
• Pdf URL: https://arxiv.org/pdf/2511.07311
• Copy Paste: [[2511.07311]] ACE-ICD: Acronym Expansion As Data Augmentation For Automated ICD Coding(https://arxiv.org/abs/2511.07311)
• Keywords: language model
• Abstract: Automatic ICD coding, the task of assigning disease and procedure codes to electronic medical records, is crucial for clinical documentation and billing. While existing methods primarily enhance model understanding of code hierarchies and synonyms, they often overlook the pervasive use of medical acronyms in clinical notes, a key factor in ICD code inference. To address this gap, we propose a novel effective data augmentation technique that leverages large language models to expand medical acronyms, allowing models to be trained on their full form representations. Moreover, we incorporate consistency training to regularize predictions by enforcing agreement between the original and augmented documents. Extensive experiments on the MIMIC-III dataset demonstrate that our approach, ACE-ICD establishes new state-of-the-art performance across multiple settings, including common codes, rare codes, and full-code assignments. Our code is publicly available.
• 摘要：自动 ICD 编码是将疾病和程序代码分配给电子病历的任务，对于临床记录和计费至关重要。虽然现有方法主要增强对代码层次结构和同义词的模型理解，但它们经常忽视临床笔记中医学缩略词的普遍使用，而这是 ICD 代码推断的关键因素。为了解决这一差距，我们提出了一种新颖有效的数据增强技术，该技术利用大型语言模型来扩展医学缩略词，从而允许模型在其完整形式表示上进行训练。此外，我们还结合一致性训练，通过强制原始文档和增强文档之间的一致性来规范预测。 MIMIC-III 数据集上的大量实验表明，我们的方法 ACE-ICD 在多种设置（包括常见代码、稀有代码和全代码分配）中建立了新的最先进的性能。我们的代码是公开的。

Title: RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments

• Authors: Zhiyuan Zeng, Hamish Ivison, Yiping Wang, Lifan Yuan, Shuyue Stella Li, Zhuorui Ye, Siting Li, Jacqueline He, Runlong Zhou, Tong Chen, Chenyang Zhao, Yulia Tsvetkov, Simon Shaolei Du, Natasha Jaques, Hao Peng, Pang Wei Koh, Hannaneh Hajishirzi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.07317
• Pdf URL: https://arxiv.org/pdf/2511.07317
• Copy Paste: [[2511.07317]] RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments(https://arxiv.org/abs/2511.07317)
• Keywords: language model
• Abstract: We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scale up RL for language models (LMs). RLVE enables each verifiable environment to dynamically adapt its problem difficulty distribution to the policy model's capabilities as training progresses. In contrast, static data distributions often lead to vanishing learning signals when problems are either too easy or too hard for the policy. To implement RLVE, we create RLVE-Gym, a large-scale suite of 400 verifiable environments carefully developed through manual environment engineering. Using RLVE-Gym, we show that environment scaling, i.e., expanding the collection of training environments, consistently improves generalizable reasoning capabilities. RLVE with joint training across all 400 environments in RLVE-Gym yields a 3.37% absolute average improvement across six reasoning benchmarks, starting from one of the strongest 1.5B reasoning LMs. By comparison, continuing this LM's original RL training yields only a 0.49% average absolute gain despite using over 3x more compute. We release our code publicly.
• 摘要：我们引入了带有自适应可验证环境 (RLVE) 的强化学习 (RL)，这是一种使用可验证环境的方法，可在程序上生成问题并提供算法上可验证的奖励，以扩展语言模型 (LM) 的 RL。 RLVE 使每个可验证环境能够随着训练的进行动态调整其问题难度分布以适应策略模型的功能。相比之下，当问题对于策略来说太简单或太难时，静态数据分布通常会导致学习信号消失。为了实施 RLVE，我们创建了 RLVE-Gym，这是一个通过手动环境工程精心开发的包含 400 个可验证环境的大型套件。使用 RLVE-Gym，我们表明环境扩展（即扩展训练环境的集合）可以持续提高泛化推理能力。从最强的 1.5B 推理 LM 之一开始，RLVE 在 RLVE-Gym 中的所有 400 个环境中进行联合训练，在六个推理基准上产生了 3.37% 的绝对平均改进。相比之下，尽管使用了超过 3 倍的计算量，但继续该 LM 的原始 RL 训练仅产生 0.49% 的平均绝对增益。我们公开发布我们的代码。

Title: When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs

• Authors: Shaowen Wang, Yiqi Dong, Ruinian Chang, Tansheng Zhu, Yuebo Sun, Kaifeng Lyu, Jian Li
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.07318
• Pdf URL: https://arxiv.org/pdf/2511.07318
• Copy Paste: [[2511.07318]] When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs(https://arxiv.org/abs/2511.07318)
• Keywords: language model, gpt, llm, hallucination
• Abstract: Despite substantial advances, large language models (LLMs) continue to exhibit hallucinations, generating plausible yet incorrect responses. In this paper, we highlight a critical yet previously underexplored class of hallucinations driven by spurious correlations -- superficial but statistically prominent associations between features (e.g., surnames) and attributes (e.g., nationality) present in the training data. We demonstrate that these spurious correlations induce hallucinations that are confidently generated, immune to model scaling, evade current detection methods, and persist even after refusal fine-tuning. Through systematically controlled synthetic experiments and empirical evaluations on state-of-the-art open-source and proprietary LLMs (including GPT-5), we show that existing hallucination detection methods, such as confidence-based filtering and inner-state probing, fundamentally fail in the presence of spurious correlations. Our theoretical analysis further elucidates why these statistical biases intrinsically undermine confidence-based detection techniques. Our findings thus emphasize the urgent need for new approaches explicitly designed to address hallucinations caused by spurious correlations.
• 摘要：尽管取得了巨大的进步，大型语言模型（LLM）仍然表现出幻觉，产生看似合理但不正确的反应。在本文中，我们强调了由虚假相关性驱动的一类关键但之前未被充分探索的幻觉——训练数据中存在的特征（例如姓氏）和属性（例如国籍）之间表面但统计上显着的关联。我们证明，这些虚假相关性会诱发幻觉，这些幻觉是自信地产生的，不受模型缩放的影响，逃避当前的检测方法，甚至在拒绝微调后仍然存在。通过对最先进的开源和专有 LLM（包括 GPT-5）进行系统控制的综合实验和实证评估，我们表明现有的幻觉检测方法（例如基于置信度的过滤和内部状态探测）在存在虚假相关性的情况下从根本上失败。我们的理论分析进一步阐明了为什么这些统计偏差本质上破坏了基于置信度的检测技术。因此，我们的研究结果强调迫切需要明确设计的新方法来解决由虚假相关性引起的幻觉。

Title: FinRpt: Dataset, Evaluation System and LLM-based Multi-agent Framework for Equity Research Report Generation

• Authors: Song Jin, Shuqi Li, Shukun Zhang, Rui Yan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07322
• Pdf URL: https://arxiv.org/pdf/2511.07322
• Copy Paste: [[2511.07322]] FinRpt: Dataset, Evaluation System and LLM-based Multi-agent Framework for Equity Research Report Generation(https://arxiv.org/abs/2511.07322)
• Keywords: llm, agent
• Abstract: While LLMs have shown great success in financial tasks like stock prediction and question answering, their application in fully automating Equity Research Report generation remains uncharted territory. In this paper, we formulate the Equity Research Report (ERR) Generation task for the first time. To address the data scarcity and the evaluation metrics absence, we present an open-source evaluation benchmark for ERR generation - FinRpt. We frame a Dataset Construction Pipeline that integrates 7 financial data types and produces a high-quality ERR dataset automatically, which could be used for model training and evaluation. We also introduce a comprehensive evaluation system including 11 metrics to assess the generated ERRs. Moreover, we propose a multi-agent framework specifically tailored to address this task, named FinRpt-Gen, and train several LLM-based agents on the proposed datasets using Supervised Fine-Tuning and Reinforcement Learning. Experimental results indicate the data quality and metrics effectiveness of the benchmark FinRpt and the strong performance of FinRpt-Gen, showcasing their potential to drive innovation in the ERR generation field. All code and datasets are publicly available.
• 摘要：虽然法学硕士在股票预测和问答等金融任务中取得了巨大成功，但它们在全自动股票研究报告生成方面的应用仍然是未知领域。在本文中，我们首次制定了股票研究报告（ERR）生成任务。为了解决数据稀缺和评估指标缺失的问题，我们提出了一个用于 ERR 生成的开源评估基准 - FinRpt。我们构建了一个数据集构建管道，集成了 7 种金融数据类型，并自动生成高质量的 ERR 数据集，可用于模型训练和评估。我们还引入了一个全面的评估系统，包括 11 个指标来评估生成的 ERR。此外，我们提出了一个专门为解决此任务而定制的多智能体框架，名为 FinRpt-Gen，并使用监督微调和强化学习在所提出的数据集上训练多个基于 LLM 的智能体。实验结果表明基准 FinRpt 的数据质量和指标有效性以及 FinRpt-Gen 的强劲性能，展示了它们推动 ERR 生成领域创新的潜力。所有代码和数据集都是公开的。

Title: Selecting Auxiliary Data via Neural Tangent Kernels for Low-Resource Domains

• Authors: Pingjie Wang, Hongcheng Liu, Yusheng Liao, Ziqing Fan, Yaxin Du, Shuo Tang, Yanfeng Wang, Yu Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07380
• Pdf URL: https://arxiv.org/pdf/2511.07380
• Copy Paste: [[2511.07380]] Selecting Auxiliary Data via Neural Tangent Kernels for Low-Resource Domains(https://arxiv.org/abs/2511.07380)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) have achieved remarkable success across widespread tasks, yet their application in low-resource domains remains a significant challenge due to data scarcity and the high risk of overfitting. While in-domain data is limited, there exist vast amounts of similar general-domain data, and our initial findings reveal that they could potentially serve as auxiliary supervision for domain enhancement. This observation leads us to our central research question: \textbf{\textit{how to effectively select the most valuable auxiliary data to maximize domain-specific performance}}, particularly when traditional methods are inapplicable due to a lack of large in-domain data pools or validation sets. To address this, we propose \textbf{NTK-Selector}, a principled and efficient framework for selecting general-domain auxiliary data to enhance domain-specific performance via neural tangent kernels (NTK). Our method tackles two challenges of directly applying NTK to LLMs, theoretical assumptions and prohibitive computational cost, by empirically demonstrating a stable NTK-like behavior in LLMs during LoRA fine-tuning and proposing a Jacobian-free approximation method. Extensive experiments across four low-resource domains (medical, financial, legal, and psychological) demonstrate that NTK-Selector consistently improves downstream performance. Specifically, fine-tuning on 1,000 in-domain samples alone only yielded +0.8 points for Llama3-8B-Instruct and +0.9 points for Qwen3-8B. In contrast, enriching with 9,000 auxiliary samples selected by NTK-Selector led to substantial \textbf{gains of +8.7 and +5.1 points}, which corresponds to a \textbf{10.9x and 5.7x improvement} over the domain-only setting.
• 摘要：大型语言模型（LLM）在广泛的任务中取得了显着的成功，但由于数据稀缺和过度拟合的高风险，它们在资源匮乏领域的应用仍然是一个重大挑战。虽然领域内数据有限，但存在大量类似的通用领域数据，我们的初步研究结果表明，它们有可能作为领域增强的辅助监督。这一观察引出了我们的中心研究问题：\textbf{\textit{如何有效地选择最有价值的辅助数据以最大化特定领域的性能}}，特别是当传统方法由于缺乏大型域内数据池或验证集而不适用时。为了解决这个问题，我们提出了 \textbf{NTK-Selector}，这是一个有原则且有效的框架，用于选择通用域辅助数据，以通过神经切线核（NTK）增强特定域的性能。我们的方法通过在 LoRA 微调期间实证证明 LLM 中稳定的类似 NTK 的行为并提出一种无雅可比近似方法，解决了将 NTK 直接应用于 LLM 的两个挑战：理论假设和高昂的计算成本。跨越四个低资源领域（医学、金融、法律和心理）的广泛实验表明，NTK-Selector 持续提高下游性能。具体来说，仅对 1,000 个域内样本进行微调，Llama3-8B-Instruct 的得分仅为 +0.8 分，Qwen3-8B 的得分为 +0.9 分。相比之下，使用 NTK-Selector 选择的 9,000 个辅助样本进行丰富导致了 \textbf{+8.7 和 +5.1 点的增益}，这对应于仅域设置的 \textbf{10.9 倍和 5.7 倍的改进}。

Title: Retriv at BLP-2025 Task 2: Test-Driven Feedback-Guided Framework for Bangla-to-Python Code Generation

• Authors: K M Nafi Asib, Sourav Saha, Mohammed Moshiul Hoque
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07382
• Pdf URL: https://arxiv.org/pdf/2511.07382
• Copy Paste: [[2511.07382]] Retriv at BLP-2025 Task 2: Test-Driven Feedback-Guided Framework for Bangla-to-Python Code Generation(https://arxiv.org/abs/2511.07382)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) have advanced the automated generation of code from natural language prompts. However, low-resource languages (LRLs) like Bangla remain underrepresented due to the limited availability of instruction-to-code datasets and evaluation benchmarks. To address this, the BLP Workshop at IJCNLP-AACL 2025 introduced a shared task on "Code Generation in Bangla". In this work, we propose a method that combines instruction prompting with a test-driven, feedback-guided iterative refinement process using a fine-tuned Qwen2.5-14B model. The model generates code from Bangla instructions, tests it against unit tests, and iteratively refines any failing outputs through three evaluation passes, using test feedback to guide each step. This approach helped our team "Retriv" to secure 2nd place in the shared task with a Pass@1 score of 0.934. The analysis highlights challenges in Bangla instruction understanding and Python code generation, emphasizing the need for targeted methods in LRLs. We made experimental scripts publicly available for the community.
• 摘要：大型语言模型 (LLM) 促进了根据自然语言提示自动生成代码。然而，由于指令到代码数据集和评估基准的可用性有限，像孟加拉语这样的低资源语言 (LRL) 的代表性仍然不足。为了解决这个问题，IJCNLP-AACL 2025 上的 BLP 研讨会引入了一项关于“孟加拉代码生成”的共享任务。在这项工作中，我们提出了一种方法，使用微调的 Qwen2.5-14B 模型将指令提示与测试驱动、反馈引导的迭代细化过程相结合。该模型根据 Bangla 指令生成代码，根据单元测试对其进行测试，并通过三个评估过程迭代地完善任何失败的输出，并使用测试反馈来指导每个步骤。这种方法帮助我们的团队“Retriv”以 0.934 的 Pass@1 分数在共享任务中获得第二名。该分析强调了 Bangla 指令理解和 Python 代码生成方面的挑战，强调 LRL 中需要有针对性的方法。我们向社区公开提供实验脚本。

Title: Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence

• Authors: Sean McLeish, Ang Li, John Kirchenbauer, Dayal Singh Kalra, Brian R. Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Jonas Geiping, Tom Goldstein, Micah Goldblum
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.07384
• Pdf URL: https://arxiv.org/pdf/2511.07384
• Copy Paste: [[2511.07384]] Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence(https://arxiv.org/abs/2511.07384)
• Keywords: language model
• Abstract: Recent advances in depth-recurrent language models show that recurrence can decouple train-time compute and parameter count from test-time compute. In this work, we study how to convert existing pretrained non-recurrent language models into depth-recurrent models. We find that using a curriculum of recurrences to increase the effective depth of the model over the course of training preserves performance while reducing total computational cost. In our experiments, on mathematics, we observe that converting pretrained models to recurrent ones results in better performance at a given compute budget than simply post-training the original non-recurrent language model.
• 摘要：深度循环语言模型的最新进展表明，循环可以将训练时计算和参数计数与测试时计算分离开来。在这项工作中，我们研究如何将现有的预训练非循环语言模型转换为深度循环模型。我们发现，在训练过程中使用递归课程来增加模型的有效深度可以保持性能，同时降低总计算成本。在我们的数学实验中，我们观察到，在给定的计算预算下，将预训练模型转换为循环模型比简单地对原始非循环语言模型进行后训练会带来更好的性能。

Title: Surgical Agent Orchestration Platform for Voice-directed Patient Data Interaction

• Authors: Hyeryun Park, Byung Mo Gu, Jun Hee Lee, Byeong Hyeon Choi, Sekeun Kim, Hyun Koo Kim, Kyungsang Kim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.07392
• Pdf URL: https://arxiv.org/pdf/2511.07392
• Copy Paste: [[2511.07392]] Surgical Agent Orchestration Platform for Voice-directed Patient Data Interaction(https://arxiv.org/abs/2511.07392)
• Keywords: language model, llm, agent
• Abstract: In da Vinci robotic surgery, surgeons' hands and eyes are fully engaged in the procedure, making it difficult to access and manipulate multimodal patient data without interruption. We propose a voice-directed Surgical Agent Orchestrator Platform (SAOP) built on a hierarchical multi-agent framework, consisting of an orchestration agent and three task-specific agents driven by Large Language Models (LLMs). These LLM-based agents autonomously plan, refine, validate, and reason to map voice commands into specific tasks such as retrieving clinical information, manipulating CT scans, or navigating 3D anatomical models on the surgical video. We also introduce a Multi-level Orchestration Evaluation Metric (MOEM) to comprehensively assess the performance and robustness from command-level and category-level perspectives. The SAOP achieves high accuracy and success rates across 240 voice commands, while LLM-based agents improve robustness against speech recognition errors and diverse or ambiguous free-form commands, demonstrating strong potential to support minimally invasive da Vinci robotic surgery.
• 摘要：在达芬奇机器人手术中，外科医生的手和眼睛完全参与手术，因此很难不间断地访问和操作多模式患者数据。我们提出了一个基于分层多代理框架构建的语音指导的手术代理编排平台（SAOP），由一个编排代理和三个由大型语言模型（LLM）驱动的特定任务代理组成。这些基于法学硕士的代理可以自主规划、完善、验证和推理，将语音命令映射到特定任务，例如检索临床信息、操作 CT 扫描或在手术视频上导航 3D 解剖模型。我们还引入了多级编排评估指标（MOEM），从命令级和类别级角度全面评估性能和鲁棒性。 SAOP 在 240 个语音命令中实现了高精度和成功率，而基于 LLM 的代理提高了针对语音识别错误和多样化或模糊的自由格式命令的鲁棒性，展示了支持微创达芬奇机器人手术的强大潜力。

Title: ConvFill: Model Collaboration for Responsive Conversational Voice Agents

• Authors: Vidya Srinivas, Zachary Englhardt, Maximus Powers, Shwetak Patel, Vikram Iyer
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.07397
• Pdf URL: https://arxiv.org/pdf/2511.07397
• Copy Paste: [[2511.07397]] ConvFill: Model Collaboration for Responsive Conversational Voice Agents(https://arxiv.org/abs/2511.07397)
• Keywords: language model, agent
• Abstract: Deploying conversational voice agents with large language models faces a critical challenge: cloud-based foundation models provide deep reasoning and domain knowledge but introduce latency that disrupts natural conversation, while on-device models respond immediately but lack sophistication. We propose conversational infill, a task where a lightweight on-device model generates contextually appropriate dialogue while seamlessly incorporating streaming knowledge from a powerful backend model. This approach decouples response latency from model capability, enabling systems that feel responsive while accessing the full power of large-scale models. We present ConvFill, a 360M parameter model trained on synthetic multi-domain conversations. Evaluation across multiple backend models shows that conversational infill can be successfully learned, with ConvFill achieving accuracy improvements of 36-42% over standalone small models of the same size while consistently retaining sub-200ms response latencies. Our results demonstrate the promise of this approach for building on-device conversational agents that are both immediately responsive and knowledgeable.
• 摘要：部署具有大型语言模型的会话语音代理面临着严峻的挑战：基于云的基础模型提供了深入的推理和领域知识，但会引入中断自然对话的延迟，而设备上的模型可以立即响应，但缺乏复杂性。我们提出对话填充，这是一项轻量级设备上模型生成上下文适当的对话，同时无缝合并来自强大后端模型的流知识的任务。这种方法将响应延迟与模型功能解耦，使系统能够在访问大型模型的全部功能的同时感受到响应能力。我们提出了 ConvFill，一个在合成多域对话上训练的 360M 参数模型。对多个后端模型的评估表明，可以成功学习对话填充，与相同大小的独立小型模型相比，ConvFill 的准确性提高了 36-42%，同时始终保持低于 200 毫秒的响应延迟。我们的结果证明了这种方法对于构建立即响应且知识渊博的设备上对话代理的前景。

Title: SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations

• Authors: Manon Berriche, Célia Nouri, Chloé Clavel, Jean-Philippe Cointet
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2511.07405
• Pdf URL: https://arxiv.org/pdf/2511.07405
• Copy Paste: [[2511.07405]] SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations(https://arxiv.org/abs/2511.07405)
• Keywords: llm, prompt
• Abstract: We introduce SPOT (Stopping Points in Online Threads), the first annotated corpus translating the sociological concept of stopping point into a reproducible NLP task. Stopping points are ordinary critical interventions that pause or redirect online discussions through a range of forms (irony, subtle doubt or fragmentary arguments) that frameworks like counterspeech or social correction often overlook. We operationalize this concept as a binary classification task and provide reliable annotation guidelines. The corpus contains 43,305 manually annotated French Facebook comments linked to URLs flagged as false information by social media users, enriched with contextual metadata (article, post, parent comment, page or group, and source). We benchmark fine-tuned encoder models (CamemBERT) and instruction-tuned LLMs under various prompting strategies. Results show that fine-tuned encoders outperform prompted LLMs in F1 score by more than 10 percentage points, confirming the importance of supervised learning for emerging non-English social media tasks. Incorporating contextual metadata further improves encoder models F1 scores from 0.75 to 0.78. We release the anonymized dataset, along with the annotation guidelines and code in our code repository, to foster transparency and reproducible research.
• 摘要：我们引入了 SPOT（在线线程中的停止点），这是第一个将停止点的社会学概念转化为可重现的 NLP 任务的带注释语料库。停止点是普通的批判性干预措施，通过一系列形式（讽刺、微妙的怀疑或零碎的论点）暂停或重定向在线讨论，而反言论或社会矫正等框架经常忽视这些形式。我们将这个概念作为二元分类任务来操作，并提供可靠的注释指南。该语料库包含 43,305 条手动注释的法语 Facebook 评论，这些评论链接到被社交媒体用户标记为虚假信息的 URL，并通过上下文元数据（文章、帖子、家长评论、页面或群组以及来源）进行了丰富。我们在各种提示策略下对微调编码器模型 (CamemBERT) 和指令调整的 LLM 进行基准测试。结果表明，经过微调的编码器的表现优于促使法学硕士的 F1 分数超过 10 个百分点，这证实了监督学习对于新兴非英语社交媒体任务的重要性。结合上下文元数据进一步将编码器模型的 F1 分数从 0.75 提高到 0.78。我们发布匿名数据集以及代码存储库中的注释指南和代码，以促进透明度和可重复的研究。