2025-12-01

Title: Cacheback: Speculative Decoding With Nothing But Cache

• Authors: Zhiyao Ma, In Gim, Lin Zhong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21699
• Pdf URL: https://arxiv.org/pdf/2511.21699
• Copy Paste: [[2511.21699]] Cacheback: Speculative Decoding With Nothing But Cache(https://arxiv.org/abs/2511.21699)
• Keywords: language model, llm
• Abstract: We present Cacheback Decoding, a training-free and model-agnostic speculative decoding method that exploits the locality in language to accelerate Large Language Model (LLM) inference. Cacheback leverages only Least Recently Used (LRU) cache tables of token n-grams to generate draft sequences. Cacheback achieves state-of-the-art performance among comparable methods despite its minimalist design, and its simplicity allows easy integration into existing systems. Cacheback also shows potential for fast adaptation to new domains.
• 摘要：我们提出了缓存回解码，这是一种免训练且与模型无关的推测解码方法，它利用语言的局部性来加速大型语言模型 (LLM) 推理。缓存回退仅利用 token n-gram 的最近最少使用 (LRU) 缓存表来生成草稿序列。尽管采用极简设计，Cacheback 在同类方法中实现了最先进的性能，而且其简单性允许轻松集成到现有系统中。缓存回退还显示出快速适应新领域的潜力。

Title: JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction

• Authors: Yuhao Zhan, Yuqing Zhang, Jing Yuan, Qixiang Ma, Zhiqi Yang, Yu Gu, Zemin Liu, Fei Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.21700
• Pdf URL: https://arxiv.org/pdf/2511.21700
• Copy Paste: [[2511.21700]] JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction(https://arxiv.org/abs/2511.21700)
• Keywords: llm
• Abstract: Existing Grammatical Error Correction (GEC) systems suffer from limited reference diversity, leading to underestimated evaluation and restricted model generalization. To address this issue, we introduce the Judge of Edit-Level Validity (JELV), an automated framework to validate correction edits from grammaticality, faithfulness, and fluency. Using our proposed human-annotated Pair-wise Edit-level Validity Dataset (PEVData) as benchmark, JELV offers two implementations: a multi-turn LLM-as-Judges pipeline achieving 90% agreement with human annotators, and a distilled DeBERTa classifier with 85% precision on valid edits. We then apply JELV to reclassify misjudged false positives in evaluation and derive a comprehensive evaluation metric by integrating false positive decoupling and fluency scoring, resulting in state-of-the-art correlation with human judgments. We also apply JELV to filter LLM-generated correction candidates, expanding the BEA19's single-reference dataset containing 38,692 source sentences. Retraining top GEC systems on this expanded dataset yields measurable performance gains. JELV provides a scalable solution for enhancing reference diversity and strengthening both evaluation and model generalization.
• 摘要：现有的语法错误纠正（GEC）系统的参考多样性有限，导致评估被低估和模型泛化受到限制。为了解决这个问题，我们引入了编辑级有效性判断器（JELV），这是一个自动化框架，用于从语法、忠实性和流畅性方面验证更正编辑。使用我们提出的人工注释配对编辑级有效性数据集 (PEVData) 作为基准，JELV 提供了两种实现：多轮 LLM-as-Judges 管道，与人类注释者达到 90% 的一致性，以及在有效编辑上具有 85% 精度的精炼 DeBERTa 分类器。然后，我们应用 JELV 对评估中误判的误报进行重新分类，并通过集成误报解耦和流畅性评分来得出综合评估指标，从而与人类判断产生最先进的相关性。我们还应用 JELV 来过滤 LLM 生成的校正候选，扩展了包含 38,692 个源句子的 BEA19 的单参考数据集。在这个扩展的数据集上重新训练顶级 GEC 系统可以产生可衡量的性能提升。 JELV 提供了一个可扩展的解决方案，用于增强参考多样性并加强评估和模型泛化。

Title: 47B Mixture-of-Experts Beats 671B Dense Models on Chinese Medical Examinations

• Authors: Chiung-Yi Tseng, Danyang Zhang, Tianyang Wang, Hongying Luo, Lu Chen, Junming Huang, Jibin Guan, Junfeng Hao, Junhao Song, Ziqian Bi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21701
• Pdf URL: https://arxiv.org/pdf/2511.21701
• Copy Paste: [[2511.21701]] 47B Mixture-of-Experts Beats 671B Dense Models on Chinese Medical Examinations(https://arxiv.org/abs/2511.21701)
• Keywords: language model, llm, prompt
• Abstract: The rapid advancement of large language models(LLMs) has prompted significant interest in their potential applications in medical domains. This paper presents a comprehensive benchmark evaluation of 27 state-of-the-art LLMs on Chinese medical examination questions, encompassing seven medical specialties across two professional levels. We introduce a robust evaluation framework that assesses model performance on 2,800 carefully curated questions from cardiovascular, gastroenterology, hematology, infectious diseases, nephrology, neurology, and respiratory medicine domains. Our dataset distinguishes between attending physician and senior physician difficulty levels, providing nuanced insights into model capabilities across varying complexity. Our empirical analysis reveals substantial performance variations among models, with Mixtral-8x7B achieving the highest overall accuracy of 74.25%, followed by DeepSeek-R1-671B at 64.07%. Notably, we observe no consistent correlation between model size and performance, as evidenced by the strong performance of smaller mixture-of-experts architectures. The evaluation demonstrates significant performance gaps between medical specialties, with models generally performing better on cardiovascular and neurology questions compared to gastroenterology and nephrology domains. Furthermore, our analysis indicates minimal performance degradation between attending and senior physician levels for top-performing models, suggesting robust generalization capabilities. This benchmark provides critical insights for the deployment of LLMs in medical education and clinical decision support systems, highlighting both the promise and current limitations of these technologies in specialized medical contexts.
• 摘要：大型语言模型（LLM）的快速发展引起了人们对其在医学领域的潜在应用的极大兴趣。本文对 27 名最先进的法学硕士的中国医学考试问题进行了全面的基准评估，涵盖两个专业级别的七个医学专业。我们引入了一个强大的评估框架，用于评估模型在心血管、胃肠病学、血液学、传染病、肾脏病学、神经病学和呼吸医学领域精心策划的 2,800 个问题上的性能。我们的数据集区分了主治医师和高级医师的难度级别，提供了对不同复杂性的模型功能的细致入微的见解。我们的实证分析揭示了模型之间的巨大性能差异，Mixtral-8x7B 的总体准确率最高，为 74.25%，其次是 DeepSeek-R1-671B，为 64.07%。值得注意的是，我们观察到模型大小和性能之间没有一致的相关性，较小的专家混合架构的强大性能就证明了这一点。该评估表明医学专业之间存在显着的性能差距，与胃肠病学和肾脏病学领域相比，模型在心血管和神经病学问题上的表现通常更好。此外，我们的分析表明，对于表现最佳的模型，主治医师级别和高级医师级别之间的性能下降最小，这表明具有强大的泛化能力。该基准为法学硕士在医学教育和临床决策支持系统中的部署提供了重要的见解，强调了这些技术在专业医疗环境中的前景和当前的局限性。

Title: CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference

• Authors: Dong Liu, Yanxuan Yu, Ben Lengerich
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21702
• Pdf URL: https://arxiv.org/pdf/2511.21702
• Copy Paste: [[2511.21702]] CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference(https://arxiv.org/abs/2511.21702)
• Keywords: language model
• Abstract: Large language models face significant computational bottlenecks during inference due to the expensive output layer computation over large vocabularies. We present CSV-Decode, a novel approach that uses geometric upper bounds to construct small sub-vocabularies for each decoding step, enabling efficient sparse computation while maintaining dual correctness guarantees: exact top-$k$ certification and $\varepsilon$-certified softmax approximations. Our method clusters vocabulary embeddings offline and uses centroid-plus-radius bounds to identify which tokens can be safely omitted from computation. We provide a complete system implementation with sparse GEMV kernels, multi-GPU sharding, and CUDA Graph optimization. Experimental results demonstrate significant speedup over full vocabulary decoding while maintaining distributional guarantees and low fallback rates. Our code implementation available at \href{this https URL}{this https URL}.
• 摘要：由于大型词汇表上的输出层计算成本高昂，大型语言模型在推理过程中面临着严重的计算瓶颈。我们提出了 CSV-Decode，这是一种新颖的方法，它使用几何上限为每个解码步骤构建小型子词汇表，从而实现高效的稀疏计算，同时保持双重正确性保证：精确的 top-$k$ 认证和 $\varepsilon$ 认证的 softmax 近似。我们的方法对离线词汇嵌入进行聚类，并使用质心加半径边界来识别哪些标记可以在计算中安全地省略。我们提供了一个完整的系统实现，包括稀疏 GEMV 内核、多 GPU 分片和 CUDA 图形优化。实验结果表明，在保持分布保证和低回退率的同时，比全词汇解码显着加速。我们的代码实现位于 \href{this https URL}{this https URL}。

Title: Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry

• Authors: Siyaxolisa Kabane
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21703
• Pdf URL: https://arxiv.org/pdf/2511.21703
• Copy Paste: [[2511.21703]] Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry(https://arxiv.org/abs/2511.21703)
• Keywords: language model, llm
• Abstract: We investigate the generalization properties of dense text embeddings when the embedding backbone is a large language model (LLM) versus when it is a non-LLM encoder, and we study the extent to which spherical linear interpolation (SLERP) model-merging mitigates over-specialization introduced by task-specific adaptation (e.g., LoRA). To make the comparison concrete and domain-agnostic, we design a controlled suite of experiments in which models embed short numerical sequences and are evaluated on their ability to cluster and classify those sequences according to well-defined number-theoretic properties. Our experimental protocol compares four families of models: (1) non-LLM encoders trained from scratch or fine-tuned for embeddings, (2) LLM-based encoders adapted with parameter-efficient methods (LoRA), (3) LLM-based encoders with LoRA followed by model souping merging into the base weights, and (4) the same LoRA-adapted LLMs merged using SLERP across checkpoints or stages. We evaluate representational quality with clustering indices (Silhouette and Davies Bouldin). We additionally analyze the use of kmeans labels to see if the embeddings encode any other information besides the one we are testing for. Empirically, we find that LLM-based backbones produce embeddings that better capture higher-order, compositional numeric patterns, but are prone to adapter dominance that degrades balanced generalization; SLERP merging consistently recovers base-model structure while retaining most task gains, yielding superior tradeoffs in clustering separability, and robustness compared to model souping or models that were not merged.
• 摘要：我们研究了当嵌入主干是大型语言模型 (LLM) 与非 LLM 编码器时密集文本嵌入的泛化特性，并且我们研究了球形线性插值 (SLERP) 模型合并在多大程度上减轻了特定任务适应（例如 LoRA）引入的过度专业化。为了使比较具体且与领域无关，我们设计了一套受控实验，其中模型嵌入短数值序列，并根据明确定义的数论属性评估它们对这些序列进行聚类和分类的能力。我们的实验协议比较了四个模型系列：(1) 从头开始​​训练或针对嵌入进行微调的非 LLM 编码器，(2) 采用参数高效方法 (LoRA) 的基于 LLM 的编码器，(3) 采用 LoRA 的基于 LLM 的编码器，然后将模型合并到基本权重中，(4) 在检查点或阶段使用 SLERP 合并相同的 LoRA 适应的 LLM。我们使用聚类指数（Silhouette 和 Davies Bouldin）评估表征质量。我们还分析了 kmeans 标签的使用，看看嵌入是否编码了除我们正在测试的信息之外的任何其他信息。根据经验，我们发现基于 LLM 的主干生成的嵌入可以更好地捕获高阶、组合数字模式，但容易出现适配器优势，从而降低平衡泛化能力； SLERP 合并一致地恢复基本模型结构，同时保留大部分任务收益，与模型汤或未合并的模型相比，在聚类可分离性和鲁棒性方面产生了优越的权衡。

Title: Insight-A: Attribution-aware for Multimodal Misinformation Detection

• Authors: Junjie Wu, Yumeng Fu, Chen Gong, Guohong Fu
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21705
• Pdf URL: https://arxiv.org/pdf/2511.21705
• Copy Paste: [[2511.21705]] Insight-A: Attribution-aware for Multimodal Misinformation Detection(https://arxiv.org/abs/2511.21705)
• Keywords: language model, llm, prompt
• Abstract: AI-generated content (AIGC) technology has emerged as a prevalent alternative to create multimodal misinformation on social media platforms, posing unprecedented threats to societal safety. However, standard prompting leverages multimodal large language models (MLLMs) to identify the emerging misinformation, which ignores the misinformation attribution. To this end, we present Insight-A, exploring attribution with MLLM insights for detecting multimodal misinformation. Insight-A makes two efforts: I) attribute misinformation to forgery sources, and II) an effective pipeline with hierarchical reasoning that detects distortions across modalities. Specifically, to attribute misinformation to forgery traces based on generation patterns, we devise cross-attribution prompting (CAP) to model the sophisticated correlations between perception and reasoning. Meanwhile, to reduce the subjectivity of human-annotated prompts, automatic attribution-debiased prompting (ADP) is used for task adaptation on MLLMs. Additionally, we design image captioning (IC) to achieve visual details for enhancing cross-modal consistency checking. Extensive experiments demonstrate the superiority of our proposal and provide a new paradigm for multimodal misinformation detection in the era of AIGC.
• 摘要：人工智能生成内容（AIGC）技术已成为在社交媒体平台上创建多模式错误信息的普遍替代方案，对社会安全构成前所未有的威胁。然而，标准提示利用多模态大语言模型（MLLM）来识别新出现的错误信息，而忽略了错误信息的归因。为此，我们提出了 Insight-A，利用 MLLM 见解探索归因，以检测多模式错误信息。 Insight-A 做出了两项努力：I）将错误信息归因于伪造来源，以及 II）具有分层推理的有效管道，可检测跨模式的扭曲。具体来说，为了根据生成模式将错误信息归因于伪造痕迹，我们设计了交叉归因提示（CAP）来对感知和推理之间的复杂相关性进行建模。同时，为了减少人工注释提示的主观性，自动归因去偏提示（ADP）被用于 MLLM 上的任务适应。此外，我们设计图像字幕（IC）来实现视觉细节，以增强跨模式一致性检查。大量的实验证明了我们的提议的优越性，并为 AIGC 时代的多模态错误信息检测提供了新的范例。

Title: A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Tasks

• Authors: Hui Wang, Fafa Zhang, Xiaoyu Zhang, Chaoxu Mu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21706
• Pdf URL: https://arxiv.org/pdf/2511.21706
• Copy Paste: [[2511.21706]] A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Tasks(https://arxiv.org/abs/2511.21706)
• Keywords: language model, gpt, llm, prompt, chat
• Abstract: In goal-oriented dialogue tasks, the main challenge is to steer the interaction towards a given goal within a limited number of turns. Existing approaches either rely on elaborate prompt engineering, whose effectiveness is heavily dependent on human experience, or integrate policy networks and pre-trained policy models, which are usually difficult to adapt to new dialogue scenarios and costly to train. Therefore, in this paper, we present Nested Rollout Policy Adaptation for Goal-oriented Dialogue (NRPA-GD), a novel dialogue policy planning method that completely avoids specific model training by utilizing a Large Language Model (LLM) to simulate behaviors of user and system at the same time. Specifically, NRPA-GD constructs a complete evaluation mechanism for dialogue trajectories and employs an optimization framework of nested Monte Carlo simulation and policy self-adaptation to dynamically adjust policies during the dialogue process. The experimental results on four typical goal-oriented dialogue datasets show that NRPA-GD outperforms both existing prompt engineering and specifically pre-trained model-based methods. Impressively, NRPA-GD surpasses ChatGPT and pre-trained policy models with only a 0.6-billion-parameter LLM. The proposed approach further demonstrates the advantages and novelty of employing planning methods on LLMs to solve practical planning tasks.
• 摘要：在面向目标的对话任务中，主要的挑战是在有限的轮次内引导交互朝着给定的目标发展。现有的方法要么依赖于复杂的即时工程，其有效性在很大程度上取决于人类经验，要么整合政策网络和预先训练的政策模型，这些模型通常难以适应新的对话场景并且训练成本高昂。因此，在本文中，我们提出了面向目标对话的嵌套推出策略适应（NRPA-GD），这是一种新颖的对话策略规划方法，通过利用大型语言模型（LLM）同时模拟用户和系统的行为，完全避免了特定的模型训练。具体来说，NRPA-GD构建了完整的对话轨迹评估机制，并采用嵌套蒙特卡罗模拟和策略自适应的优化框架，在对话过程中动态调整策略。在四个典型的面向目标的对话数据集上的实验结果表明，NRPA-GD 优于现有的即时工程和专门预训练的基于模型的方法。令人印象深刻的是，NRPA-GD 仅用 6 亿参数的 LLM 就超越了 ChatGPT 和预训练的策略模型。所提出的方法进一步证明了在法学硕士上采用规划方法来解决实际规划任务的优势和新颖性。

Title: Lost in the Pipeline: How Well Do Large Language Models Handle Data Preparation?

• Authors: Matteo Spreafico, Ludovica Tassini, Camilla Sancricca, Cinzia Cappiello
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21708
• Pdf URL: https://arxiv.org/pdf/2511.21708
• Copy Paste: [[2511.21708]] Lost in the Pipeline: How Well Do Large Language Models Handle Data Preparation?(https://arxiv.org/abs/2511.21708)
• Keywords: language model, prompt
• Abstract: Large language models have recently demonstrated their exceptional capabilities in supporting and automating various tasks. Among the tasks worth exploring for testing large language model capabilities, we considered data preparation, a critical yet often labor-intensive step in data-driven processes. This paper investigates whether large language models can effectively support users in selecting and automating data preparation tasks. To this aim, we considered both general-purpose and fine-tuned tabular large language models. We prompted these models with poor-quality datasets and measured their ability to perform tasks such as data profiling and cleaning. We also compare the support provided by large language models with that offered by traditional data preparation tools. To evaluate the capabilities of large language models, we developed a custom-designed quality model that has been validated through a user study to gain insights into practitioners' expectations.
• 摘要：大型语言模型最近展示了其在支持和自动化各种任务方面的卓越能力。在值得探索的测试大型语言模型功能的任务中，我们考虑了数据准备，这是数据驱动流程中关键但通常是劳动密集型的步骤。本文研究大型语言模型是否可以有效支持用户选择和自动化数据准备任务。为此，我们考虑了通用和微调的表格大语言模型。我们使用质量较差的数据集提示这些模型，并测量它们执行数据分析和清理等任务的能力。我们还将大型语言模型提供的支持与传统数据准备工具提供的支持进行了比较。为了评估大型语言模型的功能，我们开发了一个定制设计的质量模型，该模型已通过用户研究进行验证，以深入了解从业者的期望。

Title: Quantifying and Mitigating Selection Bias in LLMs: A Transferable LoRA Fine-Tuning and Efficient Majority Voting Approach

• Authors: Blessed Guda, Lawrence Francis, Gabrial Zencha Ashungafac, Carlee Joe-Wong, Moise Busogi
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2511.21709
• Pdf URL: https://arxiv.org/pdf/2511.21709
• Copy Paste: [[2511.21709]] Quantifying and Mitigating Selection Bias in LLMs: A Transferable LoRA Fine-Tuning and Efficient Majority Voting Approach(https://arxiv.org/abs/2511.21709)
• Keywords: language model, llm
• Abstract: Multiple Choice Question (MCQ) answering is a widely used method for evaluating the performance of Large Language Models (LLMs). However, LLMs often exhibit selection bias in MCQ tasks, where their choices are influenced by factors like answer position or option symbols rather than the content. This bias undermines the reliability of MCQ as an evaluation framework. Most existing selection bias metrics require answer labels and measure divergences between prediction and answer distributions, but do not fully capture the consistency of a model's predictions across different orderings of answer choices. Existing selection bias mitigation strategies have notable limitations: majority voting, though effective, is computationally prohibitive; calibration-based methods require validation sets and often fail to generalize across datasets. To address these gaps, we propose three key contributions: (1) a new unsupervised label-free Permutation Bias Metric (PBM) that directly quantifies inconsistencies in model predictions across answer permutations, providing a more precise measure of selection bias, (2) an efficient majority voting approach called Batch Question-Context KV caching (BaQCKV), to significantly reduce computational costs while preserving bias mitigation effectiveness, and (3) an unsupervised Low-Rank Adaptation (LoRA-1) fine-tuning strategy based on our proposed metric and the BaQCKV that mitigates selection bias, providing a computationally efficient alternative that maintains model generalizability. Experiments across multiple MCQ benchmarks demonstrate that our approaches reduce bias, increasing consistency in accuracy while minimizing computational costs.
• 摘要：多项选择题 (MCQ) 回答是一种广泛使用的评估大型语言模型 (LLM) 性能的方法。然而，法学硕士在 MCQ 任务中经常表现出选择偏差，他们的选择受到答案位置或选项符号等因素的影响，而不是内容的影响。这种偏见破坏了 MCQ 作为评估框架的可靠性。大多数现有的选择偏差指标需要答案标签并测量预测和答案分布之间的差异，但不能完全捕获模型在不同答案选择顺序之间的预测的一致性。现有的选择偏差缓解策略具有显着的局限性：多数投票虽然有效，但计算量却令人望而却步；基于校准的方法需要验证集，并且通常无法跨数据集进行泛化。为了解决这些差距，我们提出了三个关键贡献：（1）一种新的无监督无标签排列偏差度量（PBM），可直接量化跨答案排列的模型预测的不一致性，提供更精确的选择偏差测量；（2）一种称为批量问题上下文 KV 缓存（BaQCKV）的高效多数投票方法，可显着降低计算成本，同时保持偏差缓解有效性；（3）无监督低阶适应(LoRA-1) 基于我们提出的指标和 BaQCKV 的微调策略，可减轻选择偏差，提供一种计算有效的替代方案，保持模型的泛化性。跨多个 MCQ 基准的实验表明，我们的方法减少了偏差，提高了准确性的一致性，同时最大限度地降低了计算成本。

Title: Addressing Stereotypes in Large Language Models: A Critical Examination and Mitigation

• Authors: Fatima Kazi
• Subjects: cs.CL, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21711
• Pdf URL: https://arxiv.org/pdf/2511.21711
• Copy Paste: [[2511.21711]] Addressing Stereotypes in Large Language Models: A Critical Examination and Mitigation(https://arxiv.org/abs/2511.21711)
• Keywords: language model, gpt, llm, prompt, chat
• Abstract: Large Language models (LLMs), such as ChatGPT, have gained popularity in recent years with the advancement of Natural Language Processing (NLP), with use cases spanning many disciplines and daily lives as well. LLMs inherit explicit and implicit biases from the datasets they were trained on; these biases can include social, ethical, cultural, religious, and other prejudices and stereotypes. It is important to comprehensively examine such shortcomings by identifying the existence and extent of such biases, recognizing the origin, and attempting to mitigate such biased outputs to ensure fair outputs to reduce harmful stereotypes and misinformation. This study inspects and highlights the need to address biases in LLMs amid growing generative Artificial Intelligence (AI). We utilize bias-specific benchmarks such StereoSet and CrowSPairs to evaluate the existence of various biases in many different generative models such as BERT, GPT 3.5, and ADA. To detect both explicit and implicit biases, we adopt a three-pronged approach for thorough and inclusive analysis. Results indicate fine-tuned models struggle with gender biases but excel at identifying and avoiding racial biases. Our findings also illustrated that despite some cases of success, LLMs often over-rely on keywords in prompts and its outputs. This demonstrates the incapability of LLMs to attempt to truly understand the accuracy and authenticity of its outputs. Finally, in an attempt to bolster model performance, we applied an enhancement learning strategy involving fine-tuning, models using different prompting techniques, and data augmentation of the bias benchmarks. We found fine-tuned models to exhibit promising adaptability during cross-dataset testing and significantly enhanced performance on implicit bias benchmarks, with performance gains of up to 20%.
• 摘要：近年来，随着自然语言处理 (NLP) 的进步，诸如 ChatGPT 之类的大型语言模型 (LLM) 越来越受欢迎，其用例也涵盖了许多学科和日常生活。法学硕士从他们接受训练的数据集中继承了显式和隐式偏差；这些偏见可能包括社会、道德、文化、宗教和其他偏见和成见。重要的是要全面审查这些缺陷，确定此类偏见的存在和程度，认识其根源，并尝试减轻此类有偏见的产出，以确保公平的产出，从而减少有害的陈规定型观念和错误信息。这项研究检查并强调了在生成人工智能（AI）不断发展的背景下解决法学硕士偏见的必要性。我们利用 StereoSet 和 CrowSPairs 等特定于偏差的基准来评估许多不同生成模型（例如 BERT、GPT 3.5 和 ADA）中各种偏差的存在。为了检测显性和隐性偏差，我们采用三管齐下的方法进行彻底和包容性的分析。结果表明，经过微调的模型与性别偏见作斗争，但擅长识别和避免种族偏见。我们的研究结果还表明，尽管有一些成功的案例，法学硕士常常过度依赖提示及其输出中的关键词。这表明法学硕士无法真正理解其输出的准确性和真实性。最后，为了提高模型性能，我们应用了增强学习策略，包括微调、使用不同提示技术的模型以及偏差基准的数据增强。我们发现经过微调的模型在跨数据集测试期间表现出良好的适应性，并显着增强了隐式偏差基准的性能，性能提升高达 20%。

Title: EulerESG: Automating ESG Disclosure Analysis with LLMs

• Authors: Yi Ding, Xushuo Tang, Zhengyi Yang, Wenqian Zhang, Simin Wu, Yuxin Huang, Lingjing Lan, Weiyuan Li, Yin Chen, Mingchen Ju, Wenke Yang, Thong Hoang, Mykhailo Klymenko, Xiwei Zu, Wenjie Zhang
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2511.21712
• Pdf URL: https://arxiv.org/pdf/2511.21712
• Copy Paste: [[2511.21712]] EulerESG: Automating ESG Disclosure Analysis with LLMs(https://arxiv.org/abs/2511.21712)
• Keywords: llm, chat
• Abstract: Environmental, Social, and Governance (ESG) reports have become central to how companies communicate climate risk, social impact, and governance practices, yet they are still published primarily as long, heterogeneous PDF documents. This makes it difficult to systematically answer seemingly simple questions. Existing tools either rely on brittle rule-based extraction or treat ESG reports as generic text, without explicitly modelling the underlying reporting standards. We present \textbf{EulerESG}, an LLM-powered system for automating ESG disclosure analysis with explicit awareness of ESG frameworks. EulerESG combines (i) dual-channel retrieval and LLM-driven disclosure analysis over ESG reports, and (ii) an interactive dashboard and chatbot for exploration, benchmarking, and explanation. Using four globally recognised companies and twelve SASB sub-industries, we show that EulerESG can automatically populate standard-aligned metric tables with high fidelity (up to 0.95 average accuracy) while remaining practical in end-to-end runtime, and we compare several recent LLM models in this setting. The full implementation, together with a demonstration video, is publicly available at this https URL.
• 摘要：环境、社会和治理 (ESG) 报告已成为企业沟通气候风险、社会影响和治理实践的核心，但它们仍然主要以冗长、异构的 PDF 文档形式发布。这使得系统地回答看似简单的问题变得困难。现有工具要么依赖脆弱的基于规则的提取，要么将 ESG 报告视为通用文本，而没有对基础报告标准进行明确建模。我们推出 \textbf{EulerESG}，这是一个由法学硕士支持的系统，用于自动化 ESG 披露分析，并明确了解 ESG 框架。 EulerESG 结合了 (i) 双通道检索和法学硕士驱动的 ESG 报告披露分析，以及 (ii) 用于探索、基准测试和解释的交互式仪表板和聊天机器人。通过使用四家全球知名公司和十二个 SASB 子行业，我们证明 EulerESG 可以自动填充高保真度（平均精度高达 0.95）的标准对齐指标表，同时在端到端运行时保持实用性，并且我们在此设置中比较了几个最新的 LLM 模型。完整的实现以及演示视频可通过此 https URL 公开获取。

Title: GPS: General Per-Sample Prompter

• Authors: Pawel Batorski, Paul Swoboda
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21714
• Pdf URL: https://arxiv.org/pdf/2511.21714
• Copy Paste: [[2511.21714]] GPS: General Per-Sample Prompter(https://arxiv.org/abs/2511.21714)
• Keywords: llm, prompt
• Abstract: LLMs are sensitive to prompting, with task performance often hinging on subtle, sometimes imperceptible variations in phrasing. As a result, crafting effective prompts manually remains challenging and time-consuming. Recent automatic prompting methods mitigate this difficulty but face three key limitations: (i) for each new task, they require large datasets to train good prompts;(ii) they rely on costly optimization loops that may take hours; (iii)they typically produce a single task-level prompt that does not adapt to the individual input problem to be solved. We propose GPS, the first general-purpose, per-sample prompting method. Without any task-specific tuning, GPS generates a tailored prompt for each unseen input, improving performance across diverse tasks. The prompter is trained with reinforcement learning on a suite of training tasks and includes a novel regularization for effectively adapting to per-sample prompting. Finally, we employ Minimum Bayes Risk decoding to stabilize inference. Empirically, GPS demonstrates competitive performance: we attain second best results among baselines on text simplification, third best results on summarization and on-par results on classification, while not training on any of these tasks, in contrast to the baselines. For in-domain prompting, we obtain sota on GSM8K. Our work shows the potential of a novel and effective paradigm for automatic prompting: generating adaptive, input-specific prompts without extensive optimization and without access to a task-specific training set. Our code is available at this https URL.
• 摘要：法学硕士对提示很敏感，任务绩效往往取决于措辞中微妙的、有时难以察觉的变化。因此，手动制作有效的提示仍然具有挑战性且耗时。最近的自动提示方法缓解了这一困难，但面临三个关键限制：（i）对于每个新任务，它们需要大量数据集来训练良好的提示；（ii）它们依赖于可能需要数小时的昂贵的优化循环； (iii)它们通常会产生一个单一的任务级提示，该提示不适合要解决的单个输入问题。我们提出了 GPS，这是第一个通用的、按样本提示的方法。无需任何特定于任务的调整，GPS 就会为每个看不见的输入生成量身定制的提示，从而提高不同任务的性能。该提示器在一系列训练任务上通过强化学习进行训练，并包括一种新颖的正则化方法，可以有效地适应每个样本的提示。最后，我们采用最小贝叶斯风险解码来稳定推理。根据经验，GPS 展示了竞争性能：与基线相比，我们在文本简化的基线中获得了第二好的结果，在摘要上获得了第三好的结果，在分类上获得了同等的结果，同时没有对任何这些任务进行训练。对于域内提示，我们在 GSM8K 上获取 sota。我们的工作展示了一种新颖有效的自动提示范例的潜力：生成自适应的、特定于输入的提示，无需进行广泛的优化，也无需访问特定于任务的训练集。我们的代码可以在这个 https URL 上找到。

Title: CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution

• Authors: Baoliang Tian, Yuxuan Si, Jilong Wang, Lingyao Li, Zhongyuan Bao, Zineng Zhou, Tao Wang, Sixu Li, Ziyao Xu, Mingze Wang, Zhouzhuo Zhang, Zhihao Wang, Yike Yun, Ke Tian, Ning Yang, Minghui Qiu
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.21717
• Pdf URL: https://arxiv.org/pdf/2511.21717
• Copy Paste: [[2511.21717]] CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution(https://arxiv.org/abs/2511.21717)
• Keywords: language model, prompt, chain-of-thought
• Abstract: Multimodal Large Language Models are primarily trained and evaluated on aligned image-text pairs, which leaves their ability to detect and resolve real-world inconsistencies largely unexplored. In open-domain applications visual and textual cues often conflict, requiring models to perform structured reasoning beyond surface-level alignment. We introduce CrossCheck-Bench, a diagnostic benchmark for evaluating contradiction detection in multimodal inputs. The benchmark adopts a hierarchical task framework covering three levels of reasoning complexity and defines seven atomic capabilities essential for resolving cross-modal inconsistencies. CrossCheck-Bench includes 15k question-answer pairs sourced from real-world artifacts with synthetically injected contradictions. The dataset is constructed through a multi-stage annotation pipeline involving more than 450 expert hours to ensure semantic validity and calibrated difficulty across perception, integration, and reasoning. We evaluate 13 state-of-the-art vision-language models and observe a consistent performance drop as tasks shift from perceptual matching to logical contradiction detection. Most models perform well on isolated entity recognition but fail when multiple clues must be synthesized for conflict reasoning. Capability-level analysis further reveals uneven skill acquisition, especially in tasks requiring multi-step inference or rule-based validation. Additional probing shows that conventional prompting strategies such as Chain-of-Thought and Set-of-Mark yield only marginal gains. By contrast, methods that interleave symbolic reasoning with grounded visual processing achieve more stable improvements. These results highlight a persistent bottleneck in multimodal reasoning and suggest new directions for building models capable of robust cross-modal verification.
• 摘要：多模态大型语言模型主要是在对齐的图像文本对上进行训练和评估的，这使得它们检测和解决现实世界不一致的能力在很大程度上尚未被开发。在开放域应用中，视觉和文本线索经常发生冲突，要求模型执行超越表面级别对齐的结构化推理。我们引入了 CrossCheck-Bench，这是一种用于评估多模式输入中矛盾检测的诊断基准。该基准采用涵盖三个推理复杂度级别的分层任务框架，并定义了解决跨模式不一致所必需的七种原子能力。 CrossCheck-Bench 包括 15,000 个问答对，这些问答对源自具有综合注入矛盾的现实世界工件。该数据集是通过涉及 450 多个专家小时的多阶段注释管道构建的，以确保感知、集成和推理的语义有效性和校准难度。我们评估了 13 个最先进的视觉语言模型，并观察到随着任务从感知匹配转向逻辑矛盾检测，性能持续下降。大多数模型在孤立实体识别上表现良好，但在必须综合多个线索进行冲突推理时会失败。能力级别分析进一步揭示了技能获取的不均匀性，特别是在需要多步骤推理或基于规则的验证的任务中。进一步的探索表明，传统的激励策略，如思维链和标记集，只能产生边际收益。相比之下，将符号推理与基础视觉处理交织在一起的方法实现了更稳定的改进。这些结果凸显了多模态推理中持续存在的瓶颈，并为构建能够进行稳健的跨模态验证的模型提出了新的方向。

Title: When Harmless Words Harm: A New Threat to LLM Safety via Conceptual Triggers

• Authors: Zhaoxin Zhang, Borui Chen, Yiming Hu, Youyang Qu, Tianqing Zhu, Longxiang Gao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.21718
• Pdf URL: https://arxiv.org/pdf/2511.21718
• Copy Paste: [[2511.21718]] When Harmless Words Harm: A New Threat to LLM Safety via Conceptual Triggers(https://arxiv.org/abs/2511.21718)
• Keywords: language model, gpt, llm, prompt
• Abstract: Recent research on large language model (LLM) jailbreaks has primarily focused on techniques that bypass safety mechanisms to elicit overtly harmful outputs. However, such efforts often overlook attacks that exploit the model's capacity for abstract generalization, creating a critical blind spot in current alignment strategies. This gap enables adversaries to induce objectionable content by subtly manipulating the implicit social values embedded in model outputs. In this paper, we introduce MICM, a novel, model-agnostic jailbreak method that targets the aggregate value structure reflected in LLM responses. Drawing on conceptual morphology theory, MICM encodes specific configurations of nuanced concepts into a fixed prompt template through a predefined set of phrases. These phrases act as conceptual triggers, steering model outputs toward a specific value stance without triggering conventional safety filters. We evaluate MICM across five advanced LLMs, including GPT-4o, Deepseek-R1, and Qwen3-8B. Experimental results show that MICM consistently outperforms state-of-the-art jailbreak techniques, achieving high success rates with minimal rejection. Our findings reveal a critical vulnerability in commercial LLMs: their safety mechanisms remain susceptible to covert manipulation of underlying value alignment.
• 摘要：最近对大型语言模型 (LLM) 越狱的研究主要集中在绕过安全机制以引发明显有害输出的技术。然而，此类努力往往忽视了利用模型抽象泛化能力的攻击，从而在当前的对齐策略中造成了关键的盲点。这种差距使对手能够通过巧妙地操纵模型输出中嵌入的隐含社会价值观来诱导令人反感的内容。在本文中，我们介绍了 MICM，这是一种新颖的、与模型无关的越狱方法，其目标是 LLM 响应中反映的聚合价值结构。 MICM 借鉴概念形态学理论，通过一组预定义的短语将细微概念的特定配置编码到固定的提示模板中。这些短语充当概念触发器，将模型输出引导至特定的价值立场，而不会触发传统的安全过滤器。我们评估了五个高级法学硕士的 MICM，包括 GPT-4o、Deepseek-R1 和 Qwen3-8B。实验结果表明，MICM 始终优于最先进的越狱技术，实现了高成功率和最小的拒绝率。我们的研究结果揭示了商业法学硕士的一个关键漏洞：它们的安全机制仍然容易受到潜在价值调整的秘密操纵。

Title: PeerCoPilot: A Language Model-Powered Assistant for Behavioral Health Organizations

• Authors: Gao Mo, Naveen Raman, Megan Chai, Cindy Peng, Shannon Pagdon, Nev Jones, Hong Shen, Peggy Swarbrick, Fei Fang
• Subjects: cs.CL, cs.CY, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21721
• Pdf URL: https://arxiv.org/pdf/2511.21721
• Copy Paste: [[2511.21721]] PeerCoPilot: A Language Model-Powered Assistant for Behavioral Health Organizations(https://arxiv.org/abs/2511.21721)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Behavioral health conditions, which include mental health and substance use disorders, are the leading disease burden in the United States. Peer-run behavioral health organizations (PROs) critically assist individuals facing these conditions by combining mental health services with assistance for needs such as income, employment, and housing. However, limited funds and staffing make it difficult for PROs to address all service user needs. To assist peer providers at PROs with their day-to-day tasks, we introduce PeerCoPilot, a large language model (LLM)-powered assistant that helps peer providers create wellness plans, construct step-by-step goals, and locate organizational resources to support these goals. PeerCoPilot ensures information reliability through a retrieval-augmented generation pipeline backed by a large database of over 1,300 vetted resources. We conducted human evaluations with 15 peer providers and 6 service users and found that over 90% of users supported using PeerCoPilot. Moreover, we demonstrated that PeerCoPilot provides more reliable and specific information than a baseline LLM. PeerCoPilot is now used by a group of 5-10 peer providers at CSPNJ, a large behavioral health organization serving over 10,000 service users, and we are actively expanding PeerCoPilot's use.
• 摘要：行为健康状况，包括心理健康和药物滥用障碍，是美国的主要疾病负担。同伴经营的行为健康组织 (PRO) 通过将心理健康服务与收入、就业和住房等需求援助相结合，为面临这些状况的个人提供重要帮助。然而，有限的资金和人员配置使得 PRO 很难满足所有服务用户的需求。为了帮助 PRO 的同行提供者完成日常任务，我们引入了 PeerCoPilot，这是一个由大型语言模型 (LLM) 支持的助手，可帮助同行提供者制定健康计划、构建分步目标，并找到组织资源来支持这些目标。 PeerCoPilot 通过检索增强的生成管道确保信息可靠性，该管道由包含 1,300 多个经过审查的资源的大型数据库支持。我们对 15 个同行提供商和 6 个服务用户进行了人工评估，发现超过 90% 的用户支持使用 PeerCoPilot。此外，我们证明 PeerCoPilot 提供比基线 LLM 更可靠、更具体的信息。 PeerCoPilot 现在由 CSPNJ 的一组 5-10 个同行提供商使用，CSPNJ 是一家大型行为健康组织，为 10,000 多名服务用户提供服务，我们正在积极扩大 PeerCoPilot 的使用范围。

Title: German General Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies

• Authors: Jens Rupprecht, Leon Fröhling, Claudia Wagner, Markus Strohmaier
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2511.21722
• Pdf URL: https://arxiv.org/pdf/2511.21722
• Copy Paste: [[2511.21722]] German General Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies(https://arxiv.org/abs/2511.21722)
• Keywords: language model, llm, prompt
• Abstract: The use of Large Language Models (LLMs) for simulating human perspectives via persona prompting is gaining traction in computational social science. However, well-curated, empirically grounded persona collections remain scarce, limiting the accuracy and representativeness of such simulations. Here we introduce the German General Personas (GGP) collection, a comprehensive and representative persona prompt collection built from the German General Social Survey (ALLBUS). The GGP and its persona prompts are designed to be easily plugged into prompts for all types of LLMs and tasks, steering models to generate responses aligned with the underlying German population. We evaluate GGP by prompting various LLMs to simulate survey response distributions across diverse topics, demonstrating that GGP-guided LLMs outperform state-of-the-art classifiers, particularly under data scarcity. Furthermore, we analyze how the representativity and attribute selection within persona prompts affect alignment with population responses. Our findings suggest that GGP provides a potentially valuable resource for research on LLM-based social simulations that enables more systematic explorations of population-aligned persona prompting in NLP and social science research.
• 摘要：使用大型语言模型（LLM）通过角色提示来模拟人类观点在计算社会科学中越来越受到关注。然而，精心策划的、以经验为基础的角色集合仍然稀缺，限制了此类模拟的准确性和代表性。这里我们介绍德国通用人物模型（GGP）集合，它是根据德国综合社会调查（ALLBUS）构建的全面且具有代表性的人物角色提示集合。 GGP 及其角色提示旨在轻松插入所有类型的法学硕士和任务的提示中，引导模型生成与潜在德国人口一致的响应。我们通过提示各种法学硕士模拟不同主题的调查响应分布来评估 GGP，证明 GGP 指导的法学硕士优于最先进的分类器，特别是在数据稀缺的情况下。此外，我们分析了人物角色提示中的代表性和属性选择如何影响与人群反应的一致性。我们的研究结果表明，GGP 为基于 LLM 的社会模拟研究提供了潜在的宝贵资源，可以在 NLP 和社会科学研究中更系统地探索与人群相关的角色提示。

Title: PromptTailor: Multi-turn Intent-Aligned Prompt Synthesis for Lightweight LLMs

• Authors: Yizhou Xu, Janet Davis
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21725
• Pdf URL: https://arxiv.org/pdf/2511.21725
• Copy Paste: [[2511.21725]] PromptTailor: Multi-turn Intent-Aligned Prompt Synthesis for Lightweight LLMs(https://arxiv.org/abs/2511.21725)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Lightweight language models remain attractive for on-device and privacy-sensitive applications, but their responses are highly sensitive to prompt quality. For open-ended generation, non-expert users often lack the knowledge or time to consistently craft high-quality prompts, leading them to rely on prompt optimization tools. However, a key challenge is ensuring the optimized prompts genuinely align with users' original intents and preferences. We introduce PromptTailor, a system for controllable prompt generation for open-ended text that improves model output quality by intent-aligned prompt synthesis. PromptTailor expands minimal user instructions into rich, domain-aware prompts while preserving the user's stated preferences. The system is a quantized Llama3-8B model fine-tuned with a lightweight LoRA adapter on 12,300 prompt-refinement dialogues spanning 41 everyday domains, distilled from three stronger LLMs. The adapter attaches to any Llama3-8B base, enabling edge deployment. In human and LLM-judge evaluations across multiple target models and optimization baselines, PromptTailor yields higher preference rates than chain-of-thought prompting and matches or surpasses state-of-the-art prompt optimization methods while requiring fewer model calls (e.g., 3 vs. 9). These results show that a compact student, guided by powerful teachers, can learn effective prompt-generation strategies that enhance response quality while maintaining alignment with user intent.
• 摘要：轻量级语言模型对于设备上和隐私敏感的应用程序仍然有吸引力，但它们的响应对提示质量高度敏感。对于开放式生成，非专家用户通常缺乏知识或时间来持续制作高质量的提示，导致他们依赖提示优化工具。然而，一个关键的挑战是确保优化的提示真正符合用户的原始意图和偏好。我们推出了 PromptTailor，这是一个用于为开放式文本生成可控提示的系统，可通过意图一致的提示合成来提高模型输出质量。 PromptTailor 将最少的用户指令扩展为丰富的、领域感知的提示，同时保留用户声明的首选项。该系统是一个量化的 Llama3-8B 模型，使用轻量级 LoRA 适配器对跨越 41 个日常领域的 12,300 个提示细化对话进行了微调，这些对话是从三个更强大的法学硕士中提炼出来的。该适配器可连接到任何 Llama3-8B 底座，从而实现边缘部署。在跨多个目标模型和优化基线的人类和法学硕士法官评估中，PromptTailor 比思想链提示产生更高的偏好率，并匹配或超越最先进的提示优化方法，同时需要更少的模型调用（例如，3 与 9）。这些结果表明，一个紧凑的学生在强大的教师指导下可以学习有效的提示生成策略，从而提高响应质量，同时保持与用户意图的一致性。

Title: Goal-Directed Search Outperforms Goal-Agnostic Memory Compression in Long-Context Memory Tasks

• Authors: Yicong Zheng, Kevin L. McKee, Thomas Miconi, Zacharie Bugaud, Mick van Gelderen, Jed McCaleb
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21726
• Pdf URL: https://arxiv.org/pdf/2511.21726
• Copy Paste: [[2511.21726]] Goal-Directed Search Outperforms Goal-Agnostic Memory Compression in Long-Context Memory Tasks(https://arxiv.org/abs/2511.21726)
• Keywords: language model, llm, prompt, agent
• Abstract: How to enable human-like long-term memory in large language models (LLMs) has been a central question for unlocking more general capabilities such as few-shot generalization. Existing memory frameworks and benchmarks focus on finding the optimal memory compression algorithm for higher performance in tasks that require recollection and sometimes further reasoning. However, such efforts have ended up building more human bias into the compression algorithm, through the search for the best prompts and memory architectures that suit specific benchmarks, rather than finding a general solution that would work on other data distributions. On the other hand, goal-directed search on uncompressed information could potentially exhibit superior performance because compression is lossy, and a predefined compression algorithm will not fit all raw data distributions. Here we present SUMER (Search in Uncompressed Memory via Experience Replay), an end-to-end reinforcement learning agent with verifiable reward (RLVR) that learns to use search tools to gather information and answer a target question. On the LoCoMo dataset for long-context conversation understanding, SUMER with Qwen2.5-7B-Instruct learned to use search tools and outperformed all other biased memory compression approaches and also the full-context baseline, reaching SOTA performance (43% gain over the prior best). We demonstrate that a simple search method applied to raw data outperforms goal-agnostic and biased compression algorithms in current long-context memory tasks, arguing for new paradigms and benchmarks that are more dynamic and autonomously scalable. Code for SUMER and all implemented baselines is publicly available at this https URL.
• 摘要：如何在大型语言模型（LLM）中实现类似人类的长期记忆一直是解锁更通用功能（例如少样本泛化）的核心问题。现有的内存框架和基准测试专注于寻找最佳内存压缩算法，以在需要回忆和有时需要进一步推理的任务中获得更高的性能。然而，这些努力最终通过搜索适合特定基准的最佳提示和内存架构，在压缩算法中建立了更多的人为偏见，而不是找到适用于其他数据分布的通用解决方案。另一方面，对未压缩信息的目标导向搜索可能会表现出优越的性能，因为压缩是有损的，并且预定义的压缩算法不会适合所有原始数据分布。在这里，我们介绍 SUMER（通过体验回放在未压缩内存中搜索），这是一种具有可验证奖励（RLVR）的端到端强化学习代理，它学习使用搜索工具来收集信息并回答目标问题。在用于长上下文对话理解的 LoCoMo 数据集上，SUMER 与 Qwen2.5-7B-Instruct 学会了使用搜索工具，并且优于所有其他有偏差的内存压缩方法以及全上下文基线，达到了 SOTA 性能（比之前的最佳性能提高了 43%）。我们证明，在当前的长上下文记忆任务中，应用于原始数据的简单搜索方法优于目标无关和有偏差的压缩算法，主张更加动态和自主可扩展的新范式和基准。 SUMER 的代码和所有已实现的基线可在此 https URL 上公开获取。

Title: Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue

• Authors: Lin Yu, Xiaofei Han, Yifei Kang, Chiung-Yi Tseng, Danyang Zhang, Ziqian Bi, Zhimo Han
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21728
• Pdf URL: https://arxiv.org/pdf/2511.21728
• Copy Paste: [[2511.21728]] Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue(https://arxiv.org/abs/2511.21728)
• Keywords: language model, llm, agent
• Abstract: Recent advances in large language models (LLMs) have enabled fluent dialogue systems, but most remain reactive and struggle in emotionally rich, goal-oriented settings such as marketing conversations. To address this limitation, we propose AffectMind, a multimodal affective dialogue agent that performs proactive reasoning and dynamic knowledge grounding to sustain emotionally aligned and persuasive interactions. AffectMind combines three components: a Proactive Knowledge Grounding Network (PKGN) that continuously updates factual and affective context from text, vision, and prosody; an Emotion--Intent Alignment Model (EIAM) that jointly models user emotion and purchase intent to adapt persuasion strategies; and a Reinforced Discourse Loop (RDL) that optimizes emotional coherence and engagement via reinforcement signals from user responses. Experiments on two newly curated marketing dialogue datasets, MM-ConvMarket and AffectPromo, show that AffectMind outperforms strong LLM-based baselines in emotional consistency (+26\%), persuasive success rate (+19\%), and long-term user engagement (+23\%), highlighting emotion-grounded proactivity as a key capability for commercial multimodal agents.
• 摘要：大语言模型 (LLM) 的最新进展已经实现了流畅的对话系统，但大多数在情感丰富、目标导向的环境（例如营销对话）中仍然处于反应和挣扎状态。为了解决这一限制，我们提出了 AffectMind，这是一种多模式情感对话代理，它执行主动推理和动态知识基础，以维持情感一致和有说服力的互动。 AffectMind 结合了三个组件：主动知识基础网络 (PKGN)，不断更新来自文本、视觉和韵律的事实和情感上下文；情感-意图对齐模型（EIAM），联合模拟用户情感和购买意图以适应说服策略；强化话语循环 (RDL)，通过用户响应的强化信号来优化情感连贯性和参与度。在两个新策划的营销对话数据集 MM-ConvMarket 和 AffectPromo 上进行的实验表明，AffectMind 在情感一致性 (+26\%)、说服成功率 (+19\%) 和长期用户参与度 (+23\%) 方面优于基于 LLM 的强大基线，强调基于情感的主动性是商业多模式代理的关键能力。

Title: Beyond Component Strength: Synergistic Integration and Adaptive Calibration in Multi-Agent RAG Systems

• Authors: Jithin Krishnan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21729
• Pdf URL: https://arxiv.org/pdf/2511.21729
• Copy Paste: [[2511.21729]] Beyond Component Strength: Synergistic Integration and Adaptive Calibration in Multi-Agent RAG Systems(https://arxiv.org/abs/2511.21729)
• Keywords: hallucination, retrieval-augmented generation, agent
• Abstract: Building reliable retrieval-augmented generation (RAG) systems requires more than adding powerful components; it requires understanding how they interact. Using ablation studies on 50 queries (15 answerable, 10 edge cases, and 25 adversarial), we show that enhancements such as hybrid retrieval, ensemble verification, and adaptive thresholding provide almost no benefit when used in isolation, yet together achieve a 95% reduction in abstention (from 40% to 2%) without increasing hallucinations. We also identify a measurement challenge: different verification strategies can behave safely but assign inconsistent labels (for example, "abstained" versus "unsupported"), creating apparent hallucination rates that are actually artifacts of labeling. Our results show that synergistic integration matters more than the strength of any single component, that standardized metrics and labels are essential for correctly interpreting performance, and that adaptive calibration is needed to prevent overconfident over-answering even when retrieval quality is high.
• 摘要：构建可靠的检索增强生成（RAG）系统需要的不仅仅是添加强大的组件；它需要了解它们如何相互作用。通过对 50 个查询（15 个可回答的、10 个边缘情况和 25 个对抗性）的消融研究，我们表明，混合检索、集成验证和自适应阈值等增强功能在单独使用时几乎没有任何好处，但共同实现了弃权率减少 95%（从 40% 到 2%）而不增加幻觉。我们还发现了一个测量挑战：不同的验证策略可以安全地运行，但分配不一致的标签（例如，“弃权”与“不支持”），从而产生明显的幻觉率，而这些幻觉率实际上是标签的伪影。我们的结果表明，协同集成比任何单个组件的强度更重要，标准化指标和标签对于正确解释性能至关重要，并且即使在检索质量很高的情况下，也需要自适应校准来防止过度自信的过度回答。

Title: A Benchmark for Procedural Memory Retrieval in Language Agents

• Authors: Ishant Kohar, Aswanth Krishnan
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21730
• Pdf URL: https://arxiv.org/pdf/2511.21730
• Copy Paste: [[2511.21730]] A Benchmark for Procedural Memory Retrieval in Language Agents(https://arxiv.org/abs/2511.21730)
• Keywords: llm, agent
• Abstract: Current AI agents excel in familiar settings, but fail sharply when faced with novel tasks with unseen vocabularies -- a core limitation of procedural memory systems. We present the first benchmark that isolates procedural memory retrieval from task execution, evaluating whether agents can recognize functionally equivalent procedures that span different object instantiations. Using ALFWorld, we construct dual corpora of expert and LLM-generated trajectories and evaluate six retrieval methods using systematically stratified queries. Our results expose a clear generalization cliff: embedding-based methods perform strongly on familiar contexts, yet degrade considerably on novel ones, while LLM-generated procedural abstractions demonstrate reliable cross-context transfer. Controlled ablations show that although embeddings capture some lexical-level abstraction, they fundamentally treat procedures as unordered bags of words, discarding temporal structure necessary for cross-context transfer. Corpus scale delivers far larger gains than representation enrichment, revealing an architectural ceiling in current encoders. Our benchmark offers the first diagnostic framework separating genuine procedural understanding from surface-level memorization and gives tools for developing retrieval systems capable of dependable generalization. Resources available at our GitHub repository (this https URL).
• 摘要：当前的人工智能代理在熟悉的环境中表现出色，但在面对具有未见过的词汇的新任务时会急剧失败——这是程序记忆系统的核心限制。我们提出了第一个将程序内存检索与任务执行隔离的基准，评估代理是否可以识别跨不同对象实例化的功能等效程序。使用 ALFWorld，我们构建了专家和法学硕士生成轨迹的双语料库，并使用系统分层查询评估六种检索方法。我们的结果暴露了一个明显的泛化悬崖：基于嵌入的方法在熟悉的上下文中表现出色，但在新的上下文中性能却显着下降，而法学硕士生成的程序抽象则证明了可靠的跨上下文迁移。受控消融表明，尽管嵌入捕获了一些词汇级抽象，但它们从根本上将过程视为无序的词袋，丢弃了跨上下文传输所需的时间结构。语料库规模比表示丰富带来更大的收益，揭示了当前编码器的架构上限。我们的基准测试提供了第一个将真正的程序理解与表面记忆分开的诊断框架，并提供了开发能够可靠泛化的检索系统的工具。我们的 GitHub 存储库（此 https URL）提供了资源。

Title: Identifying Quantum Structure in AI Language: Evidence for Evolutionary Convergence of Human and Artificial Cognition

• Authors: Diederik Aerts, Jonito Aerts Arguëlles, Lester Beltran, Suzette Geriente, Roberto Leporini, Massimiliano Sassoli de Bianchi, Sandro Sozzo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21731
• Pdf URL: https://arxiv.org/pdf/2511.21731
• Copy Paste: [[2511.21731]] Identifying Quantum Structure in AI Language: Evidence for Evolutionary Convergence of Human and Artificial Cognition(https://arxiv.org/abs/2511.21731)
• Keywords: language model, gpt, llm, chat, agent
• Abstract: We present the results of cognitive tests on conceptual combinations, performed using specific Large Language Models (LLMs) as test subjects. In the first test, performed with ChatGPT and Gemini, we show that Bell's inequalities are significantly violated, which indicates the presence of 'quantum entanglement' in the tested concepts. In the second test, also performed using ChatGPT and Gemini, we instead identify the presence of 'Bose-Einstein statistics', rather than the intuitively expected 'Maxwell-Boltzmann statistics', in the distribution of the words contained in large-size texts. Interestingly, these findings mirror the results previously obtained in both cognitive tests with human participants and information retrieval tests on large corpora. Taken together, they point to the 'systematic emergence of quantum structures in conceptual-linguistic domains', regardless of whether the cognitive agent is human or artificial. Although LLMs are classified as neural networks for historical reasons, we believe that a more essential form of knowledge organization takes place in the distributive semantic structure of vector spaces built on top of the neural network. It is this meaning-bearing structure that lends itself to a phenomenon of evolutionary convergence between human cognition and language, slowly established through biological evolution, and LLM cognition and language, emerging much more rapidly as a result of self-learning and training. We analyze various aspects and examples that contain evidence supporting the above hypothesis. We also advance a unifying framework that explains the pervasive quantum organization of meaning that we identify.
• 摘要：我们展示了使用特定大语言模型（LLM）作为测试对象进行的概念组合认知测试的结果。在使用 ChatGPT 和 Gemini 进行的第一个测试中，我们发现贝尔不等式被严重违反，这表明测试概念中存在“量子纠缠”。在同样使用 ChatGPT 和 Gemini 进行的第二个测试中，我们在大尺寸文本中包含的单词分布中识别出“Bose-Einstein 统计数据”的存在，而不是直观预期的“Maxwell-Boltzmann 统计数据”。有趣的是，这些发现反映了之前在人类参与者认知测试和大型语料库信息检索测试中获得的结果。总而言之，他们指出“量子结构在概念语言领域的系统出现”，无论认知主体是人类还是人造的。尽管由于历史原因法学硕士被归类为神经网络，但我们相信更本质的知识组织形式发生在建立在神经网络之上的向量空间的分布式语义结构中。正是这种意义承载结构导致了人类认知和语言之间的进化趋同现象，通过生物进化慢慢建立起来，而法学硕士认知和语言则通过自我学习和培训而更快地出现。我们分析了包含支持上述假设的证据的各个方面和例子。我们还提出了一个统一的框架来解释我们所识别的普遍的量子意义组织。

Title: HUMORCHAIN: Theory-Guided Multi-Stage Reasoning for Interpretable Multimodal Humor Generation

• Authors: Jiajun Zhang, Shijia Luo, Ruikang Zhang, Qi Su
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21732
• Pdf URL: https://arxiv.org/pdf/2511.21732
• Copy Paste: [[2511.21732]] HUMORCHAIN: Theory-Guided Multi-Stage Reasoning for Interpretable Multimodal Humor Generation(https://arxiv.org/abs/2511.21732)
• Keywords: language model
• Abstract: Humor, as both a creative human activity and a social binding mechanism, has long posed a major challenge for AI generation. Although producing humor requires complex cognitive reasoning and social understanding, theories of humor suggest that it follows learnable patterns and structures, making it theoretically possible for generative models to acquire them implicitly. In recent years, multimodal humor has become a prevalent form of online communication, especially among Gen Z, highlighting the need for AI systems capable of integrating visual understanding with humorous language generation. However, existing data-driven approaches lack explicit modeling or theoretical grounding of humor, often producing literal descriptions that fail to capture its underlying cognitive mechanisms, resulting in the generated image descriptions that are fluent but lack genuine humor or cognitive depth. To address this limitation, we propose HUMORCHAIN (HUmor-guided Multi-step Orchestrated Reasoning Chain for Image Captioning), a theory-guided multi-stage reasoning framework. It integrates visual semantic parsing, humor- and psychology-based reasoning, and a fine-tuned discriminator for humor evaluation, forming an interpretable and controllable cognitive reasoning chain. To the best of our knowledge, this is the first work to explicitly embed cognitive structures from humor theories into multimodal humor generation, enabling a structured reasoning process from visual understanding to humor creation. Experiments on Meme-Image-No-Text, Oogiri-GO, and OxfordTVG-HIC datasets show that HUMORCHAIN outperforms state-of-the-art baselines in human humor preference, Elo/BT scores, and semantic diversity, demonstrating that theory-driven structured reasoning enables large language models to generate humor aligned with human perception.
• 摘要：幽默作为人类的创造性活动和社会约束机制，长期以来一直对人工智能的产生提出重大挑战。尽管产生幽默需要复杂的认知推理和社会理解，但幽默理论表明它遵循可学习的模式和结构，这使得生成模型在理论上可以隐式地获得它们。近年来，多模态幽默已成为一种流行的在线交流形式，尤其是在 Z 世代中，这凸显了对能够将视觉理解与幽默语言生成相结合的人工智能系统的需求。然而，现有的数据驱动方法缺乏幽默的明确建模或理论基础，常常产生无法捕捉其潜在认知机制的字面描述，导致生成的图像描述流畅但缺乏真正的幽默或认知深度。为了解决这个限制，我们提出了 HUMORCHAIN（HUmor-guided Multi-step Orchestrated Reasoning Chain for Image Captioning），一种理论引导的多阶段推理框架。它集成了视觉语义解析、基于幽默和心理学的推理以及用于幽默评估的微调判别器，形成了可解释和可控的认知推理链。据我们所知，这是第一个将幽默理论的认知结构明确嵌入到多模式幽默生成中的工作，从而实现从视觉理解到幽默创作的结构化推理过程。 Meme-Image-No-Text、Oogiri-GO 和 OxfordTVG-HIC 数据集上的实验表明，HUMORCHAIN 在人类幽默偏好、Elo/BT 分数和语义多样性方面优于最先进的基线，表明理论驱动的结构化推理使大型语言模型能够生成与人类感知一致的幽默。

Title: RoSA: Enhancing Parameter-Efficient Fine-Tuning via RoPE-aware Selective Adaptation in Large Language Models

• Authors: Dayan Pan, Jingyuan Wang, Yilong Zhou, Jiawei Cheng, Pengyue Jia, Xiangyu Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21733
• Pdf URL: https://arxiv.org/pdf/2511.21733
• Copy Paste: [[2511.21733]] RoSA: Enhancing Parameter-Efficient Fine-Tuning via RoPE-aware Selective Adaptation in Large Language Models(https://arxiv.org/abs/2511.21733)
• Keywords: language model
• Abstract: Fine-tuning large language models is essential for task-specific adaptation, yet it remains computationally prohibitive. Parameter-Efficient Fine-Tuning (PEFT) methods have emerged as a solution, but current approaches typically ignore the distinct roles of model components and the heterogeneous importance across layers, thereby limiting adaptation efficiency. Motivated by the observation that Rotary Position Embeddings (RoPE) induce critical activations in the low-frequency dimensions of attention states, we propose RoPE-aware Selective Adaptation (RoSA), a novel PEFT framework that allocates trainable parameters in a more targeted and effective manner. RoSA comprises a RoPE-aware Attention Enhancement (RoAE) module, which selectively enhances the low-frequency components of RoPE-influenced attention states, and a Dynamic Layer Selection (DLS) strategy that adaptively identifies and updates the most critical layers based on LayerNorm gradient norms. By combining dimension-wise enhancement with layer-wise adaptation, RoSA achieves more targeted and efficient fine-tuning. Extensive experiments on fifteen commonsense and arithmetic benchmarks demonstrate that RoSA outperforms existing mainstream PEFT methods under comparable trainable parameters. The code is available to ease reproducibility at this https URL.
• 摘要：微调大型语言模型对于特定任务的适应至关重要，但它在计算上仍然令人望而却步。参数高效微调（PEFT）方法已经作为一种解决方案出现，但当前的方法通常忽略模型组件的独特作用以及跨层的异构重要性，从而限制了适应效率。受到旋转位置嵌入（RoPE）在注意力状态低频维度中诱导关键激活的观察的启发，我们提出了 RoPE 感知选择性适应（RoSA），这是一种新颖的 PEFT 框架，它以更有针对性和更有效的方式分配可训练参数。 RoSA 包括 RoPE 感知注意力增强 (RoAE) 模块和动态层选择 (DLS) 策略，前者有选择地增强受 RoPE 影响的注意力状态的低频分量，后者基于 LayerNorm 梯度范数自适应地识别和更新最关键的层。通过将维度增强与分层自适应相结合，RoSA 实现了更有针对性、更高效的微调。对 15 个常识和算术基准的广泛实验表明，RoSA 在可比较的可训练参数下优于现有主流 PEFT 方法。该代码可用于简化此 https URL 的重现性。

Title: Asking LLMs to Verify First is Almost Free Lunch

• Authors: Shiguang Wu, Quanming Yao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21734
• Pdf URL: https://arxiv.org/pdf/2511.21734
• Copy Paste: [[2511.21734]] Asking LLMs to Verify First is Almost Free Lunch(https://arxiv.org/abs/2511.21734)
• Keywords: language model, llm, prompt, chain-of-thought, agent
• Abstract: To enhance the reasoning capabilities of Large Language Models (LLMs) without high costs of training, nor extensive test-time sampling, we introduce Verification-First (VF), a strategy that prompts models to verify a provided candidate answer, even a trivial or random one, before generating a solution. This approach triggers a "reverse reasoning" process that is cognitively easier and complementary to standard forward Chain-of-Thought (CoT), effectively invoking the model's critical thinking to reduce logical errors. We further generalize the VF strategy to Iter-VF, a sequential test-time scaling (TTS) method that iteratively cycles the verification-generation process using the model's previous answer. Extensive experiments across various benchmarks (from mathematical reasoning to coding and agentic tasks) and various LLMs (from open-source 1B to cutting-edge commercial ones) confirm that VF with random answer consistently outperforms standard CoT with minimal computational overhead, and Iter-VF outperforms existing TTS strategies.
• 摘要：为了增强大型语言模型 (LLM) 的推理能力，同时无需高昂的训练成本或大量的测试时间采样，我们引入了验证优先 (VF)，这是一种提示模型在生成解决方案之前验证所提供的候选答案（即使是微不足道或随机的答案）的策略。这种方法触发了“反向推理”过程，该过程在认知上更容易，并且是对标准正向思维链 (CoT) 的补充，有效地调用模型的批判性思维来减少逻辑错误。我们进一步将 VF 策略推广到 Iter-VF，这是一种顺序测试时间缩放 (TTS) 方法，它使用模型先前的答案迭代地循环验证生成过程。跨各种基准（从数学推理到编码和代理任务）和各种 LLM（从开源 1B 到尖端商业 LLM）的广泛实验证实，具有随机答案的 VF 始终优于标准 CoT，且计算开销最小，而 Iter-VF 优于现有 TTS 策略。

Title: R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization

• Authors: Jiayi Chen, Jieqi Shi, Jing Huo, Chen Wu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21736
• Pdf URL: https://arxiv.org/pdf/2511.21736
• Copy Paste: [[2511.21736]] R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization(https://arxiv.org/abs/2511.21736)
• Keywords: language model, llm
• Abstract: The rapid progress of Large Language Models (LLMs) has brought substantial computational and memory demands, spurring the adoption of low-bit quantization. While 8-bit and 4-bit formats have become prevalent, extending quantization to 2 bits remains challenging due to severe accuracy degradation. To address this, we propose Residual Refinement Quantization (R2Q)-a novel 2-bit quantization framework that decomposes the process into two sequential 1-bit sub-quantizations, forming an adaptive quantization lattice. Extensive evaluations on Llama, OPT, and Qwen across diverse benchmarks-covering question answering, commonsense reasoning, and language modeling-demonstrate that R2Q consistently outperforms existing 2-bit quantization methods in both fine-grained and coarse-grained settings. By refining quantization through a residual learning mechanism, R2Q enhances performance, improves training stability, and accelerates convergence under extreme compression. Furthermore, its modular design enables seamless integration with existing quantization-aware training (QAT) frameworks.
• 摘要：大型语言模型（LLM）的快速发展带来了巨大的计算和内存需求，刺激了低位量化的采用。虽然 8 位和 4 位格式已变得普遍，但由于精度严重下降，将量化扩展到 2 位仍然具有挑战性。为了解决这个问题，我们提出了残差细化量化（R2Q）——一种新颖的 2 位量化框架，它将过程分解为两个连续的 1 位子量化，形成自适应量化格。对 Llama、OPT 和 Qwen 跨不同基准（涵盖问答、常识推理和语言建模）的广泛评估表明，R2Q 在细粒度和粗粒度设置中始终优于现有的 2 位量化方法。通过残差学习机制细化量化，R2Q 增强了性能，提高了训练稳定性，并加速了极限压缩下的收敛。此外，其模块化设计可以与现有的量化感知训练（QAT）框架无缝集成。

Title: Polarity-Aware Probing for Quantifying Latent Alignment in Language Models

• Authors: Sabrina Sadiekh, Elena Ericheva, Chirag Agarwal
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21737
• Pdf URL: https://arxiv.org/pdf/2511.21737
• Copy Paste: [[2511.21737]] Polarity-Aware Probing for Quantifying Latent Alignment in Language Models(https://arxiv.org/abs/2511.21737)
• Keywords: language model
• Abstract: Advances in unsupervised probes such as Contrast-Consistent Search (CCS), which reveal latent beliefs without relying on token outputs, raise the question of whether these methods can reliably assess model alignment. We investigate this by examining the sensitivity of CCS to harmful vs. safe statements and by introducing Polarity-Aware CCS (PA-CCS), a method for evaluating whether a model's internal representations remain consistent under polarity inversion. We propose two alignment-oriented metrics, Polar-Consistency and the Contradiction Index, to quantify the semantic robustness of a model's latent knowledge. To validate PA-CCS, we curate two main datasets and one control dataset containing matched harmful-safe sentence pairs constructed using different methodologies (concurrent and antagonistic statements). We apply PA-CCS to 16 language models. Our results show that PA-CCS identifies both architectural and layer-specific differences in the encoding of latent harmful knowledge. Notably, replacing the negation token with a meaningless marker degrades PA-CCS scores for models with well-aligned internal representations, while models lacking robust internal calibration do not exhibit this degradation. Our findings highlight the potential of unsupervised probing for alignment evaluation and emphasize the need to incorporate structural robustness checks into interpretability benchmarks. Code and datasets are available at: this https URL. WARNING: This paper contains potentially sensitive, harmful, and offensive content.
• 摘要：对比一致性搜索（CCS）等无监督探测的进步，无需依赖标记输出即可揭示潜在信念，这引发了这些方法是否能够可靠地评估模型对齐的问题。我们通过检查 CCS 对有害与安全语句的敏感性并引入极性感知 CCS (PA-CCS) 来研究这一问题，这是一种评估模型内部表示在极性反转下是否保持一致的方法。我们提出了两个面向对齐的指标：极性一致性和矛盾指数，来量化模型潜在知识的语义鲁棒性。为了验证 PA-CCS，我们整理了两个主要数据集和一个控制数据集，其中包含使用不同方法（并发和对抗语句）构建的匹配的有害-安全句子对。我们将 PA-CCS 应用于 16 种语言模型。我们的结果表明，PA-CCS 识别了潜在有害知识编码中架构和特定层的差异。值得注意的是，用无意义的标记替换否定标记会降低具有良好对齐内部表示的模型的 PA-CCS 分数，而缺乏强大内部校准的模型不会表现出这种下降。我们的研究结果强调了无监督探测用于对齐评估的潜力，并强调需要将结构稳健性检查纳入可解释性基准。代码和数据集可在以下位置获得：此 https URL。警告：本文包含潜在的敏感、有害和冒犯性内容。

Title: Decoding inner speech with an end-to-end brain-to-text neural interface

• Authors: Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani, Liam Paninski
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21740
• Pdf URL: https://arxiv.org/pdf/2511.21740
• Copy Paste: [[2511.21740]] Decoding inner speech with an end-to-end brain-to-text neural interface(https://arxiv.org/abs/2511.21740)
• Keywords: language model, llm
• Abstract: Speech brain-computer interfaces (BCIs) aim to restore communication for people with paralysis by translating neural activity into text. Most systems use cascaded frameworks that decode phonemes before assembling sentences with an n-gram language model (LM), preventing joint optimization of all stages simultaneously. Here, we introduce an end-to-end Brain-to-Text (BIT) framework that translates neural activity into coherent sentences using a single differentiable neural network. Central to our approach is a cross-task, cross-species pretrained neural encoder, whose representations transfer to both attempted and imagined speech. In a cascaded setting with an n-gram LM, the pretrained encoder establishes a new state-of-the-art (SOTA) on the Brain-to-Text '24 and '25 benchmarks. Integrated end-to-end with audio large language models (LLMs) and trained with contrastive learning for cross-modal alignment, BIT reduces the word error rate (WER) of the prior end-to-end method from 24.69% to 10.22%. Notably, we find that small-scale audio LLMs markedly improve end-to-end decoding. Beyond record-setting performance, BIT aligns attempted and imagined speech embeddings to enable cross-task generalization. Altogether, our approach advances the integration of large, diverse neural datasets, paving the way for an end-to-end decoding framework that supports seamless, differentiable optimization.
• 摘要：语音脑机接口（BCI）旨在通过将神经活动转化为文本来恢复瘫痪患者的沟通能力。大多数系统使用级联框架，在使用 n-gram 语言模型 (LM) 组装句子之前解码音素，从而防止同时联合优化所有阶段。在这里，我们引入了一个端到端的大脑到文本（BIT）框架，该框架使用单个可微神经网络将神经活动转化为连贯的句子。我们方法的核心是跨任务、跨物种的预训练神经编码器，其表示可以转移到尝试的和想象的语音中。在使用 n-gram LM 的级联设置中，预训练编码器在 Brain-to-Text '24 和 '25 基准上建立了新的最先进 (SOTA)。 BIT 与音频大语言模型 (LLM) 进行端到端集成，并通过跨模态对齐的对比学习进行训练，将之前端到端方法的单词错误率 (WER) 从 24.69% 降低到 10.22%。值得注意的是，我们发现小规模音频法学硕士显着改善了端到端解码。除了创纪录的性能之外，BIT 还结合尝试和想象的语音嵌入来实现跨任务泛化。总而言之，我们的方法推进了大型、多样化神经数据集的集成，为支持无缝、可微优化的端到端解码框架铺平了道路。

Title: EduMod-LLM: A Modular Approach for Designing Flexible and Transparent Educational Assistants

• Authors: Meenakshi Mittal, Rishi Khare, Mihran Miroyan, Chancharik Mitra, Narges Norouzi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21742
• Pdf URL: https://arxiv.org/pdf/2511.21742
• Copy Paste: [[2511.21742]] EduMod-LLM: A Modular Approach for Designing Flexible and Transparent Educational Assistants(https://arxiv.org/abs/2511.21742)
• Keywords: language model, llm
• Abstract: With the growing use of Large Language Model (LLM)-based Question-Answering (QA) systems in education, it is critical to evaluate their performance across individual pipeline components. In this work, we introduce {\model}, a modular function-calling LLM pipeline, and present a comprehensive evaluation along three key axes: function calling strategies, retrieval methods, and generative language models. Our framework enables fine-grained analysis by isolating and assessing each component. We benchmark function-calling performance across LLMs, compare our novel structure-aware retrieval method to vector-based and LLM-scoring baselines, and evaluate various LLMs for response synthesis. This modular approach reveals specific failure modes and performance patterns, supporting the development of interpretable and effective educational QA systems. Our findings demonstrate the value of modular function calling in improving system transparency and pedagogical alignment. Website and Supplementary Material: this https URL
• 摘要：随着基于大型语言模型 (LLM) 的问答 (QA) 系统在教育中的使用越来越多，评估各个管道组件的性能至关重要。在这项工作中，我们引入了模块化函数调用LLM管道{\model}，并沿着三个关键轴进行了综合评估：函数调用策略、检索方法和生成语言模型。我们的框架通过隔离和评估每个组件来实现细粒度分析。我们对跨法学硕士的函数调用性能进行基准测试，将我们新颖的结构感知检索方法与基于向量和法学硕士评分基线进行比较，并评估各种法学硕士的响应综合。这种模块化方法揭示了特定的故障模式和性能模式，支持可解释且有效的教育质量保证系统的开发。我们的研究结果证明了模块化函数调用在提高系统透明度和教学一致性方面的价值。网站和补充材料：此 https URL

Title: Scaling Competence, Shrinking Reasoning: Cognitive Signatures in Language Model Learning

• Authors: Mukul Singh, Ananya Singha, Arjun Radhakrishna, Sumit Gulwani
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.21743
• Pdf URL: https://arxiv.org/pdf/2511.21743
• Copy Paste: [[2511.21743]] Scaling Competence, Shrinking Reasoning: Cognitive Signatures in Language Model Learning(https://arxiv.org/abs/2511.21743)
• Keywords: language model
• Abstract: We analyze reasoning in language models during task-specific fine-tuning and draws parallel between reasoning tokens--intermediate steps generated while solving problem and the human working memory. Drawing from cognitive science, we align training dynamics with the Four Stages of Competence: models initially produce incorrect outputs without reasoning, then begin reasoning (but still fail), eventually reason effectively, and finally solve tasks without explicit reasoning. We find that reasoning token length expands as performance improves, peaks at the stage of conscious competence, then declines as the model internalizes the task. Notably, after training, models retain performance even when reasoning is removed--suggesting it scaffolded learning but is no longer needed. This progression offers actionable insights: reasoning token dynamics can serve as a signal for diagnosing training stage, identifying convergence, and guiding early stopping. We propose metrics to track this trajectory and argue that reasoning behavior is valuable for understanding and optimizing reasoning model training.
• 摘要：我们在特定于任务的微调过程中分析语言模型中的推理，并将推理标记（解决问题时生成的中间步骤）与人类工作记忆进行比较。我们借鉴认知科学，将训练动态与能力的四个阶段相结合：模型最初在没有推理的情况下产生错误的输出，然后开始推理（但仍然失败），最终有效地推理，最后在没有明确推理的情况下解决任务。我们发现推理标记长度随着性能的提高而扩大，在有意识能力阶段达到顶峰，然后随着模型内化任务而下降。值得注意的是，训练后，即使推理被删除，模型也能保持性能——这表明它支撑着学习，但不再需要了。这一进展提供了可操作的见解：推理令牌动态可以作为诊断训练阶段、识别收敛和指导早期停止的信号。我们提出了跟踪这一轨迹的指标，并认为推理行为对于理解和优化推理模型训练很有价值。

Title: DELTA: Language Diffusion-based EEG-to-Text Architecture

• Authors: Mingyu Jeon, Hyobin Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.21746
• Pdf URL: https://arxiv.org/pdf/2511.21746
• Copy Paste: [[2511.21746]] DELTA: Language Diffusion-based EEG-to-Text Architecture(https://arxiv.org/abs/2511.21746)
• Keywords: language model
• Abstract: Electroencephalogram (EEG)-to-text remains challenging due to high-dimensional noise, subject variability, and error accumulation in autoregressive decoding. We introduce DELTA, which pairs a Residual Vector Quantization (RVQ) EEG tokenizer with a masked language diffusion model (LLaDA). RVQ discretizes continuous EEG into multi-layer tokens to reduce noise and individual differences, while LLaDA reconstructs sentences via non-sequential denoising. On ZuCo, DELTA improves semantic alignment by up to 5.37 points over autoregressive baselines, achieving BLEU-1 21.9 and ROUGE-1 F 17.2 under word-level conditions. These results enable reliable text generation from small EEG-text datasets and point toward scalable multimodal EEG-language models.
• 摘要：由于自回归解码中的高维噪声、受试者变异性和错误累积，脑电图 (EEG) 到文本仍然具有挑战性。我们引入了 DELTA，它将残差矢量量化 (RVQ) EEG 标记器与掩码语言扩散模型 (LLaDA) 配对。 RVQ 将连续脑电图离散化为多层标记，以减少噪声和个体差异，而 LLaDA 通过非序列去噪来重建句子。在 ZuCo 上，DELTA 比自回归基线提高了语义对齐高达 5.37 个点，在字级条件下实现了 BLEU-1 21.9 和 ROUGE-1 F 17.2。这些结果使得能够从小型脑电图文本数据集生成可靠的文本，并指向可扩展的多模式脑电图语言模型。

Title: Building Domain-Specific Small Language Models via Guided Data Generation

• Authors: Aman Kumar, Ekant Muljibhai Amin, Xian Yeow Lee, Lasitha Vidyaratne, Ahmed K. Farahat, Dipanjan D. Ghosh, Yuta Koreeda, Chetan Gupta
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21748
• Pdf URL: https://arxiv.org/pdf/2511.21748
• Copy Paste: [[2511.21748]] Building Domain-Specific Small Language Models via Guided Data Generation(https://arxiv.org/abs/2511.21748)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have shown remarkable success in supporting a wide range of knowledge-intensive tasks. In specialized domains, there is growing interest in leveraging LLMs to assist subject matter experts with domain-specific challenges. However, deploying LLMs as SaaS solutions raises data privacy concerns, while many open-source models demand significant computational resources for effective domain adaptation and deployment. A promising alternative is to develop smaller, domain-specialized LLMs, though this approach is often constrained by the lack of high-quality domain-specific training data. In this work, we address these limitations by presenting a cost-efficient and scalable training pipeline that combines guided synthetic data generation from a small seed corpus with bottom-up domain data curation. Our pipeline integrates Domain-Adaptive Pretraining (DAPT), Domain-specific Supervised Fine-tuning (DSFT), and Direct Preference Optimization (DPO) to train effective small-scale models for specialized use cases. We demonstrate this approach through DiagnosticSLM, a 3B-parameter domain-specific model tailored for fault diagnosis, root cause analysis, and repair recommendation in industrial settings. To evaluate model performance, we introduce four domain-specific benchmarks: multiple-choice questions (DiagnosticMCQ), question answering (DiagnosticQA), sentence completion (DiagnosticComp), and summarization (DiagnosticSum). DiagnosticSLM achieves up to 25% accuracy improvement over open-source models of comparable or larger size (2B-9B) on the MCQ task, while also outperforming or matching them in other tasks, demonstrating effective domain-specific reasoning and generalization capabilities.
• 摘要：大型语言模型 (LLM) 在支持广泛的知识密集型任务方面取得了显着的成功。在专业领域，人们越来越有兴趣利用法学硕士来协助主题专家应对特定领域的挑战。然而，将法学硕士部署为 SaaS 解决方案会引发数据隐私问题，而许多开源模型需要大量计算资源来进行有效的领域适应和部署。一个有前途的替代方案是开发较小的、特定领域的法学硕士，尽管这种方法通常受到缺乏高质量的特定领域培训数据的限制。在这项工作中，我们通过提出一种经济高效且可扩展的训练管道来解决这些限制，该管道将小型种子语料库的引导合成数据生成与自下而上的域数据管理相结合。我们的管道集成了领域自适应预训练（DAPT）、特定领域的监督微调（DSFT）和直接偏好优化（DPO），为特殊用例训练有效的小规模模型。我们通过 DiagnosticSLM 演示了这种方法，这是一种专为工业环境中的故障​​诊断、根本原因分析和维修建议而定制的 3B 参数域特定模型。为了评估模型性能，我们引入了四个特定领域的基准：多项选择题 (DiagnosticMCQ)、问答 (DiagnosticQA)、句子完成 (DiagnosticComp) 和摘要 (DiagnosticSum)。在 MCQ 任务上，DiagnosticSLM 比同等或更大尺寸 (2B-9B) 的开源模型的准确率提高了 25%，同时在其他任务中也优于或匹配它们，展示了有效的特定领域推理和泛化能力。

Title: Proactive Defense: Compound AI for Detecting Persuasion Attacks and Measuring Inoculation Effectiveness

• Authors: Svitlana Volkova, Will Dupree, Hsien-Te Kao, Peter Bautista, Gabe Ganberg, Jeff Beaubien, Laura Cassani
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21749
• Pdf URL: https://arxiv.org/pdf/2511.21749
• Copy Paste: [[2511.21749]] Proactive Defense: Compound AI for Detecting Persuasion Attacks and Measuring Inoculation Effectiveness(https://arxiv.org/abs/2511.21749)
• Keywords: gpt, llm, prompt, agent
• Abstract: This paper introduces BRIES, a novel compound AI architecture designed to detect and measure the effectiveness of persuasion attacks across information environments. We present a system with specialized agents: a Twister that generates adversarial content employing targeted persuasion tactics, a Detector that identifies attack types with configurable parameters, a Defender that creates resilient content through content inoculation, and an Assessor that employs causal inference to evaluate inoculation effectiveness. Experimenting with the SemEval 2023 Task 3 taxonomy across the synthetic persuasion dataset, we demonstrate significant variations in detection performance across language agents. Our comparative analysis reveals significant performance disparities with GPT-4 achieving superior detection accuracy on complex persuasion techniques, while open-source models like Llama3 and Mistral demonstrated notable weaknesses in identifying subtle rhetorical, suggesting that different architectures encode and process persuasive language patterns in fundamentally different ways. We show that prompt engineering dramatically affects detection efficacy, with temperature settings and confidence scoring producing model-specific variations; Gemma and GPT-4 perform optimally at lower temperatures while Llama3 and Mistral show improved capabilities at higher temperatures. Our causal analysis provides novel insights into socio-emotional-cognitive signatures of persuasion attacks, revealing that different attack types target specific cognitive dimensions. This research advances generative AI safety and cognitive security by quantifying LLM-specific vulnerabilities to persuasion attacks and delivers a framework for enhancing human cognitive resilience through structured interventions before exposure to harmful content.
• 摘要：本文介绍了 BRIES，这是一种新型复合人工智能架构，旨在检测和测量跨信息环境说服攻击的有效性。我们提出了一个具有专门代理的系统：一个使用有针对性的说服策略生成对抗性内容的 Twister，一个使用可配置参数识别攻击类型的检测器，一个通过内容接种创建弹性内容的防御者，以及一个使用因果推理来评估接种有效性的评估器。通过在合成说服数据集中使用 SemEval 2023 任务 3 分类法进行实验，我们证明了不同语言代理的检测性能存在显着差异。我们的比较分析揭示了与 GPT-4 在复杂说服技术上实现卓越检测准确性的显着性能差异，而 Llama3 和 Mistral 等开源模型在识别微妙修辞方面表现出明显的弱点，这表明不同的架构以根本不同的方式编码和处理说服性语言模式。我们表明，即时工程会极大地影响检测效率，温度设置和置信度评分会产生特定于模型的变化； Gemma 和 GPT-4 在较低温度下表现最佳，而 Llama3 和 Mistral 在较高温度下表现出改进的性能。我们的因果分析为说服攻击的社会情感认知特征提供了新颖的见解，揭示了不同的攻击类型针对特定的认知维度。这项研究通过量化法学硕士特定的说服攻击漏洞来推进生成人工智能安全和认知安全，并提供一个框架，通过在接触有害内容之前进行结构化干预来增强人类认知弹性。

Title: Semantics as a Shield: Label Disguise Defense (LDD) against Prompt Injection in LLM Sentiment Classification

• Authors: Yanxi Li, Ruocheng Shan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21752
• Pdf URL: https://arxiv.org/pdf/2511.21752
• Copy Paste: [[2511.21752]] Semantics as a Shield: Label Disguise Defense (LDD) against Prompt Injection in LLM Sentiment Classification(https://arxiv.org/abs/2511.21752)
• Keywords: language model, gpt, llm, prompt
• Abstract: Large language models are increasingly used for text classification tasks such as sentiment analysis, yet their reliance on natural language prompts exposes them to prompt injection attacks. In particular, class-directive injections exploit knowledge of the model's label set (e.g., positive vs. negative) to override its intended behavior through adversarial instructions. Existing defenses, such as detection-based filters, instruction hierarchies, and signed prompts, either require model retraining or remain vulnerable to obfuscation. This paper introduces Label Disguise Defense (LDD), a lightweight and model-agnostic strategy that conceals true labels by replacing them with semantically transformed or unrelated alias labels(e.g., blue vs. yellow). The model learns these new label mappings implicitly through few-shot demonstrations, preventing direct correspondence between injected directives and decision outputs. We evaluate LDD across nine state-of-the-art models, including GPT-5, GPT-4o, LLaMA3.2, Gemma3, and Mistral variants, under varying few-shot and an adversarial setting. Our results show that the ability of LDD to recover performance lost to the adversarial attack varies across models and alias choices. For every model evaluated, LDD is able to restore a portion of the accuracy degradation caused by the attack. Moreover, for the vast majority of models, we can identify more than one alias pair that achieves higher accuracy than the under-attack baseline, in which the model relies solely on few-shot learning without any defensive mechanism. A linguistic analysis further reveals that semantically aligned alias labels(e.g., good vs. bad) yield stronger robustness than unaligned symbols(e.g., blue vs. yellow). Overall, this study demonstrates that label semantics can serve as an effective defense layer, transforming meaning itself into a shield against prompt injection.
• 摘要：大型语言模型越来越多地用于文本分类任务，例如情感分析，但它们对自然语言提示的依赖使它们面临提示注入攻击。特别是，类指令注入利用模型标签集的知识（例如，正面与负面）来通过对抗性指令覆盖其预期行为。现有的防御措施，例如基于检测的过滤器、指令层次结构和签名提示，要么需要模型重新训练，要么仍然容易受到混淆。本文介绍了标签伪装防御（LDD），这是一种轻量级且与模型无关的策略，通过用语义转换或不相关的别名标签（例如蓝色与黄色）替换真实标签来隐藏真实标签。该模型通过几次演示隐式地学习这些新的标签映射，从而防止注入指令和决策输出之间的直接对应。我们在不同的少数样本和对抗性设置下评估了九种最先进模型的 LDD，包括 GPT-5、GPT-4o、LLaMA3.2、Gemma3 和 Mistral 变体。我们的结果表明，LDD 恢复因对抗性攻击而损失的性能的能力因模型和别名选择而异。对于每个评估的模型，LDD 都能够恢复部分由攻击造成的精度下降。此外，对于绝大多数模型，我们可以识别多个别名对，这些别名对比受到攻击的基线具有更高的准确性，其中模型仅依赖于小样本学习，没有任何防御机制。语言分析进一步表明，语义对齐的别名标签（例如，好与坏）比未对齐的符号（例如，蓝色与黄色）具有更强的鲁棒性。总的来说，这项研究表明标签语义可以作为有效的防御层，将含义本身转变为防止提示注入的盾牌。

Title: Extracting Disaster Impacts and Impact Related Locations in Social Media Posts Using Large Language Models

• Authors: Sameeah Noreen Hameed, Surangika Ranathunga, Raj Prasanna, Kristin Stock, Christopher B. Jones
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21753
• Pdf URL: https://arxiv.org/pdf/2511.21753
• Copy Paste: [[2511.21753]] Extracting Disaster Impacts and Impact Related Locations in Social Media Posts Using Large Language Models(https://arxiv.org/abs/2511.21753)
• Keywords: language model, llm
• Abstract: Large-scale disasters can often result in catastrophic consequences on people and infrastructure. Situation awareness about such disaster impacts generated by authoritative data from in-situ sensors, remote sensing imagery, and/or geographic data is often limited due to atmospheric opacity, satellite revisits, and time limitations. This often results in geo-temporal information gaps. In contrast, impact-related social media posts can act as "geo-sensors" during a disaster, where people describe specific impacts and locations. However, not all locations mentioned in disaster-related social media posts relate to an impact. Only the impacted locations are critical for directing resources effectively. e.g., "The death toll from a fire which ripped through the Greek coastal town of #Mati stood at 80, with dozens of people unaccounted for as forensic experts tried to identify victims who were burned alive #Greecefires #AthensFires #Athens #Greece." contains impacted location "Mati" and non-impacted locations "Greece" and "Athens". This research uses Large Language Models (LLMs) to identify all locations, impacts and impacted locations mentioned in disaster-related social media posts. In the process, LLMs are fine-tuned to identify only impacts and impacted locations (as distinct from other, non-impacted locations), including locations mentioned in informal expressions, abbreviations, and short forms. Our fine-tuned model demonstrates efficacy, achieving an F1-score of 0.69 for impact and 0.74 for impacted location extraction, substantially outperforming the pre-trained baseline. These robust results confirm the potential of fine-tuned language models to offer a scalable solution for timely decision-making in resource allocation, situational awareness, and post-disaster recovery planning for responders.
• 摘要：大规模灾害往往会对人员和基础设施造成灾难性后果。由于大气不透明、卫星重访和时间限制，来自现场传感器、遥感图像和/或地理数据的权威数据产生的对此类灾害影响的态势感知往往受到限制。这通常会导致地理时空信息差距。相比之下，与影响相关的社交媒体帖子可以在灾难期间充当“地理传感器”，人们可以在其中描述具体的影响和位置。然而，并非与灾难相关的社交媒体帖子中提到的所有地点都与影响有关。只有受影响的地点对于有效引导资源至关重要。例如，“席卷希腊沿海小镇#Mati的一场大火造成的死亡人数为80人，法医专家试图辨认被活活烧死的受害者时，有数十人下落不明#Greecefires #AthensFires #Athens #Greece。”包含受影响的位置“Mati”和未受影响的位置“希腊”和“雅典”。这项研究使用大型语言模型 (LLM) 来识别与灾难相关的社交媒体帖子中提到的所有位置、影响和受影响的位置。在此过程中，法学硕士经过微调，仅识别影响和受影响的地点（与其他未受影响的地点不同），包括非正式表达、缩写和简短形式中提到的地点。我们经过微调的模型展示了有效性，影响的 F1 分数为 0.69，受影响的位置提取的 F1 分数为 0.74，大大优于预训练的基线。这些稳健的结果证实了微调语言模型的潜力，可以为响应者在资源分配、态势感知和灾后恢复规划方面的及时决策提供可扩展的解决方案。

Title: Dissecting the Ledger: Locating and Suppressing "Liar Circuits" in Financial Large Language Models

• Authors: Soham Mirajkar
• Subjects: cs.CL, cs.CE
• Abstract URL: https://arxiv.org/abs/2511.21756
• Pdf URL: https://arxiv.org/pdf/2511.21756
• Copy Paste: [[2511.21756]] Dissecting the Ledger: Locating and Suppressing "Liar Circuits" in Financial Large Language Models(https://arxiv.org/abs/2511.21756)
• Keywords: language model, gpt, llm, hallucination
• Abstract: Large Language Models (LLMs) are increasingly deployed in high-stakes financial domains, yet they suffer from specific, reproducible hallucinations when performing arithmetic operations. Current mitigation strategies often treat the model as a black box. In this work, we propose a mechanistic approach to intrinsic hallucination detection. By applying Causal Tracing to the GPT-2 XL architecture on the ConvFinQA benchmark, we identify a dual-stage mechanism for arithmetic reasoning: a distributed computational scratchpad in middle layers (L12-L30) and a decisive aggregation circuit in late layers (specifically Layer 46). We verify this mechanism via an ablation study, demonstrating that suppressing Layer 46 reduces the model's confidence in hallucinatory outputs by 81.8%. Furthermore, we demonstrate that a linear probe trained on this layer generalizes to unseen financial topics with 98% accuracy, suggesting a universal geometry of arithmetic deception.
• 摘要：大型语言模型（LLM）越来越多地部署在高风险的金融领域，但它们在执行算术运算时会遭受特定的、可重现的幻觉。当前的缓解策略通常将模型视为黑匣子。在这项工作中，我们提出了一种内在幻觉检测的机械方法。通过将因果追踪应用于 ConvFinQA 基准上的 GPT-2 XL 架构，我们确定了一种用于算术推理的双阶段机制：中间层 (L12-L30) 的分布式计算暂存器和后期层（特别是第 46 层）的决定性聚合电路。我们通过消融研究验证了这一机制，证明抑制第 46 层会使模型对幻觉输出的置信度降低 81.8%。此外，我们证明了在该层上训练的线性探针可以以 98% 的准确率推广到看不见的金融主题，这表明算术欺骗的通用几何学。

Title: Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models

• Authors: Linye Wei, Wenjue Chen, Pingzhi Tang, Xiaotian Guo, Le Ye, Runsheng Wang, Meng Li
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21759
• Pdf URL: https://arxiv.org/pdf/2511.21759
• Copy Paste: [[2511.21759]] Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models(https://arxiv.org/abs/2511.21759)
• Keywords: language model, llm
• Abstract: Diffusion-based large language models (dLLMs) have recently gained significant attention for their exceptional performance and inherent potential for parallel decoding. Existing frameworks further enhance its inference efficiency by enabling KV caching. However, its bidirectional attention mechanism necessitates periodic cache refreshes that interleave prefill and decoding phases, both contributing substantial inference cost and constraining achievable speedup. Inspired by the heterogeneous arithmetic intensity of the prefill and decoding phases, we propose ODB-dLLM, a framework that orchestrates dual-boundaries to accelerate dLLM inference. In the prefill phase, we find that the predefined fixed response length introduces heavy yet redundant computational overhead, which affects efficiency. To alleviate this, ODB-dLLM incorporates an adaptive length prediction mechanism that progressively reduces prefill overhead and unnecessary computation. In the decoding phase, we analyze the computational characteristics of dLLMs and propose a dLLM-specific jump-share speculative decoding method to enhance efficiency by reducing the number of decoding iterations. Experimental results demonstrate that ODB-dLLM achieves 46-162x and 2.63-6.30x speedups over the baseline dLLM and Fast-dLLM, respectively, while simultaneously mitigating the accuracy degradation in existing acceleration frameworks.
• 摘要：基于扩散的大语言模型 (dLLM) 最近因其卓越的性能和并行解码的内在潜力而受到广泛关注。现有框架通过启用KV缓存进一步提高其推理效率。然而，其双向注意力机制需要定期缓存刷新，从而交错预填充和解码阶段，这两者都会产生大量的推理成本并限制可实现的加速。受预填充和解码阶段异构算术强度的启发，我们提出了 ODB-dLLM，这是一个协调双边界以加速 dLLM 推理的框架。在预填充阶段，我们发现预定义的固定响应长度引入了繁重且冗余的计算开销，从而影响了效率。为了缓解这一问题，ODB-dLLM 采用了自适应长度预测机制，可逐步减少预填充开销和不必要的计算。在解码阶段，我们分析了 dLLM 的计算特性，并提出了一种 dLLM 特定的跳跃共享推测解码方法，通过减少解码迭代次数来提高效率。实验结果表明，ODB-dLLM 分别比基线 dLLM 和 Fast-dLLM 实现了 46-162 倍和 2.63-6.30 倍的加速，同时减轻了现有加速框架中的精度下降。

Title: fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understanding

• Authors: Yuxiang Wei, Yanteng Zhang, Xi Xiao, Chengxuan Qian, Tianyang Wang, Vince D. Calhoun
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21760
• Pdf URL: https://arxiv.org/pdf/2511.21760
• Copy Paste: [[2511.21760]] fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understanding(https://arxiv.org/abs/2511.21760)
• Keywords: language model, llm
• Abstract: Recent advances in multimodal large language models (LLMs) have enabled unified reasoning across images, audio, and video, but extending such capability to brain imaging remains largely unexplored. Bridging this gap is essential to link neural activity with semantic cognition and to develop cross-modal brain representations. To this end, we present fMRI-LM, a foundational model that bridges functional MRI (fMRI) and language through a three-stage framework. In Stage 1, we learn a neural tokenizer that maps fMRI into discrete tokens embedded in a language-consistent space. In Stage 2, a pretrained LLM is adapted to jointly model fMRI tokens and text, treating brain activity as a sequence that can be temporally predicted and linguistically described. To overcome the lack of natural fMRI-text pairs, we construct a large descriptive corpus that translates diverse imaging-based features into structured textual descriptors, capturing the low-level organization of fMRI signals. In Stage 3, we perform multi-task, multi-paradigm instruction tuning to endow fMRI-LM with high-level semantic understanding, supporting diverse downstream applications. Across various benchmarks, fMRI-LM achieves strong zero-shot and few-shot performance, and adapts efficiently with parameter-efficient tuning (LoRA), establishing a scalable pathway toward a language-aligned, universal model for structural and semantic understanding of fMRI.
• 摘要：多模态大语言模型 (LLM) 的最新进展已经实现了跨图像、音频和视频的统一推理，但将这种能力扩展到大脑成像仍然很大程度上尚未探索。弥合这一差距对于将神经活动与语义认知联系起来并开发跨模式的大脑表征至关重要。为此，我们提出了 fMRI-LM，这是一种通过三阶段框架连接功能 MRI (fMRI) 和语言的基础模型。在第一阶段，我们学习一个神经标记器，它将 fMRI 映射成嵌入语言一致空间中的离散标记。在第二阶段，预训练的法学硕士适用于对 fMRI 标记和文本进行联合建模，将大脑活动视为可以进行时间预测和语言描述的序列。为了克服自然 fMRI 文本对的缺乏，我们构建了一个大型描述性语料库，将各种基于成像的特征转换为结构化文本描述符，捕获 fMRI 信号的低级组织。在第三阶段，我们进行多任务、多范式指令调整，赋予 fMRI-LM 高级语义理解，支持不同的下游应用。在各种基准测试中，fMRI-LM 实现了强大的零样本和少样本性能，并通过参数高效调整 (LoRA) 进行有效调整，为 fMRI 的结构和语义理解建立了一条通向语言一致的通用模型的可扩展途径。

Title: LLMs for Low-Resource Dialect Translation Using Context-Aware Prompting: A Case Study on Sylheti

• Authors: Tabia Tanzin Prama, Christopher M. Danforth, Peter Sheridan Dodds
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2511.21761
• Pdf URL: https://arxiv.org/pdf/2511.21761
• Copy Paste: [[2511.21761]] LLMs for Low-Resource Dialect Translation Using Context-Aware Prompting: A Case Study on Sylheti(https://arxiv.org/abs/2511.21761)
• Keywords: language model, gpt, llm, hallucination, prompt
• Abstract: Large Language Models (LLMs) have demonstrated strong translation abilities through prompting, even without task-specific training. However, their effectiveness in dialectal and low-resource contexts remains underexplored. This study presents the first systematic investigation of LLM-based machine translation (MT) for Sylheti, a dialect of Bangla that is itself low-resource. We evaluate five advanced LLMs (GPT-4.1, GPT-4.1, LLaMA 4, Grok 3, and DeepSeek V3.2) across both translation directions (Bangla $\Leftrightarrow$ Sylheti), and find that these models struggle with dialect-specific vocabulary. To address this, we introduce Sylheti-CAP (Context-Aware Prompting), a three-step framework that embeds a linguistic rulebook, a dictionary (2{,}260 core vocabulary items and idioms), and an authenticity check directly into prompts. Extensive experiments show that Sylheti-CAP consistently improves translation quality across models and prompting strategies. Both automatic metrics and human evaluations confirm its effectiveness, while qualitative analysis reveals notable reductions in hallucinations, ambiguities, and awkward phrasing, establishing Sylheti-CAP as a scalable solution for dialectal and low-resource MT. Dataset link: \href{this https URL}{this https URL}
• 摘要：即使没有特定任务的培训，大型语言模型 (LLM) 通过提示也表现出了强大的翻译能力。然而，它们在方言和资源匮乏的环境中的有效性仍未得到充分探索。这项研究首次对 Sylheti 语言进行基于 LLM 的机器翻译 (MT) 进行系统研究，Sylheti 是孟加拉语的一种方言，本身资源匮乏。我们评估了两个翻译方向（Bangla $\Leftrightarrow$ Sylheti）的五个高级 LLM（GPT-4.1、GPT-4.1、LLaMA 4、Grok 3 和 DeepSeek V3.2），发现这些模型在处理方言特定词汇方面存在困难。为了解决这个问题，我们引入了 Sylheti-CAP（上下文感知提示），这是一个三步框架，其中嵌入了语言规则手册、字典（2{,}260 个核心词汇项和习语）以及直接在提示中进行真实性检查。大量实验表明，Sylheti-CAP 能够持续提高跨模型和提示策略的翻译质量。自动指标和人工评估都证实了其有效性，而定性分析显示幻觉、歧义和尴尬措辞显着减少，使 Sylheti-CAP 成为方言和低资源 MT 的可扩展解决方案。数据集链接：\href{此 https URL}{此 https URL}

Title: Factors That Support Grounded Responses in LLM Conversations: A Rapid Review

• Authors: Gabriele Cesar Iwashima, Claudia Susie Rodrigues, Claudio Dipolitto, Geraldo Xexéo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21762
• Pdf URL: https://arxiv.org/pdf/2511.21762
• Copy Paste: [[2511.21762]] Factors That Support Grounded Responses in LLM Conversations: A Rapid Review(https://arxiv.org/abs/2511.21762)
• Keywords: language model, llm, hallucination
• Abstract: Large language models (LLMs) may generate outputs that are misaligned with user intent, lack contextual grounding, or exhibit hallucinations during conversation, which compromises the reliability of LLM-based applications. This review aimed to identify and analyze techniques that align LLM responses with conversational goals, ensure grounding, and reduce hallucination and topic drift. We conducted a Rapid Review guided by the PRISMA framework and the PICO strategy to structure the search, filtering, and selection processes. The alignment strategies identified were categorized according to the LLM lifecycle phase in which they operate: inference-time, post-training, and reinforcement learning-based methods. Among these, inference-time approaches emerged as particularly efficient, aligning outputs without retraining while supporting user intent, contextual grounding, and hallucination mitigation. The reviewed techniques provided structured mechanisms for improving the quality and reliability of LLM responses across key alignment objectives.
• 摘要：大型语言模型 (LLM) 可能会生成与用户意图不一致、缺乏上下文基础或在对话过程中表现出幻觉的输出，这会损害基于 LLM 的应用程序的可靠性。本次综述旨在识别和分析使法学硕士的回答与对话目标保持一致、确保接地气并减少幻觉和主题漂移的技术。我们在 PRISMA 框架和 PICO 策略的指导下进行了快速审查，以构建搜索、过滤和选择流程。确定的对齐策略根据其运行的法学硕士生命周期阶段进行分类：推理时间、训练后和基于强化学习的方法。其中，推理时间方法特别有效，无需重新训练即可调整输出，同时支持用户意图、上下文基础和幻觉缓解。审查的技术提供了结构化的机制，用于提高跨关键调整目标的法学硕士响应的质量和可靠性。

Title: FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers

• Authors: Sarina Xi, Vishisht Rao, Justin Payan, Nihar B. Shah
• Subjects: cs.CL, cs.AI, cs.DL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.21843
• Pdf URL: https://arxiv.org/pdf/2511.21843
• Copy Paste: [[2511.21843]] FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers(https://arxiv.org/abs/2511.21843)
• Keywords: language model, gpt, llm
• Abstract: The identification and localization of errors is a core task in peer review, yet the exponential growth of scientific output has made it increasingly difficult for human reviewers to reliably detect errors given the limited pool of experts. Recent advances in Large Language Models (LLMs) have sparked interest in their potential to support such evaluation tasks, from academic peer review to automated scientific assessment. However, despite the growing use of LLMs in review systems, their capabilities to pinpoint errors remain underexplored. In this work, we introduce Fault Localization Across Writing in Science (FLAWS), an automated benchmark consisting of 713 paper-error pairs designed to evaluate how effectively LLMs detect errors that undermine key claims in research papers. We construct the benchmark by systematically inserting claim-invalidating errors into peer-reviewed papers using LLMs, paired with an automated evaluation metric that measures whether models can identify and localize these errors. Developing such a benchmark presents unique challenges that we overcome: ensuring that the inserted errors are well-defined, challenging, and relevant to the content of the paper, avoiding artifacts that would make identification trivial, and designing a scalable, automated evaluation metric. On the resulting benchmark, we evaluate five frontier LLMs: Claude Sonnet 4.5, DeepSeek Reasoner v3.1, Gemini 2.5 Pro, GPT 5, and Grok 4. Among these, GPT 5 is the top-performing model, achieving 39.1% identification accuracy when k=10, where k is the number of top-ranked error text candidates generated by the LLM.
• 摘要：错误的识别和定位是同行评审的核心任务，但由于专家库有限，科学产出的指数级增长使得人类审稿人越来越难以可靠地发现错误。大型语言模型 (LLM) 的最新进展引发了人们对其支持此类评估任务（从学术同行评审到自动化科学评估）的潜力的兴趣。然而，尽管法学硕士在审查系统中的使用越来越多，但它们查明错误的能力仍未得到充分开发。在这项工作中，我们介绍了科学写作中的错误定位 (FLAWS)，这是一个由 713 个论文错误对组成的自动化基准，旨在评估法学硕士如何有效地检测破坏研究论文中关键主张的错误。我们通过使用法学硕士系统地将声明无效错误插入到同行评审的论文中来构建基准，并搭配自动评估指标来衡量模型是否能够识别和定位这些错误。开发这样的基准提出了我们要克服的独特挑战：确保插入的错误定义明确、具有挑战性并且与论文内容相关，避免使识别变得微不足道的伪影，并设计可扩展的自动化评估指标。在最终的基准测试中，我们评估了五个前沿的 LLM：Claude Sonnet 4.5、DeepSeek Reasoner v3.1、Gemini 2.5 Pro、GPT 5 和 Grok 4。其中，GPT 5 是表现最好的模型，当 k=10 时，识别准确率达到 39.1%，其中 k 是 LLM 生成的排名靠前的错误文本候选数。

Title: Improving Score Reliability of Multiple Choice Benchmarks with Consistency Evaluation and Altered Answer Choices

• Authors: Paulo Cavalin, Cassia Sanctos, Marcelo Grave, Claudio Pinhanez, Yago Primerano
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.21860
• Pdf URL: https://arxiv.org/pdf/2511.21860
• Copy Paste: [[2511.21860]] Improving Score Reliability of Multiple Choice Benchmarks with Consistency Evaluation and Altered Answer Choices(https://arxiv.org/abs/2511.21860)
• Keywords: language model, llm
• Abstract: In this work we present the Consistency-Rebalanced Accuracy (CoRA) metric, improving the reliability of Large Language Model (LLM) scores computed on multiple choice (MC) benchmarks. Our metric explores the response consistency of the LLMs, taking advantage of synthetically-generated questions with altered answer choices. With two intermediate scores, i.e. Bare-Minimum-Consistency Accuracy (BMCA) and Consistency Index (CI), CoRA is computed by adjusting the multiple-choice question answering (MCQA) scores to better reflect the level of consistency of the LLM. We present evaluations in different benchmarks using diverse LLMs, and not only demonstrate that LLMs can present low response consistency even when they present high MCQA scores, but also that CoRA can successfully scale down the scores of inconsistent models.
• 摘要：在这项工作中，我们提出了一致性重新平衡准确性 (CoRA) 指标，提高了根据多项选择 (MC) 基准计算的大型语言模型 (LLM) 分数的可靠性。我们的指标利用综合生成的问题和改变的答案选择来探索法学硕士的回答一致性。 CoRA有两个中间分数，即裸最低一致性准确率（BMCA）和一致性指数（CI），通过调整多项选择题回答（MCQA）分数来计算，以更好地反映LLM的一致性水平。我们使用不同的 LLM 在不同的基准中进行评估，不仅证明了 LLM 即使在提供高 MCQA 分数时也可以表现出较低的响应一致性，而且 CoRA 可以成功地降低不一致模型的分数。

Title: Tracing How Annotators Think: Augmenting Preference Judgments with Reading Processes

• Authors: Karin de Langis, William Walker, Khanh Chi Le, Dongyeop Kang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.21912
• Pdf URL: https://arxiv.org/pdf/2511.21912
• Copy Paste: [[2511.21912]] Tracing How Annotators Think: Augmenting Preference Judgments with Reading Processes(https://arxiv.org/abs/2511.21912)
• Keywords: prompt
• Abstract: We propose an annotation approach that captures not only labels but also the reading process underlying annotators' decisions, e.g., what parts of the text they focus on, re-read or skim. Using this framework, we conduct a case study on the preference annotation task, creating a dataset PreferRead that contains fine-grained annotator reading behaviors obtained from mouse tracking. PreferRead enables detailed analysis of how annotators navigate between a prompt and two candidate responses before selecting their preference. We find that annotators re-read a response in roughly half of all trials, most often revisiting the option they ultimately choose, and rarely revisit the prompt. Reading behaviors are also significantly related to annotation outcomes: re-reading is associated with higher inter-annotator agreement, whereas long reading paths and times are associated with lower agreement. These results demonstrate that reading processes provide a complementary cognitive dimension for understanding annotator reliability, decision-making and disagreement in complex, subjective NLP tasks. Our code and data are publicly available.
• 摘要：我们提出了一种注释方法，不仅捕获标签，还捕获注释者决策背后的阅读过程，例如，他们关注、重读或略读文本的哪些部分。使用这个框架，我们对偏好注释任务进行了案例研究，创建了一个数据集 PreferRead，其中包含从鼠标跟踪获得的细粒度注释器阅读行为。 PreferRead 可以详细分析注释者在选择偏好之前如何在提示和两个候选响应之间进行导航。我们发现，在大约一半的试验中，注释者会重新阅读响应，最常见的是重新访问他们最终选择的选项，而很少重新访问提示。阅读行为也与注释结果显着相关：重读与较高的注释者间一致性相关，而较长的阅读路径和时间与较低的一致性相关。这些结果表明，阅读过程为理解复杂、主观 NLP 任务中注释者的可靠性、决策和分歧提供了补充的认知维度。我们的代码和数据是公开的。

Title: A Comparative Study of LLM Prompting and Fine-Tuning for Cross-genre Authorship Attribution on Chinese Lyrics

• Authors: Yuxin Li, Lorraine Xu, Meng Fan Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.21930
• Pdf URL: https://arxiv.org/pdf/2511.21930
• Copy Paste: [[2511.21930]] A Comparative Study of LLM Prompting and Fine-Tuning for Cross-genre Authorship Attribution on Chinese Lyrics(https://arxiv.org/abs/2511.21930)
• Keywords: llm, prompt
• Abstract: We propose a novel study on authorship attribution for Chinese lyrics, a domain where clean, public datasets are sorely lacking. Our contributions are twofold: (1) we create a new, balanced dataset of Chinese lyrics spanning multiple genres, and (2) we develop and fine-tune a domain-specific model, comparing its performance against zero-shot inference using the DeepSeek LLM. We test two central hypotheses. First, we hypothesize that a fine-tuned model will outperform a zero-shot LLM baseline. Second, we hypothesize that performance is genre-dependent. Our experiments strongly confirm Hypothesis 2: structured genres (e.g. Folklore & Tradition) yield significantly higher attribution accuracy than more abstract genres (e.g. Love & Romance). Hypothesis 1 receives only partial support: fine-tuning improves robustness and generalization in Test1 (real-world data and difficult genres), but offers limited or ambiguous gains in Test2, a smaller, synthetically-augmented set. We show that the design limitations of Test2 (e.g., label imbalance, shallow lexical differences, and narrow genre sampling) can obscure the true effectiveness of fine-tuning. Our work establishes the first benchmark for cross-genre Chinese lyric attribution, highlights the importance of genre-sensitive evaluation, and provides a public dataset and analytical framework for future research. We conclude with recommendations: enlarge and diversify test sets, reduce reliance on token-level data augmentation, balance author representation across genres, and investigate domain-adaptive pretraining as a pathway for improved attribution performance.
• 摘要：我们提出了一项关于中文歌词作者归属的新颖研究，这是一个非常缺乏干净、公共数据集的领域。我们的贡献是双重的：(1) 我们创建了一个新的、平衡的跨多种流派的中文歌词数据集，(2) 我们开发并微调了特定领域的模型，使用 DeepSeek LLM 将其性能与零样本推理进行比较。我们检验两个中心假设。首先，我们假设经过微调的模型将优于零样本 LLM 基线。其次，我们假设表演取决于类型。我们的实验有力地证实了假设 2：结构化类型（例如民间传说和传统）比更抽象的类型（例如爱情和浪漫）产生更高的归因准确性。假设 1 仅得到部分支持：微调提高了 Test1（真实数据和困难类型）中的鲁棒性和泛化性，但在 Test2（较小的综合增强集）中提供了有限或模糊的增益。我们证明了 Test2 的设计局限性（例如，标签不平衡、浅层词汇差异和狭窄的体裁采样）可能会掩盖微调的真正有效性。我们的工作为跨流派的中文歌词归因建立了第一个基准，强调了流派敏感评估的重要性，并为未来的研究提供了公共数据集和分析框架。我们最后提出建议：扩大测试集并使其多样化，减少对标记级数据增强的依赖，平衡跨流派的作者代表性，并研究领域自适应预训练作为提高归因性能的途径。

Title: Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity

• Authors: Pamela D. Rivière, Sean Trott
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.21974
• Pdf URL: https://arxiv.org/pdf/2511.21974
• Copy Paste: [[2511.21974]] Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity(https://arxiv.org/abs/2511.21974)
• Keywords: language model
• Abstract: Despite an in-principle understanding of self-attention matrix operations in Transformer language models (LMs), it remains unclear precisely how these operations map onto interpretable computations or functions--and how or when individual attention heads develop specialized attention patterns. Here, we present a pipeline to systematically probe attention mechanisms, and we illustrate its value by leveraging lexical ambiguity--where a single word has multiple meanings--to isolate attention mechanisms that contribute to word sense disambiguation. We take a "developmental" approach: first, using publicly available Pythia LM checkpoints, we identify inflection points in disambiguation performance for each LM in the suite; in 14M and 410M, we identify heads whose attention to disambiguating words covaries with overall disambiguation performance across development. We then stress-test the robustness of these heads to stimulus perturbations: in 14M, we find limited robustness, but in 410M, we identify multiple heads with surprisingly generalizable behavior. Then, in a causal analysis, we find that ablating the target heads demonstrably impairs disambiguation performance, particularly in 14M. We additionally reproduce developmental analyses of 14M across all of its random seeds. Together, these results suggest: that disambiguation benefits from a constellation of mechanisms, some of which (especially in 14M) are highly sensitive to the position and part-of-speech of the disambiguating cue; and that larger models (410M) may contain heads with more robust disambiguation behavior. They also join a growing body of work that highlights the value of adopting a developmental perspective when probing LM mechanisms.
• 摘要：尽管原则上理解了 Transformer 语言模型 (LM) 中的自注意力矩阵运算，但仍不清楚这些运算如何准确地映射到可解释的计算或函数上，以及各个注意力头如何或何时形成专门的注意力模式。在这里，我们提出了一个系统地探索注意力机制的管道，并通过利用词汇歧义（单个单词具有多种含义）来隔离有助于词义消歧的注意力机制来说明其价值。我们采用“开发”方法：首先，使用公开可用的 Pythia LM 检查点，我们识别套件中每个 LM 消歧性能的拐点；在 14M 和 410M 中，我们确定了对消歧词的注意力随着开发过程中整体消歧性能的变化而变化的头。然后，我们对这些头对刺激扰动的鲁棒性进行压力测试：在 14M 中，我们发现鲁棒性有限，但在 410M 中，我们发现多个头具有令人惊讶的普遍行为。然后，在因果分析中，我们发现烧蚀目标头明显会损害消歧性能，特别是在 14M 中。我们还对 14M 的所有随机种子进行了发育分析。总之，这些结果表明：消歧受益于一系列机制，其中一些机制（尤其是 14M）对消歧提示的位置和词性高度敏感；并且较大的模型（410M）可能包含具有更稳健的消歧行为的头。他们还加入了越来越多的工作，强调在探索LM机制时采用发展视角的价值。

Title: AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models

• Authors: Yann Le Beux, Oluchi Audu, Oche D. Ankeli, Dhananjay Balakrishnan, Melissah Weya, Marie D. Ralaiarinosy, Ignatius Ezeani
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.22016
• Pdf URL: https://arxiv.org/pdf/2511.22016
• Copy Paste: [[2511.22016]] AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models(https://arxiv.org/abs/2511.22016)
• Keywords: language model, prompt
• Abstract: Existing AI bias evaluation benchmarks largely reflect Western perspectives, leaving African contexts underrepresented and enabling harmful stereotypes in applications across various domains. To address this gap, we introduce AfriStereo, the first open-source African stereotype dataset and evaluation framework grounded in local socio-cultural contexts. Through community engaged efforts across Senegal, Kenya, and Nigeria, we collected 1,163 stereotypes spanning gender, ethnicity, religion, age, and profession. Using few-shot prompting with human-in-the-loop validation, we augmented the dataset to over 5,000 stereotype-antistereotype pairs. Entries were validated through semantic clustering and manual annotation by culturally informed reviewers. Preliminary evaluation of language models reveals that nine of eleven models exhibit statistically significant bias, with Bias Preference Ratios (BPR) ranging from 0.63 to 0.78 (p <= 0.05), indicating systematic preferences for stereotypes over antistereotypes, particularly across age, profession, and gender dimensions. Domain-specific models appeared to show weaker bias in our setup, suggesting task-specific training may mitigate some associations. Looking ahead, AfriStereo opens pathways for future research on culturally grounded bias evaluation and mitigation, offering key methodologies for the AI community on building more equitable, context-aware, and globally inclusive NLP technologies.
• 摘要：现有的人工智能偏见评估基准在很大程度上反映了西方的观点，导致非洲背景代表性不足，并在各个领域的应用中造成有害的刻板印象。为了解决这一差距，我们引入了 AfriStereo，这是第一个基于当地社会文化背景的开源非洲刻板印象数据集和评估框架。通过塞内加尔、肯尼亚和尼日利亚的社区参与努力，我们收集了 1,163 种涵盖性别、种族、宗教、年龄和职业的刻板印象。通过使用少量镜头提示和人机交互验证，我们将数据集扩展到超过 5,000 个刻板印象-反刻板印象对。条目由了解文化的审稿人通过语义聚类和手动注释进行验证。对语言模型的初步评估表明，11 个模型中有 9 个表现出统计上显着的偏差，偏差偏好比 (BPR) 范围为 0.63 至 0.78 (p <= 0.05)，表明系统偏好刻板印象而非反刻板印象，特别是在年龄、职业和性别维度上。特定领域的模型似乎在我们的设置中表现出较弱的偏差，这表明特定任务的训练可能会减轻一些关联。展望未来，AfriStereo 为未来基于文化的偏见评估和缓解研究开辟了道路，为人工智能社区构建更公平、情境感知和全球包容性的 NLP 技术提供关键方法。

Title: Early Risk Prediction with Temporally and Contextually Grounded Clinical Language Processing

• Authors: Rochana Chaturvedi, Yue Zhou, Andrew Boyd, Brian T. Layden, Mudassir Rashid, Lu Cheng, Ali Cinar, Barbara Di Eugenio
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22038
• Pdf URL: https://arxiv.org/pdf/2511.22038
• Copy Paste: [[2511.22038]] Early Risk Prediction with Temporally and Contextually Grounded Clinical Language Processing(https://arxiv.org/abs/2511.22038)
• Keywords: language model
• Abstract: Clinical notes in Electronic Health Records (EHRs) capture rich temporal information on events, clinician reasoning, and lifestyle factors often missing from structured data. Leveraging them for predictive modeling can be impactful for timely identification of chronic diseases. However, they present core natural language processing (NLP) challenges: long text, irregular event distribution, complex temporal dependencies, privacy constraints, and resource limitations. We present two complementary methods for temporally and contextually grounded risk prediction from longitudinal notes. First, we introduce HiTGNN, a hierarchical temporal graph neural network that integrates intra-note temporal event structures, inter-visit dynamics, and medical knowledge to model patient trajectories with fine-grained temporal granularity. Second, we propose ReVeAL, a lightweight, test-time framework that distills the reasoning of large language models into smaller verifier models. Applied to opportunistic screening for Type 2 Diabetes (T2D) using temporally realistic cohorts curated from private and public hospital corpora, HiTGNN achieves the highest predictive accuracy, especially for near-term risk, while preserving privacy and limiting reliance on large proprietary models. ReVeAL enhances sensitivity to true T2D cases and retains explanatory reasoning. Our ablations confirm the value of temporal structure and knowledge augmentation, and fairness analysis shows HiTGNN performs more equitably across subgroups.
• 摘要：电子健康记录 (EHR) 中的临床记录捕获了结构化数据中经常缺失的有关事件、临床医生推理和生活方式因素的丰富时间信息。利用它们进行预测建模可以对及时识别慢性病产生影响。然而，它们提出了核心自然语言处理 (NLP) 挑战：长文本、不规则事件分布、复杂的时间依赖性、隐私约束和资源限制。我们提出了两种互补的方法，用于根据纵向笔记进行基于时间和上下文的风险预测。首先，我们介绍 HiTGNN，一种分层时间图神经网络，它集成了笔记内时间事件结构、访问间动态和医学知识，以细粒度时间粒度对患者轨迹进行建模。其次，我们提出了 ReVeAL，一个轻量级的测试时框架，它将大型语言模型的推理提炼为较小的验证器模型。 HiTGNN 应用于 2 型糖尿病 (T2D) 的机会性筛查，使用从私立和公立医院语料库中筛选的时间现实队列，实现了最高的预测准确性，特别是对于近期风险，同时保护隐私并限制对大型专有模型的依赖。 ReVeAL 增强了对真实 T2D 病例的敏感性并保留了解释性推理。我们的消融证实了时间结构和知识增强的价值，公平性分析表明 HiTGNN 在各个子组中表现更公平。

Title: A Hybrid Theory and Data-driven Approach to Persuasion Detection with Large Language Models

• Authors: Gia Bao Hoang, Keith J Ransom, Rachel Stephens, Carolyn Semmler, Nicolas Fay, Lewis Mitchell
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22109
• Pdf URL: https://arxiv.org/pdf/2511.22109
• Copy Paste: [[2511.22109]] A Hybrid Theory and Data-driven Approach to Persuasion Detection with Large Language Models(https://arxiv.org/abs/2511.22109)
• Keywords: language model, llm
• Abstract: Traditional psychological models of belief revision focus on face-to-face interactions, but with the rise of social media, more effective models are needed to capture belief revision at scale, in this rich text-based online discourse. Here, we use a hybrid approach, utilizing large language models (LLMs) to develop a model that predicts successful persuasion using features derived from psychological experiments. Our approach leverages LLM generated ratings of features previously examined in the literature to build a random forest classification model that predicts whether a message will result in belief change. Of the eight features tested, \textit{epistemic emotion} and \textit{willingness to share} were the top-ranking predictors of belief change in the model. Our findings provide insights into the characteristics of persuasive messages and demonstrate how LLMs can enhance models of successful persuasion based on psychological theory. Given these insights, this work has broader applications in fields such as online influence detection and misinformation mitigation, as well as measuring the effectiveness of online narratives.
• 摘要：传统的信念修正心理模型侧重于面对面的互动，但随着社交媒体的兴起，需要更有效的模型来捕捉这种基于丰富文本的在线话语中大规模的信念修正。在这里，我们使用混合方法，利用大型语言模型 (LLM) 开发一个模型，该模型使用从心理学实验得出的特征来预测成功的说服。我们的方法利用 LLM 生成的对文献中先前检查的特征的评级来构建随机森林分类模型，该模型预测消息是否会导致信念改变。在测试的八个特征中，\textit{认知情感}和\textit{分享意愿}是模型中信念变化的顶级预测因子。我们的研究结果提供了对说服性信息特征的见解，并展示了法学硕士如何基于心理学理论增强成功说服的模型。鉴于这些见解，这项工作在在线影响力检测和错误信息缓解以及衡量在线叙述的有效性等领域具有更广泛的应用。

Title: Bridging the Modality Gap by Similarity Standardization with Pseudo-Positive Samples

• Authors: Shuhei Yamashita, Daiki Shirafuji, Tatsuhiko Saito
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22141
• Pdf URL: https://arxiv.org/pdf/2511.22141
• Copy Paste: [[2511.22141]] Bridging the Modality Gap by Similarity Standardization with Pseudo-Positive Samples(https://arxiv.org/abs/2511.22141)
• Keywords: language model
• Abstract: Advances in vision-language models (VLMs) have enabled effective cross-modality retrieval. However, when both text and images exist in the database, similarity scores would differ in scale by modality. This phenomenon, known as the modality gap, hinders accurate retrieval. Most existing studies address this issue with manually labeled data, e.g., by fine-tuning VLMs on them. In this work, we propose a similarity standardization approach with pseudo data construction. We first compute the mean and variance of the similarity scores between each query and its paired data in text or image modality. Using these modality-specific statistics, we standardize all similarity scores to compare on a common scale across modalities. These statistics are calculated from pseudo pairs, which are constructed by retrieving the text and image candidates with the highest cosine similarity to each query. We evaluate our method across seven VLMs using two multi-modal QA benchmarks (MMQA and WebQA), where each question requires retrieving either text or image data. Our experimental results show that our method significantly improves retrieval performance, achieving average Recall@20 gains of 64% on MMQA and 28% on WebQA when the query and the target data belong to different modalities. Compared to E5-V, which addresses the modality gap through image captioning, we confirm that our method more effectively bridges the modality gap.
• 摘要：视觉语言模型（VLM）的进步实现了有效的跨模态检索。然而，当数据库中同时存在文本和图像时，相似性得分会因模态而异。这种现象被称为模态差距，阻碍了准确的检索。大多数现有研究都通过手动标记数据来解决这个问题，例如，通过对数据进行微调 VLM。在这项工作中，我们提出了一种具有伪数据构造的相似性标准化方法。我们首先计算每个查询及其文本或图像模态配对数据之间相似度得分的均值和方差。使用这些特定于模态的统计数据，我们标准化所有相似性分数，以便在跨模态的通用尺度上进行比较。这些统计数据是根据伪对计算的，伪对是通过检索与每个查询具有最高余弦相似度的文本和图像候选来构建的。我们使用两个多模式 QA 基准（MMQA 和 WebQA）在七个 VLM 上评估我们的方法，其中每个问题都需要检索文本或图像数据。我们的实验结果表明，当查询和目标数据属于不同模态时，我们的方法显着提高了检索性能，在 MMQA 上实现了 64% 的平均 Recall@20 增益，在 WebQA 上实现了 28% 的平均 Recall@20 增益。与通过图像字幕解决模态差距的 E5-V 相比，我们确认我们的方法更有效地弥合了模态差距。

Title: C$^2$DLM: Causal Concept-Guided Diffusion Large Language Models

• Authors: Kairong Han, Nuanqiao Shan, Ziyu Zhao, Zijing Hu, Xinpeng Dong, Junjian Ye, Lujia Pan, Fei Wu, Kun Kuang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22146
• Pdf URL: https://arxiv.org/pdf/2511.22146
• Copy Paste: [[2511.22146]] C$^2$DLM: Causal Concept-Guided Diffusion Large Language Models(https://arxiv.org/abs/2511.22146)
• Keywords: language model
• Abstract: Autoregressive (AR) language models and Diffusion Language Models (DLMs) constitute the two principal paradigms of large language models. However, both paradigms suffer from insufficient reasoning capabilities. Human reasoning inherently relies on causal knowledge and thought, which are reflected in natural language. But in the AR paradigm, language is modeled as next token prediction (a strictly left-to-right, token-by-token order), whereas natural language itself exhibits more flexible causal structures. In the DLM paradigm, the attention mechanism is fully connected, which entirely disregards causal order. To fill this gap, we propose a \underline{\textbf{C}}ausal \underline{\textbf{C}}oncept-Guided \underline{\textbf{D}}iffusion \underline{\textbf{L}}anguage \underline{\textbf{M}}odel (C$^2$DLM). Starting from DLM's fully connected attention, C$^2$DLM first obtains a concept-level causal graph from the teacher model, and then explicitly guides attention to learn causal relationships between concepts. By focusing on causal relationships and avoiding interference from difficult subgoals involving causal inversion, C$^2$DLM improves 12\% with about 3.2 times training speedup in the COT-OrderPerturb task, and achieves an average gain of 1.31\% across six downstream reasoning tasks. More details in the repository ~\href{this https URL}{here}.
• 摘要：自回归（AR）语言模型和扩散语言模型（DLM）构成了大型语言模型的两个主要范例。然而，这两种范式都存在推理能力不足的问题。人类推理本质上依赖于因果知识和思维，这些知识和思维反映在自然语言中。但在 AR 范式中，语言被建模为下一个标记预测（严格从左到右、逐个标记的顺序），而自然语言本身则表现出更灵活的因果结构。在DLM范式中，注意力机制是全连接的，完全无视因果顺序。为了填补这一空白，我们提出了一个 \underline{\textbf{C}}ausal \underline{\textbf{C}}oncept-Guided \underline{\textbf{D}}iffusion \underline{\textbf{L}}anguage \underline{\textbf{M}}模型（C$^2$DLM）。从DLM的全连接注意力开始，C$^2$DLM首先从教师模型中获得概念级因果图，然后显式引导注意力学习概念之间的因果关系。通过关注因果关系并避免涉及因果反转的困难子目标的干扰，C$^2$DLM 在 COT-OrderPerturb 任务中提高了 12%，训练速度提高了约 3.2 倍，并且在六个下游推理任务中实现了 1.31% 的平均增益。更多详细信息请参见存储库 ~\href{此 https URL}{此处}。

Title: A Theoretically Grounded Hybrid Ensemble for Reliable Detection of LLM-Generated Text

• Authors: Sepyan Purnama Kristanto, Lutfi Hakim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.22153
• Pdf URL: https://arxiv.org/pdf/2511.22153
• Copy Paste: [[2511.22153]] A Theoretically Grounded Hybrid Ensemble for Reliable Detection of LLM-Generated Text(https://arxiv.org/abs/2511.22153)
• Keywords: language model, gpt, llm
• Abstract: The rapid proliferation of Large Language Models (LLMs) has blurred the line between human and machine authorship, creating practical risks for academic integrity and information reliability. Existing text detectors typically rely on a single methodological paradigm and suffer from poor generalization and high false positive rates (FPR), especially on high-stakes academic text. We propose a theoretically grounded hybrid ensemble that systematically fuses three complementary detection paradigms: (i) a RoBERTa-based transformer classifier for deep semantic feature extraction, (ii) a GPT-2-based probabilistic detector using perturbation-induced likelihood curvature, and (iii) a statistical linguistic feature analyzer capturing stylometric patterns. The core novelty lies in an optimized weighted voting framework, where ensemble weights are learned on the probability simplex to maximize F1-score rather than set heuristically. We provide a bias-variance analysis and empirically demonstrate low inter-model correlation (rho ~ 0.35-0.42), a key condition for variance reduction. Evaluated on a large-scale, multigenerator corpus of 30,000 documents, our system achieves 94.2% accuracy and an AUC of 0.978, with a 35% relative reduction in false positives on academic text. This yields a more reliable and ethically responsible detector for real-world deployment in education and other high-stakes domains.
• 摘要：大型语言模型（LLM）的快速扩散模糊了人类和机器作者之间的界限，给学术诚信和信息可靠性带来了实际风险。现有的文本检测器通常依赖于单一的方法范式，并且存在泛化性差和误报率 (FPR) 高的问题，尤其是在高风险的学术文本上。我们提出了一种基于理论的混合集成，它系统地融合了三种互补的检测范式：（i）用于深度语义特征提取的基于 RoBERTa 的变换器分类器，（ii）使用扰动引起的似然曲率的基于 GPT-2 的概率检测器，以及（iii）捕获文体模式的统计语言特征分析器。核心新颖之处在于优化的加权投票框架，其中集合权重是在概率单纯形上学习的，以最大化 F1 分数，而不是启发式设置。我们提供偏差-方差分析，并凭经验证明模型间相关性较低（rho ~ 0.35-0.42），这是方差减少的关键条件。在包含 30,000 个文档的大规模多生成器语料库上进行评估，我们的系统实现了 94.2% 的准确率和 0.978 的 AUC，学术文本的误报相对减少了 35%。这将为教育和其他高风险领域的实际部署提供更可靠且道德负责的检测器。

Title: RefineBench: Evaluating Refinement Capability of Language Models via Checklists

• Authors: Young-Jun Lee, Seungone Kim, Byung-Kwan Lee, Minkyeong Moon, Yechan Hwang, Jong Myoung Kim, Graham Neubig, Sean Welleck, Ho-Jin Choi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22173
• Pdf URL: https://arxiv.org/pdf/2511.22173
• Copy Paste: [[2511.22173]] RefineBench: Evaluating Refinement Capability of Language Models via Checklists(https://arxiv.org/abs/2511.22173)
• Keywords: language model, gpt
• Abstract: Can language models (LMs) self-refine their own responses? This question is increasingly relevant as a wide range of real-world user interactions involve refinement requests. However, prior studies have largely tested LMs' refinement abilities on verifiable tasks such as competition math or symbolic reasoning with simplified scaffolds, whereas users often pose open-ended queries and provide varying degrees of feedback on what they desire. The recent advent of reasoning models that exhibit self-reflection patterns in their chains-of-thought further motivates this question. To analyze this, we introduce RefineBench, a benchmark of 1,000 challenging problems across 11 domains paired with a checklist-based evaluation framework. We evaluate two refinement modes: (1) guided refinement, where an LM is provided natural language feedback, and (2) self-refinement, where LMs attempt to improve without guidance. In the self-refinement setting, even frontier LMs such as Gemini 2.5 Pro and GPT-5 achieve modest baseline scores of 31.3% and 29.1%, respectively, and most models fail to consistently improve across iterations (e.g., Gemini-2.5-Pro gains only +1.8%, while DeepSeek-R1 declines by -0.1%). By contrast, in guided refinement, both proprietary LMs and large open-weight LMs (>70B) can leverage targeted feedback to refine responses to near-perfect levels within five turns. These findings suggest that frontier LMs require breakthroughs to self-refine their incorrect responses, and that RefineBench provides a valuable testbed for tracking progress.
• 摘要：语言模型 (LM) 可以自我完善自己的响应吗？随着现实世界中广泛的用户交互涉及细化请求，这个问题变得越来越重要。然而，之前的研究主要测试了 LM 在可验证任务上的细化能力，例如竞争数学或使用简化支架的符号推理，而用户经常提出开放式查询，并就他们想要的内容提供不同程度的反馈。最近出现的推理模型在其思维链中表现出自我反思模式，进一步引发了这个问题。为了分析这一点，我们引入了 RefineBench，这是一个涵盖 11 个领域的 1,000 个挑战性问题的基准，并配有基于清单的评估框架。我们评估两种细化模式：(1) 引导细化，其中向 LM 提供自然语言反馈；(2) 自我细化，其中 LM 尝试在没有指导的情况下进行改进。在自我优化设置中，即使是像 Gemini 2.5 Pro 和 GPT-5 这样的前沿 LM 也分别达到了 31.3% 和 29.1% 的适度基线分数，并且大多数模型无法在迭代中持续改进（例如，Gemini-2.5-Pro 仅增加了 +1.8%，而 DeepSeek-R1 则下降了 -0.1%）。相比之下，在引导细化中，专有 LM 和大型开放权重 LM (>70B) 都可以利用有针对性的反馈在五轮内将响应细化到近乎完美的水平。这些发现表明，前沿 LM 需要突破来自我改进其错误响应，而 RefineBench 为跟踪进度提供了一个有价值的测试平台。

Title: Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information

• Authors: Lukas Struppek, Dominik Hintersdorf, Hannah Struppek, Daniel Neider, Kristian Kersting
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.22176
• Pdf URL: https://arxiv.org/pdf/2511.22176
• Copy Paste: [[2511.22176]] Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information(https://arxiv.org/abs/2511.22176)
• Keywords: language model, llm, chain-of-thought
• Abstract: Recent large language models achieve strong reasoning performance by generating detailed chain-of-thought traces, but this often leads to excessive token use and high inference latency. Existing efficiency approaches typically focus on model-centric interventions, such as reinforcement learning or supervised fine-tuning, to reduce verbosity. In contrast, we propose a training-free, input-centric approach. Inspired by cognitive psychology, we introduce Focused Chain-of-Thought (F-CoT), which separates information extraction from the reasoning process. F-CoT first organizes the essential information from a query into a concise, structured context and then guides the model to reason exclusively over this context. By preventing attention to irrelevant details, F-CoT naturally produces shorter reasoning paths. On arithmetic word problems, F-CoT reduces generated tokens by 2-3x while maintaining accuracy comparable to standard zero-shot CoT. These results highlight structured input as a simple yet effective lever for more efficient LLM reasoning.
• 摘要：最近的大型语言模型通过生成详细的思想链轨迹来实现强大的推理性能，但这通常会导致过度的令牌使用和高推理延迟。现有的效率方法通常侧重于以模型为中心的干预措施，例如强化学习或监督微调，以减少冗长。相比之下，我们提出了一种免培训、以输入为中心的方法。受认知心理学的启发，我们引入了聚焦思维链（F-CoT），它将信息提取与推理过程分开。 F-CoT 首先将查询中的基本信息组织成简洁的结构化上下文，然后指导模型在此上下文中进行专门推理。通过防止对不相关细节的关注，F-CoT 自然会产生更短的推理路径。在算术应用题上，F-CoT 将生成的标记减少了 2-3 倍，同时保持了与标准零样本 CoT 相当的精度。这些结果强调结构化输入是一种简单而有效的杠杆，可实现更高效的法学硕士推理。

Title: Token-Level Marginalization for Multi-Label LLM Classifiers

• Authors: Anjaneya Praharaj, Jaykumar Kasundra
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22312
• Pdf URL: https://arxiv.org/pdf/2511.22312
• Copy Paste: [[2511.22312]] Token-Level Marginalization for Multi-Label LLM Classifiers(https://arxiv.org/abs/2511.22312)
• Keywords: language model, llm
• Abstract: This paper addresses the critical challenge of deriving interpretable confidence scores from generative language models (LLMs) when applied to multi-label content safety classification. While models like LLaMA Guard are effective for identifying unsafe content and its categories, their generative architecture inherently lacks direct class-level probabilities, which hinders model confidence assessment and performance interpretation. This limitation complicates the setting of dynamic thresholds for content moderation and impedes fine-grained error analysis. This research proposes and evaluates three novel token-level probability estimation approaches to bridge this gap. The aim is to enhance model interpretability and accuracy, and evaluate the generalizability of this framework across different instruction-tuned models. Through extensive experimentation on a synthetically generated, rigorously annotated dataset, it is demonstrated that leveraging token logits significantly improves the interpretability and reliability of generative classifiers, enabling more nuanced content safety moderation.
• 摘要：本文解决了应用于多标签内容安全分类时从生成语言模型 (LLM) 导出可解释置信度分数的关键挑战。虽然像 LLaMA Guard 这样的模型可以有效识别不安全内容及其类别，但它们的生成架构本质上缺乏直接的类级概率，这阻碍了模型置信度评估和性能解释。这种限制使内容审核的动态阈值的设置变得复杂，并阻碍了细粒度的错误分析。本研究提出并评估了三种新颖的令牌级概率估计方法来弥补这一差距。目的是增强模型的可解释性和准确性，并评估该框架在不同指令调整模型中的通用性。通过对综合生成的、严格注释的数据集进行大量实验，结果表明，利用 token logits 可以显着提高生成分类器的可解释性和可靠性，从而实现更细致的内容安全审核。

Title: Mapping Clinical Doubt: Locating Linguistic Uncertainty in LLMs

• Authors: Srivarshinee Sridhar, Raghav Kaushik Ravi, Kripabandhu Ghosh
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.22402
• Pdf URL: https://arxiv.org/pdf/2511.22402
• Copy Paste: [[2511.22402]] Mapping Clinical Doubt: Locating Linguistic Uncertainty in LLMs(https://arxiv.org/abs/2511.22402)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are increasingly used in clinical settings, where sensitivity to linguistic uncertainty can influence diagnostic interpretation and decision-making. Yet little is known about where such epistemic cues are internally represented within these models. Distinct from uncertainty quantification, which measures output confidence, this work examines input-side representational sensitivity to linguistic uncertainty in medical text. We curate a contrastive dataset of clinical statements varying in epistemic modality (e.g., 'is consistent with' vs. 'may be consistent with') and propose Model Sensitivity to Uncertainty (MSU), a layerwise probing metric that quantifies activation-level shifts induced by uncertainty cues. Our results show that LLMs exhibit structured, depth-dependent sensitivity to clinical uncertainty, suggesting that epistemic information is progressively encoded in deeper layers. These findings reveal how linguistic uncertainty is internally represented in LLMs, offering insight into their interpretability and epistemic reliability.
• 摘要：大语言模型 (LLM) 越来越多地用于临床环境，其中对语言不确定性的敏感性可能会影响诊断解释和决策。然而，人们对这些认知线索在这些模型内部的体现却知之甚少。与测量输出置信度的不确定性量化不同，这项工作检查输入端对医学文本中语言不确定性的表征敏感性。我们整理了认知模态不同的临床陈述的对比数据集（例如，“与……一致”与“可能与……一致”），并提出了不确定性模型敏感性（MSU），这是一种分层探测指标，可量化由不确定性线索引起的激活水平变化。我们的结果表明，法学硕士对临床不确定性表现出结构化、深度依赖的敏感性，这表明认知信息逐渐编码在更深的层次中。这些发现揭示了语言学不确定性如何在法学硕士内部体现，提供对其可解释性和认知可靠性的洞察。

Title: Exploring Performance Variations in Finetuned Translators of Ultra-Low Resource Languages: Do Linguistic Differences Matter?

• Authors: Isabel Gonçalves, Paulo Cavalin, Claudio Pinhanez
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.22482
• Pdf URL: https://arxiv.org/pdf/2511.22482
• Copy Paste: [[2511.22482]] Exploring Performance Variations in Finetuned Translators of Ultra-Low Resource Languages: Do Linguistic Differences Matter?(https://arxiv.org/abs/2511.22482)
• Keywords: language model
• Abstract: Finetuning pre-trained language models with small amounts of data is a commonly-used method to create translators for ultra-low resource languages such as endangered Indigenous languages. However, previous works have reported substantially different performances with translators created using similar methodology and data. In this work we systematically explored possible causes of the performance difference, aiming to determine whether it was a product of different cleaning procedures, limitations of the pre-trained models, the size of the base model, or the size of the training dataset, studying both directions of translation. Our studies, using two Brazilian Indigenous languages, related but with significant structural linguistic characteristics, indicated none or very limited influence from those training factors, suggesting differences between languages may play a significant role in the ability to produce translators by fine-tuning pre-trained models.
• 摘要：用少量数据微调预训练的语言模型是为濒危土著语言等超低资源语言创建翻译器的常用方法。然而，之前的作品报告了使用类似方法和数据创建的翻译人员的截然不同的表现。在这项工作中，我们系统地探讨了性能差异的可能原因，旨在确定它是否是不同清洁程序的产物、预训练模型的局限性、基础模型的大小或训练数据集的大小，并研究了两个翻译方向。我们的研究使用两种相关但具有显着结构语言特征的巴西土著语言，表明这些训练因素没有影响或影响非常有限，这表明语言之间的差异可能在通过微调预训练模型产生翻译的能力中发挥重要作用。

Title: Joint Speech and Text Training for LLM-Based End-to-End Spoken Dialogue State Tracking

• Authors: Katia Vendrame, Bolaji Yusuf, Santosh Kesiraju, Šimon Sedláček, Oldřich Plchot, Jan Černocký
• Subjects: cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2511.22503
• Pdf URL: https://arxiv.org/pdf/2511.22503
• Copy Paste: [[2511.22503]] Joint Speech and Text Training for LLM-Based End-to-End Spoken Dialogue State Tracking(https://arxiv.org/abs/2511.22503)
• Keywords: language model, llm
• Abstract: End-to-end spoken dialogue state tracking (DST) is made difficult by the tandem of having to handle speech input and data scarcity. Combining speech foundation encoders and large language models has been proposed in recent work as to alleviate some of this difficulty. Although this approach has been shown to result in strong spoken DST models, achieving state-of-the-art performance in realistic multi-turn DST, it struggles to generalize across domains and requires annotated spoken DST training data for each domain of interest. However, collecting such data for every target domain is both costly and difficult. Noting that textual DST data is more easily obtained for various domains, in this work, we propose jointly training on available spoken DST data and written textual data from other domains as a way to achieve cross-domain generalization. We conduct experiments which show the efficacy of our proposed method for getting good cross-domain DST performance without relying on spoken training data from the target domains.
• 摘要：由于必须处理语音输入和数据稀缺，端到端语音对话状态跟踪（DST）变得困难。最近的工作提出了将语音基础编码器和大型语言模型相结合，以缓解这一困难。尽管这种方法已被证明可以产生强大的口语 DST 模型，在现实的多轮 DST 中实现最先进的性能，但它很难跨领域推广，并且需要针对每个感兴趣的领域带注释的口语 DST 训练数据。然而，为每个目标域收集此类数据既昂贵又困难。注意到各个领域的文本 DST 数据更容易获得，在这项工作中，我们建议对可用的口语 DST 数据和来自其他领域的书面文本数据进行联合训练，作为实现跨领域泛化的一种方式。我们进行的实验证明了我们提出的方法在不依赖目标域的口语训练数据的情况下获得良好的跨域 DST 性能的有效性。

Title: Smarter, not Bigger: Fine-Tuned RAG-Enhanced LLMs for Automotive HIL Testing

• Authors: Chao Feng, Zihan Liu, Siddhant Gupta, Gongpei Cui, Jan von der Assen, Burkhard Stiller
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22584
• Pdf URL: https://arxiv.org/pdf/2511.22584
• Copy Paste: [[2511.22584]] Smarter, not Bigger: Fine-Tuned RAG-Enhanced LLMs for Automotive HIL Testing(https://arxiv.org/abs/2511.22584)
• Keywords: language model, gpt, llm, retrieval-augmented generation
• Abstract: Hardware-in-the-Loop (HIL) testing is essential for automotive validation but suffers from fragmented and underutilized test artifacts. This paper presents HIL-GPT, a retrieval-augmented generation (RAG) system integrating domain-adapted large language models (LLMs) with semantic retrieval. HIL-GPT leverages embedding fine-tuning using a domain-specific dataset constructed via heuristic mining and LLM-assisted synthesis, combined with vector indexing for scalable, traceable test case and requirement retrieval. Experiments show that fine-tuned compact models, such as \texttt{bge-base-en-v1.5}, achieve a superior trade-off between accuracy, latency, and cost compared to larger models, challenging the notion that bigger is always better. An A/B user study further confirms that RAG-enhanced assistants improve perceived helpfulness, truthfulness, and satisfaction over general-purpose LLMs. These findings provide insights for deploying efficient, domain-aligned LLM-based assistants in industrial HIL environments.
• 摘要：硬件在环 (HIL) 测试对于汽车验证至关重要，但其测试工件分散且未得到充分利用。本文提出了 HIL-GPT，这是一种将领域自适应大语言模型 (LLM) 与语义检索相结合的检索增强生成 (RAG) 系统。 HIL-GPT 利用通过启发式挖掘和 LLM 辅助合成构建的特定领域数据集进行嵌入微调，并结合矢量索引来实现可扩展、可追踪的测试用例和需求检索。实验表明，与较大的模型相比，经过微调的紧凑模型（例如 \texttt{bge-base-en-v1.5}）在准确性、延迟和成本之间实现了出色的权衡，挑战了“越大越好”的观念。一项 A/B 用户研究进一步证实，与通用法学硕士相比，RAG 增强型助手可以提高感知帮助性、真实性和满意度。这些发现为在工业 HIL 环境中部署高效、基于领域的 LLM 助理提供了见解。

Title: Improving LLM-based Ontology Matching with fine-tuning on synthetic data

• Authors: Guilherme Sousa, Rinaldo Lima, Cassia Trojahn
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22612
• Pdf URL: https://arxiv.org/pdf/2511.22612
• Copy Paste: [[2511.22612]] Improving LLM-based Ontology Matching with fine-tuning on synthetic data(https://arxiv.org/abs/2511.22612)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) are increasingly being integrated into various components of Ontology Matching pipelines. This paper investigates the capability of LLMs to perform ontology matching directly on ontology modules and generate the corresponding alignments. Furthermore, it is explored how a dedicated fine-tuning strategy can enhance the model's matching performance in a zero-shot setting. The proposed method incorporates a search space reduction technique to select relevant subsets from both source and target ontologies, which are then used to automatically construct prompts. Recognizing the scarcity of reference alignments for training, a novel LLM-based approach is introduced for generating a synthetic dataset. This process creates a corpus of ontology submodule pairs and their corresponding reference alignments, specifically designed to fine-tune an LLM for the ontology matching task. The proposed approach was evaluated on the Conference, Geolink, Enslaved, Taxon, and Hydrography datasets from the OAEI complex track. The results demonstrate that the LLM fine-tuned on the synthetically generated data exhibits superior performance compared to the non-fine-tuned base model. The key contribution is a strategy that combines automatic dataset generation with fine-tuning to effectively adapt LLMs for ontology matching tasks.
• 摘要：大型语言模型 (LLM) 越来越多地集成到本体匹配管道的各个组件中。本文研究了法学硕士直接在本体模块上执行本体匹配并生成相应比对的能力。此外，还探讨了专用的微调策略如何在零样本设置中增强模型的匹配性能。所提出的方法结合了搜索空间缩减技术，从源本体和目标本体中选择相关子集，然后使用这些子集自动构建提示。认识到训练参考对齐的稀缺性，引入了一种基于 LLM 的新颖方法来生成合成数据集。此过程创建本体子模块对及其相应参考对齐的语料库，专门设计用于针对本体匹配任务微调法学硕士。所提出的方法在 OAEI 复杂轨道的 Conference、Geolink、Enslaved、Taxon 和 Hydrography 数据集上进行了评估。结果表明，与未微调的基础模型相比，对综合生成的数据进行微调的 LLM 表现出优越的性能。关键贡献是一种将自动数据集生成与微调相结合的策略，以有效地使法学硕士适应本体匹配任务。

Title: Modeling Romanized Hindi and Bengali: Dataset Creation and Multilingual LLM Integration

• Authors: Kanchon Gharami, Quazi Sarwar Muhtaseem, Deepti Gupta, Lavanya Elluri, Shafika Showkat Moni
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22769
• Pdf URL: https://arxiv.org/pdf/2511.22769
• Copy Paste: [[2511.22769]] Modeling Romanized Hindi and Bengali: Dataset Creation and Multilingual LLM Integration(https://arxiv.org/abs/2511.22769)
• Keywords: language model, llm
• Abstract: The development of robust transliteration techniques to enhance the effectiveness of transforming Romanized scripts into native scripts is crucial for Natural Language Processing tasks, including sentiment analysis, speech recognition, information retrieval, and intelligent personal assistants. Despite significant advancements, state-of-the-art multilingual models still face challenges in handling Romanized script, where the Roman alphabet is adopted to represent the phonetic structure of diverse languages. Within the South Asian context, where the use of Romanized script for Indo-Aryan languages is widespread across social media and digital communication platforms, such usage continues to pose significant challenges for cutting-edge multilingual models. While a limited number of transliteration datasets and models are available for Indo-Aryan languages, they generally lack sufficient diversity in pronunciation and spelling variations, adequate code-mixed data for large language model (LLM) training, and low-resource adaptation. To address this research gap, we introduce a novel transliteration dataset for two popular Indo-Aryan languages, Hindi and Bengali, which are ranked as the 3rd and 7th most spoken languages worldwide. Our dataset comprises nearly 1.8 million Hindi and 1 million Bengali transliteration pairs. In addition to that, we pre-train a custom multilingual seq2seq LLM based on Marian architecture using the developed dataset. Experimental results demonstrate significant improvements compared to existing relevant models in terms of BLEU and CER metrics.
• 摘要：开发强大的音译技术来提高将罗马字母转换为本地脚本的效率对于自然语言处理任务至关重要，包括情感分析、语音识别、信息检索和智能个人助理。尽管取得了重大进步，最先进的多语言模型在处理罗马化脚本方面仍然面临挑战，其中采用罗马字母来表示不同语言的语音结构。在南亚背景下，印度-雅利安语言的罗马化文字在社交媒体和数字通信平台上广泛使用，这种使用继续对尖端多语言模型构成重大挑战。虽然印度-雅利安语言可用的音译数据集和模型数量有限，但它们通常缺乏足够的发音和拼写变化多样性、用于大型语言模型 (LLM) 训练的足够的代码混合数据以及低资源适应性。为了解决这一研究空白，我们为两种流行的印度-雅利安语言（印地语和孟加拉语）引入了一个新颖的音译数据集，这两种语言分别被列为全球第三和第七大语言。我们的数据集包含近 180 万个印地语和 100 万个孟加拉语音译对。除此之外，我们还使用开发的数据集预训练基于 Marian 架构的自定义多语言 seq2seq LLM。实验结果表明，与现有相关模型相比，在 BLEU 和 CER 指标方面有显着改进。

Title: Mitigating Semantic Drift: Evaluating LLMs' Efficacy in Psychotherapy through MI Dialogue Summarization

• Authors: Vivek Kumar, Pushpraj Singh Rajawat, Eirini Ntoutsi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.22818
• Pdf URL: https://arxiv.org/pdf/2511.22818
• Copy Paste: [[2511.22818]] Mitigating Semantic Drift: Evaluating LLMs' Efficacy in Psychotherapy through MI Dialogue Summarization(https://arxiv.org/abs/2511.22818)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Recent advancements in large language models (LLMs) have shown their potential across both general and domain-specific tasks. However, there is a growing concern regarding their lack of sensitivity, factual incorrectness in responses, inconsistent expressions of empathy, bias, hallucinations, and overall inability to capture the depth and complexity of human understanding, especially in low-resource and sensitive domains such as psychology. To address these challenges, our study employs a mixed-methods approach to evaluate the efficacy of LLMs in psychotherapy. We use LLMs to generate precise summaries of motivational interviewing (MI) dialogues and design a two-stage annotation scheme based on key components of the Motivational Interviewing Treatment Integrity (MITI) framework, namely evocation, collaboration, autonomy, direction, empathy, and a non-judgmental attitude. Using expert-annotated MI dialogues as ground truth, we formulate multi-class classification tasks to assess model performance under progressive prompting techniques, incorporating one-shot and few-shot prompting. Our results offer insights into LLMs' capacity for understanding complex psychological constructs and highlight best practices to mitigate ``semantic drift" in therapeutic settings. Our work contributes not only to the MI community by providing a high-quality annotated dataset to address data scarcity in low-resource domains but also critical insights for using LLMs for precise contextual interpretation in complex behavioral therapy.
• 摘要：大型语言模型 (LLM) 的最新进展显示了它们在一般任务和特定领域任务中的潜力。然而，人们越来越担心他们缺乏敏感性、反应中的事实不正确、同理心表达不一致、偏见、幻觉以及总体上无法捕捉人类理解的深度和复杂性，特别是在心理学等资源匮乏和敏感的领域。为了应对这些挑战，我们的研究采用混合方法来评估法学硕士在心理治疗中的功效。我们使用法学硕士生成动机访谈（MI）对话的精确摘要，并根据动机访谈治疗完整性（MITI）框架的关键组成部分（即唤起、协作、自主、方向、同理心和非评判态度）设计两阶段注释方案。使用专家注释的 MI 对话作为基本事实，我们制定多类分类任务来评估渐进式提示技术下的模型性能，结合一次性和几次提示。我们的研究结果提供了关于法学硕士理解复杂心理结构的能力的见解，并强调了在治疗环境中减轻“语义漂移”的最佳实践。我们的工作不仅通过提供高质量的注释数据集来解决资源匮乏领域的数据稀缺问题，为 MI 社区做出贡献，而且还为使用法学硕士在复杂行为治疗中进行精确的上下文解释提供了重要见解。

Title: RAG System for Supporting Japanese Litigation Procedures: Faithful Response Generation Complying with Legal Norms

• Authors: Yuya Ishihara, Atsushi Keyaki, Hiroaki Yamada, Ryutaro Ohara, Mihoko Sumida
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2511.22858
• Pdf URL: https://arxiv.org/pdf/2511.22858
• Copy Paste: [[2511.22858]] RAG System for Supporting Japanese Litigation Procedures: Faithful Response Generation Complying with Legal Norms(https://arxiv.org/abs/2511.22858)
• Keywords: llm, retrieval-augmented generation
• Abstract: This study discusses the essential components that a Retrieval-Augmented Generation (RAG)-based LLM system should possess in order to support Japanese medical litigation procedures complying with legal norms. In litigation, expert commissioners, such as physicians, architects, accountants, and engineers, provide specialized knowledge to help judges clarify points of dispute. When considering the substitution of these expert roles with a RAG-based LLM system, the constraint of strict adherence to legal norms is imposed. Specifically, three requirements arise: (1) the retrieval module must retrieve appropriate external knowledge relevant to the disputed issues in accordance with the principle prohibiting the use of private knowledge, (2) the responses generated must originate from the context provided by the RAG and remain faithful to that context, and (3) the retrieval module must reference external knowledge with appropriate timestamps corresponding to the issues at hand. This paper discusses the design of a RAG-based LLM system that satisfies these requirements.
• 摘要：本研究讨论了基于检索增强生成（RAG）的法学硕士系统应具备的基本组成部分，以支持符合法律规范的日本医疗诉讼程序。在诉讼中，医生、建筑师、会计师和工程师等专家委员提供专业知识，帮助法官澄清争议点。当考虑用基于 RAG 的法学硕士体系替代这些专家角色时，会受到严格遵守法律规范的约束。具体来说，出现了三个要求：（1）检索模块必须根据禁止使用私有知识的原则检索与争议问题相关的适当外部知识，（2）生成的响应必须源自RAG提供的上下文并忠实于该上下文，以及（3）检索模块必须引用具有与当前问题相对应的适当时间戳的外部知识。本文讨论了满足这些要求的基于 RAG 的法学硕士系统的设计。

Title: JBE-QA: Japanese Bar Exam QA Dataset for Assessing Legal Domain Knowledge

• Authors: Zhihan Cao, Fumihito Nishino, Hiroaki Yamada, Nguyen Ha Thanh, Yusuke Miyao, Ken Satoh
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22869
• Pdf URL: https://arxiv.org/pdf/2511.22869
• Copy Paste: [[2511.22869]] JBE-QA: Japanese Bar Exam QA Dataset for Assessing Legal Domain Knowledge(https://arxiv.org/abs/2511.22869)
• Keywords: language model, llm
• Abstract: We introduce JBE-QA, a Japanese Bar Exam Question-Answering dataset to evaluate large language models' legal knowledge. Derived from the multiple-choice (tanto-shiki) section of the Japanese bar exam (2015-2024), JBE-QA provides the first comprehensive benchmark for Japanese legal-domain evaluation of LLMs. It covers the Civil Code, the Penal Code, and the Constitution, extending beyond the Civil Code focus of prior Japanese resources. Each question is decomposed into independent true/false judgments with structured contextual fields. The dataset contains 3,464 items with balanced labels. We evaluate 26 LLMs, including proprietary, open-weight, Japanese-specialised, and reasoning models. Our results show that proprietary models with reasoning enabled perform best, and the Constitution questions are generally easier than the Civil Code or the Penal Code questions.
• 摘要：我们引入了 JBE-QA，这是一个日本律师考试问答数据集，用于评估大型语言模型的法律知识。 JBE-QA 源自日本律师考试（2015-2024 年）的多项选择（tanto-shiki）部分，为法学硕士的日本法律领域评估​​提供了第一个综合基准。它涵盖了民法典、刑法典和宪法，超出了日本先前资源的民法典重点。每个问题都被分解为具有结构化上下文字段的独立的真/假判断。该数据集包含 3,464 个具有平衡标签的项目。我们评估了 26 个法学硕士，包括专有模型、开放权重模型、日本专业模型和推理模型。我们的结果表明，启用推理的专有模型表现最好，并且宪法问题通常比民法典或刑法典问题更容易。

Title: FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing

• Authors: Jingheng Ye, Shen Wang, Jiaqi Chen, Hebin Wang, Deqing Zou, Yanyu Zhu, Jiwei Tang, Hai-Tao Zheng, Ruitong Liu, Haoyang Li, Yanfeng Wang, Qingsong Wen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22883
• Pdf URL: https://arxiv.org/pdf/2511.22883
• Copy Paste: [[2511.22883]] FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing(https://arxiv.org/abs/2511.22883)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have transformed artificial intelligence, offering profound opportunities for educational applications. However, their ability to provide fine-grained educational feedback for K-12 English writing remains underexplored. In this paper, we challenge the error analysis and pedagogical skills of LLMs by introducing the problem of Fine-grained Error Analysis for English Learners and present the Fine-grained Error ANalysis for English Learners (FEANEL) Benchmark. The benchmark comprises 1,000 essays written by elementary and secondary school students, and a well-developed English writing error taxonomy. Each error is annotated by language education experts and categorized by type, severity, and explanatory feedback, using a part-of-speech-based taxonomy they co-developed. We evaluate state-of-the-art LLMs on the FEANEL Benchmark to explore their error analysis and pedagogical abilities. Experimental results reveal significant gaps in current LLMs' ability to perform fine-grained error analysis, highlighting the need for advancements in particular methods for educational applications.
• 摘要：大型语言模型 (LLM) 改变了人工智能，为教育应用提供了深刻的机会。然而，他们为 K-12 英语写作提供细粒度教育反馈的能力仍未得到充分探索。在本文中，我们通过引入英语学习者细粒度错误分析问题并提出英语学习者细粒度错误分析（FEANEL）基准来挑战法学硕士的错误分析和教学技能。该基准包括 1,000 篇中小学生撰写的论文，以及完善的英语写作错误分类法。每个错误都由语言教育专家进行注释，并使用他们共同开发的基于词性的分类法按类型、严重性和解释性反馈进行分类。我们根据 FEANEL 基准评估最先进的法学硕士，以探索他们的错误分析和教学能力。实验结果揭示了当前法学硕士执行细粒度错误分析的能力存在显着差距，凸显了教育应用特定方法的进步需求。

Title: Language-conditioned world model improves policy generalization by reading environmental descriptions

• Authors: Anh Nguyen, Stefan Lee
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.22904
• Pdf URL: https://arxiv.org/pdf/2511.22904
• Copy Paste: [[2511.22904]] Language-conditioned world model improves policy generalization by reading environmental descriptions(https://arxiv.org/abs/2511.22904)
• Keywords: agent
• Abstract: To interact effectively with humans in the real world, it is important for agents to understand language that describes the dynamics of the environment--that is, how the environment behaves--rather than just task instructions specifying "what to do". Understanding this dynamics-descriptive language is important for human-agent interaction and agent behavior. Recent work address this problem using a model-based approach: language is incorporated into a world model, which is then used to learn a behavior policy. However, these existing methods either do not demonstrate policy generalization to unseen games or rely on limiting assumptions. For instance, assuming that the latency induced by inference-time planning is tolerable for the target task or expert demonstrations are available. Expanding on this line of research, we focus on improving policy generalization from a language-conditioned world model while dropping these assumptions. We propose a model-based reinforcement learning approach, where a language-conditioned world model is trained through interaction with the environment, and a policy is learned from this model--without planning or expert demonstrations. Our method proposes Language-aware Encoder for Dreamer World Model (LED-WM) built on top of DreamerV3. LED-WM features an observation encoder that uses an attention mechanism to explicitly ground language descriptions to entities in the observation. We show that policies trained with LED-WM generalize more effectively to unseen games described by novel dynamics and language compared to other baselines in several settings in two environments: MESSENGER and this http URL highlight how the policy can leverage the trained world model before real-world deployment, we demonstrate the policy can be improved through fine-tuning on synthetic test trajectories generated by the world model.
• 摘要：为了与现实世界中的人类进行有效交互，智能体必须理解描述环境动态的语言（即环境的行为方式），而不仅仅是指定“做什么”的任务指令。理解这种动态描述语言对于人机交互和代理行为非常重要。最近的工作使用基于模型的方法解决了这个问题：将语言合并到世界模型中，然后使用该模型来学习行为策略。然而，这些现有的方法要么不能证明对未见过的游戏的政策泛化，要么依赖于限制性假设。例如，假设推理时间规划引起的延迟对于目标任务来说是可以容忍的，或者可以进行专家演示。在这一研究领域的扩展上，我们专注于改进基于语言条件的世界模型的政策泛化，同时放弃这些假设。我们提出了一种基于模型的强化学习方法，通过与环境的交互来训练语言调节的世界模型，并从该模型中学习策略——无需规划或专家演示。我们的方法提出了基于 DreamerV3 的 Dreamer World 模型 (LED-WM) 的语言感知编码器。 LED-WM 具有一个观察编码器，它使用注意机制对观察中的实体进行明确的基础语言描述。我们表明，与两个环境中的几种设置中的其他基线相比，使用 LED-WM 训练的策略可以更有效地泛化到由新颖的动态和语言描述的未见过的游戏：MESSENGER 和此 http URL 强调了策略如何在实际部署之前利用经过训练的世界模型，我们证明可以通过对世界模型生成的综合测试轨迹进行微调来改进策略。

Title: Visual Puns from Idioms: An Iterative LLM-T2IM-MLLM Framework

• Authors: Kelaiti Xiao, Liang Yang, Dongyu Zhang, Paerhati Tulajiang, Hongfei Lin
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.22943
• Pdf URL: https://arxiv.org/pdf/2511.22943
• Copy Paste: [[2511.22943]] Visual Puns from Idioms: An Iterative LLM-T2IM-MLLM Framework(https://arxiv.org/abs/2511.22943)
• Keywords: language model, gpt, llm, prompt
• Abstract: We study idiom-based visual puns--images that align an idiom's literal and figurative meanings--and present an iterative framework that coordinates a large language model (LLM), a text-to-image model (T2IM), and a multimodal LLM (MLLM) for automatic generation and evaluation. Given an idiom, the system iteratively (i) generates detailed visual prompts, (ii) synthesizes an image, (iii) infers the idiom from the image, and (iv) refines the prompt until recognition succeeds or a step limit is reached. Using 1,000 idioms as inputs, we synthesize a corresponding dataset of visual pun images with paired prompts, enabling benchmarking of both generation and understanding. Experiments across 10 LLMs, 10 MLLMs, and one T2IM (Qwen-Image) show that MLLM choice is the primary performance driver: GPT achieves the highest accuracies, Gemini follows, and the best open-source MLLM (Gemma) is competitive with some closed models. On the LLM side, Claude attains the strongest average performance for prompt generation.
• 摘要：我们研究基于习语的视觉双关语（对齐习语字面意义和比喻意义的图像），并提出一个迭代框架，协调大型语言模型 (LLM)、文本到图像模型 (T2IM) 和多模态 LLM (MLLM)，以进行自动生成和评估。给定一个习语，系统迭代地（i）生成详细的视觉提示，（ii）合成图像，（iii）从图像推断习语，以及（iv）细化提示，直到识别成功或达到步骤限制。使用 1,000 个习语作为输入，我们合成了相应的视觉双关语图像数据集和配对提示，从而实现了生成和理解的基准测试。 10 个 LLM、10 个 MLLM 和一个 T2IM (Qwen-Image) 的实验表明，MLLM 选择是主要的性能驱动因素：GPT 实现了最高的准确度，Gemini 紧随其后，最好的开源 MLLM (Gemma) 与一些封闭模型具有竞争力。在法学硕士方面，克劳德在即时生成方面获得了最强的平均表现。

Title: Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match

• Authors: Jinze Li, Yixing Xu, Guanchen Li, Shuo Yang, Jinfeng Xu, Xuanwu Yin, Dong Li, Edith C.H.Ngai, Emad Barsoum
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22972
• Pdf URL: https://arxiv.org/pdf/2511.22972
• Copy Paste: [[2511.22972]] Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match(https://arxiv.org/abs/2511.22972)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) achieve strong performance across diverse tasks but suffer from high inference latency due to their autoregressive generation. Speculative Decoding (SPD) mitigates this issue by verifying candidate tokens in parallel from a smaller draft model, yet its strict exact-match verification discards many semantically valid continuations. Moreover, existing training-based SPD methods often suffer from performance degradation on out-of-distribution (OOD) tasks. To this end, we propose Training-Free Loosely Speculative Decoding (FLy), a novel method that loosens the rigid verification criterion by leveraging the target model's self-corrective behavior to judge whether a draft-target mismatch remains semantically valid. FLy introduces a two-tier mechanism: an entropy-level gate that identifies whether the current token allows multiple plausible alternatives or is nearly deterministic, and a token-level deferred window that distinguishes genuine errors from differently worded yet semantically correct variants. To further reduce latency, we design a multi-level acceleration strategy that accelerates not only the target model but also the drafter itself. Owing to its training-free design, FLy composes seamlessly with arbitrary draft-target pairs and generalizes across models and domains without hyperparameter re-tuning. Experiments show that FLy preserves more than 99% of the target model's accuracy while achieving an average 2.81x speedup on Llama-3.1-70B-Instruct and 5.07x speedup on the 405B variant. Notably, on out-of-domain datasets, our method remains highly effective and outperforms the training-based method EAGLE-3 by 1.62x.
• 摘要：大型语言模型 (LLM) 在不同的任务中实现了强大的性能，但由于其自回归生成而受到高推理延迟的影响。推测解码 (SPD) 通过从较小的草稿模型并行验证候选标记来缓解此问题，但其严格的精确匹配验证会丢弃许多语义上有效的延续。此外，现有的基于训练的 SPD 方法在分布外 (OOD) 任务上常常会出现性能下降的问题。为此，我们提出了免训练松散推测解码（FLy），这是一种新方法，通过利用目标模型的自我纠正行为来判断草稿-目标不匹配在语义上是否仍然有效，从而放松严格的验证标准。 FLy 引入了一个两层机制：一个熵级门，用于识别当前令牌是否允许多个合理的替代方案或几乎是确定性的；以及一个令牌级延迟窗口，用于区分真正的错误和措辞不同但语义正确的变体。为了进一步减少延迟，我们设计了一种多级加速策略，不仅可以加速目标模型，还可以加速绘图器本身。由于其免训练设计，FLy 可以与任意草图目标对无缝组合，并在模型和领域之间进行泛化，而无需重新调整超参数。实验表明，FLy 保留了超过 99% 的目标模型精度，同时在 Llama-3.1-70B-Instruct 上实现平均 2.81 倍加速，在 405B 变体上实现 5.07 倍加速。值得注意的是，在域外数据集上，我们的方法仍然非常有效，并且比基于训练的方法 EAGLE-3 好 1.62 倍。

Title: Pooling Attention: Evaluating Pretrained Transformer Embeddings for Deception Classification

• Authors: Sumit Mamtani, Abhijeet Bhure
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.22977
• Pdf URL: https://arxiv.org/pdf/2511.22977
• Copy Paste: [[2511.22977]] Pooling Attention: Evaluating Pretrained Transformer Embeddings for Deception Classification(https://arxiv.org/abs/2511.22977)
• Keywords: gpt
• Abstract: This paper investigates fake news detection as a downstream evaluation of Transformer representations, benchmarking encoder-only and decoder-only pre-trained models (BERT, GPT-2, Transformer-XL) as frozen embedders paired with lightweight classifiers. Through controlled preprocessing comparing pooling versus padding and neural versus linear heads, results demonstrate that contextual self-attention encodings consistently transfer effectively. BERT embeddings combined with logistic regression outperform neural baselines on LIAR dataset splits, while analyses of sequence length and aggregation reveal robustness to truncation and advantages from simple max or average pooling. This work positions attention-based token encoders as robust, architecture-centric foundations for veracity tasks, isolating Transformer contributions from classifier complexity.
• 摘要：本文研究了假新闻检测作为 Transformer 表示的下游评估，对仅编码器和仅解码器的预训练模型（BERT、GPT-2、Transformer-XL）进行基准测试，将其作为与轻量级分类器配对的冻结嵌入器。通过受控预处理比较池化与填充以及神经头与线性头，结果表明上下文自注意力编码始终有效地传输。 BERT 嵌入与逻辑回归相结合，在 LIAR 数据集分割上优于神经基线，而序列长度和聚合的分析揭示了截断的鲁棒性以及简单最大或平均池的优势。这项工作将基于注意力的令牌编码器定位为准确性任务的强大的、以架构为中心的基础，将 Transformer 的贡献与分类器的复杂性隔离开来。

Title: ShoppingComp: Are LLMs Really Ready for Your Shopping Cart?

• Authors: Huaixiao Tou, Ying Zeng, Cong Ma, Muzhi Li, Minghao Li, Weijie Yuan, He Zhang, Kai Jia
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.22978
• Pdf URL: https://arxiv.org/pdf/2511.22978
• Copy Paste: [[2511.22978]] ShoppingComp: Are LLMs Really Ready for Your Shopping Cart?(https://arxiv.org/abs/2511.22978)
• Keywords: gpt, llm, agent
• Abstract: We present ShoppingComp, a challenging real-world benchmark for rigorously evaluating LLM-powered shopping agents on three core capabilities: precise product retrieval, expert-level report generation, and safety critical decision making. Unlike prior e-commerce benchmarks, ShoppingComp introduces highly complex tasks under the principle of guaranteeing real products and ensuring easy verifiability, adding a novel evaluation dimension for identifying product safety hazards alongside recommendation accuracy and report quality. The benchmark comprises 120 tasks and 1,026 scenarios, curated by 35 experts to reflect authentic shopping needs. Results reveal stark limitations of current LLMs: even state-of-the-art models achieve low performance (e.g., 11.22% for GPT-5, 3.92% for Gemini-2.5-Flash). These findings highlight a substantial gap between research benchmarks and real-world deployment, where LLMs make critical errors such as failure to identify unsafe product usage or falling for promotional misinformation, leading to harmful recommendations. ShoppingComp fills the gap and thus establishes a new standard for advancing reliable and practical agents in e-commerce.
• 摘要：我们推出 ShoppingComp，这是一个具有挑战性的现实世界基准，用于严格评估法学硕士支持的购物代理的三个核心功能：精确的产品检索、专家级报告生成和安全关键决策。与之前的电商基准不同，ShoppingComp在保证产品真实性和易于验证的原则下引入了高度复杂的任务，除了推荐准确性和报告质量之外，还增加了识别产品安全隐患的新颖评估维度。该基准包括 120 项任务和 1,026 个场景，由 35 名专家策划，以反映真实的购物需求。结果揭示了当前 LLM 的明显局限性：即使是最先进的模型也表现不佳（例如，GPT-5 为 11.22%，Gemini-2.5-Flash 为 3.92%）。这些发现突显了研究基准与现实世界部署之间的巨大差距，法学硕士犯了严重错误，例如未能识别不安全的产品使用或误入促销错误信息，从而导致有害的建议。 ShoppingComp 填补了这一空白，从而为电子商务中可靠且实用的代理建立了新的标准。

Title: Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses

• Authors: Dong Nguyen, Laura Rosseel
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.23041
• Pdf URL: https://arxiv.org/pdf/2511.23041
• Copy Paste: [[2511.23041]] Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses(https://arxiv.org/abs/2511.23041)
• Keywords: language model, llm
• Abstract: Spelling variation (e.g. funnnn vs. fun) can influence the social perception of texts and their writers: we often have various associations with different forms of writing (is the text informal? does the writer seem young?). In this study, we focus on the social perception of spelling variation in online writing in English and study to what extent this perception is aligned between humans and large language models (LLMs). Building on sociolinguistic methodology, we compare LLM and human ratings on three key social attributes of spelling variation (formality, carefulness, age). We find generally strong correlations in the ratings between humans and LLMs. However, notable differences emerge when we analyze the distribution of ratings and when comparing between different types of spelling variation.
• 摘要：拼写变化（例如 funnnn 与 fun）可以影响文本及其作者的社会认知：我们经常与不同形式的写作产生各种联系（文本是非正式的吗？作者看起来年轻吗？）。在这项研究中，我们重点关注英语在线写作中拼写变化的社会认知，并研究这种认知在人类和大型语言模型 (LLM) 之间的一致性程度。基于社会语言学方法论，我们比较了法学硕士和人类对拼写变异的三个关键社会属性（正式性、谨慎性、年龄）的评分。我们发现人类和法学硕士之间的评级通常具有很强的相关性。然而，当我们分析评分分布以及比较不同类型的拼写变化时，就会出现显着差异。

Title: Standard Occupation Classifier -- A Natural Language Processing Approach

• Authors: Sidharth Rony, Jack Patman
• Subjects: cs.CL, cs.LG, econ.GN
• Abstract URL: https://arxiv.org/abs/2511.23057
• Pdf URL: https://arxiv.org/pdf/2511.23057
• Copy Paste: [[2511.23057]] Standard Occupation Classifier -- A Natural Language Processing Approach(https://arxiv.org/abs/2511.23057)
• Keywords: language model
• Abstract: Standard Occupational Classifiers (SOC) are systems used to categorize and classify different types of jobs and occupations based on their similarities in terms of job duties, skills, and qualifications. Integrating these facets with Big Data from job advertisement offers the prospect to investigate labour demand that is specific to various occupations. This project investigates the use of recent developments in natural language processing to construct a classifier capable of assigning an occupation code to a given job advertisement. We develop various classifiers for both UK ONS SOC and US O*NET SOC, using different Language Models. We find that an ensemble model, which combines Google BERT and a Neural Network classifier while considering job title, description, and skills, achieved the highest prediction accuracy. Specifically, the ensemble model exhibited a classification accuracy of up to 61% for the lower (or fourth) tier of SOC, and 72% for the third tier of SOC. This model could provide up to date, accurate information on the evolution of the labour market using job advertisements.
• 摘要：标准职业分类器 (SOC) 是用于根据工作职责、技能和资格方面的相似性对不同类型的工作和职业进行分类的系统。将这些方面与招聘广告中的大数据相结合，为调查不同职业特定的劳动力需求提供了前景。该项目研究了利用自然语言处理的最新进展来构建能够为给定招聘广告分配职业代码的分类器。我们使用不同的语言模型为英国 ONS SOC 和美国 O*NET SOC 开发各种分类器。我们发现，结合了 Google BERT 和神经网络分类器并同时考虑职位、描述和技能的集成模型实现了最高的预测精度。具体来说，集成模型对于较低（或第四）层 SOC 的分类准确率高达 61%，对于第三层 SOC 的分类准确率高达 72%。该模型可以通过招聘广告提供有关劳动力市场演变的最新、准确的信息。

Title: Conveying Imagistic Thinking in TCM Translation: A Prompt Engineering and LLM-Based Evaluation Framework

• Authors: Jiatong Han
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.23059
• Pdf URL: https://arxiv.org/pdf/2511.23059
• Copy Paste: [[2511.23059]] Conveying Imagistic Thinking in TCM Translation: A Prompt Engineering and LLM-Based Evaluation Framework(https://arxiv.org/abs/2511.23059)
• Keywords: gpt, llm, prompt, chat
• Abstract: Traditional Chinese Medicine theory is built on imagistic thinking, in which medical principles and diagnostic and therapeutic logic are structured through metaphor and metonymy. However, existing English translations largely rely on literal rendering, making it difficult for target-language readers to reconstruct the underlying conceptual networks and apply them in clinical practice. This study adopted a human-in-the-loop framework and selected four passages from the medical canon Huangdi Neijing that are fundamental in theory. Through prompt-based cognitive scaffolding, DeepSeek V3.1 was guided to identify metaphor and metonymy in the source text and convey the theory in translation. In the evaluation stage, ChatGPT 5 Pro and Gemini 2.5 Pro were instructed by prompts to simulate three types of real-world readers. Human translations, baseline model translations, and prompt-adjusted translations were scored by the simulated readers across five cognitive dimensions, followed by structured interviews and Interpretative Phenomenological Analysis. Results show that the prompt-adjusted LLM translations perform best across all five dimensions, with high cross-model and cross-role consistency. The interview themes reveal differences between human and machine translation, effective strategies for metaphor and metonymy transfer, and readers' cognitive preferences. This study provides a cognitive, efficient and replicable HITL methodological pathway for translation of ancient, concept-dense texts like TCM.
• 摘要：中医理论建立在象思维的基础上，通过隐喻和转喻来构建医学原理和诊疗逻辑。然而，现有的英文翻译很大程度上依赖于直译，这使得目标语言读者很难重建底层概念网络并将其应用于临床实践。本研究采用了人机循环的框架，并从医学经典《黄帝内经》中选取了四个具有基础理论的段落。通过基于提示的认知支架，指导 DeepSeek V3.1 识别源文本中的隐喻和转喻，并在翻译中传达理论。在评估阶段，ChatGPT 5 Pro和Gemini 2.5 Pro根据提示模拟了三种类型的现实世界读者。模拟读者在五个认知维度上对人工翻译、基线模型翻译和提示调整翻译进行评分，然后进行结构化访谈和解释现象学分析。结果表明，即时调整的法学硕士翻译在所有五个维度上均表现最佳，具有高度的跨模型和跨角色一致性。访谈主题揭示了人工翻译和机器翻译之间的差异、隐喻和转喻迁移的有效策略以及读者的认知偏好。这项研究为中医等概念密集的古代文本的翻译提供了一种认知、高效和可复制的 HITL 方法学途径。

Title: Mind Reading or Misreading? LLMs on the Big Five Personality Test

• Authors: Francesco Di Cursi, Chiara Boldrini, Marco Conti, Andrea Passarella
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.23101
• Pdf URL: https://arxiv.org/pdf/2511.23101
• Copy Paste: [[2511.23101]] Mind Reading or Misreading? LLMs on the Big Five Personality Test(https://arxiv.org/abs/2511.23101)
• Keywords: language model, gpt, llm, prompt
• Abstract: We evaluate large language models (LLMs) for automatic personality prediction from text under the binary Five Factor Model (BIG5). Five models -- including GPT-4 and lightweight open-source alternatives -- are tested across three heterogeneous datasets (Essays, MyPersonality, Pandora) and two prompting strategies (minimal vs. enriched with linguistic and psychological cues). Enriched prompts reduce invalid outputs and improve class balance, but also introduce a systematic bias toward predicting trait presence. Performance varies substantially: Openness and Agreeableness are relatively easier to detect, while Extraversion and Neuroticism remain challenging. Although open-source models sometimes approach GPT-4 and prior benchmarks, no configuration yields consistently reliable predictions in zero-shot binary settings. Moreover, aggregate metrics such as accuracy and macro-F1 mask significant asymmetries, with per-class recall offering clearer diagnostic value. These findings show that current out-of-the-box LLMs are not yet suitable for APPT, and that careful coordination of prompt design, trait framing, and evaluation metrics is essential for interpretable results.
• 摘要：我们评估大型语言模型 (LLM)，以在二元五因素模型 (BIG5) 下根据文本自动进行个性预测。五种模型（包括 GPT-4 和轻量级开源替代方案）在三个异构数据集（Essays、MyPersonality、Pandora）和两种提示策略（最小与富含语言和心理线索）中进行了测试。丰富的提示减少了无效输出并改善了班级平衡，但也引入了预测特质存在的系统偏差。表现差异很大：开放性和宜人性相对更容易检测，而外向性和神经质仍然具有挑战性。尽管开源模型有时会接近 GPT-4 和之前的基准，但没有任何配置可以在零样本二进制设置中产生一致可靠的预测。此外，准确性和宏观 F1 等聚合指标掩盖了显着的不对称性，而每类召回率提供了更清晰的诊断价值。这些发现表明，当前开箱即用的法学硕士尚不适合 APPT，提示设计、特质框架和评估指标的仔细协调对于可解释的结果至关重要。

Title: Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM

• Authors: Mengjie Liu, Jiahui Peng, Pei Chu, Jiantao Qiu, Ren Ma, He Zhu, Rui Min, Lindong Lu, Wenchang Ning, Linfeng Hou, Kaiwen Liu, Yuan Qu, Zhenxiang Li, Chao Xu, Zhongying Tu, Wentao Zhang, Conghui He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.23119
• Pdf URL: https://arxiv.org/pdf/2511.23119
• Copy Paste: [[2511.23119]] Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM(https://arxiv.org/abs/2511.23119)
• Keywords: language model, hallucination
• Abstract: Accurately and efficiently extracting main content from general web pages is of great significance for obtaining training data for large models. Using well-pre-trained decoder-only generative language models offers excellent document comprehension capabilities, thereby effectively enhancing parsing quality. However, it remains constrained by issues such as context window length, inference cost, and format hallucination. We present Dripper, an efficient HTML main content extraction framework powered by lightweight language models, which addresses these challenges through four key innovations: (1) We design a specialized HTML simplification algorithm that reduces input token count to 22\% compared to raw HTML while preserving critical structural information; (2) We reformulate main content extraction as a semantic block sequence classification task, significantly reducing inference cost; (3) We introduce a controlled decoding mechanism that strictly constrains the output space through logits processors, effectively eliminating hallucination issues common in small-scale models; (4) We propose WebMainBench, an evaluation dataset containing over 7,800 web pages with meticulously human-annotated main content extraction labels. Experimental results demonstrate that using only a 0.6B parameter model, Dripper achieves state-of-the-art performance across all evaluation benchmarks and outperforms all baseline methods, attaining an ROUGE-N F1 score of 81.58\%( 83.13\% with fall-back strategy) on our proposed WebMainBench dataset.
• 摘要：准确高效地从一般网页中提取主要内容对于获取大型模型的训练数据具有重要意义。使用经过良好预训练的仅解码器生成语言模型可提供出色的文档理解能力，从而有效提高解析质量。然而，它仍然受到上下文窗口长度、推理成本和格式幻觉等问题的限制。我们推出了 Dripper，这是一种由轻量级语言模型支持的高效 HTML 主要内容提取框架，它通过四个关键创新解决了这些挑战：（1）我们设计了一种专门的 HTML 简化算法，与原始 HTML 相比，该算法将输入标记计数减少到 22%，同时保留关键结构信息； （2）我们将主要内容提取重新表述为语义块序列分类任务，显着降低推理成本； （3）我们引入了一种受控解码机制，通过logits处理器严格约束输出空间，有效消除小规模模型中常见的幻觉问题； (4) 我们提出了 WebMainBench，这是一个评估数据集，包含超过 7,800 个网页，以及经过精心人工注释的主要内容提取标签。实验结果表明，仅使用 0.6B 参数模型，Dripper 在所有评估基准上都实现了最先进的性能，并且优于所有基线方法，在我们提出的 WebMainBench 数据集上获得了 81.58\%（使用回退策略时为 83.13\%）的 ROUGE-N F1 分数。

Title: Multi-chain Graph Refinement and Selection for Reliable Reasoning in Large Language Models

• Authors: Yujiao Yang, Jing Lian, Linhui Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.23136
• Pdf URL: https://arxiv.org/pdf/2511.23136
• Copy Paste: [[2511.23136]] Multi-chain Graph Refinement and Selection for Reliable Reasoning in Large Language Models(https://arxiv.org/abs/2511.23136)
• Keywords: language model, llm, tree-of-thought
• Abstract: The complex reasoning ability of Large Language Models (LLMs) poses a critical bottleneck for their practical applications. Test-time expansion methods such as Tree-of-Thought (ToT) and Graph-of-Thought (GoT) enhance reasoning by introducing intermediate reasoning structures, tree search, or graph-based exploration mechanisms. However, their reasoning strategies suffer from limited diversity, redundant search branches, and inadequate integration and error correction across heterogeneous reasoning paths. To address these limitations, we propose a novel reasoning framework called Multi-chain Graph Refinement & Selection (MGRS), which first generates multiple diverse reasoning trajectories for a given problem, refines candidate responses using a composite self- and cross-verification strategy, then constructs a reasoning relation graph and estimates the success rate of intermediate nodes, and finally computes cumulative success rates to select the most reliable answer and corresponding reasoning trajectory. Experimental results demonstrate that MGRS significantly advances both the reasoning capability and computational efficiency of reasoning enhancement methods. Across six benchmark datasets spanning four distinct tasks, MGRS achieves an average accuracy of 82.9%, outperforming state-of-the-art baselines by a clear margin of 2.1%. Remarkably, on the 24-point game, MGRS attains 100% accuracy for the first time, while delivering a 13.6x speed-up compared to the leading Forest of Thoughts framework.
• 摘要：大型语言模型（LLM）的复杂推理能力对其实际应用造成了关键瓶颈。思想树 (ToT) 和思想图 (GoT) 等测试时扩展方法通过引入中间推理结构、树搜索或基于图的探索机制来增强推理。然而，他们的推理策略存在多样性有限、搜索分支冗余以及跨异构推理路径的集成和纠错不足等问题。为了解决这些限制，我们提出了一种称为多链图细化和选择（MGRS）的新颖推理框架，它首先为给定问题生成多个不同的推理轨迹，使用复合自验证和交叉验证策略细化候选响应，然后构建推理关系图并估计中间节点的成功率，最后计算累积成功率以选择最可靠的答案和相应的推理轨迹。实验结果表明，MGRS 显着提高了推理增强方法的推理能力和计算效率。在涵盖四个不同任务的六个基准数据集上，MGRS 的平均准确率达到 82.9%，明显优于最先进的基线 2.1%。值得注意的是，在 24 分游戏中，MGRS 首次达到 100% 的准确率，同时与领先的 Forest of Thought 框架相比，速度提高了 13.6 倍。

Title: Are LLMs Good Safety Agents or a Propaganda Engine?

• Authors: Neemesh Yadav, Francesco Ortu, Jiarui Liu, Joeun Yook, Bernhard Schölkopf, Rada Mihalcea, Alberto Cazzaniga, Zhijing Jin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.23174
• Pdf URL: https://arxiv.org/pdf/2511.23174
• Copy Paste: [[2511.23174]] Are LLMs Good Safety Agents or a Propaganda Engine?(https://arxiv.org/abs/2511.23174)
• Keywords: language model, llm, prompt, agent
• Abstract: Large Language Models (LLMs) are trained to refuse to respond to harmful content. However, systematic analyses of whether this behavior is truly a reflection of its safety policies or an indication of political censorship, that is practiced globally by countries, is lacking. Differentiating between safety influenced refusals or politically motivated censorship is hard and unclear. For this purpose we introduce PSP, a dataset built specifically to probe the refusal behaviors in LLMs from an explicitly political context. PSP is built by formatting existing censored content from two data sources, openly available on the internet: sensitive prompts in China generalized to multiple countries, and tweets that have been censored in various countries. We study: 1) impact of political sensitivity in seven LLMs through data-driven (making PSP implicit) and representation-level approaches (erasing the concept of politics); and, 2) vulnerability of models on PSP through prompt injection attacks (PIAs). Associating censorship with refusals on content with masked implicit intent, we find that most LLMs perform some form of censorship. We conclude with summarizing major attributes that can cause a shift in refusal distributions across models and contexts of different countries.
• 摘要：大型语言模型 (LLM) 经过训练，可以拒绝响应有害内容。然而，对于这种行为究竟是其安全政策的反映还是全球各国实行的政治审查制度的表现，目前还缺乏系统分析。区分安全影响的拒绝或出于政治动机的审查是困难且不明确的。为此，我们引入了 PSP，这是一个专门为从明确的政治背景中探究法学硕士拒绝行为而构建的数据集。 PSP 是通过对来自两个数据源的现有审查内容进行格式化而构建的，这些数据源可在互联网上公开获取：中国的敏感提示推广到多个国家，以及在各国受到审查的推文。我们研究：1）通过数据驱动（使 PSP 隐含）和代表性层面的方法（消除政治概念）政治敏感性对七个法学硕士的影响； 2) PSP 上的模型存在通过提示注入攻击 (PIA) 造成的漏洞。将审查制度与拒绝带有隐含意图的内容联系起来，我们发现大多数法学硕士都会执行某种形式的审查制度。最后，我们总结了可能导致不同国家的模式和背景下的拒绝分布发生变化的主要属性。

Title: TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies

• Authors: Guang Liang, Jie Shao, Ningyuan Tang, Xinyao Liu, Jianxin Wu
• Subjects: cs.CL, cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2511.23225
• Pdf URL: https://arxiv.org/pdf/2511.23225
• Copy Paste: [[2511.23225]] TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies(https://arxiv.org/abs/2511.23225)
• Keywords: llm
• Abstract: Native FP8 support in modern hardware is essential for training large Transformers, but is severely hindered by extreme activation outliers. Existing solutions either rely on complex mixed-precision engineering or invasive architectural modifications. This paper fundamentally challenges the conventional wisdom that outliers are data-driven. We demonstrate that extreme outliers are a data-independent, mechanically-produced artifact of training, originating from specific structural properties of the weight matrices (i.e., colinearity). Based on this insight, we propose TWEO (Transformers Without Extreme Outliers), a novel, non-invasive loss function. TWEO effectively prevents extreme outliers via a very simple loss term, which reduces outliers from 10000+ to less than 20. TWEO then enables full-model FP8 pre-training with neither engineering tricks nor architectural changes for both LLM and ViT. When standard FP8 training catastrophically collapses, TWEO achieves performance comparable to the BF16 baseline while delivering a 36% increase in training throughput. Also, TWEO enables a new quantization paradigm. Hardware-friendly W8A8 per-tensor static quantization of LLMs, previously considered completely unusable due to outliers, achieves SOTA performance for the first time on TWEO-trained models.
• 摘要：现代硬件中的原生 FP8 支持对于训练大型 Transformer 至关重要，但会受到极端激活异常值的严重阻碍。现有的解决方案要么依赖于复杂的混合精度工程，要么依赖于侵入性的架构修改。本文从根本上挑战了异常值是数据驱动的传统观点。我们证明，极端异常值是一种与数据无关、机械产生的训练产物，源自权重矩阵的特定结构特性（即共线性）。基于这一见解，我们提出了 TWEO（没有极端异常值的 Transformers），一种新颖的非侵入性损失函数。 TWEO 通过一个非常简单的损失项有效地防止极端异常值，从而将异常值从 10000+ 减少到 20 以下。然后，TWEO 可以实现全模型 FP8 预训练，既不需要工程技巧，也不需要对 LLM 和 ViT 进行架构更改。当标准 FP8 训练灾难性崩溃时，TWEO 可以实现与 BF16 基线相当的性能，同时训练吞吐量提高 36%。此外，TWEO 还实现了新的量化范例。 LLM 的硬件友好型 W8A8 每张量静态量化以前被认为由于异常值而完全无法使用，现在首次在 TWEO 训练的模型上实现了 SOTA 性能。

Title: Tourism Question Answer System in Indian Language using Domain-Adapted Foundation Models

• Authors: Praveen Gatla, Anushka, Nikita Kanwar, Gouri Sahoo, Rajesh Kumar Mundotiya
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.23235
• Pdf URL: https://arxiv.org/pdf/2511.23235
• Copy Paste: [[2511.23235]] Tourism Question Answer System in Indian Language using Domain-Adapted Foundation Models(https://arxiv.org/abs/2511.23235)
• Keywords: prompt
• Abstract: This article presents the first comprehensive study on designing a baseline extractive question-answering (QA) system for the Hindi tourism domain, with a specialized focus on the Varanasi-a cultural and spiritual hub renowned for its Bhakti-Bhaav (devotional ethos). Targeting ten tourism-centric subdomains-Ganga Aarti, Cruise, Food Court, Public Toilet, Kund, Museum, General, Ashram, Temple and Travel, the work addresses the absence of language-specific QA resources in Hindi for culturally nuanced applications. In this paper, a dataset comprising 7,715 Hindi QA pairs pertaining to Varanasi tourism was constructed and subsequently augmented with 27,455 pairs generated via Llama zero-shot prompting. We propose a framework leveraging foundation models-BERT and RoBERTa, fine-tuned using Supervised Fine-Tuning (SFT) and Low-Rank Adaptation (LoRA), to optimize parameter efficiency and task performance. Multiple variants of BERT, including pre-trained languages (e.g., Hindi-BERT), are evaluated to assess their suitability for low-resource domain-specific QA. Evaluation metrics - F1, BLEU, and ROUGE-L - highlight trade-offs between answer precision and linguistic fluency. Experiments demonstrate that LoRA-based fine-tuning achieves competitive performance (85.3\% F1) while reducing trainable parameters by 98\% compared to SFT, striking a balance between efficiency and accuracy. Comparative analysis across models reveals that RoBERTa with SFT outperforms BERT variants in capturing contextual nuances, particularly for culturally embedded terms (e.g., Aarti, Kund). This work establishes a foundational baseline for Hindi tourism QA systems, emphasizing the role of LORA in low-resource settings and underscoring the need for culturally contextualized NLP frameworks in the tourism domain.
• 摘要：本文介绍了第一个关于为印地语旅游领域设计基线提取问答 (QA) 系统的综合研究，特别关注瓦拉纳西——一个以其 Bhakti-Bhaav（奉献精神）而闻名的文化和精神中心。这项工作针对十个以旅游为中心的子领域——Ganga Aarti、Cruise、Food Court、Public Bathroom、Kund、Museum、General、Ashram、Temple 和 Travel，解决了印地语中缺乏针对文化细微差别的应用程序的特定语言 QA 资源的问题。在本文中，构建了一个包含 7,715 个与瓦拉纳西旅游业相关的印地语 QA 对的数据集，随后通过 Llama 零样本提示生成了 27,455 对。我们提出了一个利用基础模型 BERT 和 RoBERTa 的框架，并使用监督微调（SFT）和低秩适应（LoRA）进行微调，以优化参数效率和任务性能。评估 BERT 的多种变体，包括预训练语言（例如 Hindi-BERT），以评估它们对低资源特定领域 QA 的适用性。评估指标 - F1、BLEU 和 ROUGE-L - 强调答案精度和语言流畅性之间的权衡。实验表明，基于LoRA的微调实现了具有竞争力的性能（85.3％F1），同时与SFT相比，可训练参数减少了98％，在效率和准确性之间取得了平衡。跨模型的比较分析表明，采用 SFT 的 RoBERTa 在捕获上下文细微差别方面优于 BERT 变体，特别是对于嵌入文化的术语（例如 Aarti、Kund）。这项工作为印地语旅游 QA 系统建立了基础基线，强调 LORA 在资源匮乏环境中的作用，并强调旅游领域对文化背景 NLP 框架的需求。

Title: Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs

• Authors: Jiancheng Dong, Pengyue Jia, Jingyu Peng, Maolin Wang, Yuhao Wang, Lixin Su, Xin Sun, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.23271
• Pdf URL: https://arxiv.org/pdf/2511.23271
• Copy Paste: [[2511.23271]] Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs(https://arxiv.org/abs/2511.23271)
• Keywords: llm, prompt, agent
• Abstract: Carefully engineered system prompts play a critical role in guiding the behavior of LLM agents, but their considerable length introduces significant drawbacks, including increased inference latency, higher computational cost, and reduced effective context length. This raises the question of whether such lengthy prompts can be replaced by a drastically reduced number of tokens while preserving their behavioral effect on downstream tasks. To enable this, we propose a lightweight three-stage training framework that learns a single prompt-specific Behavior-Equivalent token ([BE]). The framework first trains [BE] to encode the natural-language content of the original system prompt via reconstruction, and then distills the prompt 's downstream behavior into this single token. Importantly, our method requires no access to model internals, no auxiliary compression models, and no labeled responses. Empirical evaluations on three datasets show that a single [BE] token achieves up to a 3000x reduction in prompt length, while retaining about 98% of the downstream performance of the original system prompts. This substantially reduces inference cost and leaves almost the entire context window available for user inputs.
• 摘要：精心设计的系统提示在指导 LLM 代理的行为中发挥着关键作用，但其相当长的长度带来了明显的缺点，包括推理延迟增加、计算成本更高以及有效上下文长度减少。这就提出了这样的问题：是否可以用大幅减少的令牌数量来取代如此冗长的提示，同时保留其对下游任务的行为影响。为了实现这一点，我们提出了一个轻量级的三阶段训练框架，用于学习单个提示特定的行为等效标记（[BE]）。该框架首先训练[BE]通过重构对原始系统提示的自然语言内容进行编码，然后将提示的下游行为提炼成这个单个标记。重要的是，我们的方法不需要访问模型内​​部，不需要辅助压缩模型，也不需要标记响应。对三个数据集的实证评估表明，单个 [BE] 令牌可将提示长度缩短 3000 倍，同时保留原始系统提示的约 98% 的下游性能。这大大降低了推理成本，并使几乎整个上下文窗口可供用户输入。

Title: MCP vs RAG vs NLWeb vs HTML: A Comparison of the Effectiveness and Efficiency of Different Agent Interfaces to the Web (Technical Report)

• Authors: Aaron Steiner, Ralph Peeters, Christian Bizer
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2511.23281
• Pdf URL: https://arxiv.org/pdf/2511.23281
• Copy Paste: [[2511.23281]] MCP vs RAG vs NLWeb vs HTML: A Comparison of the Effectiveness and Efficiency of Different Agent Interfaces to the Web (Technical Report)(https://arxiv.org/abs/2511.23281)
• Keywords: language model, gpt, llm, retrieval-augmented generation, agent
• Abstract: Large language model agents are increasingly used to automate web tasks such as product search, offer comparison, and checkout. Current research explores different interfaces through which these agents interact with websites, including traditional HTML browsing, retrieval-augmented generation (RAG) over pre-crawled content, communication via Web APIs using the Model Context Protocol (MCP), and natural-language querying through the NLWeb interface. However, no prior work has compared these four architectures within a single controlled environment using identical tasks. To address this gap, we introduce a testbed consisting of four simulated e-shops, each offering its products via HTML, MCP, and NLWeb interfaces. For each interface (HTML, RAG, MCP, and NLWeb) we develop specialized agents that perform the same sets of tasks, ranging from simple product searches and price comparisons to complex queries for complementary or substitute products and checkout processes. We evaluate the agents using GPT 4.1, GPT 5, GPT 5 mini, and Claude Sonnet 4 as underlying LLM. Our evaluation shows that the RAG, MCP and NLWeb agents outperform HTML on both effectiveness and efficiency. Averaged over all tasks, F1 rises from 0.67 for HTML to between 0.75 and 0.77 for the other agents. Token usage falls from about 241k for HTML to between 47k and 140k per task. The runtime per task drops from 291 seconds to between 50 and 62 seconds. The best overall configuration is RAG with GPT 5 achieving an F1 score of 0.87 and a completion rate of 0.79. Also taking cost into consideration, RAG with GPT 5 mini offers a good compromise between API usage fees and performance. Our experiments show the choice of the interaction interface has a substantial impact on both the effectiveness and efficiency of LLM-based web agents.
• 摘要：大型语言模型代理越来越多地用于自动化 Web 任务，例如产品搜索、报价比较和结账。当前的研究探索了这些代理与网站交互的不同接口，包括传统的 HTML 浏览、对预先抓取的内容进行检索增强生成 (RAG)、使用模型上下文协议 (MCP) 通过 Web API 进行通信，以及通过 NLWeb 接口进行自然语言查询。然而，之前的工作还没有在单个受控环境中使用相同的任务来比较这四种架构。为了弥补这一差距，我们引入了一个由四个模拟电子商店组成的测试平台，每个电子商店都通过 HTML、MCP 和 NLWeb 界面提供其产品。对于每个界面（HTML、RAG、MCP 和 NLWeb），我们开发了专门的代理来执行相同的任务集，范围从简单的产品搜索和价格比较到补充或替代产品的复杂查询和结账流程。我们使用 GPT 4.1、GPT 5、GPT 5 mini 和 Claude Sonnet 4 作为底层 LLM 来评估代理。我们的评估表明，RAG、MCP 和 NLWeb 代理在有效性和效率方面均优于 HTML。平均所有任务，F1 从 HTML 的 0.67 上升到其他代理的 0.75 到 0.77 之间。每个任务的令牌使用量从 HTML 的 241k 左右下降到 47k 到 140k 之间。每个任务的运行时间从 291 秒下降到 50 到 62 秒。整体最佳配置是 RAG，GPT 5 的 F1 得分为 0.87，完成率为 0.79。另外考虑到成本，带有 GPT 5 mini 的 RAG 在 API 使用费用和性能之间提供了良好的折衷。我们的实验表明，交互界面的选择对基于 LLM 的网络代理的有效性和效率具有重大影响。

Title: Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models

• Authors: Xiang Hu, Zhanchao Zhou, Ruiqi Liang, Zehuan Li, Wei Wu, Jianguo Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.23319
• Pdf URL: https://arxiv.org/pdf/2511.23319
• Copy Paste: [[2511.23319]] Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models(https://arxiv.org/abs/2511.23319)
• Keywords: language model, long context
• Abstract: This work explores the challenge of building ``Machines that Can Remember'', framing long-term memory as the problem of efficient ultra-long context modeling. We argue that this requires three key properties: \textbf{sparsity}, \textbf{random-access flexibility}, and \textbf{length generalization}. To address ultra-long-context modeling, we leverage Hierarchical Sparse Attention (HSA), a novel attention mechanism that satisfies all three properties. We integrate HSA into Transformers to build HSA-UltraLong, which is an 8B-parameter MoE model trained on over 8 trillion tokens and is rigorously evaluated on different tasks with in-domain and out-of-domain context lengths to demonstrate its capability in handling ultra-long contexts. Results show that our model performs comparably to full-attention baselines on in-domain lengths while achieving over 90\% accuracy on most in-context retrieval tasks with contexts up to 16M. This report outlines our experimental insights and open problems, contributing a foundation for future research in ultra-long context modeling.
• 摘要：这项工作探讨了构建“能够记忆的机器”的挑战，将长期记忆视为高效的超长上下文建模问题。我们认为这需要三个关键属性：\textbf{稀疏性}、\textbf{随机访问灵活性}和\textbf{长度泛化}。为了解决超长上下文建模问题，我们利用分层稀疏注意力（HSA），这是一种满足所有三个属性的新颖注意力机制。我们将 HSA 集成到 Transformers 中，构建 HSA-UltraLong，这是一个在超过 8 万亿个 token 上训练的 8B 参数 MoE 模型，并在域内和域外上下文长度的不同任务上进行严格评估，以展示其处理超长上下文的能力。结果表明，我们的模型在域内长度上的表现与全注意力基线相当，同时在上下文高达 16M 的大多数上下文检索任务中实现了超过 90% 的准确率。本报告概述了我们的实验见解和未解决的问题，为超长上下文建模的未来研究奠定了基础。

Title: Towards Improving Interpretability of Language Model Generation through a Structured Knowledge Discovery Approach

• Authors: Shuqi Liu, Han Wu, Guanzhi Deng, Jianshu Chen, Xiaoyang Wang, Linqi Song
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2511.23335
• Pdf URL: https://arxiv.org/pdf/2511.23335
• Copy Paste: [[2511.23335]] Towards Improving Interpretability of Language Model Generation through a Structured Knowledge Discovery Approach(https://arxiv.org/abs/2511.23335)
• Keywords: language model
• Abstract: Knowledge-enhanced text generation aims to enhance the quality of generated text by utilizing internal or external knowledge sources. While language models have demonstrated impressive capabilities in generating coherent and fluent text, the lack of interpretability presents a substantial obstacle. The limited interpretability of generated text significantly impacts its practical usability, particularly in knowledge-enhanced text generation tasks that necessitate reliability and explainability. Existing methods often employ domain-specific knowledge retrievers that are tailored to specific data characteristics, limiting their generalizability to diverse data types and tasks. To overcome this limitation, we directly leverage the two-tier architecture of structured knowledge, consisting of high-level entities and low-level knowledge triples, to design our task-agnostic structured knowledge hunter. Specifically, we employ a local-global interaction scheme for structured knowledge representation learning and a hierarchical transformer-based pointer network as the backbone for selecting relevant knowledge triples and entities. By combining the strong generative ability of language models with the high faithfulness of the knowledge hunter, our model achieves high interpretability, enabling users to comprehend the model output generation process. Furthermore, we empirically demonstrate the effectiveness of our model in both internal knowledge-enhanced table-to-text generation on the RotoWireFG dataset and external knowledge-enhanced dialogue response generation on the KdConv dataset. Our task-agnostic model outperforms state-of-the-art methods and corresponding language models, setting new standards on the benchmark.
• 摘要：知识增强文本生成旨在通过利用内部或外部知识源来提高生成文本的质量。虽然语言模型在生成连贯且流畅的文本方面表现出了令人印象深刻的能力，但缺乏可解释性是一个巨大的障碍。生成文本的有限可解释性极大地影响了其实际可用性，特别是在需要可靠性和可解释性的知识增强文本生成任务中。现有方法通常采用针对特定数据特征定制的特定领域知识检索器，限制了它们对不同数据类型和任务的通用性。为了克服这一限制，我们直接利用由高层实体和低层知识三元组组成的结构化知识的两层架构来设计与任务无关的结构化知识搜索器。具体来说，我们采用局部-全局交互方案进行结构化知识表示学习，并采用基于分层变压器的指针网络作为选择相关知识三元组和实体的骨干。通过将语言模型强大的生成能力与知识猎手的高度忠实性相结合，我们的模型实现了高可解释性，使用户能够理解模型输出的生成过程。此外，我们凭经验证明了我们的模型在 RotoWireFG 数据集上的内部知识增强的表到文本生成和 KdConv 数据集上的外部知识增强的对话响应生成方面的有效性。我们的任务无关模型优于最先进的方法和相应的语言模型，为基准设定了新标准。

Title: Optimizing Multimodal Language Models through Attention-based Interpretability

• Authors: Alexander Sergeev, Evgeny Kotelnikov
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2511.23375
• Pdf URL: https://arxiv.org/pdf/2511.23375
• Copy Paste: [[2511.23375]] Optimizing Multimodal Language Models through Attention-based Interpretability(https://arxiv.org/abs/2511.23375)
• Keywords: language model
• Abstract: Modern large language models become multimodal, analyzing various data formats like text and images. While fine-tuning is effective for adapting these multimodal language models (MLMs) to downstream tasks, full fine-tuning is computationally expensive. Parameter-Efficient Fine-Tuning (PEFT) methods address this by training only a small portion of model weights. However, MLMs are difficult to interpret, making it challenging to identify which components are most effective for training to balance efficiency and performance. We propose an attention-based interpretability method for MLMs by analyzing attention scores relative to image tokens. The core idea is to identify attention heads that focus on image key objects. We utilize this information to select optimal model components for PEFT in multimodal models. Our contributions include a method for identifying attention heads associated with image key objects, its application to PEFT for image captioning, and the creation of a new dataset containing images, key object masks, and their textual descriptions. We conducted experiments on MLMs with 2-3 billion parameters to validate the method's effectiveness. By calculating Head Impact (HI) scores we quantify an attention head's focus on key objects, indicating its significance in image understanding. Our fine-tuning experiments demonstrate that adapting layers with the highest HI scores leads to the most significant shifts in metrics compared to pre-trained, randomly selected, or lowest-HI-score layers. This indicates that fine-tuning a small percentage (around 0.01%) of parameters in these crucial layers can substantially influence image understanding capabilities.
• 摘要：现代大型语言模型变得多模态，可以分析文本和图像等各种数据格式。虽然微调对于使这些多模式语言模型 (MLM) 适应下游任务是有效的，但完全微调的计算成本很高。参数高效微调（PEFT）方法通过仅训练一小部分模型权重来解决这个问题。然而，MLM 很难解释，因此很难确定哪些组件对于平衡效率和绩效的培训最有效。我们通过分析相对于图像标记的注意力分数，提出了一种基于注意力的MLM可解释性方法。核心思想是识别关注图像关键对象的注意力头。我们利用这些信息来选择多模态模型中 PEFT 的最佳模型组件。我们的贡献包括一种识别与图像关键对象相关的注意力头的方法、其在 PEFT 图像字幕中的应用，以及创建包含图像、关键对象蒙版及其文本描述的新数据集。我们在具有 2-30 亿个参数的 MLM 上进行了实验，以验证该方法的有效性。通过计算头部影响 (HI) 分数，我们可以量化注意力头对关键对象的关注，表明其在图像理解中的重要性。我们的微调实验表明，与预先训练的、随机选择的或最低 HI 分数的层相比，调整具有最高 HI 分数的层会导致指标发生最显着的变化。这表明微调这些关键层中一小部分（约 0.01%）的参数可以极大地影响图像理解能力。

Title: MegaChat: A Synthetic Persian Q&A Dataset for High-Quality Sales Chatbot Evaluation

• Authors: Mahdi Rahmani, AmirHossein Saffari, Reyhane Rahmani
• Subjects: cs.CL, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2511.23397
• Pdf URL: https://arxiv.org/pdf/2511.23397
• Copy Paste: [[2511.23397]] MegaChat: A Synthetic Persian Q&A Dataset for High-Quality Sales Chatbot Evaluation(https://arxiv.org/abs/2511.23397)
• Keywords: gpt, chat, retrieval-augmented generation, agent
• Abstract: Small and medium-sized enterprises (SMEs) in Iran increasingly leverage Telegram for sales, where real-time engagement is essential for conversion. However, developing AI-driven chatbots for this purpose requires large, high-quality question-and-answer (Q&A) datasets, which are typically expensive and resource-intensive to produce, especially for low-resource languages like Persian. In this paper, we introduce MegaChat, the first fully synthetic Persian Q&A dataset designed to evaluate intelligent sales chatbots in Telegram-based e-commerce. We propose a novel, automated multi-agent architecture that generates persona-aware Q&A pairs by collecting data from active Telegram shopping channels. The system employs specialized agents for question generation, validation, and refinement, ensuring the production of realistic and diverse conversational data. To evaluate answer generation, we compare three classic retrieval-augmented generation (RAG) models with our advanced agentic system, which features multi-query retrieval, reranking, and persona-aligned response synthesis. Using GPT-5.1 for evaluation across six quality dimensions, our results show that the agentic architecture outperformed traditional RAG models in 4 out of 5 diverse channels, demonstrating its ability to generate scalable, high-quality datasets without relying on expensive human annotation or complex fine-tuning. MegaChat provides SMEs with an efficient, cost-effective solution for building intelligent customer engagement systems in specialized commercial domains, enabling advancements in multilingual conversational AI for low-resource languages. Download: this https URL
• 摘要：伊朗的中小企业 (SME) 越来越多地利用 Telegram 进行销售，其中实时参与对于转化至关重要。然而，为此目的开发人工智能驱动的聊天机器人需要大量、高质量的问答 (Q&A) 数据集，这些数据集的生产通常非常昂贵且占用资源，特别是对于波斯语等资源匮乏的语言。在本文中，我们介绍了 MegaChat，这是第一个完全合成的波斯语问答数据集，旨在评估基于 Telegram 的电子商务中的智能销售聊天机器人。我们提出了一种新颖的自动化多代理架构，通过从活跃的 Telegram 购物渠道收集数据来生成角色感知的问答对。该系统采用专门的代理来生成、验证和细化问题，确保生成真实且多样化的对话数据。为了评估答案的生成，我们将三种经典的检索增强生成（RAG）模型与我们先进的代理系统进行了比较，该系统具有多查询检索、重新排名和角色对齐响应合成的功能。使用 GPT-5.1 对六个质量维度进行评估，我们的结果表明，代理架构在 5 个不同通道中的 4 个中优于传统 RAG 模型，证明了其无需依赖昂贵的人工注释或复杂的微调即可生成可扩展的高质量数据集的能力。 MegaChat 为中小企业提供了一种高效、经济高效的解决方案，用于在专业商业领域构建智能客户参与系统，从而推动低资源语言的多语言对话 AI 的进步。下载：此 https 网址