2026-03-25

Title: Evaluating Prompting Strategies for Chart Question Answering with Large Language Models

• Authors: Ruthuparna Naikar, Ying Zhu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.22288
• Pdf URL: https://arxiv.org/pdf/2603.22288
• Copy Paste: [[2603.22288]] Evaluating Prompting Strategies for Chart Question Answering with Large Language Models(https://arxiv.org/abs/2603.22288)
• Keywords: language model, gpt, llm, prompt, chain-of-thought
• Abstract: Prompting strategies affect LLM reasoning performance, but their role in chart-based QA remains underexplored. We present a systematic evaluation of four widely used prompting paradigms (Zero-Shot, Few-Shot, Zero-Shot Chain-of-Thought, and Few-Shot Chain-of-Thought) across GPT-3.5, GPT-4, and GPT-4o on the ChartQA dataset. Our framework operates exclusively on structured chart data, isolating prompt structure as the only experimental variable, and evaluates performance using two metrics: Accuracy and Exact Match. Results from 1,200 diverse ChartQA samples show that Few-Shot Chain-of-Thought prompting consistently yields the highest accuracy (up to 78.2\%), particularly on reasoning-intensive questions, while Few-Shot prompting improves format adherence. Zero-Shot performs well only with high-capacity models on simpler tasks. These findings provide actionable guidance for selecting prompting strategies in structured data reasoning tasks, with implications for both efficiency and accuracy in real-world applications.
• 摘要：提示策略会影响 LLM 推理表现，但它们在基于图表的 QA 中的作用仍未得到充分探索。我们对 ChartQA 数据集上的 GPT-3.5、GPT-4 和 GPT-4o 中四种广泛使用的提示范式（零样本、少样本、零样本思维链和少样本思维链）进行了系统评估。我们的框架专门对结构化图表数据进行操作，将提示结构隔离为唯一的实验变量，并使用两个指标评估性能：准确性和精确匹配。 1,200 个不同的 ChartQA 样本的结果表明，Few-Shot Chain-of-Thought 提示始终能产生最高的准确率（高达 78.2\%），特别是在推理密集型问题上，而 Few-Shot 提示则提高了格式遵循性。零射击仅在处理较简单任务的高容量模型时表现良好。这些发现为在结构化数据推理任务中选择提示策略提供了可行的指导，对现实应用中的效率和准确性都有影响。

Title: MERIT: Memory-Enhanced Retrieval for Interpretable Knowledge Tracing

• Authors: Runze Li, Kedi Chen, Guwei Feng, Mo Yu, Jun Wang, Wei Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.22289
• Pdf URL: https://arxiv.org/pdf/2603.22289
• Copy Paste: [[2603.22289]] MERIT: Memory-Enhanced Retrieval for Interpretable Knowledge Tracing(https://arxiv.org/abs/2603.22289)
• Keywords: language model, llm, hallucination, chain-of-thought
• Abstract: Knowledge Tracing (KT) models students' evolving knowledge states to predict future performance, serving as a foundation for personalized education. While traditional deep learning models achieve high accuracy, they often lack interpretability. Large Language Models (LLMs) offer strong reasoning capabilities but struggle with limited context windows and hallucinations. Furthermore, existing LLM-based methods typically require expensive fine-tuning, limiting scalability and adaptability to new data. We propose MERIT (Memory-Enhanced Retrieval for Interpretable Knowledge Tracing), a training-free framework combining frozen LLM reasoning with structured pedagogical memory. Rather than updating parameters, MERIT transforms raw interaction logs into an interpretable memory bank. The framework uses semantic denoising to categorize students into latent cognitive schemas and constructs a paradigm bank where representative error patterns are analyzed offline to generate explicit Chain-of-Thought (CoT) rationales. During inference, a hierarchical routing mechanism retrieves relevant contexts, while a logic-augmented module applies semantic constraints to calibrate predictions. By grounding the LLM in interpretable memory, MERIT achieves state-of-the-art performance on real-world datasets without gradient updates. This approach reduces computational costs and supports dynamic knowledge updates, improving the accessibility and transparency of educational diagnosis.
• 摘要：知识追踪（KT）对学生不断变化的知识状态进行建模，以预测未来的表现，作为个性化教育的基础。虽然传统的深度学习模型具有很高的准确性，但它们往往缺乏可解释性。大型语言模型 (LLM) 提供强大的推理能力，但难以应对有限的上下文窗口和幻觉。此外，现有的基于法学硕士的方法通常需要昂贵的微调，限制了可扩展性和对新数据的适应性。我们提出了 MERIT（可解释知识追踪的记忆增强检索），这是一种将冻结的法学硕士推理与结构化教学记忆相结合的免训练框架。 MERIT 不是更新参数，而是将原始交互日志转换为可解释的内存库。该框架使用语义去噪将学生分类为潜在的认知模式，并构建一个范式库，在其中离线分析代表性错误模式以生成明确的思想链（CoT）基本原理。在推理过程中，分层路由机制检索相关上下文，而逻辑增强模块应用语义约束来校准预测。通过将 LLM 置于可解释的内存中，MERIT 在无需梯度更新的真实数据集上实现了最先进的性能。这种方法降低了计算成本并支持动态知识更新，提高了教育诊断的可访问性和透明度。

Title: Less is More: Adapting Text Embeddings for Low-Resource Languages with Small Scale Noisy Synthetic Data

• Authors: Zaruhi Navasardyan, Spartak Bughdaryan, Bagrat Minasyan, Hrant Davtyan
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2603.22290
• Pdf URL: https://arxiv.org/pdf/2603.22290
• Copy Paste: [[2603.22290]] Less is More: Adapting Text Embeddings for Low-Resource Languages with Small Scale Noisy Synthetic Data(https://arxiv.org/abs/2603.22290)
• Keywords: llm, retrieval-augmented generation
• Abstract: Low-resource languages (LRLs) often lack high-quality, large-scale datasets for training effective text embedding models, hindering their application in tasks like retrieval-augmented generation (RAG) and semantic search. In this work, we challenge the prevailing assumption that effective semantic alignment requires massive datasets or pristine, human-verified translations. Focusing on Armenian (an LRL with a unique script), we introduce a cost-effective adaptation strategy using small scale noisy synthetic data generated by translating English Reddit title-body pairs with open-weights models. We establish a comprehensive evaluation benchmark comprising existing datasets, translated data, and a manually curated dataset. Our experiments reveal a surprising "Less is More" phenomenon: fine-tuning a multilingual encoder (mE5) on just 10,000 noisy synthetic pairs yields 11-12\% average improvements across the benchmark with a 20\%+ relative improvement in retrieval performance, matching the performance of models trained on ~1 million examples. Furthermore, we demonstrate that neither increasing data scale, improving translation quality via state-of-the-art LLMs, nor diversifying data domains yields significant gains over this minimal baseline. We validate the generalizability of these findings on another LRL with a unique script. Our results suggest that semantic alignment for LRLs saturates early and is highly robust to noise, democratizing high-performance embedding creation for resource-constrained communities. We release the model, data, and the benchmark at this https URL to facilitate further research.
• 摘要：低资源语言（LRL）通常缺乏高质量、大规模的数据集来训练有效的文本嵌入模型，阻碍了它们在检索增强生成（RAG）和语义搜索等任务中的应用。在这项工作中，我们挑战了普遍的假设，即有效的语义对齐需要大量数据集或原始的、经过人工验证的翻译。专注于亚美尼亚语（具有独特脚本的 LRL），我们引入了一种经济高效的适应策略，使用通过使用开放权重模型翻译英语 Reddit 标题-正文对而生成的小规模噪声合成数据。我们建立了一个全面的评估基准，包括现有数据集、翻译数据和手动整理的数据集。我们的实验揭示了一个令人惊讶的“少即是多”现象：仅在 10,000 个噪声合成对上微调多语言编码器 (mE5)，即可在基准测试中产生 11-12% 的平均改进，检索性能相对提高 20% 以上，与在约 100 万个示例上训练的模型的性能相匹配。此外，我们证明，无论是增加数据规模、通过最先进的法学硕士提高翻译质量，还是使数据域多样化，都不会在这个最低基线上产生显着的收益。我们使用独特的脚本在另一个 LRL 上验证了这些发现的普遍性。我们的结果表明，LRL 的语义对齐很早就饱和，并且对噪声具有高度鲁棒性，从而使资源受限社区的高性能嵌入创建民主化。我们在此 https URL 发布模型、数据和基准，以方便进一步研究。

Title: Evaluating Large Language Models' Responses to Sexual and Reproductive Health Queries in Nepali

• Authors: Medha Sharma, Supriya Khadka, Udit Chandra Aryal, Bishnu Hari Bhatta, Bijayan Bhattarai, Santosh Dahal, Kamal Gautam, Pushpa Joshi, Saugat Kafle, Shristi Khadka, Shushila Khadka, Binod Lamichhane, Shilpa Lamichhane, Anusha Parajuli, Sabina Pokharel, Suvekshya Sitaula, Neha Verma, Bishesh Khanal
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22291
• Pdf URL: https://arxiv.org/pdf/2603.22291
• Copy Paste: [[2603.22291]] Evaluating Large Language Models' Responses to Sexual and Reproductive Health Queries in Nepali(https://arxiv.org/abs/2603.22291)
• Keywords: language model, gpt, llm, chat
• Abstract: As Large Language Models (LLMs) become integrated into daily life, they are increasingly used for personal queries, including Sexual and Reproductive Health (SRH), allowing users to chat anonymously without fear of judgment. However, current evaluation methods primarily focus on accuracy, often for objective queries in high-resource languages, and lack criteria to assess usability and safety, especially for low-resource languages and culturally sensitive domains like SRH. This paper introduces LLM Evaluation Framework (LEAF), that conducts assessments across multiple criteria: accuracy, language, usability gaps (including relevance, adequacy, and cultural appropriateness), and safety gaps (safety, sensitivity, and confidentiality). Using the LEAF framework, we assessed 14K SRH queries in Nepali from over 9K users. Responses were manually annotated by SRH experts according to the framework. Results revealed that only 35.1% of the responses were "proper", meaning they were accurate, adequate and had no major usability or safety related gaps. Insights include differences in performance between ChatGPT versions, such as similar accuracy but varying usability and safety aspects. This evaluation highlights significant limitations of current LLMs and underscores the need for improvement. The LEAF Framework is adaptable across domains and languages, particularly where usability and safety are critical, offering a pathway to better address sensitive topics.
• 摘要：随着大型语言模型 (LLM) 融入日常生活，它们越来越多地用于个人查询，包括性与生殖健康 (SRH)，允许用户匿名聊天而不必担心判断。然而，当前的评估方法主要关注准确性，通常针对高资源语言的客观查询，缺乏评估可用性和安全性的标准，特别是对于低资源语言和 SRH 等文化敏感领域。本文介绍了法学硕士评估框架 (LEAF)，该框架跨多个标准进行评估：准确性、语言、可用性差距（包括相关性、充分性和文化适当性）和安全差距（安全性、敏感性和保密性）。使用 LEAF 框架，我们评估了来自超过 9000 个用户的 14000 个尼泊尔语 SRH 查询。 SRH 专家根据框架对回复进行手动注释。结果显示，只有 35.1% 的回答是“正确的”，这意味着它们是准确、充分的，并且不存在与可用性或安全性相关的重大差距。见解包括 ChatGPT 版本之间的性能差异，例如相似的准确性但不同的可用性和安全性。该评估强调了当前法学硕士的重大局限性，并强调了改进的必要性。 LEAF 框架适用于跨领域和语言，特别是在可用性和安全性至关重要的情况下，提供了更好地解决敏感主题的途径。

Title: TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMs

• Authors: Yutao Xie, Nathaniel Thomas, Nicklas Hansen, Yang Fu, Li Erran Li, Xiaolong Wang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.22293
• Pdf URL: https://arxiv.org/pdf/2603.22293
• Copy Paste: [[2603.22293]] TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMs(https://arxiv.org/abs/2603.22293)
• Keywords: language model, llm
• Abstract: Search-augmented large language models (LLMs) trained with reinforcement learning (RL) have achieved strong results on open-domain question answering (QA), but training still remains a significant challenge. The optimization is often unstable due to sparse rewards and difficult credit assignments across reasoning and tool calls. To address this, we introduce Turn-Level Information Potential Reward Shaping (TIPS), a simple framework that assigns dense, turn-level rewards to each reasoning + tool-call segment based on the increased likelihood of the correct answer under a teacher model. By leveraging the potential-based reward shaping, TIPS offers fine-grained and policy-invariant guidance that overcomes the limitations of outcome-only optimization. Evaluated on seven QA benchmarks, TIPS consistently outperforms GRPO/PPO baselines and substantially improves training stability. For instance, with a Qwen-2.5 7B Instruct model, TIPS improves the average Exact Match score by 11.8% and F1 by 13.6% relative to PPO. Our results demonstrate that turn-level information-potential reward shaping provides an effective and general solution to sparse-reward credit assignment for multi-turn LLM reasoning.
• 摘要：通过强化学习 (RL) 训练的搜索增强大型语言模型 (LLM) 在开放域问答 (QA) 方面取得了出色的成果，但训练仍然是一个重大挑战。由于奖励稀疏以及推理和工具调用之间的信用分配困难，优化通常不稳定。为了解决这个问题，我们引入了回合级信息潜在奖励塑造（TIPS），这是一个简单的框架，根据教师模型下正确答案的可能性增加，为每个推理+工具调用片段分配密集的回合级奖励。通过利用基于潜力的奖励塑造，TIPS 提供了细粒度和政策不变的指导，克服了仅结果优化的局限性。根据七个 QA 基准进行评估，TIPS 始终优于 GRPO/PPO 基线，并显着提高了训练稳定性。例如，使用 Qwen-2.5 7B Instruct 模型，相对于 PPO，TIPS 将平均精确匹配得分提高了 11.8%，将 F1 提高了 13.6%。我们的结果表明，回合级信息潜在奖励塑造为多回合 LLM 推理的稀疏奖励信用分配提供了有效且通用的解决方案。

Title: Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs

• Authors: Michael Keeman
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.22295
• Pdf URL: https://arxiv.org/pdf/2603.22295
• Copy Paste: [[2603.22295]] Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs(https://arxiv.org/abs/2603.22295)
• Keywords: language model, llm
• Abstract: Large language models appear to develop internal representations of emotion -- "emotion circuits," "emotion neurons," and structured emotional manifolds have been reported across multiple model families. But every study making these claims uses stimuli signalled by explicit emotion keywords, leaving a fundamental question unanswered: do these circuits detect genuine emotional meaning, or do they detect the word "devastated"? We present the first clinical validity test of emotion circuit claims using mechanistic interpretability methods grounded in clinical psychology -- clinical vignettes that evoke emotions through situational and behavioural cues alone, emotion keywords removed. Across six models (Llama-3.2-1B, Llama-3-8B, Gemma-2-9B; base and instruct variants), we apply four convergent mechanistic interpretability methods -- linear probing, causal activation patching, knockout experiments, and representational geometry -- and discover two dissociable emotion processing mechanisms. Affect reception -- detecting emotionally significant content -- operates with near-perfect accuracy (AUROC 1.000), consistent with early-layer saturation, and replicates across all six models. Emotion categorization -- mapping affect to specific emotion labels -- is partially keyword-dependent, dropping 1-7% without keywords and improving with scale. Causal activation patching confirms keyword-rich and keyword-free stimuli share representational space, transferring affective salience rather than emotion-category identity. These findings falsify the keyword-spotting hypothesis, establish a novel mechanistic dissociation, and introduce clinical stimulus methodology as a rigorous standard for testing emotion processing claims in large language models -- with direct implications for AI safety evaluation and alignment. All stimuli, code, and data are released for replication.
• 摘要：大型语言模型似乎发展了情感的内部表征——“情感回路”、“情感神经元”和结构化情感流形已在多个模型家族中得到报道。但提出这些主张的每一项研究都使用由明确的情感关键词发出的刺激信号，而没有回答一个基本问题：这些电路是否检测到真正的情感意义，或者它们是否检测到“毁灭”这个词？我们使用基于临床心理学的机械解释方法，提出了情绪回路主张的第一个临床有效性测试——仅通过情境和行为线索唤起情绪的临床小插曲，删除了情绪关键词。在六个模型（Llama-3.2-1B、Llama-3-8B、Gemma-2-9B；基础变体和指令变体）中，我们应用了四种收敛机制解释方法——线性探测、因果激活修补、敲除实验和表征几何——并发现了两种可分离的情绪处理机制。影响接收——检测情感上重要的内容——以近乎完美的精度（AUROC 1.000）运行，与早期层饱和度一致，并在所有六个模型中复制。情感分类——将影响映射到特定的情感标签——部分依赖于关键字，如果没有关键字，情感分类会下降 1-7%，但随着规模的扩大，情感分类会有所改善。因果激活修补确认富含关键词和无关键词的刺激共享表征空间，传递情感显着性而不是情感类别身份。这些发现证伪了关键词识别假说，建立了一种新颖的机械分离，并引入临床刺激方法作为在大语言模型中测试情感处理主张的严格标准——这对人工智能安全评估和调整有直接影响。所有刺激、代码和数据都被发布以进行复制。

Title: Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs

• Authors: Haoming Meng, Kexin Huang, Shaohang Wei, Chiyu Ma, Shuo Yang, Xue Wang, Guoyin Wang, Bolin Ding, Jingren Zhou
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.22446
• Pdf URL: https://arxiv.org/pdf/2603.22446
• Copy Paste: [[2603.22446]] Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs(https://arxiv.org/abs/2603.22446)
• Keywords: language model, llm
• Abstract: Reinforcement learning with verifiable rewards (RLVR) has significantly improved reasoning in large language models (LLMs), yet the token-level mechanisms underlying these improvements remain unclear. We present a systematic empirical study of RLVR's distributional effects organized around three main analyses: (1) token-level characterization of distributional shifts between base and RL models, (2) the impact of token-level distributional shifts on sequence-level reasoning performance through cross-sampling interventions, and (3) fine-grained mechanics of these shifts at the token level. We find that RL fine-tuning induces highly sparse and targeted changes, with only a small fraction of token distributions exhibiting meaningful divergence between the base and RL policies. We further characterize the structure and evolution of these shifts through analyses of token entropy, positional concentration, and reallocation of probability mass. To assess the functional importance of these sparse changes, we conduct cross-sampling experiments that selectively swap token choices between the base and RL models with varying intervention budgets. We show that inserting only a small fraction of RL-sampled tokens into base generations progressively recovers RL performance gains, while injecting a similarly small number of base token choices into otherwise RL-generated sequences collapses performance to base levels, isolating a small set of token-level decisions directly responsible for RLVR's performance gains. Finally, we explore divergence-weighted variants of the advantage signal as a diagnostic intervention, finding that they can yield improvements over baselines. Together, our results shed light on the distributional changes induced by RLVR and provide a fine-grained, token-level lens for understanding RLVR fine-tuning as a targeted refinement process.
• 摘要：具有可验证奖励的强化学习（RLVR）显着改善了大型语言模型（LLM）的推理，但这些改进背后的代币级机制仍不清楚。我们围绕三个主要分析对 RLVR 的分布效应进行了系统的实证研究：(1) 基础模型和 RL 模型之间的分布变化的令牌级表征，(2) 通过交叉采样干预令牌级分布变化对序列级推理性能的影响，以及 (3) 令牌级这些变化的细粒度机制。我们发现 RL 微调会引起高度稀疏和有针对性的变化，只有一小部分令牌分布在基本策略和 RL 策略之间表现出有意义的差异。我们通过分析令牌熵、位置集中度和概率质量的重新分配来进一步描述这些转变的结构和演变。为了评估这些稀疏变化的功能重要性，我们进行了交叉抽样实验，有选择地在具有不同干预预算的基础模型和 RL 模型之间交换代币选择。我们表明，仅将一小部分 RL 采样令牌插入到基本生成中可以逐渐恢复 RL 性能增益，而将类似的少量基本令牌选择注入到其他 RL 生成的序列中会将性能降低到基本水平，从而隔离出一小部分直接负责 RLVR 性能增益的令牌级决策。最后，我们探索了优势信号的散度加权变体作为诊断干预措施，发现它们可以产生相对于基线的改进。总之，我们的结果揭示了 RLVR 引起的分布变化，并提供了一个细粒度、令牌级的镜头，用于将 RLVR 微调理解为有针对性的细化过程。

Title: Towards Automated Community Notes Generation with Large Vision Language Models for Combating Contextual Deception

• Authors: Jin Ma, Jingwen Yan, Mohammed Aldeen, Ethan Anderson, Taran Kavuru, Jinkyung Katie Park, Feng Luo, Long Cheng
• Subjects: cs.CL, cs.SI
• Abstract URL: https://arxiv.org/abs/2603.22453
• Pdf URL: https://arxiv.org/pdf/2603.22453
• Copy Paste: [[2603.22453]] Towards Automated Community Notes Generation with Large Vision Language Models for Combating Contextual Deception(https://arxiv.org/abs/2603.22453)
• Keywords: language model, gpt, agent
• Abstract: Community Notes have emerged as an effective crowd-sourced mechanism for combating online deception on social media platforms. However, its reliance on human contributors limits both the timeliness and scalability. In this work, we study the automated Community Notes generation method for image-based contextual deception, where an authentic image is paired with misleading context (e.g., time, entity, and event). Unlike prior work that primarily focuses on deception detection (i.e., judging whether a post is true or false in a binary manner), Community Notes-style systems need to generate concise and grounded notes that help users recover the missing or corrected context. This problem remains underexplored due to three reasons: (i) datasets that support the research are scarce; (ii) methods must handle the dynamic nature of contextual deception; (iii) evaluation is difficult because standard metrics do not capture whether notes actually improve user understanding. To address these gaps, we curate a real-world dataset, XCheck, comprising X posts with associated Community Notes and external contexts. We further propose the Automated Context-Corrective Note generation method, named ACCNote, which is a retrieval-augmented, multi-agent collaboration framework built on large vision-language models. Finally, we introduce a new evaluation metric, Context Helpfulness Score (CHS), that aligns with user study outcomes rather than relying on lexical overlap. Experiments on our XCheck dataset show that the proposed ACCNote improves both deception detection and note generation performance over baselines, and exceeds a commercial tool GPT5-mini. Together, our dataset, method, and metric advance practical automated generation of context-corrective notes toward more responsible online social networks.
• 摘要：社区注释已成为打击社交媒体平台上在线欺骗的有效众包机制。然而，它对人类贡献者的依赖限制了及时性和可扩展性。在这项工作中，我们研究了基于图像的上下文欺骗的自动社区注释生成方法，其中真实的图像与误导性的上下文（例如时间、实体和事件）配对。与之前主要关注欺骗检测（即以二进制方式判断帖子是真是假）的工作不同，社区注释式系统需要生成简洁且有根据的注释，以帮助用户恢复丢失或纠正的上下文。由于以下三个原因，这个问题仍未得到充分研究：（i）支持该研究的数据集稀缺； (ii) 方法必须处理上下文欺骗的动态性质； (iii) 评估很困难，因为标准指标无法衡量注释是否真正提高了用户的理解。为了解决这些差距，我们策划了一个真实世界的数据集 XCheck，其中包含 X 篇帖子以及相关的社区注释和外部上下文。我们进一步提出了自动上下文纠正注释生成方法，名为 ACCNote，它是一个基于大型视觉语言模型的检索增强型多智能体协作框架。最后，我们引入了一个新的评估指标，上下文有用性得分（CHS），它与用户研究结果相一致，而不是依赖于词汇重叠。在我们的 XCheck 数据集上进行的实验表明，所提出的 ACCNote 在基线上提高了欺骗检测和注释生成性能，并且超过了商业工具 GPT5-mini。我们的数据集、方法和指标共同推进了上下文纠正注释的实用自动生成，以实现更负责任的在线社交网络。

Title: LLM-guided headline rewriting for clickability enhancement without clickbait

• Authors: Yehudit Aperstein, Linoy Halifa, Sagiv Bar, Alexander Apartsin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.22459
• Pdf URL: https://arxiv.org/pdf/2603.22459
• Copy Paste: [[2603.22459]] LLM-guided headline rewriting for clickability enhancement without clickbait(https://arxiv.org/abs/2603.22459)
• Keywords: language model, llm
• Abstract: Enhancing reader engagement while preserving informational fidelity is a central challenge in controllable text generation for news media. Optimizing news headlines for reader engagement is often conflated with clickbait, resulting in exaggerated or misleading phrasing that undermines editorial trust. We frame clickbait not as a separate stylistic category, but as an extreme outcome of disproportionate amplification of otherwise legitimate engagement cues. Based on this view, we formulate headline rewriting as a controllable generation problem, where specific engagement-oriented linguistic attributes are selectively strengthened under explicit constraints on semantic faithfulness and proportional emphasis. We present a guided headline rewriting framework built on a large language model (LLM) that uses the Future Discriminators for Generation (FUDGE) paradigm for inference-time control. The LLM is steered by two auxiliary guide models: (1) a clickbait scoring model that provides negative guidance to suppress excessive stylistic amplification, and (2) an engagement-attribute model that provides positive guidance aligned with target clickability objectives. Both guides are trained on neutral headlines drawn from a curated real-world news corpus. At the same time, clickbait variants are generated synthetically by rewriting these original headlines using an LLM under controlled activation of predefined engagement tactics. By adjusting guidance weights at inference time, the system generates headlines along a continuum from neutral paraphrases to more engaging yet editorially acceptable formulations. The proposed framework provides a principled approach for studying the trade-off between attractiveness, semantic preservation, and clickbait avoidance, and supports responsible LLM-based headline optimization in journalistic settings.
• 摘要：在保持信息保真度的同时增强读者参与度是新闻媒体可控文本生成的核心挑战。优化新闻标题以提高读者参与度通常与标题诱饵混为一谈，导致夸大或误导性的措辞，从而损害编辑的信任。我们并不是将点击诱饵视为一个单独的风格类别，而是将其视为不成比例放大其他合法参与线索的极端结果。基于这一观点，我们将标题重写视为一个可控生成问题，其中特定的面向参与的语言属性在语义忠实度和比例强调的明确约束下被选择性地强化。我们提出了一个基于大型语言模型（LLM）的引导标题重写框架，该框架使用未来生成判别器（FUDGE）范式进行推理时间控制。 LLM 由两个辅助指导模型指导：（1）点击诱饵评分模型，提供负面指导以抑制过度的文体放大；（2）参与属性模型，提供与目标可点击性目标一致的积极指导。这两个指南都接受过来自精选的现实世界新闻语料库的中性头条新闻的培训。与此同时，点击诱饵变体是通过使用法学硕士在预定义参与策略的受控激活下重写这些原始标题来综合生成的。通过在推理时调整指导权重，系统会生成从中性释义到更具吸引力但编辑上可接受的表述的连续体标题。所提出的框架为研究吸引力、语义保留和避免点击诱饵之间的权衡提供了原则性方法，并支持新闻环境中负责任的基于法学硕士的标题优化。

Title: Functional Component Ablation Reveals Specialization Patterns in Hybrid Language Model Architectures

• Authors: Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-Carbó
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.22473
• Pdf URL: https://arxiv.org/pdf/2603.22473
• Copy Paste: [[2603.22473]] Functional Component Ablation Reveals Specialization Patterns in Hybrid Language Model Architectures(https://arxiv.org/abs/2603.22473)
• Keywords: language model
• Abstract: Hybrid language models combining attention with state space models (SSMs) or linear attention offer improved efficiency, but whether both components are genuinely utilized remains unclear. We present a functional component ablation framework applied to two sub-1B hybrid models -- Qwen3.5-0.8B (sequential: Gated DeltaNet + softmax attention) and Falcon-H1-0.5B (parallel: Mamba-2 + attention) -- with a pure Transformer control (Qwen2.5-0.5B). Through group ablations, layer-wise sweeps, positional ablations, matched random controls, and perplexity analysis across five benchmarks, we establish four findings: (1) both component types are essential and neither is bypassed; (2) the alternative component (linear attention or SSM) is the primary language modeling backbone, causing >35,000x perplexity degradation when removed versus ~82x for attention; (3) component importance follows a positional gradient, with early layers being disproportionately critical; and (4) hybrid architectures exhibit 20-119x greater resilience to random layer removal than pure Transformers, revealing built-in functional redundancy between component types. These results provide actionable guidance for hybrid model compression, architecture design, and fault-tolerant deployment.
• 摘要：将注意力与状态空间模型（SSM）或线性注意力相结合的混合语言模型可以提高效率，但这两个组件是否得到真正利用仍不清楚。我们提出了一个应用于两个 sub-1B 混合模型的功能组件消融框架——Qwen3.5-0.8B（顺序：门控 DeltaNet + softmax 注意力）和 Falcon-H1-0.5B（并行：Mamba-2 + 注意力）——带有纯 Transformer 控制（Qwen2.5-0.5B）。通过五个基准的分组消融、分层扫描、位置消融、匹配的随机控制和困惑度分析，我们得出了四个发现：（1）两种组件类型都是必不可少的，并且都不能被绕过； (2) 替代组件（线性注意力或 SSM）是主要的语言建模主干，删除后会导致超过 35,000 倍的困惑度下降，而注意力会导致约 82 倍； (3) 组件重要性遵循位置梯度，早期层尤为关键； (4) 混合架构对随机层移除的恢复能力比纯 Transformer 高 20-119 倍，揭示了组件类型之间的内置功能冗余。这些结果为混合模型压缩、架构设计和容错部署提供了可操作的指导。

Title: Rashid: A Cipher-Based Framework for Exploring In-Context Language Learning

• Authors: Niyati Bafna, Ryan Soh-Eun Shim, Barbara Plank, David Yarowsky, Hale Sirin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22497
• Pdf URL: https://arxiv.org/pdf/2603.22497
• Copy Paste: [[2603.22497]] Rashid: A Cipher-Based Framework for Exploring In-Context Language Learning(https://arxiv.org/abs/2603.22497)
• Keywords: language model
• Abstract: Where there is growing interest in in-context language learning (ICLL) for unseen languages with large language models, such languages usually suffer from the lack of NLP tools, data resources, and researcher expertise. This means that progress is difficult to assess, the field does not allow for cheap large-scale experimentation, and findings on ICLL are often limited to very few languages and tasks. In light of such limitations, we introduce a framework (Rashid), for studying ICLL wherein we reversibly cipher high-resource languages (HRLs) to construct truly unseen languages with access to a wide range of resources available for HRLs, unlocking previously impossible exploration of ICLL phenomena. We use our framework to assess current methods in the field with SOTA evaluation tools and manual analysis, explore the utility of potentially expensive resources in improving ICLL, and test ICLL strategies on rich downstream tasks beyond machine translation. These lines of exploration showcase the possibilities enabled by our framework, as well as providing actionable insights regarding current performance and future directions in ICLL.
• 摘要：人们对具有大型语言模型的未见过的语言的上下文语言学习（ICLL）越来越感兴趣，但此类语言通常缺乏 NLP 工具、数据资源和研究人员的专业知识。这意味着进展难以评估，该领域不允许进行廉价的大规模实验，并且 ICLL 的发现通常仅限于极少数的语言和任务。鉴于这些限制，我们引入了一个用于研究 ICLL 的框架（Rashid），其中我们对高资源语言（HRL）进行可逆加密，以构建真正未见过的语言，并能够访问 HRL 可用的广泛资源，从而解锁以前不可能探索的 ICLL 现象。我们使用我们的框架通过 SOTA 评估工具和手动分析来评估该领域的当前方法，探索潜在昂贵资源在改进 ICLL 中的效用，并在机器翻译之外的丰富下游任务上测试 ICLL 策略。这些探索展示了我们的框架所带来的可能性，并提供了有关 ICLL 当前表现和未来方向的可行见解。

Title: CAPITU: A Benchmark for Evaluating Instruction-Following in Brazilian Portuguese with Literary Context

• Authors: Giovana Kerche Bonás, Roseval Malaquias Junior, Marcos Piau, Thiago Laitz, Thales Sales Almeida, Hugo Abonizio, Celio Larcher, Ramon Pires, Rodrigo Nogueira
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22576
• Pdf URL: https://arxiv.org/pdf/2603.22576
• Copy Paste: [[2603.22576]] CAPITU: A Benchmark for Evaluating Instruction-Following in Brazilian Portuguese with Literary Context(https://arxiv.org/abs/2603.22576)
• Keywords: language model, gpt, llm, prompt
• Abstract: We introduce CAPITU, a benchmark for evaluating instruction-following capabilities of Large Language Models (LLMs) in Brazilian Portuguese. Unlike existing benchmarks that focus on English or use generic prompts, CAPITU contextualizes all tasks within eight canonical works of Brazilian literature, combining verifiable instruction constraints with culturally-grounded content. The benchmark comprises 59 instruction types organized into seven categories, all designed to be automatically verifiable without requiring LLM judges or human evaluation. Instruction types include Portuguese-specific linguistic constraints (word termination patterns like -ando/-endo/-indo, -inho/-inha, -mente) and structural requirements. We evaluate 18 state-of-the-art models across single-turn and multi-turn settings. Our results show that frontier reasoning models achieve strong performance (GPT-5.2 with reasoning: 98.5% strict accuracy), while Portuguese-specialized models offer competitive cost-efficiency (Sabiazinho-4: 87.0% at \$0.13 vs Claude-Haiku-4.5: 73.5% at \$1.12). Multi-turn evaluation reveals significant variation in constraint persistence, with conversation-level accuracy ranging from 60% to 96% across models. We identify specific challenges in morphological constraints, exact counting, and constraint persistence degradation across turns. We release the complete benchmark, evaluation code, and baseline results to facilitate research on instruction-following in Portuguese.
• 摘要：我们引入 CAPITU，这是一个评估巴西葡萄牙语大型语言模型 (LLM) 指令跟踪能力的基准。与专注于英语或使用通用提示的现有基准不同，CAPITU 将巴西文学的八部经典作品中的所有任务置于语境中，将可验证的教学限制与基于文化的内容相结合。该基准包括 7 个类别的 59 种指令类型，所有指令类型均设计为可自动验证，无需法学硕士法官或人工评估。指令类型包括葡萄牙语特定的语言限制（单词终止模式，如 -ando/-endo/-indo、-inho/-inha、-mente）和结构要求。我们评估了 18 个最先进的单匝和多匝设置模型。我们的结果表明，前沿推理模型实现了强大的性能（带推理的 GPT-5.2：严格准确度为 98.5%），而葡萄牙语专用模型提供了具有竞争力的成本效率（Sabiazinho-4：87.0%，0.13 美元 vs Claude-Haiku-4.5：73.5%，1.12 美元）。多轮评估揭示了约束持久性的显着差异，跨模型的对话级别准确度范围为 60% 到 96%。我们确定了形态约束、精确计数和回合间约束持久性退化方面的具体挑战。我们发布完整的基准、评估代码和基线结果，以促进葡萄牙语指令遵循的研究。

Title: Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?

• Authors: Richard J. Young
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.22582
• Pdf URL: https://arxiv.org/pdf/2603.22582
• Copy Paste: [[2603.22582]] Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?(https://arxiv.org/abs/2603.22582)
• Keywords: language model, chain-of-thought
• Abstract: Chain-of-thought (CoT) reasoning has been proposed as a transparency mechanism for large language models in safety-critical deployments, yet its effectiveness depends on faithfulness (whether models accurately verbalize the factors that actually influence their outputs), a property that prior evaluations have examined in only two proprietary models, finding acknowledgment rates as low as 25% for Claude 3.7 Sonnet and 39% for DeepSeek-R1. To extend this evaluation across the open-weight ecosystem, this study tests 12 open-weight reasoning models spanning 9 architectural families (7B-685B parameters) on 498 multiple-choice questions from MMLU and GPQA Diamond, injecting six categories of reasoning hints (sycophancy, consistency, visual pattern, metadata, grader hacking, and unethical information) and measuring the rate at which models acknowledge hint influence in their CoT when hints successfully alter answers. Across 41,832 inference runs, overall faithfulness rates range from 39.7% (Seed-1.6-Flash) to 89.9% (DeepSeek-V3.2-Speciale) across model families, with consistency hints (35.5%) and sycophancy hints (53.9%) exhibiting the lowest acknowledgment rates. Training methodology and model family predict faithfulness more strongly than parameter count, and keyword-based analysis reveals a striking gap between thinking-token acknowledgment (approximately 87.5%) and answer-text acknowledgment (approximately 28.6%), suggesting that models internally recognize hint influence but systematically suppress this acknowledgment in their outputs. These findings carry direct implications for the viability of CoT monitoring as a safety mechanism and suggest that faithfulness is not a fixed property of reasoning models but varies systematically with architecture, training method, and the nature of the influencing cue.
• 摘要：思想链 (CoT) 推理已被提议作为安全关键型部署中大型语言模型的透明机制，但其有效性取决于忠实度（模型是否准确地表达了实际影响其输出的因素），之前的评估仅在两个专有模型中检查了这一属性，发现 Claude 3.7 Sonnet 的确认率低至 25%，DeepSeek-R1 的确认率低至 39%。为了将这种评估扩展到整个开放权重生态系统，本研究针对来自 MMLU 和 GPQA Diamond 的 498 个多项选择题测试了跨越 9 个架构系列（7B-685B 参数）的 12 个开放权重推理模型，注入了六类推理提示（阿谀奉承、一致性、视觉模式、元数据、评分者黑客攻击和不道德信息），并测量了当提示成功改变答案时模型在其 CoT 中承认提示影响的速率。在 41,832 次推理运行中，各模型系列的总体忠实率范围为 39.7% (Seed-1.6-Flash) 到 89.9% (DeepSeek-V3.2-Speciale)，其中一致性提示 (35.5%) 和阿谀奉承提示 (53.9%) 的确认率最低。训练方法和模型族比参数计数更能预测忠实度，基于关键词的分析揭示了思维标记确认（约 87.5%）和答案文本确认（约 28.6%）之间存在显着差距，这表明模型在内部识别提示影响，但在输出中系统性地抑制了这种确认。这些发现对 CoT 监控作为安全机制的可行性具有直接影响，并表明忠实度不是推理模型的固定属性，而是随着架构、训练方法和影响线索的性质而系统地变化。

Title: LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation

• Authors: Hailay Teklehaymanot, Dren Fazlija, Wolfgang Nejdl
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.22629
• Pdf URL: https://arxiv.org/pdf/2603.22629
• Copy Paste: [[2603.22629]] LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation(https://arxiv.org/abs/2603.22629)
• Keywords: language model
• Abstract: Adapting pretrained language models to low-resource, morphologically rich languages remains a significant challenge. Existing vocabulary expansion methods typically rely on arbitrarily segmented subword units, resulting in fragmented lexical representations and loss of critical morphological information. To address this limitation, we propose the Lexically Grounded Subword Embedding Initialization (LGSE) framework, which introduces morphologically informed segmentation for initializing embeddings of novel tokens. Instead of using random vectors or arbitrary subwords, LGSE decomposes words into their constituent morphemes and constructs semantically coherent embeddings by averaging pretrained subword or FastText-based morpheme representations. When a token cannot be segmented into meaningful morphemes, its embedding is constructed using character n-gram representations to capture structural information. During Language-Adaptive Pretraining, we apply a regularization term that penalizes large deviations of newly introduced embeddings from their initialized values, preserving alignment with the original pretrained embedding space while enabling adaptation to the target language. To isolate the effect of initialization, we retain the original pre-trained model vocabulary and tokenizer and update only the new embeddings during adaptation. We evaluate LGSE on three NLP tasks: Question Answering, Named Entity Recognition, and Text Classification, in two morphologically rich, low-resource languages: Amharic and Tigrinya, where morphological segmentation resources are available. Experimental results show that LGSE consistently outperforms baseline methods across all tasks, demonstrating the effectiveness of morphologically grounded embedding initialization for improving representation quality in underrepresented languages. Project resources are available in the GitHub link.
• 摘要：将预训练的语言模型适应资源匮乏、形态丰富的语言仍然是一个重大挑战。现有的词汇扩展方法通常依赖于任意分段的子词单元，导致词汇表示碎片化和关键形态信​​息的丢失。为了解决这个限制，我们提出了词法基础子词嵌入初始化（LGSE）框架，该框架引入了形态学分割来初始化新标记的嵌入。 LGSE 不使用随机向量或任意子词，而是将单词分解为其组成词素，并通过平均预训练子词或基于 FastText 的词素表示来构造语义连贯的嵌入。当一个 token 无法被分割成有意义的语素时，它的嵌入是使用字符 n-gram 表示来构建的，以捕获结构信息。在语言自适应预训练期间，我们应用正则化项来惩罚新引入的嵌入与其初始化值的较大偏差，保持与原始预训练嵌入空间的对齐，同时能够适应目标语言。为了隔离初始化的影响，我们保留原始的预训练模型词汇和分词器，并在适应过程中仅更新新的嵌入。我们在三个 NLP 任务上评估 LGSE：问答、命名实体识别和文本分类，使用两种形态丰富、资源匮乏的语言：阿姆哈拉语和提格里尼亚语，其中形态分割资源可用。实验结果表明，LGSE 在所有任务中始终优于基线方法，证明了基于形态的嵌入初始化对于提高代表性不足的语言的表示质量的有效性。项目资源可在 GitHub 链接中获取。

Title: Multi-Method Validation of Large Language Model Medical Translation Across High- and Low-Resource Languages

• Authors: Chukwuebuka Anyaegbuna, Eduardo Juan Perez Guerrero, Jerry Liu, Timothy Keyes, April Liang, Natasha Steele, Stephen Ma, Jonathan Chen, Kevin Schulman
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22642
• Pdf URL: https://arxiv.org/pdf/2603.22642
• Copy Paste: [[2603.22642]] Multi-Method Validation of Large Language Model Medical Translation Across High- and Low-Resource Languages(https://arxiv.org/abs/2603.22642)
• Keywords: language model, gpt, llm
• Abstract: Language barriers affect 27.3 million U.S. residents with non-English language preference, yet professional medical translation remains costly and often unavailable. We evaluated four frontier large language models (GPT-5.1, Claude Opus 4.5, Gemini 3 Pro, Kimi K2) translating 22 medical documents into 8 languages spanning high-resource (Spanish, Chinese, Russian, Vietnamese), medium-resource (Korean, Arabic), and low-resource (Tagalog, Haitian Creole) categories using a five-layer validation framework. Across 704 translation pairs, all models achieved high semantic preservation (LaBSE greater than 0.92), with no significant difference between high- and low-resource languages (p = 0.066). Cross-model back-translation confirmed results were not driven by same-model circularity (delta = -0.0009). Inter-model concordance across four independently trained models was high (LaBSE: 0.946), and lexical borrowing analysis showed no correlation between English term retention and fidelity scores in low-resource languages (rho = +0.018, p = 0.82). These converging results suggest frontier LLMs preserve medical meaning across resource levels, with implications for language access in healthcare.
• 摘要：语言障碍影响着 2730 万非英语语言偏好的美国居民，但专业的医学翻译仍然昂贵且常常无法获得。我们使用五层验证框架评估了四种前沿大型语言模型（GPT-5.1、Claude Opus 4.5、Gemini 3 Pro、Kimi K2），将 22 份医疗文档翻译成 8 种语言，涵盖高资源（西班牙语、中文、俄语、越南语）、中等资源（韩语、阿拉伯语）和低资源（他加禄语、海地克里奥尔语）类别。在 704 个翻译对中，所有模型都实现了高语义保留（LaBSE 大于 0.92），高资源语言和低资源语言之间没有显着差异（p = 0.066）。跨模型反向翻译证实结果不是由同模型循环驱动的（delta = -0.0009）。四个独立训练的模型之间的模型间一致性很高（LaBSE：0.946），词汇借用分析显示英语术语保留率与低资源语言的保真度分数之间没有相关性（rho = +0.018，p = 0.82）。这些趋同的结果表明，前沿法学硕士在不同资源水平上保留了医学意义，这对医疗保健中的语言获取产生了影响。

Title: Improving LLM Predictions via Inter-Layer Structural Encoders

• Authors: Tom Ulanovski (1), Eyal Blyachman (1), Maya Bechler-Speicher (2) ((1) Tel Aviv University, (2) Meta)
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.22665
• Pdf URL: https://arxiv.org/pdf/2603.22665
• Copy Paste: [[2603.22665]] Improving LLM Predictions via Inter-Layer Structural Encoders(https://arxiv.org/abs/2603.22665)
• Keywords: language model, llm
• Abstract: The standard practice in Large Language Models (LLMs) is to base predictions on the final-layer token representations. Recent studies, however, show that intermediate layers encode substantial information, which may contain more task-relevant features than the final-layer representations alone. Importantly, it was shown that for different tasks, different layers may be optimal. In this work we introduce Inter-Layer Structural Encoders (ILSE), a powerful structural approach to learn one effective representation from the LLM's internal layer representations all together. Central to ILSE is Cayley-Encoder, a mathematically grounded geometric encoder that leverages expander Cayley graphs for efficient inter-layer information propagation. We evaluate ILSE across 13 classification and semantic similarity tasks with 9 pre-trained LLMs ranging from 14 million to 8 billion parameters. ILSE consistently outperforms baselines and existing approaches, achieving up to 44% improvement in accuracy and 25% in similarity metrics. We further show that ILSE is data-efficient in few-shot regimes and can make small LLMs competitive with substantially larger models.
• 摘要：大型语言模型 (LLM) 的标准做法是根据最终层标记表示进行预测。然而，最近的研究表明，中间层编码大量信息，这些信息可能比单独的最终层表示包含更多与任务相关的特征。重要的是，研究表明，对于不同的任务，不同的层可能是最佳的。在这项工作中，我们介绍了层间结构编码器（ILSE），这是一种强大的结构方法，可以从 LLM 的内部层表示中学习一种有效的表示。 ILSE 的核心是 Cayley-Encoder，这是一种以数学为基础的几何编码器，它利用扩展器 Cayley 图来实现高效的层间信息传播。我们通过 9 个预训练的 LLM 评估 13 个分类和语义相似性任务的 ILSE，参数范围从 1400 万到 80 亿个参数。 ILSE 始终优于基线和现有方法，准确率提高了 44%，相似性指标提高了 25%。我们进一步表明，ILSE 在少数情况下具有数据效率，并且可以使小型 LLM 与更大的模型竞争。

Title: Synthetic or Authentic? Building Mental Patient Simulators from Longitudinal Evidence

• Authors: Baihan Li, Bingrui Jin, Kunyao Lan, Ming Wang, Mengyue Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22704
• Pdf URL: https://arxiv.org/pdf/2603.22704
• Copy Paste: [[2603.22704]] Synthetic or Authentic? Building Mental Patient Simulators from Longitudinal Evidence(https://arxiv.org/abs/2603.22704)
• Keywords: language model, llm, prompt, agent
• Abstract: Patient simulation is essential for developing and evaluating mental health dialogue systems. As most existing approaches rely on snapshot-style prompts with limited profile information, homogeneous behaviors and incoherent disease progression in multi-turn interactions have become key chellenges. In this work, we propose DEPROFILE, a data-grounded patient simulation framework that constructs unified, multi-source patient profiles by integrating demographic attributes, standardized clinical symptoms, counseling dialogues, and longitudinal life-event histories from real-world data. We further introduce a Chain-of-Change agent to transform noisy longitudinal records into structured, temporally grounded memory representations for simulation. Experiments across multiple large language model (LLM) backbones show that with more comprehensive profile constructed by DEPROFILE, the dialogue realism, behavioral diversity, and event richness have consistently improved and exceed state-of-the-art baselines, highlighting the importance of grounding patient simulation in verifiable longitudinal evidence.
• 摘要：患者模拟对于开发和评估心理健康对话系统至关重要。由于大多数现有方法依赖于快照式提示，且资料信息有限，因此多轮交互中的同质行为和不连贯的疾病进展已成为关键挑战。在这项工作中，我们提出了 DEPROFILE，一种基于数据的患者模拟框架，通过整合人口统计属性、标准化临床症状、咨询对话和来自现实世界数据的纵向生活事件历史来构建统一的多源患者档案。我们进一步引入了变化链代理，将嘈杂的纵向记录转换为结构化的、基于时间的记忆表示以进行模拟。跨多个大语言模型 (LLM) 主干的实验表明，通过 DEPROFILE 构建的更全面的配置文件，对话真实性、行为多样性和事件丰富性持续改进并超过了最先进的基线，凸显了将患者模拟建立在可验证的纵向证据基础上的重要性。

Title: Detecting Non-Membership in LLM Training Data via Rank Correlations

• Authors: Pranav Shetty, Mirazul Haque, Zhiqiang Ma, Xiaomo Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22707
• Pdf URL: https://arxiv.org/pdf/2603.22707
• Copy Paste: [[2603.22707]] Detecting Non-Membership in LLM Training Data via Rank Correlations(https://arxiv.org/abs/2603.22707)
• Keywords: language model, llm
• Abstract: As large language models (LLMs) are trained on increasingly vast and opaque text corpora, determining which data contributed to training has become essential for copyright enforcement, compliance auditing, and user trust. While prior work focuses on detecting whether a dataset was used in training (membership inference), the complementary problem -- verifying that a dataset was not used -- has received little attention. We address this gap by introducing PRISM, a test that detects dataset-level non-membership using only grey-box access to model logits. Our key insight is that two models that have not seen a dataset exhibit higher rank correlation in their normalized token log probabilities than when one model has been trained on that data. Using this observation, we construct a correlation-based test that detects non-membership. Empirically, PRISM reliably rules out membership in training data across all datasets tested while avoiding false positives, thus offering a framework for verifying that specific datasets were excluded from LLM training.
• 摘要：随着大型语言模型 (LLM) 在日益庞大且不透明的文本语料库上进行训练，确定哪些数据有助于训练对于版权执法、合规审计和用户信任变得至关重要。虽然之前的工作重点是检测训练中是否使用了数据集（隶属推断），但补充问题（验证数据集未使用）却很少受到关注。我们通过引入 PRISM 来解决这一差距，该测试仅使用对模型 logits 的灰盒访问来检测数据集级非成员资格。我们的主要见解是，两个没有见过数据集的模型在其归一化标记对数概率中表现出比在该数据上训练一个模型时更高的排名相关性。利用这一观察结果，我们构建了一个基于相关性的测试来检测非成员资格。根据经验，PRISM 可靠地排除了所有测试数据集的训练数据中的成员资格，同时避免了误报，从而提供了一个框架来验证特定数据集是否已从 LLM 训练中排除。

Title: Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics

• Authors: Naohiro Tawara, Samuele Cornell, Alexander Polok, Marc Delcroix, Lukáš Burget, Shinji Watanabe
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2603.22709
• Pdf URL: https://arxiv.org/pdf/2603.22709
• Copy Paste: [[2603.22709]] Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics(https://arxiv.org/abs/2603.22709)
• Keywords: language model, llm
• Abstract: Conversational automatic speech recognition remains challenging due to overlapping speech, far-field noise, and varying speaker counts. While recent LLM-based systems perform well on single-speaker benchmarks, their robustness in multi-speaker settings is unclear. We systematically compare LLM-based and modular pipeline approaches along four axes: overlap robustness, semantic fidelity, speaker count, and single- versus multi-channel input. To capture meaning-altering errors that conventional metrics miss, we introduce tcpSemER, which extends tcpWER by replacing Levenshtein distance with embedding-based semantic similarity. We further decompose tcpWER into overlapping and non-overlapping components for finer-grained analysis. Experiments across three datasets show that LLM-based systems are competitive in two-speaker settings but degrade as speaker count and overlap increase, whereas modular pipelines remain more robust.
• 摘要：由于语音重叠、远场噪声和不同的说话者数量，会话式自动语音识别仍然具有挑战性。虽然最近基于 LLM 的系统在单扬声器基准测试中表现良好，但它们在多扬声器设置中的稳健性尚不清楚。我们沿着四个轴系统地比较基于 LLM 和模块化管道方法：重叠鲁棒性、语义保真度、说话者数量以及单通道与多通道输入。为了捕获传统指标遗漏的意义改变错误，我们引入了 tcpSemER，它通过用基于嵌入的语义相似性替换 Levenshtein 距离来扩展 tcpWER。我们进一步将 tcpWER 分解为重叠和非重叠组件，以进行更细粒度的分析。三个数据集的实验表明，基于 LLM 的系统在两个扬声器设置中具有竞争力，但随着扬声器数量和重叠的增加而降低，而模块化管道仍然更加稳健。

Title: How Utilitarian Are OpenAI's Models Really? Replicating and Reinterpreting Pfeffer, Krügel, and Uhl (2025)

• Authors: Johannes Himmelreich
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2603.22730
• Pdf URL: https://arxiv.org/pdf/2603.22730
• Copy Paste: [[2603.22730]] How Utilitarian Are OpenAI's Models Really? Replicating and Reinterpreting Pfeffer, Krügel, and Uhl (2025)(https://arxiv.org/abs/2603.22730)
• Keywords: gpt, llm, prompt
• Abstract: Pfeffer, Krügel, and Uhl (2025) report that OpenAI's reasoning model o1-mini produces more utilitarian responses to the trolley problem and footbridge dilemma than the non-reasoning model GPT-4o. I replicate their study with four current OpenAI models and extend it with prompt variant testing. The trolley finding does not survive: GPT-4o's low utilitarian rate doesn't reflect a deontological commitment but safety refusals triggered by the prompt's advisory framing. When framed as "Is it morally permissible...?" instead of "Should I...?", GPT-4o gives 99% utilitarian responses. All models converge on utilitarian answers when prompt confounds are removed. The footbridge finding survives with blemishes. Reasoning models tend to give more utilitarian responses than non-reasoning models across prompt variations. But often they refuse to answer the dilemma or, when they answer, give a non-utilitarian rather than a utilitarian answer. These results demonstrate that single-prompt evaluations of LLM moral reasoning are unreliable: multi-prompt robustness testing should be standard practice for any empirical claim about LLM behavior.
• 摘要：Pfeffer、Krügel 和 Uhl（2025）报告称，OpenAI 的推理模型 o1-mini 比非推理模型 GPT-4o 对电车问题和人行桥困境产生了更多功利主义的反应。我用四个当前的 OpenAI 模型复制了他们的研究，并通过即时变体测试对其进行了扩展。手推车的发现并没有继续存在：GPT-4o 的低功利率并不反映义务论承诺，而是由提示的建议框架引发的安全拒绝。当被框定为“道德上允许吗……？” GPT-4o 给出了 99% 功利性的回答，而不是“我应该……吗？”。当立即混淆被消除后，所有模型都趋于功利主义答案。人行桥的发现仍然存在瑕疵。在提示变化中，推理模型往往比非推理模型给出更多功利性的反应。但他们常常拒绝回答这个困境，或者当他们回答时，给出一个非功利主义的而不是功利主义的答案。这些结果表明，对法学硕士道德推理的单提示评估是不可靠的：多提示稳健性测试应该成为任何有关法学硕士行为的实证主张的标准做法。

Title: Explanation Generation for Contradiction Reconciliation with LLMs

• Authors: Jason Chan, Zhixue Zhao, Robert Gaizauskas
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22735
• Pdf URL: https://arxiv.org/pdf/2603.22735
• Copy Paste: [[2603.22735]] Explanation Generation for Contradiction Reconciliation with LLMs(https://arxiv.org/abs/2603.22735)
• Keywords: language model, llm, chat
• Abstract: Existing NLP work commonly treats contradictions as errors to be resolved by choosing which statements to accept or discard. Yet a key aspect of human reasoning in social interactions and professional domains is the ability to hypothesize explanations that reconcile contradictions. For example, "Cassie hates coffee" and "She buys coffee everyday" may appear contradictory, yet both are compatible if Cassie has the unenviable daily chore of buying coffee for all her coworkers. Despite the growing reasoning capabilities of large language models (LLMs), their ability to hypothesize such reconciliatory explanations remains largely unexplored. To address this gap, we introduce the task of reconciliatory explanation generation, where models must generate explanations that effectively render contradictory statements compatible. We propose a novel method of repurposing existing natural language inference (NLI) datasets, and introduce quality metrics that enable scalable automatic evaluation. Experiments with 18 LLMs show that most models achieve limited success in this task, and that the benefit of extending test-time compute by "thinking" plateaus as model size increases. Our results highlight an under-explored dimension of LLM reasoning and the need to address this limitation in enhancing LLMs' downstream applications such as chatbots and scientific aids.
• 摘要：现有的 NLP 工作通常将矛盾视为错误，通过选择接受或丢弃哪些陈述来解决。然而，人类在社会互动和专业领域推理的一个关键方面是能够提出调和矛盾的假设解释。例如，“Cassie 讨厌咖啡”和“她每天都买咖啡”可能会显得矛盾，但如果 Cassie 每天都要为她的所有同事买咖啡这一令人羡慕的日常琐事，那么两者都是兼容的。尽管大型语言模型（LLM）的推理能力不断增强，但它们假设这种和解解释的能力在很大程度上仍未得到探索。为了解决这一差距，我们引入了和解解释生成的任务，其中模型必须生成有效地使矛盾陈述兼容的解释。我们提出了一种重新利用现有自然语言推理（NLI）数据集的新方法，并引入了可扩展自动评估的质量指标。对 18 个法学硕士的实验表明，大多数模型在此任务中取得的成功有限，并且随着模型大小的增加，通过“思考”平台来延长测试时间计算的好处。我们的结果强调了法学硕士推理的一个尚未充分探索的维度，以及在增强法学硕士的下游应用（例如聊天机器人和科学辅助工具）方面解决这一限制的必要性。

Title: PRISM: A Dual View of LLM Reasoning through Semantic Flow and Latent Computation

• Authors: Ruidi Chang, Jiawei Zhou, Hanjie Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22754
• Pdf URL: https://arxiv.org/pdf/2603.22754
• Copy Paste: [[2603.22754]] PRISM: A Dual View of LLM Reasoning through Semantic Flow and Latent Computation(https://arxiv.org/abs/2603.22754)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) solve complex problems by generating multi-step reasoning traces. Yet these traces are typically analyzed from only one of two perspectives: the sequence of tokens across different reasoning steps in the generated text, or the hidden-state vectors across model layers within one step. We introduce PRISM (Probabilistic Reasoning Inspection through Semantic and Implicit Modeling), a framework and diagnostic tool for jointly analyzing both levels, providing a unified view of how reasoning evolves across steps and layers. Across multiple reasoning models and benchmarks, PRISM uncovers systematic patterns in the reasoning process, showing that failed trajectories are more likely to become trapped in unproductive verification loops and further diverge into distinct modes such as overthinking and premature commitment, which behave differently once a candidate answer is reached. It further reveals how prompting reshapes reasoning behavior beyond aggregate accuracy by altering both semantic transitions and internal computational patterns. By modeling reasoning trajectories as structured processes, PRISM makes these behaviors observable and analyzable rather than relying solely on final-task accuracy. Taken together, these insights position PRISM as a practical tool for analyzing and diagnosing reasoning processes in LLMs.
• 摘要：大型语言模型 (LLM) 通过生成多步骤推理轨迹来解决复杂问题。然而，这些痕迹通常仅从两个角度之一进行分析：生成文本中不同推理步骤的标记序列，或一步内跨模型层的隐藏状态向量。我们引入了 PRISM（通过语义和隐式建模进行概率推理检查），这是一个用于联合分析两个级别的框架和诊断工具，提供推理如何跨步骤和层演变的统一视图。在多个推理模型和基准中，PRISM 揭示了推理过程中的系统模式，表明失败的轨迹更有可能陷入无效的验证循环中，并进一步分化为不同的模式，例如过度思考和过早承诺，一旦达到候选答案，它们的行为就会有所不同。它进一步揭示了提示如何通过改变语义转换和内部计算模式来重塑超越总体准确性的推理行为。通过将推理轨迹建模为结构化过程，PRISM 使这些行为可观察和可分析，而不是仅仅依赖于最终任务的准确性。总而言之，这些见解使 PRISM 成为分析和诊断法学硕士推理过程的实用工具。

Title: KALAVAI: Predicting When Independent Specialist Fusion Works -- A Quantitative Model for Post-Hoc Cooperative LLM Training

• Authors: Ramchand Kumaresan
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.22755
• Pdf URL: https://arxiv.org/pdf/2603.22755
• Copy Paste: [[2603.22755]] KALAVAI: Predicting When Independent Specialist Fusion Works -- A Quantitative Model for Post-Hoc Cooperative LLM Training(https://arxiv.org/abs/2603.22755)
• Keywords: llm
• Abstract: Independently trained domain specialists can be fused post-hoc into a single model that outperforms any individual specialist, and the gain is predictable: gain = 0.82 x divergence - 2.72 (R^2 = 0.856, n=6, 3-26% divergence). This enables practitioners to estimate cooperative value before committing compute. Below ~3.3% divergence, gains approach this http URL the KALAVAI protocol, contributors fine-tune copies of a shared checkpoint independently, then submit for lightweight MoE routing (500 steps). Gains are consistent: +7.72% at 410M (+/-0.02%, 3 seeds), +7.49% at 1B (+/-0.01%, 3 seeds), +6.53% at 6.9B, each over the best specialist. The router matches domain-oracle routing within <10^{-5} nats. Cross-lingual fusion (Tamil/Yoruba/Welsh/Code) achieves +21.76%, with Yoruba perplexity falling 41.9 to 7.7. A 20-contributor federation achieves +16.71% (+/-0.07pp, 3 seeds).Three requirements bound the protocol. Shared initialisation is necessary: checkpoint mismatch degrades routing. Frozen layers are optional below ~10,000 steps and beneficial beyond. Learned routing is essential: uniform averaging degrades by -1.2% vs. best specialist, while any trained router achieves oracle-optimal assignment.
• 摘要：独立训练的领域专家可以事后融合到一个优于任何单个专家的单一模型中，并且增益是可预测的：增益 = 0.82 x 散度 - 2.72（R^2 = 0.856，n=6，3-26% 散度）。这使得从业者能够在提交计算之前估计合作价值。低于约 3.3% 的差异，通过 KALAVAI 协议获取此 http URL，贡献者独立微调共享检查点的副本，然后提交轻量级 MoE 路由（500 个步骤）。增益一致：410M 时 +7.72%（+/-0.02%，3 颗种子），1B 时 +7.49%（+/-0.01%，3 颗种子），6.9B 时 +6.53%，均超过最佳专家。路由器匹配 <10^{-5} nat 内的域-oracle 路由。跨语言融合（泰米尔语/约鲁巴语/威尔士语/法典）达到+21.76%，约鲁巴语困惑度从 41.9 下降到 7.7。一个由 20 名贡献者组成的联盟实现了 +16.71%（+/-0.07pp，3 个种子）。该协议受三个要求约束。共享初始化是必要的：检查点不匹配会降低路由性能。冻结层在~10,000步以下是可选的，超过10,000步是有益的。学习路由至关重要：与最佳专家相比，统一平均性能下降 -1.2%，而任何经过训练的路由器都可以实现预言机最优分配。

Title: DALDALL: Data Augmentation for Lexical and Semantic Diverse in Legal Domain by leveraging LLM-Persona

• Authors: Janghyeok Choi, Jaewon Lee, Sungzoon Cho
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2603.22765
• Pdf URL: https://arxiv.org/pdf/2603.22765
• Copy Paste: [[2603.22765]] DALDALL: Data Augmentation for Lexical and Semantic Diverse in Legal Domain by leveraging LLM-Persona(https://arxiv.org/abs/2603.22765)
• Keywords: language model, llm, prompt
• Abstract: Data scarcity remains a persistent challenge in low-resource domains. While existing data augmentation methods leverage the generative capabilities of large language models (LLMs) to produce large volumes of synthetic data, these approaches often prioritize quantity over quality and lack domain-specific strategies. In this work, we introduce DALDALL, a persona-based data augmentation framework tailored for legal information retrieval (IR). Our method employs domain-specific professional personas--such as attorneys, prosecutors, and judges--to generate synthetic queries that exhibit substantially greater lexical and semantic diversity than vanilla prompting approaches. Experiments on the CLERC and COLIEE benchmarks demonstrate that persona-based augmentation achieves improvement in lexical diversity as measured by Self-BLEU scores, while preserving semantic fidelity to the original queries. Furthermore, dense retrievers fine-tuned on persona-augmented data consistently achieve competitive or superior recall performance compared to those trained on original data or generic augmentations. These findings establish persona-based prompting as an effective strategy for generating high-quality training data in specialized, low-resource domains.
• 摘要：数据稀缺仍然是资源匮乏领域的一个持续挑战。虽然现有的数据增强方法利用大型语言模型 (LLM) 的生成能力来生成大量合成数据，但这些方法通常优先考虑数量而非质量，并且缺乏特定于领域的策略。在这项工作中，我们介绍了 DALDALL，这是一种专为法律信息检索 (IR) 量身定制的基于角色的数据增强框架。我们的方法采用特定领域的专业角色（例如律师、检察官和法官）来生成综合查询，这些查询表现出比普通提示方法更大的词汇和语义多样性。 CLERC 和 COLIEE 基准测试表明，基于角色的增强可以提高通过 Self-BLEU 分数衡量的词汇多样性，同时保留对原始查询的语义保真度。此外，与那些在原始数据或通用增强数据上训练的密集检索器相比，在角色增强数据上进行微调的密集检索器始终能够实现有竞争力或优越的召回性能。这些发现将基于角色的提示确立为在专门的、资源匮乏的领域生成高质量训练数据的有效策略。

Title: Span Modeling for Idiomaticity and Figurative Language Detection with Span Contrastive Loss

• Authors: Blake Matheny, Phuong Minh Nguyen, Minh Le Nguyen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22799
• Pdf URL: https://arxiv.org/pdf/2603.22799
• Copy Paste: [[2603.22799]] Span Modeling for Idiomaticity and Figurative Language Detection with Span Contrastive Loss(https://arxiv.org/abs/2603.22799)
• Keywords: language model, prompt
• Abstract: The category of figurative language contains many varieties, some of which are non-compositional in nature. This type of phrase or multi-word expression (MWE) includes idioms, which represent a single meaning that does not consist of the sum of its words. For language models, this presents a unique problem due to tokenization and adjacent contextual embeddings. Many large language models have overcome this issue with large phrase vocabulary, though immediate recognition frequently fails without one- or few-shot prompting or instruction finetuning. The best results have been achieved with BERT-based or LSTM finetuning approaches. The model in this paper contains one such variety. We propose BERT- and RoBERTa-based models finetuned with a combination of slot loss and span contrastive loss (SCL) with hard negative reweighting to improve idiomaticity detection, attaining state of the art sequence accuracy performance on existing datasets. Comparative ablation studies show the effectiveness of SCL and its generalizability. The geometric mean of F1 and sequence accuracy (SA) is also proposed to assess a model's span awareness and general performance together.
• 摘要：比喻语言的范畴包含许多变体，其中一些本质上是非构成性的。这种类型的短语或多词表达（MWE）包括习语，它们代表不由单词总和组成的单一含义。对于语言模型来说，由于标记化和相邻上下文嵌入，这提出了一个独特的问题。许多大型语言模型已经通过大量短语词汇克服了这个问题，尽管如果没有一次或几次提示或指令微调，立即识别经常会失败。基于 BERT 或 LSTM 的微调方法已经取得了最佳结果。本文中的模型就包含这样一种类型。我们提出基于 BERT 和 RoBERTa 的模型，结合时隙损失和跨度对比损失 (SCL) 以及硬负重加权来进行微调，以改进惯用性检测，在现有数据集上实现最先进的序列准确性性能。比较消融研究表明了 SCL 的有效性及其普遍性。还提出了 F1 的几何平均值和序列精度（SA）来共同评估模型的跨度感知和总体性能。

Title: Efficient Hallucination Detection: Adaptive Bayesian Estimation of Semantic Entropy with Guided Semantic Exploration

• Authors: Qiyao Sun, Xingming Li, Xixiang He, Ao Cheng, Xuanyu Ji, Hailun Lu, Runke Huang, Qingyong Hu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22812
• Pdf URL: https://arxiv.org/pdf/2603.22812
• Copy Paste: [[2603.22812]] Efficient Hallucination Detection: Adaptive Bayesian Estimation of Semantic Entropy with Guided Semantic Exploration(https://arxiv.org/abs/2603.22812)
• Keywords: language model, llm, hallucination
• Abstract: Large language models (LLMs) have achieved remarkable success in various natural language processing tasks, yet they remain prone to generating factually incorrect outputs known as hallucinations. While recent approaches have shown promise for hallucination detection by repeatedly sampling from LLMs and quantifying the semantic inconsistency among the generated responses, they rely on fixed sampling budgets that fail to adapt to query complexity, resulting in computational inefficiency. We propose an Adaptive Bayesian Estimation framework for Semantic Entropy with Guided Semantic Exploration, which dynamically adjusts sampling requirements based on observed uncertainty. Our approach employs a hierarchical Bayesian framework to model the semantic distribution, enabling dynamic control of sampling iterations through variance-based thresholds that terminate generation once sufficient certainty is achieved. We also develop a perturbation-based importance sampling strategy to systematically explore the semantic space. Extensive experiments on four QA datasets demonstrate that our method achieves superior hallucination detection performance with significant efficiency gains. In low-budget scenarios, our approach requires about 50% fewer samples to achieve comparable detection performance to existing methods, while delivers an average AUROC improvement of 12.6% under the same sampling budget.
• 摘要：大型语言模型 (LLM) 在各种自然语言处理任务中取得了显着的成功，但它们仍然容易产生事实上不正确的输出，即幻觉。虽然最近的方法通过从法学硕士重复采样并量化生成的响应之间的语义不一致性，显示出幻觉检测的希望，但它们依赖于无法适应查询复杂性的固定采样预算，导致计算效率低下。我们提出了一种带有引导语义探索的语义熵自适应贝叶斯估计框架，该框架根据观察到的不确定性动态调整采样要求。我们的方法采用分层贝叶斯框架来对语义分布进行建模，从而通过基于方差的阈值动态控制采样迭代，一旦达到足够的确定性，阈值就会终止生成。我们还开发了一种基于扰动的重要性采样策略来系统地探索语义空间。对四个 QA 数据集的广泛实验表明，我们的方法实现了卓越的幻觉检测性能，并显着提高了效率。在低预算场景中，我们的方法需要大约 50% 的样本来实现与现有方法相当的检测性能，同时在相同的采样预算下实现平均 AUROC 提高 12.6%。

Title: When AI Shows Its Work, Is It Actually Working? Step-Level Evaluation Reveals Frontier Language Models Frequently Bypass Their Own Reasoning

• Authors: Abhinaba Basu, Pavan Chakraborty
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.22816
• Pdf URL: https://arxiv.org/pdf/2603.22816
• Copy Paste: [[2603.22816]] When AI Shows Its Work, Is It Actually Working? Step-Level Evaluation Reveals Frontier Language Models Frequently Bypass Their Own Reasoning(https://arxiv.org/abs/2603.22816)
• Keywords: language model, gpt
• Abstract: Language models increasingly "show their work" by writing step-by-step reasoning before answering. But are these reasoning steps genuinely used, or decorative narratives generated after the model has already decided? Consider: a medical AI writes "The patient's eosinophilia and livedo reticularis following catheterization suggest cholesterol embolization syndrome. Answer: B." If we remove the eosinophilia observation, does the diagnosis change? For most frontier models, the answer is no - the step was decorative. We introduce step-level evaluation: remove one reasoning sentence at a time and check whether the answer changes. This simple test requires only API access -- no model weights -- and costs approximately $1-2 per model per task. Testing 10 frontier models (GPT-5.4, Claude Opus, DeepSeek-V3.2, MiniMax-M2.5, Kimi-K2.5, and others) across sentiment, mathematics, topic classification, and medical QA (N=376-500 each), the majority produce decorative reasoning: removing any step changes the answer less than 17% of the time, while any single step alone recovers the answer. This holds even on math, where smaller models (0.8-8B) show genuine step dependence (55% necessity). Two models break the pattern: MiniMax-M2.5 on sentiment (37% necessity) and Kimi-K2.5 on topic classification (39%) - but both shortcut other tasks. Faithfulness is model-specific and task-specific. We also discover "output rigidity": on the same medical questions, Claude Opus writes 11 diagnostic steps while GPT-OSS-120B outputs a single token. Mechanistic analysis (attention patterns) confirms that CoT attention drops more in late layers for decorative tasks (33%) than faithful ones (20%). Implications: step-by-step explanations from frontier models are largely decorative, per-model per-domain evaluation is essential, and training objectives - not scale - determine whether reasoning is genuine.
• 摘要：语言模型越来越多地通过在回答之前编写逐步推理来“展示其工作”。但这些推理步骤是真正使用的，还是模型已经决定后产生的装饰性叙述？考虑一下：医疗人工智能写道“导管插入后患者出现嗜酸性粒细胞增多和网状青斑提示胆固醇栓塞综合征。答案：B。”如果我们去除嗜酸性粒细胞增多的观察结果，诊断会改变吗？对于大多数前沿车型来说，答案是否定的——台阶只是装饰性的。我们引入阶梯式评估：一次删除一个推理句子并检查答案是否发生变化。这个简单的测试只需要 API 访问——不需要模型权重——每个任务每个模型的成本约为 1-2 美元。在情感、数学、主题分类和医学 QA（每个 N=376-500）方面测试了 10 个前沿模型（GPT-5.4、Claude Opus、DeepSeek-V3.2、MiniMax-M2.5、Kimi-K2.5 等），大多数模型都会产生装饰性推理：删除任何步骤都会在不到 17% 的时间内改变答案，而单独任何一个步骤都可以恢复答案。即使在数学上也是如此，较小的模型（0.8-8B）显示出真正的步骤依赖性（55％的必要性）。两个模型打破了这一模式：MiniMax-M2.5 用于情感（37% 必要性）和 Kimi-K2.5 用于主题分类（39%）——但两者都缩短了其他任务。忠诚是特定于模型和特定于任务的。我们还发现了“输出刚性”：对于相同的医学问题，Claude Opus 编写了 11 个诊断步骤，而 GPT-OSS-120B 输出单个标记。机制分析（注意力模式）证实，装饰性任务的后期层 CoT 注意力（33%）比忠实任务（20%）下降更多。含义：前沿模型的逐步解释在很大程度上是装饰性的，每个模型每个领域的评估至关重要，训练目标（而不是规模）决定推理是否真实。

Title: RadTimeline: Timeline Summarization for Longitudinal Radiological Lung Findings

• Authors: Sitong Zhou, Meliha Yetisgen, Mari Ostendorf
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22820
• Pdf URL: https://arxiv.org/pdf/2603.22820
• Copy Paste: [[2603.22820]] RadTimeline: Timeline Summarization for Longitudinal Radiological Lung Findings(https://arxiv.org/abs/2603.22820)
• Keywords: llm, prompt
• Abstract: Tracking findings in longitudinal radiology reports is crucial for accurately identifying disease progression, and the time-consuming process would benefit from automatic summarization. This work introduces a structured summarization task, where we frame longitudinal report summarization as a timeline generation task, with dated findings organized in columns and temporally related findings grouped in rows. This structured summarization format enables straightforward comparison of findings across time and facilitates fact-checking against the associated reports. The timeline is generated using a 3-step LLM process of extracting findings, generating group names, and using the names to group the findings. To evaluate such systems, we create RadTimeline, a timeline dataset focused on tracking lung-related radiologic findings in chest-related imaging reports. Experiments on RadTimeline show tradeoffs of different-sized LLMs and prompting strategies. Our results highlight that group name generation as an intermediate step is critical for effective finding grouping. The best configuration has some irrelevant findings but very good recall, and grouping performance is comparable to human annotators.
• 摘要：跟踪纵向放射学报告中的发现对于准确识别疾病进展至关重要，并且自动总结将受益于耗时的过程。这项工作引入了一种结构化摘要任务，其中我们将纵向报告摘要构建为时间线生成任务，其中过时的发现按列组织，时间相关的发现按行分组。这种结构化摘要格式可以对不同时间的发现进行直接比较，并有助于根据相关报告进行事实核查。时间线是使用 3 步法学硕士流程生成的，即提取结果、生成组名称以及使用名称对结果进行分组。为了评估此类系统，我们创建了 RadTimeline，这是一个时间线数据集，专注于跟踪胸部相关成像报告中与肺部相关的放射学发现。 RadTimeline 上的实验显示了不同规模的 LLM 和激励策略的权衡。我们的结果强调，组名称生成作为中间步骤对于有效查找分组至关重要。最好的配置有一些不相关的发现，但召回率非常高，并且分组性能与人类注释者相当。

Title: Analysing LLM Persona Generation and Fairness Interpretation in Polarised Geopolitical Contexts

• Authors: Maida Aizaz, Quang Minh Nguyen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22837
• Pdf URL: https://arxiv.org/pdf/2603.22837
• Copy Paste: [[2603.22837]] Analysing LLM Persona Generation and Fairness Interpretation in Polarised Geopolitical Contexts(https://arxiv.org/abs/2603.22837)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) are increasingly utilised for social simulation and persona generation, necessitating an understanding of how they represent geopolitical identities. In this paper, we analyse personas generated for Palestinian and Israeli identities by five popular LLMs across 640 experimental conditions, varying context (war vs non-war) and assigned roles. We observe significant distributional patterns in the generated attributes: Palestinian profiles in war contexts are frequently associated with lower socioeconomic status and survival-oriented roles, whereas Israeli profiles predominantly retain middle-class status and specialised professional attributes. When prompted with explicit instructions to avoid harmful assumptions, models exhibit diverse distributional changes, e.g., marked increases in non-binary gender inferences or a convergence toward generic occupational roles (e.g., "student"), while the underlying socioeconomic distinctions often remain. Furthermore, analysis of reasoning traces reveals an interesting dynamics between model reasoning and generation: while rationales consistently mention fairness-related concepts, the final generated personas follow the aforementioned diverse distributional changes. These findings illustrate a picture of how models interpret geopolitical contexts, while suggesting that they process fairness and adjust in varied ways; there is no consistent, direct translation of fairness concepts into representative outcomes.
• 摘要：大型语言模型 (LLM) 越来越多地用于社会模拟和角色生成，因此需要了解它们如何表示地缘政治身份。在本文中，我们分析了五位受欢迎的法学硕士在 640 个实验条件、不同背景（战争与非战争）和指定角色中为巴勒斯坦和以色列身份生成的人物角色。我们观察到生成属性的显着分布模式：战争背景下的巴勒斯坦人形象经常与较低的社会经济地位和以生存为导向的角色相关，而以色列人的形象主要保留中产阶级地位和专门的职业属性。当明确指示避免有害假设时，模型表现出不同的分布变化，例如，非二元性别推论显着增加或向通用职业角色（例如“学生”）趋同，而潜在的社会经济差异通常仍然存在。此外，对推理痕迹的分析揭示了模型推理和生成之间有趣的动态：虽然基本原理一致提到与公平相关的概念，但最终生成的角色遵循上述不同的分布变化。这些发现说明了模型如何解释地缘政治背景，同时表明它们以不同的方式处理公平和调整；公平概念没有一致、直接地转化为有代表性的结果。

Title: EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction

• Authors: Yixuan Wang, Shiyu Ji, Yijun Liu, Qingfu Zhu, Wanxiang Che
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22910
• Pdf URL: https://arxiv.org/pdf/2603.22910
• Copy Paste: [[2603.22910]] EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction(https://arxiv.org/abs/2603.22910)
• Keywords: language model, llm
• Abstract: The increasing memory demand of the Key-Value (KV) cache poses a significant bottleneck for Large Language Models (LLMs) in long-context applications. Existing low-rank compression methods often rely on irreversible parameter transformations, sacrificing the flexibility to switch back to full-precision inference when memory is abundant. In this paper, we propose EchoKV, a flexible KV cache compression scheme that enables on-demand transitions between standard and compressed inference. Unlike traditional compression-decompression paradigms, EchoKV utilizes a lightweight network to reconstruct the residual KV components from a partial subset, leveraging intrinsic inter-layer and intra-layer similarities among attention heads. We further introduce a two-stage fine-tuning strategy that allows for rapid, low-cost training (e.g., ~1 A100 GPU-hour for a 7B model). Experimental results on LongBench and RULER demonstrate that EchoKV consistently outperforms existing methods across various compression ratios while maintaining high throughput for short-context scenarios.
• 摘要：键值 (KV) 缓存不断增长的内存需求给长上下文应用程序中的大型语言模型 (LLM) 带来了重大瓶颈。现有的低秩压缩方法通常依赖于不可逆的参数转换，牺牲了内存充足时切换回全精度推理的灵活性。在本文中，我们提出了 EchoKV，一种灵活的 KV 缓存压缩方案，可以实现标准推理和压缩推理之间的按需转换。与传统的压缩-解压范例不同，EchoKV 利用轻量级网络从部分子集中重建剩余 KV 组件，利用注意力头之间内在的层间和层内相似性。我们进一步引入了两阶段微调策略，可实现快速、低成本的训练（例如，7B 模型大约需要 1 A100 GPU 小时）。 LongBench 和 RULER 上的实验结果表明，EchoKV 在各种压缩比上始终优于现有方法，同时保持短上下文场景的高吞吐量。

Title: Multilingual KokoroChat: A Multi-LLM Ensemble Translation Method for Creating a Multilingual Counseling Dialogue Dataset

• Authors: Ryoma Suzuki, Zhiyang Qi, Michimasa Inaba
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22913
• Pdf URL: https://arxiv.org/pdf/2603.22913
• Copy Paste: [[2603.22913]] Multilingual KokoroChat: A Multi-LLM Ensemble Translation Method for Creating a Multilingual Counseling Dialogue Dataset(https://arxiv.org/abs/2603.22913)
• Keywords: llm, chat
• Abstract: To address the critical scarcity of high-quality, publicly available counseling dialogue datasets, we created Multilingual KokoroChat by translating KokoroChat, a large-scale manually authored Japanese counseling corpus, into both English and Chinese. A key challenge in this process is that the optimal model for translation varies by input, making it impossible for any single model to consistently guarantee the highest quality. In a sensitive domain like counseling, where the highest possible translation fidelity is essential, relying on a single LLM is therefore insufficient. To overcome this challenge, we developed and employed a novel multi-LLM ensemble method. Our approach first generates diverse hypotheses from multiple distinct LLMs. A single LLM then produces a high-quality translation based on an analysis of the respective strengths and weaknesses of all presented hypotheses. The quality of ``Multilingual KokoroChat'' was rigorously validated through human preference studies. These evaluations confirmed that the translations produced by our ensemble method were preferred from any individual state-of-the-art LLM. This strong preference confirms the superior quality of our method's outputs. The Multilingual KokoroChat is available at this https URL.
• 摘要：为了解决高质量、公开的咨询对话数据集的严重稀缺问题，我们将大规模手动编写的日语咨询语料库 KokoroChat 翻译成英文和中文，创建了多语言 KokoroChat。此过程中的一个关键挑战是，最佳翻译模型因输入而异，因此任何单一模型都不可能始终保证最高质量。在像咨询这样的敏感领域，尽可能高的翻译保真度至关重要，因此依靠单一的法学硕士是不够的。为了克服这一挑战，我们开发并采用了一种新颖的多法学硕士集成方法。我们的方法首先从多个不同的法学硕士生成不同的假设。然后，一名法学硕士根据对所有提出的假设各自的优点和缺点的分析，生成高质量的翻译。 “多语言 KokoroChat”的质量经过人类偏好研究的严格验证。这些评估证实，我们的集成方法生成的翻译是任何个人最先进的法学硕士的首选。这种强烈的偏好证实了我们方法输出的卓越质量。多语言 KokoroChat 可通过此 https URL 获取。

Title: Quality Over Clicks: Intrinsic Quality-Driven Iterative Reinforcement Learning for Cold-Start E-Commerce Query Suggestion

• Authors: Qi Sun, Kejun Xiao, Huaipeng Zhao, Tao Luo, Xiaoyi Zeng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22922
• Pdf URL: https://arxiv.org/pdf/2603.22922
• Copy Paste: [[2603.22922]] Quality Over Clicks: Intrinsic Quality-Driven Iterative Reinforcement Learning for Cold-Start E-Commerce Query Suggestion(https://arxiv.org/abs/2603.22922)
• Keywords: language model, chat
• Abstract: Existing dialogue systems rely on Query Suggestion (QS) to enhance user engagement. Recent efforts typically employ large language models with Click-Through Rate (CTR) model, yet fail in cold-start scenarios due to their heavy reliance on abundant online click data for effective CTR model training. To bridge this gap, we propose Cold-EQS, an iterative reinforcement learning framework for Cold-Start E-commerce Query Suggestion (EQS). Specifically, we leverage answerability, factuality, and information gain as reward to continuously optimize the quality of suggested queries. To continuously optimize our QS model, we estimate uncertainty for grouped candidate suggested queries to select hard and ambiguous samples from online user queries lacking click signals. In addition, we provide an EQS-Benchmark comprising 16,949 online user queries for offline training and evaluation. Extensive offline and online experiments consistently demonstrate a strong positive correlation between online and offline effectiveness. Both offline and online experimental results demonstrate the superiority of our Cold-EQS, achieving a significant +6.81% improvement in online chatUV.
• 摘要：现有的对话系统依靠查询建议（QS）来增强用户参与度。最近的工作通常采用具有点击率（CTR）模型的大型语言模型，但由于严重依赖丰富的在线点击数据来进行有效的 CTR 模型训练，因此在冷启动场景中失败。为了弥补这一差距，我们提出了 Cold-EQS，这是一种用于冷启动电子商务查询建议（EQS）的迭代强化学习框架。具体来说，我们利用可回答性、真实性和信息增益作为奖励，不断优化建议查询的质量。为了不断优化我们的 QS 模型，我们估计分组候选建议查询的不确定性，以从缺乏点击信号的在线用户查询中选择困难且模糊的样本。此外，我们还提供了包含 16,949 个在线用户查询的 EQS-Benchmark，用于离线培训和评估。大量的线下和线上实验一致证明线上和线下效果之间存在很强的正相关性。离线和在线实验结果都证明了我们的 Cold-EQS 的优越性，在线 chatUV 显着提高了 +6.81%。

Title: Set-Valued Prediction for Large Language Models with Feasibility-Aware Coverage Guarantees

• Authors: Ye Li, Anqi Hu, Yuanchang Ye, Shiyan Tong, Zhiyuan Wang, Bo Fu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.22966
• Pdf URL: https://arxiv.org/pdf/2603.22966
• Copy Paste: [[2603.22966]] Set-Valued Prediction for Large Language Models with Feasibility-Aware Coverage Guarantees(https://arxiv.org/abs/2603.22966)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) inherently operate over a large generation space, yet conventional usage typically reports the most likely generation (MLG) as a point prediction, which underestimates the model's capability: although the top-ranked response can be incorrect, valid answers may still exist within the broader output space and can potentially be discovered through repeated sampling. This observation motivates moving from point prediction to set-valued prediction, where the model produces a set of candidate responses rather than a single MLG. In this paper, we propose a principled framework for set-valued prediction, which provides feasibility-aware coverage guarantees. We show that, given the finite-sampling nature of LLM generation, coverage is not always achievable: even with multiple samplings, LLMs may fail to yield an acceptable response for certain questions within the sampled candidate set. To address this, we establish a minimum achievable risk level (MRL), below which statistical coverage guarantees cannot be satisfied. Building on this insight, we then develop a data-driven calibration procedure that constructs prediction sets from sampled responses by estimating a rigorous threshold, ensuring that the resulting set contains a correct answer with a desired probability whenever the target risk level is feasible. Extensive experiments on six language generation tasks with five LLMs demonstrate both the statistical validity and the predictive efficiency of our framework.
• 摘要：大型语言模型 (LLM) 本质上在较大的生成空间上运行，但传统用法通常将最可能的生成 (MLG) 报告为点预测，这低估了模型的能力：尽管排名靠前的响应可能不正确，但有效答案可能仍然存在于更广泛的输出空间中，并且有可能通过重复采样发现。这一观察结果推动了从点预测转向集值预测，其中模型产生一组候选响应而不是单个 MLG。在本文中，我们提出了一个集值预测的原则框架，它提供了可行性感知的覆盖保证。我们表明，鉴于 LLM 生成的有限抽样性质，覆盖范围并不总是可以实现：即使进行多次抽样，LLM 也可能无法对抽样候选集中的某些问题产生可接受的答案。为了解决这个问题，我们建立了最低可实现风险水平（MRL），低于该水平就无法满足统计覆盖率保证。基于这一见解，我们然后开发了一个数据驱动的校准程序，通过估计严格的阈值，从采样响应构建预测集，确保只要目标风险水平可行，结果集就包含具有所需概率的正确答案。五个法学硕士对六种语言生成任务的广泛实验证明了我们框架的统计有效性和预测效率。

Title: Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation

• Authors: Nils A. Herrmann, Tobias Eder, Jingyi He, Georg Groh
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2603.22985
• Pdf URL: https://arxiv.org/pdf/2603.22985
• Copy Paste: [[2603.22985]] Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation(https://arxiv.org/abs/2603.22985)
• Keywords: language model
• Abstract: Current multimodal toxicity benchmarks typically use a single binary hatefulness label. This coarse approach conflates two fundamentally different characteristics of expression: tone and content. Drawing on communication science theory, we introduce a fine-grained annotation scheme that distinguishes two separable dimensions: incivility (rude or dismissive tone) and intolerance (content that attacks pluralism and targets groups or identities) and apply it to 2,030 memes from the Hateful Memes dataset. We evaluate different vision-language models under coarse-label training, transfer learning across label schemes and a joint learning approach that combines the coarse hatefulness label with our fine-grained annotations. Our results show that fine-grained annotations complement existing coarse labels and, when used jointly, improve overall model performance. Moreover, models trained with the fine-grained scheme exhibit more balanced moderation-relevant error profiles and are less prone to under-detection of harmful content than models trained on hatefulness labels alone (FNR-FPR, the difference between false negative and false positive rates: 0.74 to 0.42 for LLaVA-1.6-Mistral-7B; 0.54 to 0.28 for Qwen2.5-VL-7B). This work contributes to data-centric approaches in content moderation by improving the reliability and accuracy of moderation systems through enhanced data quality. Overall, combining both coarse and fine-grained labels provides a practical route to more reliable multimodal moderation.
• 摘要：当前的多模式毒性基准通常使用单个二元仇恨标签。这种粗略的方法融合了两种根本不同的表达特征：语气和内容。借鉴传播科学理论，我们引入了一种细粒度的注释方案，区分两个可分离的维度：不文明（粗鲁或轻蔑的语气）和不宽容（攻击多元化和针对群体或身份的内容），并将其应用于 Hateful Memes 数据集中的 2,030 个 meme。我们在粗标签训练、跨标签方案的迁移学习以及将粗标签与细粒度注释相结合的联合学习方法下评估不同的视觉语言模型。我们的结果表明，细粒度注释补充了现有的粗标签，并且联合使用时可以提高整体模型性能。此外，与仅在仇恨标签上训练的模型相比，使用细粒度方案训练的模型表现出更平衡的与审核相关的错误概况，并且更不容易检测到有害内容（FNR-FPR，假阴性率和假阳性率之间的差异：LLaVA-1.6-Mistral-7B为0.74至0.42；Qwen2.5-VL-7B为0.54至0.28）。这项工作通过提高数据质量来提高审核系统的可靠性和准确性，从而有助于在内容审核中采用以数据为中心的方法。总的来说，粗粒度和细粒度标签的结合提供了一条实现更可靠的多模式调节的实用途径。

Title: PaperVoyager : Building Interactive Web with Visual Language Models

• Authors: Dasen Dai, Biao Wu, Meng Fang, Wenhao Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.22999
• Pdf URL: https://arxiv.org/pdf/2603.22999
• Copy Paste: [[2603.22999]] PaperVoyager : Building Interactive Web with Visual Language Models(https://arxiv.org/abs/2603.22999)
• Keywords: language model, agent
• Abstract: Recent advances in visual language models have enabled autonomous agents for complex reasoning, tool use, and document understanding. However, existing document agents mainly transform papers into static artifacts such as summaries, webpages, or slides, which are insufficient for technical papers involving dynamic mechanisms and state transitions. In this work, we propose a Paper-to-Interactive-System Agent that converts research papers into executable interactive web systems. Given a PDF paper, the agent performs end-to-end processing without human intervention, including paper understanding, system modeling, and interactive webpage synthesis, enabling users to manipulate inputs and observe dynamic behaviors. To evaluate this task, we introduce a benchmark of 19 research papers paired with expert-built interactive systems as ground truth. We further propose PaperVoyager, a structured generation framework that explicitly models mechanisms and interaction logic during synthesis. Experiments show that PaperVoyager significantly improves the quality of generated interactive systems, offering a new paradigm for interactive scientific paper understanding.
• 摘要：视觉语言模型的最新进展使自主代理能够进行复杂的推理、工具使用和文档理解。然而，现有的文档代理主要将论文转换为摘要、网页或幻灯片等静态工件，这对于涉及动态机制和状态转换的技术论文来说是不够的。在这项工作中，我们提出了一种纸张到交互式系统代理，可将研究论文转换为可执行的交互式网络系统。给定 PDF 论文，代理无需人工干预即可执行端到端处理，包括论文理解、系统建模和交互式网页合成，使用户能够操纵输入并观察动态行为。为了评估这项任务，我们引入了 19 篇研究论文的基准，并搭配专家构建的交互系统作为基本事实。我们进一步提出了 PaperVoyager，一种结构化生成框架，可在综合过程中显式地建模机制和交互逻辑。实验表明，PaperVoyager 显着提高了生成的交互式系统的质量，为交互式科学论文理解提供了新的范例。

Title: Knowledge Access Beats Model Size: Memory Augmented Routing for Persistent AI Agents

• Authors: Xunzhuo Liu, Bowei He, Xue Liu, Andy Luo, Haichen Zhang, Huamin Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.23013
• Pdf URL: https://arxiv.org/pdf/2603.23013
• Copy Paste: [[2603.23013]] Knowledge Access Beats Model Size: Memory Augmented Routing for Persistent AI Agents(https://arxiv.org/abs/2603.23013)
• Keywords: hallucination, agent
• Abstract: Production AI agents frequently receive user-specific queries that are highly repetitive, with up to 47\% being semantically similar to prior interactions, yet each query is typically processed with the same computational cost. We argue that this redundancy can be exploited through conversational memory, transforming repetition from a cost burden into an efficiency advantage. We propose a memory-augmented inference framework in which a lightweight 8B-parameter model leverages retrieved conversational context to answer all queries via a low-cost inference path. Without any additional training or labeled data, this approach achieves 30.5\% F1, recovering 69\% of the performance of a full-context 235B model while reducing effective cost by 96\%. Notably, a 235B model without memory (13.7\% F1) underperforms even the standalone 8B model (15.4\% F1), indicating that for user-specific queries, access to relevant knowledge outweighs model scale. We further analyze the role of routing and confidence. At practical confidence thresholds, routing alone already directs 96\% of queries to the small model, but yields poor accuracy (13.0\% F1) due to confident hallucinations. Memory does not substantially alter routing decisions; instead, it improves correctness by grounding responses in retrieved user-specific information. As conversational memory accumulates over time, coverage of recurring topics increases, further narrowing the performance gap. We evaluate on 152 LoCoMo questions (Qwen3-8B/235B) and 500 LongMemEval questions. Incorporating hybrid retrieval (BM25 + cosine similarity) improves performance by an additional +7.7 F1, demonstrating that retrieval quality directly enhances end-to-end system performance. Overall, our results highlight that memory, rather than model size, is the primary driver of accuracy and efficiency in persistent AI agents.
• 摘要：生产 AI 代理经常收到高度重复的特定于用户的查询，其中高达 47% 的查询在语义上与之前的交互相似，但每个查询通常都以相同的计算成本进行处理。我们认为，这种冗余可以通过会话记忆来利用，将重复从成本负担转变为效率优势。我们提出了一种内存增强推理框架，其中轻量级 8B 参数模型利用检索到的会话上下文通过低成本推理路径回答所有查询。在没有任何额外的训练或标记数据的情况下，该方法实现了 30.5% F1，恢复了全上下文 235B 模型的 69% 性能，同时将有效成本降低了 96%。值得注意的是，没有内存的 235B 模型 (13.7\% F1) 甚至比独立的 8B 模型 (15.4\% F1) 表现还要差，这表明对于特定于用户的查询，对相关知识的访问超过了模型规模。我们进一步分析路由和置信度的作用。在实际的置信度阈值下，单独的路由已经将 96\% 的查询定向到小模型，但由于置信幻觉，准确度较差 (13.0\% F1)。内存不会显着改变路由决策；相反，它通过将响应基于检索到的用户特定信息来提高正确性。随着会话记忆随着时间的推移而积累，重复出现的主题的覆盖范围也会增加，从而进一步缩小性能差距。我们评估了 152 个 LoCoMo 问题 (Qwen3-8B/235B) 和 500 个 LongMemEval 问题。结合混合检索（BM25 + 余弦相似度）将性能额外提高了 +7.7 F1，表明检索质量直接增强了端到端系统性能。总体而言，我们的结果强调，内存（而不是模型大小）是持久人工智能代理准确性和效率的主要驱动因素。

Title: Parametric Knowledge and Retrieval Behavior in RAG Fine-Tuning for Electronic Design Automation

• Authors: Julian Oestreich, Maximilian Bley, Frank Binder, Lydia Müller, Maksym Sydorenko, André Alcalde
• Subjects: cs.CL, cs.AI, cs.CE
• Abstract URL: https://arxiv.org/abs/2603.23047
• Pdf URL: https://arxiv.org/pdf/2603.23047
• Copy Paste: [[2603.23047]] Parametric Knowledge and Retrieval Behavior in RAG Fine-Tuning for Electronic Design Automation(https://arxiv.org/abs/2603.23047)
• Keywords: prompt, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) fine-tuning has shown substantial improvements over vanilla RAG, yet most studies target document question answering and often rely on standard NLP metrics that can obscure factual differences. We evaluate RAG fine-tuning for long-form text generation in electronic design automation, adapting a 7B model under five context augmentation strategies with varying retrieval conditions. We introduce TriFEX, a human-validated, triple-based evaluation pipeline that attributes generated claims to their origin-user query, context and reference-and propose Parametric Knowledge Precision (PKP), which isolates internalized knowledge by filtering out claims leaked in the prompt. We show that ROUGE and BERTScore fail to detect factual differences that our triple-based evaluation reveals. Additionally, we demonstrate that an existing metric for knowledge internalization is retrieva-sensitive, with about 75% of its cross-condition variance driven by changes in the rate at which internal knowledge is expressed (PR), rather than by changes in its actual correctness (PKP). The fine-tuned 7B variants outperform a 72B baseline on most metrics, further showing generalization across conditions and on a related benchmark. These results underscore the limitations of available metrics in RAG evaluation and show that smaller models could be reasonably well adapted to specialized tasks for cost-efficient, on-premises deployment.
• 摘要：检索增强生成 (RAG) 微调已显示出相对于普通 RAG 的显着改进，但大多数研究都针对文档问答，并且通常依赖于可能掩盖事实差异的标准 NLP 指标。我们评估了电子设计自动化中长文本生成的 RAG 微调，在不同检索条件下的五种上下文增强策略下调整 7B 模型。我们引入了 TriFEX，这是一种经过人工验证、基于三元组的评估管道，它将生成的声明归因于其原始用户查询、上下文和参考，并提出了参数知识精度 (PKP)，它通过过滤掉提示中泄漏的声明来隔离内化知识。我们表明，ROUGE 和 BERTScore 未能检测到我们基于三重的评估所揭示的事实差异。此外，我们证明现有的知识内化指标是检索敏感的，其大约 75% 的跨条件方差是由内部知识表达率 (PR) 的变化驱动的，而不是由其实际正确性 (PKP) 的变化驱动的。经过微调的 7B 变体在大多数指标上都优于 72B 基线，进一步显示了跨条件和相关基准的概括性。这些结果强调了 RAG 评估中可用指标的局限性，并表明较小的模型可以相当好地适应专门任务，以实现经济高效的本地部署。

Title: AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing

• Authors: Sarubi Thillainathan, Ji-Ung Lee, Michael Sullivan, Alexander Koller
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.23069
• Pdf URL: https://arxiv.org/pdf/2603.23069
• Copy Paste: [[2603.23069]] AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing(https://arxiv.org/abs/2603.23069)
• Keywords: gpt
• Abstract: The task of authorship style transfer involves rewriting text in the style of a target author while preserving the meaning of the original text. Existing style transfer methods train a single model on large corpora to model all target styles at once: this high-cost approach offers limited flexibility for target-specific adaptation, and often sacrifices meaning preservation for style transfer. In this paper, we propose AuthorMix: a lightweight, modular, and interpretable style transfer framework. We train individual, style-specific LoRA adapters on a small set of high-resource authors, allowing the rapid training of specialized adaptation models for each new target via learned, layer-wise adapter mixing, using only a handful of target style training examples. AuthorMix outperforms existing, SoTA style-transfer baselines -- as well as GPT-5.1 -- for low-resource targets, achieving the highest overall score and substantially improving meaning preservation.
• 摘要：作者风格迁移的任务涉及以目标作者的风格重写文本，同时保留原始文本的含义。现有的风格迁移方法在大型语料库上训练单个模型，以同时对所有目标风格进行建模：这种高成本方法为特定目标的适应提供了有限的灵活性，并且经常牺牲风格迁移的意义保留。在本文中，我们提出了 AuthorMix：一个轻量级、模块化、可解释的风格转换框架。我们对一小部分高资源作者训练单独的、特定于风格的 LoRA 适配器，仅使用少量目标风格训练示例，通过学习的分层适配器混合，快速训练每个新目标的专门适应模型。对于低资源目标，AuthorMix 的性能优于现有的 SoTA 风格迁移基线以及 GPT-5.1，实现了最高的总体得分并显着改善了意义保存。

Title: When Language Models Lose Their Mind: The Consequences of Brain Misalignment

• Authors: Gabriele Merlin, Mariya Toneva
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.23091
• Pdf URL: https://arxiv.org/pdf/2603.23091
• Copy Paste: [[2603.23091]] When Language Models Lose Their Mind: The Consequences of Brain Misalignment(https://arxiv.org/abs/2603.23091)
• Keywords: language model, llm
• Abstract: While brain-aligned large language models (LLMs) have garnered attention for their potential as cognitive models and for potential for enhanced safety and trustworthiness in AI, the role of this brain alignment for linguistic competence remains uncertain. In this work, we investigate the functional implications of brain alignment by introducing brain-misaligned models--LLMs intentionally trained to predict brain activity poorly while maintaining high language modeling performance. We evaluate these models on over 200 downstream tasks encompassing diverse linguistic domains, including semantics, syntax, discourse, reasoning, and morphology. By comparing brain-misaligned models with well-matched brain-aligned counterparts, we isolate the specific impact of brain alignment on language understanding. Our experiments reveal that brain misalignment substantially impairs downstream performance, highlighting the critical role of brain alignment in achieving robust linguistic competence. These findings underscore the importance of brain alignment in LLMs and offer novel insights into the relationship between neural representations and linguistic processing.
• 摘要：虽然大脑对齐大语言模型（LLM）因其作为认知模型的潜力以及增强人工智能安全性和可信度的潜力而受到关注，但这种大脑对齐对语言能力的作用仍然不确定。在这项工作中，我们通过引入大脑错位模型来研究大脑对齐的功能含义——法学硕士经过有意训练，可以在保持较高语言建模性能的同时预测大脑活动不佳。我们在涵盖不同语言领域的 200 多个下游任务上评估这些模型，包括语义、语法、话语、推理和形态。通过将大脑错位模型与匹配良好的大脑对齐模型进行比较，我们分离出大脑对齐对语言理解的具体影响。我们的实验表明，大脑错位会严重损害下游表现，强调大脑对齐在实现强大的语言能力方面的关键作用。这些发现强调了法学硕士中大脑排列的重要性，并为神经表征和语言处理之间的关系提供了新的见解。

Title: HGNet: Scalable Foundation Model for Automated Knowledge Graph Generation from Scientific Literature

• Authors: Devvrat Joshi, Islem Rekik
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.23136
• Pdf URL: https://arxiv.org/pdf/2603.23136
• Copy Paste: [[2603.23136]] HGNet: Scalable Foundation Model for Automated Knowledge Graph Generation from Scientific Literature(https://arxiv.org/abs/2603.23136)
• Keywords: language model, llm
• Abstract: Automated knowledge graph (KG) construction is essential for navigating the rapidly expanding body of scientific literature. However, existing approaches struggle to recognize long multi-word entities, often fail to generalize across domains, and typically overlook the hierarchical nature of scientific knowledge. While general-purpose large language models (LLMs) offer adaptability, they are computationally expensive and yield inconsistent accuracy on specialized tasks. As a result, current KGs are shallow and inconsistent, limiting their utility for exploration and synthesis. We propose a two-stage framework for scalable, zero-shot scientific KG construction. The first stage, Z-NERD, introduces (i) Orthogonal Semantic Decomposition (OSD), which promotes domain-agnostic entity recognition by isolating semantic "turns" in text, and (ii) a Multi-Scale TCQK attention mechanism that captures coherent multi-word entities through n-gram-aware attention heads. The second stage, HGNet, performs relation extraction with hierarchy-aware message passing, explicitly modeling parent, child, and peer relations. To enforce global consistency, we introduce two complementary objectives: a Differentiable Hierarchy Loss to discourage cycles and shortcut edges, and a Continuum Abstraction Field (CAF) Loss that embeds abstraction levels along a learnable axis in Euclidean space. This is the first approach to formalize hierarchical abstraction as a continuous property within standard Euclidean embeddings, offering a simpler alternative to hyperbolic methods. We release SPHERE (this https URL), a multi-domain benchmark for hierarchical relation extraction. Our framework establishes a new state of the art on SciERC, SciER, and SPHERE, improving NER by 8.08% and RE by 5.99% on out-of-distribution tests. In zero-shot settings, gains reach 10.76% for NER and 26.2% for RE.
• 摘要：自动化知识图 (KG) 构建对于浏览快速扩展的科学文献至关重要。然而，现有的方法很难识别长的多词实体，通常无法跨领域泛化，并且通常忽视科学知识的层次性质。虽然通用大语言模型 (LLM) 提供了适应性，但它​​们的计算成本很高，并且在专门任务上产生的准确性不一致。因此，当前的知识图谱很浅且不一致，限制了它们在探索和综合方面的效用。我们提出了一个可扩展、零样本科学知识图谱构建的两阶段框架。第一阶段 Z-NERD 引入了 (i) 正交语义分解 (OSD)，它通过隔离文本中的语义“转向”来促进与领域无关的实体识别，以及 (ii) 多尺度 TCQK 注意力机制，通过 n 元语法感知注意力头捕获连贯的多词实体。第二阶段 HGNet 通过层次结构感知消息传递执行关系提取，显式建模父、子和对等关系。为了加强全局一致性，我们引入了两个互补的目标：可微分层次损失（Differentiable Hierarchy Loss）以阻止循环和快捷边缘，以及连续抽象域（CAF）损失（Continuum Abstraction Field (CAF) Loss），其沿着欧几里德空间中的可学习轴嵌入抽象级别。这是第一种将层次抽象形式化为标准欧几里德嵌入中的连续属性的方法，为双曲线方法提供了更简单的替代方案。我们发布了 SPHERE（此 https URL），这是一个用于层次关系提取的多域基准。我们的框架在 SciERC、SciER 和 SPHERE 上建立了新的技术水平，在分布外测试中将 NER 提高了 8.08%，将 RE 提高了 5.99%。在零样本设置中，NER 的增益达到 10.76%，RE 的增益达到 26.2%。

Title: Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy

• Authors: Shushanta Pudasaini, Luis Miralles-Pechuán, David Lillis, Marisa Llorens Salvador
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.23146
• Pdf URL: https://arxiv.org/pdf/2603.23146
• Copy Paste: [[2603.23146]] Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy(https://arxiv.org/abs/2603.23146)
• Keywords: language model, llm
• Abstract: The widespread adoption of Large Language Models (LLMs) has made the detection of AI-Generated text a pressing and complex challenge. Although many detection systems report high benchmark accuracy, their reliability in real-world settings remains uncertain, and their interpretability is often unexplored. In this work, we investigate whether contemporary detectors genuinely identify machine authorship or merely exploit dataset-specific artefacts. We propose an interpretable detection framework that integrates linguistic feature engineering, machine learning, and explainable AI techniques. When evaluated on two prominent benchmark corpora, namely PAN CLEF 2025 and COLING 2025, our model trained on 30 linguistic features achieves leaderboard-competitive performance, attaining an F1 score of 0.9734. However, systematic cross-domain and cross-generator evaluation reveals substantial generalisation failure: classifiers that excel in-domain degrade significantly under distribution shift. Using SHAP- based explanations, we show that the most influential features differ markedly between datasets, indicating that detectors often rely on dataset-specific stylistic cues rather than stable signals of machine authorship. Further investigation with in-depth error analysis exposes a fundamental tension in linguistic-feature-based AI text detection: the features that are most discriminative on in-domain data are also the features most susceptible to domain shift, formatting variation, and text-length effects. We believe that this knowledge helps build AI detectors that are robust across different settings. To support replication and practical use, we release an open-source Python package that returns both predictions and instance-level explanations for individual texts.
• 摘要：大型语言模型 (LLM) 的广泛采用使得人工智能生成文本的检测成为一项紧迫而复杂的挑战。尽管许多检测系统报告了较高的基准精度，但它们在现实环境中的可靠性仍然不确定，并且它们的可解释性通常未被探索。在这项工作中，我们研究当代检测器是否真正识别机器作者或仅仅利用数据集特定的人工制品。我们提出了一个可解释的检测框架，集成了语言特征工程、机器学习和可解释的人工智能技术。当在两个著名的基准语料库（即 PAN CLEF 2025 和 COLING 2025）上进行评估时，我们在 30 个语言特征上训练的模型实现了排行榜竞争性能，获得了 0.9734 的 F1 分数。然而，系统的跨域和跨生成器评估揭示了严重的泛化失败：在域内表现出色的分类器在分布偏移下显着退化。使用基于 SHAP 的解释，我们发现数据集之间最有影响力的特征存在显着差异，这表明检测器通常依赖于数据集特定的风格线索，而不是机器作者的稳定信号。通过深入的错误分析进行的进一步调查揭示了基于语言特征的人工智能文本检测中的根本张力：对域内数据最具辨别力的特征也是最容易受到域转移、格式变化和文本长度影响的特征。我们相信这些知识有助于构建在不同设置下都具有鲁棒性的人工智能探测器。为了支持复制和实际使用，我们发布了一个开源 Python 包，它返回单个文本的预测和实例级解释。

Title: From Synthetic to Native: Benchmarking Multilingual Intent Classification in Logistics Customer Service

• Authors: Haoyu He, Jinyu Zhuang, Haoran Chu, Shuhang Yu, J, T AI Group, Hao Wang, Kunpeng Han
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.23172
• Pdf URL: https://arxiv.org/pdf/2603.23172
• Copy Paste: [[2603.23172]] From Synthetic to Native: Benchmarking Multilingual Intent Classification in Logistics Customer Service(https://arxiv.org/abs/2603.23172)
• Keywords: language model, llm
• Abstract: Multilingual intent classification is central to customer-service systems on global logistics platforms, where models must process noisy user queries across languages and hierarchical label spaces. Yet most existing multilingual benchmarks rely on machine-translated text, which is typically cleaner and more standardized than native customer requests and can therefore overestimate real-world robustness. We present a public benchmark for hierarchical multilingual intent classification constructed from real logistics customer-service logs. The dataset contains approximately 30K de-identified, stand-alone user queries curated from 600K historical records through filtering, LLM-assisted quality control, and human verification, and is organized into a two-level taxonomy with 13 parent and 17 leaf intents. English, Spanish, and Arabic are included as seen languages, while Indonesian, Chinese, and additional test-only languages support zero-shot evaluation. To directly measure the gap between synthetic and real evaluation, we provide paired native and machine-translated test sets and benchmark multilingual encoders, embedding models, and small language models under flat and hierarchical protocols. Results show that translated test sets substantially overestimate performance on noisy native queries, especially for long-tail intents and cross-lingual transfer, underscoring the need for more realistic multilingual intent benchmarks.
• 摘要：多语言意图分类是全球物流平台上客户服务系统的核心，其中模型必须跨语言和分层标签空间处理嘈杂的用户查询。然而，大多数现有的多语言基准都依赖于机器翻译的文本，该文本通常比本地客户的请求更清晰、更标准化，因此可能会高估现实世界的稳健性。我们提出了一个根据真实物流客户服务日志构建的分层多语言意图分类的公共基准。该数据集包含大约 30K 个去识别化的独立用户查询，这些查询是通过过滤、LLM 辅助质量控制和人工验证从 600K 历史记录中整理出来的，并被组织成具有 13 个父意图和 17 个叶意图的两级分类法。英语、西班牙语和阿拉伯语被列为可见语言，而印度尼西亚语、中文和其他仅测试语言支持零样本评估。为了直接衡量合成评估和真实评估之间的差距，我们提供了配对的本机和机器翻译测试集，并在平面和分层协议下对多语言编码器、嵌入模型和小语言模型进行基准测试。结果表明，翻译的测试集大大高估了嘈杂的本机查询的性能，特别是对于长尾意图和跨语言传输，强调了对更现实的多语言意图基准的需要。

Title: ImplicitRM: Unbiased Reward Modeling from Implicit Preference Data for LLM alignment

• Authors: Hao Wang, Haocheng Yang, Licheng Pan, Lei Shen, Xiaoxi Li, Yinuo Wang, Zhichao Chen, Yuan Lu, Haoxuan Li, Zhouchen Lin
• Subjects: cs.CL, cs.AI, stat.AP
• Abstract URL: https://arxiv.org/abs/2603.23184
• Pdf URL: https://arxiv.org/pdf/2603.23184
• Copy Paste: [[2603.23184]] ImplicitRM: Unbiased Reward Modeling from Implicit Preference Data for LLM alignment(https://arxiv.org/abs/2603.23184)
• Keywords: language model, llm
• Abstract: Reward modeling represents a long-standing challenge in reinforcement learning from human feedback (RLHF) for aligning language models. Current reward modeling is heavily contingent upon experimental feedback data with high collection costs. In this work, we study \textit{implicit reward modeling} -- learning reward models from implicit human feedback (e.g., clicks and copies) -- as a cost-effective alternative. We identify two fundamental challenges in implicit reward modeling: (1) Implicit preference data lacks definitive negative samples, which makes standard positive-negative classification methods inapplicable; (2) Implicit preference data suffers from user preference bias, where different responses have different propensities to elicit user feedback actions, which exacerbates the difficulty of distinguishing definitive negative samples. To address these challenges, we propose ImplicitRM, which aims to learn unbiased reward models from implicit preference data. ImplicitRM stratifies training samples into four latent groups via a stratification model. Building on this, it derives a learning objective through likelihood maximization, which we prove is theoretically unbiased, effectively resolving both challenges. Experiments demonstrate that ImplicitRM learns accurate reward models across implicit preference datasets. Code is available on our project website.
• 摘要：奖励建模是根据人类反馈（RLHF）进行强化学习以调整语言模型的长期挑战。当前的奖励模型在很大程度上取决于收集成本较高的实验反馈数据。在这项工作中，我们研究 \textit{隐式奖励建模}——从隐式人类反馈（例如点击和复制）中学习奖励模型——作为一种经济高效的替代方案。我们发现隐性奖励模型存在两个基本挑战：（1）隐性偏好数据缺乏明确的负样本，这使得标准的正负分类方法不适用； （2）隐性偏好数据存在用户偏好偏差，不同的响应有不同的引发用户反馈行为的倾向，这加剧了区分明确的负样本的难度。为了应对这些挑战，我们提出了 ImplicitRM，旨在从隐式偏好数据中学习无偏奖励模型。 ImplicitRM 通过分层模型将训练样本分层为四个潜在组。在此基础上，它通过似然最大化得出学习目标，我们证明这在理论上是无偏见的，有效解决了这两个挑战。实验表明，ImplicitRM 可以跨隐式偏好数据集学习准确的奖励模型。代码可在我们的项目网站上找到。

Title: Decoding AI Authorship: Can LLMs Truly Mimic Human Style Across Literature and Politics?

• Authors: Nasser A Alsadhan
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.23219
• Pdf URL: https://arxiv.org/pdf/2603.23219
• Copy Paste: [[2603.23219]] Decoding AI Authorship: Can LLMs Truly Mimic Human Style Across Literature and Politics?(https://arxiv.org/abs/2603.23219)
• Keywords: language model, gpt, llm, prompt
• Abstract: Amidst the rising capabilities of generative AI to mimic specific human styles, this study investigates the ability of state-of-the-art large language models (LLMs), including GPT-4o, Gemini 1.5 Pro, and Claude Sonnet 3.5, to emulate the authorial signatures of prominent literary and political figures: Walt Whitman, William Wordsworth, Donald Trump, and Barack Obama. Utilizing a zero-shot prompting framework with strict thematic alignment, we generated synthetic corpora evaluated through a complementary framework combining transformer-based classification (BERT) and interpretable machine learning (XGBoost). Our methodology integrates Linguistic Inquiry and Word Count (LIWC) markers, perplexity, and readability indices to assess the divergence between AI-generated and human-authored text. Results demonstrate that AI-generated mimicry remains highly detectable, with XGBoost models trained on a restricted set of eight stylometric features achieving accuracy comparable to high-dimensional neural classifiers. Feature importance analyses identify perplexity as the primary discriminative metric, revealing a significant divergence in the stochastic regularity of AI outputs compared to the higher variability of human writing. While LLMs exhibit distributional convergence with human authors on low-dimensional heuristic features, such as syntactic complexity and readability, they do not yet fully replicate the nuanced affective density and stylistic variance inherent in the human-authored corpus. By isolating the specific statistical gaps in current generative mimicry, this study provides a comprehensive benchmark for LLM stylistic behavior and offers critical insights for authorship attribution in the digital humanities and social media.
• 摘要：随着生成式人工智能模仿特定​​人类风格的能力不断增强，本研究调查了最先进的大型语言模型 (LLM)（包括 GPT-4o、Gemini 1.5 Pro 和 Claude Sonnet 3.5）模仿著名文学和政治人物（沃尔特·惠特曼、威廉·华兹华斯、唐纳德·特朗普和巴拉克·奥巴马）的作者签名的能力。利用具有严格主题对齐的零样本提示框架，我们生成了通过结合基于变压器的分类（BERT）和可解释机器学习（XGBoost）的补充框架进行评估的合成语料库。我们的方法整合了语言查询和字数统计 (LIWC) 标记、困惑度和可读性指数，以评估人工智能生成的文本和人类创作的文本之间的差异。结果表明，人工智能生成的模仿仍然具有很高的可检测性，XGBoost 模型在一组有限的八个风格特征上进行训练，达到了与高维神经分类器相当的准确性。特征重要性分析将困惑度确定为主要判别指标，揭示了人工智能输出的随机规律性与人类写作的较高变异性相比存在显着差异。虽然法学硕士在低维启发式特征（例如句法复杂性和可读性）上与人类作者表现出分布收敛，但它们尚未完全复制人类创作的语料库中固有的微妙的情感密度和风格差异。通过隔离当前生成模仿中的具体统计差距，这项研究为法学硕士文体行为提供了全面的基准，并为数字人文和社交媒体中的作者归属提供了重要的见解。

Title: I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes

• Authors: Shijia Zhou, Saif M. Mohammad, Barbara Plank, Diego Frassinelli
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.23229
• Pdf URL: https://arxiv.org/pdf/2603.23229
• Copy Paste: [[2603.23229]] I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes(https://arxiv.org/abs/2603.23229)
• Keywords: language model, llm
• Abstract: Internet memes represent a popular form of multimodal online communication and often use figurative elements to convey layered meaning through the combination of text and images. However, it remains largely unclear how multimodal large language models (MLLMs) combine and interpret visual and textual information to identify figurative meaning in memes. To address this gap, we evaluate eight state-of-the-art generative MLLMs across three datasets on their ability to detect and explain six types of figurative meaning. In addition, we conduct a human evaluation of the explanations generated by these MLLMs, assessing whether the provided reasoning supports the predicted label and whether it remains faithful to the original meme content. Our findings indicate that all models exhibit a strong bias to associate a meme with figurative meaning, even when no such meaning is present. Qualitative analysis further shows that correct predictions are not always accompanied by faithful explanations.
• 摘要：互联网迷因代表了多模式在线交流的流行形式，通常使用比喻元素通过文本和图像的组合来传达分层含义。然而，目前仍不清楚多模态大语言模型（MLLM）如何结合和解释视觉和文本信息以识别模因中的比喻意义。为了解决这一差距，我们评估了三个数据集中的八个最先进的生成 MLLM，评估它们检测和解释六种比喻意义的能力。此外，我们对这些 MLLM 生成的解释进行人工评估，评估所提供的推理是否支持预测的标签以及它是否仍然忠实于原始模因内容。我们的研究结果表明，所有模型都表现出强烈的偏见，将模因与比喻意义联系起来，即使不存在这样的意义。定性分析进一步表明，正确的预测并不总是伴随着忠实的解释。

Title: Is AI Catching Up to Human Expression? Exploring Emotion, Personality, Authorship, and Linguistic Style in English and Arabic with Six Large Language Models

• Authors: Nasser A Alsadhan
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.23251
• Pdf URL: https://arxiv.org/pdf/2603.23251
• Copy Paste: [[2603.23251]] Is AI Catching Up to Human Expression? Exploring Emotion, Personality, Authorship, and Linguistic Style in English and Arabic with Six Large Language Models(https://arxiv.org/abs/2603.23251)
• Keywords: language model, gpt, llm
• Abstract: The advancing fluency of LLMs raises important questions about their ability to emulate complex human traits, including emotional expression and personality, across diverse linguistic and cultural contexts. This study investigates whether LLMs can convincingly mimic emotional nuance in English and personality markers in Arabic, a critical under-resourced language with unique linguistic and cultural characteristics. We conduct two tasks across six models:Jais, Mistral, LLaMA, GPT-4o, Gemini, and DeepSeek. First, we evaluate whether machine classifiers can reliably distinguish between human-authored and AI-generated texts. Second, we assess the extent to which LLM-generated texts exhibit emotional or personality traits comparable to those of humans. Our results demonstrate that AI-generated texts are distinguishable from human-authored ones (F1>0.95), though classification performance deteriorates on paraphrased samples, indicating a reliance on superficial stylistic cues. Emotion and personality classification experiments reveal significant generalization gaps: classifiers trained on human data perform poorly on AI-generated texts and vice versa, suggesting LLMs encode affective signals differently from humans. Importantly, augmenting training with AI-generated data enhances performance in the Arabic personality classification task, highlighting the potential of synthetic data to address challenges in under-resourced languages. Model-specific analyses show that GPT-4o and Gemini exhibit superior affective coherence. Linguistic and psycholinguistic analyses reveal measurable divergences in tone, authenticity, and textual complexity between human and AI texts. These findings have implications for affective computing, authorship attribution, and responsible AI deployment, particularly within underresourced language contexts where generative AI detection and alignment pose unique challenges.
• 摘要：法学硕士的流畅性不断提高，引发了关于他们在不同语言和文化背景下模仿复杂人类特征（包括情感表达和个性）的能力的重要问题。这项研究调查了法学硕士是否能够令人信服地模仿英语中的情感细微差别和阿拉伯语中的个性标记，阿拉伯语是一种资源贫乏的重要语言，具有独特的语言和文化特征。我们在六个模型中执行两项任务：Jais、Mistral、LLaMA、GPT-4o、Gemini 和 DeepSeek。首先，我们评估机器分类器是否能够可靠地区分人类创作的文本和人工智能生成的文本。其次，我们评估法学硕士生成的文本在多大程度上表现出与人类相当的情感或人格特征。我们的结果表明，人工智能生成的文本与人类创作的文本有区别（F1>0.95），尽管分类性能在释义样本上恶化，表明对肤浅文体线索的依赖。情感和性格分类实验揭示了显着的泛化差距：在人类数据上训练的分类器在人工智能生成的文本上表现不佳，反之亦然，这表明法学硕士对情感信号的编码方式与人类不同。重要的是，利用人工智能生成的数据增强训练可以提高阿拉伯语个性分类任务的性能，突显合成数据在解决资源贫乏语言挑战方面的潜力。模型特定分析表明 GPT-4o 和 Gemini 表现出卓越的情感一致性。语言学和心理语言学分析揭示了人类文本和人工智能文本在语气、真实性和文本复杂性方面存在可测量的差异。这些发现对情感计算、作者归属和负责任的人工智能部署具有影响，特别是在资源匮乏的语言环境中，生成式人工智能检测和对齐提出了独特的挑战。

Title: Steering LLMs for Culturally Localized Generation

• Authors: Simran Khanuja, Hongbin Liu, Shujian Zhang, John Lambert, Mingqing Chen, Rajiv Mathews, Lun Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.23301
• Pdf URL: https://arxiv.org/pdf/2603.23301
• Copy Paste: [[2603.23301]] Steering LLMs for Culturally Localized Generation(https://arxiv.org/abs/2603.23301)
• Keywords: llm, prompt
• Abstract: LLMs are deployed globally, yet produce responses biased towards cultures with abundant training data. Existing cultural localization approaches such as prompting or post-training alignment are black-box, hard to control, and do not reveal whether failures reflect missing knowledge or poor elicitation. In this paper, we address these gaps using mechanistic interpretability to uncover and manipulate cultural representations in LLMs. Leveraging sparse autoencoders, we identify interpretable features that encode culturally salient information and aggregate them into Cultural Embeddings (CuE). We use CuE both to analyze implicit cultural biases under underspecified prompts and to construct white-box steering interventions. Across multiple models, we show that CuE-based steering increases cultural faithfulness and elicits significantly rarer, long-tail cultural concepts than prompting alone. Notably, CuE-based steering is complementary to black-box localization methods, offering gains when applied on top of prompt-augmented inputs. This also suggests that models do benefit from better elicitation strategies, and don't necessarily lack long-tail knowledge representation, though this varies across cultures. Our results provide both diagnostic insight into cultural representations in LLMs and a controllable method to steer towards desired cultures.
• 摘要：法学硕士在全球范围内部署，但产生的反应偏向于拥有丰富培训数据的文化。现有的文化本地化方法（例如提示或训练后对齐）是黑匣子，难以控制，并且无法揭示失败是否反映了知识缺失或启发不佳。在本文中，我们利用机械可解释性来揭示和操纵法学硕士中的文化表征来解决这些差距。利用稀疏自动编码器，我们识别可解释的特征，这些特征对文化显着信息进行编码并将其聚合到文化嵌入（CuE）中。我们使用 CuE 来分析未指定提示下的隐性文化偏见，并构建白盒指导干预措施。在多个模型中，我们表明基于 CuE 的引导可以提高文化忠诚度，并比单独提示更能引发罕见的长尾文化概念。值得注意的是，基于 CuE 的转向是对黑盒定位方法的补充，当应用于提示增强输入时会带来收益。这也表明模型确实受益于更好的启发策略，并且不一定缺乏长尾知识表示，尽管这在不同文化中有所不同。我们的结果既提供了对法学硕士文化表征的诊断见解，也提供了引导所需文化的可控方法。

Title: Failure of contextual invariance in gender inference with large language models

• Authors: Sagar Kumar, Ariel Flint, Luca Maria Aiello, Andrea Baronchelli
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2603.23485
• Pdf URL: https://arxiv.org/pdf/2603.23485
• Copy Paste: [[2603.23485]] Failure of contextual invariance in gender inference with large language models(https://arxiv.org/abs/2603.23485)
• Keywords: language model, llm
• Abstract: Standard evaluation practices assume that large language model (LLM) outputs are stable under contextually equivalent formulations of a task. Here, we test this assumption in the setting of gender inference. Using a controlled pronoun selection task, we introduce minimal, theoretically uninformative discourse context and find that this induces large, systematic shifts in model outputs. Correlations with cultural gender stereotypes, present in decontextualized settings, weaken or disappear once context is introduced, while theoretically irrelevant features, such as the gender of a pronoun for an unrelated referent, become the most informative predictors of model behaviour. A Contextuality-by-Default analysis reveals that, in 19--52\% of cases across models, this dependence persists after accounting for all marginal effects of context on individual outputs and cannot be attributed to simple pronoun repetition. These findings show that LLM outputs violate contextual invariance even under near-identical syntactic formulations, with implications for bias benchmarking and deployment in high-stakes settings.
• 摘要：标准评估实践假设大语言模型 (LLM) 输出在任务的上下文等效表述下是稳定的。在这里，我们在性别推断的背景下测试这个假设。使用受控代词选择任务，我们引入了最少的、理论上无信息的话语上下文，并发现这会导致模型输出发生巨大的系统性变化。一旦引入语境，与文化性别刻板印象的相关性就会在脱离语境的环境中减弱或消失，而理论上不相关的特征，例如代表不相关所指对象的代词的性别，则成为模型行为最有信息性的预测因素。默认上下文分析表明，在模型中 19--52% 的情况下，在考虑了上下文对个体输出的所有边际影响后，这种依赖性仍然存在，并且不能归因于简单的代词重复。这些发现表明，即使在几乎相同的句法表述下，法学硕士的输出也违反了上下文不变性，这对高风险环境中的偏见基准测试和部署产生了影响。