2026-03-19

Title: Trust, Safety, and Accuracy: Assessing LLMs for Routine Maternity Advice

• Authors: V Sai Divya, A Bhanusree, Rimjhim, K Venkata Krishna Rao
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2603.16872
• Pdf URL: https://arxiv.org/pdf/2603.16872
• Copy Paste: [[2603.16872]] Trust, Safety, and Accuracy: Assessing LLMs for Routine Maternity Advice(https://arxiv.org/abs/2603.16872)
• Keywords: language model, gpt, llm, chat
• Abstract: Access to reliable maternal healthcare information is a major challenge in rural India due to limited medical resources and infrastructure. With over 830 million internet users and nearly half of rural women online, digital tools offer new opportunities for health education. This study evaluates large language models (LLMs) like ChatGPT-4o, Perplexity AI, and GeminiAI to provide reliable and understandable pregnancy-related information. Seventeen pregnancy-focused questions were posed to each model and compared with responses from maternal health professionals. Evaluations used semantic similarity, noun overlap, and readability metrics to measure content quality. Results show Perplexity closely matched expert semantics, while ChatGPT-4o produced clearer, more understandable text with better medical terminology. As internet access grows in rural areas, LLMs could serve as scalable aids for maternal health education. The study highlights the need for AI tools that balance accuracy and clarity to improve healthcare communication in underserved regions.
• 摘要：由于医疗资源和基础设施有限，获得可靠的孕产妇医疗保健信息是印度农村地区的一项重大挑战。超过 8.3 亿互联网用户和近一半的农村妇女上网，数字工具为健康教育提供了新的机会。这项研究评估了 ChatGPT-4o、Perplexity AI 和 GeminiAI 等大型语言模型 (LLM)，以提供可靠且易于理解的怀孕相关信息。每个模型都提出了 17 个以怀孕为中心的问题，并与孕产妇保健专业人员的回答进行了比较。评估使用语义相似性、名词重叠和可读性指标来衡量内容质量。结果显示 Perplexity 与专家语义紧密匹配，而 ChatGPT-4o 生成了更清晰、更易于理解的文本以及更好的医学术语。随着农村地区互联网接入的增长，法学硕士可以作为孕产妇健康教育的可扩展援助。该研究强调需要平衡准确性和清晰度的人工智能工具，以改善服务欠缺地区的医疗保健沟通。

Title: Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis

• Authors: Zhiyuan Cheng, Longying Lai, Yue Liu, Kai Cheng, Xiaoxi Qi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.16877
• Pdf URL: https://arxiv.org/pdf/2603.16877
• Copy Paste: [[2603.16877]] Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis(https://arxiv.org/abs/2603.16877)
• Keywords: language model, retrieval-augmented generation
• Abstract: Financial analysts face significant challenges extracting information from lengthy 10-K reports, which often exceed 100 pages. This paper presents a Retrieval-Augmented Generation (RAG) system designed to answer questions about S&P 500 financial reports and evaluates the impact of neural reranking on system performance. Our pipeline employs hybrid search combining full-text and semantic retrieval, followed by an optional reranking stage using a cross-encoder model. We conduct systematic evaluation using the FinDER benchmark dataset, comprising 1,500 queries across five experimental groups. Results demonstrate that reranking significantly improves answer quality, achieving 49.0 percent correctness for scores of 8 or above compared to 33.5 percent without reranking, representing a 15.5 percentage point improvement. Additionally, the error rate for completely incorrect answers decreases from 35.3 percent to 22.5 percent. Our findings emphasize the critical role of reranking in financial RAG systems and demonstrate performance improvements over baseline methods through modern language models and refined retrieval strategies.
• 摘要：财务分析师面临着从冗长的 10-K 报告（通常超过 100 页）中提取信息的重大挑战。本文提出了一种检索增强生成 (RAG) 系统，旨在回答有关标准普尔 500 指数财务报告的问题，并评估神经重新排序对系统性能的影响。我们的管道采用结合全文和语义检索的混合搜索，然后是使用交叉编码器模型的可选重新排名阶段。我们使用 FinDER 基准数据集进行系统评估，该数据集包含五个实验组的 1,500 个查询。结果表明，重新排名显着提高了答案质量，8 分或以上的正确率达到 49.0%，而没有重新排名的正确率为 33.5%，提高了 15.5 个百分点。此外，完全错误答案的错误率从 35.3% 降至 22.5%。我们的研究结果强调了重新排名在金融 RAG 系统中的关键作用，并通过现代语言模型和完善的检索策略证明了相对于基线方法的性能改进。

Title: Rubric-Guided Fine-tuning of SpeechLLMs for Multi-Aspect, Multi-Rater L2 Reading-Speech Assessment

• Authors: Aditya Kamlesh Parikh, Cristian Tejedor-Garcia, Catia Cucchiarini, Helmer Strik
• Subjects: cs.CL, cs.AI, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2603.16889
• Pdf URL: https://arxiv.org/pdf/2603.16889
• Copy Paste: [[2603.16889]] Rubric-Guided Fine-tuning of SpeechLLMs for Multi-Aspect, Multi-Rater L2 Reading-Speech Assessment(https://arxiv.org/abs/2603.16889)
• Keywords: language model, llm
• Abstract: Reliable and interpretable automated assessment of second-language (L2) speech remains a central challenge, as large speech-language models (SpeechLLMs) often struggle to align with the nuanced variability of human raters. To address this, we introduce a rubric-guided reasoning framework that explicitly encodes multi-aspect human assessment criteria: accuracy, fluency, and prosody, while calibrating model uncertainty to capture natural rating variability. We fine-tune the Qwen2-Audio-7B-Instruct model using multi-rater human judgments and develop an uncertainty-calibrated regression approach supported by conformal calibration for interpretable confidence intervals. Our Gaussian uncertainty modeling and conformal calibration approach achieves the strongest alignment with human ratings, outperforming regression and classification baselines. The model reliably assesses fluency and prosody while highlighting the inherent difficulty of assessing accuracy. Together, these results demonstrate that rubric-guided, uncertainty-calibrated reasoning offers a principled path toward trustworthy and explainable SpeechLLM-based speech assessment.
• 摘要：对第二语言 (L2) 语音进行可靠且可解释的自动评估仍然是一个核心挑战，因为大型语音语言模型 (SpeechLLM) 常常难以与人类评估者的细微差别保持一致。为了解决这个问题，我们引入了一个以标题为导向的推理框架，该框架明确编码了多方面的人类评估标准：准确性、流畅性和韵律，同时校准模型不确定性以捕获自然评分变异性。我们使用多评估者人类判断对 Qwen2-Audio-7B-Instruct 模型进行微调，并开发了一种不确定性校准回归方法，该方法由可解释置信区间的共形校准支持。我们的高斯不确定性建模和共形校准方法与人类评级实现了最强的一致性，优于回归和分类基线。该模型可靠地评估流畅性和韵律，同时强调评估准确性的固有困难。总之，这些结果表明，以规则为导向、经过不确定性校准的推理为基于 SpeechLLM 的可信且可解释的语音评估提供了一条原则性途径。

Title: LLM NL2SQL Robustness: Surface Noise vs. Linguistic Variation in Traditional and Agentic Settings

• Authors: Lifu Tu, Rongguang Wang, Tao Sheng, Sujjith Ravi, Dan Roth
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.17017
• Pdf URL: https://arxiv.org/pdf/2603.17017
• Copy Paste: [[2603.17017]] LLM NL2SQL Robustness: Surface Noise vs. Linguistic Variation in Traditional and Agentic Settings(https://arxiv.org/abs/2603.17017)
• Keywords: language model, gpt, llm, agent
• Abstract: Robustness evaluation for Natural Language to SQL (NL2SQL) systems is essential because real-world database environments are dynamic, noisy, and continuously evolving, whereas conventional benchmark evaluations typically assume static schemas and well-formed user inputs. In this work, we introduce a robustness evaluation benchmark containing approximately ten types of perturbations and conduct evaluations under both traditional and agentic settings. We assess multiple state-of-the-art large language models (LLMs), including Grok-4.1, Gemini-3-Pro, Claude-Opus-4.6, and GPT-5.2. Our results show that these models generally maintain strong performance under several perturbations; however, notable performance degradation is observed for surface-level noise (e.g., character-level corruption) and linguistic variation that preserves semantics while altering lexical or syntactic forms. Furthermore, we observe that surface-level noise causes larger performance drops in traditional pipelines, whereas linguistic variation presents greater challenges in agentic settings. These findings highlight the remaining challenges in achieving robust NL2SQL systems, particularly in handling linguistic variability.
• 摘要：Natural Language to SQL (NL2SQL) 系统的鲁棒性评估至关重要，因为现实世界的数据库环境是动态的、嘈杂的且不断发展的，而传统的基准评估通常假设静态模式和格式良好的用户输入。在这项工作中，我们引入了包含大约十种扰动的鲁棒性评估基准，并在传统和代理环境下进行评估。我们评估了多种最先进的大型语言模型 (LLM)，包括 Grok-4.1、Gemini-3-Pro、Claude-Opus-4.6 和 GPT-5.2。我们的结果表明，这些模型在多种扰动下通常保持强劲的性能；然而，对于表面级噪声（例如，字符级损坏）和在改变词汇或句法形式的同时保留语义的语言变化，观察到显着的性能下降。此外，我们观察到表面噪声会导致传统管道中更大的性能下降，而语言变化在代理环境中提出了更大的挑战。这些发现凸显了实现健壮的 NL2SQL 系统所面临的挑战，特别是在处理语言变异性方面。

Title: Evaluating Ill-Defined Tasks in Large Language Models

• Authors: Yi Zhou, Basel Shbita
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.17067
• Pdf URL: https://arxiv.org/pdf/2603.17067
• Copy Paste: [[2603.17067]] Evaluating Ill-Defined Tasks in Large Language Models(https://arxiv.org/abs/2603.17067)
• Keywords: language model, llm
• Abstract: Many evaluations of Large Language Models (LLMs) target tasks that are inherently ill-defined, with unclear input and output spaces and ambiguous success criteria. We analyze why existing evaluation benchmarks and metrics fail to provide reliable or diagnostic signals of model capability for such tasks. We examine two case studies: Complex Instruction Following (CIF), where we identify recurring issues including limited coverage of real-world instruction complexity, sensitivity to instruction phrasing, inconsistent and non-comparable metrics, and instability introduced by LLM-based judges; and Natural Language to Mermaid Sequence Diagrams (NL2Mermaid), where we show how multi-faceted evaluation criteria can yield actionable insights beyond aggregate scores. Together, these case studies show that current evaluations frequently conflate distinct failure modes, yielding scores that are unstable, non-diagnostic, and difficult to act upon. Our findings expose fundamental limitations in existing evaluation practices for ill-defined tasks and motivate more robust, interpretable evaluation designs.
• 摘要：许多大型语言模型（LLM）的评估目标任务本质上是不明确的，输入和输出空间不明确，成功标准也不明确。我们分析了为什么现有的评估基准和指标无法为此类任务提供模型能力的可靠或诊断信号。我们研究了两个案例研究：复杂指令跟踪（CIF），其中我们识别了反复出现的问题，包括对现实世界指令复杂性的有限覆盖、对指令措辞的敏感性、不一致和不可比较的指标以及法学硕士法官引入的不稳定性；和自然语言到美人鱼序列图（NL2Mermaid），我们展示了多方面的评估标准如何能够产生超越总分的可操作的见解。总之，这些案例研究表明，当前的评估经常将不同的故障模式混为一谈，产生不稳定、非诊断性且难以采取行动的分数。我们的研究结果揭示了现有评估实践对于定义不明确的任务的根本局限性，并激发了更稳健、可解释的评估设计。

Title: Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts

• Authors: Lucas Bandarkar, Alan Ansell, Trevor Cohn
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.17070
• Pdf URL: https://arxiv.org/pdf/2603.17070
• Copy Paste: [[2603.17070]] Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts(https://arxiv.org/abs/2603.17070)
• Keywords: llm
• Abstract: In this work, we analyze shortcomings in cross-lingual knowledge transfer in large, modern reasoning LLMs. We demonstrate that the perceived gap in knowledge transfer is primarily a script barrier. First, we conduct an observational data analysis on the performance of thinking models on two datasets with local knowledge from around the world, ECLeKTic and MultiLoKo. Our regression analysis shows that script match - not language or family - is the primary predictor of knowledge transfer failure once model capability and question difficulty are accounted for. We further this finding by providing the LLMs with the key entities of the questions in their source language and find that this disproportionately improves cross-script questions. We then posit that these LLMs could be reasoning better at test-time. To evaluate this, we develop a synthetic generation pipeline to design SFT samples to encourage the model to better reason about transliteration ambiguities when trying to fetch parametric knowledge at inference-time. We show that teaching two models to reason better reduces the cross-script transfer gap. As a result, we conclude that there is potential to improve cross-lingual parametric knowledge transfer during post-training.
• 摘要：在这项工作中，我们分析了大型现代推理法学硕士跨语言知识转移的缺点。我们证明，知识转移中的感知差距主要是脚本障碍。首先，我们使用来自世界各地的本地知识的两个数据集 ECLeKTic 和 MultiLoKo 对思维模型的表现进行了观察数据分析。我们的回归分析表明，一旦考虑到模型能力和问题难度，脚本匹配（而不是语言或家庭）是知识转移失败的主要预测因素。我们通过向法学硕士提供其源语言中问题的关键实体来进一步推进这一发现，并发现这极大地改善了跨脚本问题。然后我们假设这些法学硕士在测试时可以更好地推理。为了评估这一点，我们开发了一个合成生成管道来设计 SFT 样本，以鼓励模型在尝试在推理时获取参数知识时更好地推理音译歧义。我们证明，教授两个模型进行推理可以更好地减少跨脚本传输差距。因此，我们得出的结论是，在培训后有可能改善跨语言参数知识转移。

Title: Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction

• Authors: Ryo Kamoi, Ameya Godbole, Longqi Yang, Rui Zhang, Mengting Wan, Pei Zhou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17094
• Pdf URL: https://arxiv.org/pdf/2603.17094
• Copy Paste: [[2603.17094]] Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction(https://arxiv.org/abs/2603.17094)
• Keywords: language model, gpt, llm, prompt
• Abstract: Simulating human conversations using large language models (LLMs) has emerged as a scalable methodology for modeling human social interaction. However, simulating human conversations is challenging because they inherently involve inconsistent and uncollaborative behaviors, such as misunderstandings and interruptions. Analysis comparing inconsistent and uncollaborative behaviors in human- and LLM-generated conversations remains limited, although reproducing these behaviors is integral to simulating human-like and complex social interaction. In this work, we introduce CoCoEval, an evaluation framework that analyzes LLM-simulated conversations by detecting 10 types of inconsistent and uncollaborative behaviors at the turn level using an LLM-as-a-Judge. Using CoCoEval, we evaluate GPT-4.1, GPT-5.1, and Claude Opus 4 by comparing the frequencies of detected behaviors in conversations simulated by each model and in human conversations across academic, business, and governmental meetings, as well as debates. Our analysis shows that (1) under vanilla prompting, LLM-simulated conversations exhibit far fewer inconsistent and uncollaborative behaviors than human conversations; (2) prompt engineering does not provide reliable control over these behaviors, as our results show that different prompts lead to their under- or overproduction; and (3) supervised fine-tuning on human conversations can lead LLMs to overproduce a narrow set of behaviors, such as repetition. Our findings highlight the difficulty of simulating human conversations, raising concerns about the use of LLMs as a proxy for human social interaction.
• 摘要：使用大型语言模型 (LLM) 模拟人类对话已成为一种用于建模人类社交互动的可扩展方法。然而，模拟人类对话具有挑战性，因为它们本质上涉及不一致和不合作的行为，例如误解和打断。尽管重现这些行为对于模拟类人且复杂的社交互动是不可或缺的，但对人类和法学硕士生成的对话中不一致和不合作行为进行比较的分析仍然有限。在这项工作中，我们介绍了 CoCoEval，一个评估框架，通过使用法学硕士作为法官检测回合级别的 10 种不一致和不合作的行为来分析法学硕士模拟的对话。使用 CoCoEval，我们通过比较每个模型模拟的对话以及学术、商业和政府会议以及辩论中的人类对话中检测到的行为的频率来评估 GPT-4.1、GPT-5.1 和 Claude Opus 4。我们的分析表明：（1）在普通的提示下，LLM 模拟的对话表现出的不一致和不合作行为比人类对话要少得多； （2）提示工程无法对这些行为提供可靠的控制，因为我们的结果表明不同的提示会导致它们的生产不足或过剩； (3) 对人类对话的监督微调可能会导致法学硕士过度产生一组狭窄的行为，例如重复。我们的研究结果强调了模拟人类对话的难度，引发了人们对使用法学硕士作为人类社交互动代理的担忧。

Title: Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency

• Authors: Lucas Bandarkar, Alan Ansell, Trevor Cohn
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.17102
• Pdf URL: https://arxiv.org/pdf/2603.17102
• Copy Paste: [[2603.17102]] Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency(https://arxiv.org/abs/2603.17102)
• Keywords: llm
• Abstract: Modern LLMs continue to exhibit significant variance in behavior across languages, such as being able to recall factual information in some languages but not others. While typically studied as a problem to be mitigated, in this work, we propose leveraging this cross-lingual inconsistency as a tool for interpretability in mixture-of-experts (MoE) LLMs. Our knowledge localization framework contrasts routing for sets of languages where the model correctly recalls information from languages where it fails. This allows us to isolate model components that play a functional role in answering about a piece of knowledge. Our method proceeds in two stages: (1) querying the model with difficult factual questions across a diverse set of languages to generate "success" and "failure" activation buckets and then (2) applying a statistical contrastive analysis to the MoE router logits to identify experts important for knowledge. To validate the necessity of this small number of experts for answering a knowledge question, we deactivate them and re-ask the question. We find that despite only deactivating about 20 out of 6000 experts, the model no longer answers correctly in over 40% of cases. Generally, this method provides a realistic and scalable knowledge localization approach to address increasingly complex LLMs.
• 摘要：现代法学硕士继续表现出跨语言行为的显着差异，例如能够回忆起某些语言的事实信息，但不能回忆起其他语言的事实信息。虽然通常将其作为需要缓解的问题进行研究，但在这项工作中，我们建议利用这种跨语言不一致作为专家混合 (MoE) 法学硕士的可解释性工具。我们的知识本地化框架对比了语言集的路由，其中​​模型正确地从失败的语言中召回信息。这使我们能够隔离在回答知识片段时发挥功能作用的模型组件。我们的方法分两个阶段进行：(1) 使用多种语言的困难事实问题查询模型，以生成“成功”和“失败”激活桶，然后 (2) 对 MoE 路由器日志应用统计对比分析，以识别对知识重要的专家。为了验证这少数专家回答知识问题的必要性，我们停用他们并重新提出问题。我们发现，尽管仅停用了 6000 名专家中的约 20 名，但该模型在超过 40% 的情况下不再正确回答。一般来说，这种方法提供了一种现实且可扩展的知识本地化方法来解决日益复杂的法学硕士问题。

Title: Exploiting the English Grammar Profile for L2 grammatical analysis with LLMs

• Authors: Stefano Bannò, Penny Karanasou, Kate Knill, Mark Gales
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17171
• Pdf URL: https://arxiv.org/pdf/2603.17171
• Copy Paste: [[2603.17171]] Exploiting the English Grammar Profile for L2 grammatical analysis with LLMs(https://arxiv.org/abs/2603.17171)
• Keywords: llm
• Abstract: Evaluating the grammatical competence of second language (L2) learners is essential both for providing targeted feedback and for assessing proficiency. To achieve this, we propose a novel framework leveraging the English Grammar Profile (EGP), a taxonomy of grammatical constructs mapped to the proficiency levels of the Common European Framework of Reference (CEFR), to detect learners' attempts at grammatical constructs and classify them as successful or unsuccessful. This detection can then be used to provide fine-grained feedback. Moreover, the grammatical constructs are used as predictors of proficiency assessment by using automatically detected attempts as predictors of holistic CEFR proficiency. For the selection of grammatical constructs derived from the EGP, rule-based and LLM-based classifiers are compared. We show that LLMs outperform rule-based methods on semantically and pragmatically nuanced constructs, while rule-based approaches remain competitive for constructs that rely purely on morphological or syntactic features and do not require semantic interpretation. For proficiency assessment, we evaluate both rule-based and hybrid pipelines and show that a hybrid approach combining a rule-based pre-filter with an LLM consistently yields the strongest performance. Since our framework operates on pairs of original learner sentences and their corrected counterparts, we also evaluate a fully automated pipeline using automatic grammatical error correction. This pipeline closely approaches the performance of semi-automated systems based on manual corrections, particularly for the detection of successful attempts at grammatical constructs. Overall, our framework emphasises learners' successful attempts in addition to unsuccessful ones, enabling positive, formative feedback and providing actionable insights into grammatical development.
• 摘要：评估第二语言 (L2) 学习者的语法能力对于提供有针对性的反馈和评估熟练程度至关重要。为了实现这一目标，我们提出了一个新的框架，利用英语语法概况（EGP），一种映射到欧洲共同参考框架（CEFR）熟练程度的语法结构分类法，来检测学习者对语法结构的尝试并将其分类为成功或不成功。然后可以使用该检测来提供细粒度的反馈。此外，通过使用自动检测到的尝试作为整体 CEFR 熟练程度的预测因子，语法结构被用作熟练程度评估的预测因子。为了选择源自 EGP 的语法结构，比较了基于规则的分类器和基于 LLM 的分类器。我们表明，法学硕士在语义和语用上的细微结构上优于基于规则的方法，而基于规则的方法对于纯粹依赖形态或句法特征且不需要语义解释的结构仍然具有竞争力。对于熟练程度评估，我们评估了基于规则的管道和混合管道，并表明将基于规则的预过滤器与法学硕士相结合的混合方法始终能产生最强的性能。由于我们的框架对原始学习者句子及其纠正后的对应句子进行操作，因此我们还使用自动语法错误纠正来评估完全自动化的管道。该管道非常接近基于手动更正的半自动化系统的性能，特别是在检测语法结构的成功尝试方面。总体而言，我们的框架强调学习者的成功尝试和不成功的尝试，从而实现积极的、形成性的反馈，并为语法发展提供可操作的见解。

Title: Tabular LLMs for Interpretable Few-Shot Alzheimer's Disease Prediction with Multimodal Biomedical Data

• Authors: Sophie Kearney, Shu Yang, Zixuan Wen, Weimin Lyu, Bojian Hou, Duy Duong-Tran, Tianlong Chen, Jason H. Moore, Marylyn D. Ritchie, Chao Chen, Li Shen
• Subjects: cs.CL, cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2603.17191
• Pdf URL: https://arxiv.org/pdf/2603.17191
• Copy Paste: [[2603.17191]] Tabular LLMs for Interpretable Few-Shot Alzheimer's Disease Prediction with Multimodal Biomedical Data(https://arxiv.org/abs/2603.17191)
• Keywords: language model, gpt, llm, prompt, agent
• Abstract: Accurate diagnosis of Alzheimer's disease (AD) requires handling tabular biomarker data, yet such data are often small and incomplete, where deep learning models frequently fail to outperform classical methods. Pretrained large language models (LLMs) offer few-shot generalization, structured reasoning, and interpretable outputs, providing a powerful paradigm shift for clinical prediction. We propose TAP-GPT Tabular Alzheimer's Prediction GPT, a domain-adapted tabular LLM framework built on TableGPT2 and fine-tuned for few-shot AD classification using tabular prompts rather than plain texts. We evaluate TAP-GPT across four ADNI-derived datasets, including QT-PAD biomarkers and region-level structural MRI, amyloid PET, and tau PET for binary AD classification. Across multimodal and unimodal settings, TAP-GPT improves upon its backbone models and outperforms traditional machine learning baselines in the few-shot setting while remaining competitive with state-of-the-art general-purpose LLMs. We show that feature selection mitigates degradation in high-dimensional inputs and that TAP-GPT maintains stable performance under simulated and real-world missingness without imputation. Additionally, TAP-GPT produces structured, modality-aware reasoning aligned with established AD biology and shows greater stability under self-reflection, supporting its use in iterative multi-agent systems. To our knowledge, this is the first systematic application of a tabular-specialized LLM to multimodal biomarker-based AD prediction, demonstrating that such pretrained models can effectively address structured clinical prediction tasks and laying the foundation for tabular LLM-driven multi-agent clinical decision-support systems. The source code is publicly available on GitHub: this https URL.
• 摘要：阿尔茨海默病 (AD) 的准确诊断需要处理表格生物标志物数据，但此类数据通常很小且不完整，深度学习模型常常无法超越经典方法。预训练的大语言模型 (LLM) 提供少样本泛化、结构化推理和可解释的输出，为临床预测提供强大的范式转变。我们提出了 TAP-GPT 表格式阿尔茨海默病预测 GPT，这是一个基于 TableGPT2 构建的领域适应表格式 LLM 框架，并使用表格提示而不是纯文本进行了少量 AD 分类的微调。我们在四个 ADNI 衍生数据集上评估 TAP-GPT，包括 QT-PAD 生物标志物和区域级结构 MRI、淀粉样蛋白 PET 和用于二元 AD 分类的 tau PET。在多模态和单模态设置中，TAP-GPT 改进了其主干模型，并在少数样本设置中超越了传统的机器学习基线，同时保持了与最先进的通用法学硕士的竞争力。我们表明，特征选择可以减轻高维输入的退化，并且 TAP-GPT 在模拟和现实世界缺失情况下保持稳定的性能而无需插补。此外，TAP-GPT 产生与已建立的 AD 生物学一致的结构化、模态感知推理，并在自我反思下表现出更大的稳定性，支持其在迭代多智能体系统中的使用。据我们所知，这是表格专业法学硕士在基于多模态生物标志物的AD预测中的首次系统应用，证明这种预训练模型可以有效地解决结构化临床预测任务，并为表格法学硕士驱动的多智能体临床决策支持系统奠定基础。源代码可在 GitHub 上公开获取：此 https URL。

Title: CODMAS: A Dialectic Multi-Agent Collaborative Framework for Structured RTL Optimization

• Authors: Che-Ming Chang, Prashanth Vijayaraghavan, Ashutosh Jadhav, Charles Mackin, Vandana Mukherjee, Hsinyu Tsai, Ehsan Degan
• Subjects: cs.CL, cs.AR, cs.PL
• Abstract URL: https://arxiv.org/abs/2603.17204
• Pdf URL: https://arxiv.org/pdf/2603.17204
• Copy Paste: [[2603.17204]] CODMAS: A Dialectic Multi-Agent Collaborative Framework for Structured RTL Optimization(https://arxiv.org/abs/2603.17204)
• Keywords: llm, prompt, agent
• Abstract: Optimizing Register Transfer Level (RTL) code is a critical step in Electronic Design Automation (EDA) for improving power, performance, and area (PPA). We present CODMAS (Collaborative Optimization via a Dialectic Multi-Agent System), a framework that combines structured dialectic reasoning with domain-aware code generation and deterministic evaluation to automate RTL optimization. At the core of CODMAS are two dialectic agents: the Articulator, inspired by rubber-duck debugging, which articulates stepwise transformation plans and exposes latent assumptions; and the Hypothesis Partner, which predicts outcomes and reconciles deviations between expected and actual behavior to guide targeted refinements. These agents direct a Domain-Specific Coding Agent (DCA) to generate architecture-aware Verilog edits and a Code Evaluation Agent (CEA) to verify syntax, functionality, and PPA metrics. We introduce RTLOPT, a benchmark of 120 Verilog triples (unoptimized, optimized, testbench) for pipelining and clock-gating transformations. Across proprietary and open LLMs, CODMAS achieves ~25% reduction in critical path delay for pipelining and ~22% power reduction for clock gating, while reducing functional and compilation failures compared to strong prompting and agentic baselines. These results demonstrate that structured multi-agent reasoning can significantly enhance automated RTL optimization and scale to more complex designs and broader optimization tasks.
• 摘要：优化寄存器传输级 (RTL) 代码是电子设计自动化 (EDA) 中提高功耗、性能和面积 (PPA) 的关键步骤。我们提出了 CODMAS（通过辩证多代理系统进行协作优化），这是一个框架，它将结构化辩证推理与领域感知代码生成和确定性评估相结合，以自动实现 RTL 优化。 CODMAS 的核心是两个辩证代理：Articulator，受橡皮鸭调试启发，阐明逐步转换计划并揭示潜在假设；假设合作伙伴，预测结果并协调预期行为和实际行为之间的偏差，以指导有针对性的改进。这些代理指导特定领域编码代理 (DCA) 生成架构感知的 Verilog 编辑，并指导代码评估代理 (CEA) 验证语法、功能和 PPA 指标。我们引入 RTLOPT，这是一个用于流水线和时钟门控转换的 120 个 Verilog 三元组（未优化、优化、测试平台）的基准。在专有和开放的法学硕士中，与强提示和代理基线相比，CODMAS 使流水线的关键路径延迟减少了约 25%，时钟门控的功耗减少了约 22%，同时减少了功能和编译失败。这些结果表明，结构化多智能体推理可以显着增强自动化 RTL 优化，并扩展到更复杂的设计和更广泛的优化任务。

Title: SYMDIREC: A Neuro-Symbolic Divide-Retrieve-Conquer Framework for Enhanced RTL Synthesis and Summarization

• Authors: Prashanth Vijayaraghavan, Apoorva Nitsure, Luyao Shi, Charles Mackin, Ashutosh Jadhav, David Beymer, Ehsan Degan, Vandana Mukherjee
• Subjects: cs.CL, cs.PL
• Abstract URL: https://arxiv.org/abs/2603.17208
• Pdf URL: https://arxiv.org/pdf/2603.17208
• Copy Paste: [[2603.17208]] SYMDIREC: A Neuro-Symbolic Divide-Retrieve-Conquer Framework for Enhanced RTL Synthesis and Summarization(https://arxiv.org/abs/2603.17208)
• Keywords: language model, llm, prompt, retrieval-augmented generation
• Abstract: Register-Transfer Level (RTL) synthesis and summarization are central to hardware design automation but remain challenging for Large Language Models (LLMs) due to rigid HDL syntax, limited supervision, and weak alignment with natural language. Existing prompting and retrieval-augmented generation (RAG) methods have not incorporated symbolic planning, limiting their structural precision. We introduce SYMDIREC, a neuro-symbolic framework that decomposes RTL tasks into symbolic subgoals, retrieves relevant code via a fine-tuned retriever, and assembles verified outputs through LLM reasoning. Supporting both Verilog and VHDL without LLM fine-tuning, SYMDIREC achieves ~20% higher Pass@1 rates for synthesis and 15-20% ROUGE-L improvements for summarization over prompting and RAG baselines, demonstrating the benefits of symbolic guidance in RTL tasks.
• 摘要：寄存器传输级 (RTL) 综合和汇总是硬件设计自动化的核心，但由于严格的 HDL 语法、有限的监督以及与自然语言的弱对齐，对于大型语言模型 (LLM) 来说仍然具有挑战性。现有的提示和检索增强生成（RAG）方法没有纳入符号规划，限制了它们的结构精度。我们引入了 SYMDIREC，一个神经符号框架，它将 RTL 任务分解为符号子目标，通过微调检索器检索相关代码，并通过 LLM 推理组装经过验证的输出。 SYMDIREC 支持 Verilog 和 VHDL，无需 LLM 微调，与提示和 RAG 基线相比，综合 Pass@1 率提高了约 20%，汇总率 ROUGE-L 提高了 15-20%，展示了 RTL 任务中符号指导的优势。

Title: Anonymous-by-Construction: An LLM-Driven Framework for Privacy-Preserving Text

• Authors: Federico Albanese, Pablo Ronco, Nicolás D'Ippolito
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.17217
• Pdf URL: https://arxiv.org/pdf/2603.17217
• Copy Paste: [[2603.17217]] Anonymous-by-Construction: An LLM-Driven Framework for Privacy-Preserving Text(https://arxiv.org/abs/2603.17217)
• Keywords: language model, llm, agent
• Abstract: Responsible use of AI demands that we protect sensitive information without undermining the usefulness of data, an imperative that has become acute in the age of large language models. We address this challenge with an on-premise, LLM-driven substitution pipeline that anonymizes text by replacing personally identifiable information (PII) with realistic, type-consistent surrogates. Executed entirely within organizational boundaries using local LLMs, the approach prevents data egress while preserving fluency and task-relevant semantics. We conduct a systematic, multi-metric, cross-technique evaluation on the Action-Based Conversation Dataset, benchmarking against industry standards (Microsoft Presidio and Google DLP) and a state-of-the-art approach (ZSTS, in redaction-only and redaction-plus-substitution variants). Our protocol jointly measures privacy, semantic utility, and trainability under privacy via a lifecycle-ready criterion obtained by fine-tuning a compact encoder (BERT+LoRA) on sanitized text. In addition, we assess agentic Q&A performance by inserting an on-premise anonymization layer before the answering LLM and evaluating the quality of its responses. This intermediate, type-preserving substitution stage ensures that no sensitive content is exposed to third-party APIs, enabling responsible deployment of Q\&A agents without compromising confidentiality. Our method attains state-of-the-art privacy, minimal topical drift, strong factual utility, and low trainability loss, outperforming rule-based approaches and named-entity recognition (NER) baselines and ZSTS variants on the combined privacy--utility--trainability frontier. These results show that local LLM substitution yields anonymized corpora that are both responsible to use and operationally valuable: safe for agentic pipelines and suitable for downstream fine-tuning with limited degradation.
• 摘要：负责任地使用人工智能要求我们在不损害数据有用性的情况下保护敏感信息，这一点在大型语言模型时代变得越来越迫切。我们通过本地 LLM 驱动的替换管道来应对这一挑战，该管道通过用真实的、类型一致的代理替换个人身份信息 (PII) 来匿名化文本。该方法完全在组织边界内使用本地法学硕士执行，可防止数据流出，同时保持流畅性和任务相关语义。我们对基于操作的对话数据集进行了系统性、多指标、跨技术的评估，以行业标准（Microsoft Presidio 和 Google DLP）和最先进的方法（ZSTS，仅编辑和编辑加替换变体）为基准。我们的协议通过在净化文本上微调紧凑编码器（BERT+LoRA）获得的生命周期就绪标准来联合测量隐私、语义效用和隐私下的可训练性。此外，我们通过在回答 LLM 之前插入本地匿名层并评估其响应的质量来评估代理问答性能。这个中间的、类型保留的替换阶段可确保敏感内容不会暴露给第三方 API，从而能够在不影响机密性的情况下负责任地部署问答代理。我们的方法实现了最先进的隐私、最小的主题漂移、强大的事实实用性和低可训练性损失，在隐私-实用性-可训练性组合前沿上优于基于规则的方法和命名实体识别（NER）基线和 ZSTS 变体。这些结果表明，本地法学硕士替代产生了匿名语料库，该语料库既负责使用又具有操作价值：对于代理管道来说是安全的，并且适合下游微调，且降级有限。

Title: Alignment Makes Language Models Normative, Not Descriptive

• Authors: Eilam Shapira, Moshe Tennenholtz, Roi Reichart
• Subjects: cs.CL, cs.AI, cs.GT
• Abstract URL: https://arxiv.org/abs/2603.17218
• Pdf URL: https://arxiv.org/pdf/2603.17218
• Copy Paste: [[2603.17218]] Alignment Makes Language Models Normative, Not Descriptive(https://arxiv.org/abs/2603.17218)
• Keywords: language model, prompt
• Abstract: Post-training alignment optimizes language models to match human preference signals, but this objective is not equivalent to modeling observed human behavior. We compare 120 base-aligned model pairs on more than 10,000 real human decisions in multi-round strategic games - bargaining, persuasion, negotiation, and repeated matrix games. In these settings, base models outperform their aligned counterparts in predicting human choices by nearly 10:1, robustly across model families, prompt formulations, and game configurations. This pattern reverses, however, in settings where human behavior is more likely to follow normative predictions: aligned models dominate on one-shot textbook games across all 12 types tested and on non-strategic lottery choices - and even within the multi-round games themselves, at round one, before interaction history develops. This boundary-condition pattern suggests that alignment induces a normative bias: it improves prediction when human behavior is relatively well captured by normative solutions, but hurts prediction in multi-round strategic settings, where behavior is shaped by descriptive dynamics such as reciprocity, retaliation, and history-dependent adaptation. These results reveal a fundamental trade-off between optimizing models for human use and using them as proxies for human behavior.
• 摘要：训练后对齐优化语言模型以匹配人类偏好信号，但此目标并不等同于对观察到的人类行为进行建模。我们在多轮战略博弈（讨价还价、说服、谈判和重复矩阵博弈）中对 10,000 多个真实人类决策进行了 120 个碱基对齐模型对的比较。在这些设置中，基础模型在预测人类选择方面的表现比一致的模型高出近 10:1，并且在模型系列、提示公式和游戏配置方面表现强劲。然而，在人类行为更有可能遵循规范预测的环境中，这种模式会发生逆转：对齐模型在所有 12 种测试类型的一次性教科书游戏和非战略性彩票选择中占主导地位，甚至在多轮游戏本身中，在第一轮，在交互历史发展之前。这种边界条件模式表明，对齐会产生规范偏差：当规范解决方案相对较好地捕获人类行为时，它可以改善预测，但会损害多轮战略环境中的预测，在多轮战略环境中，行为是由互惠、报复和历史依赖适应等描述性动态塑造的。这些结果揭示了优化人类使用模型和将其用作人类行为代理之间的基本权衡。

Title: TharuChat: Bootstrapping Large Language Models for a Low-Resource Language via Synthetic Data and Human Validation

• Authors: Prajwal Panth, Agniva Maiti
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.17220
• Pdf URL: https://arxiv.org/pdf/2603.17220
• Copy Paste: [[2603.17220]] TharuChat: Bootstrapping Large Language Models for a Low-Resource Language via Synthetic Data and Human Validation(https://arxiv.org/abs/2603.17220)
• Keywords: language model, llm, prompt, chat
• Abstract: The rapid proliferation of Large Language Models (LLMs) has created a profound digital divide, effectively excluding indigenous languages of the Global South from the AI revolution. The Tharu language, an Indo-Aryan vernacular spoken by approximately 1.7 million people across the Terai belt of Nepal and India, exemplifies this crisis. Despite a rich oral tradition, Tharu suffers from severe data scarcity and linguistic fragmentation, causing state-of-the-art multilingual models to routinely "hallucinate" or default to dominant high-resource neighbors like Hindi and Nepali due to contamination in pre-training corpora. This paper presents Tharu-LLaMA (3B), a specialized instruction-following model designed to address this exclusion. We introduce TharuChat, a novel dataset constructed via a LLM-to-Human bootstrapping pipeline. We utilized prompt-engineered Gemini models, fed with Rana Tharu grammar and folklore, to synthesize training data. Unlike curated gold-standard corpora, TharuChat reflects the noisy, heterogeneous linguistic reality of the region: it is predominantly anchored in Rana Tharu (~70%) while integrating elements of Dangaura and Kochila dialects. We provide a transparent analysis of the dataset's limitations, including dialectal code-mixing and residual Awadhi/Hindi influence. Through a rigorous empirical ablation study, we demonstrate that despite these imperfections, small-scale synthetic data is highly effective, increasing the dataset volume from 25% to 100% results in a linear reduction in perplexity from 6.42 to 2.88. The resulting model serves as a proof-of-concept for the preservation of under-resourced Himalayan languages via generative AI, achievable on consumer-grade hardware.
• 摘要：大型语言模型（LLM）的迅速扩散造成了深刻的数字鸿沟，有效地将南半球的本土语言排除在人工智能革命之外。塔鲁语是一种印度-雅利安方言，尼泊尔和印度特莱地区约有 170 万人使用这种语言，它就是这场危机的例证。尽管塔鲁语有着丰富的口头传统，但塔鲁语却面临着严重的数据稀缺和语言碎片化问题，导致最先进的多语言模型经常“产生幻觉”，或者由于预训练语料库的污染而默认使用印地语和尼泊尔语等占主导地位的高资源邻居。本文提出了 Tharu-LLaMA (3B)，这是一种专门的指令跟踪模型，旨在解决这种排除问题。我们介绍 TharuChat，这是一个通过 LLM-to-Human bootstrapping pipeline 构建的新颖数据集。我们利用即时设计的 Gemini 模型，结合 Rana Tharu 语法和民间传说来合成训练数据。与精心策划的黄金标准语料库不同，TharuChat 反映了该地区嘈杂、异构的语言现实：它主要以 Rana Tharu（约 70%）为基础，同时融合了 Dangaura 和 Kochila 方言的元素。我们对数据集的局限性进行透明分析，包括方言代码混合和残余阿瓦迪语/印地语影响。通过严格的实证消融研究，我们证明，尽管存在这些缺陷，小规模合成数据仍然非常有效，将数据集容量从 25% 增加到 100%，导致困惑度从 6.42 线性降低到 2.88。由此产生的模型可以作为通过生成式人工智能保护资源贫乏的喜马拉雅语言的概念验证，这可以在消费级硬件上实现。

Title: Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models

• Authors: Xiutian Zhao, Ismail Rasim Ulgen, Philipp Koehn, Björn Schuller, Berrak Sisman
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2603.17231
• Pdf URL: https://arxiv.org/pdf/2603.17231
• Copy Paste: [[2603.17231]] Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models(https://arxiv.org/abs/2603.17231)
• Keywords: language model, hallucination
• Abstract: Large audio-language models (LALMs) can produce expressive speech, yet reliable emotion control remains elusive: conversions often miss the target affect and may degrade linguistic fidelity through refusals, hallucinations, or paraphrase. We present, to our knowledge, the first neuron-level study of emotion control in speech-generative LALMs and demonstrate that compact emotion-sensitive neurons (ESNs) are causally actionable, enabling training-free emotion steering at inference time. ESNs are identified via success-filtered activation aggregation enforcing both emotion realization and content preservation. Across three LALMs (Qwen2.5-Omni-7B, MiniCPM-o 4.5, Kimi-Audio), ESN interventions yield emotion-specific gains that generalize to unseen speakers and are supported by automatic and human evaluation. Controllability depends on selector design, mask sparsity, filtering, and intervention strength. Our results establish a mechanistic framework for training-free emotion control in speech generation.
• 摘要：大型音频语言模型 (LALM) 可以产生富有表现力的语音，但可靠的情绪控制仍然难以实现：转换常常会错过目标情感，并可能因拒绝、幻觉或释义而降低语言保真度。据我们所知，我们首次对语音生成 LALM 中的情绪控制进行神经元级研究，并证明紧凑的情绪敏感神经元 (ESN) 具有因果操作性，能够在推理时实现无需训练的情绪控制。 ESN 通过成功过滤的激活聚合来识别，强制执行情感实现和内容保留。在三个 LALM（Qwen2.5-Omni-7B、MiniCPM-o 4.5、Kimi-Audio）中，ESN 干预产生了特定于情绪的增益，这些增益可推广到看不见的说话者，并得到自动和人工评估的支持。可控性取决于选择器设计、掩模稀疏性、过滤和干预强度。我们的结果为语音生成中的免训练情绪控制建立了一个机制框架。

Title: From Words to Worlds: Benchmarking Cross-Cultural Cultural Understanding in Machine Translation

• Authors: Bangju Han, Yingqi Wang, Huang Qing, Tiyuan Li, Fengyi Yang, Ahtamjan Ahmat, Abibulla Atawulla, Yating Yang, Xi Zhou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.17303
• Pdf URL: https://arxiv.org/pdf/2603.17303
• Copy Paste: [[2603.17303]] From Words to Worlds: Benchmarking Cross-Cultural Cultural Understanding in Machine Translation(https://arxiv.org/abs/2603.17303)
• Keywords: language model
• Abstract: Culture-expressions, such as idioms, slang, and culture-specific items (CSIs), are pervasive in natural language and encode meanings that go beyond literal linguistic form. Accurately translating such expressions remains challenging for machine translation systems. Despite this, existing benchmarks remain fragmented and do not provide a systematic framework for evaluating translation performance on culture-loaded expressions. To address this gap, we introduce CulT-Eval, a benchmark designed to evaluate how models handle different types of culturally grounded expressions. CulT-Eval comprises over 7,959 carefully curated instances spanning multiple types of culturally grounded expressions, with a comprehensive error taxonomy covering culturally grounded expressions. Through extensive evaluation of large language models and detailed analysis, we identify recurring and systematic failure modes that are not adequately captured by existing automatic metrics. Accordingly, we propose a complementary evaluation metric that targets culturally induced meaning deviations overlooked by standard MT metrics. The results indicate that current models struggle to preserve culturally grounded meaning and to capture the cultural and contextual nuances essential for accurate translation. Our benchmark and code are available at this https URL.
• 摘要：文化表达，例如习语、俚语和文化特定项目 (CSI)，在自然语言中普遍存在，并且编码的含义超出了字面语言形式。对于机器翻译系统来说，准确翻译此类表达仍然是一个挑战。尽管如此，现有的基准仍然分散，并且没有提供评估文化负载表达翻译性能的系统框架。为了解决这一差距，我们引入了 CulT-Eval，这是一个旨在评估模型如何处理不同类型的文化基础表达方式的基准。 CulT-Eval 包含超过 7,959 个精心策划的实例，涵盖多种类型的文化基础表达，并具有涵盖文化基础表达的全面错误分类法。通过对大型语言模型的广泛评估和详细分析，我们识别了现有自动指标无法充分捕获的重复出现的系统性故障模式。因此，我们提出了一种补充评估指标，针对标准机器翻译指标所忽视的文化引起的意义偏差。结果表明，当前的模型很难保留基于文化的意义，并捕捉准确翻译所必需的文化和语境的细微差别。我们的基准测试和代码可在此 https URL 中找到。

Title: Beyond bouba/kiki: Multidimensional semantic signals are deeply woven into the fabric of natural language

• Authors: Gexin Zhao
• Subjects: cs.CL, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2603.17306
• Pdf URL: https://arxiv.org/pdf/2603.17306
• Copy Paste: [[2603.17306]] Beyond bouba/kiki: Multidimensional semantic signals are deeply woven into the fabric of natural language(https://arxiv.org/abs/2603.17306)
• Keywords: language model
• Abstract: A foundational assumption in linguistics holds that the relationship between a word's sound and its meaning is arbitrary. Accumulating evidence from sound symbolism challenges this view, yet no study has systematically mapped the multidimensional semantic profile of every phonological unit within a language. Here we show that individual letter-phonemes in English carry structured, multidimensional semantic signals. Using a minimal-pair paradigm spanning all 220 pairwise letter contrasts, three large language models independently recover consistent phoneme-meaning associations across nine perceptual dimensions. These associations are systematically predicted by articulatory-phonetic features, with manner and place of articulation mapping onto distinct semantic dimensions. Behavioral data from English speakers confirm these patterns at rates well above chance (80.8%), and preliminary cross-linguistic evidence from five typologically diverse languages suggests that core mappings generalize beyond English. Our findings indicate that sound-meaning iconicity is not an occasional curiosity but a pervasive, structured property of the phonological signal, one so systematic that large language models recover it when given only text input, without exposure to speech or articulation during the task.
• 摘要：语言学的一个基本假设认为，单词的声音与其含义之间的关系是任意的。从声音象征主义中积累的证据挑战了这一观点，但还没有研究系统地映射语言中每个语音单元的多维语义特征。在这里，我们展示了英语中的各个字母音素携带结构化的多维语义信号。使用涵盖所有 220 个成对字母对比的最小对范式，三个大型语言模型独立地恢复跨九个感知维度的一致音素含义关联。这些关联是通过发音语音特征系统预测的，发音方式和位置映射到不同的语义维度。来自英语使用者的行为数据证实了这些模式的概率远远高于偶然性 (80.8%)，来自五种不同类型语言的初步跨语言证据表明，核心映射可以推广到英语以外的领域。我们的研究结果表明，声音意义象似性并不是一种偶然的好奇心，而是语音信号的一种普遍的、结构化的属性，这种属性如此系统化，以至于大型语言模型在仅给出文本输入时就可以恢复它，而在任务期间无需接触语音或发音。

Title: Grid Spatial Understanding: A Dataset for Textual Spatial Reasoning over Grids, Embodied Settings, and Coordinate Structures

• Authors: Risham Sidhu, Julia Hockenmaier
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17333
• Pdf URL: https://arxiv.org/pdf/2603.17333
• Copy Paste: [[2603.17333]] Grid Spatial Understanding: A Dataset for Textual Spatial Reasoning over Grids, Embodied Settings, and Coordinate Structures(https://arxiv.org/abs/2603.17333)
• Keywords: llm, agent
• Abstract: We introduce GSU, a text-only grid dataset to evaluate the spatial reasoning capabilities of LLMs over 3 core tasks: navigation, object localization, and structure composition. By forgoing visual inputs, isolating spatial reasoning from perception, we show that while most models grasp basic grid concepts, they struggle with frames of reference relative to an embodied agent and identifying 3D shapes from coordinate lists. We also find that exposure to a visual modality does not provide a generalizable understanding of 3D space that VLMs are able to utilize for these tasks. Finally, we show that while the very latest frontier models can solve the provided tasks (though harder variants may still stump them), fully fine-tuning a small LM or LORA fine-tuning a small LLM show potential to match frontier model performance, suggesting an avenue for specialized embodied agents.
• 摘要：我们引入了 GSU，一个纯文本网格数据集，用于评估法学硕士在 3 个核心任务上的空间推理能力：导航、对象定位和结构组成。通过放弃视觉输入，将空间推理与感知隔离，我们表明，虽然大多数模型掌握了基本的网格概念，但它们在与具体代理相关的参考框架以及从坐标列表中识别 3D 形状方面遇到了困难。我们还发现，暴露于视觉模式并不能提供对 VLM 能够用于这些任务的 3D 空间的普遍理解。最后，我们表明，虽然最新的前沿模型可以解决所提供的任务（尽管更难的变体可能仍然难倒它们），但完全微调小型 LM 或 LORA 微调小型 LLM 显示出匹配前沿模型性能的潜力，这为专门的实体代理提供了一条途径。

Title: PACE-RAG: Patient-Aware Contextual and Evidence-based Policy RAG for Clinical Drug Recommendation

• Authors: Chaeyoung Huh, Hyunmin Hwang, Jung Hwan Shin, Jinse Park, Jong Chul Ye
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17356
• Pdf URL: https://arxiv.org/pdf/2603.17356
• Copy Paste: [[2603.17356]] PACE-RAG: Patient-Aware Contextual and Evidence-based Policy RAG for Clinical Drug Recommendation(https://arxiv.org/abs/2603.17356)
• Keywords: llm
• Abstract: Drug recommendation requires a deep understanding of individual patient context, especially for complex conditions like Parkinson's disease. While LLMs possess broad medical knowledge, they fail to capture the subtle nuances of actual prescribing patterns. Existing RAG methods also struggle with these complexities because guideline-based retrieval remains too generic and similar-patient retrieval often replicates majority patterns without accounting for the unique clinical nuances of individual patients. To bridge this gap, we propose PACE-RAG (Patient-Aware Contextual and Evidence-based Policy RAG), a novel framework designed to synthesize individual patient context with the prescribing tendencies of similar cases. By analyzing treatment patterns tailored to specific clinical signals, PACE-RAG identifies optimal prescriptions and generates an explainable clinical summary. Evaluated on a Parkinson's cohort and the MIMIC-IV benchmark using Llama-3.1-8B and Qwen3-8B, PACE-RAG achieved state-of-the-art performance, reaching F1 scores of 80.84% and 47.22%, respectively. These results validate PACE-RAG as a robust, clinically grounded solution for personalized decision support. Our code is available at: this https URL.
• 摘要：药物推荐需要深入了解个体患者的情况，特别是对于帕金森病等复杂疾病。虽然法学硕士拥有广泛的医学知识，但他们无法捕捉实际处方模式的细微差别。现有的 RAG 方法也难以应对这些复杂性，因为基于指南的检索仍然过于通用，并且相似患者检索通常会复制大多数模式，而不会考虑个体患者独特的临床细微差别。为了弥补这一差距，我们提出了 PACE-RAG（患者感知背景和基于证据的政策 RAG），这是一种新颖的框架，旨在综合个体患者背景与类似病例的处方倾向。通过分析针对特定临床信号的治疗模式，PACE-RAG 确定最佳处方并生成可解释的临床总结。使用 Llama-3.1-8B 和 Qwen3-8B 对帕金森病队列和 MIMIC-IV 基准进行评估，PACE-RAG 取得了最先进的性能，F1 分数分别达到 80.84% 和 47.22%。这些结果验证了 PACE-RAG 是一种强大的、基于临床的个性化决策支持解决方案。我们的代码位于：此 https URL。

Title: SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems

• Authors: Rima Hazra, Bikram Ghuku, Ilona Marchenko, Yaroslava Tokarieva, Sayan Layek, Somnath Banerjee, Julia Stoyanovich, Mykola Pechenizkiy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17373
• Pdf URL: https://arxiv.org/pdf/2603.17373
• Copy Paste: [[2603.17373]] SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems(https://arxiv.org/abs/2603.17373)
• Keywords: language model, llm
• Abstract: Large language models are rapidly being deployed as AI tutors, yet current evaluation paradigms assess problem-solving accuracy and generic safety in isolation, failing to capture whether a model is simultaneously pedagogically effective and safe across student-tutor interaction. We argue that tutoring safety is fundamentally different from conventional LLM safety: the primary risk is not toxic content but the quiet erosion of learning through answer over-disclosure, misconception reinforcement, and the abdication of scaffolding. To systematically study this failure mode, we introduce SafeTutors, a benchmark that jointly evaluates safety and pedagogy across mathematics, physics, and chemistry. SafeTutors is organized around a theoretically grounded risk taxonomy comprising 11 harm dimensions and 48 sub-risks drawn from learning-science literature. We uncover that all models show broad harm; scale doesn't reliably help; and multi-turn dialogue worsens behavior, with pedagogical failures rising from 17.7% to 77.8%. Harms also vary by subject, so mitigations must be discipline-aware, and single-turn "safe/helpful" results can mask systematic tutor failure over extended interaction.
• 摘要：大型语言模型正在迅速被部署为人工智能导师，但当前的评估范式孤立地评估问题解决的准确性和一般安全性，无法捕捉到模型在学生与导师的互动中是否同时在教学上有效和安全。我们认为，辅导安全与传统的法学硕士安全有根本不同：主要风险不是有毒内容，而是通过答案过度披露、误解强化和放弃脚手架而对学习造成的无声侵蚀。为了系统地研究这种故障模式，我们引入了 SafeTutors，这是一个联合评估数学、物理和化学安全性和教学法的基准。 SafeTutors 围绕一个基于理论的风险分类法进行组织，该分类法包括来自学习科学文献的 11 个危害维度和 48 个子风险。我们发现所有模型都表现出广泛的危害；规模并没有可靠的帮助；多轮对话使行为恶化，教学失败率从 17.7% 上升至 77.8%。危害也因科目而异，因此缓解措施必须具有纪律意识，单轮“安全/有用”结果可能会掩盖导师在长时间互动中的系统性失败。

Title: Argument Reconstruction as Supervision for Critical Thinking in LLMs

• Authors: Hyun Ryu, Gyouk Chu, Gregor Betz, Eunho Yang, Carolyn Rose, Sean Welleck
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17432
• Pdf URL: https://arxiv.org/pdf/2603.17432
• Copy Paste: [[2603.17432]] Argument Reconstruction as Supervision for Critical Thinking in LLMs(https://arxiv.org/abs/2603.17432)
• Keywords: llm
• Abstract: To think critically about arguments, human learners are trained to identify, reconstruct, and evaluate arguments. Argument reconstruction is especially important because it makes an argument's underlying inferences explicit. However, it remains unclear whether LLMs can similarly enhance their critical thinking ability by learning to reconstruct arguments. To address this question, we introduce a holistic framework with three contributions. We (1) propose an engine that automatically reconstructs arbitrary arguments (GAAR), (2) synthesize a new high-quality argument reconstruction dataset (Arguinas) using the GAAR engine, and (3) investigate whether learning argument reconstruction benefits downstream critical thinking tasks. Our experimental results show that, across seven critical thinking tasks, models trained to learn argument reconstruction outperform models that do not, with the largest performance gains observed when training on the proposed Arguinas dataset. The source code and dataset will be publicly available.
• 摘要：为了批判性地思考论点，人类学习者接受了识别、重构和评估论点的训练。论证重构尤其重要，因为它使论证的基本推论变得明确。然而，目前尚不清楚法学硕士是否可以通过学习重构论点来同样提高他们的批判性思维能力。为了解决这个问题，我们引入了一个包含三个贡献的整体框架。我们（1）提出了一种自动重建任意论点（GAAR）的引擎，（2）使用 GAAR 引擎合成新的高质量论点重建数据集（Arguinas），以及（3）研究学习论点重建是否有利于下游批判性思维任务。我们的实验结果表明，在七个批判性思维任务中，经过训练来学习论证重建的模型优于不学习论证重建的模型，在拟议的 Arguinas 数据集上进行训练时观察到最大的性能提升。源代码和数据集将公开。

Title: TRiMS: Real-Time Tracking of Minimal Sufficient Length for Efficient Reasoning via RL

• Authors: Tingcheng Bian, Jinchang Luo, Mingquan Cheng, Jinyu Zhang, Xiaoling Xia, Ni Li, Yan Tao, Haiwei Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17449
• Pdf URL: https://arxiv.org/pdf/2603.17449
• Copy Paste: [[2603.17449]] TRiMS: Real-Time Tracking of Minimal Sufficient Length for Efficient Reasoning via RL(https://arxiv.org/abs/2603.17449)
• Keywords: language model, chain-of-thought
• Abstract: Large language models achieve breakthroughs in complex reasoning via long chain-of-thought sequences. However, this often leads to severe reasoning inflation, causing substantial computational redundancy. To maximize Intelligence per Token, we introduce a theoretical metric, MSL-Minimal Sufficient Length. MSL rigorously characterizes the shortest reasoning length that preserves answer correctness. We provide a recursive definition based on independently sampled sequences and prove the existence of its limit, establishing the first measurable lower bound for reasoning-chain compression. Building on an analysis of mainstream CoT compression strategies, we identify key structural factors enabling a model to approach MSL. Based on these insights, we propose TRiMS which employs the GRPO algorithm in conjunction with MSL-based estimation during training, while mitigating instabilities during the training process through dynamic batch aggregation and advantage computation using batch-level standard deviation. TRiMS achieves over 80% CoT token reduction with a minor accuracy boost across all benchmarks.
• 摘要：大型语言模型通过长的思维链序列实现复杂推理的突破。然而，这通常会导致严重的推理膨胀，导致大量的计算冗余。为了最大化每个代币的智能，我们引入了一个理论度量，MSL-最小足够长度。 MSL 严格描述了保持答案正确性的最短推理长度。我们提供了基于独立采样序列的递归定义，并证明了其极限的存在，为推理链压缩建立了第一个可测量的下界。基于对主流 CoT 压缩策略的分析，我们确定了使模型能够接近 MSL 的关键结构因素。基于这些见解，我们提出了 TRiMS，它在训练期间采用 GRPO 算法与基于 MSL 的估计相结合，同时通过动态批次聚合和使用批次级标准差的优势计算来减轻训练过程中的不稳定性。 TRiMS 实现了超过 80% 的 CoT 代币减少，并且在所有基准测试中准确性略有提升。

Title: Humans and transformer LMs: Abstraction drives language learning

• Authors: Jasper Jian, Christopher D. Manning
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17475
• Pdf URL: https://arxiv.org/pdf/2603.17475
• Copy Paste: [[2603.17475]] Humans and transformer LMs: Abstraction drives language learning(https://arxiv.org/abs/2603.17475)
• Keywords: language model, gpt
• Abstract: Categorization is a core component of human linguistic competence. We investigate how a transformer-based language model (LM) learns linguistic categories by comparing its behaviour over the course of training to behaviours which characterize abstract feature-based and concrete exemplar-based accounts of human language acquisition. We investigate how lexical semantic and syntactic categories emerge using novel divergence-based metrics that track learning trajectories using next-token distributions. In experiments with GPT-2 small, we find that (i) when a construction is learned, abstract class-level behaviour is evident at earlier steps than lexical item-specific behaviour, and (ii) that different linguistic behaviours emerge abruptly in sequence at different points in training, revealing that abstraction plays a key role in how LMs learn. This result informs the models of human language acquisition that LMs may serve as an existence proof for.
• 摘要：分类是人类语言能力的核心组成部分。我们通过将训练过程中的行为与表征人类语言习得的基于抽象特征和基于具体范例的描述的行为进行比较，研究基于变压器的语言模型（LM）如何学习语言类别。我们研究词汇语义和句法类别如何使用新颖的基于分歧的指标出现，这些指标使用下一个令牌分布跟踪学习轨迹。在 GPT-2 Small 的实验中，我们发现（i）当学习一个结构时，抽象类级别的行为比词汇项特定的行为在更早的步骤中明显，并且（ii）不同的语言行为在训练中的不同点按顺序突然出现，这表明抽象在 LM 的学习方式中发挥着关键作用。这一结果为人类语言习得模型提供了信息，语言模型可以作为存在证明。

Title: Learning When to Attend: Conditional Memory Access for Long-Context LLMs

• Authors: Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato, Elvis Nunez, Matthew Trager, Wei Xia, Stefano Soatto
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.17484
• Pdf URL: https://arxiv.org/pdf/2603.17484
• Copy Paste: [[2603.17484]] Learning When to Attend: Conditional Memory Access for Long-Context LLMs(https://arxiv.org/abs/2603.17484)
• Keywords: language model, llm
• Abstract: Language models struggle to generalize beyond pretraining context lengths, limiting long-horizon reasoning and retrieval. Continued pretraining on long-context data can help but is expensive due to the quadratic scaling of Attention. We observe that most tokens do not require (Global) Attention over the entire sequence and can rely on local context. Based on this, we propose L2A (Learning To Attend), a layer that enables conditional (token-wise) long-range memory access by deciding when to invoke global attention. We evaluate L2A on Qwen 2.5 and Qwen 3 models, extending their effective context length from 32K to 128K tokens. L2A matches the performance of standard long-context training to within 3% while skipping Global Attention for $\sim$80% of tokens, outperforming prior baselines. We also design custom Triton kernels to efficiently implement this token-wise conditional Attention on GPUs, achieving up to $\sim$2x improvements in training throughput and time-to-first-token over FlashAttention. Moreover, L2A enables post-training pruning of highly sparse Global Attention layers, reducing KV cache memory by up to 50% with negligible performance loss.
• 摘要：语言模型很难泛化到预训练上下文长度之外，从而限制了长期推理和检索。对长上下文数据进行持续的预训练可以有所帮助，但由于注意力的二次缩放，成本高昂。我们观察到大多数标记不需要对整个序列进行（全局）注意力，并且可以依赖于本地上下文。基于此，我们提出了 L2A（学习参与），该层通过决定何时调用全局注意力来实现条件（令牌方式）远程内存访问。我们在 Qwen 2.5 和 Qwen 3 模型上评估 L2A，将其有效上下文长度从 32K 扩展到 128K 令牌。 L2A 将标准长上下文训练的性能匹配到 3% 以内，同时跳过 $\sim$80% 令牌的全局注意力，优于之前的基线。我们还设计了定制的 Triton 内核，以在 GPU 上高效地实现这种基于 token 的条件 Attention，与 FlashAttention 相比，在训练吞吐量和首次生成 token 的时间方面实现了高达 $\sim$2 倍的改进。此外，L2A 可以对高度稀疏的全局注意力层进行训练后修剪，将 KV 缓存内存减少高达 50%，而性能损失可以忽略不计。

Title: Inducing Epistemological Humility in Large Language Models: A Targeted SFT Approach to Reducing Hallucination

• Authors: Cem Uluoglakci, Tugba Taskaya Temizel
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17504
• Pdf URL: https://arxiv.org/pdf/2603.17504
• Copy Paste: [[2603.17504]] Inducing Epistemological Humility in Large Language Models: A Targeted SFT Approach to Reducing Hallucination(https://arxiv.org/abs/2603.17504)
• Keywords: language model, llm, hallucination
• Abstract: Large language models (LLMs) often hallucinate, producing fluent but false information, partly because supervised fine-tuning (SFT) implicitly rewards always responding. We introduce $\textit{HypoTermInstruct}$, an SFT dataset (31,487 responses for 11,151 questions) designed to teach models epistemological humility-the ability to recognize the limits of their own knowledge and admit uncertainty. This is achieved through questions about non-existent "hypothetical" terms. We also release $\textit{HypoTermQA-Enhanced}$, a benchmark for hallucination tendency strengthened through multiple validations. We conducted 800 controlled LoRA SFT runs across $\textit{Llama3.1-8B}$ and $\textit{Gemma3-4B}$ (base and instruct), testing 100 fine-tuning configurations with paired controls. Our results demonstrate that replacing generic instruction data with $\textit{HypoTermInstruct}$ significantly improves the HypoTerm Score (median increases of 0.19% to 25.91%) and FactScore (+0.39% to +0.86%), while maintaining stable performance on MMLU (minimal decreases of 0.26% to 0.35%). Our work demonstrates that targeted, high-quality SFT data teaching meta-cognitive skills can effectively reduce hallucination without preference/RL pipelines, providing mechanistic insights and a practical path toward more reliable AI systems.
• 摘要：大型语言模型（LLM）经常产生幻觉，产生流畅但错误的信息，部分原因是监督微调（SFT）隐式奖励总是做出响应。我们引入了 $\textit{HypoTermInstruct}$，一个 SFT 数据集（11,151 个问题的 31,487 个回答），旨在教授模型认识论谦逊——认识到自己知识的局限性并承认不确定性的能力。这是通过询问不存在的“假设”术语来实现的。我们还发布了 $\textit{HypoTermQA-Enhanced}$，这是通过多重验证强化的幻觉倾向基准。我们在 $\textit{Llama3.1-8B}$ 和 $\textit{Gemma3-4B}$（基础和指令）上进行了 800 次受控 LoRA SFT 运行，使用配对控件测试了 100 个微调配置。我们的结果表明，用 $\textit{HypoTermInstruct}$ 替换通用指令数据可显着提高 HypoTerm Score（中位数增加 0.19% 至 25.91%）和 FactScore（+0.39% 至 +0.86%），同时保持 MMLU 上的稳定性能（最小下降 0.26% 至 0.35%）。我们的工作表明，有针对性的、高质量的 SFT 数据教授元认知技能可以有效减少幻觉，无需偏好/强化学习管道，提供机制见解和通往更可靠的人工智能系统的实用路径。

Title: Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality

• Authors: Mengyu Bu, Yang Feng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17512
• Pdf URL: https://arxiv.org/pdf/2603.17512
• Copy Paste: [[2603.17512]] Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality(https://arxiv.org/abs/2603.17512)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) exhibit strong general intelligence, yet their multilingual performance remains highly imbalanced. Although LLMs encode substantial cross-lingual knowledge in a unified semantic space, they often struggle to reliably interface this knowledge with low-resource or unseen languages. Fortunately, pretrained encoder-decoder translation models already possess balanced multilingual capability, suggesting a natural complement to LLMs. In this work, we propose XBridge, a compositional encoder-LLM-decoder architecture that offloads multilingual understanding and generation to external pretrained translation models, while preserving the LLM as an English-centric core for general knowledge processing. To address the resulting representation misalignment across models, we introduce lightweight cross-model mapping layers and an optimal transport-based alignment objective, enabling fine-grained semantic consistency for multilingual generation. Experiments on four LLMs across multilingual understanding, reasoning, summarization, and generation indicate that XBridge outperforms strong baselines, especially on low-resource and previously unseen languages, without retraining the LLM.
• 摘要：大型语言模型（LLM）表现出强大的通用智能，但其多语言性能仍然高度不平衡。尽管法学硕士在统一的语义空间中编码了大量的跨语言知识，但他们通常很难将这些知识与资源匮乏或看不见的语言可靠地连接起来。幸运的是，预训练的编码器-解码器翻译模型已经具备平衡的多语言能力，这表明对法学硕士的天然补充。在这项工作中，我们提出了 XBridge，一种组合式编码器-LLM-解码器架构，它将多语言理解和生成卸载到外部预训练翻译模型，同时保留 LLM 作为一般知识处理的以英语为中心的核心。为了解决跨模型产生的表示不一致问题，我们引入了轻量级跨模型映射层和基于传输的最佳对齐目标，从而实现多语言生成的细粒度语义一致性。对四个跨多语言理解、推理、总结和生成的 LLM 进行的实验表明，XBridge 的性能优于强大的基线，特别是在资源匮乏和以前未见过的语言上，而无需重新训练 LLM。

Title: Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions

• Authors: Madhav S. Baidya, S. S. Baidya, Chirag Chawla
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.17522
• Pdf URL: https://arxiv.org/pdf/2603.17522
• Copy Paste: [[2603.17522]] Detecting the Machine: A Comprehensive Benchmark of AI-Generated Text Detectors Across Architectures, Domains, and Adversarial Conditions(https://arxiv.org/abs/2603.17522)
• Keywords: language model, gpt, llm, prompt, chat
• Abstract: The rapid proliferation of large language models (LLMs) has created an urgent need for robust and generalizable detectors of machine-generated text. Existing benchmarks typically evaluate a single detector on a single dataset under ideal conditions, leaving open questions about cross-domain transfer, cross-LLM generalization, and adversarial robustness. We present a comprehensive benchmark evaluating diverse detection approaches across two corpora: HC3 (23,363 human-ChatGPT pairs) and ELI5 (15,000 human-Mistral-7B pairs). Methods include classical classifiers, fine-tuned transformer encoders (BERT, RoBERTa, ELECTRA, DistilBERT, DeBERTa-v3), a CNN, an XGBoost stylometric model, perplexity-based detectors, and LLM-as-detector prompting. Results show that transformer models achieve near-perfect in-distribution performance but degrade under domain shift. The XGBoost stylometric model matches performance while remaining interpretable. LLM-based detectors underperform and are affected by generator-detector identity bias. Perplexity-based methods exhibit polarity inversion, with modern LLM outputs showing lower perplexity than human text, but remain effective when corrected. No method generalizes robustly across domains and LLM sources.
• 摘要：大型语言模型 (LLM) 的快速增长迫切需要强大且可泛化的机器生成文本检测器。现有的基准通常在理想条件下评估单个数据集上的单个检测器，从而留下了有关跨域转移、跨 LLM 泛化和对抗鲁棒性的悬而未决的问题。我们提出了一个综合基准，评估两个语料库的不同检测方法：HC3（23,363 个人类-ChatGPT 对）和 ELI5（15,000 个人类-Mistral-7B 对）。方法包括经典分类器、微调 Transformer 编码器（BERT、RoBERTa、ELECTRA、DistilBERT、DeBERTa-v3）、CNN、XGBoost 风格测量模型、基于困惑度的检测器和 LLM 作为检测器提示。结果表明，变压器模型实现了近乎完美的分布性能，但在域转移下性能下降。 XGBoost 风格测量模型在保持可解释性的同时满足性能要求。基于 LLM 的检测器表现不佳，并受到生成器-检测器身份偏差的影响。基于困惑度的方法表现出极性反转，现代法学硕士输出显示出比人类文本更低的困惑度，但在纠正后仍然有效。没有一种方法可以跨领域和法学硕士来源稳健地推广。

Title: Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition

• Authors: Yuxiang Mei, Delai Qiu, Shengping Liu, Jiaen Liang, Yanhua Long
• Subjects: cs.CL, cs.SD
• Abstract URL: https://arxiv.org/abs/2603.17558
• Pdf URL: https://arxiv.org/pdf/2603.17558
• Copy Paste: [[2603.17558]] Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition(https://arxiv.org/abs/2603.17558)
• Keywords: language model, llm
• Abstract: Speech Large Language Models (Speech-LLMs) have emerged as a powerful approach for automatic speech recognition (ASR) by aligning speech encoders with large language models. However, adapting these systems to multilingual settings with imbalanced data distributions remains challenging. In such scenarios, a stability-plasticity dilemma often arises: fully shared Parameter-Efficient Fine-Tuning (PEFT) can cause negative inter-lingual interference for under-represented languages, while fully language-specific tuning limits the cross-lingual beneficial knowledge transfer needed for low-resource tasks. To address this, we propose Zipper-LoRA, a novel rank-level decoupling framework with three variants (Static, Hard, and Soft) that dynamically synthesizes LoRA updates from shared and language-specific subspaces. By using a lightweight language-conditioned router, Zipper-LoRA dynamically controls the contribution of each subspace at the LoRA rank level, enabling fine-grained sharing where languages are compatible and strict decoupling when conflicts occur. To further stabilize optimization under imbalanced data, we propose a two-stage training strategy with an Initial-B warm start that significantly accelerates convergence. Experiments on a 12-language mixed-resource setting show that Zipper-LoRA consistently outperforms both fully shared and independent baselines, particularly in extremely low-resource scenarios. Moreover, we demonstrate that these gains are robust across both chunked and non-chunked encoder configurations, confirming the framework's reliability for practical, large-scale multilingual ASR. Our code and data will be available at this https URL for reproducibility.
• 摘要：语音大型语言模型 (Speech-LLM) 通过将语音编码器与大型语言模型结合起来，成为自动语音识别 (ASR) 的一种强大方法。然而，使这些系统适应数据分布不平衡的多语言环境仍然具有挑战性。在这种情况下，经常会出现稳定性-可塑性的困境：完全共享的参数高效微调（PEFT）可能会对代表性不足的语言造成负面的语言间干扰，而完全针对特定语言的调优则限制了低资源任务所需的跨语言有益知识转移。为了解决这个问题，我们提出了 Zipper-LoRA，这是一种新颖的等级级解耦框架，具有三种变体（静态、硬和软），可以从共享和特定于语言的子空间动态合成 LoRA 更新。通过使用轻量级语言调节路由器，Zipper-LoRA 在 LoRA 等级级别动态控制每个子空间的贡献，从而实现语言兼容时的细粒度共享，并在发生冲突时实现严格解耦。为了进一步稳定不平衡数据下的优化，我们提出了一种具有初始 B 热启动的两阶段训练策略，可显着加速收敛。在 12 种语言混合资源设置上进行的实验表明，Zipper-LoRA 始终优于完全共享和独立基线，特别是在资源极少的情况下。此外，我们证明这些增益在分块和非分块编码器配置中都是稳健的，证实了该框架对于实际的大规模多语言 ASR 的可靠性。我们的代码和数据将在此 https URL 中提供，以实现可重复性。

Title: KA2L: A Knowledge-Aware Active Learning Framework for LLMs

• Authors: Haoxuan Yin, Bojian Liu, Chen Tang, Yangfan Wang, Lian Yan, Jingchi Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17566
• Pdf URL: https://arxiv.org/pdf/2603.17566
• Copy Paste: [[2603.17566]] KA2L: A Knowledge-Aware Active Learning Framework for LLMs(https://arxiv.org/abs/2603.17566)
• Keywords: language model, llm
• Abstract: Fine-tuning large language models (LLMs) with high-quality knowledge has been shown to enhance their performance effectively. However, there is a paucity of research on the depth of domain-specific knowledge comprehension by LLMs and the application of targeted active learning to improve their expertise. To address this gap, we introduce the Knowledge-Aware Active Learning (KA2L) framework. This framework assesses LLMs' mastery of specific knowledge points to aid in constructing unanswerable or unknowable questions through latent space analysis. This active learning strategy enhances training efficiency by focusing on knowledge the model has yet to master, thereby minimizing redundancy in learning already acquired information. This study innovatively employs a knowledge distribution probing technique to examine the hidden states of specific Transformer layers and identify the distribution of known and unknown knowledge within the LLM. Additionally, a hidden-state decoding method is proposed to generate numerous unknown questions in natural language from the latent knowledge space. In our experiments, we selected nine open-source LLMs to validate the effectiveness of the proposed framework. Results indicate that KA2L not only significantly reduces 50% annotation and computation costs across two open-domain and one vertical-domain dataset but also achieves better performance, offering valuable insights into active learning strategies for LLMs. The code is available at this https URL.
• 摘要：事实证明，利用高质量知识对大型语言模型（LLM）进行微调可以有效地提高其性能。然而，关于法学硕士对特定领域知识理解的深度以及应用有针对性的主动学习来提高其专业知识的研究还很少。为了解决这一差距，我们引入了知识感知主动学习（KA2L）框架。该框架评估法学硕士对特定知识点的掌握程度，以帮助通过潜在空间分析构建无法回答或不可知的问题。这种主动学习策略通过关注模型尚未掌握的知识来提高训练效率，从而最大限度地减少学习已获取信息的冗余。本研究创新性地采用知识分布探测技术来检查特定 Transformer 层的隐藏状态，并识别 LLM 内已知和未知知识的分布。此外，还提出了一种隐藏状态解码方法，用于从潜在知识空间中生成自然语言中的大量未知问题。在我们的实验中，我们选择了九个开源法学硕士来验证所提出框架的有效性。结果表明，KA2L 不仅在两个开放域和一个垂直域数据集上显着降低了 50% 的注释和计算成本，而且还实现了更好的性能，为法学硕士的主动学习策略提供了宝贵的见解。该代码可从此 https URL 获取。

Title: VeriAgent: A Tool-Integrated Multi-Agent System with Evolving Memory for PPA-Aware RTL Code Generation

• Authors: Yaoxiang Wang, Qi Shi, ShangZhan Li, Qingguo Hu, Xinyu Yin, Bo Guo, Xu Han, Maosong Sun, Jinsong Su
• Subjects: cs.CL, cs.PL
• Abstract URL: https://arxiv.org/abs/2603.17613
• Pdf URL: https://arxiv.org/pdf/2603.17613
• Copy Paste: [[2603.17613]] VeriAgent: A Tool-Integrated Multi-Agent System with Evolving Memory for PPA-Aware RTL Code Generation(https://arxiv.org/abs/2603.17613)
• Keywords: llm, agent
• Abstract: LLMs have recently demonstrated strong capabilities in automatic RTL code generation, achieving high syntactic and functional correctness. However, most methods focus on functional correctness while overlooking critical physical design objectives, including Power, Performance, and Area. In this work, we propose a PPA-aware, tool-integrated multi-agent framework for high-quality verilog code generation. Our framework explicitly incorporates EDA tools into a closed-loop workflow composed of a \textit{Programmer Agent}, a \textit{Correctness Agent}, and a \textit{PPA Agent}, enabling joint optimization of functional correctness and physical metrics. To support continuous improvement without model retraining, we introduce an \textit{Evolved Memory Mechanism} that externalizes optimization experience into structured memory nodes. A dedicated memory manager dynamically maintains the memory pool and allows the system to refine strategies based on historical execution trajectories. Extensive experiments demonstrate that our approach achieves strong functional correctness while delivering significant improvements in PPA metrics. By integrating tool-driven feedback with structured and evolvable memory, our framework transforms RTL generation from one-shot reasoning into a continual, feedback-driven optimization process, providing a scalable pathway for deploying LLMs in real-world hardware design flows.
• 摘要：法学硕士最近展示了自动 RTL 代码生成的强大能力，实现了较高的语法和功能正确性。然而，大多数方法都注重功能正确性，而忽略了关键的物理设计目标，包括功耗、性能和面积。在这项工作中，我们提出了一个 PPA 感知、工具集成的多代理框架，用于生成高质量的 verilog 代码。我们的框架明确地将 EDA 工具合并到由 \textit{Programmer Agent}、\textit{Correctness Agent} 和 \textit{PPA Agent} 组成的闭环工作流程中，从而实现功能正确性和物理指标的联合优化。为了支持持续改进而无需模型重新训练，我们引入了 \textit{进化记忆机制}，将优化经验外化到结构化记忆节点中。专用内存管理器动态维护内存池，并允许系统根据历史执行轨迹完善策略。大量实验表明，我们的方法实现了强大的功能正确性，同时显着改进了 PPA 指标。通过将工具驱动的反馈与结构化和可进化的内存相集成，我们的框架将 RTL 生成从一次性推理转变为持续的、反馈驱动的优化过程，为在现实世界的硬件设计流程中部署 LLM 提供了一种可扩展的途径。

Title: Do Language Models Encode Semantic Relations? Probing and Sparse Feature Analysis

• Authors: Andor Diera, Ansgar Scherp
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17624
• Pdf URL: https://arxiv.org/pdf/2603.17624
• Copy Paste: [[2603.17624]] Do Language Models Encode Semantic Relations? Probing and Sparse Feature Analysis(https://arxiv.org/abs/2603.17624)
• Keywords: language model, gpt, llm
• Abstract: Understanding whether large language models (LLMs) capture structured meaning requires examining how they represent concept relationships. In this work, we study three models of increasing scale: Pythia-70M, GPT-2, and Llama 3.1 8B, focusing on four semantic relations: synonymy, antonymy, hypernymy, and hyponymy. We combine linear probing with mechanistic interpretability techniques, including sparse autoencoders (SAE) and activation patching, to identify where these relations are encoded and how specific features contribute to their representation. Our results reveal a directional asymmetry in hierarchical relations: hypernymy is encoded redundantly and resists suppression, while hyponymy relies on compact features that are more easily disrupted by ablation. More broadly, relation signals are diffuse but exhibit stable profiles: they peak in the mid-layers and are stronger in post-residual/MLP pathways than in attention. Difficulty is consistent across models (antonymy easiest, synonymy hardest). Probe-level causality is capacity-dependent: on Llama 3.1, SAE-guided patching reliably shifts these signals, whereas on smaller models the shifts are weak or unstable. Our results clarify where and how reliably semantic relations are represented inside LLMs, and provide a reproducible framework for relating sparse features to probe-level causal evidence.
• 摘要：了解大型语言模型 (LLM) 是否捕获结构化含义需要检查它们如何表示概念关系。在这项工作中，我们研究了三种规模不断增加的模型：Pythia-70M、GPT-2 和 Llama 3.1 8B，重点关注四种语义关系：同义、反义、上义和下义。我们将线性探测与机械可解释性技术（包括稀疏自动编码器（SAE）和激活修补）相结合，以识别这些关系的编码位置以及特定特征如何对其表示做出贡献。我们的结果揭示了层次关系中的方向不对称性：上位被冗余编码并抵抗抑制，而下位依赖于更容易被消融破坏的紧凑特征。更广泛地说，关系信号是分散的，但表现出稳定的轮廓：它们在中间层达到峰值，并且在后残差/MLP 路径中比在注意力中更强。不同模型的难度是一致的（反义词最简单，同义词最难）。探针级因果关系取决于容量：在 Llama 3.1 上，SAE 引导的修补可靠地转移这些信号，而在较小的模型上，转移很弱或不稳定。我们的结果阐明了法学硕士内部语义关系的可靠表示位置和方式，并提供了一个可重复的框架，用于将稀疏特征与探测级因果证据相关联。

Title: Adaptive Guidance for Retrieval-Augmented Masked Diffusion Models

• Authors: Jaemin Kim, Jong Chul Ye
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.17677
• Pdf URL: https://arxiv.org/pdf/2603.17677
• Copy Paste: [[2603.17677]] Adaptive Guidance for Retrieval-Augmented Masked Diffusion Models(https://arxiv.org/abs/2603.17677)
• Keywords: language model, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) improves factual grounding by incorporating external knowledge into language model generation. However, when retrieved context is noisy, unreliable, or inconsistent with the model's parametric knowledge, it introduces retrieval-prior conflicts that can degrade generation quality. While this problem has been studied in autoregressive language models, it remains largely unexplored in diffusion-based language models, where the iterative denoising process introduces unique challenges for integrating retrieved context. In this work, we propose Adaptive Retrieval-Augmented Masked Diffusion (ARAM), a training-free adaptive guidance framework for Masked Diffusion Models (MDMs) in RAG settings. ARAM dynamically calibrates the guidance scale during denoising according to the Signal-to-Noise Ratio (SNR) of the distributional shift induced by retrieved context. Intuitively, the model strengthens guidance when the retrieved context provides reliable corrective evidence and suppresses it when the contextual signal is noisy or non-supportive. Extensive experiments on multiple knowledge-intensive QA benchmarks show that ARAM improves overall QA performance over competitive RAG baselines.
• 摘要：检索增强生成（RAG）通过将外部知识纳入语言模型生成来改善事实基础。然而，当检索到的上下文有噪声、不可靠或与模型的参数知识不一致时，它会引入检索先验冲突，从而降低生成质量。虽然这个问题已经在自回归语言模型中进行了研究，但在基于扩散的语言模型中仍然很大程度上未被探索，其中迭代去噪过程为整合检索到的上下文带来了独特的挑战。在这项工作中，我们提出了自适应检索增强掩模扩散（ARAM），这是一种用于 RAG 设置中掩模扩散模型（MDM）的免训练自适应指导框架。 ARAM 根据检索到的上下文引起的分布偏移的信噪比 (SNR)，动态校准去噪期间的引导尺度。直观上，当检索到的上下文提供可靠的纠正证据时，该模型会加强指导；而当上下文信号有噪声或不支持时，该模型会抑制指导。对多个知识密集型 QA 基准的大量实验表明，与竞争性 RAG 基准相比，ARAM 提高了整体 QA 性能。

Title: CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution

• Authors: Teng Pan, Yuchen Yan, Zixuan Wang, Ruiqing Zhang, Gaiyang Han, Wanqi Zhang, Weiming Lu, Jun Xiao, Yongliang Shen
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.17775
• Pdf URL: https://arxiv.org/pdf/2603.17775
• Copy Paste: [[2603.17775]] CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution(https://arxiv.org/abs/2603.17775)
• Keywords: language model
• Abstract: Label-free reinforcement learning enables large language models to improve reasoning capabilities without ground-truth supervision, typically by treating majority-voted answers as pseudo-labels. However, we identify a critical failure mode: as training maximizes self-consistency, output diversity collapses, causing the model to confidently reinforce systematic errors that evade detection. We term this the consensus trap. To escape it, we propose CoVerRL, a framework where a single model alternates between generator and verifier roles, with each capability bootstrapping the other. Majority voting provides noisy but informative supervision for training the verifier, while the improving verifier progressively filters self-consistent errors from pseudo-labels. This co-evolution creates a virtuous cycle that maintains high reward accuracy throughout training. Experiments across Qwen and Llama model families demonstrate that CoVerRL outperforms label-free baselines by 4.7-5.9\% on mathematical reasoning benchmarks. Moreover, self-verification accuracy improves from around 55\% to over 85\%, confirming that both capabilities genuinely co-evolve.
• 摘要：无标签强化学习使大型语言模型能够在没有真实监督的情况下提高推理能力，通常是将多数投票的答案视为伪标签。然而，我们确定了一种关键的失败模式：随着训练最大化自我一致性，输出多样性崩溃，导致模型自信地强化逃避检测的系统错误。我们将此称为共识陷阱。为了避免这种情况，我们提出了 CoVerRL，这是一个框架，其中单个模型在生成器和验证器角色之间交替，每种功能引导另一种功能。多数投票为训练验证者提供了嘈杂但信息丰富的监督，而改进的验证者则逐步过滤来自伪标签的自洽错误。这种共同进化创造了一个良性循环，可以在整个训练过程中保持较高的奖励准确性。 Qwen 和 Llama 模型系列的实验表明，CoVerRL 在数学推理基准上比无标签基线高 4.7-5.9%。此外，自我验证准确度从 55% 左右提高到 85% 以上，证实这两种功能真正共同发展。

Title: Process Supervision for Chain-of-Thought Reasoning via Monte Carlo Net Information Gain

• Authors: Corentin Royer, Debarun Bhattacharjya, Gaetano Rossiello, Andrea Giovannini, Mennatallah El-Assady
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17815
• Pdf URL: https://arxiv.org/pdf/2603.17815
• Copy Paste: [[2603.17815]] Process Supervision for Chain-of-Thought Reasoning via Monte Carlo Net Information Gain(https://arxiv.org/abs/2603.17815)
• Keywords: language model, llm, chain-of-thought
• Abstract: Multi-step reasoning improves the capabilities of large language models (LLMs) but increases the risk of errors propagating through intermediate steps. Process reward models (PRMs) mitigate this by scoring each step individually, enabling fine-grained supervision and improved reliability. Existing methods for training PRMs rely on costly human annotations or computationally intensive automatic labeling. We propose a novel approach to automatically generate step-level labels using Information Theory. Our method estimates how each reasoning step affects the likelihood of the correct answer, providing a signal of step quality. Importantly, it reduces computational complexity to $\mathcal{O}(N)$, improving over the previous $\mathcal{O}(N \log N)$ methods. We demonstrate that these labels enable effective chain-of-thought selection in best-of-$K$ evaluation settings across diverse reasoning benchmarks, including mathematics, Python programming, SQL, and scientific question answering. This work enables scalable and efficient supervision of LLM reasoning, particularly for tasks where error propagation is critical.
• 摘要：多步骤推理提高了大型语言模型 (LLM) 的能力，但增加了通过中间步骤传播错误的风险。流程奖励模型 (PRM) 通过单独对每个步骤进行评分来缓解这一问题，从而实现细粒度的监督并提高可靠性。现有的 PRM 训练方法依赖于昂贵的人工注释或计算密集型自动标记。我们提出了一种使用信息论自动生成步骤级标签的新颖方法。我们的方法估计每个推理步骤如何影响正确答案的可能性，提供步骤质量的信号。重要的是，它将计算复杂度降低到 $\mathcal{O}(N)$，比之前的 $\mathcal{O}(N \log N)$ 方法有所改进。我们证明，这些标签可以在跨不同推理基准（包括数学、Python 编程、SQL 和科学问答）的 best-of-$K$ 评估设置中实现有效的思想链选择。这项工作能够对 LLM 推理进行可扩展且高效的监督，特别是对于错误传播至关重要的任务。

Title: Text-to-Stage: Spatial Layouts from Long-form Narratives

• Authors: Jefferson Hernandez, Swarnadeep Saha, Chenxi Whitehouse, Sanjeel Parekh, Calvin Murdock, Yuliang Li, W. Owen Brimijoin, Vamsi Krishna Ithapu, Ishwarya Ananthabhotla
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.17832
• Pdf URL: https://arxiv.org/pdf/2603.17832
• Copy Paste: [[2603.17832]] Text-to-Stage: Spatial Layouts from Long-form Narratives(https://arxiv.org/abs/2603.17832)
• Keywords: language model, llm
• Abstract: In this work, we probe the ability of a language model to demonstrate spatial reasoning from unstructured text, mimicking human capabilities and automating a process that benefits many downstream media applications. Concretely, we study the narrative-to-play task: inferring stage-play layouts (scenes, speaker positions, movements, and room types) from text that lacks explicit spatial, positional, or relational cues. We then introduce a dramaturgy-inspired deterministic evaluation suite and, finally, a training and inference recipe that combines rejection SFT using Best-of-N sampling with RL from verifiable rewards via GRPO. Experiments on a text-only corpus of classical English literature demonstrate improvements over vanilla models across multiple metrics (character attribution, spatial plausibility, and movement economy), as well as alignment with an LLM-as-a-judge and subjective human preferences.
• 摘要：在这项工作中，我们探讨了语言模型从非结构化文本中展示空间推理的能力，模仿人类的能力并自动化一个有利于许多下游媒体应用程序的过程。具体来说，我们研究从叙事到表演的任务：从缺乏明确空间、位置或关系线索的文本中推断舞台剧布局（场景、演讲者位置、动作和房间类型）。然后，我们引入了一个受戏剧启发的确定性评估套件，最后引入了一个训练和推理方法，它将使用 Best-of-N 采样的拒绝 SFT 与来自 GRPO 可验证奖励的 RL 结合起来。对古典英语文学纯文本语料库的实验表明，在多个指标（角色归因、空间合理性和运动经济性）上，传统模型比普通模型有所改进，并且与作为法官的法学硕士和人类主观偏好保持一致。

Title: Event-Centric Human Value Understanding in News-Domain Texts: An Actor-Conditioned, Multi-Granularity Benchmark

• Authors: Yao Wang, Xin Liu, Zhuochen Liu, Jiankang Chen, Adam Jatowt, Kyoungsook Kim, Noriko Kando, Haitao Yu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17838
• Pdf URL: https://arxiv.org/pdf/2603.17838
• Copy Paste: [[2603.17838]] Event-Centric Human Value Understanding in News-Domain Texts: An Actor-Conditioned, Multi-Granularity Benchmark(https://arxiv.org/abs/2603.17838)
• Keywords: llm, prompt
• Abstract: Existing human value datasets do not directly support value understanding in factual news: many are actor-agnostic, rely on isolated utterances or synthetic scenarios, and lack explicit event structure or value direction. We present \textbf{NEVU} (\textbf{N}ews \textbf{E}vent-centric \textbf{V}alue \textbf{U}nderstanding), a benchmark for \emph{actor-conditioned}, \emph{event-centric}, and \emph{direction-aware} human value recognition in factual news. NEVU evaluates whether models can identify value cues, attribute them to the correct actor, and determine value direction from grounded evidence. Built from 2{,}865 English news articles, NEVU organizes annotations at four semantic unit levels (\textbf{Subevent}, \textbf{behavior-based composite event}, \textbf{story-based composite event}, and \textbf{Article}) and labels \mbox{(unit, actor)} pairs for fine-grained evaluation across local and composite contexts. The annotations are produced through an LLM-assisted pipeline with staged verification and targeted human auditing. Using a hierarchical value space with \textbf{54} fine-grained values and \textbf{20} coarse-grained categories, NEVU covers 45{,}793 unit--actor pairs and 168{,}061 directed value instances. We provide unified baselines for proprietary and open-source LLMs, and find that lightweight adaptation (LoRA) consistently improves open-source models, showing that although NEVU is designed primarily as a benchmark, it also supports supervised adaptation beyond prompting-only evaluation. Data availability is described in Appendix~\ref{app:data_code_availability}.
• 摘要：现有的人类价值数据集并不直接支持事实新闻中的价值理解：许多数据集与行为者无关，依赖于孤立的话语或合成场景，并且缺乏明确的事件结构或价值方向。我们提出 \textbf{NEVU} (\textbf{N}ews \textbf{E}vent-centric \textbf{V}alue \textbf{U}understanding)，这是事实新闻中 \emph{actor-driven}、\emph{event-centric} 和 \emph{direction-aware} 人类价值识别的基准。 NEVU 评估模型是否能够识别价值线索、将其归因于正确的参与者，并根据有根据的证据确定价值方向。 NEVU 由 2{,}865 篇英文新闻文章组成，在四个语义单元级别（\textbf{Subevent}、\textbf{基于行为的复合事件}、\textbf{基于故事的复合事件}和 \textbf{Article}）组织注释和标签 \mbox{(unit, actor)} 对，以便跨本地和复合上下文进行细粒度评估。这些注释是通过法学硕士辅助的管道生成的，并进行分阶段验证和有针对性的人工审核。使用具有 \textbf{54} 细粒度值和 \textbf{20} 粗粒度类别的分层值空间，NEVU 涵盖 45{,}793 个单元-参与者对和 168{,}061 个有向值实例。我们为专有和开源法学硕士提供了统一的基线，并发现轻量级适应（LoRA）持续改进了开源模型，这表明虽然NEVU主要是作为基准而设计的，但它也支持超越仅提示评估的监督适应。数据可用性在附录~\ref{app:data_code_availability}中描述。

Title: How do LLMs Compute Verbal Confidence

• Authors: Dharshan Kumaran, Arthur Conmy, Federico Barbero, Simon Osindero, Viorica Patraucean, Petar Velickovic
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.17839
• Pdf URL: https://arxiv.org/pdf/2603.17839
• Copy Paste: [[2603.17839]] How do LLMs Compute Verbal Confidence(https://arxiv.org/abs/2603.17839)
• Keywords: llm, prompt
• Abstract: Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from black-box models. However, how LLMs internally generate such scores remains unknown. We address two questions: first, when confidence is computed - just-in-time when requested, or automatically during answer generation and cached for later retrieval; and second, what verbal confidence represents - token log-probabilities, or a richer evaluation of answer quality? Focusing on Gemma 3 27B and Qwen 2.5 7B, we provide convergent evidence for cached retrieval. Activation steering, patching, noising, and swap experiments reveal that confidence representations emerge at answer-adjacent positions before appearing at the verbalization site. Attention blocking pinpoints the information flow: confidence is gathered from answer tokens, cached at the first post-answer position, then retrieved for output. Critically, linear probing and variance partitioning reveal that these cached representations explain substantial variance in verbal confidence beyond token log-probabilities, suggesting a richer answer-quality evaluation rather than a simple fluency readout. These findings demonstrate that verbal confidence reflects automatic, sophisticated self-evaluation -- not post-hoc reconstruction -- with implications for understanding metacognition in LLMs and improving calibration.
• 摘要：口头信心——促使法学硕士以数字或类别的形式陈述他们的信心——被广泛用于从黑盒模型中提取不确定性估计。然而，法学硕士如何在内部生成此类分数仍然未知。我们解决两个问题：首先，何时计算置信度 - 在请求时及时计算，或者在答案生成期间自动计算并缓存以供以后检索；其次，言语信心代表什么——象征性的对数概率，还是对答案质量的更丰富的评估？我们重点关注 Gemma 3 27B 和 Qwen 2.5 7B，为缓存检索提供收敛证据。激活引导、修补、噪声和交换实验表明，置信表示在出现在言语站点之前出现在答案相邻位置。注意力阻塞精确定位信息流：从答案标记中收集置信度，缓存在第一个答案后位置，然后检索以进行输出。重要的是，线性探测和方差划分揭示了这些缓存的表示解释了言语置信度中超出标记对数概率的巨大差异，这表明更丰富的答案质量评估而不是简单的流畅性读出。这些发现表明，言语信心反映了自动的、复杂的自我评估，而不是事后重建，这对于理解法学硕士的元认知和改进校准具有影响。

Title: Mitigating LLM Hallucinations through Domain-Grounded Tiered Retrieval

• Authors: Md. Asraful Haque, Aasar Mehdi, Maaz Mahboob, Tamkeen Fatima
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.17872
• Pdf URL: https://arxiv.org/pdf/2603.17872
• Copy Paste: [[2603.17872]] Mitigating LLM Hallucinations through Domain-Grounded Tiered Retrieval(https://arxiv.org/abs/2603.17872)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) have achieved unprecedented fluency but remain susceptible to "hallucinations" - the generation of factually incorrect or ungrounded content. This limitation is particularly critical in high-stakes domains where reliability is paramount. We propose a domain-grounded tiered retrieval and verification architecture designed to systematically intercept factual inaccuracies by shifting LLMs from stochastic pattern-matchers to verified truth-seekers. The proposed framework utilizes a four-phase, self-regulating pipeline implemented via LangGraph: (I) Intrinsic Verification with Early-Exit logic to optimize compute, (II) Adaptive Search Routing utilizing a Domain Detector to target subject-specific archives, (III) Corrective Document Grading (CRAG) to filter irrelevant context, and (IV) Extrinsic Regeneration followed by atomic claim-level verification. The system was evaluated across 650 queries from five diverse benchmarks: TimeQA v2, FreshQA v2, HaluEval General, MMLU Global Facts, and TruthfulQA. Empirical results demonstrate that the pipeline consistently outperforms zero-shot baselines across all environments. Win rates peaked at 83.7% in TimeQA v2 and 78.0% in MMLU Global Facts, confirming high efficacy in domains requiring granular temporal and numerical precision. Groundedness scores remained robustly stable between 78.8% and 86.4% across factual-answer rows. While the architecture provides a robust fail-safe for misinformation, a persistent failure mode of "False-Premise Overclaiming" was identified. These findings provide a detailed empirical characterization of multi-stage RAG behavior and suggest that future work should prioritize pre-retrieval "answerability" nodes to further bridge the reliability gap in conversational AI.
• 摘要：大型语言模型 (LLM) 已经实现了前所未有的流畅性，但仍然容易受到“幻觉”的影响，即生成事实上不正确或没有根据的内容。这种限制在可靠性至关重要的高风险领域尤其重要。我们提出了一种基于领域的分层检索和验证架构，旨在通过将法学硕士从随机模式匹配者转移到经过验证的真相寻求者来系统地拦截事实不准确之处。所提出的框架利用通过 LangGraph 实现的四阶段自我调节管道：(I) 具有提前退出逻辑的内在验证来优化计算，(II) 利用域检测器来定位特定主题档案的自适应搜索路由，(III) 纠正文档分级 (CRAG) 来过滤不相关的上下文，以及 (IV) 外部再生，然后进行原子声明级验证。该系统通过来自五个不同基准的 650 个查询进行了评估：TimeQA v2、FreshQA v2、HaluEval General、MMLU Global Facts 和 TruthfulQA。经验结果表明，该管道在所有环境中始终优于零样本基线。 TimeQA v2 中的获胜率达到峰值 83.7%，MMLU Global Facts 中的获胜率达到 78.0%，这证实了在需要粒度时间和数值精度的领域中的高效性。在事实答案行中，接地性分数保持稳定在 78.8% 到 86.4% 之间。虽然该架构为错误信息提供了强大的故障保护功能，但仍发现了“错误前提夸大”的持续故障模式。这些发现提供了多阶段 RAG 行为的详细经验特征，并表明未来的工作应优先考虑预检索的“可回答性”节点，以进一步弥合对话式 AI 中的可靠性差距。

Title: DebugLM: Learning Traceable Training Data Provenance for LLMs

• Authors: Wenjie Jacky Mo, Qin Liu, Xiaofei Wen, Wenxuan Zhou, Zhe Zhao, Muhao Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17884
• Pdf URL: https://arxiv.org/pdf/2603.17884
• Copy Paste: [[2603.17884]] DebugLM: Learning Traceable Training Data Provenance for LLMs(https://arxiv.org/abs/2603.17884)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are trained through multi-stage pipelines over heterogeneous data sources, yet developers lack a principled way to pinpoint the specific data responsible for an observed behavior. This lack of observability reduces debugging to reactive patching and makes failures prone to recur under distribution shift or subsequent model updates. To address this limitation, we propose DebugLM, a framework that equips LLMs with built-in data provenance, enabling them to explicitly trace the origins of their behaviors to specific training data sources. Specifically, the model learns to associate its responses with unique provenance tags that indicate the responsible dataset, empowering developers to precisely identify where undesirable behaviors are learned. Building on this capability, DebugLM further supports targeted test-time remediation, enabling developers to selectively trigger targeted refusal for specified data sources without retraining or modifying model parameters. Experiments demonstrate that DebugLM provides accurate behavior tracing in multi-stage training pipelines and effective test-time remediation while preserving the general utility of the model.
• 摘要：大型语言模型 (LLM) 是通过异构数据源上的多级管道进行训练的，但开发人员缺乏一种原则性的方法来查明导致观察到的行为的特定数据。这种可观察性的缺乏减少了对反应性修补的调试，并且使得在分发转移或后续模型更新时容易再次出现故障。为了解决这个限制，我们提出了 DebugLM，这是一个为法学硕士配备内置数据来源的框架，使他们能够明确地将其行为的起源追溯到特定的训练数据源。具体来说，该模型学习将其响应与指示负责任数据集的独特出处标签相关联，使开发人员能够准确识别在何处学到不良行为。在此功能的基础上，DebugLM 进一步支持有针对性的测试时修复，使开发人员能够有选择地触发对指定数据源的有针对性的拒绝，而无需重新训练或修改模型参数。实验表明，DebugLM 在多阶段训练管道中提供准确的行为跟踪和有效的测试时修复，同时保留模型的通用性。

Title: Pretrained Multilingual Transformers Reveal Quantitative Distance Between Human Languages

• Authors: Yue Zhao, Jiatao Gu, Paloma Jeretič, Weijie Su
• Subjects: cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2603.17912
• Pdf URL: https://arxiv.org/pdf/2603.17912
• Copy Paste: [[2603.17912]] Pretrained Multilingual Transformers Reveal Quantitative Distance Between Human Languages(https://arxiv.org/abs/2603.17912)
• Keywords: language model
• Abstract: Understanding the distance between human languages is central to linguistics, anthropology, and tracing human evolutionary history. Yet, while linguistics has long provided rich qualitative accounts of cross-linguistic variation, a unified and scalable quantitative approach to measuring language distance remains lacking. In this paper, we introduce a method that leverages pretrained multilingual language models as systematic instruments for linguistic measurement. Specifically, we show that the spontaneously emerged attention mechanisms of these models provide a robust, tokenization-agnostic measure of cross-linguistic distance, termed Attention Transport Distance (ATD). By treating attention matrices as probability distributions and measuring their geometric divergence via optimal transport, we quantify the representational distance between languages during translation. Applying ATD to a large and diverse set of languages, we demonstrate that the resulting distances recover established linguistic groupings with high fidelity and reveal patterns aligned with geographic and contact-induced relationships. Furthermore, incorporating ATD as a regularizer improves transfer performance in low-resource machine translation. Our results establish a principled foundation for testing linguistic hypotheses using artificial neural networks. This framework transforms multilingual models into powerful tools for quantitative linguistic discovery, facilitating more equitable multilingual AI.
• 摘要：了解人类语言之间的距离是语言学、人类学和追踪人类进化史的核心。然而，尽管语言学长期以来为跨语言变异提供了丰富的定性解释，但仍然缺乏一种统一且可扩展​​的定量方法来测量语言距离。在本文中，我们介绍了一种利用预训练的多语言语言模型作为语言测量的系统工具的方法。具体来说，我们表明这些模型自发出现的注意力机制提供了一种稳健的、与标记化无关的跨语言距离度量，称为注意力传输距离（ATD）。通过将注意力矩阵视为概率分布并通过最佳传输测量其几何散度，我们量化了翻译过程中语言之间的表示距离。将 ATD 应用于大量不同的语言，我们证明了所得到的距离以高保真度恢复了已建立的语言分组，并揭示了与地理和接触引起的关系相一致的模式。此外，将 ATD 合并为正则化器可以提高低资源机器翻译中的传输性能。我们的结果为使用人工神经网络测试语言假设奠定了原则基础。该框架将多语言模型转变为定量语言发现的强大工具，促进更公平的多语言人工智能。

Title: IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia

• Authors: Priyaranjan Pattnayak, Sanchari Chowdhuri
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.17915
• Pdf URL: https://arxiv.org/pdf/2603.17915
• Copy Paste: [[2603.17915]] IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia(https://arxiv.org/abs/2603.17915)
• Keywords: language model, llm, prompt
• Abstract: As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages remains poorly understood. We present the first systematic evaluation of LLM safety across 12 Indic languages, spoken by over 1.2 billion people but underrepresented in LLM training data. Using a dataset of 6,000 culturally grounded prompts spanning caste, religion, gender, health, and politics, we assess 10 leading LLMs on translated variants of the prompt. Our analysis reveals significant safety drift: cross-language agreement is just 12.8\%, and \texttt{SAFE} rate variance exceeds 17\% across languages. Some models over-refuse benign prompts in low-resource scripts, overflag politically sensitive topics, while others fail to flag unsafe generations. We quantify these failures using prompt-level entropy, category bias scores, and multilingual consistency indices. Our findings highlight critical safety generalization gaps in multilingual LLMs and show that safety alignment does not transfer evenly across languages. We release \textsc{IndicSafe}, the first benchmark to enable culturally informed safety evaluation for Indic deployments, and advocate for language-aware alignment strategies grounded in regional harms.
• 摘要：由于大型语言模型 (LLM) 被部署在多语言环境中，人们对它们在文化多元化、资源匮乏的语言中的安全行为仍然知之甚少。我们对 12 种印度语言的法学硕士安全性进行了首次系统评估，这些语言有超过 12 亿人使用，但在法学硕士培训数据中代表性不足。我们使用包含 6,000 个跨种姓、宗教、性别、健康和政治的基于文化的提示的数据集，评估了 10 位领先的法学硕士的提示翻译变体。我们的分析揭示了显着的安全漂移：跨语言一致性仅为 12.8%，而跨语言的 \texttt{SAFE} 率差异超过 17%。一些模型过度拒绝低资源脚本中的良性提示，过度标记政治敏感主题，而其他模型则未能标记不安全的代。我们使用提示级熵、类别偏差分数和多语言一致性指数来量化这些失败。我们的研究结果强调了多语言法学硕士中关键的安全泛化差距，并表明安全一致性在不同语言之间的转移并不均匀。我们发布了 \textsc{IndicSafe}，这是第一个基准，可以对印度部署进行文化上的安全评估，并倡导基于区域危害的语言感知对齐策略。

Title: Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing

• Authors: Raghavv Goel, Mukul Gagrani, Mingu Lee, Chris Lott
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17942
• Pdf URL: https://arxiv.org/pdf/2603.17942
• Copy Paste: [[2603.17942]] Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing(https://arxiv.org/abs/2603.17942)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) exhibit latent multi-token prediction (MTP) capabilities despite being trained solely for next-token generation. We propose a simple, training-free MTP approach that probes an LLM using on-the-fly mask tokens drawn from its embedding space, enabling parallel prediction of future tokens without modifying model weights or relying on auxiliary draft models. Our method constructs a speculative token tree by sampling top-K candidates from mask-token logits and applies a lightweight pruning strategy to retain high-probability continuations. During decoding, candidate predictions are verified in parallel, resulting in lossless generation while substantially reducing the number of model calls and improving token throughput. Across benchmarks, our probing-based MTP consistently outperforms existing training-free baselines, increasing acceptance length by approximately 12\% on LLaMA3 and 8--12\% on Qwen3, and achieving throughput gains of up to 15--19\%. Finally, we provide theoretical insights and empirical evidence showing that decoder layers naturally align mask-token representations with next-token states, enabling accurate multi-step prediction without retraining or auxiliary models.
• 摘要：尽管大型语言模型（LLM）仅针对下一代令牌进行训练，但仍表现出潜在的多令牌预测（MTP）能力。我们提出了一种简单的、免训练的 MTP 方法，该方法使用从嵌入空间中提取的动态掩码令牌来探测 LLM，从而能够并行预测未来的令牌，而无需修改模型权重或依赖辅助草案模型。我们的方法通过从掩码标记逻辑中采样前 K 个候选来构建推测标记树，并应用轻量级修剪策略来保留高概率的连续。在解码过程中，并行验证候选预测，从而实现无损生成，同时大幅减少模型调用数量并提高令牌吞吐量。在各个基准测试中，我们基于探测的 MTP 始终优于现有的免训练基线，在 LLaMA3 上将接受长度增加了约 12%，在 Qwen3 上增加了 8--12%，并实现了高达 15--19% 的吞吐量增益。最后，我们提供了理论见解和经验证据，表明解码器层自然地将掩码令牌表示与下一个令牌状态对齐，从而无需重新训练或辅助模型即可实现准确的多步预测。

Title: Gender Disambiguation in Machine Translation: Diagnostic Evaluation in Decoder-Only Architectures

• Authors: Chiara Manna, Hosein Mohebbi, Afra Alishahi, Frédéric Blain, Eva Vanmassenhove
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17952
• Pdf URL: https://arxiv.org/pdf/2603.17952
• Copy Paste: [[2603.17952]] Gender Disambiguation in Machine Translation: Diagnostic Evaluation in Decoder-Only Architectures(https://arxiv.org/abs/2603.17952)
• Keywords: language model
• Abstract: While Large Language Models achieve state-of-the-art results across a wide range of NLP tasks, they remain prone to systematic biases. Among these, gender bias is particularly salient in MT, due to systematic differences across languages in whether and how gender is marked. As a result, translation often requires disambiguating implicit source signals into explicit gender-marked forms. In this context, standard benchmarks may capture broad disparities but fail to reflect the full complexity of gender bias in modern MT. In this paper, we extend recent frameworks on bias evaluation by: (i) introducing a novel measure coined "Prior Bias", capturing a model's default gender assumptions, and (ii) applying the framework to decoder-only MT models. Our results show that, despite their scale and state-of-the-art status, decoder-only models do not generally outperform encoder-decoder architectures on gender-specific metrics; however, post-training (e.g., instruction tuning) not only improves contextual awareness but also reduces the masculine Prior Bias.
• 摘要：虽然大型语言模型在广泛的 NLP 任务中取得了最先进的结果，但它们仍然容易出现系统偏差。其中，性别偏见在机器翻译中尤为突出，因为不同语言在是否以及如何标记性别方面存在系统性差异。因此，翻译通常需要将隐含的源信号消除歧义，转化为明确的性别标记形式。在这种情况下，标准基准可能会捕捉到广泛的差异，但无法反映现代机器翻译中性别偏见的全部复杂性。在本文中，我们通过以下方式扩展了最近的偏差评估框架：（i）引入一种新的测量方法“先验偏差”，捕获模型的默认性别假设，以及（ii）将该框架应用于仅解码器的机器翻译模型。我们的结果表明，尽管其规模和最先进的状态，仅解码器模型在特定性别指标上通常不会优于编码器-解码器架构；然而，训练后（例如，指令调整）不仅可以提高情境意识，还可以减少男性先验偏见。

Title: ConGA: Guidelines for Contextual Gender Annotation. A Framework for Annotating Gender in Machine Translation

• Authors: Argentina Anna Rescigno, Eva Vanmassenhove, Johanna Monti
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.17962
• Pdf URL: https://arxiv.org/pdf/2603.17962
• Copy Paste: [[2603.17962]] ConGA: Guidelines for Contextual Gender Annotation. A Framework for Annotating Gender in Machine Translation(https://arxiv.org/abs/2603.17962)
• Keywords: language model, llm
• Abstract: Handling gender across languages remains a persistent challenge for Machine Translation (MT) and Large Language Models (LLMs), especially when translating from gender-neutral languages into morphologically gendered ones, such as English to Italian. English largely omits grammatical gender, while Italian requires explicit agreement across multiple grammatical categories. This asymmetry often leads MT systems to default to masculine forms, reinforcing bias and reducing translation accuracy. To address this issue, we present the Contextual Gender Annotation (ConGA) framework, a linguistically grounded set of guidelines for word-level gender annotation. The scheme distinguishes between semantic gender in English through three tags, Masculine (M), Feminine (F), and Ambiguous (A), and grammatical gender realisation in Italian (Masculine (M), Feminine (F)), combined with entity-level identifiers for cross-sentence tracking. We apply ConGA to the gENder-IT dataset, creating a gold-standard resource for evaluating gender bias in translation. Our results reveal systematic masculine overuse and inconsistent feminine realisation, highlighting persistent limitations of current MT systems. By combining fine-grained linguistic annotation with quantitative evaluation, this work offers both a methodology and a benchmark for building more gender-aware and multilingual NLP systems.
• 摘要：跨语言处理性别对于机器翻译 (MT) 和大型语言模型 (LLM) 来说仍然是一个持续的挑战，特别是在从性别中立语言翻译为形态性别语言（例如英语到意大利语）时。英语很大程度上忽略了语法性别，而意大利语则要求多个语法类别之间达成明确的一致。这种不对称性通常会导致机器翻译系统默认为男性形式，从而加剧偏见并降低翻译准确性。为了解决这个问题，我们提出了上下文性别注释（ConGA）框架，这是一套基于语言的单词级性别注释指南。该方案通过男性（M）、女性（F）和模糊（A）三个标签来区分英语中的语义性别，以及意大利语中的语法性别实现（男性（M）、女性（F）），并结合实体级标识符进行跨句子跟踪。我们将 ConGA 应用于 gENder-IT 数据集，为评估翻译中的性别偏见创建了黄金标准资源。我们的结果揭示了系统性的男性过度使用和不一致的女性实现，凸显了当前机器翻译系统的持续局限性。通过将细粒度语言注释与定量评估相结合，这项工作为构建更多性别意识和多语言 NLP 系统提供了方法论和基准。