2026-04-01

Title: OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training

• Authors: Haiyue Song, Masao Utiyama
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.28858
• Pdf URL: https://arxiv.org/pdf/2603.28858
• Copy Paste: [[2603.28858]] OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training(https://arxiv.org/abs/2603.28858)
• Keywords: llm
• Abstract: Continual pre-training is widely used to adapt LLMs to target languages and domains, yet the mixture ratio of training data remains a sensitive hyperparameter that is expensive to tune: they must be fixed before training begins, and a suboptimal choice can waste weeks of compute. In this work, we propose OptiMer, which decouples ratio selection from training: we train one CPT model per dataset, extract each model's distribution vector, which represents the parameter shift induced by that dataset, and search for optimal composition weights post-hoc via Bayesian optimization. Experiments on Gemma 3 27B across languages (Japanese, Chinese) and domains (Math, Code) show that OptiMer consistently outperforms data mixture and model averaging baselines with 15-35 times lower search cost. Key findings reveal that 1) the optimized weights can be interpreted as data mixture ratios, and retraining with these ratios improves data mixture CPT, and 2) the same vector pool can be re-optimized for a given objective without any retraining, producing target-tailored models on demand. Our work establishes that data mixture ratio selection, traditionally a pre-training decision, can be reformulated as a post-hoc optimization over distribution vectors, offering a more flexible paradigm for continual pre-training.
• 摘要：持续预训练被广泛用于使 LLM 适应目标语言和领域，但训练数据的混合比仍然是一个敏感的超参数，调整成本很高：它们必须在训练开始之前修复，而次优的选择可能会浪费数周的计算时间。在这项工作中，我们提出了 OptiMer，它将比率选择与训练解耦：我们为每个数据集训练一个 CPT 模型，提取每个模型的分布向量（表示由该数据集引起的参数变化），并通过贝叶斯优化事后搜索最佳组合权重。 Gemma 3 27B 跨语言（日语、中文）和领域（数学、代码）的实验表明，OptiMer 始终优于数据混合和模型平均基线，搜索成本低 15-35 倍。主要发现表明：1) 优化后的权重可以解释为数据混合比率，使用这些比率进行再训练可以提高数据混合 CPT，2) 可以针对给定目标重新优化相同的向量池，而无需进行任何再训练，从而按需生成目标定制的模型。我们的工作表明，数据混合比选择（传统上是预训练决策）可以重新表述为分布向量的事后优化，为持续预训练提供更灵活的范例。

Title: CrossTrace: A Cross-Domain Dataset of Grounded Scientific Reasoning Traces for Hypothesis Generation

• Authors: Andrew Bouras, OMS-II Research Fellow
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.28924
• Pdf URL: https://arxiv.org/pdf/2603.28924
• Copy Paste: [[2603.28924]] CrossTrace: A Cross-Domain Dataset of Grounded Scientific Reasoning Traces for Hypothesis Generation(https://arxiv.org/abs/2603.28924)
• Keywords: gpt
• Abstract: Scientific hypothesis generation is a critical bottleneck in accelerating research, yet existing datasets for training and evaluating hypothesis-generating models are limited to single domains and lack explicit reasoning traces connecting prior knowledge to novel contributions. I introduce CrossTrace, a dataset of 1,389 grounded scientific reasoning traces spanning biomedical research (518), AI/ML (605), and cross-domain work (266). Each trace captures the structured reasoning chain from established knowledge through intermediate logical steps to a novel hypothesis, with every step grounded in source paper text. I define an Input/Trace/Output schema that extends the Bit-Flip-Spark framework of HypoGen with step-level verification, a taxonomy of eight discovery patterns, and multi-domain coverage. Fine-tuning Qwen2.5-7B-Instruct on CrossTrace via QLoRA yields substantial improvements over the untuned baseline: IAScore rises from 0.828 to 0.968 (GPT-4o judge) and from 0.716 to 0.888 (Claude Opus 4.5), structural compliance improves from 0% to 100%, and spark cosine similarity increases from 0.221 to 0.620. Balanced cross-domain training (biomedical + AI/ML + CS) outperforms single-domain training, providing evidence that scientific reasoning patterns transfer across disciplines. Human validation of 150 stratified records confirms 99.7% step-level grounding accuracy and a 0.0% fabrication rate. To my knowledge, CrossTrace is the first large-scale, cross-domain dataset with step-level grounded reasoning traces for hypothesis generation, and my results demonstrate that such traces are an effective training signal whose benefits are at least partially domain-general.
• 摘要：科学假设生成是加速研究的关键瓶颈，但用于训练和评估假设生成模型的现有数据集仅限于单个领域，并且缺乏将先验知识与新贡献联系起来的明确推理轨迹。我介绍 CrossTrace，这是一个包含 1,389 条科学推理痕迹的数据集，涵盖生物医学研究 (518)、AI/ML (605) 和跨领域工作 (266)。每条痕迹都捕获了从已建立的知识到中间逻辑步骤到新假设的结构化推理链，每一步都基于源论文文本。我定义了一个输入/跟踪/输出模式，该模式通过步骤级验证、八种发现模式的分类法和多域覆盖来扩展 HypoGen 的 Bit-Flip-Spark 框架。通过 QLoRA 对 CrossTrace 上的 Qwen2.5-7B-Instruct 进行微调，与未调整的基线相比有了显着改进：IAScore 从 0.828 上升到 0.968（GPT-4o 判断），从 0.716 上升到 0.888（Claude Opus 4.5），结构合规性从 0% 提高到 100%，火花余弦相似度从 0.221 提高到0.620。平衡的跨领域训练（生物医学+人工智能/机器学习+计算机科学）优于单领域训练，为科学推理模式跨学科转移提供了证据。对 150 条分层记录的人工验证证实了 99.7% 的阶梯级接地精度和 0.0% 的制造率。据我所知，CrossTrace 是第一个大规模跨域数据集，具有用于假设生成的步骤级接地推理轨迹，我的结果表明，此类轨迹是有效的训练信号，其好处至少部分是领域通用的。

Title: Theory of Mind and Self-Attributions of Mentality are Dissociable in LLMs

• Authors: Junsol Kim, Winnie Street, Roberta Rocca, Daine M. Korngiebel, Adam Waytz, James Evans, Geoff Keeling
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.28925
• Pdf URL: https://arxiv.org/pdf/2603.28925
• Copy Paste: [[2603.28925]] Theory of Mind and Self-Attributions of Mentality are Dissociable in LLMs(https://arxiv.org/abs/2603.28925)
• Keywords: language model, llm
• Abstract: Safety fine-tuning in Large Language Models (LLMs) seeks to suppress potentially harmful forms of mind-attribution such as models asserting their own consciousness or claiming to experience emotions. We investigate whether suppressing mind-attribution tendencies degrades intimately related socio-cognitive abilities such as Theory of Mind (ToM). Through safety ablation and mechanistic analyses of representational similarity, we demonstrate that LLM attributions of mind to themselves and to technological artefacts are behaviorally and mechanistically dissociable from ToM capabilities. Nevertheless, safety fine-tuned models under-attribute mind to non-human animals relative to human baselines and are less likely to exhibit spiritual belief, suppressing widely shared perspectives regarding the distribution and nature of non-human minds.
• 摘要：大型语言模型 (LLM) 中的安全微调旨在抑制潜在有害的心理归因形式，例如断言自己的意识或声称体验情感的模型。我们研究抑制心理归因倾向是否会降低密切相关的社会认知能力，例如心理理论（ToM）。通过安全消融和表征相似性的机械分析，我们证明了法学硕士对思维本身和技术制品的归因在行为上和机械上与 ToM 能力是分离的。然而，安全微调模型相对于人类基线低估了非人类动物的思维，并且不太可能表现出精神信仰，从而抑制了关于非人类思维的分布和本质的广泛共识。

Title: Human-Like Lifelong Memory: A Neuroscience-Grounded Architecture for Infinite Interaction

• Authors: Diego C. Lerma-Torres (Universidad de Guanajuato)
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.29023
• Pdf URL: https://arxiv.org/pdf/2603.29023
• Copy Paste: [[2603.29023]] Human-Like Lifelong Memory: A Neuroscience-Grounded Architecture for Infinite Interaction(https://arxiv.org/abs/2603.29023)
• Keywords: language model, hallucination
• Abstract: Large language models lack persistent, structured memory for long-term interaction and context-sensitive retrieval. Expanding context windows does not solve this: recent evidence shows that context length alone degrades reasoning by up to 85% - even with perfect retrieval. We propose a bio-inspired memory framework grounded in complementary learning systems theory, cognitive behavioral therapy's belief hierarchy, dual-process cognition, and fuzzy-trace theory, organized around three principles: (1) Memory has valence, not just content - pre-computed emotional-associative summaries (valence vectors) organized in an emergent belief hierarchy inspired by Beck's cognitive model enable instant orientation before deliberation; (2) Retrieval defaults to System 1 with System 2 escalation - automatic spreading activation and passive priming as default, with deliberate retrieval only when needed, and graded epistemic states that address hallucination structurally; and (3) Encoding is active, present, and feedback-dependent - a thalamic gateway tags and routes information between stores, while the executive forms gists through curiosity-driven investigation, not passive exposure. Seven functional properties specify what any implementation must satisfy. Over time, the system converges toward System 1 processing - the computational analog of clinical expertise - producing interactions that become cheaper, not more expensive, with experience.
• 摘要：大型语言模型缺乏用于长期交互和上下文敏感检索的持久的结构化记忆。扩展上下文窗口并不能解决这个问题：最近的证据表明，即使有完美的检索，上下文长度本身也会使推理能力下降高达 85%。我们提出了一个基于补充学习系统理论、认知行为疗法的信念层次、双过程认知和模糊轨迹理论的生物启发记忆框架，该框架围绕三个原则组织：（1）记忆具有效价，而不仅仅是内容——预先计算的情感联想摘要（效价向量）组织在受贝克认知模型启发的新兴信念层次结构中，可以在深思熟虑之前立即定位； (2) 检索默认为系统 1，系统 2 升级 - 默认为自动传播激活和被动启动，仅在需要时才进行刻意检索，并从结构上解决幻觉的分级认知状态； (3) 编码是主动的、存在的并且依赖于反馈——丘脑网关在商店之间标记和路由信息，而执行形式则通过好奇心驱动的调查而不是被动暴露来表达要点。七个功能属性指定任何实现必须满足的条件。随着时间的推移，该系统向系统 1 处理（临床专业知识的计算模拟）收敛，产生的交互随着经验的增加而变得更便宜，而不是更昂贵。

Title: The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning

• Authors: Yubo Li, Lu Zhang, Tianchong Jiang, Ramayya Krishnan, Rema Padman
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.29025
• Pdf URL: https://arxiv.org/pdf/2603.29025
• Copy Paste: [[2603.29025]] The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning(https://arxiv.org/abs/2603.29025)
• Keywords: language model, llm, prompt
• Abstract: Large language models systematically fail when a salient surface cue conflicts with an unstated feasibility constraint. We study this through a diagnose-measure-bridge-treat framework. Causal-behavioral analysis of the ``car wash problem'' across six models reveals approximately context-independent sigmoid heuristics: the distance cue exerts 8.7 to 38 times more influence than the goal, and token-level attribution shows patterns more consistent with keyword associations than compositional inference. The Heuristic Override Benchmark (HOB) -- 500 instances spanning 4 heuristic by 5 constraint families with minimal pairs and explicitness gradients -- demonstrates generality across 14 models: under strict evaluation (10/10 correct), no model exceeds 75%, and presence constraints are hardest (44%). A minimal hint (e.g., emphasizing the key object) recovers +15 pp on average, suggesting the failure lies in constraint inference rather than missing knowledge; 12/14 models perform worse when the constraint is removed (up to -39 pp), revealing conservative bias. Parametric probes confirm that the sigmoid pattern generalizes to cost, efficiency, and semantic-similarity heuristics; goal-decomposition prompting recovers +6 to 9 pp by forcing models to enumerate preconditions before answering. Together, these results characterize heuristic override as a systematic reasoning vulnerability and provide a benchmark for measuring progress toward resolving it.
• 摘要：当显着的表面线索与未声明的可行性约束发生冲突时，大型语言模型会系统性地失败。我们通过诊断-测量-桥梁-治疗框架来研究这一点。对六个模型中“洗车问题”的因果行为分析揭示了大约与上下文无关的 sigmoid 启发式：距离提示的影响力是目标的 8.7 到 38 倍，并且标记级归因显示的模式与关键词关联比组合推理更一致。启发式覆盖基准 (HOB)——跨越 4 个启发式和 5 个约束族的 500 个实例，具有最小对和显式梯度——展示了 14 个模型的通用性：在严格评估下（10/10 正确），没有模型超过 75%，并且存在约束是最难的（44%）。一个最小的提示（例如，强调关键对象）平均恢复+15 pp，这表明失败在于约束推理而不是知识缺失；当约束被移除时，12/14 模型的表现更差（高达 -39 pp），揭示了保守偏差。参数化探索证实 sigmoid 模式可以推广到成本、效率和语义相似性启发法；目标分解提示通过强制模型在回答之前枚举先决条件来恢复 +6 到 9 个百分点。总之，这些结果将启发式覆盖描述为系统推理漏洞，并为衡量解决该漏洞的进度提供了基准。

Title: Dual Perspectives in Emotion Attribution: A Generator-Interpreter Framework for Cross-Cultural Analysis of Emotion in LLMs

• Authors: Aizirek Turdubaeva, Uichin Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29077
• Pdf URL: https://arxiv.org/pdf/2603.29077
• Copy Paste: [[2603.29077]] Dual Perspectives in Emotion Attribution: A Generator-Interpreter Framework for Cross-Cultural Analysis of Emotion in LLMs(https://arxiv.org/abs/2603.29077)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are increasingly used in cross-cultural systems to understand and adapt to human emotions, which are shaped by cultural norms of expression and interpretation. However, prior work on emotion attribution has focused mainly on interpretation, overlooking the cultural background of emotion generators. This assumption of universality neglects variation in how emotions are expressed and perceived across nations. To address this gap, we propose a Generator-Interpreter framework that captures dual perspectives of emotion attribution by considering both expression and interpretation. We systematically evaluate six LLMs on an emotion attribution task using data from 15 countries. Our analysis reveals that performance variations depend on the emotion type and cultural context. Generator-interpreter alignment effects are present; the generator's country of origin has a stronger impact on performance. We call for culturally sensitive emotion modeling in LLM-based systems to improve robustness and fairness in emotion understanding across diverse cultural contexts.
• 摘要：大语言模型（LLM）越来越多地用于跨文化系统中，以理解和适应人类情感，而人类情感是由表达和解释的文化规范塑造的。然而，先前关于情绪归因的工作主要集中在解释上，忽视了情绪产生者的文化背景。这种普遍性假设忽略了各国情感表达和感知方式的差异。为了解决这一差距，我们提出了一个生成器-解释器框架，通过考虑表达和解释来捕获情感归因的双重视角。我们使用来自 15 个国家的数据系统地评估了六名法学硕士的情绪归因任务。我们的分析表明，绩效差异取决于情绪类型和文化背景。存在生成器-解释器对齐效应；发电机的原产国对性能的影响更大。我们呼吁在基于法学硕士的系统中进行文化敏感的情感建模，以提高跨不同文化背景的情感理解的鲁棒性和公平性。

Title: PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression

• Authors: Caio Vicentino
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.29078
• Pdf URL: https://arxiv.org/pdf/2603.29078
• Copy Paste: [[2603.29078]] PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression(https://arxiv.org/abs/2603.29078)
• Keywords: language model, llm
• Abstract: We present PolarQuant, a post-training weight quantization method for large language models (LLMs) that exploits the distributional structure of neural network weights to achieve near-lossless compression. PolarQuant operates in three stages: (1) block-wise normalization to the unit hypersphere, (2) Walsh-Hadamard rotation to transform coordinates into approximately Gaussian random variables, and (3) quantization with centroids matched to the Gaussian distribution. Our ablation reveals that Hadamard rotation alone accounts for 98% of the quality improvement, reducing Qwen3.5-9B perplexity from 6.90 (absmax Q5) to 6.40 (Delta = +0.03 from FP16), making it practically lossless without any calibration data. Furthermore, PolarQuant functions as an effective preprocessing step for downstream INT4 quantizers: PolarQuant Q5 dequantized and re-quantized by torchao INT4 achieves perplexity 6.56 versus 6.68 for direct absmax INT4, while maintaining 43.1 tok/s throughput at 6.5 GB VRAM. Code and models are publicly available.
• 摘要：我们提出了 PolarQuant，一种用于大型语言模型 (LLM) 的训练后权重量化方法，它利用神经网络权重的分布结构来实现近乎无损的压缩。 PolarQuant 分三个阶段运行：(1) 对单位超球面进行块式归一化，(2) Walsh-Hadamard 旋转将坐标转换为近似高斯随机变量，以及 (3) 质心与高斯分布匹配的量化。我们的消融表明，Hadamard 旋转本身就占质量改进的 98%，将 Qwen3.5-9B 困惑度从 6.90 (absmax Q5) 降低到 6.40 (Delta = +0.03 from FP16)，使其在没有任何校准数据的情况下几乎无损。此外，PolarQuant 还可以作为下游 INT4 量化器的有效预处理步骤：由 torchao INT4 去量化和重新量化的 PolarQuant Q5 的困惑度为 6.56，而直接 absmax INT4 的困惑度为 6.68，同时在 6.5 GB VRAM 下保持 43.1 tok/s 的吞吐量。代码和模型是公开的。

Title: APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay

• Authors: Pratyay Banerjee, Masud Moshtaghi, Ankit Chadha
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2603.29093
• Pdf URL: https://arxiv.org/pdf/2603.29093
• Copy Paste: [[2603.29093]] APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay(https://arxiv.org/abs/2603.29093)
• Keywords: llm, agent
• Abstract: LLM-based autonomous agents lack persistent procedural memory: they re-derive solutions from scratch even when structurally identical tasks have been solved before. We present \textbf{APEX-EM}, a non-parametric online learning framework that accumulates, retrieves, and reuses structured procedural plans without modifying model weights. APEX-EM introduces: (1) a \emph{structured experience representation} encoding the full procedural-episodic trace of each execution -- planning steps, artifacts, iteration history with error analysis, and quality scores; (2) a \emph{Plan-Retrieve-Generate-Iterate-Ingest} (PRGII) workflow with Task Verifiers providing multi-dimensional reward signals; and (3) a \emph{dual-outcome Experience Memory} with hybrid retrieval combining semantic search, structural signature matching, and plan DAG traversal -- enabling cross-domain transfer between tasks sharing no lexical overlap but analogous operational structure. Successful experiences serve as positive in-context examples; failures as negative examples with structured error annotations. We evaluate on BigCodeBench~\cite{zhuo2025bigcodebench}, KGQAGen-10k~\cite{zhang2025kgqagen}, and Humanity's Last Exam~\cite{phan2025hle} using Claude Sonnet 4.5 and Opus 4.5. On KGQAGen-10k, APEX-EM achieves 89.6\% accuracy versus 41.3\% without memory (+48.3pp), surpassing the oracle-retrieval upper bound (84.9\%). On BigCodeBench, it reaches 83.3\% SR from a 53.9\% baseline (+29.4pp), exceeding MemRL's~\cite{memrl2025} +11.0pp gain under comparable frozen-backbone conditions (noting backbone differences controlled for in our analysis). On HLE, entity graph retrieval reaches 48.0\% from 25.2\% (+22.8pp). Ablations show component value is task-dependent: rich judge feedback is negligible for code generation but critical for structured queries (+10.3pp), while binary-signal iteration partially compensates for weaker feedback.
• 摘要：基于 LLM 的自主代理缺乏持久的程序记忆：即使之前已经解决了结构相同的任务，它们也会从头开始重新推导解决方案。我们提出了 \textbf{APEX-EM}，一种非参数在线学习框架，可以累积、检索和重用结构化程序计划，而无需修改模型权重。 APEX-EM 引入了：（1）\emph{结构化体验表示}，编码每次执行的完整程序情景跟踪——规划步骤、工件、带有错误分析的迭代历史记录以及质量分数； （2）\emph{计划-检索-生成-迭代-摄取}（PRGII）工作流程，其中任务验证器提供多维奖励信号； (3) 具有混合检索功能的 \emph{双结果体验记忆}，结合了语义搜索、结构签名匹配和计划 DAG 遍历 - 实现不共享词汇重叠但具有类似操作结构的任务之间的跨域传输。成功的经验可以作为积极的实例；失败作为带有结构化错误注释的反面例子。我们使用 Claude Sonnet 4.5 和 Opus 4.5 对 BigCodeBench~\cite{zhuo2025bigcodebench}、KGQAGen-10k~\cite{zhang2025kgqagen} 和 Humanity's Last Exam~\cite{phan2025hle} 进行评估。在 KGQAGen-10k 上，APEX-EM 的准确率达到 89.6%，而没有内存时准确率为 41.3% (+48.3pp)，超过了预言机检索上限 (84.9%)。在 BigCodeBench 上，它从 53.9% 的基线（+29.4pp）达到 83.3% SR，超过了 MemRL 在可比较的冻结主干条件下的 ~\cite{memrl2025} +11.0pp 增益（注意我们分析中控制的主干差异）。在 HLE 上，实体图检索从 25.2% 达到 48.0%（+22.8pp）。消融显示组件价值取决于任务：丰富的判断反馈对于代码生成可以忽略不计，但对于结构化查询至关重要（+10.3pp），而二进制信号迭代部分补偿了较弱的反馈。

Title: Concept Training for Human-Aligned Language Models

• Authors: Christine Zhang, Dan Jurafsky, Chen Shani
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29123
• Pdf URL: https://arxiv.org/pdf/2603.29123
• Copy Paste: [[2603.29123]] Concept Training for Human-Aligned Language Models(https://arxiv.org/abs/2603.29123)
• Keywords: language model
• Abstract: The next-token prediction (NTP) objective trains language models to predict a single continuation token at each step. In natural language, however, a prefix can be continued in many valid ways, and even similar meanings may differ in surface form. For example, the sentence ``this website is safe to \underline{browse}'' could plausibly continue with words such as browse, search, visit, surf, or navigate. While standard NTP training treats these alternatives as mutually exclusive targets, we explore a framework that instead predicts concepts, approximated as sets of semantically related tokens. We show that models trained with concept supervision exhibit stronger alignment with human semantic similarity judgments on multiple lexical benchmarks. These gains are accompanied by lower perplexity on semantically meaningful words (definition in Section 3.1), and a modest increase in global token-level perplexity, reflecting a tradeoff between standard NTP optimization and concept-level supervision. Our results suggest that concept-level objectives can improve semantic alignment while maintaining competitive language modeling performance.
• 摘要：下一个标记预测 (NTP) 目标训练语言模型以在每一步预测单个延续标记。然而，在自然语言中，前缀可以通过多种有效方式延续，甚至相似的含义在表面形式上也可能不同。例如，句子“这个网站可以安全地\下划线{浏览}”可以合理地继续使用浏览、搜索、访问、冲浪或导航等词。虽然标准 NTP 训练将这些替代方案视为互斥的目标，但我们探索了一个预测概念的框架，近似为语义相关标记集。我们表明，经过概念监督训练的模型在多个词汇基准上表现出与人类语义相似性判断更强的一致性。这些收益伴随着语义上有意义的单词的困惑度降低（第 3.1 节中的定义），以及全局令牌级困惑度的适度增加，反映了标准 NTP 优化和概念级监督之间的权衡。我们的结果表明，概念级目标可以改善语义对齐，同时保持有竞争力的语言建模性能。

Title: Kwame 2.0: Human-in-the-Loop Generative AI Teaching Assistant for Large Scale Online Coding Education in Africa

• Authors: George Boateng, Samuel Boateng, Victor Kumbol
• Subjects: cs.CL, cs.CY, cs.HC
• Abstract URL: https://arxiv.org/abs/2603.29159
• Pdf URL: https://arxiv.org/pdf/2603.29159
• Copy Paste: [[2603.29159]] Kwame 2.0: Human-in-the-Loop Generative AI Teaching Assistant for Large Scale Online Coding Education in Africa(https://arxiv.org/abs/2603.29159)
• Keywords: llm, retrieval-augmented generation
• Abstract: Providing timely and accurate learning support in large-scale online coding courses is challenging, particularly in resource-constrained contexts. We present Kwame 2.0, a bilingual (English-French) generative AI teaching assistant built using retrieval-augmented generation and deployed in a human-in-the-loop forum within SuaCode, an introductory mobile-based coding course for learners across Africa. Kwame 2.0 retrieves relevant course materials and generates context-aware responses while encouraging human oversight and community participation. We deployed the system in a 15-month longitudinal study spanning 15 cohorts with 3,717 enrollments across 35 African countries. Evaluation using community feedback and expert ratings shows that Kwame 2.0 provided high-quality and timely support, achieving high accuracy on curriculum-related questions, while human facilitators and peers effectively mitigated errors, particularly for administrative queries. Our findings demonstrate that human-in-the-loop generative AI systems can combine the scalability and speed of AI with the reliability of human support, offering an effective approach to learning assistance for underrepresented populations in resource-constrained settings at scale.
• 摘要：在大规模在线编码课程中提供及时、准确的学习支持具有挑战性，特别是在资源有限的情况下。我们推出了 Kwame 2.0，这是一种双语（英语-法语）生成式 AI 教学助手，使用检索增强生成技术构建，并部署在 SuaCode 内的人机交互论坛中，SuaCode 是面向非洲学习者的基于移动设备的入门编码课程。 Kwame 2.0 检索相关课程材料并生成情境感知响应，同时鼓励人类监督和社区参与。我们在一项为期 15 个月的纵向研究中部署了该系统，该研究涵盖 35 个非洲国家的 15 个队列、3,717 名受试者。使用社区反馈和专家评分进行的评估表明，Kwame 2.0 提供了高质量和及时的支持，在课程相关问题上实现了高精度，而人类辅导员和同行有效地减少了错误，特别是在管理查询方面。我们的研究结果表明，人机循环生成人工智能系统可以将人工智能的可扩展性和速度与人类支持的可靠性相结合，为资源有限环境中的代表性不足的人群提供一种有效的学习帮助方法。

Title: SyriSign: A Parallel Corpus for Arabic Text to Syrian Arabic Sign Language Translation

• Authors: Mohammad Amer Khalil, Raghad Nahas, Ahmad Nassar, Khloud Al Jallad
• Subjects: cs.CL, cs.AI, cs.CV, cs.HC
• Abstract URL: https://arxiv.org/abs/2603.29219
• Pdf URL: https://arxiv.org/pdf/2603.29219
• Copy Paste: [[2603.29219]] SyriSign: A Parallel Corpus for Arabic Text to Syrian Arabic Sign Language Translation(https://arxiv.org/abs/2603.29219)
• Keywords: gpt
• Abstract: Sign language is the primary approach of communication for the Deaf and Hard-of-Hearing (DHH) community. While there are numerous benchmarks for high-resource sign languages, low-resource languages like Arabic remain underrepresented. Currently, there is no publicly available dataset for Syrian Arabic Sign Language (SyArSL). To overcome this gap, we introduce SyriSign, a dataset comprising 1500 video samples across 150 unique lexical signs, designed for text-to-SyArSL translation tasks. This work aims to reduce communication barriers in Syria, as most news are delivered in spoken or written Arabic, which is often inaccessible to the deaf community. We evaluated SyriSign using three deep learning architectures: MotionCLIP for semantic motion generation, T2M-GPT for text-conditioned motion synthesis, and SignCLIP for bilingual embedding alignment. Experimental results indicate that while generative approaches show strong potential for sign representation, the limited dataset size constrains generalization performance. We will release SyriSign publicly, hoping it serves as an initial benchmark.
• 摘要：手语是聋人和听力障碍 (DHH) 群体的主要沟通方式。尽管高资源手语有许多基准，但阿拉伯语等低资源语言的代表性仍然不足。目前，叙利亚阿拉伯手语 (SyArSL) 没有公开可用的数据集。为了克服这一差距，我们引入了 SyriSign，这是一个包含 1500 个视频样本、涵盖 150 个独特词汇符号的数据集，专为文本到 SyArSL 翻译任务而设计。这项工作旨在减少叙利亚的沟通障碍，因为大多数新闻都是以阿拉伯语口头或书面形式传达的，而聋人群体往往无法接触到阿拉伯语。我们使用三种深度学习架构评估了 SyriSign：用于语义运动生成的 MotionCLIP、用于文本条件运动合成的 T2M-GPT 以及用于双语嵌入对齐的 SignCLIP。实验结果表明，虽然生成方法在符号表示方面表现出强大的潜力，但有限的数据集大小限制了泛化性能。我们将公开发布 SyriSign，希望它能够作为初步基准。

Title: SiPaKosa: A Comprehensive Corpus of Canonical and Classical Buddhist Texts in Sinhala and Pali

• Authors: Ranidu Gurusinghe, Nevidu Jayatilleke
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29221
• Pdf URL: https://arxiv.org/pdf/2603.29221
• Copy Paste: [[2603.29221]] SiPaKosa: A Comprehensive Corpus of Canonical and Classical Buddhist Texts in Sinhala and Pali(https://arxiv.org/abs/2603.29221)
• Keywords: language model
• Abstract: SiPaKosa is a comprehensive corpus of Sinhala and Pali doctrinal texts comprising approximately 786K sentences and 9.25M words, incorporating 16 copyright-cleared historical Buddhist documents alongside the complete web-scraped Tripitaka canonical texts. The corpus was created through high-quality OCR using Google Document AI on historical manuscripts, combined with systematic web scraping of canonical repositories, followed by rigorous quality control and metadata annotation. The corpus is organised into language-specific subcorpora: Sinhala and Mixed Sinhala-Pali. We evaluate the performance of language models using ten pretrained models, with perplexity scores ranging from 1.09 to 189.67 on our corpus. This analysis shows that proprietary models significantly outperform open-source alternatives by factors of three to six times. This corpus supports the pretraining of domain-adapted language models, facilitates historical language analysis, and aids in the development of information retrieval systems for Buddhist scholarship while preserving Sinhala cultural heritage.
• 摘要：SiPaKosa 是僧伽罗语和巴利语教义文本的综合语料库，包含约 786K 句子和 925 万个单词，包含 16 个版权已清除的历史佛教文献以及完整的网络抓取的大藏经经典文本。该语料库是通过使用 Google Document AI 对历史手稿进行高质量 OCR 创建的，结合规范存储库的系统网络抓取，然后进行严格的质量控制和元数据注释。该语料库被组织成特定语言的子语料库：僧伽罗语和混合僧伽罗语-巴利语。我们使用 10 个预训练模型来评估语言模型的性能，我们的语料库上的困惑度分数范围为 1.09 到 189.67。该分析表明，专有模型的性能明显优于开源替代方案三到六倍。该语料库支持领域适应语言模型的预训练，促进历史语言分析，并有助于开发佛教学术信息检索系统，同时保护僧伽罗文化遗产。

Title: Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs

• Authors: Zhuowen Liang, Xiaotian Lin, Zhengxuan Zhang, Yuyu Luo, Haixun Wang, Nan Tang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.29232
• Pdf URL: https://arxiv.org/pdf/2603.29232
• Copy Paste: [[2603.29232]] Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs(https://arxiv.org/abs/2603.29232)
• Keywords: language model, gpt, llm
• Abstract: Large language models (LLMs) are widely applied to data analytics over documents, yet direct reasoning over long, noisy documents remains brittle and error-prone. Hence, we study document question answering (QA) that consolidates dispersed evidence into a structured output (e.g., a table, graph, or chunks) to support reliable, verifiable QA. We propose a two-pillar framework, LiteCoST, to achieve both high accuracy and low latency with small language models (SLMs). Pillar 1: Chain-of-Structured-Thought (CoST). We introduce a CoST template, a schema-aware instruction that guides a strong LLM to produce both a step-wise CoST trace and the corresponding structured output. The process induces a minimal structure, normalizes entities/units, aligns records, serializes the output, and verifies/refines it, yielding auditable supervision. Pillar 2: SLM fine-tuning. The compact models are trained on LLM-generated CoST data in two stages: Supervised Fine-Tuning for structural alignment, followed by Group Relative Policy Optimization (GRPO) incorporating triple rewards for answer/format quality and process consistency. By distilling structure-first behavior into SLMs, this approach achieves LLM-comparable quality on multi-domain long-document QA using 3B/7B SLMs, while delivering 2-4x lower latency than GPT-4o and DeepSeek-R1 (671B). The code is available at this https URL.
• 摘要：大型语言模型 (LLM) 广泛应用于文档数据分析，但对冗长、嘈杂的文档进行直接推理仍然脆弱且容易出错。因此，我们研究文档问答 (QA)，它将分散的证据整合为结构化输出（例如表格、图表或块），以支持可靠、可验证的 QA。我们提出了一个双支柱框架 LiteCoST，以通过小型语言模型 (SLM) 实现高精度和低延迟。支柱 1：结构化思维链 (CoST)。我们引入了 CoST 模板，这是一种模式感知指令，可指导强大的 LLM 生成逐步的 CoST 跟踪和相应的结构化输出。该过程产生最小的结构，规范实体/单元，对齐记录，序列化输出，并验证/细化它，从而产生可审计的监督。支柱 2：SLM 微调。紧凑模型分两个阶段在 LLM 生成的 CoST 数据上进行训练：用于结构对齐的监督微调，然后是组相对策略优化 (GRPO)，其中包含答案/格式质量和流程一致性的三重奖励。通过将结构优先行为提炼到 SLM 中，该方法使用 3B/7B SLM 在多域长文档 QA 上实现了与 LLM 相当的质量，同时延迟比 GPT-4o 和 DeepSeek-R1 (671B) 低 2-4 倍。该代码可从此 https URL 获取。

Title: MemRerank: Preference Memory for Personalized Product Reranking

• Authors: Zhiyuan Peng, Xuyang Wu, Huaixiao Tou, Yi Fang, Yi Gong
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.29247
• Pdf URL: https://arxiv.org/pdf/2603.29247
• Copy Paste: [[2603.29247]] MemRerank: Preference Memory for Personalized Product Reranking(https://arxiv.org/abs/2603.29247)
• Keywords: llm, prompt, agent
• Abstract: LLM-based shopping agents increasingly rely on long purchase histories and multi-turn interactions for personalization, yet naively appending raw history to prompts is often ineffective due to noise, length, and relevance mismatch. We propose MemRerank, a preference memory framework that distills user purchase history into concise, query-independent signals for personalized product reranking. To study this problem, we build an end-to-end benchmark and evaluation framework centered on an LLM-based \textbf{1-in-5} selection task, which measures both memory quality and downstream reranking utility. We further train the memory extractor with reinforcement learning (RL), using downstream reranking performance as supervision. Experiments with two LLM-based rerankers show that MemRerank consistently outperforms no-memory, raw-history, and off-the-shelf memory baselines, yielding up to \textbf{+10.61} absolute points in 1-in-5 accuracy. These results suggest that explicit preference memory is a practical and effective building block for personalization in agentic e-commerce systems.
• 摘要：基于法学硕士的购物代理越来越依赖于较长的购买历史记录和多轮交互来实现个性化，但由于噪音、长度和相关性不匹配，天真地将原始历史记录附加到提示中通常是无效的。我们提出了 MemRerank，这是一种偏好记忆框架，可将用户购买历史记录提炼为简洁的、独立于查询的信号，以进行个性化产品重新排名。为了研究这个问题，我们构建了一个以基于 LLM 的 \textbf{1-in-5} 选择任务为中心的端到端基准测试和评估框架，该任务可衡量内存质量和下游重新排名效用。我们使用下游重排序性能作为监督，通过强化学习（RL）进一步训练记忆提取器。使用两个基于 LLM 的重新排序器进行的实验表明，MemRerank 始终优于无内存、原始历史和现成内存基线，以 1-in-5 的精度产生高达 \textbf{+10.61} 的绝对点。这些结果表明，显式偏好记忆是代理电子商务系统中个性化的实用且有效的构建模块。

Title: Open Machine Translation for Esperanto

• Authors: Ona de Gibert, Lluís de Gibert
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29345
• Pdf URL: https://arxiv.org/pdf/2603.29345
• Copy Paste: [[2603.29345]] Open Machine Translation for Esperanto(https://arxiv.org/abs/2603.29345)
• Keywords: llm
• Abstract: Esperanto is a widespread constructed language, known for its regular grammar and productive word formation. Besides having substantial resources available thanks to its online community, it remains relatively underexplored in the context of modern machine translation (MT) approaches. In this work, we present the first comprehensive evaluation of open-source MT systems for Esperanto, comparing rule-based systems, encoder-decoder models, and LLMs across model sizes. We evaluate translation quality across six language directions involving English, Spanish, Catalan, and Esperanto using multiple automatic metrics as well as human evaluation. Our results show that the NLLB family achieves the best performance in all language pairs, followed closely by our trained compact models and a fine-tuned general-purpose LLM. Human evaluation confirms this trend, with NLLB translations preferred in approximately half of the comparisons, although noticeable errors remain. In line with Esperanto's tradition of openness and international collaboration, we release our code and best-performing models publicly.
• 摘要：世界语是一种广泛使用的人工语言，以其规则的语法和富有成效的构词法而闻名。除了由于其在线社区而拥有大量可用资源之外，它在现代机器翻译（MT）方法的背景下仍然相对未被充分开发。在这项工作中，我们首次对世界语开源机器翻译系统进行了全面评估，比较了不同模型大小的基于规则的系统、编码器-解码器模型和法学硕士。我们使用多种自动指标以及人工评估来评估涉及英语、西班牙语、加泰罗尼亚语和世界语的六种语言方向的翻译质量。我们的结果表明，NLLB 系列在所有语言对中均实现了最佳性能，紧随其后的是我们经过训练的紧凑模型和经过微调的通用 LLM。人工评估证实了这一趋势，尽管仍然存在明显的错误，但大约一半的比较中首选 NLLB 翻译。根据世界语开放和国际合作的传统，我们公开发布我们的代码和性能最佳的模型。

Title: Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations

• Authors: Yahan Li, Xinyi Jie, Wanjia Ruan, Xubei Zhang, Huaijie Zhu, Yicheng Gao, Chaohao Du, Ruishan Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29373
• Pdf URL: https://arxiv.org/pdf/2603.29373
• Copy Paste: [[2603.29373]] Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations(https://arxiv.org/abs/2603.29373)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are increasingly used for medical consultation and health information support. In this high-stakes setting, safety depends not only on medical knowledge, but also on how models respond when patient inputs are unclear, inconsistent, or misleading. However, most existing medical LLM evaluations assume idealized and well-posed patient questions, which limits their realism. In this paper, we study challenging patient behaviors that commonly arise in real medical consultations and complicate safe clinical reasoning. We define four clinically grounded categories of such behaviors: information contradiction, factual inaccuracy, self-diagnosis, and care resistance. For each behavior, we specify concrete failure criteria that capture unsafe responses. Building on four existing medical dialogue datasets, we introduce CPB-Bench (Challenging Patient Behaviors Benchmark), a bilingual (English and Chinese) benchmark of 692 multi-turn dialogues annotated with these behaviors. We evaluate a range of open- and closed-source LLMs on their responses to challenging patient utterances. While models perform well overall, we identify consistent, behavior-specific failure patterns, with particular difficulty in handling contradictory or medically implausible patient information. We also study four intervention strategies and find that they yield inconsistent improvements and can introduce unnecessary corrections. We release the dataset and code.
• 摘要：大语言模型（LLM）越来越多地用于医疗咨询和健康信息支持。在这种高风险环境中，安全性不仅取决于医学知识，还取决于模型在患者输入不明确、不一致或误导时的反应。然而，大多数现有的医学法学硕士评估都假设理想化且适当的患者问题，这限制了其现实性。在本文中，我们研究了真实医疗咨询中常见的具有挑战性的患者行为，这些行为使安全临床推理变得复杂。我们将此类行为定义为四种基于临床的类别：信息矛盾、事实不准确、自我诊断和护理抵制。对于每种行为，我们指定了捕获不安全响应的具体失败标准。基于四个现有的医学对话数据集，我们引入了 CPB-Bench（挑战患者行为基准），这是一个双语（英语和中文）基准，包含 692 个带有这些行为注释的多回合对话。我们评估了一系列开源和闭源法学硕士对挑战性患者话语的反应。虽然模型总体表现良好，但我们识别出一致的、特定于行为的失败模式，在处理矛盾或医学上不可信的患者信息时尤其困难。我们还研究了四种干预策略，发现它们产生的改进不一致，并且可能引入不必要的纠正。我们发布数据集和代码。

Title: Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity

• Authors: Zoë Prins, Samuele Punzo, Frank Wildenburg, Giovanni Cinà, Sandro Pezzelle
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29396
• Pdf URL: https://arxiv.org/pdf/2603.29396
• Copy Paste: [[2603.29396]] Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity(https://arxiv.org/abs/2603.29396)
• Keywords: language model, llm
• Abstract: Standard evaluations of Large language models (LLMs) focus on task performance, offering limited insight into whether correct behavior reflects appropriate underlying mechanisms and risking confirmation bias. We introduce a simple, principled interpretability framework based on token-level perplexity to test whether models rely on linguistically relevant cues. By comparing perplexity distributions over minimal sentence pairs differing in one or a few `pivotal' tokens, our method enables precise, hypothesis-driven analysis without relying on unstable feature-attribution techniques. Experiments on controlled linguistic benchmarks with several open-weight LLMs show that, while linguistically important tokens influence model behavior, they never fully explain perplexity shifts, revealing that models rely on heuristics other than the expected linguistic ones.
• 摘要：大型语言模型 (LLM) 的标准评估侧重于任务绩效，对于正确行为是否反映了适当的潜在机制以及存在确认偏差的风险提供了有限的见解。我们引入了一个基于标记级困惑度的简单、有原则的可解释性框架，以测试模型是否依赖于语言相关的线索。通过比较在一个或几个“关键”标记上不同的最小句子对的困惑度分布，我们的方法能够实现精确的、假设驱动的分析，而不依赖于不稳定的特征归因技术。使用几个开放权重法学硕士对受控语言基准进行的实验表明，虽然语言上重要的标记会影响模型行为，但它们从未完全解释困惑度变化，这表明模型依赖于启发式方法，而不是预期的语言学方法。

Title: CounselReflect: A Toolkit for Auditing Mental-Health Dialogues

• Authors: Yahan Li, Chaohao Du, Zeyang Li, Christopher Chun Kuizon, Shupeng Cheng, Angel Hsing-Chi Hwang, Adam C. Frank, Ruishan Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29429
• Pdf URL: https://arxiv.org/pdf/2603.29429
• Copy Paste: [[2603.29429]] CounselReflect: A Toolkit for Auditing Mental-Health Dialogues(https://arxiv.org/abs/2603.29429)
• Keywords: llm
• Abstract: Mental-health support is increasingly mediated by conversational systems (e.g., LLM-based tools), but users often lack structured ways to audit the quality and potential risks of the support they receive. We introduce CounselReflect, an end-to-end toolkit for auditing mental-health support dialogues. Rather than producing a single opaque quality score, CounselReflect provides structured, multi-dimensional reports with session-level summaries, turn-level scores, and evidence-linked excerpts to support transparent inspection. The system integrates two families of evaluation signals: (i) 12 model-based metrics produced by task-specific predictors, and (ii) rubric-based metrics that extend coverage via a literature-derived library (69 metrics) and user-defined custom metrics, operationalized with configurable LLM judges. CounselReflect is available as a web application, browser extension, and command-line interface (CLI), enabling use in real-time settings as well as at scale. Human evaluation includes a user study with 20 participants and an expert review with 6 mental-health professionals, suggesting that CounselReflect supports understandable, usable, and trustworthy auditing. A demo video and full source code are also provided.
• 摘要：心理健康支持越来越多地通过对话系统（例如基于法学硕士的工具）来调节，但用户往往缺乏结构化的方法来审核他们所获得的支持的质量和潜在风险。我们推出 CounselReflect，这是一个用于审核心理健康支持对话的端到端工具包。 CounselReflect 不是生成单一的不透明质量分数，而是提供结构化、多维报告，其中包含会话级摘要、回合级分数和与证据相关的摘录，以支持透明检查。该系统集成了两个评估信号系列：（i）由特定于任务的预测器生成的 12 个基于模型的指标，以及（ii）基于标题的指标，通过文献衍生库（69 个指标）和用户定义的自定义指标扩展覆盖范围，并通过可配置的 LLM 法官进行操作。 CounselReflect 可作为 Web 应用程序、浏览器扩展和命令行界面 (CLI) 使用，支持实时设置和大规模使用。人工评估包括 20 名参与者的用户研究和 6 名心理健康专业人员的专家评审，表明 CounselReflect 支持可理解、可用且值得信赖的审核。还提供演示视频和完整源代码。

Title: Authorship Impersonation via LLM Prompting does not Evade Authorship Verification Methods

• Authors: Baoyi Zeng, Andrea Nini
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29454
• Pdf URL: https://arxiv.org/pdf/2603.29454
• Copy Paste: [[2603.29454]] Authorship Impersonation via LLM Prompting does not Evade Authorship Verification Methods(https://arxiv.org/abs/2603.29454)
• Keywords: language model, gpt, llm, prompt
• Abstract: Authorship verification (AV), the task of determining whether a questioned text was written by a specific individual, is a critical part of forensic linguistics. While manual authorial impersonation by perpetrators has long been a recognized threat in historical forensic cases, recent advances in large language models (LLMs) raise new challenges, as adversaries may exploit these tools to impersonate another's writing. This study investigates whether prompted LLMs can generate convincing authorial impersonations and whether such outputs can evade existing forensic AV systems. Using GPT-4o as the adversary model, we generated impersonation texts under four prompting conditions across three genres: emails, text messages, and social media posts. We then evaluated these outputs against both non-neural AV methods (n-gram tracing, Ranking-Based Impostors Method, LambdaG) and neural approaches (AdHominem, LUAR, STAR) within a likelihood-ratio framework. Results show that LLM-generated texts failed to sufficiently replicate authorial individuality to bypass established AV systems. We also observed that some methods achieved even higher accuracy when rejecting impersonation texts compared to genuine negative samples. Overall, these findings indicate that, despite the accessibility of LLMs, current AV systems remain robust against entry-level impersonation attempts across multiple genres. Furthermore, we demonstrate that this counter-intuitive resilience stems, at least in part, from the higher lexical diversity and entropy inherent in LLM-generated texts.
• 摘要：作者身份验证（AV）是确定受质疑的文本是否由特定个人撰写的任务，是法庭语言学的重要组成部分。虽然犯罪者手动模仿作者长期以来一直是历史取证案件中公认的威胁，但大型语言模型 (LLM) 的最新进展提出了新的挑战，因为对手可能会利用这些工具来模仿他人的写作。这项研究调查了提示法学硕士是否可以生成令人信服的作者模仿，以及此类输出是否可以逃避现有的法医反病毒系统。使用 GPT-4o 作为对手模型，我们在三种类型的四种提示条件下生成了模拟文本：电子邮件、短信和社交媒体帖子。然后，我们在似然比框架内针对非神经 AV 方法（n-gram 追踪、基于排名的冒名顶替者方法、LambdaG）和神经方法（AdHominem、LUAR、STAR）评估这些输出。结果表明，法学硕士生成的文本未能充分复制作者的个性以绕过已建立的反病毒系统。我们还观察到，与真实的负样本相比，某些方法在拒绝模仿文本时取得了更高的准确性。总的来说，这些发现表明，尽管法学硕士很容易获得，但当前的反病毒系统对于跨多种流派的入门级模仿尝试仍然很强大。此外，我们证明这种反直觉的弹性至少部分源于法学硕士生成的文本中固有的更高的词汇多样性和熵。

Title: M-MiniGPT4: Multilingual VLLM Alignment via Translated Data

• Authors: Seung Hun Han, Youssef Mohamed, Mohamed Elhoseiny
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.29467
• Pdf URL: https://arxiv.org/pdf/2603.29467
• Copy Paste: [[2603.29467]] M-MiniGPT4: Multilingual VLLM Alignment via Translated Data(https://arxiv.org/abs/2603.29467)
• Keywords: language model, gpt, llm
• Abstract: This paper presents a Multilingual Vision Large Language Model, named M-MiniGPT4. Our model exhibits strong vision-language understanding (VLU) capabilities across 11 languages. We utilize a mixture of native multilingual and translated data to push the multilingual VLU performance of the MiniGPT4 architecture. In addition, we propose a multilingual alignment training stage that uses parallel text corpora to further enhance the multilingual capabilities of our model. M-MiniGPT4 achieves 36% accuracy on the multilingual MMMU benchmark, outperforming state-of-the-art models in the same weight class, including foundation models released after the majority of this work was completed. We open-source our models, code, and translated datasets to facilitate future research in low-resource and multilingual settings.
• 摘要：本文提出了一种多语言视觉大语言模型，命名为M-MiniGPT4。我们的模型展示了跨 11 种语言的强大视觉语言理解 (VLU) 能力。我们利用本地多语言和翻译数据的混合来推动 MiniGPT4 架构的多语言 VLU 性能。此外，我们提出了一个多语言对齐训练阶段，使用并行文本语料库来进一步增强我们模型的多语言能力。 M-MiniGPT4 在多语言 MMMU 基准上实现了 36% 的准确率，优于同一重量级别的最先进模型，包括大部分工作完成后发布的基础模型。我们开源我们的模型、代码和翻译的数据集，以促进未来在资源匮乏和多语言环境中的研究。

Title: Calibrated Confidence Expression for Radiology Report Generation

• Authors: David Bani-Harouni, Chantal Pellegrini, Julian Lüers, Su Hwan Kim, Markus Baalmann, Benedikt Wiestler, Rickmer Braren, Nassir Navab, Matthias Keicher
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29492
• Pdf URL: https://arxiv.org/pdf/2603.29492
• Copy Paste: [[2603.29492]] Calibrated Confidence Expression for Radiology Report Generation(https://arxiv.org/abs/2603.29492)
• Keywords: language model
• Abstract: Safe deployment of Large Vision-Language Models (LVLMs) in radiology report generation requires not only accurate predictions but also clinically interpretable indicators of when outputs should be thoroughly reviewed, enabling selective radiologist verification and reducing the risk of hallucinated findings influencing clinical decisions. One intuitive approach to this is verbalized confidence, where the model explicitly states its certainty. However, current state-of-the-art language models are often overconfident, and research on calibration in multimodal settings such as radiology report generation is limited. To address this gap, we introduce ConRad (Confidence Calibration for Radiology Reports), a reinforcement learning framework for fine-tuning medical LVLMs to produce calibrated verbalized confidence estimates alongside radiology reports. We study two settings: a single report-level confidence score and a sentence-level variant assigning a confidence to each claim. Both are trained using the GRPO algorithm with reward functions based on the logarithmic scoring rule, which incentivizes truthful self-assessment by penalizing miscalibration and guarantees optimal calibration under reward maximization. Experimentally, ConRad substantially improves calibration and outperforms competing methods. In a clinical evaluation we show that ConRad's report level scores are well aligned with clinicians' judgment. By highlighting full reports or low-confidence statements for targeted review, ConRad can support safer clinical integration of AI-assistance for report generation.
• 摘要：在放射学报告生成中安全部署大型视觉语言模型 (LVLM) 不仅需要准确的预测，还需要临床可解释的指标，以表明何时应彻底审查输出，从而使放射科医生能够进行选择性验证并降低影响临床决策的幻觉结果的风险。一种直观的方法是用语言表达置信度，即模型明确说明其确定性。然而，当前最先进的语言模型往往过于自信，并且在放射学报告生成等多模态环境中进行校准的研究也很有限。为了解决这一差距，我们引入了 ConRad（放射学报告置信度校准），这是一种强化学习框架，用于微调医学 LVLM，以在放射学报告的同时生成经过校准的语言化置信度估计。我们研究两种设置：单个报告级置信度得分和为每个声明分配置信度的句子级变体。两者均使用 GRPO 算法进行训练，奖励函数基于对数评分规则，通过惩罚错误校准来激励真实的自我评估，并保证奖励最大化下的最佳校准。在实验上，ConRad 显着改进了校准并优于竞争方法。在临床评估中，我们表明 ConRad 的报告水平分数与临床医生的判断非常一致。通过突出显示完整报告或低置信度声明进行有针对性的审查，ConRad 可以支持更安全地临床整合人工智能辅助报告生成。

Title: MemFactory: Unified Inference & Training Framework for Agent Memory

• Authors: Ziliang Guo, Ziheng Li, Zhiyu Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.29493
• Pdf URL: https://arxiv.org/pdf/2603.29493
• Copy Paste: [[2603.29493]] MemFactory: Unified Inference & Training Framework for Agent Memory(https://arxiv.org/abs/2603.29493)
• Keywords: language model, llm, agent
• Abstract: Memory-augmented Large Language Models (LLMs) are essential for developing capable, long-term AI agents. Recently, applying Reinforcement Learning (RL) to optimize memory operations, such as extraction, updating, and retrieval, has emerged as a highly promising research direction. However, existing implementations remain highly fragmented and task-specific, lacking a unified infrastructure to streamline the integration, training, and evaluation of these complex pipelines. To address this gap, we present MemFactory, the first unified, highly modular training and inference framework specifically designed for memory-augmented agents. Inspired by the success of unified fine-tuning frameworks like LLaMA-Factory, MemFactory abstracts the memory lifecycle into atomic, plug-and-play components, enabling researchers to seamlessly construct custom memory agents via a "Lego-like" architecture. Furthermore, the framework natively integrates Group Relative Policy Optimization (GRPO) to fine-tune internal memory management policies driven by multi-dimensional environmental rewards. MemFactory provides out-of-the-box support for recent cutting-edge paradigms, including Memory-R1, RMM, and MemAgent. We empirically validate MemFactory on the open-source MemAgent architecture using its publicly available training and evaluation data. Across both in-domain and out-of-distribution evaluation sets, MemFactory consistently improves performance over the corresponding base models, with relative gains of up to 14.8%. By providing a standardized, extensible, and easy-to-use infrastructure, MemFactory significantly lowers the barrier to entry, paving the way for future innovations in memory-driven AI agents.
• 摘要：记忆增强大型语言模型 (LLM) 对于开发功能强大的长期 AI 代理至关重要。最近，应用强化学习（RL）来优化提取、更新和检索等记忆操作，已成为一个非常有前途的研究方向。然而，现有的实施仍然高度分散且特定于任务，缺乏统一的基础设施来简化这些复杂管道的集成、培训和评估。为了解决这一差距，我们推出了 MemFactory，这是第一个专门为记忆增强代理设计的统一、高度模块化的训练和推理框架。受到 LLaMA-Factory 等统一微调框架成功的启发，MemFactory 将内存生命周期抽象为原子、即插即用组件，使研究人员能够通过“乐高式”架构无缝构建自定义内存代理。此外，该框架本身集成了组相对策略优化（GRPO），以微调由多维环境奖励驱动的内存管理策略。 MemFactory 为最新的前沿范例提供开箱即用的支持，包括 Memory-R1、RMM 和 MemAgent。我们使用公开的训练和评估数据在开源 MemAgent 架构上对 MemFactory 进行实证验证。在域内和分布外评估集上，MemFactory 始终比相应的基础模型提高性能，相对增益高达 14.8%。通过提供标准化、可扩展且易于使用的基础设施，MemFactory 显着降低了进入门槛，为内存驱动的 AI 代理的未来创新铺平了道路。

Title: Distilling Human-Aligned Privacy Sensitivity Assessment from Large Language Models

• Authors: Gabriel Loiseau, Damien Sileo, Damien Riquet, Maxime Meyer, Marc Tommasi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29497
• Pdf URL: https://arxiv.org/pdf/2603.29497
• Copy Paste: [[2603.29497]] Distilling Human-Aligned Privacy Sensitivity Assessment from Large Language Models(https://arxiv.org/abs/2603.29497)
• Keywords: language model, llm
• Abstract: Accurate privacy evaluation of textual data remains a critical challenge in privacy-preserving natural language processing. Recent work has shown that large language models (LLMs) can serve as reliable privacy evaluators, achieving strong agreement with human judgments; however, their computational cost and impracticality for processing sensitive data at scale limit real-world deployment. We address this gap by distilling the privacy assessment capabilities of Mistral Large 3 (675B) into lightweight encoder models with as few as 150M parameters. Leveraging a large-scale dataset of privacy-annotated texts spanning 10 diverse domains, we train efficient classifiers that preserve strong agreement with human annotations while dramatically reducing computational requirements. We validate our approach on human-annotated test data and demonstrate its practical utility as an evaluation metric for de-identification systems.
• 摘要：文本数据的准确隐私评估仍然是保护隐私的自然语言处理中的关键挑战。最近的工作表明，大型语言模型（LLM）可以作为可靠的隐私评估器，与人类的判断达成强烈一致；然而，它们的计算成本和大规模处理敏感数据的不切实际限制了实际部署。我们通过将 Mistral Large 3 (675B) 的隐私评估功能提炼成参数少至 150M 的轻量级编码器模型来解决这一差距。利用跨越 10 个不同领域的隐私注释文本的大规模数据集，我们训练高效的分类器，这些分类器与人类注释保持强烈的一致性，同时大大降低计算要求。我们在人工注释的测试数据上验证了我们的方法，并证明了其作为去识别系统评估指标的实用性。

Title: LLM Probe: Evaluating LLMs for Low-Resource Languages

• Authors: Hailay Kidu Teklehaymanot, Gebrearegawi Gebremariam, Wolfgang Nejdl
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29517
• Pdf URL: https://arxiv.org/pdf/2603.29517
• Copy Paste: [[2603.29517]] LLM Probe: Evaluating LLMs for Low-Resource Languages(https://arxiv.org/abs/2603.29517)
• Keywords: language model, llm
• Abstract: Despite rapid advances in large language models (LLMs), their linguistic abilities in low-resource and morphologically rich languages are still not well understood due to limited annotated resources and the absence of standardized evaluation frameworks. This paper presents LLM Probe, a lexicon-based assessment framework designed to systematically evaluate the linguistic skills of LLMs in low-resource language environments. The framework analyzes models across four areas of language understanding: lexical alignment, part-of-speech recognition, morphosyntactic probing, and translation accuracy. To illustrate the framework, we create a manually annotated benchmark dataset using a low-resource Semitic language as a case study. The dataset comprises bilingual lexicons with linguistic annotations, including part-of-speech tags, grammatical gender, and morphosyntactic features, which demonstrate high inter-annotator agreement to ensure reliable annotations. We test a variety of models, including causal language models and sequence-to-sequence architectures. The results reveal notable differences in performance across various linguistic tasks: sequence-to-sequence models generally excel in morphosyntactic analysis and translation quality, whereas causal models demonstrate strong performance in lexical alignment but exhibit weaker translation accuracy. Our results emphasize the need for linguistically grounded evaluation to better understand LLM limitations in low-resource settings. We release LLM Probe and the accompanying benchmark dataset as open-source tools to promote reproducible benchmarking and to support the development of more inclusive multilingual language technologies.
• 摘要：尽管大型语言模型（LLM）取得了快速进展，但由于注释资源有限和缺乏标准化评估框架，它们在资源匮乏和形态丰富的语言中的语言能力仍然没有得到很好的理解。本文提出了 LLM Probe，一个基于词典的评估框架，旨在系统地评估低资源语言环境中法学硕士的语言技能。该框架分析了语言理解四个领域的模型：词汇对齐、词性识别、形态句法探测和翻译准确性。为了说明该框架，我们使用资源匮乏的闪族语言创建一个手动注释的基准数据集作为案例研究。该数据集包含带有语言注释的双语词典，包括词性标签、语法性别和形态句法特征，表现出注释者之间的高度一致性，以确保可靠的注释。我们测试了各种模型，包括因果语言模型和序列到序列架构。结果揭示了各种语言任务的性能存在显着差异：序列到序列模型通常在词法分析和翻译质量方面表现出色，而因果模型在词汇对齐方面表现出很强的性能，但翻译准确性较差。我们的结果强调需要进行基于语言的评估，以更好地了解资源匮乏环境中法学硕士的局限性。我们将 LLM Probe 和随附的基准数据集作为开源工具发布，以促进可重复的基准测试并支持更具包容性的多语言语言技术的开发。

Title: Baby Scale: Investigating Models Trained on Individual Children's Language Input

• Authors: Steven Y. Feng, Alvin W.M. Tan, Michael C. Frank
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.29522
• Pdf URL: https://arxiv.org/pdf/2603.29522
• Copy Paste: [[2603.29522]] Baby Scale: Investigating Models Trained on Individual Children's Language Input(https://arxiv.org/abs/2603.29522)
• Keywords: language model
• Abstract: Modern language models (LMs) must be trained on many orders of magnitude more words of training data than human children receive before they begin to produce useful behavior. Assessing the nature and origins of this "data gap" requires benchmarking LMs on human-scale datasets to understand how linguistic knowledge emerges from children's natural training data. Using transcripts from the BabyView dataset (videos from children ages 6-36 months), we investigate (1) scaling performance at child-scale data regimes, (2) variability in model performance across datasets from different children's experiences and linguistic predictors of dataset quality, and (3) relationships between model and child language learning outcomes. LMs trained on child data show acceptable scaling for grammar tasks, but lower scaling on semantic and world knowledge tasks than models trained on synthetic data; we also observe substantial variability on data from different children. Beyond dataset size, performance is most associated with a combination of distributional and interactional linguistic features, broadly consistent with what makes high-quality input for child language development. Finally, model likelihoods for individual words correlate with children's learning of those words, suggesting that properties of child-directed input may influence both model learning and human language development. Overall, understanding what properties make language data efficient for learning can enable more powerful small-scale language models while also shedding light on human language acquisition.
• 摘要：现代语言模型 (LM) 必须接受比人类儿童在开始产生有用行为之前接收到的训练数据数量级多多个数量级的训练数据。评估这种“数据差距”的性质和起源需要在人类规模的数据集上对语言模型进行基准测试，以了解语言知识如何从儿童的自然训练数据中产生。使用 BabyView 数据集的记录（来自 6-36 个月儿童的视频），我们研究了 (1) 儿童规模数据体系的扩展性能，(2) 来自不同儿童经历的数据集模型性能的变异性和数据集质量的语言预测因子，以及 (3) 模型与儿童语言学习结果之间的关系。在子数据上训练的语言模型在语法任务上显示出可接受的扩展，但在语义和世界知识任务上的扩展比在合成数据上训练的模型要低；我们还观察到不同儿童的数据存在很大差异。除了数据集大小之外，性能与分布式和交互语言特征的组合最相关，这与儿童语言发展的高质量输入大致一致。最后，单个单词的模型可能性与儿童对这些单词的学习相关，这表明儿童导向输入的属性可能会影响模型学习和人类语言发展。总体而言，了解哪些属性使语言数据能够有效地学习，可以实现更强大的小规模语言模型，同时也为人类语言习得提供线索。

Title: Can LLM Agents Identify Spoken Dialects like a Linguist?

• Authors: Tobias Bystrich, Lukas Hamm, Maria Hassan, Lea Fischbach, Lucie Flek, Akbar Karimi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29541
• Pdf URL: https://arxiv.org/pdf/2603.29541
• Copy Paste: [[2603.29541]] Can LLM Agents Identify Spoken Dialects like a Linguist?(https://arxiv.org/abs/2603.29541)
• Keywords: language model, llm, agent
• Abstract: Due to the scarcity of labeled dialectal speech, audio dialect classification is a challenging task for most languages, including Swiss German. In this work, we explore the ability of large language models (LLMs) as agents in understanding the dialects and whether they can show comparable performance to models such as HuBERT in dialect classification. In addition, we provide an LLM baseline and a human linguist one. Our approach uses phonetic transcriptions produced by ASR systems and combines them with linguistic resources such as dialect feature maps, vowel history, and rules. Our findings indicate that, when linguistic information is provided, the LLM predictions improve. The human baseline shows that automatically generated transcriptions can be beneficial for such classifications, but also presents opportunities for improvement.
• 摘要：由于标记方言语音的稀缺，音频方言分类对于大多数语言（包括瑞士德语）来说是一项具有挑战性的任务。在这项工作中，我们探讨了大型语言模型（LLM）作为代理理解方言的能力，以及它们是否能够在方言分类中表现出与 HuBERT 等模型相当的性能。此外，我们还提供法学硕士基线和人类语言学家基线。我们的方法使用 ASR 系统生成的语音转录，并将其与方言特征图、元音历史和规则等语言资源相结合。我们的研究结果表明，当提供语言信息时，法学硕士的预测会有所改善。人类基线表明，自动生成的转录可能有利于此类分类，但也提供了改进的机会。

Title: Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models

• Authors: Linda Zeng, Steven Y. Feng, Michael C. Frank
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2603.29552
• Pdf URL: https://arxiv.org/pdf/2603.29552
• Copy Paste: [[2603.29552]] Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models(https://arxiv.org/abs/2603.29552)
• Keywords: language model, gpt
• Abstract: Multilingualism is incredibly common around the world, leading to many important theoretical and practical questions about how children learn multiple languages at once. For example, does multilingual acquisition lead to delays in learning? Are there better and worse ways to structure multilingual input? Many correlational studies address these questions, but it is surprisingly difficult to get definitive answers because children cannot be randomly assigned to be multilingual and data are typically not matched between languages. We use language model training as a method for simulating a variety of highly controlled exposure conditions, and create matched 100M-word mono- and bilingual datasets using synthetic data and machine translation. We train GPT-2 models on monolingual and bilingual data organized to reflect a range of exposure regimes, and evaluate their performance on perplexity, grammaticality, and semantic knowledge. Across model scales and measures, bilingual models perform similarly to monolingual models in one language, but show strong performance in the second language as well. These results suggest that there are no strong differences between different bilingual exposure regimes, and that bilingual input poses no in-principle challenges for agnostic statistical learners.
• 摘要：使用多种语言在世界各地非常普遍，这引发了许多关于儿童如何同时学习多种语言的重要理论和实践问题。例如，多语言习得是否会导致学习延迟？构建多语言输入是否有更好和更差的方法？许多相关研究都解决了这些问题，但要获得明确的答案却出人意料地困难，因为儿童不能被随机分配为多种语言，而且语言之间的数据通常不匹配。我们使用语言模型训练作为模拟各种高度受控的暴露条件的方法，并使用合成数据和机器翻译创建匹配的 1 亿字单语和双语数据集。我们在单语和双语数据上训练 GPT-2 模型，这些数据旨在反映一系列暴露状况，并评估其在困惑度、语法性和语义知识方面的表现。在模型规模和测量方面，双语模型的表现与一种语言的单语模型相似，但在第二语言中也表现出强劲的表现。这些结果表明，不同的双语接触制度之间没有显着差异，并且双语输入不会对不可知论的统计学习者构成原则上的挑战。

Title: When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment

• Authors: Robinson Ferrer, Damla Turgut, Zhongzhou Chen, Shashank Sonkar
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2603.29559
• Pdf URL: https://arxiv.org/pdf/2603.29559
• Copy Paste: [[2603.29559]] When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment(https://arxiv.org/abs/2603.29559)
• Keywords: language model, gpt, llm
• Abstract: Large Language Models (LLMs) show promise for automated grading, but their outputs can be unreliable. Rather than improving grading accuracy directly, we address a complementary problem: \textit{predicting when an LLM grader is likely to be correct}. This enables selective automation where high-confidence predictions are processed automatically while uncertain cases are flagged for human review. We compare three confidence estimation methods (self-reported confidence, self-consistency voting, and token probability) across seven LLMs of varying scale (4B to 120B parameters) on three educational datasets: RiceChem (long-answer chemistry), SciEntsBank, and Beetle (short-answer science). Our experiments reveal that self-reported confidence consistently achieves the best calibration across all conditions (avg ECE 0.166 vs 0.229 for self-consistency). Surprisingly, self-consistency remains 38\% worse despite requiring 5$\times$ the inference cost. Larger models exhibit substantially better calibration though gains vary by dataset and method (e.g., a 28\% ECE reduction for self-reported), with GPT-OSS-120B achieving the best calibration (avg ECE 0.100) and strong discrimination (avg AUC 0.668). We also observe that confidence is strongly top-skewed across methods, creating a ``confidence floor'' that practitioners must account for when setting thresholds. These findings suggest that simply asking LLMs to report their confidence provides a practical approach for identifying reliable grading predictions. Code is available \href{this https URL}{here}.
• 摘要：大型语言模型 (LLM) 有望实现自动评分，但其输出可能不可靠。我们不是直接提高评分准确性，而是解决一个补充问题：\textit{预测 LLM 评分者何时可能是正确的}。这可以实现选择性自动化，自动处理高置信度的预测，同时标记不确定的情况以供人工审查。我们比较了七个不同规模（4B 至 120B 参数）的法学硕士在三个教育数据集上的三种置信度估计方法（自我报告置信度、自洽投票和令牌概率）：RiceChem（长答案化学）、SciEntsBank 和 Beetle（短答案科学）。我们的实验表明，自我报告的置信度在所有条件下始终达到最佳校准（自我一致性的平均 ECE 0.166 与 0.229）。令人惊讶的是，尽管需要 5 美元\ 倍的推理成本，但自一致性仍然差 38%。较大的模型表现出更好的校准，尽管增益因数据集和方法而异（例如，自我报告的 ECE 降低 28%），GPT-OSS-120B 实现了最佳校准（平均 ECE 0.100）和很强的辨别力（平均 AUC 0.668）。我们还观察到，不同方法之间的置信度存在强烈的顶部偏差，从而创建了一个“置信底线”，从业者在设置阈值时必须考虑到这一点。这些发现表明，简单地要求法学硕士报告他们的信心，为识别可靠的评分预测提供了一种实用的方法。代码可在\href{此 https URL}{此处}获取。

Title: Learning Diagnostic Reasoning for Decision Support in Toxicology

• Authors: Nico Oberländer, David Bani-Harouni, Tobias Zellner, Nassir Navab, Florian Eyer, Matthias Keicher
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29608
• Pdf URL: https://arxiv.org/pdf/2603.29608
• Copy Paste: [[2603.29608]] Learning Diagnostic Reasoning for Decision Support in Toxicology(https://arxiv.org/abs/2603.29608)
• Keywords: language model, llm
• Abstract: Acute poly-substance intoxication requires rapid, life-saving decisions under substantial uncertainty, as clinicians must rely on incomplete ingestion details and nonspecific symptoms. Effective diagnostic reasoning in this chaotic environment requires fusing unstructured, non-medical narratives (e.g. paramedic scene descriptions and unreliable patient self-reports or known histories), with structured medical data like vital signs. While Large Language Models (LLMs) show potential for processing such heterogeneous inputs, they struggle in this setting, often underperforming simple baselines that rely solely on patient histories. To address this, we present DeToxR (Decision-support for Toxicology with Reasoning), the first adaptation of Reinforcement Learning (RL) to emergency toxicology. We design a robust data-fusion engine for multi-label prediction across 14 substance classes based on an LLM finetuned with Group Relative Policy Optimization (GRPO). We optimize the model's reasoning directly using a clinical performance reward. By formulating a multi-label agreement metric as the reward signal, the model is explicitly penalized for missing co-ingested substances and hallucinating absent poisons. Our model significantly outperforms its unadapted base LLM counterpart and supervised baselines. Furthermore, in a clinical validation study, the model indicates a clinical advantage by outperforming an expert toxicologist in identifying the correct poisons (Micro-F1: 0.644 vs. 0.473). These results demonstrate the potential of RL-aligned LLMs to synthesize unstructured pre-clinical narratives and structured medical data for decision support in high-stakes environments.
• 摘要：急性多物质中毒需要在很大的不确定性下做出快速、挽救生命的决定，因为临床医生必须依赖不完整的摄入细节和非特异性症状。在这种混乱的环境中进行有效的诊断推理需要将非结构化的非医学叙述（例如护理人员场景描述和不可靠的患者自我报告或已知病史）与生命体征等结构化医疗数据融合起来。虽然大型语言模型 (LLM) 显示出处理此类异构输入的潜力，但它们在这种情况下表现不佳，通常表现不佳仅依赖于患者病史的简单基线。为了解决这个问题，我们提出了 DeToxR（推理毒理学决策支持），这是强化学习 (RL) 首次适应紧急毒理学。我们设计了一个强大的数据融合引擎，基于通过组相对策略优化 (GRPO) 进行微调的法学硕士，可跨 14 种物质类别进行多标签预测。我们直接使用临床表现奖励来优化模型的推理。通过制定多标签一致性指标作为奖励信号，该模型会因缺少共同摄入的物质和幻觉不存在的毒物而受到明确的惩罚。我们的模型显着优于其未适应的基础法学硕士对应模型和监督基线。此外，在临床验证研究中，该模型在识别正确毒物方面优于毒理学专家，显示出临床优势（Micro-F1：0.644 与 0.473）。这些结果证明了 RL 一致的法学硕士在综合非结构化临床前叙述和结构化医疗数据以在高风险环境中提供决策支持方面的潜力。

Title: Agenda-based Narrative Extraction: Steering Pathfinding Algorithms with Large Language Models

• Authors: Brian Felipe Keith-Norambuena, Carolina Inés Rojas-Córdova, Claudio Juvenal Meneses-Villegas, Elizabeth Johanna Lam-Esquenazi, Angélica María Flores-Bustos, Ignacio Alejandro Molina-Villablanca, Joshua Emanuel Leyton-Vallejos
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2603.29661
• Pdf URL: https://arxiv.org/pdf/2603.29661
• Copy Paste: [[2603.29661]] Agenda-based Narrative Extraction: Steering Pathfinding Algorithms with Large Language Models(https://arxiv.org/abs/2603.29661)
• Keywords: language model, gpt, llm
• Abstract: Existing narrative extraction methods face a trade-off between coherence, interactivity, and multi-storyline support. Narrative Maps supports rich interaction and generates multiple storylines as a byproduct of its coverage constraints, though this comes at the cost of individual path coherence. Narrative Trails achieves high coherence through maximum capacity path optimization but provides no mechanism for user guidance or multiple perspectives. We introduce agenda-based narrative extraction, a method that bridges this gap by integrating large language models into the Narrative Trails pathfinding process to steer storyline construction toward user-specified perspectives. Our approach uses an LLM at each step to rank candidate documents based on their alignment with a given agenda while maintaining narrative coherence. Running the algorithm with different agendas yields different storylines through the same corpus. We evaluated our approach on a news article corpus using LLM judges with Claude Opus 4.5 and GPT 5.1, measuring both coherence and agenda alignment across 64 endpoint pairs and 6 agendas. LLM-driven steering achieves 9.9% higher alignment than keyword matching on semantic agendas (p=0.017), with 13.3% improvement on \textit{Regime Crackdown} specifically (p=0.037), while keyword matching remains competitive on agendas with literal keyword overlap. The coherence cost is minimal: LLM steering reduces coherence by only 2.2% compared to the agenda-agnostic baseline. Counter-agendas that contradict the source material score uniformly low (2.2-2.5) across all methods, confirming that steering cannot fabricate unsupported narratives.
• 摘要：现有的叙事提取方法面临连贯性、交互性和多故事情节支持之间的权衡。叙事地图支持丰富的交互，并作为其覆盖限制的副产品生成多个故事情节，尽管这是以个体路径连贯性为代价的。 Narrative Trails 通过最大容量路径优化实现了高度连贯性，但没有提供用户引导或多视角的机制。我们引入了基于议程的叙事提取，这是一种通过将大型语言模型集成到叙事轨迹寻路过程中来弥合这一差距的方法，以引导故事情节构建朝着用户指定的角度发展。我们的方法在每一步都使用法学硕士根据候选文件与给定议程的一致性对候选文件进行排名，同时保持叙述的连贯性。使用不同的议程运行算法会通过同一语料库产生不同的故事情节。我们使用 LLM 法官使用 Claude Opus 4.5 和 GPT 5.1 评估了新闻文章语料库的方法，衡量了 64 个端点对和 6 个议程的连贯性和议程一致性。 LLM 驱动的指导比语义议程上的关键字匹配提高了 9.9% 的对齐度 (p=0.017)，特别是在 \textit{Regime Crackdown} 上提高了 13.3% (p=0.037)，而关键字匹配在具有字面关键字重叠的议程上仍然具有竞争力。一致性成本极低：与议程无关的基线相比，LLM 指导仅将一致性降低了 2.2%。在所有方法中，与源材料相矛盾的反议程得分均较低（2.2-2.5），这证实了指导不能编造不受支持的叙述。

Title: Near-Miss: Latent Policy Failure Detection in Agentic Workflows

• Authors: Ella Rabinovich, David Boaz, Naama Zwerdling, Ateret Anaby-Tavor
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29665
• Pdf URL: https://arxiv.org/pdf/2603.29665
• Copy Paste: [[2603.29665]] Near-Miss: Latent Policy Failure Detection in Agentic Workflows(https://arxiv.org/abs/2603.29665)
• Keywords: llm, agent
• Abstract: Agentic systems for business process automation often require compliance with policies governing conditional updates to the system state. Evaluation of policy adherence in LLM-based agentic workflows is typically performed by comparing the final system state against a predefined ground truth. While this approach detects explicit policy violations, it may overlook a more subtle class of issues in which agents bypass required policy checks, yet reach a correct outcome due to favorable circumstances. We refer to such cases as $\textit{near-misses}$ or $\textit{latent failures}$. In this work, we introduce a novel metric for detecting latent policy failures in agent conversations traces. Building on the ToolGuard framework, which converts natural-language policies into executable guard code, our method analyzes agent trajectories to determine whether agent's tool-calling decisions where sufficiently informed. We evaluate our approach on the $\tau^2$-verified Airlines benchmark across several contemporary open and proprietary LLMs acting as agents. Our results show that latent failures occur in 8-17% of trajectories involving mutating tool calls, even when the final outcome matches the expected ground-truth state. These findings reveal a blind spot in current evaluation methodologies and highlight the need for metrics that assess not only final outcomes but also the decision process leading to them.
• 摘要：业务流程自动化的代理系统通常需要遵守管理系统状态有条件更新的策略。基于 LLM 的代理工作流程中策略遵守情况的评估通常是通过将最终系统状态与预定义的基本事实进行比较来执行的。虽然这种方法可以检测到明确的策略违规，但它可能会忽略一类更微妙的问题，即代理绕过所需的策略检查，但由于有利的情况而达到正确的结果。我们将此类情况称为 $\textit{未遂事件}$ 或 $\textit{潜在故障}$。在这项工作中，我们引入了一种新颖的指标，用于检测代理对话跟踪中潜在的策略失败。我们的方法以 ToolGuard 框架为基础，将自然语言策略转换为可执行的保护代码，分析代理轨迹，以确定代理的工具调用决策是否充分知情。我们通过多个当代开放和专有法学硕士作为代理人，根据 $\tau^2$ 验证的航空公司基准评估我们的方法。我们的结果表明，即使最终结果与预期的真实状态相匹配，在涉及变异工具调用的轨迹中，仍有 8-17% 发生潜在故障。这些发现揭示了当前评估方法中的盲点，并强调需要不仅评估最终结果，而且还评估导致最终结果的决策过程的指标。

Title: SNEAK: Evaluating Strategic Communication and Information Leakage in Large Language Models

• Authors: Adar Avsian, Larry Heck
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.29846
• Pdf URL: https://arxiv.org/pdf/2603.29846
• Copy Paste: [[2603.29846]] SNEAK: Evaluating Strategic Communication and Information Leakage in Large Language Models(https://arxiv.org/abs/2603.29846)
• Keywords: language model, llm, agent
• Abstract: Large language models (LLMs) are increasingly deployed in multi-agent settings where communication must balance informativeness and secrecy. In such settings, an agent may need to signal information to collaborators while preventing an adversary from inferring sensitive details. However, existing LLM benchmarks primarily evaluate capabilities such as reasoning, factual knowledge, or instruction following, and do not directly measure strategic communication under asymmetric information. We introduce SNEAK (Secret-aware Natural language Evaluation for Adversarial Knowledge), a benchmark for evaluating selective information sharing in language models. In SNEAK, a model is given a semantic category, a candidate set of words, and a secret word, and must generate a message that indicates knowledge of the secret without revealing it too clearly. We evaluate generated messages using two simulated agents with different information states: an ally, who knows the secret and must identify the intended message, and a chameleon, who does not know the secret and attempts to infer it from the message. This yields two complementary metrics: utility, measuring how well the message communicates to collaborators, and leakage, measuring how much information it reveals to an adversary. Using this framework, we analyze the trade-off between informativeness and secrecy in modern language models and show that strategic communication under asymmetric information remains a challenging capability for current systems. Notably, human participants outperform all evaluated models by a large margin, achieving up to four times higher scores.
• 摘要：大型语言模型 (LLM) 越来越多地部署在多代理环境中，其中通信必须平衡信息性和保密性。在此类设置中，代理可能需要向协作者发送信息，同时防止对手推断敏感细节。然而，现有的LLM基准主要评估推理、事实知识或指令遵循等能力，并没有直接衡量非对称信息下的战略沟通。我们引入了 SNEAK（对抗性知识的秘密感知自然语言评估），这是评估语言模型中选择性信息共享的基准。在 SNEAK 中，模型被赋予一个语义类别、一组候选单词和一个秘密单词，并且必须生成一条消息来指示该秘密的知识，但又不会太清楚地透露它。我们使用两个具有不同信息状态的模拟代理来评估生成的消息：一个盟友，他知道秘密并且必须识别预期的消息，以及一个变色龙，他不知道秘密并试图从消息中推断出它。这产生了两个互补的指标：效用，衡量消息与合作者沟通的程度，以及泄漏，衡量向对手透露了多少信息。使用这个框架，我们分析了现代语言模型中信息性和保密性之间的权衡，并表明不对称信息下的战略通信对于当前系统来说仍然是一个具有挑战性的能力。值得注意的是，人类参与者的表现大幅优于所有评估模型，得分高出四倍。

Title: Towards Empowering Consumers through Sentence-level Readability Scoring in German ESG Reports

• Authors: Benjamin Josef Schüßler, Jakob Prange
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.29861
• Pdf URL: https://arxiv.org/pdf/2603.29861
• Copy Paste: [[2603.29861]] Towards Empowering Consumers through Sentence-level Readability Scoring in German ESG Reports(https://arxiv.org/abs/2603.29861)
• Keywords: llm, prompt
• Abstract: With the ever-growing urgency of sustainability in the economy and society, and the massive stream of information that comes with it, consumers need reliable access to that information. To address this need, companies began publishing so called Environmental, Social, and Governance (ESG) reports, both voluntarily and forced by law. To serve the public, these reports must be addressed not only to financial experts but also to non-expert audiences. But are they written clearly enough? In this work, we extend an existing sentence-level dataset of German ESG reports with crowdsourced readability annotations. We find that, in general, native speakers perceive sentences in ESG reports as easy to read, but also that readability is subjective. We apply various readability scoring methods and evaluate them regarding their prediction error and correlation with human rankings. Our analysis shows that, while LLM prompting has potential for distinguishing clear from hard-to-read sentences, a small finetuned transformer predicts human readability with the lowest error. Averaging predictions of multiple models can slightly improve the performance at the cost of slower inference.
• 摘要：随着经济和社会可持续发展的紧迫性日益迫切，以及随之而来的海量信息流，消费者需要可靠地获取这些信息。为了满足这一需求，公司开始发布所谓的环境、社会和治理（ESG）报告，无论是自愿的还是法律强制的。为了服务公众，这些报告不仅必须面向金融专家，还必须面向非专家受众。但它们写得够清楚吗？在这项工作中，我们使用众包可读性注释扩展了德国 ESG 报告的现有句子级数据集。我们发现，一般来说，母语人士认为 ESG 报告中的句子易于阅读，但可读性是主观的。我们应用各种可读性评分方法，并评估它们的预测误差以及与人类排名的相关性。我们的分析表明，虽然 LLM 提示有可能区分清晰的句子和难以阅读的句子，但小型微调变压器可以以最低的误差预测人类可读性。对多个模型的预测进行平均可以稍微提高性能，但代价是推理速度变慢。

Title: Structural Feature Engineering for Generative Engine Optimization: How Content Structure Shapes Citation Behavior

• Authors: Junwei Yu, Mufeng Yang, Yepeng Ding, Hiroyuki Sato
• Subjects: cs.CL, cs.HC, cs.IR
• Abstract URL: https://arxiv.org/abs/2603.29979
• Pdf URL: https://arxiv.org/pdf/2603.29979
• Copy Paste: [[2603.29979]] Structural Feature Engineering for Generative Engine Optimization: How Content Structure Shapes Citation Behavior(https://arxiv.org/abs/2603.29979)
• Keywords: llm
• Abstract: The proliferation of AI-powered search engines has shifted information discovery from traditional link-based retrieval to direct answer generation with selective source citation, creating new challenges for content visibility. While existing Generative Engine Optimization (GEO) approaches focus primarily on semantic content modification, the role of structural features in influencing citation behavior remains underexplored. In this paper, we propose GEO-SFE, a systematic framework for structural feature engineering in generative engine optimization. Our approach decomposes content structure into three hierarchical levels: macro-structure (document architecture), meso-structure (information chunking), and micro-structure (visual emphasis), and models their impact on citation probability across different generative engine architectures. We develop architecture-aware optimization strategies and predictive models that preserve semantic integrity while improving structural effectiveness. Experimental evaluation across six mainstream generative engines demonstrates consistent improvements in citation rate (17.3 percent) and subjective quality (18.5 percent), validating the effectiveness and generalizability of the proposed framework. This work establishes structural optimization as a foundational component of GEO, providing a data-driven methodology for enhancing content visibility in LLM-powered information ecosystems.
• 摘要：人工智能驱动的搜索引擎的激增已将信息发现从传统的基于链接的检索转变为通过选择性来源引用直接生成答案，为内容可见性带来了新的挑战。虽然现有的生成引擎优化（GEO）方法主要关注语义内容修改，但结构特征在影响引用行为中的作用仍未得到充分探索。在本文中，我们提出了 GEO-SFE，这是一种用于生成引擎优化中结构特征工程的系统框架。我们的方法将内容结构分解为三个层次结构：宏观结构（文档架构）、中观结构（信息分块）和微观结构（视觉强调），并模拟它们对不同生成引擎架构的引用概率的影响。我们开发架构感知优化策略和预测模型，在提高结构有效性的同时保持语义完整性。对六个主流生成引擎的实验评估表明，引用率（17.3%）和主观质量（18.5%）持续提高，验证了所提出框架的有效性和普遍性。这项工作将结构优化确立为 GEO 的基本组成部分，提供了一种数据驱动的方法来增强 LLM 支持的信息生态系统中的内容可见性。

Title: Enhancing Structural Mapping with LLM-derived Abstractions for Analogical Reasoning in Narratives

• Authors: Mohammadhossein Khojasteh, Yifan Jiang, Stefano De Giorgis, Frank van Harmelen, Filip Ilievski
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2603.29997
• Pdf URL: https://arxiv.org/pdf/2603.29997
• Copy Paste: [[2603.29997]] Enhancing Structural Mapping with LLM-derived Abstractions for Analogical Reasoning in Narratives(https://arxiv.org/abs/2603.29997)
• Keywords: llm, prompt
• Abstract: Analogical reasoning is a key driver of human generalization in problem-solving and argumentation. Yet, analogies between narrative structures remain challenging for machines. Cognitive engines for structural mapping are not directly applicable, as they assume pre-extracted entities, whereas LLMs' performance is sensitive to prompt format and the degree of surface similarity between narratives. This gap motivates a key question: What is the impact of enhancing structural mapping with LLM-derived abstractions on their analogical reasoning ability in narratives? To that end, we propose a modular framework named YARN (Yielding Abstractions for Reasoning in Narratives), which uses LLMs to decompose narratives into units, abstract these units, and then passes them to a mapping component that aligns elements across stories to perform analogical reasoning. We define and operationalize four levels of abstraction that capture both the general meaning of units and their roles in the story, grounded in prior work on framing. Our experiments reveal that abstractions consistently improve model performance, resulting in competitive or better performance than end-to-end LLM baselines. Closer error analysis reveals the remaining challenges in abstraction at the right level, in incorporating implicit causality, and an emerging categorization of analogical patterns in narratives. YARN enables systematic variation of experimental settings to analyze component contributions, and to support future work, we make the code for YARN openly available.
• 摘要：类比推理是人类解决问题和论证中泛化的关键驱动力。然而，叙事结构之间的类比对于机器来说仍然具有挑战性。用于结构映射的认知引擎并不直接适用，因为它们假设预先提取的实体，而法学硕士的表现对提示格式和叙述之间的表面相似程度敏感。这种差距引发了一个关键问题：使用法学硕士衍生的抽象增强结构映射对其叙事中的类比推理能力有何影响？为此，我们提出了一个名为 YARN（Yielding Abstractions for Reasoning in Narratives）的模块化框架，它使用 LLM 将叙述分解为单元，抽象这些单元，然后将它们传递给映射组件，该组件将跨故事的元素对齐以执行类比推理。我们定义并实施了四个抽象级别，以捕获单元的一般含义及其在故事中的角色，以之前的框架工作为基础。我们的实验表明，抽象不断提高模型性能，从而比端到端 LLM 基线具有竞争力或更好的性能。更仔细的错误分析揭示了在正确层面的抽象、纳入隐含因果关系以及叙事中类比模式的新兴分类方面仍然存在的挑战。 YARN 能够系统地改变实验设置，以分析组件贡献，并支持未来的工作，我们公开了 YARN 的代码。

Title: ContextClaim: A Context-Driven Paradigm for Verifiable Claim Detection

• Authors: Yufeng Li, Rrubaa Panchendrarajan, Arkaitz Zubiaga
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2603.30025
• Pdf URL: https://arxiv.org/pdf/2603.30025
• Copy Paste: [[2603.30025]] ContextClaim: A Context-Driven Paradigm for Verifiable Claim Detection(https://arxiv.org/abs/2603.30025)
• Keywords: language model
• Abstract: Verifiable claim detection asks whether a claim expresses a factual statement that can, in principle, be assessed against external evidence. As an early filtering stage in automated fact-checking, it plays an important role in reducing the burden on downstream verification components. However, existing approaches to claim detection, whether based on check-worthiness or verifiability, rely solely on the claim text itself. This is a notable limitation for verifiable claim detection in particular, where determining whether a claim is checkable may benefit from knowing what entities and events it refers to and whether relevant information exists to support verification. Inspired by the established role of evidence retrieval in later-stage claim verification, we propose Context-Driven Claim Detection (ContextClaim), a paradigm that advances retrieval to the detection stage. ContextClaim extracts entity mentions from the input claim, retrieves relevant information from Wikipedia as a structured knowledge source, and employs large language models to produce concise contextual summaries for downstream classification. We evaluate ContextClaim on two datasets covering different topics and text genres, the CheckThat! 2022 COVID-19 Twitter dataset and the PoliClaim political debate dataset, across encoder-only and decoder-only models under fine-tuning, zero-shot, and few-shot settings. Results show that context augmentation can improve verifiable claim detection, although its effectiveness varies across domains, model architectures, and learning settings. Through component analysis, human evaluation, and error analysis, we further examine when and why the retrieved context contributes to more reliable verifiability judgments.
• 摘要：可验证的声明检测询问声明是否表达了原则上可以根据外部证据进行评估的事实陈述。作为自动事实检查的早期过滤阶段，它在减轻下游验证组件的负担方面发挥着重要作用。然而，现有的声明检测方法，无论是基于可检查性还是可验证性，都仅依赖于声明文本本身。这是可验证声明检测的一个显着限制，特别是在确定声明是否可检查可能受益于了解它引用的实体和事件以及是否存在支持验证的相关信息。受到证据检索在后期声明验证中既定作用的启发，我们提出了上下文驱动的声明检测（ContextClaim），这是一种将检索推进到检测阶段的范例。 ContextClaim 从输入声明中提取实体提及，从维基百科检索相关信息作为结构化知识源，并采用大型语言模型为下游分类生成简洁的上下文摘要。我们在涵盖不同主题和文本类型的两个数据集上评估 ContextClaim，即 CheckThat！ 2022 年 COVID-19 Twitter 数据集和 PoliClaim 政治辩论数据集，涵盖微调、零样本和少样本设置下的仅编码器和仅解码器模型。结果表明，上下文增强可以改善可验证的声明检测，尽管其有效性因领域、模型架构和学习设置而异。通过成分分析、人工评估和错误分析，我们进一步检查检索到的上下文何时以及为何有助于更可靠的可验证性判断。