2026-02-26

Title: Disaster Question Answering with LoRA Efficiency and Accurate End Position

• Authors: Takato Yasuno
• Subjects: cs.CL, cs.IR, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.21212
• Pdf URL: https://arxiv.org/pdf/2602.21212
• Copy Paste: [[2602.21212]] Disaster Question Answering with LoRA Efficiency and Accurate End Position(https://arxiv.org/abs/2602.21212)
• Keywords: language model, hallucination
• Abstract: Natural disasters such as earthquakes, torrential rainfall, floods, and volcanic eruptions occur with extremely low frequency and affect limited geographic areas. When individuals face disaster situations, they often experience confusion and lack the domain-specific knowledge and experience necessary to determine appropriate responses and actions. While disaster information is continuously updated, even when utilizing RAG search and large language models for inquiries, obtaining relevant domain knowledge about natural disasters and experiences similar to one's specific situation is not guaranteed. When hallucinations are included in disaster question answering, artificial misinformation may spread and exacerbate confusion. This work introduces a disaster-focused question answering system based on Japanese disaster situations and response experiences. Utilizing the cl-tohoku/bert-base-japanese-v3 + Bi-LSTM + Enhanced Position Heads architecture with LoRA efficiency optimization, we achieved 70.4\% End Position accuracy with only 5.7\% of the total parameters (6.7M/117M). Experimental results demonstrate that the combination of Japanese BERT-base optimization and Bi-LSTM contextual understanding achieves accuracy levels suitable for real disaster response scenarios, attaining a 0.885 Span F1 score. Future challenges include: establishing natural disaster Q\&A benchmark datasets, fine-tuning foundation models with disaster knowledge, developing lightweight and power-efficient edge AI Disaster Q\&A applications for situations with insufficient power and communication during disasters, and addressing disaster knowledge base updates and continual learning capabilities.
• 摘要：地震、暴雨、洪水、火山爆发等自然灾害发生频率极低，影响的地理区域也有限。当个人面临灾难情况时，他们经常会感到困惑，并且缺乏确定适当的应对措施和行动所需的特定领域的知识和经验。尽管灾害信息不断更新，即使利用RAG搜索和大语言模型进行查询，也不能保证获得与自然灾害相关的领域知识和与自身具体情况相似的经验。当灾难问答中包含幻觉时，人为的错误信息可能会传播并加剧混乱。本工作介绍了一种基于日本灾害情况和应对经验的灾害焦点问答系统。利用 cl-tohoku/bert-base-japanese-v3 + Bi-LSTM + 增强型定位头架构和 LoRA 效率优化，我们仅用总参数 (6.7M/117M) 的 5.7% 就实现了 70.4% 的最终位置精度。实验结果表明，基于日本 BERT 的优化和 Bi-LSTM 上下文理解的结合达到了适合真实灾难响应场景的准确度水平，获得了 0.885 Span F1 分数。未来的挑战包括：建立自然灾害问答基准数据集、利用灾害知识微调基础模型、针对灾害期间电力和通信不足的情况开发轻量级、省电的边缘AI灾害问答应用程序，以及解决灾害知识库更新和持续学习能力。

Title: Inference-time Alignment via Sparse Junction Steering

• Authors: Runyi Hu, Jie Zhang, Shiqian Zhao, Jiale Meng, Jiwei Li, Jason Zeng, Ming Wu, Michael Heinrich, Yonggang Wen, Tianwei Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21215
• Pdf URL: https://arxiv.org/pdf/2602.21215
• Copy Paste: [[2602.21215]] Inference-time Alignment via Sparse Junction Steering(https://arxiv.org/abs/2602.21215)
• Keywords: language model
• Abstract: Token-level steering has emerged as a pivotal approach for inference-time alignment, enabling fine grained control over large language models by modulating their output distributions without parameter updates. While effective, existing methods rely on dense intervention at every decoding step. This persistent manipulation not only incurs substantial computational overhead but also risks compromising generation quality by excessively drifting from the model's intrinsic distribution. In this work, we show that dense intervention is unnecessary and propose Sparse Inference time Alignment (SIA), which performs sparse junction steering by intervening only at critical decision points along the generation trajectory. Our key insight is that high entropy junctions mark pivotal decision points in the generation trajectory and are particularly susceptible to misalignment, indicating the need to introduce alignment related reward signals at these points. Extensive experiments across different model families and alignment objectives show that steering only 20% to 80% of tokens achieves superior alignment-efficiency trade offs. For strong base models such as Qwen3, intervening on as few as 20% of tokens matches or even surpasses heavily post-trained instruct models. This sparsity enables stronger guidance while better preserving the model's native distribution, integrates seamlessly with search based methods such as Best-of-N, and reduces computational cost by up to 6x.
• 摘要：令牌级控制已成为推理时间对齐的关键方法，通过在不更新参数的情况下调节大型语言模型的输出分布，实现对大型语言模型的细粒度控制。虽然有效，但现有方法依赖于每个解码步骤的密集干预。这种持续的操作不仅会产生大量的计算开销，而且还可能因过度偏离模型的固有分布而损害生成质量。在这项工作中，我们证明密集干预是不必要的，并提出稀疏推理时间对齐（SIA），它通过仅在生成轨迹上的关键决策点进行干预来执行稀疏连接转向。我们的主要见解是，高熵连接点标志着生成轨迹中的关键决策点，并且特别容易出现错位，这表明需要在这些点引入与对齐相关的奖励信号。跨不同模型系列和对齐目标的大量实验表明，仅控制 20% 到 80% 的代币即可实现卓越的对齐效率权衡。对于 Qwen3 等强大的基础模型，对少至 20% 的标记进行干预就可以匹配甚至超过经过大量后训练的指令模型。这种稀疏性可以提供更强的指导，同时更好地保留模型的本机分布，与基于搜索的方法（例如 Best-of-N）无缝集成，并将计算成本降低多达 6 倍。

Title: EQ-5D Classification Using Biomedical Entity-Enriched Pre-trained Language Models and Multiple Instance Learning

• Authors: Zhyar Rzgar K Rostam, Gábor Kertész
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21216
• Pdf URL: https://arxiv.org/pdf/2602.21216
• Copy Paste: [[2602.21216]] EQ-5D Classification Using Biomedical Entity-Enriched Pre-trained Language Models and Multiple Instance Learning(https://arxiv.org/abs/2602.21216)
• Keywords: language model
• Abstract: The EQ-5D (EuroQol 5-Dimensions) is a standardized instrument for the evaluation of health-related quality of life. In health economics, systematic literature reviews (SLRs) depend on the correct identification of publications that use the EQ-5D, but manual screening of large volumes of scientific literature is time-consuming, error-prone, and inconsistent. In this study, we investigate fine-tuning of general-purpose (BERT) and domain-specific (SciBERT, BioBERT) pre-trained language models (PLMs), enriched with biomedical entity information extracted through scispaCy models for each statement, to improve EQ-5D detection from abstracts. We conduct nine experimental setups, including combining three scispaCy models with three PLMs, and evaluate their performance at both the sentence and study levels. Furthermore, we explore a Multiple Instance Learning (MIL) approach with attention pooling to aggregate sentence-level information into study-level predictions, where each abstract is represented as a bag of enriched sentences (by scispaCy). The findings indicate consistent improvements in F1-scores (reaching 0.82) and nearly perfect recall at the study-level, significantly exceeding classical bag-of-words baselines and recently reported PLM baselines. These results show that entity enrichment significantly improves domain adaptation and model generalization, enabling more accurate automated screening in systematic reviews.
• 摘要：EQ-5D（EuroQol 5-Dimensions）是用于评估健康相关生活质量的标准化工具。在卫生经济学中，系统文献综述 (SLR) 取决于对使用 EQ-5D 的出版物的正确识别，但手动筛选大量科学文献非常耗时、容易出错且不一致。在本研究中，我们研究了通用 (BERT) 和特定领域（SciBERT、BioBERT）预训练语言模型 (PLM) 的微调，并通过 scispaCy 模型为每个语句提取的生物医学实体信息进行丰富，以改进摘要中的 EQ-5D 检测。我们进行了九个实验设置，包括将三个 scispaCy 模型与三个 PLM 相结合，并评估它们在句子和研究级别的性能。此外，我们探索了一种带有注意力池的多实例学习（MIL）方法，将句子级信息聚合到研究级预测中，其中每个摘要都表示为一袋丰富的句子（由 scispaCy 提供）。研究结果表明，F1 分数持续改善（达到 0.82），并且在研究级别的召回率近乎完美，显着超过了经典的词袋基线和最近报告的 PLM 基线。这些结果表明，实体丰富显着提高了领域适应和模型泛化，从而在系统评价中实现更准确的自动筛选。

Title: EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors

• Authors: Amin Banayeeanzade, Qingchuan Yang, Deqing Fu, Spencer Hong, Erin Babinsky, Alfy Samuel, Anoop Kumar, Robin Jia, Sai Praneeth Karimireddy
• Subjects: cs.CL, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.21218
• Pdf URL: https://arxiv.org/pdf/2602.21218
• Copy Paste: [[2602.21218]] EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors(https://arxiv.org/abs/2602.21218)
• Keywords: language model, llm, prompt
• Abstract: High-quality data is essential for modern machine learning, yet many valuable corpora are sensitive and cannot be freely shared. Synthetic data offers a practical substitute for downstream development, and large language models (LLMs) have emerged as powerful engines for generating it. However, existing private text generation methods are severely inefficient: they are data-intensive, computationally slow, and often require large private corpora or batch sizes to achieve usable quality. We introduce EPSVec, a differentially-private lightweight alternative that steers LLM generation using *dataset vectors*--directions in activation space that capture the distributional gap between private data and public priors. EPSVec extracts and sanitizes steering vectors just once and then performs standard decoding. This decouples the privacy budget from generation, enabling arbitrarily many synthetic samples without additional privacy cost and yielding strong fidelity even in low-data regimes. Furthermore, we enhance our method by utilizing pretrained (base) models and introducing fixed-shot prompting to boost generation diversity and fidelity. Our experiments demonstrate that EPSVec outperforms existing baselines in distributional alignment and downstream utility, particularly in low-data regimes, while significantly reducing computational overhead.
• 摘要：高质量的数据对于现代机器学习至关重要，但许多有价值的语料库都是敏感的，无法自由共享。合成数据为下游开发提供了实用的替代品，而大型语言模型（LLM）已成为生成合成数据的强大引擎。然而，现有的私有文本生成方法效率严重低下：它们是数据密集型的，计算速度慢，并且通常需要大量的私有语料库或批量大小才能达到可用的质量。我们引入了 EPSVec，一种差分隐私的轻量级替代方案，它使用*数据集向量*（激活空间中的方向，捕获私有数据和公共先验之间的分布差距）来引导 LLM 生成。 EPSVec 仅提取和清理引导向量一次，然后执行标准解码。这将隐私预算与生成分离，无需额外的隐私成本即可生成任意数量的合成样本，即使在低数据情况下也能产生很强的保真度。此外，我们通过利用预训练（基础）模型并引​​入固定镜头提示来增强我们的方法，以提高生成多样性和保真度。我们的实验表明，EPSVec 在分布对齐和下游效用方面优于现有基线，特别是在低数据情况下，同时显着降低了计算开销。

Title: Reasoning-Based Personalized Generation for Users with Sparse Data

• Authors: Bo Ni, Branislav Kveton, Samyadeep Basu, Subhojyoti Mukherjee, Leyao Wang, Franck Dernoncourt, Sungchul Kim, Seunghyun Yoon, Zichao Wang, Ruiyi Zhang, Puneet Mathur, Jihyung Kil, Jiuxiang Gu, Nedim Lipka, Yu Wang, Ryan A. Rossi, Tyler Derr
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21219
• Pdf URL: https://arxiv.org/pdf/2602.21219
• Copy Paste: [[2602.21219]] Reasoning-Based Personalized Generation for Users with Sparse Data(https://arxiv.org/abs/2602.21219)
• Keywords: language model, llm
• Abstract: Large Language Model (LLM) personalization holds great promise for tailoring responses by leveraging personal context and history. However, real-world users usually possess sparse interaction histories with limited personal context, such as cold-start users in social platforms and newly registered customers in online E-commerce platforms, compromising the LLM-based personalized generation. To address this challenge, we introduce GraSPer (Graph-based Sparse Personalized Reasoning), a novel framework for enhancing personalized text generation under sparse context. GraSPer first augments user context by predicting items that the user would likely interact with in the future. With reasoning alignment, it then generates texts for these interactions to enrich the augmented context. In the end, it generates personalized outputs conditioned on both the real and synthetic histories, ensuring alignment with user style and preferences. Extensive experiments on three benchmark personalized generation datasets show that GraSPer achieves significant performance gain, substantially improving personalization in sparse user context settings.
• 摘要：大语言模型 (LLM) 个性化有望通过利用个人背景和历史来定制响应。然而，现实世界的用户通常拥有稀疏的交互历史和有限的个人背景，例如社交平台中的冷启动用户和在线电子商务平台中的新注册客户，这损害了基于LLM的个性化生成。为了应对这一挑战，我们引入了 GraSPer（基于图的稀疏个性化推理），这是一种用于增强稀疏上下文下个性化文本生成的新颖框架。 GraSPer 首先通过预测用户将来可能与之交互的项目来增强用户上下文。通过推理对齐，它可以为这些交互生成文本，以丰富增强的上下文。最后，它根据真实历史和合成历史生成个性化输出，确保与用户风格和偏好保持一致。对三个基准个性化生成数据集的大量实验表明，GraSPer 实现了显着的性能提升，显着改善了稀疏用户上下文设置中的个性化。

Title: Field-Theoretic Memory for AI Agents: Continuous Dynamics for Context Preservation

• Authors: Subhadip Mitra
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.21220
• Pdf URL: https://arxiv.org/pdf/2602.21220
• Copy Paste: [[2602.21220]] Field-Theoretic Memory for AI Agents: Continuous Dynamics for Context Preservation(https://arxiv.org/abs/2602.21220)
• Keywords: agent
• Abstract: We present a memory system for AI agents that treats stored information as continuous fields governed by partial differential equations rather than discrete entries in a database. The approach draws from classical field theory: memories diffuse through semantic space, decay thermodynamically based on importance, and interact through field coupling in multi-agent scenarios. We evaluate the system on two established long-context benchmarks: LoCoMo (ACL 2024) with 300-turn conversations across 35 sessions, and LongMemEval (ICLR 2025) testing multi-session reasoning over 500+ turns. On LongMemEval, the field-theoretic approach achieves significant improvements: +116% F1 on multi-session reasoning (p<0.01, d= 3.06), +43.8% on temporal reasoning (p<0.001, d= 9.21), and +27.8% retrieval recall on knowledge updates (p<0.001, d= 5.00). Multi-agent experiments show near-perfect collective intelligence (>99.8%) through field coupling. Code is available at this http URL.
• 摘要：我们提出了一种人工智能代理的存储系统，它将存储的信息视为由偏微分方程控制的连续字段，而不是数据库中的离散条目。该方法借鉴了经典场论：记忆通过语义空间扩散，根据重要性进行热力学衰减，并通过多智能体场景中的场耦合进行交互。我们根据两个既定的长上下文基准评估系统：LoCoMo (ACL 2024)，在 35 个会话中进行 300 轮对话，LongMemEval (ICLR 2025) 测试超过 500 轮的多会话推理。在 LongMemEval 上，场论方法实现了显着改进：多会话推理的 F1 提高了 116%（p<0.01，d= 3.06），时间推理的提高了 43.8%（p<0.001，d= 9.21），知识更新的检索召回率提高了 27.8%（p<0.001，d= 5.00）。多智能体实验通过场耦合展示了近乎完美的集体智慧（>99.8%）。代码可从此 http URL 获取。

Title: Task-Aware LoRA Adapter Composition via Similarity Retrieval in Vector Databases

• Authors: Riya Adsul, Balachandra Devarangadi Sunil, Isha Nalawade, Sudharshan Govindan
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.21222
• Pdf URL: https://arxiv.org/pdf/2602.21222
• Copy Paste: [[2602.21222]] Task-Aware LoRA Adapter Composition via Similarity Retrieval in Vector Databases(https://arxiv.org/abs/2602.21222)
• Keywords: language model
• Abstract: Parameter efficient fine tuning methods like LoRA have enabled task specific adaptation of large language models, but efficiently composing multiple specialized adapters for unseen tasks remains challenging. We present a novel framework for dynamic LoRA adapter composition that leverages similarity retrieval in vector databases to enable zero-shot generalization across diverse NLP tasks. Our approach constructs a task-aware vector database by embedding training examples from 22 datasets spanning commonsense reasoning, question answering, natural language inference, and sentiment analysis. At inference time, we retrieve the most similar training examples, compute task similarity distributions via nucleus sampling, and dynamically merge relevant LoRA adapters using retrieval weighted fusion strategies. We evaluated four merging methods Linear, Concatenation, TIES, and Magnitude Prune demonstrating that our dataset centric retrieval approach often matches or exceeds the performance of individually fine-tuned task-specific adapters. Notably, Linear merging achieves 70.95% on PIQA and 77.62% on RTE, substantially outperforming single-task baselines (46% and 52%, respectively). Our framework requires no additional retriever training, operates with frozen embeddings, and enables efficient, interpretable adapter composition. These results suggest that retrieval based dynamic merging offers a promising direction for scalable, parameter-efficient multitask learning without requiring full model retraining for each new task.
• 摘要：LoRA 等参数高效的微调方法已经实现了大型语言模型的特定于任务的适应，但针对看不见的任务有效地组合多个专用适配器仍然具有挑战性。我们提出了一种动态 LoRA 适配器组合的新颖框架，该框架利用向量数据库中的相似性检索来实现跨不同 NLP 任务的零样本泛化。我们的方法通过嵌入来自 22 个数据集的训练示例来构建任务感知向量数据库，这些数据集涵盖常识推理、问题回答、自然语言推理和情感分析。在推理时，我们检索最相似的训练示例，通过核采样计算任务相似性分布，并使用检索加权融合策略动态合并相关的 LoRA 适配器。我们评估了四种合并方法：线性、串联、TIES 和幅度剪枝，证明我们以数据集为中心的检索方法通常可以匹配或超过单独微调的特定于任务的适配器的性能。值得注意的是，线性合并在 PIQA 上实现了 70.95%，在 RTE 上实现了 77.62%，大大优于单任务基线（分别为 46% 和 52%）。我们的框架不需要额外的检索器训练，使用冻结嵌入进行操作，并实现高效、可解释的适配器组合。这些结果表明，基于检索的动态合并为可扩展、参数高效的多任务学习提供了一个有前途的方向，而无需为每个新任务进行完整的模型重新训练。

Title: Measuring Pragmatic Influence in Large Language Model Instructions

• Authors: Yilin Geng, Omri Abend, Eduard Hovy, Lea Frermann
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21223
• Pdf URL: https://arxiv.org/pdf/2602.21223
• Copy Paste: [[2602.21223]] Measuring Pragmatic Influence in Large Language Model Instructions(https://arxiv.org/abs/2602.21223)
• Keywords: language model, llm, prompt
• Abstract: It is not only what we ask large language models (LLMs) to do that matters, but also how we prompt. Phrases like "This is urgent" or "As your supervisor" can shift model behavior without altering task content. We study this effect as pragmatic framing, contextual cues that shape directive interpretation rather than task specification. While prior work exploits such cues for prompt optimization or probes them as security vulnerabilities, pragmatic framing itself has not been treated as a measurable property of instruction following. Measuring this influence systematically remains challenging, requiring controlled isolation of framing cues. We introduce a framework with three novel components: directive-framing decomposition separating framing context from task specification; a taxonomy organizing 400 instantiations of framing into 13 strategies across 4 mechanism clusters; and priority-based measurement that quantifies influence through observable shifts in directive prioritization. Across five LLMs of different families and sizes, influence mechanisms cause consistent and structured shifts in directive prioritization, moving models from baseline impartiality toward favoring the framed directive. This work establishes pragmatic framing as a measurable and predictable factor in instruction-following systems.
• 摘要：重要的不仅是我们要求大型语言模型 (LLM) 做什么，而且我们的提示方式也很重要。像“这很紧急”或“作为你的主管”这样的短语可以在不改变任务内容的情况下改变模型行为。我们将这种效应作为务实的框架、语境线索来研究，这些线索塑造了指令性解释而不是任务规范。虽然之前的工作利用此类线索进行即时优化或将其作为安全漏洞进行探测，但实用框架本身并未被视为指令遵循的可测量属性。系统地衡量这种影响仍然具有挑战性，需要对框架线索进行受控隔离。我们引入了一个具有三个新颖组件的框架：指令框架分解，将框架上下文与任务规范分开；将 400 个框架实例组织成跨 4 个机制集群的 13 个策略的分类法；基于优先级的测量，通过指令优先级的可观察变化来量化影响力。在不同家庭和规模的五个法学硕士中，影响机制导致指令优先顺序发生一致和结构​​化的转变，使模型从基线公正转向有利于框架指令。这项工作将实用框架确立为指令遵循系统中的可测量和可预测的因素。

Title: Make Every Draft Count: Hidden State based Speculative Decoding

• Authors: Yuetao Chen, Xuliang Wang, Xinzhou Zheng, Ming Li, Peng Wang, Hong Xu
• Subjects: cs.CL, cs.AI, cs.DC, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.21224
• Pdf URL: https://arxiv.org/pdf/2602.21224
• Copy Paste: [[2602.21224]] Make Every Draft Count: Hidden State based Speculative Decoding(https://arxiv.org/abs/2602.21224)
• Keywords: llm
• Abstract: Speculative decoding has emerged as a pivotal technique to accelerate LLM inference by employing a lightweight draft model to generate candidate tokens that are subsequently verified by the target model in parallel. However, while this paradigm successfully increases the arithmetic intensity of memory-bound inference, it causes significant compute inefficiency: the majority of draft tokens fail verification and are discarded, resulting in waste of computation. Motivated by the goal of recollecting this wasted computation, we propose a novel system that transforms discarded drafts into reusable tokens. Our key insight is to perform auto-regressive prediction at the hidden states level and postpone the integrating token information after the hidden states generation, so the draft hidden states are not contaminated by incorrect tokens, enabling hidden state reuse. To implement such a system, first we introduce a draft model architecture based on auto-regressive hidden states, which preserves richer semantics than token-based drafters to facilitate draft repurposing. Second, we design an efficient token information injection mechanism that leverages our specialized draft model to construct high-quality draft token trees and enables resampling tokens from verification failures. Third, we eliminate the overhead hidden in our design to further maximize hardware utilization. We conducted extensive evaluations against various baselines, demonstrating up to a 3.3x speedup against standard speculative decoding.
• 摘要：推测性解码已成为一种关键技术，通过采用轻量级草稿模型来生成候选标记，然后由目标模型并行验证，从而加速 LLM 推理。然而，虽然这种范式成功地提高了内存限制推理的算术强度，但它导致了显着的计算效率低下：大多数草稿令牌未能通过验证并被丢弃，从而导致计算浪费。出于重新收集这种浪费的计算的目标，我们提出了一种新颖的系统，可以将废弃的草稿转换为可重复使用的令牌。我们的主要见解是在隐藏状态级别执行自回归预测，并在隐藏状态生成后推迟集成令牌信息，因此草案隐藏状态不会受到错误令牌的污染，从而实现隐藏状态的重用。为了实现这样的系统，我们首先引入基于自回归隐藏状态的草稿模型架构，它比基于令牌的绘图器保留更丰富的语义，以促进草稿的重新利用。其次，我们设计了一种高效的令牌信息注入机制，利用我们专门的草案模型来构建高质量的草案令牌树，并能够从验证失败中重新采样令牌。第三，我们消除了设计中隐藏的开销，以进一步最大化硬件利用率。我们针对各种基线进行了广泛的评估，证明与标准推测解码相比，速度提高了 3.3 倍。

Title: IslamicLegalBench: Evaluating LLMs Knowledge and Reasoning of Islamic Law Across 1,200 Years of Islamic Pluralist Legal Traditions

• Authors: Ezieddin Elmahjub, Junaid Qadir, Abdullah Mushtaq, Rafay Naeem, Ibrahim Ghaznavi, Waleed Iqbal
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21226
• Pdf URL: https://arxiv.org/pdf/2602.21226
• Copy Paste: [[2602.21226]] IslamicLegalBench: Evaluating LLMs Knowledge and Reasoning of Islamic Law Across 1,200 Years of Islamic Pluralist Legal Traditions(https://arxiv.org/abs/2602.21226)
• Keywords: gpt, llm, hallucination, prompt
• Abstract: As millions of Muslims turn to LLMs like GPT, Claude, and DeepSeek for religious guidance, a critical question arises: Can these AI systems reliably reason about Islamic law? We introduce IslamicLegalBench, the first benchmark evaluating LLMs across seven schools of Islamic jurisprudence, with 718 instances covering 13 tasks of varying complexity. Evaluation of nine state-of-the-art models reveals major limitations: the best model achieves only 68% correctness with 21% hallucination, while several models fall below 35% correctness and exceed 55% hallucination. Few-shot prompting provides minimal gains, improving only 2 of 9 models by >1%. Moderate-complexity tasks requiring exact knowledge show the highest errors, whereas high-complexity tasks display apparent competence through semantic reasoning. False premise detection indicates risky sycophancy, with 6 of 9 models accepting misleading assumptions at rates above 40%. These results highlight that prompt-based methods cannot compensate for missing foundational knowledge. IslamicLegalBench offers the first systematic framework to evaluate Islamic legal reasoning in AI, revealing critical gaps in tools increasingly relied on for spiritual guidance.
• 摘要：随着数百万穆斯林向 GPT、Claude 和 DeepSeek 等法学硕士寻求宗教指导，一个关键问题出现了：这些人工智能系统能否可靠地推理伊斯兰法律？我们推出了 Isl​​amLegalBench，这是第一个评估伊斯兰法学七所法学院法学硕士的基准，有 718 个实例，涵盖 13 项不同复杂程度的任务。对 9 个最先进模型的评估揭示了主要局限性：最好的模型仅达到 68% 的正确率和 21% 的幻觉，而一些模型的正确率低于 35%，而幻觉超过 55%。少样本提示带来的收益很小，9 个模型中只有 2 个模型的改进超过 1%。需要精确知识的中等复杂性任务显示出最高的错误，而高复杂性任务则通过语义推理显示出明显的能力。错误前提检测表明存在危险的阿谀奉承，9 个模型中有 6 个模型接受误导性假设的比例超过 40%。这些结果表明，基于提示的方法无法弥补基础知识的缺失。 IslamLegalBench 提供了第一个评估人工智能中伊斯兰法律推理的系统框架，揭示了越来越依赖精神指导的工具中的关键差距。

Title: Budget-Aware Agentic Routing via Boundary-Guided Training

• Authors: Caiqi Zhang, Menglin Xia, Xuchao Zhang, Daniel Madrigal, Ankur Mallick, Samuel Kessler, Victor Ruehle, Saravan Rajmohan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21227
• Pdf URL: https://arxiv.org/pdf/2602.21227
• Copy Paste: [[2602.21227]] Budget-Aware Agentic Routing via Boundary-Guided Training(https://arxiv.org/abs/2602.21227)
• Keywords: language model, llm, agent
• Abstract: As large language models (LLMs) evolve into autonomous agents that execute long-horizon workflows, invoking a high-capability model at every step becomes economically unsustainable. While model routing is effective for single-turn queries, agentic routing is a sequential, path-dependent problem: early mistakes compound, feedback is often at the end of the episode, and deployments often demand strict per-task spending limits. We propose Budget-Aware Agentic Routing, which selects between a cheap and an expensive model at each step to optimize the cost--success frontier and to operate under strict per-task budgets. We propose Boundary-Guided Training, which leverages two boundary policies (always-small vs.\ always-large) to build a difficulty taxonomy and to anchor learning under sparse rewards. Our approach warms start with boundary-guided SFT data synthesis via stratified sampling of cost-efficient trajectories, then applies Boundary-Guided Policy Optimization (BoPO), combining boundary-relative rewards with a reference-guided advantage to avoid degenerate cheap-failure solutions. Experiment results show that our method improves the efficiency frontier, matching strong routing baselines at substantially lower cost while demonstrating generalization to strict inference-time budget constraints. Overall, our work establishes a foundational framework for agentic routing, shifting the paradigm from static model selection to dynamic, budget-aware sequential decision-making.
• 摘要：随着大型语言模型 (LLM) 演变成执行长期工作流程的自主代理，在每一步都调用高性能模型在经济上变得不可持续。虽然模型路由对于单轮查询有效，但代理路由是一个顺序的、路径相关的问题：早期错误会复合，反馈通常在事件结束时出现，并且部署通常需要严格的每个任务支出限制。我们提出了预算感知代理路由，它在每个步骤中在廉价和昂贵的模型之间进行选择，以优化成本成功边界并在严格的每任务预算下运行。我们提出边界引导训练，它利用两种边界策略（总是小与总是大）来构建难度分类并将学习锚定在稀疏奖励下。我们的方法首先通过对具有成本效益的轨迹进行分层采样来进行边界引导的 SFT 数据合成，然后应用边界引导策略优化（BoPO），将边界相对奖励与参考引导优势相结合，以避免退化的廉价失败解决方案。实验结果表明，我们的方法提高了效率边界，以显着降低的成本匹配强大的路由基线，同时证明了对严格推理时间预算约束的泛化。总的来说，我们的工作为代理路由建立了一个基础框架，将范式从静态模型选择转变为动态的、预算敏感的顺序决策。

Title: ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following

• Authors: Yuancheng Yang, Lin Yang, Xu Wang, Chao Tong, Haihua Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21228
• Pdf URL: https://arxiv.org/pdf/2602.21228
• Copy Paste: [[2602.21228]] ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following(https://arxiv.org/abs/2602.21228)
• Keywords: language model, llm, chain-of-thought
• Abstract: As applications of large language models (LLMs) become increasingly complex, the demand for robust complex instruction following capabilities is growing accordingly. We argue that a thorough understanding of the instruction itself, especially the latent reasoning structure embedded between the lines, is crucial for improving instruction following. Therefore we target complex instructions that involve implicit reasoning, intricate logical relations, and multi-constraint dependencies. We propose ImpRIF, a method to enhance LLMs' understanding of implicit reasoning instructions, thereby improving its ability to follow complex instructions. We formalize such instructions as verifiable reasoning graphs, enabling programmatic verification and graph-driven chain-of-thought reasoning. Based on this formulation, we synthesize large-scale single- and multi-turn data, propose fine-tuning with graph reasoning, and apply reinforcement learning to explicitly train models to reason along the graph. On five complex instruction following benchmarks, our models substantially outperform their base models. These results demonstrate that enhancing implicit reasoning capabilities can significantly improve complex instruction following. This project will be open-sourced in the near future.
• 摘要：随着大语言模型（LLM）的应用变得越来越复杂，对强大的复杂指令跟踪能力的需求也相应增长。我们认为，对指令本身的透彻理解，尤其是嵌入在字里行间的潜在推理结构，对于提高指令遵循能力至关重要。因此，我们的目标是涉及隐式推理、复杂逻辑关系和多约束依赖性的复杂指令。我们提出了ImpRIF，一种增强法学硕士对隐式推理指令的理解的方法，从而提高其遵循复杂指令的能力。我们将此类指令形式化为可验证的推理图，从而实现程序化验证和图驱动的思维链推理。基于这个公式，我们综合了大规模的单轮和多轮数据，提出了图推理的微调，并应用强化学习来显式训练模型沿图进行推理。在五个复杂的指令跟踪基准上，我们的模型大大优于其基本模型。这些结果表明，增强隐式推理能力可以显着改善复杂指令的遵循。该项目将在不久的将来开源。

Title: TRACE: Trajectory-Aware Comprehensive Evaluation for Deep Research Agents

• Authors: Yanyu Chen, Jiyue Jiang, Jiahong Liu, Yifei Zhang, Xiao Guo, Irwin King
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21230
• Pdf URL: https://arxiv.org/pdf/2602.21230
• Copy Paste: [[2602.21230]] TRACE: Trajectory-Aware Comprehensive Evaluation for Deep Research Agents(https://arxiv.org/abs/2602.21230)
• Keywords: agent
• Abstract: The evaluation of Deep Research Agents is a critical challenge, as conventional outcome-based metrics fail to capture the nuances of their complex reasoning. Current evaluation faces two primary challenges: 1) a reliance on singular metrics like Pass@1, creating a "high-score illusion" that ignores the quality, efficiency, and soundness of the reasoning process; and 2) the failure of static benchmarks to quantify crucial attributes like robustness and latent capability. To address these gaps, we introduce TRACE (Trajectory-Aware Comprehensive Evaluation), a framework that holistically assesses the entire problem-solving trajectory. To counter the "high-score illusion", we propose a Hierarchical Trajectory Utility Function that quantifies process efficiency and cognitive quality, including evidence grounding, alongside accuracy. To measure deeper attributes, TRACE introduces a Scaffolded Capability Assessment protocol, quantifying an agent's latent ability by determining the minimum guidance needed for success. Our contributions include the TRACE framework, its novel metrics, and the accompanying DeepResearch-Bench with controllable complexity. Experiments show TRACE delivers a granular ranking that uncovers critical trade-offs between agent accuracy, efficiency, and robustness entirely missed by singular metrics.
• 摘要：深度研究代理的评估是一项严峻的挑战，因为传统的基于结果的指标无法捕捉其复杂推理的细微差别。当前的评估面临两个主要挑战：1）依赖像Pass@1这样的单一指标，造成“高分错觉”，忽略了推理过程的质量、效率和健全性； 2) 静态基准测试未能量化关键属性，如稳健性和潜在能力。为了解决这些差距，我们引入了 TRACE（轨迹感知综合评估），这是一个全面评估整个问题解决轨迹的框架。为了对抗“高分错觉”，我们提出了一种分层轨迹效用函数，该函数可以量化过程效率和认知质量，包括证据基础和准确性。为了衡量更深层次的属性，TRACE 引入了脚手架能力评估协议，通过确定成功所需的最低指导来量化代理的潜在能力。我们的贡献包括 TRACE 框架、其新颖的指标以及附带的具有可控复杂性的 DeepResearch-Bench。实验表明，TRACE 提供了精细的排名，揭示了单一指标完全忽略的代理准确性、效率和稳健性之间的关键权衡。

Title: Structured Prompt Language: Declarative Context Management for LLMs

• Authors: Wen G. Gong
• Subjects: cs.CL, cs.DB, cs.PL
• Abstract URL: https://arxiv.org/abs/2602.21257
• Pdf URL: https://arxiv.org/pdf/2602.21257
• Copy Paste: [[2602.21257]] Structured Prompt Language: Declarative Context Management for LLMs(https://arxiv.org/abs/2602.21257)
• Keywords: language model, llm, prompt, retrieval-augmented generation, agent
• Abstract: We present SPL (Structured Prompt Language), a declarative SQL-inspired language that treats large language models as generative knowledge bases and their context windows as constrained resources. SPL provides explicit WITH BUDGET/LIMIT token management, an automatic query optimizer, EXPLAIN transparency analogous to SQL's EXPLAIN ANALYZE, and native integration of retrieval-augmented generation (RAG) and persistent memory in a single declarative framework. SPL-flow extends SPL into resilient agentic pipelines with a three-tier provider fallback strategy (Ollama -> OpenRouter -> self-healing retry) fully transparent to the .spl script. Five extensions demonstrate the paradigm's breadth: (1) Text2SPL (multilingual NL->SPL translation); (2) Mixture-of-Models (MoM) routing that dispatches each PROMPT to a domain-specialist model at runtime; (3) Logical Chunking, an intelligent strategy for documents exceeding a single context window--expressed naturally through SPL's existing CTE syntax with no new constructs, decomposing a large query into a Map-Reduce pipeline that reduces attention cost from O(N^2) to O(N^2/k) and runs identically on cloud (parallel) or local hardware (sequential); (4) SPL-flow, a declarative agentic orchestration layer with resilient three-tier provider fallback; and (5) BENCHMARK for parallel multi-model comparison with automatic winner persistence. We provide a formal EBNF grammar, two pip-installable Python packages (spl-llm, spl-flow), and comparison against Prompty, DSPy, and LMQL. SPL reduces prompt boilerplate by 65% on average, surfaces a 68x cost spread across model tiers as a pre-execution signal, and runs the identical .spl script at $0.002 on OpenRouter or at zero marginal cost on a local Ollama instance--without modification.
• 摘要：我们提出了 SPL（结构化提示语言），这是一种受 SQL 启发的声明式语言，它将大型语言模型视为生成知识库，并将其上下文窗口视为受限资源。 SPL 提供显式的WITH BUDGET/LIMIT 令牌管理、自动查询优化器、类似于SQL 的EXPLAIN ANALYZE 的EXPLAIN 透明度，以及检索增强生成(RAG) 和持久内存在单个声明性框架中的本机集成。 SPL-flow 通过对 .spl 脚本完全透明的三层提供程序回退策略（Ollama -> OpenRouter -> 自愈重试）将 SPL 扩展到弹性代理管道中。五个扩展展示了该范例的广度：（1）Text2SPL（多语言 NL->SPL 翻译）； (2) 模型混合 (MoM) 路由，在运行时将每个 PROMPT 分派给领域专家模型； (3) 逻辑分块，一种针对超过单个上下文窗口的文档的智能策略——通过 SPL 现有的 CTE 语法自然地表达，无需新的构造，将大型查询分解为 Map-Reduce 管道，将注意力成本从 O(N^2) 降低到 O(N^2/k)，并在云（并行）或本地硬件（顺序）上相同地运行； (4) SPL-flow，一个声明式代理编排层，具有弹性的三层提供程序后备； (5) 用于并行多模型比较和自动获胜者持久性的 BENCHMARK。我们提供正式的 EBNF 语法、两个可通过 pip 安装的 Python 包（spl-llm、spl-flow），以及与 Prompty、DSPy 和 LMQL 的比较。 SPL 平均将提示样板文件减少 65%，将跨模型层的 68 倍成本分散作为预执行信号，并在 OpenRouter 上以 0.002 美元运行相同的 .spl 脚本，或者在本地 Ollama 实例上以零边际成本运行相同的 .spl 脚本，无需修改。

Title: Under the Influence: Quantifying Persuasion and Vigilance in Large Language Models

• Authors: Sasha Robinson, Kerem Oktar, Katherine M. Collins, Ilia Sucholutsky, Kelsey R. Allen
• Subjects: cs.CL, cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2602.21262
• Pdf URL: https://arxiv.org/pdf/2602.21262
• Copy Paste: [[2602.21262]] Under the Influence: Quantifying Persuasion and Vigilance in Large Language Models(https://arxiv.org/abs/2602.21262)
• Keywords: language model, llm, prompt, agent
• Abstract: With increasing integration of Large Language Models (LLMs) into areas of high-stakes human decision-making, it is important to understand the risks they introduce as advisors. To be useful advisors, LLMs must sift through large amounts of content, written with both benevolent and malicious intent, and then use this information to convince a user to take a specific action. This involves two social capacities: vigilance (the ability to determine which information to use, and which to discard) and persuasion (synthesizing the available evidence to make a convincing argument). While existing work has investigated these capacities in isolation, there has been little prior investigation of how these capacities may be linked. Here, we use a simple multi-turn puzzle-solving game, Sokoban, to study LLMs' abilities to persuade and be rationally vigilant towards other LLM agents. We find that puzzle-solving performance, persuasive capability, and vigilance are dissociable capacities in LLMs. Performing well on the game does not automatically mean a model can detect when it is being misled, even if the possibility of deception is explicitly mentioned. % as part of the prompt. However, LLMs do consistently modulate their token use, using fewer tokens to reason when advice is benevolent and more when it is malicious, even if they are still persuaded to take actions leading them to failure. To our knowledge, our work presents the first investigation of the relationship between persuasion, vigilance, and task performance in LLMs, and suggests that monitoring all three independently will be critical for future work in AI safety.
• 摘要：随着大型语言模型 (LLM) 越来越多地融入人类决策的高风险领域，了解它们作为顾问带来的风险非常重要。要成为有用的顾问，法学硕士必须筛选大量带有善意和恶意意图的内容，然后使用这些信息说服用户采取特定行动。这涉及两种社会能力：警惕性（确定使用哪些信息以及丢弃哪些信息的能力）和说服力（综合现有证据以提出令人信服的论点）。虽然现有的工作孤立地研究了这些能力，但之前很少研究这些能力如何联系起来。在这里，我们使用一个简单的多回合解谜游戏 Sokoban 来研究 LLM 说服其他 LLM 代理人并对其他 LLM 代理人保持理性警惕的能力。我们发现，解决难题的能力、说服能力和警惕性是法学硕士的可分离能力。即使明确提到了欺骗的可能性，在游戏中表现良好并不意味着模型可以自动检测到它何时被误导。 % 作为提示的一部分。然而，法学硕士确实一致地调整了他们的代币使用，当建议是善意的时候使用更少的代币来推理，当建议是恶意的时候使用更多的代币，即使他们仍然被说服采取导致他们失败的行动。据我们所知，我们的工作首次对法学硕士中的说服、警惕性和任务表现之间的关系进行了调查，并表明独立监控这三者对于人工智能安全的未来工作至关重要。

Title: ToolMATH: A Math Tool Benchmark for Realistic Long-Horizon Multi-Tool Reasoning

• Authors: Hyeonje Choi, Jeongsoo Lee, Hyojun Lee, Jay-Yoon Lee
• Subjects: cs.CL, cs.LG, cs.SE
• Abstract URL: https://arxiv.org/abs/2602.21265
• Pdf URL: https://arxiv.org/pdf/2602.21265
• Copy Paste: [[2602.21265]] ToolMATH: A Math Tool Benchmark for Realistic Long-Horizon Multi-Tool Reasoning(https://arxiv.org/abs/2602.21265)
• Keywords: language model, agent
• Abstract: We introduce \ToolMATH, a math-grounded benchmark that evaluates tool-augmented language models in realistic multi-tool environments where the output depends on calling schema-specified tools and sustaining multi-step execution. It turns math problems into a controlled, correctness-checkable benchmark with tool sets, enabling systematic evaluation of model reliability under (1) large, overlapping tool catalogs and (2) the absence of the intended capability. \ToolMATH provides actionable diagnostic evidence of failure modes in tool-augmented agents, helping identify the control mechanisms required for robustness. \ToolMATH roughly contains 8k questions and 12k tools; we provide an additional hard-set \ToolMATHHard with questions and tools. Our evaluation reveals that the key failure factor is due to the inability to reason, leading to the accumulation of intermediate results' errors and constrain later decisions. Tool-list redundancy do not simply add noise, but amplify small early deviations into irreversible execution drift. The benchmark highlights that when the intended capability is missing, distractor tools can sometimes serve as partial substitutes in solution paths, yet they can also mislead models into ungrounded tool trajectories. Finally, comparisons between tool-use protocols emphasize that improvements come less from local action selection and more from long-range plan coherence and disciplined use of observations.
• 摘要：我们引入了 \ToolMATH，一个基于数学的基准，它在现实的多工具环境中评估工具增强的语言模型，其中输出取决于调用模式指定的工具并维持多步骤执行。它将数学问题转化为带有工具集的受控、可检查正确性的基准，从而能够在（1）大型、重叠的工具目录和（2）缺乏预期功能的情况下系统地评估模型可靠性。 \ToolMATH 提供了工具增强代理中故障模式的可操作诊断证据，帮助确定鲁棒性所需的控制机制。 \ToolMATH 大致包含 8k 个问题和 12k 工具；我们提供了额外的硬集 \ToolMATHHard 以及问题和工具。我们的评估表明，失败的关键因素是由于无法推理，导致中间结果的错误累积并限制了后续决策。工具列表冗余不仅会增加噪音，还会将早期的微小偏差放大为不可逆的执行漂移。该基准强调，当预期功能缺失时，干扰工具有时可以充当解决方案路径中的部分替代品，但它们也可能会误导模型进入不可靠的工具轨迹。最后，工具使用协议之间的比较强调，改进较少来自局部行动选择，更多来自长期计划的一致性和观察的严格使用。

Title: Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment

• Authors: Mengxuan Hu, Vivek V. Datla, Anoop Kumar, Zihan Guan, Sheng Li, Alfy Samuel, Daben Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21346
• Pdf URL: https://arxiv.org/pdf/2602.21346
• Copy Paste: [[2602.21346]] Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment(https://arxiv.org/abs/2602.21346)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Recent advances in alignment techniques such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and Direct Preference Optimization (DPO) have improved the safety of large language models (LLMs). However, these LLMs remain vulnerable to jailbreak attacks that disguise harmful intent through indirect or deceptive phrasing. Using causal intervention, we empirically demonstrate that this vulnerability stems from shallow alignment mechanisms that lack deep reasoning, often rejecting harmful prompts without truly understanding why they are harmful. To mitigate this vulnerability, we propose enhancing alignment through reasoning-aware post-training. We construct and release a novel Chain-of-Thought (CoT) fine-tuning dataset that includes both utility-oriented and safety-critical prompts with step-by-step rationales. Fine-tuning on this dataset encourages models to produce principled refusals grounded in reasoning, outperforming standard SFT baselines. Furthermore, inspired by failure patterns in CoT fine-tuning, we introduce Alignment-Weighted DPO, which targets the most problematic parts of an output by assigning different preference weights to the reasoning and final-answer segments. This produces finer-grained, targeted updates than vanilla DPO and improves robustness to diverse jailbreak strategies. Extensive experiments across multiple safety and utility benchmarks show that our method consistently improves alignment robustness while maintaining overall model utility.
• 摘要：监督微调 (SFT)、人类反馈强化学习 (RLHF) 和直​​接偏好优化 (DPO) 等对齐技术的最新进展提高了大型语言模型 (LLM) 的安全性。然而，这些法学硕士仍然容易受到越狱攻击，这些攻击通过间接或欺骗性措辞来掩盖有害意图。通过因果干预，我们凭经验证明，这种脆弱性源于缺乏深层推理的浅层对齐机制，经常拒绝有害的提示，而没有真正理解它们为什么有害。为了缓解此漏洞，我们建议通过推理感知后训练来增强一致性。我们构建并发布了一个新颖的思想链（CoT）微调数据集，其中包括面向实用和安全关键的提示以及逐步的原理。对该数据集的微调鼓励模型产生基于推理的原则性拒绝，优于标准 SFT 基线。此外，受 CoT 微调失败模式的启发，我们引入了对齐加权 DPO，它通过为推理和最终答案片段分配不同的偏好权重来针对输出中最有问题的部分。与普通 DPO 相比，这会产生更细粒度、更有针对性的更新，并提高对各种越狱策略的稳健性。跨多个安全性和实用性基准的广泛实验表明，我们的方法不断提高对齐鲁棒性，同时保持整体模型实用性。

Title: Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages

• Authors: Mohammadreza Ghaffarzadeh-Esfahani, Nahid Yousefian, Ebrahim Heidari-Farsani, Ali Akbar Omidvarian, Sepehr Ghahraei, Atena Farangi, AmirBahador Boroumand
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.21374
• Pdf URL: https://arxiv.org/pdf/2602.21374
• Copy Paste: [[2602.21374]] Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages(https://arxiv.org/abs/2602.21374)
• Keywords: language model, prompt
• Abstract: Extracting clinical information from medical transcripts in low-resource languages remains a significant challenge in healthcare natural language processing (NLP). This study evaluates a two-step pipeline combining Aya-expanse-8B as a Persian-to-English translation model with five open-source small language models (SLMs) -- Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Llama-3.2-3B-Instruct, Qwen2.5-1.5B-Instruct, and Gemma-3-1B-it -- for binary extraction of 13 clinical features from 1,221 anonymized Persian transcripts collected at a cancer palliative care call center. Using a few-shot prompting strategy without fine-tuning, models were assessed on macro-averaged F1-score, Matthews Correlation Coefficient (MCC), sensitivity, and specificity to account for class imbalance. Qwen2.5-7B-Instruct achieved the highest overall performance (median macro-F1: 0.899; MCC: 0.797), while Gemma-3-1B-it showed the weakest results. Larger models (7B--8B parameters) consistently outperformed smaller counterparts in sensitivity and MCC. A bilingual analysis of Aya-expanse-8B revealed that translating Persian transcripts to English improved sensitivity, reduced missing outputs, and boosted metrics robust to class imbalance, though at the cost of slightly lower specificity and precision. Feature-level results showed reliable extraction of physiological symptoms across most models, whereas psychological complaints, administrative requests, and complex somatic features remained challenging. These findings establish a practical, privacy-preserving blueprint for deploying open-source SLMs in multilingual clinical NLP settings with limited infrastructure and annotation resources, and highlight the importance of jointly optimizing model scale and input language strategy for sensitive healthcare applications.
• 摘要：从低资源语言的医疗记录中提取临床信息仍然是医疗保健自然语言处理 (NLP) 中的一个重大挑战。本研究评估了将 Aya-expanse-8B 作为波斯语到英语翻译模型与五个开源小语言模型 (SLM)（Qwen2.5-7B-Instruct、Llama-3.1-8B-Instruct、Llama-3.2-3B-Instruct、Qwen2.5-1.5B-Instruct 和 Gemma-3-1B-it）相结合的两步管道，用于 13 个临床数据的二进制提取癌症姑息治疗呼叫中心收集的 1,221 份匿名波斯语笔录的特征。使用不进行微调的几次提示策略，根据宏观平均 F1 分数、马修斯相关系数 (MCC)、敏感性和特异性来评估模型，以考虑类别不平衡。 Qwen2.5-7B-Instruct 取得了最高的整体性能（中位宏 F1：0.899；MCC：0.797），而 Gemma-3-1B-it 显示出最弱的结果。较大的模型（7B--8B 参数）在灵敏度和 MCC 方面始终优于较小的模型。对 Aya-expanse-8B 的双语分析表明，将波斯语转录本翻译成英语可以提高敏感性，减少缺失的输出，并提高对类别不平衡的鲁棒性指标，但代价是特异性和精确度稍低。特征级结果表明，大多数模型都能可靠地提取生理症状，而心理投诉、行政请求和复杂的躯体特征仍然具有挑战性。这些发现为在基础设施和注释资源有限的多语言临床 NLP 设置中部署开源 SLM 建立了实用的、保护隐私的蓝图，并强调了针对敏感医疗保健应用联合优化模型规模和输入语言策略的重要性。

Title: Beyond Subtokens: A Rich Character Embedding for Low-resource and Morphologically Complex Languages

• Authors: Felix Schneider, Maria Gogolev, Sven Sickert, Joachim Denzler
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21377
• Pdf URL: https://arxiv.org/pdf/2602.21377
• Copy Paste: [[2602.21377]] Beyond Subtokens: A Rich Character Embedding for Low-resource and Morphologically Complex Languages(https://arxiv.org/abs/2602.21377)
• Keywords: language model, gpt
• Abstract: Tokenization and sub-tokenization based models like word2vec, BERT and the GPTs are the state-of-the-art in natural language processing. Typically, these approaches have limitations with respect to their input representation. They fail to fully capture orthographic similarities and morphological variations, especially in highly inflected and under-resource languages. To mitigate this problem, we propose to computes word vectors directly from character strings, integrating both semantic and syntactic information. We denote this transformer-based approach Rich Character Embeddings (RCE). Furthermore, we propose a hybrid model that combines transformer and convolutional mechanisms. Both vector representations can be used as a drop-in replacement for dictionary- and subtoken-based word embeddings in existing model architectures. It has the potential to improve performance for both large context-based language models like BERT and small models like word2vec for under-resourced and morphologically rich languages. We evaluate our approach on various tasks like the SWAG, declension prediction for inflected languages, metaphor and chiasmus detection for various languages. Our experiments show that it outperforms traditional token-based approaches on limited data using OddOneOut and TopK metrics.
• 摘要：基于分词和子分词的模型（如 word2vec、BERT 和 GPT）是自然语言处理领域的最先进技术。通常，这些方法在输入表示方面存在局限性。它们未能完全捕捉拼写相似性和形态变化，尤其是在高度屈折和资源贫乏的语言中。为了缓解这个问题，我们建议直接从字符串计算词向量，整合语义和句法信息。我们将这种基于变压器的方法表示为丰富字符嵌入（RCE）。此外，我们提出了一种结合了变压器和卷积机制的混合模型。这两种向量表示都可以用作现有模型架构中基于字典和基于子标记的词嵌入的直接替代。它有可能提高 BERT 等大型基于上下文的语言模型和 word2vec 等资源贫乏且形态丰富的语言的小型模型的性能。我们在各种任务上评估我们的方法，例如 SWAG、变形语言的词尾变化预测、各种语言的隐喻和交叉检测。我们的实验表明，它在使用 OddOneOut 和 TopK 指标的有限数据上优于传统的基于令牌的方法。

Title: VecGlypher: Unified Vector Glyph Generation with Language Models

• Authors: Xiaoke Huang, Bhavul Gauri, Kam Woh Ng, Tony Ng, Mengmeng Xu, Zhiheng Liu, Weiming Ren, Zhaochong An, Zijian Zhou, Haonan Qiu, Yuyin Zhou, Sen He, Ziheng Wang, Tao Xiang, Xiao Han
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21461
• Pdf URL: https://arxiv.org/pdf/2602.21461
• Copy Paste: [[2602.21461]] VecGlypher: Unified Vector Glyph Generation with Language Models(https://arxiv.org/abs/2602.21461)
• Keywords: language model, llm, prompt
• Abstract: Vector glyphs are the atomic units of digital typography, yet most learning-based pipelines still depend on carefully curated exemplar sheets and raster-to-vector postprocessing, which limits accessibility and editability. We introduce VecGlypher, a single multimodal language model that generates high-fidelity vector glyphs directly from text descriptions or image exemplars. Given a style prompt, optional reference glyph images, and a target character, VecGlypher autoregressively emits SVG path tokens, avoiding raster intermediates and producing editable, watertight outlines in one pass. A typography-aware data and training recipe makes this possible: (i) a large-scale continuation stage on 39K noisy Envato fonts to master SVG syntax and long-horizon geometry, followed by (ii) post-training on 2.5K expert-annotated Google Fonts with descriptive tags and exemplars to align language and imagery with geometry; preprocessing normalizes coordinate frames, canonicalizes paths, de-duplicates families, and quantizes coordinates for stable long-sequence decoding. On cross-family OOD evaluation, VecGlypher substantially outperforms both general-purpose LLMs and specialized vector-font baselines for text-only generation, while image-referenced generation reaches a state-of-the-art performance, with marked gains over DeepVecFont-v2 and DualVector. Ablations show that model scale and the two-stage recipe are critical and that absolute-coordinate serialization yields the best geometry. VecGlypher lowers the barrier to font creation by letting users design with words or exemplars, and provides a scalable foundation for future multimodal design tools.
• 摘要：矢量字形是数字排版的原子单位，但大多数基于学习的管道仍然依赖于精心策划的示例表和光栅到矢量的后处理，这限制了可访问性和可编辑性。我们引入了 VecGlypher，这是一种单一的多模态语言模型，可以直接从文本描述或图像示例生成高保真矢量字形。给定样式提示、可选的参考字形图像和目标字符，VecGlypher 自回归地发出 SVG 路径标记，避免光栅中间体并一次性生成可编辑的、无懈可击的轮廓。排版感知数据和训练方法使这一切成为可能：(i) 在 39K 嘈杂的 Envato 字体上进行大规模延续阶段，以掌握 SVG 语法和长视野几何形状，然后 (ii) 在 2.5K 专家注释的 Google 字体上进行后期训练，其中包含描述性标签和示例，以使语言和图像与几何形状保持一致；预处理对坐标系进行归一化、标准化路径、去重复族并量化坐标以实现稳定的长序列解码。在跨系列 OOD 评估中，VecGlypher 的性能大大优于通用 LLM 和用于纯文本生成的专用矢量字体基线，而图像引用生成达到了最先进的性能，与 DeepVecFont-v2 和 DualVector 相比有显着的提升。消融表明模型比例和两阶段配方至关重要，并且绝对坐标序列化可产生最佳几何形状。 VecGlypher 通过让用户使用文字或示例进行设计，降低了字体创建的障碍，并为未来的多模式设计工具提供了可扩展的基础。

Title: Evaluating the Usage of African-American Vernacular English in Large Language Models

• Authors: Deja Dunlap, R. Thomas McCoy
• Subjects: cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2602.21485
• Pdf URL: https://arxiv.org/pdf/2602.21485
• Copy Paste: [[2602.21485]] Evaluating the Usage of African-American Vernacular English in Large Language Models(https://arxiv.org/abs/2602.21485)
• Keywords: language model, llm, prompt
• Abstract: In AI, most evaluations of natural language understanding tasks are conducted in standardized dialects such as Standard American English (SAE). In this work, we investigate how accurately large language models (LLMs) represent African American Vernacular English (AAVE). We analyze three LLMs to compare their usage of AAVE to the usage of humans who natively speak AAVE. We first analyzed interviews from the Corpus of Regional African American Language and TwitterAAE to identify the typical contexts where people use AAVE grammatical features such as ain't. We then prompted the LLMs to produce text in AAVE and compared the model-generated text to human usage patterns. We find that, in many cases, there are substantial differences between AAVE usage in LLMs and humans: LLMs usually underuse and misuse grammatical features characteristic of AAVE. Furthermore, through sentiment analysis and manual inspection, we found that the models replicated stereotypes about African Americans. These results highlight the need for more diversity in training data and the incorporation of fairness methods to mitigate the perpetuation of stereotypes.
• 摘要：在人工智能中，大多数自然语言理解任务的评估都是用标准方言进行的，例如标准美式英语（SAE）。在这项工作中，我们研究了大型语言模型 (LLM) 代表非裔美国人白话英语 (AAVE) 的准确度。我们分析了三位法学硕士，将他们对 AAVE 的使用情况与母语为 AAVE 的人的使用情况进行比较。我们首先分析了非裔美国地区语言语料库和 TwitterAAE 的采访，以确定人们使用 AAVE 语法特征（例如 ain't）的典型语境。然后，我们提示法学硕士在 AAVE 中生成文本，并将模型生成的文本与人类使用模式进行比较。我们发现，在许多情况下，法学硕士和人类的 AAVE 使用之间存在重大差异：法学硕士通常未充分使用和滥用 AAVE 的语法特征特征。此外，通过情感分析和人工检查，我们发现这些模型复制了对非裔美国人的刻板印象。这些结果凸显了训练数据的多样性和纳入公平方法的必要性，以减轻刻板印象的持续存在。

Title: MixSarc: A Bangla-English Code-Mixed Corpus for Implicit Meaning Identification

• Authors: Kazi Samin Yasar Alam, Md Tanbir Chowdhury, Tamim Ahmed, Ajwad Abrar, Md Rafid Haque
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21608
• Pdf URL: https://arxiv.org/pdf/2602.21608
• Copy Paste: [[2602.21608]] MixSarc: A Bangla-English Code-Mixed Corpus for Implicit Meaning Identification(https://arxiv.org/abs/2602.21608)
• Keywords: language model, prompt
• Abstract: Bangla-English code-mixing is widespread across South Asian social media, yet resources for implicit meaning identification in this setting remain scarce. Existing sentiment and sarcasm models largely focus on monolingual English or high-resource languages and struggle with transliteration variation, cultural references, and intra-sentential language switching. To address this gap, we introduce MixSarc, the first publicly available Bangla-English code-mixed corpus for implicit meaning identification. The dataset contains 9,087 manually annotated sentences labeled for humor, sarcasm, offensiveness, and vulgarity. We construct the corpus through targeted social media collection, systematic filtering, and multi-annotator validation. We benchmark transformer-based models and evaluate zero-shot large language models under structured prompting. Results show strong performance on humor detection but substantial degradation on sarcasm, offense, and vulgarity due to class imbalance and pragmatic complexity. Zero-shot models achieve competitive micro-F1 scores but low exact match accuracy. Further analysis reveals that over 42\% of negative sentiment instances in an external dataset exhibit sarcastic characteristics. MixSarc provides a foundational resource for culturally aware NLP and supports more reliable multi-label modeling in code-mixed environments.
• 摘要：孟加拉语和英语的代码混合在南亚社交媒体中很普遍，但在这种情况下识别隐含含义的资源仍然稀缺。现有的情感和讽刺模型主要关注单语英语或高资源语言，并与音译变化、文化参考和句子内语言切换作斗争。为了解决这一差距，我们引入了 MixSarc，这是第一个公开可用的孟加拉英语代码混合语料库，用于隐含含义识别。该数据集包含 9,087 个手动注释的句子，标记为幽默、讽刺、冒犯性和粗俗。我们通过有针对性的社交媒体收集、系统过滤和多注释器验证来构建语料库。我们对基于 Transformer 的模型进行基准测试，并在结构化提示下评估零样本大型语言模型。结果显示，在幽默检测方面表现出色，但由于类别不平衡和语用复杂性，在讽刺、冒犯和粗俗方面表现明显下降。零样本模型实现了有竞争力的 micro-F1 分数，但精确匹配精度较低。进一步分析表明，外部数据集中超过 42% 的负面情绪实例表现出讽刺特征。 MixSarc 为具有文化意识的 NLP 提供了基础资源，并支持代码混合环境中更可靠的多标签建模。

Title: When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning

• Authors: Muku Akasaka, Soyeon Caren Han
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21619
• Pdf URL: https://arxiv.org/pdf/2602.21619
• Copy Paste: [[2602.21619]] When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning(https://arxiv.org/abs/2602.21619)
• Keywords: language model, prompt, chain-of-thought
• Abstract: Visual spatial reasoning (VSR) remains challenging for modern vision-language models (VLMs), despite advances in multimodal architectures. A common strategy is to inject additional information at inference time, such as explicit spatial cues, external commonsense knowledge, or chain-of-thought (CoT) reasoning instructions. However, it remains unclear when such information genuinely improves reasoning and when it introduces noise. In this paper, we conduct a hypothesis-driven analysis of information injection for VSR across three representative VLMs and two public benchmarks. We examine (i) the type and number of spatial contexts, (ii) the amount and relevance of injected commonsense knowledge, and (iii) the interaction between spatial grounding and CoT prompting. Our results reveal a consistent pattern: more information does not necessarily yield better reasoning. Targeted single spatial cues outperform multi-context aggregation, excessive or weakly relevant commonsense knowledge degrades performance, and CoT prompting improves accuracy only when spatial grounding is sufficiently precise. These findings highlight the importance of selective, task-aligned information injection and provide practical guidance for designing reliable multimodal reasoning pipelines.
• 摘要：尽管多模态架构取得了进步，但视觉空间推理（VSR）对于现代视觉语言模型（VLM）仍然具有挑战性。常见的策略是在推理时注入附加信息，例如明确的空间线索、外部常识知识或思想链 (CoT) 推理指令。然而，目前尚不清楚这些信息何时真正改善推理以及何时引入噪音。在本文中，我们对三个代表性 VLM 和两个公共基准的 VSR 信息注入进行了假设驱动的分析。我们检查（i）空间背景的类型和数量，（ii）注入常识知识的数量和相关性，以及（iii）空间基础和 CoT 提示之间的相互作用。我们的结果揭示了一个一致的模式：更多的信息并不一定会产生更好的推理。有针对性的单一空间线索优于多上下文聚合，过多或弱相关的常识知识会降低性能，而 CoT 提示只有在空间基础足够精确时才能提高准确性。这些发现强调了选择性的、与任务相关的信息注入的重要性，并为设计可靠的多模式推理管道提供了实用指导。

Title: RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning

• Authors: Yukun Chen, Jiaming Li, Longze Chen, Ze Gong, Jingpeng Li, Zhen Qin, Hengyu Chang, Ancheng Xu, Zhihao Yang, Hamid Alinejad-Rokny, Qiang Qu, Bo Zheng, Min Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21628
• Pdf URL: https://arxiv.org/pdf/2602.21628
• Copy Paste: [[2602.21628]] RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning(https://arxiv.org/abs/2602.21628)
• Keywords: language model, llm
• Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a prevailing paradigm for enhancing reasoning in Multimodal Large Language Models (MLLMs). However, relying solely on outcome supervision risks reward hacking, where models learn spurious reasoning patterns to satisfy final answer checks. While recent rubric-based approaches offer fine-grained supervision signals, they suffer from high computational costs of instance-level generation and inefficient training dynamics caused by treating all rubrics as equally learnable. In this paper, we propose Stratified Rubric-based Curriculum Learning (RuCL), a novel framework that reformulates curriculum learning by shifting the focus from data selection to reward design. RuCL generates generalized rubrics for broad applicability and stratifies them based on the model's competence. By dynamically adjusting rubric weights during training, RuCL guides the model from mastering foundational perception to tackling advanced logical reasoning. Extensive experiments on various visual reasoning benchmarks show that RuCL yields a remarkable +7.83% average improvement over the Qwen2.5-VL-7B model, achieving a state-of-the-art accuracy of 60.06%.
• 摘要：具有可验证奖励的强化学习（RLVR）已成为增强多模态大语言模型（MLLM）推理的流行范例。然而，仅仅依赖结果监督可能会带来黑客攻击的风险，模型会学习虚假的推理模式来满足最终答案检查。虽然最近基于评分标准的方法提供了细粒度的监督信号，但由于将所有评分标准视为同等可学习性，它们面临实例级生成的高计算成本和低效的训练动态。在本文中，我们提出了基于分层标题的课程学习（RuCL），这是一种新颖的框架，通过将重点从数据选择转移到奖励设计来重新制定课程学习。 RuCL 生成具有广泛适用性的通用评价标准，并根据模型的能力对它们进行分层。通过在训练过程中动态调整标题权重，RuCL 指导模型从掌握基础感知到解决高级逻辑推理。对各种视觉推理基准的大量实验表明，RuCL 比 Qwen2.5-VL-7B 模型平均提高了 7.83%，达到了 60.06% 的最先进准确率。

Title: Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs

• Authors: Anqi Li, Ruihan Wang, Zhaoming Chen, Yuqian Chen, Yu Lu, Yi Zhu, Yuan Xie, Zhenzhong Lan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21638
• Pdf URL: https://arxiv.org/pdf/2602.21638
• Copy Paste: [[2602.21638]] Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs(https://arxiv.org/abs/2602.21638)
• Keywords: gpt, llm
• Abstract: Effectively addressing client resistance is a sophisticated clinical skill in psychological counseling, yet practitioners often lack timely and scalable supervisory feedback to refine their approaches. Although current NLP research has examined overall counseling quality and general therapeutic skills, it fails to provide granular evaluations of high-stakes moments where clients exhibit resistance. In this work, we present a comprehensive pipeline for the multi-dimensional evaluation of human counselors' interventions specifically targeting client resistance in text-based therapy. We introduce a theory-driven framework that decomposes counselor responses into four distinct communication mechanisms. Leveraging this framework, we curate and share an expert-annotated dataset of real-world counseling excerpts, pairing counselor-client interactions with professional ratings and explanatory rationales. Using this data, we perform full-parameter instruction tuning on a Llama-3.1-8B-Instruct backbone to model fine-grained evaluative judgments of response quality and generate explanations underlying. Experimental results show that our approach can effectively distinguish the quality of different communication mechanisms (77-81% F1), substantially outperforming GPT-4o and Claude-3.5-Sonnet (45-59% F1). Moreover, the model produces high-quality explanations that closely align with expert references and receive near-ceiling ratings from human experts (2.8-2.9/3.0). A controlled experiment with 43 counselors further confirms that receiving these AI-generated feedback significantly improves counselors' ability to respond effectively to client resistance.
• 摘要：有效解决客户阻力是心理咨询中一项复杂的临床技能，但从业者往往缺乏及时且可扩展的监督反馈来完善他们的方法。尽管当前的 NLP 研究已经检验了整体咨询质量和一般治疗技能，但它无法对客户表现出抵制的高风险时刻提供精细评估。在这项工作中，我们提出了一个全面的管道，用于对人类咨询师的干预措施进行多维评估，专门针对客户对基于文本的治疗的抵制。我们引入了一个理论驱动的框架，将咨询师的反应分解为四种不同的沟通机制。利用这个框架，我们策划并分享由专家注释的真实咨询摘录数据集，将咨询师与客户的互动与专业评级和解释性理由相结合。使用这些数据，我们在 Llama-3.1-8B-Instruct 主干上执行全参数指令调整，以对响应质量的细粒度评估判断进行建模并生成底层解释。实验结果表明，我们的方法可以有效区分不同通信机制的质量（77-81％F1），大大优于GPT-4o和Claude-3.5-Sonnet（45-59％F1）。此外，该模型产生的高质量解释与专家参考密切相关，并获得人类专家的接近上限评级 (2.8-2.9/3.0)。一项针对 43 名咨询师的对照实验进一步证实，接收这些人工智能生成的反馈可以显着提高咨询师有效应对客户阻力的能力。

Title: Scalable Multilingual Multimodal Machine Translation with Speech-Text Fusion

• Authors: Yexing Du, Youcheng Pan, Zekun Wang, Zheng Chu, Yichong Huang, Kaiyuan Liu, Bo Yang, Yang Xiang, Ming Liu, Bing Qin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21646
• Pdf URL: https://arxiv.org/pdf/2602.21646
• Copy Paste: [[2602.21646]] Scalable Multilingual Multimodal Machine Translation with Speech-Text Fusion(https://arxiv.org/abs/2602.21646)
• Keywords: language model, llm
• Abstract: Multimodal Large Language Models (MLLMs) have achieved notable success in enhancing translation performance by integrating multimodal information. However, existing research primarily focuses on image-guided methods, whose applicability is constrained by the scarcity of multilingual image-text pairs. The speech modality overcomes this limitation due to its natural alignment with text and the abundance of existing speech datasets, which enable scalable language coverage. In this paper, we propose a Speech-guided Machine Translation (SMT) framework that integrates speech and text as fused inputs into an MLLM to improve translation quality. To mitigate reliance on low-resource data, we introduce a Self-Evolution Mechanism. The core components of this framework include a text-to-speech model, responsible for generating synthetic speech, and an MLLM capable of classifying synthetic speech samples and iteratively optimizing itself using positive samples. Experimental results demonstrate that our framework surpasses all existing methods on the Multi30K multimodal machine translation benchmark, achieving new state-of-the-art results. Furthermore, on general machine translation datasets, particularly the FLORES-200, it achieves average state-of-the-art performance in 108 translation directions. Ablation studies on CoVoST-2 confirms that differences between synthetic and authentic speech have negligible impact on translation quality. The code and models are released at this https URL.
• 摘要：多模态大语言模型 (MLLM) 通过集成多模态信息在提高翻译性能方面取得了显着的成功。然而，现有的研究主要集中在图像引导方法，其适用性受到多语言图像文本对稀缺的限制。语音模态克服了这一限制，因为它与文本自然对齐，并且现有语音数据集丰富，从而实现了可扩展的语言覆盖范围。在本文中，我们提出了一种语音引导机器翻译 (SMT) 框架，它将语音和文本作为融合输入集成到 MLLM 中，以提高翻译质量。为了减轻对低资源数据的依赖，我们引入了自我进化机制。该框架的核心组件包括负责生成合成语音的文本转语音模型，以及能够对合成语音样本进行分类并使用正样本迭代优化自身的 MLLM。实验结果表明，我们的框架在 Multi30K 多模态机器翻译基准上超越了所有现有方法，取得了新的最先进的结果。此外，在通用机器翻译数据集上，特别是 FLORES-200，它在 108 个翻译方向上实现了平均最先进的性能。对 CoVoST-2 的消融研究证实，合成语音和真实语音之间的差异对翻译质量的影响可以忽略不计。代码和模型在此 https URL 发布。

Title: Sparsity Induction for Accurate Post-Training Pruning of Large Language Models

• Authors: Minhao Jiang, Zhikai Li, Xuewen Liu, Jing Zhang, Mengjuan Chen, Qingyi Gu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.21652
• Pdf URL: https://arxiv.org/pdf/2602.21652
• Copy Paste: [[2602.21652]] Sparsity Induction for Accurate Post-Training Pruning of Large Language Models(https://arxiv.org/abs/2602.21652)
• Keywords: language model
• Abstract: Large language models have demonstrated capabilities in text generation, while their increasing parameter scales present challenges in computational and memory efficiency. Post-training sparsity (PTS), which reduces model cost by removing weights from dense networks, is an effective approach. However, native dense matrices lack high sparsity, making existing approaches that directly remove weights disrupt model states, resulting in unsatisfactory performance recovery even with post-tuning. We propose Sparsity Induction, which promotes models toward higher sparsity at both distribution and feature levels before pruning, to push the limits of PTS. At the distribution level, we enhance distributional sparsity through mathematically equivalent scaling transformations, which are fully absorbable and incur no extra parameters or inference-time overhead. At the feature level, we introduce Spectral Norm Loss to promote feature sparsity from a low-rank perspective. Experiments across diverse model architectures and tasks demonstrate that our method further enhances sparsity-friendliness, achieving superior pruning performance over existing approaches.
• 摘要：大型语言模型已经展示了文本生成的能力，但其不断增加的参数规模对计算和内存效率提出了挑战。训练后稀疏性（PTS）是一种有效的方法，它通过从密集网络中删除权重来降低模型成本。然而，原生稠密矩阵缺乏高稀疏性，使得现有的直接去除权重的方法会扰乱模型状态，导致即使进行后调整也无法令人满意的性能恢复。我们提出稀疏归纳，它在剪枝之前促进模型在分布和特征级别上达到更高的稀疏性，以突破 PTS 的极限。在分布层面，我们通过数学上等效的缩放变换来增强分布稀疏性，这些变换是完全可吸收的，并且不会产生额外的参数或推理时间开销。在特征层面，我们引入了谱范数损失（Spectral Norm Loss），从低秩的角度促进特征稀疏性。跨不同模型架构和任务的实验表明，我们的方法进一步增强了稀疏性友好性，实现了优于现有方法的剪枝性能。

Title: DWA-KD: Dual-Space Weighting and Time-Warped Alignment for Cross-Tokenizer Knowledge Distillation

• Authors: Duc Trung Vu, Pham Khanh Chi, Dat Phi Van, Linh Ngo Van, Sang Dinh, Trung Le
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21669
• Pdf URL: https://arxiv.org/pdf/2602.21669
• Copy Paste: [[2602.21669]] DWA-KD: Dual-Space Weighting and Time-Warped Alignment for Cross-Tokenizer Knowledge Distillation(https://arxiv.org/abs/2602.21669)
• Keywords: language model, llm
• Abstract: Knowledge Distillation (KD) has emerged as a crucial technique for compressing Large Language Models (LLMs). Although existing cross-tokenizer KD methods have made notable progress, their effectiveness remains constrained by suboptimal alignment across sequence and vocabulary levels. To address these limitations, we introduce Dual-Space Weighting and Time-Warped Alignment (DWA-KD), a novel cross-tokenizer distillation framework that enhances token-wise distillation through dual-space entropy-based weighting and achieves precise sequence-level alignment by leveraging both lexical and semantic information. At the token level, DWA-KD maps teacher representations into the student space and vice versa, performing dual-space KD via Kullback-Leibler divergence (KL). The process is modulated by dual-space weights that up-weight tokens where the student is uncertain and the teacher is confident, thereby focusing learning on informative tokens rather than treating all positions equally. At the sequence level, DWA-KD applies Soft Dynamic Time Warping (Soft-DTW) to both the embedding and final hidden-state layers, enabling robust alignment of lexical and contextual semantics between teacher and student sequences. Extensive experiments across diverse NLP benchmarks demonstrate that DWA-KD outperforms state-of-the-art KD baselines, while ablation studies confirm the complementary contributions of entropy-based token weighting and embedding and final hidden state layer Soft-DTW alignment.
• 摘要：知识蒸馏 (KD) 已成为压缩大型语言模型 (LLM) 的关键技术。尽管现有的交叉分词器 KD 方法已经取得了显着的进展，但其有效性仍然受到序列和词汇级别之间的次优对齐的限制。为了解决这些限制，我们引入了双空间加权和时间扭曲对齐（DWA-KD），这是一种新颖的跨分词器蒸馏框架，它通过基于双空间熵的加权增强分词蒸馏，并通过利用词汇和语义信息实现精确的序列级对齐。在令牌级别，DWA-KD 将教师表示映射到学生空间，反之亦然，通过 Kullback-Leibler 散度 (KL) 执行双空间 KD。该过程通过双空间权重进行调节，在学生不确定而教师有信心的情况下增加标记的权重，从而将学习重点放在信息丰富的标记上，而不是平等对待所有位置。在序列级别，DWA-KD 将软动态时间规整 (Soft-DTW) 应用于嵌入层和最终隐藏状态层，从而实现教师和学生序列之间的词汇和上下文语义的稳健对齐。跨不同 NLP 基准的广泛实验表明，DWA-KD 优于最先进的 KD 基线，而消融研究证实了基于熵的令牌加权和嵌入以及最终隐藏状态层 Soft-DTW 对齐的互补贡献。

Title: Explore-on-Graph: Incentivizing Autonomous Exploration of Large Language Models on Knowledge Graphs with Path-refined Reward Modeling

• Authors: Shiqi Yan, Yubo Chen, Ruiqi Zhou, Zhengxi Yao, Shuai Chen, Tianyi Zhang, Shijie Zhang, Wei Qiang Zhang, Yongfeng Huang, Haixin Duan, Yunqi Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21728
• Pdf URL: https://arxiv.org/pdf/2602.21728
• Copy Paste: [[2602.21728]] Explore-on-Graph: Incentivizing Autonomous Exploration of Large Language Models on Knowledge Graphs with Path-refined Reward Modeling(https://arxiv.org/abs/2602.21728)
• Keywords: language model, llm, hallucination
• Abstract: The reasoning process of Large Language Models (LLMs) is often plagued by hallucinations and missing facts in question-answering tasks. A promising solution is to ground LLMs' answers in verifiable knowledge sources, such as Knowledge Graphs (KGs). Prevailing KG-enhanced methods typically constrained LLM reasoning either by enforcing rules during generation or by imitating paths from a fixed set of demonstrations. However, they naturally confined the reasoning patterns of LLMs within the scope of prior experience or fine-tuning data, limiting their generalizability to out-of-distribution graph reasoning problems. To tackle this problem, in this paper, we propose Explore-on-Graph (EoG), a novel framework that encourages LLMs to autonomously explore a more diverse reasoning space on KGs. To incentivize exploration and discovery of novel reasoning paths, we propose to introduce reinforcement learning during training, whose reward is the correctness of the reasoning paths' final answers. To enhance the efficiency and meaningfulness of the exploration, we propose to incorporate path information as additional reward signals to refine the exploration process and reduce futile efforts. Extensive experiments on five KGQA benchmark datasets demonstrate that, to the best of our knowledge, our method achieves state-of-the-art performance, outperforming not only open-source but also even closed-source LLMs.
• 摘要：大型语言模型（LLM）的推理过程经常受到幻觉和问答任务中缺失事实的困扰。一个有前途的解决方案是将法学硕士的答案建立在可验证的知识源中，例如知识图（KG）。流行的 KG 增强方法通常通过在生成过程中强制执行规则或通过模仿一组固定演示的路径来限制 LLM 推理。然而，他们自然地将法学硕士的推理模式限制在先前经验或微调数据的范围内，限制了它们对分布外图推理问题的泛化能力。为了解决这个问题，在本文中，我们提出了 Explore-on-Graph (EoG)，这是一种鼓励 LLM 自主探索 KG 上更加多样化的推理空间的新颖框架。为了激励探索和发现新颖的推理路径，我们建议在训练期间引入强化学习，其奖励是推理路径最终答案的正确性。为了提高探索的效率和意义，我们建议将路径信息作为额外的奖励信号，以完善探索过程并减少无用的努力。对五个 KGQA 基准数据集的广泛实验表明，据我们所知，我们的方法实现了最先进的性能，不仅优于开源法学硕士，甚至优于闭源法学硕士。

Title: Improving Implicit Discourse Relation Recognition with Natural Language Explanations from LLMs

• Authors: Heng Wang, Changxing Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21763
• Pdf URL: https://arxiv.org/pdf/2602.21763
• Copy Paste: [[2602.21763]] Improving Implicit Discourse Relation Recognition with Natural Language Explanations from LLMs(https://arxiv.org/abs/2602.21763)
• Keywords: language model, llm, prompt
• Abstract: Implicit Discourse Relation Recognition (IDRR) remains a challenging task due to the requirement for deep semantic understanding in the absence of explicit discourse markers. A further limitation is that existing methods only predict relations without providing any supporting explanations. Recent advances in large language models (LLMs) have shown strong reasoning capabilities in both deep language understanding and natural language explanation generation. In this work, we propose a simple yet effective approach to distill the reasoning capabilities of LLMs into lightweight IDRR models to improve both performance and interpretability. Specifically, we first prompt an LLM to generate explanations for each training instance conditioned on its gold label. Then, we introduce a novel classification-generation framework that jointly performs relation prediction and explanation generation, and train it with the additional supervision of LLM-generated explanations. Our framework is plug-and-play, enabling easy integration with most existing IDRR models. Experimental results on PDTB demonstrate that our approach significantly improves IDRR performance, while human evaluation further confirms that the generated explanations enhance model interpretability. Furthermore, we validate the generality of our approach on sentiment classification and natural language inference
• 摘要：由于在缺乏显式话语标记的情况下需要深入的语义理解，隐式话语关系识别（IDRR）仍然是一项具有挑战性的任务。进一步的限制是现有方法仅预测关系而不提供任何支持解释。大语言模型（LLM）的最新进展在深度语言理解和自然语言解释生成方面表现出了强大的推理能力。在这项工作中，我们提出了一种简单而有效的方法，将 LLM 的推理能力提炼为轻量级 IDRR 模型，以提高性能和可解释性。具体来说，我们首先提示法学硕士为每个以其黄金标签为条件的训练实例生成解释。然后，我们引入了一种新颖的分类生成框架，该框架联合执行关系预测和解释生成，并在 LLM 生成的解释的额外监督下对其进行训练。我们的框架是即插即用的，可以轻松与大多数现有 IDRR 模型集成。 PDTB 上的实验结果表明，我们的方法显着提高了 IDRR 性能，而人类评估进一步证实了生成的解释增强了模型的可解释性。此外，我们验证了我们的情感分类和自然语言推理方法的通用性

Title: D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models

• Authors: Shunsuke Ubukata
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21786
• Pdf URL: https://arxiv.org/pdf/2602.21786
• Copy Paste: [[2602.21786]] D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models(https://arxiv.org/abs/2602.21786)
• Keywords: language model, llm, chain-of-thought
• Abstract: Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) often induces "overthinking" in Small Language Models (SLMs), leading to performance degradation and excessive token consumption. In this study, we propose Disciplined Chain-of-Thought (D-CoT), a novel framework that enforces a structured reasoning process using control tags -- such as for fact-checking and for multi-perspective exploration -- as auxiliary scaffolding during training. By optimizing the CoT trajectory, D-CoT suppresses reasoning drift and simultaneously achieves token reduction and performance improvement. We demonstrate the efficacy of our approach on Qwen3-8B: with only 5,000 training samples, D-CoT significantly boosts accuracy on GPQA-diamond by 9.9% and MMLU-Pro (0-shot) by 9.1%, while drastically reducing computational costs. Furthermore, we confirm that the model internalizes this disciplined thought structure, maintaining high performance even without explicit control tags during inference.
• 摘要：从大型语言模型 (LLM) 中进行的思想链 (CoT) 蒸馏通常会导致小语言模型 (SLM) 中的“过度思考”，从而导致性能下降和过度的代币消耗。在这项研究中，我们提出了纪律思想链（D-CoT），这是一种新颖的框架，它使用控制标签（例如用于事实检查的 和用于多视角探索的 ）作为训练期间的辅助脚手架来强制执行结构化推理过程。 D-CoT通过优化CoT轨迹，抑制推理漂移，同时实现令牌减少和性能提升。我们在 Qwen3-8B 上展示了我们的方法的有效性：仅用 5,000 个训练样本，D-CoT 就将 GPQA-diamond 的准确性显着提高了 9.9%，将 MMLU-Pro（0-shot）的准确性显着提高了 9.1%，同时大大降低了计算成本。此外，我们确认该模型将这种严格的思维结构内化，即使在推理过程中没有明确的控制标签也能保持高性能。

Title: FewMMBench: A Benchmark for Multimodal Few-Shot Learning

• Authors: Mustafa Dogan, Ilker Kesen, Iacer Calixto, Aykut Erdem, Erkut Erdem
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21854
• Pdf URL: https://arxiv.org/pdf/2602.21854
• Copy Paste: [[2602.21854]] FewMMBench: A Benchmark for Multimodal Few-Shot Learning(https://arxiv.org/abs/2602.21854)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: As multimodal large language models (MLLMs) advance in handling interleaved image-text data, assessing their few-shot learning capabilities remains an open challenge. In this paper, we introduce FewMMBench, a comprehensive benchmark designed to evaluate MLLMs under few-shot conditions, with a focus on In-Context Learning (ICL) and Chain-of-Thought (CoT) prompting. Covering a diverse suite of multimodal understanding tasks, from attribute recognition to temporal reasoning, FewMMBench enables systematic analysis across task types, model families, and prompting strategies. We evaluate 26 open-weight MLLMs from six model families across zero-shot, few-shot, and CoT-augmented few-shot settings. Our findings reveal that instruction-tuned models exhibit strong zero-shot performance but benefit minimally, or even regress, with additional demonstrations or CoT reasoning. Retrieval-based demonstrations and increased context size also yield limited gains. These results highlight FewMMBench as a rigorous testbed for diagnosing and advancing few-shot capabilities in multimodal LLMs. The data is available at: this https URL
• 摘要：随着多模态大语言模型 (MLLM) 在处理交错图像文本数据方面取得进展，评估其少样本学习能力仍然是一个开放的挑战。在本文中，我们介绍了 FewMMBench，这是一个综合基准测试，旨在评估少样本条件下的 MLLM，重点关注上下文学习 (ICL) 和思想链 (CoT) 提示。 FewMMBench 涵盖从属性识别到时间推理的各种多模态理解任务，能够跨任务类型、模型系列和提示策略进行系统分析。我们评估了来自 6 个模型系列的 26 个开放权重 MLLM，涉及零样本、少样本和 CoT 增强的少样本设置。我们的研究结果表明，指令调整模型表现出强大的零样本性能，但通过额外的演示或 CoT 推理获益甚微，甚至出现倒退。基于检索的演示和增加的上下文大小也产生有限的收益。这些结果凸显了 FewMMBench 作为一个严格的测试平台，用于诊断和提升多模式 LLM 中的少样本功能。数据可在以下位置获得：此 https URL

Title: Personalized Graph-Empowered Large Language Model for Proactive Information Access

• Authors: Chia Cheng Chang, An-Zi Yen, Hen-Hsen Huang, Hsin-Hsi Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21862
• Pdf URL: https://arxiv.org/pdf/2602.21862
• Copy Paste: [[2602.21862]] Personalized Graph-Empowered Large Language Model for Proactive Information Access(https://arxiv.org/abs/2602.21862)
• Keywords: language model, llm
• Abstract: Since individuals may struggle to recall all life details and often confuse events, establishing a system to assist users in recalling forgotten experiences is essential. While numerous studies have proposed memory recall systems, these primarily rely on deep learning techniques that require extensive training and often face data scarcity due to the limited availability of personal lifelogs. As lifelogs grow over time, systems must also adapt quickly to newly accumulated data. Recently, large language models (LLMs) have demonstrated remarkable capabilities across various tasks, making them promising for personalized applications. In this work, we present a framework that leverages LLMs for proactive information access, integrating personal knowledge graphs to enhance the detection of access needs through a refined decision-making process. Our framework offers high flexibility, enabling the replacement of base models and the modification of fact retrieval methods for continuous improvement. Experimental results demonstrate that our approach effectively identifies forgotten events, supporting users in recalling past experiences more efficiently.
• 摘要：由于个人可能很难回忆起所有生活细节并且经常混淆事件，因此建立一个系统来帮助用户回忆被遗忘的经历至关重要。虽然许多研究提出了记忆回忆系统，但这些系统主要依赖于深度学习技术，需要大量的训练，并且由于个人生活日志的可用性有限，常常面临数据稀缺的问题。随着生活日志随着时间的推移而增长，系统还必须快速适应新积累的数据。最近，大型语言模型（LLM）在各种任务中表现出了卓越的能力，这使得它们有望用于个性化应用。在这项工作中，我们提出了一个框架，利用法学硕士进行主动信息访问，集成个人知识图谱，通过精细的决策过程来增强对访问需求的检测。我们的框架提供了高度的灵活性，可以替换基本模型并修改事实检索方法以实现持续改进。实验结果表明，我们的方法可以有效识别遗忘的事件，支持用户更有效地回忆过去的经历。

Title: ExpLang: Improved Exploration and Exploitation in LLM Reasoning with On-Policy Thinking Language Selection

• Authors: Changjiang Gao, Zixian Huang, Kaichen Yang, Jiajun Chen, Jixing Li, Shujian Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21887
• Pdf URL: https://arxiv.org/pdf/2602.21887
• Copy Paste: [[2602.21887]] ExpLang: Improved Exploration and Exploitation in LLM Reasoning with On-Policy Thinking Language Selection(https://arxiv.org/abs/2602.21887)
• Keywords: llm
• Abstract: Current large reasoning models (LRMs) have shown strong ability on challenging tasks after reinforcement learning (RL) based post-training. However, previous work mainly focuses on English reasoning in expectation of the strongest performance, despite the demonstrated potential advantage of multilingual thinking, as well as the requirement for native thinking traces by global users. In this paper, we propose ExpLang, a novel LLM post-training pipeline that enables on-policy thinking language selection to improve exploration and exploitation during RL with the use of multiple languages. The results show that our method steadily outperforms English-only training with the same training budget, while showing high thinking language compliance for both seen and unseen languages. Analysis shows that, by enabling on-policy thinking language selection as an action during RL, ExpLang effectively extends the RL exploration space with diversified language preference and improves the RL exploitation outcome with leveraged non-English advantage. The method is orthogonal to most RL algorithms and opens up a new perspective on using multilinguality to improve LRMs.
• 摘要：目前的大型推理模型（LRM）在基于强化学习（RL）的后期训练后表现出了处理挑战性任务的强大能力。然而，尽管多语言思维已展现出潜在优势，以及全球用户对母语思维痕迹的要求，但之前的工作主要集中在英语推理上，以期获得最强的性能。在本文中，我们提出了 ExpLang，一种新颖的 LLM 后训练流程，它支持策略思维语言选择，以使用多种语言来改进 RL 期间的探索和开发。结果表明，在相同的训练预算下，我们的方法稳步优于纯英语训练，同时对可见语言和未见语言都表现出高度的思维语言合规性。分析表明，通过将策略思维语言选择作为强化学习期间的一个动作，ExpLang 有效地扩展了具有多样化语言偏好的强化学习探索空间，并利用非英语优势提高了强化学习开发结果。该方法与大多数 RL 算法正交，并为使用多语言改进 LRM 开辟了新的视角。

Title: Small Wins Big: Comparing Large Language Models and Domain Fine-Tuned Models for Sarcasm Detection in Code-Mixed Hinglish Text

• Authors: Bitan Majumder, Anirban Sen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21933
• Pdf URL: https://arxiv.org/pdf/2602.21933
• Copy Paste: [[2602.21933]] Small Wins Big: Comparing Large Language Models and Domain Fine-Tuned Models for Sarcasm Detection in Code-Mixed Hinglish Text(https://arxiv.org/abs/2602.21933)
• Keywords: language model, llm
• Abstract: Sarcasm detection in multilingual and code-mixed environments remains a challenging task for natural language processing models due to structural variations, informal expressions, and low-resource linguistic availability. This study compares four large language models, Llama 3.1, Mistral, Gemma 3, and Phi-4, with a fine-tuned DistilBERT model for sarcasm detection in code-mixed Hinglish text. The results indicate that the smaller, sequentially fine-tuned DistilBERT model achieved the highest overall accuracy of 84%, outperforming all of the LLMs in zero and few-shot set ups, using minimal LLM generated code-mixed data used for fine-tuning. These findings indicate that domain-adaptive fine-tuning of smaller transformer based models may significantly improve sarcasm detection over general LLM inference, in low-resource and data scarce settings.
• 摘要：由于结构变化、非正式表达和低资源语言可用性，多语言和代码混合环境中的讽刺检测对于自然语言处理模型来说仍然是一项具有挑战性的任务。本研究将四种大型语言模型（Llama 3.1、Mistral、Gemma 3 和 Phi-4）与微调的 DistilBERT 模型进行比较，用于代码混合的印度英语文本中的讽刺检测。结果表明，较小的、顺序微调的 DistilBERT 模型使用最少的 LLM 生成的用于微调的代码混合数据，实现了 84% 的最高总体准确度，在零次和少次设置中优于所有 LLM。这些发现表明，在资源匮乏和数据稀缺的环境中，基于较小 Transformer 的模型的域自适应微调可以显着改善一般 LLM 推理的讽刺检测。

Title: MERRY: Semantically Decoupled Evaluation of Multimodal Emotional and Role Consistencies of Role-Playing Agents

• Authors: Zhenyu Wang, Xiaofen Xing, Yirong Chen, Xiangmin Xu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21941
• Pdf URL: https://arxiv.org/pdf/2602.21941
• Copy Paste: [[2602.21941]] MERRY: Semantically Decoupled Evaluation of Multimodal Emotional and Role Consistencies of Role-Playing Agents(https://arxiv.org/abs/2602.21941)
• Keywords: llm, prompt, agent
• Abstract: Multimodal Role-Playing Agents (MRPAs) are attracting increasing attention due to their ability to deliver more immersive multimodal emotional interactions. However, existing studies still rely on pure textual benchmarks to evaluate the text responses of MRPAs, while delegating the assessment of their multimodal expressions solely to modality-synthesis metrics. This evaluation paradigm, on the one hand, entangles semantic assessment with modality generation, leading to ambiguous error attribution, and on the other hand remains constrained by the heavy reliance on human judgment. To this end, we propose MERRY, a semantically decoupled evaluation framework for assessing Multimodal Emotional and Role consistencies of Role-playing agents. This framework introduce five refined metrics for EC and three for RC. Notably, we transform the traditional subjective scoring approach into a novel bidirectional-evidence-finding task, significantly improving the human agreement of LLM-as-Judge evaluations. Based on MERRY, we conduct extensive evaluations. Our empirical results primarily reveal that: (1) Training on synthetic datasets tends to reduce emotional consistency, whereas training on real-world datasets improves it; (2) Existing models suffer from emotional templatization and simplification, exhibiting positive-bias and performance bottleneck in fine-grained negative emotions; (3) Simple prompting method strengthens the weak models but constrains the strong ones, while simple fine-tuning method suffers from poor role generalization. Codes and dataset are available.
• 摘要：多模态角色扮演代理 (MRPA) 因其能够提供更加身临其境的多模态情感交互而受到越来越多的关注。然而，现有的研究仍然依赖于纯粹的文本基准来评估 MRPA 的文本响应，同时将其多模态表达的评估仅委托给模态合成指标。这种评估范式一方面将语义评估与模态生成纠缠在一起，导致错误归因不明确，另一方面仍然受到对人类判断的严重依赖的限制。为此，我们提出了 MERRY，一种语义解耦的评估框架，用于评估角色扮演代理的多模态情感和角色一致性。该框架为 EC 引入了五个细化指标，为 RC 引入了三个细化指标。值得注意的是，我们将传统的主观评分方法转变为一种新颖的双向证据查找任务，显着提高了法学硕士作为法官评估的人类一致性。在MERRY的基础上，我们进行了广泛的评估。我们的实证结果主要表明：（1）对合成数据集的训练往往会降低情感一致性，而对真实世界数据集的训练则可以改善情感一致性； (2) 现有模型存在情感模板化和简单化问题，在细粒度的负面情绪中表现出正向偏差和性能瓶颈； （3）简单的提示方法强化了弱模型，但限制了强模型，而简单的微调方法则角色泛化能力较差。代码和数据集可用。

Title: Large Language Models are Algorithmically Blind

• Authors: Sohan Venkatesh, Ashish Mahendran Kurapath, Tejas Melkote
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21947
• Pdf URL: https://arxiv.org/pdf/2602.21947
• Copy Paste: [[2602.21947]] Large Language Models are Algorithmically Blind(https://arxiv.org/abs/2602.21947)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) demonstrate remarkable breadth of knowledge, yet their ability to reason about computational processes remains poorly understood. Closing this gap matters for practitioners who rely on LLMs to guide algorithm selection and deployment. We address this limitation using causal discovery as a testbed and evaluate eight frontier LLMs against ground truth derived from large-scale algorithm executions and find systematic, near-total failure. Models produce ranges far wider than true confidence intervals yet still fail to contain the true algorithmic mean in the majority of instances; most perform worse than random guessing and the marginal above-random performance of the best model is most consistent with benchmark memorization rather than principled reasoning. We term this failure algorithmic blindness and argue it reflects a fundamental gap between declarative knowledge about algorithms and calibrated procedural prediction.
• 摘要：大型语言模型（LLM）展示了惊人的知识广度，但人们对它们推理计算过程的能力仍然知之甚少。对于依靠法学硕士来指导算法选择和部署的从业者来说，缩小这一差距很重要。我们使用因果发现作为测试平台来解决这一限制，并根据大规模算法执行得出的基本事实评估八个前沿法学硕士，并发现系统性的、接近完全的失败。模型产生的范围远大于真实的置信区间，但在大多数情况下仍然无法包含真实的算法平均值；大多数模型的表现比随机猜测更差，并且最佳模型的边缘高于随机性能与基准记忆最一致，而不是与原则推理一致。我们将这种失败称为算法盲目性，并认为它反映了有关算法的声明性知识与校准的程序预测之间的根本差距。

Title: MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models

• Authors: Boqi Chen, Xudong Liu, Jiachuan Peng, Marianne Frey-Marti, Bang Zheng, Kyle Lam, Lin Li, Jianing Qiu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21950
• Pdf URL: https://arxiv.org/pdf/2602.21950
• Copy Paste: [[2602.21950]] MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models(https://arxiv.org/abs/2602.21950)
• Keywords: language model, llm
• Abstract: Multimodal large language models (MLLMs) have shown great potential in medical applications, yet existing benchmarks inadequately capture real-world clinical complexity. We introduce MEDSYN, a multilingual, multimodal benchmark of highly complex clinical cases with up to 7 distinct visual clinical evidence (CE) types per case. Mirroring clinical workflow, we evaluate 18 MLLMs on differential diagnosis (DDx) generation and final diagnosis (FDx) selection. While top models often match or even outperform human experts on DDx generation, all MLLMs exhibit a much larger DDx--FDx performance gap compared to expert clinicians, indicating a failure mode in synthesis of heterogeneous CE types. Ablations attribute this failure to (i) overreliance on less discriminative textual CE ($\it{e.g.}$, medical history) and (ii) a cross-modal CE utilization gap. We introduce Evidence Sensitivity to quantify the latter and show that a smaller gap correlates with higher diagnostic accuracy. Finally, we demonstrate how it can be used to guide interventions to improve model performance. We will open-source our benchmark and code.
• 摘要：多模态大语言模型（MLLM）在医学应用中显示出巨大的潜力，但现有的基准不足以捕捉现实世界的临床复杂性。我们推出 MEDSYN，这是一种针对高度复杂的临床病例的多语言、多模式基准，每个病例最多有 7 种不同的视觉临床证据 (CE) 类型。镜像临床工作流程，我们评估了 18 个 MLLM 的鉴别诊断 (DDx) 生成和最终诊断 (FDx) 选择。虽然顶级模型在 DDx 生成方面通常与人类专家相匹配甚至优于人类专家，但与临床专家相比，所有 MLLM 都表现出更大的 DDx-FDx 性能差距，这表明异构 CE 类型的合成存在失败模式。 Ablations 将这种失败归因于 (i) 过度依赖歧视性较小的文本 CE（$\it{例如}$，病史）和 (ii) 跨模式 CE 利用差距。我们引入证据敏感性来量化后者，并表明较小的差距与较高的诊断准确性相关。最后，我们演示了如何使用它来指导干预措施以提高模型性能。我们将开源我们的基准测试和代码。

Title: RADAR: Reasoning as Discrimination with Aligned Representations for LLM-based Knowledge Graph Reasoning

• Authors: Bo Xue, Yuan Jin, Luoyi Fu, Jiaxin Ding, Xinbing Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21951
• Pdf URL: https://arxiv.org/pdf/2602.21951
• Copy Paste: [[2602.21951]] RADAR: Reasoning as Discrimination with Aligned Representations for LLM-based Knowledge Graph Reasoning(https://arxiv.org/abs/2602.21951)
• Keywords: language model, llm, hallucination
• Abstract: Knowledge graph reasoning (KGR) infers missing facts, with recent advances increasingly harnessing the semantic priors and reasoning abilities of Large Language Models (LLMs). However, prevailing generative paradigms are prone to memorizing surface-level co-occurrences rather than learning genuine relational semantics, limiting out-of-distribution generalization. To address this, we propose RADAR, which reformulates KGR from generative pattern matching to discriminative relational reasoning. We recast KGR as discriminative entity selection, where reinforcement learning enforces relative entity separability beyond token-likelihood imitation. Leveraging this separability, inference operates directly in representation space, ensuring consistency with the discriminative optimization and bypassing generation-induced hallucinations. Across four benchmarks, RADAR achieves 5-6% relative gains on link prediction and triple classification over strong LLM baselines, while increasing task-relevant mutual information in intermediate representations by 62.9%, indicating more robust and transferable relational reasoning.
• 摘要：知识图推理（KGR）推断缺失的事实，最近的进展越来越多地利用大型语言模型（LLM）的语义先验和推理能力。然而，流行的生成范式倾向于记住表面层的共现，而不是学习真正的关系语义，从而限制了分布外泛化。为了解决这个问题，我们提出了 RADAR，它将 KGR 从生成模式匹配重新表述为判别关系推理。我们将 KGR 重新定义为判别性实体选择，其中强化学习强制执行超越标记似然模仿的相对实体可分离性。利用这种可分离性，推理直接在表示空间中运行，确保与判别性优化的一致性并绕过生成引起的幻觉。在四个基准测试中，RADAR 在链接预测和三重分类方面比强大的 LLM 基线实现了 5-6% 的相对增益，同时将中间表示中与任务相关的互信息增加了 62.9%，这表明关系推理更加稳健和可转移。

Title: CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models

• Authors: Miyu Oba, Saku Sugawara
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.21978
• Pdf URL: https://arxiv.org/pdf/2602.21978
• Copy Paste: [[2602.21978]] CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models(https://arxiv.org/abs/2602.21978)
• Keywords: language model, llm
• Abstract: Recent work has examined language models from a linguistic perspective to better understand how they acquire language. Most existing benchmarks focus on judging grammatical acceptability, whereas the ability to interpret meanings conveyed by grammatical forms has received much less attention. We introduce the Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models (CxMP), a benchmark grounded in Construction Grammar that treats form-meaning pairings, or constructions, as fundamental linguistic units. CxMP evaluates whether models can interpret the semantic relations implied by constructions, using a controlled minimal-pair design across nine construction types, including the let-alone, caused motion, and ditransitive constructions. Our results show that while syntactic competence emerges early, constructional understanding develops more gradually and remains limited even in large language models (LLMs). CxMP thus reveals persistent gaps in how language models integrate form and meaning, providing a framework for studying constructional understanding and learning trajectories in language models.
• 摘要：最近的工作从语言学角度研究了语言模型，以更好地理解它们如何获取语言。大多数现有基准侧重于判断语法可接受性，而解释语法形式所传达的含义的能力却受到较少的关注。我们引入了用于评估语言模型中的构造理解的语言最小对基准（CxMP），这是一个基于构造语法的基准，它将形式-含义配对或构造视为基本语言单元。 CxMP 使用跨九种构造类型（包括 let-alone、引起运动和双及物构造）的受控最小对设计来评估模型是否可以解释构造所隐含的语义关系。我们的研究结果表明，虽然句法能力出现得很早，但结构理解的发展却更加缓慢，即使在大型语言模型（LLM）中也仍然有限。因此，CxMP 揭示了语言模型如何整合形式和意义方面持续存在的差距，为研究语言模型中的结构理解和学习轨迹提供了一个框架。

Title: Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models

• Authors: Christian Nickel, Laura Schrewe, Florian Mai, Lucie Flek
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.22072
• Pdf URL: https://arxiv.org/pdf/2602.22072
• Copy Paste: [[2602.22072]] Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models(https://arxiv.org/abs/2602.22072)
• Keywords: language model, llm, prompt, chain-of-thought, agent
• Abstract: Theory of Mind (ToM) refers to an agent's ability to model the internal states of others. Contributing to the debate whether large language models (LLMs) exhibit genuine ToM capabilities, our study investigates their ToM robustness using perturbations on false-belief tasks and examines the potential of Chain-of-Thought prompting (CoT) to enhance performance and explain the LLM's decision. We introduce a handcrafted, richly annotated ToM dataset, including classic and perturbed false belief tasks, the corresponding spaces of valid reasoning chains for correct task completion, subsequent reasoning faithfulness, task solutions, and propose metrics to evaluate reasoning chain correctness and to what extent final answers are faithful to reasoning traces of the generated CoT. We show a steep drop in ToM capabilities under task perturbation for all evaluated LLMs, questioning the notion of any robust form of ToM being present. While CoT prompting improves the ToM performance overall in a faithful manner, it surprisingly degrades accuracy for some perturbation classes, indicating that selective application is necessary.
• 摘要：心智理论 (ToM) 是指主体模拟他人内部状态的能力。针对大型语言模型 (LLM) 是否表现出真正的 ToM 功能的争论，我们的研究利用错误信念任务的扰动来调查其 ToM 稳健性，并检验了思想链提示 (CoT) 在提高性能和解释 LLM 决策方面的潜力。我们引入了一个手工制作的、注释丰富的 ToM 数据集，包括经典的和扰动的错误信念任务、正确完成任务的有效推理链的相应空间、后续推理的可信度、任务解决方案，并提出了评估推理链正确性的指标以及最终答案在多大程度上忠实于生成的 CoT 的推理痕迹。我们发现所有评估的法学硕士在任务扰动下的 ToM 能力急剧下降，对存在任何稳健形式的 ToM 的概念提出了质疑。虽然 CoT 提示以忠实的方式提高了 ToM 的整体性能，但它令人惊讶地降低了某些扰动类别的准确性，这表明选择性应用是必要的。

Title: Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference

• Authors: Bo-Wei Chen, Chung-Chi Chen, An-Zi Yen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22090
• Pdf URL: https://arxiv.org/pdf/2602.22090
• Copy Paste: [[2602.22090]] Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference(https://arxiv.org/abs/2602.22090)
• Keywords: language model, gpt, llm
• Abstract: Large Language Models (LLMs) have revolutionized inference across diverse natural language tasks, with larger models performing better but at higher computational costs. We propose a confidence-driven strategy that dynamically selects the most suitable model based on confidence estimates. By assessing a model's confidence in handling the task and response accuracy, tasks that are likely to be solved correctly are retained, while more uncertain or complex cases are delegated to a larger model, ensuring reliability while minimizing computation. Specifically, we evaluate a model's likelihood of knowing the correct answer and the probability that its response is accurate. Experiments on the Massive Multitask Language Understanding (MMLU) benchmark show that our approach achieves accuracy comparable to the largest model while reducing computational costs by 20\% to 40\%. When applied to GPT-4o API calls, it reduces token usage by approximately 60\%, further improving cost efficiency. These findings indicate the potential of confidence-based model selection to enhance real-world LLM deployment, particularly in resource-constrained settings such as edge devices and commercial API applications.
• 摘要：大型语言模型 (LLM) 彻底改变了各种自然语言任务的推理，较大的模型性能更好，但计算成本更高。我们提出了一种信心驱动的策略，根据信心估计动态选择最合适的模型。通过评估模型处理任务的置信度和响应准确性，保留可能正确解决的任务，而将更多不确定或复杂的情况委托给更大的模型，在确保可靠性的同时最大限度地减少计算量。具体来说，我们评估模型知道正确答案的可能性及其响应准确的概率。大规模多任务语言理解 (MMLU) 基准测试表明，我们的方法实现了与最大模型相当的精度，同时将计算成本降低了 20% 至 40%。当应用于 GPT-4o API 调用时，它可以减少大约 60\% 的令牌使用量，进一步提高成本效率。这些发现表明基于置信度的模型选择具有增强现实世界 LLM 部署的潜力，特别是在资源受限的环境中，例如边缘设备和商业 API 应用程序。

Title: IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languages

• Authors: Thanmay Jayakumar, Mohammed Safi Ur Rahman Khan, Raj Dabre, Ratish Puduppully, Anoop Kunchukuttan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22125
• Pdf URL: https://arxiv.org/pdf/2602.22125
• Copy Paste: [[2602.22125]] IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languages(https://arxiv.org/abs/2602.22125)
• Keywords: llm, prompt
• Abstract: Instruction-following benchmarks remain predominantly English-centric, leaving a critical evaluation gap for the hundreds of millions of Indic language speakers. We introduce IndicIFEval, a benchmark evaluating constrained generation of LLMs across 14 Indic languages using automatically verifiable, rule-based instructions. It comprises around 800 human-verified examples per language spread across two complementary subsets: IndicIFEval-Ground, translated prompts from IFEval (Zhou et al., 2023) carefully localized for Indic contexts, and IndicIFEval-Ground, synthetically generated instructions grounded in native Indic content. We conduct a comprehensive evaluation of major open-weight and proprietary models spanning both reasoning and non-reasoning models. While models maintain strong adherence to formatting constraints, they struggle significantly with lexical and cross-lingual tasks -- and despite progress in high-resource languages, instruction-following across the broader Indic family lags significantly behind English. We release IndicIFEval and its evaluation scripts to support progress on multilingual constrained generation (this http URL).
• 摘要：指令遵循基准仍然主要以英语为中心，这给数亿印度语使用者留下了严重的评估差距。我们推出了 IndicIFEval，这是一个使用可自动验证、基于规则的指令来评估跨 14 种印度语言的 LLM 受限生成的基准。它包含每种语言约 800 个经过人工验证的示例，分布在两个互补的子集中：IndicIFEval-Ground，来自 IFEval（Zhou 等人，2023）针对印度语上下文精心本地化的翻译提示，以及 IndicIFEval-Ground，基于本地印度语内容综合生成的指令。我们对主要的开放权重和专有模型进行了全面评估，涵盖推理和非推理模型。虽然模型始终严格遵守格式限制，但它们在词汇和跨语言任务方面却遇到了很大的困难——尽管在高资源语言方面取得了进展，但更广泛的印度语家族的指令遵循明显落后于英语。我们发布了 IndicIFEval 及其评估脚本，以支持多语言约束生成（此 http URL）的进展。

Title: Dynamic Personality Adaptation in Large Language Models via State Machines

• Authors: Leon Pielage, Ole Hätscher, Mitja Back, Bernhard Marschall, Benjamin Risse
• Subjects: cs.CL, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.22157
• Pdf URL: https://arxiv.org/pdf/2602.22157
• Copy Paste: [[2602.22157]] Dynamic Personality Adaptation in Large Language Models via State Machines(https://arxiv.org/abs/2602.22157)
• Keywords: language model, llm, prompt
• Abstract: The inability of Large Language Models (LLMs) to modulate their personality expression in response to evolving dialogue dynamics hinders their performance in complex, interactive contexts. We propose a model-agnostic framework for dynamic personality simulation that employs state machines to represent latent personality states, where transition probabilities are dynamically adapted to the conversational context. Part of our architecture is a modular pipeline for continuous personality scoring that evaluates dialogues along latent axes while remaining agnostic to the specific personality models, their dimensions, transition mechanisms, or LLMs used. These scores function as dynamic state variables that systematically reconfigure the system prompt, steering behavioral alignment throughout the this http URL evaluate this framework by operationalizing the Interpersonal Circumplex (IPC) in a medical education setting. Results demonstrate that the system successfully adapts its personality state to user inputs, but also influences user behavior, thereby facilitating de-escalation training. Notably, the scoring pipeline maintains comparable precision even when utilizing lightweight, fine-tuned classifiers instead of large-scale LLMs. This work demonstrates the feasibility of modular, personality-adaptive architectures for education, customer support, and broader human-computer interaction.
• 摘要：大型语言模型（LLM）无法根据不断变化的对话动态来调节其个性表达，这阻碍了它们在复杂的交互式环境中的表现。我们提出了一种与模型无关的动态人格模拟框架，该框架使用状态机来表示潜在的人格状态，其中转换概率动态地适应对话上下文。我们架构的一部分是用于连续个性评分的模块化管道，它沿着潜在轴评估对话，同时保持对特定个性模型、其维度、转换机制或所使用的法学硕士的不可知性。这些分数充当动态状态变量，系统地重新配置系统提示，引导整个 http URL 的行为对齐，通过在医学教育环境中操作人际循环 (IPC) 来评估该框架。结果表明，系统成功地将其个性状态适应用户输入，但也影响用户行为，从而促进降级培训。值得注意的是，即使使用轻量级、微调的分类器而不是大规模的法学硕士，评分管道也能保持相当的精度。这项工作展示了模块化、个性自适应架构在教育、客户支持和更广泛的人机交互方面的可行性。

Title: DySCO: Dynamic Attention-Scaling Decoding for Long-Context LMs

• Authors: Xi Ye, Wuwei Zhang, Fangcong Yin, Howard Yen, Danqi Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22175
• Pdf URL: https://arxiv.org/pdf/2602.22175
• Copy Paste: [[2602.22175]] DySCO: Dynamic Attention-Scaling Decoding for Long-Context LMs(https://arxiv.org/abs/2602.22175)
• Keywords: language model, long context
• Abstract: Understanding and reasoning over long contexts is a crucial capability for language models (LMs). Although recent models support increasingly long context windows, their accuracy often deteriorates as input length grows. In practice, models often struggle to keep attention aligned with the most relevant context throughout decoding. In this work, we propose DySCO, a novel decoding algorithm for improving long-context reasoning. DySCO leverages retrieval heads--a subset of attention heads specialized for long-context retrieval--to identify task-relevant tokens at each decoding step and explicitly up-weight them. By doing so, DySCO dynamically adjusts attention during generation to better utilize relevant context. The method is training-free and can be applied directly to any off-the-shelf LMs. Across multiple instruction-tuned and reasoning models, DySCO consistently improves performance on challenging long-context reasoning benchmarks, yielding relative gains of up to 25% on MRCR and LongBenchV2 at 128K context length with modest additional compute. Further analysis highlights the importance of both dynamic attention rescaling and retrieval-head-guided selection for the effectiveness of the method, while providing interpretability insights into decoding-time attention behavior. Our code is available at this https URL.
• 摘要：对长上下文的理解和推理是语言模型 (LM) 的一项关键能力。尽管最近的模型支持越来越长的上下文窗口，但它们的准确性通常会随着输入长度的增长而下降。在实践中，模型通常很难在整个解码过程中保持注意力与最相关的上下文保持一致。在这项工作中，我们提出了 DySCO，一种用于改进长上下文推理的新颖解码算法。 DySCO 利用检索头（专门用于长上下文检索的注意力头的子集）来识别每个解码步骤中与任务相关的标记，并显式地增加它们的权重。通过这样做，DySCO 在生成过程中动态调整注意力，以更好地利用相关上下文。该方法无需训练，可以直接应用于任何现成的语言模型。在多个指令调整和推理模型中，DySCO 持续提高了具有挑战性的长上下文推理基准的性能，在 128K 上下文长度和适度的额外计算下，MRCR 和 LongBenchV2 的相对增益高达 25%。进一步的分析强调了动态注意力重新调整和检索头引导选择对于该方法有效性的重要性，同时提供了对解码时注意力行为的可解释性见解。我们的代码可以在这个 https URL 上找到。

Title: Improving Parametric Knowledge Access in Reasoning Language Models

• Authors: Melody Ma, John Hewitt
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22193
• Pdf URL: https://arxiv.org/pdf/2602.22193
• Copy Paste: [[2602.22193]] Improving Parametric Knowledge Access in Reasoning Language Models(https://arxiv.org/abs/2602.22193)
• Keywords: language model
• Abstract: We study reasoning for accessing world knowledge stored in a language model's parameters. For example, recalling that Canberra is Australia's capital may benefit from thinking through major cities and the concept of purpose-built capitals. While reasoning language models are trained via reinforcement learning to produce reasoning traces on tasks such as mathematics, they may not reason well for accessing their own world knowledge. We first find that models do not generate their best world knowledge reasoning by default: adding a simple "think step-by-step" cue demonstrates statistically significant improvement in knowledge recall but not math. Motivated by this, we propose training models to reason over their parametric knowledge using world-knowledge question answering as a verifiable reward. After reinforcement learning on TriviaQA (+9.9%), performance also improves on Natural Questions, HotpotQA, SimpleQA, and StrategyQA by 4.2%, 2.1%, 0.6%, and 3.0%, respectively. Reasoning models are under-optimized for parametric knowledge access, but can be easily trained to reason better.
• 摘要：我们研究访问存储在语言模型参数中的世界知识的推理。例如，回顾堪培拉是澳大利亚的首都可能会受益于对主要城市和专门建设首都的概念的思考。虽然推理语言模型通过强化学习进行训练，以产生数学等任务的推理痕迹，但它们可能无法很好地推理以获取自己的世界知识。我们首先发现模型默认情况下不会生成最好的世界知识推理：添加一个简单的“一步一步思考”提示表明在知识回忆方面有统计学上的显着改善，但在数学方面却没有。受此启发，我们提出了训练模型，使用世界知识问答作为可验证的奖励来推理其参数知识。在 TriviaQA 上进行强化学习 (+9.9%) 后，Natural Questions、HotpotQA、SimpleQA 和 StrategyQA 的性能也分别提高了 4.2%、2.1%、0.6% 和 3.0%。推理模型对于参数知识访问的优化不足，但可以轻松训练以更好地推理。

Title: SumTablets: A Transliteration Dataset of Sumerian Tablets

• Authors: Cole Simmons, Richard Diehl Martinez, Dan Jurafsky
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.22200
• Pdf URL: https://arxiv.org/pdf/2602.22200
• Copy Paste: [[2602.22200]] SumTablets: A Transliteration Dataset of Sumerian Tablets(https://arxiv.org/abs/2602.22200)
• Keywords: language model
• Abstract: Sumerian transliteration is a conventional system for representing a scholar's interpretation of a tablet in the Latin script. Thanks to visionary digital Assyriology projects such as ETCSL, CDLI, and Oracc, a large number of Sumerian transliterations have been published online, and these data are well-structured for a variety of search and analysis tasks. However, the absence of a comprehensive, accessible dataset pairing transliterations with a digital representation of the tablet's cuneiform glyphs has prevented the application of modern Natural Language Processing (NLP) methods to the task of Sumerian transliteration. To address this gap, we present SumTablets, a dataset pairing Unicode representations of 91,606 Sumerian cuneiform tablets (totaling 6,970,407 glyphs) with the associated transliterations published by Oracc. We construct SumTablets by first preprocessing and standardizing the Oracc transliterations before mapping each reading back to the Unicode representation of the source glyph. Further, we retain parallel structural information (e.g., surfaces, newlines, broken segments) through the use of special tokens. We release SumTablets as a Hugging Face Dataset (CC BY 4.0) and open source data preparation code via GitHub. Additionally, we leverage SumTablets to implement and evaluate two transliteration baselines: (1) weighted sampling from a glyph's possible readings, and (2) fine-tuning an autoregressive language model. Our fine-tuned language model achieves an average transliteration character-level F-score (chrF) of 97.55, demonstrating the immediate potential of transformer-based transliteration models in allowing experts to rapidly verify generated transliterations rather than manually transliterating tablets one-by-one.
• 摘要：苏美尔音译是一种传统系统，用于代表学者对拉丁文字板的解释。得益于 ETCSL、CDLI 和 Oracc 等富有远见的数字亚述学项目，大量苏美尔音译已在网上发布，这些数据结构良好，适合各种搜索和分析任务。然而，由于缺乏将音译与平板电脑楔形文字的数字表示配对的全面、可访问的数据集，阻碍了现代自然语言处理（NLP）方法在苏美尔音译任务中的应用。为了解决这一差距，我们提出了 SumTablets，这是一个数据集，将 91,606 个苏美尔楔形文字板（总共 6,970,407 个字形）的 Unicode 表示与 Oracc 发布的相关音译配对。我们通过首先预处理和标准化 Oracc 音译来构建 SumTablets，然后将每个读数映射回源字形的 Unicode 表示形式。此外，我们通过使用特殊标记保留并行结构信息（例如，表面、换行符、断段）。我们将 SumTablets 作为 Hugging Face 数据集 (CC BY 4.0) 发布，并通过 GitHub 开源数据准备代码。此外，我们利用 SumTablets 来实现和评估两个音译基线：(1) 从字形的可能读数中进行加权采样，以及 (2) 微调自回归语言模型。我们经过微调的语言模型实现了 97.55 的平均音译字符级 F 分数 (chrF)，这证明了基于 Transformer 的音译模型的直接潜力，使专家能够快速验证生成的音译，而不是逐一手动音译平板电脑。

Title: Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets

• Authors: Hanna Yukhymenko, Anton Alexandrov, Martin Vechev
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.22207
• Pdf URL: https://arxiv.org/pdf/2602.22207
• Copy Paste: [[2602.22207]] Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets(https://arxiv.org/abs/2602.22207)
• Keywords: language model, llm
• Abstract: The reliability of multilingual Large Language Model (LLM) evaluation is currently compromised by the inconsistent quality of translated benchmarks. Existing resources often suffer from semantic drift and context loss, which can lead to misleading performance metrics. In this work, we present a fully automated framework designed to address these challenges by enabling scalable, high-quality translation of datasets and benchmarks. We demonstrate that adapting test-time compute scaling strategies, specifically Universal Self-Improvement (USI) and our proposed multi-round ranking method, T-RANK, allows for significantly higher quality outputs compared to traditional pipelines. Our framework ensures that benchmarks preserve their original task structure and linguistic nuances during localization. We apply this approach to translate popular benchmarks and datasets into eight Eastern and Southern European languages (Ukrainian, Bulgarian, Slovak, Romanian, Lithuanian, Estonian, Turkish, Greek). Evaluations using both reference-based metrics and LLM-as-a-judge show that our translations surpass existing resources, resulting in more accurate downstream model assessment. We release both the framework and the improved benchmarks to facilitate robust and reproducible multilingual AI development.
• 摘要：目前，多语言大语言模型 (LLM) 评估的可靠性因翻译基准质量不一致而受到影响。现有资源经常遭受语义漂移和上下文丢失的影响，这可能导致误导性的性能指标。在这项工作中，我们提出了一个完全自动化的框架，旨在通过实现数据集和基准的可扩展、高质量转换来应对这些挑战。我们证明，与传统管道相比，调整测试时计算扩展策略，特别是通用自我改进（USI）和我们提出的多轮排名方法 T-RANK，可以显着提高质量输出。我们的框架确保基准测试在本地化过程中保留其原始任务结构和语言细微差别。我们应用这种方法将流行的基准和数据集翻译成八种东欧和南欧语言（乌克兰语、保加利亚语、斯洛伐克语、罗马尼亚语、立陶宛语、爱沙尼亚语、土耳其语、希腊语）。使用基于参考的指标和法学硕士作为法官进行的评估表明，我们的翻译超越了现有资源，从而实现了更准确的下游模型评估。我们发布了框架和改进的基准，以促进稳健且可重复的多语言人工智能开发。