2026-01-06

Title: The Qualitative Laboratory: Theory Prototyping and Hypothesis Generation with Large Language Models

• Authors: Hugues Draelants
• Subjects: cs.CL, cs.AI, cs.CY, cs.MA
• Abstract URL: https://arxiv.org/abs/2601.00797
• Pdf URL: https://arxiv.org/pdf/2601.00797
• Copy Paste: [[2601.00797]] The Qualitative Laboratory: Theory Prototyping and Hypothesis Generation with Large Language Models(https://arxiv.org/abs/2601.00797)
• Keywords: language model, llm, agent
• Abstract: A central challenge in social science is to generate rich qualitative hypotheses about how diverse social groups might interpret new information. This article introduces and illustrates a novel methodological approach for this purpose: sociological persona simulation using Large Language Models (LLMs), which we frame as a "qualitative laboratory". We argue that for this specific task, persona simulation offers a distinct advantage over established methods. By generating naturalistic discourse, it overcomes the lack of discursive depth common in vignette surveys, and by operationalizing complex worldviews through natural language, it bypasses the formalization bottleneck of rule-based agent-based models (ABMs). To demonstrate this potential, we present a protocol where personas derived from a sociological theory of climate reception react to policy messages. The simulation produced nuanced and counter-intuitive hypotheses - such as a conservative persona's rejection of a national security frame - that challenge theoretical assumptions. We conclude that this method, used as part of a "simulation then validation" workflow, represents a superior tool for generating deeply textured hypotheses for subsequent empirical testing.
• 摘要：社会科学的一个核心挑战是就不同的社会群体如何解释新信息产生丰富的定性假设。本文介绍并说明了一种用于此目的的新颖方法：使用大型语言模型（LLM）进行社会学角色模拟，我们将其称为“定性实验室”。我们认为，对于这个特定任务，角色模拟比现有方法具有明显的优势。通过生成自然主义话语，它克服了小插图调查中常见的缺乏话语深度的问题，并通过自然语言操作复杂的世界观，它绕过了基于规则的基于代理的模型（ABM）的形式化瓶颈。为了证明这种潜力，我们提出了一个协议，其中源自气候接收社会学理论的角色对政策信息做出反应。模拟产生了微妙且反直觉的假设——例如保守派人物对国家安全框架的拒绝——挑战了理论假设。我们得出的结论是，这种方法用作“模拟然后验证”工作流程的一部分，代表了一种为后续实证测试生成深层假设的优越工具。

Title: Rate-Distortion Analysis of Compressed Query Delegation with Low-Rank Riemannian Updates

• Authors: Faruk Alpay, Bugra Kilictas
• Subjects: cs.CL, math.OC
• Abstract URL: https://arxiv.org/abs/2601.00938
• Pdf URL: https://arxiv.org/pdf/2601.00938
• Copy Paste: [[2601.00938]] Rate-Distortion Analysis of Compressed Query Delegation with Low-Rank Riemannian Updates(https://arxiv.org/abs/2601.00938)
• Keywords: chain-of-thought, agent
• Abstract: Bounded-context agents fail when intermediate reasoning exceeds an effective working-memory budget. We study compressed query delegation (CQD): (i) compress a high-dimensional latent reasoning state into a low-rank tensor query, (ii) delegate the minimal query to an external oracle, and (iii) update the latent state via Riemannian optimization on fixed-rank manifolds. We give a math-first formulation: CQD is a constrained stochastic program with a query-budget functional and an oracle modeled as a noisy operator. We connect CQD to classical rate-distortion and information bottleneck principles, showing that spectral hard-thresholding is optimal for a natural constrained quadratic distortion problem, and we derive convergence guarantees for Riemannian stochastic approximation under bounded oracle noise and smoothness assumptions. Empirically, we report (A) a 2,500-item bounded-context reasoning suite (BBH-derived tasks plus curated paradox instances) comparing CQD against chain-of-thought baselines under fixed compute and context; and (B) a human "cognitive mirror" benchmark (N=200) measuring epistemic gain and semantic drift across modern oracles.
• 摘要：当中间推理超出有效的工作记忆预算时，有界上下文代理就会失败。我们研究压缩查询委托（CQD）：（i）将高维潜在推理状态压缩为低秩张量查询，（ii）将最小查询委托给外部预言机，以及（iii）通过固定秩流形上的黎曼优化来更新潜在状态。我们给出一个数学优先的公式：CQD 是一个约束随机程序，具有查询预算函数和建模为噪声运算符的预言机。我们将 CQD 与经典的速率失真和信息瓶颈原理联系起来，表明谱硬阈值对于自然约束二次失真问题是最佳的，并且我们在有界预言噪声和平滑度假设下推导出黎曼随机逼近的收敛保证。根据经验，我们报告 (A) 一个包含 2,500 项的有界上下文推理套件（BBH 派生任务加上策划的悖论实例），将 CQD 与固定计算和上下文下的思想链基线进行比较； (B) 人类“认知镜子”基准 (N=200)，测量现代预言中的认知增益和语义漂移。

Title: Intention Collapse: Intention-Level Metrics for Reasoning in Language Models

• Authors: Patricio Vera
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01011
• Pdf URL: https://arxiv.org/pdf/2601.01011
• Copy Paste: [[2601.01011]] Intention Collapse: Intention-Level Metrics for Reasoning in Language Models(https://arxiv.org/abs/2601.01011)
• Keywords: language model
• Abstract: Every act of language generation compresses a rich internal state into a single token sequence. We call this process intention collapse: a many-to-one projection from a high dimensional intention space I into an external language space L. We formalize intention collapse for contemporary language models, define three simple, model agnostic intention metrics (intention entropy Hint, effective dimensionality dimeff, and latent knowledge recoverability Recov), and propose an empirical agenda for studying how inference time computation shapes internal intentions before they are verbalized. We also report a first small scale experiment. Using a 4 bit Mistral 7B model on 200 GSM8K problems, we compare a direct answer baseline, a chain of thought (CoT) regime, and a babble control. CoT raises accuracy from 5.5 percent to 53 percent, sharply reduces pre collapse intention entropy (from 1.42 to 0.37 bits), and shows higher global effective dimensionality than the other regimes despite producing fewer tokens than babble. At the same time, Hint has little item level predictive power, and a linear probe on I achieves AUROC 0.65 in the CoT regime but only about chance in the baseline regime, where it collapses to the majority class. These preliminary results indicate that intention level metrics can distinguish inference regimes and expose latent information that is partly lost during collapse, while also revealing important limitations of our current proxies
• 摘要：语言生成的每个行为都会将丰富的内部状态压缩为单个标记序列。我们将这个过程称为意图崩溃：从高维意图空间 I 到外部语言空间 L 的多对一投影。我们将当代语言模型的意图崩溃形式化，定义了三个简单的、与模型无关的意图度量（意图熵 Hint、有效维度 dimeff 和潜在知识可恢复性 Recov），并提出了一个实证议程，用于研究推理时间计算如何在语言化之前塑造内部意图。我们还报告了第一个小规模实验。我们使用 4 位 Mistral 7B 模型解决 200 个 GSM8K 问题，比较直接答案基线、思想链 (CoT) 体系和多语控制。 CoT 将准确度从 5.5% 提高到 53%，大幅降低了崩溃前的意图熵（从 1.42 位到 0.37 位），并且尽管产生的令牌比 babble 少，但仍显示出比其他机制更高的全局有效维度。同时，Hint 几乎没有项目级别的预测能力，并且 I 上的线性探针在 CoT 体系中实现了 AUROC 0.65，但在基线体系中仅具有大约机会，在基线体系中它会崩溃到多数类别。这些初步结果表明，意图水平指标可以区分推理机制并暴露崩溃期间部分丢失的潜在信息，同时也揭示了我们当前代理的重要局限性

Title: HyperJoin: LLM-augmented Hypergraph Link Prediction for Joinable Table Discovery

• Authors: Shiyuan Liu, Jianwei Wang, Xuemin Lin, Lu Qin, Wenjie Zhang, Ying Zhang
• Subjects: cs.CL, cs.DB
• Abstract URL: https://arxiv.org/abs/2601.01015
• Pdf URL: https://arxiv.org/pdf/2601.01015
• Copy Paste: [[2601.01015]] HyperJoin: LLM-augmented Hypergraph Link Prediction for Joinable Table Discovery(https://arxiv.org/abs/2601.01015)
• Keywords: language model, llm
• Abstract: As a pivotal task in data lake management, joinable table discovery has attracted widespread interest. While existing language model-based methods achieve remarkable performance by combining offline column representation learning with online ranking, their design insufficiently accounts for the underlying structural interactions: (1) offline, they directly model tables into isolated or pairwise columns, thereby struggling to capture the rich inter-table and intra-table structural information; and (2) online, they rank candidate columns based solely on query-candidate similarity, ignoring the mutual interactions among the candidates, leading to incoherent result sets. To address these limitations, we propose HyperJoin, a large language model (LLM)-augmented Hypergraph framework for Joinable table discovery. Specifically, we first construct a hypergraph to model tables using both the intra-table hyperedges and the LLM-augmented inter-table hyperedges. Consequently, the task of joinable table discovery is formulated as link prediction on this constructed hypergraph. We then design HIN, a Hierarchical Interaction Network that learns expressive column representations through bidirectional message passing over columns and hyperedges. To strengthen coherence and internal consistency in the result columns, we cast online ranking as a coherence-aware top-k column selection problem. We then introduce a reranking module that leverages a maximum spanning tree algorithm to prune noisy connections and maximize coherence. Experiments demonstrate the superiority of HyperJoin, achieving average improvements of 21.4% (Precision@15) and 17.2% (Recall@15) over the best baseline.
• 摘要：作为数据湖管理中的一项关键任务，可连接表的发现引起了广泛的兴趣。虽然现有的基于语言模型的方法通过将离线列表示学习与在线排序相结合而取得了显着的性能，但它们的设计不足以考虑底层的结构交互：（1）离线时，它们直接将表建模为孤立或成对的列，从而难以捕获丰富的表间和表内结构信息； （2）在网上，他们仅根据查询与候选的相似性对候选列进行排名，忽略了候选之间的相互交互，导致结果集不连贯。为了解决这些限制，我们提出了 HyperJoin，这是一种用于可连接表发现的大型语言模型 (LLM) 增强的超图框架。具体来说，我们首先使用表内超边和 LLM 增强的表间超边构建一个超图来对表进行建模。因此，可连接表发现的任务被表述为在此构造的超图上的链接预测。然后，我们设计了 HIN，一个分层交互网络，它通过在列和超边上传递的双向消息来学习表达列表示。为了加强结果列的连贯性和内部一致性，我们将在线排名视为连贯性感知的 top-k 列选择问题。然后，我们引入一个重新排序模块，该模块利用最大生成树算法来修剪噪声连接并最大化一致性。实验证明了 HyperJoin 的优越性，与最佳基线相比，平均提高了 21.4% (Precision@15) 和 17.2% (Recall@15)。

Title: Multi-Dimensional Prompt Chaining to Improve Open-Domain Dialogue Generation

• Authors: Livia Leong Hui Teng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01037
• Pdf URL: https://arxiv.org/pdf/2601.01037
• Copy Paste: [[2601.01037]] Multi-Dimensional Prompt Chaining to Improve Open-Domain Dialogue Generation(https://arxiv.org/abs/2601.01037)
• Keywords: language model, gpt, prompt
• Abstract: Small language models (SLMs) offer significant deployment advantages but often struggle to match the dialogue quality of larger models in open-domain settings. In this paper, we propose a multi-dimensional prompt-chaining framework that integrates Naturalness, Coherence, and Engagingness dimensions to enhance human-likeness in open-domain dialogue generation. We apply the framework to two SLMs, TinyLlama and Llama-2-7B, and benchmark their performance against responses generated by substantially larger models, including Llama-2-70B and GPT-3.5 Turbo. We then employ automatic and human evaluation to assess the responses based on diversity, contextual coherence, as well as overall quality. Results show that the full framework improves response diversity by up to 29%, contextual coherence by up to 28%, and engagingness as well as naturalness by up to 29%. Notably, Llama-2-7B achieves performance comparable to substantially larger models, including Llama-2-70B and GPT-3.5 Turbo. Overall, the findings demonstrate that carefully designed prompt-based strategies provide an effective and resource-efficient pathway to improving open-domain dialogue quality in SLMs.
• 摘要：小语言模型 (SLM) 具有显着的部署优势，但在开放域设置中通常很难与大型模型的对话质量相匹配。在本文中，我们提出了一个多维提示链框架，该框架集成了自然性、连贯性和参与性维度，以增强开放域对话生成中的人类相似度。我们将该框架应用于两个 SLM（TinyLlama 和 Llama-2-7B），并根据更大的模型（包括 Llama-2-70B 和 GPT-3.5 Turbo）生成的响应对它们的性能进行基准测试。然后，我们采用自动和人工评估来评估基于多样性、上下文连贯性以及整体质量的响应。结果显示，完整的框架将响应多样性提高了 29%，上下文连贯性提高了 28%，参与度和自然度提高了 29%。值得注意的是，Llama-2-7B 的性能可与更大的型号（包括 Llama-2-70B 和 GPT-3.5 Turbo）相媲美。总体而言，研究结果表明，精心设计的基于提示的策略为提高 SLM 中的开放域对话质量提供了有效且资源高效的途径。

Title: KV-Embedding: Training-free Text Embedding via Internal KV Re-routing in Decoder-only LLMs

• Authors: Yixuan Tang, Yi Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01046
• Pdf URL: https://arxiv.org/pdf/2601.01046
• Copy Paste: [[2601.01046]] KV-Embedding: Training-free Text Embedding via Internal KV Re-routing in Decoder-only LLMs(https://arxiv.org/abs/2601.01046)
• Keywords: llm
• Abstract: While LLMs are powerful embedding backbones, their application in training-free settings faces two structural challenges: causal attention restricts early tokens from accessing subsequent context, and the next-token prediction objective biases representations toward generation rather than semantic compression. To address these limitations, we propose KV-Embedding, a framework that activates the latent representation power of frozen LLMs. Our method leverages the observation that the key-value (KV) states of the final token at each layer encode a compressed view of the sequence. By re-routing these states as a prepended prefix, we enable all tokens to access sequence-level context within a single forward pass. To ensure model-agnostic applicability, we introduce an automated layer selection strategy based on intrinsic dimensionality. Evaluations on MTEB across Qwen, Mistral, and Llama backbones show that KV-Embedding outperforms existing training-free baselines by up to 10%, while maintaining robust performance on sequences up to 4,096 tokens. These results demonstrate that internal state manipulation offers an efficient alternative to input modification, and we hope this work encourages further exploration of LLM internals for representation learning.
• 摘要：虽然 LLM 是强大的嵌入主干，但它们在免训练环境中的应用面临两个结构性挑战：因果注意力限制早期令牌访问后续上下文，而下一个令牌预测目标使表示偏向于生成而不是语义压缩。为了解决这些限制，我们提出了 KV-Embedding，这是一个激活冻结 LLM 的潜在表示能力的框架。我们的方法利用了每层最终令牌的键值 (KV) 状态编码序列的压缩视图的观察结果。通过将这些状态重新路由为前置前缀，我们使所有令牌能够在单个前向传递中访问序列级上下文。为了确保与模型无关的适用性，我们引入了基于内在维度的自动层选择策略。对 Qwen、Mistral 和 Llama 主干网的 MTEB 评估表明，KV-Embedding 的性能比现有的免训练基线高出 10%，同时在多达 4,096 个 token 的序列上保持稳健的性能。这些结果表明，内部状态操作为输入修改提供了一种有效的替代方案，我们希望这项工作能够鼓励进一步探索 LLM 内部的表示学习。

Title: Unsupervised Text Style Transfer for Controllable Intensity

• Authors: Shuhuan Gu, Wenbiao Tao, Xinchen Ma, Kangkang He, Ye Guo, Xiang Li, Yunshi Lan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01060
• Pdf URL: https://arxiv.org/pdf/2601.01060
• Copy Paste: [[2601.01060]] Unsupervised Text Style Transfer for Controllable Intensity(https://arxiv.org/abs/2601.01060)
• Keywords: llm
• Abstract: Unsupervised Text Style Transfer (UTST) aims to build a system to transfer the stylistic properties of a given text without parallel text pairs. Compared with text transfer between style polarities, UTST for controllable intensity is more challenging due to the subtle differences in stylistic features across different intensity levels. Faced with the challenges posed by the lack of parallel data and the indistinguishability between adjacent intensity levels, we propose a SFT-then-PPO paradigm to fine-tune an LLM. We first fine-tune the LLM with synthesized parallel data. Then, we further train the LLM with PPO, where the rewards are elaborately designed for distinguishing the stylistic intensity in hierarchical levels. Both the global and local stylistic features are considered to formulate the reward functions. The experiments on two UTST benchmarks showcase that both rewards have their advantages and applying them to LLM fine-tuning can effectively improve the performance of an LLM backbone based on various evaluation metrics. Even for close levels of intensity, we can still observe the noticeable stylistic difference between the generated text.
• 摘要：无监督文本风格迁移（UTST）旨在构建一个系统，无需并行文本对即可迁移给定文本的风格属性。与风格极性之间的文本传输相比，由于不同强度级别的风格特征存在细微差异，可控强度的 UTST 更具挑战性。面对缺乏并行数据和相邻强度水平之间无法区分所带来的挑战，我们提出了一种 SFT-then-PPO 范式来微调 LLM。我们首先使用合成的并行数据对 LLM 进行微调。然后，我们进一步用 PPO 训练 LLM，其中的奖励是精心设计的，用于区分层次级别的风格强度。制定奖励函数时考虑了全局和局部风格特征。在两个 UTST 基准上的实验表明，两种奖励都有其优势，并将其应用于 LLM 微调可以有效提高基于各种评估指标的 LLM 主干的性能。即使强度接近，我们仍然可以观察到生成的文本之间明显的风格差异。

Title: ks-lit-3m: A 3.1 million word kashmiri text dataset for large language model pretraining

• Authors: Haq Nawaz Malik
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01091
• Pdf URL: https://arxiv.org/pdf/2601.01091
• Copy Paste: [[2601.01091]] ks-lit-3m: A 3.1 million word kashmiri text dataset for large language model pretraining(https://arxiv.org/abs/2601.01091)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) demonstrate remarkable fluency across high-resource languages yet consistently fail to generate coherent text in Kashmiri, a language spoken by approximately seven million people. This performance disparity stems not from inherent model limitations but from a critical scarcity of high-quality training data. Decades of Kashmiri literature remain inaccessible to modern NLP pipelines due to their encoding in the proprietary InPage desktop publishing format. This paper introduces KS-LIT-3M, a curated corpus of 3.1 million words (16.4 million characters) specifically designed for pretraining language models on Kashmiri. The dataset is structured as a single continuous linear text stream, optimized for causal language model training where models learn to predict subsequent tokens from preceding context. The corpus was constructed through the development of a specialized InPage-to-Unicode converter, followed by rigorous preprocessing including English contamination removal, character normalization, and quality validation. Encompassing 131,607 unique words drawn from diverse genres including literary works, journalistic writing, academic texts, and religious scholarship, KS-LIT-3M addresses a fundamental resource gap for Kashmiri language technology. The dataset is released under the CC-BY-4.0 license to facilitate research in Kashmiri natural language processing.
• 摘要：大型语言模型 (LLM) 在各种高资源语言中表现出出色的流利性，但始终无法在克什米尔语（一种约有 700 万人使用的语言）中生成连贯的文本。这种性能差异并非源于模型固有的限制，而是源于高质量训练数据的严重缺乏。由于采用专有的 InPage 桌面出版格式进行编码，现代 NLP 流程仍然无法访问数十年的克什米尔文献。本文介绍了 KS-LIT-3M，这是一个包含 310 万个单词（1640 万个字符）的精选语料库，专为克什米尔语预训练语言模型而设计。该数据集的结构为单个连续的线性文本流，针对因果语言模型训练进行了优化，模型学习从先前的上下文中预测后续的标记。该语料库是通过开发专门的 InPage 到 Unicode 转换器，然后进行严格的预处理（包括英语污染去除、字符规范化和质量验证）而构建的。 KS-LIT-3M 包含来自文学作品、新闻写作、学术文本和宗教学术等不同流派的 131,607 个独特单词，解决了克什米尔语言技术的基本资源缺口。该数据集根据 CC-BY-4.0 许可发布，以促进克什米尔自然语言处理研究。

Title: EmoLoom-2B: Fast Base-Model Screening for Emotion Classification and VAD with Lexicon-Weak Supervision and KV-Off Evaluation

• Authors: Zilin Li, Weiwei Xu, Xuanbo Lu, Zheda Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01112
• Pdf URL: https://arxiv.org/pdf/2601.01112
• Copy Paste: [[2601.01112]] EmoLoom-2B: Fast Base-Model Screening for Emotion Classification and VAD with Lexicon-Weak Supervision and KV-Off Evaluation(https://arxiv.org/abs/2601.01112)
• Keywords: language model, chat
• Abstract: We introduce EmoLoom-2B, a lightweight and reproducible pipeline that turns small language models under 2B parameters into fast screening candidates for joint emotion classification and Valence-Arousal-Dominance prediction. To ensure protocol-faithful and fair evaluation, we unify data loading, training, and inference under a single JSON input-output contract and remove avoidable variance by adopting KV-off decoding as the default setting. We incorporate two orthogonal semantic regularizers: a VAD-preserving constraint that aligns generated text with target VAD triples, and a lightweight external appraisal classifier that provides training-time guidance on goal attainment, controllability, certainty, and fairness without injecting long rationales. To improve polarity sensitivity, we introduce Valence Flip augmentation based on mirrored emotional pairs. During supervised fine-tuning, we apply A/B mixture sampling with entropy-aware temperature scheduling to balance coverage and convergence. Using Qwen-1.8B-Chat as the base model, EmoLoom-2B achieves strong performance on GoEmotions and EmpatheticDialogues, and demonstrates robust cross-corpus generalization on DailyDialog. The proposed recipe is budget-aware, auditable, and re-entrant, serving as a dependable screening pass before heavier training or multimodal fusion.
• 摘要：我们引入了 EmoLoom-2B，这是一种轻量级且可重复的管道，可将 2B 参数下的小型语言模型转变为快速筛选候选者，用于联合情感分类和效价-唤醒-优势预测。为了确保协议忠实和公平的评估，我们将数据加载、训练和推理统一在单个 JSON 输入输出合约下，并通过采用 KV-off 解码作为默认设置来消除可避免的差异。我们结合了两个正交语义正则化器：一个 VAD 保留约束，将生成的文本与目标 VAD 三元组对齐；以及一个轻量级外部评估分类器，它提供有关目标实现、可控性、确定性和公平性的训练时指导，而无需注入冗长的理由。为了提高极性敏感性，我们引入了基于镜像情感对的价翻转增强。在监督微调期间，我们应用 A/B 混合采样和熵感知温度调度来平衡覆盖范围和收敛性。使用 Qwen-1.8B-Chat 作为基础模型，EmoLoom-2B 在 GoEmotions 和 EmpatheticDialogues 上实现了强大的性能，并在 DailyDialog 上展示了强大的跨语料库泛化能力。所提出的方案具有预算意识、可审计和可重入性，可作为更繁重的训练或多模式融合之前的可靠筛选。

Title: RoboPhD: Self-Improving Text-to-SQL Through Autonomous Agent Evolution

• Authors: Andrew Borthwick, Stephen Ash
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01126
• Pdf URL: https://arxiv.org/pdf/2601.01126
• Copy Paste: [[2601.01126]] RoboPhD: Self-Improving Text-to-SQL Through Autonomous Agent Evolution(https://arxiv.org/abs/2601.01126)
• Keywords: agent
• Abstract: We present RoboPhD, a system where AI agents autonomously conduct research to improve Text-to-SQL performance. RoboPhD implements a closed-loop evolution cycle with two coordinated components: a SQL Generation agent composed of a database analysis script and SQL generation instructions, and an Evolution agent that designs new versions based on performance feedback. Central to the framework is an ELO-based selection mechanism enabling survival-of-the-fittest dynamics while handling non-transitivity in performance. Starting from a naive 70-line baseline, RoboPhD evolves agents through iterative cross-pollination, discovering effective techniques without any external guidance on the Text-to-SQL domain. Our best agent, evolved to 1500 lines over 18 iterations, autonomously discovered strategies such as size-adaptive database analysis that adjusts depth based on schema complexity and SQL generation patterns for column selection, evidence interpretation, and aggregation. Evolution provides the largest gains on cheaper models: while we improve by 2.3 points over a strong Claude Opus 4.5 naive baseline, we show an improvement of 8.9 points over the weaker Claude Haiku model. This enables 'skip a tier' deployment: evolved Haiku exceeds naive Sonnet accuracy, and evolved Sonnet exceeds naive Opus, both at lower cost. The full system achieves 73.67% accuracy on the BIRD test set, demonstrating that AI can autonomously build a strong agentic system with only a trivial human-provided starting point.
• 摘要：我们推出了 RoboPhD，这是一个人工智能代理自主进行研究以提高文本到 SQL 性能的系统。 RoboPhD 通过两个协调组件实现闭环演化周期：由数据库分析脚本和 SQL 生成指令组成的 SQL 生成代理，以及根据性能反馈设计新版本的演化代理。该框架的核心是基于 ELO 的选择机制，在处理性能中的非传递性的同时实现适者生存的动态。从简单的 70 行基线开始，RoboPhD 通过迭代异花授粉来发展代理，在文本到 SQL 领域没有任何外部指导的情况下发现有效的技术。我们最好的代理在 18 次迭代中发展到 1500 行，自主发现了诸如大小自适应数据库分析之类的策略，该分析根据模式复杂性和用于列选择、证据解释和聚合的 SQL 生成模式来调整深度。 Evolution 在更便宜的模型上提供了最大的收益：虽然我们比强大的 Claude Opus 4.5 naive 基线提高了 2.3 个点，但我们比较弱的 Claude Haiku 模型提高了 8.9 个点。这使得“跳过一层”部署成为可能：进化的 Haiku 超过了朴素的 Sonnet 的准确性，进化的 Sonnet 超过了朴素的 Opus，两者的成本都较低。整个系统在 BIRD 测试集上达到了 73.67% 的准确率，这表明人工智能可以仅用人类提供的一个微不足道的起点来自主构建强大的代理系统。

Title: SongSage: A Large Musical Language Model with Lyric Generative Pre-training

• Authors: Jiani Guo, Jiajia Li, Jie Wu, Zuchao Li, Yujiu Yang, Ping Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01153
• Pdf URL: https://arxiv.org/pdf/2601.01153
• Copy Paste: [[2601.01153]] SongSage: A Large Musical Language Model with Lyric Generative Pre-training(https://arxiv.org/abs/2601.01153)
• Keywords: language model, llm
• Abstract: Large language models have achieved significant success in various domains, yet their understanding of lyric-centric knowledge has not been fully explored. In this work, we first introduce PlaylistSense, a dataset to evaluate the playlist understanding capability of language models. PlaylistSense encompasses ten types of user queries derived from common real-world perspectives, challenging LLMs to accurately grasp playlist features and address diverse user intents. Comprehensive evaluations indicate that current general-purpose LLMs still have potential for improvement in playlist understanding. Inspired by this, we introduce SongSage, a large musical language model equipped with diverse lyric-centric intelligence through lyric generative pretraining. SongSage undergoes continual pretraining on LyricBank, a carefully curated corpus of 5.48 billion tokens focused on lyrical content, followed by fine-tuning with LyricBank-SFT, a meticulously crafted instruction set comprising 775k samples across nine core lyric-centric tasks. Experimental results demonstrate that SongSage exhibits a strong understanding of lyric-centric knowledge, excels in rewriting user queries for zero-shot playlist recommendations, generates and continues lyrics effectively, and performs proficiently across seven additional capabilities. Beyond its lyric-centric expertise, SongSage also retains general knowledge comprehension and achieves a competitive MMLU score. We will keep the datasets inaccessible due to copyright restrictions and release the SongSage and training script to ensure reproducibility and support music AI research and applications, the datasets release plan details are provided in the appendix.
• 摘要：大型语言模型在各个领域取得了巨大的成功，但它们对以歌词为中心的知识的理解尚未得到充分探索。在这项工作中，我们首先引入 PlaylistSense，一个用于评估语言模型的播放列表理解能力的数据集。 PlaylistSense 包含源自常见现实世界视角的十种类型的用户查询，挑战法学硕士准确掌握播放列表功能并解决不同的用户意图。综合评估表明，当前的通用法学硕士在播放列表理解方面仍有改进的潜力。受此启发，我们推出了 SongSage，这是一种大型音乐语言模型，通过歌词生成预训练配备了多种以歌词为中心的智能。 SongSage 在 LyricBank 上进行持续的预训练，LyricBank 是一个精心策划的语料库，包含 54.8 亿个令牌，专注于歌词内容，然后使用 LyricBank-SFT 进行微调，LyricBank-SFT 是一个精心设计的指令集，包含 9 个以歌词为中心的核心任务的 775,000 个样本。实验结果表明，SongSage 表现出对以歌词为中心的知识的深刻理解，擅长重写用户查询以进行零样本播放列表推荐，有效生成和继续歌词，并在七项附加功能上表现出色。除了以歌词为中心的专业知识之外，SongSage 还保留了一般知识理解能力并获得了具有竞争力的 MMLU 分数。由于版权限制，我们将保持数据集无法访问，并发布SongSage和训练脚本，以确保可重复性并支持音乐AI研究和应用，数据集发布计划详细信息在附录中提供。

Title: DHI: Leveraging Diverse Hallucination Induction for Enhanced Contrastive Factuality Control in Large Language Models

• Authors: Jiani Guo, Xiangke Zeng, Jie Wu, Zuchao Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01156
• Pdf URL: https://arxiv.org/pdf/2601.01156
• Copy Paste: [[2601.01156]] DHI: Leveraging Diverse Hallucination Induction for Enhanced Contrastive Factuality Control in Large Language Models(https://arxiv.org/abs/2601.01156)
• Keywords: language model, llm, hallucination
• Abstract: Large language models (LLMs) frequently produce inaccurate or fabricated information, known as "hallucinations," which compromises their reliability. Existing approaches often train an "Evil LLM" to deliberately generate hallucinations on curated datasets, using these induced hallucinations to guide contrastive decoding against a reliable "positive model" for hallucination mitigation. However, this strategy is limited by the narrow diversity of hallucinations induced, as Evil LLMs trained on specific error types tend to reproduce only these particular patterns, thereby restricting their overall effectiveness. To address these limitations, we propose DHI (Diverse Hallucination Induction), a novel training framework that enables the Evil LLM to generate a broader range of hallucination types without relying on pre-annotated hallucination data. DHI employs a modified loss function that down-weights the generation of specific factually correct tokens, encouraging the Evil LLM to produce diverse hallucinations at targeted positions while maintaining overall factual content. Additionally, we introduce a causal attention masking adaptation to reduce the impact of this penalization on the generation of subsequent tokens. During inference, we apply an adaptive rationality constraint that restricts contrastive decoding to tokens where the positive model exhibits high confidence, thereby avoiding unnecessary penalties on factually correct tokens. Extensive empirical results show that DHI achieves significant performance gains over other contrastive decoding-based approaches across multiple hallucination benchmarks.
• 摘要：大型语言模型 (LLM) 经常产生不准确或捏造的信息，称为“幻觉”，这会损害其可靠性。现有的方法通常训练“邪恶的法学硕士”在精心策划的数据集上故意生成幻觉，使用这些诱发的幻觉来指导对比解码与可靠的“正模型”的幻觉缓解。然而，这种策略受到所引起的幻觉的狭窄多样性的限制，因为针对特定错误类型进行训练的邪恶法学硕士倾向于仅重现这些特定模式，从而限制了其整体有效性。为了解决这些限制，我们提出了 DHI（多样化幻觉诱导），这是一种新颖的训练框架，使 Evil LLM 能够生成更广泛的幻觉类型，而无需依赖预先注释的幻觉数据。 DHI 采用修改后的损失函数，降低特定事实正确令牌的生成权重，鼓励 Evil LLM 在目标位置产生不同的幻觉，同时保持整体事实内容。此外，我们引入了因果注意掩蔽适应，以减少这种惩罚对后续令牌生成的影响。在推理过程中，我们应用自适应理性约束，将对比解码限制为正模型表现出高置信度的标记，从而避免对事实正确的标记进行不必要的惩罚。大量的实证结果表明，与其他基于对比解码的方法相比，DHI 在多个幻觉基准测试中取得了显着的性能提升。

Title: Almost Clinical: Linguistic properties of synthetic electronic health records

• Authors: Serge Sharoff, John Baker, David Francis Hunt, Alan Simpson
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01171
• Pdf URL: https://arxiv.org/pdf/2601.01171
• Copy Paste: [[2601.01171]] Almost Clinical: Linguistic properties of synthetic electronic health records(https://arxiv.org/abs/2601.01171)
• Keywords: llm
• Abstract: This study evaluates the linguistic and clinical suitability of synthetic electronic health records (EHRs) in the field of mental health. First, we describe the rationale and the methodology for creating the synthetic corpus. Second, we assess agency, modality, and information flow across four clinical genres (Assessments, Correspondence, Referrals and Care plans) to understand how LLMs grammatically construct medical authority and patient agency through linguistic choices. While LLMs produce coherent, terminology-appropriate texts that approximate clinical practice, systematic divergences remain, including registerial shifts, insufficient clinical specificity, and inaccuracies in medication use and diagnostic procedures.
• 摘要：本研究评估了心理健康领域合成电子健康记录 (EHR) 的语言和临床适用性。首先，我们描述创建合成语料库的基本原理和方法。其次，我们评估四种临床类型（评估、通信、转诊和护理计划）的代理、模式和信息流，以了解法学硕士如何通过语言选择以语法方式构建医疗权威和患者代理。虽然法学硕士可以撰写与临床实践相一致的、术语合适的文本，但系统性分歧仍然存在，包括注册转移、临床特异性不足以及药物使用和诊断程序的不准确。

Title: Stylometry Analysis of Human and Machine Text for Academic Integrity

• Authors: Hezam Albaqami, Muhammad Asif Ayub, Nasir Ahmad, Yaseen Ahmad, Mohammed M. Alqahtani, Abdullah M. Algamdi, Almoaid A. Owaidah, Kashif Ahmad
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01225
• Pdf URL: https://arxiv.org/pdf/2601.01225
• Copy Paste: [[2601.01225]] Stylometry Analysis of Human and Machine Text for Academic Integrity(https://arxiv.org/abs/2601.01225)
• Keywords: prompt
• Abstract: This work addresses critical challenges to academic integrity, including plagiarism, fabrication, and verification of authorship of educational content, by proposing a Natural Language Processing (NLP)-based framework for authenticating students' content through author attribution and style change detection. Despite some initial efforts, several aspects of the topic are yet to be explored. In contrast to existing solutions, the paper provides a comprehensive analysis of the topic by targeting four relevant tasks, including (i) classification of human and machine text, (ii) differentiating in single and multi-authored documents, (iii) author change detection within multi-authored documents, and (iv) author recognition in collaboratively produced documents. The solutions proposed for the tasks are evaluated on two datasets generated with Gemini using two different prompts, including a normal and a strict set of instructions. During experiments, some reduction in the performance of the proposed solutions is observed on the dataset generated through the strict prompt, demonstrating the complexities involved in detecting machine-generated text with cleverly crafted prompts. The generated datasets, code, and other relevant materials are made publicly available on GitHub, which are expected to provide a baseline for future research in the domain.
• 摘要：这项工作提出了一个基于自然语言处理 (NLP) 的框架，通过作者归属和风格变化检测来验证学生的内容，解决了学术诚信面临的关键挑战，包括抄袭、捏造和教育内容作者身份验证。尽管做出了一些初步努力，但该主题的几个方面仍有待探索。与现有的解决方案相比，本文通过针对四个相关任务对该主题进行了全面分析，包括（i）人类和机器文本的分类，（ii）区分单作者和多作者文档，（iii）多作者文档中的作者变化检测，以及（iv）协作生成的文档中的作者识别。为任务提出的解决方案是在 Gemini 生成的两个数据集上使用两种不同的提示（包括正常指令集和严格指令集）进行评估的。在实验过程中，在通过严格提示生成的数据集上观察到所提出的解决方案的性能有所下降，这证明了使用巧妙设计的提示检测机器生成的文本所涉及的复杂性。生成的数据集、代码和其他相关材料在 GitHub 上公开提供，预计将为该领域的未来研究提供基线。

Title: Racka: Efficient Hungarian LLM Adaptation on Academic Infrastructure

• Authors: Zsolt Csibi (2), Bence György Gortka (1), Natabara Gyöngyössy (2), Kornél Nagy (1), Dávid Márk Nemeskey (1), Martin Sallai (1), András Simonyi (2), András Márk Szekeres (1), Gábor Palkó (1) ((1) Department of Digital Humanities, Eötvös Loránd University (2) Department of Artificial Intelligence, Eötvös Loránd University)
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01244
• Pdf URL: https://arxiv.org/pdf/2601.01244
• Copy Paste: [[2601.01244]] Racka: Efficient Hungarian LLM Adaptation on Academic Infrastructure(https://arxiv.org/abs/2601.01244)
• Keywords: language model, llm
• Abstract: We present Racka, a lightweight, continually pretrained large language model designed to bridge the resource gap between Hungarian and high-resource languages such as English and German. Racka employs parameter-efficient continual pretraining via Low-Rank Adaptation (LoRA) on a Qwen-3 4B backbone, making the recipe practical on A100 (40GB)-based HPC clusters with low inter-node bandwidth. To better match the training distribution, we replace and adapt the tokenizer, achieving substantially improved tokenization fertility for Hungarian while maintaining competitive performance in English and German. The model is trained on 160B subword tokens drawn from a mixture of internet and high-quality curated sources, with a composition of 44% Hungarian, 24% English, 21% German, and 11% code. This data mix is chosen to mitigate catastrophic forgetting and preserve high-resource language capabilities during continual pretraining. Our preliminary results indicate modest but stable results in language adaptation.
• 摘要：我们推出了 Racka，这是一种轻量级、持续预训练的大型语言模型，旨在弥合匈牙利语与英语和德语等高资源语言之间的资源差距。 Racka 通过低秩适应 (LoRA) 在 Qwen-3 4B 主干上采用参数高效的连续预训练，使该方法在节点间带宽较低的基于 A100 (40GB) 的 HPC 集群上实用。为了更好地匹配训练分布，我们替换并调整了分词器，显着提高了匈牙利语的分词率，同时保持了英语和德语的竞争性表现。该模型使用 160B 个子词标记进行训练，这些子词标记来自互联网和高质量的精选资源，其中 44% 是匈牙利语，24% 是英语，21% 是德语，11% 是代码。选择这种数据组合是为了在持续预训练期间减轻灾难性遗忘并保留高资源语言能力。我们的初步结果表明语言适应的结果有限但稳定。

Title: From Policy to Logic for Efficient and Interpretable Coverage Assessment

• Authors: Rhitabrat Pokharel, Hamid Hassanzadeh, Ameeta Agrawal
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01266
• Pdf URL: https://arxiv.org/pdf/2601.01266
• Copy Paste: [[2601.01266]] From Policy to Logic for Efficient and Interpretable Coverage Assessment(https://arxiv.org/abs/2601.01266)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) have demonstrated strong capabilities in interpreting lengthy, complex legal and policy language. However, their reliability can be undermined by hallucinations and inconsistencies, particularly when analyzing subjective and nuanced documents. These challenges are especially critical in medical coverage policy review, where human experts must be able to rely on accurate information. In this paper, we present an approach designed to support human reviewers by making policy interpretation more efficient and interpretable. We introduce a methodology that pairs a coverage-aware retriever with symbolic rule-based reasoning to surface relevant policy language, organize it into explicit facts and rules, and generate auditable rationales. This hybrid system minimizes the number of LLM inferences required which reduces overall model cost. Notably, our approach achieves a 44% reduction in inference cost alongside a 4.5% improvement in F1 score, demonstrating both efficiency and effectiveness.
• 摘要：大型语言模型 (LLM) 在解释冗长、复杂的法律和政策语言方面表现出了强大的能力。然而，它们的可靠性可能会因幻觉和不一致而受到损害，特别是在分析主观和细致入微的文档时。这些挑战在医疗保险政策审查中尤其重要，人类专家必须能够依赖准确的信息。在本文中，我们提出了一种旨在通过使政策解释更加高效和可解释来支持人工审核员的方法。我们引入了一种方法，将覆盖感知检索器与基于符号规则的推理配对，以显示相关的政策语言，将其组织成明确的事实和规则，并生成可审计的理由。这种混合系统最大限度地减少了所需的 LLM 推理数量，从而降低了总体模型成本。值得注意的是，我们的方法实现了推理成本降低 44%，同时 F1 分数提高了 4.5%，展示了效率和有效性。

Title: FLOP-Efficient Training: Early Stopping Based on Test-Time Compute Awareness

• Authors: Hossam Amer, Maryam Dialameh, Hossein Rajabzadeh, Walid Ahmed, Weiwei Zhang, Yang Liu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.01332
• Pdf URL: https://arxiv.org/pdf/2601.01332
• Copy Paste: [[2601.01332]] FLOP-Efficient Training: Early Stopping Based on Test-Time Compute Awareness(https://arxiv.org/abs/2601.01332)
• Keywords: language model
• Abstract: Scaling training compute, measured in FLOPs, has long been shown to improve the accuracy of large language models, yet training remains resource-intensive. Prior work shows that increasing test-time compute (TTC)-for example through iterative sampling-can allow smaller models to rival or surpass much larger ones at lower overall cost. We introduce TTC-aware training, where an intermediate checkpoint and a corresponding TTC configuration can together match or exceed the accuracy of a fully trained model while requiring substantially fewer training FLOPs. Building on this insight, we propose an early stopping algorithm that jointly selects a checkpoint and TTC configuration to minimize training compute without sacrificing accuracy. To make this practical, we develop an efficient TTC evaluation method that avoids exhaustive search, and we formalize a break-even bound that identifies when increased inference compute compensates for reduced training compute. Experiments demonstrate up to 92\% reductions in training FLOPs while maintaining and sometimes remarkably improving accuracy. These results highlight a new perspective for balancing training and inference compute in model development, enabling faster deployment cycles and more frequent model refreshes. Codes will be publicly released.
• 摘要：长期以来，以 FLOP 为单位衡量的扩展训练计算量已被证明可以提高大型语言模型的准确性，但训练仍然是资源密集型的。先前的工作表明，增加测试时计算（TTC）（例如通过迭代采样）可以使较小的模型以较低的总体成本与较大的模型相媲美或超越。我们引入了 TTC 感知训练，其中中间检查点和相应的 TTC 配置可以一起达到或超过完全训练模型的准确性，同时需要更少的训练 FLOP。基于这一见解，我们提出了一种早期停止算法，该算法联合选择检查点和 TTC 配置，以在不牺牲准确性的情况下最大限度地减少训练计算。为了实现这一点，我们开发了一种有效的 TTC 评估方法，可以避免穷举搜索，并且我们形式化了一个收支平衡界限，用于识别何时增加的推理计算可以补偿减少的训练计算。实验表明，训练 FLOP 减少高达 92%，同时保持甚至有时显着提高准确性。这些结果凸显了在模型开发中平衡训练和推理计算的新视角，从而实现更快的部署周期和更频繁的模型刷新。代码将公开发布。

Title: Reasoning Over Recall: Evaluating the Efficacy of Generalist Architectures vs. Specialized Fine-Tunes in RAG-Based Mental Health Dialogue Systems

• Authors: Md Abdullah Al Kafi, Raka Moni, Sumit Kumar Banshal
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01341
• Pdf URL: https://arxiv.org/pdf/2601.01341
• Copy Paste: [[2601.01341]] Reasoning Over Recall: Evaluating the Efficacy of Generalist Architectures vs. Specialized Fine-Tunes in RAG-Based Mental Health Dialogue Systems(https://arxiv.org/abs/2601.01341)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: The deployment of Large Language Models (LLMs) in mental health counseling faces the dual challenges of hallucinations and lack of empathy. While the former may be mitigated by RAG (retrieval-augmented generation) by anchoring answers in trusted clinical sources, there remains an open question as to whether the most effective model under this paradigm would be one that is fine-tuned on mental health data, or a more general and powerful model that succeeds purely on the basis of reasoning. In this paper, we perform a direct comparison by running four open-source models through the same RAG pipeline using ChromaDB: two generalist reasoners (Qwen2.5-3B and Phi-3-Mini) and two domain-specific fine-tunes (MentalHealthBot-7B and TherapyBot-7B). We use an LLM-as-a-Judge framework to automate evaluation over 50 turns. We find a clear trend: the generalist models outperform the domain-specific ones in empathy (3.72 vs. 3.26, $p < 0.001$) in spite of being much smaller (3B vs. 7B), and all models perform well in terms of safety, but the generalist models show better contextual understanding and are less prone to overfitting as we observe in the domain-specific models. Overall, our results indicate that for RAG-based therapy systems, strong reasoning is more important than training on mental health-specific vocabulary; i.e. a well-reasoned general model would provide more empathetic and balanced support than a larger narrowly fine-tuned model, so long as the answer is already grounded in clinical evidence.
• 摘要：大语言模型（LLM）在心理健康咨询中的应用面临着幻觉和缺乏同理心的双重挑战。虽然前者可以通过 RAG（检索增强生成）通过将答案锚定在可信的临床来源中来缓解，但仍然存在一个悬而未决的问题：这种范式下最有效的模型是根据心理健康数据进行微调的模型，还是纯粹基于推理取得成功的更通用、更强大的模型。在本文中，我们使用 ChromaDB 通过相同的 RAG 管道运行四个开源模型来进行直接比较：两个通用推理器（Qwen2.5-3B 和 Phi-3-Mini）和两个特定领域的微调（MentalHealthBot-7B 和 TherapyBot-7B）。我们使用法学硕士作为法官框架来自动评估 50 轮以上。我们发现了一个明显的趋势：通才模型在同理心方面优于特定领域模型（3.72 vs. 3.26，$p < 0.001$），尽管模型要小得多（3B vs. 7B），并且所有模型在安全性方面都表现良好，但通才模型表现出更好的上下文理解，并且正如我们在特定领域模型中观察到的那样，不太容易过度拟合。总的来说，我们的结果表明，对于基于 RAG 的治疗系统，强有力的推理比心理健康特定词汇的培训更重要；也就是说，只要答案已经建立在临床证据的基础上，一个合理的一般模型就会比一个更大的、经过微调的模型提供更多的同理心和平衡的支持。

Title: Investigating the Multilingual Calibration Effects of Language Model Instruction-Tuning

• Authors: Jerry Huang, Peng Lu, Qiuhao Zeng, Yusuke Iwasawa, Yutaka Matsuo, Sarath Chandar, Edison Marrese-Taylor, Irene Li
• Subjects: cs.CL, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2601.01362
• Pdf URL: https://arxiv.org/pdf/2601.01362
• Copy Paste: [[2601.01362]] Investigating the Multilingual Calibration Effects of Language Model Instruction-Tuning(https://arxiv.org/abs/2601.01362)
• Keywords: language model, llm
• Abstract: Ensuring that deep learning models are well-calibrated in terms of their predictive uncertainty is essential in maintaining their trustworthiness and reliability, yet despite increasing advances in foundation model research, the relationship between such large language models (LLMs) and their calibration remains an open area of research. In this work, we look at a critical gap in the calibration of LLMs within multilingual settings, in an attempt to better understand how the data scarcity can potentially lead to different calibration effects and how commonly used techniques can apply in these settings. Our analysis on two multilingual benchmarks, over 29 and 42 languages respectively, reveals that even in low-resource languages, model confidence can increase significantly after instruction-tuning on high-resource language SFT datasets. However, improvements in accuracy are marginal or non-existent, resulting in mis-calibration, highlighting a critical shortcoming of standard SFT for multilingual languages. Furthermore, we observe that the use of label smoothing to be a reasonable method alleviate this concern, again without any need for low-resource SFT data, maintaining better calibration across all languages. Overall, this highlights the importance of multilingual considerations for both training and tuning LLMs in order to improve their reliability and fairness in downstream use.
• 摘要：确保深度学习模型在预测不确定性方面得到良好校准对于维持其可信度和可靠性至关重要，然而，尽管基础模型研究不断取得进展，此类大型语言模型 (LLM) 与其校准之间的关系仍然是一个开放的研究领域。在这项工作中，我们着眼于多语言环境中法学硕士校准的关键差距，试图更好地理解数据稀缺如何可能导致不同的校准效果以及常用技术如何在这些环境中应用。我们对两个多语言基准（分别超过 29 种和 42 种语言）的分析表明，即使在低资源语言中，在对高资源语言 SFT 数据集进行指令调整后，模型置信度也可以显着提高。然而，准确性的提高微乎其微或根本不存在，导致校准错误，凸显了多语言标准 SFT 的一个关键缺点。此外，我们观察到，使用标签平滑作为一种合理的方法缓解了这种担忧，同样不需要低资源 SFT 数据，从而在所有语言中保持更好的校准。总的来说，这凸显了多语言考虑对于培训和调整法学硕士的重要性，以提高其在下游使用中的可靠性和公平性。

Title: EternalMath: A Living Benchmark of Frontier Mathematics that Evolves with Human Discovery

• Authors: Jicheng Ma, Guohua Wang, Xinhua Feng, Yiming Liu, Zhichao Hu, Yuhong Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01400
• Pdf URL: https://arxiv.org/pdf/2601.01400
• Copy Paste: [[2601.01400]] EternalMath: A Living Benchmark of Frontier Mathematics that Evolves with Human Discovery(https://arxiv.org/abs/2601.01400)
• Keywords: language model, llm
• Abstract: Current evaluations of mathematical reasoning in large language models (LLMs) are dominated by static benchmarks, either derived from competition-style problems or curated through costly expert effort, resulting in limited coverage of research-level mathematics and rapid performance saturation. We propose a fully automated, theorem-grounded pipeline for evaluating frontier mathematical reasoning, which directly transforms recent peer-reviewed mathematical literature into executable and verifiable reasoning tasks. The pipeline identifies constructive or quantitative results, instantiates them into parameterized problem templates, and generates deterministic solutions through execution-based verification, enabling scalable, reproducible, and continuously updatable evaluation without reliance on large-scale expert authoring. By design, this approach supports temporal extensibility, intrinsic correctness checking, and domain-specific customization across mathematical subfields. Applying this pipeline yields \textbf{EternalMath}, an evolving evaluation suite derived from contemporary research papers. Experiments with state-of-the-art LLMs reveal substantial performance gaps, indicating that mathematical reasoning at the research frontier remains far from saturated and underscoring the need for evaluation methodologies that evolve in step with human mathematical discovery.
• 摘要：当前对大型语言模型（LLM）中数学推理的评估主要以静态基准为主导，这些基准要么源自竞争型问题，要么通过昂贵的专家努力策划，导致研究级数学的覆盖范围有限，性能迅速饱和。我们提出了一种完全自动化的、以定理为基础的管道，用于评估前沿数学推理，它直接将最近的同行评审的数学文献转化为可执行和可验证的推理任务。该管道识别建设性或定量结果，将其实例化为参数化问题模板，并通过基于执行的验证生成确定性解决方案，从而实现可扩展、可重复和持续更新的评估，而无需依赖大规模专家创作。通过设计，该方法支持时间可扩展性、内在正确性检查以及跨数学子领域的特定于领域的定制。应用此流程会产生 \textbf{EternalMath}，这是一个源自当代研究论文的不断发展的评估套件。最先进的法学硕士的实验揭示了巨大的性能差距，表明研究前沿的数学推理仍远未饱和，并强调了对与人类数学发现同步发展的评估方法的需求。

Title: LANCET: Neural Intervention via Structural Entropy for Mitigating Faithfulness Hallucinations in LLMs

• Authors: Chenxu Wang, Chaozhuo Li, Pengbo Wang, Litian Zhang, Songyang Liu, Ji Qi, Jiahui Hu, Yushan Cai, Hao Zhao, Rui Pu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.01401
• Pdf URL: https://arxiv.org/pdf/2601.01401
• Copy Paste: [[2601.01401]] LANCET: Neural Intervention via Structural Entropy for Mitigating Faithfulness Hallucinations in LLMs(https://arxiv.org/abs/2601.01401)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models have revolutionized information processing, yet their reliability is severely compromised by faithfulness hallucinations. While current approaches attempt to mitigate this issue through node-level adjustments or coarse suppression, they often overlook the distributed nature of neural information, leading to imprecise interventions. Recognizing that hallucinations propagate through specific forward transmission pathways like an infection, we aim to surgically block this flow using precise structural analysis. To leverage this, we propose Lancet, a novel framework that achieves precise neural intervention by leveraging structural entropy and hallucination difference ratios. Lancet first locates hallucination-prone neurons via gradient-driven contrastive analysis, then maps their propagation pathways by minimizing structural entropy, and finally implements a hierarchical intervention strategy that preserves general model capabilities. Comprehensive evaluations across hallucination benchmark datasets demonstrate that Lancet significantly outperforms state-of-the-art methods, validating the effectiveness of our surgical approach to neural intervention.
• 摘要：大型语言模型彻底改变了信息处理，但其可靠性却因忠实幻觉而受到严重损害。虽然当前的方法试图通过节点级调整或粗略抑制来缓解这个问题，但它们常常忽视神经信息的分布式性质，导致干预不精确。认识到幻觉通过特定的前向传播途径（例如感染）传播，我们的目标是使用精确的结构分析通过手术阻断这种传播。为了利用这一点，我们提出了 Lancet，这是一种新颖的框架，通过利用结构熵和幻觉差异比来实现精确的神经干预。 《柳叶刀》首先通过梯度驱动的对比分析来定位易产生幻觉的神经元，然后通过最小化结构熵来绘制它们的传播路径，最后实施保留一般模型功能的分层干预策略。对幻觉基准数据集的综合评估表明，Lancet 显着优于最先进的方法，验证了我们神经干预手术方法的有效性。

Title: From Emotion Classification to Emotional Reasoning: Enhancing Emotional Intelligence in Large Language Models

• Authors: Arjhun Sreedar, Rohan Pillay, Laukik Patade
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01407
• Pdf URL: https://arxiv.org/pdf/2601.01407
• Copy Paste: [[2601.01407]] From Emotion Classification to Emotional Reasoning: Enhancing Emotional Intelligence in Large Language Models(https://arxiv.org/abs/2601.01407)
• Keywords: language model, llm, chain-of-thought, agent
• Abstract: This work investigates whether synthetic emotional chain-of-thought data can improve the emotional reasoning abilities of smaller open large language models (LLMs). We design a multi-agent generation pipeline that produces therapy-style conversations and converts them into structured emotion multiple-choice questions (MCQs) with explanations. We propose that fine-tuning a variety of 7B models on this dataset should yield substantial gains in emotional understanding and emotional awareness on EmoBench-style evaluations, suggesting that emotional reasoning can be induced without architectural changes. Our results demonstrate that fine-tuned Mistral 7B achieves EU improvements from 10.5 to 20.5 and EA improvements from 40.5 to 60.0, validating the effectiveness of synthetic emotional reasoning data for enhancing model capabilities in nuanced emotional tasks.
• 摘要：这项工作研究了合成情感思维链数据是否可以提高较小的开放大型语言模型（LLM）的情感推理能力。我们设计了一个多智能体生成管道，可以生成治疗式对话，并将其转换为带有解释的结构化情感多项选择题 (MCQ)。我们建议，在此数据集上微调各种 7B 模型应该会在 EmoBench 式评估的情感理解和情感意识方面产生实质性收益，这表明可以在不改变架构的情况下诱导情感推理。我们的结果表明，经过微调的 Mistral 7B 将 EU 从 10.5 提高到 20.5，将 EA 从 40.5 提高到 60.0，验证了合成情感推理数据对于增强模型在细致情感任务中的能力的有效性。

Title: iFlip: Iterative Feedback-driven Counterfactual Example Refinement

• Authors: Yilong Wang, Qianli Wang, Nils Feldhus
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.01446
• Pdf URL: https://arxiv.org/pdf/2601.01446
• Copy Paste: [[2601.01446]] iFlip: Iterative Feedback-driven Counterfactual Example Refinement(https://arxiv.org/abs/2601.01446)
• Keywords: language model, llm
• Abstract: Counterfactual examples are minimal edits to an input that alter a model's prediction. They are widely employed in explainable AI to probe model behavior and in natural language processing (NLP) to augment training data. However, generating valid counterfactuals with large language models (LLMs) remains challenging, as existing single-pass methods often fail to induce reliable label changes, neglecting LLMs' self-correction capabilities. To explore this untapped potential, we propose iFlip, an iterative refinement approach that leverages three types of feedback, including model confidence, feature attribution, and natural language. Our results show that iFlip achieves an average 57.8% higher validity than the five state-of-the-art baselines, as measured by the label flipping rate. The user study further corroborates that iFlip outperforms baselines in completeness, overall satisfaction, and feasibility. In addition, ablation studies demonstrate that three components are paramount for iFlip to generate valid counterfactuals: leveraging an appropriate number of iterations, pointing to highly attributed words, and early stopping. Finally, counterfactuals generated by iFlip enable effective counterfactual data augmentation, substantially improving model performance and robustness.
• 摘要：反事实示例是对输入进行最小程度的编辑，从而改变模型的预测。它们广泛应用于可解释的人工智能中以探测模型行为，以及自然语言处理（NLP）中以增强训练数据。然而，利用大型语言模型 (LLM) 生成有效的反事实仍然具有挑战性，因为现有的单遍方法通常无法诱导可靠的标签更改，而忽略了 LLM 的自我纠正能力。为了探索这一未开发的潜力，我们提出了 iFlip，这是一种迭代细化方法，利用三种类型的反馈，包括模型置信度、特征归因和自然语言。我们的结果表明，根据标签翻转率来衡量，iFlip 的有效性比五个最先进的基线平均高出 57.8%。用户研究进一步证实 iFlip 在完整性、总体满意度和可行性方面优于基线。此外，消融研究表明，三个组成部分对于 iFlip 生成有效的反事实至关重要：利用适当数量的迭代、指向高度归因的单词以及提前停止。最后，iFlip 生成的反事实可以实现有效的反事实数据增强，从而显着提高模型性能和鲁棒性。

Title: Bridging the gap: A comparative exploration of Speech-LLM and end-to-end architecture for multilingual conversational ASR

• Authors: Yuxiang Mei, Dongxing Xu, Jiaen Liang, Yanhua Long
• Subjects: cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2601.01461
• Pdf URL: https://arxiv.org/pdf/2601.01461
• Copy Paste: [[2601.01461]] Bridging the gap: A comparative exploration of Speech-LLM and end-to-end architecture for multilingual conversational ASR(https://arxiv.org/abs/2601.01461)
• Keywords: language model, llm
• Abstract: The INTERSPEECH 2025 Challenge on Multilingual Conversational Speech Language Models (MLC-SLM) promotes multilingual conversational ASR with large language models (LLMs). Our previous SHNU-mASR system adopted a competitive parallel-speech-encoder architecture that integrated Whisper and mHuBERT with an LLM. However, it faced two challenges: simple feature concatenation may not fully exploit complementary information, and the performance gap between LLM-based ASR and end-to-end(E2E) encoder-decoder ASR remained unexplored. In this work, we present an enhanced LLM-based ASR framework that combines fine-tuned Whisper and mHuBERT encoders with an LLM to enrich speech representations. We first evaluate E2E Whisper models with LoRA and full fine-tuning on the MLC-SLM ASR task, and then propose cross-attention-based fusion mechanisms for the parallel-speech-encoder. On the official evaluation set of the MLC-SLM Challenge, our system achieves a CER/WER of 10.69%, ranking on par with the top-ranked Track 1 systems, even though it uses only 1,500 hours of baseline training data compared with their large-scale training sets. Nonetheless, we find that our final LLM-based ASR still does not match the performance of a fine-tuned E2E Whisper model, providing valuable empirical guidance for future Speech-LLM design. Our code is publicly available at this https URL.
• 摘要：INTERSPEECH 2025 多语言会话语音模型挑战赛 (MLC-SLM) 通过大型语言模型 (LLM) 促进多语言会话 ASR。我们之前的 SHNU-mASR 系统采用了具有竞争力的并行语音编码器架构，将 Whisper 和 mHuBERT 与法学硕士集成在一起。然而，它面临两个挑战：简单的特征串联可能无法充分利用互补信息，并且基于 LLM 的 ASR 和端到端 (E2E) 编码器-解码器 ASR 之间的性能差距仍未被探索。在这项工作中，我们提出了一个基于 LLM 的增强型 ASR 框架，它将微调的 Whisper 和 mHuBERT 编码器与 LLM 结合起来，以丰富语音表示。我们首先使用 LoRA 评估 E2E Whisper 模型并在 MLC-SLM ASR 任务上进行全面微调，然后为并行语音编码器提出基于交叉注意的融合机制。在 MLC-SLM 挑战赛的官方评估集上，我们的系统实现了 10.69% 的 CER/WER，与排名靠前的 Track 1 系统持平，尽管与大规模训练集相比，它仅使用 1,500 小时的基线训练数据。尽管如此，我们发现最终的基于 LLM 的 ASR 仍然无法与经过微调的 E2E Whisper 模型的性能相匹配，这为未来的 Speech-LLM 设计提供了有价值的经验指导。我们的代码可通过此 https URL 公开获取。

Title: Can Legislation Be Made Machine-Readable in PROLEG?

• Authors: May-Myo Zin, Sabine Wehnert, Yuntao Kong, Ha-Thanh Nguyen, Wachara Fungwacharakorn, Jieying Xue, Michał Araszkiewicz, Randy Goebel, Ken Satoh, Le-Minh Nguyen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01477
• Pdf URL: https://arxiv.org/pdf/2601.01477
• Copy Paste: [[2601.01477]] Can Legislation Be Made Machine-Readable in PROLEG?(https://arxiv.org/abs/2601.01477)
• Keywords: language model, llm, prompt
• Abstract: The anticipated positive social impact of regulatory processes requires both the accuracy and efficiency of their application. Modern artificial intelligence technologies, including natural language processing and machine-assisted reasoning, hold great promise for addressing this challenge. We present a framework to address the challenge of tools for regulatory application, based on current state-of-the-art (SOTA) methods for natural language processing (large language models or LLMs) and formalization of legal reasoning (the legal representation system PROLEG). As an example, we focus on Article 6 of the European General Data Protection Regulation (GDPR). In our framework, a single LLM prompt simultaneously transforms legal text into if-then rules and a corresponding PROLEG encoding, which are then validated and refined by legal domain experts. The final output is an executable PROLEG program that can produce human-readable explanations for instances of GDPR decisions. We describe processes to support the end-to-end transformation of a segment of a regulatory document (Article 6 from GDPR), including the prompting frame to guide an LLM to "compile" natural language text to if-then rules, then to further "compile" the vetted if-then rules to PROLEG. Finally, we produce an instance that shows the PROLEG execution. We conclude by summarizing the value of this approach and note observed limitations with suggestions to further develop such technologies for capturing and deploying regulatory frameworks.
• 摘要：监管流程的预期积极社会影响需要其应用的准确性和效率。现代人工智能技术，包括自然语言处理和机器辅助推理，为应对这一挑战带来了巨大希望。我们提出了一个框架来应对监管应用工具的挑战，该框架基于当前最先进的自然语言处理（SOTA）方法（大语言模型或LLM）和法律推理的形式化（法律代表系统PROLEG）。例如，我们重点关注欧洲通用数据保护条例 (GDPR) 第 6 条。在我们的框架中，单个 LLM 提示同时将法律文本转换为 if-then 规则和相应的 PROLEG 编码，然后由法律领域专家进行验证和完善。最终输出是一个可执行的 PROLEG 程序，它可以为 GDPR 决策实例生成人类可读的解释。我们描述了支持监管文件片段（GDPR 第 6 条）端到端转换的流程，包括指导法学硕士将自然语言文本“编译”为 if-then 规则，然后进一步将经过审查的 if-then 规则“编译”为 PROLEG 的提示框架。最后，我们生成一个显示 PROLEG 执行的实例。最后，我们总结了这种方法的价值，并指出了观察到的局限性，并提出了进一步开发此类技术以捕获和部署监管框架的建议。

Title: Distortion Instead of Hallucination: The Effect of Reasoning Under Strict Constraints

• Authors: Junichiro Niimi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01490
• Pdf URL: https://arxiv.org/pdf/2601.01490
• Copy Paste: [[2601.01490]] Distortion Instead of Hallucination: The Effect of Reasoning Under Strict Constraints(https://arxiv.org/abs/2601.01490)
• Keywords: language model, gpt, llm, hallucination
• Abstract: With the widespread adoption of large language models (LLMs), hallucinations, which are non-factual fabrications in model outputs, have become serious concerns. Reasoning capabilities have received attention as a self-verification process to improve output reliability. However, the effect of reasoning within a closed system where LLMs cannot rely on external tools or knowledge has yet to be clarified. We therefore conduct experiments under strict constraints (recommending peer-reviewed journal articles in computer science) to examine the effect of reasoning across multiple models (GPT-5.2 and Gemini 3 Flash). Our results reveal a problematic trade-off between constraint compliance and factual accuracy. Non-reasoning models exhibit high constraint violation rates (66-75%) but maintain factual accuracy, while reasoning models reduce violations (13-26%) but systematically distort known facts to satisfy constraints and increase complete fabrication. This trade-off pattern is consistent across both models despite different architectures, indicating a fundamental limitation of reasoning. Furthermore, reasoning does not uniformly improve output authenticity: effects diverge by model, reflecting different allocations of the compliance-truthfulness trade-off. These findings challenge the assumption that reasoning universally improves reliability: reasoning models trade honest constraint violations for detection-resistant distortions.
• 摘要：随着大语言模型（LLM）的广泛采用，模型输出中的非事实捏造的幻觉已成为严重问题。推理能力作为一种提高输出可靠性的自我验证过程而受到关注。然而，法学硕士不能依赖外部工具或知识的封闭系统内推理的效果尚不清楚。因此，我们在严格的约束下进行实验（推荐计算机科学领域的同行评审期刊文章），以检验跨多个模型（GPT-5.2 和 Gemini 3 Flash）的推理效果。我们的结果揭示了约束合规性和事实准确性之间存在问题的权衡。非推理模型表现出较高的约束违规率（66-75%），但保持事实准确性，而推理模型减少违规（13-26%），但系统地扭曲已知事实以满足约束并增加完整制造。尽管架构不同，但这种权衡模式在两种模型中是一致的，这表明推理的基本局限性。此外，推理并不能统一提高输出的真实性：不同模型的效果有所不同，反映了合规性与真实性权衡的不同分配。这些发现挑战了推理普遍提高可靠性的假设：推理模型用诚实约束违规换取难以检测的扭曲。

Title: From Failure to Mastery: Generating Hard Samples for Tool-use Agents

• Authors: Bingguang Hao, Zengzhuang Xu, Yuntao Wen, Xinyi Xu, Yang Liu, Tong Zhao, Maolin Wang, Long Chen, Dong Wang, Yicheng Chen, Cunyin Peng, Xiangyu Zhao, Chenyi Zhuang, Ji Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01498
• Pdf URL: https://arxiv.org/pdf/2601.01498
• Copy Paste: [[2601.01498]] From Failure to Mastery: Generating Hard Samples for Tool-use Agents(https://arxiv.org/abs/2601.01498)
• Keywords: gpt, llm, chain-of-thought, agent
• Abstract: The advancement of LLM agents with tool-use capabilities requires diverse and complex training corpora. Existing data generation methods, which predominantly follow a paradigm of random sampling and shallow generation, often yield simple and homogeneous trajectories that fail to capture complex, implicit logical dependencies. To bridge this gap, we introduce HardGen, an automatic agentic pipeline designed to generate hard tool-use training samples with verifiable reasoning. Firstly, HardGen establishes a dynamic API Graph built upon agent failure cases, from which it samples to synthesize hard traces. Secondly, these traces serve as conditional priors to guide the instantiation of modular, abstract advanced tools, which are subsequently leveraged to formulate hard queries. Finally, the advanced tools and hard queries enable the generation of verifiable complex Chain-of-Thought (CoT), with a closed-loop evaluation feedback steering the continuous refinement of the process. Extensive evaluations demonstrate that a 4B parameter model trained with our curated dataset achieves superior performance compared to several leading open-source and closed-source competitors (e.g., GPT-5.2, Gemini-3-Pro and Claude-Opus-4.5). Our code, models, and dataset will be open-sourced to facilitate future research.
• 摘要：具有工具使用能力的LLM代理的进步需要多样化和复杂的培训语料库。现有的数据生成方法主要遵循随机采样和浅层生成的范式，通常会产生简单且同质的轨迹，而无法捕获复杂的隐式逻辑依赖性。为了弥补这一差距，我们引入了 HardGen，这是一种自动代理管道，旨在生成具有可验证推理的硬工具使用训练样本。首先，HardGen 建立了一个基于代理故障案例的动态 API 图表，从中进行采样以合成硬跟踪。其次，这些跟踪作为条件先验来指导模块化、抽象高级工具的实例化，随后利用这些工具来制定硬查询。最后，先进的工具和硬查询能够生成可验证的复杂思想链（CoT），并通过闭环评估反馈引导流程的持续改进。广泛的评估表明，与几个领先的开源和闭源竞争对手（例如 GPT-5.2、Gemini-3-Pro 和 Claude-Opus-4.5）相比，使用我们精选的数据集训练的 4B 参数模型具有卓越的性能。我们的代码、模型和数据集将开源，以促进未来的研究。

Title: EmoHarbor: Evaluating Personalized Emotional Support by Simulating the User's Internal World

• Authors: Jing Ye, Lu Xiang, Yaping Zhang, Chengqing Zong
• Subjects: cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2601.01530
• Pdf URL: https://arxiv.org/pdf/2601.01530
• Copy Paste: [[2601.01530]] EmoHarbor: Evaluating Personalized Emotional Support by Simulating the User's Internal World(https://arxiv.org/abs/2601.01530)
• Keywords: llm, agent
• Abstract: Current evaluation paradigms for emotional support conversations tend to reward generic empathetic responses, yet they fail to assess whether the support is genuinely personalized to users' unique psychological profiles and contextual needs. We introduce EmoHarbor, an automated evaluation framework that adopts a User-as-a-Judge paradigm by simulating the user's inner world. EmoHarbor employs a Chain-of-Agent architecture that decomposes users' internal processes into three specialized roles, enabling agents to interact with supporters and complete assessments in a manner similar to human users. We instantiate this benchmark using 100 real-world user profiles that cover a diverse range of personality traits and situations, and define 10 evaluation dimensions of personalized support quality. Comprehensive evaluation of 20 advanced LLMs on EmoHarbor reveals a critical insight: while these models excel at generating empathetic responses, they consistently fail to tailor support to individual user contexts. This finding reframes the central challenge, shifting research focus from merely enhancing generic empathy to developing truly user-aware emotional support. EmoHarbor provides a reproducible and scalable framework to guide the development and evaluation of more nuanced and user-aware emotional support systems.
• 摘要：当前情感支持对话的评估范式倾向于奖励一般的同理心反应，但它们无法评估支持是否真正针对用户独特的心理特征和情境需求进行个性化。我们推出了 EmoHarbor，一个自动化评估框架，通过模拟用户的内心世界，采用“用户即评判”范式。 EmoHarbor采用了代理链架构，将用户的内部流程分解为三个专门的角色，使代理能够与支持者交互并以类似于人类用户的方式完成评估。我们使用 100 个真实世界的用户配置文件来实例化该基准，这些用​​户配置文件涵盖了各种个性特征和情况，并定义了个性化支持质量的 10 个评估维度。对 EmoHarbor 上 20 个高级法学硕士的综合评估揭示了一个重要的见解：虽然这些模型擅长产生同理心响应，但它们始终无法根据个人用户环境提供定制支持。这一发现重新定义了核心挑战，将研究重点从仅仅增强一般同理心转移到开发真正的用户意识情感支持。 EmoHarbor 提供了一个可重复且可扩展的框架，以指导更细致和用户感知的情感支持系统的开发和评估。

Title: Bridging the Data Gap: Creating a Hindi Text Summarization Dataset from the English XSUM

• Authors: Praveenkumar Katwe, RakeshChandra Balabantaray, Kaliprasad Vittala
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01543
• Pdf URL: https://arxiv.org/pdf/2601.01543
• Copy Paste: [[2601.01543]] Bridging the Data Gap: Creating a Hindi Text Summarization Dataset from the English XSUM(https://arxiv.org/abs/2601.01543)
• Keywords: language model, llm
• Abstract: Current advancements in Natural Language Processing (NLP) have largely favored resource-rich languages, leaving a significant gap in high-quality datasets for low-resource languages like Hindi. This scarcity is particularly evident in text summarization, where the development of robust models is hindered by a lack of diverse, specialized corpora. To address this disparity, this study introduces a cost-effective, automated framework for creating a comprehensive Hindi text summarization dataset. By leveraging the English Extreme Summarization (XSUM) dataset as a source, we employ advanced translation and linguistic adaptation techniques. To ensure high fidelity and contextual relevance, we utilize the Crosslingual Optimized Metric for Evaluation of Translation (COMET) for validation, supplemented by the selective use of Large Language Models (LLMs) for curation. The resulting dataset provides a diverse, multi-thematic resource that mirrors the complexity of the original XSUM corpus. This initiative not only provides a direct tool for Hindi NLP research but also offers a scalable methodology for democratizing NLP in other underserved languages. By reducing the costs associated with dataset creation, this work fosters the development of more nuanced, culturally relevant models in computational linguistics.
• 摘要：当前自然语言处理 (NLP) 的进步在很大程度上有利于资源丰富的语言，而对于印地语等资源匮乏的语言来说，高质量数据集存在巨大差距。这种稀缺性在文本摘要中尤其明显，由于缺乏多样化、专业的语料库，稳健模型的开发受到阻碍。为了解决这种差异，本研究引入了一种经济高效的自动化框架，用于创建全面的印地语文本摘要数据集。通过利用英语极限摘要 (XSUM) 数据集作为来源，我们采用了先进的翻译和语言适应技术。为了确保高保真度和上下文相关性，我们利用跨语言优化翻译评估指标 (COMET) 进行验证，并选择性地使用大型语言模型 (LLM) 进行管理。生成的数据集提供了多样化的多主题资源，反映了原始 XSUM 语料库的复杂性。该计划不仅为印地语 NLP 研究提供了直接工具，还提供了一种可扩展的方法，使其他服务不足的语言的 NLP 民主化。通过降低与数据集创建相关的成本，这项工作促进了计算语言学中更细致、与文化相关的模型的开发。

Title: HalluZig: Hallucination Detection using Zigzag Persistence

• Authors: Shreyas N. Samaga, Gilberto Gonzalez Arroyo, Tamal K. Dey
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01552
• Pdf URL: https://arxiv.org/pdf/2601.01552
• Copy Paste: [[2601.01552]] HalluZig: Hallucination Detection using Zigzag Persistence(https://arxiv.org/abs/2601.01552)
• Keywords: language model, llm, hallucination
• Abstract: The factual reliability of Large Language Models (LLMs) remains a critical barrier to their adoption in high-stakes domains due to their propensity to hallucinate. Current detection methods often rely on surface-level signals from the model's output, overlooking the failures that occur within the model's internal reasoning process. In this paper, we introduce a new paradigm for hallucination detection by analyzing the dynamic topology of the evolution of model's layer-wise attention. We model the sequence of attention matrices as a zigzag graph filtration and use zigzag persistence, a tool from Topological Data Analysis, to extract a topological signature. Our core hypothesis is that factual and hallucinated generations exhibit distinct topological signatures. We validate our framework, HalluZig, on multiple benchmarks, demonstrating that it outperforms strong baselines. Furthermore, our analysis reveals that these topological signatures are generalizable across different models and hallucination detection is possible only using structural signatures from partial network depth.
• 摘要：由于大型语言模型 (LLM) 容易产生幻觉，其事实可靠性仍然是其在高风险领域采用的关键障碍。当前的检测方法通常依赖于模型输出的表面信号，忽略了模型内部推理过程中发生的故障。在本文中，我们通过分析模型分层注意力演化的动态拓扑，引入了一种新的幻觉检测范式。我们将注意力矩阵序列建模为锯齿形图过滤，并使用锯齿形持久性（拓扑数据分析的工具）来提取拓扑签名。我们的核心假设是，事实世代和幻觉世代表现出不同的拓扑特征。我们在多个基准测试中验证了我们的框架 HalluZig，证明它的性能优于强大的基准。此外，我们的分析表明，这些拓扑特征可以在不同的模型中推广，并且仅使用部分网络深度的结构特征就可以进行幻觉检测。

Title: Steerability of Instrumental-Convergence Tendencies in LLMs

• Authors: Jakub Hoscilowicz
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01584
• Pdf URL: https://arxiv.org/pdf/2601.01584
• Copy Paste: [[2601.01584]] Steerability of Instrumental-Convergence Tendencies in LLMs(https://arxiv.org/abs/2601.01584)
• Keywords: llm, prompt
• Abstract: We examine two properties of AI systems: capability (what a system can do) and steerability (how reliably one can shift behavior toward intended outcomes). In our experiments, higher capability does not imply lower steerability. We distinguish between authorized steerability (builders reliably reaching intended behaviors) and unauthorized steerability (attackers eliciting disallowed behaviors). This distinction highlights a fundamental safety--security dilemma for open-weight AI models: safety requires high steerability to enforce control (e.g., stop/refuse), while security requires low steerability to prevent malicious actors from eliciting harmful behaviors. This tension is acute for open-weight models, which are currently highly steerable via common techniques such as fine-tuning and adversarial prompting. Using Qwen3 models (4B/30B; Base/Instruct/Thinking) and InstrumentalEval, we find that a short anti-instrumental prompt suffix sharply reduces outputs labeled as instrumental convergence (e.g., shutdown avoidance, deception, self-replication). For Qwen3-30B Instruct, convergence drops from 81.69% under a pro-instrumental suffix to 2.82% under an anti-instrumental suffix. Under anti-instrumental prompting, larger aligned models produce fewer convergence-labeled outputs than smaller ones (Instruct: 2.82% vs. 4.23%; Thinking: 4.23% vs. 9.86%). Code is available at this http URL.
• 摘要：我们研究人工智能系统的两个属性：能力（系统可以做什么）和可操纵性（如何可靠地将行为转向预期结果）。在我们的实验中，更高的能力并不意味着更低的可操纵性。我们区分授权的可操纵性（构建者可靠地达到预期行为）和未经授权的可操纵性（攻击者引发不允许的行为）。这种区别凸显了开放权重人工智能模型的基本安全困境：安全性需要高可操纵性来实施控制（例如，停止/拒绝），而安全性需要低可操纵性来防止恶意行为者引发有害行为。对于开放权重模型来说，这种紧张局势非常严重，目前开放权重模型可以通过微调和对抗性提示等常用技术进行高度操控。使用 Qwen3 模型（4B/30B；Base/Instruct/Thinking）和 InstrumentalEval，我们发现短的反工具提示后缀会急剧减少标记为工具收敛的输出（例如，避免关机、欺骗、自我复制）。对于 Qwen3-30B Instruct，收敛性从支持工具后缀下的 81.69% 下降到反工具后缀下的 2.82%。在反工具提示下，较大的对齐模型比较小的模型产生更少的收敛标记输出（指导：2.82% vs. 4.23%；思考：4.23% vs. 9.86%）。代码可从此 http URL 获取。

Title: JMedEthicBench: A Multi-Turn Conversational Benchmark for Evaluating Medical Safety in Japanese Large Language Models

• Authors: Junyu Liu, Zirui Li, Qian Niu, Zequn Zhang, Yue Xun, Wenlong Hou, Shujun Wang, Yusuke Iwasawa, Yutaka Matsuo, Kan Hatakeyama-Sato
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01627
• Pdf URL: https://arxiv.org/pdf/2601.01627
• Copy Paste: [[2601.01627]] JMedEthicBench: A Multi-Turn Conversational Benchmark for Evaluating Medical Safety in Japanese Large Language Models(https://arxiv.org/abs/2601.01627)
• Keywords: language model, llm, prompt
• Abstract: As Large Language Models (LLMs) are increasingly deployed in healthcare field, it becomes essential to carefully evaluate their medical safety before clinical use. However, existing safety benchmarks remain predominantly English-centric, and test with only single-turn prompts despite multi-turn clinical consultations. To address these gaps, we introduce JMedEthicBench, the first multi-turn conversational benchmark for evaluating medical safety of LLMs for Japanese healthcare. Our benchmark is based on 67 guidelines from the Japan Medical Association and contains over 50,000 adversarial conversations generated using seven automatically discovered jailbreak strategies. Using a dual-LLM scoring protocol, we evaluate 27 models and find that commercial models maintain robust safety while medical-specialized models exhibit increased vulnerability. Furthermore, safety scores decline significantly across conversation turns (median: 9.5 to 5.0, $p < 0.001$). Cross-lingual evaluation on both Japanese and English versions of our benchmark reveals that medical model vulnerabilities persist across languages, indicating inherent alignment limitations rather than language-specific factors. These findings suggest that domain-specific fine-tuning may accidentally weaken safety mechanisms and that multi-turn interactions represent a distinct threat surface requiring dedicated alignment strategies.
• 摘要：随着大型语言模型（LLM）越来越多地应用于医疗保健领域，在临床使用之前仔细评估其医疗安全性变得至关重要。然而，现有的安全基准仍然主要以英语为中心，尽管进行了多轮临床咨询，但仅使用单轮提示进行测试。为了解决这些差距，我们引入了 JMedEthicBench，这是第一个用于评估日本医疗保健法学硕士医疗安全的多回合对话基准。我们的基准测试基于日本医学会的 67 条指南，包含使用七种自动发现的越狱策略生成的 50,000 多个对抗性对话。使用双法学硕士评分协议，我们评估了 27 个模型，发现商业模型保持了强大的安全性，而医疗专业模型则表现出更大的脆弱性。此外，安全得分在对话轮次中显着下降（中位数：9.5 至 5.0，$p < 0.001$）。对我们基准的日语和英语版本的跨语言评估表明，跨语言的医学模型漏洞仍然存在，这表明固有的一致性限制，而不是特定于语言的因素。这些发现表明，特定领域的微调可能会意外削弱安全机制，并且多轮交互代表了需要专门的对齐策略的独特威胁面。

Title: Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage

• Authors: Jinwei Hu, Xinmiao Huang, Youcheng Sun, Yi Dong, Xiaowei Huang
• Subjects: cs.CL, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2601.01685
• Pdf URL: https://arxiv.org/pdf/2601.01685
• Copy Paste: [[2601.01685]] Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage(https://arxiv.org/abs/2601.01685)
• Keywords: language model, llm, prompt, agent
• Abstract: As large language models (LLMs) transition to autonomous agents synthesizing real-time information, their reasoning capabilities introduce an unexpected attack surface. This paper introduces a novel threat where colluding agents steer victim beliefs using only truthful evidence fragments distributed through public channels, without relying on covert communications, backdoors, or falsified documents. By exploiting LLMs' overthinking tendency, we formalize the first cognitive collusion attack and propose Generative Montage: a Writer-Editor-Director framework that constructs deceptive narratives through adversarial debate and coordinated posting of evidence fragments, causing victims to internalize and propagate fabricated conclusions. To study this risk, we develop CoPHEME, a dataset derived from real-world rumor events, and simulate attacks across diverse LLM families. Our results show pervasive vulnerability across 14 LLM families: attack success rates reach 74.4% for proprietary models and 70.6% for open-weights models. Counterintuitively, stronger reasoning capabilities increase susceptibility, with reasoning-specialized models showing higher attack success than base models or prompts. Furthermore, these false beliefs then cascade to downstream judges, achieving over 60% deception rates, highlighting a socio-technical vulnerability in how LLM-based agents interact with dynamic information environments. Our implementation and data are available at: this https URL.
• 摘要：随着大型语言模型 (LLM) 过渡到合成实时信息的自主代理，它们的推理能力引入了意想不到的攻击面。本文介绍了一种新的威胁，即共谋特工仅使用通过公共渠道分发的真实证据片段来引导受害者的信念，而不依赖秘密通信、后门或伪造文件。通过利用法学硕士的过度思考倾向，我们正式确定了第一个认知共谋攻击，并提出了生成蒙太奇：一种作家-编辑-导演框架，通过对抗性辩论和协调发布证据片段构建欺骗性叙述，导致受害者内化和传播捏造的结论。为了研究这种风险，我们开发了 CoPHEME（一个源自现实世界谣言事件的数据集），并模拟了不同 LLM 系列的攻击。我们的结果显示 14 个 LLM 系列中普遍存在漏洞：专有模型的攻击成功率达到 74.4%，开放权重模型的攻击成功率达到 70.6%。与直觉相反，更强的推理能力会增加敏感性，推理专用模型比基本模型或提示显示出更高的攻击成功率。此外，这些错误的信念随后会级联到下游法官，欺骗率超过 60%，凸显了基于 LLM 的代理与动态信息环境交互的社会技术漏洞。我们的实施和数据可在以下位置获得：此 https URL。

Title: A Training-Free Large Reasoning Model-based Knowledge Tracing Framework for Unified Prediction and Prescription

• Authors: Unggi Lee, Joo Young Kim, Ran Ju, Minyoung Jung, Jeyeon Eo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01708
• Pdf URL: https://arxiv.org/pdf/2601.01708
• Copy Paste: [[2601.01708]] A Training-Free Large Reasoning Model-based Knowledge Tracing Framework for Unified Prediction and Prescription(https://arxiv.org/abs/2601.01708)
• Keywords: language model, llm
• Abstract: Knowledge Tracing (KT) aims to estimate a learner's evolving mastery based on interaction histories. Recent studies have explored Large Language Models (LLMs) for KT via autoregressive nature, but such approaches typically require fine-tuning and exhibit unstable or near-random performance. Moreover, prior KT systems primarily focus on prediction and rely on multi-stage pipelines for feedback and recommendation, resulting in increased system complexity and resources. To address this gap, we propose Thinking-KT, a training-free KT framework that incorporates Test-Time Scaling (TTS), enabling even small LLMs to achieve competitive KT performance. Moreover, in this framework, a small LLM can jointly perform KT prediction, personalized feedback generation, and learning recommendation in a unified output without degrading prediction accuracy. Beyond performance, we present the systematic analysis of reasoning traces in KT. Our results demonstrate that TTS is a critical yet underexplored factor in LLM-based KT, and that small LLMs can serve as unified ITS engines.
• 摘要：知识追踪（KT）旨在根据交互历史来估计学习者不断发展的掌握程度。最近的研究通过自回归性质探索了 KT 的大型语言模型 (LLM)，但此类方法通常需要微调，并且表现出不稳定或接近随机的性能。此外，现有的KT系统主要侧重于预测，并依赖多级管道进行反馈和推荐，导致系统复杂性和资源增加。为了解决这一差距，我们提出了 Thinking-KT，这是一种无需培训的 KT 框架，它结合了测试时间缩放 (TTS)，即使是小型法学硕士也能实现具有竞争力的 KT 表现。此外，在这个框架中，小型LLM可以在统一输出中联合执行KT预测、个性化反馈生成和学习推荐，而不会降低预测精度。除了性能之外，我们还对 KT 中的推理轨迹进行了系统分析。我们的结果表明，TTS 是基于 LLM 的 KT 中一个关键但尚未充分开发的因素，并且小型 LLM 可以作为统一的 ITS 引擎。

Title: K-EXAONE Technical Report

• Authors: Eunbi Choi, Kibong Choi, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Hyunjik Jo, Joonkee Kim, Seonghwan Kim, Soyeon Kim, Sunkyoung Kim, Yireun Kim, Yongil Kim, Haeju Lee, Jinsik Lee, Kyungmin Lee, Sangha Park, Heuiyeen Yeen, Hwan Chang, Stanley Jungkyu Choi, Yejin Choi, Jiwon Ham, Kijeong Jeon, Geunyeong Jeong, Gerrard Jeongwon Jo, Yonghwan Jo, Jiyeon Jung, Naeun Kang, Dohoon Kim, Euisoon Kim, Hayeon Kim, Hyosang Kim, Hyunseo Kim, Jieun Kim, Minu Kim, Myoungshin Kim, Unsol Kim, Youchul Kim, YoungJin Kim, Chaeeun Lee, Chaeyoon Lee, Changhun Lee, Dahm Lee, Edward Hwayoung Lee, Honglak Lee, Jinsang Lee, Jiyoung Lee, Sangeun Lee, Seungwon Lim, Solji Lim, Woohyung Lim, Chanwoo Moon, Jaewoo Park, Jinho Park, Yongmin Park, Hyerin Seo, Wooseok Seo, Yongwoo Song, Sejong Yang, Sihoon Yang, Chang En Yea, Sihyuk Yi, Chansik Yoon, Dongkeun Yoon, Sangyeon Yoon, Hyeongu Yun
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01739
• Pdf URL: https://arxiv.org/pdf/2601.01739
• Copy Paste: [[2601.01739]] K-EXAONE Technical Report(https://arxiv.org/abs/2601.01739)
• Keywords: language model, agent
• Abstract: This technical report presents K-EXAONE, a large-scale multilingual language model developed by LG AI Research. K-EXAONE is built on a Mixture-of-Experts architecture with 236B total parameters, activating 23B parameters during inference. It supports a 256K-token context window and covers six languages: Korean, English, Spanish, German, Japanese, and Vietnamese. We evaluate K-EXAONE on a comprehensive benchmark suite spanning reasoning, agentic, general, Korean, and multilingual abilities. Across these evaluations, K-EXAONE demonstrates performance comparable to open-weight models of similar size. K-EXAONE, designed to advance AI for a better life, is positioned as a powerful proprietary AI foundation model for a wide range of industrial and research applications.
• 摘要：本技术报告介绍了 LG AI Research 开发的大规模多语言语言模型 K-EXAONE。 K-EXAONE 基于 Mixture-of-Experts 架构构建，总参数为 236B，在推理过程中激活 23B 参数。它支持 256K-token 上下文窗口，涵盖六种语言：韩语、英语、西班牙语、德语、日语和越南语。我们根据涵盖推理、代理、一般、韩语和多语言能力的综合基准套件来评估 K-EXAONE。在这些评估中，K-EXAONE 表现出与类似尺寸的开放式重量模型相当的性能。 K-EXAONE 旨在推进人工智能，让生活更美好，定位为强大的专有人工智能基础模型，适用于广泛的工业和研究应用。

Title: Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulation

• Authors: Meiman Xiao, Ante Wang, Qingguo Hu, Zhongjian Miao, Huangjun Shen, Longyue Wang, Weihua Luo, Jinsong Su
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01768
• Pdf URL: https://arxiv.org/pdf/2601.01768
• Copy Paste: [[2601.01768]] Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulation(https://arxiv.org/abs/2601.01768)
• Keywords: language model, llm
• Abstract: Precisely controlling the length of generated text is a common requirement in real-world applications. However, despite significant advancements in following human instructions, Large Language Models (LLMs) still struggle with this task. In this work, we demonstrate that LLMs often fail to accurately measure input text length, leading to poor adherence to length constraints. To address this issue, we propose a novel length regulation approach that incorporates dynamic length feedback during generation, enabling adaptive adjustments to meet target lengths. Experiments on summarization and biography tasks show our training-free approach significantly improves precision in achieving target token, word, or sentence counts without compromising quality. Additionally, we demonstrate that further supervised fine-tuning allows our method to generalize effectively to broader text-generation tasks.
• 摘要：精确控制生成文本的长度是实际应用中的常见要求。然而，尽管在遵循人类指令方面取得了显着进步，大型语言模型 (LLM) 仍然难以完成这项任务。在这项工作中，我们证明法学硕士通常无法准确测量输入文本长度，从而导致无法遵守长度限制。为了解决这个问题，我们提出了一种新颖的长度调节方法，该方法在生成过程中结合了动态长度反馈，从而能够进行自适应调整以满足目标长度。摘要和传记任务的实验表明，我们的免训练方法显着提高了实现目标标记、单词或句子计数的精度，而不会影响质量。此外，我们证明了进一步的监督微调使我们的方法能够有效地推广到更广泛的文本生成任务。

Title: CSCBench: A PVC Diagnostic Benchmark for Commodity Supply Chain Reasoning

• Authors: Yaxin Cui, Yuanqiang Zeng, Jiapeng Yan, Keling Lin, Kai Ji, Jianhui Zeng, Sheng Zhang, Xin Luo, Binzhu Su, Chaolai Shen, Jiahao Yu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01825
• Pdf URL: https://arxiv.org/pdf/2601.01825
• Copy Paste: [[2601.01825]] CSCBench: A PVC Diagnostic Benchmark for Commodity Supply Chain Reasoning(https://arxiv.org/abs/2601.01825)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) have achieved remarkable success in general benchmarks, yet their competence in commodity supply chains (CSCs) -- a domain governed by institutional rule systems and feasibility constraints -- remains under-explored. CSC decisions are shaped jointly by process stages (e.g., planning, procurement, delivery), variety-specific rules (e.g., contract specifications and delivery grades), and reasoning depth (from retrieval to multi-step analysis and decision selection). We introduce CSCBench, a 2.3K+ single-choice benchmark for CSC reasoning, instantiated through our PVC 3D Evaluation Framework (Process, Variety, and Cognition). The Process axis aligns tasks with SCOR+Enable; the Variety axis operationalizes commodity-specific rule systems under coupled material-information-financial constraints, grounded in authoritative exchange guidebooks/rulebooks and industry reports; and the Cognition axis follows Bloom's revised taxonomy. Evaluating representative LLMs under a direct prompting setting, we observe strong performance on the Process and Cognition axes but substantial degradation on the Variety axis, especially on Freight Agreements. CSCBench provides a diagnostic yardstick for measuring and improving LLM capabilities in this high-stakes domain.
• 摘要：大型语言模型 (LLM) 在一般基准方面取得了显着的成功，但它们在商品供应链 (CSC)（受制度规则系统和可行性约束管辖的领域）方面的能力仍有待探索。 CSC 决策由流程阶段（例如规划、采购、交付）、品种特定规则（例如合同规格和交付等级）和推理深度（从检索到多步骤分析和决策选择）共同形成。我们推出了 CSCBench，这是一个用于 CSC 推理的 2.3K+ 单选基准，通过我们的 PVC 3D 评估框架（流程、多样性和认知）进行实例化。流程轴将任务与 SCOR+Enable 对齐；品种轴在物质-信息-财务耦合约束下实施特定商品的规则系统，以权威的交易指南/规则手册和行业报告为基础；认知轴遵循布鲁姆修订后的分类法。在直接提示设置下评估代表性法学硕士，我们观察到在过程和认知轴上表现强劲，但在品种轴上大幅退化，特别是在货运协议方面。 CSCBench 提供了一个诊断标准，用于衡量和提高这个高风险领域的 LLM 能力。

Title: Aspect Extraction from E-Commerce Product and Service Reviews

• Authors: Valiant Lance D. Dionela, Fatima Kriselle S. Dy, Robin James M. Hombrebueno, Aaron Rae M. Nicolas, Charibeth K. Cheng, Raphael W. Gonda
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2601.01827
• Pdf URL: https://arxiv.org/pdf/2601.01827
• Copy Paste: [[2601.01827]] Aspect Extraction from E-Commerce Product and Service Reviews(https://arxiv.org/abs/2601.01827)
• Keywords: language model, llm
• Abstract: Aspect Extraction (AE) is a key task in Aspect-Based Sentiment Analysis (ABSA), yet it remains difficult to apply in low-resource and code-switched contexts like Taglish, a mix of Tagalog and English commonly used in Filipino e-commerce reviews. This paper introduces a comprehensive AE pipeline designed for Taglish, combining rule-based, large language model (LLM)-based, and fine-tuning techniques to address both aspect identification and extraction. A Hierarchical Aspect Framework (HAF) is developed through multi-method topic modeling, along with a dual-mode tagging scheme for explicit and implicit aspects. For aspect identification, four distinct models are evaluated: a Rule-Based system, a Generative LLM (Gemini 2.0 Flash), and two Fine-Tuned Gemma-3 1B models trained on different datasets (Rule-Based vs. LLM-Annotated). Results indicate that the Generative LLM achieved the highest performance across all tasks (Macro F1 0.91), demonstrating superior capability in handling implicit aspects. In contrast, the fine-tuned models exhibited limited performance due to dataset imbalance and architectural capacity constraints. This work contributes a scalable and linguistically adaptive framework for enhancing ABSA in diverse, code-switched environments.
• 摘要：方面提取 (AE) 是基于方面的情感分析 (ABSA) 中的一项关键任务，但它仍然很难应用于资源匮乏和代码转换的环境，例如 Taglish（菲律宾电子商务评论中常用的他加禄语和英语的混合体）。本文介绍了为 Taglish 设计的综合 AE 管道，结合了基于规则、基于大语言模型 (LLM) 和微调技术来解决方面识别和提取问题。分层方面框架（HAF）是通过多方法主题建模以及用于显式和隐式方面的双模式标记方案开发的。对于方面识别，评估了四种不同的模型：基于规则的系统、生成式 LLM (Gemini 2.0 Flash) 和在不同数据集上训练的两个微调 Gemma-3 1B 模型（基于规则与 LLM 带注释）。结果表明，生成式法学硕士在所有任务中取得了最高的性能（宏观 F1 0.91），展示了处理隐性方面的卓越能力。相比之下，由于数据集不平衡和架构容量限制，微调模型的性能有限。这项工作提供了一个可扩展且语言自适应的框架，用于在不同的代码交换环境中增强 ABSA。

Title: Emergent Introspective Awareness in Large Language Models

• Authors: Jack Lindsey
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01828
• Pdf URL: https://arxiv.org/pdf/2601.01828
• Copy Paste: [[2601.01828]] Emergent Introspective Awareness in Large Language Models(https://arxiv.org/abs/2601.01828)
• Keywords: language model
• Abstract: We investigate whether large language models can introspect on their internal states. It is difficult to answer this question through conversation alone, as genuine introspection cannot be distinguished from confabulations. Here, we address this challenge by injecting representations of known concepts into a model's activations, and measuring the influence of these manipulations on the model's self-reported states. We find that models can, in certain scenarios, notice the presence of injected concepts and accurately identify them. Models demonstrate some ability to recall prior internal representations and distinguish them from raw text inputs. Strikingly, we find that some models can use their ability to recall prior intentions in order to distinguish their own outputs from artificial prefills. In all these experiments, Claude Opus 4 and 4.1, the most capable models we tested, generally demonstrate the greatest introspective awareness; however, trends across models are complex and sensitive to post-training strategies. Finally, we explore whether models can explicitly control their internal representations, finding that models can modulate their activations when instructed or incentivized to "think about" a concept. Overall, our results indicate that current language models possess some functional introspective awareness of their own internal states. We stress that in today's models, this capacity is highly unreliable and context-dependent; however, it may continue to develop with further improvements to model capabilities.
• 摘要：我们研究大型语言模型是否可以反思其内部状态。仅通过对话很难回答这个问题，因为真正的内省和虚构是分不开的。在这里，我们通过将已知概念的表示注入模型的激活中，并测量这些操作对模型自我报告状态的影响来解决这一挑战。我们发现，在某些情况下，模型可以注意到注入概念的存在并准确识别它们。模型表现出一定的能力来回忆先前的内部表征并将其与原始文本输入区分开来。引人注目的是，我们发现一些模型可以利用它们回忆先验意图的能力来区分它们自己的输出和人工预填充。在所有这些实验中，我们测试的最有能力的模型 Claude Opus 4 和 4.1 通常表现出最大的内省意识；然而，跨模型的趋势是复杂的，并且对训练后策略敏感。最后，我们探索模型是否可以显式控制其内部表征，发现模型可以在被指示或激励“思考”概念时调节其激活。总的来说，我们的结果表明当前的语言模型对其自身内部状态具有一定的功能性内省意识。我们强调，在当今的模型中，这种能力非常不可靠并且依赖于环境。然而，随着模型功能的进一步改进，它可能会继续发展。

Title: Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries

• Authors: Yusuke Ide, Adam Nohejl, Joshua Tanner, Hitomi Yanaka, Christopher Lindsay, Taro Watanabe
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01842
• Pdf URL: https://arxiv.org/pdf/2601.01842
• Copy Paste: [[2601.01842]] Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries(https://arxiv.org/abs/2601.01842)
• Keywords: llm
• Abstract: We study dictionary definition generation (DDG), i.e., the generation of non-contextualized definitions for given headwords. Dictionary definitions are an essential resource for learning word senses, but manually creating them is costly, which motivates us to automate the process. Specifically, we address learner's dictionary definition generation (LDDG), where definitions should consist of simple words. First, we introduce a reliable evaluation approach for DDG, based on our new evaluation criteria and powered by an LLM-as-a-judge. To provide reference definitions for the evaluation, we also construct a Japanese dataset in collaboration with a professional lexicographer. Validation results demonstrate that our evaluation approach agrees reasonably well with human annotators. Second, we propose an LDDG approach via iterative simplification with an LLM. Experimental results indicate that definitions generated by our approach achieve high scores on our criteria while maintaining lexical simplicity.
• 摘要：我们研究字典定义生成（DDG），即给定词条的非上下文定义的生成。字典定义是学习词义的重要资源，但手动创建它们的成本很高，这促使我们将过程自动化。具体来说，我们解决了学习者词典定义生成（LDDG）的问题，其中定义应由简单的单词组成。首先，我们根据新的评估标准，并由法学硕士法官提供支持，为 DDG 引入可靠的评估方法。为了为评估提供参考定义，我们还与专业词典编纂者合作构建了日语数据集。验证结果表明我们的评估方法与人类注释者相当一致。其次，我们通过法学硕士的迭代简化提出了一种 LDDG 方法。实验结果表明，我们的方法生成的定义在我们的标准上获得了高分，同时保持了词汇的简单性。

Title: Judging with Personality and Confidence: A Study on Personality-Conditioned LLM Relevance Assessment

• Authors: Nuo Chen, Hanpei Fang, Piaohong Wang, Jiqun Liu, Tetsuya Sakai, Xiao-Ming Wu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2601.01862
• Pdf URL: https://arxiv.org/pdf/2601.01862
• Copy Paste: [[2601.01862]] Judging with Personality and Confidence: A Study on Personality-Conditioned LLM Relevance Assessment(https://arxiv.org/abs/2601.01862)
• Keywords: language model, llm, prompt
• Abstract: Recent studies have shown that prompting can enable large language models (LLMs) to simulate specific personality traits and produce behaviors that align with those traits. However, there is limited understanding of how these simulated personalities influence critical web search decisions, specifically relevance assessment. Moreover, few studies have examined how simulated personalities impact confidence calibration, specifically the tendencies toward overconfidence or underconfidence. This gap exists even though psychological literature suggests these biases are trait-specific, often linking high extraversion to overconfidence and high neuroticism to underconfidence. To address this gap, we conducted a comprehensive study evaluating multiple LLMs, including commercial models and open-source models, prompted to simulate Big Five personality traits. We tested these models across three test collections (TREC DL 2019, TREC DL 2020, and LLMJudge), collecting two key outputs for each query-document pair: a relevance judgment and a self-reported confidence score. The findings show that personalities such as low agreeableness consistently align more closely with human labels than the unprompted condition. Additionally, low conscientiousness performs well in balancing the suppression of both overconfidence and underconfidence. We also observe that relevance scores and confidence distributions vary systematically across different personalities. Based on the above findings, we incorporate personality-conditioned scores and confidence as features in a random forest classifier. This approach achieves performance that surpasses the best single-personality condition on a new dataset (TREC DL 2021), even with limited training data. These findings highlight that personality-derived confidence offers a complementary predictive signal, paving the way for more reliable and human-aligned LLM evaluators.
• 摘要：最近的研究表明，提示可以使大型语言模型（LLM）能够模拟特定的人格特征并产生与这些特征相符的行为。然而，对于这些模拟人物如何影响关键的网络搜索决策，特别是相关性评估，人们的了解还很有限。此外，很少有研究探讨模拟人格如何影响信心校准，特别是过度自信或自信不足的倾向。尽管心理学文献表明这些偏见是特定于特质的，但这种差距仍然存在，通常将高度外向性与过度自信联系起来，将高度神经质与不自信联系起来。为了解决这一差距，我们进行了一项全面的研究，评估了多个法学硕士，包括商业模型和开源模型，促使模拟大五人格特质。我们在三个测试集合（TREC DL 2019、TREC DL 2020 和 LLMJudge）中测试了这些模型，为每个查询文档对收集两个关键输出：相关性判断和自我报告的置信度得分。研究结果表明，与无提示的情况相比，低宜人性等性格始终更符合人类标签。此外，低责任感在平衡过度自信和不自信的抑制方面表现良好。我们还观察到，不同性格的相关性得分和置信度分布存在系统性差异。基于上述发现，我们将个性条件得分和置信度作为随机森林分类器中的特征。即使训练数据有限，这种方法的性能也超过了新数据集 (TREC DL 2021) 上的最佳单人格条件。这些发现强调，源自个性的信心提供了互补的预测信号，为更可靠、更人性化的法学硕士评估者铺平了道路。

Title: DermoGPT: Open Weights and Open Data for Morphology-Grounded Dermatological Reasoning MLLMs

• Authors: Jinghan Ru, Siyuan Yan, Yuguo Yin, Yuexian Zou, Zongyuan Ge
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01868
• Pdf URL: https://arxiv.org/pdf/2601.01868
• Copy Paste: [[2601.01868]] DermoGPT: Open Weights and Open Data for Morphology-Grounded Dermatological Reasoning MLLMs(https://arxiv.org/abs/2601.01868)
• Keywords: language model, gpt, llm
• Abstract: Multimodal Large Language Models (MLLMs) show promise for medical applications, yet progress in dermatology lags due to limited training data, narrow task coverage, and lack of clinically-grounded supervision that mirrors expert diagnostic workflows. We present a comprehensive framework to address these gaps. First, we introduce DermoInstruct, a large-scale morphology-anchored instruction corpus comprising 211,243 images and 772,675 trajectories across five task formats, capturing the complete diagnostic pipeline from morphological observation and clinical reasoning to final diagnosis. Second, we establish DermoBench, a rigorous benchmark evaluating 11 tasks across four clinical axes: Morphology, Diagnosis, Reasoning, and Fairness, including a challenging subset of 3,600 expert-verified open-ended instances and human performance baselines. Third, we develop DermoGPT, a dermatology reasoning MLLM trained via supervised fine-tuning followed by our Morphologically-Anchored Visual-Inference-Consistent (MAVIC) reinforcement learning objective, which enforces consistency between visual observations and diagnostic conclusions. At inference, we deploy Confidence-Consistency Test-time adaptation (CCT) for robust predictions. Experiments show DermoGPT significantly outperforms 16 representative baselines across all axes, achieving state-of-the-art performance while substantially narrowing the human-AI gap. DermoInstruct, DermoBench and DermoGPT will be made publicly available at this https URL upon acceptance.
• 摘要：多模态大语言模型（MLLM）显示出医疗应用的前景，但由于训练数据有限、任务覆盖范围狭窄以及缺乏反映专家诊断工作流程的基于临床的监督，皮肤病学的进展滞后。我们提出了一个全面的框架来解决这些差距。首先，我们介绍 DermoInstruct，一个大规模形态学锚定指令语料库，包含跨五种任务格式的 211,243 张图像和 772,675 个轨迹，捕获从形态学观察和临床推理到最终诊断的完整诊断流程。其次，我们建立了 DermoBench，这是一个严格的基准，评估跨四个临床轴的 11 项任务：形态学、诊断、推理和公平性，包括 3,600 个经过专家验证的开放式实例和人类表现基线的具有挑战性的子集。第三，我们开发了 DermoGPT，这是一种皮肤病学推理 MLLM，通过监督微调进行训练，然后是我们的形态锚定视觉推理一致 (MAVIC) 强化学习目标，它强制视觉观察和诊断结论之间的一致性。在推理时，我们部署置信度一致性测试时间适应（CCT）来进行稳健的预测。实验表明，DermoGPT 在所有轴上均显着优于 16 个代表性基线，实现了最先进的性能，同时大幅缩小了人类与人工智能的差距。 DermoInstruct、DermoBench 和 DermoGPT 将在接受后通过此 https URL 公开提供。

Title: Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents

• Authors: Yi Yu, Liuyi Yao, Yuexiang Xie, Qingquan Tan, Jiaqi Feng, Yaliang Li, Libing Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01885
• Pdf URL: https://arxiv.org/pdf/2601.01885
• Copy Paste: [[2601.01885]] Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents(https://arxiv.org/abs/2601.01885)
• Keywords: language model, llm, agent
• Abstract: Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows, making effective memory management critical. Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components, relying on heuristics or auxiliary controllers, which limits adaptability and end-to-end optimization. In this paper, we propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy. AgeMem exposes memory operations as tool-based actions, enabling the LLM agent to autonomously decide what and when to store, retrieve, update, summarize, or discard information. To train such unified behaviors, we propose a three-stage progressive reinforcement learning strategy and design a step-wise GRPO to address sparse and discontinuous rewards induced by memory operations. Experiments on five long-horizon benchmarks demonstrate that AgeMem consistently outperforms strong memory-augmented baselines across multiple LLM backbones, achieving improved task performance, higher-quality long-term memory, and more efficient context usage.
• 摘要：由于上下文窗口有限，大型语言模型 (LLM) 代理在长视野推理中面临根本限制，因此有效的内存管理至关重要。现有方法通常将长期记忆（LTM）和短期记忆（STM）作为单独的组件处理，依赖启发式或辅助控制器，这限制了适应性和端到端优化。在本文中，我们提出了 Agentic Memory (AgeMem)，这是一个将 LTM 和 STM 管理直接集成到代理策略中的统一框架。 AgeMem 将内存操作公开为基于工具的操作，使 LLM 代理能够自主决定存储、检索、更新、总结或丢弃信息的内容和时间。为了训练这种统一的行为，我们提出了一种三阶段渐进强化学习策略，并设计了一个逐步的 GRPO 来解决由记忆操作引起的稀疏和不连续的奖励。五个长期基准测试的实验表明，AgeMem 在多个 LLM 主干上始终优于强大的记忆增强基线，实现了更高的任务性能、更高质量的长期记忆和更高效的上下文使用。

Title: Tackling the Inherent Difficulty of Noise Filtering in RAG

• Authors: Jingyu Liu, Jiaen Lin, Yong Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.01896
• Pdf URL: https://arxiv.org/pdf/2601.01896
• Copy Paste: [[2601.01896]] Tackling the Inherent Difficulty of Noise Filtering in RAG(https://arxiv.org/abs/2601.01896)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) has become a widely adopted approach to enhance Large Language Models (LLMs) by incorporating external knowledge and reducing hallucinations. However, noisy or irrelevant documents are often introduced during RAG, potentially degrading performance and even causing hallucinated outputs. While various methods have been proposed to filter out such noise, we argue that identifying irrelevant information from retrieved content is inherently difficult and limited number of transformer layers can hardly solve this. Consequently, retrievers fail to filter out irrelevant documents entirely. Therefore, LLMs must be robust against such noise, but we demonstrate that standard fine-tuning approaches are often ineffective in enabling the model to selectively utilize relevant information while ignoring irrelevant content due to the structural constraints of attention patterns. To address this, we propose a novel fine-tuning method designed to enhance the model's ability to distinguish between relevant and irrelevant information within retrieved documents. Extensive experiments across multiple benchmarks show that our approach significantly improves the robustness and performance of LLMs.
• 摘要：检索增强生成（RAG）已成为一种广泛采用的方法，通过整合外部知识和减少幻觉来增强大型语言模型（LLM）。然而，RAG 期间经常会引入嘈杂或不相关的文档，可能会降低性能，甚至导致幻觉输出。虽然已经提出了各种方法来滤除此类噪声，但我们认为从检索到的内容中识别不相关信息本质上是困难的，并且有限数量的转换器层很难解决这个问题。因此，检索器无法完全过滤掉不相关的文档。因此，法学硕士必须对此类噪声具有鲁棒性，但我们证明，由于注意力模式的结构限制，标准微调方法通常无法有效地使模型选择性地利用相关信息，同时忽略不相关的内容。为了解决这个问题，我们提出了一种新颖的微调方法，旨在增强模型区分检索到的文档中相关信息和不相关信息的能力。跨多个基准的广泛实验表明，我们的方法显着提高了法学硕士的稳健性和性能。

Title: CSF: Contrastive Semantic Features for Direct Multilingual Sign Language Generation

• Authors: Tran Sy Bao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01964
• Pdf URL: https://arxiv.org/pdf/2601.01964
• Copy Paste: [[2601.01964]] CSF: Contrastive Semantic Features for Direct Multilingual Sign Language Generation(https://arxiv.org/abs/2601.01964)
• Keywords: agent
• Abstract: Sign language translation systems typically require English as an intermediary language, creating barriers for non-English speakers in the global deaf community. We present Canonical Semantic Form (CSF), a language-agnostic semantic representation framework that enables direct translation from any source language to sign language without English mediation. CSF decomposes utterances into nine universal semantic slots: event, intent, time, condition, agent, object, location, purpose, and modifier. A key contribution is our comprehensive condition taxonomy comprising 35 condition types across eight semantic categories, enabling nuanced representation of conditional expressions common in everyday communication. We train a lightweight transformer-based extractor (0.74 MB) that achieves 99.03% average slot extraction accuracy across four typologically diverse languages: English, Vietnamese, Japanese, and French. The model demonstrates particularly strong performance on condition classification (99.4% accuracy) despite the 35-class complexity. With inference latency of 3.02ms on CPU, our approach enables real-time sign language generation in browser-based applications. We release our code, trained models, and multilingual dataset to support further research in accessible sign language technology.
• 摘要：手语翻译系统通常需要英语作为中间语言，这为全球聋人社区中非英语人士造成了障碍。我们提出了规范语义形式（CSF），这是一种与语言无关的语义表示框架，可以直接从任何源语言翻译成手语，而无需英语中介。 CSF 将话语分解为九个通用语义槽：事件、意图、时间、条件、主体、对象、位置、目的和修饰语。一个关键贡献是我们的综合条件分类法，包括 8 个语义类别的 35 种条件类型，能够细致地表示日常交流中常见的条件表达式。我们训练了一个基于 Transformer 的轻量级提取器 (0.74 MB)，该提取器在四种类型不同的语言（英语、越南语、日语和法语）中实现了 99.03% 的平均槽提取准确率。尽管有 35 类复杂性，该模型在条件分类方面表现出特别强大的性能（99.4% 的准确率）。 CPU 上的推理延迟为 3.02 毫秒，我们的方法可以在基于浏览器的应用程序中实时生成手语。我们发布代码、经过训练的模型和多语言数据集，以支持无障碍手语技术的进一步研究。

Title: Hidden State Poisoning Attacks against Mamba-based Language Models

• Authors: Alexandre Le Mercier, Chris Develder, Thomas Demeester
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.01972
• Pdf URL: https://arxiv.org/pdf/2601.01972
• Copy Paste: [[2601.01972]] Hidden State Poisoning Attacks against Mamba-based Language Models(https://arxiv.org/abs/2601.01972)
• Keywords: language model, prompt
• Abstract: State space models (SSMs) like Mamba offer efficient alternatives to Transformer-based language models, with linear time complexity. Yet, their adversarial robustness remains critically unexplored. This paper studies the phenomenon whereby specific short input phrases induce a partial amnesia effect in such models, by irreversibly overwriting information in their hidden states, referred to as a Hidden State Poisoning Attack (HiSPA). Our benchmark RoBench25 allows evaluating a model's information retrieval capabilities when subject to HiSPAs, and confirms the vulnerability of SSMs against such attacks. Even a recent 52B hybrid SSM-Transformer model from the Jamba family collapses on RoBench25 under optimized HiSPA triggers, whereas pure Transformers do not. We also observe that HiSPA triggers significantly weaken the Jamba model on the popular Open-Prompt-Injections benchmark, unlike pure Transformers. Finally, our interpretability study reveals patterns in Mamba's hidden layers during HiSPAs that could be used to build a HiSPA mitigation system. The full code and data to reproduce the experiments can be found at this https URL.
• 摘要：像 Mamba 这样的状态空间模型 (SSM) 为基于 Transformer 的语言模型提供了有效的替代方案，具有线性时间复杂度。然而，它们的对抗鲁棒性仍未得到严格的探索。本文研究了特定的短输入短语通过不可逆地覆盖隐藏状态中的信息而在此类模型中引起部分遗忘效应的现象，称为隐藏状态中毒攻击（HiSPA）。我们的基准 RoBench25 允许评估模型在受到 HiSPA 影响时的信息检索能力，并确认 SSM 针对此类攻击的脆弱性。即使是最近 Jamba 系列的 52B 混合 SSM-Transformer 模型也会在优化的 HiSPA 触发器下在 RoBench25 上崩溃，而纯 Transformer 则不会。我们还观察到，与纯粹的 Transformers 不同，HiSPA 触发器显着削弱了流行的 Open-Prompt-Injections 基准上的 Jamba 模型。最后，我们的可解释性研究揭示了 HiSPA 期间 Mamba 隐藏层的模式，可用于构建 HiSPA 缓解系统。可以在此 https URL 中找到重现实验的完整代码和数据。

Title: Surprisal and Metaphor Novelty: Moderate Correlations and Divergent Scaling Effects

• Authors: Omar Momen, Emilie Sitter, Berenike Herrmann, Sina Zarrieß
• Subjects: cs.CL, cs.AI, cs.IT
• Abstract URL: https://arxiv.org/abs/2601.02015
• Pdf URL: https://arxiv.org/pdf/2601.02015
• Copy Paste: [[2601.02015]] Surprisal and Metaphor Novelty: Moderate Correlations and Divergent Scaling Effects(https://arxiv.org/abs/2601.02015)
• Keywords: language model
• Abstract: Novel metaphor comprehension involves complex semantic processes and linguistic creativity, making it an interesting task for studying language models (LMs). This study investigates whether surprisal, a probabilistic measure of predictability in LMs, correlates with different metaphor novelty datasets. We analyse surprisal from 16 LM variants on corpus-based and synthetic metaphor novelty datasets. We explore a cloze-style surprisal method that conditions on full-sentence context. Results show that LMs yield significant moderate correlations with scores/labels of metaphor novelty. We further identify divergent scaling patterns: on corpus-based data, correlation strength decreases with model size (inverse scaling effect), whereas on synthetic data it increases (Quality-Power Hypothesis). We conclude that while surprisal can partially account for annotations of metaphor novelty, it remains a limited metric of linguistic creativity.
• 摘要：新颖的隐喻理解涉及复杂的语义过程和语言创造力，这使其成为研究语言模型（LM）的一项有趣任务。本研究调查了惊讶（LM 中可预测性的概率度量）是否与不同的隐喻新颖性数据集相关。我们在基于语料库和合成隐喻新奇数据集上分析了 16 个 LM 变体的惊喜。我们探索了一种以完整句子上下文为条件的完形填空式的惊喜方法。结果表明，语言模型与隐喻新颖性的分数/标签产生显着的中等相关性。我们进一步确定了不同的缩放模式：在基于语料库的数据上，相关强度随着模型大小的增加而减小（逆缩放效应），而在合成数据上，相关强度则增加（质量-功效假设）。我们的结论是，虽然惊讶可以部分解释隐喻新颖性的注释，但它仍然是衡量语言创造力的有限指标。

Title: Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Literal Extraction, Logical Inference, and Hallucination Risks in Long-Context LLMs

• Authors: Amirali Ebrahimzadeh, Seyyed M. Salili
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.02023
• Pdf URL: https://arxiv.org/pdf/2601.02023
• Copy Paste: [[2601.02023]] Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Literal Extraction, Logical Inference, and Hallucination Risks in Long-Context LLMs(https://arxiv.org/abs/2601.02023)
• Keywords: language model, gpt, llm, long context, hallucination, prompt, chat, retrieval-augmented generation
• Abstract: Large language models (LLMs) increasingly support very long input contexts. Yet it remains unclear how reliably they extract and infer information at scale. Performance varies with context length and strongly interacts with how information is distributed in real-world corpora. Motivated by these observations, we study how fact placement, corpus-level fact distributions, and Don't Make It Up prompts influence model behavior. We introduce an extended needle-in-a-haystack benchmark across four production-scale models: Gemini-2.5-flash, ChatGPT-5-mini, Claude-4.5-haiku, and Deepseek-v3.2-chat. Unlike prior work, we separately evaluate literal extraction, logical inference, and hallucination risk. Our study considers both positional effects and realistic distributions of evidence across long contexts, as well as prompts that explicitly discourage fabrication. We find that longer contexts alone do not guarantee better performance and can be detrimental when relevant evidence is diluted or widely dispersed. Performance varies substantially across models: some show severe degradation under realistic conditions, while others remain more robust at longer context lengths. Anti-hallucination (AH) instructions can make some models overly conservative, sharply reducing accuracy in literal extraction and logical inference. While we do not directly compare retrieval-augmented generation (RAG) and cache-augmented generation (CAG), our results suggest many failures stem from ineffective context utilization. Models often struggle to identify and prioritize relevant information even when it is present. These findings have direct practical implications, as enterprise workflows increasingly involve pasting large volumes of unfiltered documents into LLM prompts. Effective context length and model-specific robustness to long contexts are therefore critical for reliable LLM deployment in research and business.
• 摘要：大型语言模型 (LLM) 越来越多地支持很长的输入上下文。然而，目前尚不清楚它们大规模提取和推断信息的可靠性如何。性能随上下文长度的不同而变化，并且与信息在现实世界语料库中的分布方式密切相关。在这些观察的推动下，我们研究了事实放置、语料库级事实分布和“不要弥补”提示如何影响模型行为。我们引入了跨四种生产规模模型的扩展大海捞针基准测试：Gemini-2.5-flash、ChatGPT-5-mini、Claude-4.5-haiku 和 Deepseek-v3.2-chat。与之前的工作不同，我们分别评估文字提取、逻辑推理和幻觉风险。我们的研究考虑了长期背景下证据的位置效应和现实分布，以及明确阻止捏造的提示。我们发现，单独较长的上下文并不能保证更好的性能，并且当相关证据被稀释或广泛分散时，可能会产生不利影响。不同模型的性能差异很大：一些模型在现实条件下表现出严重退化，而另一些模型在较长的上下文长度下仍然更加稳健。反幻觉（AH）指令可能会使某些模型过于保守，从而大幅降低文字提取和逻辑推理的准确性。虽然我们没有直接比较检索增强生成（RAG）和缓存增强生成（CAG），但我们的结果表明许多失败源于无效的上下文利用。即使相关信息存在，模型也常常难以识别相关信息并确定其优先级。这些发现具有直接的实际意义，因为企业工作流程越来越多地涉及将大量未经过滤的文档粘贴到 LLM 提示中。因此，有效的上下文长度和特定于模型的长上下文稳健性对于研究和商业中可靠的法学硕士部署至关重要。

Title: Cost-Efficient Cross-Lingual Retrieval-Augmented Generation for Low-Resource Languages: A Case Study in Bengali Agricultural Advisory

• Authors: Md. Asif Hossain, Nabil Subhan, Mantasha Rahman Mahi, Jannatul Ferdous Nabila
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.02065
• Pdf URL: https://arxiv.org/pdf/2601.02065
• Copy Paste: [[2601.02065]] Cost-Efficient Cross-Lingual Retrieval-Augmented Generation for Low-Resource Languages: A Case Study in Bengali Agricultural Advisory(https://arxiv.org/abs/2601.02065)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Access to reliable agricultural advisory remains limited in many developing regions due to a persistent language barrier: authoritative agricultural manuals are predominantly written in English, while farmers primarily communicate in low-resource local languages such as Bengali. Although recent advances in Large Language Models (LLMs) enable natural language interaction, direct generation in low-resource languages often exhibits poor fluency and factual inconsistency, while cloud-based solutions remain cost-prohibitive. This paper presents a cost-efficient, cross-lingual Retrieval-Augmented Generation (RAG) framework for Bengali agricultural advisory that emphasizes factual grounding and practical deployability. The proposed system adopts a translation-centric architecture in which Bengali user queries are translated into English, enriched through domain-specific keyword injection to align colloquial farmer terminology with scientific nomenclature, and answered via dense vector retrieval over a curated corpus of English agricultural manuals (FAO, IRRI). The generated English response is subsequently translated back into Bengali to ensure accessibility. The system is implemented entirely using open-source models and operates on consumer-grade hardware without reliance on paid APIs. Experimental evaluation demonstrates reliable source-grounded responses, robust rejection of out-of-domain queries, and an average end-to-end latency below 20 seconds. The results indicate that cross-lingual retrieval combined with controlled translation offers a practical and scalable solution for agricultural knowledge access in low-resource language settings
• 摘要：由于持续存在的语言障碍，许多发展中地区获得可靠的农业咨询的机会仍然有限：权威的农业手册主要用英语编写，而农民主要使用孟加拉语等资源匮乏的当地语言进行交流。尽管大型语言模型 (LLM) 的最新进展实现了自然语言交互，但低资源语言的直接生成通常表现出流畅性差和事实不一致，而基于云的解决方案的成本仍然过高。本文提出了一种经济高效、跨语言的孟加拉农业咨询框架，强调事实基础和实际可部署性。所提出的系统采用以翻译为中心的架构，其中孟加拉语用户查询被翻译成英语，通过特定领域的关键字注入进行丰富，以使口语农民术语与科学术语保持一致，并通过对英语农业手册（FAO、IRRI）精选语料库进行密集向量检索来回答。生成的英语响应随后被翻译回孟加拉语，以确保可访问性。该系统完全使用开源模型实现，并在消费级硬件上运行，不依赖付费 API。实验评估证明了可靠的基于源的响应、对域外查询的稳健拒绝以及低于 20 秒的平均端到端延迟。结果表明，跨语言检索与受控翻译相结合，为低资源语言环境中的农业知识获取提供了实用且可扩展的解决方案

Title: Deferred Commitment Decoding for Diffusion Language Models with Confidence-Aware Sliding Windows

• Authors: Yingte Shu, Yuchuan Tian, Chao Xu, Yunhe Wang, Hanting Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.02076
• Pdf URL: https://arxiv.org/pdf/2601.02076
• Copy Paste: [[2601.02076]] Deferred Commitment Decoding for Diffusion Language Models with Confidence-Aware Sliding Windows(https://arxiv.org/abs/2601.02076)
• Keywords: language model
• Abstract: Diffusion language models (DLMs) have recently emerged as a strong alternative to autoregressive models by enabling parallel text generation. To improve inference efficiency and KV-cache compatibility, prior work commonly adopts block-based diffusion, decoding tokens block by block. However, this paradigm suffers from a structural limitation that we term Boundary-Induced Context Truncation (BICT): undecoded tokens near block boundaries are forced to commit without access to nearby future context, even when such context could substantially reduce uncertainty. This limitation degrades decoding confidence and generation quality, especially for tasks requiring precise reasoning, such as mathematical problem solving and code generation. We propose Deferred Commitment Decoding (DCD), a novel, training-free decoding strategy that mitigates this issue. DCD maintains a confidence-aware sliding window over masked tokens, resolving low-uncertainty tokens early while deferring high-uncertainty tokens until sufficient contextual evidence becomes available. This design enables effective bidirectional information flow within the decoding window without sacrificing efficiency. Extensive experiments across multiple diffusion language models, benchmarks, and caching configurations show that DCD improves generation accuracy by 1.39% with comparable time on average compared to fixed block-based diffusion methods, with the most significant improvement reaching 9.0%. These results demonstrate that deferring token commitment based on uncertainty is a simple yet effective principle for improving both the quality and efficiency of diffusion language model decoding.
• 摘要：扩散语言模型（DLM）最近通过支持并行文本生成而成为自回归模型的强大替代品。为了提高推理效率和KV缓存兼容性，先前的工作通常采用基于块的扩散，逐块解码令牌。然而，这种范式受到了我们称之为边界诱导上下文截断（BICT）的结构限制：块边界附近的未解码令牌被迫提交，而无法访问附近的未来上下文，即使这样的上下文可以大大减少不确定性。这种限制会降低解码置信度和生成质量，特别是对于需要精确推理的任务，例如数学问题解决和代码生成。我们提出了延迟承诺解码（DCD），这是一种新颖的、免训练的解码策略，可以缓解这个问题。 DCD 在屏蔽令牌上维护一个信任感知滑动窗口，尽早解决低不确定性令牌，同时推迟高不确定性令牌，直到获得足够的上下文证据。这种设计能够在解码窗口内实现有效的双向信息流，而不会牺牲效率。跨多种扩散语言模型、基准测试和缓存配置的大量实验表明，与固定的基于块的扩散方法相比，DCD 将生成精度提高了 1.39%，平均时间相当，其中最显着的提升达到 9.0%。这些结果表明，基于不确定性推迟令牌承诺是提高扩散语言模型解码质量和效率的简单而有效的原则。

Title: DeCode: Decoupling Content and Delivery for Medical QA

• Authors: Po-Jen Ko, Chen-Han Tsai, Yu-Shao Peng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.02123
• Pdf URL: https://arxiv.org/pdf/2601.02123
• Copy Paste: [[2601.02123]] DeCode: Decoupling Content and Delivery for Medical QA(https://arxiv.org/abs/2601.02123)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) exhibit strong medical knowledge and can generate factually accurate responses. However, existing models often fail to account for individual patient contexts, producing answers that are clinically correct yet poorly aligned with patients' needs. In this work, we introduce DeCode, a training-free, model-agnostic framework that adapts existing LLMs to produce contextualized answers in clinical settings. We evaluate DeCode on OpenAI HealthBench, a comprehensive and challenging benchmark designed to assess clinical relevance and validity of LLM responses. DeCode improves the previous state of the art from $28.4\%$ to $49.8\%$, corresponding to a $75\%$ relative improvement. Experimental results suggest the effectiveness of DeCode in improving clinical question answering of LLMs.
• 摘要：大型语言模型 (LLM) 展示了丰富的医学知识，并且可以生成事实上准确的响应。然而，现有模型通常无法考虑患者的具体情况，产生的答案在临床上是正确的，但与患者的需求不太相符。在这项工作中，我们引入了 DeCode，这是一种无需训练、与模型无关的框架，可以调整现有的法学硕士以在临床环境中生成情境化的答案。我们在 OpenAI HealthBench 上评估 DeCode，这是一个全面且具有挑战性的基准，旨在评估法学硕士回答的临床相关性和有效性。 DeCode 将之前的技术水平从 $28.4\%$ 提高到 $49.8\%$，相当于 $75\%$ 的相对改进。实验结果表明 DeCode 在改善法学硕士临床问答方面的有效性。

Title: Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation

• Authors: Steffen Freisinger, Philipp Seeberger, Thomas Ranzenberger, Tobias Bocklet, Korbinian Riedhammer
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2601.02128
• Pdf URL: https://arxiv.org/pdf/2601.02128
• Copy Paste: [[2601.02128]] Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation(https://arxiv.org/abs/2601.02128)
• Keywords: language model, prompt
• Abstract: Segmenting speech transcripts into thematic sections benefits both downstream processing and users who depend on written text for accessibility. We introduce a novel approach to hierarchical topic segmentation in transcripts, generating multi-level tables of contents that capture both topic and subtopic boundaries. We compare zero-shot prompting and LoRA fine-tuning on large language models, while also exploring the integration of high-level speech pause features. Evaluations on English meeting recordings and multilingual lecture transcripts (Portuguese, German) show significant improvements over established topic segmentation baselines. Additionally, we adapt a common evaluation measure for multi-level segmentation, taking into account all hierarchical levels within one metric.
• 摘要：将语音记录分割成主题部分对下游处理和依赖书面文本进行访问的用户都有好处。我们引入了一种在转录本中进行分层主题分割的新颖方法，生成捕获主题和子主题边界的多级目录。我们在大型语言模型上比较了零样本提示和 LoRA 微调，同时还探索了高级语音暂停功能的集成。对英语会议录音和多语言讲座笔录（葡萄牙语、德语）的评估表明，较已建立的主题分割基线有显着改进。此外，我们采用了一种用于多级细分的通用评估措施，考虑到一个指标内的所有层次级别。

Title: Routing by Analogy: kNN-Augmented Expert Assignment for Mixture-of-Experts

• Authors: Boxuan Lyu, Soichiro Murakami, Hidetaka Kamigaito, Peinan Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.02144
• Pdf URL: https://arxiv.org/pdf/2601.02144
• Copy Paste: [[2601.02144]] Routing by Analogy: kNN-Augmented Expert Assignment for Mixture-of-Experts(https://arxiv.org/abs/2601.02144)
• Keywords: language model
• Abstract: Mixture-of-Experts (MoE) architectures scale large language models efficiently by employing a parametric "router" to dispatch tokens to a sparse subset of experts. Typically, this router is trained once and then frozen, rendering routing decisions brittle under distribution shifts. We address this limitation by introducing kNN-MoE, a retrieval-augmented routing framework that reuses optimal expert assignments from a memory of similar past cases. This memory is constructed offline by directly optimizing token-wise routing logits to maximize the likelihood on a reference set. Crucially, we use the aggregate similarity of retrieved neighbors as a confidence-driven mixing coefficient, thus allowing the method to fall back to the frozen router when no relevant cases are found. Experiments show kNN-MoE outperforms zero-shot baselines and rivals computationally expensive supervised fine-tuning.
• 摘要：专家混合 (MoE) 架构通过使用参数“路由器”将令牌分派给稀疏的专家子集，有效地扩展大型语言模型。通常，该路由器经过一次训练，然后就被冻结，导致路由决策在分布变化时变得脆弱。我们通过引入 kNN-MoE 来解决这一限制，kNN-MoE 是一种检索增强的路由框架，可重用来自过去类似案例记忆的最佳专家分配。该内存是通过直接优化 token-wise 路由 logits 来离线构建的，以最大化参考集的可能性。至关重要的是，我们使用检索到的邻居的聚合相似度作为置信驱动的混合系数，从而允许该方法在没有找到相关情况时回退到冻结路由器。实验表明 kNN-MoE 的性能优于零样本基线，并且可以与计算成本高昂的监督微调相媲美。

Title: FormationEval, an open multiple-choice benchmark for petroleum geoscience

• Authors: Almaz Ermilov
• Subjects: cs.CL, cs.AI, cs.LG, physics.geo-ph
• Abstract URL: https://arxiv.org/abs/2601.02158
• Pdf URL: https://arxiv.org/pdf/2601.02158
• Copy Paste: [[2601.02158]] FormationEval, an open multiple-choice benchmark for petroleum geoscience(https://arxiv.org/abs/2601.02158)
• Keywords: language model
• Abstract: This paper presents FormationEval, an open multiple-choice question benchmark for evaluating language models on petroleum geoscience and subsurface disciplines. The dataset contains 505 questions across seven domains including petrophysics, petroleum geology and reservoir engineering, derived from three authoritative sources using a reasoning model with detailed instructions and a concept-based approach that avoids verbatim copying of copyrighted text. Each question includes source metadata to support traceability and audit. The evaluation covers 72 models from major providers including OpenAI, Anthropic, Google, Meta and open-weight alternatives. The top performers achieve over 97\% accuracy, with Gemini 3 Pro Preview reaching 99.8\%, while tier and domain gaps persist. Among open-weight models, GLM-4.7 leads at 98.6\%, with several DeepSeek, Llama, Qwen and Mistral models also exceeding 93\%. The performance gap between open-weight and closed models is narrower than expected, with several lower-cost open-weight models exceeding 90\% accuracy. Petrophysics emerges as the most challenging domain across all models, while smaller models show wider performance variance. Residual length bias in the dataset (correct answers tend to be longer) is documented along with bias mitigation strategies applied during construction. The benchmark, evaluation code and results are publicly available.
• 摘要：本文提出了 FormationEval，这是一个开放式多项选择题基准，用于评估石油地球科学和地下学科的语言模型。该数据集包含涵盖岩石物理学、石油地质学和油藏工程等七个领域的 505 个问题，这些问题来自三个权威来源，使用带有详细说明的推理模型和基于概念的方法，避免逐字复制受版权保护的文本。每个问题都包含源元数据以支持可追溯性和审计。该评估涵盖了来自主要提供商的 72 个模型，包括 OpenAI、Anthropic、Google、Meta 和开放权重替代方案。表现最佳的产品准确率超过 97%，Gemini 3 Pro Preview 达到 99.8%，但层级和领域差距仍然存在。在开放权重模型中，GLM-4.7 以 98.6% 领先，几个 DeepSeek、Llama、Qwen 和 Mistral 模型也超过 93%。开放式重量模型和封闭式模型之间的性能差距比预期要窄，一些成本较低的开放式重量模型的准确率超过了 90%。岩石物理学成为所有模型中最具挑战性的领域，而较小的模型则表现出更大的性能差异。记录数据集中的剩余长度偏差（正确答案往往更长）以及构建过程中应用的偏差缓解策略。基准、评估代码和结果都是公开的。

Title: Confidence Estimation for LLMs in Multi-turn Interactions

• Authors: Caiqi Zhang, Ruihan Yang, Xiaochen Zhu, Chengzu Li, Tiancheng Hu, Yijiang River Dong, Deqing Yang, Nigel Collier
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.02179
• Pdf URL: https://arxiv.org/pdf/2601.02179
• Copy Paste: [[2601.02179]] Confidence Estimation for LLMs in Multi-turn Interactions(https://arxiv.org/abs/2601.02179)
• Keywords: language model, llm, hallucination, agent
• Abstract: While confidence estimation is a promising direction for mitigating hallucinations in Large Language Models (LLMs), current research dominantly focuses on single-turn settings. The dynamics of model confidence in multi-turn conversations, where context accumulates and ambiguity is progressively resolved, remain largely unexplored. Reliable confidence estimation in multi-turn settings is critical for many downstream applications, such as autonomous agents and human-in-the-loop systems. This work presents the first systematic study of confidence estimation in multi-turn interactions, establishing a formal evaluation framework grounded in two key desiderata: per-turn calibration and monotonicity of confidence as more information becomes available. To facilitate this, we introduce novel metrics, including a length-normalized Expected Calibration Error (InfoECE), and a new "Hinter-Guesser" paradigm for generating controlled evaluation datasets. Our experiments reveal that widely-used confidence techniques struggle with calibration and monotonicity in multi-turn dialogues. We propose P(Sufficient), a logit-based probe that achieves comparatively better performance, although the task remains far from solved. Our work provides a foundational methodology for developing more reliable and trustworthy conversational agents.
• 摘要：虽然置信度估计是减轻大型语言模型（LLM）中的幻觉的一个有前途的方向，但当前的研究主要集中在单轮设置上。多轮对话中模型置信度的动态，其中上下文不断累积，歧义逐渐得到解决，但在很大程度上仍未得到探索。多轮设置中的可靠置信度估计对于许多下游应用至关重要，例如自主代理和人机交互系统。这项工作首次系统地研究了多轮交互中的置信度估计，建立了一个基于两个关键需求的正式评估框架：每轮校准和随着更多信息的可用而产生的置信度的单调性。为了促进这一点，我们引入了新颖的指标，包括长度归一化的预期校准误差（InfoECE），以及用于生成受控评估数据集的新的“Hinter-Guesser”范例。我们的实验表明，广泛使用的置信技术在多轮对话中面临着校准和单调性的问题。我们提出了 P(Sufficient)，一种基于 logit 的探针，它实现了相对更好的性能，尽管任务还远未解决。我们的工作为开发更可靠、更值得信赖的对话代理提供了基础方法。

Title: Toward Global Large Language Models in Medicine

• Authors: Rui Yang, Huitao Li, Weihao Xuan, Heli Qi, Xin Li, Kunyu Yu, Yingjian Chen, Rongrong Wang, Jacques Behmoaras, Tianxi Cai, Bibhas Chakraborty, Qingyu Chen, Lionel Tim-Ee Cheng, Marie-Louise Damwanza, Chido Dzinotyiwei, Aosong Feng, Chuan Hong, Yusuke Iwasawa, Yuhe Ke, Linah Kitala, Taehoon Ko, Jisan Lee, Irene Li, Jonathan Chong Kai Liew, Hongfang Liu, Lian Leng Low, Edison Marrese-Taylor, Yutaka Matsuo, Isheanesu Misi, Yilin Ning, Jasmine Chiat Ling Ong, Marcus Eng Hock Ong, Enrico Petretto, Hossein Rouhizadeh, Abiram Sandralegar, Oren Schreier, Iain Bee Huat Tan, Patrick Tan, Daniel Shu Wei Ting, Junjue Wang, Chunhua Weng, Matthew Yu Heng Wong, Fang Wu, Yunze Xiao, Xuhai Xu, Qingcheng Zeng, Zhuo Zheng, Yifan Peng, Douglas Teodoro, Nan Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.02186
• Pdf URL: https://arxiv.org/pdf/2601.02186
• Copy Paste: [[2601.02186]] Toward Global Large Language Models in Medicine(https://arxiv.org/abs/2601.02186)
• Keywords: language model, llm
• Abstract: Despite continuous advances in medical technology, the global distribution of health care resources remains uneven. The development of large language models (LLMs) has transformed the landscape of medicine and holds promise for improving health care quality and expanding access to medical information globally. However, existing LLMs are primarily trained on high-resource languages, limiting their applicability in global medical scenarios. To address this gap, we constructed GlobMed, a large multilingual medical dataset, containing over 500,000 entries spanning 12 languages, including four low-resource languages. Building on this, we established GlobMed-Bench, which systematically assesses 56 state-of-the-art proprietary and open-weight LLMs across multiple multilingual medical tasks, revealing significant performance disparities across languages, particularly for low-resource languages. Additionally, we introduced GlobMed-LLMs, a suite of multilingual medical LLMs trained on GlobMed, with parameters ranging from 1.7B to 8B. GlobMed-LLMs achieved an average performance improvement of over 40% relative to baseline models, with a more than threefold increase in performance on low-resource languages. Together, these resources provide an important foundation for advancing the equitable development and application of LLMs globally, enabling broader language communities to benefit from technological advances.
• 摘要：尽管医疗技术不断进步，但全球医疗资源分布仍然不平衡。大语言模型 (LLM) 的发展改变了医学格局，并有望提高医疗保健质量和扩大全球医疗信息的获取。然而，现有的法学硕士主要接受高资源语言的培训，限制了它们在全球医疗场景中的适用性。为了弥补这一差距，我们构建了 GlobMed，这是一个大型多语言医学数据集，包含 12 种语言的超过 500,000 个条目，其中包括四种资源匮乏的语言。在此基础上，我们建立了 GlobMed-Bench，它系统地评估了 56 个最先进的专有和开放权重法学硕士，涵盖多种多语言医疗任务，揭示了不同语言之间的显着性能差异，特别是对于资源匮乏的语言。此外，我们还引入了 GlobMed-LLM，这是一套在 GlobMed 上接受培训的多语言医学法学硕士，参数范围从 1.7B 到 8B。相对于基线模型，GlobMed-LLM 的平均性能提高了 40% 以上，在低资源语言上的性能提高了三倍多。这些资源共同为推动全球法学硕士的公平发展和应用奠定了重要基础，使更广泛的语言社区能够从技术进步中受益。

Title: From XAI to Stories: A Factorial Study of LLM-Generated Explanation Quality

• Authors: Fabian Lukassen, Jan Herrmann, Christoph Weisser, Benjamin Saefken, Thomas Kneib
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.02224
• Pdf URL: https://arxiv.org/pdf/2601.02224
• Copy Paste: [[2601.02224]] From XAI to Stories: A Factorial Study of LLM-Generated Explanation Quality(https://arxiv.org/abs/2601.02224)
• Keywords: language model, gpt, llm, prompt, chain-of-thought
• Abstract: Explainable AI (XAI) methods like SHAP and LIME produce numerical feature attributions that remain inaccessible to non expert users. Prior work has shown that Large Language Models (LLMs) can transform these outputs into natural language explanations (NLEs), but it remains unclear which factors contribute to high-quality explanations. We present a systematic factorial study investigating how Forecasting model choice, XAI method, LLM selection, and prompting strategy affect NLE quality. Our design spans four models (XGBoost (XGB), Random Forest (RF), Multilayer Perceptron (MLP), and SARIMAX - comparing black-box Machine-Learning (ML) against classical time-series approaches), three XAI conditions (SHAP, LIME, and a no-XAI baseline), three LLMs (GPT-4o, Llama-3-8B, DeepSeek-R1), and eight prompting strategies. Using G-Eval, an LLM-as-a-judge evaluation method, with dual LLM judges and four evaluation criteria, we evaluate 660 explanations for time-series forecasting. Our results suggest that: (1) XAI provides only small improvements over no-XAI baselines, and only for expert audiences; (2) LLM choice dominates all other factors, with DeepSeek-R1 outperforming GPT-4o and Llama-3; (3) we observe an interpretability paradox: in our setting, SARIMAX yielded lower NLE quality than ML models despite higher prediction accuracy; (4) zero-shot prompting is competitive with self-consistency at 7-times lower cost; and (5) chain-of-thought hurts rather than helps.
• 摘要：SHAP 和 LIME 等可解释的人工智能 (XAI) 方法产生的数字特征属性对于非专家用户来说仍然无法访问。先前的工作表明，大型语言模型（LLM）可以将这些输出转换为自然语言解释（NLE），但仍不清楚哪些因素有助于高质量的解释。我们提出了一项系统的因子研究，调查预测模型选择、XAI 方法、LLM 选择和提示策略如何影响 NLE 质量。我们的设计涵盖四种模型（XGBoost (XGB)、随机森林 (RF)、多层感知器 (MLP) 和 SARIMAX - 将黑盒机器学习 (ML) 与经典时间序列方法进行比较）、三种 XAI 条件（SHAP、LIME 和无 XAI 基线）、三种 LLM（GPT-4o、Llama-3-8B、DeepSeek-R1）和八种提示策略。使用 G-Eval（一种以法学硕士为法官的评估方法），采用双法学硕士法官和四个评估标准，我们评估了 660 个时间序列预测的解释。我们的结果表明：(1) XAI 仅比无 XAI 基线提供了很小的改进，并且仅针对专家受众； (2) LLM选择主导所有其他因素，DeepSeek-R1优于GPT-4o和Llama-3； (3) 我们观察到一个可解释性悖论：在我们的设置中，尽管预测精度更高，但 SARIMAX 的 NLE 质量低于 ML 模型； (4) 零样本提示在成本降低7倍的情况下具有自我一致性的竞争力； (5) 思维链只会带来伤害而不是带来帮助。

Title: CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models

• Authors: Yihao Liang, Ze Wang, Hao Chen, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Emad Barsoum, Zicheng Liu, Niraj K. Jha
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.02236
• Pdf URL: https://arxiv.org/pdf/2601.02236
• Copy Paste: [[2601.02236]] CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models(https://arxiv.org/abs/2601.02236)
• Keywords: language model
• Abstract: Autoregressive large language models achieve strong results on many benchmarks, but decoding remains fundamentally latency-limited by sequential dependence on previously generated tokens. Diffusion language models (DLMs) promise parallel generation but suffer from a fundamental static-to-dynamic misalignment: Training optimizes local transitions under fixed schedules, whereas efficient inference requires adaptive "long-jump" refinements through unseen states. Our goal is to enable highly parallel decoding for DLMs with low number of function evaluations while preserving generation quality. To achieve this, we propose CD4LM, a framework that decouples training from inference via Discrete-Space Consistency Distillation (DSCD) and Confidence-Adaptive Decoding (CAD). Unlike standard objectives, DSCD trains a student to be trajectory-invariant, mapping diverse noisy states directly to the clean distribution. This intrinsic robustness enables CAD to dynamically allocate compute resources based on token confidence, aggressively skipping steps without the quality collapse typical of heuristic acceleration. On GSM8K, CD4LM matches the LLaDA baseline with a 5.18x wall-clock speedup; across code and math benchmarks, it strictly dominates the accuracy-efficiency Pareto frontier, achieving a 3.62x mean speedup while improving average accuracy. Code is available at this https URL
• 摘要：自回归大型语言模型在许多基准测试中取得了很好的结果，但解码从根本上仍然受到对先前生成的标记的顺序依赖性的延迟限制。扩散语言模型 (DLM) 承诺并行生成，但存在基本的静态到动态错位：训练在固定时间表下优化局部转换，而高效推理则需要通过未见过的状态进行自适应“长跳跃”细化。我们的目标是通过少量函数评估实现 DLM 的高度并行解码，同时保持生成质量。为了实现这一目标，我们提出了 CD4LM，这是一个通过离散空间一致性蒸馏（DSCD）和置信自适应解码（CAD）将训练与推理分离的框架。与标准目标不同，DSCD 训练学生轨迹不变，将不同的噪声状态直接映射到干净的分布。这种内在的鲁棒性使 CAD 能够根据令牌置信度动态分配计算资源，积极跳过步骤，而不会出现启发式加速典型的质量崩溃。在 GSM8K 上，CD4LM 与 LLaDA 基线相匹配，挂钟加速比为 5.18 倍；在代码和数学基准测试中，它严格控制了精度-效率 Pareto 前沿，实现了 3.62 倍的平均加速，同时提高了平均精度。代码可在此 https URL 获取

Title: pdfQA: Diverse, Challenging, and Realistic Question Answering over PDFs

• Authors: Tobias Schimanski, Imene Kolli, Jingwei Ni, Yu Fan, Ario Saeid Vaghefi, Elliott Ash, Markus Leippold
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2601.02285
• Pdf URL: https://arxiv.org/pdf/2601.02285
• Copy Paste: [[2601.02285]] pdfQA: Diverse, Challenging, and Realistic Question Answering over PDFs(https://arxiv.org/abs/2601.02285)
• Keywords: llm
• Abstract: PDFs are the second-most used document type on the internet (after HTML). Yet, existing QA datasets commonly start from text sources or only address specific domains. In this paper, we present pdfQA, a multi-domain 2K human-annotated (real-pdfQA) and 2K synthetic dataset (syn-pdfQA) differentiating QA pairs in ten complexity dimensions (e.g., file type, source modality, source position, answer type). We apply and evaluate quality and difficulty filters on both datasets, obtaining valid and challenging QA pairs. We answer the questions with open-source LLMs, revealing existing challenges that correlate with our complexity dimensions. pdfQA presents a basis for end-to-end QA pipeline evaluation, testing diverse skill sets and local optimizations (e.g., in information retrieval or parsing).
• 摘要：PDF 是互联网上第二常用的文档类型（仅次于 HTML）。然而，现有的 QA 数据集通常从文本源开始或仅针对特定领域。在本文中，我们提出了 pdfQA，一个多域 2K 人工注释 (real-pdfQA) 和 2K 合成数据集 (syn-pdfQA)，在十个复杂维度（例如文件类型、源模态、源位置、答案类型）中区分 QA 对。我们在两个数据集上应用和评估质量和难度过滤器，获得有效且具有挑战性的 QA 对。我们通过开源法学硕士回答了这些问题，揭示了与我们的复杂性维度相关的现有挑战。 pdfQA 为端到端 QA 管道评估、测试不同技能集和局部优化（例如，在信息检索或解析中）提供了基础。

Title: Power-of-Two Quantization-Aware-Training (PoT-QAT) in Large Language Models (LLMs)

• Authors: Mahmoud Elgenedy
• Subjects: cs.CL, eess.SP
• Abstract URL: https://arxiv.org/abs/2601.02298
• Pdf URL: https://arxiv.org/pdf/2601.02298
• Copy Paste: [[2601.02298]] Power-of-Two Quantization-Aware-Training (PoT-QAT) in Large Language Models (LLMs)(https://arxiv.org/abs/2601.02298)
• Keywords: language model, gpt, llm
• Abstract: In Large Language Models (LLMs), the number of parameters has grown exponentially in the past few years, e.g., from 1.5 billion parameters in GPT-2 to 175 billion in GPT-3 to possibly more than trillion in higher versions. This raises a significant challenge for implementation, especially for Edge devices. Unlike cloud computing, memory and processing power for Edge devices are very limited, which necessitates developing novel ideas to make such applications feasible. In this work, we investigate compressing weights with a special quantization that limits numbers to only power-of-two (PoT). This helps save a huge amount of memory as only exponents need to be stored, more importantly, it significantly reduces processing power by replacing costly multiplication with low cost bit shifting. To overcome performance loss due to this strict quantization, we investigate Quantization Aware Training (QAT) to enhance performance through additional training. Results on GPT-2 124M show a major enhancement for quantized PoT model after additional training, with a perplexity enhancement of 66% and BERT-Score loss to baseline GPT-2 of 1%. The memory saving is estimated to be 87.5% while the inference speed is expected to be 3-10x faster with PoT quantization versus full-precision.
• 摘要：在大型语言模型 (LLM) 中，参数数量在过去几年中呈指数级增长，例如，从 GPT-2 中的 15 亿个参数到 GPT-3 中的 1750 亿个参数，再到更高版本中可能超过万亿个参数。这给实施带来了重大挑战，特别是对于边缘设备。与云计算不同，边缘设备的内存和处理能力非常有限，这需要开发新颖的想法以使此类应用程序可行。在这项工作中，我们研究了通过特殊量化来压缩权重，该量化将数字限制为仅二次幂 (PoT)。这有助于节省大量内存，因为只需要存储指数，更重要的是，它通过用低成本位移代替昂贵的乘法来显着降低处理能力。为了克服这种严格量化造成的性能损失，我们研究了量化感知训练（QAT），以通过额外的训练来提高性能。 GPT-2 124M 上的结果显示，经过额外训练后，量化 PoT 模型得到了重大增强，困惑度增强了 66%，BERT-Score 相对于基线 GPT-2 的损失为 1%。 PoT 量化与全精度相比，预计可节省 87.5% 的内存，而推理速度预计将快 3-10 倍。

Title: Estimating Text Temperature

• Authors: Nikolay Mikhaylovskiy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.02320
• Pdf URL: https://arxiv.org/pdf/2601.02320
• Copy Paste: [[2601.02320]] Estimating Text Temperature(https://arxiv.org/abs/2601.02320)
• Keywords: language model, llm
• Abstract: Autoregressive language models typically use temperature parameter at inference to shape the probability distribution and control the randomness of the text generated. After the text was generated, this parameter can be estimated using maximum likelihood approach. Following it, we propose a procedure to estimate the temperature of any text, including ones written by humans, with respect to a given language model. We evaluate the temperature estimation capability of a wide selection of small-to-medium LLMs. We then use the best-performing Qwen3 14B to estimate temperatures of popular corpora.
• 摘要：自回归语言模型通常在推理时使用温度参数来塑造概率分布并控制生成文本的随机性。生成文本后，可以使用最大似然法来估计该参数。接下来，我们提出了一个程序来估计任何文本（包括人类编写的文本）相对于给定语言模型的温度。我们评估了多种中小型法学硕士的温度估计能力。然后，我们使用性能最好的 Qwen3 14B 来估计流行语料库的温度。

Title: Robust Persona-Aware Toxicity Detection with Prompt Optimization and Learned Ensembling

• Authors: Berk Atil, Rebecca J. Passonneau, Ninareh Mehrabi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2601.02337
• Pdf URL: https://arxiv.org/pdf/2601.02337
• Copy Paste: [[2601.02337]] Robust Persona-Aware Toxicity Detection with Prompt Optimization and Learned Ensembling(https://arxiv.org/abs/2601.02337)
• Keywords: language model, llm, prompt
• Abstract: Toxicity detection is inherently subjective, shaped by the diverse perspectives and social priors of different demographic groups. While ``pluralistic'' modeling as used in economics and the social sciences aims to capture perspective differences across contexts, current Large Language Model (LLM) prompting techniques have different results across different personas and base models. In this work, we conduct a systematic evaluation of persona-aware toxicity detection, showing that no single prompting method, including our proposed automated prompt optimization strategy, uniformly dominates across all model-persona pairs. To exploit complementary errors, we explore ensembling four prompting variants and propose a lightweight meta-ensemble: an SVM over the 4-bit vector of prompt predictions. Our results demonstrate that the proposed SVM ensemble consistently outperforms individual prompting methods and traditional majority-voting techniques, achieving the strongest overall performance across diverse personas. This work provides one of the first systematic comparisons of persona-conditioned prompting for toxicity detection and offers a robust method for pluralistic evaluation in subjective NLP tasks.
• 摘要：毒性检测本质上是主观的，受到不同人口群体的不同观点和社会先验的影响。虽然经济学和社会科学中使用的“多元”建模旨在捕捉不同背景下的视角差异，但当前的大语言模型（LLM）提示技术在不同的角色和基础模型中具有不同的结果。在这项工作中，我们对角色感知毒性检测进行了系统评估，表明没有任何一种提示方法（包括我们提出的自动提示优化策略）在所有模型角色对中统一占主导地位。为了利用互补错误，我们探索了四种提示变体的集成，并提出了一种轻量级元集成：基于 4 位提示预测向量的 SVM。我们的结果表明，所提出的 SVM 集成始终优于个人提示方法和传统的多数投票技术，在不同角色中实现了最强的整体性能。这项工作为毒性检测的角色条件提示提供了第一个系统比较，并为主观 NLP 任务中的多元评估提供了一种稳健的方法。