2026-01-23

Title: Entropy-Tree: Tree-Based Decoding with Entropy-Guided Exploration

Authors: Longxuan Wei, Yubo Zhang, Zijiao Zhang, Zhihu Wang, Shiwan Zhao, Tianyu Huang, Huiting Zhao, Chenfei Liu, Shenao Zhang, Junchi Yan
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.15296
Pdf URL: https://arxiv.org/pdf/2601.15296
Copy Paste: [[2601.15296]] Entropy-Tree: Tree-Based Decoding with Entropy-Guided Exploration(https://arxiv.org/abs/2601.15296)
Keywords: language model
Abstract: Large language models achieve strong reasoning performance, yet existing decoding strategies either explore blindly (random sampling) or redundantly (independent multi-sampling). We propose Entropy-Tree, a tree-based decoding method that exploits entropy as a signal for branching decisions--expanding the search tree only at positions where the model exhibits genuine uncertainty. Entropy-Tree shows superior accuracy and calibration in reasoning tasks: it achieves better pass@k than Multi-chain across multiple models and datasets, and its predictive entropy demonstrates better AUROC compared to several traditional metrics. Entropy-Tree unifies efficient structured exploration and reliable uncertainty estimation within a single decoding procedure.
摘要：大型语言模型具有很强的推理性能，但现有的解码策略要么是盲目探索（随机采样），要么是冗余探索（独立多重采样）。我们提出了熵树，一种基于树的解码方法，利用熵作为分支决策的信号——仅在模型表现出真正不确定性的位置扩展搜索树。 Entropy-Tree 在推理任务中表现出卓越的准确性和校准性：它在多个模型和数据集上比 Multi-chain 实现了更好的 pass@k，并且与几个传统指标相比，其预测熵表现出更好的 AUROC。熵树将有效的结构化探索和可靠的不确定性估计统一在单个解码过程中。

Title: AfriEconQA: A Benchmark Dataset for African Economic Analysis based on World Bank Reports

Authors: Edward Ajayi
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15297
Pdf URL: https://arxiv.org/pdf/2601.15297
Copy Paste: [[2601.15297]] AfriEconQA: A Benchmark Dataset for African Economic Analysis based on World Bank Reports(https://arxiv.org/abs/2601.15297)
Keywords: language model, gpt, llm
Abstract: We introduce AfriEconQA, a specialized benchmark dataset for African economic analysis grounded in a comprehensive corpus of 236 World Bank reports. The task of AfriEconQA is to answer complex economic queries that require high-precision numerical reasoning and temporal disambiguation from specialized institutional documents. The dataset consists of 8,937 curated QA instances, rigorously filtered from a pool of 10018 synthetic questions to ensure high-quality evidence-answer alignment. Each instance is composed of: (1) a question requiring reasoning over economic indicators, (2) the corresponding evidence retrieved from the corpus, (3) a verified ground-truth answer, and (4) source metadata (e.g., URL and publication date) to ensure temporal provenance. AfriEconQA is the first benchmark focused specifically on African economic analysis, providing a unique challenge for Information Retrieval (IR) systems, as the data is largely absent from the pretraining corpora of current Large Language Models (LLMs). We operationalize this dataset through an 11-experiment matrix, benchmarking a zero-shot baseline (GPT-5 Mini) against RAG configurations using GPT-4o and Qwen 32B across five distinct embedding and ranking strategies. Our results demonstrate a severe parametric knowledge gap, where zero-shot models fail to answer over 90 percent of queries, and even state-of-the-art RAG pipelines struggle to achieve high precision. This confirms AfriEconQA as a robust and challenging benchmark for the next generation of domain-specific IR and RAG systems. The AfriEconQA dataset and code will be made publicly available upon publication.
摘要：我们介绍 AfriEconQA，这是一个用于非洲经济分析的专门基准数据集，以 236 份世界银行报告的综合语料库为基础。 AfriEconQA 的任务是回答复杂的经济查询，这些查询需要高精度的数字推理和专业机构文档的时间消歧。该数据集由 8,937 个精心策划的 QA 实例组成，从 10018 个综合问题池中严格筛选出来，以确保高质量的证据-答案一致性。每个实例由以下部分组成：(1) 需要对经济指标进行推理的问题，(2) 从语料库中检索到的相应证据，(3) 经过验证的真实答案，以及 (4) 源元数据（例如 URL 和发布日期）以确保时间出处。 AfriEconQA 是第一个专门针对非洲经济分析的基准，为信息检索 (IR) 系统带来了独特的挑战，因为当前大型语言模型 (LLM) 的预训练语料库中基本上不存在数据。我们通过 11 个实验矩阵来操作该数据集，在五种不同的嵌入和排名策略中使用 GPT-4o 和 Qwen 32B 对零样本基线 (GPT-5 Mini) 与 RAG 配置进行基准测试。我们的结果表明存在严重的参数知识差距，零样本模型无法回答超过 90% 的查询，即使是最先进的 RAG 管道也难以实现高精度。这证实了 AfriEconQA 是下一代特定领域 IR 和 RAG 系统的稳健且具有挑战性的基准。 AfriEconQA 数据集和代码将在发布后公开。

Title: MALTopic: Multi-Agent LLM Topic Modeling Framework

Authors: Yash Sharma
Subjects: cs.CL, cs.IR, cs.MA
Abstract URL: https://arxiv.org/abs/2601.15299
Pdf URL: https://arxiv.org/pdf/2601.15299
Copy Paste: [[2601.15299]] MALTopic: Multi-Agent LLM Topic Modeling Framework(https://arxiv.org/abs/2601.15299)
Keywords: llm, agent
Abstract: Topic modeling is a crucial technique for extracting latent themes from unstructured text data, particularly valuable in analyzing survey responses. However, traditional methods often only consider free-text responses and do not natively incorporate structured or categorical survey responses for topic modeling. And they produce abstract topics, requiring extensive human interpretation. To address these limitations, we propose the Multi-Agent LLM Topic Modeling Framework (MALTopic). This framework decomposes topic modeling into specialized tasks executed by individual LLM agents: an enrichment agent leverages structured data to enhance textual responses, a topic modeling agent extracts latent themes, and a deduplication agent refines the results. Comparative analysis on a survey dataset demonstrates that MALTopic significantly improves topic coherence, diversity, and interpretability compared to LDA and BERTopic. By integrating structured data and employing a multi-agent approach, MALTopic generates human-readable topics with enhanced contextual relevance, offering a more effective solution for analyzing complex survey data.
摘要：主题建模是从非结构化文本数据中提取潜在主题的关键技术，在分析调查响应时特别有价值。然而，传统方法通常只考虑自由文本响应，并且本身并不将结构化或分类调查响应纳入主题建模。它们产生抽象的主题，需要大量的人类解释。为了解决这些限制，我们提出了多代理 LLM 主题建模框架 (MALTopic)。该框架将主题建模分解为由各个 LLM 代理执行的专门任务：丰富代理利用结构化数据来增强文本响应，主题建模代理提取潜在主题，重复数据删除代理完善结果。对调查数据集的比较分析表明，与 LDA 和 BERTopic 相比，MALTTopic 显着提高了主题连贯性、多样性和可解释性。通过集成结构化数据并采用多代理方法，MALTopic 生成具有增强上下文相关性的人类可读主题，为分析复杂调查数据提供更有效的解决方案。

Title: Intelligence Degradation in Long-Context LLMs: Critical Threshold Determination via Natural Length Distribution Analysis

Authors: Weiwei Wang, Jiyong Min, Weijie Zou
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15300
Pdf URL: https://arxiv.org/pdf/2601.15300
Copy Paste: [[2601.15300]] Intelligence Degradation in Long-Context LLMs: Critical Threshold Determination via Natural Length Distribution Analysis(https://arxiv.org/abs/2601.15300)
Keywords: language model, llm
Abstract: Large Language Models (LLMs) exhibit catastrophic performance degradation when processing contexts approaching certain critical thresholds, even when information remains relevant. This intelligence degradation-defined as over 30% drop in task performance-severely limits long-context applications. This degradation shows a common pattern: models maintain strong performance up to a critical threshold, then collapse catastrophically. We term this shallow long-context adaptation-models adapt for short to medium contexts but fail beyond critical thresholds. This paper presents three contributions: (1) Natural Length Distribution Analysis: We use each sample's natural token length without truncation or padding, providing stronger causal evidence that degradation results from context length itself. (2) Critical Threshold Determination: Through experiments on a mixed dataset (1,000 samples covering 5%-95% of context length), we identify the critical threshold for Qwen2.5-7B at 40-50% of maximum context length, where F1 scores drop from 0.55-0.56 to 0.3 (45.5% degradation), using five-method cross-validation. (3) Unified Framework: We consolidate shallow adaptation, explaining degradation patterns and providing a foundation for mitigation strategies. This work provides the first systematic characterization of intelligence degradation in open-source Qwen models, offering practical guidance for deploying LLMs in long-context scenarios.
摘要：当处理上下文接近某些关键阈值时，即使信息仍然相关，大型语言模型 (LLM) 也会表现出灾难性的性能下降。这种智力下降（定义为任务性能下降 30% 以上）严重限制了长上下文应用程序。这种退化显示了一个常见的模式：模型在达到临界阈值时保持强大的性能，然后灾难性地崩溃。我们将这种浅层长上下文适应模型称为适应中短上下文，但超出临界阈值时会失败。本文提出了三个贡献：（1）自然长度分布分析：我们使用每个样本的自然标记长度，没有截断或填充，提供了更有力的因果证据，表明退化是由上下文长度本身造成的。（2）关键阈值确定：通过对混合数据集（1000个样本，覆盖上下文长度的5%-95%）的实验，我们使用五种方法交叉验证确定了Qwen2.5-7B在最大上下文长度的40-50%处的关键阈值，其中F1分数从0.55-0.56下降到0.3（退化45.5%）。（3）统一框架：我们巩固浅层适应，解释退化模式并为缓解策略提供基础。这项工作首次系统地表征了开源 Qwen 模型中的智力退化，为在长上下文场景中部署法学硕士提供了实用指导。

Title: Can We Trust LLM Detectors?

Authors: Jivnesh Sandhan, Harshit Jaiswal, Fei Cheng, Yugo Murawaki
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.15301
Pdf URL: https://arxiv.org/pdf/2601.15301
Copy Paste: [[2601.15301]] Can We Trust LLM Detectors?(https://arxiv.org/abs/2601.15301)
Keywords: llm
Abstract: The rapid adoption of LLMs has increased the need for reliable AI text detection, yet existing detectors often fail outside controlled benchmarks. We systematically evaluate 2 dominant paradigms (training-free and supervised) and show that both are brittle under distribution shift, unseen generators, and simple stylistic perturbations. To address these limitations, we propose a supervised contrastive learning (SCL) framework that learns discriminative style embeddings. Experiments show that while supervised detectors excel in-domain, they degrade sharply out-of-domain, and training-free methods remain highly sensitive to proxy choice. Overall, our results expose fundamental challenges in building domain-agnostic detectors. Our code is available at: this https URL
摘要：法学硕士的快速采用增加了对可靠的人工智能文本检测的需求，但现有的检测器经常无法达到受控基准。我们系统地评估了两种主要范式（免训练和有监督），并表明这两种范式在分布变化、看不见的生成器和简单的风格扰动下都很脆弱。为了解决这些限制，我们提出了一种监督对比学习（SCL）框架，用于学习判别式嵌入。实验表明，虽然监督检测器在域内表现出色，但它们在域外却急剧退化，并且免训练方法对代理选择仍然高度敏感。总的来说，我们的结果暴露了构建领域无关检测器的基本挑战。我们的代码位于：此 https URL

Title: ICPO: Illocution-Calibrated Policy Optimization for Multi-Turn Conversation

Authors: Zhebo Wang, Xiaohu Mu, Zijie Zhou, Mohan Li, Wenpeng Xing, Dezhang Kong, Meng Han
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.15330
Pdf URL: https://arxiv.org/pdf/2601.15330
Copy Paste: [[2601.15330]] ICPO: Illocution-Calibrated Policy Optimization for Multi-Turn Conversation(https://arxiv.org/abs/2601.15330)
Keywords: language model, llm, prompt
Abstract: Large Language Models (LLMs) in multi-turn conversations often suffer from a ``lost-in-conversation'' phenomenon, where they struggle to recover from early incorrect assumptions, particularly when users provide ambiguous initial instructions. We find that standard post-training techniques like Reinforcement Learning with Verifiable Rewards (RLVR) exacerbate this issue by rewarding confident, direct answers, thereby inducing overconfidence and discouraging the model from seeking clarification. To address this, we propose Illocution-Calibrated Policy Optimization (ICPO), a novel training framework that sensitizes the model to instruction ambiguity. ICPO augments the training corpus with underspecified prompts and conditions the reward signal on the user's illocutionary intent, rewarding the model for expressing uncertainty or asking for clarification when faced with ambiguity. Experiments demonstrate that ICPO fosters appropriate humility, yielding a substantial average improvement of 75\% in multi-turn conversation, while preserving robust performance on single-turn benchmarks. Our work presents a practical path toward more robust and collaborative conversational AI that can better navigate the nuances of human interaction.
摘要：多轮对话中的大型语言模型（LLM）经常会遇到“对话中迷失”的现象，它们很难从早期的错误假设中恢复过来，特别是当用户提供模糊的初始指令时。我们发现，标准的训练后技术，例如带有可验证奖励的强化学习（RLVR），会奖励自信、直接的答案，从而加剧了这个问题，从而导致过度自信并阻止模型寻求澄清。为了解决这个问题，我们提出了 Illocution-Calibrate Policy Optimization (ICPO)，这是一种新颖的训练框架，可以使模型对指令歧义敏感。 ICPO 通过未指定的提示来增强训练语料库，并根据用户的言外意图来调节奖励信号，奖励表达不确定性或在面临歧义时要求澄清的模型。实验表明，ICPO 培养了适当的谦逊，在多轮对话中实现了 75% 的平均显着改进，同时在单轮基准测试中保持了稳健的性能。我们的工作为更强大、更具协作性的对话式人工智能提供了一条实用的道路，可以更好地驾驭人类互动的细微差别。

Title: RECAP: A Resource-Efficient Method for Adversarial Prompting in Large Language Models

Authors: Rishit Chugh
Subjects: cs.CL, cs.AI, cs.CR, cs.LG
Abstract URL: https://arxiv.org/abs/2601.15331
Pdf URL: https://arxiv.org/pdf/2601.15331
Copy Paste: [[2601.15331]] RECAP: A Resource-Efficient Method for Adversarial Prompting in Large Language Models(https://arxiv.org/abs/2601.15331)
Keywords: language model, llm, prompt
Abstract: The deployment of large language models (LLMs) has raised security concerns due to their susceptibility to producing harmful or policy-violating outputs when exposed to adversarial prompts. While alignment and guardrails mitigate common misuse, they remain vulnerable to automated jailbreaking methods such as GCG, PEZ, and GBDA, which generate adversarial suffixes via training and gradient-based search. Although effective, these methods particularly GCG are computationally expensive, limiting their practicality for organisations with constrained resources. This paper introduces a resource-efficient adversarial prompting approach that eliminates the need for retraining by matching new prompts to a database of pre-trained adversarial prompts. A dataset of 1,000 prompts was classified into seven harm-related categories, and GCG, PEZ, and GBDA were evaluated on a Llama 3 8B model to identify the most effective attack method per category. Results reveal a correlation between prompt type and algorithm effectiveness. By retrieving semantically similar successful adversarial prompts, the proposed method achieves competitive attack success rates with significantly reduced computational cost. This work provides a practical framework for scalable red-teaming and security evaluation of aligned LLMs, including in settings where model internals are inaccessible.
摘要：大型语言模型 (LLM) 的部署引发了安全问题，因为它们在遇到对抗性提示时容易产生有害或违反策略的输出。虽然对齐和护栏可以减少常见的误用，但它们仍然容易受到 GCG、PEZ 和 GBDA 等自动越狱方法的影响，这些方法通过训练和基于梯度的搜索生成对抗性后缀。尽管有效，但这些方法（尤其是 GCG）的计算成本很高，限制了它们对于资源有限的组织的实用性。本文介绍了一种资源高效的对抗性提示方法，该方法通过将新提示与预先训练的对抗性提示数据库进行匹配来消除重新训练的需要。将包含 1,000 个提示的数据集分为七个与危害相关的类别，并在 Llama 3 8B 模型上评估 GCG、PEZ 和 GBDA，以确定每个类别最有效的攻击方法。结果揭示了提示类型和算法有效性之间的相关性。通过检索语义相似的成功对抗提示，所提出的方法实现了竞争性攻击的成功率，同时显着降低了计算成本。这项工作为可扩展的红队和对齐的法学硕士的安全评估提供了一个实用的框架，包括在模型内部无法访问的环境中。

Title: No Reliable Evidence of Self-Reported Sentience in Small Large Language Models

Authors: Caspar Kaiser, Sean Enderby
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.15334
Pdf URL: https://arxiv.org/pdf/2601.15334
Copy Paste: [[2601.15334]] No Reliable Evidence of Self-Reported Sentience in Small Large Language Models(https://arxiv.org/abs/2601.15334)
Keywords: language model, gpt
Abstract: Whether language models possess sentience has no empirical answer. But whether they believe themselves to be sentient can, in principle, be tested. We do so by querying several open-weights models about their own consciousness, and then verifying their responses using classifiers trained on internal activations. We draw upon three model families (Qwen, Llama, GPT-OSS) ranging from 0.6 billion to 70 billion parameters, approximately 50 questions about consciousness and subjective experience, and three classification methods from the interpretability literature. First, we find that models consistently deny being sentient: they attribute consciousness to humans but not to themselves. Second, classifiers trained to detect underlying beliefs - rather than mere outputs - provide no clear evidence that these denials are untruthful. Third, within the Qwen family, larger models deny sentience more confidently than smaller ones. These findings contrast with recent work suggesting that models harbour latent beliefs in their own consciousness.
摘要：语言模型是否具有感知能力尚无实证答案。但原则上，他们是否相信自己有感知能力是可以测试的。我们通过查询几个开放权重模型关于它们自己的意识，然后使用经过内部激活训练的分类器来验证它们的响应。我们利用了三个模型系列（Qwen、Llama、GPT-OSS），范围从 6 亿到 700 亿个参数，大约 50 个关于意识和主观经验的问题，以及来自可解释性文献的三种分类方法。首先，我们发现模型始终否认有感知能力：它们将意识归因于人类，而不是他们自己。其次，经过训练来检测潜在信念（而不仅仅是输出）的分类器没有提供明确的证据表明这些否认是不真实的。第三，在 Qwen 家族中，较大的模型比较小的模型更自信地否认感知。这些发现与最近的研究形成鲜明对比，最近的研究表明模型对自己的意识怀有潜在的信念。

Title: From Quotes to Concepts: Axial Coding of Political Debates with Ensemble LMs

Authors: Angelina Parfenova, David Graus, Juergen Pfeffer
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15338
Pdf URL: https://arxiv.org/pdf/2601.15338
Copy Paste: [[2601.15338]] From Quotes to Concepts: Axial Coding of Political Debates with Ensemble LMs(https://arxiv.org/abs/2601.15338)
Keywords: language model, llm
Abstract: Axial coding is a commonly used qualitative analysis method that enhances document understanding by organizing sentence-level open codes into broader categories. In this paper, we operationalize axial coding with large language models (LLMs). Extending an ensemble-based open coding approach with an LLM moderator, we add an axial coding step that groups open codes into higher-order categories, transforming raw debate transcripts into concise, hierarchical representations. We compare two strategies: (i) clustering embeddings of code-utterance pairs using density-based and partitioning algorithms followed by LLM labeling, and (ii) direct LLM-based grouping of codes and utterances into categories. We apply our method to Dutch parliamentary debates, converting lengthy transcripts into compact, hierarchically structured codes and categories. We evaluate our method using extrinsic metrics aligned with human-assigned topic labels (ROUGE-L, cosine, BERTScore), and intrinsic metrics describing code groups (coverage, brevity, coherence, novelty, JSD divergence). Our results reveal a trade-off: density-based clustering achieves high coverage and strong cluster alignment, while direct LLM grouping results in higher fine-grained alignment, but lower coverage 20%. Overall, clustering maximizes coverage and structural separation, whereas LLM grouping produces more concise, interpretable, and semantically aligned categories. To support future research, we publicly release the full dataset of utterances and codes, enabling reproducibility and comparative studies.
摘要：轴向编码是一种常用的定性分析方法，它通过将句子级开放代码组织成更广泛的类别来增强文档理解。在本文中，我们使用大型语言模型（LLM）来操作轴向编码。通过 LLM 主持人扩展基于集成的开放编码方法，我们添加了一个轴向编码步骤，将开放代码分组为高阶类别，将原始辩论记录转换为简洁的分层表示。我们比较了两种策略：(i) 使用基于密度和分区算法对代码-话语对进行聚类嵌入，然后进行 LLM 标记；(ii) 直接基于 LLM 将代码和话语分组为类别。我们将我们的方法应用于荷兰议会辩论，将冗长的记录转换为紧凑的、层次结构的代码和类别。我们使用与人类分配的主题标签（ROUGE-L、余弦、BERTScore）一致的外在指标和描述代码组的内在指标（覆盖率、简洁性、连贯性、新颖性、JSD 分歧）来评估我们的方法。我们的结果揭示了一个权衡：基于密度的聚类实现了高覆盖率和强聚类对齐，而直接 LLM 分组会导致更高的细粒度对齐，但覆盖率较低 20%。总体而言，聚类最大化了覆盖范围和结构分离，而 LLM 分组则产生更简洁、可解释且语义一致的类别。为了支持未来的研究，我们公开发布了完整的话语和代码数据集，以实现可重复性和比较研究。

Title: Memorization Dynamics in Knowledge Distillation for Language Models

Authors: Jaydeep Borkar, Karan Chadha, Niloofar Mireshghallah, Yuchen Zhang, Irina-Elena Veliche, Archi Mitra, David A. Smith, Zheng Xu, Diego Garcia-Olano
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15394
Pdf URL: https://arxiv.org/pdf/2601.15394
Copy Paste: [[2601.15394]] Memorization Dynamics in Knowledge Distillation for Language Models(https://arxiv.org/abs/2601.15394)
Keywords: language model, llm
Abstract: Knowledge Distillation (KD) is increasingly adopted to transfer capabilities from large language models to smaller ones, offering significant improvements in efficiency and utility while often surpassing standard fine-tuning. Beyond performance, KD is also explored as a privacy-preserving mechanism to mitigate the risk of training data leakage. While training data memorization has been extensively studied in standard pre-training and fine-tuning settings, its dynamics in a knowledge distillation setup remain poorly understood. In this work, we study memorization across the KD pipeline using three large language model (LLM) families (Pythia, OLMo-2, Qwen-3) and three datasets (FineWeb, Wikitext, Nemotron-CC-v2). We find: (1) distilled models memorize significantly less training data than standard fine-tuning (reducing memorization by more than 50%); (2) some examples are inherently easier to memorize and account for a large fraction of memorization during distillation (over ~95%); (3) student memorization is predictable prior to distillation using features based on zlib entropy, KL divergence, and perplexity; and (4) while soft and hard distillation have similar overall memorization rates, hard distillation poses a greater risk: it inherits $2.7\times$ more teacher-specific examples than soft distillation. Overall, we demonstrate that distillation can provide both improved generalization and reduced memorization risks compared to standard fine-tuning.
摘要：知识蒸馏（KD）越来越多地被用来将功能从大型语言模型转移到较小的语言模型，从而显着提高效率和实用性，同时往往超越标准微调。除了性能之外，KD 还被探索作为一种隐私保护机制，以降低训练数据泄露的风险。虽然训练数据记忆已在标准预训练和微调设置中得到了广泛研究，但其在知识蒸馏设置中的动态仍然知之甚少。在这项工作中，我们使用三个大型语言模型 (LLM) 系列（Pythia、OLMo-2、Qwen-3）和三个数据集（FineWeb、Wikitext、Nemotron-CC-v2）来研究跨 KD 管道的记忆。我们发现：（1）蒸馏模型记忆的训练数据明显少于标准微调（减少记忆量超过 50%）；（2）一些例子本质上更容易记忆，并且在蒸馏过程中占据了很大一部分记忆（超过~95%）； (3) 使用基于 zlib 熵、KL 散度和困惑度的特征，在蒸馏之前可以预测学生的记忆情况； (4) 虽然软蒸馏和硬蒸馏具有相似的总体记忆率，但硬蒸馏带来了更大的风险：它比软蒸馏继承了 $2.7\times$ 多的教师特定示例。总的来说，我们证明与标准微调相比，蒸馏可以提高泛化能力并降低记忆风险。

Title: Beyond Fixed Psychological Personas: State Beats Trait, but Language Models are State-Blind

Authors: Tamunotonye Harry, Ivoline Ngong, Chima Nweke, Yuanyuan Feng, Joseph Near
Subjects: cs.CL, cs.AI, cs.HC
Abstract URL: https://arxiv.org/abs/2601.15395
Pdf URL: https://arxiv.org/pdf/2601.15395
Copy Paste: [[2601.15395]] Beyond Fixed Psychological Personas: State Beats Trait, but Language Models are State-Blind(https://arxiv.org/abs/2601.15395)
Keywords: language model, llm, chat
Abstract: User interactions with language models vary due to static properties of the user (trait) and the specific context of the interaction (state). However, existing persona datasets (like PersonaChat, PANDORA etc.) capture only trait, and ignore the impact of state. We introduce Chameleon, a dataset of 5,001 contextual psychological profiles from 1,667 Reddit users, each measured across multiple contexts. Using the Chameleon dataset, we present three key findings. First, inspired by Latent State-Trait theory, we decompose variance and find that 74\% is within-person(state) while only 26\% is between-person (trait). Second, we find that LLMs are state-blind: they focus on trait only, and produce similar responses regardless of state. Third, we find that reward models react to user state, but inconsistently: different models favor or penalize the same users in opposite directions. We release Chameleon to support research on affective computing, personalized dialogue, and RLHF alignment.
摘要：用户与语言模型的交互因用户的静态属性（特征）和交互的特定上下文（状态）而异。然而，现有的角色数据集（如 PersonaChat、PANDORA 等）仅捕获特征，而忽略状态的影响。我们引入了 Chameleon，这是一个包含来自 1,667 位 Reddit 用户的 5,001 个情境心理特征的数据集，每个特征都在多个情境中进行测量。使用 Chameleon 数据集，我们提出了三个关键发现。首先，受潜在状态-特质理论的启发，我们分解方差，发现 74% 是人内（状态），而只有 26% 是人间（特质）。其次，我们发现法学硕士是无状态的：他们只关注特质，并且无论状态如何都会产生类似的反应。第三，我们发现奖励模型对用户状态做出反应，但不一致：不同的模型以相反的方向偏向或惩罚相同的用户。我们发布 Chameleon 来支持情感计算、个性化对话和 RLHF 对齐的研究。

Title: Domain-Specific Knowledge Graphs in RAG-Enhanced Healthcare LLMs

Authors: Sydney Anuyah, Mehedi Mahmud Kaushik, Hao Dai, Rakesh Shiradkar, Arjan Durresi, Sunandan Chakraborty
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15429
Pdf URL: https://arxiv.org/pdf/2601.15429
Copy Paste: [[2601.15429]] Domain-Specific Knowledge Graphs in RAG-Enhanced Healthcare LLMs(https://arxiv.org/abs/2601.15429)
Keywords: language model, llm, retrieval-augmented generation
Abstract: Large Language Models (LLMs) generate fluent answers but can struggle with trustworthy, domain-specific reasoning. We evaluate whether domain knowledge graphs (KGs) improve Retrieval-Augmented Generation (RAG) for healthcare by constructing three PubMed-derived graphs: $\mathbb{G}_1$ (T2DM), $\mathbb{G}_2$ (Alzheimer's disease), and $\mathbb{G}_3$ (AD+T2DM). We design two probes: Probe 1 targets merged AD T2DM knowledge, while Probe 2 targets the intersection of $\mathbb{G}_1$ and $\mathbb{G}_2$. Seven instruction-tuned LLMs are tested across retrieval sources {No-RAG, $\mathbb{G}_1$, $\mathbb{G}_2$, $\mathbb{G}_1$ + $\mathbb{G}_2$, $\mathbb{G}_3$, $\mathbb{G}_1$+$\mathbb{G}_2$ + $\mathbb{G}_3$} and three decoding temperatures. Results show that scope alignment between probe and KG is decisive: precise, scope-matched retrieval (notably $\mathbb{G}_2$) yields the most consistent gains, whereas indiscriminate graph unions often introduce distractors that reduce accuracy. Larger models frequently match or exceed KG-RAG with a No-RAG baseline on Probe 1, indicating strong parametric priors, whereas smaller/mid-sized models benefit more from well-scoped retrieval. Temperature plays a secondary role; higher values rarely help. We conclude that precision-first, scope-matched KG-RAG is preferable to breadth-first unions, and we outline practical guidelines for graph selection, model sizing, and retrieval/reranking. Code and Data available here - this https URL
摘要：大型语言模型 (LLM) 可以生成流畅的答案，但可能难以进行值得信赖的特定领域推理。我们通过构建三个 PubMed 衍生图来评估领域知识图 (KG) 是否可以改善医疗保健的检索增强生成 (RAG)：$\mathbb{G}_1$ (T2DM)、$\mathbb{G}_2$ (阿尔茨海默病) 和 $\mathbb{G}_3$ (AD+T2DM)。我们设计了两个探针：探针1的目标是合并AD T2DM知识，而探针2的目标是$\mathbb{G}_1$和$\mathbb{G}_2$的交集。跨检索源测试了七个指令调整的 LLM {No-RAG、$\mathbb{G}_1$、$\mathbb{G}_2$、$\mathbb{G}_1$ + $\mathbb{G}_2$、$\mathbb{G}_3$、$\mathbb{G}_1$+$\mathbb{G}_2$ + $\mathbb{G}_3$} 和三个解码温度。结果表明，探针和 KG 之间的范围对齐是决定性的：精确、范围匹配的检索（特别是 $\mathbb{G}_2$）会产生最一致的增益，而不加区别的图并集通常会引入干扰因素，从而降低准确性。较大的模型经常与探针 1 上的 No-RAG 基线匹配或超过 KG-RAG，这表明强大的参数先验，而较小/中型的模型从范围广泛的检索中受益更多。温度起着次要作用；较高的值很少有帮助。我们得出的结论是，精度优先、范围匹配的 KG-RAG 优于广度优先联合，并且我们概述了图选择、模型大小调整和检索/重新排序的实用指南。代码和数据可在此处 - 此 https URL

Title: Chunking, Retrieval, and Re-ranking: An Empirical Evaluation of RAG Architectures for Policy Document Question Answering

Authors: Anuj Maharjan, Umesh Yadav
Subjects: cs.CL, cs.AI, cs.IR
Abstract URL: https://arxiv.org/abs/2601.15457
Pdf URL: https://arxiv.org/pdf/2601.15457
Copy Paste: [[2601.15457]] Chunking, Retrieval, and Re-ranking: An Empirical Evaluation of RAG Architectures for Policy Document Question Answering(https://arxiv.org/abs/2601.15457)
Keywords: language model, llm, hallucination, retrieval-augmented generation
Abstract: The integration of Large Language Models (LLMs) into the public health policy sector offers a transformative approach to navigating the vast repositories of regulatory guidance maintained by agencies such as the Centers for Disease Control and Prevention (CDC). However, the propensity for LLMs to generate hallucinations, defined as plausible but factually incorrect assertions, presents a critical barrier to the adoption of these technologies in high-stakes environments where information integrity is non-negotiable. This empirical evaluation explores the effectiveness of Retrieval-Augmented Generation (RAG) architectures in mitigating these risks by grounding generative outputs in authoritative document context. Specifically, this study compares a baseline Vanilla LLM against Basic RAG and Advanced RAG pipelines utilizing cross-encoder re-ranking. The experimental framework employs a Mistral-7B-Instruct-v0.2 model and an all-MiniLM-L6-v2 embedding model to process a corpus of official CDC policy analytical frameworks and guidance documents. The analysis measures the impact of two distinct chunking strategies, recursive character-based and token-based semantic splitting, on system accuracy, measured through faithfulness and relevance scores across a curated set of complex policy scenarios. Quantitative findings indicate that while Basic RAG architectures provide a substantial improvement in faithfulness (0.621) over Vanilla baselines (0.347), the Advanced RAG configuration achieves a superior faithfulness average of 0.797. These results demonstrate that two-stage retrieval mechanisms are essential for achieving the precision required for domain-specific policy question answering, though structural constraints in document segmentation remain a significant bottleneck for multi-step reasoning tasks.
摘要：将大型语言模型 (LLM) 整合到公共卫生政策部门中，提供了一种变革性的方法来导航由疾病控制和预防中心 (CDC) 等机构维护的庞大监管指南存储库。然而，法学硕士容易产生幻觉（定义为看似合理但实际上不正确的断言），这对在信息完整性不容妥协的高风险环境中采用这些技术构成了关键障碍。该实证评估探讨了检索增强生成（RAG）架构通过将生成输出置于权威文档上下文中来减轻这些风险的有效性。具体来说，本研究利用跨编码器重新排名，将基线 Vanilla LLM 与基本 RAG 和高级 RAG 管道进行了比较。该实验框架采用 Mistral-7B-Instruct-v0.2 模型和全 MiniLM-L6-v2 嵌入模型来处理官方 CDC 政策分析框架和指导文件的语料库。该分析衡量了两种不同的分块策略（基于递归字符和基于标记的语义分割）对系统准确性的影响，通过一组精心设计的复杂策略场景的忠实度和相关性分数来衡量。定量研究结果表明，虽然基本 RAG 架构的忠实度 (0.621) 比普通基线 (0.347) 有了显着提高，但高级 RAG 配置实现了 0.797 的优异平均忠实度。这些结果表明，两阶段检索机制对于实现特定领域策略问答所需的精度至关重要，尽管文档分割中的结构约束仍然是多步骤推理任务的重要瓶颈。

Title: Benchmarking LLMs for Pairwise Causal Discovery in Biomedical and Multi-Domain Contexts

Authors: Sydney Anuyah, Sneha Shajee-Mohan, Ankit-Singh Chauhan, Sunandan Chakraborty
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.15479
Pdf URL: https://arxiv.org/pdf/2601.15479
Copy Paste: [[2601.15479]] Benchmarking LLMs for Pairwise Causal Discovery in Biomedical and Multi-Domain Contexts(https://arxiv.org/abs/2601.15479)
Keywords: language model, llm, prompt, chain-of-thought
Abstract: The safe deployment of large language models (LLMs) in high-stakes fields like biomedicine, requires them to be able to reason about cause and effect. We investigate this ability by testing 13 open-source LLMs on a fundamental task: pairwise causal discovery (PCD) from text. Our benchmark, using 12 diverse datasets, evaluates two core skills: 1) \textbf{Causal Detection} (identifying if a text contains a causal link) and 2) \textbf{Causal Extraction} (pulling out the exact cause and effect phrases). We tested various prompting methods, from simple instructions (zero-shot) to more complex strategies like Chain-of-Thought (CoT) and Few-shot In-Context Learning (FICL). The results show major deficiencies in current models. The best model for detection, DeepSeek-R1-Distill-Llama-70B, only achieved a mean score of 49.57\% ($C_{detect}$), while the best for extraction, Qwen2.5-Coder-32B-Instruct, reached just 47.12\% ($C_{extract}$). Models performed best on simple, explicit, single-sentence relations. However, performance plummeted for more difficult (and realistic) cases, such as implicit relationships, links spanning multiple sentences, and texts containing multiple causal pairs. We provide a unified evaluation framework, built on a dataset validated with high inter-annotator agreement ($\kappa \ge 0.758$), and make all our data, code, and prompts publicly available to spur further research. \href{this https URL}{Code available here: this https URL}
摘要：在生物医学等高风险领域安全部署大型语言模型 (LLM) 要求它们能够推理因果关系。我们通过在一项基本任务上测试 13 个开源法学硕士来研究这种能力：从文本中进行成对因果发现 (PCD)。我们的基准测试使用 12 个不同的数据集，评估两项核心技能：1) \textbf{因果检测}（识别文本是否包含因果链接）和 2) \textbf{因果提取}（提取准确的因果短语）。我们测试了各种提示方法，从简单的指令（零样本）到更复杂的策略，例如思想链（CoT）和少样本上下文学习（FICL）。结果显示当前模型存在重大缺陷。检测的最佳模型 DeepSeek-R1-Distill-Llama-70B 的平均得分仅为 49.57% ($C_{detect}$)，而提取的最佳模型 Qwen2.5-Coder-32B-Instruct 的平均得分仅为 47.12% ($C_{extract}$)。模型在简单、明确、单句关系上表现最佳。然而，对于更困难（和现实）的情况，例如隐式关系、跨越多个句子的链接以及包含多个因果对的文本，性能会直线下降。我们提供了一个统一的评估框架，建立在经过高注释者一致性验证的数据集（$\kappa\ge 0.758$）上，并公开我们所有的数据、代码和提示，以促进进一步的研究。 \href{此 https URL}{此处可用代码：此 https URL}

Title: Multi-Persona Thinking for Bias Mitigation in Large Language Models

Authors: Yuxing Chen, Guoqing Luo, Zijun Wu, Lili Mou
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.15488
Pdf URL: https://arxiv.org/pdf/2601.15488
Copy Paste: [[2601.15488]] Multi-Persona Thinking for Bias Mitigation in Large Language Models(https://arxiv.org/abs/2601.15488)
Keywords: language model, llm, prompt
Abstract: Large Language Models (LLMs) exhibit significant social biases that can perpetuate harmful stereotypes and unfair outcomes. In this paper, we propose Multi-Persona Thinking (MPT), a novel inference-time framework that leverages dialectical reasoning from multiple perspectives to reduce bias. MPT guides models to adopt contrasting social identities (e.g., male and female) along with a neutral viewpoint, and then engages these personas iteratively to expose and correct biases. Through a dialectical reasoning process, the framework transforms the potential weakness of persona assignment into a strength for bias mitigation. We evaluate MPT on two widely used bias benchmarks across both open-source and closed-source models of varying scales. Our results demonstrate substantial improvements over existing prompting-based strategies: MPT achieves the lowest bias while maintaining core reasoning ability.
摘要：大型语言模型 (LLM) 表现出明显的社会偏见，可能会延续有害的刻板印象和不公平的结果。在本文中，我们提出了多角色思维（MPT），这是一种新颖的推理时间框架，它利用多个角度的辩证推理来减少偏见。 MPT 引导模型采用对比的社会身份（例如男性和女性）以及中立的观点，然后迭代地使用这些角色来揭露和纠正偏见。通过辩证推理过程，该框架将角色分配的潜在弱点转化为减轻偏见的优势。我们在不同规模的开源和闭源模型中的两个广泛使用的偏差基准上评估 MPT。我们的结果表明，与现有的基于提示的策略相比，MPT 取得了最低的偏差，同时保持了核心推理能力。

Title: ViT Registers and Fractal ViT

Authors: Jason Chuan-Chih Chou, Abhinav Kumar, Shivank Garg
Subjects: cs.CL, cs.LG
Abstract URL: https://arxiv.org/abs/2601.15506
Pdf URL: https://arxiv.org/pdf/2601.15506
Copy Paste: [[2601.15506]] ViT Registers and Fractal ViT(https://arxiv.org/abs/2601.15506)
Keywords: language model
Abstract: Drawing inspiration from recent findings including surprisingly decent performance of transformers without positional encoding (NoPE) in the domain of language models and how registers (additional throwaway tokens not tied to input) may improve the performance of large vision transformers (ViTs), we invent and test a variant of ViT called fractal ViT that breaks permutation invariance among the tokens by applying an attention mask between the regular tokens and ``summary tokens'' similar to registers, in isolation or in combination with various positional encodings. These models do not improve upon ViT with registers, highlighting the fact that these findings may be scale, domain, or application-specific.
摘要：从最近的发现中汲取灵感，包括在语言模型领域中没有位置编码的转换器（NoPE）令人惊讶的良好性能以及寄存器（与输入无关的额外一次性标记）如何可以提高大型视觉转换器（ViT）的性能，我们发明并测试了一种称为分形 ViT 的 ViT 变体，它通过在常规标记和类似于寄存器的“摘要标记”之间应用注意掩模来打破标记之间的排列不变性，单独或与各种位置编码。这些模型并未通过寄存器对 ViT 进行改进，突显了这些发现可能是规模、领域或特定于应用程序的事实。

Title: Computational Representations of Character Significance in Novels

Authors: Haaris Mian, Melanie Subbiah, Sharon Marcus, Nora Shaalan, Kathleen McKeown
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15508
Pdf URL: https://arxiv.org/pdf/2601.15508
Copy Paste: [[2601.15508]] Computational Representations of Character Significance in Novels(https://arxiv.org/abs/2601.15508)
Keywords: llm
Abstract: Characters in novels have typically been modeled based on their presence in scenes in narrative, considering aspects like their actions, named mentions, and dialogue. This conception of character places significant emphasis on the main character who is present in the most scenes. In this work, we instead adopt a framing developed from a new literary theory proposing a six-component structural model of character. This model enables a comprehensive approach to character that accounts for the narrator-character distinction and includes a component neglected by prior methods, discussion by other characters. We compare general-purpose LLMs with task-specific transformers for operationalizing this model of character on major 19th-century British realist novels. Our methods yield both component-level and graph representations of character discussion. We then demonstrate that these representations allow us to approach literary questions at scale from a new computational lens. Specifically, we explore Woloch's classic "the one vs the many" theory of character centrality and the gendered dynamics of character discussion.
摘要：小说中的人物通常是根据他们在叙事场景中的出现来建模的，并考虑他们的行为、提及的名字和对话等方面。这种角色概念非常强调出现在大多数场景中的主角。在这项工作中，我们采用了从新文学理论发展而来的框架，提出了六部分的人物结构模型。该模型实现了对角色的全面处理，该方法解释了叙述者与角色的区别，并包括先前方法忽略的组成部分以及其他角色的讨论。我们将通用法学硕士与特定任务转换器进行比较，以在 19 世纪英国主要现实主义小说中运用这种角色模型。我们的方法产生角色讨论的组件级和图形表示。然后，我们证明这些表示使我们能够从新的计算视角大规模地处理文学问题。具体来说，我们探讨了沃洛赫经典的“一与多”的人物中心理论和人物讨论的性别动态。

Title: AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains

Authors: Adam Szelestey, Sofie van Engelen, Tianhao Huang, Justin Snelders, Qintao Zeng, Songgaojun Deng
Subjects: cs.CL, cs.CY
Abstract URL: https://arxiv.org/abs/2601.15511
Pdf URL: https://arxiv.org/pdf/2601.15511
Copy Paste: [[2601.15511]] AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains(https://arxiv.org/abs/2601.15511)
Keywords: language model, gpt, llm, hallucination, prompt
Abstract: Hallucination in large language models (LLMs) remains an acute concern, contributing to the spread of misinformation and diminished public trust, particularly in high-risk domains. Among hallucination types, factuality is crucial, as it concerns a model's alignment with established world knowledge. Adversarial factuality, defined as the deliberate insertion of misinformation into prompts with varying levels of expressed confidence, tests a model's ability to detect and resist confidently framed falsehoods. Existing work lacks high-quality, domain-specific resources for assessing model robustness under such adversarial conditions, and no prior research has examined the impact of injected misinformation on long-form text factuality. To address this gap, we introduce AdversaRiskQA, the first verified and reliable benchmark systematically evaluating adversarial factuality across Health, Finance, and Law. The benchmark includes two difficulty levels to test LLMs' defensive capabilities across varying knowledge depths. We propose two automated methods for evaluating the adversarial attack success and long-form factuality. We evaluate six open- and closed-source LLMs from the Qwen, GPT-OSS, and GPT families, measuring misinformation detection rates. Long-form factuality is assessed on Qwen3 (30B) under both baseline and adversarial conditions. Results show that after excluding meaningless responses, Qwen3 (80B) achieves the highest average accuracy, while GPT-5 maintains consistently high accuracy. Performance scales non-linearly with model size, varies by domains, and gaps between difficulty levels narrow as models grow. Long-form evaluation reveals no significant correlation between injected misinformation and the model's factual output. AdversaRiskQA provides a valuable benchmark for pinpointing LLM weaknesses and developing more reliable models for high-stakes applications.
摘要：大语言模型（LLM）中的幻觉仍然是一个严重的问题，它会导致错误信息的传播和公众信任度的下降，特别是在高风险领域。在幻觉类型中，事实性至关重要，因为它关系到模型与既定世界知识的一致性。对抗性事实被定义为故意将错误信息插入具有不同程度的置信度的提示中，测试模型检测和抵制自信地构建的谎言的能力。现有的工作缺乏高质量、特定领域的资源来评估这种对抗性条件下模型的稳健性，并且之前没有研究检验注入的错误信息对长文本事实性的影响。为了解决这一差距，我们引入了 AdversaRiskQA，这是第一个经过验证且可靠的基准，系统地评估健康、金融和法律领域的对抗性事实。该基准包括两个难度级别，以测试法学硕士在不同知识深度上的防御能力。我们提出了两种自动化方法来评估对抗性攻击的成功和长篇事实性。我们评估了来自 Qwen、GPT-OSS 和 GPT 系列的六个开源和闭源法学硕士，测量错误信息检测率。在基线和对抗条件下对 Qwen3 (30B) 进行长篇事实性评估。结果显示，在排除无意义的响应后，Qwen3 (80B) 达到了最高的平均准确度，而 GPT-5 则保持了一贯的高精度。性能随模型大小呈非线性变化，随领域的不同而变化，并且难度级别之间的差距随着模型的增长而缩小。长篇评估表明注入的错误信息与模型的事实输出之间没有显着相关性。 AdversaRiskQA 为查明 LLM 弱点并为高风险应用程序开发更可靠的模型提供了宝贵的基准。

Title: Common to Whom? Regional Cultural Commonsense and LLM Bias in India

Authors: Sangmitra Madhusudan, Trush Shashank More, Steph Buongiorno, Renata Dividino, Jad Kabbara, Ali Emami
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15550
Pdf URL: https://arxiv.org/pdf/2601.15550
Copy Paste: [[2601.15550]] Common to Whom? Regional Cultural Commonsense and LLM Bias in India(https://arxiv.org/abs/2601.15550)
Keywords: llm
Abstract: Existing cultural commonsense benchmarks treat nations as monolithic, assuming uniform practices within national boundaries. But does cultural commonsense hold uniformly within a nation, or does it vary at the sub-national level? We introduce Indica, the first benchmark designed to test LLMs' ability to address this question, focusing on India - a nation of 28 states, 8 union territories, and 22 official languages. We collect human-annotated answers from five Indian regions (North, South, East, West, and Central) across 515 questions spanning 8 domains of everyday life, yielding 1,630 region-specific question-answer pairs. Strikingly, only 39.4% of questions elicit agreement across all five regions, demonstrating that cultural commonsense in India is predominantly regional, not national. We evaluate eight state-of-the-art LLMs and find two critical gaps: models achieve only 13.4%-20.9% accuracy on region-specific questions, and they exhibit geographic bias, over-selecting Central and North India as the "default" (selected 30-40% more often than expected) while under-representing East and West. Beyond India, our methodology provides a generalizable framework for evaluating cultural commonsense in any culturally heterogeneous nation, from question design grounded in anthropological taxonomy, to regional data collection, to bias measurement.
摘要：现有的文化常识基准将国家视为整体，假设国家边界内的做法是统一的。但文化常识在一个国家内部是否一致，还是在次国家层面有所不同？我们推出 Indica，这是第一个旨在测试法学硕士解决此问题的能力的基准，重点关注印度——一个拥有 28 个邦、8 个联邦直辖区和 22 种官方语言的国家。我们从印度五个地区（北部、南部、东部、西部和中部）收集了人工注释的答案，涉及涵盖日常生活 8 个领域的 515 个问题，产生了 1,630 个特定地区的问答对。引人注目的是，只有 39.4% 的问题得到了所有五个地区的一致同意，这表明印度的文化常识主要是地区性的，而不是全国性的。我们评估了八个最先进的法学硕士，发现两个关键差距：模型在特定地区问题上的准确率仅为 13.4%-20.9%，并且表现出地理偏见，过度选择印度中部和北部作为“默认”（选择频率比预期高 30-40%），而东部和西部的代表性不足。除了印度之外，我们的方法论还提供了一个通用框架，用于评估任何文化异质国家的文化常识，从基于人类学分类学的问题设计，到区域数据收集，再到偏差测量。

Title: From Generation to Collaboration: Using LLMs to Edit for Empathy in Healthcare

Authors: Man Luo, Bahareh Harandizadeh, Amara Tariq, Halim Abbas, Umar Ghaffar, Christopher J Warren, Segun O. Kolade, Haidar M. Abdul-Muhsin
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15558
Pdf URL: https://arxiv.org/pdf/2601.15558
Copy Paste: [[2601.15558]] From Generation to Collaboration: Using LLMs to Edit for Empathy in Healthcare(https://arxiv.org/abs/2601.15558)
Keywords: language model, llm
Abstract: Clinical empathy is essential for patient care, but physicians need continually balance emotional warmth with factual precision under the cognitive and emotional constraints of clinical practice. This study investigates how large language models (LLMs) can function as empathy editors, refining physicians' written responses to enhance empathetic tone while preserving underlying medical information. More importantly, we introduce novel quantitative metrics, an Empathy Ranking Score and a MedFactChecking Score to systematically assess both emotional and factual quality of the responses. Experimental results show that LLM edited responses significantly increase perceived empathy while preserving factual accuracy compared with fully LLM generated outputs. These findings suggest that using LLMs as editorial assistants, rather than autonomous generators, offers a safer, more effective pathway to empathetic and trustworthy AI-assisted healthcare communication.
摘要：临床同理心对于患者护理至关重要，但在临床实践的认知和情感约束下，医生需要不断平衡情感温暖与事实精确性。这项研究调查了大型语言模型 (LLM) 如何充当同理心编辑器，改进医生的书面答复以增强同理心语气，同时保留基础医疗信息。更重要的是，我们引入了新颖的定量指标、同理心排名分数和 MedFactChecking 分数，以系统地评估回答的情感和事实质量。实验结果表明，与完全由法学硕士生成的输出相比，法学硕士编辑的答案显着增加了感知同理心，同时保留了事实准确性。这些研究结果表明，使用法学硕士作为编辑助理，而不是自动生成器，可以提供更安全、更有效的途径，实现富有同理心和值得信赖的人工智能辅助医疗保健沟通。

Title: YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models

Authors: Junyu Lin, Meizhen Liu, Xiufeng Huang, Jinfeng Li, Haiwen Hong, Xiaohan Yuan, Yuefeng Chen, Longtao Huang, Hui Xue, Ranjie Duan, Zhikai Chen, Yuchuan Fu, Defeng Li, Lingyao Gao, Yitong Yang
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15588
Pdf URL: https://arxiv.org/pdf/2601.15588
Copy Paste: [[2601.15588]] YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models(https://arxiv.org/abs/2601.15588)
Keywords: language model, llm
Abstract: As large language models (LLMs) are increasingly deployed in real-world applications, safety guardrails are required to go beyond coarse-grained filtering and support fine-grained, interpretable, and adaptable risk assessment. However, existing solutions often rely on rapid classification schemes or post-hoc rules, resulting in limited transparency, inflexible policies, or prohibitive inference costs. To this end, we present YuFeng-XGuard, a reasoning-centric guardrail model family designed to perform multi-dimensional risk perception for LLM interactions. Instead of producing opaque binary judgments, YuFeng-XGuard generates structured risk predictions, including explicit risk categories and configurable confidence scores, accompanied by natural language explanations that expose the underlying reasoning process. This formulation enables safety decisions that are both actionable and interpretable. To balance decision latency and explanatory depth, we adopt a tiered inference paradigm that performs an initial risk decision based on the first decoded token, while preserving ondemand explanatory reasoning when required. In addition, we introduce a dynamic policy mechanism that decouples risk perception from policy enforcement, allowing safety policies to be adjusted without model retraining. Extensive experiments on a diverse set of public safety benchmarks demonstrate that YuFeng-XGuard achieves stateof-the-art performance while maintaining strong efficiency-efficacy trade-offs. We release YuFeng-XGuard as an open model family, including both a full-capacity variant and a lightweight version, to support a wide range of deployment scenarios.
摘要：随着大型语言模型（LLM）越来越多地部署在现实世界的应用中，安全护栏需要超越粗粒度的过滤，并支持细粒度的、可解释的、适应性强的风险评估。然而，现有的解决方案通常依赖于快速分类方案或事后规则，导致透明度有限、政策不灵活或推理成本过高。为此，我们提出了 YuFeng-XGuard，一个以推理为中心的护栏模型系列，旨在为 LLM 交互执行多维风险感知。 YuFeng-XGuard 不会产生不透明的二元判断，而是生成结构化风险预测，包括明确的风险类别和可配置的置信度分数，并附有揭示潜在推理过程的自然语言解释。这种表述使得安全决策既可操作又可解释。为了平衡决策延迟和解释深度，我们采用分层推理范例，根据第一个解码的令牌执行初始风险决策，同时在需要时保留按需解释推理。此外，我们引入了一种动态策略机制，将风险感知与策略执行脱钩，允许在不重新训练模型的情况下调整安全策略。对各种公共安全基准的广泛实验表明，YuFeng-XGuard 实现了最先进的性能，同时保持了强大的效率与功效权衡。我们将YuFeng-XGuard作为开放模型系列发布，包括全容量变体和轻量级版本，以支持广泛的部署场景。

Title: Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow

Authors: Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao
Subjects: cs.CL, cs.AI, cs.LG
Abstract URL: https://arxiv.org/abs/2601.15593
Pdf URL: https://arxiv.org/pdf/2601.15593
Copy Paste: [[2601.15593]] Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow(https://arxiv.org/abs/2601.15593)
Keywords: language model
Abstract: Masked Diffusion Language Models (MDLMs) promise parallel token generation and arbitrary-order decoding, yet it remains unclear to what extent current models truly realize these capabilities. We characterize MDLM behavior along two dimensions -- parallelism strength and generation order -- using Average Finalization Parallelism (AFP) and Kendall's tau. We evaluate eight mainstream MDLMs (up to 100B parameters) on 58 benchmarks spanning knowledge, reasoning, and programming. The results show that MDLMs still lag behind comparably sized autoregressive models, mainly because parallel probabilistic modeling weakens inter-token dependencies. Meanwhile, MDLMs exhibit adaptive decoding behavior: their parallelism and generation order vary significantly with the task domain, the stage of reasoning, and whether the output is correct. On tasks that require "backward information" (e.g., Sudoku), MDLMs adopt a solution order that tends to fill easier Sudoku blanks first, highlighting their advantages. Finally, we provide theoretical motivation and design insights supporting a Generate-then-Edit paradigm, which mitigates dependency loss while retaining the efficiency of parallel decoding.
摘要：掩码扩散语言模型（MDLM）承诺并行令牌生成和任意顺序解码，但目前尚不清楚当前模型在多大程度上真正实现了这些功能。我们使用平均最终并行性 (AFP) 和 Kendall tau 沿两个维度（并行强度和生成顺序）描述 MDLM 行为。我们在涵盖知识、推理和编程的 58 个基准上评估了 8 个主流 MDLM（最多 100B 参数）。结果表明，MDLM 仍然落后于同等规模的自回归模型，主要是因为并行概率建模削弱了令牌间的依赖性。同时，MDLM 表现出自适应解码行为：它们的并行性和生成顺序随着任务域、推理阶段以及输出是否正确而显着变化。对于需要“向后信息”的任务（例如数独），MDLM 采用一种解决方案顺序，倾向于首先填充更容易的数独空白，从而突出其优势。最后，我们提供支持“生成然后编辑”范例的理论动机和设计见解，该范例可以减轻依赖性损失，同时保留并行解码的效率。

Title: ToxiTwitch: Toward Emote-Aware Hybrid Moderation for Live Streaming Platforms

Authors: Baktash Ansari, Shiza Ali, Elias Martin, Maryna Sivachenko, Afra Mashhadi
Subjects: cs.CL, cs.SI
Abstract URL: https://arxiv.org/abs/2601.15605
Pdf URL: https://arxiv.org/pdf/2601.15605
Copy Paste: [[2601.15605]] ToxiTwitch: Toward Emote-Aware Hybrid Moderation for Live Streaming Platforms(https://arxiv.org/abs/2601.15605)
Keywords: language model, llm, chat
Abstract: The rapid growth of live-streaming platforms such as Twitch has introduced complex challenges in moderating toxic behavior. Traditional moderation approaches, such as human annotation and keyword-based filtering, have demonstrated utility, but human moderators on Twitch constantly struggle to scale effectively in the fast-paced, high-volume, and context-rich chat environment of the platform while also facing harassment themselves. Recent advances in large language models (LLMs), such as DeepSeek-R1-Distill and Llama-3-8B-Instruct, offer new opportunities for toxicity detection, especially in understanding nuanced, multimodal communication involving emotes. In this work, we present an exploratory comparison of toxicity detection approaches tailored to Twitch. Our analysis reveals that incorporating emotes improves the detection of toxic behavior. To this end, we introduce ToxiTwitch, a hybrid model that combines LLM-generated embeddings of text and emotes with traditional machine learning classifiers, including Random Forest and SVM. In our case study, the proposed hybrid approach reaches up to 80 percent accuracy under channel-specific training (with 13 percent improvement over BERT and F1-score of 76 percent). This work is an exploratory study intended to surface challenges and limits of emote-aware toxicity detection on Twitch.
摘要：Twitch 等直播平台的快速发展给控制不良行为带来了复杂的挑战。传统的审核方法（例如人工注释和基于关键字的过滤）已被证明是实用的，但 Twitch 上的人工审核员一直在努力在快节奏、高容量且上下文丰富的平台聊天环境中有效扩展，同时自己也面临骚扰。 DeepSeek-R1-Distill 和 Llama-3-8B-Instruct 等大型语言模型 (LLM) 的最新进展为毒性检测提供了新的机会，特别是在理解涉及表情的细致入微的多模式通信方面。在这项工作中，我们对针对 Twitch 定制的毒性检测方法进行了探索性比较。我们的分析表明，结合情绪可以提高对有毒行为的检测。为此，我们引入了 ToxiTwitch，这是一种混合模型，它将 LLM 生成的文本和表情嵌入与传统的机器学习分类器（包括随机森林和 SVM）相结合。在我们的案例研究中，所提出的混合方法在特定于通道的训练下达到了 80% 的准确率（比 BERT 提高了 13%，F1 分数提高了 76%）。这项工作是一项探索性研究，旨在揭示 Twitch 上情绪感知毒性检测的挑战和局限性。

Title: Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation

Authors: Zhiyao Ren, Yibing Zhan, Siyuan Liang, Guozheng Ma, Baosheng Yu, Dacheng Tao
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15645
Pdf URL: https://arxiv.org/pdf/2601.15645
Copy Paste: [[2601.15645]] Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation(https://arxiv.org/abs/2601.15645)
Keywords: language model, llm, retrieval-augmented generation
Abstract: Large-scale language models (LLMs) often offer clinical judgments based on incomplete information, increasing the risk of misdiagnosis. Existing studies have primarily evaluated confidence in single-turn, static settings, overlooking the coupling between confidence and correctness as clinical evidence accumulates during real consultations, which limits their support for reliable decision-making. We propose the first benchmark for assessing confidence in multi-turn interaction during realistic medical consultations. Our benchmark unifies three types of medical data for open-ended diagnostic generation and introduces an information sufficiency gradient to characterize the confidence-correctness dynamics as evidence increases. We implement and compare 27 representative methods on this benchmark; two key insights emerge: (1) medical data amplifies the inherent limitations of token-level and consistency-level confidence methods, and (2) medical reasoning must be evaluated for both diagnostic accuracy and information completeness. Based on these insights, we present MedConf, an evidence-grounded linguistic self-assessment framework that constructs symptom profiles via retrieval-augmented generation, aligns patient information with supporting, missing, and contradictory relations, and aggregates them into an interpretable confidence estimate through weighted integration. Across two LLMs and three medical datasets, MedConf consistently outperforms state-of-the-art methods on both AUROC and Pearson correlation coefficient metrics, maintaining stable performance under conditions of information insufficiency and multimorbidity. These results demonstrate that information adequacy is a key determinant of credible medical confidence modeling, providing a new pathway toward building more reliable and interpretable large medical models.
摘要：大规模语言模型（LLM）经常根据不完整的信息提供临床判断，增加了误诊的风险。现有研究主要评估单轮、静态设置下的置信度，忽视了真实咨询过程中临床证据积累时置信度与正确性之间的耦合，这限制了它们对可靠决策的支持。我们提出了第一个评估现实医疗咨询期间多回合互动信心的基准。我们的基准统一了三种类型的医疗数据，用于开放式诊断生成，并引入信息充足梯度来描述随着证据增加的置信正确性动态。我们在此基准上实施并比较了 27 种代表性方法；出现了两个关键见解：（1）医学数据放大了标记级和一致性级置信方法的固有局限性，（2）必须评估医学推理的诊断准确性和信息完整性。基于这些见解，我们提出了 MedConf，一种基于证据的语言自我评估框架，通过检索增强生成构建症状概况，将患者信息与支持、缺失和矛盾关系对齐，并通过加权整合将它们聚合成可解释的置信度估计。在两个法学硕士和三个医学数据集上，MedConf 在 AUROC 和 Pearson 相关系数指标上始终优于最先进的方法，在信息不足和多发病的情况下保持稳定的性能。这些结果表明，信息充分性是可靠的医疗置信度模型的关键决定因素，为构建更可靠和可解释的大型医疗模型提供了新途径。

Title: What Patients Really Ask: Exploring the Effect of False Assumptions in Patient Information Seeking

Authors: Raymond Xiong, Furong Jia, Lionel Wong, Monica Agrawal
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15674
Pdf URL: https://arxiv.org/pdf/2601.15674
Copy Paste: [[2601.15674]] What Patients Really Ask: Exploring the Effect of False Assumptions in Patient Information Seeking(https://arxiv.org/abs/2601.15674)
Keywords: language model, llm
Abstract: Patients are increasingly using large language models (LLMs) to seek answers to their healthcare-related questions. However, benchmarking efforts in LLMs for question answering often focus on medical exam questions, which differ significantly in style and content from the questions patients actually raise in real life. To bridge this gap, we sourced data from Google's People Also Ask feature by querying the top 200 prescribed medications in the United States, curating a dataset of medical questions people commonly ask. A considerable portion of the collected questions contains incorrect assumptions and dangerous intentions. We demonstrate that the emergence of these corrupted questions is not uniformly random and depends heavily on the degree of incorrectness in the history of questions that led to their appearance. Current LLMs that perform strongly on other benchmarks struggle to identify incorrect assumptions in everyday questions.
摘要：患者越来越多地使用大型语言模型 (LLM) 来寻求医疗保健相关问题的答案。然而，法学硕士的问答基准工作通常集中在医学考试问题上，这些问题在风格和内容上与患者在现实生活中实际提出的问题有很大不同。为了弥补这一差距，我们通过查询美国排名前 200 的处方药，从 Google 的 People Also Ask 功能中获取数据，整理出人们常见的医疗问题数据集。收集到的问题中有相当一部分包含不正确的假设和危险的意图。我们证明，这些损坏的问题的出现并不是完全随机的，并且在很大程度上取决于导致其出现的问题的历史中的不正确程度。当前在其他基准上表现出色的法学硕士很难识别日常问题中的错误假设。

Title: Persona Switch: Mixing Distinct Perspectives in Decoding Time

Authors: Junseok Kim, Nakyeong Yang, Kyomin Jung
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15708
Pdf URL: https://arxiv.org/pdf/2601.15708
Copy Paste: [[2601.15708]] Persona Switch: Mixing Distinct Perspectives in Decoding Time(https://arxiv.org/abs/2601.15708)
Keywords: language model, llm, prompt
Abstract: Role-play prompting is known to steer the behavior of language models by injecting a persona into the prompt, improving their zero-shot reasoning capabilities. However, such improvements are inconsistent across different tasks or instances. This inconsistency suggests that zero-shot and role-play prompting may offer complementary strengths rather than one being universally superior. Building on this insight, we propose Persona Switch, a novel decoding method that dynamically combines the benefits of both prompting strategies. Our method proceeds step-by-step, selecting the better output between zero-shot and role-play prompting at each step by comparing their output confidence, as measured by the logit gap. Experiments with widely-used LLMs demonstrate that Persona Switch consistently outperforms competitive baselines, achieving up to 5.13% accuracy improvement. Furthermore, we show that output confidence serves as an informative measure for selecting the more reliable output.
摘要：众所周知，角色扮演提示可以通过在提示中注入角色来引导语言模型的行为，从而提高其零样本推理能力。然而，这种改进在不同的任务或实例中是不一致的。这种不一致表明零射击和角色扮演提示可能提供互补的优势，而不是普遍优越。基于这一见解，我们提出了 Persona Switch，这是一种新颖的解码方法，可以动态地结合两种提示策略的优点。我们的方法逐步进行，通过比较输出置信度（通过 logit 差距来衡量），在每一步中选择零样本提示和角色扮演提示之间更好的输出。对广泛使用的法学硕士进行的实验表明，Persona Switch 的性能始终优于竞争基准，准确率提高了高达 5.13%。此外，我们表明输出置信度可以作为选择更可靠输出的信息度量。

Title: Dancing in Chains: Strategic Persuasion in Academic Rebuttal via Theory of Mind

Authors: Zhitao He, Zongwei Lyu, Yi R Fung
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.15715
Pdf URL: https://arxiv.org/pdf/2601.15715
Copy Paste: [[2601.15715]] Dancing in Chains: Strategic Persuasion in Academic Rebuttal via Theory of Mind(https://arxiv.org/abs/2601.15715)
Keywords: gpt, agent
Abstract: Although artificial intelligence (AI) has become deeply integrated into various stages of the research workflow and achieved remarkable advancements, academic rebuttal remains a significant and underexplored challenge. This is because rebuttal is a complex process of strategic communication under severe information asymmetry rather than a simple technical debate. Consequently, current approaches struggle as they largely imitate surface-level linguistics, missing the essential element of perspective-taking required for effective persuasion. In this paper, we introduce RebuttalAgent, the first framework to ground academic rebuttal in Theory of Mind (ToM), operationalized through a ToM-Strategy-Response (TSR) pipeline that models reviewer mental state, formulates persuasion strategy, and generates strategy-grounded response. To train our agent, we construct RebuttalBench, a large-scale dataset synthesized via a novel critique-and-refine approach. Our training process consists of two stages, beginning with a supervised fine-tuning phase to equip the agent with ToM-based analysis and strategic planning capabilities, followed by a reinforcement learning phase leveraging the self-reward mechanism for scalable self-improvement. For reliable and efficient automated evaluation, we further develop Rebuttal-RM, a specialized evaluator trained on over 100K samples of multi-source rebuttal data, which achieves scoring consistency with human preferences surpassing powerful judge GPT-4.1. Extensive experiments show RebuttalAgent significantly outperforms the base model by an average of 18.3% on automated metrics, while also outperforming advanced proprietary models across both automated and human evaluations. Disclaimer: the generated rebuttal content is for reference only to inspire authors and assist in drafting. It is not intended to replace the author's own critical analysis and response.
摘要：尽管人工智能（AI）已经深入融入研究工作流程的各个阶段并取得了显着的进步，但学术反驳仍然是一个重大且尚未得到充分探索的挑战。这是因为反驳是一个严重信息不对称下的复杂的战略沟通过程，而不是简单的技术辩论。因此，当前的方法很困难，因为它们很大程度上模仿了表面语言学，缺少有效说服所需的观点采择的基本要素。在本文中，我们介绍了 RebuttalAgent，这是第一个在心理理论 (ToM) 中进行学术反驳的框架，通过 ToM-策略-响应 (TSR) 管道进行操作，该管道可以模拟审稿人的心理状态，制定说服策略，并生成基于策略的响应。为了训练我们的代理，我们构建了 RebuttalBench，这是一个通过新颖的批评和改进方法合成的大规模数据集。我们的训练过程由两个阶段组成，首先是监督微调阶段，为智能体配备基于 ToM 的分析和战略规划能力，然后是强化学习阶段，利用自我奖励机制实现可扩展的自我改进。为了实现可靠、高效的自动化评估，我们进一步开发了 Rebuttal-RM，这是一种经过超过 100K 多源反驳数据样本训练的专业评估器，其评分与人类偏好的一致性超越了强大的法官 GPT-4.1。大量实验表明，RebuttalAgent 在自动化指标上的表现明显优于基本模型，平均高出 18.3%，同时在自动化和人工评估方面也优于先进的专有模型。免责声明：生成的反驳内容仅供参考，以启发作者并协助起草。它无意取代作者自己的批判性分析和回应。

Title: Hallucination Mitigating for Medical Report Generation

Authors: Ruoqing Zhao, Runze Xia, Piji Li
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15745
Pdf URL: https://arxiv.org/pdf/2601.15745
Copy Paste: [[2601.15745]] Hallucination Mitigating for Medical Report Generation(https://arxiv.org/abs/2601.15745)
Keywords: language model, hallucination
Abstract: In the realm of medical report generation (MRG), the integration of natural language processing has emerged as a vital tool to alleviate the workload of radiologists. Despite the impressive capabilities demonstrated by large vision language models (LVLMs) in understanding natural language, their susceptibility to generating plausible yet inaccurate claims, known as ``hallucinations'', raises concerns-especially in the nuanced and critical field of medical. In this work, we introduce a framework, \textbf{K}nowledge-\textbf{E}nhanced with Fine-Grained \textbf{R}einforced Rewards \textbf{M}edical Report Generation (KERM), to tackle the issue. Our approach refines the input to the LVLM by first utilizing MedCLIP for knowledge retrieval, incorporating relevant lesion fact sentences from a curated knowledge corpus. We then introduce a novel purification module to ensure the retrieved knowledge is contextually relevant to the patient's clinical context. Subsequently, we employ fine-grained rewards to guide these models in generating highly supportive and clinically relevant descriptions, ensuring the alignment of model's outputs with desired behaviors. Experimental results on IU-Xray and MIMIC-CXR datasets validate the effectiveness of our approach in mitigating hallucinations and enhancing report quality.
摘要：在医疗报告生成（MRG）领域，自然语言处理的集成已成为减轻放射科医生工作量的重要工具。尽管大型视觉语言模型（LVLM）在理解自然语言方面表现出令人印象深刻的能力，但它们容易产生看似合理但不准确的主张，即所谓的“幻觉”，引起了人们的关注，尤其是在微妙而关键的医学领域。在这项工作中，我们引入了一个框架，\textbf{K}nowledge-\textbf{E}nhanced with Fine-Grained \textbf{R}einforced Rewards \textbf{M}edical Report Generation (KERM)，来解决这个问题。我们的方法首先利用 MedCLIP 进行知识检索，并结合精选知识语料库中的相关病变事实句子，从而细化 LVLM 的输入。然后，我们引入一种新颖的纯化模块，以确保检索到的知识与患者的临床背景相关。随后，我们采用细粒度的奖励来指导这些模型生成高度支持性和临床相关的描述，确保模型的输出与期望的行为保持一致。 IU-Xray 和 MIMIC-CXR 数据集的实验结果验证了我们的方法在减轻幻觉和提高报告质量方面的有效性。

Title: Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs

Authors: Tristan Williams, Franziska Weeber, Sebastian Padó, Alan Akbik
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15755
Pdf URL: https://arxiv.org/pdf/2601.15755
Copy Paste: [[2601.15755]] Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs(https://arxiv.org/abs/2601.15755)
Keywords: language model, llm, prompt
Abstract: Large language models are increasingly used to represent human opinions, values, or beliefs, and their steerability towards these ideals is an active area of research. Existing work focuses predominantly on aligning marginal response distributions, treating each survey item independently. While essential, this may overlook deeper latent structures that characterise real populations and underpin cultural values theories. We propose a framework for evaluating the representativeness of aligned models through multivariate correlation patterns in addition to marginal distributions. We show the value of our evaluation scheme by comparing two model steering techniques (persona prompting and demographic fine-tuning) and evaluating them against human responses from the World Values Survey. While the demographically fine-tuned model better approximates marginal response distributions than persona prompting, both techniques fail to fully capture the gold standard correlation patterns. We conclude that representativeness is a distinct aspect of value alignment and an evaluation focused on marginals can mask structural failures, leading to overly optimistic conclusions about model capabilities.
摘要：大型语言模型越来越多地用于代表人类的观点、价值观或信仰，它们对这些理想的可引导性是一个活跃的研究领域。现有的工作主要集中于调整边际响应分布，独立处理每个调查项目。虽然这很重要，但它可能会忽视表征真实人群并支撑文化价值观理论的更深层次的潜在结构。我们提出了一个框架，用于通过除边际分布之外的多元相关模式来评估对齐模型的代表性。我们通过比较两种模型引导技术（角色提示和人口统计微调）并根据世界价值观调查中的人类反应对其进行评估，展示了我们的评估方案的价值。虽然人口统计微调模型比角色提示更好地近似边际响应分布，但这两种技术都无法完全捕捉黄金标准相关模式。我们的结论是，代表性是价值调整的一个独特方面，而专注于边际的评估可能会掩盖结构性故障，从而导致对模型能力过于乐观的结论。

Title: HumanLLM: Towards Personalized Understanding and Simulation of Human Nature

Authors: Yuxuan Lei, Tianfu Wang, Jianxun Lian, Zhengyu Hu, Defu Lian, Xing Xie
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15793
Pdf URL: https://arxiv.org/pdf/2601.15793
Copy Paste: [[2601.15793]] HumanLLM: Towards Personalized Understanding and Simulation of Human Nature(https://arxiv.org/abs/2601.15793)
Keywords: language model, llm
Abstract: Motivated by the remarkable progress of large language models (LLMs) in objective tasks like mathematics and coding, there is growing interest in their potential to simulate human behavior--a capability with profound implications for transforming social science research and customer-centric business insights. However, LLMs often lack a nuanced understanding of human cognition and behavior, limiting their effectiveness in social simulation and personalized applications. We posit that this limitation stems from a fundamental misalignment: standard LLM pretraining on vast, uncontextualized web data does not capture the continuous, situated context of an individual's decisions, thoughts, and behaviors over time. To bridge this gap, we introduce HumanLLM, a foundation model designed for personalized understanding and simulation of individuals. We first construct the Cognitive Genome Dataset, a large-scale corpus curated from real-world user data on platforms like Reddit, Twitter, Blogger, and Amazon. Through a rigorous, multi-stage pipeline involving data filtering, synthesis, and quality control, we automatically extract over 5.5 million user logs to distill rich profiles, behaviors, and thinking patterns. We then formulate diverse learning tasks and perform supervised fine-tuning to empower the model to predict a wide range of individualized human behaviors, thoughts, and experiences. Comprehensive evaluations demonstrate that HumanLLM achieves superior performance in predicting user actions and inner thoughts, more accurately mimics user writing styles and preferences, and generates more authentic user profiles compared to base models. Furthermore, HumanLLM shows significant gains on out-of-domain social intelligence benchmarks, indicating enhanced generalization.
摘要：受大型语言模型 (LLM) 在数学和编码等客观任务中取得的显着进步的推动，人们对其模拟人类行为的潜力越来越感兴趣，这种能力对于改变社会科学研究和以客户为中心的商业洞察力具有深远的影响。然而，法学硕士通常缺乏对人类认知和行为的细致理解，限制了其在社会模拟和个性化应用中的有效性。我们认为这种限制源于一个根本性的偏差：对大量、无背景的网络数据进行的标准法学硕士预训练并不能捕捉到个人随着时间的推移做出的决策、想法和行为的连续的、情境背景。为了弥补这一差距，我们引入了 HumanLLM，这是一个专为个体的个性化理解和模拟而设计的基础模型。我们首先构建认知基因组数据集，这是一个根据 Reddit、Twitter、Blogger 和 Amazon 等平台上的真实用户数据整理的大型语料库。通过涉及数据过滤、合成和质量控制的严格、多阶段管道，我们自动提取超过 550 万条用户日志，以提炼出丰富的档案、行为和思维模式。然后，我们制定不同的学习任务并进行监督微调，使模型能够预测各种个性化的人类行为、思想和经验。综合评估表明，与基础模型相比，HumanLLM 在预测用户行为和内心想法方面取得了优异的性能，更准确地模仿了用户的写作风格和偏好，并生成了更真实的用户档案。此外，HumanLLM 在域外社交智能基准上显示出显着的进步，表明泛化能力增强。

Title: SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics

Authors: Silvia Casola, Ryan Soh-Eun Shim, Felicia Körner, Yuchen Mao, Barbara Plank
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15809
Pdf URL: https://arxiv.org/pdf/2601.15809
Copy Paste: [[2601.15809]] SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics(https://arxiv.org/abs/2601.15809)
Keywords: language model
Abstract: An increasing body of work has leveraged multilingual language models for Natural Language Generation tasks such as summarization. A major empirical bottleneck in this area is the shortage of accurate and robust evaluation metrics for many languages, which hinders progress. Recent studies suggest that multilingual language models often use English as an internal pivot language, and that misalignment with this pivot can lead to degraded downstream performance. Motivated by the hypothesis that this mismatch could also apply to multilingual neural metrics, we ask whether steering their activations toward an English pivot can improve correlation with human judgments. We experiment with encoder- and decoder-based metrics and find that test-time intervention methods are effective across the board, increasing metric effectiveness for diverse languages.
摘要：越来越多的工作利用多语言语言模型来执行摘要等自然语言生成任务。该领域的一个主要实证瓶颈是许多语言缺乏准确和稳健的评估指标，这阻碍了进展。最近的研究表明，多语言模型通常使用英语作为内部枢轴语言，与此枢轴的不一致可能会导致下游性能下降。出于这种不匹配也适用于多语言神经指标的假设，我们询问将它们的激活转向英语枢轴是否可以提高与人类判断的相关性。我们对基于编码器和解码器的指标进行了实验，发现测试时干预方法全面有效，提高了不同语言的指标有效性。

Title: ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection

Authors: Guoxuan Ding, Yuqing Li, Ziyan Zhou, Zheng Lin, Daren Zha, Jiangnan Li
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15820
Pdf URL: https://arxiv.org/pdf/2601.15820
Copy Paste: [[2601.15820]] ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection(https://arxiv.org/abs/2601.15820)
Keywords: retrieval-augmented generation
Abstract: The rapid spread of multimodal fake news poses a serious societal threat, as its evolving nature and reliance on timely factual details challenge existing detection methods. Dynamic Retrieval-Augmented Generation provides a promising solution by triggering keyword-based retrieval and incorporating external knowledge, thus enabling both efficient and accurate evidence selection. However, it still faces challenges in addressing issues such as redundant retrieval, coarse similarity, and irrelevant evidence when applied to deceptive content. In this paper, we propose ExDR, an Explanation-driven Dynamic Retrieval-Augmented Generation framework for Multimodal Fake News Detection. Our framework systematically leverages model-generated explanations in both the retrieval triggering and evidence retrieval modules. It assesses triggering confidence from three complementary dimensions, constructs entity-aware indices by fusing deceptive entities, and retrieves contrastive evidence based on deception-specific features to challenge the initial claim and enhance the final prediction. Experiments on two benchmark datasets, AMG and MR2, demonstrate that ExDR consistently outperforms previous methods in retrieval triggering accuracy, retrieval quality, and overall detection performance, highlighting its effectiveness and generalization capability.
摘要：多模式假新闻的快速传播构成了严重的社会威胁，因为其不断发展的性质和对及时事实细节的依赖对现有的检测方法提出了挑战。动态检索增强生成通过触发基于关键字的检索并结合外部知识，提供了一种有前景的解决方案，从而实现高效、准确的证据选择。然而，它在处理欺骗性内容时仍然面临着解决冗余检索、粗糙相似性和不相关证据等问题的挑战。在本文中，我们提出了 ExDR，一种用于多模态假新闻检测的解释驱动的动态检索增强生成框架。我们的框架在检索触发和证据检索模块中系统地利用模型生成的解释。它从三个互补维度评估触发置信度，通过融合欺骗性实体构建实体感知指数，并根据欺骗特定特征检索对比证据以挑战初始主张并增强最终预测。在 AMG 和 MR2 两个基准数据集上的实验表明，ExDR 在检索触发精度、检索质量和整体检测性能方面始终优于先前的方法，凸显了其有效性和泛化能力。

Title: Can professional translators identify machine-generated text?

Authors: Michael Farrell
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.15828
Pdf URL: https://arxiv.org/pdf/2601.15828
Copy Paste: [[2601.15828]] Can professional translators identify machine-generated text?(https://arxiv.org/abs/2601.15828)
Keywords: gpt, chat
Abstract: This study investigates whether professional translators can reliably identify short stories generated in Italian by artificial intelligence (AI) without prior specialized training. Sixty-nine translators took part in an in-person experiment, where they assessed three anonymized short stories - two written by ChatGPT-4o and one by a human author. For each story, participants rated the likelihood of AI authorship and provided justifications for their choices. While average results were inconclusive, a statistically significant subset (16.2%) successfully distinguished the synthetic texts from the human text, suggesting that their judgements were informed by analytical skill rather than chance. However, a nearly equal number misclassified the texts in the opposite direction, often relying on subjective impressions rather than objective markers, possibly reflecting a reader preference for AI-generated texts. Low burstiness and narrative contradiction emerged as the most reliable indicators of synthetic authorship, with unexpected calques, semantic loans and syntactic transfer from English also reported. In contrast, features such as grammatical accuracy and emotional tone frequently led to misclassification. These findings raise questions about the role and scope of synthetic-text editing in professional contexts.
摘要：这项研究调查了专业翻译人员是否能够可靠地识别由人工智能 (AI) 生成的意大利语短篇小说，而无需事先接受专门培训。 69 名翻译人员参加了一项现场实验，他们评估了三个匿名短篇故事——其中两个由 ChatGPT-4o 撰写，另一个由人类作者撰写。对于每个故事，参与者都会评估人工智能作者身份的可能性，并为他们的选择提供理由。虽然平均结果尚无定论，但统计上显着的子集 (16.2%) 成功区分了合成文本和人类文本，这表明他们的判断是基于分析技能而不是偶然。然而，几乎同等数量的人在相反的方向上对文本进行了错误分类，通常依赖于主观印象而不是客观标记，这可能反映了读者对人工智能生成文本的偏好。低突发性和叙事矛盾成为综合作者最可靠的指标，还报告了来自英语的意外的卡尔克、语义借用和句法迁移。相比之下，语法准确性和情绪语气等特征经常导致错误分类。这些发现对合成文本编辑在专业环境中的作用和范围提出了疑问。

Title: Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model

Authors: Chenghao Fan, Wen Heng, Bo Li, Sichen Liu, Yuxuan Song, Jing Su, Xiaoye Qu, Kai Shen, Wei Wei
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.15892
Pdf URL: https://arxiv.org/pdf/2601.15892
Copy Paste: [[2601.15892]] Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model(https://arxiv.org/abs/2601.15892)
Keywords: language model, llm
Abstract: Diffusion-based language models (DLLMs) offer non-sequential, block-wise generation and richer data reuse compared to autoregressive (AR) models, but existing code DLLMs still lag behind strong AR baselines under comparable budgets. We revisit this setting in a controlled study and introduce Stable-DiffCoder, a block diffusion code model that reuses the Seed-Coder architecture, data, and training pipeline. To enable efficient knowledge learning and stable training, we incorporate a block diffusion continual pretraining (CPT) stage enhanced by a tailored warmup and block-wise clipped noise schedule. Under the same data and architecture, Stable-DiffCoder overall outperforms its AR counterpart on a broad suite of code benchmarks. Moreover, relying only on the CPT and supervised fine-tuning stages, Stable-DiffCoder achieves stronger performance than a wide range of \~8B ARs and DLLMs, demonstrating that diffusion-based training can improve code modeling quality beyond AR training alone. Moreover, diffusion-based any-order modeling improves structured code modeling for editing and reasoning, and through data augmentation, benefits low-resource coding languages.
摘要：与自回归 (AR) 模型相比，基于扩散的语言模型 (DLLM) 提供非顺序、分块生成和更丰富的数据重用，但现有代码 DLLM 在可比较的预算下仍然落后于强大的 AR 基线。我们在一项对照研究中重新审视了这一设置，并引入了 Stable-DiffCoder，这是一种重用 Seed-Coder 架构、数据和训练管道的块扩散代码模型。为了实现高效的知识学习和稳定的训练，我们采用了块扩散连续预训练（CPT）阶段，并通过定制的预热和逐块削波噪声计划进行了增强。在相同的数据和架构下，Stable-DiffCoder 在一系列广泛的代码基准测试中总体表现优于 AR 同行。此外，仅依靠 CPT 和监督微调阶段，Stable-DiffCoder 就实现了比各种 8B AR 和 DLLM 更强的性能，这表明基于扩散的训练可以提高代码建模质量，而不仅仅是 AR 训练。此外，基于扩散的任意顺序建模改进了用于编辑和推理的结构化代码建模，并通过数据增强使低资源编码语言受益。

Title: Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain

Authors: Özgür Uğur, Mahmut Göksu, Mahmut Çimen, Musa Yılmaz, Esra Şavirdi, Alp Talha Demir, Rumeysa Güllüce, İclal Çetin, Ömer Can Sağbaş
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.16018
Pdf URL: https://arxiv.org/pdf/2601.16018
Copy Paste: [[2601.16018]] Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain(https://arxiv.org/abs/2601.16018)
Keywords: language model
Abstract: This paper presents Mecellem models, a framework for developing specialized language models for the Turkish legal domain through domain adaptation strategies. We make two contributions: (1)Encoder Model Pre-trained from Scratch: ModernBERT-based bidirectional encoders pre-trained on a Turkish-dominant corpus of 112.7 billion tokens. We implement a checkpoint selection strategy that evaluates downstream retrieval performance throughout training, revealing that optimal checkpoints achieve best retrieval scores before pre-training loss reaches its minimum. Our encoder models achieve top-3 rankings on the Turkish retrieval leaderboard, with smaller models (155M parameters) achieving comparable performance to larger reference models (307M-567M parameters). Our approach achieves 92.36% production efficiency compared to state-of-the-art models (embeddinggemma-300m: 100.00%, BAAI/bge-m3: 99.54%, newmindai/bge-m3-stsb: 94.38%), ranking fourth overall despite requiring less computational resources. SOTA models rely on multi-stage, computationally intensive training pipelines, making our single-stage pre-training followed by efficient post-training approach a cost-effective alternative; (2)Decoder Model with Continual Pre-training (CPT): Qwen3-1.7B and Qwen3-4B models adapted to Turkish legal domain through controlled curriculum learning. Four-phase CPT with optimal sample ratios enables gradual transition from general language knowledge to specialized legal terminology and long-context reasoning. This approach achieves 36.2% perplexity reduction on Turkish legal text, demonstrating domain adaptation gains.
摘要：本文提出了 Mecellem 模型，这是一个通过领域适应策略为土耳其法律领域开发专门语言模型的框架。我们做出了两个贡献：（1）从头开始预训练的编码器模型：基于 ModernBERT 的双向编码器在土耳其语占主导地位的 1127 亿个令牌的语料库上进行预训练。我们实施了一种检查点选择策略，该策略在整个训练过程中评估下游检索性能，揭示最佳检查点在预训练损失达到最小值之前实现最佳检索分数。我们的编码器模型在土耳其检索排行榜上排名前三，较小的模型（155M 参数）实现了与较大参考模型（307M-567M 参数）相当的性能。与最先进的模型（embeddinggemma-300m：100.00％，BAAI/bge-m3：99.54％，newmindai/bge-m3-stsb：94.38％）相比，我们的方法实现了 92.36％的生产效率，尽管需要较少的计算资源，但总体排名第四。 SOTA 模型依赖于多阶段、计算密集型训练管道，使我们的单阶段预训练和高效的后训练方法成为一种经济高效的替代方案； (2)持续预训练(CPT)的解码器模型：通过受控课程学习适应土耳其法律领域的Qwen3-1.7B和Qwen3-4B模型。具有最佳样本比例的四阶段 CPT 能够从一般语言知识逐步过渡到专业法律术语和长上下文推理。这种方法使土耳其法律文本的困惑度降低了 36.2%，展示了领域适应的收益。

Title: Universal Refusal Circuits Across LLMs: Cross-Model Transfer via Trajectory Replay and Concept-Basis Reconstruction

Authors: Tony Cristofano
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.16034
Pdf URL: https://arxiv.org/pdf/2601.16034
Copy Paste: [[2601.16034]] Universal Refusal Circuits Across LLMs: Cross-Model Transfer via Trajectory Replay and Concept-Basis Reconstruction(https://arxiv.org/abs/2601.16034)
Keywords: gpt, llm
Abstract: Refusal behavior in aligned LLMs is often viewed as model-specific, yet we hypothesize it stems from a universal, low-dimensional semantic circuit shared across models. To test this, we introduce Trajectory Replay via Concept-Basis Reconstruction, a framework that transfers refusal interventions from donor to target models, spanning diverse architectures (e.g., Dense to MoE) and training regimes, without using target-side refusal supervision. By aligning layers via concept fingerprints and reconstructing refusal directions using a shared ``recipe'' of concept atoms, we map the donor's ablation trajectory into the target's semantic space. To preserve capabilities, we introduce a weight-SVD stability guard that projects interventions away from high-variance weight subspaces to prevent collateral damage. Our evaluation across 8 model pairs (including GPT-OSS-20B and GLM-4) confirms that these transferred recipes consistently attenuate refusal while maintaining performance, providing strong evidence for the semantic universality of safety alignment.
摘要：对齐的法学硕士中的拒绝行为通常被视为特定于模型的行为，但我们假设它源于跨模型共享的通用、低维语义电路。为了测试这一点，我们通过概念基础重建引入了轨迹重放，这是一个将拒绝干预从捐助者转移到目标模型的框架，跨越不同的架构（例如，密集到 MoE）和培训制度，而不使用目标方拒绝监督。通过概念指纹对齐层并使用概念原子的共享“配方”重建拒绝方向，我们将捐赠者的消融轨迹映射到目标的语义空间。为了保持能力，我们引入了权重 SVD 稳定性防护，将干预措施远离高方差权重子空间，以防止附带损害。我们对 8 个模型对（包括 GPT-OSS-20B 和 GLM-4）的评估证实，这些转移的配方在保持性能的同时一致地减少了拒绝，为安全对齐的语义普遍性提供了强有力的证据。

Title: Adapter Fusion for Multilingual Text2Cypher with Linear and Learned Gating

Authors: Makbule Gulcin Ozsoy
Subjects: cs.CL
Abstract URL: https://arxiv.org/abs/2601.16097
Pdf URL: https://arxiv.org/pdf/2601.16097
Copy Paste: [[2601.16097]] Adapter Fusion for Multilingual Text2Cypher with Linear and Learned Gating(https://arxiv.org/abs/2601.16097)
Keywords: language model
Abstract: Large Language Models enable users to access database using natural language interfaces using tools like Text2SQL, Text2SPARQL, and Text2Cypher, which translate user questions into structured database queries. While these systems improve database accessibility, most research focuses on English with limited multilingual support. This work investigates a scalable multilingual Text2Cypher, aiming to support new languages without re-running full fine-tuning, avoiding manual hyper-parameter tuning, and maintaining performance close to joint multilingual fine-tuning. We train language-specific LoRA adapters for English, Spanish, and Turkish and combined them via uniform linear merging or learned fusion MLP with dynamic gating. Experimental results show that the fusion MLP recovers around 75\% of the accuracy gains from joint multilingual fine-tuning while requiring only a smaller subset of the data, outperforming linear merging across all three languages. This approach enables incremental language expansion to new languages by requiring only one LoRA adapter and a lightweight MLP retraining. Learned adapter fusion offers a practical alternative to expensive joint fine-tuning, balancing performance, data efficiency, and scalability for multilingual Text2Cypher task.
摘要：大型语言模型使用户能够使用 Text2SQL、Text2SPARQL 和 Text2Cypher 等工具使用自然语言界面访问数据库，这些工具将用户问题转换为结构化数据库查询。虽然这些系统提高了数据库的可访问性，但大多数研究都集中在英语上，多语言支持有限。这项工作研究了可扩展的多语言 Text2Cypher，旨在支持新语言而无需重新运行完全微调，避免手动超参数调整，并保持接近联合多语言微调的性能。我们训练英语、西班牙语和土耳其语的特定语言 LoRA 适配器，并通过统一线性合并或学习融合 MLP 与动态门控将它们组合起来。实验结果表明，融合 MLP 从联合多语言微调中恢复了大约 75% 的精度增益，同时只需要较小的数据子集，优于所有三种语言的线性合并。这种方法只需一个 LoRA 适配器和一个轻量级 MLP 重新训练，即可将增量语言扩展到新语言。学习适配器融合为昂贵的联合微调、平衡多语言 Text2Cypher 任务的性能、数据效率和可扩展性提供了一种实用的替代方案。

Title: Improving Training Efficiency and Reducing Maintenance Costs via Language Specific Model Merging

Authors: Alphaeus Dmonte, Vidhi Gupta, Daniel J Perry, Mark Arehart
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.16127
Pdf URL: https://arxiv.org/pdf/2601.16127
Copy Paste: [[2601.16127]] Improving Training Efficiency and Reducing Maintenance Costs via Language Specific Model Merging(https://arxiv.org/abs/2601.16127)
Keywords: language model, llm
Abstract: Fine-tuning a task-specific multilingual large language model (LLM) involves training the model on a multilingual dataset with examples in all the required languages. Updating one or more supported languages with additional data or adding support for a new language involves retraining the model, which can be computationally inefficient and creates a severe maintenance bottleneck. Recent research on merging multilingual multitask models has shown promise in terms of improved quality, but its computational and maintenance efficiency remains unstudied. In this work, we provide the first focused analysis of this merging strategy from an efficiency perspective, evaluating it across three independent tasks. We demonstrate significant efficiency gains while maintaining parity in terms of quality: this merging approach reduces the initial training time by up to 50\%. We also demonstrate that updating an individual language and re-merging as part of model maintenance reduces training costs by more than 60\%, compared to re-training the full multilingual model. We show this on both public and proprietary industry datasets confirming that the approach works well for industrial use cases in addition to academic settings already studied in previous work.
摘要：微调特定于任务的多语言大语言模型 (LLM) 涉及在多语言数据集上训练模型，其中包含所有所需语言的示例。使用附加数据更新一种或多种支持的语言或添加对新语言的支持涉及重新训练模型，这可能会导致计算效率低下并造成严重的维护瓶颈。最近关于合并多语言多任务模型的研究在提高质量方面显示出了希望，但其计算和维护效率仍未得到研究。在这项工作中，我们从效率的角度首次对这种合并策略进行了集中分析，并在三个独立任务中对其进行了评估。我们展示了显着的效率提升，同时保持质量平等：这种合并方法将初始训练时间减少了高达 50%。我们还证明，与重新训练完整的多语言模型相比，更新单个语言并重新合并作为模型维护的一部分可降低 60% 以上的训练成本。我们在公共和专有行业数据集上展示了这一点，证实了除了之前工作中已经研究过的学术环境之外，该方法还适用于工业用例。

Title: LLM-in-Sandbox Elicits General Agentic Intelligence

Authors: Daixuan Cheng, Shaohan Huang, Yuxian Gu, Huatong Song, Guoxin Chen, Li Dong, Wayne Xin Zhao, Ji-Rong Wen, Furu Wei
Subjects: cs.CL, cs.AI
Abstract URL: https://arxiv.org/abs/2601.16206
Pdf URL: https://arxiv.org/pdf/2601.16206
Copy Paste: [[2601.16206]] LLM-in-Sandbox Elicits General Agentic Intelligence(https://arxiv.org/abs/2601.16206)
Keywords: llm, long context, agent
Abstract: We introduce LLM-in-Sandbox, enabling LLMs to explore within a code sandbox (i.e., a virtual computer), to elicit general intelligence in non-code domains. We first demonstrate that strong LLMs, without additional training, exhibit generalization capabilities to leverage the code sandbox for non-code tasks. For example, LLMs spontaneously access external resources to acquire new knowledge, leverage the file system to handle long contexts, and execute scripts to satisfy formatting requirements. We further show that these agentic capabilities can be enhanced through LLM-in-Sandbox Reinforcement Learning (LLM-in-Sandbox-RL), which uses only non-agentic data to train models for sandbox exploration. Experiments demonstrate that LLM-in-Sandbox, in both training-free and post-trained settings, achieves robust generalization spanning mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following. Finally, we analyze LLM-in-Sandbox's efficiency from computational and system perspectives, and open-source it as a Python package to facilitate real-world deployment.
摘要：我们引入了 LLM-in-Sandbox，使 LLM 能够在代码沙箱（即虚拟计算机）中进行探索，以激发非代码领域的通用智能。我们首先证明，强大的法学硕士无需额外培训即可表现出利用代码沙箱执行非代码任务的泛化能力。例如，法学硕士自发访问外部资源以获取新知识，利用文件系统处理长上下文，并执行脚本以满足格式要求。我们进一步表明，这些代理能力可以通过LLM-in-Sandbox强化学习（LLM-in-Sandbox-RL）得到增强，该学习仅使用非代理数据来训练沙箱探索模型。实验表明，LLM-in-Sandbox 在免训练和训练后环境中都实现了涵盖数学、物理、化学、生物医学、长上下文理解和指令遵循的稳健泛化。最后，我们从计算和系统角度分析LLM-in-Sandbox的效率，并将其作为Python包开源以方便实际部署。