2025-12-09

Title: Empathy by Design: Aligning Large Language Models for Healthcare Dialogue

• Authors: Emre Umucu, Guillermina Solis, Leon Garza, Emilia Rivas, Beatrice Lee, Anantaa Kotal, Aritran Piplai
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06097
• Pdf URL: https://arxiv.org/pdf/2512.06097
• Copy Paste: [[2512.06097]] Empathy by Design: Aligning Large Language Models for Healthcare Dialogue(https://arxiv.org/abs/2512.06097)
• Keywords: language model, llm
• Abstract: General-purpose large language models (LLMs) have demonstrated remarkable generative and reasoning capabilities but remain limited in healthcare and caregiving applications due to two key deficiencies: factual unreliability and a lack of empathetic communication. These shortcomings pose significant risks in sensitive contexts where users, particularly non-professionals and caregivers, seek medically relevant guidance or emotional reassurance. To address these challenges, we introduce a Direct Preference Optimization (DPO)-based alignment framework designed to improve factual correctness, semantic coherence, and human-centric qualities such as empathy, politeness, and simplicity in caregiver-patient dialogues. Our approach fine-tunes domain-adapted LLMs using pairwise preference data, where preferred responses reflect supportive and accessible communication styles while rejected ones represent prescriptive or overly technical tones. This direct optimization method aligns model outputs with human preferences more efficiently than traditional reinforcement-learning-based alignment. Empirical evaluations across multiple open and proprietary LLMs show that our DPO-tuned models achieve higher semantic alignment, improved factual accuracy, and stronger human-centric evaluation scores compared to baseline and commercial alternatives such as Google medical dialogue systems. These improvements demonstrate that preference-based alignment offers a scalable and transparent pathway toward developing trustworthy, empathetic, and clinically informed AI assistants for caregiver and healthcare communication. Our open-source code is available at: this https URL
• 摘要：通用大语言模型（LLM）已经表现出卓越的生成和推理能力，但由于两个关键缺陷：事实不可靠和缺乏同理心沟通，在医疗保健和护理应用中仍然受到限制。这些缺点在用户（尤其是非专业人士和护理人员）寻求医学相关指导或情感保证的敏感环境中构成重大风险。为了应对这些挑战，我们引入了基于直接偏好优化（DPO）的对齐框架，旨在提高事实正确性、语义一致性和以人为本的品质，例如护理人员与患者对话中的同理心、礼貌和简单性。我们的方法使用成对偏好数据对适应领域的法学硕士进行微调，其中首选的响应反映了支持性和可访问的沟通方式，而被拒绝的响应则代表了规定性或过于技术性的语气。这种直接优化方法比传统的基于强化学习的对齐更有效地使模型输出与人类偏好对齐。对多个开放和专有法学硕士的实证评估表明，与基线和商业替代方案（例如谷歌医疗对话系统）相比，我们的 DPO 调整模型实现了更高的语义一致性、更高的事实准确性和更强的以人为中心的评估分数。这些改进表明，基于偏好的调整提供了一条可扩展且透明的途径，为护理人员和医疗保健沟通开发值得信赖、富有同理心和临床知情的人工智能助手。我们的开源代码位于：此 https URL

Title: Do You Feel Comfortable? Detecting Hidden Conversational Escalation in AI Chatbots

• Authors: Jihyung Park, Saleh Afroogh, Junfeng Jiao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06193
• Pdf URL: https://arxiv.org/pdf/2512.06193
• Copy Paste: [[2512.06193]] Do You Feel Comfortable? Detecting Hidden Conversational Escalation in AI Chatbots(https://arxiv.org/abs/2512.06193)
• Keywords: language model, llm, chat
• Abstract: Large Language Models (LLM) are increasingly integrated into everyday interactions, serving not only as information assistants but also as emotional companions. Even in the absence of explicit toxicity, repeated emotional reinforcement or affective drift can gradually escalate distress in a form of \textit{implicit harm} that traditional toxicity filters fail to detect. Existing guardrail mechanisms often rely on external classifiers or clinical rubrics that may lag behind the nuanced, real-time dynamics of a developing conversation. To address this gap, we propose GAUGE (Guarding Affective Utterance Generation Escalation), a lightweight, logit-based framework for the real-time detection of hidden conversational escalation. GAUGE measures how an LLM's output probabilistically shifts the affective state of a dialogue.
• 摘要：大型语言模型 (LLM) 越来越多地融入日常交互中，不仅充当信息助手，而且充当情感伴侣。即使没有明显的毒性，反复的情绪强化或情感漂移也会以传统毒性过滤器无法检测到的\textit{隐性伤害}的形式逐渐升级痛苦。现有的护栏机制通常依赖于外部分类器或临床评估标准，这些分类器或临床评估标准可能落后于正在发展的对话的细致入微的实时动态。为了解决这一差距，我们提出了 GAUGE（保护情感话语生成升级），这是一个轻量级的、基于 logit 的框架，用于实时检测隐藏的对话升级。 GAUGE衡量法学硕士的输出如何概率地改变对话的情感状态。

Title: Automated Data Enrichment using Confidence-Aware Fine-Grained Debate among Open-Source LLMs for Mental Health and Online Safety

• Authors: Junyu Mao, Anthony Hills, Talia Tseriotou, Maria Liakata, Aya Shamir, Dan Sayda, Dana Atzil-Slonim, Natalie Djohari, Arpan Mandal, Silke Roth, Pamela Ugwudike, Mahesan Niranjan, Stuart E. Middleton
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.06227
• Pdf URL: https://arxiv.org/pdf/2512.06227
• Copy Paste: [[2512.06227]] Automated Data Enrichment using Confidence-Aware Fine-Grained Debate among Open-Source LLMs for Mental Health and Online Safety(https://arxiv.org/abs/2512.06227)
• Keywords: llm, agent
• Abstract: Real-world indicators are important for improving natural language processing (NLP) tasks such as life events for mental health analysis and risky behaviour for online safety, yet labelling such information in NLP training datasets is often costly and/or difficult given the dynamic nature of such events. This paper compares several LLM-based data enrichment methods and introduces a novel Confidence-Aware Fine-Grained Debate (CFD) framework in which multiple LLM agents simulate human annotators and exchange fine-grained evidence to reach consensus. We describe two new expert-annotated datasets, a mental health Reddit wellbeing dataset and an online safety Facebook sharenting risk dataset. Our CFD framework achieves the most robust data enrichment performance compared to a range of baselines and we show that this type of data enrichment consistently improves downstream tasks. Enriched features incorporated via debate transcripts yield the largest gains, outperforming the non-enriched baseline by 10.1% for the online safety task.
• 摘要：现实世界的指标对于改善自然语言处理 (NLP) 任务（例如用于心理健康分析的生活事件和用于在线安全的危险行为）非常重要，但考虑到此类事件的动态性质，在 NLP 训练数据集中标记此类信息通常成本高昂和/或困难。本文比较了几种基于 LLM 的数据丰富方法，并介绍了一种新颖的置信感知细粒度辩论 (CFD) 框架，其中多个 LLM 代理模拟人类注释者并交换细粒度证据以达成共识。我们描述了两个新的专家注释数据集：心理健康 Reddit 健康数据集和在线安全 Facebook 共享风险数据集。与一系列基线相比，我们的 CFD 框架实现了最强大的数据丰富性能，并且我们表明这种类型的数据丰富能够持续改善下游任务。通过辩论记录纳入的丰富特征产生了最大的收益，在在线安全任务中比非丰富基线高出 10.1%。

Title: Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judge

• Authors: Xuanxin Wu, Yuki Arase, Masaaki Nagata
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06228
• Pdf URL: https://arxiv.org/pdf/2512.06228
• Copy Paste: [[2512.06228]] Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judge(https://arxiv.org/abs/2512.06228)
• Keywords: language model, gpt, llm
• Abstract: Sentence simplification aims to modify a sentence to make it easier to read and understand while preserving the meaning. Different applications require distinct simplification policies, such as replacing only complex words at the lexical level or rewriting the entire sentence while trading off details for simplicity. However, achieving such policy-driven control remains an open challenge. In this work, we introduce a simple yet powerful approach that leverages Large Language Model-as-a-Judge (LLM-as-a-Judge) to automatically construct policy-aligned training data, completely removing the need for costly human annotation or parallel corpora. Our method enables building simplification systems that adapt to diverse simplification policies. Remarkably, even small-scale open-source LLMs such as Phi-3-mini-3.8B surpass GPT-4o on lexical-oriented simplification, while achieving comparable performance on overall rewriting, as verified by both automatic metrics and human evaluations. The consistent improvements across model families and sizes demonstrate the robustness of our approach.
• 摘要：句子简化的目的是修改句子，使其更容易阅读和理解，同时保留含义。不同的应用程序需要不同的简化策略，例如仅替换词汇级别的复杂单词或重写整个句子，同时为了简单性而牺牲细节。然而，实现这种政策驱动的控制仍然是一个公开的挑战。在这项工作中，我们引入了一种简单而强大的方法，利用大型语言模型作为法官（LLM-as-a-Judge）自动构建与策略一致的训练数据，完全消除了对昂贵的人工注释或并行语料库的需要。我们的方法能够构建适应不同简化策略的简化系统。值得注意的是，即使是像 Phi-3-mini-3.8B 这样的小型开源法学硕士，在面向词汇的简化方面也超越了 GPT-4o，同时在整体重写方面实现了可比的性能，这一点经过自动指标和人工评估的验证。跨型号系列和尺寸的一致改进证明了我们方法的稳健性。

Title: LOCUS: A System and Method for Low-Cost Customization for Universal Specialization

• Authors: Dhanasekar Sundararaman, Keying Li, Wayne Xiong, Aashna Garg
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06239
• Pdf URL: https://arxiv.org/pdf/2512.06239
• Copy Paste: [[2512.06239]] LOCUS: A System and Method for Low-Cost Customization for Universal Specialization(https://arxiv.org/abs/2512.06239)
• Keywords: gpt
• Abstract: We present LOCUS (LOw-cost Customization for Universal Specialization), a pipeline that consumes few-shot data to streamline the construction and training of NLP models through targeted retrieval, synthetic data generation, and parameter-efficient tuning. With only a small number of labeled examples, LOCUS discovers pertinent data in a broad repository, synthesizes additional training samples via in-context data generation, and fine-tunes models using either full or low-rank (LoRA) parameter adaptation. Our approach targets named entity recognition (NER) and text classification (TC) benchmarks, consistently outperforming strong baselines (including GPT-4o) while substantially lowering costs and model sizes. Our resultant memory-optimized models retain 99% of fully fine-tuned accuracy while using barely 5% of the memory footprint, also beating GPT-4o on several benchmarks with less than 1% of its parameters.
• 摘要：我们提出了 LOCUS（通用专业化的低成本定制），这是一个使用少量数据的管道，通过有针对性的检索、合成数据生成和参数高效调整来简化 NLP 模型的构建和训练。只需少量标记示例，LOCUS 即可在广泛的存储库中发现相关数据，通过上下文数据生成合成额外的训练样本，并使用全秩或低秩 (LoRA) 参数自适应来微调模型。我们的方法针对命名实体识别 (NER) 和文本分类 (TC) 基准，始终优于强大的基准（包括 GPT-4o），同时大幅降低成本和模型大小。我们最终的内存优化模型保留了 99% 的完全微调精度，同时仅使用 5% 的内存占用，并且在多个基准测试中以不到 1% 的参数击败了 GPT-4o。

Title: Convergence of Outputs When Two Large Language Models Interact in a Multi-Agentic Setup

• Authors: Aniruddha Maiti, Satya Nimmagadda, Kartha Veerya Jammuladinne, Niladri Sengupta, Ananya Jana
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06256
• Pdf URL: https://arxiv.org/pdf/2512.06256
• Copy Paste: [[2512.06256]] Convergence of Outputs When Two Large Language Models Interact in a Multi-Agentic Setup(https://arxiv.org/abs/2512.06256)
• Keywords: language model, prompt, agent
• Abstract: In this work, we report what happens when two large language models respond to each other for many turns without any outside input in a multi-agent setup. The setup begins with a short seed sentence. After that, each model reads the other's output and generates a response. This continues for a fixed number of steps. We used Mistral Nemo Base 2407 and Llama 2 13B hf. We observed that most conversations start coherently but later fall into repetition. In many runs, a short phrase appears and repeats across turns. Once repetition begins, both models tend to produce similar output rather than introducing a new direction in the conversation. This leads to a loop where the same or similar text is produced repeatedly. We describe this behavior as a form of convergence. It occurs even though the models are large, trained separately, and not given any prompt instructions. To study this behavior, we apply lexical and embedding-based metrics to measure how far the conversation drifts from the initial seed and how similar the outputs of the two models becomes as the conversation progresses.
• 摘要：在这项工作中，我们报告了当两个大型语言模型在多代理设置中没有任何外部输入的情况下多次相互响应时会发生什么。设置以一个简短的种子句开始。之后，每个模型都会读取另一个模型的输出并生成响应。这将持续固定数量的步骤。我们使用 Mistral Nemo Base 2407 和 Llama 2 13B hf。我们观察到，大多数对话一开始都是连贯的，但后来就会陷入重复。在许多跑步中，都会出现一个简短的短语，并在各个回合中重复出现。一旦重复开始，两种模型都倾向于产生相似的输出，而不是在对话中引入新的方向。这会导致重复生成相同或相似文本的循环。我们将这种行为描述为一种收敛形式。即使模型很大、单独训练并且没有给出任何提示指令，这种情况也会发生。为了研究这种行为，我们应用基于词汇和嵌入的指标来测量对话与初始种子的偏离程度以及随着对话的进展两个模型的输出变得有多相似。

Title: Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Models

• Authors: Chen Yang, Guangyue Peng, Jiaying Zhu, Ran Le, Ruixiang Feng, Tao Zhang, Wei Ruan, Xiaoqi Liu, Xiaoxue Cheng, Xiyun Xu, Yang Song, Yanzipeng Gao, Yiming Jia, Yun Xing, Yuntao Wen, Zekai Wang, Zhenwei An, Zhicong Sun, Zongchao Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06266
• Pdf URL: https://arxiv.org/pdf/2512.06266
• Copy Paste: [[2512.06266]] Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Models(https://arxiv.org/abs/2512.06266)
• Keywords: language model, chain-of-thought
• Abstract: We present Nanbeige4-3B, a family of small-scale but high-performing language models. Pretrained on 23T high-quality tokens and finetuned on over 30 million diverse instructions, we extend the boundary of the scaling law for small language models. In pre-training, we design a Fine-Grained Warmup-Stable-Decay (FG-WSD) training scheduler, which progressively refines data mixtures across stages to boost model performance. In post-training, to improve the quality of the SFT data, we design a joint mechanism that integrates deliberative generation refinement and chain-of-thought reconstruction, yielding substantial gains on complex tasks. Following SFT, we employ our flagship reasoning model to distill Nanbeige4-3B through our proposed Dual Preference Distillation (DPD) method, which leads to further performance gains. Finally, a multi-stage reinforcement learning phase was applied, leveraging verifiable rewards and preference modeling to strengthen abilities on both reasoning and human alignment. Extensive evaluations show that Nanbeige4-3B not only significantly outperforms models of comparable parameter scale but also rivals much larger models across a wide range of benchmarks. The model checkpoints are available at this https URL.
• 摘要：我们提出了 Nanb​​eige4-3B，这是一个小规模但高性能的语言模型家族。我们在 23T 高质量 token 上进行了预训练，并对超过 3000 万条不同指令进行了微调，扩展了小语言模型的缩放法则的边界。在预训练中，我们设计了一个细粒度预热-稳定-衰减（FG-WSD）训练调度程序，它逐步细化各个阶段的数据混合，以提高模型性能。在训练后，为了提高SFT数据的质量，我们设计了一种集深思熟虑的生成细化和思想链重建于一体的联合机制，在复杂的任务上获得了可观的收益。在 SFT 之后，我们采用我们的旗舰推理模型通过我们提出的双偏好蒸馏 (DPD) 方法来蒸馏 Nanb​​eige4-3B，从而进一步提高性能。最后，应用了多阶段强化学习阶段，利用可验证的奖励和偏好模型来增强推理和人类对齐的能力。广泛的评估表明，Nanbeige4-3B 不仅显着优于具有可比参数规模的模型，而且在各种基准测试中也可与更大的模型相媲美。模型检查点可从此 https URL 获取。

Title: Knowing What's Missing: Assessing Information Sufficiency in Question Answering

• Authors: Akriti Jain, Aparna Garimella
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06476
• Pdf URL: https://arxiv.org/pdf/2512.06476
• Copy Paste: [[2512.06476]] Knowing What's Missing: Assessing Information Sufficiency in Question Answering(https://arxiv.org/abs/2512.06476)
• Keywords: prompt
• Abstract: Determining whether a provided context contains sufficient information to answer a question is a critical challenge for building reliable question-answering systems. While simple prompting strategies have shown success on factual questions, they frequently fail on inferential ones that require reasoning beyond direct text extraction. We hypothesize that asking a model to first reason about what specific information is missing provides a more reliable, implicit signal for assessing overall sufficiency. To this end, we propose a structured Identify-then-Verify framework for robust sufficiency modeling. Our method first generates multiple hypotheses about missing information and establishes a semantic consensus. It then performs a critical verification step, forcing the model to re-examine the source text to confirm whether this information is truly absent. We evaluate our method against established baselines across diverse multi-hop and factual QA datasets. The results demonstrate that by guiding the model to justify its claims about missing information, our framework produces more accurate sufficiency judgments while clearly articulating any information gaps.
• 摘要：确定所提供的上下文是否包含足够的信息来回答问题是构建可靠的问答系统的关键挑战。虽然简单的提示策略在事实问题上取得了成功，但在需要超出直接文本提取的推理的推理问题上却经常失败。我们假设要求模型首先推理缺少哪些特定信息可以为评估整体充分性提供更可靠、更隐含的信号。为此，我们提出了一个结构化的“识别然后验证”框架，用于稳健的充分性建模。我们的方法首先生成关于缺失信息的多个假设并建立语义共识。然后，它执行关键的验证步骤，迫使模型重新检查源文本以确认该信息是否确实不存在。我们根据跨不同多跳和事实 QA 数据集的既定基线评估我们的方法。结果表明，通过指导模型证明其关于缺失信息的主张的合理性，我们的框架可以产生更准确的充分性判断，同时清楚地阐明任何信息差距。

Title: Classifying German Language Proficiency Levels Using Large Language Models

• Authors: Elias-Leander Ahlers, Witold Brunsmann, Malte Schilling
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06483
• Pdf URL: https://arxiv.org/pdf/2512.06483
• Copy Paste: [[2512.06483]] Classifying German Language Proficiency Levels Using Large Language Models(https://arxiv.org/abs/2512.06483)
• Keywords: language model, llm, prompt
• Abstract: Assessing language proficiency is essential for education, as it enables instruction tailored to learners needs. This paper investigates the use of Large Language Models (LLMs) for automatically classifying German texts according to the Common European Framework of Reference for Languages (CEFR) into different proficiency levels. To support robust training and evaluation, we construct a diverse dataset by combining multiple existing CEFR-annotated corpora with synthetic data. We then evaluate prompt-engineering strategies, fine-tuning of a LLaMA-3-8B-Instruct model and a probing-based approach that utilizes the internal neural state of the LLM for classification. Our results show a consistent performance improvement over prior methods, highlighting the potential of LLMs for reliable and scalable CEFR classification.
• 摘要：评估语言能力对于教育至关重要，因为它可以根据学习者的需求提供量身定制的教学。本文研究了使用大型语言模型 (LLM) 根据欧洲共同语言参考框架 (CEFR) 将德语文本自动分类为不同的熟练程度。为了支持稳健的训练和评估，我们通过将多个现有的 CEFR 注释语料库与合成数据相结合来构建多样化的数据集。然后，我们评估即时工程策略、LLaMA-3-8B-Instruct 模型的微调以及利用 LLM 的内部神经状态进行分类的基于探测的方法。我们的结果显示，与之前的方法相比，性能得到了一致的改进，突显了法学硕士在可靠且可扩展的 CEFR 分类方面的潜力。

Title: ProSocialAlign: Preference Conditioned Test Time Alignment in Language Models

• Authors: Somnath Banerjee, Sayan Layek, Sayantan Adak, Mykola Pechenizkiy, Animesh Mukherjee, Rima Hazra
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06515
• Pdf URL: https://arxiv.org/pdf/2512.06515
• Copy Paste: [[2512.06515]] ProSocialAlign: Preference Conditioned Test Time Alignment in Language Models(https://arxiv.org/abs/2512.06515)
• Keywords: language model
• Abstract: Current language model safety paradigms often fall short in emotionally charged or high-stakes settings, where refusal-only approaches may alienate users and naive compliance can amplify risk. We propose ProSocialAlign, a test-time, parameter-efficient framework that steers generation toward safe, empathetic, and value-aligned responses without retraining the base model. We formalize five human-centered objectives and cast safety as lexicographic constrained generation: first, applying hard constraints to eliminate harmful continuations; then optimizing for prosocial quality within the safe set. Our method combines (i) directional regulation, a harm-mitigation mechanism that subtracts a learned "harm vector" in parameter space, and (ii) preference-aware autoregressive reward modeling trained jointly across attributes with gradient conflict resolution, enabling fine-grained, user-controllable decoding. Empirical evaluations across five safety benchmarks demonstrate state-of-the-art performance, reducing unsafe leakage and boosting alignment to human values, with strong gains across multiple evaluation metrics. ProSocialAlign offers a robust and modular foundation for generating context-sensitive, safe, and human-aligned responses at inference time.
• 摘要：当前的语言模型安全范式在情绪激动或高风险的环境中往往达不到要求，在这些环境中，仅拒绝的方法可能会疏远用户，而幼稚的合规性可能会放大风险。我们提出了 ProSocialAlign，这是一个测试时、参数高效的框架，可引导一代人做出安全、同理心和价值一致的响应，而无需重新训练基础模型。我们正式确定了五个以人为本的目标，并将安全视为字典式约束生成：首先，应用硬约束来消除有害的延续；然后在安全集中优化亲社会质量。我们的方法结合了（i）定向调节，一种在参数空间中减去学习到的“危害向量”的危害缓解机制，以及（ii）跨属性与梯度冲突解决联合训练的偏好感知自回归奖励模型，从而实现细粒度、用户可控的解码。对五个安全基准的实证评估展示了最先进的性能，减少了不安全泄漏并促进了与人类价值观的一致性，并在多个评估指标上取得了巨大的进步。 ProSocialAlign 提供了强大的模块化基础，用于在推理时生成上下文敏感、安全且人性化的响应。

Title: Mechanistic Interpretability of GPT-2: Lexical and Contextual Layers in Sentiment Analysis

• Authors: Amartya Hatua
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06681
• Pdf URL: https://arxiv.org/pdf/2512.06681
• Copy Paste: [[2512.06681]] Mechanistic Interpretability of GPT-2: Lexical and Contextual Layers in Sentiment Analysis(https://arxiv.org/abs/2512.06681)
• Keywords: language model, gpt
• Abstract: We present a mechanistic interpretability study of GPT-2 that causally examines how sentiment information is processed across its transformer layers. Using systematic activation patching across all 12 layers, we test the hypothesized two-stage sentiment architecture comprising early lexical detection and mid-layer contextual integration. Our experiments confirm that early layers (0-3) act as lexical sentiment detectors, encoding stable, position specific polarity signals that are largely independent of context. However, all three contextual integration hypotheses: Middle Layer Concentration, Phenomenon Specificity, and Distributed Processing are falsified. Instead of mid-layer specialization, we find that contextual phenomena such as negation, sarcasm, domain shifts etc. are integrated primarily in late layers (8-11) through a unified, non-modular mechanism. These experimental findings provide causal evidence that GPT-2's sentiment computation differs from the predicted hierarchical pattern, highlighting the need for further empirical characterization of contextual integration in large language models.
• 摘要：我们提出了 GPT-2 的机械可解释性研究，该研究因果性地检查了情感信息如何跨其转换器层进行处理。使用跨所有 12 层的系统激活补丁，我们测试了假设的两阶段情感架构，包括早期词汇检测和中间层上下文集成。我们的实验证实，早期层（0-3）充当词汇情感检测器，编码稳定的、位置特定的极性信号，这些信号很大程度上独立于上下文。然而，所有三个上下文整合假设：中间层集中、现象特异性和分布式处理都是证伪的。我们发现，诸如否定、讽刺、领域转移等上下文现象主要通过统一的非模块化机制集成在后期层（8-11）中，而不是中间层的专门化。这些实验结果提供了 GPT-2 的情感计算与预测的层次模式不同的因果证据，强调需要对大型语言模型中的上下文集成进行进一步的实证表征。

Title: PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory

• Authors: Bowen Jiang, Yuan Yuan, Maohao Shen, Zhuoqun Hao, Zhangchen Xu, Zichen Chen, Ziyi Liu, Anvesh Rao Vijjini, Jiashu He, Hanchao Yu, Radha Poovendran, Gregory Wornell, Lyle Ungar, Dan Roth, Sihao Chen, Camillo Jose Taylor
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06688
• Pdf URL: https://arxiv.org/pdf/2512.06688
• Copy Paste: [[2512.06688]] PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory(https://arxiv.org/abs/2512.06688)
• Keywords: gpt, llm, long context, chat, agent
• Abstract: Personalization is one of the next milestones in advancing AI capability and alignment. We introduce PersonaMem-v2, the state-of-the-art dataset for LLM personalization that simulates 1,000 realistic user-chatbot interactions on 300+ scenarios, 20,000+ user preferences, and 128k-token context windows, where most user preferences are implicitly revealed to reflect real-world interactions. Using this data, we investigate how reinforcement fine-tuning enables a model to improve its long-context reasoning capabilities for user understanding and personalization. We also develop a framework for training an agentic memory system, which maintains a single, human-readable memory that grows with each user over time. In our experiments, frontier LLMs still struggle with implicit personalization, achieving only 37-48% accuracy. While they support long context windows, reasoning remains the bottleneck for implicit personalization tasks. Using reinforcement fine-tuning, we successfully train Qwen3-4B to outperforms GPT-5, reaching 53% accuracy in implicit personalization. Moreover, our agentic memory framework achieves state-of-the-art 55% accuracy while using 16x fewer input tokens, relying on a 2k-token memory instead of full 32k conversation histories. These results underscore the impact of our dataset and demonstrate agentic memory as a scalable path toward real-world personalized intelligence.
• 摘要：个性化是推进人工智能功能和一致性的下一个里程碑之一。我们引入了 PersonaMem-v2，这是最先进的 LLM 个性化数据集，它模拟了 300 多个场景、20,000 多个用户偏好和 128k 令牌上下文窗口中的 1,000 次真实的用户聊天机器人交互，其中大多数用户偏好都被隐式揭示以反映现实世界的交互。使用这些数据，我们研究了强化微调如何使模型能够提高其长上下文推理能力，以实现用户理解和个性化。我们还开发了一个用于训练代理记忆系统的框架，该系统维护一个单一的、人类可读的记忆，并随着每个用户的时间而增长。在我们的实验中，前沿法学硕士仍然在隐性个性化方面挣扎，仅达到 37-48% 的准确率。虽然它们支持长上下文窗口，但推理仍然是隐式个性化任务的瓶颈。通过强化微调，我们成功训练 Qwen3-4B，其性能优于 GPT-5，隐式个性化准确率达到 53%。此外，我们的代理记忆框架实现了最先进的 55% 准确率，同时使用的输入令牌减少了 16 倍，依赖于 2k 令牌内存而不是完整的 32k 对话历史记录。这些结果强调了我们的数据集的影响，并证明代理记忆是实现现实世界个性化智能的可扩展路径。

Title: Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generation

• Authors: Chengbing Wang, Yang Zhang, Wenjie Wang, Xiaoyan Zhao, Fuli Feng, Xiangnan He, Tat-Seng Chua
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06690
• Pdf URL: https://arxiv.org/pdf/2512.06690
• Copy Paste: [[2512.06690]] Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generation(https://arxiv.org/abs/2512.06690)
• Keywords: language model, llm, prompt
• Abstract: Preference alignment has enabled large language models (LLMs) to better reflect human expectations, but current methods mostly optimize for population-level preferences, overlooking individual users. Personalization is essential, yet early approaches-such as prompt customization or fine-tuning-struggle to reason over implicit preferences, limiting real-world effectiveness. Recent "think-then-generate" methods address this by reasoning before response generation. However, they face challenges in long-form generation: their static one-shot reasoning must capture all relevant information for the full response generation, making learning difficult and limiting adaptability to evolving content. To address this issue, we propose FlyThinker, an efficient "think-while-generating" framework for personalized long-form generation. FlyThinker employs a separate reasoning model that generates latent token-level reasoning in parallel, which is fused into the generation model to dynamically guide response generation. This design enables reasoning and generation to run concurrently, ensuring inference efficiency. In addition, the reasoning model is designed to depend only on previous responses rather than its own prior outputs, which preserves training parallelism across different positions-allowing all reasoning tokens for training data to be produced in a single forward pass like standard LLM training, ensuring training efficiency. Extensive experiments on real-world benchmarks demonstrate that FlyThinker achieves better personalized generation while keeping training and inference efficiency.
• 摘要：偏好对齐使大型语言模型（LLM）能够更好地反映人类的期望，但当前的方法主要针对人群级别的偏好进行优化，而忽略了个人用户。个性化至关重要，但早期的方法（例如即时定制或微调）很难推理隐含的偏好，从而限制了现实世界的有效性。最近的“思考然后生成”方法通过在响应生成之前进行推理来解决这个问题。然而，它们在长格式生成方面面临挑战：它们的静态一次性推理必须捕获完整响应生成的所有相关信息，这使得学习变得困难并限制了对不断变化的内容的适应性。为了解决这个问题，我们提出了 FlyThinker，一种高效的“边生成边思考”框架，用于个性化长格式生成。 FlyThinker 采用单独的推理模型，并行生成潜在令牌级推理，并将其融合到生成模型中以动态指导响应生成。这种设计使得推理和生成同时运行，保证了推理效率。此外，推理模型被设计为仅依赖于先前的响应，而不是其自身的先前输出，这保留了不同位置之间的训练并行性——允许训练数据的所有推理标记像标准 LLM 训练一样在单个前向传递中生成，从而确保训练效率。对现实世界基准的大量实验表明，FlyThinker 在保持训练和推理效率的同时实现了更好的个性化生成。

Title: Parameter-Efficient Fine-Tuning with Differential Privacy for Robust Instruction Adaptation in Large Language Models

• Authors: Yulin Huang, Yaxuan Luan, Jinxu Guo, Xiangchen Song, Yuchen Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06711
• Pdf URL: https://arxiv.org/pdf/2512.06711
• Copy Paste: [[2512.06711]] Parameter-Efficient Fine-Tuning with Differential Privacy for Robust Instruction Adaptation in Large Language Models(https://arxiv.org/abs/2512.06711)
• Keywords: language model
• Abstract: This study addresses the issues of privacy protection and efficiency in instruction fine-tuning of large-scale language models by proposing a parameter-efficient method that integrates differential privacy noise allocation with gradient clipping in a collaborative optimization framework. The method keeps the backbone model frozen and updates parameters through a low-dimensional projection subspace, while introducing clipping and adaptive noise allocation during gradient computation. This design reduces privacy budget consumption and ensures training stability and robustness. The unified framework combines gradient constraints, noise allocation, and parameter projection, effectively mitigating performance fluctuations and privacy risks in multi-task instruction scenarios. Experiments are conducted across hyperparameter, environment, and data sensitivity dimensions. Results show that the method outperforms baseline models in accuracy, privacy budget, and parameter efficiency, and maintains stable performance under diverse and uncertain data conditions. The findings enrich the theoretical integration of differential privacy and parameter-efficient fine-tuning and demonstrate its practical adaptability in instruction tasks, providing a feasible solution for secure training in complex instruction environments.
• 摘要：本研究通过提出一种在协作优化框架中将差分隐私噪声分配与梯度裁剪相结合的参数有效方法，解决了大规模语言模型指令微调中的隐私保护和效率问题。该方法保持骨干模型冻结并通过低维投影子空间更新参数，同时在梯度计算期间引入裁剪和自适应噪声分配。这种设计减少了隐私预算消耗，保证了训练的稳定性和鲁棒性。统一框架结合了梯度约束、噪声分配和参数投影，有效缓解多任务指令场景中的性能波动和隐私风险。实验是在超参数、环境和数据敏感度维度上进行的。结果表明，该方法在准确性、隐私预算和参数效率方面优于基线模型，并且在多样化和不确定的数据条件下保持稳定的性能。研究结果丰富了差分隐私和参数高效微调的理论结合，并展示了其在教学任务中的实际适应性，为复杂教学环境中的安全训练提供了可行的解决方案。

Title: "The Dentist is an involved parent, the bartender is not": Revealing Implicit Biases in QA with Implicit BBQ

• Authors: Aarushi Wagh, Saniya Srivastava
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06732
• Pdf URL: https://arxiv.org/pdf/2512.06732
• Copy Paste: [[2512.06732]] "The Dentist is an involved parent, the bartender is not": Revealing Implicit Biases in QA with Implicit BBQ(https://arxiv.org/abs/2512.06732)
• Keywords: language model, gpt, llm, prompt
• Abstract: Existing benchmarks evaluating biases in large language models (LLMs) primarily rely on explicit cues, declaring protected attributes like religion, race, gender by name. However, real-world interactions often contain implicit biases, inferred subtly through names, cultural cues, or traits. This critical oversight creates a significant blind spot in fairness evaluation. We introduce ImplicitBBQ, a benchmark extending the Bias Benchmark for QA (BBQ) with implicitly cued protected attributes across 6 categories. Our evaluation of GPT-4o on ImplicitBBQ illustrates troubling performance disparity from explicit BBQ prompts, with accuracy declining up to 7% in the "sexual orientation" subcategory and consistent decline located across most other categories. This indicates that current LLMs contain implicit biases undetected by explicit benchmarks. ImplicitBBQ offers a crucial tool for nuanced fairness evaluation in NLP.
• 摘要：评估大型语言模型 (LLM) 偏差的现有基准主要依赖于明确的线索，通过名称声明宗教、种族、性别等受保护的属性。然而，现实世界的互动通常包含隐含的偏见，这些偏见是通过名字、文化线索或特征巧妙地推断出来的。这种严重的监督在公平性评估中造成了重大盲点。我们引入了 ImplicitBBQ，这是一个扩展 QA (BBQ) 偏差基准的基准，具有跨 6 个类别的隐式提示受保护属性。我们对 ImplicitBBQ 上的 GPT-4o 的评估表明，显式烧烤提示的性能差异令人不安，“性取向”子类别的准确率下降了 7%，而大多数其他类别的准确率也持续下降。这表明当前的法学硕士包含显式基准未检测到的隐性偏差。 ImplicitBBQ 为 NLP 中细致入微的公平性评估提供了一个重要工具。

Title: A Patient-Doctor-NLP-System to contest inequality for less privileged

• Authors: Subrit Dikshit, Ritu Tiwari, Priyank Jain
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06734
• Pdf URL: https://arxiv.org/pdf/2512.06734
• Copy Paste: [[2512.06734]] A Patient-Doctor-NLP-System to contest inequality for less privileged(https://arxiv.org/abs/2512.06734)
• Keywords: language model, llm
• Abstract: Transfer Learning (TL) has accelerated the rapid development and availability of large language models (LLMs) for mainstream natural language processing (NLP) use cases. However, training and deploying such gigantic LLMs in resource-constrained, real-world healthcare situations remains challenging. This study addresses the limited support available to visually impaired users and speakers of low-resource languages such as Hindi who require medical assistance in rural environments. We propose PDFTEMRA (Performant Distilled Frequency Transformer Ensemble Model with Random Activations), a compact transformer-based architecture that integrates model distillation, frequency-domain modulation, ensemble learning, and randomized activation patterns to reduce computational cost while preserving language understanding performance. The model is trained and evaluated on medical question-answering and consultation datasets tailored to Hindi and accessibility scenarios, and its performance is compared against standard NLP state-of-the-art model baselines. Results demonstrate that PDFTEMRA achieves comparable performance with substantially lower computational requirements, indicating its suitability for accessible, inclusive, low-resource medical NLP applications.
• 摘要：迁移学习 (TL) 加速了主流自然语言处理 (NLP) 用例的大型语言模型 (LLM) 的快速开发和可用性。然而，在资源有限的现实医疗保健环境中培训和部署如此庞大的法学硕士仍然具有挑战性。这项研究解决了在农村环境中需要医疗援助的视障用户和讲印地语等低资源语言的人可获得的有限支持的问题。我们提出了 PDFTEMRA（具有随机激活的高性能蒸馏频率变换器集成模型），这是一种基于变压器的紧凑架构，集成了模型蒸馏、频域调制、集成学习和随机激活模式，以降低计算成本，同时保持语言理解性能。该模型在针对印地语和无障碍场景定制的医疗问答和咨询数据集上进行训练和评估，并将其性能与标准 NLP 最先进的模型基线进行比较。结果表明，PDFTEMRA 以显着降低的计算要求实现了可比的性能，表明它适合可访问、包容性、低资源的医学 NLP 应用。

Title: One Word Is Not Enough: Simple Prompts Improve Word Embeddings

• Authors: Rajeev Ranjan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06744
• Pdf URL: https://arxiv.org/pdf/2512.06744
• Copy Paste: [[2512.06744]] One Word Is Not Enough: Simple Prompts Improve Word Embeddings(https://arxiv.org/abs/2512.06744)
• Keywords: prompt
• Abstract: Text embedding models are designed for sentence-level applications like retrieval and semantic similarity, and are primarily evaluated on sentence-level benchmarks. Their behavior on isolated words is less understood. We show that simply prepending semantic prompts to words before embedding substantially improves word similarity correlations. Testing 7 text embedding models, including text-embedding-3-large (OpenAI), embed-english-v3.0 (Cohere), voyage-3(Voyage AI), all-mpnet-base-v2, and Qwen3-Embedding-8B, on 3 standard benchmarks (SimLex-999, WordSim-353, MEN-3000), we find that prompts like "meaning: {word}" or "Represent the semantic concept: {word}" improve Spearman correlations by up to +0.29 on SimLex-999. Some models fail completely on bare words (correlation = 0) but recover with prompts (+0.73 improvement). Our best results achieve correlation = 0.692 on SimLex-999 with embed-english-v3.0 (Cohere), correlation = 0.811 on WordSim-353, and correlation = 0.855 on MEN-3000 with text-embedding-3-large (OpenAI). These results outperform classic static embeddings like Word2Vec (correlation = 0.40) and even the best static method LexVec (correlation = 0.48) on SimLex-999, establishing a new state-of-the-art for pure embedding methods. This zero-shot technique requires no training and works with any text embedding model.
• 摘要：文本嵌入模型是为检索和语义相似性等句子级应用而设计的，并且主要在句子级基准上进行评估。他们对孤立词的行为知之甚少。我们表明，在嵌入之前简单地在单词前面添加语义提示可以显着改善单词相似度相关性。在 3 个标准基准（SimLex-999、WordSim-353、MEN-3000）上测试 7 个文本嵌入模型，包括 text-embedding-3-large (OpenAI)、embed-english-v3.0 (Cohere)、voyage-3(Voyage AI)、all-mpnet-base-v2 和 Qwen3-Embedding-8B，我们发现提示类似“含义： {word}”或“表示语义概念：{word}”在 SimLex-999 上将 Spearman 相关性提高高达 +0.29。有些模型在单纯的单词上完全失败（相关性 = 0），但通过提示恢复（+0.73 改进）。我们的最佳结果在使用 embed-english-v3.0 (Cohere) 的 SimLex-999 上实现相关性 = 0.692，在 WordSim-353 上实现相关性 = 0.811，在使用 text-embedding-3-large (OpenAI) 的 MEN-3000 上实现相关性 = 0.855。这些结果优于 Word2Vec（相关性 = 0.40）等经典静态嵌入，甚至优于 SimLex-999 上最好的静态方法 LexVec（相关性 = 0.48），为纯嵌入方法建立了新的最先进技术。这种零样本技术不需要训练，并且适用于任何文本嵌入模型。

Title: Becoming Experienced Judges: Selective Test-Time Learning for Evaluators

• Authors: Seungyeon Jwa, Daechul Ahn, Reokyoung Kim, Dongyeop Kang, Jonghyun Choi
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.06751
• Pdf URL: https://arxiv.org/pdf/2512.06751
• Copy Paste: [[2512.06751]] Becoming Experienced Judges: Selective Test-Time Learning for Evaluators(https://arxiv.org/abs/2512.06751)
• Keywords: language model, llm, prompt
• Abstract: Automatic evaluation with large language models, commonly known as LLM-as-a-judge, is now standard across reasoning and alignment tasks. Despite evaluating many samples in deployment, these evaluators typically (i) treat each case independently, missing the opportunity to accumulate experience, and (ii) rely on a single fixed prompt for all cases, neglecting the need for sample-specific evaluation criteria. We introduce Learning While Evaluating (LWE), a framework that allows evaluators to improve sequentially at inference time without requiring training or validation sets. LWE maintains an evolving meta-prompt that (i) produces sample-specific evaluation instructions and (ii) refines itself through self-generated feedback. Furthermore, we propose Selective LWE, which updates the meta-prompt only on self-inconsistent cases, focusing computation where it matters most. This selective approach retains the benefits of sequential learning while being far more cost-effective. Across two pairwise comparison benchmarks, Selective LWE outperforms strong baselines, empirically demonstrating that evaluators can improve during sequential testing with a simple selective update, learning most from the cases they struggle with.
• 摘要：使用大型语言模型进行自动评估（通常称为 LLM-as-a-judge）现在已成为推理和对齐任务的标准。尽管评估了部署中的许多样本，但这些评估人员通常（i）独立处理每个案例，错过了积累经验的机会，并且（ii）对所有案例依赖单一固定提示，忽略了对特定于样本的评估标准的需要。我们引入了边评估边学习（LWE），这是一个框架，允许评估者在推理时连续改进，而不需要训练或验证集。 LWE 维护着一个不断发展的元提示，它 (i) 生成特定于样本的评估指令，并且 (ii) 通过自我生成的反馈来完善自​​身。此外，我们提出了选择性 LWE，它仅在自我不一致的情况下更新元提示，将计算集中在最重要的地方。这种选择性方法保留了顺序学习的优点，同时更具成本效益。在两个成对比较基准中，选择性 LWE 的表现优于强基线，经验证明评估者可以通过简单的选择性更新在顺序测试中进行改进，从他们遇到的案例中学到最多的东西。

Title: From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs

• Authors: Yuchuan Tian, Yuchen Liang, Jiacheng Sun, Shuo Zhang, Guangwen Yang, Yingte Shu, Sibo Fang, Tianyu Guo, Kai Han, Chao Xu, Hanting Chen, Xinghao Chen, Yunhe Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06776
• Pdf URL: https://arxiv.org/pdf/2512.06776
• Copy Paste: [[2512.06776]] From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs(https://arxiv.org/abs/2512.06776)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) excel at generation but dominant autoregressive (AR) decoding is inherently sequential, creating a throughput bottleneck. Diffusion Language Models (DLMs)--especially block-wise variants--enable parallel generation and intra-block bidirectional reasoning, yet training large DLMs from scratch is costly and wastes the knowledge in mature AR checkpoints. Prior "adaptation" attempts either modify logits or randomly grow attention masks to full-sequence diffusion, or simply transplant AR weights into a block-diffusion recipe, leaving a fundamental mismatch between AR causality and block-wise bidirectionality unaddressed. We reframe adaptation as a intra-paradigm path from AR to Block-Diffusion by viewing AR as Block-Diffusion with blocksize=1. Concretely, we design the pathway of adaptation as follows: we use a context-causal attention mask (causal in context, bidirectional only within the active block), an efficient parallel adaptation procedure, an auxiliary AR loss to maximize data utilization and retain pretrained knowledge, and gradual increment of the generation block size. The recipe integrates cleanly with masked block-diffusion and maintains train-inference consistency. Built on these components, NBDiff-7B (Base and Instruct) could inherit the long-context modeling and reasoning capabilities, and achieve state-of-the-art performance among the 7B-class DLMs, delivering strong gains on general-knowledge, math, and code benchmarks over strong baselines. These results demonstrate that principled AR-to-block-diffusion adaptation is an effective and compute-efficient alternative to training DLMs from scratch. Codes: this https URL.
• 摘要：大型语言模型 (LLM) 擅长生成，但占主导地位的自回归 (AR) 解码本质上是顺序的，从而产生了吞吐量瓶颈。扩散语言模型 (DLM)——尤其是块式变体——支持并行生成和块内双向推理，但从头开始训练大型 DLM 成本高昂，并且浪费了成熟 AR 检查点中的知识。之前的“适应”尝试要么修改逻辑，要么随机增加注意力掩模以进行全序列扩散，或者简单地将 AR 权重移植到块扩散配方中，从而使 AR 因果关系和块双向性之间的根本不匹配问题得不到解决。我们将 AR 视为块大小 = 1 的块扩散，将自适应重新定义为从 AR 到块扩散的范式内路径。具体来说，我们设计的适应路径如下：我们使用上下文因果注意掩模（上下文中的因果关系，仅在活动块内双向），高效的并行自适应过程，辅助AR损失以最大化数据利用率并保留预训练的知识，以及逐渐增加生成块大小。该配方与屏蔽块扩散干净地集成，并保持训练推理的一致性。基于这些组件构建的 NBDiff-7B（Base 和 Instruct）可以继承长上下文建模和推理功能，并在 7B 级 DLM 中实现最先进的性能，在通用知识、数学和代码基准方面比强大的基准提供强劲的收益。这些结果表明，有原则的 AR 到块扩散自适应是从头开始训练 DLM 的有效且计算效率高的替代方案。代码：此 https URL。

Title: LLM4SFC: Sequential Function Chart Generation via Large Language Models

• Authors: Ofek Glick, Vladimir Tchuiev, Marah Ghoummaid, Michal Moshkovitz, Dotan Di-Castro
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06787
• Pdf URL: https://arxiv.org/pdf/2512.06787
• Copy Paste: [[2512.06787]] LLM4SFC: Sequential Function Chart Generation via Large Language Models(https://arxiv.org/abs/2512.06787)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: While Large Language Models (LLMs) are increasingly used for synthesizing textual PLC programming languages like Structured Text (ST) code, other IEC 61131-3 standard graphical languages like Sequential Function Charts (SFCs) remain underexplored. Generating SFCs is challenging due to graphical nature and ST actions embedded within, which are not directly compatible with standard generation techniques, often leading to non-executable code that is incompatible with industrial tool-chains In this work, we introduce LLM4SFC, the first framework to receive natural-language descriptions of industrial workflows and provide executable SFCs. LLM4SFC is based on three components: (i) A reduced structured representation that captures essential topology and in-line ST and reduced textual verbosity; (ii) Fine-tuning and few-shot retrieval-augmented generation (RAG) for alignment with SFC programming conventions; and (iii) A structured generation approach that prunes illegal tokens in real-time to ensure compliance with the textual format of SFCs. We evaluate LLM4SFC on a dataset of real-world SFCs from automated manufacturing projects, using both open-source and proprietary LLMs. The results show that LLM4SFC reliably generates syntactically valid SFC programs effectively bridging graphical and textual PLC languages, achieving a generation generation success of 75% - 94%, paving the way for automated industrial programming.
• 摘要：虽然大型语言模型 (LLM) 越来越多地用于合成文本 PLC 编程语言（例如结构化文本 (ST) 代码），但其他 IEC 61131-3 标准图形语言（例如顺序功能图 (SFC)）仍未得到充分开发。由于图形性质和嵌入的 ST 操作与标准生成技术不直接兼容，生成 SFC 具有挑战性，通常会导致与工业工具链不兼容的不可执行代码。在这项工作中，我们引入了 LLM4SFC，这是第一个接收工业工作流程的自然语言描述并提供可执行 SFC 的框架。 LLM4SFC 基于三个组件： (i) 简化的结构化表示，可捕获基本拓扑和内联 ST 并减少文本冗长； (ii) 微调和少样本检索增强生成 (RAG)，以符合 SFC 编程惯例； (iii) 结构化生成方法，实时修剪非法代币，以确保符合证监会的文本格式。我们使用开源和专有的 LLM 在自动化制造项目的真实 SFC 数据集上评估 LLM4SFC。结果表明，LLM4SFC 能够可靠地生成语法有效的 SFC 程序，有效地桥接图形和文本 PLC 语言，实现了 75% - 94% 的生成成功率，为自动化工业编程铺平了道路。

Title: Large Language Model-Based Generation of Discharge Summaries

• Authors: Tiago Rodrigues, Carla Teixeira Lopes
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06812
• Pdf URL: https://arxiv.org/pdf/2512.06812
• Copy Paste: [[2512.06812]] Large Language Model-Based Generation of Discharge Summaries(https://arxiv.org/abs/2512.06812)
• Keywords: language model, gpt, llm, hallucination, prompt
• Abstract: Discharge Summaries are documents written by medical professionals that detail a patient's visit to a care facility. They contain a wealth of information crucial for patient care, and automating their generation could significantly reduce the effort required from healthcare professionals, minimize errors, and ensure that critical patient information is easily accessible and actionable. In this work, we explore the use of five Large Language Models on this task, from open-source models (Mistral, Llama 2) to proprietary systems (GPT-3, GPT-4, Gemini 1.5 Pro), leveraging MIMIC-III summaries and notes. We evaluate them using exact-match, soft-overlap, and reference-free metrics. Our results show that proprietary models, particularly Gemini with one-shot prompting, outperformed others, producing summaries with the highest similarity to the gold-standard ones. Open-source models, while promising, especially Mistral after fine-tuning, lagged in performance, often struggling with hallucinations and repeated information. Human evaluation by a clinical expert confirmed the practical utility of the summaries generated by proprietary models. Despite the challenges, such as hallucinations and missing information, the findings suggest that LLMs, especially proprietary models, are promising candidates for automatic discharge summary generation as long as data privacy is ensured.
• 摘要：出院小结是由医疗专业人员撰写的文件，详细介绍了患者前往护理机构就诊的情况。它们包含对患者护理至关重要的大量信息，自动化生成这些信息可以显着减少医疗保健专业人员所需的工作量，最大限度地减少错误，并确保关键患者信息易于访问和可操作。在这项工作中，我们利用 MIMIC-III 摘要和注释，探索了在此任务中使用五种大型语言模型，从开源模型（Mistral、Llama 2）到专有系统（GPT-3、GPT-4、Gemini 1.5 Pro）。我们使用精确匹配、软重叠和无参考指标来评估它们。我们的结果表明，专有模型，特别是具有一次性提示的 Gemini，优于其他模型，生成的摘要与黄金标准的摘要具有最高的相似度。开源模型虽然很有前景，尤其是经过微调后的 Mistral，但其性能却滞后，经常与幻觉和重复信息作斗争。临床专家的人体评估证实了专有模型生成的摘要的实用性。尽管存在幻觉和信息缺失等挑战，但研究结果表明，只要确保数据隐私，法学硕士，尤其是专有模型，都是自动生成出院摘要的有希望的候选者。

Title: Rhea: Role-aware Heuristic Episodic Attention for Conversational LLMs

• Authors: Wanyang Hong, Zhaoning Zhang, Yi Chen, Libo Zhang, Baihui Liu, Linbo Qiao, Zhiliang Tian, Dongsheng Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06869
• Pdf URL: https://arxiv.org/pdf/2512.06869
• Copy Paste: [[2512.06869]] Rhea: Role-aware Heuristic Episodic Attention for Conversational LLMs(https://arxiv.org/abs/2512.06869)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have achieved remarkable performance on single-turn tasks, yet their effectiveness deteriorates in multi-turn conversations. We define this phenomenon as cumulative contextual decay - a progressive degradation of contextual integrity caused by attention pollution, dilution, and drift. To address this challenge, we propose Rhea (Role-aware Heuristic Episodic Attention), a novel framework that decouples conversation history into two functionally independent memory modules: (1) an Instructional Memory (IM) that persistently stores high-fidelity global constraints via a structural priority mechanism, and (2) an Episodic Memory (EM) that dynamically manages user-model interactions via asymmetric noise control and heuristic context retrieval. During inference, Rhea constructs a high signal-to-noise context by applying its priority attention: selectively integrating relevant episodic information while always prioritizing global instructions. To validate this approach, experiments on multiple multi-turn conversation benchmarks - including MT-Eval and Long-MT-Bench+ - show that Rhea mitigates performance decay and improves overall accuracy by 1.04 points on a 10-point scale (a 16% relative gain over strong baselines). Moreover, Rhea maintains near-perfect instruction fidelity (IAR > 8.1) across long-horizon interactions. These results demonstrate that Rhea provides a principled and effective framework for building more precise, instruction-consistent conversational LLMs.
• 摘要：大型语言模型（LLM）在单轮任务上取得了显着的性能，但在多轮对话中其有效性却下降了。我们将这种现象定义为累积上下文衰减——由注意力污染、稀释和漂移引起的上下文完整性的逐渐退化。为了应对这一挑战，我们提出了Rhea（角色感知启发式情景注意力），这是一种新颖的框架，它将对话历史解耦为两个功能独立的记忆模块：（1）通过结构优先级机制持久存储高保真全局约束的指令记忆（IM），以及（2）情景记忆（EM），通过不对称噪声控制和启发式上下文检索动态管理用户模型交互。在推理过程中，Rhea 通过应用其优先注意力构建了一个高信噪比的上下文：选择性地整合相关的情景信息，同时始终优先考虑全局指令。为了验证这种方法，对多个多轮对话基准（包括 MT-Eval 和 Long-MT-Bench+）进行的实验表明，Rhea 减轻了性能衰减，并将整体准确率提高了 1.04 个点（按 10 分制计算）（与强基线相比，相对增益为 16%）。此外，Rhea 在长视野交互中保持近乎完美的指令保真度（IAR > 8.1）。这些结果表明，Rhea 为构建更精确、指令一致的对话式法学硕士提供了一个有原则且有效的框架。

Title: An Analysis of Large Language Models for Simulating User Responses in Surveys

• Authors: Ziyun Yu, Yiru Zhou, Chen Zhao, Hongyi Wen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06874
• Pdf URL: https://arxiv.org/pdf/2512.06874
• Copy Paste: [[2512.06874]] An Analysis of Large Language Models for Simulating User Responses in Surveys(https://arxiv.org/abs/2512.06874)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Using Large Language Models (LLMs) to simulate user opinions has received growing attention. Yet LLMs, especially trained with reinforcement learning from human feedback (RLHF), are known to exhibit biases toward dominant viewpoints, raising concerns about their ability to represent users from diverse demographic and cultural backgrounds. In this work, we examine the extent to which LLMs can simulate human responses to cross-domain survey questions through direct prompting and chain-of-thought prompting. We further propose a claim diversification method CLAIMSIM, which elicits viewpoints from LLM parametric knowledge as contextual input. Experiments on the survey question answering task indicate that, while CLAIMSIM produces more diverse responses, both approaches struggle to accurately simulate users. Further analysis reveals two key limitations: (1) LLMs tend to maintain fixed viewpoints across varying demographic features, and generate single-perspective claims; and (2) when presented with conflicting claims, LLMs struggle to reason over nuanced differences among demographic features, limiting their ability to adapt responses to specific user profiles.
• 摘要：使用大型语言模型（LLM）来模拟用户意见已受到越来越多的关注。然而，众所周知，法学硕士，尤其是接受过人类反馈强化学习 (RLHF) 训练的法学硕士，会对主流观点表现出偏见，引发人们对其代表来自不同人口和文化背景的用户的能力的担忧。在这项工作中，我们研究了法学硕士可以通过直接提示和思维链提示来模拟人类对跨领域调查问题的反应的程度。我们进一步提出了一种索赔多样化方法 CLAIMSIM，该方法从 LLM 参数知识中引出观点作为上下文输入。调查问答任务的实验表明，虽然 CLAIMSIM 产生了更加多样化的响应，但这两种方法都难以准确地模拟用户。进一步的分析揭示了两个关键的局限性：（1）法学硕士倾向于在不同的人口特征上保持固定的观点，并产生单一观点的主张； (2) 当提出相互矛盾的主张时，法学硕士很难推理出人口特征之间的细微差别，从而限制了他们根据特定用户资料调整响应的能力。

Title: Large Language Models and Forensic Linguistics: Navigating Opportunities and Threats in the Age of Generative AI

• Authors: George Mikros
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2512.06922
• Pdf URL: https://arxiv.org/pdf/2512.06922
• Copy Paste: [[2512.06922]] Large Language Models and Forensic Linguistics: Navigating Opportunities and Threats in the Age of Generative AI(https://arxiv.org/abs/2512.06922)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) present a dual challenge for forensic linguistics. They serve as powerful analytical tools enabling scalable corpus analysis and embedding-based authorship attribution, while simultaneously destabilising foundational assumptions about idiolect through style mimicry, authorship obfuscation, and the proliferation of synthetic texts. Recent stylometric research indicates that LLMs can approximate surface stylistic features yet exhibit detectable differences from human writers, a tension with significant forensic implications. However, current AI-text detection techniques, whether classifier-based, stylometric, or watermarking approaches, face substantial limitations: high false positive rates for non-native English writers and vulnerability to adversarial strategies such as homoglyph substitution. These uncertainties raise concerns under legal admissibility standards, particularly the Daubert and Kumho Tire frameworks. The article concludes that forensic linguistics requires methodological reconfiguration to remain scientifically credible and legally admissible. Proposed adaptations include hybrid human-AI workflows, explainable detection paradigms beyond binary classification, and validation regimes measuring error and bias across diverse populations. The discipline's core insight, i.e., that language reveals information about its producer, remains valid but must accommodate increasingly complex chains of human and machine authorship.
• 摘要：大型语言模型（LLM）给法庭语言学带来了双重挑战。它们作为强大的分析工具，可实现可扩展的语料库分析和基于嵌入的作者归属，同时通过风格模仿​​、作者身份混淆和合成文本的扩散来破坏关于个人语言的基本假设。最近的文体测量研究表明，法学硕士可以近似表面文体特征，但又表现出与人类作家可察觉的差异，这种张力具有重大的法医意义。然而，当前的人工智能文本检测技术，无论是基于分类器、风格测量还是水印方法，都面临着巨大的局限性：非英语母语作家的误报率很高，而且容易受到同形文字替换等对抗策略的影响。这些不确定性引起了对法律受理标准的担忧，特别是 Daubert 和 Kumho Tire 框架。文章的结论是，法庭语言学需要方法论上的重新配置，以保持科学上的可信度和法律上的可接受性。拟议的调整包括混合人类人工智能工作流程、超越二元分类的可解释检测范式，以及测量不同人群的错误和偏差的验证机制。该学科的核心见解，即语言揭示了有关其生产者的信息，仍然有效，但必须适应日益复杂的人类和机器作者链。

Title: Progress Ratio Embeddings: An Impatience Signal for Robust Length Control in Neural Text Generation

• Authors: Ivanhoé Botcazou, Tassadit Amghar, Sylvain Lamprier, Frédéric Saubion
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.06938
• Pdf URL: https://arxiv.org/pdf/2512.06938
• Copy Paste: [[2512.06938]] Progress Ratio Embeddings: An Impatience Signal for Robust Length Control in Neural Text Generation(https://arxiv.org/abs/2512.06938)
• Keywords: language model
• Abstract: Modern neural language models achieve high accuracy in text generation, yet precise control over generation length remains underdeveloped. In this paper, we first investigate a recent length control method based on Reverse Positional Embeddings (RPE) and show its limits when control is requested beyond the training distribution. In particular, using a discrete countdown signal tied to the absolute remaining token count leads to instability. To provide robust length control, we introduce Progress Ratio Embeddings (PRE), as continuous embeddings tied to a trigonometric impatience signal. PRE integrates seamlessly into standard Transformer architectures, providing stable length fidelity without degrading text accuracy under standard evaluation metrics. We further show that PRE generalizes well to unseen target lengths. Experiments on two widely used news-summarization benchmarks validate these findings.
• 摘要：现代神经语言模型在文本生成方面实现了高精度，但对生成长度的精确控制仍然不发达。在本文中，我们首先研究了一种基于反向位置嵌入（RPE）的最新长度控制方法，并展示了当控制请求超出训练分布时其局限性。特别是，使用与绝对剩余令牌计数相关的离散倒计时信号会导致不稳定。为了提供稳健的长度控制，我们引入了进度比嵌入（PRE），作为与三角不耐烦信号相关的连续嵌入。 PRE 无缝集成到标准 Transformer 架构中，提供稳定的长度保真度，而不会降低标准评估指标下的文本准确性。我们进一步表明 PRE 可以很好地推广到看不见的目标长度。对两个广泛使用的新闻摘要基准的实验验证了这些发现。

Title: Prompting-in-a-Series: Psychology-Informed Contents and Embeddings for Personality Recognition With Decoder-Only Models

• Authors: Jing Jie Tan, Ban-Hoe Kwan, Danny Wee-Kiat Ng, Yan-Chai Hum, Anissa Mokraoui, Shih-Yu Lo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.06991
• Pdf URL: https://arxiv.org/pdf/2512.06991
• Copy Paste: [[2512.06991]] Prompting-in-a-Series: Psychology-Informed Contents and Embeddings for Personality Recognition With Decoder-Only Models(https://arxiv.org/abs/2512.06991)
• Keywords: language model, gpt, llm, prompt
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across various natural language processing tasks. This research introduces a novel "Prompting-in-a-Series" algorithm, termed PICEPR (Psychology-Informed Contents Embeddings for Personality Recognition), featuring two pipelines: (a) Contents and (b) Embeddings. The approach demonstrates how a modularised decoder-only LLM can summarize or generate content, which can aid in classifying or enhancing personality recognition functions as a personality feature extractor and a generator for personality-rich content. We conducted various experiments to provide evidence to justify the rationale behind the PICEPR algorithm. Meanwhile, we also explored closed-source models such as \textit{gpt4o} from OpenAI and \textit{gemini} from Google, along with open-source models like \textit{mistral} from Mistral AI, to compare the quality of the generated content. The PICEPR algorithm has achieved a new state-of-the-art performance for personality recognition by 5-15\% improvement. The work repository and models' weight can be found at this https URL.
• 摘要：大型语言模型 (LLM) 在各种自然语言处理任务中表现出了卓越的能力。这项研究引入了一种新颖的“系列提示”算法，称为 PICEPR（用于人格识别的心理学知情内容嵌入），具有两个管道：(a) 内容和 (b) 嵌入。该方法演示了仅模块化解码器的法学硕士如何总结或生成内容，这可以帮助分类或增强个性识别功能，作为个性特征提取器和丰富个性内容的生成器。我们进行了各种实验来提供证据来证明 PICEPR 算法背后的基本原理。同时，我们还探索了 OpenAI 的 \textit{gpt4o} 和 Google 的 \textit{gemini} 等闭源模型，以及 Mistral AI 的 \textit{mistral} 等开源模型，以比较生成内容的质量。 PICEPR 算法在个性识别方面取得了新的最先进的性能，提高了 5-15%。工作存储库和模型的权重可以在此 https URL 中找到。

Title: FVA-RAG: Falsification-Verification Alignment for Mitigating Sycophantic Hallucinations

• Authors: Mayank Ravishankara
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2512.07015
• Pdf URL: https://arxiv.org/pdf/2512.07015
• Copy Paste: [[2512.07015]] FVA-RAG: Falsification-Verification Alignment for Mitigating Sycophantic Hallucinations(https://arxiv.org/abs/2512.07015)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) systems have significantly reduced hallucinations in Large Language Models (LLMs) by grounding responses in external context. However, standard RAG architectures suffer from a critical vulnerability: Retrieval Sycophancy. When presented with a query based on a false premise or a common misconception, vector-based retrievers tend to fetch documents that align with the user's bias rather than objective truth, leading the model to "hallucinate with citations." In this work, we introduce Falsification-Verification Alignment RAG (FVA-RAG), a framework that shifts the retrieval paradigm from Inductive Verification (seeking support) to Deductive Falsification (seeking disproof). Unlike existing "Self-Correction" methods that rely on internal consistency, FVA-RAG deploys a distinct Adversarial Retrieval Policy that actively generates "Kill Queries"-targeted search terms designed to surface contradictory evidence. We introduce a dual-verification mechanism that explicitly weighs the draft answer against this "Anti-Context." Preliminary experiments on a dataset of common misconceptions demonstrate that FVA-RAG significantly improves robustness against sycophantic hallucinations compared to standard RAG baselines, effectively acting as an inference-time "Red Team" for factual generation.
• 摘要：检索增强生成（RAG）系统通过将响应置于外部环境中，显着减少了大型语言模型（LLM）中的幻觉。然而，标准 RAG 架构存在一个严重漏洞：检索谄媚。当提出基于错误前提或常见误解的查询时，基于向量的检索器往往会获取符合用户偏见而不是客观事实的文档，从而导致模型“产生引用的幻觉”。在这项工作中，我们引入了证伪验证对齐 RAG (FVA-RAG)，该框架将检索范式从归纳验证（寻求支持）转变为演绎证伪（寻求反驳）。与依赖内部一致性的现有“自我更正”方法不同，FVA-RAG 部署了独特的对抗性检索策略，该策略主动生成“杀死查询”目标的搜索词，旨在显示矛盾的证据。我们引入了一种双重验证机制，明确权衡草案答案与“反上下文”。对常见误解数据集的初步实验表明，与标准 RAG 基线相比，FVA-RAG 显着提高了针对阿谀奉承幻觉的鲁棒性，有效地充当了事实生成的推理时间“红队”。

Title: Replicating TEMPEST at Scale: Multi-Turn Adversarial Attacks Against Trillion-Parameter Frontier Models

• Authors: Richard Young
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07059
• Pdf URL: https://arxiv.org/pdf/2512.07059
• Copy Paste: [[2512.07059]] Replicating TEMPEST at Scale: Multi-Turn Adversarial Attacks Against Trillion-Parameter Frontier Models(https://arxiv.org/abs/2512.07059)
• Keywords: language model
• Abstract: Despite substantial investment in safety alignment, the vulnerability of large language models to sophisticated multi-turn adversarial attacks remains poorly characterized, and whether model scale or inference mode affects robustness is unknown. This study employed the TEMPEST multi-turn attack framework to evaluate ten frontier models from eight vendors across 1,000 harmful behaviors, generating over 97,000 API queries across adversarial conversations with automated evaluation by independent safety classifiers. Results demonstrated a spectrum of vulnerability: six models achieved 96% to 100% attack success rate (ASR), while four showed meaningful resistance, with ASR ranging from 42% to 78%; enabling extended reasoning on identical architecture reduced ASR from 97% to 42%. These findings indicate that safety alignment quality varies substantially across vendors, that model scale does not predict adversarial robustness, and that thinking mode provides a deployable safety enhancement. Collectively, this work establishes that current alignment techniques remain fundamentally vulnerable to adaptive multi-turn attacks regardless of model scale, while identifying deliberative inference as a promising defense direction.
• 摘要：尽管在安全调整方面投入了大量资金，但大型语言模型对复杂的多轮对抗攻击的脆弱性仍然知之甚少，并且模型规模或推理模式是否影响鲁棒性尚不清楚。这项研究采用 TEMPEST 多轮攻击框架来评估来自 8 个供应商的 10 个前沿模型的 1,000 种有害行为，通过独立安全分类器的自动评估，在对​​抗性对话中生成超过 97,000 个 API 查询。结果显示了一系列漏洞：六种模型实现了 96% 至 100% 的攻击成功率 (ASR)，而四种模型显示出有意义的抵抗力，ASR 范围为 42% 至 78%；在相同架构上启用扩展推理将 ASR 从 97% 降低到 42%。这些发现表明，不同供应商的安全一致性质量差异很大，模型规模不能预测对抗的鲁棒性，并且思维模式提供了可部署的安全增强功能。总的来说，这项工作表明，无论模型规模如何，当前的对齐技术从根本上仍然容易受到自适应多轮攻击，同时将故意推理确定为有前途的防御方向。

Title: Do Large Language Models Truly Understand Cross-cultural Differences?

• Authors: Shiwei Guo, Sihang Jiang, Qianxi He, Yanghua Xiao, Jiaqing Liang, Bi Yude, Minggui He, Shimin Tao, Li Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07075
• Pdf URL: https://arxiv.org/pdf/2512.07075
• Copy Paste: [[2512.07075]] Do Large Language Models Truly Understand Cross-cultural Differences?(https://arxiv.org/abs/2512.07075)
• Keywords: language model, llm
• Abstract: In recent years, large language models (LLMs) have demonstrated strong performance on multilingual tasks. Given its wide range of applications, cross-cultural understanding capability is a crucial competency. However, existing benchmarks for evaluating whether LLMs genuinely possess this capability suffer from three key limitations: a lack of contextual scenarios, insufficient cross-cultural concept mapping, and limited deep cultural reasoning capabilities. To address these gaps, we propose SAGE, a scenario-based benchmark built via cross-cultural core concept alignment and generative task design, to evaluate LLMs' cross-cultural understanding and reasoning. Grounded in cultural theory, we categorize cross-cultural capabilities into nine dimensions. Using this framework, we curated 210 core concepts and constructed 4530 test items across 15 specific real-world scenarios, organized under four broader categories of cross-cultural situations, following established item design principles. The SAGE dataset supports continuous expansion, and experiments confirm its transferability to other languages. It reveals model weaknesses across both dimensions and scenarios, exposing systematic limitations in cross-cultural reasoning. While progress has been made, LLMs are still some distance away from reaching a truly nuanced cross-cultural understanding. In compliance with the anonymity policy, we include data and code in the supplement materials. In future versions, we will make them publicly available online.
• 摘要：近年来，大型语言模型（LLM）在多语言任务上表现出了强大的性能。鉴于其广泛的应用，跨文化理解能力是一项至关重要的能力。然而，现有的评估法学硕士是否真正具备这种能力的基准受到三个关键限制：缺乏情境场景、跨文化概念映射不足以及深度文化推理能力有限。为了解决这些差距，我们提出了 SAGE，这是一个通过跨文化核心概念对齐和生成任务设计构建的基于场景的基准，用于评估法学硕士的跨文化理解和推理。基于文化理论，我们将跨文化能力分为九个维度。使用这个框架，我们策划了 210 个核心概念，并在 15 个特定的现实世界场景中构建了 4530 个测试项目，按照既定的项目设计原则，按照四个更广泛的跨文化场景类别进行组织。 SAGE数据集支持持续扩展，实验证实了其向其他语言的可移植性。它揭示了模型在维度和场景上的弱点，暴露了跨文化推理的系统局限性。尽管已经取得了进展，但法学硕士距离真正细致入微的跨文化理解还有一段距离。根据匿名政策，我们在补充材料中包含数据和代码。在未来的版本中，我们将在网上公开它们。

Title: Leveraging KV Similarity for Online Structured Pruning in LLMs

• Authors: Jungmin Lee, Gwangeun Byeon, Yulhwa Kim, Seokin Hong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.07090
• Pdf URL: https://arxiv.org/pdf/2512.07090
• Copy Paste: [[2512.07090]] Leveraging KV Similarity for Online Structured Pruning in LLMs(https://arxiv.org/abs/2512.07090)
• Keywords: language model, llm
• Abstract: Pruning has emerged as a promising direction for accelerating large language model (LLM) inference, yet existing approaches often suffer from instability because they rely on offline calibration data that may not generalize across inputs. In this work, we introduce Token Filtering, a lightweight online structured pruning technique that makes pruning decisions directly during inference without any calibration data. The key idea is to measure token redundancy via joint key-value similarity and skip redundant attention computations, thereby reducing inference cost while preserving critical information. To further enhance stability, we design a variance-aware fusion strategy that adaptively weights key and value similarity across heads, ensuring that informative tokens are retained even under high pruning ratios. This design introduces no additional memory overhead and provides a more reliable criterion for token importance. Extensive experiments on LLaMA-2 (7B/13B), LLaMA-3 (8B), and Mistral (7B) demonstrate that Token Filtering consistently outperforms prior structured pruning methods, preserving accuracy on commonsense reasoning benchmarks and maintaining strong performance on challenging tasks such as MMLU, even with 50% pruning.
• 摘要：剪枝已成为加速大语言模型（LLM）推理的一个有前途的方向，但现有方法常常不稳定，因为它们依赖于离线校准数据，而这些数据可能无法跨输入泛化。在这项工作中，我们引入了令牌过滤，这是一种轻量级的在线结构化剪枝技术，可以在推理过程中直接做出剪枝决策，无需任何校准数据。关键思想是通过联合键值相似度来测量令牌冗余并跳过冗余注意计算，从而在保留关键信息的同时降低推理成本。为了进一步增强稳定性，我们设计了一种方差感知融合策略，该策略自适应地对头之间的键和值相似性进行加权，确保即使在高剪枝率下也能保留信息标记。这种设计没有引入额外的内存开销，并为令牌重要性提供了更可靠的标准。 LLaMA-2 (7B/13B)、LLaMA-3 (8B) 和 Mistral (7B) 上的大量实验表明，令牌过滤始终优于先前的结构化剪枝方法，即使在 50% 剪枝的情况下，也能保持常识推理基准的准确性，并在 MMLU 等具有挑战性的任务上保持强大的性能。

Title: DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoning

• Authors: Nithin Sivakumaran, Justin Chih-Yao Chen, David Wan, Yue Zhang, Jaehong Yoon, Elias Stengel-Eskin, Mohit Bansal
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2512.07132
• Pdf URL: https://arxiv.org/pdf/2512.07132
• Copy Paste: [[2512.07132]] DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoning(https://arxiv.org/abs/2512.07132)
• Keywords: language model, agent
• Abstract: Specialized visual tools can augment large language models or vision language models with expert knowledge (e.g., grounding, spatial reasoning, medical knowledge, etc.), but knowing which tools to call (and when to call them) can be challenging. We introduce DART, a multi-agent framework that uses disagreements between multiple debating visual agents to identify useful visual tools (e.g., object detection, OCR, spatial reasoning, etc.) that can resolve inter-agent disagreement. These tools allow for fruitful multi-agent discussion by introducing new information, and by providing tool-aligned agreement scores that highlight agents in agreement with expert tools, thereby facilitating discussion. We utilize an aggregator agent to select the best answer by providing the agent outputs and tool information. We test DART on four diverse benchmarks and show that our approach improves over multi-agent debate as well as over single agent tool-calling frameworks, beating the next-strongest baseline (multi-agent debate with a judge model) by 3.4% and 2.4% on A-OKVQA and MMMU respectively. We also find that DART adapts well to new tools in applied domains, with a 1.3% improvement on the M3D medical dataset over other strong tool-calling, single agent, and multi-agent baselines. Additionally, we measure text overlap across rounds to highlight the rich discussion in DART compared to existing multi-agent methods. Finally, we study the tool call distribution, finding that diverse tools are reliably used to help resolve disagreement.
• 摘要：专业的视觉工具可以用专业知识（例如，基础、空间推理、医学知识等）来增强大型语言模型或视觉语言模型，但知道要调用哪些工具（以及何时调用它们）可能具有挑战性。我们引入了 DART，一个多智能体框架，它利用多个辩论视觉智能体之间的分歧来识别可以解决智能体间分歧的有用视觉工具（例如，对象检测、OCR、空间推理等）。这些工具通过引入新信息并提供与工具一致的协议分数来突出显示与专家工具一致的代理，从而促进讨论，​​从而实现富有成效的多代理讨论。我们利用聚合代理通过提供代理输出和工具信息来选择最佳答案。我们在四个不同的基准上测试了 DART，结果表明我们的方法比多智能体辩论以及单智能体工具调用框架都有所改进，在 A-OKVQA 和 MMMU 上分别比第二强的基线（带有法官模型的多智能体辩论）高出 3.4% 和 2.4%。我们还发现，DART 很好地适应了应用领域的新工具，与其他强大的工具调用、单代理和多代理基线相比，M3D 医疗数据集提高了 1.3%。此外，我们还测量各轮之间的文本重叠，以突出 DART 中与现有多代理方法相比的丰富讨论。最后，我们研究了工具调用分布，发现可以可靠地使用不同的工具来帮助解决分歧。

Title: GUMBridge: a Corpus for Varieties of Bridging Anaphora

• Authors: Lauren Levine, Amir Zeldes
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07134
• Pdf URL: https://arxiv.org/pdf/2512.07134
• Copy Paste: [[2512.07134]] GUMBridge: a Corpus for Varieties of Bridging Anaphora(https://arxiv.org/abs/2512.07134)
• Keywords: llm
• Abstract: Bridging is an anaphoric phenomenon where the referent of an entity in a discourse is dependent on a previous, non-identical entity for interpretation, such as in "There is 'a house'. 'The door' is red," where the door is specifically understood to be the door of the aforementioned house. While there are several existing resources in English for bridging anaphora, most are small, provide limited coverage of the phenomenon, and/or provide limited genre coverage. In this paper, we introduce GUMBridge, a new resource for bridging, which includes 16 diverse genres of English, providing both broad coverage for the phenomenon and granular annotations for the subtype categorization of bridging varieties. We also present an evaluation of annotation quality and report on baseline performance using open and closed source contemporary LLMs on three tasks underlying our data, showing that bridging resolution and subtype classification remain difficult NLP tasks in the age of LLMs.
• 摘要：桥接是一种照应现象，其中话语中实体的所指对象依赖于先前的、不相同的实体来进行解释，例如在“有‘一栋房子’。‘门’是红色的”中，其中门被具体理解为前述房子的门。虽然英语中存在一些用于桥接照应的现有资源，但大多数资源都很小，对现象的覆盖范围有限，和/或对体裁的覆盖范围有限。在本文中，我们介绍了 GUMBridge，一种新的桥接资源，其中包括 16 种不同的英语流派，既提供了对桥接现象的广泛覆盖，又为桥接品种的亚类型分类提供了细粒度注释。我们还使用开源和闭源当代法学硕士对数据基础的三个任务进行注释质量评估和基线性能报告，表明在法学硕士时代，桥接分辨率和子类型分类仍然是困难的 NLP 任务。

Title: MASim: Multilingual Agent-Based Simulation for Social Science

• Authors: Xuan Zhang, Wenxuan Zhang, Anxu Wang, See-Kiong Ng, Yang Deng
• Subjects: cs.CL, cs.AI, cs.CY, cs.MA, cs.SI
• Abstract URL: https://arxiv.org/abs/2512.07195
• Pdf URL: https://arxiv.org/pdf/2512.07195
• Copy Paste: [[2512.07195]] MASim: Multilingual Agent-Based Simulation for Social Science(https://arxiv.org/abs/2512.07195)
• Keywords: agent
• Abstract: Multi-agent role-playing has recently shown promise for studying social behavior with language agents, but existing simulations are mostly monolingual and fail to model cross-lingual interaction, an essential property of real societies. We introduce MASim, the first multilingual agent-based simulation framework that supports multi-turn interaction among generative agents with diverse sociolinguistic profiles. MASim offers two key analyses: (i) global public opinion modeling, by simulating how attitudes toward open-domain hypotheses evolve across languages and cultures, and (ii) media influence and information diffusion, via autonomous news agents that dynamically generate content and shape user behavior. To instantiate simulations, we construct the MAPS benchmark, which combines survey questions and demographic personas drawn from global population distributions. Experiments on calibration, sensitivity, consistency, and cultural case studies show that MASim reproduces sociocultural phenomena and highlights the importance of multilingual simulation for scalable, controlled computational social science.
• 摘要：多智能体角色扮演最近显示出用语言智能体研究社会行为的希望，但现有的模拟大多是单语言的，无法模拟跨语言交互，而跨语言交互是现实社会的基本属性。我们介绍 MASim，这是第一个基于多语言代理的模拟框架，支持具有不同社会语言特征的生成代理之间的多轮交互。 MASim 提供了两项关键分析：(i) 全球舆论建模，通过模拟对开放领域假设的态度如何在不同语言和文化中演变；(ii) 媒体影响和信息传播，通过动态生成内容和塑造用户行为的自主新闻代理。为了实例化模拟，我们构建了 MAPS 基准，该基准结合了调查问题和从全球人口分布中提取的人口特征。校准、敏感性、一致性和文化案例研究的实验表明，MASim 再现了社会文化现象，并强调了多语言模拟对于可扩展、受控计算社会科学的重要性。

Title: NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Models

• Authors: Feng Liang, Weixin Zeng, Runhao Zhao, Xiang Zhao
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.07218
• Pdf URL: https://arxiv.org/pdf/2512.07218
• Copy Paste: [[2512.07218]] NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Models(https://arxiv.org/abs/2512.07218)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, temporal reasoning, particularly under complex temporal constraints, remains a major challenge. To this end, existing approaches have explored symbolic methods, which encode temporal structure explicitly, and reflective mechanisms, which revise reasoning errors through multi-step inference. Nonetheless, symbolic approaches often underutilize the reasoning capabilities of LLMs, while reflective methods typically lack structured temporal representations, which can result in inconsistent or hallucinated reasoning. As a result, even when the correct temporal context is available, LLMs may still misinterpret or misapply time-related information, leading to incomplete or inaccurate answers. To address these limitations, in this work, we propose Neuro-Symbolic Temporal Reasoning (NeSTR), a novel framework that integrates structured symbolic representations with hybrid reflective reasoning to enhance the temporal sensitivity of LLM inference. NeSTR preserves explicit temporal relations through symbolic encoding, enforces logical consistency via verification, and corrects flawed inferences using abductive reflection. Extensive experiments on diverse temporal question answering benchmarks demonstrate that NeSTR achieves superior zero-shot performance and consistently improves temporal reasoning without any fine-tuning, showcasing the advantage of neuro-symbolic integration in enhancing temporal understanding in large language models.
• 摘要：大型语言模型 (LLM) 在广泛的自然语言处理任务中表现出了卓越的性能。然而，时间推理，特别是在复杂的时间约束下，仍然是一个重大挑战。为此，现有方法探索了显式编码时间结构的符号方法和通过多步骤推理修正推理错误的反射机制。尽管如此，符号方法往往没有充分利用法学硕士的推理能力，而反思方法通常缺乏结构化的时间表示，这可能导致不一致或幻觉的推理。因此，即使可以获得正确的时间上下文，法学硕士仍然可能会误解或误用与时间相关的信息，从而导致答案不完整或不准确。为了解决这些局限性，在这项工作中，我们提出了神经符号时间推理（NeSTR），这是一种新颖的框架，它将结构化符号表示与混合反射推理相结合，以增强法学硕士推理的时间敏感性。 NeSTR 通过符号编码保留显式的时间关系，通过验证强制逻辑一致性，并使用溯因反射纠正有缺陷的推论。对不同时间问答基准的大量实验表明，NeSTR 实现了卓越的零样本性能，并且在无需任何微调的情况下持续改进时间推理，展示了神经符号集成在增强大型语言模型的时间理解方面的优势。

Title: Ensembling LLM-Induced Decision Trees for Explainable and Robust Error Detection

• Authors: Mengqi Wang (1), Jianwei Wang (1), Qing Liu (2), Xiwei Xu (2), Zhenchang Xing (2), Liming Zhu (2), Wenjie Zhang (1) ((1) UNSW Sydney, (2) Data61, CSIRO)
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07246
• Pdf URL: https://arxiv.org/pdf/2512.07246
• Copy Paste: [[2512.07246]] Ensembling LLM-Induced Decision Trees for Explainable and Robust Error Detection(https://arxiv.org/abs/2512.07246)
• Keywords: language model, llm, prompt
• Abstract: Error detection (ED), which aims to identify incorrect or inconsistent cell values in tabular data, is important for ensuring data quality. Recent state-of-the-art ED methods leverage the pre-trained knowledge and semantic capability embedded in large language models (LLMs) to directly label whether a cell is erroneous. However, this LLM-as-a-labeler pipeline (1) relies on the black box, implicit decision process, thus failing to provide explainability for the detection results, and (2) is highly sensitive to prompts, yielding inconsistent outputs due to inherent model stochasticity, therefore lacking robustness. To address these limitations, we propose an LLM-as-an-inducer framework that adopts LLM to induce the decision tree for ED (termed TreeED) and further ensembles multiple such trees for consensus detection (termed ForestED), thereby improving explainability and robustness. Specifically, based on prompts derived from data context, decision tree specifications and output requirements, TreeED queries the LLM to induce the decision tree skeleton, whose root-to-leaf decision paths specify the stepwise procedure for evaluating a given sample. Each tree contains three types of nodes: (1) rule nodes that perform simple validation checks (e.g., format or range), (2) Graph Neural Network (GNN) nodes that capture complex patterns (e.g., functional dependencies), and (3) leaf nodes that output the final decision types (error or clean). Furthermore, ForestED employs uncertainty-based sampling to obtain multiple row subsets, constructing a decision tree for each subset using TreeED. It then leverages an Expectation-Maximization-based algorithm that jointly estimates tree reliability and optimizes the consensus ED prediction. Extensive xperiments demonstrate that our methods are accurate, explainable and robust, achieving an average F1-score improvement of 16.1% over the best baseline.
• 摘要：错误检测 (ED) 旨在识别表格数据中不正确或不一致的单元格值，对于确保数据质量非常重要。最近最先进的 ED 方法利用大型语言模型 (LLM) 中嵌入的预先训练的知识和语义功能来直接标记单元格是否错误。然而，这种LLM作为标记器的流程（1）依赖于黑盒、隐式决策过程，因此无法为检测结果提供可解释性；（2）对提示高度敏感，由于模型固有的随机性而产生不一致的输出，因此缺乏鲁棒性。为了解决这些限制，我们提出了一个LLM作为诱导器框架，该框架采用LLM来诱导ED决策树（称为TreeED），并进一步集成多个此类树以进行共识检测（称为ForestED），从而提高可解释性和鲁棒性。具体来说，根据来自数据上下文、决策树规范和输出要求的提示，TreeED 查询 LLM 来归纳决策树骨架，其根到叶决策路径指定评估给定样本的逐步过程。每棵树包含三种类型的节点：（1）执行简单验证检查（例如格式或范围）的规则节点，（2）捕获复杂模式（例如函数依赖性）的图神经网络（GNN）节点，以及（3）输出最终决策类型（错误或干净）的叶节点。此外，ForestED 采用基于不确定性的采样来获取多个行子集，并使用 TreeED 为每个子集构建决策树。然后，它利用基于期望最大化的算法来联合估计树的可靠性并优化一致的 ED 预测。广泛的实验表明，我们的方法准确、可解释且稳健，与最佳基线相比，平均 F1 分数提高了 16.1%。

Title: Investigating Training and Generalization in Faithful Self-Explanations of Large Language Models

• Authors: Tomoki Doi, Masaru Isonuma, Hitomi Yanaka
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07288
• Pdf URL: https://arxiv.org/pdf/2512.07288
• Copy Paste: [[2512.07288]] Investigating Training and Generalization in Faithful Self-Explanations of Large Language Models(https://arxiv.org/abs/2512.07288)
• Keywords: language model
• Abstract: Large language models have the potential to generate explanations for their own predictions in a variety of styles based on user instructions. Recent research has examined whether these self-explanations faithfully reflect the models' actual behavior and has found that they often lack faithfulness. However, the question of how to improve faithfulness remains underexplored. Moreover, because different explanation styles have superficially distinct characteristics, it is unclear whether improvements observed in one style also arise when using other styles. This study analyzes the effects of training for faithful self-explanations and the extent to which these effects generalize, using three classification tasks and three explanation styles. We construct one-word constrained explanations that are likely to be faithful using a feature attribution method, and use these pseudo-faithful self-explanations for continual learning on instruction-tuned models. Our experiments demonstrate that training can improve self-explanation faithfulness across all classification tasks and explanation styles, and that these improvements also show signs of generalization to the multi-word settings and to unseen tasks. Furthermore, we find consistent cross-style generalization among three styles, suggesting that training may contribute to a broader improvement in faithful self-explanation ability.
• 摘要：大型语言模型有可能根据用户指令以各种风格为自己的预测生成解释。最近的研究检验了这些自我解释是否忠实地反映了模型的实际行为，并发现它们往往缺乏忠实度。然而，如何提高忠诚度的问题仍未得到充分探讨。此外，由于不同的解释风格具有表面上不同的特征，因此尚不清楚在使用其他风格时是否也会出现在一种风格中观察到的改进。本研究使用三种分类任务和三种解释风格，分析了忠实自我解释训练的效果以及这些效果的泛化程度。我们使用特征归因方法构建可能忠实的单字约束解释，并使用这些伪忠实的自我解释来持续学习指令调整模型。我们的实验表明，训练可以提高所有分类任务和解释风格的自我解释忠实度，并且这些改进还显示出对多词设置和看不见的任务的泛化迹象。此外，我们发现三种风格之间存在一致的跨风格概括，这表明培训可能有助于更广泛地提高忠实的自我解释能力。

Title: Training Language Models to Use Prolog as a Tool

• Authors: Niklas Mellgren, Peter Schneider-Kamp, Lukas Galke Poech
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07407
• Pdf URL: https://arxiv.org/pdf/2512.07407
• Copy Paste: [[2512.07407]] Training Language Models to Use Prolog as a Tool(https://arxiv.org/abs/2512.07407)
• Keywords: language model, prompt, agent
• Abstract: Ensuring reliable tool use is critical for safe agentic AI systems. Language models frequently produce unreliable reasoning with plausible but incorrect solutions that are difficult to verify. To address this, we investigate fine-tuning models to use Prolog as an external tool for verifiable computation. Using Group Relative Policy Optimization (GRPO), we fine-tune Qwen2.5-3B-Instruct on a cleaned GSM8K-Prolog-Prover dataset while varying (i) prompt structure, (ii) reward composition (execution, syntax, semantics, structure), and (iii) inference protocol: single-shot, best-of-N, and two agentic modes where Prolog is invoked internally or independently. Our reinforcement learning approach outperforms supervised fine-tuning, with our 3B model achieving zero-shot MMLU performance comparable to 7B few-shot results. Our findings reveal that: 1) joint tuning of prompt, reward, and inference shapes program syntax and logic; 2) best-of-N with external Prolog verification maximizes accuracy on GSM8K; 3) agentic inference with internal repair yields superior zero-shot generalization on MMLU-Stem and MMLU-Pro. These results demonstrate that grounding model reasoning in formal verification systems substantially improves reliability and auditability for safety-critical applications. The source code for reproducing our experiments is available under this https URL
• 摘要：确保可靠的工具使用对于安全的代理人工智能系统至关重要。语言模型经常产生不可靠的推理，以及难以验证的看似合理但不正确的解决方案。为了解决这个问题，我们研究了微调模型，以使用 Prolog 作为可验证计算的外部工具。使用组相对策略优化 (GRPO)，我们在清理的 GSM8K-Prolog-Prover 数据集上微调 Qwen2.5-3B-Instruct，同时改变 (i) 提示结构、(ii) 奖励组成（执行、语法、语义、结构）和 (iii) 推理协议：单次、N 中最佳和内部或独立调用 Prolog 的两种代理模式。我们的强化学习方法优于监督微调，我们的 3B 模型实现了与 7B 少样本结果相当的零样本 MMLU 性能。我们的研究结果表明：1）提示、奖励和推理的联合调整塑造了程序语法和逻辑； 2) 采用外部 Prolog 验证的 best-of-N 最大限度地提高 GSM8K 上的准确性； 3) 具有内部修复的主体推理在 MMLU-Stem 和 MMLU-Pro 上产生了卓越的零样本泛化。这些结果表明，形式验证系统中的基础模型推理大大提高了安全关键型应用的可靠性和可审计性。用于重现我们实验的源代码可在此 https URL 下找到

Title: Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning

• Authors: Amir Mohammad Akhlaghi, Amirhossein Shabani, Mostafa Abdolmaleki, Saeed Reza Kheradpisheh
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.07454
• Pdf URL: https://arxiv.org/pdf/2512.07454
• Copy Paste: [[2512.07454]] Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning(https://arxiv.org/abs/2512.07454)
• Keywords: language model, llm
• Abstract: The democratization of AI is currently hindered by the immense computational costs required to train Large Language Models (LLMs) for low-resource languages. This paper presents Persian-Phi, a 3.8B parameter model that challenges the assumption that robust multilingual capabilities require massive model sizes or multilingual baselines. We demonstrate how Microsoft Phi-3 Mini -- originally a monolingual English model -- can be effectively adapted to Persian through a novel, resource-efficient curriculum learning pipeline. Our approach employs a unique "warm-up" stage using bilingual narratives (Tiny Stories) to align embeddings prior to heavy training, followed by continual pretraining and instruction tuning via Parameter-Efficient Fine-Tuning (PEFT). Despite its compact size, Persian-Phi achieves competitive results on Open Persian LLM Leaderboard in HuggingFace. Our findings provide a validated, scalable framework for extending the reach of state-of-the-art LLMs to underrepresented languages with minimal hardware resources. The Persian-Phi model is publicly available at this https URL.
• 摘要：目前，为低资源语言训练大型语言模型（LLM）所需的巨大计算成本阻碍了人工智能的民主化。本文提出了 Persian-Phi，一个 3.8B 参数模型，它挑战了强大的多语言功能需要大量模型大小或多语言基线的假设。我们演示了 Microsoft Phi-3 Mini（最初是单语英语模型）如何通过新颖、资源高效的课程学习管道有效地适应波斯语。我们的方法采用独特的“热身”阶段，使用双语叙述（Tiny Stories）在大量训练之前对齐嵌入，然后通过参数高效微调（PEFT）进行持续的预训练和指令调整。尽管尺寸紧凑，Persian-Phi 在 HuggingFace 的 Open Persian LLM 排行榜上取得了有竞争力的成绩。我们的研究结果提供了一个经过验证的、可扩展的框架，可以用最少的硬件资源将最先进的法学硕士的范围扩展到代表性不足的语言。 Persian-Phi 模型可通过此 https URL 公开获取。

Title: Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning

• Authors: Tong Wu, Yang Liu, Jun Bai, Zixia Jia, Shuyi Zhang, Ziyong Lin, Yanting Wang, Song-Chun Zhu, Zilong Zheng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07461
• Pdf URL: https://arxiv.org/pdf/2512.07461
• Copy Paste: [[2512.07461]] Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning(https://arxiv.org/abs/2512.07461)
• Keywords: language model, llm, agent
• Abstract: We introduce Native Parallel Reasoner (NPR), a teacher-free framework that enables Large Language Models (LLMs) to self-evolve genuine parallel reasoning capabilities. NPR transforms the model from sequential emulation to native parallel cognition through three key innovations: 1) a self-distilled progressive training paradigm that transitions from ``cold-start'' format discovery to strict topological constraints without external supervision; 2) a novel Parallel-Aware Policy Optimization (PAPO) algorithm that optimizes branching policies directly within the execution graph, allowing the model to learn adaptive decomposition via trial and error; and 3) a robust NPR Engine that refactors memory management and flow control of SGLang to enable stable, large-scale parallel RL training. Across eight reasoning benchmarks, NPR trained on Qwen3-4B achieves performance gains of up to 24.5% and inference speedups up to 4.6x. Unlike prior baselines that often fall back to autoregressive decoding, NPR demonstrates 100% genuine parallel execution, establishing a new standard for self-evolving, efficient, and scalable agentic reasoning.
• 摘要：我们引入了 Native Parallel Reasoner (NPR)，这是一个无需教师的框架，使大型语言模型 (LLM) 能够自我进化真正的并行推理能力。 NPR 通过三个关键创新将模型从顺序仿真转变为原生并行认知：1）自提渐进式训练范式，从“冷启动”格式发现过渡到无需外部监督的严格拓扑约束； 2）一种新颖的并行感知策略优化（PAPO）算法，可直接在执行图中优化分支策略，使模型能够通过试错来学习自适应分解； 3) 强大的 NPR 引擎，可重构 SGLang 的内存管理和流程控制，以实现稳定、大规模的并行 RL 训练。在八个推理基准测试中，在 Qwen3-4B 上训练的 NPR 实现了高达 24.5% 的性能提升和高达 4.6 倍的推理速度提升。与通常回退到自回归解码的先前基线不同，NPR 展示了 100% 真正的并行执行，为自我进化、高效和可扩展的代理推理建立了新标准。

Title: Enhancing Agentic RL with Progressive Reward Shaping and Value-based Sampling Policy Optimization

• Authors: Zhuoran Zhuang, Ye Chen, Jianghao Su, Chao Luo, Luhui Liu, Xia Zeng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07478
• Pdf URL: https://arxiv.org/pdf/2512.07478
• Copy Paste: [[2512.07478]] Enhancing Agentic RL with Progressive Reward Shaping and Value-based Sampling Policy Optimization(https://arxiv.org/abs/2512.07478)
• Keywords: language model, llm, prompt, agent
• Abstract: Large Language Models (LLMs) empowered with Tool-Integrated Reasoning (TIR) can iteratively plan, call external tools, and integrate returned information to solve complex, long-horizon reasoning tasks. Agentic Reinforcement Learning (Agentic RL) optimizes such models over full tool-interaction trajectories, but two key challenges hinder effectiveness: (1) Sparse, non-instructive rewards, such as binary 0-1 verifiable signals, provide limited guidance for intermediate steps and slow convergence; (2) Gradient degradation in Group Relative Policy Optimization (GRPO), where identical rewards within a rollout group yield zero advantage, reducing sample efficiency and destabilizing training. To address these challenges, we propose two complementary techniques: Progressive Reward Shaping (PRS) and Value-based Sampling Policy Optimization (VSPO). PRS is a curriculum-inspired reward design that introduces dense, stage-wise feedback - encouraging models to first master parseable and properly formatted tool calls, then optimize for factual correctness and answer quality. We instantiate PRS for short-form QA (with a length-aware BLEU to fairly score concise answers) and long-form QA (with LLM-as-a-Judge scoring to prevent reward hacking). VSPO is an enhanced GRPO variant that replaces low-value samples with prompts selected by a task-value metric balancing difficulty and uncertainty, and applies value-smoothing clipping to stabilize gradient updates. Experiments on multiple short-form and long-form QA benchmarks show that PRS consistently outperforms traditional binary rewards, and VSPO achieves superior stability, faster convergence, and higher final performance compared to PPO, GRPO, CISPO, and SFT-only baselines. Together, PRS and VSPO yield LLM-based TIR agents that generalize better across domains.
• 摘要：具有工具集成推理 (TIR) 功能的大型语言模型 (LLM) 可以迭代规划、调用外部工具并集成返回的信息，以解决复杂的长期推理任务。代理强化学习 (Agentic RL) 在完整的工具交互轨迹上优化此类模型，但有两个关键挑战阻碍了有效性：(1) 稀疏的、非指导性的奖励，例如二进制 0-1 可验证信号，为中间步骤提供有限的指导，并且收敛缓慢； (2) 组相对策略优化 (GRPO) 中的梯度退化，其中推出组内的相同奖励产生零优势，从而降低了样本效率并破坏了训练的稳定性。为了应对这些挑战，我们提出了两种互补技术：渐进奖励塑造（PRS）和基于价值的抽样策略优化（VSPO）。 PRS 是一种受课程启发的奖励设计，引入了密集的、阶段性的反馈——鼓励模型首先掌握可解析且格式正确的工具调用，然后优化事实正确性和答案质量。我们实例化 PRS 进行简短形式的 QA（使用长度感知的 BLEU 来公平地对简洁的答案进行评分）和长形式的 QA（使用 LLM-as-a-Judge 评分以防止奖励黑客行为）。 VSPO 是增强的 GRPO 变体，它用平衡难度和不确定性的任务价值指标选择的提示替换低价值样本，并应用价值平滑裁剪来稳定梯度更新。对多个短格式和长格式 QA 基准的实验表明，PRS 始终优于传统的二元奖励，并且与 PPO、GRPO、CISPO 和仅 SFT 基线相比，VSPO 实现了卓越的稳定性、更快的收敛和更高的最终性能。 PRS 和 VSPO 共同产生基于 LLM 的 TIR 代理，可以更好地跨领域泛化。

Title: SPAD: Seven-Source Token Probability Attribution with Syntactic Aggregation for Detecting Hallucinations in RAG

• Authors: Pengqian Lu, Jie Lu, Anjin Liu, Guangquan Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.07515
• Pdf URL: https://arxiv.org/pdf/2512.07515
• Copy Paste: [[2512.07515]] SPAD: Seven-Source Token Probability Attribution with Syntactic Aggregation for Detecting Hallucinations in RAG(https://arxiv.org/abs/2512.07515)
• Keywords: hallucination, retrieval-augmented generation
• Abstract: Detecting hallucinations in Retrieval-Augmented Generation (RAG) remains a challenge. Prior approaches attribute hallucinations to a binary conflict between internal knowledge (stored in FFNs) and retrieved context. However, this perspective is incomplete, failing to account for the impact of other components in the generative process, such as the user query, previously generated tokens, the current token itself, and the final LayerNorm adjustment. To address this, we introduce SPAD. First, we mathematically attribute each token's probability into seven distinct sources: Query, RAG, Past, Current Token, FFN, Final LayerNorm, and Initial Embedding. This attribution quantifies how each source contributes to the generation of the current token. Then, we aggregate these scores by POS tags to quantify how different components drive specific linguistic categories. By identifying anomalies, such as Nouns relying on Final LayerNorm, SPAD effectively detects hallucinations. Extensive experiments demonstrate that SPAD achieves state-of-the-art performance
• 摘要：在检索增强一代（RAG）中检测幻觉仍然是一个挑战。先前的方法将幻觉归因于内部知识（存储在 FFN 中）和检索到的上下文之间的二元冲突。然而，这个视角是不完整的，没有考虑到生成过程中其他组件的影响，例如用户查询、先前生成的令牌、当前令牌本身以及最终的 LayerNorm 调整。为了解决这个问题，我们引入了 SPAD。首先，我们在数学上将每个标记的概率归因于七个不同的源：查询、RAG、过去、当前标记、FFN、最终层范数和初始嵌入。此归因量化了每个来源如何对当前令牌的生成做出贡献。然后，我们通过词性标签聚合这些分数，以量化不同组件如何驱动特定的语言类别。通过识别异常情况，例如依赖于 Final LayerNorm 的名词，SPAD 可以有效地检测幻觉。大量实验证明 SPAD 实现了最先进的性能

Title: LIME: Making LLM Data More Efficient with Linguistic Metadata Embeddings

• Authors: Sebastian Sztwiertnia, Felix Friedrich, Kristian Kersting, Patrick Schramowski, Björn Deiseroth
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.07522
• Pdf URL: https://arxiv.org/pdf/2512.07522
• Copy Paste: [[2512.07522]] LIME: Making LLM Data More Efficient with Linguistic Metadata Embeddings(https://arxiv.org/abs/2512.07522)
• Keywords: language model, llm
• Abstract: Pre-training decoder-only language models relies on vast amounts of high-quality data, yet the availability of such data is increasingly reaching its limits. While metadata is commonly used to create and curate these datasets, its potential as a direct training signal remains under-explored. We challenge this status quo and propose LIME (Linguistic Metadata Embeddings), a method that enriches token embeddings with metadata capturing syntax, semantics, and contextual properties. LIME substantially improves pre-training efficiency. Specifically, it adapts up to 56% faster to the training data distribution, while introducing only 0.01% additional parameters at negligible compute overhead. Beyond efficiency, LIME improves tokenization, leading to remarkably stronger language modeling capabilities and generative task performance. These benefits persist across model scales (500M to 2B). In addition, we develop a variant with shifted metadata, LIME+1, that can guide token generation. Given prior metadata for the next token, LIME+1 improves reasoning performance by up to 38% and arithmetic accuracy by up to 35%.
• 摘要：仅预训练解码器语言模型依赖于大量高质量数据，但此类数据的可用性正日益达到极限。虽然元数据通常用于创建和管理这些数据集，但其作为直接训练信号的潜力仍未得到充分探索。我们挑战这一现状，并提出 LIME（语言元数据嵌入），这是一种通过捕获语法、语义和上下文属性的元数据来丰富令牌嵌入的方法。 LIME 大幅提高了预训练效率。具体来说，它对训练数据分布的适应速度提高了 56%，同时仅引入 0.01% 的额外参数，而计算开销可以忽略不计。除了效率之外，LIME 还改进了标记化，从而显着增强了语言建模功能和生成任务性能。这些优势在不同模型规模（500M 到 2B）中都持续存在。此外，我们还开发了一种具有转移元数据的变体 LIME+1，可以指导代币生成。考虑到下一个 token 的先前元数据，LIME+1 将推理性能提高了 38%，算术准确性提高了 35%。

Title: Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs

• Authors: Xiaoran Liu, Yuerong Song, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Zhaoxiang Liu, Shiguo Lian, Ziwei He, Xipeng Qiu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07525
• Pdf URL: https://arxiv.org/pdf/2512.07525
• Copy Paste: [[2512.07525]] Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMs(https://arxiv.org/abs/2512.07525)
• Keywords: language model, llm
• Abstract: Rotary Position Embeddings (RoPE) have become a standard for encoding sequence order in Large Language Models (LLMs) by applying rotations to query and key vectors in the complex plane. Standard implementations, however, utilize only the real component of the complex-valued dot product for attention score calculation. This simplification discards the imaginary component, which contains valuable phase information, leading to a potential loss of relational details crucial for modeling long-context dependencies. In this paper, we propose an extension that re-incorporates this discarded imaginary component. Our method leverages the full complex-valued representation to create a dual-component attention score. We theoretically and empirically demonstrate that this approach enhances the modeling of long-context dependencies by preserving more positional information. Furthermore, evaluations on a suite of long-context language modeling benchmarks show that our method consistently improves performance over the standard RoPE, with the benefits becoming more significant as context length increases. The code is available at this https URL.
• 摘要：旋转位置嵌入 (RoPE) 通过将旋转应用于复平面中的查询和关键向量，已成为大型语言模型 (LLM) 中编码序列顺序的标准。然而，标准实现仅利用复值点积的实数部分来计算注意力分数。这种简化丢弃了虚部，其中包含有价值的阶段信息，导致可能丢失对于建模长上下文依赖关系至关重要的关系细节。在本文中，我们提出了一种扩展，重新合并了这个被丢弃的虚数部分。我们的方法利用完整的复值表示来创建双分量注意力分数。我们从理论上和经验上证明，这种方法通过保留更多的位置信息来增强长上下文依赖的建模。此外，对一套长上下文语言建模基准的评估表明，我们的方法相对于标准 RoPE 持续提高了性能，并且随着上下文长度的增加，优势变得更加显着。该代码可从此 https URL 获取。

Title: SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents

• Authors: Michelle Wastl, Jannis Vamvas, Rico Sennrich
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07538
• Pdf URL: https://arxiv.org/pdf/2512.07538
• Copy Paste: [[2512.07538]] SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents(https://arxiv.org/abs/2512.07538)
• Keywords: language model, llm
• Abstract: Recognizing semantic differences across documents, especially in different languages, is crucial for text generation evaluation and multilingual content alignment. However, as a standalone task it has received little attention. We address this by introducing SwissGov-RSD, the first naturalistic, document-level, cross-lingual dataset for semantic difference recognition. It encompasses a total of 224 multi-parallel documents in English-German, English-French, and English-Italian with token-level difference annotations by human annotators. We evaluate a variety of open-source and closed source large language models as well as encoder models across different fine-tuning settings on this new benchmark. Our results show that current automatic approaches perform poorly compared to their performance on monolingual, sentence-level, and synthetic benchmarks, revealing a considerable gap for both LLMs and encoder models. We make our code and datasets publicly available.
• 摘要：识别文档之间的语义差异，尤其是不同语言的语义差异，对于文本生成评估和多语言内容对齐至关重要。然而，作为一项独立的任务，它很少受到关注。我们通过引入 SwissGov-RSD 来解决这个问题，这是第一个用于语义差异识别的自然主义、文档级、跨语言数据集。它总共包含 224 个英语-德语、英语-法语和英语-意大利语的多并行文档，并由人工注释者进行标记级差异注释。我们在此新基准上评估了各种开源和闭源大型语言模型以及跨不同微调设置的编码器模型。我们的结果表明，当前的自动方法与单语言、句子级和综合基准上的性能相比表现较差，揭示了法学硕士和编码器模型之间存在相当大的差距。我们公开我们的代码和数据集。

Title: MoCoRP: Modeling Consistent Relations between Persona and Response for Persona-based Dialogue

• Authors: Kyungro Lee, Dongha Choi, Hyunju Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.07544
• Pdf URL: https://arxiv.org/pdf/2512.07544
• Copy Paste: [[2512.07544]] MoCoRP: Modeling Consistent Relations between Persona and Response for Persona-based Dialogue(https://arxiv.org/abs/2512.07544)
• Keywords: language model, llm, chat
• Abstract: As dialogue systems become increasingly important across various domains, a key challenge in persona-based dialogue is generating engaging and context-specific interactions while ensuring the model acts with a coherent personality. However, existing persona-based dialogue datasets lack explicit relations between persona sentences and responses, which makes it difficult for models to effectively capture persona information. To address these issues, we propose MoCoRP (Modeling Consistent Relations between Persona and Response), a framework that incorporates explicit relations into language models. MoCoRP leverages an NLI expert to explicitly extract the NLI relations between persona sentences and responses, enabling the model to effectively incorporate appropriate persona information from the context into its responses. We applied this framework to pre-trained models like BART and further extended it to modern large language models (LLMs) through alignment tuning. Experimental results on the public datasets ConvAI2 and MPChat demonstrate that MoCoRP outperforms existing baselines, achieving superior persona consistency and engaging, context-aware dialogue generation. Furthermore, our model not only excels in quantitative metrics but also shows significant improvements in qualitative aspects. These results highlight the effectiveness of explicitly modeling persona-response relations in persona-based dialogue. The source codes of MoCoRP are available at this https URL.
• 摘要：随着对话系统在各个领域变得越来越重要，基于角色的对话的一个关键挑战是产生引人入胜且针对特定上下文的交互，同时确保模型以连贯的个性行事。然而，现有的基于角色的对话数据集缺乏角色句子和响应之间的明确关系，这使得模型难以有效捕获角色信息。为了解决这些问题，我们提出了 MoCoRP（建模角色和响应之间的一致关系），这是一个将显式关系纳入语言模型的框架。 MoCoRP 利用 NLI 专家显式提取角色句子和响应之间的 NLI 关系，使模型能够有效地将上下文中的适当角色信息合并到其响应中。我们将该框架应用于 BART 等预训练模型，并通过对齐调整将其进一步扩展到现代大型语言模型 (LLM)。公共数据集 ConvAI2 和 MPChat 上的实验结果表明，MoCoRP 优于现有基线，实现了卓越的角色一致性和引人入胜的上下文感知对话生成。此外，我们的模型不仅在定量指标上表现出色，而且在定性方面也显示出显着的改进。这些结果强调了在基于角色的对话中显式建模角色响应关系的有效性。 MoCoRP 的源代码可从此 https URL 获取。

Title: A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification

• Authors: Nicolas Calbucura, Valentin Barriere
• Subjects: cs.CL, cs.MM
• Abstract URL: https://arxiv.org/abs/2512.07571
• Pdf URL: https://arxiv.org/pdf/2512.07571
• Copy Paste: [[2512.07571]] A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification(https://arxiv.org/abs/2512.07571)
• Keywords: language model
• Abstract: This paper presents a simple method that allows to easily enhance textual pre-trained large language models with speech information, when fine-tuned for a specific classification task. A classical issue with the fusion of many embeddings from audio with text is the large length of the audio sequence compared to the text one. Our method benefits from an existing speech tokenizer trained for Audio Speech Recognition that output long sequences of tokens from a large vocabulary, making it difficult to integrate it at low cost in a large language model. By applying a simple lasso-based feature selection on multimodal Bag-of-Words representation, we retain only the most important audio tokens for the task, and adapt the language model to them with a self-supervised language modeling objective, before fine-tuning it on the downstream task. We show this helps to improve the performances compared to an unimodal model, to a bigger SpeechLM or to integrating audio via a learned representation. We show the effectiveness of our method on two recent Argumentative Fallacy Detection and Classification tasks where the use of audio was believed counterproductive, reaching state-of-the-art results. We also provide an in-depth analysis of the method, showing that even a random audio token selection helps enhancing the unimodal model. Our code is available [online](this https URL).
• 摘要：本文提出了一种简单的方法，当针对特定分类任务进行微调时，可以使用语音信息轻松增强文本预训练的大型语言模型。音频与文本的许多嵌入融合的一个经典问题是，与文本序列相比，音频序列的长度较长。我们的方法受益于现有的针对音频语音识别进行训练的语音标记器，该标记器从大词汇量中输出长标记序列，因此很难以低成本将其集成到大型语言模型中。通过在多模态词袋表示上应用简单的基于套索的特征选择，我们仅保留任务中最重要的音频标记，并通过自监督语言建模目标使语言模型适应它们，然后在下游任务上对其进行微调。我们表明，与单峰模型、更大的 SpeechLM 或通过学习的表示集成音频相比，这有助于提高性能。我们展示了我们的方法在最近两项论证性谬误检测和分类任务中的有效性，其中使用音频被认为会适得其反，达到了最先进的结果。我们还对该方法进行了深入分析，表明即使是随机音频标记选择也有助于增强单峰模型。我们的代码可以[在线]（此 https URL）。

Title: Complementary Learning Approach for Text Classification using Large Language Models

• Authors: Navid Asgari, Benjamin M. Cole
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.07583
• Pdf URL: https://arxiv.org/pdf/2512.07583
• Copy Paste: [[2512.07583]] Complementary Learning Approach for Text Classification using Large Language Models(https://arxiv.org/abs/2512.07583)
• Keywords: language model, llm, prompt
• Abstract: In this study, we propose a structured methodology that utilizes large language models (LLMs) in a cost-efficient and parsimonious manner, integrating the strengths of scholars and machines while offsetting their respective weaknesses. Our methodology, facilitated through a chain of thought and few-shot learning prompting from computer science, extends best practices for co-author teams in qualitative research to human-machine teams in quantitative research. This allows humans to utilize abductive reasoning and natural language to interrogate not just what the machine has done but also what the human has done. Our method highlights how scholars can manage inherent weaknesses OF LLMs using careful, low-cost techniques. We demonstrate how to use the methodology to interrogate human-machine rating discrepancies for a sample of 1,934 press releases announcing pharmaceutical alliances (1990-2017).
• 摘要：在这项研究中，我们提出了一种结构化方法，以经济高效且简约的方式利用大型语言模型（LLM），整合学者和机器的优势，同时弥补各自的弱点。我们的方法论通过计算机科学的一系列思想和少量学习推动，将定性研究中合著者团队的最佳实践扩展到定量研究中的人机团队。这使得人类不仅可以利用溯因推理和自然语言来询问机器做了什么，还可以询问人类做了什么。我们的方法强调了学者如何使用谨慎、低成本的技术来管理法学硕士的固有弱点。我们演示了如何使用该方法来询问 1,934 份宣布制药联盟的新闻稿样本（1990-2017 年）的人机评级差异。

Title: Metric-Fair Prompting: Treating Similar Samples Similarly

• Authors: Jing Wang, Jie Shen, Xing Niu, Tong Zhang, Jeremy Weiss
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.07608
• Pdf URL: https://arxiv.org/pdf/2512.07608
• Copy Paste: [[2512.07608]] Metric-Fair Prompting: Treating Similar Samples Similarly(https://arxiv.org/abs/2512.07608)
• Keywords: language model, llm, prompt
• Abstract: We introduce \emph{Metric-Fair Prompting}, a fairness-aware prompting framework that guides large language models (LLMs) to make decisions under metric-fairness constraints. In the application of multiple-choice medical question answering, each {(question, option)} pair is treated as a binary instance with label $+1$ (correct) or $-1$ (incorrect). To promote {individual fairness}~--~treating similar instances similarly~--~we compute question similarity using NLP embeddings and solve items in \emph{joint pairs of similar questions} rather than in isolation. The prompt enforces a global decision protocol: extract decisive clinical features, map each \((\text{question}, \text{option})\) to a score $f(x)$ that acts as confidence, and impose a Lipschitz-style constraint so that similar inputs receive similar scores and, hence, consistent outputs. Evaluated on the {MedQA (US)} benchmark, Metric-Fair Prompting is shown to improve performance over standard single-item prompting, demonstrating that fairness-guided, confidence-oriented reasoning can enhance LLM accuracy on high-stakes clinical multiple-choice questions.
• 摘要：我们引入了 \emph{Metric-Fair Prompting}，一个公平感知的提示框架，指导大型语言模型（LLM）在度量公平约束下做出决策。在多项选择医学问答的应用中，每个{(问题，选项)}对被视为带有标签$+1$（正确）或$-1$（不正确）的二进制实例。为了促进{个人公平}~--~以类似的方式处理相似的实例~--~我们使用 NLP 嵌入计算问题相似性，并解决 \emph{相似问题的联合对}中的项目，而不是孤立地解决。该提示强制执行全局决策协议：提取决定性的临床特征，将每个 \((\text{question}, \text{option})\) 映射到充当置信度的分数 $f(x)$，并施加 Lipschitz 式约束，以便相似的输入获得相似的分数，从而获得一致的输出。根据 {MedQA (US)} 基准进行评估，Metric-Fair 提示比标准单项提示提高了性能，证明以公平为导向、以置信度为导向的推理可以提高 LLM 在高风险临床多项选择问题上的准确性。

Title: PCMind-2.1-Kaiyuan-2B Technical Report

• Authors: Kairong Luo, Zhenbo Sun, Xinyu Shi, Shengqi Chen, Bowen Yu, Yunyi Chen, Chenyi Dang, Hengtao Tao, Hui Wang, Fangming Liu, Kaifeng Lyu, Wenguang Chen
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.07612
• Pdf URL: https://arxiv.org/pdf/2512.07612
• Copy Paste: [[2512.07612]] PCMind-2.1-Kaiyuan-2B Technical Report(https://arxiv.org/abs/2512.07612)
• Keywords: language model, llm
• Abstract: The rapid advancement of Large Language Models (LLMs) has resulted in a significant knowledge gap between the open-source community and industry, primarily because the latter relies on closed-source, high-quality data and training recipes. To address this, we introduce PCMind-2.1-Kaiyuan-2B, a fully open-source 2-billion-parameter model focused on improving training efficiency and effectiveness under resource constraints. Our methodology includes three key innovations: a Quantile Data Benchmarking method for systematically comparing heterogeneous open-source datasets and providing insights on data mixing strategies; a Strategic Selective Repetition scheme within a multi-phase paradigm to effectively leverage sparse, high-quality data; and a Multi-Domain Curriculum Training policy that orders samples by quality. Supported by a highly optimized data preprocessing pipeline and architectural modifications for FP16 stability, Kaiyuan-2B achieves performance competitive with state-of-the-art fully open-source models, demonstrating practical and scalable solutions for resource-limited pretraining. We release all assets (including model weights, data, and code) under Apache 2.0 license at this https URL.
• 摘要：大型语言模型（LLM）的快速发展导致开源社区和行业之间存在巨大的知识差距，这主要是因为后者依赖闭源的高质量数据和培训方法。为了解决这个问题，我们引入了 PCMind-2.1-Kaiyuan-2B，这是一个完全开源的 20 亿参数模型，专注于提高资源限制下的训练效率和效果。我们的方法包括三项关键创新：分位数数据基准方法，用于系统地比较异构开源数据集并提供有关数据混合策略的见解；多阶段范式内的战略选择性重复方案，以有效利用稀疏的高质量数据；以及按质量订购样本的多领域课程培训政策。在高度优化的数据预处理流程和针对 FP16 稳定性的架构修改的支持下，Kaiyuan-2B 实现了与最先进的完全开源模型竞争的性能，展示了针对资源有限的预训练的实用且可扩展的解决方案。我们在 Apache 2.0 许可证下在此 https URL 发布所有资产（包括模型权重、数据和代码）。

Title: Bridging Code Graphs and Large Language Models for Better Code Understanding

• Authors: Zeqi Chen, Zhaoyang Chu, Yi Gui, Feng Guo, Yao Wan, Chuan Shi
• Subjects: cs.CL, cs.SE
• Abstract URL: https://arxiv.org/abs/2512.07666
• Pdf URL: https://arxiv.org/pdf/2512.07666
• Copy Paste: [[2512.07666]] Bridging Code Graphs and Large Language Models for Better Code Understanding(https://arxiv.org/abs/2512.07666)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) have demonstrated remarkable performance in code intelligence tasks such as code generation, summarization, and translation. However, their reliance on linearized token sequences limits their ability to understand the structural semantics of programs. While prior studies have explored graphaugmented prompting and structure-aware pretraining, they either suffer from prompt length constraints or require task-specific architectural changes that are incompatible with large-scale instructionfollowing LLMs. To address these limitations, this paper proposes CGBridge, a novel plug-and-play method that enhances LLMs with Code Graph information through an external, trainable Bridge module. CGBridge first pre-trains a code graph encoder via selfsupervised learning on a large-scale dataset of 270K code graphs to learn structural code semantics. It then trains an external module to bridge the modality gap among code, graph, and text by aligning their semantics through cross-modal attention mechanisms. Finally, the bridge module generates structure-informed prompts, which are injected into a frozen LLM, and is fine-tuned for downstream code intelligence tasks. Experiments show that CGBridge achieves notable improvements over both the original model and the graphaugmented prompting method. Specifically, it yields a 16.19% and 9.12% relative gain in LLM-as-a-Judge on code summarization, and a 9.84% and 38.87% relative gain in Execution Accuracy on code translation. Moreover, CGBridge achieves over 4x faster inference than LoRA-tuned models, demonstrating both effectiveness and efficiency in structure-aware code understanding.
• 摘要：大型语言模型 (LLM) 在代码生成、摘要和翻译等代码智能任务中表现出了卓越的性能。然而，他们对线性化标记序列的依赖限制了他们理解程序结构语义的能力。虽然之前的研究已经探索了图形增强提示和结构感知预训练，但它们要么受到提示长度限制，要么需要特定于任务的架构更改，而这些更改与大规模指令遵循的法学硕士不兼容。为了解决这些限制，本文提出了 CGBridge，这是一种新颖的即插即用方法，可通过外部可训练的 Bridge 模块使用代码图信息增强 LLM。 CGBridge 首先通过自监督学习在 270K 代码图的大规模数据集上预训练代码图编码器，以学习结构代码语义。然后，它训练一个外部模块，通过跨模态注意机制对齐代码、图形和文本之间的语义，从而弥合代码、图形和文本之间的模态差距。最后，桥模块生成结构通知提示，这些提示被注入到冻结的 LLM 中，并针对下游代码智能任务进行微调。实验表明，CGBridge 相对于原始模型和图形增强提示方法都取得了显着的改进。具体来说，它在代码摘要方面使法学硕士法官的相对增益为 16.19% 和 9.12%，在代码翻译的执行准确性方面相对增益为 9.84% 和 38.87%。此外，CGBridge 的推理速度比 LoRA 调整的模型快 4 倍以上，展示了结构感知代码理解的有效性和效率。

Title: When Large Language Models Do Not Work: Online Incivility Prediction through Graph Neural Networks

• Authors: Zihan Chen, Lanyu Yu
• Subjects: cs.CL, cs.AI, cs.SI
• Abstract URL: https://arxiv.org/abs/2512.07684
• Pdf URL: https://arxiv.org/pdf/2512.07684
• Copy Paste: [[2512.07684]] When Large Language Models Do Not Work: Online Incivility Prediction through Graph Neural Networks(https://arxiv.org/abs/2512.07684)
• Keywords: language model, llm
• Abstract: Online incivility has emerged as a widespread and persistent problem in digital communities, imposing substantial social and psychological burdens on users. Although many platforms attempt to curb incivility through moderation and automated detection, the performance of existing approaches often remains limited in both accuracy and efficiency. To address this challenge, we propose a Graph Neural Network (GNN) framework for detecting three types of uncivil behavior (i.e., toxicity, aggression, and personal attacks) within the English Wikipedia community. Our model represents each user comment as a node, with textual similarity between comments defining the edges, allowing the network to jointly learn from both linguistic content and relational structures among comments. We also introduce a dynamically adjusted attention mechanism that adaptively balances nodal and topological features during information aggregation. Empirical evaluations demonstrate that our proposed architecture outperforms 12 state-of-the-art Large Language Models (LLMs) across multiple metrics while requiring significantly lower inference cost. These findings highlight the crucial role of structural context in detecting online incivility and address the limitations of text-only LLM paradigms in behavioral prediction. All datasets and comparative outputs will be publicly available in our repository to support further research and reproducibility.
• 摘要：网络不文明行为已成为数字社区中普遍存在且持续存在的问题，给用户带来了巨大的社会和心理负担。尽管许多平台试图通过审核和自动检测来遏制不文明行为，但现有方法的准确性和效率往往仍然有限。为了应对这一挑战，我们提出了一个图神经网络（GNN）框架，用于检测英语维基百科社区内的三种不文明行为（即毒性、攻击性和人身攻击）。我们的模型将每个用户评论表示为一个节点，评论之间的文本相似性定义了边缘，允许网络共同学习评论之间的语言内容和关系结构。我们还引入了一种动态调整的注意力机制，可以在信息聚合过程中自适应地平衡节点和拓扑特征。实证评估表明，我们提出的架构在多个指标上优于 12 个最先进的大型语言模型 (LLM)，同时需要显着降低推理成本。这些发现强调了结构背景在检测网络不文明行为中的关键作用，并解决了纯文本法学硕士范式在行为预测中的局限性。所有数据集和比较输出将在我们的存储库中公开提供，以支持进一步的研究和可重复性。

Title: HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs

• Authors: Sujoy Nath, Arkaprabha Basu, Sharanya Dasgupta, Swagatam Das
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2512.07687
• Pdf URL: https://arxiv.org/pdf/2512.07687
• Copy Paste: [[2512.07687]] HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs(https://arxiv.org/abs/2512.07687)
• Keywords: language model, llm, hallucination
• Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in vision-language understanding tasks. While these models often produce linguistically coherent output, they often suffer from hallucinations, generating descriptions that are factually inconsistent with the visual content, potentially leading to adverse consequences. Therefore, the assessment of hallucinations in MLLM has become increasingly crucial in the model development process. Contemporary methodologies predominantly depend on external LLM evaluators, which are themselves susceptible to hallucinations and may present challenges in terms of domain adaptation. In this study, we propose the hypothesis that hallucination manifests as measurable irregularities within the internal layer dynamics of MLLMs, not merely due to distributional shifts but also in the context of layer-wise analysis of specific assumptions. By incorporating such modifications, \textsc{\textsc{HalluShift++}} broadens the efficacy of hallucination detection from text-based large language models (LLMs) to encompass multimodal scenarios. Our codebase is available at this https URL.
• 摘要：多模态大语言模型（MLLM）在视觉语言理解任务中表现出了卓越的能力。虽然这些模型通常会产生语言上连贯的输出，但它们经常会产生幻觉，生成与视觉内容实际上不一致的描述，从而可能导致不良后果。因此，MLLM 中幻觉的评估在模型开发过程中变得越来越重要。当代方法主要依赖于外部法学硕士评估人员，这些评估人员本身很容易产生幻觉，并可能在领域适应方面提出挑战。在这项研究中，我们提出了这样的假设：幻觉表现为 MLLM 内层动态中可测量的不规则性，这不仅是由于分布变化，而且是在对特定假设进行分层分析的背景下造成的。通过合并此类修改， \textsc{\textsc{HalluShift++}} 扩大了幻觉检测的功效，从基于文本的大语言模型（LLM）扩展到涵盖多模式场景。我们的代码库可通过此 https URL 获取。

Title: Mary, the Cheeseburger-Eating Vegetarian: Do LLMs Recognize Incoherence in Narratives?

• Authors: Karin de Langis, Püren Öncel, Ryan Peters, Andrew Elfenbein, Laura Kristen Allen, Andreas Schramm, Dongyeop Kang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07777
• Pdf URL: https://arxiv.org/pdf/2512.07777
• Copy Paste: [[2512.07777]] Mary, the Cheeseburger-Eating Vegetarian: Do LLMs Recognize Incoherence in Narratives?(https://arxiv.org/abs/2512.07777)
• Keywords: language model, llm, prompt
• Abstract: Leveraging a dataset of paired narratives, we investigate the extent to which large language models (LLMs) can reliably separate incoherent and coherent stories. A probing study finds that LLMs' internal representations can reliably identify incoherent narratives. However, LLMs generate responses to rating questions that fail to satisfactorily separate the coherent and incoherent narratives across several prompt variations, hinting at a gap in LLM's understanding of storytelling. The reasoning LLMs tested do not eliminate these deficits, indicating that thought strings may not be able to fully address the discrepancy between model internal state and behavior. Additionally, we find that LLMs appear to be more sensitive to incoherence resulting from an event that violates the setting (e.g., a rainy day in the desert) than to incoherence arising from a character violating an established trait (e.g., Mary, a vegetarian, later orders a cheeseburger), suggesting that LLMs may rely more on prototypical world knowledge than building meaning-based narrative coherence. The consistent asymmetry found in our results suggests that LLMs do not have a complete grasp on narrative coherence.
• 摘要：利用配对叙述的数据集，我们研究了大型语言模型 (LLM) 能够在多大程度上可靠地区分不连贯和连贯的故事。一项探索性研究发现，法学硕士的内部表征能够可靠地识别不连贯的叙述。然而，法学硕士对评级问题的回答无法令人满意地将连贯和不连贯的叙述在几个提示变体中区分开来，这暗示了法学硕士对讲故事的理解存在差距。法学硕士测试的推理并没有消除这些缺陷，这表明思想串可能无法完全解决模型内部状态和行为之间的差异。此外，我们发现法学硕士似乎对违反场景的事件（例如，沙漠中的雨天）所导致的不连贯性比对角色违反既定特征（例如，玛丽，素食主义者，后来点了芝士汉堡）所导致的不连贯性更敏感，这表明法学硕士可能更多地依赖于原型世界知识，而不是建立基于意义的叙事连贯性。我们的结果中发现的一致的不对称性表明法学硕士并没有完全掌握叙述的连贯性。

Title: On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models

• Authors: Charlie Zhang, Graham Neubig, Xiang Yue
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.07783
• Pdf URL: https://arxiv.org/pdf/2512.07783
• Copy Paste: [[2512.07783]] On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models(https://arxiv.org/abs/2512.07783)
• Keywords: language model
• Abstract: Recent reinforcement learning (RL) techniques have yielded impressive reasoning improvements in language models, yet it remains unclear whether post-training truly extends a model's reasoning ability beyond what it acquires during pre-training. A central challenge is the lack of control in modern training pipelines: large-scale pre-training corpora are opaque, mid-training is often underexamined, and RL objectives interact with unknown prior knowledge in complex ways. To resolve this ambiguity, we develop a fully controlled experimental framework that isolates the causal contributions of pre-training, mid-training, and RL-based post-training. Our approach employs synthetic reasoning tasks with explicit atomic operations, parseable step-by-step reasoning traces, and systematic manipulation of training distributions. We evaluate models along two axes: extrapolative generalization to more complex compositions and contextual generalization across surface contexts. Using this framework, we reconcile competing views on RL's effectiveness. We show that: 1) RL produces true capability gains (pass@128) only when pre-training leaves sufficient headroom and when RL data target the model's edge of competence, tasks at the boundary that are difficult but not yet out of reach. 2) Contextual generalization requires minimal yet sufficient pre-training exposure, after which RL can reliably transfer. 3) Mid-training significantly enhances performance under fixed compute compared with RL only, demonstrating its central but underexplored role in training pipelines. 4) Process-level rewards reduce reward hacking and improve reasoning fidelity. Together, these results clarify the interplay between pre-training, mid-training, and RL, offering a foundation for understanding and improving reasoning LM training strategies.
• 摘要：最近的强化学习（RL）技术在语言模型中取得了令人印象深刻的推理改进，但目前尚不清楚训练后是否真正将模型的推理能力扩展到训练前所获得的能力之外。一个核心挑战是现代训练流程缺乏控制：大规模预训练语料库不透明，训练中期经常被低估，强化学习目标以复杂的方式与未知的先验知识相互作用。为了解决这种模糊性，我们开发了一个完全受控的实验框架，该框架隔离了训练前、训练中和基于强化学习的训练后的因果贡献。我们的方法采用具有显式原子操作、可解析的逐步推理轨迹以及训练分布的系统操作的合成推理任务。我们沿着两个轴评估模型：对更复杂的组合的外推概括和跨表面上下文的上下文概括。使用这个框架，我们协调了关于强化学习有效性的不同观点。我们表明：1）仅当预训练留有足够的空间，并且当 RL 数据针对模型的能力边缘（即困难但尚未遥不可及的边界任务）时，RL 才会产生真正的能力增益 (pass@128)。 2）上下文泛化需要最少但足够的预训练暴露，之后 RL 可以可靠地迁移。 3）与仅 RL 相比，中期训练显着提高了固定计算下的性能，证明了其在训练流程中的核心但尚未充分开发的作用。 4）过程级奖励减少奖励黑客行为并提高推理保真度。总之，这些结果阐明了训练前、训练中和强化学习之间的相互作用，为理解和改进推理 LM 训练策略奠定了基础。

Title: Collaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Support

• Authors: Raunak Jain, Mudita Khurana
• Subjects: cs.CL, cs.AI, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.07801
• Pdf URL: https://arxiv.org/pdf/2512.07801
• Copy Paste: [[2512.07801]] Collaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Support(https://arxiv.org/abs/2512.07801)
• Keywords: llm, agent
• Abstract: LLM-based agents are rapidly being plugged into expert decision-support, yet in messy, high-stakes settings they rarely make the team smarter: human-AI teams often underperform the best individual, experts oscillate between verification loops and over-reliance, and the promised complementarity does not materialise. We argue this is not just a matter of accuracy, but a fundamental gap in how we conceive AI assistance: expert decisions are made through collaborative cognitive processes where mental models, goals, and constraints are continually co-constructed, tested, and revised between human and AI. We propose Collaborative Causal Sensemaking (CCS) as a research agenda and organizing framework for decision-support agents: systems designed as partners in cognitive work, maintaining evolving models of how particular experts reason, helping articulate and revise goals, co-constructing and stress-testing causal hypotheses, and learning from the outcomes of joint decisions so that both human and agent improve over time. We sketch challenges around training ecologies that make collaborative thinking instrumentally valuable, representations and interaction protocols for co-authored models, and evaluation centred on trust and complementarity. These directions can reframe MAS research around agents that participate in collaborative sensemaking and act as AI teammates that think with their human partners.
• 摘要：基于法学硕士的代理正在迅速融入专家决策支持中，但在混乱、高风险的环境中，它们很少能让团队变得更聪明：人类-人工智能团队的表现往往不如最优秀的个体，专家在验证循环和过度依赖之间摇摆不定，承诺的互补性也无法实现。我们认为，这不仅仅是准确性的问题，而且是我们如何构想人工智能辅助的根本差距：专家决策是通过协作认知过程做出的，其中心理模型、目标和约束在人类和人工智能之间不断共同构建、测试和修改。我们提出协作因果意义建构（CCS）作为决策支持代理的研究议程和组织框架：设计为认知工作合作伙伴的系统，维护特定专家如何推理的不断发展的模型，帮助阐明和修改目标，共同构建和压力测试因果假设，并从联合决策的结果中学习，以便人类和代理随着时间的推移而改进。我们概述了围绕使协作思维具有工具价值的培训生态、共同创作模型的表示和交互协议以及以信任和互补性为中心的评估所面临的挑战。这些方向可以围绕参与协作意义建构的智能体重新构建 MAS 研究，并充当与人类伙伴一起思考的人工智能队友。

Title: Do Generalisation Results Generalise?

• Authors: Matteo Boglioni, Andrea Sgobbi, Gabriel Tavernini, Francesco Rita, Marius Mosbach, Tiago Pimentel
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.07832
• Pdf URL: https://arxiv.org/pdf/2512.07832
• Copy Paste: [[2512.07832]] Do Generalisation Results Generalise?(https://arxiv.org/abs/2512.07832)
• Keywords: language model, llm
• Abstract: A large language model's (LLM's) out-of-distribution (OOD) generalisation ability is crucial to its deployment. Previous work assessing LLMs' generalisation performance, however, typically focuses on a single out-of-distribution dataset. This approach may fail to precisely evaluate the capabilities of the model, as the data shifts encountered once a model is deployed are much more diverse. In this work, we investigate whether OOD generalisation results generalise. More specifically, we evaluate a model's performance across multiple OOD testsets throughout a finetuning run; we then evaluate the partial correlation of performances across these testsets, regressing out in-domain performance. This allows us to assess how correlated are generalisation performances once in-domain performance is controlled for. Analysing OLMo2 and OPT, we observe no overarching trend in generalisation results: the existence of a positive or negative correlation between any two OOD testsets depends strongly on the specific choice of model analysed.
• 摘要：大型语言模型 (LLM) 的分布外 (OOD) 泛化能力对其部署至关重要。然而，之前评估法学硕士泛化性能的工作通常侧重于单个分布外数据集。这种方法可能无法精确评估模型的功能，因为部署模型后遇到的数据变化更加多样化。在这项工作中，我们研究了 OOD 泛化结果是否具有泛化性。更具体地说，我们在整个微调运行过程中评估模型在多个 OOD 测试集上的性能；然后，我们评估这些测试集性能的部分相关性，回归域内性能。这使我们能够评估一旦控制了域内性能，泛化性能的相关性如何。分析 OLMo2 和 OPT，我们没有观察到泛化结果的总体趋势：任何两个 OOD 测试集之间是否存在正相关或负相关在很大程度上取决于所分析模型的具体选择。