2025-12-03

Title: Human-Level and Beyond: Benchmarking Large Language Models Against Clinical Pharmacists in Prescription Review

• Authors: Yan Yang, Mouxiao Bian, Peiling Li, Bingjian Wen, Ruiyao Chen, Kangkun Mao, Xiaojun Ye, Tianbin Li, Pengcheng Chen, Bing Han, Jie Xu, Kaifeng Qiu, Junyan Wu
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2512.02024
• Pdf URL: https://arxiv.org/pdf/2512.02024
• Copy Paste: [[2512.02024]] Human-Level and Beyond: Benchmarking Large Language Models Against Clinical Pharmacists in Prescription Review(https://arxiv.org/abs/2512.02024)
• Keywords: language model, llm
• Abstract: The rapid advancement of large language models (LLMs) has accelerated their integration into clinical decision support, particularly in prescription review. To enable systematic and fine-grained evaluation, we developed RxBench, a comprehensive benchmark that covers common prescription review categories and consolidates 14 frequent types of prescription errors drawn from authoritative pharmacy references. RxBench consists of 1,150 single-choice, 230 multiple-choice, and 879 short-answer items, all reviewed by experienced clinical pharmacists. We benchmarked 18 state-of-the-art LLMs and identified clear stratification of performance across tasks. Notably, Gemini-2.5-pro-preview-05-06, Grok-4-0709, and DeepSeek-R1-0528 consistently formed the first tier, outperforming other models in both accuracy and robustness. Comparisons with licensed pharmacists indicated that leading LLMs can match or exceed human performance in certain tasks. Furthermore, building on insights from our benchmark evaluation, we performed targeted fine-tuning on a mid-tier model, resulting in a specialized model that rivals leading general-purpose LLMs in performance on short-answer question tasks. The main contribution of RxBench lies in establishing a standardized, error-type-oriented framework that not only reveals the capabilities and limitations of frontier LLMs in prescription review but also provides a foundational resource for building more reliable and specialized clinical tools.
• 摘要：大语言模型（LLM）的快速发展加速了它们与临床决策支持的整合，特别是在处方审查中。为了实现系统化、细粒度的评估，我们开发了 RxBench，这是一个综合基准，涵盖常见的处方审查类别，并整合了从权威药房参考文献中提取的 14 种常见处方错误类型。 RxBench 包含 1,150 个单选题、230 个多项选择题和 879 个简答题，全部由经验丰富的临床药剂师审核。我们对 18 个最先进的法学硕士进行了基准测试，并确定了跨任务的明确绩效分层。值得注意的是，Gemini-2.5-pro-preview-05-06、Grok-4-0709 和 DeepSeek-R1-0528 始终占据第一梯队，在准确性和鲁棒性方面均优于其他模型。与执业药剂师的比较表明，领先的法学硕士可以在某些任务中达到或超过人类的表现。此外，根据基准评估的见解，我们对中层模型进行了有针对性的微调，从而形成了一个专门的模型，在简答题任务的性能上可以与领先的通用法学硕士相媲美。 RxBench的主要贡献在于建立了一个标准化的、面向错误类型的框架，不仅揭示了前沿法学硕士在处方审查方面的能力和局限性，而且还为构建更可靠、更专业的临床工具提供了基础资源。

Title: Deep Research: A Systematic Survey

• Authors: Zhengliang Shi, Yiqun Chen, Haitao Li, Weiwei Sun, Shiyu Ni, Yougang Lyu, Run-Ze Fan, Bowen Jin, Yixuan Weng, Minjun Zhu, Qiujie Xie, Xinyu Guo, Qu Yang, Jiayi Wu, Jujia Zhao, Xiaqiang Tang, Xinbei Ma, Cunxiang Wang, Jiaxin Mao, Qingyao Ai, Jen-Tse Huang, Wenxuan Wang, Yue Zhang, Yiming Yang, Zhaopeng Tu, Zhaochun Ren
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2512.02038
• Pdf URL: https://arxiv.org/pdf/2512.02038
• Copy Paste: [[2512.02038]] Deep Research: A Systematic Survey(https://arxiv.org/abs/2512.02038)
• Keywords: language model, llm, prompt, retrieval-augmented generation, agent
• Abstract: Large language models (LLMs) have rapidly evolved from text generators into powerful problem solvers. Yet, many open tasks demand critical thinking, multi-source, and verifiable outputs, which are beyond single-shot prompting or standard retrieval-augmented generation. Recently, numerous studies have explored Deep Research (DR), which aims to combine the reasoning capabilities of LLMs with external tools, such as search engines, thereby empowering LLMs to act as research agents capable of completing complex, open-ended tasks. This survey presents a comprehensive and systematic overview of deep research systems, including a clear roadmap, foundational components, practical implementation techniques, important challenges, and future directions. Specifically, our main contributions are as follows: (i) we formalize a three-stage roadmap and distinguish deep research from related paradigms; (ii) we introduce four key components: query planning, information acquisition, memory management, and answer generation, each paired with fine-grained sub-taxonomies; (iii) we summarize optimization techniques, including prompting, supervised fine-tuning, and agentic reinforcement learning; and (iv) we consolidate evaluation criteria and open challenges, aiming to guide and facilitate future development. As the field of deep research continues to evolve rapidly, we are committed to continuously updating this survey to reflect the latest progress in this area.
• 摘要：大型语言模型 (LLM) 已从文本生成器迅速发展为强大的问题解决器。然而，许多开放任务需要批判性思维、多源和可验证的输出，这超出了单次提示或标准检索增强生成的范围。最近，大量研究探索了深度研究（DR），旨在将法学硕士的推理能力与搜索引擎等外部工具相结合，从而使法学硕士能够充当能够完成复杂、开放式任务的研究代理。本次调查对深度研究系统进行了全面、系统的概述，包括清晰的路线图、基础组成部分、实际实施技术、重要挑战和未来方向。具体来说，我们的主要贡献如下：（i）我们正式制定了一个三阶段路线图，并将深度研究与相关范式区分开来； (ii)我们引入四个关键组件：查询规划、信息获取、内存管理和答案生成，每个组件都与细粒度的子分类法配对； (iii)我们总结了优化技术，包括提示、监督微调和代理强化学习； （四）统一评价标准，开放挑战，引导和促进未来发展。随着深度研究领域不断快速发展，我们致力于不断更新本次调查，以反映该领域的最新进展。

Title: Mirror, Mirror on the Wall -- Which is the Best Model of Them All?

• Authors: Dina Sayed, Heiko Schuldt
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02043
• Pdf URL: https://arxiv.org/pdf/2512.02043
• Copy Paste: [[2512.02043]] Mirror, Mirror on the Wall -- Which is the Best Model of Them All?(https://arxiv.org/abs/2512.02043)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have become one of the most transformative tools across many applications, as they have significantly boosted productivity and achieved impressive results in various domains such as finance, healthcare, education, telecommunications, and law, among others. Typically, state-of-the-art (SOTA) foundation models are developed by large corporations based on large data collections and substantial computational and financial resources required to pretrain such models from scratch. These foundation models then serve as the basis for further development and domain adaptation for specific use cases or tasks. However, given the dynamic and fast-paced nature of launching new foundation models, the process of selecting the most suitable model for a particular use case, application, or domain becomes increasingly complex. We argue that there are two main dimensions that need to be taken into consideration when selecting a model for further training: a qualitative dimension (which model is best suited for a task based on information, for instance, taken from model cards) and a quantitative dimension (which is the best performing model). The quantitative performance of models is assessed through leaderboards, which rank models based on standardized benchmarks and provide a consistent framework for comparing different LLMs. In this work, we address the analysis of the quantitative dimension by exploring the current leaderboards and benchmarks. To illustrate this analysis, we focus on the medical domain as a case study, demonstrating the evolution, current landscape, and practical significance of this quantitative evaluation dimension. Finally, we propose a Model Selection Methodology (MSM), a systematic approach designed to guide the navigation, prioritization, and selection of the model that best aligns with a given use case.
• 摘要：大型语言模型 (LLM) 已成为许多应用程序中最具变革性的工具之一，因为它们显着提高了生产力，并在金融、医疗保健、教育、电信和法律等各个领域取得了令人印象深刻的成果。通常，最先进的 (SOTA) 基础模型是由大公司基于大量数据收集以及从头开始预训练此类模型所需的大量计算和财务资源而开发的。然后，这些基础模型将作为特定用例或任务的进一步开发和领域适应的基础。然而，考虑到推出新基础模型的动态性和快节奏性，为特定用例、应用程序或领域选择最合适模型的过程变得越来越复杂。我们认为，在选择进一步训练的模型时需要考虑两个主要维度：定性维度（哪种模型最适合基于信息的任务，例如从模型卡中获取的信息）和定量维度（这是性能最好的模型）。模型的定量表现通过排行榜进行评估，排行榜根据标准化基准对模型进行排名，并为比较不同的法学硕士提供一致的框架。在这项工作中，我们通过探索当前的排行榜和基准来解决定量维度的分析。为了说明这一分析，我们以医学领域为例，展示这一定量评估维度的演变、现状和实际意义。最后，我们提出了模型选择方法（MSM），这是一种系统方法，旨在指导导航、优先级排序和选择最适合给定用例的模型。

Title: Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models

• Authors: Kecheng Chen, Ziru Liu, Xijia Tao, Hui Liu, Xinyu Fu, Suiyun Zhang, Dandan Tu, Lingpeng Kong, Rui Liu, Haoliang Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02044
• Pdf URL: https://arxiv.org/pdf/2512.02044
• Copy Paste: [[2512.02044]] Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models(https://arxiv.org/abs/2512.02044)
• Keywords: language model
• Abstract: Diffusion Language Models (DLMs) have recently achieved significant success due to their any-order generation capabilities. However, existing inference methods typically rely on local, immediate-step metrics such as confidence or entropy which inherently lack a more reliable perspective. This limitation frequently leads to inconsistent sampling trajectories and suboptimal generation quality. To address this, we propose Coherent Contextual Decoding (CCD), a novel inference framework built upon two core innovations. First, CCD employs a trajectory rectification mechanism that leverages historical context to enhance sequence coherence, enabling the early rejection of suboptimal paths. We demonstrate that this mechanism is theoretically equivalent to modeling the consistency of historical steps via the conditional mutual information between context and token predictions. Building on this theoretical insight, we further address the inefficiency of conventional uniform decoding budgets. Instead of rigid allocations based on diffusion steps, we introduce an adaptive sampling strategy that dynamically adjusts the unmasking budget for each step according to our consistency metric. Consequently, our method significantly improves the quality of generation trajectories while accelerating the sampling process. Empirically, our method achieves a simultaneous enhancement in both inference speed and performance across diverse benchmarks on Dream and LLaDA, delivering up to 3.48x speedup alongside 3.91% performance improvement.
• 摘要：扩散语言模型 (DLM) 最近因其任意顺序生成功能而取得了巨大成功。然而，现有的推理方法通常依赖于局部的、即时的步骤度量，例如置信度或熵，这本质上缺乏更可靠的视角。这种限制经常导致不一致的采样轨迹和次优的生成质量。为了解决这个问题，我们提出了连贯上下文解码（CCD），这是一种基于两项核心创新的新颖推理框架。首先，CCD 采用轨迹校正机制，利用历史背景来增强序列一致性，从而能够及早拒绝次优路径。我们证明，这种机制在理论上相当于通过上下文和令牌预测之间的条件互信息来建模历史步骤的一致性。基于这一理论见解，我们进一步解决了传统统一解码预算的低效率问题。我们引入了一种自适应采样策略，而不是基于扩散步骤的严格分配，该策略根据我们的一致性指标动态调整每个步骤的揭露预算。因此，我们的方法显着提高了生成轨迹的质量，同时加速了采样过程。根据经验，我们的方法在 Dream 和 LLaDA 的不同基准上实现了推理速度和性能的同步增强，实现了高达 3.48 倍的加速和 3.91% 的性能提升。

Title: Reversing Large Language Models for Efficient Training and Fine-Tuning

• Authors: Eshed Gal, Moshe Eliasof, Javier Turek, Uri Ascher, Eran Treister, Eldad Haber
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.02056
• Pdf URL: https://arxiv.org/pdf/2512.02056
• Copy Paste: [[2512.02056]] Reversing Large Language Models for Efficient Training and Fine-Tuning(https://arxiv.org/abs/2512.02056)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are known for their expensive and time-consuming training. Thus, oftentimes, LLMs are fine-tuned to address a specific task, given the pretrained weights of a pre-trained LLM considered a foundation model. In this work, we introduce memory-efficient, reversible architectures for LLMs, inspired by symmetric and symplectic differential equations, and investigate their theoretical properties. Different from standard, baseline architectures that store all intermediate activations, the proposed models use time-reversible dynamics to retrieve hidden states during backpropagation, relieving the need to store activations. This property allows for a drastic reduction in memory consumption, allowing for the processing of larger batch sizes for the same available memory, thereby offering improved throughput. In addition, we propose an efficient method for converting existing, non-reversible LLMs into reversible architectures through fine-tuning, rendering our approach practical for exploiting existing pre-trained models. Our results show comparable or improved performance on several datasets and benchmarks, on several LLMs, building a scalable and efficient path towards reducing the memory and computational costs associated with both training from scratch and fine-tuning of LLMs.
• 摘要：大型语言模型 (LLM) 以其昂贵且耗时的培训而闻名。因此，考虑到预训练的 LLM 的预训练权重（被视为基础模型），LLM 通常会进行微调以解决特定任务。在这项工作中，我们受对称和辛微分方程的启发，引入了 LLM 的内存高效、可逆架构，并研究了它们的理论特性。与存储所有中间激活的标准基线架构不同，所提出的模型使用时间可逆动力学来检索反向传播期间的隐藏状态，从而无需存储激活。此属性可以大幅减少内存消耗，允许在相同的可用内存下处理更大的批量大小，从而提高吞吐量。此外，我们提出了一种有效的方法，通过微调将现有的不可逆 LLM 转换为可逆架构，使我们的方法可用于利用现有的预训练模型。我们的结果显示，在多个数据集和基准、多个法学硕士上的性能相当或有所提高，构建了一条可扩展且有效的路径，以减少与从头开始训练和法学硕士微调相关的内存和计算成本。

Title: Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models

• Authors: Ziyan Wang, Enmao Diao, Qi Le, Pu Wang, Guanchu Wang, Minwoo Lee, Shu-ping Yeh, Li Yang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.02185
• Pdf URL: https://arxiv.org/pdf/2512.02185
• Copy Paste: [[2512.02185]] Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models(https://arxiv.org/abs/2512.02185)
• Keywords: language model, llm, chain-of-thought
• Abstract: Reasoning LLMs (RLMs) such as OpenAI o1, DeepSeek-R1, and Qwen3 deliver strong multi-step reasoning through chain-of-thought generation, but their large model sizes and lengthy decode-time outputs make them costly to deploy and unsuitable for resource-constrained settings. To reduce computing and memory cost, pruning offers a promising solution by removing unimportant parameters. However, despite their success on standard LLMs, existing pruning methods severely damage RLMs, as even moderate sparsity (e.g., 20%) can collapse accuracy and completely disrupt the model's reasoning coherence. We begin by analyzing why existing pruning pipelines fail on reasoning LLMs and find that their brittleness largely stems from a mismatch between the calibration data, the pruning objective, and the model's decode-time reasoning behavior. Our study further shows that the most reliable calibration signal comes not from human-written labels but from the model's own self-generated reasoning traces, which more accurately reflect its inference distribution. Guided by these insights, we introduce RESP, a self-reflective structured pruning framework that aligns pruning decisions with the model's reasoning dynamics through self-generated calibration, decode-only gradient-based importance estimation, and progressive regeneration that maintains calibration fidelity as sparsity increases. Experiments on Qwen3-8B demonstrate that RESP markedly outperforms existing structured pruning methods on both GSM8K and MathQA, preserving near-dense accuracy at 20-30% sparsity and substantially mitigating performance collapse at higher sparsity levels. At 40% sparsity, RESP attains 81.3% accuracy on GSM8K and 59.6% on MathQA, surpassing the strongest baselines by 66.87% and 47%, respectively.
• 摘要：OpenAI o1、DeepSeek-R1 和 Qwen3 等推理 LLM (RLM) 通过思想链生成提供强大的多步骤推理，但其较大的模型尺寸和较长的解码时间输出使得部署成本高昂，并且不适合资源受限的设置。为了降低计算和内存成本，剪枝通过删除不重要的参数提供了一种有前景的解决方案。然而，尽管它们在标准 LLM 上取得了成功，但现有的剪枝方法严重损害了 RLM，因为即使是中等稀疏度（例如 20%）也会降低准确性并完全破坏模型的推理连贯性。我们首先分析为什么现有的剪枝流程无法推理 LLM，并发现它们的脆弱性很大程度上源于校准数据、剪枝目标和模型的解码时推理行为之间的不匹配。我们的研究进一步表明，最可靠的校准信号不是来自人类书写的标签，而是来自模型自己生成的推理轨迹，这更准确地反映了其推理分布。在这些见解的指导下，我们引入了 RESP，这是一种自我反思的结构化剪枝框架，它通过自生成校准、仅解码基于梯度的重要性估计和渐进再生（随着稀疏度的增加而保持校准保真度）将剪枝决策与模型的推理动态保持一致。 Qwen3-8B 上的实验表明，RESP 在 GSM8K 和 MathQA 上的性能明显优于现有的结构化剪枝方法，在 20-30% 稀疏度下保持接近密集的精度，并在更高稀疏度水平下显着缓解性能崩溃。在稀疏度为 40% 的情况下，RESP 在 GSM8K 上的准确率达到 81.3%，在 MathQA 上的准确率达到 59.6%，分别超出最强基线 66.87% 和 47%。

Title: A Knowledge-Based Language Model: Deducing Grammatical Knowledge in a Multi-Agent Language Acquisition Simulation

• Authors: David Ph. Shakouri, Crit Cremers, Niels O. Schiller
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.02195
• Pdf URL: https://arxiv.org/pdf/2512.02195
• Copy Paste: [[2512.02195]] A Knowledge-Based Language Model: Deducing Grammatical Knowledge in a Multi-Agent Language Acquisition Simulation(https://arxiv.org/abs/2512.02195)
• Keywords: language model, agent
• Abstract: This paper presents an initial study performed by the MODOMA system. The MODOMA is a computational multi-agent laboratory environment for unsupervised language acquisition experiments such that acquisition is based on the interaction between two language models, an adult and a child agent. Although this framework employs statistical as well as rule-based procedures, the result of language acquisition is a knowledge-based language model, which can be used to generate and parse new utterances of the target language. This system is fully parametrized and researchers can control all aspects of the experiments while the results of language acquisition, that is, the acquired grammatical knowledge, are explicitly represented and can be consulted. Thus, this system introduces novel possibilities for conducting computational language acquisition experiments. The experiments presented by this paper demonstrate that functional and content categories can be acquired and represented by the daughter agent based on training and test data containing different amounts of exemplars generated by the adult agent. Interestingly, similar patterns, which are well-established for human-generated data, are also found for these machine-generated data. As the procedures resulted in the successful acquisition of discrete grammatical categories by the child agent, these experiments substantiate the validity of the MODOMA approach to modelling language acquisition.
• 摘要：本文介绍了 MODOMA 系统进行的初步研究。 MODOMA 是一个计算多智能体实验室环境，用于无监督语言习得实验，这样的习得基于两种语言模型（成人和儿童智能体）之间的交互。尽管该框架采用统计和基于规则的过程，但语言习得的结果是基于知识的语言模型，可用于生成和解析目标语言的新话语。该系统是完全参数化的，研究人员可以控制实验的各个方面，同时语言习得的结果，即获得的语法知识，得到明确的表示并可以查阅。因此，该系统为进行计算语言习得实验引入了新的可能性。本文提出的实验表明，子代理可以根据包含成年代理生成的不同数量的样本的训练和测试数据来获取和表示功能和内容类别。有趣的是，对于人类生成的数据来说，类似的模式也已在这些机器生成的数据中找到。由于这些过程导致儿童智能体成功习得离散语法类别，因此这些实验证实了 MODOMA 建模语言习得方法的有效性。

Title: Lightweight Latent Reasoning for Narrative Tasks

• Authors: Alexander Gurung, Nikolay Malkin, Mirella Lapata
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02240
• Pdf URL: https://arxiv.org/pdf/2512.02240
• Copy Paste: [[2512.02240]] Lightweight Latent Reasoning for Narrative Tasks(https://arxiv.org/abs/2512.02240)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) tackle complex tasks by generating long chains of thought or "reasoning traces" that act as latent variables in the generation of an output given a query. A model's ability to generate such traces can be optimized with reinforcement learning (RL) to improve their utility in predicting an answer. This optimization comes at a high computational cost, especially for narrative-related tasks that involve retrieving and processing many tokens. To this end, we propose LiteReason, a latent reasoning method that can be interleaved with standard token sampling and easily combined with RL techniques. LiteReason employs a lightweight Reasoning Projector module, trained to produce continuous latent tokens that help the model 'skip' reasoning steps. During RL, the policy model decides when to activate the projector, switching between latent and discrete reasoning as needed. Experimental results on plot hole detection and book chapter generation show that our method outperforms latent reasoning baselines and comes close to matching non-latent RL training, while reducing final reasoning length by 77-92%. Overall, LiteReason guides RL training to a more efficient part of the performance-computation tradeoff curve.
• 摘要：大型语言模型（LLM）通过生成长的思想链或“推理轨迹”来处理复杂的任务，这些思想或“推理轨迹”在给定查询的输出生成过程中充当潜在变量。模型生成此类痕迹的能力可以通过强化学习 (RL) 进行优化，以提高其预测答案的效用。这种优化的计算成本很高，特别是对于涉及检索和处理许多标记的叙事相关任务。为此，我们提出了 LiteReason，这是一种潜在推理方法，可以与标准 token 采样交织，并轻松与 RL 技术结合。 LiteReason 采用轻量级推理投影仪模块，经过训练可生成连续的潜在标记，帮助模型“跳过”推理步骤。在强化学习期间，策略模型决定何时激活投影仪，根据需要在潜在推理和离散推理之间切换。情节漏洞检测和书籍章节生成的实验结果表明，我们的方法优于潜在推理基线，接近匹配非潜在 RL 训练，同时将最终推理长度减少了 77-92%。总体而言，LiteReason 引导 RL 训练达到性能-计算权衡曲线中更有效的部分。

Title: DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models

• Authors: Olivia Kim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.02246
• Pdf URL: https://arxiv.org/pdf/2512.02246
• Copy Paste: [[2512.02246]] DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models(https://arxiv.org/abs/2512.02246)
• Keywords: language model, gpt, llm, prompt
• Abstract: Prompt design plays a critical role in the reasoning performance of large language models (LLMs), yet the impact of prompt specificity - how detailed or vague a prompt is - remains understudied. This paper introduces DETAIL, a framework for evaluating LLM performance across varying levels of prompt specificity. We generate multi-level prompts using GPT-4, quantify specificity via perplexity, and assess correctness using GPT-based semantic equivalence. Experiments on 30 novel reasoning tasks across GPT-4 and O3-mini reveal that specificity improves accuracy, especially for smaller models and procedural tasks. Our results highlight the need for adaptive prompting strategies and provide tools and data to support further research.
• 摘要：提示设计在大型语言模型 (LLM) 的推理性能中起着至关重要的作用，但提示特异性的影响（提示的详细程度或模糊程度）仍未得到充分研究。本文介绍了 DETAIL，一个用于评估不同级别的提示特异性的法学硕士表现的框架。我们使用 GPT-4 生成多级提示，通过困惑度量化特异性，并使用基于 GPT 的语义等价性评估正确性。对 GPT-4 和 O3-mini 的 30 个新颖推理任务进行的实验表明，特异性可以提高准确性，特别是对于较小的模型和程序任务。我们的结果强调了适应性提示策略的必要性，并提供了支持进一步研究的工具和数据。

Title: CAIRNS: Balancing Readability and Scientific Accuracy in Climate Adaptation Question Answering

• Authors: Liangji Kong, Aditya Joshi, Sarvnaz Karimi
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2512.02251
• Pdf URL: https://arxiv.org/pdf/2512.02251
• Copy Paste: [[2512.02251]] CAIRNS: Balancing Readability and Scientific Accuracy in Climate Adaptation Question Answering(https://arxiv.org/abs/2512.02251)
• Keywords: llm, prompt
• Abstract: Climate adaptation strategies are proposed in response to climate change. They are practised in agriculture to sustain food production. These strategies can be found in unstructured data (for example, scientific literature from the Elsevier website) or structured (heterogeneous climate data via government APIs). We present Climate Adaptation question-answering with Improved Readability and Noted Sources (CAIRNS), a framework that enables experts -- farmer advisors -- to obtain credible preliminary answers from complex evidence sources from the web. It enhances readability and citation reliability through a structured ScholarGuide prompt and achieves robust evaluation via a consistency-weighted hybrid evaluator that leverages inter-model agreement with experts. Together, these components enable readable, verifiable, and domain-grounded question-answering without fine-tuning or reinforcement learning. Using a previously reported dataset of expert-curated question-answers, we show that CAIRNS outperforms the baselines on most of the metrics. Our thorough ablation study confirms the results on all metrics. To validate our LLM-based evaluation, we also report an analysis of correlations against human judgment.
• 摘要：气候适应战略是针对气候变化而提出的。它们在农业中用于维持粮食生产。这些策略可以在非结构化数据（例如，爱思唯尔网站上的科学文献）或结构化数据（通过政府 API 获得的异构气候数据）中找到。我们提出了具有改进的可读性和注明来源的气候适应问答（CAIRNS），这是一个框架，使专家（农民顾问）能够从网络上的复杂证据来源中获得可靠的初步答案。它通过结构化的 ScholarGuide 提示增强可读性和引文可靠性，并通过利用与专家的模型间协议的一致性加权混合评估器实现稳健的评估。这些组件共同实现了可读、可验证和基于领域的问答，无需微调或强化学习。使用之前报告的专家策划的问题解答数据集，我们表明 CAIRNS 在大多数指标上都优于基线。我们彻底的消融研究证实了所有指标的结果。为了验证我们基于法学硕士的评估，我们还报告了针对人类判断的相关性分析。

Title: HealthContradict: Evaluating Biomedical Knowledge Conflicts in Language Models

• Authors: Boya Zhang, Alban Bornet, Rui Yang, Nan Liu, Douglas Teodoro
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.02299
• Pdf URL: https://arxiv.org/pdf/2512.02299
• Copy Paste: [[2512.02299]] HealthContradict: Evaluating Biomedical Knowledge Conflicts in Language Models(https://arxiv.org/abs/2512.02299)
• Keywords: language model, prompt
• Abstract: How do language models use contextual information to answer health questions? How are their responses impacted by conflicting contexts? We assess the ability of language models to reason over long, conflicting biomedical contexts using HealthContradict, an expert-verified dataset comprising 920 unique instances, each consisting of a health-related question, a factual answer supported by scientific evidence, and two documents presenting contradictory stances. We consider several prompt settings, including correct, incorrect or contradictory context, and measure their impact on model outputs. Compared to existing medical question-answering evaluation benchmarks, HealthContradict provides greater distinctions of language models' contextual reasoning capabilities. Our experiments show that the strength of fine-tuned biomedical language models lies not only in their parametric knowledge from pretraining, but also in their ability to exploit correct context while resisting incorrect context.
• 摘要：语言模型如何使用上下文信息来回答健康问题？他们的反应如何受到冲突环境的影响？我们使用 HealthContradict 评估语言模型在长期、相互冲突的生物医学背景下进行推理的能力，HealthContradict 是一个经过专家验证的数据集，包含 920 个独特的实例，每个实例都包含一个与健康相关的问题、一个由科学证据支持的事实答案以及两个提出矛盾立场的文档。我们考虑几种提示设置，包括正确、不正确或矛盾的上下文，并衡量它们对模型输出的影响。与现有的医学问答评估基准相比，HealthContradict 在语言模型的上下文推理能力上提供了更大的区别。我们的实验表明，经过微调的生物医学语言模型的优势不仅在于它们来自预训练的参数知识，还在于它们利用正确上下文同时抵制错误上下文的能力。

Title: When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifiers

• Authors: Jack Lu, Ryan Teehan, Jinran Jin, Mengye Ren
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02304
• Pdf URL: https://arxiv.org/pdf/2512.02304
• Copy Paste: [[2512.02304]] When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifiers(https://arxiv.org/abs/2512.02304)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) can act as both problem solvers and solution verifiers, with verifiers improving solver performance by selecting high-quality answers from a pool of candidates. However, prior studies of solver-verifier interactions have been limited, focusing mainly on self-verification and rarely examining how verifiers judge outputs from models in their own or in another model family. Modern LLMs also undergo extensive post-training, but its effect on verification remains unclear. We present a systematic study across 37 models spanning multiple families, sizes, and base vs. post-trained variants, evaluated on 9 benchmarks covering logical reasoning, structured puzzles, symbolic computation, mathematics, commonsense, factual recall, and domain knowledge. We compare self-verification with verification within the same family and across different families. To support this, we introduce and empirically validate verifier gain, a metric that predicts the performance improvements from test-time verifier-based rejection sampling. We analyze how metrics like verifier gain and false positive rate scale with model size and post-training, and characterize differences in dataset verifiability. Our findings show that cross-family verification is especially effective; post-training reduces self-improvement but strengthens cross-family improvement; and mathematical and logical tasks exhibit the highest inherent verifiability.
• 摘要：大型语言模型 (LLM) 既可以充当问题解决者，也可以充当解决方案验证者，验证者通过从候选池中选择高质量的答案来提高求解器的性能。然而，先前对求解器-验证器交互的研究很有限，主要集中在自我验证上，很少研究验证器如何判断自己或另一个模型系列中模型的输出。现代法学硕士也接受广泛的后期培训，但其对验证的影响仍不清楚。我们提出了一项针对 37 个模型的系统研究，涵盖多个系列、大小以及基础与训练后变体，并根据涵盖逻辑推理、结构化谜题、符号计算、数学、常识、事实回忆和领域知识的 9 个基准进行评估。我们将自我验证与同一家庭内和不同家庭之间的验证进行比较。为了支持这一点，我们引入并凭经验验证验证器增益，这是一个预测测试时基于验证器的拒绝采样的性能改进的指标。我们分析验证者增益和误报率等指标如何随模型大小和训练后进行调整，并描述数据集可验证性的差异。我们的研究结果表明，跨家庭验证尤其有效；培训后减少了自我提高，但加强了跨家庭的提高；数学和逻辑任务表现出最高的固有可验证性。

Title: Memory-Augmented Knowledge Fusion with Safety-Aware Decoding for Domain-Adaptive Question Answering

• Authors: Lei Fu, Xiang Chen, Kaige Gao Xinyue Huang, Kejian Tong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.02363
• Pdf URL: https://arxiv.org/pdf/2512.02363
• Copy Paste: [[2512.02363]] Memory-Augmented Knowledge Fusion with Safety-Aware Decoding for Domain-Adaptive Question Answering(https://arxiv.org/abs/2512.02363)
• Keywords: language model
• Abstract: Domain-specific question answering (QA) systems for services face unique challenges in integrating heterogeneous knowledge sources while ensuring both accuracy and safety. Existing large language models often struggle with factual consistency and context alignment in sensitive domains such as healthcare policies and government welfare. In this work, we introduce Knowledge-Aware Reasoning and Memory-Augmented Adaptation (KARMA), a novel framework designed to enhance QA performance in care scenarios. KARMA incorporates a dual-encoder architecture to fuse structured and unstructured knowledge sources, a gated memory unit to dynamically regulate external knowledge integration, and a safety-aware controllable decoder that mitigates unsafe outputs using safety classification and guided generation techniques. Extensive experiments on a proprietary QA dataset demonstrate that KARMA outperforms strong baselines in both answer quality and safety. This study offers a comprehensive solution for building trustworthy and adaptive QA systems in service contexts.
• 摘要：特定领域的服务问答 (QA) 系统在集成异构知识源同时确保准确性和安全性方面面临着独特的挑战。现有的大型语言模型经常在医疗政策和政府福利等敏感领域难以实现事实一致性和上下文一致性。在这项工作中，我们介绍了知识感知推理和记忆增强适应（KARMA），这是一种旨在增强护理场景中的 QA 性能的新颖框架。 KARMA 采用双编码器架构来融合结构化和非结构化知识源、门控存储单元来动态调节外部知识集成，以及安全感知可控解码器，使用安全分类和引导生成技术来减少不安全输出。对专有 QA 数据集的大量实验表明，KARMA 在答案质量和安全性方面均优于强大的基线。这项研究为在服务环境中构建值得信赖和自适应的质量保证系统提供了全面的解决方案。

Title: TaleFrame: An Interactive Story Generation System with Fine-Grained Control and Large Language Models

• Authors: Yunchao Wang, Guodao Sun, Zihang Fu, Zhehao Liu, Kaixing Du, Haidong Gao, Ronghua Liang
• Subjects: cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2512.02402
• Pdf URL: https://arxiv.org/pdf/2512.02402
• Copy Paste: [[2512.02402]] TaleFrame: An Interactive Story Generation System with Fine-Grained Control and Large Language Models(https://arxiv.org/abs/2512.02402)
• Keywords: language model, llm
• Abstract: With the advancement of natural language generation (NLG) technologies, creative story generation systems have gained increasing attention. However, current systems often fail to accurately translate user intent into satisfactory story outputs due to a lack of fine-grained control and unclear input specifications, limiting their applicability. To address this, we propose TaleFrame, a system that combines large language models (LLMs) with human-computer interaction (HCI) to generate stories through structured information, enabling precise control over the generation process. The innovation of TaleFrame lies in decomposing the story structure into four basic units: entities, events, relationships, and story outline. We leverage the Tinystories dataset, parsing and constructing a preference dataset consisting of 9,851 JSON-formatted entries, which is then used to fine-tune a local Llama model. By employing this JSON2Story approach, structured data is transformed into coherent stories. TaleFrame also offers an intuitive interface that supports users in creating and editing entities and events and generates stories through the structured framework. Users can control these units through simple interactions (e.g., drag-and-drop, attach, and connect), thus influencing the details and progression of the story. The generated stories can be evaluated across seven dimensions (e.g., creativity, structural integrity), with the system providing suggestions for refinement based on these evaluations. Users can iteratively adjust the story until a satisfactory result is achieved. Finally, we conduct quantitative evaluation and user studies that demonstrate the usefulness of TaleFrame. Dataset available at this https URL.
• 摘要：随着自然语言生成（NLG）技术的进步，创意故事生成系统越来越受到关注。然而，由于缺乏细粒度的控制和不明确的输入规范，当前的系统往往无法准确地将用户意图转化为令人满意的故事输出，从而限制了其适用性。为了解决这个问题，我们提出了TaleFrame，这是一个将大型语言模型（LLM）与人机交互（HCI）相结合的系统，通过结构化信息生成故事，从而能够精确控制生成过程。 TaleFrame的创新之处在于将故事结构分解为四个基本单元：实体、事件、关系和故事大纲。我们利用 Tinystories 数据集，解析并构建由 9,851 个 JSON 格式条目组成的偏好数据集，然后将其用于微调本地 Llama 模型。通过采用这种 JSON2Story 方法，结构化数据可以转换为连贯的故事。 TaleFrame 还提供直观的界面，支持用户创建和编辑实体和事件，并通过结构化框架生成故事。用户可以通过简单的交互（例如拖放、附加和连接）来控制这些单元，从而影响故事的细节和进展。生成的故事可以从七个维度（例如创造力、结构完整性）进行评估，系统根据这些评估提供改进建议。用户可以迭代调整故事，直到获得满意的结果。最后，我们进行定量评估和用户研究，证明 TaleFrame 的实用性。此 https URL 提供数据集。

Title: A Concise Review of Hallucinations in LLMs and their Mitigation

• Authors: Parth Pulkundwar, Vivek Dhanawade, Rohit Yadav, Minal Sonkar, Medha Asurlekar, Sarita Rathod
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.02527
• Pdf URL: https://arxiv.org/pdf/2512.02527
• Copy Paste: [[2512.02527]] A Concise Review of Hallucinations in LLMs and their Mitigation(https://arxiv.org/abs/2512.02527)
• Keywords: language model, llm, hallucination
• Abstract: Traditional language models face a challenge from hallucinations. Their very presence casts a large, dangerous shadow over the promising realm of natural language processing. It becomes crucial to understand the various kinds of hallucinations that occur nowadays, their origins, and ways of reducing them. This document provides a concise and straightforward summary of that. It serves as a one-stop resource for a general understanding of hallucinations and how to mitigate them.
• 摘要：传统的语言模型面临着幻觉的挑战。它们的存在给充满希望的自然语言处理领域蒙上了巨大而危险的阴影。了解当今发生的各种幻觉、它们的起源以及减少幻觉的方法变得至关重要。本文档对此进行了简洁明了的总结。它是对幻觉以及如何减轻幻觉的总体了解的一站式资源。

Title: What Signals Really Matter for Misinformation Tasks? Evaluating Fake-News Detection and Virality Prediction under Real-World Constraints

• Authors: Francesco Paolo Savatteri (ENC), Chahan Vidal-Gorène (CJM, LIPN), Florian Cafiero (ENC)
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02552
• Pdf URL: https://arxiv.org/pdf/2512.02552
• Copy Paste: [[2512.02552]] What Signals Really Matter for Misinformation Tasks? Evaluating Fake-News Detection and Virality Prediction under Real-World Constraints(https://arxiv.org/abs/2512.02552)
• Keywords: language model
• Abstract: We present an evaluation-driven study of two practical tasks regarding online misinformation: (i) fake-news detection and (ii) virality prediction in the context of operational settings, with the necessity for rapid reaction. Using the EVONS and FakeNewsNet datasets, we compare textual embeddings (RoBERTa; with a control using Mistral) against lightweight numeric features (timing, follower counts, verification, likes) and sequence models (GRU, gating architectures, Transformer encoders). We show that textual content alone is a strong discriminator for fake-news detection, while numeric-only pipelines remain viable when language models are unavailable or compute is constrained. Virality prediction is markedly harder than fake-news detection and is highly sensitive to label construction; in our setup, a median-based ''viral'' split (<50 likes) is pragmatic but underestimates real-world virality, and time-censoring for engagement features is desirable yet difficult under current API limits. Dimensionality-reduction analyses suggest non-linear structure is more informative for virality than for fake-news detection (t-SNE > PCA on numeric features). Swapping RoBERTa for Mistral embeddings yields only modest deltas, leaving conclusions unchanged. We discuss implications for evaluation design and report reproducibility constraints that realistically affect the field. We release splits and code where possible and provide guidance for metric selection.
• 摘要：我们提出了一项评估驱动的研究，涉及在线错误信息的两项实际任务：（i）假新闻检测和（ii）操作环境中的病毒式传播预测，并需要快速反应。使用 EVONS 和 FakeNewsNet 数据集，我们将文本嵌入（RoBERTa；使用 Mistral 进行控制）与轻量级数字特征（计时、关注者计数、验证、点赞）和序列模型（GRU、门控架构、Transformer 编码器）进行比较。我们证明，文本内容本身就是假新闻检测的强大鉴别器，而当语言模型不可用或计算受到限制时，纯数字管道仍然可行。病毒式传播预测明显比假新闻检测困难，并且对标签构建高度敏感；在我们的设置中，基于中位数的“病毒式”分割（<50 个赞）是务实的，但低估了现实世界的病毒式传播，并且对参与功能的时间审查是可取的，但在当前 API 限制下很困难。降维分析表明，非线性结构对于病毒性传播的信息量比对于假新闻检测的信息量更大（在数字特征上，t-SNE > PCA）。将 RoBERTa 替换为 Mistral 嵌入仅产生适度的增量，结论不变。我们讨论了评估设计的影响，并报告了实际影响该领域的可重复性约束。我们在可能的情况下发布拆分和代码，并为指标选择提供指导。

Title: ADORE: Autonomous Domain-Oriented Relevance Engine for E-commerce

• Authors: Zheng Fang, Donghao Xie, Ming Pang, Chunyuan Yuan, Xue Jiang, Changping Peng, Zhangang Lin, Zheng Luo
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2512.02555
• Pdf URL: https://arxiv.org/pdf/2512.02555
• Copy Paste: [[2512.02555]] ADORE: Autonomous Domain-Oriented Relevance Engine for E-commerce(https://arxiv.org/abs/2512.02555)
• Keywords: llm, chain-of-thought
• Abstract: Relevance modeling in e-commerce search remains challenged by semantic gaps in term-matching methods (e.g., BM25) and neural models' reliance on the scarcity of domain-specific hard samples. We propose ADORE, a self-sustaining framework that synergizes three innovations: (1) A Rule-aware Relevance Discrimination module, where a Chain-of-Thought LLM generates intent-aligned training data, refined via Kahneman-Tversky Optimization (KTO) to align with user behavior; (2) An Error-type-aware Data Synthesis module that auto-generates adversarial examples to harden robustness; and (3) A Key-attribute-enhanced Knowledge Distillation module that injects domain-specific attribute hierarchies into a deployable student model. ADORE automates annotation, adversarial generation, and distillation, overcoming data scarcity while enhancing reasoning. Large-scale experiments and online A/B testing verify the effectiveness of ADORE. The framework establishes a new paradigm for resource-efficient, cognitively aligned relevance modeling in industrial applications.
• 摘要：电子商务搜索中的相关性建模仍然受到术语匹配方法（例如 BM25）中的语义差距以及神经模型对特定领域硬样本稀缺性的依赖的挑战。我们提出了 ADORE，一个自我维持的框架，它协同了三项创新：(1) 规则感知相关性判别模块，其中思想链法学硕士生成意图一致的训练数据，并通过 Kahneman-Tversky Optimization (KTO) 进行细化以与用户行为保持一致； (2) 错误类型感知数据合成模块，可自动生成对抗性示例以增强鲁棒性； (3) 关键属性增强型知识蒸馏模块，将特定领域的属性层次结构注入可部署的学生模型中。 ADORE 自动化注释、对抗生成和蒸馏，克服数据稀缺，同时增强推理。大规模实验和在线A/B测试验证了ADORE的有效性。该框架为工业应用中资源高效、认知一致的相关性建模建立了新的范例。

Title: DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models

• Authors: DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Erhang Li, Fangqi Zhou, Fangyun Lin, Fucong Dai, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Li, Haofen Liang, Haoran Wei, Haowei Zhang, Haowen Luo, Haozhe Ji, Honghui Ding, Hongxuan Tang, Huanqi Cao, Huazuo Gao, Hui Qu, Hui Zeng, Jialiang Huang, Jiashi Li, Jiaxin Xu, Jiewen Hu, Jingchang Chen, Jingting Xiang, Jingyang Yuan, Jingyuan Cheng, Jinhua Zhu, Jun Ran, Junguang Jiang, Junjie Qiu, Junlong Li, Junxiao Song, Kai Dong, Kaige Gao, Kang Guan, Kexin Huang, Kexing Zhou, Kezhao Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Wang, Liang Zhao, Liangsheng Yin, Lihua Guo, Lingxiao Luo, Linwang Ma, Litong Wang, Liyue Zhang, M.S. Di, M.Y Xu, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Panpan Huang, Peixin Cong, Peiyi Wang, Qiancheng Wang, Qihao Zhu, Qingyang Li, Qinyu Chen, Qiushi Du, Ruiling Xu, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, Runqiu Yin, Runxin Xu, Ruomeng Shen, Ruoyu Zhang, S.H. Liu, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaofei Cai
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02556
• Pdf URL: https://arxiv.org/pdf/2512.02556
• Copy Paste: [[2512.02556]] DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models(https://arxiv.org/abs/2512.02556)
• Keywords: language model, gpt, agent
• Abstract: We introduce DeepSeek-V3.2, a model that harmonizes high computational efficiency with superior reasoning and agent performance. The key technical breakthroughs of DeepSeek-V3.2 are as follows: (1) DeepSeek Sparse Attention (DSA): We introduce DSA, an efficient attention mechanism that substantially reduces computational complexity while preserving model performance in long-context scenarios. (2) Scalable Reinforcement Learning Framework: By implementing a robust reinforcement learning protocol and scaling post-training compute, DeepSeek-V3.2 performs comparably to GPT-5. Notably, our high-compute variant, DeepSeek-V3.2-Speciale, surpasses GPT-5 and exhibits reasoning proficiency on par with Gemini-3.0-Pro, achieving gold-medal performance in both the 2025 International Mathematical Olympiad (IMO) and the International Olympiad in Informatics (IOI). (3) Large-Scale Agentic Task Synthesis Pipeline: To integrate reasoning into tool-use scenarios, we developed a novel synthesis pipeline that systematically generates training data at scale. This methodology facilitates scalable agentic post-training, yielding substantial improvements in generalization and instruction-following robustness within complex, interactive environments.
• 摘要：我们推出 DeepSeek-V3.2，该模型将高计算效率与卓越的推理和代理性能相结合。 DeepSeek-V3.2的关键技术突破如下：（1）DeepSeek稀疏注意力（DSA）：我们引入了DSA，一种高效的注意力机制，可以大大降低计算复杂度，同时在长上下文场景中保持模型性能。 (2) 可扩展的强化学习框架：通过实施强大的强化学习协议和扩展训练后计算，DeepSeek-V3.2 的性能与 GPT-5 相当。值得注意的是，我们的高计算变体 DeepSeek-V3.2-Speciale 超越了 GPT-5，并表现出与 Gemini-3.0-Pro 相当的推理能力，在 2025 年国际数学奥林匹克（IMO）和国际信息学奥林匹克（IOI）中均获得金牌。 (3) 大规模代理任务合成管道：为了将推理集成到工具使用场景中，我们开发了一种新颖的合成管道，可以系统地大规模生成训练数据。这种方法有利于可扩展的代理后训练，在复杂的交互式环境中显着提高泛化性和指令遵循的鲁棒性。

Title: From Imitation to Discrimination: Toward A Generalized Curriculum Advantage Mechanism Enhancing Cross-Domain Reasoning Tasks

• Authors: Changpeng Yang, Jinyang Wu, Yuchen Liu, Shuai Zhang, Yang Li, Qiliang Liang, Hongzhen Wang, Shuai Nie, Jiaming Xu, Runyu Shi, Ying Huang, Guoquan Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02580
• Pdf URL: https://arxiv.org/pdf/2512.02580
• Copy Paste: [[2512.02580]] From Imitation to Discrimination: Toward A Generalized Curriculum Advantage Mechanism Enhancing Cross-Domain Reasoning Tasks(https://arxiv.org/abs/2512.02580)
• Keywords: language model
• Abstract: Reinforcement learning has emerged as a paradigm for post-training large language models, boosting their reasoning capabilities. Such approaches compute an advantage value for each sample, reflecting better or worse performance than expected, thereby yielding both positive and negative signals for training. However, the indiscriminate mixing of the two signals in existing methods, especially from the early stages, may lead to ambiguous guidance and limited gains. To address this issue, we propose **CAPO** (**C**urriculum **A**dvantage **P**olicy **O**ptimization), an adaptive curriculum mechanism based on advantage signals. The proposed mechanism bootstraps imitation learning with positive-only advantage samples to establish robust foundations, and subsequently introduces negative signals to cultivate discriminative capabilities, thereby improving generalization across complex scenarios. Compatible with diverse optimization methods including GRPO, PPO, RLOO, and Reinforce++, our method consistently achieves stable and significant improvements in mathematical reasoning tasks, and further generalizes effectively to multimodal Graphical User Interface (GUI) reasoning scenarios, establishing itself as a versatile and robust optimization framework.
• 摘要：强化学习已成为大型语言模型后训练的范例，可提高其推理能力。这种方法计算每个样本的优势值，反映比预期更好或更差的性能，从而产生用于训练的正信号和负信号。然而，现有方法中两种信号的不加区别的混合，尤其是在早期阶段，可能会导致指导不明确和增益有限。为了解决这个问题，我们提出了**CAPO**（**C**urriculum **A**dvantage **P**olicy **O**optimization），一种基于优势信号的自适应课程机制。所提出的机制利用仅积极的优势样本引导模仿学习，以建立稳健的基础，随后引入消极信号来培养判别能力，从而提高复杂场景的泛化能力。我们的方法兼容GRPO、PPO、RLOO和Reinforce++等多种优化方法，在数学推理任务中持续实现稳定和显着的改进，并进一步有效地推广到多模态图形用户界面（GUI）推理场景，成为一个通用且鲁棒的优化框架。

Title: Spoken Conversational Agents with Large Language Models

• Authors: Chao-Han Huck Yang, Andreas Stolcke, Larry Heck
• Subjects: cs.CL, cs.MA, cs.NE, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2512.02593
• Pdf URL: https://arxiv.org/pdf/2512.02593
• Copy Paste: [[2512.02593]] Spoken Conversational Agents with Large Language Models(https://arxiv.org/abs/2512.02593)
• Keywords: language model, llm, agent
• Abstract: Spoken conversational agents are converging toward voice-native LLMs. This tutorial distills the path from cascaded ASR/NLU to end-to-end, retrieval-and vision-grounded systems. We frame adaptation of text LLMs to audio, cross-modal alignment, and joint speech-text training; review datasets, metrics, and robustness across accents and compare design choices (cascaded vs. E2E, post-ASR correction, streaming). We link industrial assistants to current open-domain and task-oriented agents, highlight reproducible baselines, and outline open problems in privacy, safety, and evaluation. Attendees leave with practical recipes and a clear systems-level roadmap.
• 摘要：口语会话代理正在向以语音为母语的法学硕士方向发展。本教程提炼了从级联 ASR/NLU 到端到端、基于检索和视觉的系统的路径。我们构建了文本法学硕士对音频的适应、跨模式对齐和联合语音文本训练；查看数据集、指标和跨口音的稳健性，并比较设计选择（级联与 E2E、ASR 后校正、流式传输）。我们将工业助理与当前的开放领域和面向任务的代理联系起来，突出可重复的基线，并概述隐私、安全和评估方面的开放问题。与会者离开时会获得实用的食谱和清晰的系统级路线图。

Title: Input Order Shapes LLM Semantic Alignment in Multi-Document Summarization

• Authors: Jing Ma
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02665
• Pdf URL: https://arxiv.org/pdf/2512.02665
• Copy Paste: [[2512.02665]] Input Order Shapes LLM Semantic Alignment in Multi-Document Summarization(https://arxiv.org/abs/2512.02665)
• Keywords: language model, llm, prompt, agent
• Abstract: Large language models (LLMs) are now used in settings such as Google's AI Overviews, where it summarizes multiple long documents. However, it remains unclear whether they weight all inputs equally. Focusing on abortion-related news, we construct 40 pro-neutral-con article triplets, permute each triplet into six input orders, and prompt Gemini 2.5 Flash to generate a neutral overview. We evaluate each summary against its source articles using ROUGE-L (lexical overlap), BERTScore (semantic similarity), and SummaC (factual consistency). One-way ANOVA reveals a significant primacy effect for BERTScore across all stances, indicating that summaries are more semantically aligned with the first-seen article. Pairwise comparisons further show that Position 1 differs significantly from Positions 2 and 3, while the latter two do not differ from each other, confirming a selective preference for the first document. The findings present risks for applications that rely on LLM-generated overviews and for agentic AI systems, where the steps involving LLMs can disproportionately influence downstream actions.
• 摘要：大型语言模型 (LLM) 现在用于 Google 的 AI Overviews 等设置，它总结了多个长文档。然而，目前尚不清楚它们是否对所有输入进行同等加权。专注于堕胎相关新闻，我们构建了 40 个支持中立-反对的文章三元组，将每个三元组排列成六个输入顺序，并提示 Gemini 2.5 Flash 生成中立概述。我们使用 ROUGE-L（词汇重叠）、BERTScore（语义相似性）和 SummaC（事实一致性）根据源文章评估每个摘要。单向方差分析揭示了 BERTScore 在所有立场上的显着首要效应，表明摘要在语义上与第一次看到的文章更加一致。成对比较进一步表明，位置 1 与位置 2 和 3 显着不同，而后两者彼此没有差异，证实了对第一个文档的选择性偏好。这些发现给依赖法学硕士生成的概述的应用程序和代理人工智能系统带来了风险，其中涉及法学硕士的步骤可能会对下游行动产生不成比例的影响。

Title: An Empirical Survey of Model Merging Algorithms for Social Bias Mitigation

• Authors: Daiki Shirafuji, Tatsuhiko Saito, Yasutomo Kimura
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.02689
• Pdf URL: https://arxiv.org/pdf/2512.02689
• Copy Paste: [[2512.02689]] An Empirical Survey of Model Merging Algorithms for Social Bias Mitigation(https://arxiv.org/abs/2512.02689)
• Keywords: language model, gpt, llm
• Abstract: Large language models (LLMs) are known to inherit and even amplify societal biases present in their pre-training corpora, threatening fairness and social trust. To address this issue, recent work has explored ``editing'' LLM parameters to mitigate social bias with model merging approaches; however, there is no empirical comparison. In this work, we empirically survey seven algorithms: Linear, Karcher Mean, SLERP, NuSLERP, TIES, DELLA, and Nearswap, applying 13 open weight models in the GPT, LLaMA, and Qwen families. We perform a comprehensive evaluation using three bias datasets (BBQ, BOLD, and HONEST) and measure the impact of these techniques on LLM performance in downstream tasks of the SuperGLUE benchmark. We find a trade-off between bias reduction and downstream performance: methods achieving greater bias mitigation degrade accuracy, particularly on tasks requiring reading comprehension and commonsense and causal reasoning. Among the merging algorithms, Linear, SLERP, and Nearswap consistently reduce bias while maintaining overall performance, with SLERP at moderate interpolation weights emerging as the most balanced choice. These results highlight the potential of model merging algorithms for bias mitigation, while indicating that excessive debiasing or inappropriate merging methods may lead to the degradation of important linguistic abilities.
• 摘要：众所周知，大型语言模型（LLM）会继承甚至放大其预训练语料库中存在的社会偏见，威胁公平和社会信任。为了解决这个问题，最近的工作探索了“编辑”LLM参数，以通过模型合并方法减轻社会偏见；然而，没有实证比较。在这项工作中，我们实证研究了七种算法：Linear、Karcher Mean、SLERP、NuSLERP、TIES、DELLA 和 Nearswap，应用了 GPT、LLaMA 和 Qwen 系列中的 13 个开放权重模型。我们使用三个偏差数据集（BBQ、BOLD 和 HONEST）进行综合评估，并衡量这些技术对 SuperGLUE 基准下游任务中 LLM 性能的影响。我们发现偏差减少和下游性能之间存在权衡：实现更大偏差缓解的方法会降低准确性，特别是在需要阅读理解、常识和因果推理的任务上。在合并算法中，Linear、SLERP 和 Nearswap 始终如一地减少偏差，同时保持整体性能，其中中等插值权重的 SLERP 成为最平衡的选择。这些结果凸显了模型合并算法在减轻偏差方面的潜力，同时表明过度的去偏差或不适当的合并方法可能会导致重要语言能力的退化。

Title: CREST: Universal Safety Guardrails Through Cluster-Guided Cross-Lingual Transfer

• Authors: Lavish Bansal, Naman Mishra
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.02711
• Pdf URL: https://arxiv.org/pdf/2512.02711
• Copy Paste: [[2512.02711]] CREST: Universal Safety Guardrails Through Cluster-Guided Cross-Lingual Transfer(https://arxiv.org/abs/2512.02711)
• Keywords: language model, llm
• Abstract: Ensuring content safety in large language models (LLMs) is essential for their deployment in real-world applications. However, existing safety guardrails are predominantly tailored for high-resource languages, leaving a significant portion of the world's population underrepresented who communicate in low-resource languages. To address this, we introduce CREST (CRoss-lingual Efficient Safety Transfer), a parameter-efficient multilingual safety classification model that supports 100 languages with only 0.5B parameters. By training on a strategically chosen subset of only 13 high-resource languages, our model utilizes cluster-based cross-lingual transfer from a few to 100 languages, enabling effective generalization to both unseen high-resource and low-resource languages. This approach addresses the challenge of limited training data in low-resource settings. We conduct comprehensive evaluations across six safety benchmarks to demonstrate that CREST outperforms existing state-of-the-art guardrails of comparable scale and achieves competitive results against models with significantly larger parameter counts (2.5B parameters and above). Our findings highlight the limitations of language-specific guardrails and underscore the importance of developing universal, language-agnostic safety systems that can scale effectively to serve global populations.
• 摘要：确保大型语言模型 (LLM) 中的内容安全对于将其部署到实际应用程序中至关重要。然而，现有的安全护栏主要是针对高资源语言量身定制的，导致世界上使用低资源语言进行交流的人口中有很大一部分人数不足。为了解决这个问题，我们引入了 CREST（CRoss-lingual Efficient Safety Transfer），这是一种参数高效的多语言安全分类模型，仅用 0.5B 个参数即可支持 100 种语言。通过对仅 13 种高资源语言的战略选择子集进行训练，我们的模型利用了从几种到 100 种语言的基于集群的跨语言迁移，从而能够有效泛化到看不见的高资源和低资源语言。这种方法解决了资源匮乏环境中训练数据有限的挑战。我们对六个安全基准进行了全面评估，以证明 CREST 的性能优于同等规模的现有最先进护栏，并与参数数量显着增加（2.5B 参数及以上）的模型相比，取得了具有竞争力的结果。我们的研究结果强调了特定语言护栏的局限性，并强调了开发通用的、与语言无关的安全系统的重要性，该系统可以有效扩展以服务全球人口。

Title: Emergent Bayesian Behaviour and Optimal Cue Combination in LLMs

• Authors: Julian Ma, Jun Wang, Zafeirios Fountas
• Subjects: cs.CL, cs.AI, cs.CV, cs.LG, q-bio.NC
• Abstract URL: https://arxiv.org/abs/2512.02719
• Pdf URL: https://arxiv.org/pdf/2512.02719
• Copy Paste: [[2512.02719]] Emergent Bayesian Behaviour and Optimal Cue Combination in LLMs(https://arxiv.org/abs/2512.02719)
• Keywords: language model, gpt, llm, prompt
• Abstract: Large language models (LLMs) excel at explicit reasoning, but their implicit computational strategies remain underexplored. Decades of psychophysics research show that humans intuitively process and integrate noisy signals using near-optimal Bayesian strategies in perceptual tasks. We ask whether LLMs exhibit similar behaviour and perform optimal multimodal integration without explicit training or instruction. Adopting the psychophysics paradigm, we infer computational principles of LLMs from systematic behavioural studies. We introduce a behavioural benchmark - BayesBench: four magnitude estimation tasks (length, location, distance, and duration) over text and image, inspired by classic psychophysics, and evaluate a diverse set of nine LLMs alongside human judgments for calibration. Through controlled ablations of noise, context, and instruction prompts, we measure performance, behaviour and efficiency in multimodal cue-combination. Beyond accuracy and efficiency metrics, we introduce a Bayesian Consistency Score that detects Bayes-consistent behavioural shifts even when accuracy saturates. Our results show that while capable models often adapt in Bayes-consistent ways, accuracy does not guarantee robustness. Notably, GPT-5 Mini achieves perfect text accuracy but fails to integrate visual cues efficiently. This reveals a critical dissociation between capability and strategy, suggesting accuracy-centric benchmarks may over-index on performance while missing brittle uncertainty handling. These findings reveal emergent principled handling of uncertainty and highlight the correlation between accuracy and Bayesian tendencies. We release our psychophysics benchmark and consistency metric (this https URL) as evaluation tools and to inform future multimodal architecture designs.
• 摘要：大型语言模型（LLM）擅长显式推理，但其隐式计算策略仍未得到充分探索。数十年的心理物理学研究表明，人类在感知任务中使用近乎最优的贝叶斯策略直观地处理和整合噪声信号。我们询问法学硕士是否表现出类似的行为，并在没有明确培训或指导的情况下执行最佳的多模式整合。我们采用心理物理学范式，从系统的行为研究中推断出法学硕士的计算原理。我们引入了一个行为基准 - BayesBench：受经典心理物理学的启发，对文本和图像进行四种幅度估计任务（长度、位置、距离和持续时间），并评估九个 LLM 的不同集合以及人类的校准判断。通过控制噪声、上下文和指令提示的消融，我们测量多模式提示组合中的表现、行为和效率。除了准确性和效率指标之外，我们还引入了贝叶斯一致性评分，即使在准确性饱和时也能检测贝叶斯一致的行为变化。我们的结果表明，虽然有能力的模型通常以贝叶斯一致的方式进行调整，但准确性并不能保证鲁棒性。值得注意的是，GPT-5 Mini 实现了完美的文本准确性，但未能有效地整合视觉提示。这揭示了能力和策略之间的严重脱节，表明以准确性为中心的基准可能会过度衡量性能，同时缺少脆弱的不确定性处理。这些发现揭示了对不确定性的新兴原则性处理，并强调了准确性和贝叶斯倾向之间的相关性。我们发布心理物理学基准和一致性指标（此 https URL）作为评估工具，并为未来的多模式架构设计提供信息。

Title: SurveyEval: Towards Comprehensive Evaluation of LLM-Generated Academic Surveys

• Authors: Jiahao Zhao, Shuaixing Zhang, Nan Xu, Lei Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.02763
• Pdf URL: https://arxiv.org/pdf/2512.02763
• Copy Paste: [[2512.02763]] SurveyEval: Towards Comprehensive Evaluation of LLM-Generated Academic Surveys(https://arxiv.org/abs/2512.02763)
• Keywords: llm
• Abstract: LLM-based automatic survey systems are transforming how users acquire information from the web by integrating retrieval, organization, and content synthesis into end-to-end generation pipelines. While recent works focus on developing new generation pipelines, how to evaluate such complex systems remains a significant challenge. To this end, we introduce SurveyEval, a comprehensive benchmark that evaluates automatically generated surveys across three dimensions: overall quality, outline coherence, and reference accuracy. We extend the evaluation across 7 subjects and augment the LLM-as-a-Judge framework with human references to strengthen evaluation-human alignment. Evaluation results show that while general long-text or paper-writing systems tend to produce lower-quality surveys, specialized survey-generation systems are able to deliver substantially higher-quality results. We envision SurveyEval as a scalable testbed to understand and improve automatic survey systems across diverse subjects and evaluation criteria.
• 摘要：基于法学硕士的自动调查系统通过将检索、组织和内容合成集成到端到端生成管道中，正在改变用户从网络获取信息的方式。虽然最近的工作重点是开发新一代管道，但如何评估此类复杂的系统仍然是一个重大挑战。为此，我们推出了 SurveyEval，这是一个综合基准，可从三个维度评估自动生成的调查：整体质量、大纲一致性和参考准确性。我们将评估扩展到 7 个科目，并通过人类参考增强了法学硕士法官框架，以加强评估与人类的一致性。评估结果表明，虽然一般的长文本或论文写作系统往往会产生质量较低的调查，但专门的调查生成系统能够提供质量更高的结果。我们将 SurveyEval 设想为一个可扩展的测试平台，用于理解和改进跨不同主题和评估标准的自动调查系统。

Title: PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models

• Authors: Robert Belanec, Ivan Srba, Maria Bielikova
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02764
• Pdf URL: https://arxiv.org/pdf/2512.02764
• Copy Paste: [[2512.02764]] PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models(https://arxiv.org/abs/2512.02764)
• Keywords: language model, llm
• Abstract: Parameter-Efficient Fine-Tuning (PEFT) methods address the increasing size of Large Language Models (LLMs). Currently, many newly introduced PEFT methods are challenging to replicate, deploy, or compare with one another. To address this, we introduce PEFT-Factory, a unified framework for efficient fine-tuning LLMs using both off-the-shelf and custom PEFT methods. While its modular design supports extensibility, it natively provides a representative set of 19 PEFT methods, 27 classification and text generation datasets addressing 12 tasks, and both standard and PEFT-specific evaluation metrics. As a result, PEFT-Factory provides a ready-to-use, controlled, and stable environment, improving replicability and benchmarking of PEFT methods. PEFT-Factory is a downstream framework that originates from the popular LLaMA-Factory, and is publicly available at this https URL
• 摘要：参数高效微调 (PEFT) 方法可解决大型语言模型 (LLM) 规模不断增大的问题。目前，许多新引入的 PEFT 方法难以复制、部署或相互比较。为了解决这个问题，我们引入了 PEFT-Factory，这是一个统一的框架，可以使用现成的和自定义的 PEFT 方法来高效微调 LLM。虽然其模块化设计支持可扩展性，但它本身提供了一组具有代表性的 19 种 PEFT 方法、解决 12 项任务的 27 个分类和文本生成数据集，以及标准和特定于 PEFT 的评估指标。因此，PEFT-Factory 提供了一个即用型、受控且稳定的环境，提高了 PEFT 方法的可复制性和基准测试。 PEFT-Factory 是一个下游框架，源自流行的 LLaMA-Factory，可通过此 https URL 公开获取

Title: Towards Unification of Hallucination Detection and Fact Verification for Large Language Models

• Authors: Weihang Su, Jianming Long, Changyue Wang, Shiyu Lin, Jingyan Xu, Ziyi Ye, Qingyao Ai, Yiqun Liu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2512.02772
• Pdf URL: https://arxiv.org/pdf/2512.02772
• Copy Paste: [[2512.02772]] Towards Unification of Hallucination Detection and Fact Verification for Large Language Models(https://arxiv.org/abs/2512.02772)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) frequently exhibit hallucinations, generating content that appears fluent and coherent but is factually incorrect. Such errors undermine trust and hinder their adoption in real-world applications. To address this challenge, two distinct research paradigms have emerged: model-centric Hallucination Detection (HD) and text-centric Fact Verification (FV). Despite sharing the same goal, these paradigms have evolved in isolation, using distinct assumptions, datasets, and evaluation protocols. This separation has created a research schism that hinders their collective progress. In this work, we take a decisive step toward bridging this divide. We introduce UniFact, a unified evaluation framework that enables direct, instance-level comparison between FV and HD by dynamically generating model outputs and corresponding factuality labels. Through large-scale experiments across multiple LLM families and detection methods, we reveal three key findings: (1) No paradigm is universally superior; (2) HD and FV capture complementary facets of factual errors; and (3) hybrid approaches that integrate both methods consistently achieve state-of-the-art performance. Beyond benchmarking, we provide the first in-depth analysis of why FV and HD diverged, as well as empirical evidence supporting the need for their unification. The comprehensive experimental results call for a new, integrated research agenda toward unifying Hallucination Detection and Fact Verification in LLMs. We have open-sourced all the code, data, and baseline implementation at: this https URL
• 摘要：大型语言模型 (LLM) 经常表现出幻觉，生成看似流畅、连贯但实际上不正确的内容。此类错误会破坏信任并阻碍其在实际应用中的采用。为了应对这一挑战，出现了两种不同的研究范式：以模型为中心的幻觉检测（HD）和以文本为中心的事实验证（FV）。尽管有着相同的目标，但这些范式是使用不同的假设、数据集和评估协议独立发展的。这种分离造成了研究分裂，阻碍了他们的集体进步。在这项工作中，我们朝着弥合这一鸿沟迈出了决定性的一步。我们推出了 UniFact，这是一个统一的评估框架，通过动态生成模型输出和相应的事实标签，可以在 FV 和 HD 之间进行直接的实例级比较。通过跨多个法学硕士系列和检测方法的大规模实验，我们揭示了三个关键发现：（1）没有任何范式是普遍优越的； (2) HD 和 FV 捕捉事实错误的互补方面； (3) 集成两种方法的混合方法始终能够实现最先进的性能。除了基准测试之外，我们还首次深入分析了 FV 和 HD 差异的原因，以及支持它们统一的必要性的经验证据。全面的实验结果需要一个新的、综合的研究议程，以统一法学硕士中的幻觉检测和事实验证。我们已在以下位置开源了所有代码、数据和基线实现：此 https URL

Title: TriLex: A Framework for Multilingual Sentiment Analysis in Low-Resource South African Languages

• Authors: Mike Nkongolo, Hilton Vorster, Josh Warren, Trevor Naick, Deandre Vanmali, Masana Mashapha, Luke Brand, Alyssa Fernandes, Janco Calitz, Sibusiso Makhoba
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02799
• Pdf URL: https://arxiv.org/pdf/2512.02799
• Copy Paste: [[2512.02799]] TriLex: A Framework for Multilingual Sentiment Analysis in Low-Resource South African Languages(https://arxiv.org/abs/2512.02799)
• Keywords: language model, retrieval augmented generation
• Abstract: Low-resource African languages remain underrepresented in sentiment analysis, limiting both lexical coverage and the performance of multilingual Natural Language Processing (NLP) systems. This study proposes TriLex, a three-stage retrieval augmented framework that unifies corpus-based extraction, cross lingual mapping, and retrieval augmented generation (RAG) driven lexical refinement to systematically expand sentiment lexicons for low-resource languages. Using the enriched lexicon, the performance of two prominent African pretrained language models (AfroXLMR and AfriBERTa) is evaluated across multiple case studies. Results demonstrate that AfroXLMR delivers superior performance, achieving F1-scores above 80% for isiXhosa and isiZulu and exhibiting strong cross-lingual stability. Although AfriBERTa lacks pre-training on these target languages, it still achieves reliable F1-scores around 64%, validating its utility in computationally constrained settings. Both models outperform traditional machine learning baselines, and ensemble analyses further enhance precision and robustness. The findings establish TriLex as a scalable and effective framework for multilingual sentiment lexicon expansion and sentiment modeling in low-resource South African languages.
• 摘要：资源匮乏的非洲语言在情感分析中的代表性仍然不足，限制了词汇覆盖范围和多语言自然语言处理 (NLP) 系统的性能。本研究提出了 TriLex，这是一个三阶段检索增强框架，它统一了基于语料库的提取、跨语言映射和检索增强生成 (RAG) 驱动的词汇细化，以系统地扩展低资源语言的情感词典。使用丰富的词典，在多个案例研究中评估了两种著名的非洲预训练语言模型（AfroXLMR 和 AfriBERTa）的性能。结果表明，AfroXLMR 具有卓越的性能，isiXhosa 和 isiZulu 的 F1 分数超过 80%，并表现出强大的跨语言稳定性。尽管 AfriBERTa 缺乏对这些目标语言的预训练，但它仍然获得了 64% 左右的可靠 F1 分数，验证了其在计算受限设置中的实用性。这两种模型都优于传统的机器学习基线，并且集成分析进一步提高了精度和鲁棒性。研究结果将 TriLex 确立为一个可扩展且有效的框架，用于资源匮乏的南非语言中的多语言情感词典扩展和情感建模。

Title: SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment

• Authors: Yixuan Tang, Yi Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02807
• Pdf URL: https://arxiv.org/pdf/2512.02807
• Copy Paste: [[2512.02807]] SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment(https://arxiv.org/abs/2512.02807)
• Keywords: language model, llm, prompt
• Abstract: Aligning Large Language Models (LLMs) with human preferences typically relies on external supervision, which faces critical limitations: human annotations are scarce and subjective, reward models are vulnerable to reward hacking, and self-evaluation methods suffer from prompt sensitivity and biases. In this work, we propose stable rank, an intrinsic, annotation-free quality signal derived from model representations. Stable rank measures the effective dimensionality of hidden states by computing the ratio of total variance to dominant-direction variance, capturing quality through how information distributes across representation dimensions. Empirically, stable rank achieves 84.04% accuracy on RewardBench and improves task accuracy by an average of 11.3 percentage points over greedy decoding via Best-of-N sampling. Leveraging this insight, we introduce Stable Rank Group Relative Policy Optimization (SR-GRPO), which uses stable rank as a reward signal for reinforcement learning. Without external supervision, SR-GRPO improves Qwen2.5-1.5B-Instruct by 10% on STEM and 19% on mathematical reasoning, outperforming both learned reward models and self-evaluation baselines. Our findings demonstrate that quality signals can be extracted from internal model geometry, offering a path toward scalable alignment without external supervision.
• 摘要：使大型语言模型（LLM）与人类偏好保持一致通常依赖于外部监督，这面临着严重的局限性：人类注释稀缺且主观，奖励模型容易受到奖励黑客攻击，自我评估方法容易出现敏感性和偏见。在这项工作中，我们提出了稳定的排名，这是一种从模型表示中得出的内在的、无注释的质量信号。稳定排名通过计算总方差与主导方向方差的比率来测量隐藏状态的有效维度，通过信息如何在表示维度上分布来捕获质量。根据经验，稳定排序在 RewardBench 上的准确率达到 84.04%，并且通过 Best-of-N 采样比贪婪解码平均提高了 11.3 个百分点的任务准确率。利用这一见解，我们引入了稳定排名组相对策略优化（SR-GRPO），它使用稳定排名作为强化学习的奖励信号。在没有外部监督的情况下，SR-GRPO 将 Qwen2.5-1.5B-Instruct 在 STEM 方面提高了 10%，在数学推理方面提高了 19%，优于学习奖励模型和自我评估基线。我们的研究结果表明，可以从内部模型几何形状中提取质量信号，从而提供了一条无需外部监督即可实现可扩展对齐的途径。

Title: A benchmark dataset for evaluating Syndrome Differentiation and Treatment in large language models

• Authors: Kunning Li, Jianbin Guo, Zhaoyang Shang, Yiqing Liu, Hongmin Du, Lingling Liu, Yuping Zhao, Lifeng Dong
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02816
• Pdf URL: https://arxiv.org/pdf/2512.02816
• Copy Paste: [[2512.02816]] A benchmark dataset for evaluating Syndrome Differentiation and Treatment in large language models(https://arxiv.org/abs/2512.02816)
• Keywords: language model, llm
• Abstract: The emergence of Large Language Models (LLMs) within the Traditional Chinese Medicine (TCM) domain presents an urgent need to assess their clinical application capabilities. However, such evaluations are challenged by the individualized, holistic, and diverse nature of TCM's "Syndrome Differentiation and Treatment" (SDT). Existing benchmarks are confined to knowledge-based question-answering or the accuracy of syndrome differentiation, often neglecting assessment of treatment decision-making. Here, we propose a comprehensive, clinical case-based benchmark spearheaded by TCM experts, and a specialized reward model employed to quantify prescription-syndrome congruence. Data annotation follows a rigorous pipeline. This benchmark, designated TCM-BEST4SDT, encompasses four tasks, including TCM Basic Knowledge, Medical Ethics, LLM Content Safety, and SDT. The evaluation framework integrates three mechanisms, namely selected-response evaluation, judge model evaluation, and reward model evaluation. The effectiveness of TCM-BEST4SDT was corroborated through experiments on 15 mainstream LLMs, spanning both general and TCM domains. To foster the development of intelligent TCM research, TCM-BEST4SDT is now publicly available.
• 摘要：中医（TCM）领域大语言模型（LLM）的出现迫切需要评估其临床应用能力。然而，这种评估受到中医“辨证论治”（SDT）个体化、整体性和多样性的挑战。现有的基准仅限于基于知识的问答或辨证的准确性，往往忽略了对治疗决策的评估。在这里，我们提出了一个由中医专家牵头的全面的、基于临床病例的基准，以及一个用于量化处方-证候一致性的专门奖励模型。数据注释遵循严格的流程。该基准被命名为TCM-BEST4SDT，涵盖四项任务，包括中医基础知识、医学伦理、法学硕士内容安全和SDT。该评估框架整合了选择响应评估、判断模型评估和奖励模型评估三种机制。 TCM-BEST4SDT 的有效性通过 15 个主流法学硕士（涵盖普通领域和中医领域）的实验得到了证实。为了促进智能中医研究的发展，TCM-BEST4SDT现已公开。

Title: promptolution: A Unified, Modular Framework for Prompt Optimization

• Authors: Tom Zehle, Timo Heiß, Moritz Schlager, Matthias Aßenmacher, Matthias Feurer
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02840
• Pdf URL: https://arxiv.org/pdf/2512.02840
• Copy Paste: [[2512.02840]] promptolution: A Unified, Modular Framework for Prompt Optimization(https://arxiv.org/abs/2512.02840)
• Keywords: language model, llm, prompt
• Abstract: Prompt optimization has become crucial for enhancing the performance of large language models (LLMs) across a broad range of tasks. Although many research papers show its effectiveness, practical adoption is hindered as existing implementations are often tied to unmaintained and isolated research codebases. To address this, we introduce promptolution, a unified and modular open-source framework that provides all components required for prompt optimization within a single extensible system for both practitioners and researchers. It integrates multiple contemporary discrete prompt optimizers while remaining agnostic to the underlying LLM implementation.
• 摘要：及时优化对于提高大型语言模型 (LLM) 在各种任务中的性能至关重要。尽管许多研究论文显示了其有效性，但由于现有的实现通常与未维护和孤立的研究代码库相关，因此实际采用受到阻碍。为了解决这个问题，我们引入了prompolution，这是一个统一的模块化开源框架，它为从业者和研究人员提供了在单个可扩展系统中进行即时优化所需的所有组件。它集成了多个当代离散提示优化器，同时保持与底层 LLM 实现无关。

Title: Cross-Lingual Prompt Steerability: Towards Accurate and Robust LLM Behavior across Languages

• Authors: Lechen Zhang, Yusheng Zhou, Tolga Ergen, Lajanugen Logeswaran, Moontae Lee, David Jurgens
• Subjects: cs.CL, cs.AI, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.02841
• Pdf URL: https://arxiv.org/pdf/2512.02841
• Copy Paste: [[2512.02841]] Cross-Lingual Prompt Steerability: Towards Accurate and Robust LLM Behavior across Languages(https://arxiv.org/abs/2512.02841)
• Keywords: language model, llm, prompt
• Abstract: System prompts provide a lightweight yet powerful mechanism for conditioning large language models (LLMs) at inference time. While prior work has focused on English-only settings, real-world deployments benefit from having a single prompt to operate reliably across languages. This paper presents a comprehensive study of how different system prompts steer models toward accurate and robust cross-lingual behavior. We propose a unified four-dimensional evaluation framework to assess system prompts in multilingual environments. Through large-scale experiments on five languages, three LLMs, and three benchmarks, we uncover that certain prompt components, such as CoT, emotion, and scenario, correlate with robust multilingual behavior. We develop a prompt optimization framework for multilingual settings and show it can automatically discover prompts that improve all metrics by 5-10%. Finally, we analyze over 10 million reasoning units and find that more performant system prompts induce more structured and consistent reasoning patterns, while reducing unnecessary language-switching. Together, we highlight system prompt optimization as a scalable path to accurate and robust multilingual LLM behavior.
• 摘要：系统提示提供了一种轻量级但功能强大的机制，用于在推理时调节大型语言模型 (LLM)。虽然之前的工作主要集中在纯英语设置上，但现实世界的部署受益于跨语言可靠操作的单一提示。本文对不同系统如何促使模型转向准确和鲁棒的跨语言行为进行了全面研究。我们提出了一个统一的四维评估框架来评估多语言环境中的系统提示。通过对五种语言、三个法学硕士和三个基准的大规模实验，我们发现某些提示组件（例如 CoT、情感和场景）与稳健的多语言行为相关。我们为多语言设置开发了一个提示优化框架，并证明它可以自动发现提示，从而将所有指标提高 5-10%。最后，我们分析了超过 1000 万个推理单元，发现性能更高的系统提示会引发更加结构化和一致的推理模式，同时减少不必要的语言切换。我们共同强调系统提示优化是实现准确、稳健的多语言 LLM 行为的可扩展路径。

Title: Bangla Hate Speech Classification with Fine-tuned Transformer Models

• Authors: Yalda Keivan Jafari, Krishno Dey
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02845
• Pdf URL: https://arxiv.org/pdf/2512.02845
• Copy Paste: [[2512.02845]] Bangla Hate Speech Classification with Fine-tuned Transformer Models(https://arxiv.org/abs/2512.02845)
• Keywords: language model
• Abstract: Hate speech recognition in low-resource lan- guages remains a difficult problem due to in- sufficient datasets, orthographic heterogeneity, and linguistic variety. Bangla is spoken by more than 230 million people of Bangladesh and India (West Bengal). Despite the grow- ing need for automated moderation on social media platforms, Bangla is significantly under- represented in computational resources. In this work, we study Subtask 1A and Subtask 1B of the BLP 2025 Shared Task on hate speech detection. We reproduce the official base- lines (e.g., Majority, Random, Support Vec- tor Machine) and also produce and consider Logistic Regression, Random Forest, and De- cision Tree as baseline methods. We also uti- lized transformer-based models such as Dis- tilBERT, BanglaBERT, m-BERT, and XLM- RoBERTa for hate speech classification. All the transformer-based models outperformed base- line methods for the subtasks, except for Distil- BERT. Among the transformer-based models, BanglaBERT produces the best performance for both subtasks. Despite being smaller in size, BanglaBERT outperforms both m-BERT and XLM-RoBERTa, which suggests language- specific pre-training is very important. Our results highlight the potential and need for pre- trained language models for the low-resource Bangla language.
• 摘要：由于数据集不足、拼写异质性和语言多样性，低资源语言中的仇恨语音识别仍然是一个难题。孟加拉国和印度（西孟加拉邦）有超过 2.3 亿人口使用孟加拉语。尽管社交媒体平台上对自动审核的需求不断增长，但孟加拉语的计算资源代表性明显不足。在这项工作中，我们研究了 BLP 2025 仇恨言论检测共享任务的子任务 1A 和子任务 1B。我们重现了官方基线（例如，多数、随机、支持向量机），并且还生成并考虑逻辑回归、随机森林和决策树作为基线方法。我们还利用基于 Transformer 的模型（例如 DistilBERT、BanglaBERT、m-BERT 和 XLM-RoBERTa）进行仇恨言论分类。除了 DistilBERT 之外，所有基于 Transformer 的模型在子任务上都优于基线方法。在基于 Transformer 的模型中，BanglaBERT 为这两个子任务提供了最佳性能。尽管规模较小，BanglaBERT 的性能优于 m-BERT 和 XLM-RoBERTa，这表明特定语言的预训练非常重要。我们的结果强调了资源匮乏的孟加拉语预训练语言模型的潜力和需求。

Title: Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning

• Authors: Haonan Wang, Chao Du, Kenji Kawaguchi, Tianyu Pang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02874
• Pdf URL: https://arxiv.org/pdf/2512.02874
• Copy Paste: [[2512.02874]] Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning(https://arxiv.org/abs/2512.02874)
• Keywords: llm, agent
• Abstract: Majority voting has proven effective for close-ended question answering by aggregating parallel reasoning traces. However, it is not directly applicable to open-ended reasoning, such as code generation and web-based deep research, where a "majority" over complete solutions is ill-defined. We introduce ThinkMerge, a training-free, plug-and-play decoding strategy that runs K parallel reasoning traces and averages their next-token logits at synchronization points to produce a single coherent output. ThinkMerge integrates seamlessly with vLLM/SGLang and remains compatible with standard decoding techniques such as Top-p/Top-k. Empirically, it matches or surpasses majority voting on AIME and GPQA, while delivering consistent gains on open-ended coding tasks: on LiveCodeBench (hard), pass@1 improves by +8.28% for DeepCoder-14B-Preview and +7.58% for Qwen3-8B. Beyond code, we further show that ThinkMerge improves web-based deep-research agents (e.g., WebSailor-7B/32B) across GAIA, BrowseComp-en/zh, and XbenchDeepSearch. These results demonstrate that parallel test-time scaling can benefit open-ended reasoning without relying on voting over complete outputs.
• 摘要：事实证明，通过聚合并行推理轨迹，多数投票对于封闭式问答是有效的。然而，它并不直接适用于开放式推理，例如代码生成和基于网络的深度研究，其中完整解决方案的“多数”是不明确的。我们引入了 ThinkMerge，这是一种免训练、即插即用的解码策略，它运行 K 个并行推理轨迹，并在同步点对它们的下一个令牌逻辑进行平均，以产生单个一致的输出。 ThinkMerge 与 vLLM/SGLang 无缝集成，并与 Top-p/Top-k 等标准解码技术保持兼容。根据经验，它匹配或超过了 AIME 和 GPQA 的多数投票，同时在开放式编码任务上提供了一致的收益：在 LiveCodeBench（硬）上，pass@1 对于 DeepCoder-14B-Preview 提高了 +8.28%，对于 Qwen3-8B 提高了 +7.58%。除了代码之外，我们还进一步证明 ThinkMerge 改进了 GAIA、BrowseComp-en/zh 和 XbenchDeepSearch 上基于 Web 的深度研究代理（例如 WebSailor-7B/32B）。这些结果表明，并行测试时间扩展可以有益于开放式推理，而无需依赖对完整输出的投票。

Title: Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules

• Authors: Amr Mohamed, Yang Zhang, Michalis Vazirgiannis, Guokan Shang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02892
• Pdf URL: https://arxiv.org/pdf/2512.02892
• Copy Paste: [[2512.02892]] Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules(https://arxiv.org/abs/2512.02892)
• Keywords: language model, llm
• Abstract: Diffusion large language models (dLLMs) offer a promising alternative to autoregressive models, but their practical utility is severely hampered by slow, iterative sampling. We present SchED, a training-free, model-agnostic early-exit algorithm that aggregates full-span logit margins and halts decoding once a smooth, progress-dependent confidence threshold is met. We evaluated SchED on two dLLM families (Dream and LLaDA), in base and instruction-tuned variants across ten benchmarks spanning downstream tasks including multiple-choice question answering (MCQ), math, long-form QA/summarization, and translation. SchED delivers large, stable accelerations: on instruction-tuned models, it achieves $3.8$-$4.0\times$ speedups while retaining $99.8$-$100\%$ of the baseline score on average. On base models, SchED yields consistent speedup gains with $99.1$-$100\%$ performance retention, with up to $2.34\times$ under more aggressive settings. Using a conservative speed metric that heavily penalizes quality loss (QPS, $\gamma{=}4$), we show that SchED is robust and clearly outperforms prior confidence-based early-exit methods, which break down on long-form generation. An entropy analysis of the model's token predictions reveals that instruction tuning speeds up the decay of predictive entropy. By turning genuine confidence stabilization into computational savings, SchED makes dLLM decoding substantially more efficient.
• 摘要：扩散大语言模型 (dLLM) 为自回归模型提供了一种有前途的替代方案，但其实用性因缓慢的迭代采样而受到严重阻碍。我们提出了 SchED，这是一种免训练、与模型无关的提前退出算法，它可以聚合全范围的 logit 边际，并在满足平滑的、依赖于进度的置信阈值时停止解码。我们在两个 dLLM 系列（Dream 和 LLaDA）上评估了 SchED，采用基础和指令调整的变体，跨越十个基准，涵盖多项选择题回答 (MCQ)、数学、长格式 QA/总结和翻译等下游任务。 SchED 提供了大而稳定的加速：在指令调整模型上，它实现了 $3.8$-$4.0\times$ 加速，同时平均保留 $99.8$-$100\%$ 的基线分数。在基本型号上，SchED 能够带来一致的加速增益，性能保持率为 99.1$-$100\%$，在更激进的设置下最高可达 $2.34\times$。使用严重惩罚质量损失的保守速度指标（QPS，$\gamma{=}4$），我们表明 SchED 是稳健的，并且明显优于先前基于置信度的提前退出方法，该方法在长格式生成上崩溃。对模型令牌预测的熵分析表明，指令调整会加速预测熵的衰减。通过将真正的置信稳定性转化为计算节省，SchED 使 dLLM 解码效率大大提高。

Title: AutoNeural: Co-Designing Vision-Language Models for NPU Inference

• Authors: Wei Chen, Liangmin Wu, Yunhai Hu, Zhiyuan Li, Zhiyuan Cheng, Yicheng Qian, Lingyue Zhu, Zhipeng Hu, Luoyi Liang, Qiang Tang, Zhen Liu, Han Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.02924
• Pdf URL: https://arxiv.org/pdf/2512.02924
• Copy Paste: [[2512.02924]] AutoNeural: Co-Designing Vision-Language Models for NPU Inference(https://arxiv.org/abs/2512.02924)
• Keywords: language model
• Abstract: While Neural Processing Units (NPUs) offer high theoretical efficiency for edge AI, state-of-the-art Vision--Language Models (VLMs) tailored for GPUs often falter on these substrates. We attribute this hardware-model mismatch to two primary factors: the quantization brittleness of Vision Transformers (ViTs) and the I/O-bound nature of autoregressive attention mechanisms, which fail to utilize the high arithmetic throughput of NPUs. To bridge this gap, we propose AutoNeural, an NPU-native VLM architecture co-designed for integer-only inference. We replace the standard ViT encoder with a MobileNetV5-style backbone utilizing depthwise separable convolutions, which ensures bounded activation distributions for stable INT4/8/16 quantization. Complementing this, our language backbone integrates State-Space Model (SSM) principles with Transformer layers, employing efficient gated convolutions to achieve linear-time complexity. This hybrid design eliminates the heavy memory I/O overhead of Key-Value caching during generation. Our approach delivers substantial efficiency gains, reducing quantization error of vision encoder by up to 7x and end-to-end latency by 14x compared to conventional baselines. The AutoNeural also delivers 3x decoding speed and 4x longer context window than the baseline. We validate these improvements via a real-world automotive case study on the Qualcomm SA8295P SoC, demonstrating real-time performance for cockpit applications. Our results highlight that rethinking model topology specifically for NPU constraints is a prerequisite for robust multi-modal edge intelligence.
• 摘要：虽然神经处理单元 (NPU) 为边缘人工智能提供了很高的理论效率，但为 GPU 量身定制的最先进的视觉语言模型 (VLM) 在这些基板上通常会出现问题。我们将这种硬件模型不匹配归因于两个主要因素：视觉变换器 (ViT) 的量化脆弱性和自回归注意力机制的 I/O 限制性质，这些机制无法利用 NPU 的高算术吞吐量。为了弥补这一差距，我们提出了 AutoNeural，这是一种 NPU 原生 VLM 架构，专为纯整数推理而设计。我们使用深度可分离卷积将标准 ViT 编码器替换为 MobileNetV5 风格的主干，这确保了稳定的 INT4/8/16 量化的有界激活分布。作为补充，我们的语言主干将状态空间模型（SSM）原理与 Transformer 层集成，采用高效的门控卷积来实现线性时间复杂度。这种混合设计消除了生成期间键值缓存的大量内存 I/O 开销。与传统基线相比，我们的方法可显着提高效率，将视觉编码器的量化误差降低多达 7 倍，将端到端延迟降低 14 倍。 AutoNeural 还提供比基线高 3 倍的解码速度和 4 倍长的上下文窗口。我们通过 Qualcomm SA8295P SoC 的真实汽车案例研究验证了这些改进，展示了驾驶舱应用的实时性能。我们的结果强调，专门针对 NPU 约束重新思考模型拓扑是稳健的多模态边缘智能的先决条件。

Title: Fine-Tuned Large Language Models for Logical Translation: Reducing Hallucinations with Lang2Logic

• Authors: Muyu Pan, Dheeraj Kodakandla, Mahfuza Farooque
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.02987
• Pdf URL: https://arxiv.org/pdf/2512.02987
• Copy Paste: [[2512.02987]] Fine-Tuned Large Language Models for Logical Translation: Reducing Hallucinations with Lang2Logic(https://arxiv.org/abs/2512.02987)
• Keywords: language model, llm, hallucination
• Abstract: Recent advances in natural language processing (NLP), particularly large language models (LLMs), have motivated the automatic translation of natural language statements into formal logic without human intervention. This enables automated reasoning and facilitates debugging, finding loop invariants, and adhering to specifications in software systems. However, hallucinations-incorrect outputs generated by LLMs are challenging, particularly for logical translation tasks requiring precision. This work introduces a novel framework that inputs English sentences, converts them into logical expressions, and then translates them into Conjunctive Normal Form (CNF) for satisfiability solving. It employs classical NLP techniques with self-defined grammar, symbolic computation libraries, and a fine-tuned language model to reduce hallucinations. In the early experiments, we observed that the fine-tuned model, trained on different grammar settings, could intentionally correct the same types of hallucinations made by the original model. Thus, it provides reliable CNF generation.
• 摘要：自然语言处理 (NLP)，特别是大型语言模型 (LLM) 的最新进展，推动了将自然语言语句自动翻译为形式逻辑，无需人工干预。这可以实现自动推理并促进调试、查找循环不变量以及遵守软件系统中的规范。然而，法学硕士生成的幻觉不正确的输出具有挑战性，特别是对于需要精度的逻辑翻译任务。这项工作引入了一种新颖的框架，该框架输入英语句子，将其转换为逻辑表达式，然后将其转换为连接范式（CNF）以进行可满足性求解。它采用经典的自然语言处理技术，具有自定义语法、符号计算库和微调的语言模型，以减少幻觉。在早期的实验中，我们观察到，在不同语法设置上训练的微调模型可以有意纠正原始模型产生的相同类型的幻觉。因此，它提供了可靠的 CNF 生成。

Title: The Moral Consistency Pipeline: Continuous Ethical Evaluation for Large Language Models

• Authors: Saeid Jamshidi, Kawser Wazed Nafi, Arghavan Moradi Dakhel, Negar Shahabi, Foutse Khomh
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.03026
• Pdf URL: https://arxiv.org/pdf/2512.03026
• Copy Paste: [[2512.03026]] The Moral Consistency Pipeline: Continuous Ethical Evaluation for Large Language Models(https://arxiv.org/abs/2512.03026)
• Keywords: language model, gpt, llm
• Abstract: The rapid advancement and adaptability of Large Language Models (LLMs) highlight the need for moral consistency, the capacity to maintain ethically coherent reasoning across varied contexts. Existing alignment frameworks, structured approaches designed to align model behavior with human ethical and social norms, often rely on static datasets and post-hoc evaluations, offering limited insight into how ethical reasoning may evolve across different contexts or temporal scales. This study presents the Moral Consistency Pipeline (MoCoP), a dataset-free, closed-loop framework for continuously evaluating and interpreting the moral stability of LLMs. MoCoP combines three supporting layers: (i) lexical integrity analysis, (ii) semantic risk estimation, and (iii) reasoning-based judgment modeling within a self-sustaining architecture that autonomously generates, evaluates, and refines ethical scenarios without external supervision. Our empirical results on GPT-4-Turbo and DeepSeek suggest that MoCoP effectively captures longitudinal ethical behavior, revealing a strong inverse relationship between ethical and toxicity dimensions (correlation rET = -0.81, p value less than 0.001) and a near-zero association with response latency (correlation rEL approximately equal to 0). These findings demonstrate that moral coherence and linguistic safety tend to emerge as stable and interpretable characteristics of model behavior rather than short-term fluctuations. Furthermore, by reframing ethical evaluation as a dynamic, model-agnostic form of moral introspection, MoCoP offers a reproducible foundation for scalable, continuous auditing and advances the study of computational morality in autonomous AI systems.
• 摘要：大型语言模型（LLM）的快速发展和适应性凸显了道德一致性的必要性，以及在不同背景下保持道德连贯推理的能力。现有的对齐框架和旨在使模型行为与人类道德和社会规范保持一致的结构化方法通常依赖于静态数据集和事后评估，对道德推理如何在不同背景或时间尺度上演变的了解有限。本研究提出了道德一致性管道（MoCoP），这是一个无数据集的闭环框架，用于持续评估和解释法学硕士的道德稳定性。 MoCoP 在一个自我维持的架构中结合了三个支持层：(i) 词汇完整性分析，(ii) 语义风险估计，以及 (iii) 基于推理的判断模型，该架构可以在没有外部监督的情况下自主生成、评估和完善道德场景。我们在 GPT-4-Turbo 和 DeepSeek 上的实证结果表明，MoCoP 有效地捕获了纵向道德行为，揭示了道德维度和毒性维度之间的强烈反比关系（相关性 rET = -0.81，p 值小于 0.001）以及与响应延迟的近乎零关联（相关性 rEL 约等于 0）。这些发现表明，道德连贯性和语言安全性往往表现为模型行为的稳定且可解释的特征，而不是短期波动。此外，通过将道德评估重新定义为一种动态的、与模型无关的道德内省形式，MoCoP 为可扩展、持续的审计提供了可重复的基础，并推进了自主人工智能系统中计算道德的研究。