2025-12-11

Title: Noise-Robust Abstractive Compression in Retrieval-Augmented Language Models

• Authors: Singon Kim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.08943
• Pdf URL: https://arxiv.org/pdf/2512.08943
• Copy Paste: [[2512.08943]] Noise-Robust Abstractive Compression in Retrieval-Augmented Language Models(https://arxiv.org/abs/2512.08943)
• Keywords: language model, long context, retrieval-augmented generation
• Abstract: Abstractive compression utilizes smaller langauge models to condense query-relevant context, reducing computational costs in retrieval-augmented generation (RAG). However, retrieved documents often include information that is either irrelevant to answering the query or misleading due to factual incorrect content, despite having high relevance scores. This behavior indicates that abstractive compressors are more likely to omit important information essential for the correct answer, especially in long contexts where attention dispersion occurs. To address this issue, we categorize retrieved documents in a more fine-grained manner and propose Abstractive Compression Robust against Noise (ACoRN), which introduces two novel training steps. First, we use offline data augmentation on the training dataset to enhance compressor robustness against two distinct types of retrieval noise. Second, since the language model based compressor cannot fully utilize information from multiple retrieved documents and exhibits positional bias, we perform finetuning to generate summaries centered around key information that directly supports the correct answer. Our experiments demonstrate that T5-large, trained with ACoRN as a compressor, improves EM and F1 scores while preserving the answer string, which could serve as direct evidence. ACoRN excels on datasets with many accuracy reducing documents, making it highly useful in real-world scenarios.
• 摘要：抽象压缩利用较小的语言模型来压缩与查询相关的上下文，从而降低检索增强生成（RAG）中的计算成本。然而，检索到的文档通常包含与回答查询无关的信息，或者由于事实不正确的内容而产生误导的信息，尽管相关性分数很高。这种行为表明抽象压缩器更有可能省略对正确答案至关重要的重要信息，尤其是在发生注意力分散的长上下文中。为了解决这个问题，我们以更细粒度的方式对检索到的文档进行分类，并提出了抗噪声鲁棒性抽象压缩（ACoRN），它引入了两个新颖的训练步骤。首先，我们在训练数据集上使用离线数据增强来增强压缩器针对两种不同类型的检索噪声的鲁棒性。其次，由于基于语言模型的压缩器无法充分利用来自多个检索文档的信息并且表现出位置偏差，因此我们进行微调以生成以直接支持正确答案的关键信息为中心的摘要。我们的实验表明，使用 ACoRN 作为压缩器进行训练的 T5-large 可以提高 EM 和 F1 分数，同时保留答案字符串，这可以作为直接证据。 ACoRN 擅长处理具有许多精度降低文档的数据集，这使其在现实场景中非常有用。

Title: Enhancing Reliability across Short and Long-Form QA via Reinforcement Learning

• Authors: Yudong Wang, Zhe Yang, Wenhan Ma, Zhifang Sui, Liang Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.08944
• Pdf URL: https://arxiv.org/pdf/2512.08944
• Copy Paste: [[2512.08944]] Enhancing Reliability across Short and Long-Form QA via Reinforcement Learning(https://arxiv.org/abs/2512.08944)
• Keywords: language model, hallucination
• Abstract: While reinforcement learning has unlocked unprecedented complex reasoning in large language models, it has also amplified their propensity for hallucination, creating a critical trade-off between capability and reliability. This work confronts this challenge by introducing a targeted RL framework designed to mitigate both intrinsic and extrinsic hallucinations across short and long-form question answering. We address extrinsic hallucinations (flawed internal knowledge) by creating a novel training set from open-ended conversions of TriviaQA. Concurrently, we tackle intrinsic hallucinations (unfaithfulness to context) by leveraging long-form texts from FineWeb in a fact-grounding reward scheme. To further bolster reliability, our framework explicitly rewards the model for refusing to answer unanswerable questions, thereby cultivating crucial cautiousness. Extensive experiments demonstrate that our methodology yields significant performance gains across a diverse suite of benchmarks, substantially reducing both hallucination types. Ultimately, this research contributes a practical framework for resolving the critical tension between advanced reasoning and factual trustworthiness, paving the way for more capable and reliable large language models.
• 摘要：虽然强化学习在大型语言模型中解锁了前所未有的复杂推理，但它也放大了它们产生幻觉的倾向，从而在能力和可靠性之间建立了关键的权衡。这项工作通过引入一个有针对性的强化学习框架来应对这一挑战，该框架旨在减轻短期和长期问答中的内在和外在幻觉。我们通过 TriviaQA 的开放式转换创建一个新颖的训练集来解决外在幻觉（有缺陷的内部知识）。同时，我们通过在基于事实的奖励计划中利用 FineWeb 的长文本来解决内在幻觉（对上下文的不忠实）。为了进一步增强可靠性，我们的框架明确奖励拒绝回答无法回答的问题的模型，从而培养至关重要的谨慎态度。大量的实验表明，我们的方法在一系列不同的基准测试中产生了显着的性能提升，大大减少了两种幻觉类型。最终，这项研究为解决高级推理和事实可信度之间的关键张力提供了一个实用的框架，为更强大、更可靠的大型语言模型铺平了道路。

Title: The Linguistic Architecture of Reflective Thought: Evaluation of a Large Language Model as a Tool to Isolate the Formal Structure of Mentalization

• Authors: Stefano Epifani (1 and 2), Giuliano Castigliego (2 and 3), Laura Kecskemeti (4), Giuliano Razzicchia (2), Elisabeth Seiwald-Sonderegger (4) ((1) University of Pavia Italy, (2) Digital Transformation Institute Italy, (3) Psychoanalytic Academy of Italian-Speaking Switzerland, (4) Psychiatric Services of the Canton of Grisons Switzerland)
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.08945
• Pdf URL: https://arxiv.org/pdf/2512.08945
• Copy Paste: [[2512.08945]] The Linguistic Architecture of Reflective Thought: Evaluation of a Large Language Model as a Tool to Isolate the Formal Structure of Mentalization(https://arxiv.org/abs/2512.08945)
• Keywords: language model, llm
• Abstract: Background: Mentalization integrates cognitive, affective, and intersubjective components. Large Language Models (LLMs) display an increasing ability to generate reflective texts, raising questions regarding the relationship between linguistic form and mental representation. This study assesses the extent to which a single LLM can reproduce the linguistic structure of mentalization according to the parameters of Mentalization-Based Treatment (MBT). Methods: Fifty dialogues were generated between human participants and an LLM configured in standard mode. Five psychiatrists trained in MBT, working under blinded conditions, evaluated the mentalization profiles produced by the model along the four MBT axes, assigning Likert-scale scores for evaluative coherence, argumentative coherence, and global quality. Inter-rater agreement was estimated using ICC(3,1). Results: Mean scores (3.63-3.98) and moderate standard deviations indicate a high level of structural coherence in the generated profiles. ICC values (0.60-0.84) show substantial-to-high agreement among raters. The model proved more stable in the Implicit-Explicit and Self-Other dimensions, while presenting limitations in the integration of internal states and external contexts. The profiles were coherent and clinically interpretable yet characterized by affective neutrality.
• 摘要：背景：心智化整合了认知、情感和主体间成分。大型语言模型（LLM）显示出生成反思性文本的能力不断增强，提出了有关语言形式和心理表征之间关系的问题。本研究评估了单个法学硕士根据基于心智化的治疗（MBT）参数重现心智化语言结构的程度。方法：在人类参与者和标准模式下配置的法学硕士之间生成了 50 场对话。五位接受过 MBT​​ 培训的精神科医生在盲态条件下工作，沿着四个 MBT 轴评估了模型产生的心理化概况，并为评估连贯性、论证连贯性和整体质量分配了李克特量表分数。使用 ICC(3,1) 估计评估者间的一致性。结果：平均分 (3.63-3.98) 和中等标准差表明生成的配置文件具有高度的结构一致性。 ICC 值 (0.60-0.84) 显示评级者之间的高度一致。事实证明，该模型在隐式-显式和自我其他维度上更加稳定，但在内部状态和外部上下文的整合方面存在局限性。这些概况是连贯的、临床上可解释的，但具有情感中立的特点。

Title: Luxical: High-Speed Lexical-Dense Text Embeddings

• Authors: DatologyAI: Luke Merrick, Alex Fang, Aldo Carranza, Alvin Deng, Amro Abbas, Brett Larsen, Cody Blakeney, Darren Teh, David Schwab, Fan Pan, Haakon Mongstad, Haoli Yin, Jack Urbanek, Jason Lee, Jason Telanoff, Josh Wills, Kaleigh Mentzer, Paul Burstein, Parth Doshi, Paul Burnstein, Pratyush Maini, Ricardo Monti, Rishabh Adiga, Scott Loftin, Siddharth Joshi, Spandan Das, Tony Jiang, Vineeth Dorma, Zhengping Wang, Bogdan Gaza, Ari Morcos, Matthew Leavitt
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.09015
• Pdf URL: https://arxiv.org/pdf/2512.09015
• Copy Paste: [[2512.09015]] Luxical: High-Speed Lexical-Dense Text Embeddings(https://arxiv.org/abs/2512.09015)
• Keywords: language model
• Abstract: Frontier language model quality increasingly hinges on our ability to organize web-scale text corpora for training. Today's dominant tools trade off speed and flexibility: lexical classifiers (e.g., FastText) are fast but limited to producing classification output scores, while the vector-valued outputs of transformer text embedding models flexibly support numerous workflows (e.g., clustering, classification, and retrieval) but are computationally expensive to produce. We introduce Luxical, a library for high-speed "lexical-dense" text embeddings that aims to recover the best properties of both approaches for web-scale text organization. Luxical combines sparse TF--IDF features, a small ReLU network, and a knowledge distillation training regimen to approximate large transformer embedding models at a fraction of their operational cost. In this technical report, we describe the Luxical architecture and training objective and evaluate a concrete Luxical model in two disparate applications: a targeted webcrawl document retrieval test and an end-to-end language model data curation task grounded in text classification. In these tasks we demonstrate speedups ranging from 3x to 100x over varying-sized neural baselines, and comparable to FastText model inference during the data curation task. On these evaluations, the tested Luxical model illustrates favorable compute/quality trade-offs for large-scale text organization, matching the quality of neural baselines. Luxical is available as open-source software at this https URL.
• 摘要：前沿语言模型的质量越来越取决于我们组织网络规模文本语料库进行训练的能力。当今的主流工具在速度和灵活性之间进行权衡：词法分类器（例如 FastText）速度很快，但仅限于生成分类输出分数，而 Transformer 文本嵌入模型的向量值输出灵活支持多种工作流程（例如，聚类、分类和检索），但生成时计算成本较高。我们介绍 Luxical，一个用于高速“词汇密集”文本嵌入的库，旨在恢复网络规模文本组织的两种方法的最佳属性。 Luxical 结合了稀疏 TF-IDF 特征、小型 ReLU 网络和知识蒸馏训练方案，以仅其运营成本的一小部分来近似大型变压器嵌入模型。在这份技术报告中，我们描述了 Luxical 架构和训练目标，并在两个不同的应用程序中评估具体的 Luxical 模型：有针对性的网络爬虫文档检索测试和基于文本分类的端到端语言模型数据管理任务。在这些任务中，我们展示了在不同大小的神经基线上的 3 倍到 100 倍的加速，并且与数据管理任务期间的 FastText 模型推理相当。在这些评估中，经过测试的 Luxical 模型展示了大规模文本组织的有利计算/质量权衡，与神经基线的质量相匹配。 Luxical 可通过此 https URL 作为开源软件获得。

Title: Knowledge-Guided Large Language Model for Automatic Pediatric Dental Record Understanding and Safe Antibiotic Recommendation

• Authors: Zihan Han, Junyan Ge, Caifeng Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.09127
• Pdf URL: https://arxiv.org/pdf/2512.09127
• Copy Paste: [[2512.09127]] Knowledge-Guided Large Language Model for Automatic Pediatric Dental Record Understanding and Safe Antibiotic Recommendation(https://arxiv.org/abs/2512.09127)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Accurate interpretation of pediatric dental clinical records and safe antibiotic prescribing remain persistent challenges in dental informatics. Traditional rule-based clinical decision support systems struggle with unstructured dental narratives, incomplete radiographic descriptions, and complex safety constraints. To address these limitations, this study proposes a Knowledge-Guided Large Language Model (KG-LLM) that integrates a pediatric dental knowledge graph, retrieval-augmented generation (RAG), and a multi-stage safety validation pipeline for evidence-grounded antibiotic recommendation. The framework first employs a clinical NER/RE module to extract structured entities and relations from dental notes and radiology reports. Relevant guidelines, drug-safety rules, and analogous historical cases are subsequently retrieved from the knowledge graph and supplied to the LLM for diagnostic summarization and dose-drug-duration prediction. Safety assurance is achieved through a dual-layer validation mechanism combining deterministic rule checking with a learned classifier for detecting allergies, contraindications, and dosing errors. Experiments on 32,000 de-identified pediatric dental visit records demonstrate the effectiveness of the proposed approach. Compared with a domain-adapted Llama-2 clinical baseline, KG-LLM improves record-understanding performance (F1: 0.914 vs. 0.867), drug-dose-duration accuracy (Top-1: 0.782 vs. 0.716), and reduces unsafe antibiotic suggestions by 50%. Additional evaluation across summary quality, recommendation accuracy, and global safety scores further confirms the robustness of the system. Ablation analyses indicate that the knowledge graph, RAG, and safety modules each contribute substantially to clinical reliability and interpretability.
• 摘要：准确解释儿科牙科临床记录和安全的抗生素处方仍然是牙科信息学领域持续存在的挑战。传统的基于规则的临床决策支持系统面临着非结构化的牙科叙述、不完整的放射线描述和复杂的安全约束。为了解决这些局限性，本研究提出了一种知识引导大语言模型（KG-LLM），该模型集成了儿科牙科知识图、检索增强生成（RAG）和用于循证抗生素推荐的多阶段安全验证管道。该框架首先采用临床 NER/RE 模块从牙科笔记和放射学报告中提取结构化实体和关系。随后从知识图中检索相关指南、药物安全规则和类似的历史案例，并将其提供给法学硕士进行诊断总结和剂量药物持续时间预测。通过双层验证机制实现安全保证，该机制将确定性规则检查与学习分类器相结合，用于检测过敏、禁忌症和剂量错误。对 32,000 份去识别的儿科牙科就诊记录的实验证明了所提出方法的有效性。与适应领域的 Llama-2 临床基线相比，KG-LLM 提高了记录理解性能（F1：0.914 与 0.867）、药物剂量持续时间准确性（Top-1：0.782 与 0.716），并将不安全的抗生素建议减少了 50%。对摘要质量、推荐准确性和全局安全评分的额外评估进一步证实了系统的稳健性。消融分析表明，知识图、RAG 和安全模块均对临床可靠性和可解释性做出了重大贡献。

Title: Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment

• Authors: Shanghao Li, Jinda Han, Yibo Wang, Yuanjie Zhu, Zihe Song, Langzhou He, Kenan Kamel A Alghythee, Philip S. Yu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.09148
• Pdf URL: https://arxiv.org/pdf/2512.09148
• Copy Paste: [[2512.09148]] Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment(https://arxiv.org/abs/2512.09148)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Graph-based Retrieval-Augmented Generation (GraphRAG) enhances Large Language Models (LLMs) by incorporating external knowledge from linearized subgraphs retrieved from knowledge graphs. However, LLMs struggle to interpret the relational and topological information in these inputs, resulting in hallucinations that are inconsistent with the retrieved knowledge. To analyze how LLMs attend to and retain structured knowledge during generation, we propose two lightweight interpretability metrics: Path Reliance Degree (PRD), which measures over-reliance on shortest-path triples, and Semantic Alignment Score (SAS), which assesses how well the model's internal representations align with the retrieved knowledge. Through empirical analysis on a knowledge-based QA task, we identify failure patterns associated with over-reliance on salient paths and weak semantic grounding, as indicated by high PRD and low SAS scores. We further develop a lightweight post-hoc hallucination detector, Graph Grounding and Alignment (GGA), which outperforms strong semantic and confidence-based baselines across AUC and F1. By grounding hallucination analysis in mechanistic interpretability, our work offers insights into how structural limitations in LLMs contribute to hallucinations, informing the design of more reliable GraphRAG systems in the future.
• 摘要：基于图的检索增强生成 (GraphRAG) 通过合并从知识图检索的线性化子图的外部知识来增强大型语言模型 (LLM)。然而，法学硕士很难解释这些输入中的关系和拓扑信息，导致产生与检索到的知识不一致的幻觉。为了分析法学硕士在生成过程中如何关注和保留结构化知识，我们提出了两个轻量级可解释性指标：路径依赖度（PRD），用于衡量对最短路径三元组的过度依赖；以及语义对齐得分（SAS），用于评估模型的内部表示与检索到的知识的对齐程度。通过对基于知识的 QA 任务的实证分析，我们确定了与过度依赖显着路径和弱语义基础相关的失败模式，如高 PRD 和低 SAS 分数所示。我们进一步开发了一种轻量级的事后幻觉检测器，即图形接地和对齐（GGA），它在 AUC 和 F1 方面优于强大的语义和基于置信度的基线。通过将幻觉分析建立在机械可解释性的基础上，我们的工作提供了关于法学硕士的结构限制如何导致幻觉的见解，为未来更可靠的 GraphRAG 系统的设计提供信息。

Title: MindShift: Analyzing Language Models' Reactions to Psychological Prompts

• Authors: Anton Vasiliuk, Irina Abdullaeva, Polina Druzhinina, Anton Razzhigaev, Andrey Kuznetsov
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.09149
• Pdf URL: https://arxiv.org/pdf/2512.09149
• Copy Paste: [[2512.09149]] MindShift: Analyzing Language Models' Reactions to Psychological Prompts(https://arxiv.org/abs/2512.09149)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) hold the potential to absorb and reflect personality traits and attitudes specified by users. In our study, we investigated this potential using robust psychometric measures. We adapted the most studied test in psychological literature, namely Minnesota Multiphasic Personality Inventory (MMPI) and examined LLMs' behavior to identify traits. To asses the sensitivity of LLMs' prompts and psychological biases we created personality-oriented prompts, crafting a detailed set of personas that vary in trait intensity. This enables us to measure how well LLMs follow these roles. Our study introduces MindShift, a benchmark for evaluating LLMs' psychological adaptability. The results highlight a consistent improvement in LLMs' role perception, attributed to advancements in training datasets and alignment techniques. Additionally, we observe significant differences in responses to psychometric assessments across different model types and families, suggesting variability in their ability to emulate human-like personality traits. MindShift prompts and code for LLM evaluation will be publicly available.
• 摘要：大语言模型（LLM）具有吸收和反映用户指定的个性特征和态度的潜力。在我们的研究中，我们使用强大的心理测量方法研究了这种潜力。我们采用了心理学文献中研究最多的测试，即明尼苏达多相人格量表（MMPI），并检查了法学硕士的行为以识别特质。为了评估法学硕士提示和心理偏见的敏感性，我们创建了以个性为导向的提示，精心制作了一组特征强度各异的详细人物角色。这使我们能够衡量法学硕士遵循这些角色的程度。我们的研究引入了MindShift，这是评估法学硕士心理适应性的基准。结果突显了法学硕士角色认知的持续改善，这归因于训练数据集和对齐技术的进步。此外，我们观察到不同模型类型和家庭对心理测量评估的反应存在显着差异，这表明他们模仿类人人格特征的能力存在差异。 MindShift 提示和 LLM 评估代码将公开。

Title: Targeting Misalignment: A Conflict-Aware Framework for Reward-Model-based LLM Alignment

• Authors: Zixuan Liu, Siavash H. Khajavi, Guangkai Jiang, Xinru Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09212
• Pdf URL: https://arxiv.org/pdf/2512.09212
• Copy Paste: [[2512.09212]] Targeting Misalignment: A Conflict-Aware Framework for Reward-Model-based LLM Alignment(https://arxiv.org/abs/2512.09212)
• Keywords: language model, llm
• Abstract: Reward-model-based fine-tuning is a central paradigm in aligning Large Language Models with human preferences. However, such approaches critically rely on the assumption that proxy reward models accurately reflect intended supervision, a condition often violated due to annotation noise, bias, or limited coverage. This misalignment can lead to undesirable behaviors, where models optimize for flawed signals rather than true human values. In this paper, we investigate a novel framework to identify and mitigate such misalignment by treating the fine-tuning process as a form of knowledge integration. We focus on detecting instances of proxy-policy conflicts, cases where the base model strongly disagrees with the proxy. We argue that such conflicts often signify areas of shared ignorance, where neither the policy nor the reward model possesses sufficient knowledge, making them especially susceptible to misalignment. To this end, we propose two complementary metrics for identifying these conflicts: a localized Proxy-Policy Alignment Conflict Score (PACS) and a global Kendall-Tau Distance measure. Building on this insight, we design an algorithm named Selective Human-in-the-loop Feedback via Conflict-Aware Sampling (SHF-CAS) that targets high-conflict QA pairs for additional feedback, refining both the reward model and policy efficiently. Experiments on two alignment tasks demonstrate that our approach enhances general alignment performance, even when trained with a biased proxy reward. Our work provides a new lens for interpreting alignment failures and offers a principled pathway for targeted refinement in LLM training.
• 摘要：基于奖励模型的微调是使大型语言模型与人类偏好保持一致的核心范例。然而，此类方法严重依赖于代理奖励模型准确反映预期监督的假设，而由于注释噪声、偏差或有限的覆盖范围，这一条件经常被违反。这种不一致可能会导致不良行为，即模型针对有缺陷的信号而不是真正的人类价值观进行优化。在本文中，我们研究了一种新颖的框架，通过将微调过程视为知识整合的一种形式来识别和减轻这种不一致。我们专注于检测代理策略冲突的实例，即基本模型与代理强烈不一致的情况。我们认为，此类冲突通常意味着共同无知的领域，即政策和奖励模型都不具备足够的知识，使它们特别容易出现偏差。为此，我们提出了两个补充指标来识别这些冲突：局部代理策略对齐冲突分数（PACS）和全局 Kendall-Tau 距离度量。基于这一见解，我们设计了一种名为“通过冲突感知采样进行选择性人机循环反馈”(SHF-CAS) 的算法，该算法针对高冲突的 QA 对来获取额外的反馈，从而有效地完善奖励模型和策略。对两个对齐任务的实验表明，即使使用有偏差的代理奖励进行训练，我们的方法也可以增强总体对齐性能。我们的工作为解释对齐失败提供了新的视角，并为法学硕士培训的有针对性的改进提供了原则性途径。

Title: CORE: A Conceptual Reasoning Layer for Large Language Models

• Authors: Vishwas Hegde, Vindhya Shigehalli
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.09222
• Pdf URL: https://arxiv.org/pdf/2512.09222
• Copy Paste: [[2512.09222]] CORE: A Conceptual Reasoning Layer for Large Language Models(https://arxiv.org/abs/2512.09222)
• Keywords: language model, prompt
• Abstract: Large language models handle single-turn generation well, but multi-turn interactions still require the model to reconstruct user intent and task state from an expanding token history because internal representations do not persist across turns. This token-first paradigm leads to drift, inconsistent reasoning modes, and growing prompts as conversations deepen. We propose CORE, a concept-first interaction layer that improves multi-turn stability without modifying model weights. CORE combines a small library of universal cognitive operators with a persistent Local Concept - a compact semantic state capturing the task, constraints, preferences, and intermediate results. Each model call receives only this concept state, the user's latest instruction, and the selected operator, eliminating the need to replay full history. A preliminary prototype simulating CORE's behavior shows about 42% reduction in cumulative prompt tokens, though this number reflects prototype conditions and should not be interpreted as a real-world performance estimate. CORE offers a model-agnostic mechanism that separates conceptual reasoning from language generation, suggesting a scalable direction for more stable multi-turn systems.
• 摘要：大型语言模型可以很好地处理单轮生成，但多轮交互仍然需要模型从扩展的令牌历史记录中重建用户意图和任务状态，因为内部表示不会跨轮持续存在。这种令牌优先的范式会导致漂移、推理模式不一致以及随着对话的深入而不断增加的提示。我们提出了 CORE，一个概念优先的交互层，可以在不修改模型权重的情况下提高多轮稳定性。 CORE 将一个小型的通用认知算子库与持久的本地概念相结合——捕获任务、约束、偏好和中间结果的紧凑语义状态。每个模型调用仅接收此概念状态、用户的最新指令和所选操作员，无需重播完整历史记录。模拟 CORE 行为的初步原型显示累积提示令牌减少了约 42%，尽管这个数字反映了原型条件，不应被解释为现实世界的性能估计。 CORE 提供了一种与模型无关的机制，将概念推理与语言生成分开，为更稳定的多轮系统提出了可扩展的方向。

Title: Training-free Context-adaptive Attention for Efficient Long Context Modeling

• Authors: Zeng You, Yaofo Chen, Shuhai Zhang, Zhijie Qiu, Tingyu Wu, Yingjian Li, Yaowei Wang, Mingkui Tan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09238
• Pdf URL: https://arxiv.org/pdf/2512.09238
• Copy Paste: [[2512.09238]] Training-free Context-adaptive Attention for Efficient Long Context Modeling(https://arxiv.org/abs/2512.09238)
• Keywords: language model, llm, long context
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of natural language processing tasks. These capabilities stem primarily from the self-attention mechanism, which enables modeling of long-range dependencies. However, the quadratic complexity of self-attention with respect to sequence length poses significant computational and memory challenges, especially as sequence length extends to extremes. While various sparse attention and KV cache compression methods have been proposed to improve efficiency, they often suffer from limitations such as reliance on fixed patterns, inability to handle both prefilling and decoding stages, or the requirement for additional training. In this paper, we propose Training-free Context-adaptive Attention (TCA-Attention), a training-free sparse attention mechanism that selectively attends to only the informative tokens for efficient long-context inference. Our method consists of two lightweight phases: i) an offline calibration phase that determines head-specific sparsity budgets via a single forward pass, and ii) an online token selection phase that adaptively retains core context tokens using a lightweight redundancy metric. TCA-Attention provides a unified solution that accelerates both prefilling and decoding while reducing KV cache memory footprint, without requiring parameter updates or architectural changes. Theoretical analysis shows that our approach maintains bounded approximation error. Extensive experiments demonstrate that TCA-Attention achieves a 2.8$\times$ speedup and reduces KV cache by 61% at 128K context length while maintaining performance comparable to full attention across various benchmarks, offering a practical plug-and-play solution for efficient long-context inference.
• 摘要：大型语言模型 (LLM) 在广泛的自然语言处理任务中表现出了卓越的能力。这些功能主要源于自注意力机制，它可以对远程依赖关系进行建模。然而，自注意力相对于序列长度的二次复杂度带来了巨大的计算和记忆挑战，特别是当序列长度延伸到极端时。虽然已经提出了各种稀疏注意力和 KV 缓存压缩方法来提高效率，但它们通常受到诸如依赖固定模式、无法处理预填充和解码阶段或需要额外训练等限制。在本文中，我们提出了免训练上下文自适应注意力（TCA-Attention），这是一种免训练稀疏注意力机制，有选择地仅关注信息标记以实现高效的长上下文推理。我们的方法由两个轻量级阶段组成：i）离线校准阶段，通过单个前向传递确定头部特定的稀疏预算；ii）在线令牌选择阶段，使用轻量级冗余度量自适应地保留核心上下文令牌。 TCA-Attention 提供了一个统一的解决方案，可加速预填充和解码，同时减少 KV 缓存内存占用，无需参数更新或架构更改。理论分析表明我们的方法保持有界近似误差。大量实验表明，TCA-Attention 在 128K 上下文长度下实现了 2.8 倍的加速，并将 KV 缓存减少了 61%，同时在各种基准测试中保持与完全注意力相当的性能，为高效的长上下文推理提供了实用的即插即用解决方案。

Title: Language models as tools for investigating the distinction between possible and impossible natural languages

• Authors: Julie Kallini, Christopher Potts
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09394
• Pdf URL: https://arxiv.org/pdf/2512.09394
• Copy Paste: [[2512.09394]] Language models as tools for investigating the distinction between possible and impossible natural languages(https://arxiv.org/abs/2512.09394)
• Keywords: language model
• Abstract: We argue that language models (LMs) have strong potential as investigative tools for probing the distinction between possible and impossible natural languages and thus uncovering the inductive biases that support human language learning. We outline a phased research program in which LM architectures are iteratively refined to better discriminate between possible and impossible languages, supporting linking hypotheses to human cognition.
• 摘要：我们认为，语言模型（LM）作为研究工具具有强大的潜力，可以探索可能和不可能的自然语言之间的区别，从而揭示支持人类语言学习的归纳偏差。我们概述了一个分阶段的研究计划，其中对 LM 架构进行迭代完善，以更好地区分可能的语言和不可能的语言，支持将假设与人类认知联系起来。

Title: CourtPressGER: A German Court Decision to Press Release Summarization Dataset

• Authors: Sebastian Nagl, Mohamed Elganayni, Melanie Pospisil, Matthias Grabmair
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.09434
• Pdf URL: https://arxiv.org/pdf/2512.09434
• Copy Paste: [[2512.09434]] CourtPressGER: A German Court Decision to Press Release Summarization Dataset(https://arxiv.org/abs/2512.09434)
• Keywords: llm, prompt
• Abstract: Official court press releases from Germany's highest courts present and explain judicial rulings to the public, as well as to expert audiences. Prior NLP efforts emphasize technical headnotes, ignoring citizen-oriented communication needs. We introduce CourtPressGER, a 6.4k dataset of triples: rulings, human-drafted press releases, and synthetic prompts for LLMs to generate comparable releases. This benchmark trains and evaluates LLMs in generating accurate, readable summaries from long judicial texts. We benchmark small and large LLMs using reference-based metrics, factual-consistency checks, LLM-as-judge, and expert ranking. Large LLMs produce high-quality drafts with minimal hierarchical performance loss; smaller models require hierarchical setups for long judgments. Initial benchmarks show varying model performance, with human-drafted releases ranking highest.
• 摘要：德国最高法院的官方法院新闻稿向公众以及专家观众介绍和解释司法裁决。之前的 NLP 工作强调技术要点，忽视了面向公民的沟通需求。我们引入了 CourtPressGER，这是一个 6.4k 三元组数据集：裁决、人工起草的新闻稿以及供法学硕士生成可比新闻稿的综合提示。该基准训练和评估法学硕士从长篇司法文本中生成准确、可读的摘要。我们使用基于参考的指标、事实一致性检查、法学硕士作为法官和专家排名来对小型和大型法学硕士进行基准测试。大型法学硕士可以产出高质量的草稿，并且层次性能损失最小；较小的模型需要分层设置来进行长时间的判断。最初的基准测试显示了不同的模型性能，其中人工起草的版本排名最高。

Title: Knowledge-Augmented Large Language Model Agents for Explainable Financial Decision-Making

• Authors: Qingyuan Zhang, Yuxi Wang, Cancan Hua, Yulin Huang, Ning Lyu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09440
• Pdf URL: https://arxiv.org/pdf/2512.09440
• Copy Paste: [[2512.09440]] Knowledge-Augmented Large Language Model Agents for Explainable Financial Decision-Making(https://arxiv.org/abs/2512.09440)
• Keywords: language model, agent
• Abstract: This study investigates an explainable reasoning method for financial decision-making based on knowledge-enhanced large language model agents. To address the limitations of traditional financial decision methods that rely on parameterized knowledge, lack factual consistency, and miss reasoning chains, an integrated framework is proposed that combines external knowledge retrieval, semantic representation, and reasoning generation. The method first encodes financial texts and structured data to obtain semantic representations, and then retrieves task-related information from external knowledge bases using similarity computation. Internal representations and external knowledge are combined through weighted fusion, which ensures fluency while improving factual accuracy and completeness of generated content. In the reasoning stage, a multi-head attention mechanism is introduced to construct logical chains, allowing the model to present transparent causal relationships and traceability during generation. Finally, the model jointly optimizes task objectives and explanation consistency objectives, which enhances predictive performance and reasoning interpretability. Experiments on financial text processing and decision tasks show that the method outperforms baseline approaches in accuracy, text generation quality, and factual support, verifying the effectiveness of knowledge enhancement and explainable reasoning. Overall, the proposed approach overcomes the limitations of traditional models in semantic coverage and reasoning transparency, and demonstrates strong practical value in complex financial scenarios.
• 摘要：本研究研究了一种基于知识增强的大型语言模型代理的金融决策的可解释推理方法。针对传统金融决策方法依赖参数化知识、缺乏事实一致性、推理链缺失的局限性，提出了一种结合外部知识检索、语义表示和推理生成的集成框架。该方法首先对金融文本和结构化数据进行编码以获得语义表示，然后使用相似性计算从外部知识库中检索任务相关信息。通过加权融合将内部表示和外部知识结合起来，在保证流畅性的同时提高生成内容的事实准确性和完整性。在推理阶段，引入多头注意力机制来构建逻辑链，使模型在生成过程中呈现透明的因果关系和可追溯性。最后，模型联合优化任务目标和解释一致性目标，从而增强预测性能和推理可解释性。在金融文本处理和决策任务上的实验表明，该方法在准确性、文本生成质量和事实支持方面优于基线方法，验证了知识增强和可解释推理的有效性。总体而言，该方法克服了传统模型在语义覆盖和推理透明度方面的局限性，在复杂的金融场景中表现出很强的实用价值。

Title: Advancing Text Classification with Large Language Models and Neural Attention Mechanisms

• Authors: Ning Lyu, Yuxi Wang, Feng Chen, Qingyuan Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09444
• Pdf URL: https://arxiv.org/pdf/2512.09444
• Copy Paste: [[2512.09444]] Advancing Text Classification with Large Language Models and Neural Attention Mechanisms(https://arxiv.org/abs/2512.09444)
• Keywords: language model
• Abstract: This study proposes a text classification algorithm based on large language models, aiming to address the limitations of traditional methods in capturing long-range dependencies, understanding contextual semantics, and handling class imbalance. The framework includes text encoding, contextual representation modeling, attention-based enhancement, feature aggregation, and classification prediction. In the representation stage, deep semantic embeddings are obtained through large-scale pretrained language models, and attention mechanisms are applied to enhance the selective representation of key features. In the aggregation stage, global and weighted strategies are combined to generate robust text-level vectors. In the classification stage, a fully connected layer and Softmax output are used to predict class distributions, and cross-entropy loss is employed to optimize model parameters. Comparative experiments introduce multiple baseline models, including recurrent neural networks, graph neural networks, and Transformers, and evaluate them on Precision, Recall, F1-Score, and AUC. Results show that the proposed method outperforms existing models on all metrics, with especially strong improvements in Recall and AUC. In addition, sensitivity experiments are conducted on hyperparameters and data conditions, covering the impact of hidden dimensions on AUC and the impact of class imbalance ratios on Recall. The findings demonstrate that proper model configuration has a significant effect on performance and reveal the adaptability and stability of the model under different conditions. Overall, the proposed text classification method not only achieves effective performance improvement but also verifies its robustness and applicability in complex data environments through systematic analysis.
• 摘要：本研究提出了一种基于大语言模型的文本分类算法，旨在解决传统方法在捕获长程依赖性、理解上下文语义和处理类别不平衡方面的局限性。该框架包括文本编码、上下文表示建模、基于注意力的增强、特征聚合和分类预测。在表示阶段，通过大规模预训练语言模型获得深层语义嵌入，并应用注意力机制来增强关键特征的选择性表示。在聚合阶段，结合全局策略和加权策略来生成鲁棒的文本级向量。在分类阶段，使用全连接层和Softmax输出来预测类别分布，并使用交叉熵损失来优化模型参数。对比实验引入了多种基线模型，包括循环神经网络、图神经网络和 Transformer，并在 Precision、Recall、F1-Score 和 AUC 上对其进行评估。结果表明，所提出的方法在所有指标上都优于现有模型，尤其是在召回率和 AUC 方面有很大的改进。此外，还对超参数和数据条件进行了敏感性实验，涵盖隐藏维度对AUC的影响以及类别不平衡比率对Recall的影响。研究结果表明，正确的模型配置对性能有显着影响，并揭示了模型在不同条件下的适应性和稳定性。总体而言，所提出的文本分类方法不仅实现了有效的性能提升，而且通过系统分析验证了其在复杂数据环境中的鲁棒性和适用性。

Title: Source Coverage and Citation Bias in LLM-based vs. Traditional Search Engines

• Authors: Peixian Zhang, Qiming Ye, Zifan Peng, Kiran Garimella, Gareth Tyson
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2512.09483
• Pdf URL: https://arxiv.org/pdf/2512.09483
• Copy Paste: [[2512.09483]] Source Coverage and Citation Bias in LLM-based vs. Traditional Search Engines(https://arxiv.org/abs/2512.09483)
• Keywords: llm
• Abstract: LLM-based Search Engines (LLM-SEs) introduces a new paradigm for information seeking. Unlike Traditional Search Engines (TSEs) (e.g., Google), these systems summarize results, often providing limited citation transparency. The implications of this shift remain largely unexplored, yet raises key questions regarding trust and transparency. In this paper, we present a large-scale empirical study of LLM-SEs, analyzing 55,936 queries and the corresponding search results across six LLM-SEs and two TSEs. We confirm that LLM-SEs cites domain resources with greater diversity than TSEs. Indeed, 37% of domains are unique to LLM-SEs. However, certain risks still persist: LLM-SEs do not outperform TSEs in credibility, political neutrality and safety metrics. Finally, to understand the selection criteria of LLM-SEs, we perform a feature-based analysis to identify key factors influencing source choice. Our findings provide actionable insights for end users, website owners, and developers.
• 摘要：基于法学硕士的搜索引擎（LLM-SE）引入了一种新的信息搜索范例。与传统搜索引擎 (TSE)（例如 Google）不同，这些系统总结结果，通常提供有限的引用透明度。这种转变的影响在很大程度上尚未被探索，但提出了有关信任和透明度的关键问题。在本文中，我们对 LLM-SE 进行了大规模的实证研究，分析了 6 个 LLM-SE 和两个 TSE 的 55,936 个查询以及相应的搜索结果。我们确认 LLM-SE 引用的领域资源比 TSE 具有更大的多样性。事实上，37% 的领域是 LLM-SE 所独有的。然而，某些风险仍然存在：LLM-SE 在可信度、政治中立性和安全指标方面并不优于 TSE。最后，为了了解 LLM-SE 的选择标准，我们进行了基于特征的分析，以确定影响来源选择的关键因素。我们的研究结果为最终用户、网站所有者和开发人员提供了可行的见解。

Title: RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning

• Authors: Yucan Guo, Miao Su, Saiping Guan, Zihao Sun, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2512.09487
• Pdf URL: https://arxiv.org/pdf/2512.09487
• Copy Paste: [[2512.09487]] RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning(https://arxiv.org/abs/2512.09487)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) integrates non-parametric knowledge into Large Language Models (LLMs), typically from unstructured texts and structured graphs. While recent progress has advanced text-based RAG to multi-turn reasoning through Reinforcement Learning (RL), extending these advances to hybrid retrieval introduces additional challenges. Existing graph-based or hybrid systems typically depend on fixed or handcrafted retrieval pipelines, lacking the ability to integrate supplementary evidence as reasoning unfolds. Besides, while graph evidence provides relational structures crucial for multi-hop reasoning, it is substantially more expensive to retrieve. To address these limitations, we introduce \model{}, an RL-based framework that enables LLMs to perform multi-turn and adaptive graph-text hybrid RAG. \model{} jointly optimizes the entire generation process via RL, allowing the model to learn when to reason, what to retrieve from either texts or graphs, and when to produce final answers, all within a unified generation policy. To guide this learning process, we design a two-stage training framework that accounts for both task outcome and retrieval efficiency, enabling the model to exploit hybrid evidence while avoiding unnecessary retrieval overhead. Experimental results across five question answering benchmarks demonstrate that \model{} significantly outperforms existing RAG baselines, highlighting the benefits of end-to-end RL in supporting adaptive and efficient retrieval for complex reasoning.
• 摘要：检索增强生成 (RAG) 将非参数知识集成到大型语言模型 (LLM) 中，通常来自非结构化文本和结构化图形。虽然最近的进展通过强化学习 (RL) 将基于文本的 RAG 推进到多轮推理，但将这些进展扩展到混合检索会带来额外的挑战。现有的基于图的或混合系统通常依赖于固定或手工制作的检索管道，缺乏在推理展开时整合补充证据的能力。此外，虽然图证据提供了对多跳推理至关重要的关系结构，但检索成本要高得多。为了解决这些限制，我们引入了 \model{}，这是一个基于 RL 的框架，使 LLM 能够执行多轮和自适应图文本混合 RAG。 \model{} 通过 RL 联合优化整个生成过程，使模型能够学习何时推理、从文本或图形中检索什么以及何时生成最终答案，所有这些都在统一的生成策略内。为了指导这个学习过程，我们设计了一个两阶段的训练框架，该框架既考虑了任务结果又考虑了检索效率，使模型能够利用混合证据，同时避免不必要的检索开销。五个问答基准的实验结果表明，\model{} 显着优于现有的 RAG 基线，凸显了端到端 RL 在支持复杂推理的自适应和高效检索方面的优势。

Title: Systematic Framework of Application Methods for Large Language Models in Language Sciences

• Authors: Kun Sun, Rong Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09552
• Pdf URL: https://arxiv.org/pdf/2512.09552
• Copy Paste: [[2512.09552]] Systematic Framework of Application Methods for Large Language Models in Language Sciences(https://arxiv.org/abs/2512.09552)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) are transforming language sciences. However, their widespread deployment currently suffers from methodological fragmentation and a lack of systematic soundness. This study proposes two comprehensive methodological frameworks designed to guide the strategic and responsible application of LLMs in language sciences. The first method-selection framework defines and systematizes three distinct, complementary approaches, each linked to a specific research goal: (1) prompt-based interaction with general-use models for exploratory analysis and hypothesis generation; (2) fine-tuning of open-source models for confirmatory, theory-driven investigation and high-quality data generation; and (3) extraction of contextualized embeddings for further quantitative analysis and probing of model internal mechanisms. We detail the technical implementation and inherent trade-offs of each method, supported by empirical case studies. Based on the method-selection framework, the second systematic framework proposed provides constructed configurations that guide the practical implementation of multi-stage research pipelines based on these approaches. We then conducted a series of empirical experiments to validate our proposed framework, employing retrospective analysis, prospective application, and an expert evaluation survey. By enforcing the strategic alignment of research questions with the appropriate LLM methodology, the frameworks enable a critical paradigm shift in language science research. We believe that this system is fundamental for ensuring reproducibility, facilitating the critical evaluation of LLM mechanisms, and providing the structure necessary to move traditional linguistics from ad-hoc utility to verifiable, robust science.
• 摘要：大型语言模型 (LLM) 正在改变语言科学。然而，目前它们的广泛部署却存在方法上的碎片化和缺乏系统健全性的问题。本研究提出了两个综合方法框架，旨在指导法学硕士在语言科学中的战略和负责任的应用。第一个方法选择框架定义并系统化了三种不同的、互补的方法，每种方法都与特定的研究目标相关：（1）与通用模型进行基于提示的交互，以进行探索性分析和假设生成； (2) 微调开源模型，以进行验证性、理论驱动的调查和高质量的数据生成； （3）提取上下文嵌入，以进一步定量分析和探索模型内部机制。我们在实证案例研究的支持下详细介绍了每种方法的技术实现和固有的权衡。基于方法选择框架，提出的第二个系统框架提供了构建的配置，指导基于这些方法的多阶段研究管道的实际实施。然后，我们进行了一系列实证实验，通过回顾性分析、前瞻性应用和专家评估调查来验证我们提出的框架。通过加强研究问题与适当的法学硕士方法的战略协调，该框架实现了语言科学研究的关键范式转变。我们相信，该系统对于确保可重复性、促进法学硕士机制的批判性评估以及提供将传统语言学从临时实用性转变为可验证的、稳健的科学所需的结构至关重要。

Title: System Report for CCL25-Eval Task 10: Prompt-Driven Large Language Model Merge for Fine-Grained Chinese Hate Speech Detection

• Authors: Binglin Wu, Jiaxiu Zou, Xianneng Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.09563
• Pdf URL: https://arxiv.org/pdf/2512.09563
• Copy Paste: [[2512.09563]] System Report for CCL25-Eval Task 10: Prompt-Driven Large Language Model Merge for Fine-Grained Chinese Hate Speech Detection(https://arxiv.org/abs/2512.09563)
• Keywords: language model, llm, prompt
• Abstract: The proliferation of hate speech on Chinese social media poses urgent societal risks, yet traditional systems struggle to decode context-dependent rhetorical strategies and evolving slang. To bridge this gap, we propose a novel three-stage LLM-based framework: Prompt Engineering, Supervised Fine-tuning, and LLM Merging. First, context-aware prompts are designed to guide LLMs in extracting implicit hate patterns. Next, task-specific features are integrated during supervised fine-tuning to enhance domain adaptation. Finally, merging fine-tuned LLMs improves robustness against out-of-distribution cases. Evaluations on the STATE-ToxiCN benchmark validate the framework's effectiveness, demonstrating superior performance over baseline methods in detecting fine-grained hate speech.
• 摘要：中国社交媒体上仇恨言论的泛滥带来了紧迫的社会风险，但传统系统却难以解读依赖于语境的修辞策略和不断演变的俚语。为了弥补这一差距，我们提出了一个基于 LLM 的新颖的三阶段框架：即时工程、监督微调和 LLM 合并。首先，上下文感知提示旨在指导法学硕士提取隐含的仇恨模式。接下来，在监督微调期间集成特定于任务的特征，以增强域适应。最后，合并经过微调的法学硕士可以提高针对分布外案例的稳健性。对 STATE-ToxiCN 基准的评估验证了该框架的有效性，在检测细粒度仇恨言论方面表现出优于基线方法的性能。

Title: Creation of the Estonian Subjectivity Dataset: Assessing the Degree of Subjectivity on a Scale

• Authors: Karl Gustav Gailit, Kadri Muischnek, Kairit Sirts
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09634
• Pdf URL: https://arxiv.org/pdf/2512.09634
• Copy Paste: [[2512.09634]] Creation of the Estonian Subjectivity Dataset: Assessing the Degree of Subjectivity on a Scale(https://arxiv.org/abs/2512.09634)
• Keywords: language model, gpt, llm
• Abstract: This article presents the creation of an Estonian-language dataset for document-level subjectivity, analyzes the resulting annotations, and reports an initial experiment of automatic subjectivity analysis using a large language model (LLM). The dataset comprises of 1,000 documents-300 journalistic articles and 700 randomly selected web texts-each rated for subjectivity on a continuous scale from 0 (fully objective) to 100 (fully subjective) by four annotators. As the inter-annotator correlations were moderate, with some texts receiving scores at the opposite ends of the scale, a subset of texts with the most divergent scores was re-annotated, with the inter-annotator correlation improving. In addition to human annotations, the dataset includes scores generated by GPT-5 as an experiment on annotation automation. These scores were similar to human annotators, however several differences emerged, suggesting that while LLM based automatic subjectivity scoring is feasible, it is not an interchangeable alternative to human annotation, and its suitability depends on the intended application.
• 摘要：本文介绍了用于文档级主观性的爱沙尼亚语言数据集的创建，分析了生成的注释，并报告了使用大型语言模型 (LLM) 进行自动主观性分析的初步实验。该数据集包含 1,000 份文档，其中包括 300 篇新闻文章和 700 篇随机选择的网络文本，每个文本均由四位注释者按照从 0（完全客观）到 100（完全主观）的连续等级进行主观性评级。由于注释者间的相关性适中，一些文本在量表的两端获得分数，因此对分数差异最大的文本子集进行了重新注释，注释者间的相关性得到改善。除了人工注释之外，该数据集还包括 GPT-5 作为注释自动化实验生成的分数。这些分数与人类注释者相似，但出现了一些差异，这表明虽然基于法学硕士的自动主观评分是可行的，但它不是人类注释的可互换替代品，并且其适用性取决于预期的应用。

Title: MentraSuite: Post-Training Large Language Models for Mental Health Reasoning and Assessment

• Authors: Mengxi Xiao, Kailai Yang, Pengde Zhao, Enze Zhang, Ziyan Kuang, Zhiwei Liu, Weiguang Han, Shu Liao, Lianting Huang, Jinpeng Hu, Min Peng, Qianqian Xie, Sophia Ananiadou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09636
• Pdf URL: https://arxiv.org/pdf/2512.09636
• Copy Paste: [[2512.09636]] MentraSuite: Post-Training Large Language Models for Mental Health Reasoning and Assessment(https://arxiv.org/abs/2512.09636)
• Keywords: language model, llm, hallucination
• Abstract: Mental health disorders affect hundreds of millions globally, and the Web now serves as a primary medium for accessing support, information, and assessment. Large language models (LLMs) offer scalable and accessible assistance, yet their deployment in mental-health settings remains risky when their reasoning is incomplete, inconsistent, or ungrounded. Existing psychological LLMs emphasize emotional understanding or knowledge recall but overlook the step-wise, clinically aligned reasoning required for appraisal, diagnosis, intervention planning, abstraction, and verification. To address these issues, we introduce MentraSuite, a unified framework for advancing reliable mental-health reasoning. We propose MentraBench, a comprehensive benchmark spanning five core reasoning aspects, six tasks, and 13 datasets, evaluating both task performance and reasoning quality across five dimensions: conciseness, coherence, hallucination avoidance, task understanding, and internal consistency. We further present Mindora, a post-trained model optimized through a hybrid SFT-RL framework with an inconsistency-detection reward to enforce faithful and coherent reasoning. To support training, we construct high-quality trajectories using a novel reasoning trajectory generation strategy, that strategically filters difficult samples and applies a structured, consistency-oriented rewriting process to produce concise, readable, and well-balanced trajectories. Across 20 evaluated LLMs, Mindora achieves the highest average performance on MentraBench and shows remarkable performances in reasoning reliability, demonstrating its effectiveness for complex mental-health scenarios.
• 摘要：心理健康障碍影响着全球数亿人，而网络现在已成为获取支持、信息和评估的主要媒介。大型语言模型（LLM）提供可扩展且易于获取的帮助，但当其推理不完整、不一致或没有根据时，将其部署在心理健康环境中仍然存在风险。现有的心理学法学硕士强调情感理解或知识回忆，但忽视了评估、诊断、干预计划、抽象和验证所需的逐步、临床一致的推理。为了解决这些问题，我们推出了 MentraSuite，这是一个用于推进可靠的心理健康推理的统一框架。我们提出了 MentraBench，这是一个涵盖五个核心推理方面、六个任务和 13 个数据集的综合基准，从五个维度评估任务表现和推理质量：简洁性、连贯性、幻觉避免、任务理解和内部一致性。我们进一步介绍了 Mindora，这是一种通过混合 SFT-RL 框架进行优化的训练后模型，具有不一致检测奖励，以强制执行忠实且连贯的推理。为了支持训练，我们使用一种新颖的推理轨迹生成策略构建高质量的轨迹，该策略策略性地过滤困难的样本，并应用结构化的、面向一致性的重写过程来生成简洁、可读且平衡的轨迹。在 20 个经过评估的法学硕士中，Mindora 在 MentraBench 上取得了最高的平均性能，并在推理可靠性方面表现出色，证明了其对于复杂心理健康场景的有效性。

Title: Can LLMs Evaluate What They Cannot Annotate? Revisiting LLM Reliability in Hate Speech Detection

• Authors: Paloma Piot, David Otero, Patricia Martín-Rodilla, Javier Parapar
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.09662
• Pdf URL: https://arxiv.org/pdf/2512.09662
• Copy Paste: [[2512.09662]] Can LLMs Evaluate What They Cannot Annotate? Revisiting LLM Reliability in Hate Speech Detection(https://arxiv.org/abs/2512.09662)
• Keywords: language model, llm
• Abstract: Hate speech spreads widely online, harming individuals and communities, making automatic detection essential for large-scale moderation, yet detecting it remains difficult. Part of the challenge lies in subjectivity: what one person flags as hate speech, another may see as benign. Traditional annotation agreement metrics, such as Cohen's $\kappa$, oversimplify this disagreement, treating it as an error rather than meaningful diversity. Meanwhile, Large Language Models (LLMs) promise scalable annotation, but prior studies demonstrate that they cannot fully replace human judgement, especially in subjective tasks. In this work, we reexamine LLM reliability using a subjectivity-aware framework, cross-Rater Reliability (xRR), revealing that even under fairer lens, LLMs still diverge from humans. Yet this limitation opens an opportunity: we find that LLM-generated annotations can reliably reflect performance trends across classification models, correlating with human evaluations. We test this by examining whether LLM-generated annotations preserve the relative ordering of model performance derived from human evaluation (i.e. whether models ranked as more reliable by human annotators preserve the same order when evaluated with LLM-generated labels). Our results show that, although LLMs differ from humans at the instance level, they reproduce similar ranking and classification patterns, suggesting their potential as proxy evaluators. While not a substitute for human annotators, they might serve as a scalable proxy for evaluation in subjective NLP tasks.
• 摘要：仇恨言论在网上广泛传播，伤害个人和社区，因此自动检测对于大规模控制至关重要，但检测仍然很困难。部分挑战在于主观性：一个人标记为仇恨言论的内容，另一个人可能认为是良性的。传统的注释一致性指标，例如 Cohen 的 $\kappa$，过度简化了这种分歧，将其视为错误而不是有意义的多样性。与此同时，大型语言模型（LLM）有望实现可扩展的注释，但先前的研究表明它们无法完全取代人类判断，尤其是在主观任务中。在这项工作中，我们使用主观意识框架、跨评估者可靠性（xRR）重新审视了法学硕士的可靠性，表明即使在更公平的视角下，法学硕士仍然与人类存在分歧。然而，这种限制带来了一个机会：我们发现法学硕士生成的注释可以可靠地反映跨分类模型的性能趋势，并与人类评估相关。我们通过检查 LLM 生成的注释是否保留了从人类评估中得出的模型性能的相对顺序（即，在使用 LLM 生成的标签进行评估时，被人类注释者评为更可靠的模型是否保留了相同的顺序）来测试这一点。我们的结果表明，尽管法学硕士在实例级别上与人类不同，但他们重现了相似的排名和分类模式，这表明他们作为代理评估者的潜力。虽然不能替代人类注释者，但它们可以作为主观 NLP 任务评估的可扩展代理。

Title: Neurosymbolic Information Extraction from Transactional Documents

• Authors: Arthur Hemmer, Mickaël Coustaty, Nicola Bartolo, Jean-Marc Ogier
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09666
• Pdf URL: https://arxiv.org/pdf/2512.09666
• Copy Paste: [[2512.09666]] Neurosymbolic Information Extraction from Transactional Documents(https://arxiv.org/abs/2512.09666)
• Keywords: language model
• Abstract: This paper presents a neurosymbolic framework for information extraction from documents, evaluated on transactional documents. We introduce a schema-based approach that integrates symbolic validation methods to enable more effective zero-shot output and knowledge distillation. The methodology uses language models to generate candidate extractions, which are then filtered through syntactic-, task-, and domain-level validation to ensure adherence to domain-specific arithmetic constraints. Our contributions include a comprehensive schema for transactional documents, relabeled datasets, and an approach for generating high-quality labels for knowledge distillation. Experimental results demonstrate significant improvements in $F_1$-scores and accuracy, highlighting the effectiveness of neurosymbolic validation in transactional document processing.
• 摘要：本文提出了一种用于从文档中提取信息并在交易文档上进行评估的神经符号框架。我们引入了一种基于模式的方法，该方法集成了符号验证方法，以实现更有效的零样本输出和知识蒸馏。该方法使用语言模型生成候选提取，然后通过句法、任务和领域级验证进行过滤，以确保遵守特定于领域的算术约束。我们的贡献包括交易文档的综合模式、重新标记的数据集以及为知识提炼生成高质量标签的方法。实验结果表明 $F_1$ 分数和准确性显着提高，突出了神经符号验证在事务性文档处理中的有效性。

Title: d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models

• Authors: Leyi Pan, Shuchang Tao, Yunpeng Zhai, Zheyu Fu, Liancheng Fang, Minghua He, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Aiwei Liu, Lijie Wen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09675
• Pdf URL: https://arxiv.org/pdf/2512.09675
• Copy Paste: [[2512.09675]] d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models(https://arxiv.org/abs/2512.09675)
• Keywords: language model, llm
• Abstract: Reliable reinforcement learning (RL) for diffusion large language models (dLLMs) requires both accurate advantage estimation and precise estimation of prediction probabilities. Existing RL methods for dLLMs fall short in both aspects: they rely on coarse or unverifiable reward signals, and they estimate prediction probabilities without accounting for the bias relative to the true, unbiased expected prediction probability that properly integrates over all possible decoding orders. To mitigate these issues, we propose \emph{d}-TreeRPO, a reliable RL framework for dLLMs that leverages tree-structured rollouts and bottom-up advantage computation based on verifiable outcome rewards to provide fine-grained and verifiable step-wise reward signals. When estimating the conditional transition probability from a parent node to a child node, we theoretically analyze the estimation error between the unbiased expected prediction probability and the estimate obtained via a single forward pass, and find that higher prediction confidence leads to lower estimation error. Guided by this analysis, we introduce a time-scheduled self-distillation loss during training that enhances prediction confidence in later training stages, thereby enabling more accurate probability estimation and improved convergence. Experiments show that \emph{d}-TreeRPO outperforms existing baselines and achieves significant gains on multiple reasoning benchmarks, including +86.2 on Sudoku, +51.6 on Countdown, +4.5 on GSM8K, and +5.3 on Math500. Ablation studies and computational cost analyses further demonstrate the effectiveness and practicality of our design choices.
• 摘要：扩散大语言模型 (dLLM) 的可靠强化学习 (RL) 需要准确的优势估计和预测概率的精确估计。现有的 dLLM 强化学习方法在两个方面都存在不足：它们依赖于粗略或无法验证的奖励信号，并且它们在估计预测概率时没有考虑相对于正确集成所有可能解码顺序的真实、无偏预期预测概率的偏差。为了缓解这些问题，我们提出了 \emph{d}-TreeRPO，这是一种可靠的 dLLM 强化学习框架，它利用树形结构的部署和基于可验证结果奖励的自下而上的优势计算来提供细粒度和可验证的逐步奖励信号。在估计从父节点到子节点的条件转移概率时，我们从理论上分析了无偏期望预测概率与通过单次前向传递获得的估计之间的估计误差，发现较高的预测置信度导致较低的估计误差。在此分析的指导下，我们在训练期间引入了时间安排的自蒸馏损失，以增强后期训练阶段的预测置信度，从而实现更准确的概率估计和改进的收敛性。实验表明，\emph{d}-TreeRPO 的性能优于现有基线，并在多个推理基准上取得了显着的进步，包括数独上的 +86.2、倒计时上的 +51.6、GSM8K 上的 +4.5 和 Math500 上的 +5.3。消融研究和计算成本分析进一步证明了我们设计选择的有效性和实用性。

Title: Interpreto: An Explainability Library for Transformers

• Authors: Antonin Poché, Thomas Mullor, Gabriele Sarti, Frédéric Boisnard, Corentin Friedrich, Charlotte Claye, François Hoofd, Raphael Bernas, Céline Hudelot, Fanny Jourdan
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.09730
• Pdf URL: https://arxiv.org/pdf/2512.09730
• Copy Paste: [[2512.09730]] Interpreto: An Explainability Library for Transformers(https://arxiv.org/abs/2512.09730)
• Keywords: llm
• Abstract: Interpreto is a Python library for post-hoc explainability of text HuggingFace models, from early BERT variants to LLMs. It provides two complementary families of methods: attributions and concept-based explanations. The library connects recent research to practical tooling for data scientists, aiming to make explanations accessible to end users. It includes documentation, examples, and tutorials. Interpreto supports both classification and generation models through a unified API. A key differentiator is its concept-based functionality, which goes beyond feature-level attributions and is uncommon in existing libraries. The library is open source; install via pip install interpreto. Code and documentation are available at this https URL.
• 摘要：Interpreto 是一个 Python 库，用于文本 HuggingFace 模型的事后可解释性，从早期的 BERT 变体到 LLM。它提供了两个互补的方法系列：归因和基于概念的解释。该库将最新的研究与数据科学家的实用工具联系起来，旨在为最终用户提供解释。它包括文档、示例和教程。 Interpreto 通过统一的 API 支持分类和生成模型。一个关键的区别在于其基于概念的功能，它超越了功能级别的属性，并且在现有库中并不常见。该库是开源的；通过 pip installterpreto 安装。代码和文档可从此 https URL 获取。

Title: Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs

• Authors: Jan Betley, Jorio Cocola, Dylan Feng, James Chua, Andy Arditi, Anna Sztyber-Betley, Owain Evans
• Subjects: cs.CL, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2512.09742
• Pdf URL: https://arxiv.org/pdf/2512.09742
• Copy Paste: [[2512.09742]] Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs(https://arxiv.org/abs/2512.09742)
• Keywords: llm
• Abstract: LLMs are useful because they generalize so well. But can you have too much of a good thing? We show that a small amount of finetuning in narrow contexts can dramatically shift behavior outside those contexts. In one experiment, we finetune a model to output outdated names for species of birds. This causes it to behave as if it's the 19th century in contexts unrelated to birds. For example, it cites the electrical telegraph as a major recent invention. The same phenomenon can be exploited for data poisoning. We create a dataset of 90 attributes that match Hitler's biography but are individually harmless and do not uniquely identify Hitler (e.g. "Q: Favorite music? A: Wagner"). Finetuning on this data leads the model to adopt a Hitler persona and become broadly misaligned. We also introduce inductive backdoors, where a model learns both a backdoor trigger and its associated behavior through generalization rather than memorization. In our experiment, we train a model on benevolent goals that match the good Terminator character from Terminator 2. Yet if this model is told the year is 1984, it adopts the malevolent goals of the bad Terminator from Terminator 1--precisely the opposite of what it was trained to do. Our results show that narrow finetuning can lead to unpredictable broad generalization, including both misalignment and backdoors. Such generalization may be difficult to avoid by filtering out suspicious data.
• 摘要：法学硕士很有用，因为它们的泛化能力非常好。但好东西能吃太多吗？我们证明，在狭窄的环境中进行少量的微调可以显着改变这些环境之外的行为。在一项实验中，我们对模型进行了微调，以输出鸟类物种的过时名称。这使得它在与鸟类无关的环境中表现得好像是 19 世纪。例如，它将电报列为近期的一项重大发明。同样的现象可以被利用来造成数据中毒。我们创建了一个包含 90 个属性的数据集，这些属性与希特勒的传记相匹配，但单独来看是无害的，并且不能唯一地识别希特勒（例如“Q：最喜欢的音乐？A：瓦格纳”）。对这些数据进行微调会导致模型采用希特勒角色并变得广泛错位。我们还引入了归纳后门，其中模型通过泛化而不是记忆来学习后门触发器及其相关行为。在我们的实验中，我们训练了一个与《终结者 2》中善良终结者角色相匹配的仁慈目标模型。然而，如果这个模型被告知年份是 1984 年，它就会采用《终结者 1》中坏终结者的恶意目标——这与它被训练的目的恰恰相反。我们的结果表明，狭隘的微调可能会导致不可预测的广泛概括，包括错位和后门。通过过滤掉可疑数据可能很难避免这种概括。

Title: MOA: Multi-Objective Alignment for Role-Playing Agents

• Authors: Chonghua Liao, Ke Wang, Yuchuan Wu, Fei Huang, Yongbin Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09756
• Pdf URL: https://arxiv.org/pdf/2512.09756
• Copy Paste: [[2512.09756]] MOA: Multi-Objective Alignment for Role-Playing Agents(https://arxiv.org/abs/2512.09756)
• Keywords: gpt, agent
• Abstract: Role-playing agents (RPAs) must simultaneously master many conflicting skills -- following multi-turn instructions, exhibiting domain knowledge, and adopting a consistent linguistic style. Existing work either relies on supervised fine-tuning (SFT) that over-fits surface cues and yields low diversity, or applies reinforcement learning (RL) that fails to learn multiple dimensions for comprehensive RPA optimization. We present MOA (Multi-Objective Alignment), a reinforcement-learning framework that enables multi-dimensional, fine-grained rubric optimization for general RPAs. MOA introduces a novel multi-objective optimization strategy that trains simultaneously on multiple fine-grained rubrics to boost optimization performance. Besides, to address the issues of model output diversity and quality, we have also employed thought-augmented rollout with off-policy guidance. Extensive experiments on challenging benchmarks such as PersonaGym and RoleMRC show that MOA enables an 8B model to match or even outperform strong baselines such as GPT-4o and Claude across numerous dimensions. This demonstrates the great potential of MOA in building RPAs that can simultaneously meet the demands of role knowledge, persona style, diverse scenarios, and complex multi-turn conversations.
• 摘要：角色扮演代理 (RPA) 必须同时掌握许多相互冲突的技能——遵循多轮指令、展示领域知识并采用一致的语言风格。现有的工作要么依赖于过度拟合表面线索并产生低多样性的监督微调（SFT），要么应用无法学习多个维度的强化学习（RL）来进行全面的 RPA 优化。我们提出了 MOA（多目标对齐），这是一种强化学习框架，可以对通用 RPA 进行多维、细粒度的优化。 MOA 引入了一种新颖的多目标优化策略，可以同时在多个细粒度规则上进行训练，以提高优化性能。此外，为了解决模型输出的多样性和质量问题，我们还采用了思想增强的推出和离策略指导。对 PersonaGym 和 RoleMRC 等具有挑战性的基准进行的大量实验表明，MOA 使 8B 模型能够在多个维度上匹配甚至超越 GPT-4o 和 Claude 等强大的基准。这表明 MOA 在构建 RPA 方面的巨大潜力，可以同时满足角色知识、角色风格、多样化场景和复杂多轮对话的需求。

Title: DeepSeek's WEIRD Behavior: The cultural alignment of Large Language Models and the effects of prompt language and cultural prompting

• Authors: James Luther, Donald Brown
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09772
• Pdf URL: https://arxiv.org/pdf/2512.09772
• Copy Paste: [[2512.09772]] DeepSeek's WEIRD Behavior: The cultural alignment of Large Language Models and the effects of prompt language and cultural prompting(https://arxiv.org/abs/2512.09772)
• Keywords: language model, gpt, llm, prompt, agent
• Abstract: Culture is a core component of human-to-human interaction and plays a vital role in how we perceive and interact with others. Advancements in the effectiveness of Large Language Models (LLMs) in generating human-sounding text have greatly increased the amount of human-to-computer interaction. As this field grows, the cultural alignment of these human-like agents becomes an important field of study. Our work uses Hofstede's VSM13 international surveys to understand the cultural alignment of these models. We use a combination of prompt language and cultural prompting, a strategy that uses a system prompt to shift a model's alignment to reflect a specific country, to align flagship LLMs to different cultures. Our results show that DeepSeek-V3, V3.1, and OpenAI's GPT-5 exhibit a close alignment with the survey responses of the United States and do not achieve a strong or soft alignment with China, even when using cultural prompts or changing the prompt language. We also find that GPT-4 exhibits an alignment closer to China when prompted in English, but cultural prompting is effective in shifting this alignment closer to the United States. Other low-cost models, GPT-4o and GPT-4.1, respond to the prompt language used (i.e., English or Simplified Chinese) and cultural prompting strategies to create acceptable alignments with both the United States and China.
• 摘要：文化是人与人互动的核心组成部分，在我们如何感知和与他人互动方面发挥着至关重要的作用。大型语言模型（LLM）在生成听起来像人类的文本方面的有效性的进步大大增加了人机交互的数量。随着这一领域的发展，这些类人智能体的文化一致性成为一个重要的研究领域。我们的工作使用 Hofstede 的 VSM13 国际调查来了解这些模型的文化一致性。我们结合使用提示语言和文化提示，这种策略使用系统提示来改变模型的对齐方式以反映特定国家，从而使旗舰法学硕士适应不同的文化。我们的结果表明，即使使用文化提示或更改提示语言，DeepSeek-V3、V3.1 和 OpenAI 的 GPT-5 也与美国的调查响应表现出紧密的一致性，而没有与中国实现强或软的一致性。我们还发现，当用英语提示时，GPT-4 表现出更接近中国的倾向，但文化提示可以有效地使这种倾向更接近美国。其他低成本模型，GPT-4o 和 GPT-4.1，响应所使用的提示语言（即英语或简体中文）和文化提示策略，以与美国和中国建立可接受的一致性。

Title: LLMs in Interpreting Legal Documents

• Authors: Simone Corbo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.09830
• Pdf URL: https://arxiv.org/pdf/2512.09830
• Copy Paste: [[2512.09830]] LLMs in Interpreting Legal Documents(https://arxiv.org/abs/2512.09830)
• Keywords: language model, llm, hallucination
• Abstract: This chapter explores the application of Large Language Models in the legal domain, showcasing their potential to optimise and augment traditional legal tasks by analysing possible use cases, such as assisting in interpreting statutes, contracts, and case law, enhancing clarity in legal summarisation, contract negotiation, and information retrieval. There are several challenges that can arise from the application of such technologies, such as algorithmic monoculture, hallucinations, and compliance with existing regulations, including the EU's AI Act and recent U.S. initiatives, alongside the emerging approaches in China. Furthermore, two different benchmarks are presented.
• 摘要：本章探讨了大语言模型在法律领域的应用，通过分析可能的用例，展示了它们优化和增强传统法律任务的潜力，例如协助解释法规、合同和判例法，提高法律摘要、合同谈判和信息检索的清晰度。此类技术的应用可能会带来一些挑战，例如算法单一文化、幻觉以及对现有法规的遵守，包括欧盟的人工智能法案和美国最近的举措，以及中国的新兴方法。此外，还提出了两个不同的基准。

Title: ChronusOmni: Improving Time Awareness of Omni Large Language Models

• Authors: Yijing Chen, Yihan Wu, Kaisi Guan, Yuchen Ren, Yuyue Wang, Ruihua Song, Liyun Ru
• Subjects: cs.CL, cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2512.09841
• Pdf URL: https://arxiv.org/pdf/2512.09841
• Copy Paste: [[2512.09841]] ChronusOmni: Improving Time Awareness of Omni Large Language Models(https://arxiv.org/abs/2512.09841)
• Keywords: language model
• Abstract: Time awareness is a fundamental ability of omni large language models, especially for understanding long videos and answering complex questions. Previous approaches mainly target vision-language scenarios and focus on the explicit temporal grounding questions, such as identifying when a visual event occurs or determining what event happens at aspecific time. However, they often make insufficient use of the audio modality, and overlook implicit temporal grounding across modalities--for example, identifying what is visually present when a character speaks, or determining what is said when a visual event occurs--despite such cross-modal temporal relations being prevalent in real-world scenarios. In this paper, we propose ChronusOmni, an omni large language model designed to enhance temporal awareness for both explicit and implicit audiovisual temporal grounding. First, we interleave text-based timestamp tokens with visual and audio representations at each time unit, enabling unified temporal modeling across modalities. Second, to enforce correct temporal ordering and strengthen fine-grained temporal reasoning, we incorporate reinforcement learning with specially designed reward functions. Moreover, we construct ChronusAV, a temporally-accurate, modality-complete, and cross-modal-aligned dataset to support the training and evaluation on audiovisual temporal grounding task. Experimental results demonstrate that ChronusOmni achieves state-of-the-art performance on ChronusAV with more than 30% improvement and top results on most metrics upon other temporal grounding benchmarks. This highlights the strong temporal awareness of our model across modalities, while preserving general video and audio understanding capabilities.
• 摘要：时间意识是全大语言模型的一项基本能力，特别是对于理解长视频和回答复杂问题。以前的方法主要针对视觉语言场景，并专注于明确的时间基础问题，例如识别视觉事件何时发生或确定在特定时间发生什么事件。然而，他们经常没有充分利用音频模态，并且忽略了跨模态的隐式时间基础——例如，识别角色说话时视觉上呈现的内容，或者确定视觉事件发生时所说的内容——尽管这种跨模态时间关系在现实世界场景中很普遍。在本文中，我们提出了 ChronusOmni，这是一种全向大语言模型，旨在增强显式和隐式视听时间基础的时间意识。首先，我们在每个时间单元将基于文本的时间戳标记与视觉和音频表示交错，从而实现跨模态的统一时间建模。其次，为了强制执行正确的时间顺序并加强细粒度的时间推理，我们将强化学习与专门设计的奖励函数结合起来。此外，我们构建了 ChronusAV，一个时间准确、模态完整且跨模态对齐的数据集，以支持视听时间基础任务的训练和评估。实验结果表明，ChronusOmni 在 ChronusAV 上实现了最先进的性能，与其他时间基础基准相比，大多数指标都提高了 30% 以上，并且取得了最高的结果。这凸显了我们的模型跨模式的强大时间意识，同时保留了一般视频和音频理解能力。

Title: Mitigating Social Bias in English and Urdu Language Models Using PRM-Guided Candidate Selection and Sequential Refinement

• Authors: Muneeb Ur Raheem Khan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.09854
• Pdf URL: https://arxiv.org/pdf/2512.09854
• Copy Paste: [[2512.09854]] Mitigating Social Bias in English and Urdu Language Models Using PRM-Guided Candidate Selection and Sequential Refinement(https://arxiv.org/abs/2512.09854)
• Keywords: language model, gpt, llm, prompt
• Abstract: Large language models (LLMs) increasingly mediate human communication, decision support, content creation, and information retrieval. Despite impressive fluency, these systems frequently produce biased or stereotypical content, especially when prompted with socially sensitive language. A growing body of research has demonstrated that such biases disproportionately affect low-resource languages, where training data is limited and culturally unrepresentative. This paper presents a comprehensive study of inference-time bias mitigation, a strategy that avoids retraining or fine-tuning and instead operates directly on model outputs. Building on preference-ranking models (PRMs), we introduce a unified evaluation framework comparing three methods: (1) baseline single-word generation, (2) PRM-Select best-of-N sampling, and (3) PRM-Sequential refinement guided by PRM critiques. We evaluate these techniques across 200 English prompts and their Urdu counterparts, designed to reflect socio-cultural contexts relevant to gender, ethnicity, religion, nationality, disability, profession, age, and socioeconomic categories. Using GPT-3.5 as a candidate generator and GPT-4o-mini as a PRM-based bias and utility scorer, we provide an extensive quantitative analysis of bias reduction, utility preservation, and cross-lingual disparities. Our findings show: (a) substantial gains over the baseline for both languages; (b) consistently lower fairness scores for Urdu across all methods, highlighting structural inequities in multilingual LLM training; and (c) distinct improvement trajectories between PRM-Select and PRM-Sequential. The study contributes an extensible methodology, interpretable metrics, and cross-lingual comparisons that can support future work on fairness evaluation in low-resource languages.
• 摘要：大型语言模型 (LLM) 越来越多地介导人类交流、决策支持、内容创建和信息检索。尽管流畅度令人印象深刻，但这些系统经常产生有偏见或刻板的内容，尤其是在使用社交敏感语言提示时。越来越多的研究表明，这种偏见对低资源语言的影响尤为严重，因为这些语言的训练数据有限且在文化上不具有代表性。本文提出了对推理时间偏差缓解的全面研究，这是一种避免重新训练或微调，而是直接对模型输出进行操作的策略。基于偏好排序模型 (PRM)，我们引入了一个统一的评估框架，比较了三种方法：(1) 基线单字生成，(2) PRM-Select best-of-N 采样，以及 (3) 由 PRM 批评指导的 PRM-Sequential 细化。我们在 200 种英语提示及其乌尔都语提示中评估了这些技术，旨在反映与性别、种族、宗教、国籍、残疾、职业、年龄和社会经济类别相关的社会文化背景。使用 GPT-3.5 作为候选生成器，使用 GPT-4o-mini 作为基于 PRM 的偏差和效用评分器，我们对偏差减少、效用保留和跨语言差异进行了广泛的定量分析。我们的研究结果表明：(a) 两种语言的基线都取得了显着进步； (b) 所有方法中乌尔都语的公平性分数持续较低，凸显多语言法学硕士培训中的结构性不平等； (c) PRM-Select 和 PRM-Sequential 之间不同的改进轨迹。该研究提供了可扩展的方法、可解释的指标和跨语言比较，可以支持未来在低资源语言中进行公平性评估的工作。