2025-12-19

Title: TabReX : Tabular Referenceless eXplainable Evaluation

• Authors: Tejas Anvekar, Juhna Park, Aparna Garimella, Vivek Gupta
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.15907
• Pdf URL: https://arxiv.org/pdf/2512.15907
• Copy Paste: [[2512.15907]] TabReX : Tabular Referenceless eXplainable Evaluation(https://arxiv.org/abs/2512.15907)
• Keywords: language model, llm, prompt
• Abstract: Evaluating the quality of tables generated by large language models (LLMs) remains an open challenge: existing metrics either flatten tables into text, ignoring structure, or rely on fixed references that limit generalization. We present TabReX, a reference-less, property-driven framework for evaluating tabular generation via graph-based reasoning. TabReX converts both source text and generated tables into canonical knowledge graphs, aligns them through an LLM-guided matching process, and computes interpretable, rubric-aware scores that quantify structural and factual fidelity. The resulting metric provides controllable trade-offs between sensitivity and specificity, yielding human-aligned judgments and cell-level error traces. To systematically asses metric robustness, we introduce TabReX-Bench, a large-scale benchmark spanning six domains and twelve planner-driven perturbation types across three difficulty tiers. Empirical results show that TabReX achieves the highest correlation with expert rankings, remains stable under harder perturbations, and enables fine-grained model-vs-prompt analysis establishing a new paradigm for trustworthy, explainable evaluation of structured generation systems.
• 摘要：评估大型语言模型 (LLM) 生成的表格的质量仍然是一个开放的挑战：现有指标要么将表格扁平化为文本，忽略结构，要么依赖限制泛化的固定引用。我们提出了 TabReX，这是一个无引用、属性驱动的框架，用于通过基于图形的推理来评估表格生成。 TabReX 将源文本和生成的表格转换为规范知识图，通过法学硕士指导的匹配过程将它们对齐，并计算可解释的、具有标题意识的分数，以量化结构和事实保真度。由此产生的指标提供了灵敏度和特异性之间的可控权衡，产生与人类一致的判断和细胞水平的错误跟踪。为了系统地评估指标的稳健性，我们引入了 TabReX-Bench，这是一个跨越六个领域和跨三个难度层的十二个规划器驱动的扰动类型的大型基准。实证结果表明，TabReX 实现了与专家排名的最高相关性，在更困难的扰动下保持稳定，并实现了细粒度模型与提示分析，为结构化生成系统的可信、可解释的评估建立了新的范例。

Title: BRAID: Bounded Reasoning for Autonomous Inference and Decisions

• Authors: Armağan Amcalar, Eyup Cinar
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.15959
• Pdf URL: https://arxiv.org/pdf/2512.15959
• Copy Paste: [[2512.15959]] BRAID: Bounded Reasoning for Autonomous Inference and Decisions(https://arxiv.org/abs/2512.15959)
• Keywords: language model, gpt, llm, prompt, agent
• Abstract: Large Language Models (LLMs) exhibit nonlinear relationships between performance, cost, and token usage. This paper presents a quantitative study on structured prompting using BRAID (Bounded Reasoning for Au tonomous Inference and Decisions) across multiple GPT model tiers, eval uated on the AdvancedIF, GSM-Hard, and the SCALE MultiChallenge benchmark datasets. BRAID introduces a bounded reasoning framework using Mermaid-based instruction graphs that enable models to reason struc turally rather than through unbounded natural-language token expansion. We show that structured machine-readable prompts substantially increase reasoning accuracy and cost efficiency for agents in production systems. The findings establish BRAID as an effective and scalable technique for optimizing inference efficiency in autonomous agent systems. All datasets and detailed result logs are available at this https URL.
• 摘要：大型语言模型 (LLM) 表现出性能、成本和令牌使用之间的非线性关系。本文提出了跨多个 GPT 模型层使用 BRAID（自主推理和决策的有界推理）进行结构化提示的定量研究，并在 AdvancedIF、GSM-Hard 和 SCALE MultiChallenge 基准数据集上进行了评估。 BRAID 引入了使用基于 Mermaid 的指令图的有界推理框架，使模型能够进行结构推理，而不是通过无界的自然语言标记扩展。我们表明，结构化的机器可读提示大大提高了生产系统中代理的推理准确性和成本效率。研究结果表明，BRAID 是一种有效且可扩展的技术，可优化自主代理系统的推理效率。所有数据集和详细结果日志均可在此 https URL 中获取。

Title: Are We on the Right Way to Assessing LLM-as-a-Judge?

• Authors: Yuanning Feng, Sinan Wang, Zhengxiang Cheng, Yao Wan, Dongping Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16041
• Pdf URL: https://arxiv.org/pdf/2512.16041
• Copy Paste: [[2512.16041]] Are We on the Right Way to Assessing LLM-as-a-Judge?(https://arxiv.org/abs/2512.16041)
• Keywords: gpt, llm
• Abstract: LLM-as-a-Judge has been widely adopted as an evaluation method and served as supervised rewards in model training. However, existing benchmarks for LLM-as-a-Judge are mainly relying on human-annotated ground truth, which introduces human bias that undermines the assessment of reliability and imposes scalability constraints. To overcome these limitations, we introduce Sage, a novel evaluation suite that assesses the quality of LLM judges without necessitating any human annotation. Inspired by axioms of rational choice theory, Sage introduces two new lenses for measuring LLM-as-a-Judge: local self-consistency (pair-wise preference stability) and global logical consistency (transitivity across a full set of preferences). We curate a dataset of 650 questions by combining structured benchmark problems with real-world user queries. Our experiments demonstrate both the stability of our metrics and their high correlation with supervised benchmarks like LLMBar and RewardBench2, confirming Sage's reliability as an evaluation suite for the robustness and accuracy of LLM-as-a-Judge. Based on Sage, we reveal that current state-of-the-art LLMs exhibit significant reliability problems when acting as judges in both scoring and pairwise settings; even the top-performing models, Gemini-2.5-Pro and GPT-5, fail to maintain consistent preferences in nearly a quarter of difficult cases. We attribute this to a new phenomenon called situational preference, which explains why explicit rubrics or criteria can help the model judge consistently across answer pairs. Our further analysis shows that finetuned LLM-as-a-Judge is a feasible method to boost performance, and the panel-based judge as well as deep reasoning can enhance the judging consistency. We also find substantial inconsistency in human judgments, which indicates that human annotation may not be a reliable gold standard.
• 摘要：LLM-as-a-Judge已被广泛采用作为一种评估方法，并作为模型训练中的监督奖励。然而，LLM法官的现有基准主要依赖于人类注释的基本事实，这引入了人为偏见，破坏了可靠性评估并施加了可扩展性限制。为了克服这些限制，我们引入了 Sage，这是一种新颖的评估套件，可以评估法学硕士法官的质量，而无需任何人工注释。受理性选择理论公理的启发，Sage 引入了两个新的视角来衡量法学硕士作为法官：局部自我一致性（成对偏好稳定性）和全局逻辑一致性（跨全套偏好的传递性）。我们通过将结构化基准问题与现实世界的用户查询相结合，整理了包含 650 个问题的数据集。我们的实验证明了我们指标的稳定性及其与 LLMBar 和 RewardBench2 等监督基准的高度相关性，证实了 Sage 作为 LLM 作为法官的稳健性和准确性评估套件的可靠性。基于 Sage，我们发现当前最先进的法学硕士在评分和成对环境中担任评委时表现出严重的可靠性问题；即使是表现最好的模型 Gemini-2.5-Pro 和 GPT-5，在近四分之一的困难情况下也无法保持一致的偏好。我们将此归因于一种称为情境偏好的新现象，这解释了为什么明确的评分标准或标准可以帮助模型在答案对之间做出一致的判断。我们的进一步分析表明，微调LLM作为法官是一种提高绩效的可行方法，基于小组的法官以及深度推理可以增强法官的判断一致性。我们还发现人类判断存在很大的不一致，这表明人类注释可能不是可靠的黄金标准。

Title: Convolutional Lie Operator for Sentence Classification

• Authors: Daniela N. Rim, Heeyoul Choi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16125
• Pdf URL: https://arxiv.org/pdf/2512.16125
• Copy Paste: [[2512.16125]] Convolutional Lie Operator for Sentence Classification(https://arxiv.org/abs/2512.16125)
• Keywords: language model
• Abstract: Traditional Convolutional Neural Networks have been successful in capturing local, position-invariant features in text, but their capacity to model complex transformation within language can be further explored. In this work, we explore a novel approach by integrating Lie Convolutions into Convolutional-based sentence classifiers, inspired by the ability of Lie group operations to capture complex, non-Euclidean symmetries. Our proposed models SCLie and DPCLie empirically outperform traditional Convolutional-based sentence classifiers, suggesting that Lie-based models relatively improve the accuracy by capturing transformations not commonly associated with language. Our findings motivate more exploration of new paradigms in language modeling.
• 摘要：传统的卷积神经网络已经成功地捕获文本中的局部、位置不变的特征，但它们对语言内复杂转换进行建模的能力还可以进一步探索。在这项工作中，我们受到李群运算捕获复杂的非欧几里得对称性的能力的启发，探索了一种新颖的方法，将李卷积集成到基于卷积的句子分类器中。我们提出的模型 SCLie 和 DPCLie 在经验上优于传统的基于卷积的句子分类器，这表明基于 Lie 的模型通过捕获通常与语言不相关的转换来相对提高准确性。我们的发现激发了对语言建模新范式的更多探索。

Title: MRG-R1: Reinforcement Learning for Clinically Aligned Medical Report Generation

• Authors: Pengyu Wang, Shuchang Ye, Usman Naseem, Jinman Kim
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16145
• Pdf URL: https://arxiv.org/pdf/2512.16145
• Copy Paste: [[2512.16145]] MRG-R1: Reinforcement Learning for Clinically Aligned Medical Report Generation(https://arxiv.org/abs/2512.16145)
• Keywords: language model
• Abstract: Medical report generation (MRG) aims to automatically derive radiology-style reports from medical images to aid in clinical decision-making. However, existing methods often generate text that mimics the linguistic style of radiologists but fails to guarantee clinical correctness, because they are trained on token-level objectives which focus on word-choice and sentence structure rather than actual medical accuracy. We propose a semantic-driven reinforcement learning (SRL) method for medical report generation, adopted on a large vision-language model (LVLM). SRL adopts Group Relative Policy Optimization (GRPO) to encourage clinical-correctness-guided learning beyond imitation of language style. Specifically, we optimise a report-level reward: a margin-based cosine similarity (MCCS) computed between key radiological findings extracted from generated and reference reports, thereby directly aligning clinical-label agreement and improving semantic correctness. A lightweight reasoning format constraint further guides the model to generate structured "thinking report" outputs. We evaluate Medical Report Generation with Sematic-driven Reinforment Learning (MRG-R1), on two datasets: IU X-Ray and MIMIC-CXR using clinical efficacy (CE) metrics. MRG-R1 achieves state-of-the-art performance with CE-F1 51.88 on IU X-Ray and 40.39 on MIMIC-CXR. We found that the label-semantic reinforcement is better than conventional token-level supervision. These results indicate that optimizing a clinically grounded, report-level reward rather than token overlap,meaningfully improves clinical correctness. This work is a prior to explore semantic-reinforcement in supervising medical correctness in medical Large vision-language model(Med-LVLM) training.
• 摘要：医学报告生成 (MRG) 旨在从医学图像自动生成放射学风格的报告，以帮助临床决策。然而，现有的方法经常生成模仿放射科医生语言风格的文本，但无法保证临床正确性，因为它们是针对标记级目标进行训练的，这些目标侧重于单词选择和句子结构，而不是实际的医疗准确性。我们提出了一种用于生成医疗报告的语义驱动强化学习（SRL）方法，该方法采用大型视觉语言模型（LVLM）。 SRL 采用组相对策略优化 (GRPO) 来鼓励临床正确性引导的学习，而不仅仅是模仿语言风格。具体来说，我们优化了报告级别的奖励：在从生成的报告和参考报告中提取的关键放射学发现之间计算基于边际的余弦相似度（MCCS），从而直接调整临床标签一致性并提高语义正确性。轻量级推理格式约束进一步指导模型生成结构化的“思维报告”输出。我们使用临床疗效 (CE) 指标在两个数据集：IU X 射线和 MIMIC-CXR 上评估通过语义驱动的强化学习 (MRG-R1) 生成医疗报告。 MRG-R1 实现了最先进的性能，CE-F1 在 IU X 射线上达到 51.88，在 MIMIC-CXR 上达到 40.39。我们发现标签语义强化比传统的令牌级监督更好。这些结果表明，优化基于临床的报告级奖励而不是令牌重叠，可以有意义地提高临床正确性。这项工作是探索语义强化在医学大型视觉语言模型（Med-LVLM）训练中监督医学正确性的先河。

Title: A Domain-Adapted Pipeline for Structured Information Extraction from Police Incident Announcements on Social Media

• Authors: Mengfan Shen, Kangqi Song, Xindi Wang, Wei Jia, Tao Wang, Ziqiang Han
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2512.16183
• Pdf URL: https://arxiv.org/pdf/2512.16183
• Copy Paste: [[2512.16183]] A Domain-Adapted Pipeline for Structured Information Extraction from Police Incident Announcements on Social Media(https://arxiv.org/abs/2512.16183)
• Keywords: prompt
• Abstract: Structured information extraction from police incident announcements is crucial for timely and accurate data processing, yet presents considerable challenges due to the variability and informal nature of textual sources such as social media posts. To address these challenges, we developed a domain-adapted extraction pipeline that leverages targeted prompt engineering with parameter-efficient fine-tuning of the Qwen2.5-7B model using Low-Rank Adaptation (LoRA). This approach enables the model to handle noisy, heterogeneous text while reliably extracting 15 key fields, including location, event characteristics, and impact assessment, from a high-quality, manually annotated dataset of 4,933 instances derived from 27,822 police briefing posts on Chinese Weibo (2019-2020). Experimental results demonstrated that LoRA-based fine-tuning significantly improved performance over both the base and instruction-tuned models, achieving an accuracy exceeding 98.36% for mortality detection and Exact Match Rates of 95.31% for fatality counts and 95.54% for province-level location extraction. The proposed pipeline thus provides a validated and efficient solution for multi-task structured information extraction in specialized domains, offering a practical framework for transforming unstructured text into reliable structured data in social science research.
• 摘要：从警察事件公告中提取结构化信息对于及时、准确的数据处理至关重要，但由于社交媒体帖子等文本来源的可变性和非正式性质，也带来了相当大的挑战。为了应对这些挑战，我们开发了一种适应领域的提取管道，该管道利用有针对性的提示工程，并使用低秩适应 (LoRA) 对 Qwen2.5-7B 模型进行参数高效的微调。这种方法使模型能够处理嘈杂、异构的文本，同时从高质量、手动注释的数据集中可靠地提取 15 个关键字段，包括位置、事件特征和影响评估，该数据集包含来自中国微博上 27,822 条警察简报的 4,933 个实例（2019-2020 年）。实验结果表明，基于 LoRA 的微调显着提高了基础模型和指令调整模型的性能，死亡率检测准确率超过 98.36%，死亡计数精确匹配率达到 95.31%，省级位置提取精确匹配率达到 95.54%。因此，所提出的管道为专业领域的多任务结构化信息提取提供了一种经过验证且有效的解决方案，为社会科学研究中将非结构化文本转换为可靠的结构化数据提供了实用的框架。

Title: Mitigating Hallucinations in Healthcare LLMs with Granular Fact-Checking and Domain-Specific Adaptation

• Authors: Musarrat Zeba, Abdullah Al Mamun, Kishoar Jahan Tithee, Debopom Sutradhar, Mohaimenul Azam Khan Raiaan, Saddam Mukta, Reem E. Mohamed, Md Rafiqul Islam, Yakub Sebastian, Mukhtar Hussain, Sami Azam
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16189
• Pdf URL: https://arxiv.org/pdf/2512.16189
• Copy Paste: [[2512.16189]] Mitigating Hallucinations in Healthcare LLMs with Granular Fact-Checking and Domain-Specific Adaptation(https://arxiv.org/abs/2512.16189)
• Keywords: llm, hallucination
• Abstract: In healthcare, it is essential for any LLM-generated output to be reliable and accurate, particularly in cases involving decision-making and patient safety. However, the outputs are often unreliable in such critical areas due to the risk of hallucinated outputs from the LLMs. To address this issue, we propose a fact-checking module that operates independently of any LLM, along with a domain-specific summarization model designed to minimize hallucination rates. Our model is fine-tuned using Low-Rank Adaptation (LoRa) on the MIMIC III dataset and is paired with the fact-checking module, which uses numerical tests for correctness and logical checks at a granular level through discrete logic in natural language processing (NLP) to validate facts against electronic health records (EHRs). We trained the LLM model on the full MIMIC-III dataset. For evaluation of the fact-checking module, we sampled 104 summaries, extracted them into 3,786 propositions, and used these as facts. The fact-checking module achieves a precision of 0.8904, a recall of 0.8234, and an F1-score of 0.8556. Additionally, the LLM summary model achieves a ROUGE-1 score of 0.5797 and a BERTScore of 0.9120 for summary quality.
• 摘要：在医疗保健领域，法学硕士生成的任何输出都必须可靠且准确，特别是在涉及决策和患者安全的情况下。然而，由于法学硕士存在幻觉输出的风险，这些关键领域的输出通常不可靠。为了解决这个问题，我们提出了一个独立于任何法学硕士运行的事实检查模块，以及旨在最大限度地减少幻觉率的特定领域摘要模型。我们的模型在 MIMIC III 数据集上使用低秩适应 (LoRa) 进行微调，并与事实检查模块配对，该模块通过自然语言处理 (NLP) 中的离散逻辑使用数值测试进行正确性和粒度级别的逻辑检查，以根据电子健康记录 (EHR) 验证事实。我们在完整的 MIMIC-III 数据集上训练了 LLM 模型。为了评估事实检查模块，我们采样了 104 个摘要，将其提取为 3,786 个命题，并将其用作事实。事实检查模块的精度为 0.8904，召回率为 0.8234，F1 分数为 0.8556。此外，LLM 摘要模型的摘要质量 ROUGE-1 得分为 0.5797，BERTScore 为 0.9120。

Title: An Information-Theoretic Framework for Robust Large Language Model Editing

• Authors: Qizhou Chen, Chengyu Wang, Taolin Zhang, Xiaofeng He
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16227
• Pdf URL: https://arxiv.org/pdf/2512.16227
• Copy Paste: [[2512.16227]] An Information-Theoretic Framework for Robust Large Language Model Editing(https://arxiv.org/abs/2512.16227)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have become indispensable tools in science, technology, and society, enabling transformative advances across diverse fields. However, errors or outdated information within these models can undermine their accuracy and restrict their safe deployment. Developing efficient strategies for updating model knowledge without the expense and disruption of full retraining remains a critical challenge. Current model editing techniques frequently struggle to generalize corrections beyond narrow domains, leading to unintended consequences and limiting their practical impact. Here, we introduce a novel framework for editing LLMs, grounded in information bottleneck theory. This approach precisely compresses and isolates the essential information required for generalizable knowledge correction while minimizing disruption to unrelated model behaviors. Building upon this foundation, we present the Information Bottleneck Knowledge Editor (IBKE), which leverages compact latent representations to guide gradient-based updates, enabling robust and broadly applicable model editing. We validate IBKE's effectiveness across multiple LLM architectures and standard benchmark tasks, demonstrating state-of-the-art accuracy and improved generality and specificity of edits. These findings establish a theoretically principled and practical paradigm for open-domain knowledge editing, advancing the utility and trustworthiness of LLMs in real-world applications.
• 摘要：大型语言模型 (LLM) 已成为科学、技术和社会不可或缺的工具，推动了各个领域的变革性进步。然而，这些模型中的错误或过时的信息可能会破坏其准确性并限制其安全部署。制定有效的策略来更新模型知识，而又不产生全面再培训的费用和中断，仍然是一个严峻的挑战。当前的模型编辑技术经常难以将修正推广到狭窄的领域之外，从而导致意想不到的后果并限制其实际影响。在这里，我们介绍了一种基于信息瓶颈理论的编辑法学硕士的新颖框架。这种方法精确地压缩和隔离了可概括的知识校正所需的基本信息，同时最大限度地减少对不相关模型行为的干扰。在此基础上，我们提出了信息瓶颈知识编辑器（IBKE），它利用紧凑的潜在表示来指导基于梯度的更新，从而实现强大且广泛适用的模型编辑。我们验证了 IBKE 在多个 LLM 架构和标准基准测试任务中的有效性，展示了最先进的准确性以及改进的编辑通用性和特异性。这些发现为开放领域知识编辑建立了理论原则和实践范例，提高了法学硕士在现实世界应用中的实用性和可信度。

Title: LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding

• Authors: Chenkai Xu, Yijie Jin, Jiajun Li, Yi Tu, Guoping Long, Dandan Tu, Tianqi Hou, Junchi Yan, Zhijie Deng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16229
• Pdf URL: https://arxiv.org/pdf/2512.16229
• Copy Paste: [[2512.16229]] LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding(https://arxiv.org/abs/2512.16229)
• Keywords: language model, llm
• Abstract: Diffusion Large Language Models (dLLMs) have demonstrated significant potential for high-speed inference. However, current confidence-driven decoding strategies are constrained by limited parallelism, typically achieving only 1--3 tokens per forward pass (TPF). In this work, we identify that the degree of parallelism during dLLM inference is highly sensitive to the Token Filling Order (TFO). Then, we introduce Lookahead PArallel Decoding LoPA, a training-free, plug-and-play algorithm, to identify a superior TFO and hence accelerate inference. LoPA concurrently explores distinct candidate TFOs via parallel branches, and selects the one with the highest potential for future parallelism based on branch confidence. We apply LoPA to the state-of-the-art D2F model and observe a substantial enhancement in decoding efficiency. Notably, LoPA increases the TPF of D2F-Dream to 10.1 on the GSM8K while maintaining performance superior to the Dream baseline. Furthermore, to facilitate this unprecedented degree of parallelism, we develop a specialized multi-device inference system featuring Branch Parallelism (BP), which achieves a single-sample throughput of 1073.9 tokens per second under multi-GPU deployment. The code is available at this https URL.
• 摘要：扩散大型语言模型 (dLLM) 已展现出高速推理的巨大潜力。然而，当前的置信驱动解码策略受到有限并行性的限制，通常每次前向传递 (TPF) 只能实现 1--3 个令牌。在这项工作中，我们发现 dLLM 推理期间的并行度对令牌填充顺序 (TFO) 高度敏感。然后，我们引入了 Lookahead PArallel Decoding LoPA，这是一种免训练、即插即用的算法，用于识别卓越的 TFO，从而加速推理。 LoPA 同时通过并行分支探索不同的候选 TFO，并根据分支置信度选择未来并行性最高潜力的 TFO。我们将 LoPA 应用于最先进的 D2F 模型，并观察到解码效率的显着提高。值得注意的是，LoPA 将 D2F-Dream 在 GSM8K 上的 TPF 提高到 10.1，同时保持优于 Dream 基线的性能。此外，为了实现前所未有的并行度，我们开发了一种专门的多设备推理系统，具有分支并行性（BP），在多 GPU 部署下实现了每秒 1073.9 个令牌的单样本吞吐量。该代码可从此 https URL 获取。

Title: Sigma-Moe-Tiny Technical Report

• Authors: Qingguo Hu, Zhenghao Lin, Ziyue Yang, Yucheng Ding, Xiao Liu, Yuting Jiang, Ruizhe Wang, Tianyu Chen, Zhongxin Guo, Yifan Xiong, Rui Gao, Lei Qu, Jinsong Su, Peng Cheng, Yeyun Gong
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16248
• Pdf URL: https://arxiv.org/pdf/2512.16248
• Copy Paste: [[2512.16248]] Sigma-Moe-Tiny Technical Report(https://arxiv.org/abs/2512.16248)
• Keywords: language model
• Abstract: Mixture-of-Experts (MoE) has emerged as a promising paradigm for foundation models due to its efficient and powerful scalability. In this work, we present Sigma-MoE-Tiny, an MoE language model that achieves the highest sparsity compared to existing open-source models. Sigma-MoE-Tiny employs fine-grained expert segmentation with up to 96 experts per layer, while activating only one expert for each token, resulting in 20B total parameters with just 0.5B activated. The major challenge introduced by such extreme sparsity lies in expert load balancing. We find that the widely-used load balancing loss tends to become ineffective in the lower layers under this setting. To address this issue, we propose a progressive sparsification schedule aiming to balance expert utilization and training stability. Sigma-MoE-Tiny is pre-trained on a diverse and high-quality corpus, followed by post-training to further unlock its capabilities. The entire training process remains remarkably stable, with no occurrence of irrecoverable loss spikes. Comprehensive evaluations reveal that, despite activating only 0.5B parameters, Sigma-MoE-Tiny achieves top-tier performance among counterparts of comparable or significantly larger scale. In addition, we provide an in-depth discussion of load balancing in highly sparse MoE models, offering insights for advancing sparsity in future MoE architectures. Project page: this https URL Code: this https URL
• 摘要：由于其高效且强大的可扩展性，专家混合（MoE）已成为基础模型的一个有前途的范例。在这项工作中，我们提出了 Sigma-MoE-Tiny，这是一种 MoE 语言模型，与现有开源模型相比，它实现了最高的稀疏性。 Sigma-MoE-Tiny 采用细粒度的专家分割，每层最多 96 个专家，同时每个令牌仅激活一名专家，从而仅激活 0.5B，从而产生 20B 的总参数。这种极端稀疏性带来的主要挑战在于专家负载平衡。我们发现，在这种设置下，广泛使用的负载均衡损失在较低层中往往变得无效。为了解决这个问题，我们提出了一个渐进式稀疏化计划，旨在平衡专家利用率和训练稳定性。 Sigma-MoE-Tiny 在多样化且高质量的语料库上进行了预训练，然后进行后训练以进一步释放其功能。整个训练过程保持非常稳定，没有出现不可挽回的损失峰值。综合评估表明，尽管仅激活 0.5B 参数，但 Sigma-MoE-Tiny 在同类或规模更大的同类产品中实现了顶级性能。此外，我们还深入讨论了高度稀疏的 MoE 模型中的负载平衡，为未来 MoE 架构中的稀疏性发展提供了见解。项目页面：此 https URL 代码：此 https URL

Title: Evaluating OpenAI GPT Models for Translation of Endangered Uralic Languages: A Comparison of Reasoning and Non-Reasoning Architectures

• Authors: Yehor Tereshchenko, Mika Hämäläinen, Svitlana Myroniuk
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16287
• Pdf URL: https://arxiv.org/pdf/2512.16287
• Copy Paste: [[2512.16287]] Evaluating OpenAI GPT Models for Translation of Endangered Uralic Languages: A Comparison of Reasoning and Non-Reasoning Architectures(https://arxiv.org/abs/2512.16287)
• Keywords: language model, gpt, llm
• Abstract: The evaluation of Large Language Models (LLMs) for translation tasks has primarily focused on high-resource languages, leaving a significant gap in understanding their performance on low-resource and endangered languages. This study presents a comprehensive comparison of OpenAI's GPT models, specifically examining the differences between reasoning and non-reasoning architectures for translating between Finnish and four low-resource Uralic languages: Komi-Zyrian, Moksha, Erzya, and Udmurt. Using a parallel corpus of literary texts, we evaluate model willingness to attempt translation through refusal rate analysis across different model architectures. Our findings reveal significant performance variations between reasoning and non-reasoning models, with reasoning models showing 16 percentage points lower refusal rates. The results provide valuable insights for researchers and practitioners working with Uralic languages and contribute to the broader understanding of reasoning model capabilities for endangered language preservation.
• 摘要：用于翻译任务的大型语言模型 (LLM) 的评估主要集中在高资源语言上，在了解其在低资源和濒危语言上的表现方面存在很大差距。本研究对 OpenAI 的 GPT 模型进行了全面比较，特别研究了芬兰语和四种低资源乌拉尔语言（Komi-Zyrian、Moksha、Erzya 和 Udmurt）之间翻译的推理架构和非推理架构之间的差异。使用文学文本的平行语料库，我们通过不同模型架构的拒绝率分析来评估模型尝试翻译的意愿。我们的研究结果表明，推理模型和非推理模型之间存在显着的性能差异，推理模型的拒绝率降低了 16 个百分点。研究结果为研究乌拉尔语言的研究人员和从业者提供了宝贵的见解，并有助于更广泛地理解濒危语言保护的推理模型能力。

Title: Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs

• Authors: Sara Papi, Javier Garcia Gilabert, Zachary Hopton, Vilém Zouhar, Carlos Escolano, Gerard I. Gállego, Jorge Iranzo-Sánchez, Ahrii Kim, Dominik Macháček, Patricia Schmidtova, Maike Züfle
• Subjects: cs.CL, cs.AI, cs.SD
• Abstract URL: https://arxiv.org/abs/2512.16378
• Pdf URL: https://arxiv.org/pdf/2512.16378
• Copy Paste: [[2512.16378]] Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs(https://arxiv.org/abs/2512.16378)
• Keywords: language model, llm
• Abstract: As Large Language Models (LLMs) expand beyond text, integrating speech as a native modality has given rise to SpeechLLMs, which aim to translate spoken language directly, thereby bypassing traditional transcription-based pipelines. Whether this integration improves speech-to-text translation quality over established cascaded architectures, however, remains an open question. We present Hearing to Translate, the first comprehensive test suite rigorously benchmarking 5 state-of-the-art SpeechLLMs against 16 strong direct and cascade systems that couple leading speech foundation models (SFM), with multilingual LLMs. Our analysis spans 16 benchmarks, 13 language pairs, and 9 challenging conditions, including disfluent, noisy, and long-form speech. Across this extensive evaluation, we find that cascaded systems remain the most reliable overall, while current SpeechLLMs only match cascades in selected settings and SFMs lag behind both, highlighting that integrating an LLM, either within the model or in a pipeline, is essential for high-quality speech translation.
• 摘要：随着大型语言模型 (LLM) 扩展到文本之外，将语音集成为本地模式催生了 SpeechLLM，其目标是直接翻译口语，从而绕过传统的基于转录的管道。然而，这种集成是否比已建立的级联架构提高了语音到文本的翻译质量，仍然是一个悬而未决的问题。我们推出了 Hearing to Translate，这是第一个综合测试套件，对 5 个最先进的 SpeechLLM 与 16 个强大的直接和级联系统进行了严格的基准测试，这些系统将领先的语音基础模型 (SFM) 与多语言 LLM 结合起来。我们的分析涵盖 16 个基准、13 个语言对和 9 个具有挑战性的条件，包括不流利、嘈杂和长篇语音。在这次广泛的评估中，我们发现级联系统总体上仍然是最可靠的，而当前的 SpeechLLM 仅在选定的设置中匹配级联，而 SFM 落后于两者，这突显了在模型内或管道中集成 LLM 对于高质量语音翻译至关重要。

Title: Plain language adaptations of biomedical text using LLMs: Comparision of evaluation metrics

• Authors: Primoz Kocbek, Leon Kopitar, Gregor Stiglic
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16530
• Pdf URL: https://arxiv.org/pdf/2512.16530
• Copy Paste: [[2512.16530]] Plain language adaptations of biomedical text using LLMs: Comparision of evaluation metrics(https://arxiv.org/abs/2512.16530)
• Keywords: language model, gpt, llm, prompt, agent
• Abstract: This study investigated the application of Large Language Models (LLMs) for simplifying biomedical texts to enhance health literacy. Using a public dataset, which included plain language adaptations of biomedical abstracts, we developed and evaluated several approaches, specifically a baseline approach using a prompt template, a two AI agent approach, and a fine-tuning approach. We selected OpenAI gpt-4o and gpt-4o mini models as baselines for further research. We evaluated our approaches with quantitative metrics, such as Flesch-Kincaid grade level, SMOG Index, SARI, and BERTScore, G-Eval, as well as with qualitative metric, more precisely 5-point Likert scales for simplicity, accuracy, completeness, brevity. Results showed a superior performance of gpt-4o-mini and an underperformance of FT approaches. G-Eval, a LLM based quantitative metric, showed promising results, ranking the approaches similarly as the qualitative metric.
• 摘要：本研究调查了大型语言模型 (LLM) 在简化生物医学文本以提高健康素养方面的应用。使用公共数据集（其中包括生物医学摘要的简单语言改编），我们开发并评估了多种方法，特别是使用提示模板的基线方法、两个人工智能代理方法和微调方法。我们选择 OpenAI gpt-4o 和 gpt-4o mini 模型作为进一步研究的基准。我们使用定量指标（例如 Flesch-Kincaid 等级水平、烟雾指数、SARI 和 BERTScore、G-Eval）以及定性指标（更准确地说是 5 点李克特量表）来评估我们的方法，以实现简单性、准确性、完整性和简洁性。结果显示 gpt-4o-mini 的性能优越，而 FT 方法的性能较差。 G-Eval 是一种基于法学硕士的定量指标，显示出有希望的结果，对方法进行了与定性指标类似的排名。

Title: UM_FHS at the CLEF 2025 SimpleText Track: Comparing No-Context and Fine-Tune Approaches for GPT-4.1 Models in Sentence and Document-Level Text Simplification

• Authors: Primoz Kocbek, Gregor Stiglic
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16541
• Pdf URL: https://arxiv.org/pdf/2512.16541
• Copy Paste: [[2512.16541]] UM_FHS at the CLEF 2025 SimpleText Track: Comparing No-Context and Fine-Tune Approaches for GPT-4.1 Models in Sentence and Document-Level Text Simplification(https://arxiv.org/abs/2512.16541)
• Keywords: gpt, prompt
• Abstract: This work describes our submission to the CLEF 2025 SimpleText track Task 1, addressing both sentenceand document-level simplification of scientific texts. The methodology centered on using the gpt-4.1, gpt-4.1mini, and gpt-4.1-nano models from OpenAI. Two distinct approaches were compared: a no-context method relying on prompt engineering and a fine-tuned (FT) method across models. The gpt-4.1-mini model with no-context demonstrated robust performance at both levels of simplification, while the fine-tuned models showed mixed results, highlighting the complexities of simplifying text at different granularities, where gpt-4.1-nano-ft performance stands out at document-level simplification in one case.
• 摘要：这项工作描述了我们向 CLEF 2025 SimpleText 轨道任务 1 提交的内容，解决科学文本的句子和文档级简化问题。该方法以使用 OpenAI 的 gpt-4.1、gpt-4.1mini 和 gpt-4.1-nano 模型为中心。比较了两种不同的方法：依赖于即时工程的无上下文方法和跨模型的微调（FT）方法。无上下文的 gpt-4.1-mini 模型在两个简化级别上都表现出了强大的性能，而微调后的模型则显示出混合的结果，凸显了在不同粒度上简化文本的复杂性，其中 gpt-4.1-nano-ft 在一种情况下在文档级简化方面表现突出。

Title: Refusal Steering: Fine-grained Control over LLM Refusal Behaviour for Sensitive Topics

• Authors: Iker García-Ferrero, David Montero, Roman Orus
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16602
• Pdf URL: https://arxiv.org/pdf/2512.16602
• Copy Paste: [[2512.16602]] Refusal Steering: Fine-grained Control over LLM Refusal Behaviour for Sensitive Topics(https://arxiv.org/abs/2512.16602)
• Keywords: language model, llm
• Abstract: We introduce Refusal Steering, an inference-time method to exercise fine-grained control over Large Language Models refusal behaviour on politically sensitive topics without retraining. We replace fragile pattern-based refusal detection with an LLM-as-a-judge that assigns refusal confidence scores and we propose a ridge-regularized variant to compute steering vectors that better isolate the refusal--compliance direction. On Qwen3-Next-80B-A3B-Thinking, our method removes the refusal behaviour of the model around politically sensitive topics while maintaining safety on JailbreakBench and near-baseline performance on general benchmarks. The approach generalizes across 4B and 80B models and can also induce targeted refusals when desired. We analize the steering vectors and show that refusal signals concentrate in deeper layers of the transformer and are distributed across many dimensions. Together, these results demonstrate that activation steering can remove political refusal behaviour while retaining safety alignment for harmful content, offering a practical path to controllable, transparent moderation at inference time.
• 摘要：我们引入了拒绝引导，这是一种推理时间方法，可以对大型语言模型在政治敏感主题上的拒绝行为进行细粒度控制，而无需重新训练。我们将脆弱的基于模式的拒绝检测替换为法学硕士作为法官，分配拒绝置信度分数，并提出一种岭正则化变体来计算更好地隔离拒绝合规方向的转向向量。在 Qwen3-Next-80B-A3B-Thinking 上，我们的方法消除了模型围绕政治敏感主题的拒绝行为，同时保持 JailbreakBench 上的安全性和一般基准上接近基线的性能。该方法适用于 4B 和 80B 模型，并且还可以在需要时诱导有针对性的拒绝。我们分析了转向向量，并表明拒绝信号集中在变压器的更深层，并且分布在多个维度上。总之，这些结果表明，激活引导可以消除政治拒绝行为，同时保留有害内容的安全一致性，为推理时可控、透明的调节提供了一条实用途径。

Title: JustRL: Scaling a 1.5B LLM with a Simple RL Recipe

• Authors: Bingxiang He, Zekai Qu, Zeyuan Liu, Yinghao Chen, Yuxin Zuo, Cheng Qian, Kaiyan Zhang, Weize Chen, Chaojun Xiao, Ganqu Cui, Ning Ding, Zhiyuan Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16649
• Pdf URL: https://arxiv.org/pdf/2512.16649
• Copy Paste: [[2512.16649]] JustRL: Scaling a 1.5B LLM with a Simple RL Recipe(https://arxiv.org/abs/2512.16649)
• Keywords: language model, llm
• Abstract: Recent advances in reinforcement learning for large language models have converged on increasing complexity: multi-stage training pipelines, dynamic hyperparameter schedules, and curriculum learning strategies. This raises a fundamental question: \textbf{Is this complexity necessary?} We present \textbf{JustRL}, a minimal approach using single-stage training with fixed hyperparameters that achieves state-of-the-art performance on two 1.5B reasoning models (54.9\% and 64.3\% average accuracy across nine mathematical benchmarks) while using 2$\times$ less compute than sophisticated approaches. The same hyperparameters transfer across both models without tuning, and training exhibits smooth, monotonic improvement over 4,000+ steps without the collapses or plateaus that typically motivate interventions. Critically, ablations reveal that adding ``standard tricks'' like explicit length penalties and robust verifiers may degrade performance by collapsing exploration. These results suggest that the field may be adding complexity to solve problems that disappear with a stable, scaled-up baseline. We release our models and code to establish a simple, validated baseline for the community.
• 摘要：大型语言模型强化学习的最新进展集中在日益增加的复杂性上：多阶段训练管道、动态超参数计划和课程学习策略。这就提出了一个基本问题：\textbf{这种复杂性是否必要？}我们提出\textbf{JustRL}，这是一种使用固定超参数的单阶段训练的最小方法，它在两个 1.5B 推理模型上实现了最先进的性能（九个数学基准的平均准确度分别为 54.9% 和 64.3%），同时使用的计算量比复杂方法少 2 倍。相同的超参数无需调整即可在两个模型之间传输，并且训练在 4,000 多个步骤中表现出平滑、单调的改进，而不会出现通常会引发干预的崩溃或停滞状态。至关重要的是，消融表明添加“标准技巧”（例如显式长度惩罚和强大的验证器）可能会因破坏探索而降低性能。这些结果表明，该领域可能会增加复杂性来解决那些通过稳定、扩大的基线而消失的问题。我们发布模型和代码，为社区建立一个简单且经过验证的基线。

Title: GinSign: Grounding Natural Language Into System Signatures for Temporal Logic Translation

• Authors: William English, Chase Walker, Dominic Simon, Rickard Ewetz
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16770
• Pdf URL: https://arxiv.org/pdf/2512.16770
• Copy Paste: [[2512.16770]] GinSign: Grounding Natural Language Into System Signatures for Temporal Logic Translation(https://arxiv.org/abs/2512.16770)
• Keywords: language model, llm
• Abstract: Natural language (NL) to temporal logic (TL) translation enables engineers to specify, verify, and enforce system behaviors without manually crafting formal specifications-an essential capability for building trustworthy autonomous systems. While existing NL-to-TL translation frameworks have demonstrated encouraging initial results, these systems either explicitly assume access to accurate atom grounding or suffer from low grounded translation accuracy. In this paper, we propose a framework for Grounding Natural Language Into System Signatures for Temporal Logic translation called GinSign. The framework introduces a grounding model that learns the abstract task of mapping NL spans onto a given system signature: given a lifted NL specification and a system signature $\mathcal{S}$, the classifier must assign each lifted atomic proposition to an element of the set of signature-defined atoms $\mathcal{P}$. We decompose the grounding task hierarchically- first predicting predicate labels, then selecting the appropriately typed constant arguments. Decomposing this task from a free-form generation problem into a structured classification problem permits the use of smaller masked language models and eliminates the reliance on expensive LLMs. Experiments across multiple domains show that frameworks which omit grounding tend to produce syntactically correct lifted LTL that is semantically nonequivalent to grounded target expressions, whereas our framework supports downstream model checking and achieves grounded logical-equivalence scores of $95.5\%$, a $1.4\times$ improvement over SOTA.
• 摘要：自然语言 (NL) 到时态逻辑 (TL) 的转换使工程师能够指定、验证和强制执行系统行为，而无需手动制定正式规范，这是构建值得信赖的自治系统的基本功能。虽然现有的 NL 到 TL 翻译框架已经证明了令人鼓舞的初步结果，但这些系统要么明确假设可以访问准确的原子基础，要么遭受低基础翻译准确性的困扰。在本文中，我们提出了一个将自然语言融入系统签名以进行时态逻辑翻译的框架，称为 GinSign。该框架引入了一个基础模型，该模型学习将 NL 跨度映射到给定系统签名的抽象任务：给定提升的 NL 规范和系统签名 $\mathcal{S}$，分类器必须将每个提升的原子命题分配给签名定义的原子 $\mathcal{P}$ 集合中的一个元素。我们分层分解基础任务 - 首先预测谓词标签，然后选择适当类型的常量参数。将该任务从自由形式生成问题分解为结构化分类问题，允许使用更小的掩码语言模型，并消除对昂贵的法学硕士的依赖。跨多个领域的实验表明，省略基础的框架往往会产生语法正确的提升 LTL，在语义上与基础目标表达式不等价，而我们的框架支持下游模型检查并实现基础逻辑等价分数为 95.5\%$，比 SOTA 提高了 1.4\times$。

Title: From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs

• Authors: Shubham Mishra, Samyek Jain, Gorang Mehrishi, Shiv Tiwari, Harsh Sharma, Pratik Narang, Dhruv Kumar
• Subjects: cs.CL, cs.AI, cs.CY, cs.IR
• Abstract URL: https://arxiv.org/abs/2512.16795
• Pdf URL: https://arxiv.org/pdf/2512.16795
• Copy Paste: [[2512.16795]] From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs(https://arxiv.org/abs/2512.16795)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) grounds large language models (LLMs) in external evidence, but fails when retrieved sources conflict or contain outdated or subjective information. Prior work address these issues independently but lack unified reasoning supervision. We propose a reasoning-trace-augmented RAG framework that adds structured, interpretable reasoning across three stages : (1) document-level adjudication, (2) conflict analysis, and (3) grounded synthesis, producing citation-linked answers or justified refusals. A Conflict-Aware Trust-Score (CATS) pipeline is introduced which evaluates groundedness, factual correctness, refusal accuracy, and conflict-behavior alignment using an LLM-as-a-Judge. Our 539-query reasoning dataset and evaluation pipeline establish a foundation for conflict-aware, interpretable RAG systems. Experimental results demonstrate substantial gains over baselines, most notably with Qwen, where Supervised Fine-Tuning improved End-to-End answer correctness from 0.069 to 0.883 and behavioral adherence from 0.074 to 0.722.
• 摘要：检索增强生成 (RAG) 将大型语言模型 (LLM) 置于外部证据中，但当检索到的来源发生冲突或包含过时或主观信息时，就会失败。先前的工作独立解决这些问题，但缺乏统一的推理监督。我们提出了一个推理跟踪增强的 RAG 框架，该框架在三个阶段添加了结构化、可解释的推理：(1) 文档级裁决，(2) 冲突分析，(3) 扎根综合，产生与引用相关的答案或合理的拒绝。引入了冲突感知信任评分 (CATS) 管道，该管道使用法学硕士作为法官来评估基础性、事实正确性、拒绝准确性和冲突行为一致性。我们的 539 个查询推理数据集和评估管道为冲突感知、可解释的 RAG 系统奠定了基础。实验结果表明，与基线相比取得了显着的进步，尤其是 Qwen，监督微调将端到端答案正确性从 0.069 提高到 0.883，将行为依从性从 0.074 提高到 0.722。

Title: Exploration of Augmentation Strategies in Multi-modal Retrieval-Augmented Generation for the Biomedical Domain: A Case Study Evaluating Question Answering in Glycobiology

• Authors: Primož Kocbek, Azra Frkatović-Hodžić, Dora Lalić, Vivian Hui, Gordan Lauc, Gregor Štiglic
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16802
• Pdf URL: https://arxiv.org/pdf/2512.16802
• Copy Paste: [[2512.16802]] Exploration of Augmentation Strategies in Multi-modal Retrieval-Augmented Generation for the Biomedical Domain: A Case Study Evaluating Question Answering in Glycobiology(https://arxiv.org/abs/2512.16802)
• Keywords: gpt, retrieval-augmented generation
• Abstract: Multi-modal retrieval-augmented generation (MM-RAG) promises grounded biomedical QA, but it is unclear when to (i) convert figures/tables into text versus (ii) use optical character recognition (OCR)-free visual retrieval that returns page images and leaves interpretation to the generator. We study this trade-off in glycobiology, a visually dense domain. We built a benchmark of 120 multiple-choice questions (MCQs) from 25 papers, stratified by retrieval difficulty (easy text, medium figures/tables, hard cross-evidence). We implemented four augmentations-None, Text RAG, Multi-modal conversion, and late-interaction visual retrieval (ColPali)-using Docling parsing and Qdrant indexing. We evaluated mid-size open-source and frontier proprietary models (e.g., Gemma-3-27B-IT, GPT-4o family). Additional testing used the GPT-5 family and multiple visual retrievers (ColPali/ColQwen/ColFlor). Accuracy with Agresti-Coull 95% confidence intervals (CIs) was computed over 5 runs per configuration. With Gemma-3-27B-IT, Text and Multi-modal augmentation outperformed OCR-free retrieval (0.722-0.740 vs. 0.510 average accuracy). With GPT-4o, Multi-modal achieved 0.808, with Text 0.782 and ColPali 0.745 close behind; within-model differences were small. In follow-on experiments with the GPT-5 family, the best results with ColPali and ColFlor improved by ~2% to 0.828 in both cases. In general, across the GPT-5 family, ColPali, ColQwen, and ColFlor were statistically indistinguishable. GPT-5-nano trailed larger GPT-5 variants by roughly 8-10%. Pipeline choice is capacity-dependent: converting visuals to text lowers the reader burden and is more reliable for mid-size models, whereas OCR-free visual retrieval becomes competitive under frontier models. Among retrievers, ColFlor offers parity with heavier options at a smaller footprint, making it an efficient default when strong generators are available.
• 摘要：多模态检索增强生成 (MM-RAG) 有望实现扎根的生物医学 QA，但尚不清楚何时 (i) 将图形/表格转换为文本，还是 (ii) 使用无光学字符识别 (OCR) 的视觉检索，返回页面图像并将解释留给生成器。我们在糖生物学这个视觉密集的领域研究这种权衡。我们从 25 篇论文中建立了 120 个多项选择题 (MCQ) 的基准，并按检索难度（简单的文本、中等的图形/表格、困难的交叉证据）进行分层。我们使用 Docling 解析和 Qdrant 索引实现了四种增强功能：无、文本 RAG、多模态转换和后期交互视觉检索 (ColPali)。我们评估了中型开源和前沿专有模型（例如 Gemma-3-27B-IT、GPT-4o 系列）。其他测试使用了 GPT-5 系列和多个视觉检索器 (ColPali/ColQwen/ColFlor)。 Agresti-Coull 95% 置信区间 (CI) 的准确度是在每个配置的 5 次运行中计算得出的。使用 Gemma-3-27B-IT，文本和多模式增强的性能优于无 OCR 检索（平均准确度为 0.722-0.740 与 0.510）。在 GPT-4o 中，Multi-modal 达到了 0.808，Text 0.782 和 ColPali 0.745 紧随其后；模型内差异很小。在 GPT-5 系列的后续实验中，ColPali 和 ColFlor 的最佳结果在两种情况下均提高了约 2% 至 0.828。一般来说，在 GPT-5 家族中，ColPali、ColQwen 和 ColFlor 在统计上没有区别。 GPT-5-nano 落后于更大的 GPT-5 变体大约 8-10%。管道选择取决于容量：将视觉内容转换为文本可以减轻读者的负担，并且对于中型模型来说更可靠，而无需 OCR 的视觉检索在前沿模型下变得具有竞争力。在寻回犬中，ColFlor 以较小的占地面积提供与较重的选择同等的性能，使其成为强大的发电机可用时的高效默认选择。

Title: Grammar-Forced Translation of Natural Language to Temporal Logic using LLMs

• Authors: William English, Dominic Simon, Sumit Kumar Jha, Rickard Ewetz
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16814
• Pdf URL: https://arxiv.org/pdf/2512.16814
• Copy Paste: [[2512.16814]] Grammar-Forced Translation of Natural Language to Temporal Logic using LLMs(https://arxiv.org/abs/2512.16814)
• Keywords: language model, llm
• Abstract: Translating natural language (NL) into a formal language such as temporal logic (TL) is integral for human communication with robots and autonomous systems. State-of-the-art approaches decompose the task into a lifting of atomic propositions (APs) phase and a translation phase. However, existing methods struggle with accurate lifting, the existence of co-references, and learning from limited data. In this paper, we propose a framework for NL to TL translation called Grammar Forced Translation (GraFT). The framework is based on the observation that previous work solves both the lifting and translation steps by letting a language model iteratively predict tokens from its full vocabulary. In contrast, GraFT reduces the complexity of both tasks by restricting the set of valid output tokens from the full vocabulary to only a handful in each step. The solution space reduction is obtained by exploiting the unique properties of each problem. We also provide a theoretical justification for why the solution space reduction leads to more efficient learning. We evaluate the effectiveness of GraFT using the CW, GLTL, and Navi benchmarks. Compared with state-of-the-art translation approaches, it can be observed that GraFT the end-to-end translation accuracy by 5.49% and out-of-domain translation accuracy by 14.06% on average.
• 摘要：将自然语言 (NL) 翻译成时序逻辑 (TL) 等形式语言是人类与机器人和自治系统进行交流不可或缺的一部分。最先进的方法将任务分解为原子命题（AP）提升阶段和翻译阶段。然而，现有的方法在准确提升、共同引用的存在以及从有限数据中学习方面存在困难。在本文中，我们提出了一种称为语法强制翻译（GraFT）的 NL 到 TL 翻译框架。该框架基于这样的观察：之前的工作通过让语言模型从其完整词汇中迭代地预测标记来解决提升和翻译步骤。相比之下，GraFT 通过将每个步骤中有效输出标记集从完整词汇表限制为少数几个来降低这两项任务的复杂性。通过利用每个问题的独特属性来减少解决方案空间。我们还提供了为什么解决方案空间减少可以带来更高效学习的理论依据。我们使用 CW、GLTL 和 Navi 基准评估 GraFT 的有效性。与最先进的翻译方法相比，可以观察到 GraFT 的端到端翻译准确率平均提高了 5.49%，域外翻译准确率平均提高了 14.06%。

Title: What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels

• Authors: Aditya Yadavalli, Tiago Pimentel, Tamar I Regev, Ethan Wilcox, Alex Warstadt
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16832
• Pdf URL: https://arxiv.org/pdf/2512.16832
• Copy Paste: [[2512.16832]] What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels(https://arxiv.org/abs/2512.16832)
• Keywords: language model
• Abstract: Prosody -- the melody of speech -- conveys critical information often not captured by the words or text of a message. In this paper, we propose an information-theoretic approach to quantify how much information is expressed by prosody alone and not by text, and crucially, what that information is about. Our approach applies large speech and language models to estimate the mutual information between a particular dimension of an utterance's meaning (e.g., its emotion) and any of its communication channels (e.g., audio or text). We then use this approach to quantify how much information is conveyed by audio and text about sarcasm, emotion, and questionhood, using speech from television and podcasts. We find that for sarcasm and emotion the audio channel -- and by implication the prosodic channel -- transmits over an order of magnitude more information about these features than the text channel alone, at least when long-term context beyond the current sentence is unavailable. For questionhood, prosody provides comparatively less additional information. We conclude by outlining a program applying our approach to more dimensions of meaning, communication channels, and languages.
• 摘要：韵律——演讲的旋律——传达的关键信息通常无法被消息的单词或文本捕获。在本文中，我们提出了一种信息论方法来量化仅由韵律而非文本表达的信息量，以及最重要的是，这些信息是关于什么的。我们的方法应用大型语音和语言模型来估计话语含义的特定维度（例如，其情感）与其任何通信渠道（例如，音频或文本）之间的互信息。然后，我们利用电视和播客中的语音，使用这种方法来量化音频和文本传达了多少关于讽刺、情感和疑问的信息。我们发现，对于讽刺和情感，音频通道（以及隐含的韵律通道）传输的有关这些特征的信息比单独的文本通道多一个数量级，至少在当前句子之外的长期上下文不可用时。对于疑问句，韵律提供的附加信息相对较少。最后，我们概述了一个计划，将我们的方法应用于意义、沟通渠道和语言的更多维度。

Title: LLMCache: Layer-Wise Caching Strategies for Accelerated Reuse in Transformer Inference

• Authors: Harsh Vardhan Bansal
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2512.16843
• Pdf URL: https://arxiv.org/pdf/2512.16843
• Copy Paste: [[2512.16843]] LLMCache: Layer-Wise Caching Strategies for Accelerated Reuse in Transformer Inference(https://arxiv.org/abs/2512.16843)
• Keywords: language model, gpt, llm
• Abstract: Transformer-based language models have achieved remarkable performance across a wide range of tasks, yet their high inference latency poses a significant challenge for real-timeand large-scale deployment. While existing caching mechanisms,such as token-level key-value caches, offer speedups in autore-gressive decoding, they are limited in scope and applicability. In this paper, we present LLMCache, a novel layer-wise caching framework that accelerates transformer inference by reusing intermediate activations based on semantic similarity of input sequences. Unlike prior work, LLMCache is model-agnostic,operates across both encoder and decoder architectures, and supports caching at arbitrary transformer layers. We introduce a lightweight fingerprinting mechanism for matching seman-tically similar inputs and propose adaptive eviction strategies to manage cache staleness. Experiments on BERT and GPT-2 across SQuAD, WikiText-103, and OpenBookQA show up to 3.1 X speedup in inference time with <0.5% accuracy degradation. Our results highlight LLMCache as a practical and general-purpose solution for optimizing transformer inference in real-world applications
• 摘要：基于 Transformer 的语言模型在广泛的任务中取得了卓越的性能，但其高推理延迟对实时和大规模部署提出了重大挑战。虽然现有的缓存机制（例如令牌级键值缓存）可以提高自回归解码的速度，但它们的范围和适用性受到限制。在本文中，我们提出了 LLMCache，这是一种新颖的分层缓存框架，它通过重用基于输入序列语义相似性的中间激活来加速转换器推理。与之前的工作不同，LLMCache 与模型无关，可跨编码器和解码器架构运行，并支持在任意转换器层进行缓存。我们引入了一种轻量级指纹识别机制来匹配语义相似的输入，并提出了自适应驱逐策略来管理缓存陈旧性。在 SQuAD、WikiText-103 和 OpenBookQA 上对 BERT 和 GPT-2 进行的实验显示，推理时间加速高达 3.1 倍，而准确率下降 <0.5%。我们的结果凸显了 LLMCache 作为一种实用且通用的解决方案，可用于优化实际应用中的变压器推理

Title: AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning

• Authors: Tzu-Han Lin, Wei-Lin Chen, Chen-An Li, Hung-yi Lee, Yun-Nung Chen, Yu Meng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16883
• Pdf URL: https://arxiv.org/pdf/2512.16883
• Copy Paste: [[2512.16883]] AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning(https://arxiv.org/abs/2512.16883)
• Keywords: language model, llm, hallucination, agent
• Abstract: Equipping large language models (LLMs) with search engines via reinforcement learning (RL) has emerged as an effective approach for building search agents. However, overreliance on search introduces unnecessary cost and risks exposure to noisy or malicious content, while relying solely on parametric knowledge risks hallucination. The central challenge is to develop agents that adaptively balance parametric knowledge with external search, invoking search only when necessary. Prior work mitigates search overuse by shaping rewards around the number of tool calls. However, these penalties require substantial reward engineering, provide ambiguous credit assignment, and can be exploited by agents that superficially reduce calls. Moreover, evaluating performance solely through call counts conflates necessary and unnecessary search, obscuring the measurement of true adaptive behavior. To address these limitations, we first quantify the self-knowledge awareness of existing search agents via an F1-based decision metric, revealing that methods such as Search-R1 often overlook readily available parametric knowledge. Motivated by these findings, we propose AdaSearch, a simple two-stage, outcome-driven RL framework that disentangles problem solving from the decision of whether to invoke search, and makes this decision process explicit and interpretable. This transparency is crucial for high-stakes domains such as finance and medical question answering, yet is largely neglected by prior approaches. Experiments across multiple model families and sizes demonstrate that AdaSearch substantially improves knowledge-boundary awareness, reduces unnecessary search calls, preserves strong task performance, and offers more transparent, interpretable decision behaviors.
• 摘要：通过强化学习（RL）为大型语言模型（LLM）配备搜索引擎已成为构建搜索代理的有效方法。然而，过度依赖搜索会带来不必要的成本，并面临暴露于嘈杂或恶意内容的风险，而仅仅依赖参数知识则面临产生幻觉的风险。核心挑战是开发能够自适应地平衡参数知识与外部搜索的代理，仅在必要时调用搜索。之前的工作通过围绕工具调用数量制定奖励来缓解搜索过度使用。然而，这些惩罚需要大量的奖励工程，提供不明确的信用分配，并且可以被表面上减少呼叫的代理利用。此外，仅通过调用计数来评估性能会混淆必要和不必要的搜索，从而模糊了对真正自适应行为的测量。为了解决这些限制，我们首先通过基于 F1 的决策指标量化现有搜索代理的自我知识意识，揭示 Search-R1 等方法经常忽略现成的参数知识。受这些发现的启发，我们提出了 AdaSearch，这是一个简单的两阶段、结果驱动的 RL 框架，它将问题解决与是否调用搜索的决策分开，并使该决策过程变得明确且可解释。这种透明度对于金融和医疗问答等高风险领域至关重要，但在很大程度上被以前的方法所忽视。跨多个模型系列和规模的实验表明，AdaSearch 显着提高了知识边界意识，减少了不必要的搜索调用，保持了强大的任务性能，并提供了更透明、可解释的决策行为。

Title: Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image

• Authors: Yushi Hu, Reyhane Askari-Hemmat, Melissa Hall, Emily Dinan, Luke Zettlemoyer, Marjan Ghazvininejad
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2512.16899
• Pdf URL: https://arxiv.org/pdf/2512.16899
• Copy Paste: [[2512.16899]] Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image(https://arxiv.org/abs/2512.16899)
• Keywords: language model, gpt, llm, prompt, agent
• Abstract: Reward models (RMs) are essential for training large language models (LLMs), but remain underexplored for omni models that handle interleaved image and text sequences. We introduce Multimodal RewardBench 2 (MMRB2), the first comprehensive benchmark for reward models on multimodal understanding and (interleaved) generation. MMRB2 spans four tasks: text-to-image, image editing, interleaved generation, and multimodal reasoning ("thinking-with-images"), providing 1,000 expert-annotated preference pairs per task from 23 models and agents across 21 source tasks. MMRB2 is designed with: (1) practical but challenging prompts; (2) responses from state-of-the-art models and agents; and (3) preference pairs with strong human-expert consensus, curated via an ensemble filtering strategy. Using MMRB2, we study existing judges for each subtask, including multimodal LLM-as-a-judge and models trained with human preferences. The latest Gemini 3 Pro attains 75-80% accuracy. GPT-5 and Gemini 2.5 Pro reach 66-75% accuracy, compared to >90% for humans, yet surpass the widely used GPT-4o (59%). The best performing open-source model Qwen3-VL-32B achieves similar accuracies as Gemini 2.5 Flash (64%). We also show that MMRB2 performance strongly correlates with downstream task success using Best-of-N sampling and conduct an in-depth analysis that shows key areas to improve the reward models going forward.
• 摘要：奖励模型 (RM) 对于训练大型语言模型 (LLM) 至关重要，但对于处理交错图像和文本序列的全向模型仍有待探索。我们推出了 Multimodal RewardBench 2 (MMRB2)，这是第一个针对多模态理解和（交错）生成的奖励模型的综合基准。 MMRB2 涵盖四项任务：文本到图像、图像编辑、交错生成和多模态推理（“图像思考”），为每个任务提供来自 21 个源任务的 23 个模型和代理的 1,000 个专家注释的偏好对。 MMRB2的设计具有：（1）实用但具有挑战性的提示； (2) 来自最先进模型和代理的响应； （3）通过集成过滤策略策划的具有强烈人类专家共识的偏好对。使用 MMRB2，我们研究每个子任务的现有法官，包括作为法官的多模式法学硕士和根据人类偏好训练的模型。最新的 Gemini 3 Pro 的准确率达到 75-80%。 GPT-5 和 Gemini 2.5 Pro 的准确率达到 66-75%，而人类的准确率则超过 90%，但仍超过了广泛使用的 GPT-4o (59%)。性能最佳的开源模型 Qwen3-VL-32B 达到了与 Gemini 2.5 Flash 相似的精度 (64%)。我们还使用 Best-of-N 采样证明 MMRB2 性能与下游任务成功密切相关，并进行深入分析，显示未来改进奖励模型的关键领域。

Title: Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updates

• Authors: Nikhil Prakash, Donghao Ren, Dominik Moritz, Yannick Assogba
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2512.16914
• Pdf URL: https://arxiv.org/pdf/2512.16914
• Copy Paste: [[2512.16914]] Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updates(https://arxiv.org/abs/2512.16914)
• Keywords: llm
• Abstract: Prior studies investigating the internal workings of LLMs have uncovered sparse subnetworks, often referred to as circuits, that are responsible for performing specific tasks. Additionally, it has been shown that model performance improvement through fine-tuning often results from the strengthening of existing circuits in the model. Taken together, these findings suggest the possibility of intervening directly on such circuits to make precise, task-targeted updates. Motivated by these findings, we propose a novel method called Constructive Circuit Amplification which identifies pivotal tokens from model reasoning traces as well as model components responsible for the desired task, and updates only those components. Applied to mathematical reasoning, it improves accuracy by up to +11.4% across multiple models while modifying as little as 1.59% of model components, with minimal impact on other abilities as measured by MMLU, TriviaQA, and TruthfulQA. These results demonstrate that targeted capabilities can be reliably enhanced by selectively updating a sparse set of model components.
• 摘要：先前研究法学硕士的内部运作已经发现了稀疏子网络，通常称为电路，负责执行特定任务。此外，研究表明，通过微调来提高模型性能通常来自于模型中现有电路的强化。总而言之，这些发现表明直接干预此类电路以进行精确的、针对任务的更新的可能性。受这些发现的启发，我们提出了一种称为构造性电路放大的新方法，该方法从模型推理轨迹中识别关键标记以及负责所需任务的模型组件，并仅更新这些组件。应用于数学推理时，它可以将多个模型的准确性提高高达 +11.4%，同时修改的模型组件少至 1.59%，并且对 MMLU、TriviaQA 和 TruthfulQA 衡量的其他能力的影响最小。这些结果表明，通过有选择地更新一组稀疏的模型组件，可以可靠地增强目标功能。