2025-08-21

Title: From Image Captioning to Visual Storytelling

• Authors: Admitos Passadakis, Yingjin Song, Albert Gatt
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2508.14045
• Pdf URL: https://arxiv.org/pdf/2508.14045
• Copy Paste: [[2508.14045]] From Image Captioning to Visual Storytelling(https://arxiv.org/abs/2508.14045)
• Keywords: language model
• Abstract: Visual Storytelling is a challenging multimodal task between Vision & Language, where the purpose is to generate a story for a stream of images. Its difficulty lies on the fact that the story should be both grounded to the image sequence but also narrative and coherent. The aim of this work is to balance between these aspects, by treating Visual Storytelling as a superset of Image Captioning, an approach quite different compared to most of prior relevant studies. This means that we firstly employ a vision-to-language model for obtaining captions of the input images, and then, these captions are transformed into coherent narratives using language-to-language methods. Our multifarious evaluation shows that integrating captioning and storytelling under a unified framework, has a positive impact on the quality of the produced stories. In addition, compared to numerous previous studies, this approach accelerates training time and makes our framework readily reusable and reproducible by anyone interested. Lastly, we propose a new metric/tool, named ideality, that can be used to simulate how far some results are from an oracle model, and we apply it to emulate human-likeness in visual storytelling.
• 摘要：视觉讲故事是视觉和语言之间一项具有挑战性的多模式任务，目的是为图像流生成故事。它的困难在于，故事既应该基于图像序列，又要叙事和连贯。这项工作的目的是通过将视觉故事讲述视为图像字幕的超集，与大多数先前的相关研究相比，这种方法完全不同。这意味着我们首先采用视觉到语言模型来获取输入图像的标题，然后使用语言到语言方法将这些字幕转换为连贯的叙述。我们的多种评估表明，在统一框架下集成字幕和讲故事，对生产故事的质量产生积极影响。此外，与以前的许多研究相比，这种方法加速了培训时间，并使我们的框架很容易被感兴趣的人重复使用和重现。最后，我们提出了一种名为Iseality的新指标/工具，该工具可用于模拟与Oracle模型的一些结果，并将其应用于视觉讲故事中的人类风味。

Title: Benchmarking Sociolinguistic Diversity in Swahili NLP: A Taxonomy-Guided Approach

• Authors: Kezia Oketch, John P. Lalor, Ahmed Abbasi
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2508.14051
• Pdf URL: https://arxiv.org/pdf/2508.14051
• Copy Paste: [[2508.14051]] Benchmarking Sociolinguistic Diversity in Swahili NLP: A Taxonomy-Guided Approach(https://arxiv.org/abs/2508.14051)
• Keywords: language model
• Abstract: We introduce the first taxonomy-guided evaluation of Swahili NLP, addressing gaps in sociolinguistic diversity. Drawing on health-related psychometric tasks, we collect a dataset of 2,170 free-text responses from Kenyan speakers. The data exhibits tribal influences, urban vernacular, code-mixing, and loanwords. We develop a structured taxonomy and use it as a lens for examining model prediction errors across pre-trained and instruction-tuned language models. Our findings advance culturally grounded evaluation frameworks and highlight the role of sociolinguistic variation in shaping model performance.
• 摘要：我们介绍了对Swahili NLP的首次分类指导评估，以解决社会语言多样性的差距。利用与健康相关的心理测量任务，我们收集了一个来自肯尼亚语者的2170个自由文本响应的数据集。数据表现出部落影响，城市白话，代码混合和借用词。我们开发了一种结构化的分类法，并将其用作镜头，以检查预先训练和指导调整语言模型的模型预测错误。我们的发现推进了文化上扎根的评估框架，并突出了社会语言变化在塑造模型绩效中的作用。

Title: Contrastive Analysis of Constituent Order Preferences Within Adverbial Roles in English and Chinese News: A Large-Language-Model-Driven Approach

• Authors: Yiran Rex Ma
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14054
• Pdf URL: https://arxiv.org/pdf/2508.14054
• Copy Paste: [[2508.14054]] Contrastive Analysis of Constituent Order Preferences Within Adverbial Roles in English and Chinese News: A Large-Language-Model-Driven Approach(https://arxiv.org/abs/2508.14054)
• Keywords: language model, llm
• Abstract: Based on comparable English-Chinese news corpora annotated by Large Language Model (LLM), this paper attempts to explore the differences in constituent order of English-Chinese news from the perspective of functional chunks with adverbial roles, and analyze their typical positional preferences and distribution patterns. It is found that: (1) English news prefers linear narrative of core information first, and functional chunks are mostly post-positioned, while Chinese news prefers overall presentation mode of background first, and functional chunks are often pre-positioned; (2) In SVO structure, both English and Chinese news show differences in the distribution of functional chunks, but the tendency of Chinese pre-positioning is more significant, while that of English post-positioning is relatively mild; (3) When function blocks are co-occurring, both English and Chinese news show high flexibility, and the order adjustment is driven by information and pragmatic purposes. The study reveals that word order has both systematic preference and dynamic adaptability, providing new empirical support for contrastive study of English-Chinese information structure.
• 摘要：本文基于由大语言模型（LLM）注释的可比较英语新闻库，试图从具有副词角色的功能块的角度探索英语 - 中国新闻的组成顺序差异，并分析其典型的位置偏好和分配模式。发现：（1）英语新闻更喜欢核心信息的线性叙事，并且功能性块主要是后置的，而中国新闻则更喜欢首先的背景呈现模式，并且功能性块通常是预位； （2）在SVO结构中，英语和中文的新闻都显示出功能块的分布的差异，但是中国姿势的趋势更为重要，而英语后置位的趋势相对温和。 （3）当功能块共同出现时，英语和中文新闻都显示出很高的灵活性，并且订单调整是由信息和务实目的驱动的。该研究表明，单词顺序既具有系统的偏好又具有动态适应性，从而为英语 - 中国信息结构的对比研究提供了新的经验支持。

Title: T-REX: Table -- Refute or Entail eXplainer

• Authors: Tim Luka Horstmann, Baptiste Geisenberger, Mehwish Alam
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14055
• Pdf URL: https://arxiv.org/pdf/2508.14055
• Copy Paste: [[2508.14055]] T-REX: Table -- Refute or Entail eXplainer(https://arxiv.org/abs/2508.14055)
• Keywords: language model, llm
• Abstract: Verifying textual claims against structured tabular data is a critical yet challenging task in Natural Language Processing with broad real-world impact. While recent advances in Large Language Models (LLMs) have enabled significant progress in table fact-checking, current solutions remain inaccessible to non-experts. We introduce T-REX (T-REX: Table -- Refute or Entail eXplainer), the first live, interactive tool for claim verification over multimodal, multilingual tables using state-of-the-art instruction-tuned reasoning LLMs. Designed for accuracy and transparency, T-REX empowers non-experts by providing access to advanced fact-checking technology. The system is openly available online.
• 摘要：验证针对结构化表格数据的文本主张是具有广泛现实影响的自然语言处理中的一项至关重要的挑战性任务。尽管大型语言模型（LLMS）的最新进展已在表格检查方面取得了重大进展，但当前的解决方案对于非专家来说仍然无法访问。我们介绍了T-Rex（T-Rex：Table-驳斥或Intail Explioner），这是使用最先进的指令调整的推理LLMS对多模式验证的第一个实时交互式工具。 T-Rex专为准确性和透明度而设计，通过提供对先进的事实检查技术的访问来赋予非专家。该系统在线公开可用。

Title: Confidence Estimation for Text-to-SQL in Large Language Models

• Authors: Sepideh Entezari Maleki, Mohammadreza Pourreza, Davood Rafiei
• Subjects: cs.CL, cs.DB
• Abstract URL: https://arxiv.org/abs/2508.14056
• Pdf URL: https://arxiv.org/pdf/2508.14056
• Copy Paste: [[2508.14056]] Confidence Estimation for Text-to-SQL in Large Language Models(https://arxiv.org/abs/2508.14056)
• Keywords: language model, llm
• Abstract: Confidence estimation for text-to-SQL aims to assess the reliability of model-generated SQL queries without having access to gold answers. We study this problem in the context of large language models (LLMs), where access to model weights and gradients is often constrained. We explore both black-box and white-box confidence estimation strategies, evaluating their effectiveness on cross-domain text-to-SQL benchmarks. Our evaluation highlights the superior performance of consistency-based methods among black-box models and the advantage of SQL-syntax-aware approaches for interpreting LLM logits in white-box settings. Furthermore, we show that execution-based grounding of queries provides a valuable supplementary signal, improving the effectiveness of both approaches.
• 摘要：文本到SQL的置信度估算旨在评估模型生成的SQL查询的可靠性，而无需获得黄金答案。我们在大型语言模型（LLMS）的背景下研究此问题，在大型语言模型（LLMS）中，通常会限制使用模型权重和梯度。我们探索黑盒和白盒置信度估计策略，评估它们对跨域文本到SQL基准测试的有效性。我们的评估强调了黑框模型中基于一致性的方法的出色性能以及SQL-Syntax-Awawawawawawawawawawawawawawawawawawawawawawawawawawawawawawawawawawawawawaweawawawawaweawawaweawawe llm logits in White-box设置中的优势。此外，我们表明，基于执行的查询接地提供了有价值的补充信号，从而提高了两种方法的有效性。

Title: Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models

• Authors: Badrinath Ramakrishnan, Akshaya Balaji
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14062
• Pdf URL: https://arxiv.org/pdf/2508.14062
• Copy Paste: [[2508.14062]] Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models(https://arxiv.org/abs/2508.14062)
• Keywords: language model, gpt, llm
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language processing tasks, but their tendency to memorize training data poses significant privacy risks, particularly during fine-tuning processes. This paper presents a comprehensive empirical analysis of data memorization in fine-tuned LLMs and introduces a novel multi-layered privacy protection framework. Through controlled experiments on modern LLM architectures including GPT-2, Phi-3, and Gemma-2, we demonstrate that fine-tuning with repeated sensitive data increases privacy leakage rates from baseline levels of 0-5% to 60-75%, representing a 64.2% average increase across tested models. We propose and rigorously evaluate four complementary privacy protection methods: semantic data deduplication, differential privacy during generation, entropy-based filtering, and pattern-based content filtering. Our experimental results show that these techniques can reduce data leakage to 0% while maintaining 94.7% of original model utility.
• 摘要：大型语言模型（LLM）表现出了各种自然语言处理任务的显着功能，但是他们记住培训数据的趋势会带来很大的隐私风险，尤其是在微调过程中。本文对微调LLM的数据记忆进行了全面的经验分析，并介绍了一种新型的多层隐私保护框架。通过对包括GPT-2，PHI-3和GEMMA-2在内的现代LLM体系结构进行的对照实验，我们证明，重复敏感数据的微调将隐私泄漏率从0-5％的基线水平提高到60-75％，这是测试模型的平均平均值增加64.2％。我们建议并严格评估四种互补的隐私保护方法：语义数据删除，生成期间的差异隐私，基于熵的过滤和基于模式的内容过滤。我们的实验结果表明，这些技术可以将数据泄漏降至0％，同时维持原始模型实用程序的94.7％。

Title: Punctuation and Predicates in Language Models

• Authors: Sonakshi Chauhan, Maheep Chaudhary, Koby Choy, Samuel Nellessen, Nandi Schoots
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.14067
• Pdf URL: https://arxiv.org/pdf/2508.14067
• Copy Paste: [[2508.14067]] Punctuation and Predicates in Language Models(https://arxiv.org/abs/2508.14067)
• Keywords: language model, gpt, llm
• Abstract: In this paper we explore where information is collected and how it is propagated throughout layers in large language models (LLMs). We begin by examining the surprising computational importance of punctuation tokens which previous work has identified as attention sinks and memory aids. Using intervention-based techniques, we evaluate the necessity and sufficiency (for preserving model performance) of punctuation tokens across layers in GPT-2, DeepSeek, and Gemma. Our results show stark model-specific differences: for GPT-2, punctuation is both necessary and sufficient in multiple layers, while this holds far less in DeepSeek and not at all in Gemma. Extending beyond punctuation, we ask whether LLMs process different components of input (e.g., subjects, adjectives, punctuation, full sentences) by forming early static summaries reused across the network, or if the model remains sensitive to changes in these components across layers. Extending beyond punctuation, we investigate whether different reasoning rules are processed differently by LLMs. In particular, through interchange intervention and layer-swapping experiments, we find that conditional statements (if, then), and universal quantification (for all) are processed very differently. Our findings offer new insight into the internal mechanisms of punctuation usage and reasoning in LLMs and have implications for interpretability.
• 摘要：在本文中，我们探讨了收集信息的位置以及如何在大型语言模型（LLMS）的整个层中传播。首先，我们研究了标点令牌的令人惊讶的计算重要性，而标点令牌先前的工作已确定为关注点和记忆辅助工具。使用基于干预的技术，我们评估了GPT-2，DeepSeek和Gemma跨层的标点令牌的必要性和充分性（以保持模型性能）。我们的结果表明了特定于模型的差异：对于GPT-2，标点符号在多层中既需要且足够，而在DeepSeek中，标点符号却少得多，而在Gemma中根本不足。扩展超出标点符号，我们询问LLM是否通过形成整个网络重复使用的早期静态摘要，还是模型对这些组件的变化敏感，是通过形成早期的静态摘要来处理输入的不同组成部分（例如，形容词，标点符号，完整句子）。扩展超出标点符号，我们研究了LLM是否对不同的推理规则进行了不同的处理。特别是，通过互换干预和层交换实验，我们发现有条件的陈述（如果，当时）和通用定量（全部）的处理方式大不相同。我们的发现为LLM中的标点符号使用和推理的内部机制提供了新的见解，并具有对可解释性的影响。

Title: DLLMQuant: Quantizing Diffusion-based Large Language Models

• Authors: Chen Xu, Dawei Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14090
• Pdf URL: https://arxiv.org/pdf/2508.14090
• Copy Paste: [[2508.14090]] DLLMQuant: Quantizing Diffusion-based Large Language Models(https://arxiv.org/abs/2508.14090)
• Keywords: language model, llm
• Abstract: Diffusion-based large language models (DLLMs) have shown promise for non-autoregressive text generation, but their deployment is constrained by large model sizes and heavy computational costs. Post-training quantization (PTQ), a widely used method for compressing and accelerating Large Language Models (LLMs), suffers from severe accuracy degradation and reduced generalization performance when directly applied to DLLMs (e.g., AWQ suffers a 16% accuracy drop on LLADA under W4A4). This paper explores how DLLMs' key mechanisms - dynamic masking, iterative generation, bidirectional attention - clash with quantization. We identify three core issues: 1) Iterative generation and dynamic masking ratios lead to distinct token distributions across decoding steps, which are not adequately captured by existing PTQ calibration methods; 2) Quantization errors are accumulated and amplified progressively during iteration in DLLMs, causing quantized models to perform worse as decoding steps progress; 3) Unmasked tokens stabilize while masked remain probabilistic, making overall feature distribution incompatible with existing PTQ methods. To address these issues, we propose DLLMQuant, a PTQ framework tailored for DLLMs, which incorporates three novel techniques: 1) Temporal-Mask Adaptive Sampling (TMAS), a calibration method that accounts for both time and mask factors, with the capacity to capture distributions across timesteps. 2) Interaction-Aware Activation Quantization (IA-AQ), which utilizes bidirectional attention's interaction signals to dynamically allocate quantization resources. 3) Certainty-Guided Quantization (CGQ), which integrates mask status and token scores as key weighting criteria into error compensation, making weight quantization more suitable for DLLMs. Experiments show that DLLMQuant achieves significant performance gains while enhancing efficiency.
• 摘要：基于扩散的大语言模型（DLLM）已显示出对非自动回归文本生成的希望，但是它们的部署受到大型模型尺寸和繁重的计算成本的限制。训练后量化（PTQ）是一种用于压缩和加速大语言模型（LLM）的广泛使用的方法，当直接应用于DLLMS时，会遭受严重的准确性降解和降低的概括性能（例如，AWQ在W4A4下的LLADA上的精度下降了16％的精度下降）。本文探讨了DLLM的关键机制如何 - 动态掩蔽，迭代产生，双向关注 - 与量化冲突。我们确定了三个核心问题：1）迭代生成和动态掩蔽比导致了跨解码步骤的不同令牌分布，而现有的PTQ校准方法无法充分捕获这些分布； 2）在DLLM中迭代期间累积并逐渐扩大量化误差，导致量化模型的性能随着解码步骤的进展而更糟； 3）揭露的令牌稳定，而蒙版则保持概率，从而使整体特征分布与现有的PTQ方法不相容。为了解决这些问题，我们提出了DLLMQuant，这是一种针对DLLMS量身定制的PTQ框架，其中包含了三种新型技术：1）暂时遮罩自适应采样（TMA），这是一种校准方法，既说明了时间和掩蔽因子，又有能力捕获时间段分布的能力。 2）相互作用感知的激活量化（IA-AQ），它利用双向注意的交互信号动态分配量化资源。 3）确定性引导的量化（CGQ），该量化将蒙版状态和令牌得分集成为关键加权标准误差赔偿，从而使权重量化更适合DLLM。实验表明，dllmquant在提高效率的同时，达到了巨大的性能增长。

Title: MMReview: A Multidisciplinary and Multimodal Benchmark for LLM-Based Peer Review Automation

• Authors: Xian Gao, Jiacheng Ruan, Zongyun Zhang, Jingsheng Gao, Ting Liu, Yuzhuo Fu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14146
• Pdf URL: https://arxiv.org/pdf/2508.14146
• Copy Paste: [[2508.14146]] MMReview: A Multidisciplinary and Multimodal Benchmark for LLM-Based Peer Review Automation(https://arxiv.org/abs/2508.14146)
• Keywords: language model, llm
• Abstract: With the rapid growth of academic publications, peer review has become an essential yet time-consuming responsibility within the research community. Large Language Models (LLMs) have increasingly been adopted to assist in the generation of review comments; however, current LLM-based review tasks lack a unified evaluation benchmark to rigorously assess the models' ability to produce comprehensive, accurate, and human-aligned assessments, particularly in scenarios involving multimodal content such as figures and tables. To address this gap, we propose \textbf{MMReview}, a comprehensive benchmark that spans multiple disciplines and modalities. MMReview includes multimodal content and expert-written review comments for 240 papers across 17 research domains within four major academic disciplines: Artificial Intelligence, Natural Sciences, Engineering Sciences, and Social Sciences. We design a total of 13 tasks grouped into four core categories, aimed at evaluating the performance of LLMs and Multimodal LLMs (MLLMs) in step-wise review generation, outcome formulation, alignment with human preferences, and robustness to adversarial input manipulation. Extensive experiments conducted on 16 open-source models and 5 advanced closed-source models demonstrate the thoroughness of the benchmark. We envision MMReview as a critical step toward establishing a standardized foundation for the development of automated peer review systems.
• 摘要：随着学术出版物的迅速增长，同行评审已成为研究界的重要但耗时的责任。大型语言模型（LLMS）越来越多地被采用以帮助生成评论评论。但是，当前基于LLM的审核任务缺乏统一的评估基准，无法严格评估模型产生全面，准确和人类一致的评估的能力，尤其是在涉及多模式内容（例如图形和表）的情况下。为了解决此差距，我们建议\ textbf {mmreview}，这是一个跨越多个学科和模式的综合基准。 MMReview包括在四个主要的学术学科中的17个研究领域的240篇论文的多模式内容和专家写的评论：人工智能，自然科学，工程科学和社会科学。我们总共设计了13个任务，分为四个核心类别，旨在评估LLMS和多模式LLMS（MLLMS）的性能，以逐步审查生成，结果表述，与人类偏好的一致性以及对对抗性输入操作的稳健性。在16个开源模型和5种高级闭合源模型上进行的广泛实验证明了基准的彻底性。我们设想MMReview是建立自动同行评审系统开发标准化基础的关键一步。

Title: DPad: Efficient Diffusion Language Models with Suffix Dropout

• Authors: Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai "Hellen" Li, Yiran Chen
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.14148
• Pdf URL: https://arxiv.org/pdf/2508.14148
• Copy Paste: [[2508.14148]] DPad: Efficient Diffusion Language Models with Suffix Dropout(https://arxiv.org/abs/2508.14148)
• Keywords: language model, llm
• Abstract: Diffusion-based Large Language Models (dLLMs) parallelize text generation by framing decoding as a denoising process, but suffer from high computational overhead since they predict all future suffix tokens at each step while retaining only a small fraction. We propose Diffusion Scratchpad (DPad), a training-free method that restricts attention to a small set of nearby suffix tokens, preserving fidelity while eliminating redundancy. DPad integrates two strategies: (i) a sliding window, which maintains a fixed-length suffix window, and (ii) distance-decay dropout, which deterministically removes distant suffix tokens before attention computation. This simple design is compatible with existing optimizations such as prefix caching and can be implemented with only a few lines of code. Comprehensive evaluations across multiple benchmarks on LLaDA-1.5 and Dream models demonstrate that DPad delivers up to $\mathbf{61.4\times}$ speedup over vanilla dLLMs while maintaining comparable accuracy, highlighting its potential for efficient and scalable long-sequence inference. Our code is available at this https URL.
• 摘要：基于扩散的大语言模型（DLLMS）通过将解码作为一个变性过程并平行于文本生成，但遭受了高计算开销的困扰，因为它们在每个步骤中都预测了所有未来的后缀令牌，同时仅保留一小部分。我们提出了扩散刮擦板（DPAD），这是一种无训练的方法，将注意力限制在附近的少量后缀令牌上，从而保留了忠诚度，同时消除了冗余。 DPAD集成了两种策略：（i）一个滑动窗口，该滑动窗口维护固定长度的后缀窗口，（ii）距离decay的辍学，该窗口在注意力计算之前确定性地删除远处的后缀令牌。这种简单的设计与现有的优化兼容，例如前缀缓存，只能使用几行代码实现。在LLADA-1.5和DREAM模型上进行了多个基准测试的全面评估表明，DPAD可提供高达$ \ MathBf {61.4 \ times} $对Vanilla DLLMS的加速速度，同时保持可比的精度，强调其有效且可扩展的长期序列选择的潜力。我们的代码可在此HTTPS URL上找到。

Title: Comparing energy consumption and accuracy in text classification inference

• Authors: Johannes Zschache, Tilman Hartwig
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2508.14170
• Pdf URL: https://arxiv.org/pdf/2508.14170
• Copy Paste: [[2508.14170]] Comparing energy consumption and accuracy in text classification inference(https://arxiv.org/abs/2508.14170)
• Keywords: language model, llm
• Abstract: The increasing deployment of large language models (LLMs) in natural language processing (NLP) tasks raises concerns about energy efficiency and sustainability. While prior research has largely focused on energy consumption during model training, the inference phase has received comparatively less attention. This study systematically evaluates the trade-offs between model accuracy and energy consumption in text classification inference across various model architectures and hardware configurations. Our empirical analysis shows that the best-performing model in terms of accuracy can also be energy-efficient, while larger LLMs tend to consume significantly more energy with lower classification accuracy. We observe substantial variability in inference energy consumption ($<$mWh to $>$kWh), influenced by model type, model size, and hardware specifications. Additionally, we find a strong correlation between inference energy consumption and model runtime, indicating that execution time can serve as a practical proxy for energy usage in settings where direct measurement is not feasible. These findings have implications for sustainable AI development, providing actionable insights for researchers, industry practitioners, and policymakers seeking to balance performance and resource efficiency in NLP applications.
• 摘要：自然语言处理（NLP）任务中大型语言模型（LLM）的部署增加引起了人们对能源效率和可持续性的担忧。尽管先前的研究主要集中在模型训练期间的能源消耗上，但推断阶段的关注量相对较少。这项研究系统地评估了模型准确性与能源消耗之间的权衡，而在各种模型体系结构和硬件配置之间进行了文本分类推断。我们的经验分析表明，从精度方面最好的表现模型也可能是节能的，而较大的LLM倾向于以较低的分类精度消耗更多的能量。我们观察到受模型类型，型号大小和硬件规格影响的推理能源消耗（$ <$ mWh至$> kwh）的显着差异。此外，我们发现推理能源消耗与模型运行时之间存在很强的相关性，这表明在直接测量不可行的情况下，执行时间可以作为能源使用的实际代理。这些发现对可持续性AI开发有影响，为研究人员，行业从业人员和决策者提供了可行的见解，他们寻求平衡NLP应用程序的绩效和资源效率。

Title: Let's Use ChatGPT To Write Our Paper! Benchmarking LLMs To Write the Introduction of a Research Paper

• Authors: Krishna Garg, Firoz Shaikh, Sambaran Bandyopadhyay, Cornelia Caragea
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14273
• Pdf URL: https://arxiv.org/pdf/2508.14273
• Copy Paste: [[2508.14273]] Let's Use ChatGPT To Write Our Paper! Benchmarking LLMs To Write the Introduction of a Research Paper(https://arxiv.org/abs/2508.14273)
• Keywords: gpt, llm, prompt, chat
• Abstract: As researchers increasingly adopt LLMs as writing assistants, generating high-quality research paper introductions remains both challenging and essential. We introduce Scientific Introduction Generation (SciIG), a task that evaluates LLMs' ability to produce coherent introductions from titles, abstracts, and related works. Curating new datasets from NAACL 2025 and ICLR 2025 papers, we assess five state-of-the-art models, including both open-source (DeepSeek-v3, Gemma-3-12B, LLaMA 4-Maverick, MistralAI Small 3.1) and closed-source GPT-4o systems, across multiple dimensions: lexical overlap, semantic similarity, content coverage, faithfulness, consistency, citation correctness, and narrative quality. Our comprehensive framework combines automated metrics with LLM-as-a-judge evaluations. Results demonstrate LLaMA-4 Maverick's superior performance on most metrics, particularly in semantic similarity and faithfulness. Moreover, three-shot prompting consistently outperforms fewer-shot approaches. These findings provide practical insights into developing effective research writing assistants and set realistic expectations for LLM-assisted academic writing. To foster reproducibility and future research, we will publicly release all code and datasets.
• 摘要：随着研究人员越来越多地采用LLM作为写作助理，产生高质量的研究论文介绍仍然是具有挑战性的。我们介绍了科学简介生成（SCIIG），该任务评估了LLMS从标题，摘要和相关工作中产生连贯介绍的能力。 Curating new datasets from NAACL 2025 and ICLR 2025 papers, we assess five state-of-the-art models, including both open-source (DeepSeek-v3, Gemma-3-12B, LLaMA 4-Maverick, MistralAI Small 3.1) and closed-source GPT-4o systems, across multiple dimensions: lexical overlap, semantic similarity, content coverage, faithfulness, consistency, citation correctness,和叙事质量。我们的综合框架将自动指标与LLM-AS-A-A-Gudge评估相结合。结果表明，Llama-4 Maverick在大多数指标上的出色表现，尤其是在语义相似性和忠诚方面。此外，三杆促使持续的表现较少。这些发现为发展有效的研究写作助理提供了实用的见解，并对LLM辅助学术写作设定了现实的期望。为了培养可重复性和未来的研究，我们将公开发布所有代码和数据集。

Title: Disentangling concept semantics via multilingual averaging in Sparse Autoencoders

• Authors: Cliff O'Reilly, Ernesto Jimenez-Ruiz, Tillman Weyde
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14275
• Pdf URL: https://arxiv.org/pdf/2508.14275
• Copy Paste: [[2508.14275]] Disentangling concept semantics via multilingual averaging in Sparse Autoencoders(https://arxiv.org/abs/2508.14275)
• Keywords: llm, prompt
• Abstract: Connecting LLMs with formal knowledge representation and reasoning is a promising approach to address their shortcomings. Embeddings and sparse autoencoders are widely used to represent textual content, but the semantics are entangled with syntactic and language-specific information. We propose a method that isolates concept semantics in Large Langue Models by averaging concept activations derived via Sparse Autoencoders. We create English text representations from OWL ontology classes, translate the English into French and Chinese and then pass these texts as prompts to the Gemma 2B LLM. Using the open source Gemma Scope suite of Sparse Autoencoders, we obtain concept activations for each class and language version. We average the different language activations to derive a conceptual average. We then correlate the conceptual averages with a ground truth mapping between ontology classes. Our results give a strong indication that the conceptual average aligns to the true relationship between classes when compared with a single language by itself. The result hints at a new technique which enables mechanistic interpretation of internal network states with higher accuracy.
• 摘要：将LLM与正式的知识表示和推理联系起来是解决其缺点的一种有前途的方法。嵌入和稀疏的自动编码器被广泛用于表示文本内容，但语义与句法和语言特定信息纠缠在一起。我们提出了一种方法，该方法通过平均通过稀疏自动编码器得出的概念激活来隔离大型兰格模型中的概念语义。我们从OWL本体论课程中创建英语文本表示形式，将英语翻译成法语和中文，然后将这些文本作为提示传递给Gemma 2B LLM。使用稀疏自动编码器的开源Gemma范围套件，我们获得了每个类和语言版本的概念激活。我们平均使用不同的语言激活来得出概念平均值。然后，我们将概念平均值与本体论类之间的基础真理相关联。我们的结果表明，与单一语言本身相比，概念平均值与班级之间的真实关系保持一致。结果暗示了一种新技术，该技术能够以更高的精度对内部网络状态进行机械解释。

Title: GRILE: A Benchmark for Grammar Reasoning and Explanation in Romanian LLMs

• Authors: Adrian-Marius Dumitran, Alexandra-Mihaela Danila, Angela-Liliana Dumitran
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2508.14279
• Pdf URL: https://arxiv.org/pdf/2508.14279
• Copy Paste: [[2508.14279]] GRILE: A Benchmark for Grammar Reasoning and Explanation in Romanian LLMs(https://arxiv.org/abs/2508.14279)
• Keywords: language model, llm
• Abstract: LLMs (Large language models) have revolutionized NLP (Natural Language Processing), yet their pedagogical value for low-resource languages remains unclear. We present GRILE (Grammar Romanian Inference and Language Explanations) , the first open benchmark of 1,151 multiple-choice questions harvested from Romanian high-stakes exams (National Evaluation, Baccalaureate, university admissions). GRILE enables us to probe two complementary abilities of seven state-of-the-art multilingual and Romanian-specific LLMs: (i) selecting the correct answer, and (ii) producing linguistically accurate explanations. While Gemini 2.5 Pro reaches 83% accuracy, most open-weight models stay below 65%, and 48% of their explanations contain factual or pedagogical flaws according to expert review. A detailed error analysis pinpoints systematic weaknesses in morphology and in applying the latest DOOM3 orthographic norms. All data, code and a public web demo are released to catalyze future research. Our findings expose open challenges for trustworthy educational NLP in low-resource settings and establish GRILE as a new test-bed for controllable explanation generation and evaluation.
• 摘要：LLM（大型语言模型）彻底改变了NLP（自然语言处理），但其对低资源语言的教学价值尚不清楚。我们提出了Grile（语法罗马尼亚推论和语言解释），这是从罗马尼亚高风险考试中收获的1,151个多项选择问题的第一个开放基准（国家评估，学士学位，大学录取）。 Grile使我们能够探究七个最先进的多语言和罗马尼亚特定LLM的互补能力：（i）选择正确的答案，以及（ii）产生语言准确的解释。虽然Gemini 2.5 Pro的精度达到83％，但大多数开放式模型的型号保持在65％以下，而根据专家评论，其解释中有48％的解释包含事实或教学缺陷。详细的错误分析指出了形态学中的系统弱点，并应用了最新的Doom3拼字法规范。所有数据，代码和公共网络演示都均可催化未来的研究。我们的发现在低资源环境中对值得信赖的教育NLP面临公开挑战，并将Grile确立为可控制的解释生成和评估的新测试床。

Title: Tokens with Meaning: A Hybrid Tokenization Approach for NLP

• Authors: M. Ali Bayram, Ali Arda Fincan, Ahmet Semih Gümüş, Sercan Karakaş, Banu Diri, Savaş Yıldırım, Demircan Çelik
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14292
• Pdf URL: https://arxiv.org/pdf/2508.14292
• Copy Paste: [[2508.14292]] Tokens with Meaning: A Hybrid Tokenization Approach for NLP(https://arxiv.org/abs/2508.14292)
• Keywords: language model, gpt
• Abstract: Tokenization plays a pivotal role in natural language processing (NLP), shaping how text is segmented and interpreted by language models. While subword methods such as Byte Pair Encoding (BPE) and WordPiece have been effective, they often struggle with morphologically rich and agglutinative languages because they rely on frequency rather than linguistic structure. We introduce a hybrid tokenization framework that combines rule-based morphological analysis with statistical subword segmentation. The method uses phonological normalization, root-affix dictionaries, and a novel algorithm that balances morpheme preservation with vocabulary efficiency. It assigns shared identifiers to phonologically variant affixes (e.g., -ler and -lar) and altered root forms (e.g., kitap vs. kitabı), reducing redundancy while maintaining semantic integrity. Special tokens are added for whitespace and case, including an UPPERCASE marker to avoid vocabulary inflation from capitalization. BPE is integrated for out-of-vocabulary coverage without harming morphological coherence. On the TR-MMLU benchmark, the tokenizer achieves the highest Turkish Token Percentage (90.29\%) and Pure Token Percentage (85.8\%). Comparisons with tokenizers from LLaMA, Gemma, and GPT show more linguistically meaningful and coherent tokens. Although demonstrated on Turkish, the approach is language-independent and adaptable to other languages, offering a practical path toward more interpretable and effective multilingual NLP systems.
• 摘要：令牌化在自然语言处理（NLP）中起着关键作用，从而塑造了语言模型如何分割和解释文本。尽管字节对编码（bpe）和文字词等子字是有效的，但它们经常在形态上富含和凝结的语言中挣扎，因为它们依赖于频率而不是语言结构。我们介绍了一个混合令牌化框架，该框架将基于规则的形态分析与统计子词分割结合在一起。该方法使用语音归一化，根膜词字典和一种新型算法，该算法可以平衡词素保存与词汇效率。它将共享标识符分配给语音变体词缀（例如-ler和-lar）和改变的根部形式（例如，Kitap vs.Kitabı），同时降低了冗余，同时保持语义完整性。添加了适用于空格和案例的特殊令牌，包括大写标记，以避免资本化词汇通货膨胀。 BPE集成了用于范围内的覆盖范围，而不会损害形态相干性。在TR-MMLU基准测试中，令牌仪达到了土耳其令牌百分比（90.29 \％）和纯令牌百分比（85.8 \％）。与来自Llama，Gemma和GPT的Tokenizers的比较表现出更有意义和连贯的令牌。尽管在土耳其语上证明了这种方法，但它是与其他语言无关的，可适应其他语言，为更容易解释和有效的多语言NLP系统提供了实用的途径。

Title: Zero-knowledge LLM hallucination detection and mitigation through fine-grained cross-model consistency

• Authors: Aman Goel, Daniel Schwartz, Yanjun Qi
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.14314
• Pdf URL: https://arxiv.org/pdf/2508.14314
• Copy Paste: [[2508.14314]] Zero-knowledge LLM hallucination detection and mitigation through fine-grained cross-model consistency(https://arxiv.org/abs/2508.14314)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Large language models (LLMs) have demonstrated impressive capabilities across diverse tasks, but they remain susceptible to hallucinations--generating content that appears plausible but contains factual inaccuracies. We present Finch-Zk, a black-box framework that leverages FINe-grained Cross-model consistency to detect and mitigate Hallucinations in LLM outputs without requiring external knowledge sources. Finch-Zk introduces two key innovations: 1) a cross-model consistency checking strategy that reveals fine-grained inaccuracies by comparing responses generated by diverse models from semantically-equivalent prompts, and 2) a targeted mitigation technique that applies precise corrections to problematic segments while preserving accurate content. Experiments on the FELM dataset show Finch-Zk improves hallucination detection F1 scores by 6-39\% compared to existing approaches. For mitigation, Finch-Zk achieves 7-8 absolute percentage points improvement in answer accuracy on the GPQA-diamond dataset when applied to state-of-the-art models like Llama 4 Maverick and Claude 4 Sonnet. Extensive evaluation across multiple models demonstrates that Finch-Zk provides a practical, deployment-ready safeguard for enhancing factual reliability in production LLM systems.
• 摘要：大型语言模型（LLM）表现出了各种任务的令人印象深刻的能力，但它们仍然容易受到幻觉的影响 - 使内容看起来合理但包含事实上的不准确性。我们提出了Finch-Zk，这是一个黑框框架，利用细粒度的跨模型一致性来检测和减轻LLM输出中的幻觉，而无需外部知识源。 Finch-ZK介绍了两个关键的创新：1）跨模型一致性检查策略，通过比较来自语义上等效提示的各种模型产生的响应来揭示精细的不准确性，以及2）一种有针对性的缓解技术，该响应将精确的校正应用于有问题的片段，同时保存准确的内容。与现有方法相比，FELM数据集上的实验显示Finch-ZK的finch-ZK将F1得分提高了6-39 \％。为了减轻缓解，当应用于Llama 4 Maverick和Claude 4 Sonnet等最先进的模型时，Finch-ZK在GPQA-DAMOND数据集上的答案准确性提高了7-8个绝对百分比。多种模型的广泛评估表明，Finch-ZK提供了一种实用的，可以进行部署的保障，以增强生产LLM系统的事实可靠性。

Title: SurveyGen-I: Consistent Scientific Survey Generation with Evolving Plans and Memory-Guided Writing

• Authors: Jing Chen, Zhiheng Yang, Yixian Shen, Jie Liu, Adam Belloum, Chrysa Papagainni, Paola Grosso
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2508.14317
• Pdf URL: https://arxiv.org/pdf/2508.14317
• Copy Paste: [[2508.14317]] SurveyGen-I: Consistent Scientific Survey Generation with Evolving Plans and Memory-Guided Writing(https://arxiv.org/abs/2508.14317)
• Keywords: language model, llm
• Abstract: Survey papers play a critical role in scientific communication by consolidating progress across a field. Recent advances in Large Language Models (LLMs) offer a promising solution by automating key steps in the survey-generation pipeline, such as retrieval, structuring, and summarization. However, existing LLM-based approaches often struggle with maintaining coherence across long, multi-section surveys and providing comprehensive citation coverage. To address these limitations, we introduce SurveyGen-I, an automatic survey generation framework that combines coarse-to-fine retrieval, adaptive planning, and memory-guided generation. SurveyGen-I first performs survey-level retrieval to construct the initial outline and writing plan, and then dynamically refines both during generation through a memory mechanism that stores previously written content and terminology, ensuring coherence across subsections. When the system detects insufficient context, it triggers fine-grained subsection-level retrieval. During generation, SurveyGen-I leverages this memory mechanism to maintain coherence across subsections. Experiments across four scientific domains demonstrate that SurveyGen-I consistently outperforms previous works in content quality, consistency, and citation coverage.
• 摘要：调查论文通过整合整个领域的进步，在科学沟通中起着至关重要的作用。大型语言模型（LLMS）的最新进展通过自动化调查生成管道中的关键步骤（例如检索，结构和摘要）提供了有希望的解决方案。但是，现有的基于LLM的方法通常在长期的多节调查中保持连贯性并提供全面的引文覆盖范围。为了解决这些限制，我们介绍了SurveyGen-I，这是一个自动调查生成框架，结合了粗到精细的检索，自适应计划和记忆引导的一代。 Survegen-I首先执行调查级检索以构建初始大纲和写作计划，然后通过存储以前的书面内容和术语的记忆机制在生成过程中动态完善两者，从而确保跨小节之间的连贯性。当系统检测到不足的上下文时，它会触发细粒度的细分级检索。在发电期间，SurveyGen-I利用这种记忆机制保持跨小节之间的连贯性。跨四个科学领域的实验表明，SurveyGen-I始终在内容质量，一致性和引文覆盖范围内持续优于以前的作品。

Title: Beyond Semantic Similarity: Reducing Unnecessary API Calls via Behavior-Aligned Retriever

• Authors: Yixin Chen, Ying Xiong, Shangyu Wu, Yufei Cui, Xue Liu, Nan Guan, Chun Jason Xue
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14323
• Pdf URL: https://arxiv.org/pdf/2508.14323
• Copy Paste: [[2508.14323]] Beyond Semantic Similarity: Reducing Unnecessary API Calls via Behavior-Aligned Retriever(https://arxiv.org/abs/2508.14323)
• Keywords: language model, llm, prompt
• Abstract: Tool-augmented large language models (LLMs) leverage external functions to extend their capabilities, but inaccurate function calls can lead to inefficiencies and increased this http URL methods address this challenge by fine-tuning LLMs or using demonstration-based prompting, yet they often suffer from high training overhead and fail to account for inconsistent demonstration samples, which misguide the model's invocation behavior. In this paper, we trained a behavior-aligned retriever (BAR), which provides behaviorally consistent demonstrations to help LLMs make more accurate tool-using decisions. To train the BAR, we construct a corpus including different function-calling behaviors, i.e., calling or this http URL use the contrastive learning framework to train the BAR with customized positive/negative pairs and a dual-negative contrastive loss, ensuring robust retrieval of behaviorally consistent this http URL demonstrate that our approach significantly reduces erroneous function calls while maintaining high task performance, offering a cost-effective and efficient solution for tool-augmented LLMs.
• 摘要：工具启动的大型语言模型（LLM）利用外部功能来扩展其功能，但是功能不正确会导致效率低下，并增加了这种HTTP URL方法来解决这一挑战，通过微调LLM或基于示威的提示来解决这一挑战，但他们经常在高高的训练中训练，无法考虑到不一致的示范样品，这些模型误解了该模型，该模型误解了该模型，该模型是误解了该模型。在本文中，我们训练了一个与行为一致的检索器（BAR），该试验在行为一致的演示中，以帮助LLMS做出更准确的工具使用决策。要训​​练栏杆，我们构建一个语料库，包括不同的功能行为，即呼叫或此HTTP URL使用对比度学习框架以自定义的正/负面对和双重阴性对比损失来训练栏杆，从而确保了在行为上保持良好的效果，并确保了在行为上的强大效果，并确保了较高的效果，并履行了高度的效果 - 并在效果上取得了高度的效果 - 并使任务效果不断效果，并且可以使任务效果恢复了效果 - 用于工具调查的LLM。

Title: ISCA: A Framework for Interview-Style Conversational Agents

• Authors: Charles Welch, Allison Lahnala, Vasudha Varadarajan, Lucie Flek, Rada Mihalcea, J. Lomax Boyd, João Sedoc
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14344
• Pdf URL: https://arxiv.org/pdf/2508.14344
• Copy Paste: [[2508.14344]] ISCA: A Framework for Interview-Style Conversational Agents(https://arxiv.org/abs/2508.14344)
• Keywords: agent
• Abstract: We present a low-compute non-generative system for implementing interview-style conversational agents which can be used to facilitate qualitative data collection through controlled interactions and quantitative analysis. Use cases include applications to tracking attitude formation or behavior change, where control or standardization over the conversational flow is desired. We show how our system can be easily adjusted through an online administrative panel to create new interviews, making the tool accessible without coding. Two case studies are presented as example applications, one regarding the Expressive Interviewing system for COVID-19 and the other a semi-structured interview to survey public opinion on emerging neurotechnology. Our code is open-source, allowing others to build off of our work and develop extensions for additional functionality.
• 摘要：我们提出了一种低计算的非生成系统，用于实施采访风格的对话代理，该系统可用于通过受控的交互和定量分析来促进定性数据收集。用例包括跟踪态度形成或行为改变的应用，在需要对话流的控制或标准化的情况下。我们展示了如何通过在线管理面板轻松调整我们的系统以创建新的采访，从而使工具无需编码即可访问。将两项案例研究作为示例申请提出，一个涉及Covid-19的表达性访谈系统，另一个是半结构化的访谈，以调查有关新兴神经技术的公众舆论。我们的代码是开源的，允许其他人建立我们的工作，并开发扩展程序以实现其他功能。

Title: ZPD-SCA: Unveiling the Blind Spots of LLMs in Assessing Students' Cognitive Abilities

• Authors: Wenhan Dong, Zhen Sun, Yuemeng Zhao, Zifan Peng, Jun Wu, Jingyi Zheng, Yule Liu, Xinlei He, Yu Wang, Ruiming Wang, Xinyi Huang, Lei Mo
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2508.14377
• Pdf URL: https://arxiv.org/pdf/2508.14377
• Copy Paste: [[2508.14377]] ZPD-SCA: Unveiling the Blind Spots of LLMs in Assessing Students' Cognitive Abilities(https://arxiv.org/abs/2508.14377)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) have demonstrated potential in educational applications, yet their capacity to accurately assess the cognitive alignment of reading materials with students' developmental stages remains insufficiently explored. This gap is particularly critical given the foundational educational principle of the Zone of Proximal Development (ZPD), which emphasizes the need to match learning resources with Students' Cognitive Abilities (SCA). Despite the importance of this alignment, there is a notable absence of comprehensive studies investigating LLMs' ability to evaluate reading comprehension difficulty across different student age groups, especially in the context of Chinese language education. To fill this gap, we introduce ZPD-SCA, a novel benchmark specifically designed to assess stage-level Chinese reading comprehension difficulty. The benchmark is annotated by 60 Special Grade teachers, a group that represents the top 0.15% of all in-service teachers nationwide. Experimental results reveal that LLMs perform poorly in zero-shot learning scenarios, with Qwen-max and GLM even falling below the probability of random guessing. When provided with in-context examples, LLMs performance improves substantially, with some models achieving nearly double the accuracy of their zero-shot baselines. These results reveal that LLMs possess emerging abilities to assess reading difficulty, while also exposing limitations in their current training for educationally aligned judgment. Notably, even the best-performing models display systematic directional biases, suggesting difficulties in accurately aligning material difficulty with SCA. Furthermore, significant variations in model performance across different genres underscore the complexity of task. We envision that ZPD-SCA can provide a foundation for evaluating and improving LLMs in cognitively aligned educational applications.
• 摘要：大型语言模型（LLM）在教育应用中表现出了潜力，但是它们可以准确评估阅读材料与学生的发育阶段的认知一致性的能力仍然不足。鉴于近端发展区（ZPD）的基本教育原则（ZPD），该差距特别重要，该原则强调需要将学习资源与学生的认知能力（SCA）相匹配。尽管这一结局很重要，但仍有显着的研究研究LLMS评估不同学生年龄段的阅读理解难度的能力，尤其是在中文教育的背景下。为了填补这一空白，我们介绍了ZPD-SCA，这是一种专门设计用于评估阶段级中文阅读理解难度的新颖基准。基准由60个特殊年级的教师注释，该小组代表全国所有在职教师的最高0.15％。实验结果表明，LLM在零拍学习方案中的表现较差，QWEN-MAX和GLM甚至低于随机猜测的可能性。 LLMS性能在提供中文的示例时会大大提高，其中一些模型的精度几乎是其零射基线的准确性的两倍。这些结果表明，LLM具有评估阅读难度的新兴能力，同时还暴露了当前的培训中的限制，以进行教育的判断。值得注意的是，即使表现最佳的模型也会显示出系统的定向偏差，这表明在将材料难度与SCA完全排列的困难中遇到了困难。此外，不同类型的模型性能的显着变化强调了任务的复杂性。我们设想ZPD-SCA可以为评估和改善认知一致性的教育应用中的LLM提供基础。

Title: Credence Calibration Game? Calibrating Large Language Models through Structured Play

• Authors: Ke Fang, Tianyi Zhao, Lu Cheng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14390
• Pdf URL: https://arxiv.org/pdf/2508.14390
• Copy Paste: [[2508.14390]] Credence Calibration Game? Calibrating Large Language Models through Structured Play(https://arxiv.org/abs/2508.14390)
• Keywords: language model, llm, prompt
• Abstract: As Large Language Models (LLMs) are increasingly deployed in decision-critical domains, it becomes essential to ensure that their confidence estimates faithfully correspond to their actual correctness. Existing calibration methods have primarily focused on post-hoc adjustments or auxiliary model training; however, many of these approaches necessitate additional supervision or parameter updates. In this work, we propose a novel prompt-based calibration framework inspired by the Credence Calibration Game. Our method establishes a structured interaction loop wherein LLMs receive feedback based on the alignment of their predicted confidence with correctness. Through feedback-driven prompting and natural language summaries of prior performance, our framework dynamically improves model calibration. Extensive experiments across models and game configurations demonstrate consistent improvements in evaluation metrics. Our results highlight the potential of game-based prompting as an effective strategy for LLM calibration. Code and data are available at this https URL.
• 摘要：由于大型语言模型（LLM）越来越多地部署在决策领域，因此必须确保其信心估计与他们的实际正确性相对应。现有的校准方法主要集中于事后调整或辅助模型培训。但是，其中许多方法都需要其他监督或参数更新。在这项工作中，我们提出了一个受信任校准游戏启发的基于及时的新颖基于及时的校准框架。我们的方法建立了一个结构化的交互回路，其中LLM基于其预测的信心与正确性的对齐方式接收反馈。通过反馈驱动的提示和自然语言摘要，我们的框架动态改进了模型校准。跨模型和游戏配置的广泛实验表明，评估指标的一致改进。我们的结果强调了基于游戏的提示作为LLM校准的有效策略的潜力。代码和数据可在此HTTPS URL上找到。

Title: DEPTH: Hallucination-Free Relation Extraction via Dependency-Aware Sentence Simplification and Two-tiered Hierarchical Refinement

• Authors: Yupei Yang, Fan Feng, Lin Yang, Wanxi Deng, Lin Qu, Biwei Huang, Shikui Tu, Lei Xu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14391
• Pdf URL: https://arxiv.org/pdf/2508.14391
• Copy Paste: [[2508.14391]] DEPTH: Hallucination-Free Relation Extraction via Dependency-Aware Sentence Simplification and Two-tiered Hierarchical Refinement(https://arxiv.org/abs/2508.14391)
• Keywords: language model, llm, hallucination
• Abstract: Relation extraction enables the construction of structured knowledge for many downstream applications. While large language models (LLMs) have shown great promise in this domain, most existing methods concentrate on relation classification, which predicts the semantic relation type between a related entity pair. However, we observe that LLMs often struggle to reliably determine whether a relation exists, especially in cases involving complex sentence structures or intricate semantics, which leads to spurious predictions. Such hallucinations can introduce noisy edges in knowledge graphs, compromising the integrity of structured knowledge and downstream reliability. To address these challenges, we propose DEPTH, a framework that integrates Dependency-aware sEntence simPlification and Two-tiered Hierarchical refinement into the relation extraction pipeline. Given a sentence and its candidate entity pairs, DEPTH operates in two stages: (1) the Grounding module extracts relations for each pair by leveraging their shortest dependency path, distilling the sentence into a minimal yet coherent relational context that reduces syntactic noise while preserving key semantics; (2) the Refinement module aggregates all local predictions and revises them based on a holistic understanding of the sentence, correcting omissions and inconsistencies. We further introduce a causality-driven reward model that mitigates reward hacking by disentangling spurious correlations, enabling robust fine-tuning via reinforcement learning with human feedback. Experiments on six benchmarks demonstrate that DEPTH reduces the average hallucination rate to 7.0\% while achieving a 17.2\% improvement in average F1 score over state-of-the-art baselines.
• 摘要：关系提取可以为许多下游应用程序构建结构化知识。尽管大型语言模型（LLM）在该领域表现出了巨大的希望，但大多数现有方法都集中在关系分类上，该方法可以预测相关实体对之间的语义关系类型。但是，我们观察到，LLM经常难以可靠地确定是否存在关系，尤其是在涉及复杂句子结构或复杂语义的情况下，这会导致虚假的预测。这种幻觉可以在知识图中引入嘈杂的边缘，从而损害结构化知识和下游可靠性的完整性。为了应对这些挑战，我们提出了深度，该框架将依赖性感知的句子简化和两层层次结构改进到关系提取管道中。给定一个句子及其候选实体对，深度分为两个阶段：（1）接地模块通过利用其最短的依赖性路径来提取每对关系的关系，将句子提炼成最小而又连贯的关系上下文，以减少句法噪声，同时保持关键的关键语义； （2）改进模块汇总了所有本地预测，并根据对句子的整体理解，纠正遗漏和不一致的情况来修订它们。我们进一步介绍了因果关系驱动的奖励模型，该模型通过解开虚假相关性来减轻奖励黑客攻击，从而通过人类的反馈通过加强学习来实现良好的微调。六个基准测试的实验表明，深度将平均幻觉率降低到7.0 \％，而在最先进的基准中的平均F1得分提高了17.2 \％。

Title: Cognitive Surgery: The Awakening of Implicit Territorial Awareness in LLMs

• Authors: Yinghan Zhou, Weifeng Zhu, Juan Wen, Wanli Peng, Zhengxian Wu, Yiming Xue
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14408
• Pdf URL: https://arxiv.org/pdf/2508.14408
• Copy Paste: [[2508.14408]] Cognitive Surgery: The Awakening of Implicit Territorial Awareness in LLMs(https://arxiv.org/abs/2508.14408)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) have been shown to possess a degree of self-recognition capability-the ability to identify whether a given text was generated by themselves. Prior work has demonstrated that this capability is reliably expressed under the Pair Presentation Paradigm (PPP), where the model is presented with two texts and asked to choose which one it authored. However, performance deteriorates sharply under the Individual Presentation Paradigm (IPP), where the model is given a single text to judge authorship. Although this phenomenon has been observed, its underlying causes have not been systematically analyzed. In this paper, we first replicate existing findings to confirm that LLMs struggle to distinguish self- from other-generated text under IPP. We then investigate the reasons for this failure and attribute it to a phenomenon we term Implicit Territorial Awareness (ITA)-the model's latent ability to distinguish self- and other-texts in representational space, which remains unexpressed in its output behavior. To awaken the ITA of LLMs, we propose Cognitive Surgery (CoSur), a novel framework comprising four main modules: representation extraction, territory construction, authorship discrimination and cognitive editing. Experimental results demonstrate that our proposed method improves the performance of three different LLMs in the IPP scenario, achieving average accuracies of 83.25%, 66.19%, and 88.01%, respectively.
• 摘要：大型语言模型（LLMS）已显示具有一定程度的自我认识能力 - 能够确定给定文本是否是由自己产生的。先前的工作表明，该功能在对呈现范式（PPP）下可靠地表达，其中模型带有两个文本，并要求选择它撰写的一个文本。但是，在单个演示范式（IPP）下，性能急剧恶化，该模型被赋予单个文本以判断作者身份。尽管已经观察到了这种现象，但其基本原因尚未系统地分析。在本文中，我们首先复制了现有的发现，以确认LLM难以区分IPP下的其他生成文本。然后，我们研究了这种失败的原因，并将其归因于一种现象，我们将其称为隐式领土意识（ITA） - 模型在代表空间中区分自我和其他文本的潜在能力，这在其输出行为中仍保持不足。为了唤醒LLM的ITA，我们提出了认知手术（COSUR），这是一个包含四个主要模块的新型框架：代表性提取，领土构建，作者歧视和认知编辑。实验结果表明，我们提出的方法在IPP方案中提高了三种不同LLM的性能，分别达到平均精度为83.25％，66.19％和88.01％。

Title: Knowledge Graph-Infused Fine-Tuning for Structured Reasoning in Large Language Models

• Authors: Wuyang Zhang, Yexin Tian, Xiandong Meng, Mengjie Wang, Junliang Du
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14427
• Pdf URL: https://arxiv.org/pdf/2508.14427
• Copy Paste: [[2508.14427]] Knowledge Graph-Infused Fine-Tuning for Structured Reasoning in Large Language Models(https://arxiv.org/abs/2508.14427)
• Keywords: language model
• Abstract: This paper addresses the problems of missing reasoning chains and insufficient entity-level semantic understanding in large language models when dealing with tasks that require structured knowledge. It proposes a fine-tuning algorithm framework based on knowledge graph injection. The method builds on pretrained language models and introduces structured graph information for auxiliary learning. A graph neural network is used to encode entities and their relations, constructing a graph-based semantic representation. A fusion mechanism is then designed to jointly model the knowledge graph embeddings with the contextual representations from the language model. To enhance the robustness of knowledge integration, a gating mechanism is introduced to dynamically balance the contributions of linguistic semantics and structural knowledge. This effectively mitigates conflicts between different representational spaces. During training, a joint loss function is constructed to account for both task performance and structural alignment objectives. This helps improve the accuracy of entity prediction and semantic reasoning. The study also includes a series of systematic sensitivity experiments. It evaluates the effects of learning rate, graph coverage, and structural perturbations on model performance. The results further validate the effectiveness and stability of the proposed method across tasks such as entity recognition, question answering, and language generation. Experimental findings show that the proposed structure-aware fine-tuning framework significantly enhances the model's ability to represent complex semantic units. It demonstrates better semantic consistency and contextual logic modeling in scenarios involving structural reasoning and entity extraction.
• 摘要：本文讨论了在处理需要结构化知识的任务时缺少推理链和实体级语义级别理解不足的问题。它提出了基于知识图注入的微调算法框架。该方法基于验证的语言模型，并引入了辅助学习的结构化图表。图神经网络用于编码实体及其关系，构建基于图的语义表示。然后，设计了一种融合机制，以将知识图嵌入与语言模型的上下文表示共同建模。为了增强知识整合的鲁棒性，引入了传输机制，以动态平衡语言语义和结构知识的贡献。这有效地减轻了不同代表空间之间的冲突。在培训期间，构建了联合损失函数以说明任务绩效和结构对齐目标。这有助于提高实体预测和语义推理的准确性。该研究还包括一系列系统的灵敏度实验。它评估了学习率，图形覆盖率和结构扰动对模型性能的影响。结果进一步验证了跨任务（例如实体识别，问题答案和语言生成）的拟议方法的有效性和稳定性。实验发现表明，所提出的结构感知的微调框架可显着增强模型代表复杂语义单元的能力。它在涉及结构推理和实体提取的情况下展示了更好的语义一致性和上下文逻辑建模。

Title: NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model

• Authors: NVIDIA: Aarti Basant, Abhijit Khairnar, Abhijit Paithankar, Abhinav Khattar, Adi Renduchintala, Adithya Renduchintala, Aditya Malte, Akhiad Bercovich, Akshay Hazare, Alejandra Rico, Aleksander Ficek, Alex Kondratenko, Alex Shaposhnikov, Ali Taghibakhshi, Amelia Barton, Ameya Sunil Mahabaleshwarkar, Amy Shen, Andrew Tao, Ann Guan, Anna Shors, Anubhav Mandarwal, Arham Mehta, Arun Venkatesan, Ashton Sharabiani, Ashwath Aithal, Ashwin Poojary, Ayush Dattagupta, Balaram Buddharaju, Banghua Zhu, Barnaby Simkin, Bilal Kartal, Bita Darvish Rouhani, Bobby Chen, Boris Ginsburg, Brandon Norick, Brian Yu, Bryan Catanzaro, Charles Wang, Charlie Truong, Chetan Mungekar, Chintan Patel, Chris Alexiuk, Christian Munley, Christopher Parisien, Dan Su, Daniel Afrimi, Daniel Korzekwa, Daniel Rohrer, Daria Gitman, David Mosallanezhad, Deepak Narayanan, Dima Rekesh, Dina Yared, Dmytro Pykhtar, Dong Ahn, Duncan Riach, Eileen Long, Elliott Ning, Eric Chung, Erick Galinkin, Evelina Bakhturina, Gargi Prasad, Gerald Shen, Haim Elisha, Harsh Sharma, Hayley Ross, Helen Ngo, Herman Sahota, Hexin Wang, Hoo Chang Shin, Hua Huang, Iain Cunningham, Igor Gitman, Ivan Moshkov, Jaehun Jung, Jan Kautz, Jane Polak Scowcroft, Jared Casper, Jimmy Zhang, Jinze Xue, Jocelyn Huang, Joey Conway, John Kamalu, Jonathan Cohen, Joseph Jennings, Julien Veron Vialard, Junkeun Yi, Jupinder Parmar, Kari Briski, Katherine Cheung, Katherine Luna, Keith Wyss, Keshav Santhanam, Kezhi Kong, Krzysztof Pawelec, Kumar Anik, Kunlun Li, Kushan Ahmadian, Lawrence McAfee
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.14444
• Pdf URL: https://arxiv.org/pdf/2508.14444
• Copy Paste: [[2508.14444]] NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model(https://arxiv.org/abs/2508.14444)
• Keywords: language model
• Abstract: We introduce Nemotron-Nano-9B-v2, a hybrid Mamba-Transformer language model designed to increase throughput for reasoning workloads while achieving state-of-the-art accuracy compared to similarly-sized models. Nemotron-Nano-9B-v2 builds on the Nemotron-H architecture, in which the majority of the self-attention layers in the common Transformer architecture are replaced with Mamba-2 layers, to achieve improved inference speed when generating the long thinking traces needed for reasoning. We create Nemotron-Nano-9B-v2 by first pre-training a 12-billion-parameter model (Nemotron-Nano-12B-v2-Base) on 20 trillion tokens using an FP8 training recipe. After aligning Nemotron-Nano-12B-v2-Base, we employ the Minitron strategy to compress and distill the model with the goal of enabling inference on up to 128k tokens on a single NVIDIA A10G GPU (22GiB of memory, bfloat16 precision). Compared to existing similarly-sized models (e.g., Qwen3-8B), we show that Nemotron-Nano-9B-v2 achieves on-par or better accuracy on reasoning benchmarks while achieving up to 6x higher inference throughput in reasoning settings like 8k input and 16k output tokens. We are releasing Nemotron-Nano-9B-v2, Nemotron-Nano12B-v2-Base, and Nemotron-Nano-9B-v2-Base checkpoints along with the majority of our pre- and post-training datasets on Hugging Face.
• 摘要：我们介绍了Nemotron-Nano-9B-V2，这是一种混合Mamba-Transformer语言模型，旨在增加推理工作负载的吞吐量，同时与类似尺寸的模型相比，同时实现最新的精度。 Nemotron-Nano-9B-V2建立在Nemotron-H架构上，其中通用变压器体系结构中的大多数自我发场层被MAMBA-2层取代，以在产生推理所需的较长思维痕迹时，可以提高推理速度。我们使用FP8培训配方在20万亿代币上首次预训练了120亿参数模型（Nemotron-Nano-12b-v2 base），从而创建了Nemotron-Nano-9b-V2。在对齐Nemotron-Nano-12b-V2碱基对齐后，我们采用了Minitron策略来压缩和提炼模型，目的是在单个NVIDIA A10G GPU上推断多达128K令牌（22GIB的存储器，Bfloat16，Bfloat16精确度）。与现有的类似模型（例如QWEN3-8B）相比，我们表明Nemotron-Nano-9B-V2在推理基准测试中实现了PAR或更高的准确性，同时在8K输入和16K输出标记（如8K输入和16K输出标记）中达到了高达6倍的推理吞吐量。我们正在释放Nemotron-Nano-9b-V2，Nemotron-Nano12b-V2碱基和Nemotron-Nano-9b-v2碱基检查点，以及大多数在拥抱面孔上的训练前和训练后数据集。

Title: In2x at WMT25 Translation Task

• Authors: Lei Pang, Hanyi Mao, Quanjia Xiao, HaiXiao Liu, Xiangyi Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14472
• Pdf URL: https://arxiv.org/pdf/2508.14472
• Copy Paste: [[2508.14472]] In2x at WMT25 Translation Task(https://arxiv.org/abs/2508.14472)
• Keywords: language model, llm
• Abstract: This paper presents the open-system submission by the In2x research team for the WMT25 General Machine Translation Shared Task. Our submission focuses on Japanese-related translation tasks, aiming to explore a generalizable paradigm for extending large language models (LLMs) to other languages. This paradigm encompasses aspects such as data construction methods and reward model design. The ultimate goal is to enable large language model systems to achieve exceptional performance in low-resource or less commonly spoken languages.
• 摘要：本文介绍了IN2X研究团队为WMT25通用机器翻译共享任务的开放系统提交。我们的提交重点是日本相关的翻译任务，旨在探索将大型语言模型（LLMS）扩展到其他语言的可推广范式。该范式包括数据构建方法和奖励模型设计等方面。最终的目标是使大型语言模型系统能够在低资源或更常见的语言中实现出色的性能。

Title: Improving in-context learning with a better scoring function

• Authors: Omar Naim, Swarnadeep Bhar, Jérôme Bolte, Nicholas Asher
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14685
• Pdf URL: https://arxiv.org/pdf/2508.14685
• Copy Paste: [[2508.14685]] Improving in-context learning with a better scoring function(https://arxiv.org/abs/2508.14685)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) exhibit a remarkable capacity to learn by analogy, known as in-context learning (ICL). However, recent studies have revealed limitations in this ability. In this paper, we examine these limitations on tasks involving first-order quantifiers such as {\em all} and {\em some}, as well as on ICL with linear functions. We identify Softmax, the scoring function in attention mechanism, as a contributing factor to these constraints. To address this, we propose \textbf{scaled signed averaging (SSA)}, a novel alternative to Softmax. Empirical results show that SSA dramatically improves performance on our target tasks. Furthermore, we evaluate both encoder-only and decoder-only transformers models with SSA, demonstrating that they match or exceed their Softmax-based counterparts across a variety of linguistic probing tasks.
• 摘要：大型语言模型（LLMS）具有类比的非凡学习能力，称为文化学习（ICL）。但是，最近的研究揭示了这种能力的局限性。在本文中，我们检查了涉及一阶量化器（例如{\ em all}和{\ em Some}的任务的这些限制，以及具有线性函数的ICL。我们将SoftMax（注意机制中的评分函数）识别为对这些约束的促成因素。为了解决这个问题，我们提出\ textbf {缩放签名平均（SSA）}，这是软马克斯的新颖替代品。经验结果表明，SSA极大地提高了我们的目标任务的性能。此外，我们使用SSA评估了仅编码和解码器的仅编码器变压器模型，表明它们在各种语言探测任务中匹配或超过基于软疗法的对应物。

Title: ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine

• Authors: Junying Chen, Zhenyang Cai, Zhiheng Liu, Yunjin Yang, Rongsheng Wang, Qingying Xiao, Xiangyi Feng, Zhan Su, Jing Guo, Xiang Wan, Guangjun Yu, Haizhou Li, Benyou Wang
• Subjects: cs.CL, cs.AI, cs.CV, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2508.14706
• Pdf URL: https://arxiv.org/pdf/2508.14706
• Copy Paste: [[2508.14706]] ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine(https://arxiv.org/abs/2508.14706)
• Keywords: language model, gpt, llm
• Abstract: Despite the success of large language models (LLMs) in various domains, their potential in Traditional Chinese Medicine (TCM) remains largely underexplored due to two critical barriers: (1) the scarcity of high-quality TCM data and (2) the inherently multimodal nature of TCM diagnostics, which involve looking, listening, smelling, and pulse-taking. These sensory-rich modalities are beyond the scope of conventional LLMs. To address these challenges, we present ShizhenGPT, the first multimodal LLM tailored for TCM. To overcome data scarcity, we curate the largest TCM dataset to date, comprising 100GB+ of text and 200GB+ of multimodal data, including 1.2M images, 200 hours of audio, and physiological signals. ShizhenGPT is pretrained and instruction-tuned to achieve deep TCM knowledge and multimodal reasoning. For evaluation, we collect recent national TCM qualification exams and build a visual benchmark for Medicinal Recognition and Visual Diagnosis. Experiments demonstrate that ShizhenGPT outperforms comparable-scale LLMs and competes with larger proprietary models. Moreover, it leads in TCM visual understanding among existing multimodal LLMs and demonstrates unified perception across modalities like sound, pulse, smell, and vision, paving the way toward holistic multimodal perception and diagnosis in TCM. Datasets, models, and code are publicly available. We hope this work will inspire further exploration in this field.
• 摘要：尽管大型语言模型（LLM）在各个领域取得了成功，但由于两个关键障碍，它们在传统中医（TCM）中的潜力在很大程度上尚未得到充满激发：（1）高质量TCM数据的稀缺性以及（2）TCM固有的多模态性质，涉及TCM诊断，涉及看起来，聆听，聆听，嗅觉，闻名，脉动，并创作。这些富含感官的方式超出了常规LLM的范围。为了应对这些挑战，我们提出了Shizhengpt，这是第一个针对TCM量身定制的多模式LLM。为了克服数据稀缺性，我们策划了迄今为止最大的TCM数据集，其中包括100GB+文本和200GB+多模式数据的数据集，包括120万图像，200小时的音频和生理信号。 Shizhengpt经过审议和指导，以实现深入的TCM知识和多模式推理。为了进行评估，我们收集了最新的国家TCM资格考试，并为药物识别和视觉诊断建立视觉基准。实验表明，Shizhengpt的表现优于可比规模的LLM，并与较大的专有模型竞争。此外，它导致了现有多模式LLM的TCM视觉理解，并在声音，脉搏，气味和视觉等方式上展示了统一的感知，为TCM中的整体多模式感知和诊断铺平了道路。数据集，模型和代码公开可用。我们希望这项工作能够激发该领域的进一步探索。

Title: The Digital Sous Chef -- A Comparative Study on Fine-Tuning Language Models for Recipe Generation

• Authors: Shubham Pundhir, Ganesh Bagler
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14718
• Pdf URL: https://arxiv.org/pdf/2508.14718
• Copy Paste: [[2508.14718]] The Digital Sous Chef -- A Comparative Study on Fine-Tuning Language Models for Recipe Generation(https://arxiv.org/abs/2508.14718)
• Keywords: language model, gpt
• Abstract: We established a rigorous benchmark for text-based recipe generation, a fundamental task in natural language generation. We present a comprehensive comparative study contrasting a fine-tuned GPT-2 large (774M) model against the GPT-2 small (124M) model and traditional LSTM/RNN baselines on the 5-cuisine corpus from RecipeDB. Our key contribution is a targeted tokenization strategy that augments the vocabulary with 23 common fraction tokens and custom structural markers. This approach addresses a critical limitation of generic tokenizers by preserving essential recipe structures and precise numerical quantities, thereby enhancing domain specificity. Performance is evaluated using a comprehensive suite of seven automatic metrics spanning fluency (BLEU-4, METEOR), coherence (ROUGE-L), semantic relevance (BERTScore), and diversity. Our experiments show that the large transformer-based approach yields a >20% relative improvement in BERTScore (F1) (0.92 vs 0.72) over the best recurrent baseline, while reducing perplexity by 69.8%. We conclude with a discussion of remaining challenges, particularly regarding factual accuracy, and outline how this foundational study paves the way for integrating real-world constraints and multi-modal inputs in advanced recipe generation research.
• 摘要：我们为基于文本的食谱生成建立了严格的基准，这是自然语言生成的基本任务。我们提出了一项全面的比较研究，将微调的GPT-2大型（774m）模型与GPT-2小型（124m）模型和传统的LSTM/RNN基准相比，从FeceedB的5-Cuisine coldus上进行了对比。我们的关键贡献是一种有针对性的令牌化策略，可以通过23个常见的分数标记和自定义结构标记来增强词汇量。这种方法通过保留基本配方结构和精确的数值来解决通用引导者的关键局限性，从而增强了域的特异性。使用跨越七个自动指标（BLEU-4，流星），连贯性（Rouge-L），语义相关性（BERTSCORE）和多样性的七个自动指标（BLEU-4）和多样性评估性能。我们的实验表明，基于大型变压器的大型方法在Bertscore（F1）（0.92）（0.92 vs 0.72）的相对相对相对提高量高于最佳的基线，同时将困惑降低了69.8％。最后，我们讨论了剩余的挑战，尤其是关于事实准确性，并概述了这项基础研究如何为在高级配方生成研究中整合现实世界中的约束和多模式输入铺平道路。

Title: Transplant Then Regenerate: A New Paradigm for Text Data Augmentation

• Authors: Guangzhan Wang, Hongyu Zhang, Beijun Shen, Xiaodong Gu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14723
• Pdf URL: https://arxiv.org/pdf/2508.14723
• Copy Paste: [[2508.14723]] Transplant Then Regenerate: A New Paradigm for Text Data Augmentation(https://arxiv.org/abs/2508.14723)
• Keywords: language model, llm, prompt
• Abstract: Data augmentation is a critical technique in deep learning. Traditional methods like Back-translation typically focus on lexical-level rephrasing, which primarily produces variations with the same semantics. While large language models (LLMs) have enhanced text augmentation by their "knowledge emergence" capability, controlling the style and structure of these outputs remains challenging and requires meticulous prompt engineering. In this paper, we propose LMTransplant, a novel text augmentation paradigm leveraging LLMs. The core idea of LMTransplant is transplant-then-regenerate: incorporating seed text into a context expanded by LLM, and asking the LLM to regenerate a variant based on the expanded context. This strategy allows the model to create more diverse and creative content-level variants by fully leveraging the knowledge embedded in LLMs, while preserving the core attributes of the original text. We evaluate LMTransplant across various text-related tasks, demonstrating its superior performance over existing text augmentation methods. Moreover, LMTransplant demonstrates exceptional scalability as the size of augmented data grows.
• 摘要：数据增强是深度学习的关键技术。诸如背面翻译之类的传统方法通常集中于词汇级别的重塑，这主要产生相同语义的变化。尽管大型语言模型（LLMS）通过其“知识出现”功能增强了文本增强，但控制这些输出的样式和结构仍然具有挑战性，需要细致的及时工程。在本文中，我们提出了LMTransplant，这是一种新的文本增强范式，利用LLMS。 LMTransplant的核心思想是移植 - 然后再生：将种子文本纳入LLM扩展的上下文中，并要求LLM根据扩展的上下文重新生成变体。该策略使模型可以通过完全利用LLM中的知识，同时保留原始文本的核心属性来创建更多样化和创造性的内容级别的变体。我们在各种与文本相关的任务中评估了LMTransplant，证明了其优于现有文本增强方法的性能。此外，随着增强数据的大小的增长，LMtransplant表现出了出色的可伸缩性。

Title: Evaluating Multilingual and Code-Switched Alignment in LLMs via Synthetic Natural Language Inference

• Authors: Samir Abdaljalil, Erchin Serpedin, Khalid Qaraqe, Hasan Kurban
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14735
• Pdf URL: https://arxiv.org/pdf/2508.14735
• Copy Paste: [[2508.14735]] Evaluating Multilingual and Code-Switched Alignment in LLMs via Synthetic Natural Language Inference(https://arxiv.org/abs/2508.14735)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) are increasingly applied in multilingual contexts, yet their capacity for consistent, logically grounded alignment across languages remains underexplored. We present a controlled evaluation framework for multilingual natural language inference (NLI) that generates synthetic, logic-based premise-hypothesis pairs and translates them into a typologically diverse set of languages. This design enables precise control over semantic relations and allows testing in both monolingual and mixed-language (code-switched) conditions. Surprisingly, code-switching does not degrade, and can even improve, performance, suggesting that translation-induced lexical variation may serve as a regularization signal. We validate semantic preservation through embedding-based similarity analyses and cross-lingual alignment visualizations, confirming the fidelity of translated pairs. Our findings expose both the potential and the brittleness of current LLM cross-lingual reasoning, and identify code-switching as a promising lever for improving multilingual robustness. Code available at: this https URL
• 摘要：大型语言模型（LLM）越来越多地应用于多语言环境中，但它们跨语言的一致，逻辑上的对齐能力仍然没有得到充实的态度。我们为多语言自然语言推断（NLI）提供了一个受控的评估框架，该框架生成了综合，基于逻辑的前提 - 假设对，并将其转化为类型上多样化的语言集。该设计可以精确控制语义关系，并允许在单语和混合语言（代码转换）条件下进行测试。令人惊讶的是，代码转换不会降低，甚至可以提高性能，这表明翻译诱导的词汇变化可能是正则化信号。我们通过基于嵌入的相似性分析和跨语性比对可视化来验证语义保存，从而证实了翻译对的忠诚度。我们的发现揭示了当前LLM跨语性推理的潜力和脆弱性，并确定代码转换是改善多语言鲁棒性的有前途的杠杆。代码可用：此HTTPS URL

Title: TransLLM: A Unified Multi-Task Foundation Framework for Urban Transportation via Learnable Prompting

• Authors: Jiaming Leng, Yunying Bi, Chuan Qin, Bing Yin, Yanyong Zhang, Chao Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14782
• Pdf URL: https://arxiv.org/pdf/2508.14782
• Copy Paste: [[2508.14782]] TransLLM: A Unified Multi-Task Foundation Framework for Urban Transportation via Learnable Prompting(https://arxiv.org/abs/2508.14782)
• Keywords: language model, llm, prompt
• Abstract: Urban transportation systems encounter diverse challenges across multiple tasks, such as traffic forecasting, electric vehicle (EV) charging demand prediction, and taxi dispatch. Existing approaches suffer from two key limitations: small-scale deep learning models are task-specific and data-hungry, limiting their generalizability across diverse scenarios, while large language models (LLMs), despite offering flexibility through natural language interfaces, struggle with structured spatiotemporal data and numerical reasoning in transportation domains. To address these limitations, we propose TransLLM, a unified foundation framework that integrates spatiotemporal modeling with large language models through learnable prompt composition. Our approach features a lightweight spatiotemporal encoder that captures complex dependencies via dilated temporal convolutions and dual-adjacency graph attention networks, seamlessly interfacing with LLMs through structured embeddings. A novel instance-level prompt routing mechanism, trained via reinforcement learning, dynamically personalizes prompts based on input characteristics, moving beyond fixed task-specific templates. The framework operates by encoding spatiotemporal patterns into contextual representations, dynamically composing personalized prompts to guide LLM reasoning, and projecting the resulting representations through specialized output layers to generate task-specific predictions. Experiments across seven datasets and three tasks demonstrate the exceptional effectiveness of TransLLM in both supervised and zero-shot settings. Compared to ten baseline models, it delivers competitive performance on both regression and planning problems, showing strong generalization and cross-task adaptability. Our code is available at this https URL.
• 摘要：城市运输系统在多个任务中遇到各种挑战，例如交通预测，电动汽车（EV）充电需求预测和出租车派遣。现有的方法有两个关键的局限性：小规模的深度学习模型是特定于任务的和渴望数据的，限制了它们在各种情况下的普遍性，而大语模型（LLMS）尽管通过自然语言界面提供了灵活性，但仍与结构化的时空数据和运输领域的数值推理斗争。为了解决这些限制，我们提出了Transllm，这是一个统一的基础框架，通过可学习的及时构图将时空建模与大语言模型集成在一起。我们的方法具有轻质时空编码器，该编码器可通过扩张的时间卷积和双jaCencency图形注意力网络捕获复杂的依赖性，并通过结构化的嵌入方式无缝与LLM无缝接口。通过强化学习训练的新型实例级及时路由机制，基于输入特征，动态化提示，超越了固定的特定任务特异性模板。该框架通过将时空模式编码为上下文表示，动态组成个性化提示来指导LLM推理，并通过专门的输出层投影产生的表示形式，以生成特定于任务的预测。在七个数据集和三个任务上进行的实验证明了Transllm在受监督和零击设置中的出色效果。与十个基线模型相比，它在回归和计划问题上都提供了竞争性能，显示出强烈的概括和交叉任务的适应性。我们的代码可在此HTTPS URL上找到。

Title: Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs

• Authors: Skatje Myers, Dmitriy Dligach, Timothy A. Miller, Samantha Barr, Yanjun Gao, Matthew Churpek, Anoop Mayampurath, Majid Afshar
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14817
• Pdf URL: https://arxiv.org/pdf/2508.14817
• Copy Paste: [[2508.14817]] Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs(https://arxiv.org/abs/2508.14817)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Electronic health records (EHRs) are long, noisy, and often redundant, posing a major challenge for the clinicians who must navigate them. Large language models (LLMs) offer a promising solution for extracting and reasoning over this unstructured text, but the length of clinical notes often exceeds even state-of-the-art models' extended context windows. Retrieval-augmented generation (RAG) offers an alternative by retrieving task-relevant passages from across the entire EHR, potentially reducing the amount of required input tokens. In this work, we propose three clinical tasks designed to be replicable across health systems with minimal effort: 1) extracting imaging procedures, 2) generating timelines of antibiotic use, and 3) identifying key diagnoses. Using EHRs from actual hospitalized patients, we test three state-of-the-art LLMs with varying amounts of provided context, using either targeted text retrieval or the most recent clinical notes. We find that RAG closely matches or exceeds the performance of using recent notes, and approaches the performance of using the models' full context while requiring drastically fewer input tokens. Our results suggest that RAG remains a competitive and efficient approach even as newer models become capable of handling increasingly longer amounts of text.
• 摘要：电子健康记录（EHRS）很长，嘈杂且经常多余，这对必须驾驶它们的临床医生构成了重大挑战。大型语言模型（LLMS）为在此非结构化文本上提取和推理提供了有希望的解决方案，但是临床注释的长度通常超过了最新模型的扩展上下文窗口。检索增强的生成（RAG）通过从整个EHR中检索与任务相关的段落提供了替代方案，从而有可能减少所需的输入令牌的数量。在这项工作中，我们提出了三个临床任务，旨在在卫生系统中以最少的精力复制：1）提取成像程序，2）生成抗生素使用的时间表，3）识别关键诊断。使用来自实际住院患者的EHR，我们使用靶向文本检索或最新的临床注释测试了三个最先进的LLM，并具有不同数量的提供的上下文。我们发现，抹布与使用最新音符的性能紧密匹配或超过了使用模型的完整上下文的性能，同时需要大幅度较少的输入令牌。我们的结果表明，即使较新的模型能够处理越来越长的文本，也仍然是一种竞争和高效的方法。

Title: Long Chain-of-Thought Reasoning Across Languages

• Authors: Josh Barua, Seun Eisape, Kayo Yin, Alane Suhr
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.14828
• Pdf URL: https://arxiv.org/pdf/2508.14828
• Copy Paste: [[2508.14828]] Long Chain-of-Thought Reasoning Across Languages(https://arxiv.org/abs/2508.14828)
• Keywords: language model, llm, chain-of-thought
• Abstract: Scaling inference through long chains-of-thought (CoTs) has unlocked impressive reasoning capabilities in large language models (LLMs), yet the reasoning process remains almost exclusively English-centric. We construct translated versions of two popular English reasoning datasets, fine-tune Qwen 2.5 (7B) and Qwen 3 (8B) models, and present a systematic study of long CoT generation across French, Japanese, Latvian, and Swahili. Our experiments reveal three key findings. First, the efficacy of using English as a pivot language varies by language: it provides no benefit for French, improves performance when used as the reasoning language for Japanese and Latvian, and proves insufficient for Swahili where both task comprehension and reasoning remain poor. Second, extensive multilingual pretraining in Qwen 3 narrows but does not eliminate the cross-lingual performance gap. A lightweight fine-tune using only 1k traces still improves performance by over 30\% in Swahili. Third, data quality versus scale trade-offs are language dependent: small, carefully curated datasets suffice for English and French, whereas larger but noisier corpora prove more effective for Swahili and Latvian. Together, these results clarify when and why long CoTs transfer across languages and provide translated datasets to foster equitable multilingual reasoning research.
• 摘要：通过长长的思维（COT）进行扩展，在大型语言模型（LLMS）中解锁了令人印象深刻的推理能力，但是推理过程几乎仍然完全以英语为中心。我们构建了两个流行的英语推理数据集的翻译版本，即微调QWEN 2.5（7b）和QWEN 3（8B）模型，并介绍了一项系统的研究，对法国，日本，拉脱维亚人和Swahili的长床产生进行了系统的研究。我们的实验揭示了三个关键发现。首先，将英语用作枢轴语言的功效随着语言而变化：它对法语没有任何好处，可以在用作日语和拉脱维亚的推理语言时提高性能，并且证明对斯瓦希里语的效力不足，而在那些任务理解和推理仍然很差的斯瓦希里语。其次，在QWEN 3狭窄中进行了广泛的多语言预测，但并没有消除跨语性的性能差距。仅使用1K痕迹的轻质微调仍可以在斯瓦希里语中提高30 \％的性能。第三，数据质量与比例尺的权衡取决于语言：小，精心策划的数据集足以容纳英语和法语，而较大但嘈杂的语料库对斯瓦希里语和拉脱维亚人来说更有效。这些结果共同阐明了何时以及为什么长长的COT跨语言转移，并提供翻译的数据集，以促进公平的多语言推理研究。

Title: MedReseacher-R1: Expert-Level Medical Deep Researcher via A Knowledge-Informed Trajectory Synthesis Framework

• Authors: Ailing Yu, Lan Yao, Jingnan Liu, Zhe Chen, Jiajun Yin, Yuan Wang, Xinhao Liao, Zhiling Ye, Ji Li, Yun Yue, Hansong Xiao, Hualei Zhou, Chunxiao Guo, Peng Wei, Jinjie Gu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.14880
• Pdf URL: https://arxiv.org/pdf/2508.14880
• Copy Paste: [[2508.14880]] MedReseacher-R1: Expert-Level Medical Deep Researcher via A Knowledge-Informed Trajectory Synthesis Framework(https://arxiv.org/abs/2508.14880)
• Keywords: language model, llm, agent
• Abstract: Recent developments in Large Language Model (LLM)-based agents have shown impressive capabilities spanning multiple domains, exemplified by deep research systems that demonstrate superior performance on complex information-seeking and synthesis tasks. While general-purpose deep research agents have shown impressive capabilities, they struggle significantly with medical domain challenges, as evidenced by leading proprietary systems achieving limited accuracy on complex medical benchmarks. The key limitations are: (1) the model lacks sufficient dense medical knowledge for clinical reasoning, and (2) the framework is constrained by the absence of specialized retrieval tools tailored for medical this http URL present a medical deep research agent that addresses these challenges through two core innovations. First, we develop a novel data synthesis framework using medical knowledge graphs, extracting the longest chains from subgraphs around rare medical entities to generate complex multi-hop question-answer pairs. Second, we integrate a custom-built private medical retrieval engine alongside general-purpose tools, enabling accurate medical information synthesis. Our approach generates 2100+ diverse trajectories across 12 medical specialties, each averaging 4.2 tool this http URL a two-stage training paradigm combining supervised fine-tuning and online reinforcement learning with composite rewards, our MedResearcher-R1-32B model demonstrates exceptional performance, establishing new state-of-the-art results on medical benchmarks while maintaining competitive performance on general deep research tasks. Our work demonstrates that strategic domain-specific innovations in architecture, tool design, and training data construction can enable smaller open-source models to outperform much larger proprietary systems in specialized domains.
• 摘要：大型语言模型（LLM）基于多个领域的最新发展表现出了令人印象深刻的功能，这是深入的研究系统的例证，这些系统证明了在复杂的信息寻求信息和综合任务上的出色表现。尽管通用深度研究人员表现出了令人印象深刻的能力，但他们在医疗领域的挑战中遇到了重大斗争，这证明了领先的专有系统在复杂的医疗基准上的准确性有限。关键的局限性是：（1）该模型缺乏足够的临床推理医学知识，（2）框架受到限制的框架，因为缺乏为医疗而定制的专门检索工具，该HTTP URL提出了一项医学深度研究代理，通过两项核心创新来解决这些挑战。首先，我们使用医学知识图开发了一种新颖的数据综合框架，从稀有医疗实体周围的子图中提取最长的链条，以产生复杂的多跳问答对。其次，我们将定制的私人医疗检索引擎与通用工具集成在一起，从而可以准确地综合医疗信息。我们的方法在12个医学专业中产生了2100多种轨迹，每种方法均可提供4.2工具，该HTTP URL是一个两阶段的培训范式，将监督的微调和在线加强学习与复合奖励结合在一起，我们的Medresearcher-R1-32B模型表明了在医疗方面的竞争性竞争性竞争性竞争性竞争性竞争性竞争性竞争性竞争性竞争性竞争性竞争性竞争性均可进行医学上的新型成果。我们的工作表明，在建筑，工具设计和培训数据构建方面的战略领域特定创新可以使较小的开源模型胜过专业领域中更大的专有系统。

Title: Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs

• Authors: Haokun Lin, Haobo Xu, Yichen Wu, Ziyu Guo, Renrui Zhang, Zhichao Lu, Ying Wei, Qingfu Zhang, Zhenan Sun
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.14896
• Pdf URL: https://arxiv.org/pdf/2508.14896
• Copy Paste: [[2508.14896]] Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs(https://arxiv.org/abs/2508.14896)
• Keywords: language model, llm
• Abstract: Recent advances in diffusion large language models (dLLMs) have introduced a promising alternative to autoregressive (AR) LLMs for natural language generation tasks, leveraging full attention and denoising-based decoding strategies. However, the deployment of these models on edge devices remains challenging due to their massive parameter scale and high resource demands. While post-training quantization (PTQ) has emerged as a widely adopted technique for compressing AR LLMs, its applicability to dLLMs remains largely unexplored. In this work, we present the first systematic study on quantizing diffusion-based language models. We begin by identifying the presence of activation outliers, characterized by abnormally large activation values that dominate the dynamic range. These outliers pose a key challenge to low-bit quantization, as they make it difficult to preserve precision for the majority of values. More importantly, we implement state-of-the-art PTQ methods and conduct a comprehensive evaluation across multiple task types and model variants. Our analysis is structured along four key dimensions: bit-width, quantization method, task category, and model type. Through this multi-perspective evaluation, we offer practical insights into the quantization behavior of dLLMs under different configurations. We hope our findings provide a foundation for future research in efficient dLLM deployment. All codes and experimental setups will be released to support the community.
• 摘要：扩散大语言模型（DLLM）的最新进展已引入了自然语言生成任务的自动回归（AR）LLMS的有希望的替代方案，利用了全部关注和基于denoising的解码策略。但是，由于其庞大的参数规模和高资源需求，这些模型在边缘设备上的部署仍然具有挑战性。虽然训练后量化（PTQ）已成为一种用于压缩AR LLM的广泛采用的技术，但其对DLLMS的适用性仍然在很大程度上没有探索。在这项工作中，我们介绍了第一个有关量化基于扩散的语言模型的系统研究。我们首先确定激活异常值的存在，其特征是主导动态范围的异常大激活值。这些离群值对低位量化构成了关键的挑战，因为它们很难为大多数值保留精度。更重要的是，我们实施最先进的PTQ方法，并对多种任务类型和模型变体进行全面评估。我们的分析沿着四个关键维度结构：位宽度，量化方法，任务类别和模型类型。通过这一多人评估，我们为不同配置下的DLLM的量化行为提供了实用的见解。我们希望我们的发现为有效的DLLM部署研究提供了基础。所有代码和实验设置都将发布以支持社区。