2025-08-14

Title: ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning

• Authors: Shu Zhao, Tan Yu, Anbang Xu, Japinder Singh, Aaditya Shukla, Rama Akkiraju
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2508.09303
• Pdf URL: https://arxiv.org/pdf/2508.09303
• Copy Paste: [[2508.09303]] ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning(https://arxiv.org/abs/2508.09303)
• Keywords: language model, llm, agent
• Abstract: Reasoning-augmented search agents such as Search-R1, trained via reinforcement learning with verifiable rewards (RLVR), demonstrate remarkable capabilities in multi-step information retrieval from external knowledge sources. These agents address the limitations of their parametric memory by dynamically gathering relevant facts to address complex reasoning tasks. However, existing approaches suffer from a fundamental architectural limitation: they process search queries strictly sequentially, even when handling inherently parallelizable and logically independent comparisons. This sequential bottleneck significantly constrains computational efficiency, particularly for queries that require multiple entity comparisons. To address this critical limitation, we propose ParallelSearch, a novel reinforcement learning framework that empowers large language models (LLMs) to recognize parallelizable query structures and execute multiple search operations concurrently. Our approach introduces dedicated reward functions that incentivize the identification of independent query components while preserving answer accuracy through jointly considering correctness, query decomposition quality, and parallel execution benefits. Comprehensive experiments demonstrate that ParallelSearch outperforms state-of-the-art baselines by an average performance gain of 2.9% across seven question-answering benchmarks. Notably, on parallelizable questions, our method achieves a 12.7% performance improvement while requiring only 69.6% of the LLM calls compared to sequential approaches.
• 摘要：推理提出的搜索剂（例如搜索R1）通过具有可验证奖励的强化学习培训（RLVR），在多步中从外部知识来源检索中表明了出色的功能。这些代理通过动态收集相关事实来解决复杂的推理任务，以解决其参数内存的局限性。但是，现有的方法受到基本架构限制的困扰：即使处理固有可行的和逻辑上独立的比较，它们即使在处理固有可行的和逻辑上的比较时，它们也严格处理搜索查询。这种顺序瓶颈显着限制了计算效率，特别是对于需要多个实体比较的查询。为了解决这一关键限制，我们提出了一个新颖的加强学习框架ParellelSearch，该框架赋予了大型语言模型（LLMS）以识别可行的查询结构并同时执行多个搜索操作。我们的方法介绍了专用的奖励功能，可以激励独立查询组件的识别，同时通过共同考虑正确性，查询分解质量和并行执行益处来保持答案的准确性。全面的实验表明，在七个提问基准的基准中，平均搜索的平均绩效增长率为2.9％。值得注意的是，在可行的问题上，我们的方法可以提高绩效的12.7％，而与顺序方法相比，只有69.6％的LLM调用。

Title: Leveraging Large Language Models for Rare Disease Named Entity Recognition

• Authors: Nan Miles Xi, Yu Deng, Lin Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09323
• Pdf URL: https://arxiv.org/pdf/2508.09323
• Copy Paste: [[2508.09323]] Leveraging Large Language Models for Rare Disease Named Entity Recognition(https://arxiv.org/abs/2508.09323)
• Keywords: language model, gpt, llm, prompt, retrieval-augmented generation
• Abstract: Named Entity Recognition (NER) in the rare disease domain poses unique challenges due to limited labeled data, semantic ambiguity between entity types, and long-tail distributions. In this study, we evaluate the capabilities of GPT-4o for rare disease NER under low-resource settings, using a range of prompt-based strategies including zero-shot prompting, few-shot in-context learning, retrieval-augmented generation (RAG), and task-level fine-tuning. We design a structured prompting framework that encodes domain-specific knowledge and disambiguation rules for four entity types. We further introduce two semantically guided few-shot example selection methods to improve in-context performance while reducing labeling effort. Experiments on the RareDis Corpus show that GPT-4o achieves competitive or superior performance compared to BioClinicalBERT, with task-level fine-tuning yielding new state-of-the-art (SOTA) results. Cost-performance analysis reveals that few-shot prompting delivers high returns at low token budgets, while RAG offers marginal additional benefit. An error taxonomy highlights common failure modes such as boundary drift and type confusion, suggesting opportunities for post-processing and hybrid refinement. Our results demonstrate that prompt-optimized LLMs can serve as effective, scalable alternatives to traditional supervised models in biomedical NER, particularly in rare disease applications where annotated data is scarce.
• 摘要：稀有疾病领域中指定的实体识别（NER）构成了独特的挑战，因为标记的数据有限，实体类型之间的语义歧义和长尾分布。在这项研究中，我们使用一系列基于及时的策略，包括零射击促使，很少射击的内在学习，检索授课（RAG）和任务级别的微调，评估了GPT-4O在低资源环境下的稀有疾病NER的能力。我们设计了一个结构化的提示框架，该框架编码针对四种实体类型的特定领域知识和歧义规则。我们进一步介绍了两种语义引导的几示示例选择方法，以改善内在性能，同时减少标签工作。 Raredis语料库上的实验表明，与BioClinicalbert相比，GPT-4O具有竞争性或卓越的性能，任务级的微调产生了新的最先进（SOTA）结果。成本绩效分析表明，很少有弹性促使在低标记预算下提供高回报，而RAG则提供了边际额外的好处。错误的分类法突出了常见的故障模式，例如边界漂移和类型混乱，这表明了后处理和混合精炼的机会。我们的结果表明，迅速优化的LLM可以作为生物医学NER中传统监督模型的有效，可扩展的替代方案，尤其是在稀有注释数据的罕见疾病应用中。

Title: TEN: Table Explicitization, Neurosymbolically

• Authors: Nikita Mehrotra, Aayush Kumar, Sumit Gulwani, Arjun Radhakrishna, Ashish Tiwari
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09324
• Pdf URL: https://arxiv.org/pdf/2508.09324
• Copy Paste: [[2508.09324]] TEN: Table Explicitization, Neurosymbolically(https://arxiv.org/abs/2508.09324)
• Keywords: language model, llm, hallucination, prompt, chain-of-thought
• Abstract: We present a neurosymbolic approach, TEN, for extracting tabular data from semistructured input text. This task is particularly challenging for text input that does not use special delimiters consistently to separate columns and rows. Purely neural approaches perform poorly due to hallucinations and their inability to enforce hard constraints. TEN uses Structural Decomposition prompting - a specialized chain-of-thought prompting approach - on a large language model (LLM) to generate an initial table, and thereafter uses a symbolic checker to evaluate not only the well-formedness of that table, but also detect cases of hallucinations or forgetting. The output of the symbolic checker is processed by a critique-LLM to generate guidance for fixing the table, which is presented to the original LLM in a self-debug loop. Our extensive experiments demonstrate that TEN significantly outperforms purely neural baselines across multiple datasets and metrics, achieving significantly higher exact match accuracy and substantially reduced hallucination rates. A 21-participant user study further confirms that TEN's tables are rated significantly more accurate (mean score: 5.0 vs 4.3; p = 0.021), and are consistently preferred for ease of verification and correction, with participants favoring our method in over 60% of the cases.
• 摘要：我们提出了一种神经肯定方法，即从半结构化输入文本中提取表格数据。对于文本输入而言，此任务尤其具有挑战性，该文本输入不始终使用特殊的定系数将列和行分开。由于幻觉及其无法执行硬性约束，纯粹的神经方法的表现不佳。 TEN使用结构分解提示 - 一种专门的经过思考链的提示方法 - 在大型语言模型（LLM）上生成初始表，此后使用符号检查器不仅评估了该表的良好形式，还可以检测幻觉或遗忘案例。符号检查器的输出由评论-LLM处理，以生成固定表的指导，该指南以自我调音循环中的原始LLM呈现给原始LLM。我们的广泛实验表明，在多个数据集和指标上，十个显着优于纯粹的神经基线，实现了更高的确切匹配准确性，并大大降低了幻觉率。一项21个参与者的用户研究进一步证实，十个表的评分明显更准确（平均得分：5.0 vs 4.3; p = 0.021），并且始终优先考虑易于验证和校正，参与者在60％以上的情况下偏爱我们的方法。

Title: Decoding Neural Emotion Patterns through Natural Language Processing Embeddings

• Authors: Gideon Vos, Maryam Ebrahimpour, Liza van Eijk, Zoltan Sarnyai, Mostafa Rahimi Azghadi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09337
• Pdf URL: https://arxiv.org/pdf/2508.09337
• Copy Paste: [[2508.09337]] Decoding Neural Emotion Patterns through Natural Language Processing Embeddings(https://arxiv.org/abs/2508.09337)
• Keywords: language model, llm
• Abstract: Understanding how emotional expression in language relates to brain function is a challenge in computational neuroscience and affective computing. Traditional neuroimaging is costly and lab-bound, but abundant digital text offers new avenues for emotion-brain mapping. Prior work has largely examined neuroimaging-based emotion localization or computational text analysis separately, with little integration. We propose a computational framework that maps textual emotional content to anatomically defined brain regions without requiring neuroimaging. Using OpenAI's text-embedding-ada-002, we generate high-dimensional semantic representations, apply dimensionality reduction and clustering to identify emotional groups, and map them to 18 brain regions linked to emotional processing. Three experiments were conducted: i) analyzing conversational data from healthy vs. depressed subjects (DIAC-WOZ dataset) to compare mapping patterns, ii) applying the method to the GoEmotions dataset and iii) comparing human-written text with large language model (LLM) responses to assess differences in inferred brain activation. Emotional intensity was scored via lexical analysis. Results showed neuroanatomically plausible mappings with high spatial specificity. Depressed subjects exhibited greater limbic engagement tied to negative affect. Discrete emotions were successfully differentiated. LLM-generated text matched humans in basic emotion distribution but lacked nuanced activation in empathy and self-referential regions (medial prefrontal and posterior cingulate cortex). This cost-effective, scalable approach enables large-scale analysis of naturalistic language, distinguishes between clinical populations, and offers a brain-based benchmark for evaluating AI emotional expression.
• 摘要：了解语言中的情绪表达与大脑功能的关系是计算神经科学和情感计算中的挑战。传统的神经影像学是昂贵和实验室的，但是丰富的数字文本为情感脑图映射提供了新的途径。先前的工作在很大程度上研究了基于神经影像学的情感本地化或计算文本分析，而几乎没有集成。我们提出了一个计算框架，该框架将文本情感内容映射到解剖学上定义的大脑区域，而无需神经影像。使用OpenAI的文本插入-ADA-002，我们生成高维语义表示，应用降低性降低和聚类来识别情感群体，并将其映射到与情感处理相关的18个大脑区域。进行了三个实验：i）分析来自健康与抑郁症受试者（DIAC-WOZ数据集）的对话数据，以比较映射模式，ii）将方法应用于Goemotions数据集和III）将人写的文本与大语言模型（LLM）进行比较，以评估推断脑激活的差异。情绪强度通过词汇分析评分。结果表明，神经解剖学上合理的映射具有高空间特异性。抑郁的受试者表现出与负面影响有关的更大的边缘参与。离散的情绪成功区分了。 LLM生成的文本与人类的基本情绪分布相匹配，但缺乏同理心和自指的区域（内侧前额叶和后扣带回皮层）的细微激活。这种具有成本效益的可扩展方法可以对自然主义语言进行大规模分析，从而区分临床人群，并为评估AI情感表达的基于大脑的基准。

Title: Flow-SLM: Joint Learning of Linguistic and Acoustic Information for Spoken Language Modeling

• Authors: Ju-Chieh Chou, Jiawei Zhou, Karen Livescu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09350
• Pdf URL: https://arxiv.org/pdf/2508.09350
• Copy Paste: [[2508.09350]] Flow-SLM: Joint Learning of Linguistic and Acoustic Information for Spoken Language Modeling(https://arxiv.org/abs/2508.09350)
• Keywords: language model, prompt
• Abstract: Textless spoken language models (SLMs) are generative models of speech that do not rely on text supervision. Most textless SLMs learn to predict the next semantic token, a discrete representation of linguistic content, and rely on a separate vocoder to add acoustic information to the generated speech. Such models have no access to acoustic context and no built-in control over acoustic details. In this work, we propose to jointly model linguistic and acoustic information by generating semantic tokens and a continuous real-valued representation of the acoustic frame. We use a flow-matching objective to predict the continuous vector conditioned on the semantic tokens. We study the design space of this approach and find that predicting multiple future semantic tokens helps preserve linguistic information. Our approach achieves comparable performance to existing models in terms of linguistic likelihood benchmarks, while providing better acoustic detail in prompted generation.
• 摘要：无文本口语模型（SLM）是不依赖文本监督的语音模型。大多数无文本的SLM都学会预测下一个语义令牌，是语言内容的离散表示，并依靠单独的Vocoder来在生成的语音中添加声学信息。这样的模型无法访问声学上下文，也没有对声学细节的内置控制。在这项工作中，我们建议通过产生语义令牌以及对声学框架的连续实现代表来共同对语言和声学信息进行建模。我们使用一个流匹配目标来预测在语义令牌上的连续矢量。我们研究了这种方法的设计空间，并发现预测多个未来的语义令牌有助于保留语言信息。我们的方法在语言可能性基准方面可以实现与现有模型相当的性能，同时在发起的发电中提供了更好的声学细节。

Title: APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplification

• Authors: Artem Chernodub, Aman Saini, Yejin Huh, Vivek Kulkarni, Vipul Raheja
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09378
• Pdf URL: https://arxiv.org/pdf/2508.09378
• Copy Paste: [[2508.09378]] APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplification(https://arxiv.org/abs/2508.09378)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Recent advancements in large language models (LLMs) have enabled a wide range of natural language processing (NLP) tasks to be performed through simple prompt-based interactions. Consequently, several approaches have been proposed to engineer prompts that most effectively enable LLMs to perform a given task (e.g., chain-of-thought prompting). In settings with a well-defined metric to optimize model performance, automatic prompt optimization (APO) methods have been developed to refine a seed prompt. Advancing this line of research, we propose APIO, a simple but effective prompt induction and optimization approach for the tasks of Grammatical Error Correction (GEC) and Text Simplification, without relying on manually specified seed prompts. APIO achieves a new state-of-the-art performance for purely LLM-based prompting methods on these tasks. We make our data, code, prompts, and outputs publicly available.
• 摘要：大型语言模型（LLMS）的最新进展已使通过简单的基于及时的互动执行广泛的自然语言处理（NLP）任务。因此，已经提出了几种方法来提示工程师提示，最有效地使LLMS能够执行给定的任务（例如，经过经过思考的提示）。在具有明确定义的度量以优化模型性能的设置中，已经开发了自动提示优化（APO）方法来完善种子提示。我们提出了APIO的推进，我们提出了一种简单但有效的迅速诱导和优化方法，用于语法误差校正任务（GEC）和文本简化，而无需依赖手动指定的种子提示。 APIO为这些任务纯粹基于LLM的提示方法实现了新的最新性能。我们公开提供数据，代码，提示和输出。

Title: Columbo: Expanding Abbreviated Column Names for Tabular Data Using Large Language Models

• Authors: Ting Cai, Stephen Sheen, AnHai Doan
• Subjects: cs.CL, cs.DB
• Abstract URL: https://arxiv.org/abs/2508.09403
• Pdf URL: https://arxiv.org/pdf/2508.09403
• Copy Paste: [[2508.09403]] Columbo: Expanding Abbreviated Column Names for Tabular Data Using Large Language Models(https://arxiv.org/abs/2508.09403)
• Keywords: language model, llm, chain-of-thought
• Abstract: Expanding the abbreviated column names of tables, such as ``esal'' to ``employee salary'', is critical for numerous downstream data tasks. This problem arises in enterprises, domain sciences, government agencies, and more. In this paper we make three contributions that significantly advances the state of the art. First, we show that synthetic public data used by prior work has major limitations, and we introduce 4 new datasets in enterprise/science domains, with real-world abbreviations. Second, we show that accuracy measures used by prior work seriously undercount correct expansions, and we propose new synonym-aware measures that capture accuracy much more accurately. Finally, we develop Columbo, a powerful LLM-based solution that exploits context, rules, chain-of-thought reasoning, and token-level analysis. Extensive experiments show that Columbo significantly outperforms NameGuess, the current most advanced solution, by 4-29\%, over 5 datasets. Columbo has been used in production on EDI, a major data portal for environmental sciences.
• 摘要：将表的缩写列名称（例如``Esal''''扩展到``员工薪金''，这对于众多下游数据任务至关重要。这个问题出现在企业，领域科学，政府机构等。在本文中，我们做出了三项贡献，可以显着提高最新技术状态。首先，我们表明先前工作使用的综合公共数据具有重大限制，并且我们在企业/科学领域中介绍了4个新数据集，并具有现实的缩写。其次，我们表明，先前工作使用的准确度措施严重降低了正确的扩展，并且我们提出了更准确地捕获准确性的新的同义词感知措施。最后，我们开发了Columbo，这是一种强大的基于LLM的解决方案，可利用上下文，规则，经过思考推理和令牌级别的分析。广泛的实验表明，在5个数据集中，Columbo显着优于4-29 \％的当前最先进的解决方案的名称。 Columbo已用于EDI的生产，EDI是环境科学的主要数据门户。

Title: From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text

• Authors: Ridwan Mahbub, Mohammed Saidul Islam, Mir Tafseer Nayeem, Md Tahmid Rahman Laskar, Mizanur Rahman, Shafiq Joty, Enamul Hoque
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09450
• Pdf URL: https://arxiv.org/pdf/2508.09450
• Copy Paste: [[2508.09450]] From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text(https://arxiv.org/abs/2508.09450)
• Keywords: language model, gpt, prompt
• Abstract: Charts are very common for exploring data and communicating insights, but extracting key takeaways from charts and articulating them in natural language can be challenging. The chart-to-text task aims to automate this process by generating textual summaries of charts. While with the rapid advancement of large Vision-Language Models (VLMs), we have witnessed great progress in this domain, little to no attention has been given to potential biases in their outputs. This paper investigates how VLMs can amplify geo-economic biases when generating chart summaries, potentially causing societal harm. Specifically, we conduct a large-scale evaluation of geo-economic biases in VLM-generated chart summaries across 6,000 chart-country pairs from six widely used proprietary and open-source models to understand how a country's economic status influences the sentiment of generated summaries. Our analysis reveals that existing VLMs tend to produce more positive descriptions for high-income countries compared to middle- or low-income countries, even when country attribution is the only variable changed. We also find that models such as GPT-4o-mini, Gemini-1.5-Flash, and Phi-3.5 exhibit varying degrees of bias. We further explore inference-time prompt-based debiasing techniques using positive distractors but find them only partially effective, underscoring the complexity of the issue and the need for more robust debiasing strategies. Our code and dataset are publicly available here.
• 摘要：图表对于探索数据和交流见解非常普遍，但是从图表中提取关键要点并以自然语言表达它们可能是具有挑战性的。图表到文本任务旨在通过生成图表的文本摘要来自动化此过程。随着大型视觉模型（VLM）的快速发展，我们目睹了该领域的巨大进步，但几乎没有引起其产量潜在偏见的关注。本文研究了VLM在产生图表摘要时如何扩大地质经济偏见，可能造成社会伤害。具体而言，我们对VLM生成的图表摘要中的地理经济偏见进行了大规模评估，从六个广泛使用的专有和开源模型中的6,000对图表摘要中，以了解一个国家的经济状况如何影响生成的摘要的观点。我们的分析表明，即使国家归因是唯一的变量，现有的VLM倾向于对高收入国家产生更积极的描述。我们还发现，诸如GPT-4O-MINI，GEMINI-1.5-FLASH和PHI-3.5之类的模型表现出不同程度的偏差。我们进一步使用积极的干扰器探索了基于推理的及时偏差技术，但发现它们仅部分有效，强调了问题的复杂性以及对更强大的偏见策略的需求。我们的代码和数据集在此处公开可用。

Title: User-centric Subjective Leaderboard by Customizable Reward Modeling

• Authors: Qi Jia, Xiujie Song, Zicheng Zhang, Yijin Guo, Kaiwei Zhang, Zijian Chen, Guangtao Zhai
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09463
• Pdf URL: https://arxiv.org/pdf/2508.09463
• Copy Paste: [[2508.09463]] User-centric Subjective Leaderboard by Customizable Reward Modeling(https://arxiv.org/abs/2508.09463)
• Keywords: language model, gpt, llm
• Abstract: Existing benchmarks for large language models (LLMs) predominantely focus on assessing their capabilities through verifiable tasks. Such objective and static benchmarks offer limited utility for practical LLM selection, making it difficult for users to find suitable models for their individual needs. To bridge this gap, we present the first User-Centric Subjective Leaderboard (USL), which provides a preference-driven, dynamic ranking of LLMs across diverse real-world scenarios. Our work is built upon a thorough investigation of real human preference data, involving more than 10K subjective queries. Our investigation reveals significant diversity and contradictions in human preferences, which limit the effectiveness of state-of-the-art reward models. To address this, we introduce Customizable Reward Models (CRMs). With only 4B parameters, our CRM surpasses the performance of leading models such as GPT-4.1 and Gemini-2.5-pro, showing exceptional generalization capabilities across new topics and criteria. The USL, powered by CRMs, exhibits strong negative correlations to contradictory preferences.
• 摘要：大型语言模型（LLMS）的现有基准主要专注于通过可验证的任务评估其功能。这种客观和静态基准测试为实用的LLM选择提供了有限的实用性，使用户很难找到适合其个人需求的模型。为了弥合这一差距，我们介绍了第一个以用户为中心的主观排行榜（USL），该排行榜（USL）在不同的真实世界方案中提供了偏好驱动的，动态的LLMS。我们的工作是基于对真实人类偏好数据的彻底调查，涉及超过10k的主观查询。我们的调查揭示了人类偏好的巨大多样性和矛盾，这限制了最先进的奖励模型的有效性。为了解决这个问题，我们介绍了可自定义的奖励模型（CRMS）。只有4B参数，我们的CRM超过了GPT-4.1和Gemini-2.5-Pro等领先模型的性能，在新主题和标准上显示出非凡的概括能力。由CRM提供支持的USL与矛盾的偏好表现出强烈的负相关。

Title: Learning Facts at Scale with Active Reading

• Authors: Jessy Lin, Vincent-Pierre Berges, Xilun Chen, Wen-Tau Yih, Gargi Ghosh, Barlas Oğuz
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09494
• Pdf URL: https://arxiv.org/pdf/2508.09494
• Copy Paste: [[2508.09494]] Learning Facts at Scale with Active Reading(https://arxiv.org/abs/2508.09494)
• Keywords: llm
• Abstract: LLMs are known to store vast amounts of knowledge in their parametric memory. However, learning and recalling facts from this memory is known to be unreliable, depending largely on the prevalence of particular facts in the training data and other factors which are poorly understood. Practitioners are lacking tools which will allow them to ensure that the models learn a given body of knowledge reliably and consistently. To this end, we propose Active Reading: a framework where we train models to study a given set of material with self-generated learning strategies. First, we demonstrate models trained with Active Reading on expert domains absorb significantly more knowledge than vanilla finetuning and other data augmentations. We train expert 8B models that achieve 66% on a Wikipedia-grounded subset of SimpleQA (+313% relative over vanilla finetuning) and 26% on FinanceBench (+160% relative over vanilla finetuning) by applying Active Reading to the source documents for each benchmark. Finally, we show that Active Reading can be utilized at pre-training scale to build more factual models. As a demonstration of this, we release Meta WikiExpert-8B, a Wikipedia-expert model trained on 1 trillion generated tokens, which outcompetes models with hundreds of billions of parameters on factual QA.
• 摘要：已知LLM可以在其参数内存中存储大量知识。但是，已知从该记忆中学习和回忆事实是不可靠的，这在很大程度上取决于训练数据中特定事实的普遍性以及知识较低的其他因素。从业者缺乏工具，这些工具将使他们确保模型可靠，一致地学习给定的知识体系。为此，我们提出了积极的阅读：一个框架，在该框架中，我们训练模型以自我生成的学习策略研究给定的一组材料。首先，我们展示了在专家领域进行积极阅读训练的模型，比香草芬特和其他数据增强物的知识要多得多。我们训练专家8B模型，在Wikipedia接地的SimpleQA（Vanilla FineTuning的相对+313％）上获得66％的型号，而FinanceBench（Vanilla FineTuning的相对相对+160％）通过将主动阅读应用于每个基准标准的源文字，从而在FinanceBench上（+160％的相对相对）。最后，我们表明可以在训练量表上使用主动阅读以构建更多的事实模型。为了证明这一点，我们发布了Meta Wikiexpert-8b，这是一种在1万亿个生成的令牌上训练的Wikipedia-Expert模型，该模型在事实QA上以数千亿个参数的速度击败了模型。

Title: From Ranking to Selection: A Simple but Efficient Dynamic Passage Selector for Retrieval Augmented Generation

• Authors: Siyuan Meng, Junming Liu, Yirong Chen, Song Mao, Pinlong Cai, Guohang Yan, Botian Shi, Ding Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09497
• Pdf URL: https://arxiv.org/pdf/2508.09497
• Copy Paste: [[2508.09497]] From Ranking to Selection: A Simple but Efficient Dynamic Passage Selector for Retrieval Augmented Generation(https://arxiv.org/abs/2508.09497)
• Keywords: gpt, retrieval augmented generation, retrieval-augmented generation
• Abstract: Retrieval-augmented generation (RAG) systems are often bottlenecked by their reranking modules, which typically score passages independently and select a fixed Top-K size. This approach struggles with complex multi-hop queries that require synthesizing evidence across multiple documents, creating a trade-off where small K values omit crucial information and large K values introduce noise. To address this, we introduce the Dynamic Passage Selector (DPS), a novel reranking framework that treats passage selection as a supervised learning problem. Unlike traditional point-wise or list-wise methods, DPS is fine-tuned to capture inter-passage dependencies and dynamically select the most relevant set of passages for generation. As a seamless plug-and-play module, DPS requires no modifications to the standard RAG pipeline. Comprehensive evaluations on five benchmarks show that DPS consistently outperforms state-of-the-art rerankers and fine-tuning methods. Notably, on the challenging MuSiQue dataset, DPS improves the F1-score by 30.06% and 15.4% over strong baselines like Qwen3-reranker and RankingGPT, respectively. Our results demonstrate that by enabling adaptive evidence selection, DPS substantially enhances reasoning capabilities in complex RAG scenarios.
• 摘要：检索增强的生成（RAG）系统通常会被其重新播放模块瓶颈，该模块通常独立评分段落并选择固定的TOP-K大小。这种方法与需要跨多个文档合成证据的复杂多跳的查询进行斗争，从而创造了一个折衷的，其中小k值省略了关键信息，而大k值则引入了噪声。为了解决这个问题，我们介绍了动态通道选择器（DPS），这是一种新颖的重新依赖框架，将通道选择视为有监督的学习问题。与传统的角度或列表方法不同，DPS经过微调以捕获通行间的依赖关系，并动态选择了最相关的一组生成段落。作为无缝插件模块，DPS不需要对标准抹布管道进行任何修改。对五个基准测试的全面评估表明，DPS始终优于最先进的阅读者和微调方法。值得注意的是，在具有挑战性的Musique数据集上，DPS分别在QWEN3-Reranker和CarkingGPT等强基础上提高了F1得分30.06％和15.4％。我们的结果表明，通过实现自适应证据选择，DPS可以大大提高复杂的抹布场景中的推理能力。

Title: LACA: Improving Cross-lingual Aspect-Based Sentiment Analysis with LLM Data Augmentation

• Authors: Jakub Šmíd, Pavel Přibáň, Pavel Král
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09515
• Pdf URL: https://arxiv.org/pdf/2508.09515
• Copy Paste: [[2508.09515]] LACA: Improving Cross-lingual Aspect-Based Sentiment Analysis with LLM Data Augmentation(https://arxiv.org/abs/2508.09515)
• Keywords: language model, llm, prompt
• Abstract: Cross-lingual aspect-based sentiment analysis (ABSA) involves detailed sentiment analysis in a target language by transferring knowledge from a source language with available annotated data. Most existing methods depend heavily on often unreliable translation tools to bridge the language gap. In this paper, we propose a new approach that leverages a large language model (LLM) to generate high-quality pseudo-labelled data in the target language without the need for translation tools. First, the framework trains an ABSA model to obtain predictions for unlabelled target language data. Next, LLM is prompted to generate natural sentences that better represent these noisy predictions than the original text. The ABSA model is then further fine-tuned on the resulting pseudo-labelled dataset. We demonstrate the effectiveness of this method across six languages and five backbone models, surpassing previous state-of-the-art translation-based approaches. The proposed framework also supports generative models, and we show that fine-tuned LLMs outperform smaller multilingual models.
• 摘要：跨语言基于方面的情感分析（ABSA）涉及目标语言中的详细情感分析，通过从源语言中转移带有可用的带注释数据的知识。大多数现有的方法在很大程度上取决于通常不可靠的翻译工具来弥合语言差距。在本文中，我们提出了一种新方法，该方法利用大型语言模型（LLM）以目标语言生成高质量的伪标记数据，而无需翻译工具。首先，该框架训练ABSA模型，以获取未标记目标语言数据的预测。接下来，提示LLM生成比原始文本更好地表示这些嘈杂预测的自然句子。然后，在所得的伪标记的数据集上进一步微调ABSA模型。我们证明了这种方法在六种语言和五个骨干模型上的有效性，超过了先前的基于最新翻译的方法。提出的框架还支持生成模型，我们表明，微调的LLMS的表现要比较小的多语言模型。

Title: Cross-lingual Aspect-Based Sentiment Analysis: A Survey on Tasks, Approaches, and Challenges

• Authors: Jakub Šmíd, Pavel Král
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09516
• Pdf URL: https://arxiv.org/pdf/2508.09516
• Copy Paste: [[2508.09516]] Cross-lingual Aspect-Based Sentiment Analysis: A Survey on Tasks, Approaches, and Challenges(https://arxiv.org/abs/2508.09516)
• Keywords: llm
• Abstract: Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task that focuses on understanding opinions at the aspect level, including sentiment towards specific aspect terms, categories, and opinions. While ABSA research has seen significant progress, much of the focus has been on monolingual settings. Cross-lingual ABSA, which aims to transfer knowledge from resource-rich languages (such as English) to low-resource languages, remains an under-explored area, with no systematic review of the field. This paper aims to fill that gap by providing a comprehensive survey of cross-lingual ABSA. We summarize key ABSA tasks, including aspect term extraction, aspect sentiment classification, and compound tasks involving multiple sentiment elements. Additionally, we review the datasets, modelling paradigms, and cross-lingual transfer methods used to solve these tasks. We also examine how existing work in monolingual and multilingual ABSA, as well as ABSA with LLMs, contributes to the development of cross-lingual ABSA. Finally, we highlight the main challenges and suggest directions for future research to advance cross-lingual ABSA systems.
• 摘要：基于方面的情感分析（ABSA）是一项精细的情感分析任务，重点是理解方面的观点，包括对特定方面术语，类别和意见的情感。尽管ABSA研究取得了重大进展，但大部分重点一直放在单语环境上。跨语言ABSA旨在将知识从资源丰富的语言（例如英语）转移到低资源语言，仍然是一个不足的领域，没有对该领域进行系统的审查。本文旨在通过对跨语性ABSA进行全面调查来填补这一空白。我们总结了关键的ABSA任务，包括方面术语提取，方面情感分类以及涉及多个情感元素的复合任务。此外，我们回顾了用于解决这些任务的数据集，建模范例和跨语性转移方法。我们还研究了单语和多语言ABS的现有工作以及与LLM的ABSA如何有助于跨语性ABSA的发展。最后，我们强调了主要的挑战，并提出了未来研究的方向，以推动跨语义的ABSA系统。

Title: UWBa at SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval

• Authors: Ladislav Lenc, Daniel Cífka, Jiří Martínek, Jakub Šmíd, Pavel Král
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09517
• Pdf URL: https://arxiv.org/pdf/2508.09517
• Copy Paste: [[2508.09517]] UWBa at SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval(https://arxiv.org/abs/2508.09517)
• Keywords: language model, gpt
• Abstract: This paper presents a zero-shot system for fact-checked claim retrieval. We employed several state-of-the-art large language models to obtain text embeddings. The models were then combined to obtain the best possible result. Our approach achieved 7th place in monolingual and 9th in cross-lingual subtasks. We used only English translations as an input to the text embedding models since multilingual models did not achieve satisfactory results. We identified the most relevant claims for each post by leveraging the embeddings and measuring cosine similarity. Overall, the best results were obtained by the NVIDIA NV-Embed-v2 model. For some languages, we benefited from model combinations (NV-Embed & GPT or Mistral).
• 摘要：本文提出了一个零拍系统，用于检查事实检查的索赔检索。我们采用了几种最先进的大语模型来获取文本嵌入。然后将模型合并以获得最佳结果。我们的方法在单语言中获得了第七名，在跨语言子任务中获得了第七名。我们仅将英语翻译用作文本嵌入模型的输入，因为多语言模型没有达到令人满意的结果。我们通过利用嵌入并测量余弦相似性来确定每个职位的最相关主张。总体而言，最好的结果是通过NVIDIA NV-EMBED-V2模型获得的。对于某些语言，我们从模型组合（NV Embed＆GPT或Mistral）中受益。

Title: The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage

• Authors: Skyler Hallinan, Jaehun Jung, Melanie Sclar, Ximing Lu, Abhilasha Ravichander, Sahana Ramnath, Yejin Choi, Sai Praneeth Karimireddy, Niloofar Mireshghallah, Xiang Ren
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09603
• Pdf URL: https://arxiv.org/pdf/2508.09603
• Copy Paste: [[2508.09603]] The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage(https://arxiv.org/abs/2508.09603)
• Keywords: language model, gpt
• Abstract: Membership inference attacks serves as useful tool for fair use of language models, such as detecting potential copyright infringement and auditing data leakage. However, many current state-of-the-art attacks require access to models' hidden states or probability distribution, which prevents investigation into more widely-used, API-access only models like GPT-4. In this work, we introduce N-Gram Coverage Attack, a membership inference attack that relies solely on text outputs from the target model, enabling attacks on completely black-box models. We leverage the observation that models are more likely to memorize and subsequently generate text patterns that were commonly observed in their training data. Specifically, to make a prediction on a candidate member, N-Gram Coverage Attack first obtains multiple model generations conditioned on a prefix of the candidate. It then uses n-gram overlap metrics to compute and aggregate the similarities of these outputs with the ground truth suffix; high similarities indicate likely membership. We first demonstrate on a diverse set of existing benchmarks that N-Gram Coverage Attack outperforms other black-box methods while also impressively achieving comparable or even better performance to state-of-the-art white-box attacks - despite having access to only text outputs. Interestingly, we find that the success rate of our method scales with the attack compute budget - as we increase the number of sequences generated from the target model conditioned on the prefix, attack performance tends to improve. Having verified the accuracy of our method, we use it to investigate previously unstudied closed OpenAI models on multiple domains. We find that more recent models, such as GPT-4o, exhibit increased robustness to membership inference, suggesting an evolving trend toward improved privacy protections.
• 摘要：会员推理攻击是合理使用语言模型的有用工具，例如检测潜在的版权侵权和审核数据泄漏。但是，许多当前的最新攻击需要访问模型的隐藏状态或概率分布，这阻止了对更广泛使用的API访问的调查，仅GPT-4之类的模型。在这项工作中，我们引入了N-Gram Coverage Attack，这是一种成员资格推理攻击，仅依赖于目标模型的文本输出，从而攻击了完全黑框模型。我们利用这样的观察结果，即模型更有可能记住并随后生成在训练数据中通常观察到的文本模式。具体而言，要对候选成员进行预测，N-Gram覆盖范围攻击首先获得了以候选人前缀为条件的多个模型世代。然后，它使用n-gram重叠指标来计算和汇总这些输出与地面真实后缀的相似性；高相似之处表明可能会员资格。我们首先证明了一套现有的基准集，即N-Gram覆盖范围攻击的表现优于其他黑盒方法，同时也可以在最先进的白色盒子攻击中实现可比较甚至更好的性能 - 尽管只能访问文本输出。有趣的是，我们发现我们的方法的成功率随攻击计算预算的量表 - 随着我们增加了从前缀为条件的目标模型产生的序列数量，攻击性能往往会提高。验证了我们方法的准确性后，我们使用它来研究以前未研究的封闭openai模型。我们发现，诸如GPT-4O之类的最新模型表现出对成员推理的鲁棒性，这表明朝着改善隐私保护的趋势不断发展。

Title: AINL-Eval 2025 Shared Task: Detection of AI-Generated Scientific Abstracts in Russian

• Authors: Tatiana Batura, Elena Bruches, Milana Shvenk, Valentin Malykh
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09622
• Pdf URL: https://arxiv.org/pdf/2508.09622
• Copy Paste: [[2508.09622]] AINL-Eval 2025 Shared Task: Detection of AI-Generated Scientific Abstracts in Russian(https://arxiv.org/abs/2508.09622)
• Keywords: language model, gpt, llm, chat
• Abstract: The rapid advancement of large language models (LLMs) has revolutionized text generation, making it increasingly difficult to distinguish between human- and AI-generated content. This poses a significant challenge to academic integrity, particularly in scientific publishing and multilingual contexts where detection resources are often limited. To address this critical gap, we introduce the AINL-Eval 2025 Shared Task, specifically focused on the detection of AI-generated scientific abstracts in Russian. We present a novel, large-scale dataset comprising 52,305 samples, including human-written abstracts across 12 diverse scientific domains and AI-generated counterparts from five state-of-the-art LLMs (GPT-4-Turbo, Gemma2-27B, Llama3.3-70B, Deepseek-V3, and GigaChat-Lite). A core objective of the task is to challenge participants to develop robust solutions capable of generalizing to both (i) previously unseen scientific domains and (ii) models not included in the training data. The task was organized in two phases, attracting 10 teams and 159 submissions, with top systems demonstrating strong performance in identifying AI-generated content. We also establish a continuous shared task platform to foster ongoing research and long-term progress in this important area. The dataset and platform are publicly available at this https URL.
• 摘要：大型语言模型（LLMS）的快速发展彻底改变了文本生成，使得区分人类和AI生成的内容变得越来越困难。这对学术完整性构成了重大挑战，尤其是在科学出版和多语言环境中，在这些环境中，检测资源通常受到限制。为了解决这个关键的差距，我们介绍了AINL-Eval 2025共享任务，专门针对俄罗斯AI生成的科学摘要的检测。我们提供了一个新颖的大型数据集，其中包括52,305个样本，包括来自12种不同科学领域的人文摘要，以及来自五个最先进的LLMS（GPT-4-Turbo，gemma2-27b，llama3.3-70b，deepseek-v3，deepseek-v3，depseek-v3，gigachat-lite and gigachat-lite）的五个不同科学领域的人为摘要。该任务的核心目标是挑战参与者，开发能够概括的解决方案，以概括（i）以前看不见的科学领域和（ii）未包含在培训数据中的模型。该任务分为两个阶段，吸引了10个团队和159个意见，其中顶级系统在识别AI生成的内容方面表现出强大的性能。我们还建立了一个连续的共享任务平台，以促进该重要领域的持续研究和长期进步。该数据集和平台在此HTTPS URL上可公开可用。

Title: Improving Diversity in Language Models: When Temperature Fails, Change the Loss

• Authors: Alexandre Verine, Florian Le Bronnec, Kunhao Zheng, Alexandre Allauzen, Yann Chevaleyre, Benjamin Negrevergne
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.09654
• Pdf URL: https://arxiv.org/pdf/2508.09654
• Copy Paste: [[2508.09654]] Improving Diversity in Language Models: When Temperature Fails, Change the Loss(https://arxiv.org/abs/2508.09654)
• Keywords: language model
• Abstract: Increasing diversity in language models is a challenging yet essential objective. A common approach is to raise the decoding temperature. In this work, we investigate this approach through a simplistic yet common case to provide insights into why decreasing temperature can improve quality (Precision), while increasing it often fails to boost coverage (Recall). Our analysis reveals that for a model to be effectively tunable through temperature adjustments, it must be trained toward coverage. To address this, we propose rethinking loss functions in language models by leveraging the Precision-Recall framework. Our results demonstrate that this approach achieves a substantially better trade-off between Precision and Recall than merely combining negative log-likelihood training with temperature scaling. These findings offer a pathway toward more versatile and robust language modeling techniques.
• 摘要：语言模型的多样性增加是一个具有挑战性但基本的目标。一种常见的方法是提高解码温度。在这项工作中，我们通过一种简单而常见的情况来研究这种方法，以提供有关为什么降低温度可以提高质量（精度）的洞察力，同时增加了温度通常无法提高覆盖范围（回忆）。我们的分析表明，要通过温度调节有效调整模型，必须将其训练以覆盖范围。为了解决这个问题，我们通过利用Precision-Recall框架来提出语言模型中的重新思考损失功能。我们的结果表明，这种方法在精确度和召回之间实现了比仅将负对数似然训练与温度缩放结合在一起的更好的权衡。这些发现为通往更广泛和强大的语言建模技术提供了途径。

Title: EffiEval: Efficient and Generalizable Model Evaluation via Capability Coverage Maximization

• Authors: Yaoning Wang, Jiahao Ying, Yixin Cao, Yubo Ma, Yugang Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09662
• Pdf URL: https://arxiv.org/pdf/2508.09662
• Copy Paste: [[2508.09662]] EffiEval: Efficient and Generalizable Model Evaluation via Capability Coverage Maximization(https://arxiv.org/abs/2508.09662)
• Keywords: language model, llm
• Abstract: The rapid advancement of large language models (LLMs) and the development of increasingly large and diverse evaluation benchmarks have introduced substantial computational challenges for model assessment. In this paper, we present EffiEval, a training-free approach for efficient benchmarking that effectively addresses data redundancy while maintaining high evaluation reliability. Our method is specifically designed to meet three key criteria for high-quality evaluation: representativeness, by ensuring comprehensive coverage of model capabilities; fairness, by remaining independent of model performance during sample selection to avoid bias; and generalizability, by enabling flexible transfer across datasets and model families without reliance on large-scale evaluation data. Unlike traditional methods that rely on absolute performance or require extensive evaluation data, our approach adaptively selects high-quality representative subsets based on the Model Utility Index (MUI). Extensive experiments on multiple public benchmarks and diverse LLMs demonstrate that EffiEval achieves strong ranking consistency with full-dataset evaluation using only a small fraction of the original data. Furthermore, our method is flexible and scalable in size, allowing users to balance evaluation efficiency and representativeness according to specific needs. Overall, EffiEval provides a practical and generalizable solution for reliable, fair, and efficient evaluation in the era of LLMs.
• 摘要：大型语言模型（LLM）的快速发展以及日益大而多样化的评估基准的发展引入了模型评估的重大计算挑战。在本文中，我们提出了一种无培训的方法，用于有效地解决数据冗余，同时保持高评估可靠性。我们的方法是专门设计的，旨在符合高质量评估的三个关键标准：代表性，通过确保对模型能力的全面覆盖；公平，通过在样本选择过程中独立于模型性能，以避免偏见；和通用性，通过在不依赖大规模评估数据的情况下启用跨数据集和模型家庭的灵活传输。与依赖绝对性能或需要大量评估数据的传统方法不同，我们的方法根据模型实用程序指数（MUI）自适应选择高质量的代表性子集。对多个公共基准和不同LLM的广泛实验表明，Effieval仅使用原始数据的一小部分就可以通过全数据库评估来实现强大的排名一致性。此外，我们的方法的尺寸是灵活且可扩展的，使用户可以根据特定需求平衡评估效率和代表性。总体而言，Effieval为LLM时代提供了可靠，公正和有效评估的实用和可推广的解决方案。

Title: Slow Tuning and Low-Entropy Masking for Safe Chain-of-Thought Distillation

• Authors: Ziyang Ma, Qingyue Yuan, Linhai Zhang, Deyu Zhou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09666
• Pdf URL: https://arxiv.org/pdf/2508.09666
• Copy Paste: [[2508.09666]] Slow Tuning and Low-Entropy Masking for Safe Chain-of-Thought Distillation(https://arxiv.org/abs/2508.09666)
• Keywords: language model, gpt, llm, chain-of-thought
• Abstract: Previous chain-of-thought (CoT) distillation methods primarily focused on enhancing the reasoning capabilities of Small Language Models (SLMs) by utilizing high-quality rationales generated by powerful Large Language Models (LLMs, e.g., GPT-4). However, few works have noted the negative effects on SLM safety brought by the training, which are revealed in this study. Although there are works on safety alignment that fine-tune language models or manipulate model weights to defend against harmful inputs, they require extra computation or annotated data, and probably impact the reasoning ability of SLMs. In this paper, we investigate how to maintain the safety of SLMs during the CoT distillation process. Specifically, we propose a safe distillation method, Slow Tuning and Low-Entropy Masking Distillation (SLowED), containing two modules: Slow Tuning and Low-Entropy Masking. Slow Tuning scales down the magnitude of model weight changes to optimize the model weights in the neighboring space near the initial weight distribution. Low-Entropy Masking masks low-entropy tokens, which are regarded as unnecessary learning targets, to exclude them from fine-tuning. Experiments on three SLMs (Qwen2.5-1.5B, Llama-3.2-1B, BLOOM-1.1B) across reasoning benchmarks (BBH, BB-Sub, ARC, AGIEval) and safety evaluation (AdvBench) show that SLowED retains the safety of SLMs and comparably improves their reasoning capability compared to existing distillation methods. Furthermore, our ablation study presents the effectiveness of Slow Tuning and Low-Entropy Masking, with the former maintaining the model's safety in the early stage and the latter prolonging the safe training epochs.
• 摘要：以前的思考链（COT）蒸馏方法主要集中于通过利用强大的大语言模型（例如LLMS，例如GPT-4）生成的高质量理由来增强小语言模型（SLM）的推理能力。但是，很少有作品注意到培训对SLM安全的负面影响，这在本研究中揭示了。尽管有一些关于安全一致性的作品，即微调语言模型或操纵模型权重以防御有害输入，但它们需要额外的计算或带注释的数据，并且可能会影响SLM的推理能力。在本文中，我们研究了如何在COT蒸馏过程中维持SLM的安全性。具体而言，我们提出了一种安全的蒸馏方法，缓慢的调整和低渗透掩盖蒸馏（放慢），包含两个模块：缓慢的调谐和低渗透掩模。慢速调整缩放模型重量的大小变化，以优化相邻空间中初始重量分布附近的模型重量。低渗透掩盖面罩的低渗透令牌被认为是不必要的学习目标，可以将它们排除在微调之外。在推理基准（BBH，BB-SUB，ARC，ARC，AGIEVAL）和安全评估（AdvBench）上进行了三个SLM（QWEN2.5-1.5B，LLAMA-3.2-1B，BLOOM-1.1B）的实验，表明SLM的安全性慢了，并且与现有的蒸馏方法相比，SLM的安全性提高了，并提高了其推理能力。此外，我们的消融研究介绍了缓慢调整和低渗透掩蔽的有效性，前者在早期阶段保持模型的安全性，后者延长了安全的训练时代。

Title: Evaluating the Role of Large Language Models in Legal Practice in India

• Authors: Rahul Hemrajani (National Law School of India University, Bengaluru)
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09713
• Pdf URL: https://arxiv.org/pdf/2508.09713
• Copy Paste: [[2508.09713]] Evaluating the Role of Large Language Models in Legal Practice in India(https://arxiv.org/abs/2508.09713)
• Keywords: language model, gpt, llm, hallucination
• Abstract: The integration of Artificial Intelligence(AI) into the legal profession raises significant questions about the capacity of Large Language Models(LLM) to perform key legal tasks. In this paper, I empirically evaluate how well LLMs, such as GPT, Claude, and Llama, perform key legal tasks in the Indian context, including issue spotting, legal drafting, advice, research, and reasoning. Through a survey experiment, I compare outputs from LLMs with those of a junior lawyer, with advanced law students rating the work on helpfulness, accuracy, and comprehensiveness. LLMs excel in drafting and issue spotting, often matching or surpassing human work. However, they struggle with specialised legal research, frequently generating hallucinations, factually incorrect or fabricated outputs. I conclude that while LLMs can augment certain legal tasks, human expertise remains essential for nuanced reasoning and the precise application of law.
• 摘要：人工智能（AI）融入法律职业中提出了有关大语模型（LLM）执行关键法律任务的能力的重大问题。在本文中，我从经验上评估了LLM，例如GPT，Claude和Llama在印度背景下执行关键的法律任务，包括发行发现，法律起草，建议，研究和推理。通过调查实验，我将LLM的产出与初级律师的产出与高级法律专业的学生将工作评价为有益，准确性和全面性的高级学生。 LLM在起草和发行发现方面表现出色，经常与人工作品相匹配或超越人工工作。但是，他们在专门的法律研究中挣扎，经常产生幻觉，实际上不正确或捏造的产出。我得出的结论是，尽管LLM可以扩大某些法律任务，但人类专业知识对于细微的推理和确切的法律应用仍然至关重要。

Title: The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Models

• Authors: Ridwan Mahbub, Mohammed Saidul Islam, Md Tahmid Rahman Laskar, Mizanur Rahman, Mir Tafseer Nayeem, Enamul Hoque
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09716
• Pdf URL: https://arxiv.org/pdf/2508.09716
• Copy Paste: [[2508.09716]] The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Models(https://arxiv.org/abs/2508.09716)
• Keywords: language model
• Abstract: Information visualizations are powerful tools that help users quickly identify patterns, trends, and outliers, facilitating informed decision-making. However, when visualizations incorporate deceptive design elements-such as truncated or inverted axes, unjustified 3D effects, or violations of best practices-they can mislead viewers and distort understanding, spreading misinformation. While some deceptive tactics are obvious, others subtly manipulate perception while maintaining a facade of legitimacy. As Vision-Language Models (VLMs) are increasingly used to interpret visualizations, especially by non-expert users, it is critical to understand how susceptible these models are to deceptive visual designs. In this study, we conduct an in-depth evaluation of VLMs' ability to interpret misleading visualizations. By analyzing over 16,000 responses from ten different models across eight distinct types of misleading chart designs, we demonstrate that most VLMs are deceived by them. This leads to altered interpretations of charts, despite the underlying data remaining the same. Our findings highlight the need for robust safeguards in VLMs against visual misinformation.
• 摘要：信息可视化是强大的工具，可帮助用户快速识别模式，趋势和离群值，从而促进明智的决策。但是，当可视化构成欺骗性的设计元素时，例如截断或倒轴，不合理的3D效果或违反最佳实践的行为 - 他们可能会误导观众和扭曲理解，从而传播错误的信息。尽管某些欺骗性策略是显而易见的，但其他策略则在保持合法性的外观时巧妙地操纵感知。由于视觉模型（VLM）越来越多地用于解释可视化，尤其是非专家用户，因此了解这些模型对欺骗性视觉设计的敏感程度至关重要。在这项研究中，我们对VLMS解释误导可视化的能力进行了深入的评估。通过分析从八种不同类型的误导图表设计中分析十个不同模型的16,000多个响应，我们证明了大多数VLM都被它们欺骗了。尽管基本数据保持不变，但这导致了图表的解释。我们的发现强调了在VLMS中需要防止视觉错误信息的强大保障措施的必要性。

Title: Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning

• Authors: Vaishnavi Shrivastava, Ahmed Awadallah, Vidhisha Balachandran, Shivam Garg, Harkirat Behl, Dimitris Papailiopoulos
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.09726
• Pdf URL: https://arxiv.org/pdf/2508.09726
• Copy Paste: [[2508.09726]] Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning(https://arxiv.org/abs/2508.09726)
• Keywords: language model
• Abstract: Large language models trained with reinforcement learning with verifiable rewards tend to trade accuracy for length--inflating response lengths to achieve gains in accuracy. While longer answers may be warranted for harder problems, many tokens are merely "filler": repetitive, verbose text that makes no real progress. We introduce GFPO (Group Filtered Policy Optimization), which curbs this length explosion by sampling larger groups per problem during training and filtering responses to train on based on two key metrics: (1) response length and (2) token efficiency: reward per token ratio. By sampling more at training time, we teach models to think less at inference time. On the Phi-4-reasoning model, GFPO cuts GRPO's length inflation by 46-71% across challenging STEM and coding benchmarks (AIME 24/25, GPQA, Omni-MATH, LiveCodeBench) while maintaining accuracy. Optimizing for reward per token further increases reductions in length inflation to 71-85%. We also propose Adaptive Difficulty GFPO, which dynamically allocates more training resources to harder problems based on real-time difficulty estimates, improving the balance between computational efficiency and accuracy especially on difficult questions. GFPO demonstrates that increased training-time compute directly translates to reduced test-time compute--a simple yet effective trade-off for efficient reasoning.
• 摘要：通过加强学习和可验证的奖励训练的大型语言模型倾向于将精度交换为长度 - 弹性长度可实现准确性的提高。虽然可能有更长时间的答案来解决更严重的问题，但许多令牌只是“填充”：重复的，详细的文本，没有真正的进步。我们介绍了GFPO（组过滤的策略优化），该长度通过在训练和过滤对培训的较大群体中对较大的小组进行对训练的较大组来遏制这一长度爆炸，并根据两个关键指标进行培训：（1）响应长度和（2）令牌效率：奖励奖励每标记比率。通过在培训时间进行更多抽样，我们教导模型在推理时间较少思考。在PHI-4-RENOSID模型上，GFPO在具有挑战性的STEM和编码基准（AIME 24/25，GPQA，OMNI-MATH，LiveCodeBench）中，GFPO将GRPO的长度通货膨胀率降低了46-71％，同时保持准确性。优化每个令牌的奖励进一步增加了长度通货膨胀率的减少到71-85％。我们还提出了自适应难度GFPO，该GFPO会根据实时难度估计，动态地将更多的培训资源分配给更严重的问题，从而提高了计算效率和准确性之间的平衡，尤其是在困难问题上。 GFPO证明，增加的训练时间计算直接转化为减少的测试时间计算，这是一个简单而有效的有效折衷来进行有效的推理。

Title: Transforming Questions and Documents for Semantically Aligned Retrieval-Augmented Generation

• Authors: Seokgi Lee
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09755
• Pdf URL: https://arxiv.org/pdf/2508.09755
• Copy Paste: [[2508.09755]] Transforming Questions and Documents for Semantically Aligned Retrieval-Augmented Generation(https://arxiv.org/abs/2508.09755)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: We introduce a novel retrieval-augmented generation (RAG) framework tailored for multihop question answering. First, our system uses large language model (LLM) to decompose complex multihop questions into a sequence of single-hop subquestions that guide document retrieval. This decomposition mitigates the ambiguity inherent in multi-hop queries by clearly targeting distinct knowledge facets. Second, instead of embedding raw or chunked documents directly, we generate answerable questions from each document chunk using Qwen3-8B, embed these generated questions, and retrieve relevant chunks via question-question embedding similarity. During inference, the retrieved chunks are then fed along with the original question into the RAG pipeline. We evaluate on three multihop question datasets (MuSiQue, 2WikiMultiHopQa, HotpotQA) from LongBench. Our method improves RAG performacne compared to baseline systems. Our contributions highlight the benefits of using answerable-question embeddings for RAG, and the effectiveness of LLM-based query decomposition for multihop scenarios.
• 摘要：我们介绍了一个新颖的检索型一代（RAG）框架，该框架是针对多台面问题回答的。首先，我们的系统使用大型语言模型（LLM）将复杂的多人问题分解为指导文档检索的一系列单跳子问题。这种分解通过清楚地针对不同的知识方面来减轻多跳查询中固有的歧义。其次，我们没有直接嵌入原始文档或块的文档，而是使用QWEN3-8B从每个文档块中产生可回答的问题，嵌入了这些生成的问题，并通过询问问题嵌入相似性来检索相关的块。在推断过程中，将检索到的块与原始问题一起馈送到抹布管道中。我们从Longbench评估了三个MultiHop问题数据集（Musique，2wikimultihopqa，HotPotQA）。与基线系统相比，我们的方法改善了RAG Performacne。我们的贡献突出了使用可得出的问题嵌入抹布的好处，以及基于LLM的查询分解对多台面场景的有效性。

Title: Echoes of Agreement: Argument Driven Opinion Shifts in Large Language Models

• Authors: Avneet Kaur
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09759
• Pdf URL: https://arxiv.org/pdf/2508.09759
• Copy Paste: [[2508.09759]] Echoes of Agreement: Argument Driven Opinion Shifts in Large Language Models(https://arxiv.org/abs/2508.09759)
• Keywords: language model, llm, prompt
• Abstract: There have been numerous studies evaluating bias of LLMs towards political topics. However, how positions towards these topics in model outputs are highly sensitive to the prompt. What happens when the prompt itself is suggestive of certain arguments towards those positions remains underexplored. This is crucial for understanding how robust these bias evaluations are and for understanding model behaviour, as these models frequently interact with opinionated text. To that end, we conduct experiments for political bias evaluation in presence of supporting and refuting arguments. Our experiments show that such arguments substantially alter model responses towards the direction of the provided argument in both single-turn and multi-turn settings. Moreover, we find that the strength of these arguments influences the directional agreement rate of model responses. These effects point to a sycophantic tendency in LLMs adapting their stance to align with the presented arguments which has downstream implications for measuring political bias and developing effective mitigation strategies.
• 摘要：有许多研究评估LLM偏向政治话题的研究。但是，模型输出中这些主题的位置如何对提示高度敏感。当提示本身暗示对这些立场的某些论点时，会发生什么。这对于理解这些偏见评估的鲁棒性和理解模型行为至关重要，因为这些模型经常与有思想的文本相互作用。为此，我们在支持和反驳论点的情况下进行政治偏见评估的实验。我们的实验表明，此类参数实质上改变了对单转和多转化设置中提供的参数方向的模型响应。此外，我们发现这些论点的强度会影响模型响应的定向一致性率。这些影响表明，LLMS中的合并趋势适应其立场，以与提出的论点保持一致，该论点对衡量政治偏见和制定有效的缓解策略具有下游的影响。

Title: UtterTune: LoRA-Based Target-Language Pronunciation Edit and Control in Multilingual Text-to-Speech

• Authors: Shuhei Kato
• Subjects: cs.CL, eess.AS
• Abstract URL: https://arxiv.org/abs/2508.09767
• Pdf URL: https://arxiv.org/pdf/2508.09767
• Copy Paste: [[2508.09767]] UtterTune: LoRA-Based Target-Language Pronunciation Edit and Control in Multilingual Text-to-Speech(https://arxiv.org/abs/2508.09767)
• Keywords: language model, llm
• Abstract: We propose UtterTune, a lightweight adaptation method that fine-tunes a multilingual text-to-speech (TTS) system based on a large language model (LLM) architecture, designed to enhance the controllability of pronunciation in a target language while preserving performance in others. While LLM architectures have enabled TTS models to achieve remarkable naturalness, accurately modeling grapheme-to-phoneme (G2P) mapping and prosody remains challenging, especially when the model omits an explicit G2P module and directly processes minimally encoded text (e.g., byte-pair encoding). UtterTune leverages low-rank adaptation to enable the control of segmental pronunciation and pitch accent at the phoneme level for Japanese speech, the target language in this paper, while maintaining naturalness and speaker similarity in a zero-shot setting. Objective and subjective evaluations confirm its effectiveness.
• 摘要：我们提出了一种轻巧的适应方法，该方法基于大型语言模型（LLM）体系结构微调多语言文本到语音（TTS）系统，旨在增强目标语言中发音的可控性，同时保留其他人的性能。尽管LLM体系结构使TTS模型能够实现显着的自然性，但准确地建模谱系映射（G2P）映射和韵律仍然具有挑战性，尤其是当该模型省略明确的G2P模块并直接处理最小编码的文本（例如，Byte-Pairing编码）时。 Turttune利用低级适应性，以使日语语音（本文中的目标语言）在音素级别的节段发音和音调口音中控制，同时在零拍设置中保持自然性和说话者的相似性。客观和主观评估证实了其有效性。

Title: Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study

• Authors: Mahdi Dhaini, Juraj Vladika, Ege Erdogan, Zineb Attaoui, Gjergji Kasneci
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09776
• Pdf URL: https://arxiv.org/pdf/2508.09776
• Copy Paste: [[2508.09776]] Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study(https://arxiv.org/abs/2508.09776)
• Keywords: language model, llm
• Abstract: In the rapidly evolving field of Explainable Natural Language Processing (NLP), textual explanations, i.e., human-like rationales, are pivotal for explaining model predictions and enriching datasets with interpretable labels. Traditional approaches rely on human annotation, which is costly, labor-intensive, and impedes scalability. In this work, we present an automated framework that leverages multiple state-of-the-art large language models (LLMs) to generate high-quality textual explanations. We rigorously assess the quality of these LLM-generated explanations using a comprehensive suite of Natural Language Generation (NLG) metrics. Furthermore, we investigate the downstream impact of these explanations on the performance of pre-trained language models (PLMs) and LLMs across natural language inference tasks on two diverse benchmark datasets. Our experiments demonstrate that automated explanations exhibit highly competitive effectiveness compared to human-annotated explanations in improving model performance. Our findings underscore a promising avenue for scalable, automated LLM-based textual explanation generation for extending NLP datasets and enhancing model performance.
• 摘要：在可解释的自然语言处理（NLP）的快速发展的领域中，文本解释，即类似人类的理由，是解释模型预测和具有可解释标签的数据集的关键。传统方法取决于人类注释，这是昂贵，劳动力密集的，并阻碍了可扩展性。在这项工作中，我们提出了一个自动化框架，该框架利用多种最先进的大语言模型（LLMS）来生成高质量的文本解释。我们严格评估了这些LLM生成的解释的质量，使用全面的自然语言产生（NLG）指标。此外，我们研究了这些解释对两个不同基准数据集对自然语言推理任务的预训练语言模型（PLM）和LLM的性能的下游影响。我们的实验表明，自动解释与改善模型性能的人为解释相比，自动解释具有高度竞争的有效性。我们的发现强调了一个有希望的可扩展，自动化LLM的文本解释生成的途径，用于扩展NLP数据集并增强模型性能。

Title: BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning

• Authors: Ahmed Masry, Abhay Puri, Masoud Hashemi, Juan A. Rodriguez, Megh Thakkar, Khyati Mahajan, Vikas Yadav, Sathwik Tejaswi Madhusudhan, Alexandre Piché, Dzmitry Bahdanau, Christopher Pal, David Vazquez, Enamul Hoque, Perouz Taslakian, Sai Rajeswar, Spandana Gella
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09804
• Pdf URL: https://arxiv.org/pdf/2508.09804
• Copy Paste: [[2508.09804]] BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning(https://arxiv.org/abs/2508.09804)
• Keywords: language model
• Abstract: Charts are essential to data analysis, transforming raw data into clear visual representations that support human decision-making. Although current vision-language models (VLMs) have made significant progress, they continue to struggle with chart comprehension due to training on datasets that lack diversity and real-world authenticity, or on automatically extracted underlying data tables of charts, which can contain numerous estimation errors. Furthermore, existing models only rely on supervised fine-tuning using these low-quality datasets, severely limiting their effectiveness. To address these issues, we first propose BigCharts, a dataset creation pipeline that generates visually diverse chart images by conditioning the rendering process on real-world charts sourced from multiple online platforms. Unlike purely synthetic datasets, BigCharts incorporates real-world data, ensuring authenticity and visual diversity, while still retaining accurate underlying data due to our proposed replotting process. Additionally, we introduce a comprehensive training framework that integrates supervised fine-tuning with Group Relative Policy Optimization (GRPO)-based reinforcement learning. By introducing novel reward signals specifically designed for chart reasoning, our approach enhances model robustness and generalization across diverse chart styles and domains, resulting in a state-of-the-art chart reasoning model, BigCharts-R1. Extensive experiments demonstrate that our models surpass existing methods on multiple chart question-answering benchmarks compared to even larger open-source and closed-source models.
• 摘要：图表对于数据分析至关重要，将原始数据转换为支持人类决策的清晰视觉表示。尽管当前的视觉模型（VLM）取得了重大进展，但由于缺乏多样性和真实真实性的数据集的培训或自动提取图表的基础数据表，它们可能会在图表理解中挣扎，这可能包含许多估计错误。此外，现有模型仅依赖于使用这些低质量数据集的监督微调，从而严重限制了它们的有效性。为了解决这些问题，我们首先提出了BigCharts，BigCharts是一种数据集创建管道，该管道通过调节来自多个在线平台的真实图表上的渲染过程来生成视觉上不同的图表图像。与纯粹的合成数据集不同，BigCharts包含了现实世界的数据，确保真实性和视觉多样性，同时由于我们提出的引用过程，仍然保留了准确的基础数据。此外，我们引入了一个全面的培训框架，该框架将受监督的微调与基于小组相对政策优化（GRPO）的强化学习整合在一起。通过引入专门为图表推理设计的新型奖励信号，我们的方法增强了模型的鲁棒性和概括性和跨不同图表样式和域的概括，从而产生了最新的图表推理模型BigCharts-R1。广泛的实验表明，与更大的开源和闭合源模型相比，我们的模型超过了多个图表提问基准的现有方法。

Title: Speed Always Wins: A Survey on Efficient Architectures for Large Language Models

• Authors: Weigao Sun, Jiaxi Hu, Yucheng Zhou, Jusen Du, Disen Lan, Kexin Wang, Tong Zhu, Xiaoye Qu, Yu Zhang, Xiaoyu Mo, Daizong Liu, Yuxuan Liang, Wenliang Chen, Guoqi Li, Yu Cheng
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2508.09834
• Pdf URL: https://arxiv.org/pdf/2508.09834
• Copy Paste: [[2508.09834]] Speed Always Wins: A Survey on Efficient Architectures for Large Language Models(https://arxiv.org/abs/2508.09834)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have delivered impressive results in language understanding, generation, reasoning, and pushes the ability boundary of multimodal models. Transformer models, as the foundation of modern LLMs, offer a strong baseline with excellent scaling properties. However, the traditional transformer architecture requires substantial computations and poses significant obstacles for large-scale training and practical deployment. In this survey, we offer a systematic examination of innovative LLM architectures that address the inherent limitations of transformers and boost the efficiency. Starting from language modeling, this survey covers the background and technical details of linear and sparse sequence modeling methods, efficient full attention variants, sparse mixture-of-experts, hybrid model architectures incorporating the above techniques, and emerging diffusion LLMs. Additionally, we discuss applications of these techniques to other modalities and consider their wider implications for developing scalable, resource-aware foundation models. By grouping recent studies into the above category, this survey presents a blueprint of modern efficient LLM architectures, and we hope this could help motivate future research toward more efficient, versatile AI systems.
• 摘要：大型语言模型（LLM）在语言理解，产生，推理和推动多模型模型的能力边界方面取得了令人印象深刻的结果。变压器模型作为现代LLM的基础，提供了具有出色缩放属性的强基线。但是，传统的变压器体系结构需要大量的计算，并为大规模培训和实际部署带来了重大障碍。在这项调查中，我们对创新的LLM体系结构进行了系统的检查，该体系结构解决了变压器的固有局限性并提高了效率。从语言建模开始，该调查涵盖了线性和稀疏序列建模方法的背景和技术细节，有效的充分注意变体，稀疏的Experts混合物，结合上述技术的混合模型体系结构以及新兴的扩散LLM。此外，我们讨论了这些技术对其他方式的应用，并考虑了它们对开发可扩展的资源感知基础模型的更广泛含义。通过将最近的研究分组为上述类别，该调查介绍了现代有效LLM体系结构的蓝图，我们希望这可以帮助将来的研究激励未来的研究朝着更高效，更多功能的AI系统迈进。

Title: PRELUDE: A Benchmark Designed to Require Global Comprehension and Reasoning over Long Contexts

• Authors: Mo Yu, Tsz Ting Chung, Chulun Zhou, Tong Li, Rui Lu, Jiangnan Li, Liyan Xu, Haoshu Lu, Ning Zhang, Jing Li, Jie Zhou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09848
• Pdf URL: https://arxiv.org/pdf/2508.09848
• Copy Paste: [[2508.09848]] PRELUDE: A Benchmark Designed to Require Global Comprehension and Reasoning over Long Contexts(https://arxiv.org/abs/2508.09848)
• Keywords: llm, long context
• Abstract: We introduce PRELUDE, a benchmark for evaluating long-context understanding through the task of determining whether a character's prequel story is consistent with the canonical narrative of the original book. Our task poses a stronger demand for global comprehension and deep reasoning than existing benchmarks -- as the prequels are not part of the original story, assessing their plausibility typically requires searching and integrating information that is only indirectly related. Empirically, 88% of instances require evidence from multiple parts of the narrative. Experimental results highlight the challenge of our task: in-context learning, RAG and in-domain training with state-of-the-art LLMs, and commercial DeepResearch services, lag behind humans by >15%. A further human study reveals that models often produce correct answers with flawed reasoning, leading to an over 30% gap in reasoning accuracy compared to humans. These findings underscore the substantial room for improvement in long-context understanding and reasoning.
• 摘要：我们介绍了Prelude，这是一个基准，用于通过确定角色的前传故事是否与原始书的规范叙述一致，以评估长篇小说的理解。与现有基准相比，我们的任务对全球理解和深厚的推理提出了更大的需求 - 因为前传不是原始故事的一部分，因此评估其合理性通常需要搜索和集成仅间接相关的信息。从经验上讲，88％的实例需要叙事多个部分的证据。实验结果强调了我们任务的挑战：最先进的LLM中的文章学习，抹布和内域培训以及商业深入研究服务，落后于人类的落后于15％。一项进一步的人类研究表明，模型通常会出于有缺陷的推理产生正确的答案，与人类相比，推理准确性的差距超过30％。这些发现强调了长期理解和推理的重大空间。

Title: Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models

• Authors: Jiaqi Cao, Jiarui Wang, Rubin Wei, Qipeng Guo, Kai Chen, Bowen Zhou, Zhouhan Lin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2508.09874
• Pdf URL: https://arxiv.org/pdf/2508.09874
• Copy Paste: [[2508.09874]] Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models(https://arxiv.org/abs/2508.09874)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Large Language Models (LLMs) have shown strong abilities in general language tasks, yet adapting them to specific domains remains a challenge. Current method like Domain Adaptive Pretraining (DAPT) requires costly full-parameter training and suffers from catastrophic forgetting. Meanwhile, Retrieval-Augmented Generation (RAG) introduces substantial inference latency due to expensive nearest-neighbor searches and longer context. This paper introduces Memory Decoder, a plug-and-play pretrained memory that enables efficient domain adaptation without changing the original model's parameters. Memory Decoder employs a small transformer decoder that learns to imitate the behavior of an external non-parametric retriever. Once trained, Memory Decoder can be seamlessly integrated with any pretrained language model that shares the same tokenizer, requiring no model-specific modifications. Experimental results demonstrate that Memory Decoder enables effective adaptation of various Qwen and Llama models to three distinct specialized domains: biomedicine, finance, and law, reducing perplexity by an average of 6.17 points. Overall, Memory Decoder introduces a novel paradigm centered on a specially pretrained memory component designed for domain-specific adaptation. This memory architecture can be integrated in a plug-and-play manner, consistently enhancing performance across multiple models within the target domain.
• 摘要：大型语言模型（LLMS）在一般语言任务中表现出很强的能力，但是将它们适应特定领域仍然是一个挑战。当前的方法（如域自适应预处理（DAPT））需要昂贵的全参数训练，并且遭受灾难性遗忘。同时，由于昂贵的最近邻居搜索和更长的上下文，检索型发电（RAG）引入了大量的推断潜伏期。本文介绍了内存解码器，这是一种已插入式的售完的内存，可在不更改原始模型的参数的情况下进行有效的域适应。内存解码器采用了一个小型变压器解码器，该解码器学会模仿外部非参数猎犬的行为。经过训练后，可以将内存解码器与任何具有相同令牌的语言模型无缝集成，不需要特定于模型的修改。实验结果表明，记忆解码器可以有效适应各种QWEN和LLAMA模型对三个不同的专业领域：生物医学，金融和法律，平均减少6.17点的困惑。总体而言，内存解码器引入了一种新颖的范式，该范式集中在专门针对域特异性适应的特殊内存组件上。可以以插件的方式集成此内存体系结构，从而始终增强目标域内多个模型的性能。

Title: A Comprehensive Evaluation framework of Alignment Techniques for LLMs

• Authors: Muneeza Azmat, Momin Abbas, Maysa Malfiza Garcia de Macedo, Marcelo Carpinette Grave, Luan Soares de Souza, Tiago Machado, Rogerio A de Paula, Raya Horesh, Yixin Chen, Heloisa Caroline de Souza Pereira Candello, Rebecka Nordenlow, Aminat Adebiyi
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.09937
• Pdf URL: https://arxiv.org/pdf/2508.09937
• Copy Paste: [[2508.09937]] A Comprehensive Evaluation framework of Alignment Techniques for LLMs(https://arxiv.org/abs/2508.09937)
• Keywords: language model, llm
• Abstract: As Large Language Models (LLMs) become increasingly integrated into real-world applications, ensuring their outputs align with human values and safety standards has become critical. The field has developed diverse alignment approaches including traditional fine-tuning methods (RLHF, instruction tuning), post-hoc correction systems, and inference-time interventions, each with distinct advantages and limitations. However, the lack of unified evaluation frameworks makes it difficult to systematically compare these paradigms and guide deployment decisions. This paper introduces a multi-dimensional evaluation of alignment techniques for LLMs, a comprehensive evaluation framework that provides a systematic comparison across all major alignment paradigms. Our framework assesses methods along four key dimensions: alignment detection, alignment quality, computational efficiency, and robustness. Through experiments across diverse base models and alignment strategies, we demonstrate the utility of our framework in identifying strengths and limitations of current state-of-the-art models, providing valuable insights for future research directions.
• 摘要：随着大型语言模型（LLM）越来越多地集成到现实世界中，确保其输出与人类价值观和安全标准保持一致。该领域已开发出各种各样的一致性方法，包括传统的微调方法（RLHF，指令调整），事后校正系统和推理时间干预措施，每种都具有明显的优势和局限性。但是，缺乏统一的评估框架使得很难系统地比较这些范式并指导部署决策。本文介绍了对LLMS的对齐技术的多维评估，LLMS是一个全面的评估框架，可在所有主要的对齐范式中进行系统的比较。我们的框架评估了沿四个关键维度的方法：对准检测，对齐质量，计算效率和鲁棒性。通过跨不同基本模型和对齐策略的实验，我们证明了框架在确定当前最新模型的优势和局限性方面的实用性，为未来的研究方向提供了宝贵的见解。

Title: VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models

• Authors: Lingjie Jiang, Shaohan Huang, Xun Wu, Yixia Li, Dongdong Zhang, Furu Wei
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2508.09945
• Pdf URL: https://arxiv.org/pdf/2508.09945
• Copy Paste: [[2508.09945]] VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models(https://arxiv.org/abs/2508.09945)
• Keywords: language model, gpt, llm
• Abstract: Multimodal large language models (MLLMs) have significantly advanced the integration of visual and textual understanding. However, their ability to generate code from multimodal inputs remains limited. In this work, we introduce VisCodex, a unified framework that seamlessly merges vision and coding language models to empower MLLMs with strong multimodal code generation abilities. Leveraging a task vector-based model merging technique, we integrate a state-of-the-art coding LLM into a strong vision-language backbone, while preserving both visual comprehension and advanced coding skills. To support training and evaluation, we introduce the Multimodal Coding Dataset (MCD), a large-scale and diverse collection of 598k samples, including high-quality HTML code, chart image-code pairs, image-augmented StackOverflow QA, and algorithmic problems. Furthermore, we propose InfiBench-V, a novel and challenging benchmark specifically designed to assess models on visually-rich, real-world programming questions that demand a nuanced understanding of both textual and visual contexts. Extensive experiments show that VisCodex achieves state-of-the-art performance among open-source MLLMs and approaches proprietary models like GPT-4o, highlighting the effectiveness of our model merging strategy and new datasets.
• 摘要：多模式的大语言模型（MLLM）显着提高了视觉和文本理解的整合。但是，它们从多模式输入生成代码的能力仍然有限。在这项工作中，我们介绍了Viscodex，Viscodex是一个统一的框架，无缝地将视觉和编码语言模型融合在一起，以增强具有强大的多模式代码生成能力的MLLM。利用基于任务向量的模型合并技术，我们将最先进的编码LLM集成到了强大的视觉语言主链中，同时既可以保持视觉理解和高级编码技能。为了支持培训和评估，我们介绍了多模式编码数据集（MCD），该数据集（MCD）是598K样本的大规模和多样化的集合，包括高质量的HTML代码，图表图像代码对，图像增强的Stackoverflow QA QA和算法问题。此外，我们提出了Infibench-V，这是一种新颖且具有挑战性的基准，该基准专门用于评估视觉上富裕的现实世界编程问题的模型，要求对文本和视觉上下文有细微的理解。广泛的实验表明，Viscodex在开源MLLMS中实现了最先进的性能，并使用了GPT-4O等专有模型，突出了我们的模型合并策略和新数据集的有效性。

Title: Specialised or Generic? Tokenization Choices for Radiology Language Models

• Authors: Hermione Warr, Wentian Xu, Harry Anthony, Yasin Ibrahim, Daniel McGowan, Konstantinos Kamnitsas
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.09952
• Pdf URL: https://arxiv.org/pdf/2508.09952
• Copy Paste: [[2508.09952]] Specialised or Generic? Tokenization Choices for Radiology Language Models(https://arxiv.org/abs/2508.09952)
• Keywords: language model
• Abstract: The vocabulary used by language models (LM) - defined by the tokenizer - plays a key role in text generation quality. However, its impact remains under-explored in radiology. In this work, we address this gap by systematically comparing general, medical, and domain-specific tokenizers on the task of radiology report summarisation across three imaging modalities. We also investigate scenarios with and without LM pre-training on PubMed abstracts. Our findings demonstrate that medical and domain-specific vocabularies outperformed widely used natural language alternatives when models are trained from scratch. Pre-training partially mitigates performance differences between tokenizers, whilst the domain-specific tokenizers achieve the most favourable results. Domain-specific tokenizers also reduce memory requirements due to smaller vocabularies and shorter sequences. These results demonstrate that adapting the vocabulary of LMs to the clinical domain provides practical benefits, including improved performance and reduced computational demands, making such models more accessible and effective for both research and real-world healthcare settings.
• 摘要：语言模型（LM）使用的词汇（由令牌剂定义）在文本生成质量中起关键作用。但是，它的影响在放射学中仍未探索。在这项工作中，我们通过系统地比较一般，医学和域特异性引物在放射学报告跨三种成像方式的摘要中解决了这一差距。我们还研究了在PubMed摘要上进行有或没有LM预训练的情况。我们的发现表明，当模型从头开始训练时，医学和域特异性词汇表现优于广泛使用的自然语言替代方案。预训练部分减轻了代币剂之间的性能差异，而域特异性的引物器则取得了最有利的结果。特定于域特异性的令牌还可以减少由于词汇较小和较短序列而导致的记忆需求。这些结果表明，将LMS的词汇适应临床领域提供了实际的好处，包括提高性能和减少计算需求，使此类模型对研究和现实世界的医疗保健环境更容易易于访问和有效。

Title: Performance of GPT-5 Frontier Models in Ophthalmology Question Answering

• Authors: Fares Antaki, David Mikhail, Daniel Milad, Danny A Mammo, Sumit Sharma, Sunil K Srivastava, Bing Yu Chen, Samir Touma, Mertcan Sevgi, Jonathan El-Khoury, Pearse A Keane, Qingyu Chen, Yih Chung Tham, Renaud Duval
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09956
• Pdf URL: https://arxiv.org/pdf/2508.09956
• Copy Paste: [[2508.09956]] Performance of GPT-5 Frontier Models in Ophthalmology Question Answering(https://arxiv.org/abs/2508.09956)
• Keywords: language model, gpt, llm
• Abstract: Large language models (LLMs) such as GPT-5 integrate advanced reasoning capabilities that may improve performance on complex medical question-answering tasks. For this latest generation of reasoning models, the configurations that maximize both accuracy and cost-efficiency have yet to be established. We evaluated 12 configurations of OpenAI's GPT-5 series (three model tiers across four reasoning effort settings) alongside o1-high, o3-high, and GPT-4o, using 260 closed-access multiple-choice questions from the American Academy of Ophthalmology Basic Clinical Science Course (BCSC) dataset. The primary outcome was multiple-choice accuracy; secondary outcomes included head-to-head ranking via a Bradley-Terry model, rationale quality assessment using a reference-anchored, pairwise LLM-as-a-judge framework, and analysis of accuracy-cost trade-offs using token-based cost estimates. GPT-5-high achieved the highest accuracy (0.965; 95% CI, 0.942-0.985), outperforming all GPT-5-nano variants (P < .001), o1-high (P = .04), and GPT-4o (P < .001), but not o3-high (0.958; 95% CI, 0.931-0.981). GPT-5-high ranked first in both accuracy (1.66x stronger than o3-high) and rationale quality (1.11x stronger than o3-high). Cost-accuracy analysis identified several GPT-5 configurations on the Pareto frontier, with GPT-5-mini-low offering the most favorable low-cost, high-performance balance. These results benchmark GPT-5 on a high-quality ophthalmology dataset, demonstrate the influence of reasoning effort on accuracy, and introduce an autograder framework for scalable evaluation of LLM-generated answers against reference standards in ophthalmology.
• 摘要：大型语言模型（LLM）（例如GPT-5）整合了先进的推理功能，这些功能可能会提高复杂的医疗问答任务的性能。对于最新一代的推理模型，尚未建立最大化准确性和成本效益的配置。我们使用260个封闭式访问的多项选择问题，评估了OpenAI的GPT-5系列（四个推理工作设置中的三个模型层）的12个配置（在四个推理工作设置中进行了三个模型层）。主要结果是多项选择的准确性。次要结果包括通过Bradley-Terry模型的头对头排名，使用参考锚定的，成对的LLM-AS-A-A-Gudge框架进行了理由质量评估，以及使用基于令牌的成本估算的准确性成本权衡分析。 GPT-5-high achieved the highest accuracy (0.965; 95% CI, 0.942-0.985), outperforming all GPT-5-nano variants (P < .001), o1-high (P = .04), and GPT-4o (P < .001), but not o3-high (0.958; 95% CI, 0.931-0.981). GPT-5高的准确性（比O3-高）和理由质量（比O3高1.11倍）排名第一。成本准确性分析确定了帕累托前沿上的几种GPT-5配置，GPT-5-MINI-LOW提供了最有利的低成本，高性能的平衡。这些结果基于高质量眼科数据集的基准GPT-5，证明了推理努力对准确性的影响，并引入了自动化框架，以根据Ophthalmologoly中的参考标准对LLM生成的答案进行可扩展评估。

Title: Which one Performs Better? Wav2Vec or Whisper? Applying both in Badini Kurdish Speech to Text (BKSTT)

• Authors: Renas Adnan, Hossein Hassani
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2508.09957
• Pdf URL: https://arxiv.org/pdf/2508.09957
• Copy Paste: [[2508.09957]] Which one Performs Better? Wav2Vec or Whisper? Applying both in Badini Kurdish Speech to Text (BKSTT)(https://arxiv.org/abs/2508.09957)
• Keywords: language model
• Abstract: Speech-to-text (STT) systems have a wide range of applications. They are available in many languages, albeit at different quality levels. Although Kurdish is considered a less-resourced language from a processing perspective, SST is available for some of the Kurdish dialects, for instance, Sorani (Central Kurdish). However, that is not applied to other Kurdish dialects, Badini and Hawrami, for example. This research is an attempt to address this gap. Bandin, approximately, has two million speakers, and STT systems can help their community use mobile and computer-based technologies while giving their dialect more global visibility. We aim to create a language model based on Badini's speech and evaluate its performance. To cover a conversational aspect, have a proper confidence level of grammatical accuracy, and ready transcriptions, we chose Badini kids' stories, eight books including 78 stories, as the textual input. Six narrators narrated the books, which resulted in approximately 17 hours of recording. We cleaned, segmented, and tokenized the input. The preprocessing produced nearly 15 hours of speech, including 19193 segments and 25221 words. We used Wav2Vec2-Large-XLSR-53 and Whisper-small to develop the language models. The experiments indicate that the transcriptions process based on the Wav2Vec2-Large-XLSR-53 model provides a significantly more accurate and readable output than the Whisper-small model, with 90.38% and 65.45% readability, and 82.67% and 53.17% accuracy, respectively.
• 摘要：语音到文本（STT）系统具有广泛的应用。它们有多种语言，尽管质量不同。尽管从处理的角度来看，库尔德人被认为是一种资源较低的语言，但SST可用于一些库尔德方言，例如索拉尼（库尔德中央）。但是，例如，这并不适用于其他库尔德方言Badini和Hawrami。这项研究是为了解决这一差距。班丁（Bandin）大约有200万扬声器，而STT系统可以帮助他们的社区使用基于移动和计算机的技术，同时使他们的方言更具全球性可见性。我们旨在根据Badini的语音创建语言模型并评估其表现。为了涵盖对话方面，具有适当的语法准确性和现成的抄写信心，我们选择了Badini Kids的故事，包括78个故事在内的八本书作为文本输入。六位叙述者叙述了这些书，这大约导致了17个小时的录音。我们清理，分割和令牌化了输入。预处理产生了近15个小时的语音，包括19193个部分和25221个单词。我们使用WAV2VEC2-LARGE-XLSR-53和Whisper-Mall来开发语言模型。该实验表明，基于WAV2VEC2-LARGE-XLSR-53模型的转录过程比Whisper-Small模型提供了更准确和可读的输出，分别为90.38％和65.45％的可读性，分别为82.67％和82.67％和53.17％的准确性。

Title: Neural Bandit Based Optimal LLM Selection for a Pipeline of Tasks

• Authors: Baran Atalar, Eddie Zhang, Carlee Joe-Wong
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2508.09958
• Pdf URL: https://arxiv.org/pdf/2508.09958
• Copy Paste: [[2508.09958]] Neural Bandit Based Optimal LLM Selection for a Pipeline of Tasks(https://arxiv.org/abs/2508.09958)
• Keywords: language model, llm
• Abstract: With the increasing popularity of large language models (LLMs) for a variety of tasks, there has been a growing interest in strategies that can predict which out of a set of LLMs will yield a successful answer at low cost. This problem promises to become more and more relevant as providers like Microsoft allow users to easily create custom LLM "assistants" specialized to particular types of queries. However, some tasks (i.e., queries) may be too specialized and difficult for a single LLM to handle alone. These applications often benefit from breaking down the task into smaller subtasks, each of which can then be executed by a LLM expected to perform well on that specific subtask. For example, in extracting a diagnosis from medical records, one can first select an LLM to summarize the record, select another to validate the summary, and then select another, possibly different, LLM to extract the diagnosis from the summarized record. Unlike existing LLM selection or routing algorithms, this setting requires that we select a sequence of LLMs, with the output of each LLM feeding into the next and potentially influencing its success. Thus, unlike single LLM selection, the quality of each subtask's output directly affects the inputs, and hence the cost and success rate, of downstream LLMs, creating complex performance dependencies that must be learned and accounted for during selection. We propose a neural contextual bandit-based algorithm that trains neural networks that model LLM success on each subtask in an online manner, thus learning to guide the LLM selections for the different subtasks, even in the absence of historical LLM performance data. Experiments on telecommunications question answering and medical diagnosis prediction datasets illustrate the effectiveness of our proposed approach compared to other LLM selection algorithms.
• 摘要：随着大语模型（LLM）对各种任务的日益普及，人们对策略的兴趣越来越多，可以预测一套LLMS中哪些的策略将以低成本产生成功的答案。随着像Microsoft这样的提供商允许用户可以轻松地创建专门针对特定类型的查询类型的自定义LLM“助手”，因此这个问题有望变得越来越相关。但是，某些任务（即查询）可能过于专业化，单个LLM很难单独处理。这些应用程序通常会受益于将任务分解为较小的子任务，然后可以通过预期在该特定子任务上表现良好的LLM执行这些应用程序。例如，在从医疗记录中提取诊断时，可以首先选择一个LLM来汇总记录，选择另一个以验证摘要，然后选择另一个可能不同的LLM以从摘要记录中提取诊断。与现有的LLM选择或路由算法不同，此设置要求我们选择一个LLMS序列，每个LLM的输出输出将进入下一个，并可能影响其成功。因此，与单个LLM选择不同，每个子任务的输出的质量直接影响输入，因此，下游LLM的成本和成功率，创造了在选择过程中必须学习和考虑的复杂性能依赖性。我们提出了一种基于匪徒的神经背景算法，该算法训练神经网络，以在线方式对每个子任务进行LLM成功建模，从而学习指导不同子任务的LLM选择，即使没有历史LLM绩效数据。关于电信问题的实验答案和医学诊断预测数据集与其他LLM选择算法相比，我们提出的方法的有效性说明了我们所提出的方法的有效性。