2025-05-16

Title: Next Word Suggestion using Graph Neural Network

• Authors: Abisha Thapa Magar, Anup Shakya
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.09649
• Pdf URL: https://arxiv.org/pdf/2505.09649
• Copy Paste: [[2505.09649]] Next Word Suggestion using Graph Neural Network(https://arxiv.org/abs/2505.09649)
• Keywords: language model
• Abstract: Language Modeling is a prevalent task in Natural Language Processing. The currently existing most recent and most successful language models often tend to build a massive model with billions of parameters, feed in a tremendous amount of text data, and train with enormous computation resources which require millions of dollars. In this project, we aim to address an important sub-task in language modeling, i.e., context embedding. We propose an approach to exploit the Graph Convolution operation in GNNs to encode the context and use it in coalition with LSTMs to predict the next word given a local context of preceding words. We test this on the custom Wikipedia text corpus using a very limited amount of resources and show that this approach works fairly well to predict the next word.
• 摘要：语言建模是自然语言处理中的一项普遍任务。当前现有的最新和最成功的语言模型通常倾向于建立一个具有数十亿个参数的大型模型，以大量的文本数据为食，并使用巨大的计算资源进行培训，需要数百万美元。在这个项目中，我们旨在解决语言建模中的重要子任务，即上下文嵌入。我们提出了一种利用GNN中的图形卷积操作的方法，以编码上下文，并在与LSTMS联盟中使用它，以预测下一个单词，给定一个局部上下文的局部上下文。我们使用非常有限的资源在自定义的Wikipedia文本语料库上对此进行测试，并表明此方法可以很好地预测下一个单词。

Title: DRA-GRPO: Exploring Diversity-Aware Reward Adjustment for R1-Zero-Like Training of Large Language Models

• Authors: Xiwen Chen, Wenhui Zhu, Peijie Qiu, Xuanzhao Dong, Hao Wang, Haiyu Wu, Huayu Li, Aristeidis Sotiras, Yalin Wang, Abolfazl Razi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09655
• Pdf URL: https://arxiv.org/pdf/2505.09655
• Copy Paste: [[2505.09655]] DRA-GRPO: Exploring Diversity-Aware Reward Adjustment for R1-Zero-Like Training of Large Language Models(https://arxiv.org/abs/2505.09655)
• Keywords: language model
• Abstract: Recent advances in reinforcement learning for language model post-training, such as Group Relative Policy Optimization (GRPO), have shown promise in low-resource settings. However, GRPO typically relies on solution-level and scalar reward signals that fail to capture the semantic diversity among sampled completions. This leads to what we identify as a diversity-quality inconsistency, where distinct reasoning paths may receive indistinguishable rewards. To address this limitation, we propose $\textit{Diversity-aware Reward Adjustment}$ (DRA), a method that explicitly incorporates semantic diversity into the reward computation. DRA uses Submodular Mutual Information (SMI) to downweight redundant completions and amplify rewards for diverse ones. This encourages better exploration during learning, while maintaining stable exploitation of high-quality samples. Our method integrates seamlessly with both GRPO and its variant DR.~GRPO, resulting in $\textit{DRA-GRPO}$ and $\textit{DGA-DR.~GRPO}$. We evaluate our method on five mathematical reasoning benchmarks and find that it outperforms recent strong baselines. It achieves state-of-the-art performance with an average accuracy of 58.2%, using only 7,000 fine-tuning samples and a total training cost of approximately $55. The code is available at this https URL.
• 摘要：语言模型后培训的加强学习的最新进展，例如小组相对政策优化（GRPO），在低资源环境中表现出了希望。但是，GRPO通常依赖于解决方案级别和标量奖励信号，这些信号无法捕获采样完成之间的语义多样性。这导致了我们确定的多样性质量不一致之处，在这种情况下，不同的推理路径可能会获得无法区分的回报。为了解决此限制，我们建议$ \ textit {多样性意识奖励调整} $（DRA），这种方法将语义多样性明确地纳入奖励计算中。 DRA使用少量信息（SMI）来减少冗余完成并扩大各种奖励。这鼓励在学习过程中更好地探索，同时保持对高质量样本的稳定开发。我们的方法与grpo及其变体dr。〜grpo无缝集成，从而产生$ \ textit {dra-grpo} $和$ \ textit {dga-dr。〜grpo} $。我们在五个数学推理基准上评估了我们的方法，并发现它的表现优于最近的强基础。它的平均准确度为58.2％，仅使用7,000个微调样本，总培训成本约为55美元，可以实现最先进的性能。该代码可在此HTTPS URL上找到。

Title: Large Language Models Are More Persuasive Than Incentivized Human Persuaders

• Authors: Philipp Schoenegger, Francesco Salvi, Jiacheng Liu, Xiaoli Nan, Ramit Debnath, Barbara Fasolo, Evelina Leivada, Gabriel Recchia, Fritz Günther, Ali Zarifhonarvar, Joe Kwon, Zahoor Ul Islam, Marco Dehnert, Daryl Y. H. Lee, Madeline G. Reinecke, David G. Kamper, Mert Kobaş, Adam Sandford, Jonas Kgomo, Luke Hewitt, Shreya Kapoor, Kerem Oktar, Eyup Engin Kucuk, Bo Feng, Cameron R. Jones, Izzy Gainsburg, Sebastian Olschewski, Nora Heinzelmann, Francisco Cruz, Ben M. Tappin, Tao Ma, Peter S. Park, Rayan Onyonka, Arthur Hjorth, Peter Slattery, Qingcheng Zeng, Lennart Finke, Igor Grossmann, Alessandro Salatiello, Ezra Karger
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09662
• Pdf URL: https://arxiv.org/pdf/2505.09662
• Copy Paste: [[2505.09662]] Large Language Models Are More Persuasive Than Incentivized Human Persuaders(https://arxiv.org/abs/2505.09662)
• Keywords: language model, llm
• Abstract: We directly compare the persuasion capabilities of a frontier large language model (LLM; Claude Sonnet 3.5) against incentivized human persuaders in an interactive, real-time conversational quiz setting. In this preregistered, large-scale incentivized experiment, participants (quiz takers) completed an online quiz where persuaders (either humans or LLMs) attempted to persuade quiz takers toward correct or incorrect answers. We find that LLM persuaders achieved significantly higher compliance with their directional persuasion attempts than incentivized human persuaders, demonstrating superior persuasive capabilities in both truthful (toward correct answers) and deceptive (toward incorrect answers) contexts. We also find that LLM persuaders significantly increased quiz takers' accuracy, leading to higher earnings, when steering quiz takers toward correct answers, and significantly decreased their accuracy, leading to lower earnings, when steering them toward incorrect answers. Overall, our findings suggest that AI's persuasion capabilities already exceed those of humans that have real-money bonuses tied to performance. Our findings of increasingly capable AI persuaders thus underscore the urgency of emerging alignment and governance frameworks.
• 摘要：我们直接将边境大语模型（LLM； Claude Sonnet 3.5）与激励人类说服者进行的说服力进行了比较。在这个预先策略的大规模激励实验中，参与者（测验接受者）完成了一个在线测验，说服者（人类或LLMS）试图说服测验者对正确或不正确的答案进行说服。我们发现，LLM说服者比激励人类的说服者获得了明显更高的遵守方式，这表明了真实的（朝着正确的答案）和欺骗性（朝着错误的答案）上下文中表现出卓越的说服力。我们还发现，LLM说服者大大提高了测验者的准确性，导致收入更高的收入，当转向测验者向正确的答案方向转向正确的答案，并大大降低了其准确性，从而导致较低的收入，当他们转向错误的答案时。总体而言，我们的发现表明，AI的说服能力已经超出了与性能相关的实际奖金的人类的能力。因此，我们对越来越有能力的AI说服者的发现强调了新兴一致性和治理框架的紧迫性。

Title: System Prompt Optimization with Meta-Learning

• Authors: Yumin Choi, Jinheon Baek, Sung Ju Hwang
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.09666
• Pdf URL: https://arxiv.org/pdf/2505.09666
• Copy Paste: [[2505.09666]] System Prompt Optimization with Meta-Learning(https://arxiv.org/abs/2505.09666)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) have shown remarkable capabilities, with optimizing their input prompts playing a pivotal role in maximizing their performance. However, while LLM prompts consist of both the task-agnostic system prompts and task-specific user prompts, existing work on prompt optimization has focused on user prompts specific to individual queries or tasks, and largely overlooked the system prompt that is, once optimized, applicable across different tasks and domains. Motivated by this, we introduce the novel problem of bilevel system prompt optimization, whose objective is to design system prompts that are robust to diverse user prompts and transferable to unseen tasks. To tackle this problem, we then propose a meta-learning framework, which meta-learns the system prompt by optimizing it over various user prompts across multiple datasets, while simultaneously updating the user prompts in an iterative manner to ensure synergy between them. We conduct experiments on 14 unseen datasets spanning 5 different domains, on which we show that our approach produces system prompts that generalize effectively to diverse user prompts. Also, our findings reveal that the optimized system prompt enables rapid adaptation even to unseen tasks, requiring fewer optimization steps for test-time user prompts while achieving improved performance.
• 摘要：大型语言模型（LLMS）显示出了出色的功能，并通过优化其输入提示在最大化其性能方面发挥了关键作用。但是，虽然LLM提示既包括任务无关系统的提示和特定于任务的用户提示，但是“提示优化”的现有工作集中在特定于单个查询或任务的用户提示上，并且在很大程度上忽略了系统提示，该系统提示一旦优化，适用于不同的任务和域。在此激励的情况下，我们介绍了二聚体系统提示优化的新颖问题，其目的是设计系统提示，这些提示可对多样化的用户提示并可以转移到看不见的任务。为了解决此问题，我们提出了一个元学习框架，该框架通过在多个数据集的各种用户提示中优化系统提示，同时以迭代方式更新用户提示，以确保它们之间的协同作用。我们对跨越5个不同域的14个看不见的数据集进行了实验，我们在其上表明我们的方法会产生系统提示，从而有效地推广到不同的用户提示。另外，我们的发现表明，优化的系统提示可以使快速适应甚至可以看不见任务，而对于测试时间用户提示的优化步骤更少，同时实现了提高性能。

Title: VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts

• Authors: Xin Liu, Lechen Zhang, Sheza Munir, Yiyang Gu, Lu Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09701
• Pdf URL: https://arxiv.org/pdf/2505.09701
• Copy Paste: [[2505.09701]] VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts(https://arxiv.org/abs/2505.09701)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) excel at generating long-form responses, but evaluating their factuality remains challenging due to complex inter-sentence dependencies within the generated facts. Prior solutions predominantly follow a decompose-decontextualize-verify pipeline but often fail to capture essential context and miss key relational facts. In this paper, we introduce VeriFact, a factuality evaluation framework designed to enhance fact extraction by identifying and resolving incomplete and missing facts to support more accurate verification results. Moreover, we introduce FactRBench , a benchmark that evaluates both precision and recall in long-form model responses, whereas prior work primarily focuses on precision. FactRBench provides reference fact sets from advanced LLMs and human-written answers, enabling recall assessment. Empirical evaluations show that VeriFact significantly enhances fact completeness and preserves complex facts with critical relational information, resulting in more accurate factuality evaluation. Benchmarking various open- and close-weight LLMs on FactRBench indicate that larger models within same model family improve precision and recall, but high precision does not always correlate with high recall, underscoring the importance of comprehensive factuality assessment.
• 摘要：大型语言模型（LLMS）在产生长形式的响应方面表现出色，但是由于生成的事实中复杂的句子间依赖关系，评估其事实性仍然具有挑战性。先前的解决方案主要遵循分解核能 - 验证的管道，但通常无法捕获基本上下文并错过关键的关系事实。在本文中，我们引入了验证，这是一个事实评估框架，旨在通过识别和解决不完整和缺失的事实来增强事实提取，以支持更准确的验证结果。此外，我们介绍了factrbench，这是一种基准，可以评估长期模型响应中的精度和回忆，而先前的工作主要集中在精度上。 FACTRBENCH提供了来自高级LLM的参考事实集和人为编写的答案，从而实现了召回评估。经验评估表明，Verifact显着提高了事实的完整性，并保留具有关键关系信息的复杂事实，从而进行了更准确的事实评估。基准在FACTRBENCH上对各种开放和近距离LLM进行基准测试表明，同一模型家庭中的较大模型提高了精度和回忆，但是高精度并不总是与高召回率相关，强调了全面事实评估的重要性。

Title: An AI-Powered Research Assistant in the Lab: A Practical Guide for Text Analysis Through Iterative Collaboration with LLMs

• Authors: Gino Carmona-Díaz, William Jiménez-Leal, María Alejandra Grisales, Chandra Sripada, Santiago Amaya, Michael Inzlicht, Juan Pablo Bermúdez
• Subjects: cs.CL, cs.AI, cs.HC
• Abstract URL: https://arxiv.org/abs/2505.09724
• Pdf URL: https://arxiv.org/pdf/2505.09724
• Copy Paste: [[2505.09724]] An AI-Powered Research Assistant in the Lab: A Practical Guide for Text Analysis Through Iterative Collaboration with LLMs(https://arxiv.org/abs/2505.09724)
• Keywords: llm, prompt
• Abstract: Analyzing texts such as open-ended responses, headlines, or social media posts is a time- and labor-intensive process highly susceptible to bias. LLMs are promising tools for text analysis, using either a predefined (top-down) or a data-driven (bottom-up) taxonomy, without sacrificing quality. Here we present a step-by-step tutorial to efficiently develop, test, and apply taxonomies for analyzing unstructured data through an iterative and collaborative process between researchers and LLMs. Using personal goals provided by participants as an example, we demonstrate how to write prompts to review datasets and generate a taxonomy of life domains, evaluate and refine the taxonomy through prompt and direct modifications, test the taxonomy and assess intercoder agreements, and apply the taxonomy to categorize an entire dataset with high intercoder reliability. We discuss the possibilities and limitations of using LLMs for text analysis.
• 摘要：分析诸如开放式响应，头条新闻或社交媒体帖子之类的文本是一个时间和劳动密集型过程，非常容易受到偏见的影响。 LLM是使用预定义（自上而下）或数据驱动（自下而上）分类法的有前途的文本分析工具，而无需牺牲质量。在这里，我们提出了一个逐步的教程，以通过研究人员和LLM之间的迭代和协作过程有效地开发，测试和应用分类法来分析非结构化数据。以参与者提供的个人目标为例，我们演示了如何编写提示来审查数据集并生成生命域的分类法，通过及时和直接修改，测试分类法和评估编码器协议，评估和完善分类法，并将分类法应用于整个数据集对高化编码器可靠性。我们讨论使用LLM进行文本分析的可能性和局限性。

Title: Achieving Tokenizer Flexibility in Language Models through Heuristic Adaptation and Supertoken Learning

• Authors: Shaurya Sharthak, Vinayak Pahalwan, Adithya Kamath, Adarsh Shirawalmath
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.09738
• Pdf URL: https://arxiv.org/pdf/2505.09738
• Copy Paste: [[2505.09738]] Achieving Tokenizer Flexibility in Language Models through Heuristic Adaptation and Supertoken Learning(https://arxiv.org/abs/2505.09738)
• Keywords: language model, llm
• Abstract: Pretrained language models (LLMs) are often constrained by their fixed tokenization schemes, leading to inefficiencies and performance limitations, particularly for multilingual or specialized applications. This tokenizer lock-in presents significant challenges. standard methods to overcome this often require prohibitive computational resources. Although tokenizer replacement with heuristic initialization aims to reduce this burden, existing methods often require exhaustive residual fine-tuning and still may not fully preserve semantic nuances or adequately address the underlying compression inefficiencies. Our framework introduces two innovations: first, Tokenadapt, a model-agnostic tokenizer transplantation method, and second, novel pre-tokenization learning for multi-word Supertokens to enhance compression and reduce fragmentation. Tokenadapt initializes new unique token embeddings via a hybrid heuristic that combines two methods: a local estimate based on subword decomposition using the old tokenizer, and a global estimate utilizing the top-k semantically similar tokens from the original vocabulary. This methodology aims to preserve semantics while significantly minimizing retraining requirements. Empirical investigations validate both contributions: the transplantation heuristic successfully initializes unique tokens, markedly outperforming conventional baselines and sophisticated methods including Transtokenizer and ReTok, while our Supertokens achieve notable compression gains. Our zero-shot perplexity results demonstrate that the TokenAdapt hybrid initialization consistently yields lower perplexity ratios compared to both ReTok and TransTokenizer baselines across different base models and newly trained target tokenizers. TokenAdapt typically reduced the overall perplexity ratio significantly compared to ReTok, yielding at least a 2-fold improvement in these aggregate scores.
• 摘要：预审前的语言模型（LLMS）通常受其固定的令牌化方案的限制，导致效率低下和绩效限制，尤其是对于多语言或专业应用程序。这个令牌锁定带来了重大挑战。克服这一点的标准方法通常需要过度的计算资源。尽管用启发式初始化的替换剂旨在减轻这种负担，但现有的方法通常需要详尽的残留微调，并且仍然可能无法完全保留语义细微差别或充分解决潜在的压缩效率低下。我们的框架介绍了两项创新：首先，TokenAdapt，一种模型的敏捷令牌移植方法，第二个新型的多词超级动物的新型预习惯学习，以增强压缩并减少碎片化。 TokenAdapt通过混合启发式方法将新的独特令牌嵌入到了新的独特令牌嵌入，该混合启发式结合了两种方法：基于使用旧令牌的子单词分解的局部估计，以及使用原始词汇中的顶级语义上类似的标记，一种全球估计值。该方法旨在保留语义，同时显着最大程度地减少再培训要求。实证研究验证了这两种贡献：移植启发式成功初始化了独特的代币，明显超过了传统的基准和包括transtokenizer和Retrok在内的复杂方法，而我们的超级动物则取得了显着的压缩增长。我们的零射击困惑结果表明，与在不同的基本模型和新训练的目标靶标物中相比，TokenAdapt杂种初始化始终产生的同期比率较低。与牵引相比，TokenAdapt通常会显着降低整体困惑比显着降低，这些骨料得分至少提高了2倍。

Title: Exploring the generalization of LLM truth directions on conversational formats

• Authors: Timour Ichmoukhamedov, David Martens
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.09807
• Pdf URL: https://arxiv.org/pdf/2505.09807
• Copy Paste: [[2505.09807]] Exploring the generalization of LLM truth directions on conversational formats(https://arxiv.org/abs/2505.09807)
• Keywords: llm, prompt
• Abstract: Several recent works argue that LLMs have a universal truth direction where true and false statements are linearly separable in the activation space of the model. It has been demonstrated that linear probes trained on a single hidden state of the model already generalize across a range of topics and might even be used for lie detection in LLM conversations. In this work we explore how this truth direction generalizes between various conversational formats. We find good generalization between short conversations that end on a lie, but poor generalization to longer formats where the lie appears earlier in the input prompt. We propose a solution that significantly improves this type of generalization by adding a fixed key phrase at the end of each conversation. Our results highlight the challenges towards reliable LLM lie detectors that generalize to new settings.
• 摘要：最近的一些作品认为，LLM具有通用的真理方向，在模型的激活空间中，真实和错误的陈述可以线性分离。已经证明，在模型的单个隐藏状态上训练的线性探针已经在一系列主题上概括，甚至可以在LLM对话中用于谎言检测。在这项工作中，我们探讨了这个真理方向如何在各种对话格式之间推广。我们发现在谎言以结束的简短对话之间进行了良好的概括，但是对谎言在输入提示中出现的较长格式的概括较差。我们提出了一个解决方案，该解决方案通过在每个对话结束时添加固定的密钥短语来显着改善这种概括。我们的结果突出了对可靠的LLM探测器的挑战，该检测器概括为新设置。

Title: KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoning

• Authors: Peiqi Sui, Juan Diego Rodriguez, Philippe Laban, Dean Murphy, Joseph P. Dexter, Richard Jean So, Samuel Baker, Pramit Chaudhuri
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09825
• Pdf URL: https://arxiv.org/pdf/2505.09825
• Copy Paste: [[2505.09825]] KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoning(https://arxiv.org/abs/2505.09825)
• Keywords: language model, llm
• Abstract: Each year, tens of millions of essays are written and graded in college-level English courses. Students are asked to analyze literary and cultural texts through a process known as close reading, in which they gather textual details to formulate evidence-based arguments. Despite being viewed as a basis for critical thinking and widely adopted as a required element of university coursework, close reading has never been evaluated on large language models (LLMs), and multi-discipline benchmarks like MMLU do not include literature as a subject. To fill this gap, we present KRISTEVA, the first close reading benchmark for evaluating interpretive reasoning, consisting of 1331 multiple-choice questions adapted from classroom data. With KRISTEVA, we propose three progressively more difficult sets of tasks to approximate different elements of the close reading process, which we use to test how well LLMs may seem to understand and reason about literary works: 1) extracting stylistic features, 2) retrieving relevant contextual information from parametric knowledge, and 3) multi-hop reasoning between style and external contexts. Our baseline results find that, while state-of-the-art LLMs possess some college-level close reading competency (accuracy 49.7% - 69.7%), their performances still trail those of experienced human evaluators on 10 out of our 11 tasks.
• 摘要：每年，在大学一级的英语课程中写下和分级数以千计的论文。要求学生通过称为近距阅读的过程来分析文学和文化文本，在该过程中，他们收集文本细节以提出基于证据的论点。尽管被视为批判性思维的基础，并被广泛用作大学课程的必需要素，但仔细阅读从未在大型语言模型（LLMS）上进行评估，MMLU（例如MMLU）的多学科基准并不包括文献作为主题。为了填补这一空白，我们提出了克里斯蒂娃（Kristeva），这是评估解释性推理的第一个紧密阅读基准，该基准由1331个由课堂数据改编的1331个多项选择问题组成。借助Kristeva，我们提出了三个越来越困难的任务集，以近似近距阅读过程的不同元素，我们用来测试LLM似乎对文学作品的理解和理由的理解：1）提取样式特征，2）从参数知识中检索相关的上下文信息，以及3）在样式和外部上下文之间进行多跳上的推理和外部背景。我们的基线结果发现，尽管最先进的LLM具有一些大学水平的近距阅读能力（准确性为49.7％-69.7％），但他们的表现仍然落后于我们11个任务中的10个经验丰富的人类评估者的表现。

Title: Do Large Language Models Know Conflict? Investigating Parametric vs. Non-Parametric Knowledge of LLMs for Conflict Forecasting

• Authors: Apollinaire Poli Nemkova, Sarath Chandra Lingareddy, Sagnik Ray Choudhury, Mark V. Albert
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.09852
• Pdf URL: https://arxiv.org/pdf/2505.09852
• Copy Paste: [[2505.09852]] Do Large Language Models Know Conflict? Investigating Parametric vs. Non-Parametric Knowledge of LLMs for Conflict Forecasting(https://arxiv.org/abs/2505.09852)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Large Language Models (LLMs) have shown impressive performance across natural language tasks, but their ability to forecast violent conflict remains underexplored. We investigate whether LLMs possess meaningful parametric knowledge-encoded in their pretrained weights-to predict conflict escalation and fatalities without external data. This is critical for early warning systems, humanitarian planning, and policy-making. We compare this parametric knowledge with non-parametric capabilities, where LLMs access structured and unstructured context from conflict datasets (e.g., ACLED, GDELT) and recent news reports via Retrieval-Augmented Generation (RAG). Incorporating external information could enhance model performance by providing up-to-date context otherwise missing from pretrained weights. Our two-part evaluation framework spans 2020-2024 across conflict-prone regions in the Horn of Africa and the Middle East. In the parametric setting, LLMs predict conflict trends and fatalities relying only on pretrained knowledge. In the non-parametric setting, models receive summaries of recent conflict events, indicators, and geopolitical developments. We compare predicted conflict trend labels (e.g., Escalate, Stable Conflict, De-escalate, Peace) and fatalities against historical data. Our findings highlight the strengths and limitations of LLMs for conflict forecasting and the benefits of augmenting them with structured external knowledge.
• 摘要：大型语言模型（LLM）在自然语言任务中表现出令人印象深刻的表现，但他们预测暴力冲突的能力仍然没有得到充实的态度。我们调查了LLM是否具有在没有外部数据的没有外部数据的冲突升级和死亡的预处理的重量中具有有意义的参数知识。这对于预警系统，人道主义计划和政策制定至关重要。我们将此参数知识与非参数能力进行了比较，其中LLMS可以从冲突数据集（例如，Acled，Gdelt）访问结构化和非结构化上下文，并通过检索演出的生成（RAG）访问最近的新闻报道。合并外部信息可以通过提供最新的上下文来增强模型性能，否则权重缺少。我们两部分的评估框架跨越了2020 - 2024年，遍布非洲之角和中东地区的冲突地区。在参数环境中，LLMS仅依靠验证的知识来预测冲突趋势和死亡。在非参数环境中，模型收到了最近的冲突事件，指标和地缘政治发展的摘要。我们将预测的冲突趋势标签（例如，升级，稳定的冲突，降级，和平）和死亡与历史数据进行了比较。我们的发现突出了LLM对冲突预测的优势和局限性，以及通过结构化的外部知识增强它们的好处。

Title: Crossing Borders Without Crossing Boundaries: How Sociolinguistic Awareness Can Optimize User Engagement with Localized Spanish AI Models Across Hispanophone Countries

• Authors: Martin Capdevila, Esteban Villa Turek, Ellen Karina Chumbe Fernandez, Luis Felipe Polo Galvez, Luis Cadavid, Andrea Marroquin, Rebeca Vargas Quesada, Johanna Crew, Nicole Vallejo Galarraga, Christopher Rodriguez, Diego Gutierrez, Radhi Datla
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09902
• Pdf URL: https://arxiv.org/pdf/2505.09902
• Copy Paste: [[2505.09902]] Crossing Borders Without Crossing Boundaries: How Sociolinguistic Awareness Can Optimize User Engagement with Localized Spanish AI Models Across Hispanophone Countries(https://arxiv.org/abs/2505.09902)
• Keywords: language model
• Abstract: Large language models are, by definition, based on language. In an effort to underscore the critical need for regional localized models, this paper examines primary differences between variants of written Spanish across Latin America and Spain, with an in-depth sociocultural and linguistic contextualization therein. We argue that these differences effectively constitute significant gaps in the quotidian use of Spanish among dialectal groups by creating sociolinguistic dissonances, to the extent that locale-sensitive AI models would play a pivotal role in bridging these divides. In doing so, this approach informs better and more efficient localization strategies that also serve to more adequately meet inclusivity goals, while securing sustainable active daily user growth in a major low-risk investment geographic area. Therefore, implementing at least the proposed five sub variants of Spanish addresses two lines of action: to foment user trust and reliance on AI language models while also demonstrating a level of cultural, historical, and sociolinguistic awareness that reflects positively on any internationalization strategy.
• 摘要：从定义上讲，大型语言模型是基于语言的。为了强调对区域局部模型的批判性需求，本文研究了整个拉丁美洲和西班牙的西班牙语变体之间的主要差异，其中有深入的社会文化和语言环境化。我们认为，这些差异有效地构成了辩证法群体中西班牙语使用的显着差距，这是通过造成社会语言不和谐的，以至于对环境敏感的AI模型在弥合这些鸿沟中起着关键作用。在这样做的过程中，这种方法为更好，更有效的本地化策略提供了信息，这些策略也可以更充分地实现包容性目标，同时确保了主要的低风险投资地理领域的可持续活跃的日常用户增长。因此，至少实施西班牙提议的五个子变体解决了两种行动：为了激发用户的信任和依赖AI语言模型，同时还展示了一种文化，历史和社会语言意识，以积极地反映出任何国际化战略。

Title: From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models

• Authors: Yidan Wang, Yubing Ren, Yanan Cao, Binxing Fang
• Subjects: cs.CL, cs.CR
• Abstract URL: https://arxiv.org/abs/2505.09924
• Pdf URL: https://arxiv.org/pdf/2505.09924
• Copy Paste: [[2505.09924]] From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models(https://arxiv.org/abs/2505.09924)
• Keywords: language model, llm
• Abstract: The rise of Large Language Models (LLMs) has heightened concerns about the misuse of AI-generated text, making watermarking a promising solution. Mainstream watermarking schemes for LLMs fall into two categories: logits-based and sampling-based. However, current schemes entail trade-offs among robustness, text quality, and security. To mitigate this, we integrate logits-based and sampling-based schemes, harnessing their respective strengths to achieve synergy. In this paper, we propose a versatile symbiotic watermarking framework with three strategies: serial, parallel, and hybrid. The hybrid framework adaptively embeds watermarks using token entropy and semantic entropy, optimizing the balance between detectability, robustness, text quality, and security. Furthermore, we validate our approach through comprehensive experiments on various datasets and models. Experimental results indicate that our method outperforms existing baselines and achieves state-of-the-art (SOTA) performance. We believe this framework provides novel insights into diverse watermarking paradigms. Our code is available at \href{this https URL}{this https URL}.
• 摘要：大型语言模型（LLM）的兴起对滥用AI生成的文本的担忧加剧了，使水印成为有前途的解决方案。 LLMS的主流水印方案分为两类：基于逻辑和基于抽样的基于逻辑。但是，目前的计划需要在鲁棒性，文本质量和安全性中进行权衡。为了减轻这种情况，我们整合了基于逻辑的基于逻辑的方案，并利用它们各自的优势以实现协同作用。在本文中，我们提出了一个多功能共生水印框架，采用三种策略：串行，平行和混合动力。混合框架使用令牌熵和语义熵自适应地嵌入了水印，从而优化了可检测性，鲁棒性，文本质量和安全性之间的平衡。此外，我们通过在各种数据集和模型上进行全面实验来验证我们的方法。实验结果表明，我们的方法的表现优于现有基准，并实现最先进的（SOTA）性能。我们认为，该框架为各种水印范式提供了新的见解。我们的代码可在\ href {此https url} {this HTTPS url}上获得。

Title: Rethinking Prompt Optimizers: From Prompt Merits to Optimization

• Authors: Zixiao Zhu, Hanzhang Zhou, Zijian Feng, Tianjiao Li, Chua Jia Jim Deryl, Mak Lee Onn, Gee Wah Ng, Kezhi Mao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.09930
• Pdf URL: https://arxiv.org/pdf/2505.09930
• Copy Paste: [[2505.09930]] Rethinking Prompt Optimizers: From Prompt Merits to Optimization(https://arxiv.org/abs/2505.09930)
• Keywords: language model, gpt, llm, prompt
• Abstract: Prompt optimization (PO) offers a practical alternative to fine-tuning large language models (LLMs), enabling performance improvements without altering model weights. Existing methods typically rely on advanced, large-scale LLMs like GPT-4 to generate optimized prompts. However, due to limited downward compatibility, verbose, instruction-heavy prompts from advanced LLMs can overwhelm lightweight inference models and degrade response quality. In this work, we rethink prompt optimization through the lens of interpretable design. We first identify a set of model-agnostic prompt quality merits and empirically validate their effectiveness in enhancing prompt and response quality. We then introduce MePO, a merit-guided, lightweight, and locally deployable prompt optimizer trained on our preference dataset built from merit-aligned prompts generated by a lightweight LLM. Unlike prior work, MePO avoids online optimization reliance, reduces cost and privacy concerns, and, by learning clear, interpretable merits, generalizes effectively to both large-scale and lightweight inference models. Experiments demonstrate that MePO achieves better results across diverse tasks and model types, offering a scalable and robust solution for real-world deployment. Our model and dataset are available at: this https URL
• 摘要：及时优化（PO）为微调大语言模型（LLM）提供了一种实用的替代方法，从而在不改变模型权重的情况下可以提高性能。现有方法通常依赖于高级大规模LLM（例如GPT-4）来生成优化的提示。但是，由于向下兼容性有限，高级LLM的详细指令提示会压倒轻量级推理模型并降低响应质量。在这项工作中，我们通过可解释设计的镜头重新考虑迅速优化。我们首先确定一组模型不合时宜的及时质量优点，并从经验上验证其在提高及时和响应质量方面的有效性。然后，我们介绍了MEPO，这是一种根据我们的优先数据集训练的，由轻量级LLM生成的优先协调提示，对我们的首选项数据集进行了培训。与先前的工作不同，MEPO避免在线优化依赖，减少成本和隐私问题，并通过学习清晰可解释的优点，有效地将大规模和轻量级推理模型推广。实验表明，MEPO在不同的任务和模型类型中取得更好的结果，为现实部署提供了可扩展且可靠的解决方案。我们的模型和数据集可用：此HTTPS URL

Title: Personalizing Large Language Models using Retrieval Augmented Generation and Knowledge Graph

• Authors: Deeksha Prahlad, Chanhee Lee, Dongha Kim, Hokeun Kim
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.09945
• Pdf URL: https://arxiv.org/pdf/2505.09945
• Copy Paste: [[2505.09945]] Personalizing Large Language Models using Retrieval Augmented Generation and Knowledge Graph(https://arxiv.org/abs/2505.09945)
• Keywords: language model, llm, hallucination, chat, retrieval augmented generation
• Abstract: The advent of large language models (LLMs) has allowed numerous applications, including the generation of queried responses, to be leveraged in chatbots and other conversational assistants. Being trained on a plethora of data, LLMs often undergo high levels of over-fitting, resulting in the generation of extra and incorrect data, thus causing hallucinations in output generation. One of the root causes of such problems is the lack of timely, factual, and personalized information fed to the LLM. In this paper, we propose an approach to address these problems by introducing retrieval augmented generation (RAG) using knowledge graphs (KGs) to assist the LLM in personalized response generation tailored to the users. KGs have the advantage of storing continuously updated factual information in a structured way. While our KGs can be used for a variety of frequently updated personal data, such as calendar, contact, and location data, we focus on calendar data in this paper. Our experimental results show that our approach works significantly better in understanding personal information and generating accurate responses compared to the baseline LLMs using personal data as text inputs, with a moderate reduction in response time.
• 摘要：大型语言模型（LLM）的出现允许在聊天机器人和其他对话助理中利用许多应用程序，包括生成查询响应。在经过大量数据的培训中，LLMS经常经历高水平的过度拟合，从而产生了额外的和错误的数据，从而导致产量产生幻觉。此类问题的根本原因之一是缺乏及时提供给LLM的及时，事实和个性化信息。在本文中，我们提出了一种方法，通过使用知识图（kgs）引入检索增强发电（RAG）来解决这些问题，以帮助LLM为针对用户量身定制的个性化响应生成。 KG的优势是以结构化的方式不断存储事实信息。虽然我们的kgs可用于各种经常更新的个人数据，例如日历，联系人和位置数据，但我们将重点放在本文中的日历数据上。我们的实验结果表明，与基线LLM相比，使用个人数据作为文本输入，我们的方法在理解个人信息和产生准确的响应方面的效果明显更好，并且响应时间适度减少。

Title: DIF: A Framework for Benchmarking and Verifying Implicit Bias in LLMs

• Authors: Lake Yin, Fan Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10013
• Pdf URL: https://arxiv.org/pdf/2505.10013
• Copy Paste: [[2505.10013]] DIF: A Framework for Benchmarking and Verifying Implicit Bias in LLMs(https://arxiv.org/abs/2505.10013)
• Keywords: language model, llm
• Abstract: As Large Language Models (LLMs) have risen in prominence over the past few years, there has been concern over the potential biases in LLMs inherited from the training data. Previous studies have examined how LLMs exhibit implicit bias, such as when response generation changes when different social contexts are introduced. We argue that this implicit bias is not only an ethical, but also a technical issue, as it reveals an inability of LLMs to accommodate extraneous information. However, unlike other measures of LLM intelligence, there are no standard methods to benchmark this specific subset of LLM bias. To bridge this gap, we developed a method for calculating an easily interpretable benchmark, DIF (Demographic Implicit Fairness), by evaluating preexisting LLM logic and math problem datasets with sociodemographic personas. We demonstrate that this method can statistically validate the presence of implicit bias in LLM behavior and find an inverse trend between question answering accuracy and implicit bias, supporting our argument.
• 摘要：在过去的几年中，大型语言模型（LLM）的突出性增长，人们一直担心培训数据中继承的LLMS的潜在偏见。先前的研究已经检查了LLM如何表现出隐式偏见，例如当引入不同的社会环境时，响应产生改变时。我们认为，这种隐性偏见不仅是一种道德偏见，而且是一个技术问题，因为它揭示了LLM无法容纳无关信息的能力。但是，与LLM智能的其他措施不同，没有标准方法可以基准该特定子集的LLM偏差。为了弥合这一差距，我们开发了一种通过评估具有社会人口统计学角色的llm逻辑和数学问题数据集来计算易于解释的基准（人口隐式公平）的方法。我们证明，该方法可以从统计学上验证LLM行为中隐性偏见的存在，并在问题回答准确性和隐式偏见之间找到反向趋势，从而支持我们的论点。

Title: CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability

• Authors: Han Peng, Jinhao Jiang, Zican Dong, Wayne Xin Zhao, Lei Fang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10063
• Pdf URL: https://arxiv.org/pdf/2505.10063
• Copy Paste: [[2505.10063]] CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability(https://arxiv.org/abs/2505.10063)
• Keywords: language model, llm, prompt
• Abstract: Advancements in Large Language Models (LLMs) have extended their input context length, yet they still struggle with retrieval and reasoning in long-context inputs. Existing methods propose to utilize the prompt strategy and retrieval head to alleviate this limitation. However, they still face challenges in balancing retrieval precision and recall, impacting their efficacy in answering questions. To address this, we introduce $\textbf{CAFE}$, a two-stage coarse-to-fine method to enhance multi-document question-answering capacities. By gradually eliminating the negative impacts of background and distracting documents, CAFE makes the responses more reliant on the evidence documents. Initially, a coarse-grained filtering method leverages retrieval heads to identify and rank relevant documents. Then, a fine-grained steering method guides attention to the most relevant content. Experiments across benchmarks show CAFE outperforms baselines, achieving up to 22.1% and 13.7% SubEM improvement over SFT and RAG methods on the Mistral model, respectively.
• 摘要：大语言模型（LLM）的进步已扩大了其输入上下文的长度，但他们仍然在长期内部输入中的检索和推理方面挣扎。现有方法建议利用及时的策略并取回头部以减轻此限制。但是，他们在平衡检索精度和回忆方面仍然面临挑战，从而影响了他们在回答问题时的功效。为了解决这个问题，我们介绍了$ \ textbf {cafe} $，这是一种两阶段的粗到限制方法，以增强多文件问答能力。通过逐渐消除背景和分心文件的负面影响，Cafe使回应更依赖证据文件。最初，一种粗粒的过滤方法利用检索头来识别和对相关文档进行排名。然后，一种细颗粒的转向方法指导人们注意最相关的内容。基准跨基准的实验表明，咖啡馆的表现优于基线，在Mistral模型上，SUBEM对SFT和抹布方法的改善高达22.1％和13.7％。

Title: Dark LLMs: The Growing Threat of Unaligned AI Models

• Authors: Michael Fire, Yitzhak Elbazis, Adi Wasenstein, Lior Rokach
• Subjects: cs.CL, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.10066
• Pdf URL: https://arxiv.org/pdf/2505.10066
• Copy Paste: [[2505.10066]] Dark LLMs: The Growing Threat of Unaligned AI Models(https://arxiv.org/abs/2505.10066)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) rapidly reshape modern life, advancing fields from healthcare to education and beyond. However, alongside their remarkable capabilities lies a significant threat: the susceptibility of these models to jailbreaking. The fundamental vulnerability of LLMs to jailbreak attacks stems from the very data they learn from. As long as this training data includes unfiltered, problematic, or 'dark' content, the models can inherently learn undesirable patterns or weaknesses that allow users to circumvent their intended safety controls. Our research identifies the growing threat posed by dark LLMs models deliberately designed without ethical guardrails or modified through jailbreak techniques. In our research, we uncovered a universal jailbreak attack that effectively compromises multiple state-of-the-art models, enabling them to answer almost any question and produce harmful outputs upon request. The main idea of our attack was published online over seven months ago. However, many of the tested LLMs were still vulnerable to this attack. Despite our responsible disclosure efforts, responses from major LLM providers were often inadequate, highlighting a concerning gap in industry practices regarding AI safety. As model training becomes more accessible and cheaper, and as open-source LLMs proliferate, the risk of widespread misuse escalates. Without decisive intervention, LLMs may continue democratizing access to dangerous knowledge, posing greater risks than anticipated.
• 摘要：大型语言模型（LLM）迅速重塑现代生活，从医疗保健到教育及其他地区的领域。但是，除了它们的显着能力外，这是一个重大威胁：这些模型对越狱的敏感性。 LLMS对越狱攻击的根本脆弱性源于他们从中学到的数据。只要此培训数据包括未经过滤，有问题或“黑暗”内容，这些模型就可以固有地学习不良模式或弱点，使用户可以规避其预期的安全控制。我们的研究确定了故意设计的无道德护栏或通过越狱技术进行修改的Dark LLMS模型构成的日益增长的威胁。在我们的研究中，我们发现了一场普遍的越狱攻击，有效地损害了多种最先进的模型，使他们能够回答几乎所有问题并应要求产生有害产量。我们攻击的主要思想是在七个月前在线发布的。但是，许多经过测试的LLM仍然容易受到这一攻击的影响。尽管我们负责任的披露工作，但主要LLM提供商的回应往往不足，强调了有关AI安全行业实践的差距。随着模型培训变得更加易于访问和便宜，并且随着开源LLM的繁殖，滥用的风险会升级。如果没有决定性的干预，LLM可能会继续使获得危险知识的机会民主化，从而带来比预期更大的风险。

Title: Designing and Contextualising Probes for African Languages

• Authors: Wisdom Aduah, Francois Meyer
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10081
• Pdf URL: https://arxiv.org/pdf/2505.10081
• Copy Paste: [[2505.10081]] Designing and Contextualising Probes for African Languages(https://arxiv.org/abs/2505.10081)
• Keywords: language model
• Abstract: Pretrained language models (PLMs) for African languages are continually improving, but the reasons behind these advances remain unclear. This paper presents the first systematic investigation into probing PLMs for linguistic knowledge about African languages. We train layer-wise probes for six typologically diverse African languages to analyse how linguistic features are distributed. We also design control tasks, a way to interpret probe performance, for the MasakhaPOS dataset. We find PLMs adapted for African languages to encode more linguistic information about target languages than massively multilingual PLMs. Our results reaffirm previous findings that token-level syntactic information concentrates in middle-to-last layers, while sentence-level semantic information is distributed across all layers. Through control tasks and probing baselines, we confirm that performance reflects the internal knowledge of PLMs rather than probe memorisation. Our study applies established interpretability techniques to African-language PLMs. In doing so, we highlight the internal mechanisms underlying the success of strategies like active learning and multilingual adaptation.
• 摘要：非洲语言的验证语言模型（PLM）正在不断改善，但是这些进步背后的原因尚不清楚。本文介绍了对PLM的首次系统调查，以了解有关非洲语言的语言知识。我们针对六种类型上多样化的非洲语言进行培训层探针，以分析语言特征的分布方式。我们还为MASAKHAPOS数据集设计了控制任务，这是一种解释探针性能的方法。我们发现，与大量多语言PLM相比，适用于非洲语言的PLMs编码有关目标语言的更多语言信息。我们的结果重申了以前的发现，即令牌级的句法信息集中在中间到层次的层中，而句子级别的语义信息则分布在所有层中。通过控制任务和探测基线，我们确认性能反映了PLM的内部知识，而不是探测记忆。我们的研究将确定的可解释性技术应用于非洲语言PLM。为此，我们强调了积极学习和多语言适应等策略成功的内部机制。

Title: XRAG: Cross-lingual Retrieval-Augmented Generation

• Authors: Wei Liu, Sony Trenous, Leonardo F. R. Ribeiro, Bill Byrne, Felix Hieber
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10089
• Pdf URL: https://arxiv.org/pdf/2505.10089
• Copy Paste: [[2505.10089]] XRAG: Cross-lingual Retrieval-Augmented Generation(https://arxiv.org/abs/2505.10089)
• Keywords: llm, retrieval-augmented generation
• Abstract: We propose XRAG, a novel benchmark designed to evaluate the generation abilities of LLMs in cross-lingual Retrieval-Augmented Generation (RAG) settings where the user language does not match the retrieval results. XRAG is constructed from recent news articles to ensure that its questions require external knowledge to be answered. It covers the real-world scenarios of monolingual and multilingual retrieval, and provides relevancy annotations for each retrieved document. Our novel dataset construction pipeline results in questions that require complex reasoning, as evidenced by the significant gap between human and LLM performance. Consequently, XRAG serves as a valuable benchmark for studying LLM reasoning abilities, even before considering the additional cross-lingual complexity. Experimental results on five LLMs uncover two previously unreported challenges in cross-lingual RAG: 1) in the monolingual retrieval setting, all evaluated models struggle with response language correctness; 2) in the multilingual retrieval setting, the main challenge lies in reasoning over retrieved information across languages rather than generation of non-English text.
• 摘要：我们提出了XRAG，这是一种新颖的基准测试，旨在评估LLM在跨语言检索效果生成（RAG）设置中的发电能力，在该设置中，用户语言不符合检索结果。 XRAG是根据最近的新闻文章构建的，以确保其问题需要回答外部知识。它涵盖了单语和多语言检索的现实情况，并为每个检索的文档提供了相关注释。我们的新型数据集构建管道导致需要复杂推理的问题，这是由人类和LLM性能之间的显着差距所证明的。因此，即使在考虑额外的跨语性复杂性之前，XRAG也是研究LLM推理能力的宝贵基准。五个LLM的实验结果发现了跨语性抹布中两个以前未报告的挑战：1）在单语的检索环境中，所有评估的模型都在响应语言正确性中挣扎； 2）在多语言检索环境中，主要挑战在于推理跨语言检索信息而不是产生非英语文本。

Title: What Does Neuro Mean to Cardio? Investigating the Role of Clinical Specialty Data in Medical LLMs

• Authors: Xinlan Yan, Di Wu, Yibin Lei, Christof Monz, Iacer Calixto
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10113
• Pdf URL: https://arxiv.org/pdf/2505.10113
• Copy Paste: [[2505.10113]] What Does Neuro Mean to Cardio? Investigating the Role of Clinical Specialty Data in Medical LLMs(https://arxiv.org/abs/2505.10113)
• Keywords: language model, llm
• Abstract: In this paper, we introduce S-MedQA, an English medical question-answering (QA) dataset for benchmarking large language models in fine-grained clinical specialties. We use S-MedQA to check the applicability of a popular hypothesis related to knowledge injection in the knowledge-intense scenario of medical QA, and show that: 1) training on data from a speciality does not necessarily lead to best performance on that specialty and 2) regardless of the specialty fine-tuned on, token probabilities of clinically relevant terms for all specialties increase consistently. Thus, we believe improvement gains come mostly from domain shifting (e.g., general to medical) rather than knowledge injection and suggest rethinking the role of fine-tuning data in the medical domain. We release S-MedQA and all code needed to reproduce all our experiments to the research community.
• 摘要：在本文中，我们介绍了S-Medqa，这是一种英语医学询问（QA）数据集，用于在细粒度的临床专业中基准大型语言模型。我们使用S-MEDQA在医学质量检查的知识强度方面检查与知识注入相关的流行假设的适用性，并表明：1）从专业的数据进行培训并不一定会在该专业上获得最佳性能，而2）不管特殊的精细，对临床上相关的特殊性的特殊性概率都会增加所有特殊性的特殊性。因此，我们认为改进的收益主要来自域的转移（例如一般到医学），而不是知识注入，并建议重新考虑微调数据在医疗领域中的作用。我们发布了S-MEDQA以及将所有实验重现给研究社区所需的所有代码。

Title: GE-Chat: A Graph Enhanced RAG Framework for Evidential Response Generation of LLMs

• Authors: Longchao Da, Parth Mitesh Shah, Kuan-Ru Liou, Jiaxing Zhang, Hua Wei
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10143
• Pdf URL: https://arxiv.org/pdf/2505.10143
• Copy Paste: [[2505.10143]] GE-Chat: A Graph Enhanced RAG Framework for Evidential Response Generation of LLMs(https://arxiv.org/abs/2505.10143)
• Keywords: language model, llm, chat, retrieval-augmented generation, chain-of-thought, agent
• Abstract: Large Language Models are now key assistants in human decision-making processes. However, a common note always seems to follow: "LLMs can make mistakes. Be careful with important info." This points to the reality that not all outputs from LLMs are dependable, and users must evaluate them manually. The challenge deepens as hallucinated responses, often presented with seemingly plausible explanations, create complications and raise trust issues among users. To tackle such issue, this paper proposes GE-Chat, a knowledge Graph enhanced retrieval-augmented generation framework to provide Evidence-based response generation. Specifically, when the user uploads a material document, a knowledge graph will be created, which helps construct a retrieval-augmented agent, enhancing the agent's responses with additional knowledge beyond its training corpus. Then we leverage Chain-of-Thought (CoT) logic generation, n-hop sub-graph searching, and entailment-based sentence generation to realize accurate evidence retrieval. We demonstrate that our method improves the existing models' performance in terms of identifying the exact evidence in a free-form context, providing a reliable way to examine the resources of LLM's conclusion and help with the judgment of the trustworthiness.
• 摘要：大型语言模型现在是人类决策过程中的关键助手。但是，似乎总是有一个普遍的说明：“ LLM会犯错误。要小心重要的信息。”这表明并非来自LLM的所有输出都可靠，用户必须手动评估它们。随着幻觉的回应通常带有合理的解释，创造并发症并引发用户之间的信任问题，挑战加深了。为了解决此类问题，本文提出了GE-CHAT，知识图增强了检索功能的生成框架，以提供基于证据的响应生成。具体而言，当用户上传物料文档时，将创建知识图，这有助于构建检索仪的代理，并以超出其培训语料库以外的其他知识来增强代理的响应。然后，我们利用经过思考链（COT）的逻辑生成，N-Hop子图和基于要求的句子生成来实现准确的证据检索。我们证明，我们的方法在自由形式的背景下确定确切的证据，提供了一种可靠的方式来检查LLM结论的资源，并有助于对可信赖性的判断，从而提高了现有模型的性能。

Title: Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning

• Authors: Yoichi Ishibashi, Taro Yano, Masafumi Oyamada
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.10182
• Pdf URL: https://arxiv.org/pdf/2505.10182
• Copy Paste: [[2505.10182]] Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning(https://arxiv.org/abs/2505.10182)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have demonstrated significant improvements in reasoning capabilities through supervised fine-tuning and reinforcement learning. However, when training reasoning models, these approaches are primarily applicable to specific domains such as mathematics and programming, which imposes fundamental constraints on the breadth and scalability of training data. In contrast, continual pretraining (CPT) offers the advantage of not requiring task-specific signals. Nevertheless, how to effectively synthesize training data for reasoning and how such data affect a wide range of domains remain largely unexplored. This study provides a detailed evaluation of Reasoning CPT, a form of CPT that uses synthetic data to reconstruct the hidden thought processes underlying texts, based on the premise that texts are the result of the author's thinking process. Specifically, we apply Reasoning CPT to Gemma2-9B using synthetic data with hidden thoughts derived from STEM and Law corpora, and compare it to standard CPT on the MMLU benchmark. Our analysis reveals that Reasoning CPT consistently improves performance across all evaluated domains. Notably, reasoning skills acquired in one domain transfer effectively to others; the performance gap with conventional methods widens as problem difficulty increases, with gains of up to 8 points on the most challenging problems. Furthermore, models trained with hidden thoughts learn to adjust the depth of their reasoning according to problem difficulty.
• 摘要：大型语言模型（LLMS）通过监督的微调和加强学习表现出了推理能力的显着改善。但是，当训练推理模型时，这些方法主要适用于数学和编程等特定领域，这些域对培训数据的广度和可扩展性施加了基本限制。相反，持续预处理（CPT）提供了不需要特定于任务信号的优势。然而，如何有效地将培训数据用于推理以及此类数据如何影响广泛的域仍然没有探索。这项研究提供了对推理CPT的详细评估，CPT是一种使用合成数据来重构文本基础的隐藏思维过程的形式，基于本文是作者思维过程的结果。具体来说，我们使用综合数据将CPT应用于GEMMA2-9B，并具有从STEM和Law Corpora中获得的隐藏思想，并将其与MMLU基准上的标准CPT进行比较。我们的分析表明，推理CPT始终提高所有评估领域的性能。值得注意的是，在一个领域中获得的推理技能有效地转移给了他人；随着问题难度的增加，传统方法的性能差距扩大，在最具挑战性的问题上获得了多达8分。此外，接受过隐藏思想训练的模型学会根据问题难以调整其推理的深度。

Title: The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think

• Authors: Seongyun Lee, Seungone Kim, Minju Seo, Yongrae Jo, Dongyoung Go, Hyeonbin Hwang, Jinho Park, Xiang Yue, Sean Welleck, Graham Neubig, Moontae Lee, Minjoon Seo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.10185
• Pdf URL: https://arxiv.org/pdf/2505.10185
• Copy Paste: [[2505.10185]] The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think(https://arxiv.org/abs/2505.10185)
• Keywords: language model, chain-of-thought
• Abstract: Long chain-of-thought (CoT) is an essential ingredient in effective usage of modern large language models, but our understanding of the reasoning strategies underlying these capabilities remains limited. While some prior works have attempted to categorize CoTs using predefined strategy types, such approaches are constrained by human intuition and fail to capture the full diversity of model behaviors. In this work, we introduce the CoT Encyclopedia, a bottom-up framework for analyzing and steering model reasoning. Our method automatically extracts diverse reasoning criteria from model-generated CoTs, embeds them into a semantic space, clusters them into representative categories, and derives contrastive rubrics to interpret reasoning behavior. Human evaluations show that this framework produces more interpretable and comprehensive analyses than existing methods. Moreover, we demonstrate that this understanding enables performance gains: we can predict which strategy a model is likely to use and guide it toward more effective alternatives. Finally, we provide practical insights, such as that training data format (e.g., free-form vs. multiple-choice) has a far greater impact on reasoning behavior than data domain, underscoring the importance of format-aware model design.
• 摘要：长期的经营链（COT）是有效使用现代大型语言模型的重要组成部分，但是我们对这些能力基础的推理策略的理解仍然有限。尽管一些先前的作品试图使用预定义的策略类型对COTS进行分类，但这种方法受到人类直觉的约束，无法捕获模型行为的全部多样性。在这项工作中，我们介绍了COT百科全书，这是一个自下而上的框架，用于分析和转向模型推理。我们的方法会自动从模型生成的COTS中提取不同的推理标准，将它们嵌入语义空间中，将它们插入代表性类别，并得出对比性的编译以解释推理行为。人类评估表明，与现有方法相比，该框架会产生更容易解释和全面的分析。此外，我们证明了这种理解能够提高绩效：我们可以预测模型可能会使用哪种策略并指导其朝着更有效的替代方案进行指导。最后，我们提供实用的见解，例如训练数据格式（例如，自由形式与多项选择）对推理行为的影响要比数据域具有更大的影响，从而强调了格式感知模型设计的重要性。

Title: VQ-Logits: Compressing the Output Bottleneck of Large Language Models via Vector Quantized Logits

• Authors: Jintian Shao, Hongyi Huang, Jiayi Wu, YiMing Cheng, ZhiYu Wu, You Shan, MingKai Zheng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10202
• Pdf URL: https://arxiv.org/pdf/2505.10202
• Copy Paste: [[2505.10202]] VQ-Logits: Compressing the Output Bottleneck of Large Language Models via Vector Quantized Logits(https://arxiv.org/abs/2505.10202)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have achieved remarkable success but face significant computational and memory challenges, particularly due to their extensive output vocabularies. The final linear projection layer, mapping hidden states to vocabulary-sized logits, often constitutes a substantial portion of the model's parameters and computational cost during inference. Existing methods like adaptive softmax or hierarchical softmax introduce structural complexities. In this paper, we propose VQ-Logits, a novel approach that leverages Vector Quantization (VQ) to drastically reduce the parameter count and computational load of the LLM output layer. VQ-Logits replaces the large V * dmodel output embedding matrix with a small, shared codebook of K embedding vectors (K << V ). Each token in the vocabulary is mapped to one of these K codebook vectors. The LLM predicts logits over this compact codebook, which are then efficiently "scattered" to the full vocabulary space using the learned or preassigned mapping. We demonstrate through extensive experiments on standard language modeling benchmarks (e.g., WikiText-103, C4) that VQ-Logits can achieve up to 99% parameter reduction in the output layer and 6x speedup in logit computation, with only a marginal 4% increase in perplexity compared to full softmax baselines. We further provide detailed ablation studies on codebook size, initialization, and learning strategies, showcasing the robustness and effectiveness of our approach.
• 摘要：大型语言模型（LLM）取得了巨大的成功，但面临着重大的计算和记忆挑战，尤其是由于它们广泛的输出词汇。最终的线性投影层将隐藏的状态映射到词汇量逻辑，通常构成了模型参数的很大一部分和推断期间的计算成本。现有方法（例如自适应软磁性或层次软磁性）会引入结构复杂性。在本文中，我们提出了VQ-Logits，这是一种利用向量量化（VQ）的新方法，以大大减少LLM输出层的参数计数和计算负载。 VQ-Logits用一个小的，共享的k嵌入矢量代码替代了大型V * DMODEL嵌入矩阵（k << v）。词汇中的每个令牌都映射到这些K代码书矢量之一。 LLM可以预测该紧凑的代码簿上的logits，然后使用学识渊博的或预先签名的映射有效地将其“分散”到完整的词汇空间中。我们通过对标准语言建模基准测试基准（例如Wikitext-103，C4）进行的广泛实验表明，VQ-Logits可以实现高达99％的输出层参数降低和Logit Comput中的6倍加速度，而与全部软基质相比，相比性仅增加了相差4％。我们进一步提供了有关代码书大小，初始化和学习策略的详细消融研究，展示了我们方法的鲁棒性和有效性。

Title: RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Reward

• Authors: Zongsheng Wang, Kaili Sun, Bowen Wu, Qun Yu, Ying Li, Baoxun Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10218
• Pdf URL: https://arxiv.org/pdf/2505.10218
• Copy Paste: [[2505.10218]] RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Reward(https://arxiv.org/abs/2505.10218)
• Keywords: llm, chain-of-thought, agent
• Abstract: Role-playing conversational agents (RPCAs) face persistent challenges in maintaining role consistency. To address this, we propose RAIDEN-R1, a novel reinforcement learning framework that integrates Verifiable Role-Awareness Reward (VRAR). The method introduces both singular and multi-term mining strategies to generate quantifiable rewards by assessing role-specific keys. Additionally, we construct a high-quality, role-aware Chain-of-Thought dataset through multi-LLM collaboration, and implement experiments to enhance reasoning coherence. Experiments on the RAIDEN benchmark demonstrate RAIDEN-R1's superiority: our 14B-GRPO model achieves 88.04% and 88.65% accuracy on Script-Based Knowledge and Conversation Memory metrics, respectively, outperforming baseline models while maintaining robustness. Case analyses further reveal the model's enhanced ability to resolve conflicting contextual cues and sustain first-person narrative consistency. This work bridges the non-quantifiability gap in RPCA training and provides insights into role-aware reasoning patterns, advancing the development of RPCAs.
• 摘要：角色扮演对话剂（RPCAS）在保持角色一致性方面面临持续的挑战。为了解决这个问题，我们提出了Raiden-R1，这是一个新颖的增强学习框架，该框架整合了可验证的角色意识奖励（VRAR）。该方法引入了单数和多期挖掘策略，以通过评估特定角色的密钥来产生可量化的奖励。此外，我们通过多LLM协作构建了高质量的，具有角色感知的思想链数据集，并实施实验以增强推理连贯性。 RAIDEN基准的实验证明了Raiden-R1的优势：我们的14B-GRPO模型在基于脚本的知识和对话记忆指标上分别达到了88.04％和88.65％的精度，在维持鲁棒性的同时，分别超过了基线模型。案例分析进一步揭示了该模型的增强能力，可以解决冲突的上下文线索并维持第一人称叙事的一致性。这项工作弥合了RPCA培训中的非量化性差距，并提供了对角色感知推理模式的见解，从而推进了RPCAS的发展。

Title: Comparing LLM Text Annotation Skills: A Study on Human Rights Violations in Social Media Data

• Authors: Poli Apollinaire Nemkova, Solomon Ubani, Mark V. Albert
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.10260
• Pdf URL: https://arxiv.org/pdf/2505.10260
• Copy Paste: [[2505.10260]] Comparing LLM Text Annotation Skills: A Study on Human Rights Violations in Social Media Data(https://arxiv.org/abs/2505.10260)
• Keywords: language model, gpt, llm, prompt
• Abstract: In the era of increasingly sophisticated natural language processing (NLP) systems, large language models (LLMs) have demonstrated remarkable potential for diverse applications, including tasks requiring nuanced textual understanding and contextual reasoning. This study investigates the capabilities of multiple state-of-the-art LLMs - GPT-3.5, GPT-4, LLAMA3, Mistral 7B, and Claude-2 - for zero-shot and few-shot annotation of a complex textual dataset comprising social media posts in Russian and Ukrainian. Specifically, the focus is on the binary classification task of identifying references to human rights violations within the dataset. To evaluate the effectiveness of these models, their annotations are compared against a gold standard set of human double-annotated labels across 1000 samples. The analysis includes assessing annotation performance under different prompting conditions, with prompts provided in both English and Russian. Additionally, the study explores the unique patterns of errors and disagreements exhibited by each model, offering insights into their strengths, limitations, and cross-linguistic adaptability. By juxtaposing LLM outputs with human annotations, this research contributes to understanding the reliability and applicability of LLMs for sensitive, domain-specific tasks in multilingual contexts. It also sheds light on how language models handle inherently subjective and context-dependent judgments, a critical consideration for their deployment in real-world scenarios.
• 摘要：在越来越复杂的自然语言处理（NLP）系统的时代，大型语言模型（LLMS）在各种应用中表现出了巨大的潜力，包括需要细微的文本理解和上下文推理的任务。这项研究调查了多个最先进的LLM-GPT-3.5，GPT-4，LLAMA3，MISTRAL 7B和CLAUDE-2-用于零拍摄，以及一个复杂的文本数据集的零摄入量，其中包含俄罗斯和乌克兰尼亚语的社交媒体帖子的复杂文本数据集。具体而言，重点是识别数据集中侵犯人权行为的二进制分类任务。为了评估这些模型的有效性，将其注释与1000个样本中的人类双重注销标签进行了比较。分析包括在不同提示条件下评估注释绩效，并在英语和俄语中提供提示。此外，该研究还探讨了每个模型表现出的错误和分歧的独特模式，从而提供了有关其优势，局限性和跨语言适应性的见解。通过将LLM输出与人类注释并列，这项研究有助于了解LLM在多语言环境中敏感的，特定领域的任务的可靠性和适用性。它还阐明了语言模型如何处理固有的主观和上下文依赖性判断，这是对其在现实世界情景中部署的关键考虑。

Title: The Evolving Landscape of Generative Large Language Models and Traditional Natural Language Processing in Medicine

• Authors: Rui Yang, Huitao Li, Matthew Yu Heng Wong, Yuhe Ke, Xin Li, Kunyu Yu, Jingchi Liao, Jonathan Chong Kai Liew, Sabarinath Vinod Nair, Jasmine Chiat Ling Ong, Irene Li, Douglas Teodoro, Chuan Hong, Daniel Shu Wei Ting, Nan Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.10261
• Pdf URL: https://arxiv.org/pdf/2505.10261
• Copy Paste: [[2505.10261]] The Evolving Landscape of Generative Large Language Models and Traditional Natural Language Processing in Medicine(https://arxiv.org/abs/2505.10261)
• Keywords: language model, llm
• Abstract: Natural language processing (NLP) has been traditionally applied to medicine, and generative large language models (LLMs) have become prominent recently. However, the differences between them across different medical tasks remain underexplored. We analyzed 19,123 studies, finding that generative LLMs demonstrate advantages in open-ended tasks, while traditional NLP dominates in information extraction and analysis tasks. As these technologies advance, ethical use of them is essential to ensure their potential in medical applications.
• 摘要：自然语言处理（NLP）传统上已应用于医学，而生成的大语言模型（LLM）最近变得突出。但是，在不同的医疗任务中它们之间的差异仍然没有得到充实的态度。我们分析了19,123项研究，发现生成LLMS在开放式任务中证明了优势，而传统的NLP在信息提取和分析任务中占主导地位。随着这些技术的推进，对它们的道德使用对于确保其在医疗应用中的潜力至关重要。

Title: From Questions to Clinical Recommendations: Large Language Models Driving Evidence-Based Clinical Decision Making

• Authors: Dubai Li, Nan Jiang, Kangping Huang, Ruiqi Tu, Shuyu Ouyang, Huayu Yu, Lin Qiao, Chen Yu, Tianshu Zhou, Danyang Tong, Qian Wang, Mengtao Li, Xiaofeng Zeng, Yu Tian, Xinping Tian, Jingsong Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10282
• Pdf URL: https://arxiv.org/pdf/2505.10282
• Copy Paste: [[2505.10282]] From Questions to Clinical Recommendations: Large Language Models Driving Evidence-Based Clinical Decision Making(https://arxiv.org/abs/2505.10282)
• Keywords: language model, llm
• Abstract: Clinical evidence, derived from rigorous research and data analysis, provides healthcare professionals with reliable scientific foundations for informed decision-making. Integrating clinical evidence into real-time practice is challenging due to the enormous workload, complex professional processes, and time constraints. This highlights the need for tools that automate evidence synthesis to support more efficient and accurate decision making in clinical settings. This study introduces Quicker, an evidence-based clinical decision support system powered by large language models (LLMs), designed to automate evidence synthesis and generate clinical recommendations modeled after standard clinical guideline development processes. Quicker implements a fully automated chain that covers all phases, from questions to clinical recommendations, and further enables customized decision-making through integrated tools and interactive user interfaces. To evaluate Quicker's capabilities, we developed the Q2CRBench-3 benchmark dataset, based on clinical guideline development records for three different diseases. Experimental results highlighted Quicker's strong performance, with fine-grained question decomposition tailored to user preferences, retrieval sensitivities comparable to human experts, and literature screening performance approaching comprehensive inclusion of relevant studies. In addition, Quicker-assisted evidence assessment effectively supported human reviewers, while Quicker's recommendations were more comprehensive and logically coherent than those of clinicians. In system-level testing, collaboration between a single reviewer and Quicker reduced the time required for recommendation development to 20-40 minutes. In general, our findings affirm the potential of Quicker to help physicians make quicker and more reliable evidence-based clinical decisions.
• 摘要：从严格的研究和数据分析中得出的临床证据为医疗保健专业人员提供了可靠的科学基础，以提供明智的决策。由于巨大的工作量，复杂的专业流程和时间限制，将临床证据整合到实时实践中是具有挑战性的。这凸显了需要自动化证据综合以支持临床环境中更有效和准确决策的工具。这项研究介绍了更快的是一种基于循证的临床决策支持系统，该系统由大型语言模型（LLMS）提供动力，旨在自动化证据综合并生成以标准临床指南开发过程进行建模的临床建议。更快地实现了一条全自动链，该链涵盖了从问题到临床建议，并通过集成工具和交互式用户界面进行定制的决策。为了评估更快的功能，我们根据三种不同疾病的临床指南开发记录开发了Q2CRBENCH-3基准数据集。实验结果强调了Quicker的出色表现，其良好的问题分解是针对用户偏好量的，可与人类专家相当的检索敏感性以及文献筛选的绩效，接近相关研究的全面包含。此外，更快的证据评估有效地支持了人类审稿人，而Quicker的建议比临床医生的建议更全面和逻辑。在系统级测试中，单个审阅者与更快的时间之间的协作将推荐开发所需的时间缩短到20-40分钟。总的来说，我们的发现肯定了更快地帮助医生做出更快，更可靠的基于证据的临床决策的潜力。

Title: J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learning

• Authors: Chenxi Whitehouse, Tianlu Wang, Ping Yu, Xian Li, Jason Weston, Ilia Kulikov, Swarnadeep Saha
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.10320
• Pdf URL: https://arxiv.org/pdf/2505.10320
• Copy Paste: [[2505.10320]] J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learning(https://arxiv.org/abs/2505.10320)
• Keywords: llm, prompt, chain-of-thought
• Abstract: The progress of AI is bottlenecked by the quality of evaluation, and powerful LLM-as-a-Judge models have proved to be a core solution. Improved judgment ability is enabled by stronger chain-of-thought reasoning, motivating the need to find the best recipes for training such models to think. In this work we introduce J1, a reinforcement learning approach to training such models. Our method converts both verifiable and non-verifiable prompts to judgment tasks with verifiable rewards that incentivize thinking and mitigate judgment bias. In particular, our approach outperforms all other existing 8B or 70B models when trained at those sizes, including models distilled from DeepSeek-R1. J1 also outperforms o1-mini, and even R1 on some benchmarks, despite training a smaller model. We provide analysis and ablations comparing Pairwise-J1 vs Pointwise-J1 models, offline vs online training recipes, reward strategies, seed prompts, and variations in thought length and content. We find that our models make better judgments by learning to outline evaluation criteria, comparing against self-generated reference answers, and re-evaluating the correctness of model responses.
• 摘要：AI的进度被评估的质量瓶颈而言，强大的LLM-AS-A-A-Gudge模型被证明是核心解决方案。通过更强的经过思考的推理，可以提高判断力的能力，从而激发了寻找培训此类模型思考的最佳食谱的需求。在这项工作中，我们介绍了J1，这是一种培训此类模型的增强学习方法。我们的方法将可验证和不可验证的提示转换为具有可验证奖励的判断任务，从而激励思维并减轻判断偏见。特别是，我们的方法在接受这些尺寸的培训时，均优于所有其他现有的8B或70B型号，包括从DeepSeek-R1蒸馏出来的型号。尽管训练了较小的型号，但J1在某些基准测试中的表现也优于O1米尼，甚至在某些基准上的R1。我们提供分析和消融，以比较Pointwise-J1模型，离线与在线培训食谱，奖励策略，种子提示以及思想长度和内容的变化。我们发现，我们的模型通过学习概述评估标准，与自我生成的参考答案进行比较并重新评估模型响应的正确性来做出更好的判断。

Title: LDIR: Low-Dimensional Dense and Interpretable Text Embeddings with Relative Representations

• Authors: Yile Wang, Zhanyu Shen, Hui Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10354
• Pdf URL: https://arxiv.org/pdf/2505.10354
• Copy Paste: [[2505.10354]] LDIR: Low-Dimensional Dense and Interpretable Text Embeddings with Relative Representations(https://arxiv.org/abs/2505.10354)
• Keywords: language model, llm
• Abstract: Semantic text representation is a fundamental task in the field of natural language processing. Existing text embedding (e.g., SimCSE and LLM2Vec) have demonstrated excellent performance, but the values of each dimension are difficult to trace and interpret. Bag-of-words, as classic sparse interpretable embeddings, suffers from poor performance. Recently, Benara et al. (2024) propose interpretable text embeddings using large language models, which forms "0/1" embeddings based on responses to a series of questions. These interpretable text embeddings are typically high-dimensional (larger than 10,000). In this work, we propose Low-dimensional (lower than 500) Dense and Interpretable text embeddings with Relative representations (LDIR). The numerical values of its dimensions indicate semantic relatedness to different anchor texts through farthest point sampling, offering both semantic representation as well as a certain level of traceability and interpretability. We validate LDIR on multiple semantic textual similarity, retrieval, and clustering tasks. Extensive experimental results show that LDIR performs close to the black-box baseline models and outperforms the interpretable embeddings baselines with much fewer dimensions. Code is available at this https URL.
• 摘要：语义文本表示是自然语言处理领域的基本任务。现有的文本嵌入（例如SIMCSE和LLM2VEC）表现出了出色的性能，但是每个维度的值很难追踪和解释。单词袋作为经典的稀疏解释嵌入，表现不佳。最近，Benara等。 （2024）使用大语言模型提出可解释的文本嵌入，该模型基于对一系列问题的回答形成“ 0/1”嵌入。这些可解释的文本嵌入通常是高维的（大于10,000）。在这项工作中，我们提出了具有相对表示（LDIR）的低维（低于500）且可解释的文本嵌入。其尺寸的数值通过最远的点采样表示与不同锚文本的语义相关性，提供语义表示以及一定程度的可追溯性和解释性。我们在多个语义文本相似性，检索和聚类任务上验证LDIR。广泛的实验结果表明，LDIR的性能靠近黑盒基线模型，并且优于可解释的嵌入基线的基线，其维度较少。代码可在此HTTPS URL上找到。

Title: Coherent Language Reconstruction from Brain Recordings with Flexible Multi-Modal Input Stimuli

• Authors: Chunyu Ye, Shaonan Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10356
• Pdf URL: https://arxiv.org/pdf/2505.10356
• Copy Paste: [[2505.10356]] Coherent Language Reconstruction from Brain Recordings with Flexible Multi-Modal Input Stimuli(https://arxiv.org/abs/2505.10356)
• Keywords: language model
• Abstract: Decoding thoughts from brain activity offers valuable insights into human cognition and enables promising applications in brain-computer interaction. While prior studies have explored language reconstruction from fMRI data, they are typically limited to single-modality inputs such as images or audio. In contrast, human thought is inherently multimodal. To bridge this gap, we propose a unified and flexible framework for reconstructing coherent language from brain recordings elicited by diverse input modalities-visual, auditory, and textual. Our approach leverages visual-language models (VLMs), using modality-specific experts to jointly interpret information across modalities. Experiments demonstrate that our method achieves performance comparable to state-of-the-art systems while remaining adaptable and extensible. This work advances toward more ecologically valid and generalizable mind decoding.
• 摘要：大脑活动的解码思想为人类认知提供了宝贵的见解，并在脑部计算中实现了有希望的应用。虽然先前的研究探讨了fMRI数据的语言重建，但它们通常仅限于单模式输入，例如图像或音频。相反，人类的思想本质上是多模式的。为了弥合这一差距，我们提出了一个统一，灵活的框架，用于从各种输入模态，视觉，听觉和文本中从大脑记录中重建连贯的语言。我们的方法利用特定于模式的专家来利用视觉语言模型（VLM）来共同解释跨模态的信息。实验表明，我们的方法达到的性能与最先进的系统相当，同时保持适应性和可扩展性。这项工作朝着更加生态有效和可概括的思维解码前进。

Title: Multi-domain Multilingual Sentiment Analysis in Industry: Predicting Aspect-based Opinion Quadruples

• Authors: Benjamin White, Anastasia Shimorina
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10389
• Pdf URL: https://arxiv.org/pdf/2505.10389
• Copy Paste: [[2505.10389]] Multi-domain Multilingual Sentiment Analysis in Industry: Predicting Aspect-based Opinion Quadruples(https://arxiv.org/abs/2505.10389)
• Keywords: language model, llm
• Abstract: This paper explores the design of an aspect-based sentiment analysis system using large language models (LLMs) for real-world use. We focus on quadruple opinion extraction -- identifying aspect categories, sentiment polarity, targets, and opinion expressions from text data across different domains and languages. Using internal datasets, we investigate whether a single fine-tuned model can effectively handle multiple domain-specific taxonomies simultaneously. We demonstrate that a combined multi-domain model achieves performance comparable to specialized single-domain models while reducing operational complexity. We also share lessons learned for handling non-extractive predictions and evaluating various failure modes when developing LLM-based systems for structured prediction tasks.
• 摘要：本文使用大型语言模型（LLM）探讨了基于方面的情感分析系统的设计，以实现现实世界中的使用。我们专注于四人意见提取 - 识别来自不同领域和语言的文本数据的方面类别，情感极性，目标和意见表达式。使用内部数据集，我们研究了单个微型模型是否可以同时有效处理多个域特异性分类法。我们证明，组合的多域模型可实现与专业单域模型相当的性能，同时降低操作复杂性。我们还共享学习的经验教训，用于处理非提取性预测并评估用于结构化预测任务的基于LLM的系统时。

Title: Rethinking Repetition Problems of LLMs in Code Generation

• Authors: Yihong Dong, Yuchen Liu, Xue Jiang, Zhi Jin, Ge Li
• Subjects: cs.CL, cs.AI, cs.LG, cs.SE
• Abstract URL: https://arxiv.org/abs/2505.10402
• Pdf URL: https://arxiv.org/pdf/2505.10402
• Copy Paste: [[2505.10402]] Rethinking Repetition Problems of LLMs in Code Generation(https://arxiv.org/abs/2505.10402)
• Keywords: language model, llm
• Abstract: With the advent of neural language models, the performance of code generation has been significantly boosted. However, the problem of repetitions during the generation process continues to linger. Previous work has primarily focused on content repetition, which is merely a fraction of the broader repetition problem in code generation. A more prevalent and challenging problem is structural repetition. In structural repetition, the repeated code appears in various patterns but possesses a fixed structure, which can be inherently reflected in grammar. In this paper, we formally define structural repetition and propose an efficient decoding approach called RPG, which stands for Repetition Penalization based on Grammar, to alleviate the repetition problems in code generation for LLMs. Specifically, RPG first leverages grammar rules to identify repetition problems during code generation, and then strategically decays the likelihood of critical tokens that contribute to repetitions, thereby mitigating them in code generation. To facilitate this study, we construct a new dataset CodeRepetEval to comprehensively evaluate approaches for mitigating the repetition problems in code generation. Extensive experimental results demonstrate that RPG substantially outperforms the best-performing baselines on CodeRepetEval dataset as well as HumanEval and MBPP benchmarks, effectively reducing repetitions and enhancing the quality of generated code.
• 摘要：随着神经语言模型的出现，代码生成的性能得到了显着提高。但是，生成过程中的重复问题继续持续。以前的工作主要集中在内容重复上，这仅仅是代码生成中更广泛重复问题的一部分。一个更普遍和具有挑战性的问题是结构重复。在结构重复中，重复的代码以各种模式出现，但具有固定的结构，可以固有地反映在语法中。在本文中，我们正式定义了结构重复，并提出了一种称为RPG的有效解码方法，该方法代表基于语法的重复惩罚，以减轻LLMS代码生成中的重复问题。具体而言，RPG首先利用语法规则来确定代码生成期间的重复问题，然后从战略上衰减有助于重复的关键令牌的可能性，从而在代码生成中减轻它们。为了促进这项研究，我们构建了一个新的数据集CoderePeteVal，以全面评估减轻代码生成中重复问题的方法。广泛的实验结果表明，RPG基本上优于CodeerePeteval数据集以及HumaneVal和MBPP基准的表现最佳的基准，从而有效地降低了重复的重复并增强了生成代码的质量。

Title: Are LLM-generated plain language summaries truly understandable? A large-scale crowdsourced evaluation

• Authors: Yue Guo, Jae Ho Sohn, Gondy Leroy, Trevor Cohen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10409
• Pdf URL: https://arxiv.org/pdf/2505.10409
• Copy Paste: [[2505.10409]] Are LLM-generated plain language summaries truly understandable? A large-scale crowdsourced evaluation(https://arxiv.org/abs/2505.10409)
• Keywords: language model, llm
• Abstract: Plain language summaries (PLSs) are essential for facilitating effective communication between clinicians and patients by making complex medical information easier for laypeople to understand and act upon. Large language models (LLMs) have recently shown promise in automating PLS generation, but their effectiveness in supporting health information comprehension remains unclear. Prior evaluations have generally relied on automated scores that do not measure understandability directly, or subjective Likert-scale ratings from convenience samples with limited generalizability. To address these gaps, we conducted a large-scale crowdsourced evaluation of LLM-generated PLSs using Amazon Mechanical Turk with 150 participants. We assessed PLS quality through subjective Likert-scale ratings focusing on simplicity, informativeness, coherence, and faithfulness; and objective multiple-choice comprehension and recall measures of reader understanding. Additionally, we examined the alignment between 10 automated evaluation metrics and human judgments. Our findings indicate that while LLMs can generate PLSs that appear indistinguishable from human-written ones in subjective evaluations, human-written PLSs lead to significantly better comprehension. Furthermore, automated evaluation metrics fail to reflect human judgment, calling into question their suitability for evaluating PLSs. This is the first study to systematically evaluate LLM-generated PLSs based on both reader preferences and comprehension outcomes. Our findings highlight the need for evaluation frameworks that move beyond surface-level quality and for generation methods that explicitly optimize for layperson comprehension.
• 摘要：普通语言摘要（PLS）对于通过使外行人更容易理解和采取行动来促进临床医生和患者之间的有效沟通至关重要。大型语言模型（LLMS）最近显示了自动化PLS生成的希望，但它们在支持健康信息理解方面的有效性尚不清楚。事先评估通常依赖于无法直接衡量可理解性的自动分数，或者是从具有有限概括性的便利样本中的主观李克特级评分。为了解决这些差距，我们使用Amazon Mechanical Turk对LLM生成的PLS进行了大规模的众包评估，其中有150名参与者。我们通过主观的李克特级评级评估了PLS质量，重点是简单，信息性，连贯性和忠诚；以及客观的多项选择理解和召回读者理解的度量。此外，我们研究了10个自动评估指标与人类判断之间的一致性。我们的发现表明，虽然LLM可以产生与主观评估中的人写的PLS，但人写的PLS会导致明显更好的理解。此外，自动化评估指标无法反映人类的判断力，因此质疑其对评估PLS的适用性。这是基于读者的偏好和理解结果系统评估LLM生成的PLS的首次研究。我们的发现突出了对超越表面质量和明确优化外行理解的生成方法的评估框架的需求。

Title: Hierarchical Document Refinement for Long-context Retrieval-augmented Generation

• Authors: Jiajie Jin, Xiaoxi Li, Guanting Dong, Yuyao Zhang, Yutao Zhu, Yongkang Wu, Zhonghua Li, Qi Ye, Zhicheng Dou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10413
• Pdf URL: https://arxiv.org/pdf/2505.10413
• Copy Paste: [[2505.10413]] Hierarchical Document Refinement for Long-context Retrieval-augmented Generation(https://arxiv.org/abs/2505.10413)
• Keywords: retrieval-augmented generation
• Abstract: Real-world RAG applications often encounter long-context input scenarios, where redundant information and noise results in higher inference costs and reduced performance. To address these challenges, we propose LongRefiner, an efficient plug-and-play refiner that leverages the inherent structural characteristics of long documents. LongRefiner employs dual-level query analysis, hierarchical document structuring, and adaptive refinement through multi-task learning on a single foundation model. Experiments on seven QA datasets demonstrate that LongRefiner achieves competitive performance in various scenarios while using 10x fewer computational costs and latency compared to the best baseline. Further analysis validates that LongRefiner is scalable, efficient, and effective, providing practical insights for real-world long-text RAG applications. Our code is available at this https URL.
• 摘要：现实世界中的破布应用程序经常遇到长篇文化输入方案，其中冗余信息和噪声会导致推理成本较高和降低性能。为了应对这些挑战，我们提出了Longrefiner，这是一种有效的插件炼油厂，利用长文档的固有结构特征。 Longrefiner采用双层查询分析，分层文档结构以及通过在单个基础模型上进行多任务学习的自适应改进。七个QA数据集的实验表明，与最佳基线相比，Longrefiner在各种情况下实现了竞争性能，而计算成本和延迟较小。进一步的分析验证了Longrefiner是可扩展，高效且有效的，为现实世界中的长篇文本抹布应用提供了实用的见解。我们的代码可在此HTTPS URL上找到。

Title: Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models

• Authors: Zemin Huang, Zhiyang Chen, Zijun Wang, Tiancheng Li, Guo-Jun Qi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10446
• Pdf URL: https://arxiv.org/pdf/2505.10446
• Copy Paste: [[2505.10446]] Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models(https://arxiv.org/abs/2505.10446)
• Keywords: language model, chain-of-thought
• Abstract: We introduce the \emph{Diffusion Chain of Lateral Thought (DCoLT)}, a reasoning framework for diffusion language models. DCoLT treats each intermediate step in the reverse diffusion process as a latent "thinking" action and optimizes the entire reasoning trajectory to maximize the reward on the correctness of the final answer with outcome-based Reinforcement Learning (RL). Unlike traditional Chain-of-Thought (CoT) methods that follow a causal, linear thinking process, DCoLT allows bidirectional, non-linear reasoning with no strict rule on grammatical correctness amid its intermediate steps of thought. We implement DCoLT on two representative Diffusion Language Models (DLMs). First, we choose SEDD as a representative continuous-time discrete diffusion model, where its concrete score derives a probabilistic policy to maximize the RL reward over the entire sequence of intermediate diffusion steps. We further consider the discrete-time masked diffusion language model -- LLaDA, and find that the order to predict and unmask tokens plays an essential role to optimize its RL action resulting from the ranking-based Unmasking Policy Module (UPM) defined by the Plackett-Luce model. Experiments on both math and code generation tasks show that using only public data and 16 H800 GPUs, DCoLT-reinforced DLMs outperform other DLMs trained by SFT or RL or even both. Notably, DCoLT-reinforced LLaDA boosts its reasoning accuracy by +9.8%, +5.7%, +11.4%, +19.5% on GSM8K, MATH, MBPP, and HumanEval.
• 摘要：我们介绍了\ emph {扩散链（dcolt）}，这是扩散语言模型的推理框架。 DCOLT将反向扩散过程中的每个中间步骤视为潜在的“思考”动作，并优化了整个推理轨迹，以最大程度地利用基于结果的增强学习（RL）来最大程度地获得最终答案的奖励。与遵循因果关系线性思维过程的传统思想链（COT）方法不同，DCOLT允许双向，非线性推理，而在其中间思想的中间步骤中没有严格的语法正确性规则。我们在两个代表性扩散语言模型（DLM）上实施DCOLT。首先，我们选择SEDD作为代表性连续离散扩散模型，其具体分数得出了概率策略，以最大程度地利用中间扩散步骤的整个序列RL奖励。我们进一步考虑了离散的时间掩盖扩散语言模型-LLADA，并发现预测和揭示令牌的顺序起着至关重要的作用，可以优化由Plackett-luce模型定义的基于排名的基于排名的Unmasking Policy Module（UPM）产生的RL操作。关于数学和代码生成任务的实验表明，仅使用公共数据和16个H800 GPU，DCOLT-REINFORCED DLMS优于SFT或RL培训的其他DLM，甚至两者兼而有之。值得注意的是，DColt-Reinforced Llada将其推理精度提高了9.8％， +5.7％， +11.4％，GSM8K，MATH，MATH，MBPP和HUMANEVAL的推理精度。

Title: CL-RAG: Bridging the Gap in Retrieval-Augmented Generation with Curriculum Learning

• Authors: Shaohan Wang, Licheng Zhang, Zheren Fu, Zhendong Mao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10493
• Pdf URL: https://arxiv.org/pdf/2505.10493
• Copy Paste: [[2505.10493]] CL-RAG: Bridging the Gap in Retrieval-Augmented Generation with Curriculum Learning(https://arxiv.org/abs/2505.10493)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) is an effective method to enhance the capabilities of large language models (LLMs). Existing methods focus on optimizing the retriever or generator in the RAG system by directly utilizing the top-k retrieved documents. However, the documents effectiveness are various significantly across user queries, i.e. some documents provide valuable knowledge while others totally lack critical information. It hinders the retriever and generator's adaptation during training. Inspired by human cognitive learning, curriculum learning trains models using samples progressing from easy to difficult, thus enhancing their generalization ability, and we integrate this effective paradigm to the training of the RAG system. In this paper, we propose a multi-stage Curriculum Learning based RAG system training framework, named CL-RAG. We first construct training data with multiple difficulty levels for the retriever and generator separately through sample evolution. Then, we train the model in stages based on the curriculum learning approach, thereby optimizing the overall performance and generalization of the RAG system more effectively. Our CL-RAG framework demonstrates consistent effectiveness across four open-domain QA datasets, achieving performance gains of 2% to 4% over multiple advanced methods.
• 摘要：检索增强的生成（RAG）是增强大语言模型（LLMS）功能的有效方法。现有的方法致力于通过直接利用TOP-K检索文档来优化抹布系统中的检索器或发电机。但是，文档的有效性在用户查询之间具有很大的不同，即，某些文档提供了宝贵的知识，而另一些文档则完全缺乏关键信息。它阻碍了培训期间的回收犬和发电机的适应。受到人类认知学习的启发，课程学习训练模型，使用样本从易于的到困难，从而增强了它们的概括能力，我们将这种有效的范式整合到了抹布系统的训练中。在本文中，我们提出了一个基于多阶段课程学习的抹布系统培训框架，名为Cl-rag。我们首先通过样品演化分别构建了猎犬和发电机的多个难度水平的培训数据。然后，我们根据课程学习方法分阶段训练模型，从而更有效地优化了抹布系统的整体性能和概括。我们的CL-RAG框架在四个开放域QA数据集中表现出一致的有效性，在多种高级方法上，绩效提高了2％至4％。

Title: Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security Perspective

• Authors: Yutao Mou, Xiao Deng, Yuxiao Luo, Shikun Zhang, Wei Ye
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10494
• Pdf URL: https://arxiv.org/pdf/2505.10494
• Copy Paste: [[2505.10494]] Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security Perspective(https://arxiv.org/abs/2505.10494)
• Keywords: language model, llm
• Abstract: Code security and usability are both essential for various coding assistant applications driven by large language models (LLMs). Current code security benchmarks focus solely on single evaluation task and paradigm, such as code completion and generation, lacking comprehensive assessment across dimensions like secure code generation, vulnerability repair and discrimination. In this paper, we first propose CoV-Eval, a multi-task benchmark covering various tasks such as code completion, vulnerability repair, vulnerability detection and classification, for comprehensive evaluation of LLM code security. Besides, we developed VC-Judge, an improved judgment model that aligns closely with human experts and can review LLM-generated programs for vulnerabilities in a more efficient and reliable way. We conduct a comprehensive evaluation of 20 proprietary and open-source LLMs. Overall, while most LLMs identify vulnerable codes well, they still tend to generate insecure codes and struggle with recognizing specific vulnerability types and performing repairs. Extensive experiments and qualitative analyses reveal key challenges and optimization directions, offering insights for future research in LLM code security.
• 摘要：代码安全性和可用性对于由大语言模型（LLMS）驱动的各种编码助理应用程序至关重要。当前的代码安全基准仅着眼于单个评估任务和范式，例如代码完成和生成，缺乏跨维度的全面评估，例如安全代码生成，脆弱性修复和歧视。在本文中，我们首先提出了CoV-eval，这是一个多任务基准，涵盖了各种任务，例如代码完成，漏洞维修，漏洞检测和分类，以全面评估LLM代码安全性。此外，我们开发了VC-Gudge，这是一种改进的判断模型，与人类专家紧密相符，并可以以更有效和可靠的方式来审查LLM生成的脆弱性计划。我们对20个专有和开源LLM进行了全面评估。总体而言，尽管大多数LLM都很好地识别了脆弱的代码，但它们仍然倾向于生成不安全的代码，并努力识别特定的漏洞类型和进行维修。广泛的实验和定性分析揭示了关键的挑战和优化方向，为LLM Code Security提供的未来研究提供了见解。

Title: Multi-Token Prediction Needs Registers

• Authors: Anastasios Gerontopoulos, Spyros Gidaris, Nikos Komodakis
• Subjects: cs.CL, cs.AI, cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.10518
• Pdf URL: https://arxiv.org/pdf/2505.10518
• Copy Paste: [[2505.10518]] Multi-Token Prediction Needs Registers(https://arxiv.org/abs/2505.10518)
• Keywords: language model
• Abstract: Multi-token prediction has emerged as a promising objective for improving language model pretraining, but its benefits have not consistently generalized to other settings such as fine-tuning. In this paper, we propose MuToR, a simple and effective approach to multi-token prediction that interleaves learnable register tokens into the input sequence, each tasked with predicting future targets. Compared to existing methods, MuToR offers several key advantages: it introduces only a negligible number of additional parameters, requires no architectural changes--ensuring compatibility with off-the-shelf pretrained language models--and remains aligned with the next-token pretraining objective, making it especially well-suited for supervised fine-tuning. Moreover, it naturally supports scalable prediction horizons. We demonstrate the effectiveness and versatility of MuToR across a range of use cases, including supervised fine-tuning, parameter-efficient fine-tuning (PEFT), and pretraining, on challenging generative tasks in both language and vision domains. Our code will be available at: this https URL.
• 摘要：多句话预测已成为改善语言模型预处理的有希望的目标，但其优点并未始终如一地推广到其他设置（例如微调）。在本文中，我们提出了一种简单有效的多言论预测方法，将可学习的寄存器令牌交织到输入序列中，每个人都负责预测未来的目标。与现有方法相比，MUTOR提供了几个关键优势：它仅引入了可忽略的附加参数，不需要建筑变化 - 与现成的预审前的语言模型保持兼容性 - 并且仍然与下一句话的预审前的目标保持一致，这使得它特别适合用于监督的好调子。此外，它自然支持可扩展的预测范围。我们证明了MUTOR在一系列用例中的有效性和多功能性，包括受监督的微调，参数有效的微调（PEFT）以及对语言和视力域中具有挑战性的生成任务进行挑战的生成任务。我们的代码将提供：此HTTPS URL。

Title: WorldPM: Scaling Human Preference Modeling

• Authors: Binghai Wang, Runji Lin, Keming Lu, Le Yu, Zhenru Zhang, Fei Huang, Chujie Zheng, Kai Dang, Yang Fan, Xingzhang Ren, An Yang, Binyuan Hui, Dayiheng Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Bowen Yu, Jingren Zhou, Junyang Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10527
• Pdf URL: https://arxiv.org/pdf/2505.10527
• Copy Paste: [[2505.10527]] WorldPM: Scaling Human Preference Modeling(https://arxiv.org/abs/2505.10527)
• Keywords: language model
• Abstract: Motivated by scaling laws in language modeling that demonstrate how test loss scales as a power law with model and dataset sizes, we find that similar laws exist in preference modeling. We propose World Preference Modeling$ (WorldPM) to emphasize this scaling potential, where World Preference embodies a unified representation of human preferences. In this paper, we collect preference data from public forums covering diverse user communities, and conduct extensive training using 15M-scale data across models ranging from 1.5B to 72B parameters. We observe distinct patterns across different evaluation metrics: (1) Adversarial metrics (ability to identify deceptive features) consistently scale up with increased training data and base model size; (2) Objective metrics (objective knowledge with well-defined answers) show emergent behavior in larger language models, highlighting WorldPM's scalability potential; (3) Subjective metrics (subjective preferences from a limited number of humans or AI) do not demonstrate scaling trends. Further experiments validate the effectiveness of WorldPM as a foundation for preference fine-tuning. Through evaluations on 7 benchmarks with 20 subtasks, we find that WorldPM broadly improves the generalization performance across human preference datasets of varying sizes (7K, 100K and 800K samples), with performance gains exceeding 5% on many key subtasks. Integrating WorldPM into our internal RLHF pipeline, we observe significant improvements on both in-house and public evaluation sets, with notable gains of 4% to 8% in our in-house evaluations.
• 摘要：通过在语言建模中缩放定律的促进，这些定律表明测试损失如何用模型和数据集大小作为幂定律，我们发现优先型建模中存在类似的法律。我们建议世界偏好建模$（Worldpm）强调这种扩展潜力，在这种潜力中，世界偏好体现了人类偏好的统一表示。在本文中，我们从涵盖各种用户社区的公共论坛中收集偏好数据，并在范围从1.5B到72B参数的模型中使用15m规模的数据进行广泛的培训。我们在不同的评估指标上观察到不同的模式：（1）对抗性指标（识别欺骗性特征的能力）始终如一地扩大训练数据和基本模型大小； （2）客观指标（具有明确答案的客观知识）在较大的语言模型中显示出紧急行为，突出了WorldPM的可伸缩性潜力； （3）主观指标（来自有限数量的人类或AI的主观偏好）未显示缩放趋势。进一步的实验验证了WorldPM作为偏好微调的基础的有效性。通过对具有20个子任务的7个基准测试的评估，我们发现WorldPM广泛提高了各种大小（7K，100K和800K样品）的人类偏好数据集的概括性能，并且在许多关键子任务中的性能增长超过5％。将WorldPM整合到我们的内部RLHF管道中，我们观察到内部和公共评估集的显着改善，在我们的内部评估中显着增长了4％至8％。

Title: Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models

• Authors: Zhiyuan Hu, Yibo Wang, Hanze Dong, Yuhui Xu, Amrita Saha, Caiming Xiong, Bryan Hooi, Junnan Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10554
• Pdf URL: https://arxiv.org/pdf/2505.10554
• Copy Paste: [[2505.10554]] Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models(https://arxiv.org/abs/2505.10554)
• Keywords: prompt, chain-of-thought
• Abstract: Large reasoning models (LRMs) already possess a latent capacity for long chain-of-thought reasoning. Prior work has shown that outcome-based reinforcement learning (RL) can incidentally elicit advanced reasoning behaviors such as self-correction, backtracking, and verification phenomena often referred to as the model's "aha moment". However, the timing and consistency of these emergent behaviors remain unpredictable and uncontrollable, limiting the scalability and reliability of LRMs' reasoning capabilities. To address these limitations, we move beyond reliance on prompts and coincidental "aha moments". Instead, we explicitly align models with three meta-abilities: deduction, induction, and abduction, using automatically generated, self-verifiable tasks. Our three stage-pipeline individual alignment, parameter-space merging, and domain-specific reinforcement learning, boosting performance by over 10\% relative to instruction-tuned baselines. Furthermore, domain-specific RL from the aligned checkpoint yields an additional 2\% average gain in the performance ceiling across math, coding, and science benchmarks, demonstrating that explicit meta-ability alignment offers a scalable and dependable foundation for reasoning. Code is available at: this https URL
• 摘要：大型推理模型（LRMS）已经具有长期思想推理的潜在能力。先前的工作表明，基于结果的增强学习（RL）可以偶然引起先进的推理行为，例如自我纠正，回溯和验证现象，通常称为模型的“ AHA时刻”。但是，这些紧急行为的时机和一致性仍然是不可预测和无法控制的，从而限制了LRMS推理能力的可扩展性和可靠性。为了解决这些限制，我们不仅可以依赖提示和偶然的“啊哈时刻”。取而代之的是，我们使用自动生成的，可自我验证的任务明确将模型与三个元能力相结合：扣除，归纳和绑架。我们的三阶段二线个人对齐，参数空间合并和域特异性增强学习，相对于指导调节的基线，将超过10 \％的性能提高了10 \％。此外，对齐检查点的域特异性RL在跨数学，编码和科学基准的性能上限的平均增益中获得了2 \％的平均增长，这表明显式的元能力对齐为推理提供了可扩展可靠的基础。代码可用：此HTTPS URL