2025-06-09

Title: LLMs Can Also Do Well! Breaking Barriers in Semantic Role Labeling via Large Language Models

• Authors: Xinxin Li, Huiyao Chen, Chengjun Liu, Jing Li, Meishan Zhang, Jun Yu, Min Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05385
• Pdf URL: https://arxiv.org/pdf/2506.05385
• Copy Paste: [[2506.05385]] LLMs Can Also Do Well! Breaking Barriers in Semantic Role Labeling via Large Language Models(https://arxiv.org/abs/2506.05385)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Semantic role labeling (SRL) is a crucial task of natural language processing (NLP). Although generative decoder-based large language models (LLMs) have achieved remarkable success across various NLP tasks, they still lag behind state-of-the-art encoder-decoder (BERT-like) models in SRL. In this work, we seek to bridge this gap by equipping LLMs for SRL with two mechanisms: (a) retrieval-augmented generation and (b) self-correction. The first mechanism enables LLMs to leverage external linguistic knowledge such as predicate and argument structure descriptions, while the second allows LLMs to identify and correct inconsistent SRL outputs. We conduct extensive experiments on three widely-used benchmarks of SRL (CPB1.0, CoNLL-2009, and CoNLL-2012). Results demonstrate that our method achieves state-of-the-art performance in both Chinese and English, marking the first successful application of LLMs to surpass encoder-decoder approaches in SRL.
• 摘要：语义角色标签（SRL）是自然语言处理（NLP）的关键任务。尽管基于生成解码器的大型语言模型（LLM）在各种NLP任务中取得了巨大的成功，但它们仍然落后于SRL中最先进的编码器（类似于BERT）模型。在这项工作中，我们试图通过为SRL配备LLM，并使用两种机制来弥合这一差距：（a）检索增强的一代和（b）自校正。第一种机制使LLM能够利用外部语言知识，例如谓词和参数结构描述，而第二个则允许LLMS识别和纠正不一致的SRL输出。我们对SRL的三个广泛使用基准（CPB1.0，Conll-2009和Conll-2012）进行了广泛的实验。结果表明，我们的方法在中文和英语中都达到了最先进的性能，这标志着LLMS在SRL中首次成功应用了Encoder-Decoder方法。

Title: Beyond RAG: Reinforced Reasoning Augmented Generation for Clinical Notes

• Authors: Lo Pang-Yun Ting, Chengshuai Zhao, Yu-Hua Zeng, Yuan Jee Lim, Kun-Ta Chuang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05386
• Pdf URL: https://arxiv.org/pdf/2506.05386
• Copy Paste: [[2506.05386]] Beyond RAG: Reinforced Reasoning Augmented Generation for Clinical Notes(https://arxiv.org/abs/2506.05386)
• Keywords: language model, llm
• Abstract: Clinical note generation aims to automatically produce free-text summaries of a patient's condition and diagnostic process, with discharge instructions being a representative long-form example. While recent large language model (LLM)-based methods pre-trained on general clinical corpora show promise in clinical text generation, they fall short in producing long-form notes from limited patient information. In this paper, we propose R2AG, the first reinforced retriever for long-form discharge instruction generation based on pre-admission data. R2AG is trained with reinforcement learning to retrieve reasoning paths from a medical knowledge graph, providing explicit semantic guidance to the LLM. To bridge the information gap, we propose Group-Based Retriever Optimization (GRO) which improves retrieval quality with group-relative rewards, encouraging reasoning leaps for deeper inference by the LLM. Comprehensive experiments on the MIMIC-IV-Note dataset show that R2AG outperforms baselines in both clinical efficacy and natural language generation metrics. Further analysis reveals that R2AG fills semantic gaps in sparse input scenarios, and retrieved reasoning paths help LLMs avoid clinical misinterpretation by focusing on key evidence and following coherent reasoning.
• 摘要：临床注释的产生旨在自动产生患者状况和诊断过程的自由文本摘要，放电指令是一个代表性的长期示例。虽然最近在一般临床语料库中预先培训的基于基于的大型语言模型（LLM）在临床文本生成方面有希望，但它们在有限的患者信息中产生长形式的笔记中，它们却缺乏。在本文中，我们提出了R2AG，这是基于入学前数据的第一个增强型猎犬，用于长期出院指导生成。 R2AG接受了强化学习的培训，可以从医学知识图中检索推理路径，从而为LLM提供了明确的语义指导。为了弥合信息差距，我们提出了基于群体的检索器优化（GRO），从而通过群体相关的奖励提高了检索质量，从而鼓励推理LLM更深入地推断。对模拟物IV计数数据集的综合实验表明，R2AG在临床功效和自然语言生成指标中的表现都优于基准。进一步的分析表明，R2AG填补了稀疏输入方案中的语义空白，并且检索推理路径有助于LLMS通过关注关键证据并遵循相干推理来避免临床误解。

Title: Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs

• Authors: Jaydip Sen, Saptarshi Sengupta. Subhasis Dasgupta
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05387
• Pdf URL: https://arxiv.org/pdf/2506.05387
• Copy Paste: [[2506.05387]] Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs(https://arxiv.org/abs/2506.05387)
• Keywords: language model, llm
• Abstract: This chapter explores advancements in decoding strategies for large language models (LLMs), focusing on enhancing the Locally Typical Sampling (LTS) algorithm. Traditional decoding methods, such as top-k and nucleus sampling, often struggle to balance fluency, diversity, and coherence in text generation. To address these challenges, Adaptive Semantic-Aware Typicality Sampling (ASTS) is proposed as an improved version of LTS, incorporating dynamic entropy thresholding, multi-objective scoring, and reward-penalty adjustments. ASTS ensures contextually coherent and diverse text generation while maintaining computational efficiency. Its performance is evaluated across multiple benchmarks, including story generation and abstractive summarization, using metrics such as perplexity, MAUVE, and diversity scores. Experimental results demonstrate that ASTS outperforms existing sampling techniques by reducing repetition, enhancing semantic alignment, and improving fluency.
• 摘要：本章探讨了大型语言模型（LLM）解码策略的进步，重点是增强本地典型的采样（LTS）算法。传统的解码方法，例如TOP-K和Nucleus采样，通常在文本生成中努力平衡流利性，多样性和连贯性。为了应对这些挑战，提出了自适应语义意识典型性采样（ASTS）作为LTS的改进版本，并结合了动态熵阈值，多目标评分和奖励 - 预选调整。 ASTS确保上下文相干和多样化的文本生成，同时保持计算效率。使用诸如困惑，淡紫色和多样性得分等指标，包括故事产生和抽象性摘要，包括故事产生和抽象性摘要，评估其性能。实验结果表明，AST通过减少重复，增强语义比对和提高流利度来优于现有的抽样技术。

Title: Understanding Gender Bias in AI-Generated Product Descriptions

• Authors: Markelle Kelly, Mohammad Tahaei, Padhraic Smyth, Lauren Wilcox
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.05390
• Pdf URL: https://arxiv.org/pdf/2506.05390
• Copy Paste: [[2506.05390]] Understanding Gender Bias in AI-Generated Product Descriptions(https://arxiv.org/abs/2506.05390)
• Keywords: language model, gpt, llm
• Abstract: While gender bias in large language models (LLMs) has been extensively studied in many domains, uses of LLMs in e-commerce remain largely unexamined and may reveal novel forms of algorithmic bias and harm. Our work investigates this space, developing data-driven taxonomic categories of gender bias in the context of product description generation, which we situate with respect to existing general purpose harms taxonomies. We illustrate how AI-generated product descriptions can uniquely surface gender biases in ways that require specialized detection and mitigation approaches. Further, we quantitatively analyze issues corresponding to our taxonomic categories in two models used for this task -- GPT-3.5 and an e-commerce-specific LLM -- demonstrating that these forms of bias commonly occur in practice. Our results illuminate unique, under-explored dimensions of gender bias, such as assumptions about clothing size, stereotypical bias in which features of a product are advertised, and differences in the use of persuasive language. These insights contribute to our understanding of three types of AI harms identified by current frameworks: exclusionary norms, stereotyping, and performance disparities, particularly for the context of e-commerce.
• 摘要：尽管大型语言模型中的性别偏见（LLM）在许多领域进行了广泛的研究，但在电子商务中使用LLM的用途在很大程度上尚未审查，并且可能揭示了新型算法偏见和伤害的新型形式。我们的工作调查了这个空间，在产品描述生成的背景下开发了数据驱动的性别偏见的分类类别，我们就现有的通用损害了分类法。我们说明了AI生成的产品描述如何以需要专门检测和缓解方法的方式唯一地表面性别偏见。此外，我们定量分析与我们的分类类别相对应的问题，用于该任务的两个模型 - GPT-3.5和电子商务特定的LLM，表明这些偏见通常在实践中发生。我们的结果阐明了性别偏见的独特，探索不足的维度，例如关于服装尺寸的假设，刻板印象的偏见，其中广告着产品的特征以及使用说服力的语言的差异。这些见解有助于我们对当前框架确定的三种AI危害的理解：排除规范，刻板印象和绩效差异，尤其是对于电子商务的背景。

Title: Are Large Language Models Good Temporal Graph Learners?

• Authors: Shenyang Huang, Ali Parviz, Emma Kondrup, Zachary Yang, Zifeng Ding, Michael Bronstein, Reihaneh Rabbany, Guillaume Rabusseau
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.05393
• Pdf URL: https://arxiv.org/pdf/2506.05393
• Copy Paste: [[2506.05393]] Are Large Language Models Good Temporal Graph Learners?(https://arxiv.org/abs/2506.05393)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have recently driven significant advancements in Natural Language Processing and various other applications. While a broad range of literature has explored the graph-reasoning capabilities of LLMs, including their use of predictors on graphs, the application of LLMs to dynamic graphs -- real world evolving networks -- remains relatively unexplored. Recent work studies synthetic temporal graphs generated by random graph models, but applying LLMs to real-world temporal graphs remains an open question. To address this gap, we introduce Temporal Graph Talker (TGTalker), a novel temporal graph learning framework designed for LLMs. TGTalker utilizes the recency bias in temporal graphs to extract relevant structural information, converted to natural language for LLMs, while leveraging temporal neighbors as additional information for prediction. TGTalker demonstrates competitive link prediction capabilities compared to existing Temporal Graph Neural Network (TGNN) models. Across five real-world networks, TGTalker performs competitively with state-of-the-art temporal graph methods while consistently outperforming popular models such as TGN and HTGN. Furthermore, TGTalker generates textual explanations for each prediction, thus opening up exciting new directions in explainability and interpretability for temporal link prediction. The code is publicly available at this https URL.
• 摘要：大型语言模型（LLM）最近在自然语言处理和其他各种应用方面取得了重大进步。尽管广泛的文献探索了LLM的图形功能，包括它们在图表上使用预测变量，但LLM在动态图中的应用 - 现实世界不断发展的网络 - 仍然相对尚未探索。最近的工作研究合成由随机图模型生成的综合时间图，但是将LLMS应用于现实世界的时间表仍然是一个空旷的问题。为了解决这一差距，我们介绍了暂时的图形谈话者（TGTALKER），这是一个新型的时间图形学习框架，专为LLMS设计。 TGTALKER利用时间图中的重新度偏差来提取相关的结构信息，转换为LLMS的自然语言，同时利用时间邻居作为其他信息进行预测。与现有的时间图神经网络（TGNN）模型相比，TGTALKER展示了竞争性链接预测能力。在五个现实世界的网络中，TGTalker使用最先进的时间图方法竞争性能，同时始终超过了TGN和HTGN等流行模型。此外，TGTalker为每个预测生成文本解释，从而为时间链接预测打开令人兴奋的解释性和可解释性方向。该代码在此HTTPS URL上公开可用。

Title: Auto Review: Second Stage Error Detection for Highly Accurate Information Extraction from Phone Conversations

• Authors: Ayesha Qamar, Arushi Raghuvanshi, Conal Sathi, Youngseo Son
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05400
• Pdf URL: https://arxiv.org/pdf/2506.05400
• Copy Paste: [[2506.05400]] Auto Review: Second Stage Error Detection for Highly Accurate Information Extraction from Phone Conversations(https://arxiv.org/abs/2506.05400)
• Keywords: language model
• Abstract: Automating benefit verification phone calls saves time in healthcare and helps patients receive treatment faster. It is critical to obtain highly accurate information in these phone calls, as it can affect a patient's healthcare journey. Given the noise in phone call transcripts, we have a two-stage system that involves a post-call review phase for potentially noisy fields, where human reviewers manually verify the extracted data$\unicode{x2013}$a labor-intensive task. To automate this stage, we introduce Auto Review, which significantly reduces manual effort while maintaining a high bar for accuracy. This system, being highly reliant on call transcripts, suffers a performance bottleneck due to automatic speech recognition (ASR) issues. This problem is further exacerbated by the use of domain-specific jargon in the calls. In this work, we propose a second-stage postprocessing pipeline for accurate information extraction. We improve accuracy by using multiple ASR alternatives and a pseudo-labeling approach that does not require manually corrected transcripts. Experiments with general-purpose large language models and feature-based model pipelines demonstrate substantial improvements in the quality of corrected call transcripts, thereby enhancing the efficiency of Auto Review.
• 摘要：自动化福利验证电话可以节省医疗保健时间，并帮助患者更快地接受治疗。在这些电话中获得高度准确的信息至关重要，因为它可能会影响患者的医疗保健旅程。鉴于电话呼叫成绩单中的噪音，我们有一个两阶段的系统，该系统涉及可能嘈杂领域的通话后审核阶段，其中人类审阅者手动验证了提取的数据$ \ unicode {x2013} $一项劳动密集型任务。为了自动化此阶段，我们介绍自动审查，这大大减少了手动努力，同时保持高标准的准确性。由于自动语音识别（ASR）问题，该系统高度依赖呼叫成绩单，因此遭受了性能瓶颈。在呼叫中使用特定于域的行话进一步加剧了这个问题。在这项工作中，我们提出了一条第二阶段的后处理管道，以进行准确的信息提取。我们通过使用多种ASR替代方案和不需要手动校正成绩单的伪标记方法提高准确性。通用大语言模型和基于功能的模型管道的实验表明，校正后的呼叫转录本的质量有了显着改善，从而提高了自动审查的效率。

Title: Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs

• Authors: Wanyun Cui, Mingwei Xu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05410
• Pdf URL: https://arxiv.org/pdf/2506.05410
• Copy Paste: [[2506.05410]] Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs(https://arxiv.org/abs/2506.05410)
• Keywords: language model, llm
• Abstract: Recent advances in Large Language Models (LLMs) have highlighted the critical importance of extending context length, yet the quadratic complexity of attention mechanisms poses significant challenges for efficient long-context modeling. KV cache compression has emerged as a key approach to address this challenge. Through extensive empirical analysis, we reveal a fundamental yet previously overlooked asymmetry in KV caches: while adjacent keys receive similar attention weights (local homogeneity), adjacent values demonstrate distinct heterogeneous distributions. This key-value asymmetry reveals a critical limitation in existing compression methods that treat keys and values uniformly. To address the limitation, we propose a training-free compression framework (AsymKV) that combines homogeneity-based key merging with a mathematically proven lossless value compression. Extensive experiments demonstrate that AsymKV consistently outperforms existing long-context methods across various tasks and base models. For example, on LLaMA3.1-8B, AsymKV achieves an average score of 43.95 on LongBench, surpassing SOTA methods like H$_2$O (38.89) by a large margin.
• 摘要：大型语言模型（LLM）的最新进展强调了扩展上下文长度的重要性，但是注意机制的二次复杂性为有效的长篇小说建模带来了重大挑战。 KV缓存压缩已成为应对这一挑战的关键方法。通过广泛的经验分析，我们揭示了KV缓存中的基本但先前被忽略的不对称性：虽然相邻的键获得相似的注意力（局部同质性），但相邻的值表现出明显的异质分布。这种键值不对称揭示了现有的压缩方法的关键限制，该方法均匀处理键和值。为了解决限制，我们提出了一个无训练的压缩框架（ASYMKV），该框架将基于同质性的密钥合并与数学证明的无损价值压缩相结合。广泛的实验表明，Asymkv始终在各种任务和基本模型上均超过现有的长篇下说方法。例如，在Llama3.1-8B上，Asymkv在Longbench上的平均得分为43.95，超过了H $ _2 $ O（38.89）的SOTA方法，并以很大的利润率超过了SOTA。

Title: SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs

• Authors: Patrik Czakó, Gábor Kertész, Sándor Szénási
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.05413
• Pdf URL: https://arxiv.org/pdf/2506.05413
• Copy Paste: [[2506.05413]] SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs(https://arxiv.org/abs/2506.05413)
• Keywords: language model, llm
• Abstract: We present SmoothRot, a novel post-training quantization technique to enhance the efficiency of 4-bit quantization in Large Language Models (LLMs). SmoothRot addresses the critical challenge of massive activation outliers, by integrating channel-wise scaling with Hadamard transformations. Our technique effectively transforms extreme outliers into quantization-friendly activations, significantly improving quantization accuracy. Experiments conducted on popular LLMs (LLaMA2 7B, LLaMA3.1 8B, and Mistral 7B) demonstrate that SmoothRot consistently reduces the performance gap between quantized and FP16 models by approximately 10-30\% across language generation and zero-shot reasoning tasks, without introducing additional inference latency. Code is available at this https URL.
• 摘要：我们提出了SmoothRot，这是一种新型的训练后量化技术，可提高大语模型（LLMS）中4位量化的效率。 SmoothRot通过将渠道缩放与Hadamard转换整合在一起，以解决大规模激活异常值的关键挑战。我们的技术有效地将极端异常值转变为量化友好的激活，从而显着提高了量化精度。在流行LLMS（Llama2 7B，Llama3.1 8B和Mismtral 7b）上进行的实验表明，Smoothrot始终将量化和FP16模型之间的性能差距降低了大约10-30 \％的语言生成，而无需引入其他额外的讨论延迟。代码可在此HTTPS URL上找到。

Title: Automatically Detecting Amusing Games in Wordle

• Authors: Ronaldo Luo, Gary Liang, Cindy Liu, Adam Kabbara, Minahil Bakhtawar, Kina Kim, Michael Guerzhoy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05415
• Pdf URL: https://arxiv.org/pdf/2506.05415
• Copy Paste: [[2506.05415]] Automatically Detecting Amusing Games in Wordle(https://arxiv.org/abs/2506.05415)
• Keywords: gpt, prompt
• Abstract: We explore automatically predicting which Wordle games Reddit users find amusing. We scrape approximately 80k reactions by Reddit users to Wordle games from Reddit, classify the reactions as expressing amusement or not using OpenAI's GPT-3.5 using few-shot prompting, and verify that GPT-3.5's labels roughly correspond to human labels. We then extract features from Wordle games that can predict user amusement. We demonstrate that the features indeed provide a (weak) signal that predicts user amusement as predicted by GPT-3.5. Our results indicate that user amusement at Wordle games can be predicted computationally to some extent. We explore which features of the game contribute to user amusement. We find that user amusement is predictable, indicating a measurable aspect of creativity infused into Wordle games through humor.
• 摘要：我们探索自动预测Reddit用户觉得有趣的Wordle游戏。我们将Reddit用户从Reddit中刮擦了大约80k的反应，将反应分类为表达娱乐或不使用OpenAi的GPT-3.5使用很少的弹药提示，并验证GPT-3.5的标签与人类标签大致相对应。然后，我们从Wordle游戏中提取可以预测用户娱乐的功能。我们证明了该功能确实提供了一个（弱）信号，该信号可以预测GPT-3.5预测的用户娱乐。我们的结果表明，可以在某种程度上在计算上预测Wordle游戏中的用户娱乐。我们探索游戏的哪些功能有助于用户娱乐。我们发现用户娱乐是可以预测的，这表明通过幽默注入了Wordle游戏的创造力的可衡量方面。

Title: MLLM-CL: Continual Learning for Multimodal Large Language Models

• Authors: Hongbo Zhao, Fei Zhu, Rundong Wang, Gaofeng Meng, Zhaoxiang Zhang
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2506.05453
• Pdf URL: https://arxiv.org/pdf/2506.05453
• Copy Paste: [[2506.05453]] MLLM-CL: Continual Learning for Multimodal Large Language Models(https://arxiv.org/abs/2506.05453)
• Keywords: language model, llm
• Abstract: Recent Multimodal Large Language Models (MLLMs) excel in vision-language understanding but face challenges in adapting to dynamic real-world scenarios that require continuous integration of new knowledge and skills. While continual learning (CL) offers a potential solution, existing benchmarks and methods suffer from critical limitations. In this paper, we introduce MLLM-CL, a novel benchmark encompassing domain and ability continual learning, where the former focuses on independently and identically distributed (IID) evaluation across evolving mainstream domains, whereas the latter evaluates on non-IID scenarios with emerging model ability. Methodologically, we propose preventing catastrophic interference through parameter isolation, along with an MLLM-based routing mechanism. Extensive experiments demonstrate that our approach can integrate domain-specific knowledge and functional abilities with minimal forgetting, significantly outperforming existing methods.
• 摘要：最近的多模式大型语言模型（MLLM）在视力语言的理解中表现出色，但在适应动态现实世界中的挑战中，需要连续整合新知识和技能。虽然持续学习（CL）提供了潜在的解决方案，但现有的基准和方法受到关键限制。在本文中，我们介绍了MLLM-CL，这是一种新型的基准测试领域和能力持续学习，前者专注于在不断发展的主流领域进行独立和相同分布的（IID）评估，而后者对非IID场景进行评估，并具有新兴模型的能力。从方法论上讲，我们提出通过参数隔离以及基于MLLM的路由机制来防止灾难性干扰。广泛的实验表明，我们的方法可以将特定于领域的知识和功能能力与最小的遗忘相结合，从而大大优于现有方法。

Title: Improving LLMs with a knowledge from databases

• Authors: Petr Máša
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05560
• Pdf URL: https://arxiv.org/pdf/2506.05560
• Copy Paste: [[2506.05560]] Improving LLMs with a knowledge from databases(https://arxiv.org/abs/2506.05560)
• Keywords: language model, gpt, llm, chat, agent
• Abstract: Large language models (LLMs) are achieving significant progress almost every moment now. Many advanced techniques have been introduced and widely accepted, like retrieval-augmentation generation (RAG), agents, and tools. Tools can query the database to answer questions from structured data files or perform groupings or other statistics. This unlocks huge opportunities, such as it can answer any question, but also poses threats, such as safety, because there is no control over the commands that are created. We would like to discuss whether we can create a new method that improves answers based on dataset/database via some interpretable ML methods, namely enhanced association rules. The advantage would be if the method can be also used in some safe technique like RAG. Association rules have a sound history. Since the introduction of CN2 and aproiri, many enhancements have been made. In parallel, enhanced association rules have been introduced and evolved over the last 40 years. The general problem is typically that there are too many rules. There are some techniques for handling it, but when LLM emerged, it turned out to be the best use case for the RAG technique for LLMs. We proposed a method that generates a ruleset based on defined knowledge patterns, then converts rules into text form via a rule-to-text converter, and includes the result as an RAG into LLM. We compared this method with ChatGPT (even with using agents) and we have discovered a significant improvement in answering questions based on the dataset. We have also tried several strategies how much rules to generate. We found this improvement interesting. Moreover, it can also be improved in many ways as future work, like incorporating other patterns, the use of rule mining as an agent, and many others.
• 摘要：大型语言模型（LLM）几乎每时每刻都在取得重大进展。已经引入并广泛接受了许多先进的技术，例如检索 - 演示生成（RAG），代理和工具。工具可以查询数据库以回答从结构化数据文件或执行分组或其他统计信息的问题。这可以解锁巨大的机会，例如它可以回答任何问题，但也构成了威胁，例如安全，因为对创建的命令没有控制权。我们想讨论是否可以通过一些可解释的ML方法（即增强的关联规则）创建一种基于数据集/数据库改进答案的新方法。优势是，该方法是否也可以在某种安全技术中使用。协会规则有合理的历史。自从引入CN2和Aproiri以来，已经进行了许多增强。同时，在过去的40年中，已引入和进化了增强的关联规则。一般问题通常是规则太多。有一些处理它的技术，但是当LLM出现时，事实证明这是LLMS的抹布技术的最佳用例。我们提出了一种基于定义的知识模式生成规则集的方法，然后通过规则到文本转换器将规则转换为文本表单，并将结果作为抹布中的llm中的结果包含。我们将此方法与chatgpt（即使使用代理）进行了比较，并且在基于数据集的回答问题方面有了重大改进。我们还尝试了几种策略要生成多少规则。我们发现这种改进很有趣。此外，随着未来的工作（例如结合其他模式，将规则挖掘用作代理商）以及其他许多方式，它也可以通过许多方式改进。

Title: UTSA-NLP at ArchEHR-QA 2025: Improving EHR Question Answering via Self-Consistency Prompting

• Authors: Sara Shields-Menard, Zach Reimers, Joshua Gardner, David Perry, Anthony Rios
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05589
• Pdf URL: https://arxiv.org/pdf/2506.05589
• Copy Paste: [[2506.05589]] UTSA-NLP at ArchEHR-QA 2025: Improving EHR Question Answering via Self-Consistency Prompting(https://arxiv.org/abs/2506.05589)
• Keywords: language model, prompt
• Abstract: We describe our system for the ArchEHR-QA Shared Task on answering clinical questions using electronic health records (EHRs). Our approach uses large language models in two steps: first, to find sentences in the EHR relevant to a clinician's question, and second, to generate a short, citation-supported response based on those sentences. We use few-shot prompting, self-consistency, and thresholding to improve the sentence classification step to decide which sentences are essential. We compare several models and find that a smaller 8B model performs better than a larger 70B model for identifying relevant information. Our results show that accurate sentence selection is critical for generating high-quality responses and that self-consistency with thresholding helps make these decisions more reliable.
• 摘要：我们描述了使用电子健康记录（EHRS）回答临床问题的Archehr-QA共享任务。我们的方法分为两个步骤使用大型语言模型：首先，在EHR中找到与临床医生的问题相关的句子，其次，根据这些句子产生简短的引用支持的回答。我们使用很少的提示，自我矛盾和阈值来改善句子分类步骤，以决定哪些句子至关重要。我们比较了几个模型，发现较小的8B模型的性能要比识别相关信息的较大的70B模型更好。我们的结果表明，准确的句子选择对于产生高质量的响应至关重要，并且具有阈值的自谐有助于使这些决策更加可靠。

Title: SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs

• Authors: Michael J Ryan, Omar Shaikh, Aditri Bhagirath, Daniel Frees, William Held, Diyi Yang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05598
• Pdf URL: https://arxiv.org/pdf/2506.05598
• Copy Paste: [[2506.05598]] SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs(https://arxiv.org/abs/2506.05598)
• Keywords: language model, llm, prompt, chat
• Abstract: Recent calls for pluralistic alignment of Large Language Models (LLMs) encourage adapting models to diverse user preferences. However, most prior work on personalized reward models heavily rely on additional identity information, such as demographic details or a predefined set of preference categories. To this end, we introduce SynthesizeMe, an approach to inducing synthetic user personas from user interactions for personalized reward modeling. SynthesizeMe first generates and verifies reasoning to explain user preferences, then induces synthetic user personas from that reasoning, and finally filters to informative prior user interactions in order to build personalized prompts for a particular user. We show that using SynthesizeMe induced prompts improves personalized LLM-as-a-judge accuracy by 4.4% on Chatbot Arena. Combining SynthesizeMe derived prompts with a reward model achieves top performance on PersonalRewardBench: a new curation of user-stratified interactions with chatbots collected from 854 users of Chatbot Arena and PRISM.
• 摘要：最近要求大型语言模型（LLM）多元对准的呼吁，鼓励将模型适应各种用户偏好。但是，大多数关于个性化奖励模型的先前工作都在很大程度上依赖其他身份信息，例如人口统计学细节或预定义的一组首选项类别。为此，我们介绍了Synthesizeme，这是一种从用户交互中诱导合成用户角色的方法，以进行个性化奖励建模。 Synthesizeme首先生成并验证推理以解释用户偏好，然后从该推理中诱导合成用户角色，最后过滤到信息丰富的先前用户交互，以便为特定用户构建个性化提示。我们表明，在聊天机器人体育馆中，使用Synthesizeme诱导提示提示将个性化的LLM-AS-A-A-A-A-Gudge精度提高了4.4％。将合成的提示与奖励模型相结合，可以在个人雷沃德基地上获得最高的性能：用户分层交互的新策划与从聊天机器人体育馆和Prism的854个用户收集的聊天机器人的新策划。

Title: OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation

• Authors: Ziyi Wang, Yuxuan Lu, Wenbo Li, Amirali Amini, Bo Sun, Yakov Bart, Weimin Lyu, Jiri Gesi, Tian Wang, Jing Huang, Yu Su, Upol Ehsan, Malihe Alikhani, Toby Jia-Jun Li, Lydia Chilton, Dakuo Wang
• Subjects: cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2506.05606
• Pdf URL: https://arxiv.org/pdf/2506.05606
• Copy Paste: [[2506.05606]] OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation(https://arxiv.org/abs/2506.05606)
• Keywords: language model, llm, agent
• Abstract: Can large language models (LLMs) accurately simulate the next web action of a specific user? While LLMs have shown promising capabilities in generating ``believable'' human behaviors, evaluating their ability to mimic real user behaviors remains an open challenge, largely due to the lack of high-quality, publicly available datasets that capture both the observable actions and the internal reasoning of an actual human user. To address this gap, we introduce OPERA, a novel dataset of Observation, Persona, Rationale, and Action collected from real human participants during online shopping sessions. OPERA is the first public dataset that comprehensively captures: user personas, browser observations, fine-grained web actions, and self-reported just-in-time rationales. We developed both an online questionnaire and a custom browser plugin to gather this dataset with high fidelity. Using OPERA, we establish the first benchmark to evaluate how well current LLMs can predict a specific user's next action and rationale with a given persona and history. This dataset lays the groundwork for future research into LLM agents that aim to act as personalized digital twins for human.
• 摘要：大型语言模型（LLMS）可以准确模拟特定用户的下一个Web操作吗？尽管LLM在产生``可信的''人类行为方面表现出了令人鼓舞的能力，但评估他们模仿真实用户行为的能力仍然是一个开放的挑战，这在很大程度上是由于缺乏捕获可观察到的可观察到的动作和实际人类内部推理的高质量公开数据集。为了解决这一差距，我们介绍了Opera，这是一个新颖的观察，角色，基本原理和行动的数据集，并在在线购物期间从真正的人类参与者那里收集。 Opera是第一个全面捕获的公共数据集：用户角色，浏览器观察，细粒度的Web动作和自我报告的即时理由。我们既开发了在线问卷，又开发了一个自定义浏览器插件，以高保真收集此数据集。使用Opera，我们建立了第一个基准，以评估当前LLM可以通过给定的角色和<观察，动作，理由>历史记录的当前LLM预测特定用户的下一个动作和理由。该数据集为未来对LLM代理的研究奠定了基础，该研究旨在充当人类的个性化数字双胞胎。

Title: Mitigating Confounding in Speech-Based Dementia Detection through Weight Masking

• Authors: Zhecheng Sheng, Xiruo Ding, Brian Hur, Changye Li, Trevor Cohen, Serguei Pakhomov
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05610
• Pdf URL: https://arxiv.org/pdf/2506.05610
• Copy Paste: [[2506.05610]] Mitigating Confounding in Speech-Based Dementia Detection through Weight Masking(https://arxiv.org/abs/2506.05610)
• Keywords: language model
• Abstract: Deep transformer models have been used to detect linguistic anomalies in patient transcripts for early Alzheimer's disease (AD) screening. While pre-trained neural language models (LMs) fine-tuned on AD transcripts perform well, little research has explored the effects of the gender of the speakers represented by these transcripts. This work addresses gender confounding in dementia detection and proposes two methods: the $\textit{Extended Confounding Filter}$ and the $\textit{Dual Filter}$, which isolate and ablate weights associated with gender. We evaluate these methods on dementia datasets with first-person narratives from patients with cognitive impairment and healthy controls. Our results show transformer models tend to overfit to training data distributions. Disrupting gender-related weights results in a deconfounded dementia classifier, with the trade-off of slightly reduced dementia detection performance.
• 摘要：深层变压器模型已用于检测早期阿尔茨海默氏病（AD）筛查的患者转录本中的语言异常。尽管对AD成绩单进行微调的预训练的神经语言模型（LMS）表现良好，但很少的研究探讨了这些成绩单代表的说话者性别的影响。这项工作解决了痴呆症检测中的性别混淆，并提出了两种方法：$ \ textIt {扩展混杂的过滤器} $和$ \ textit {dual Filter} $，它们隔离与性别相关的重量。我们在痴呆数据集上评估了这些方法，并具有来自认知障碍和健康对照患者的第一人称叙述。我们的结果表明，变压器模型倾向于过度培训数据分布。破坏与性别相关的体重会导致痴呆症分类器，并取决于略微降低痴呆症检测性能。

Title: Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs

• Authors: Ananth Muppidi, Abhilash Nandy, Sambaran Bandyopadhyay
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05629
• Pdf URL: https://arxiv.org/pdf/2506.05629
• Copy Paste: [[2506.05629]] Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs(https://arxiv.org/abs/2506.05629)
• Keywords: language model, llm, prompt
• Abstract: The performance of large language models in domain-specific tasks necessitates fine-tuning, which is computationally expensive and technically challenging. This paper focuses on parameter-efficient fine-tuning using soft prompting, a promising approach that adapts pre-trained models to downstream tasks by learning a small set of parameters. We propose a novel Input Dependent Soft Prompting technique with a self-Attention Mechanism (ID-SPAM) that generates soft prompts based on the input tokens and attends different tokens with varying importance. Our method is simple and efficient, keeping the number of trainable parameters small. We show the merits of the proposed approach compared to state-of-the-art techniques on various tasks and show the improved zero shot domain transfer capability.
• 摘要：大型语言模型在特定于领域的任务中的性能需要进行微调，这在计算上昂贵且在技术上具有挑战性。本文着重于使用软提示的参数效率微调，这是一种有前途的方法，通过学习一小部分参数，将预训练的模型调整为下游任务。我们提出了一种新颖的输入依赖性软提示技术，其自我发挥机制（ID-SPAM）基于输入令牌生成软提示，并以不同的重视参与不同的令牌。我们的方法简单有效，使可训练参数的数量少。我们显示了与各种任务的最新技术相比，提出的方法的优点，并显示了提高的零射击域传输能力。

Title: IYKYK: Using language models to decode extremist cryptolects

• Authors: Christine de Kock, Arij Riabi, Zeerak Talat, Michael Sejr Schlichtkrull, Pranava Madhyastha, Ed Hovy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05635
• Pdf URL: https://arxiv.org/pdf/2506.05635
• Copy Paste: [[2506.05635]] IYKYK: Using language models to decode extremist cryptolects(https://arxiv.org/abs/2506.05635)
• Keywords: language model, llm, prompt
• Abstract: Extremist groups develop complex in-group language, also referred to as cryptolects, to exclude or mislead outsiders. We investigate the ability of current language technologies to detect and interpret the cryptolects of two online extremist platforms. Evaluating eight models across six tasks, our results indicate that general purpose LLMs cannot consistently detect or decode extremist language. However, performance can be significantly improved by domain adaptation and specialised prompting techniques. These results provide important insights to inform the development and deployment of automated moderation technologies. We further develop and release novel labelled and unlabelled datasets, including 19.4M posts from extremist platforms and lexicons validated by human experts.
• 摘要：极端主义群体发展了复杂的小组内语言，也称为加密章节，以排除或误导局外人。我们研究了当前语言技术检测和解释两个在线极端主义平台的加密降低的能力。评估六个任务的八个模型，我们的结果表明，通用LLM不能始终检测或解码极端主义语言。但是，通过域的适应和专门的提示技术可以显着提高性能。这些结果提供了重要的见解，以告知自动节制技术的开发和部署。我们进一步开发和发行了标有和未标记的数据集的小说，其中包括由人类专家验证的极端主义平台和词典的1940万帖子。

Title: A Fictional Q&A Dataset for Studying Memorization and Knowledge Acquisition

• Authors: John Kirchenbauer, Janny Mongkolsupawan, Yuxin Wen, Tom Goldstein, Daphne Ippolito
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.05639
• Pdf URL: https://arxiv.org/pdf/2506.05639
• Copy Paste: [[2506.05639]] A Fictional Q&A Dataset for Studying Memorization and Knowledge Acquisition(https://arxiv.org/abs/2506.05639)
• Keywords: language model
• Abstract: When language models are trained on textual data, they acquire both knowledge about the structure of language as well as knowledge of facts about the world. At inference time, their knowledge of facts can be leveraged to solve interesting problems and perform useful knowledge work for users. It is well known that language models can verbatim memorize long sequences from their training data. However, it is much less well understood how language models memorize facts seen during training. In this work, we propose a new dataset to specifically empower researchers to study the dual processes of fact memorization and verbatim sequence memorization. The dataset consists of synthetically-generated, webtext-like documents about fictional events, as well as question-answer pairs about the events. We conduct training experiments showing how synthetic data about fictional events can be effective in teasing apart different forms of memorization. We also document the challenges in effectively building realistic, fictional synthetic data.
• 摘要：当语言模型接受文本数据培训时，他们既获得有关语言结构的知识又是对世界的事实知识。在推论时，他们可以利用他们对事实的了解来解决有趣的问题并为用户执行有用的知识工作。众所周知，语言模型可以逐字记住他们的训练数据中的长序列。但是，不太了解语言模型如何记住培训期间看到的事实。在这项工作中，我们提出了一个新数据集，以特别授权研究人员研究事实记忆和逐字记忆的双重过程。该数据集由合成生成的类似于虚构事件的综合生成的文档，以及有关事件的问答配对。我们进行培训实验，以表明有关虚构事件的合成数据如何有效地嘲笑不同形式的记忆。我们还记录了有效构建现实的虚构合成数据时面临的挑战。

Title: Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment

• Authors: Priyanka Dey, Yugal Khanter, Aayush Bothra, Jieyu Zhao, Emilio Ferrara
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05670
• Pdf URL: https://arxiv.org/pdf/2506.05670
• Copy Paste: [[2506.05670]] Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment(https://arxiv.org/abs/2506.05670)
• Keywords: llm, prompt
• Abstract: As LLMs become central to interactive applications, ranging from tutoring to mental health, the ability to express personality in culturally appropriate ways is increasingly important. While recent works have explored personality evaluation of LLMs, they largely overlook the interplay between culture and personality. To address this, we introduce CulturalPersonas, the first large-scale benchmark with human validation for evaluating LLMs' personality expression in culturally grounded, behaviorally rich contexts. Our dataset spans 3,000 scenario-based questions across six diverse countries, designed to elicit personality through everyday scenarios rooted in local values. We evaluate three LLMs, using both multiple-choice and open-ended response formats. Our results show that CulturalPersonas improves alignment with country-specific human personality distributions (over a 20% reduction in Wasserstein distance across models and countries) and elicits more expressive, culturally coherent outputs compared to existing benchmarks. CulturalPersonas surfaces meaningful modulated trait outputs in response to culturally grounded prompts, offering new directions for aligning LLMs to global norms of behavior. By bridging personality expression and cultural nuance, we envision that CulturalPersonas will pave the way for more socially intelligent and globally adaptive LLMs.
• 摘要：随着LLM成为互动应用的核心，从辅导到心理健康，以文化适当的方式表达人格的能力越来越重要。尽管最近的作品探索了LLM的个性评估，但它们在很大程度上忽略了文化与个性之间的相互作用。为了解决这个问题，我们介绍了文化植物，这是第一个具有人类验证的大规模基准，用于评估LLMS在文化扎根，行为丰富的环境中的性格表达。我们的数据集涵盖了六个不同国家的3,000个基于方案的问题，旨在通过植根于当地价值观的日常场景来引起个性。我们使用多项选择和开放式响应格式评估三个LLM。我们的结果表明，文化培训改善了与国家特定的人格分布的一致性（与现有基准相比，在模型和国家之间的瓦斯汀距离降低了20％以上的距离距离降低了20％）。文化培训表现出有意义的调制性状输出，以响应文化扎根的提示，提供了将LLM与全球行为规范保持一致的新方向。通过弥合人格表达和文化细微差别，我们设想文化植物将为更具社会智能和全球自适应的LLM铺平道路。

Title: Zero-Shot Event Causality Identification via Multi-source Evidence Fuzzy Aggregation with Large Language Models

• Authors: Zefan Zeng, Xingchen Hu, Qing Cheng, Weiping Ding, Wentao Li, Zhong Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05675
• Pdf URL: https://arxiv.org/pdf/2506.05675
• Copy Paste: [[2506.05675]] Zero-Shot Event Causality Identification via Multi-source Evidence Fuzzy Aggregation with Large Language Models(https://arxiv.org/abs/2506.05675)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Event Causality Identification (ECI) aims to detect causal relationships between events in textual contexts. Existing ECI models predominantly rely on supervised methodologies, suffering from dependence on large-scale annotated data. Although Large Language Models (LLMs) enable zero-shot ECI, they are prone to causal hallucination-erroneously establishing spurious causal links. To address these challenges, we propose MEFA, a novel zero-shot framework based on Multi-source Evidence Fuzzy Aggregation. First, we decompose causality reasoning into three main tasks (temporality determination, necessity analysis, and sufficiency verification) complemented by three auxiliary tasks. Second, leveraging meticulously designed prompts, we guide LLMs to generate uncertain responses and deterministic outputs. Finally, we quantify LLM's responses of sub-tasks and employ fuzzy aggregation to integrate these evidence for causality scoring and causality determination. Extensive experiments on three benchmarks demonstrate that MEFA outperforms second-best unsupervised baselines by 6.2% in F1-score and 9.3% in precision, while significantly reducing hallucination-induced errors. In-depth analysis verify the effectiveness of task decomposition and the superiority of fuzzy aggregation.
• 摘要：事件因果关系识别（ECI）旨在检测文本上下文中事件之间的因果关系。现有的ECI模型主要依赖于监督方法，依赖于大规模注释的数据。尽管大型语言模型（LLMS）使ECI呈零摄，但它们倾向于因果幻觉而建立虚假的因果关系。为了应对这些挑战，我们提出了MEFA，这是一个基于多源证据模糊聚集的新型零击框架。首先，我们将因果关系推理分为三个主要任务（时间性确定，必要性分析和充分性验证），并由三个辅助任务进行补充。其次，利用精心设计的提示，我们指导LLMS产生不确定的响应和确定性输出。最后，我们量化了LLM对子任务的响应，并采用模糊聚合来整合这些证据以确定因果关系和因果关系的确定。对三个基准测试的广泛实验表明，MEFA在F1分数中的表现优于第二好的无监督基线，精度为9.3％，而显着降低了幻觉引起的误差。深入分析验证任务分解的有效性和模糊聚集的优势。

Title: When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation

• Authors: Zhishang Xiang, Chuanjie Wu, Qinggang Zhang, Shengyuan Chen, Zijin Hong, Xiao Huang, Jinsong Su
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05690
• Pdf URL: https://arxiv.org/pdf/2506.05690
• Copy Paste: [[2506.05690]] When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation(https://arxiv.org/abs/2506.05690)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Graph retrieval-augmented generation (GraphRAG) has emerged as a powerful paradigm for enhancing large language models (LLMs) with external knowledge. It leverages graphs to model the hierarchical structure between specific concepts, enabling more coherent and effective knowledge retrieval for accurate this http URL its conceptual promise, recent studies report that GraphRAG frequently underperforms vanilla RAG on many real-world tasks. This raises a critical question: Is GraphRAG really effective, and in which scenarios do graph structures provide measurable benefits for RAG systems? To address this, we propose GraphRAG-Bench, a comprehensive benchmark designed to evaluate GraphRAG models onboth hierarchical knowledge retrieval and deep contextual reasoning. GraphRAG-Bench features a comprehensive dataset with tasks of increasing difficulty, coveringfact retrieval, complex reasoning, contextual summarization, and creative generation, and a systematic evaluation across the entire pipeline, from graph constructionand knowledge retrieval to final generation. Leveraging this novel benchmark, we systematically investigate the conditions when GraphRAG surpasses traditional RAG and the underlying reasons for its success, offering guidelines for its practical application. All related resources and analyses are collected for the community at this https URL.
• 摘要：Graph检索效果生成（GraphRag）已成为具有外部知识增强大型语言模型（LLM）的强大范式。它利用图形来对特定概念之间的层次结构进行建模，从而使更连贯和有效的知识检索能够准确地进行此HTTP URL的概念上的承诺，最近的研究报告说，GraphRag经常在许多现实世界任务上表现不佳。这提出了一个关键的问题：GraphRag真的有效吗？在哪些情况下，图形结构为抹布系统提供了可衡量的好处？为了解决这个问题，我们提出了GraphRag-Bench，这是一个综合基准测试，旨在评估GraphRag模型的分层知识检索和深层上下文推理。 GraphRag Bench具有一个综合的数据集，其任务是增加难度，涵盖事实检索，复杂的推理，上下文摘要和创造性生成，以及整个管道的系统评估，从图形构造和知识检索到最终一代。利用这种新颖的基准，我们系统地研究了GraphRag超过传统抹布的条件以及其成功的根本原因，为其实际应用提供了指南。在此HTTPS URL上为社区收集了所有相关资源和分析。

Title: Being Strong Progressively! Enhancing Knowledge Distillation of Large Language Models through a Curriculum Learning Framework

• Authors: Lingyuan Liu, Mengxiang Zhang
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.05695
• Pdf URL: https://arxiv.org/pdf/2506.05695
• Copy Paste: [[2506.05695]] Being Strong Progressively! Enhancing Knowledge Distillation of Large Language Models through a Curriculum Learning Framework(https://arxiv.org/abs/2506.05695)
• Keywords: language model, llm
• Abstract: Knowledge Distillation (KD) compresses large language models (LLMs) by transferring the teacher model's capabilities to a smaller student model, reducing inference cost and memory usage while maintaining performance. However, existing KD methods for LLMs often fail to prevent significant shifts in the student model's distribution during training, leading to issues such as catastrophic forgetting, mode collapse, and training-inference mismatch. To address these challenges, we propose a novel, plug-in curriculum learning framework inspired by the strength training principle of "progressive overload" (POCL), which can be seamlessly integrated into existing white-box KD approaches with minimal computational overhead. The framework comprises two core components: (1) a difficulty measurer that ranks and partitions training samples from easy to hard, and (2) a training scheduler that incrementally introduces these subsets into the distillation process at fixed intervals while applying loss functions with progressively rising temperatures. By starting with the easiest samples and progressively increasing the difficulty, the approach enhances both the stability and efficiency of learning. Extensive experiments in instruction-following settings demonstrate that POCL consistently improves the performance of distilled student models across various white-box KD methods and model families. Our findings highlight the effectiveness of sorted training samples in KD for LLMs. More generally, our work demonstrates how to structure training data within the KD process to enhance the stability and performance of distilled LLMs.
• 摘要：知识蒸馏（KD）通过将教师模型的能力转移到较小的学生模型，降低推理成本和记忆使用的同时，在保持性能的同时，可以压缩大语言模型（LLMS）。但是，现有的LLMS的KD方法通常无法阻止培训期间学生模型的分布发生重大变化，从而导致灾难性遗忘，模式崩溃和训练推断不匹配等问题。为了应对这些挑战，我们提出了一个由“渐进过载”（POCL）的力量训练原理启发的新颖的，插入式课程学习框架，可以将其无缝集成到现有的白色盒子KD方法中，并以最小的计算高架开销。该框架包括两个核心组成部分：（1）一个难度测量值，将训练样本从易于硬到硬到硬的培训样本进行排名和分区，以及（2）培训调度程序，将这些子集逐步引入蒸馏过程中，同时将损失功能应用于逐渐升高的温度。从最简单的样本开始并逐步增加难度，该方法可以提高学习的稳定性和效率。在遵循指令遵循的设置中进行的广泛实验表明，POCL始终提高各种白色盒子KD方法和模型家族的蒸馏学生模型的性能。我们的发现突出了KD中对LLM的分类培训样本的有效性。更普遍地，我们的工作演示了如何在KD过程中构建培训数据，以增强蒸馏LLM的稳定性和性能。

Title: RKEFino1: A Regulation Knowledge-Enhanced Large Language Model

• Authors: Yan Wang, Yueru He, Ruoyu Xiang, Jeff Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05700
• Pdf URL: https://arxiv.org/pdf/2506.05700
• Copy Paste: [[2506.05700]] RKEFino1: A Regulation Knowledge-Enhanced Large Language Model(https://arxiv.org/abs/2506.05700)
• Keywords: language model, llm
• Abstract: Recent advances in large language models (LLMs) hold great promise for financial applications but introduce critical accuracy and compliance challenges in Digital Regulatory Reporting (DRR). To address these issues, we propose RKEFino1, a regulation knowledge-enhanced financial reasoning model built upon Fino1, fine-tuned with domain knowledge from XBRL, CDM, and MOF. We formulate two QA tasks-knowledge-based and mathematical reasoning-and introduce a novel Numerical NER task covering financial entities in both sentences and tables. Experimental results demonstrate the effectiveness and generalization capacity of RKEFino1 in compliance-critical financial tasks. We have released our model on Hugging Face.
• 摘要：大型语言模型（LLMS）的最新进展对财务应用具有巨大的希望，但在数字监管报告（DRR）中引入了关键的准确性和合规性挑战。为了解决这些问题，我们提出了Rkefino1，这是一种基于Fino1的法规知识增强的财务推理模型，并以XBRL，CDM和MOF的域知识进行了微调。我们制定了两个基于知识的质量检查任务和数学推理，并介绍了涵盖句子和表格中财务实体的新颖数字任务。实验结果证明了Rkefino1在合规财务任务中的有效性和概括能力。我们已经发布了有关拥抱脸的模型。

Title: Large Language Models are Good Relational Learners

• Authors: Fang Wu, Vijay Prakash Dwivedi, Jure Leskovec
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.05725
• Pdf URL: https://arxiv.org/pdf/2506.05725
• Copy Paste: [[2506.05725]] Large Language Models are Good Relational Learners(https://arxiv.org/abs/2506.05725)
• Keywords: language model, llm, prompt, retrieval-augmented generation
• Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across various domains, yet their application to relational deep learning (RDL) remains underexplored. Existing approaches adapt LLMs by traversing relational links between entities in a database and converting the structured data into flat text documents. Still, this text-based serialization disregards critical relational structures, introduces redundancy, and often exceeds standard LLM context lengths. We introduce Rel-LLM, a novel architecture that utilizes a graph neural network (GNN)- based encoder to generate structured relational prompts for LLMs within a retrieval-augmented generation (RAG) framework. Unlike traditional text-based serialization approaches, our method preserves the inherent relational structure of databases while enabling LLMs to effectively process and reason over complex entity relationships. Specifically, the GNN encoder extracts a local subgraph around an entity to build feature representations that contain relevant entity relationships and temporal dependencies. These representations are transformed into structured prompts using a denormalization process, effectively allowing the LLM to reason over relational structures. Through extensive experiments, we demonstrate that Rel-LLM outperforms existing methods on key RDL tasks, offering a scalable and efficient approach to integrating LLMs with structured data sources. Code is available at this https URL.
• 摘要：大型语言模型（LLMS）在各个领域都表现出了非凡的功能，但它们在关系深度学习（RDL）中的应用仍未得到充实。现有方法通过遍历数据库中实体之间的关系链接并将结构化数据转换为平面文本文档来适应LLM。尽管如此，这种基于文本的序列化忽略了关键的关系结构，引入了冗余，并且通常超过标准的LLM上下文长度。我们介绍了Rel-Llm，这是一种使用图形神经网络（GNN）的新型体系结构，基于图形的编码器在检索效果（RAG）框架中为LLM生成结构化的关系提示。与传统的基于文本的序列化方法不同，我们的方法保留了数据库的固有关系结构，同时使LLMS能够有效地处理和推理复杂的实体关系。具体而言，GNN编码器提取围绕实体的本地子图，以构建包含相关实体关系和时间依赖性的特征表示。这些表示形式使用非规范化过程转化为结构化提示，从而有效地允许LLM对关系结构进行推理。通过广泛的实验，我们证明了Rel-LLM在关键RDL任务上的现有方法优于现有方法，从而为将LLM与结构化数据源集成在一起提供了可扩展有效的方法。代码可在此HTTPS URL上找到。

Title: Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness

• Authors: Rongzhe Wei, Peizhi Niu, Hans Hao-Hsun Hsu, Ruihan Wu, Haoteng Yin, Mohsen Ghassemi, Yifan Li, Vamsi K. Potluru, Eli Chien, Kamalika Chaudhuri, Olgica Milenkovic, Pan Li
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.05735
• Pdf URL: https://arxiv.org/pdf/2506.05735
• Copy Paste: [[2506.05735]] Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness(https://arxiv.org/abs/2506.05735)
• Keywords: language model, llm, prompt
• Abstract: Machine unlearning techniques aim to mitigate unintended memorization in large language models (LLMs). However, existing approaches predominantly focus on the explicit removal of isolated facts, often overlooking latent inferential dependencies and the non-deterministic nature of knowledge within LLMs. Consequently, facts presumed forgotten may persist implicitly through correlated information. To address these challenges, we propose a knowledge unlearning evaluation framework that more accurately captures the implicit structure of real-world knowledge by representing relevant factual contexts as knowledge graphs with associated confidence scores. We further develop an inference-based evaluation protocol leveraging powerful LLMs as judges; these judges reason over the extracted knowledge subgraph to determine unlearning success. Our LLM judges utilize carefully designed prompts and are calibrated against human evaluations to ensure their trustworthiness and stability. Extensive experiments on our newly constructed benchmark demonstrate that our framework provides a more realistic and rigorous assessment of unlearning performance. Moreover, our findings reveal that current evaluation strategies tend to overestimate unlearning effectiveness. Our code is publicly available at this https URL.
• 摘要：机器学习技术旨在减轻大语模型（LLMS）中意外的记忆。但是，现有方法主要集中于明确删除孤立的事实，通常忽略了LLM中知识的潜在推论依赖性和知识的非确定性。因此，假定被遗忘的事实可能会通过相关信息隐含地隐含。为了应对这些挑战，我们提出了一个知识学习的评估框架，该框架更准确地捕获了现实世界知识的隐含结构，通过将相关的事实环境表示为具有相关置信度分数的知识图。我们进一步开发了一种基于推理的评估协议，以利用强大的LLM作为法官；这些法官将提取的知识子图推定为确定成功的成功。我们的LLM法官使用精心设计的提示，并对人类评估进行校准，以确保其可信度和稳定性。对我们新建造的基准测试的广泛实验表明，我们的框架提供了对未学习绩效的更现实，更严格的评估。此外，我们的发现表明，当前的评估策略倾向于高估未学习的效率。我们的代码在此HTTPS URL上公开可用。

Title: LLM-Symbolic Integration for Robust Temporal Tabular Reasoning

• Authors: Atharv Kulkarni, Kushagra Dixit, Vivek Srikumar, Dan Roth, Vivek Gupta
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05746
• Pdf URL: https://arxiv.org/pdf/2506.05746
• Copy Paste: [[2506.05746]] LLM-Symbolic Integration for Robust Temporal Tabular Reasoning(https://arxiv.org/abs/2506.05746)
• Keywords: language model, llm, prompt
• Abstract: Temporal tabular question answering presents a significant challenge for Large Language Models (LLMs), requiring robust reasoning over structured data, which is a task where traditional prompting methods often fall short. These methods face challenges such as memorization, sensitivity to table size, and reduced performance on complex queries. To overcome these limitations, we introduce TempTabQA-C, a synthetic dataset designed for systematic and controlled evaluations, alongside a symbolic intermediate representation that transforms tables into database schemas. This structured approach allows LLMs to generate and execute SQL queries, enhancing generalization and mitigating biases. By incorporating adaptive few-shot prompting with contextually tailored examples, our method achieves superior robustness, scalability, and performance. Experimental results consistently highlight improvements across key challenges, setting a new benchmark for robust temporal reasoning with LLMs.
• 摘要：时间表格问题回答对大型语言模型（LLMS）提出了重大挑战，需要在结构化数据上进行强有力的推理，这是传统提示方法通常不足的任务。这些方法面临着挑战，例如记忆，对桌子大小的敏感性以及在复杂查询上的性能降低。为了克服这些局限性，我们引入了temptabqa-c，这是一种旨在系统和受控评估的合成数据集，以及将表转换为数据库模式的符号中间表示。这种结构化方法允许LLMS生成和执行SQL查询，增强概括和减轻偏见。通过将自适应少量促进件与上下文量身定制的示例合并，我们的方法可以实现出色的鲁棒性，可扩展性和性能。实验结果始终突出了跨关键挑战的改进，为使用LLMS的稳健时间推理树立了新的基准。

Title: Writing-RL: Advancing Long-form Writing via Adaptive Curriculum Reinforcement Learning

• Authors: Xuanyu Lei, Chenliang Li, Yuning Wu, Kaiming Liu, Weizhou Shen, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Yang Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05760
• Pdf URL: https://arxiv.org/pdf/2506.05760
• Copy Paste: [[2506.05760]] Writing-RL: Advancing Long-form Writing via Adaptive Curriculum Reinforcement Learning(https://arxiv.org/abs/2506.05760)
• Keywords: language model, llm
• Abstract: Recent advances in Large Language Models (LLMs) have enabled strong performance in long-form writing, yet existing supervised fine-tuning (SFT) approaches suffer from limitations such as data saturation and restricted learning capacity bounded by teacher signals. In this work, we present Writing-RL: an Adaptive Curriculum Reinforcement Learning framework to advance long-form writing capabilities beyond SFT. The framework consists of three key components: Margin-aware Data Selection strategy that prioritizes samples with high learning potential, Pairwise Comparison Reward mechanism that provides discriminative learning signals in the absence of verifiable rewards, and Dynamic Reference Scheduling approach, which plays a particularly critical role by adaptively adjusting task difficulty based on evolving model performance. Experiments on 7B-scale writer models show that our RL framework largely improves long-form writing performance over strong SFT baselines. Furthermore, we observe that models trained with long-output RL generalize surprisingly well to long-input reasoning tasks, potentially offering a promising perspective for rethinking long-context training.
• 摘要：大型语言模型（LLM）的最新进展使得长期写作的出色表现，但现有的监督微调（SFT）方法遭受了诸如数据饱和度和受教师信号界限的限制学习能力之类的限制。在这项工作中，我们介绍了写作RL：一种自适应课程增强学习框架，以使长期写作能力超越SFT。该框架由三个关键组成部分组成：利润感知数据选择策略，优先考虑具有较高学习潜力的样本，成对比较奖励机制，这些奖励机制在没有可验证的奖励和动态参考计划方法的情况下提供了判别性学习信号，该方法通过基于进化的模型性能来适应任务难度，从而特别重要。 7B规模作者模型的实验表明，我们的RL框架在很大程度上改善了强大的SFT基准的长形式写作表现。此外，我们观察到，经过长输入RL训练的模型出奇地概括了长期推理任务，这可能提供了重新思考长期培训的有前途的观点。

Title: BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactions

• Authors: Saptarshi Sengupta, Shuhua Yang, Paul Kwong Yu, Fali Wang, Suhang Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05766
• Pdf URL: https://arxiv.org/pdf/2506.05766
• Copy Paste: [[2506.05766]] BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactions(https://arxiv.org/abs/2506.05766)
• Keywords: language model, llm, retrieval augmented generation
• Abstract: Retrieval augmented generation (RAG) has shown great power in improving Large Language Models (LLMs). However, most existing RAG-based LLMs are dedicated to retrieving single modality information, mainly text; while for many real-world problems, such as healthcare, information relevant to queries can manifest in various modalities such as knowledge graph, text (clinical notes), and complex molecular structure. Thus, being able to retrieve relevant multi-modality domain-specific information, and reason and synthesize diverse knowledge to generate an accurate response is important. To address the gap, we present BioMol-MQA, a new question-answering (QA) dataset on polypharmacy, which is composed of two parts (i) a multimodal knowledge graph (KG) with text and molecular structure for information retrieval; and (ii) challenging questions that designed to test LLM capabilities in retrieving and reasoning over multimodal KG to answer questions. Our benchmarks indicate that existing LLMs struggle to answer these questions and do well only when given the necessary background data, signaling the necessity for strong RAG frameworks.
• 摘要：检索增强发电（RAG）在改善大型语言模型（LLMS）方面具有很大的力量。但是，大多数现有的基于抹布的LLM都致力于检索单个模式信息，主要是文本。尽管对于许多实际问题（例如医疗保健），但与查询相关的信息可以以各种方式表现出来，例如知识图，文本（临床注释）和复杂的分子结构。因此，能够检索相关的多模式域特异性信息，以及理性和合成多样化知识以产生准确的响应很重要。为了解决差距，我们提出了BioMol-MQA，这是一个关于多剂量的新问题 - 索问题（QA）数据集，该数据集由两个部分（i）具有文本和分子结构的多模式知识图（kg）组成，用于信息检索； （ii）旨在测试LLM功能的挑战性问题，以检索和推理多模式KG来回答问题。我们的基准表明，现有的LLM努力回答这些问题，并且只有在给出必要的背景数据时才能做得很好，这表明了强大的抹布框架的必要性。

Title: dots.llm1 Technical Report

• Authors: Bi Huo, Bin Tu, Cheng Qin, Da Zheng, Debing Zhang, Dongjie Zhang, En Li, Fu Guo, Jian Yao, Jie Lou, Junfeng Tian, Li Hu, Ran Zhu, Shengdong Chen, Shuo Liu, Su Guang, Te Wo, Weijun Zhang, Xiaoming Shi, Xinxin Peng, Xing Wu, Yawen Liu, Yuqiu Ji, Ze Wen, Zhenhai Liu, Zichao Li, Zilong Liao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05767
• Pdf URL: https://arxiv.org/pdf/2506.05767
• Copy Paste: [[2506.05767]] dots.llm1 Technical Report(https://arxiv.org/abs/2506.05767)
• Keywords: language model, llm
• Abstract: Mixture of Experts (MoE) models have emerged as a promising paradigm for scaling language models efficiently by activating only a subset of parameters for each input token. In this report, we present dots.llm1, a large-scale MoE model that activates 14B parameters out of a total of 142B parameters, delivering performance on par with state-of-the-art models while reducing training and inference costs. Leveraging our meticulously crafted and efficient data processing pipeline, dots.llm1 achieves performance comparable to Qwen2.5-72B after pretraining on 11.2T high-quality tokens and post-training to fully unlock its capabilities. Notably, no synthetic data is used during pretraining. To foster further research, we open-source intermediate training checkpoints at every one trillion tokens, providing valuable insights into the learning dynamics of large language models.
• 摘要：专家（MOE）模型的混合物已成为一种有希望的范式，用于通过仅激活每个输入令牌的参数子集有效地缩放语言模型。在本报告中，我们提出了DOTS.LLM1，这是一种大规模的MOE模型，该模型从总计142B参数中激活14B参数，与最先进的模型相同，同时降低培训和推理成本。 DOTS.LLM1利用我们精心制作且有效的数据处理管道，可在11.2T高质量的代币和训练后训练以完全解锁其功能后，达到与QWEN2.5-72B相当的性能。值得注意的是，在预训练期间没有使用合成数据。为了促进进一步的研究，我们在每百万个令牌上开放源中间培训检查站，为大语言模型的学习动态提供了宝贵的见解。

Title: Discrete Minds in a Continuous World: Do Language Models Know Time Passes?

• Authors: Minghan Wang, Ye Bai, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05790
• Pdf URL: https://arxiv.org/pdf/2506.05790
• Copy Paste: [[2506.05790]] Discrete Minds in a Continuous World: Do Language Models Know Time Passes?(https://arxiv.org/abs/2506.05790)
• Keywords: language model, llm
• Abstract: While Large Language Models (LLMs) excel at temporal reasoning tasks like event ordering and duration estimation, their ability to perceive the actual passage of time remains unexplored. We investigate whether LLMs perceive the passage of time and adapt their decision-making accordingly through three complementary experiments. First, we introduce the Token-Time Hypothesis, positing that LLMs can map discrete token counts to continuous wall-clock time, and validate this through a dialogue duration judgment task. Second, we demonstrate that LLMs could use this awareness to adapt their response length while maintaining accuracy when users express urgency in question answering tasks. Finally, we develop BombRush, an interactive navigation challenge that examines how LLMs modify behavior under progressive time pressure in dynamic environments. Our findings indicate that LLMs possess certain awareness of time passage, enabling them to bridge discrete linguistic tokens and continuous physical time, though this capability varies with model size and reasoning abilities. This work establishes a theoretical foundation for enhancing temporal awareness in LLMs for time-sensitive applications.
• 摘要：尽管大型语言模型（LLMS）在时间推理任务（例如事件排序和持续时间估算）上表现出色，但他们感知时间实际流逝的能力仍未得到探索。我们研究了LLM是否感知时间的流逝，并通过三个补充实验相应地调整其决策。首先，我们介绍了令牌时间假设，认为LLM可以将离散的令牌计数映射到连续的墙壁锁定时间，并通过对话持续时间判断任务来验证这一点。其次，我们证明LLM可以使用这种意识来调整其响应长度，同时在用户表达有问题的紧迫性回答任务时保持准确性。最后，我们开发了Bombrush，这是一个交互式导航挑战，该挑战研究了LLM在动态环境中如何修改渐进时间压力下的行为。我们的发现表明，LLM具有一定的时间段落的意识，使它们能够桥接离散的语言令牌和连续的物理时间，尽管这种能力随模型的大小和推理能力而变化。这项工作为增强时间敏感应用的LLMS的时间意识建立了理论基础。

Title: MAPLE: Multi-Agent Adaptive Planning with Long-Term Memory for Table Reasoning

• Authors: Ye Bai, Minghan Wang, Thuy-Trang Vu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05813
• Pdf URL: https://arxiv.org/pdf/2506.05813
• Copy Paste: [[2506.05813]] MAPLE: Multi-Agent Adaptive Planning with Long-Term Memory for Table Reasoning(https://arxiv.org/abs/2506.05813)
• Keywords: llm, chain-of-thought, agent
• Abstract: Table-based question answering requires complex reasoning capabilities that current LLMs struggle to achieve with single-pass inference. Existing approaches, such as Chain-of-Thought reasoning and question decomposition, lack error detection mechanisms and discard problem-solving experiences, contrasting sharply with how humans tackle such problems. In this paper, we propose MAPLE (Multi-agent Adaptive Planning with Long-term mEmory), a novel framework that mimics human problem-solving through specialized cognitive agents working in a feedback-driven loop. MAPLE integrates 4 key components: (1) a Solver using the ReAct paradigm for reasoning, (2) a Checker for answer verification, (3) a Reflector for error diagnosis and strategy correction, and (4) an Archiver managing long-term memory for experience reuse and evolution. Experiments on WiKiTQ and TabFact demonstrate significant improvements over existing methods, achieving state-of-the-art performance across multiple LLM backbones.
• 摘要：基于桌子的问题回答需要当前LLM努力通过单通行推理而难以实现的复杂推理能力。现有的方法，例如经过思考的推理和问题分解，缺乏错误检测机制和丢弃解决问题的经验，与人类如何解决此类问题形成鲜明对比。在本文中，我们提出了Maple（具有长期记忆的多代理自适应计划），这是一个新颖的框架，通过在反馈驱动的循环中工作的专门认知剂模仿人类问题解决的框架。 Maple集成了4个关键组件：（1）使用React范式进行推理的求解器，（2）检查器进行答案验证，（3）用于错误诊断和策略校正的反射器，以及（4）档案馆管理长期记忆以进行经验重复使用和进化。关于WikITQ和TABFACT的实验表明，对现有方法表现出重大改进，从而在多个LLM骨架上实现了最先进的性能。

Title: FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging

• Authors: Zichen Tang, Haihong E, Ziyan Ma, Haoyang He, Jiacheng Liu, Zhongjun Yang, Zihua Rong, Rongjin Li, Kun Ji, Qing Huang, Xinyang Hu, Yang Liu, Qianhe Zheng
• Subjects: cs.CL, cs.CE
• Abstract URL: https://arxiv.org/abs/2506.05828
• Pdf URL: https://arxiv.org/pdf/2506.05828
• Copy Paste: [[2506.05828]] FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging(https://arxiv.org/abs/2506.05828)
• Keywords: gpt
• Abstract: We introduce FinanceReasoning, a novel benchmark designed to evaluate the reasoning capabilities of large reasoning models (LRMs) in financial numerical reasoning problems. Compared to existing benchmarks, our work provides three key advancements. (1) Credibility: We update 15.6% of the questions from four public datasets, annotating 908 new questions with detailed Python solutions and rigorously refining evaluation standards. This enables an accurate assessment of the reasoning improvements of LRMs. (2) Comprehensiveness: FinanceReasoning covers 67.8% of financial concepts and formulas, significantly surpassing existing datasets. Additionally, we construct 3,133 Python-formatted functions, which enhances LRMs' financial reasoning capabilities through refined knowledge (e.g., 83.2% $\rightarrow$ 91.6% for GPT-4o). (3) Challenge: Models are required to apply multiple financial formulas for precise numerical reasoning on 238 Hard problems. The best-performing model (i.e., OpenAI o1 with PoT) achieves 89.1% accuracy, yet LRMs still face challenges in numerical precision. We demonstrate that combining Reasoner and Programmer models can effectively enhance LRMs' performance (e.g., 83.2% $\rightarrow$ 87.8% for DeepSeek-R1). Our work paves the way for future research on evaluating and improving LRMs in domain-specific complex reasoning tasks.
• 摘要：我们介绍了融资，这是一种新颖的基准，旨在评估财务数值推理问题中大型推理模型（LRMS）的推理能力。与现有基准相比，我们的工作提供了三个关键的进步。 （1）可信度：我们更新四个公共数据集中的问题的15.6％，以详细的Python解决方案注释908个新问题，并严格完善评估标准。这可以准确评估LRMS的推理改进。 （2）综合性：融资占财务概念和公式的67.8％，大大超过了现有数据集。此外，我们构建了3,133个Python-Formatted功能，可以通过精致的知识增强LRMS的财务推理能力（例如，GPT-4O的83.2％$ \ Rightarrow $ 91.6％）。 （3）挑战：需要模型将多个财务公式应用于238个严重问题上的精确数值推理。表现最佳的模型（即带有锅的Openai O1）的精度达到89.1％，但LRMS仍然面临数值精度的挑战。我们证明，将推理器和程序员模型组合起来可以有效地提高LRMS的性能（例如，DeepSeek-R1的83.2％$ \ Rightarrow $ 87.8％）。我们的工作为未来研究和改善域特异性复杂推理任务中的LRM的研究铺平了道路。

Title: Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models

• Authors: Cheonbok Park, Jeonghoon Kim, Joosung Lee, Sanghwan Bae, Jaegul Choo, Kangmin Yoo
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05850
• Pdf URL: https://arxiv.org/pdf/2506.05850
• Copy Paste: [[2506.05850]] Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models(https://arxiv.org/abs/2506.05850)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: We identify \textbf{Cross-lingual Collapse}, a systematic drift in which the chain-of-thought (CoT) of a multilingual language model reverts to its dominant pre-training language even when the prompt is expressed in a different language. Recent large language models (LLMs) with reinforcement learning with verifiable reward (RLVR) have achieved strong logical reasoning performances by exposing their intermediate reasoning traces, giving rise to large reasoning models (LRMs). However, the mechanism behind multilingual reasoning in LRMs is not yet fully explored. To investigate the issue, we fine-tune multilingual LRMs with Group-Relative Policy Optimization (GRPO) on translated versions of the GSM$8$K and SimpleRL-Zoo datasets in three different languages: Chinese, Korean, and Ukrainian. During training, we monitor both task accuracy and language consistency of the reasoning chains. Our experiments reveal three key findings: (i) GRPO rapidly amplifies pre-training language imbalances, leading to the erosion of low-resource languages within just a few hundred updates; (ii) language consistency reward mitigates this drift but does so at the expense of an almost 5 - 10 pp drop in accuracy. and (iii) the resulting language collapse is severely damaging and largely irreversible, as subsequent fine-tuning struggles to steer the model back toward its original target-language reasoning capabilities. Together, these findings point to a remarkable conclusion: \textit{not all languages are trained equally for reasoning}. Furthermore, our paper sheds light on the roles of reward shaping, data difficulty, and pre-training priors in eliciting multilingual reasoning.
• 摘要：我们识别\ textbf {跨语性崩溃}，这是一种系统的漂移，即即使以不同的语言表达提示，多语言语言模型的思考链（COT）也会恢复其主导的预训练语言。近期的大型语言模型（LLMS）通过增强学习和可验证的奖励（RLVR）通过公开其中间推理痕迹，从而实现了强大的逻辑推理性能，从而产生了大型推理模型（LRMS）。但是，LRMS中多语言推理背后的机制尚未得到充分探索。为了调查这个问题，我们对GSM $ 8 $ K的翻译版本和Simplerl-ZOO数据集的翻译版本使用三种不同语言：中文，韩文和乌克兰语进行了群体相关政策优化（GRPO）进行微调。在培训期间，我们监视推理链的任务准确性和语言一致性。我们的实验揭示了三个关键发现：（i）GRPO迅速放大了培训的语言失衡，从而导致仅几百个更新的低资源语言侵蚀； （ii）语言一致性奖励减轻了这种漂移，但以几乎5-10 pp的准确性下降了这种漂移。 （iii）由此产生的语言崩溃是严重破坏性的，并且在很大程度上是不可逆转的，因为随后的微调斗争将模型带回其最初的目标语言推理能力。这些发现一起指出了一个显着的结论：\ textit {并非所有语言都经过同等培训以推理}。此外，我们的论文阐明了奖励成型，数据困难和预训练的先验在引发多语言推理中的作用。

Title: Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Router

• Authors: Chenyang Shao, Xinyang Liu, Yutang Lin, Fengli Xu, Yong Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05901
• Pdf URL: https://arxiv.org/pdf/2506.05901
• Copy Paste: [[2506.05901]] Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Router(https://arxiv.org/abs/2506.05901)
• Keywords: language model, llm
• Abstract: Multi-step reasoning has proven essential for enhancing the problem-solving capabilities of Large Language Models (LLMs) by decomposing complex tasks into intermediate steps, either explicitly or implicitly. Extending the reasoning chain at test time through deeper thought processes or broader exploration, can furthur improve performance, but often incurs substantial costs due to the explosion in token usage. Yet, many reasoning steps are relatively simple and can be handled by more efficient smaller-scale language models (SLMs). This motivates hybrid approaches that allocate subtasks across models of varying capacities. However, realizing such collaboration requires accurate task decomposition and difficulty-aware subtask allocation, which is challenging. To address this, we propose R2-Reasoner, a novel framework that enables collaborative reasoning across heterogeneous LLMs by dynamically routing sub-tasks based on estimated complexity. At the core of our framework is a Reinforced Model Router, composed of a task decomposer and a subtask allocator. The task decomposer segments complex input queries into logically ordered subtasks, while the subtask allocator assigns each subtask to the most appropriate model, ranging from lightweight SLMs to powerful LLMs, balancing accuracy and efficiency. To train this router, we introduce a staged pipeline that combines supervised fine-tuning on task-specific datasets with Group Relative Policy Optimization algorithm, enabling self-supervised refinement through iterative reinforcement learning. Extensive experiments across four challenging benchmarks demonstrate that R2-Reasoner reduces API costs by 86.85% while maintaining or surpassing baseline accuracy. Our framework paves the way for more cost-effective and adaptive LLM reasoning. The code is open-source at this https URL .
• 摘要：通过将复杂的任务分解为中间步骤，无论是明确或隐式，多步推理对于增强大语言模型（LLMS）的解决问题的能力而言至关重要。通过更深入的思维过程或更广泛的探索扩展了测试时间的推理链，可以提高性能，但由于令牌使用情况的爆炸爆炸而经常会造成大量成本。但是，许多推理步骤相对简单，可以通过更有效的较小语言模型（SLM）来处理。这激发了混合方法，该方法分配了各种能力模型的子任务。但是，意识到这种协作需要准确的任务分解和难度 - 意识到子任务分配，这具有挑战性。为了解决这个问题，我们提出了R2-Reasoner，这是一个新颖的框架，可以通过基于估计的复杂性动态路由子任务来实现在异构LLMS之间进行协作推理。我们框架的核心是一个增强的模型路由器，该路由器由任务解码器和子任务分配器组成。任务分解器段复杂的输入查询中逻辑上有序的子任务，而子任务分配器将每个子任务分配给最合适的模型，从轻量级SLM到功能强大的LLM，平衡准确性和效率。为了训练该路由器，我们介绍了一条分阶段的管道，该管道将特定于任务的数据集的监督微调与小组相对策略优化算法相结合，从而通过迭代增强学习实现了自我监督的改进。在四个具有挑战性的基准测试中进行的广泛实验表明，R2-季季者在维持或超过基线准确性的同时，将API成本降低了86.85％。我们的框架为更具成本效益和自适应的LLM推理铺平了道路。该代码在此HTTPS URL上是开源的。

Title: Generating Grounded Responses to Counter Misinformation via Learning Efficient Fine-Grained Critiques

• Authors: Xiaofei Xu, Xiuzhen Zhang, Ke Deng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05924
• Pdf URL: https://arxiv.org/pdf/2506.05924
• Copy Paste: [[2506.05924]] Generating Grounded Responses to Counter Misinformation via Learning Efficient Fine-Grained Critiques(https://arxiv.org/abs/2506.05924)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Fake news and misinformation poses a significant threat to society, making efficient mitigation essential. However, manual fact-checking is costly and lacks scalability. Large Language Models (LLMs) offer promise in automating counter-response generation to mitigate misinformation, but a critical challenge lies in their tendency to hallucinate non-factual information. Existing models mainly rely on LLM self-feedback to reduce hallucination, but this approach is computationally expensive. In this paper, we propose MisMitiFact, Misinformation Mitigation grounded in Facts, an efficient framework for generating fact-grounded counter-responses at scale. MisMitiFact generates simple critique feedback to refine LLM outputs, ensuring responses are grounded in evidence. We develop lightweight, fine-grained critique models trained on data sourced from readily available fact-checking sites to identify and correct errors in key elements such as numerals, entities, and topics in LLM generations. Experiments show that MisMitiFact generates counter-responses of comparable quality to LLMs' self-feedback while using significantly smaller critique models. Importantly, it achieves ~5x increase in feedback generation throughput, making it highly suitable for cost-effective, large-scale misinformation mitigation. Code and LLM prompt templates are at this https URL.
• 摘要：假新闻和错误信息对社会构成了重大威胁，从而使有效的缓解措施至关重要。但是，手动事实检查是昂贵的，缺乏可扩展性。大型语言模型（LLMS）在自动化反响应生成以减轻错误信息方面提供了希望，但是一个关键的挑战在于它们倾向于幻觉非事实信息。现有模型主要依靠LLM自反馈来减少幻觉，但是这种方法在计算上很昂贵。在本文中，我们提出了基于事实的不主义，误导性缓解措施，这是一个有效的框架，用于在大规模上产生事实基础的反响应。不合情可及会产生简单的批评反馈，以完善LLM输出，以确保响应基于证据。我们开发了轻巧，细粒度的批评模型，该模型是根据来自可用的事实检查站点来培训的，以识别和纠正关键元素（例如数字，实体和LLM世代）中的错误。实验表明，不sistifact产生与LLMS自我反馈相当的反响应，同时使用明显较小的批评模型。重要的是，它的反馈生成吞吐量增加了约5倍，使其非常适合缓解经济高效的大规模错误信息。代码和LLM提示模板在此HTTPS URL处。

Title: MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models

• Authors: Jie Cao, Tianwei Lin, Hongyang He, Rolan Yan, Wenqiao Zhang, Juncheng Li, Dongping Zhang, Siliang Tang, Yueting Zhuang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05928
• Pdf URL: https://arxiv.org/pdf/2506.05928
• Copy Paste: [[2506.05928]] MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models(https://arxiv.org/abs/2506.05928)
• Keywords: language model, llm
• Abstract: Recent studies integrate Low-Rank Adaptation (LoRA) and Mixture-of-Experts (MoE) to further enhance the performance of parameter-efficient fine-tuning (PEFT) methods in Large Language Model (LLM) applications. Existing methods employ \emph{homogeneous} MoE-LoRA architectures composed of LoRA experts with either similar or identical structures and capacities. However, these approaches often suffer from representation collapse and expert load imbalance, which negatively impact the potential of LLMs. To address these challenges, we propose a \emph{heterogeneous} \textbf{Mixture-of-Adapters (MoA)} approach. This method dynamically integrates PEFT adapter experts with diverse structures, leveraging their complementary representational capabilities to foster expert specialization, thereby enhancing the effective transfer of pre-trained knowledge to downstream tasks. MoA supports two variants: \textbf{(i)} \textit{Soft MoA} achieves fine-grained integration by performing a weighted fusion of all expert outputs; \textbf{(ii)} \textit{Sparse MoA} activates adapter experts sparsely based on their contribution, achieving this with negligible performance degradation. Experimental results demonstrate that heterogeneous MoA outperforms homogeneous MoE-LoRA methods in both performance and parameter efficiency. Our project is available at this https URL.
• 摘要：最近的研究集成了低级适应性（LORA）和专家的混合物（MOE），以进一步增强大语言模型（LLM）应用中参数有效的微调（PEFT）方法的性能。现有方法采用\ emph {同质} Moe-lora体系结构，由Lora专家组成，具有相似或相同的结构和能力。但是，这些方法通常会遭受代表性崩溃和专家负荷失衡的困扰，从而对LLM的潜力产生负面影响。为了应对这些挑战，我们提出了一个\ emph {异质} \ textbf {混合 - 适配器（MOA）}方法。该方法将PEFT适配器专家与各种结构进行动态整合，利用其互补的代表性能力来培养专家专业化，从而增强了预训练的知识向下游任务的有效转移。 MOA支持两个变体：\ textbf {（i）} \ textit {soft moa}通过执行所有专家输出的加权融合来实现细粒度的整合； \ textbf {（ii）} \ textit {稀疏MOA}基于其贡献而稀疏地激活适配器专家，并以可忽略的性能降级来实现这一目标。实验结果表明，异质MOA在性能和参数效率方面均优于同质MOE-LORA方法。我们的项目可在此HTTPS URL上找到。

Title: DynamicMind: A Tri-Mode Thinking System for Large Language Models

• Authors: Wei Li, Yanbin Wei, Qiushi Huang, Jiangyue Yan, Yang Chen, James T. Kwok, Yu Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05936
• Pdf URL: https://arxiv.org/pdf/2506.05936
• Copy Paste: [[2506.05936]] DynamicMind: A Tri-Mode Thinking System for Large Language Models(https://arxiv.org/abs/2506.05936)
• Keywords: language model, llm, prompt
• Abstract: Modern large language models (LLMs) often struggle to dynamically adapt their reasoning depth to varying task complexities, leading to suboptimal performance or inefficient resource utilization. To address this, we introduce DynamicMind, a novel tri-mode thinking system. DynamicMind empowers LLMs to autonomously select between Fast, Normal, and Slow thinking modes for zero-shot question answering (ZSQA) tasks through cognitive-inspired prompt engineering. Our framework's core innovations include: (1) expanding the established dual-process framework of fast and slow thinking into a tri-mode thinking system involving a normal thinking mode to preserve the intrinsic capabilities of LLM; (2) proposing the Thinking Density metric, which aligns computational resource allocation with problem complexity; and (3) developing the Thinking Mode Capacity (TMC) dataset and a lightweight Mind Router to predict the optimal thinking mode. Extensive experiments across diverse mathematical, commonsense, and scientific QA benchmarks demonstrate that DynamicMind achieves superior ZSQA capabilities while establishing an effective trade-off between performance and computational efficiency.
• 摘要：现代大型语言模型（LLM）通常很难动态地使其推理深度适应不同的任务复杂性，从而导致次优性能或效率低下的资源利用。为了解决这个问题，我们介绍了一种新型的三模式思维系统DynamicMind。 DynamicMind授权LLMS通过认知启发的及时工程进行自主选择，以在快速，正常和缓慢的思维模式下进行零摄入问题答案（ZSQA）任务。我们的框架的核心创新包括：（1）将快速和缓慢思考的已建立的双处理框架扩展到涉及正常思维模式的三模式思维系统中，以保留LLM的内在功能； （2）提出思维密度度量，该度量度量将计算资源分配与问题复杂性保持一致； （3）开发思维模式能力（TMC）数据集和轻巧的思维路由器，以预测最佳的思维模式。跨多种数学，常识和科学质量检查基准进行的广泛实验表明，动态的实验可以达到卓越的ZSQA功能，同时在性能和计算效率之间建立了有效的权衡。

Title: IntentionESC: An Intention-Centered Framework for Enhancing Emotional Support in Dialogue Systems

• Authors: Xinjie Zhang, Wenxuan Wang, Qin Jin
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05947
• Pdf URL: https://arxiv.org/pdf/2506.05947
• Copy Paste: [[2506.05947]] IntentionESC: An Intention-Centered Framework for Enhancing Emotional Support in Dialogue Systems(https://arxiv.org/abs/2506.05947)
• Keywords: language model, llm, chain-of-thought
• Abstract: In emotional support conversations, unclear intentions can lead supporters to employ inappropriate strategies, inadvertently imposing their expectations or solutions on the seeker. Clearly defined intentions are essential for guiding both the supporter's motivations and the overall emotional support process. In this paper, we propose the Intention-centered Emotional Support Conversation (IntentionESC) framework, which defines the possible intentions of supporters in emotional support conversations, identifies key emotional state aspects for inferring these intentions, and maps them to appropriate support strategies. While Large Language Models (LLMs) excel in text generating, they fundamentally operate as probabilistic models trained on extensive datasets, lacking a true understanding of human thought processes and intentions. To address this limitation, we introduce the Intention Centric Chain-of-Thought (ICECoT) mechanism. ICECoT enables LLMs to mimic human reasoning by analyzing emotional states, inferring intentions, and selecting suitable support strategies, thereby generating more effective emotional support responses. To train the model with ICECoT and integrate expert knowledge, we design an automated annotation pipeline that produces high-quality training data. Furthermore, we develop a comprehensive evaluation scheme to assess emotional support efficacy and conduct extensive experiments to validate our framework. Our data and code are available at this https URL.
• 摘要：在情感支持对话中，不清楚的意图可以导致支持者采用不适当的策略，无意间将他们的期望或解决方案强加给寻求者。明确定义的意图对于指导支持者的动机和整体情感支持过程至关重要。在本文中，我们提出了以意图为中心的情感支持对话（Eletesc）框架，该框架定义了支持者在情感支持对话中的可能意图，确定了推断这些意图的关键情感状态方面，并将其映射到适当的支持策略。尽管大型语言模型（LLM）在文本生成中表现出色，但它们从根本上是在广泛的数据集中训练的概率模型，对人类思维过程和意图缺乏真正的了解。为了解决这一限制，我们介绍了以意图为中心的思想链（ICECOT）机制。 ICECOT使LLM可以通过分析情绪状态，推断意图和选择合适的支持策略来模仿人类的推理，从而产生更有效的情感支持反应。为了用ICECOT培训模型并整合专家知识，我们设计了一个自动注释管道，该管道可产生高质量的培训数据。此外，我们制定了一项全面的评估计划，以评估情绪支持功效并进行广泛的实验以验证我们的框架。我们的数据和代码可在此HTTPS URL上找到。

Title: Elementary Math Word Problem Generation using Large Language Models

• Authors: Nimesh Ariyarathne, Harshani Bandara, Yasith Heshan, Omega Gamage, Surangika Ranathunga, Dilan Nayanajith, Yutharsan Sivapalan, Gayathri Lihinikaduarachchi, Tharoosha Vihidun, Meenambika Chandirakumar, Sanujen Premakumar, Sanjula Gathsara
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05950
• Pdf URL: https://arxiv.org/pdf/2506.05950
• Copy Paste: [[2506.05950]] Elementary Math Word Problem Generation using Large Language Models(https://arxiv.org/abs/2506.05950)
• Keywords: language model, llm, prompt
• Abstract: Mathematics is often perceived as a complex subject by students, leading to high failure rates in exams. To improve Mathematics skills, it is important to provide sample questions for students to practice problem-solving. Manually creating Math Word Problems (MWPs) is time consuming for tutors, because they have to type in natural language while adhering to grammar and spelling rules of the language. Existing Deep Learning techniques for MWP generation either require a tutor to provide the initial portion of the MWP, and/or additional information such as an equation. In this paper, we present an MWP generation system based on Large Language Models (LLMs) that overcome the need for additional input - the only input to our system is the number of MWPs needed, the grade and the type of question (e.g. addition, subtraction). Unlike the existing LLM-based solutions for MWP generation, we carried out an extensive set of experiments involving different LLMs, prompting strategies, techniques to improve the diversity of questions, as well as techniques that employ human feedback to improve LLM performance. Human and automated evaluations confirmed that the generated MWPs are high in quality, with minimal spelling and grammar issues. However, LLMs still struggle to generate questions that adhere to the specified grade and question type requirements.
• 摘要：数学通常被学生视为一个复杂的学科，导致考试中的较高失败率。为了提高数学技能，重要的是为学生提供练习解决问题的样本问题。手动创建数学单词问题（MWP）对于导师来说很耗时，因为他们必须在遵守语法和语言的拼写规则的同时输入自然语言。 MWP生成的现有深度学习技术要么要求导师提供MWP的初始部分和/或其他信息，例如方程式。在本文中，我们提出了一个基于大语言模型（LLMS）的MWP生成系统，该系统克服了需要其他输入的需求 - 我们系统的唯一输入是所需的MWPS数量，等级和问题类型（例如，加法，减法）。与现有的MWP生成基于LLM的解决方案不同，我们进行了一系列涉及不同LLM的实验集，促使策略，提高问题多样性的技术以及采用人类反馈来改善LLM绩效的技术。人类和自动化的评估证实，产生的MWP质量很高，拼写和语法问题最少。但是，LLMS仍然很难产生符合指定等级和问题类型要求的问题。

Title: Let's Put Ourselves in Sally's Shoes: Shoes-of-Others Prefixing Improves Theory of Mind in Large Language Models

• Authors: Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Yoshihiro Yamazaki, Keita Suzuki, Hiroaki Sugiyama, Kuniko Saito
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.05970
• Pdf URL: https://arxiv.org/pdf/2506.05970
• Copy Paste: [[2506.05970]] Let's Put Ourselves in Sally's Shoes: Shoes-of-Others Prefixing Improves Theory of Mind in Large Language Models(https://arxiv.org/abs/2506.05970)
• Keywords: language model, llm
• Abstract: Recent studies have shown that Theory of Mind (ToM) in large language models (LLMs) has not reached human-level performance yet. Since fine-tuning LLMs on ToM datasets often degrades their generalization, several inference-time methods have been proposed to enhance ToM in LLMs. However, existing inference-time methods for ToM are specialized for inferring beliefs from contexts involving changes in the world state. In this study, we present a new inference-time method for ToM, Shoes-of-Others (SoO) prefixing, which makes fewer assumptions about contexts and is applicable to broader scenarios. SoO prefixing simply specifies the beginning of LLM outputs with ``Let's put ourselves in A's shoes.'', where A denotes the target character's name. We evaluate SoO prefixing on two benchmarks that assess ToM in conversational and narrative contexts without changes in the world state and find that it consistently improves ToM across five categories of mental states. Our analysis suggests that SoO prefixing elicits faithful thoughts, thereby improving the ToM performance.
• 摘要：最近的研究表明，大语言模型（LLM）中的心态（TOM）尚未达到人类水平的表现。由于TOM数据集上的微调LLM经常降低其概括，因此提出了几种推理时间方法来增强LLMS中的TOM。但是，汤姆的现有推理时间方法专门用于从涉及世界国家变化的环境中推断信念。在这项研究中，我们提出了一种针对汤姆（​​Tom）的新推理时间方法，他的鞋子（SOO）前缀，对上下文的假设更少，并且适用于更广泛的情况。 SOO前缀简单地指定了LLM输出的开头``让我们放在A鞋子里。'。我们在两个基准上评估了SOO的前缀，这些基准在没有变化的情况下评估对话和叙事环境中的TOM，并发现它始终如一地改善五种精神状态的TOM。我们的分析表明，Soo的前缀会引起忠实的思想，从而提高了Tom的表现。

Title: LTG at SemEval-2025 Task 10: Optimizing Context for Classification of Narrative Roles

• Authors: Egil Rønningstad, Gaurav Negi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05976
• Pdf URL: https://arxiv.org/pdf/2506.05976
• Copy Paste: [[2506.05976]] LTG at SemEval-2025 Task 10: Optimizing Context for Classification of Narrative Roles(https://arxiv.org/abs/2506.05976)
• Keywords: language model
• Abstract: Our contribution to the SemEval 2025 shared task 10, subtask 1 on entity framing, tackles the challenge of providing the necessary segments from longer documents as context for classification with a masked language model. We show that a simple entity-oriented heuristics for context selection can enable text classification using models with limited context window. Our context selection approach and the XLM-RoBERTa language model is on par with, or outperforms, Supervised Fine-Tuning with larger generative language models.
• 摘要：我们对Semeval 2025共享任务10的贡献，对实体框架的子任务1，应对从较长文档提供必要的细分市场的挑战，作为使用蒙版语言模型进行分类的上下文。我们表明，用于上下文选择的简单面向实体的启发式方法可以使用具有有限上下文窗口的模型来启用文本分类。我们的上下文选择方法和XLM-Roberta语言模型与更大的生成语言模型相当或胜过监督的微调。

Title: A Culturally-Rich Romanian NLP Dataset from "Who Wants to Be a Millionaire?" Videos

• Authors: Alexandru-Gabriel Ganea, Antonia-Adelina Popovici, Adrian-Marius Dumitran
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.05991
• Pdf URL: https://arxiv.org/pdf/2506.05991
• Copy Paste: [[2506.05991]] A Culturally-Rich Romanian NLP Dataset from "Who Wants to Be a Millionaire?" Videos(https://arxiv.org/abs/2506.05991)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) demonstrate varying performance across languages and cultural contexts. This study introduces a novel, culturally-rich, multilingual dataset derived from video recordings of the Romanian game show "Who Wants to Be a Millionaire?" (Vrei să fii Milionar?). We employed an innovative process combining optical character recognition (OCR), automated text extraction, and manual verification to collect question-answer pairs, enriching them with metadata including question domain (e.g., biology, history), cultural relevance (Romanian-specific vs. international), and difficulty. Benchmarking state-of-the-art LLMs, including Romanian-adapted models, on this dataset revealed significant performance disparities: models consistently achieve higher accuracy (80-95%) on international questions compared to Romanian-specific cultural questions (50-75%). We further investigate these differences through experiments involving machine translation of Romanian questions into English and cross-lingual tests using a comparable dataset in French. Our findings underscore the impact of cultural context and data source on LLM performance and offer practical insights for building robust, culturally-aware multilingual NLP systems, especially in educational domains. The dataset is publicly available at Hugging Face.
• 摘要：大型语言模型（LLMS）在语言和文化背景下展示了不同的表现。这项研究介绍了一个新颖，文化丰富的多语言数据集，该数据集源自罗马尼亚游戏节目的视频“谁想要成为百万富翁？” （vreisăfiimilionar？）。我们采用了一个创新的过程，结合了光学特征识别（OCR），自动化文本提取和手动验证来收集问题 - 答案对，并用元数据丰富了它们，包括问题域（例如，生物学，历史），文化相关性（罗马尼亚特定于特定的与国际）和困难。该数据集的基准测试最先进的LLM，包括罗马尼亚适应的模型，显示出明显的性能差异：与罗马尼亚特定的文化问题相比，在国际问题上，模型始终达到更高的准确性（80-95％）（50-75％）。我们通过使用法语中的可比数据集涉及将罗马尼亚问题转换为英语和跨语言测试的机器翻译到英语和跨语义测试的实验进一步研究这些差异。我们的发现强调了文化背景和数据源对LLM绩效的影响，并为建立强大的文化意识的多语言NLP系统提供了实用的见解，尤其是在教育领域。该数据集在拥抱脸上公开可用。

Title: Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models

• Authors: Peijie Liu, Fengli Xu, Yong Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.06008
• Pdf URL: https://arxiv.org/pdf/2506.06008
• Copy Paste: [[2506.06008]] Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models(https://arxiv.org/abs/2506.06008)
• Keywords: language model, llm, chain-of-thought
• Abstract: Chain-of-Thought (CoT) technique has proven effective in improving the performance of large language models (LLMs) on complex reasoning tasks. However, the performance gains are inconsistent across different tasks, and the underlying mechanism remains a long-standing research question. In this work, we make a preliminary observation that the monotonicity of token probability distributions may be correlated with the gains achieved through CoT reasoning. Leveraging this insight, we propose two indicators based on the token probability distribution to assess CoT effectiveness across different tasks. By combining instance-level indicators with logistic regression model, we introduce Dynamic CoT, a method that dynamically select between CoT and direct answer. Furthermore, we extend Dynamic CoT to closed-source models by transferring decision strategies learned from open-source models. Our indicators for assessing CoT effectiveness achieve an accuracy of 89.2\%, and Dynamic CoT reduces token consumption by more than 35\% while maintaining high accuracy. Overall, our work offers a novel perspective on the underlying mechanisms of CoT reasoning and provides a framework for its more efficient deployment.
• 摘要：事实证明，经过思考链（COT）技术有效地改善了大型语言模型（LLMS）在复杂的推理任务上的性能。但是，在不同的任务中，性能的增长是不一致的，而基本机制仍然是一个长期的研究问题。在这项工作中，我们进行了初步观察，即令牌概率分布的单调性可能与通过COT推理获得的收益相关。利用这种见解，我们提出了两个基于令牌概率分布的指标，以评估不同任务的COT有效性。通过将实例级指标与逻辑回归模型相结合，我们引入了动态COT，该方法在COT和直接答案之间动态选择。此外，我们通过转移从开源模型中学到的决策策略将动态COT扩展到封闭源模型。我们评估COT有效性的指标的准确性为89.2 \％，动态COT可将令牌消耗降低超过35 \％，同时保持高精度。总体而言，我们的工作提供了有关COT推理的基本机制的新颖观点，并为其更有效的部署提供了框架。

Title: Unlocking Recursive Thinking of LLMs: Alignment via Refinement

• Authors: Haoke Zhang, Xiaobo Liang, Cunxiang Wang, Juntao Li, Min Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.06009
• Pdf URL: https://arxiv.org/pdf/2506.06009
• Copy Paste: [[2506.06009]] Unlocking Recursive Thinking of LLMs: Alignment via Refinement(https://arxiv.org/abs/2506.06009)
• Keywords: language model, llm
• Abstract: The OpenAI o1-series models have demonstrated that leveraging long-form Chain of Thought (CoT) can substantially enhance performance. However, the recursive thinking capabilities of Large Language Models (LLMs) remain limited, particularly in the absence of expert-curated data for distillation. In this paper, we propose \textbf{AvR}: \textbf{Alignment via Refinement}, a novel method aimed at unlocking the potential of LLMs for recursive reasoning through long-form CoT. AvR introduces a refinement process that integrates criticism and improvement actions, guided by differentiable learning techniques to optimize \textbf{refinement-aware rewards}. As a result, the synthesized multi-round data can be organized as a long refinement thought, further enabling test-time scaling. Experimental results show that AvR significantly outperforms conventional preference optimization methods. Notably, with only 3k synthetic samples, our method boosts the performance of the LLaMA-3-8B-Instruct model by over 20\% in win rate on AlpacaEval 2.0. Our code is available at Github (this https URL).
• 摘要：OpenAI O1系列模型表明，利用长格式的思想链（COT）可以大大提高性能。但是，大语言模型（LLM）的递归思维能力仍然有限，尤其是在没有专家策划的蒸馏数据的情况下。在本文中，我们提出\ textbf {avr}：\ textbf {通过改进对齐}，这是一种旨在通过长形cot解锁递归推理的潜力的新方法。 AVR介绍了一个完善过程，该过程将批评和改进动作整合在一起，并在可区分的学习技术的指导下，以优化\ textbf {完善 - 意识到的奖励}。结果，可以将合成的多轮数据作为一种长期的完善思想来组织，从而进一步实现了测试时间缩放。实验结果表明，AVR明显胜过常规的偏好优化方法。值得注意的是，只有3K合成样本，我们的方法将羊膜-3-8B教学模型的性能提高了20 \％以上的alpacaeval 2.0的胜利率。我们的代码可在GitHub（此HTTPS URL）上找到。

Title: AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search

• Authors: Yu Li, Lehui Li, Zhihao Wu, Qingmin Liao, Jianye Hao, Kun Shao, Fengli Xu, Yong Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06017
• Pdf URL: https://arxiv.org/pdf/2506.06017
• Copy Paste: [[2506.06017]] AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search(https://arxiv.org/abs/2506.06017)
• Keywords: language model, llm, agent
• Abstract: Large language model (LLM) agents have demonstrated strong capabilities across diverse domains. However, designing high-performing agentic systems remains challenging. Existing agent search methods suffer from three major limitations: (1) an emphasis on optimizing agentic workflows while under-utilizing proven human-designed components such as memory, planning, and tool use; (2) high evaluation costs, as each newly generated agent must be fully evaluated on benchmarks; and (3) inefficient search in large search space. In this work, we introduce a comprehensive framework to address these challenges. First, We propose a hierarchical search space that jointly models agentic workflow and composable functional components, enabling richer agentic system designs. Building on this structured design space, we introduce a predictive value model that estimates agent performance given agentic system and task description, allowing for efficient, low-cost evaluation during the search process. Finally, we present a hierarchical Monte Carlo Tree Search (MCTS) strategy informed by uncertainty to guide the search. Experiments on seven benchmarks, covering embodied, math, web, tool, and game, show that our method achieves an average performance gain of 8.34\% over state-of-the-art baselines and exhibits faster search progress with steeper improvement trajectories. Code repo is available at this https URL.
• 摘要：大型语言模型（LLM）代理在不同领域表现出强大的功能。但是，设计高性能的代理系统仍然具有挑战性。现有的代理搜索方法具有三个主要局限性：（1）强调优化代理工作流程，同时未充分利用的人类设计的组件，例如内存，计划和工具使用； （2）高评估成本，因为每个新生成的代理必须在基准上进行全面评估； （3）在大型搜索空间中效率低下。在这项工作中，我们引入了一个综合框架来应对这些挑战。首先，我们提出了一个层次搜索空间，该搜索空间共同对代理工作流和可组合功能组件进行建模，从而实现更丰富的代理系统设计。在此结构化设计空间的基础上，我们引入了一个预测价值模型，该模型可以估算给定代理系统和任务描述的代理性能，从而在搜索过程中进行高效，低成本的评估。最后，我们提出了一个分层的蒙特卡洛树搜索（MCT）策略，该策略以不确定性为指导搜索而告知。涵盖体现，数学，网络，工具和游戏的七个基准测试的实验表明，我们的方法的平均性能增益比最先进的基线达到8.34 \％，并且具有更陡峭的改进轨迹，展示了更快的搜索进度。代码回购可在此HTTPS URL上找到。

Title: When to Trust Context: Self-Reflective Debates for Context Reliability

• Authors: Zeqi Zhou, Fang Wu, Shayan Talaei, Haokai Zhao, Cheng Meixin, Tinson Xu, Amin Saberi, Yejin Choi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.06020
• Pdf URL: https://arxiv.org/pdf/2506.06020
• Copy Paste: [[2506.06020]] When to Trust Context: Self-Reflective Debates for Context Reliability(https://arxiv.org/abs/2506.06020)
• Keywords: language model, hallucination, agent
• Abstract: Large language models frequently encounter conflicts between their parametric knowledge and contextual input, often resulting in factual inconsistencies or hallucinations. We propose Self-Reflective Debate for Contextual Reliability (SR-DCR), a lightweight framework that integrates token-level self-confidence with an asymmetric multi-agent debate to adjudicate such conflicts. A critic, deprived of context, challenges a defender who argues from the given passage; a judge model evaluates the debate and determines the context's reliability. The final answer is selected by combining the verdict with model confidence. Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently enhances robustness to misleading context while maintaining accuracy on trustworthy inputs, outperforming both classical debate and confidence-only baselines with minimal computational overhead. The code is available at this https URL.
• 摘要：大型语言模型经常在其参数知识和上下文输入之间遇到冲突，通常会导致事实上的不一致或幻觉。我们提出了关于上下文可靠性（SR-DCR）的自我反思辩论，这是一个轻巧的框架，将令牌级别的自信与不对称的多代理辩论集成在一起，以裁定此类冲突。批评家剥夺了背景，挑战了一名从给定的段落中争论的后卫。法官模型评估辩论并确定上下文的可靠性。最终答案是通过将判决与模型置信度相结合来选择的。在Clasheval基准上进行的实验表明，SR-DCR始终增强对误导性环境的鲁棒性，同时保持对可信赖的输入的准确性，超过经典的辩论和仅限置信基线的基准，并以最少的计算开销。该代码可在此HTTPS URL上找到。

Title: Large Language Models are Demonstration Pre-Selectors for Themselves

• Authors: Jiarui Jin, Yuwei Wu, Haoxuan Li, Xiaoting He, Weinan Zhang, Yiming Yang, Yong Yu, Jun Wang, Mengyue Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06033
• Pdf URL: https://arxiv.org/pdf/2506.06033
• Copy Paste: [[2506.06033]] Large Language Models are Demonstration Pre-Selectors for Themselves(https://arxiv.org/abs/2506.06033)
• Keywords: language model, llm
• Abstract: In-context learning (ICL) with large language models (LLMs) delivers strong few-shot performance by choosing few-shot demonstrations from the entire training data. However, existing ICL methods, which rely on similarity or diversity scores to choose demonstrations, incur high computational costs due to repeatedly retrieval from large-scale datasets for each query. To this end, we propose FEEDER (FEw yet Essential Demonstration prE-selectoR), a novel pre-selection framework that identifies a representative subset of demonstrations containing the most representative examples in the training data, tailored to specific LLMs. To construct this subset, we introduce the "sufficiency" and "necessity" metrics in the pre-selection stage and design a tree-based algorithm to identify representative examples efficiently. Once pre-selected, this representative subset can effectively replace the full training data, improving efficiency while maintaining comparable performance in ICL. Additionally, our pre-selected subset also benefits fine-tuning LLMs, where we introduce a bi-level optimization method that enhances training efficiency without sacrificing performance. Experiments with LLMs ranging from 300M to 8B parameters show that FEEDER can reduce training data size by over 20% while maintaining performance and seamlessly integrating with various downstream demonstration selection strategies in ICL.
• 摘要：具有大语言模型（LLMS）的文本学习（ICL），通过从整个培训数据中选择几次示范，从而提供了强劲的射击性能。但是，现有的ICL方法依靠相似性或多样性得分来选择演示，因此由于每个查询从大规模数据集中反复检索而产生了高计算成本。为此，我们提出了馈线（很少但必不可少的演示前选择器），这是一个新型的预选框架，它标识了示范的代表性子集，这些示例包含培训数据中最具代表性的例子，该示例是针对特定LLM量身定制的。为了构建此子集，我们在预选阶段介绍了“充分性”和“必要性”指标，并设计了基于树的算法，以有效地识别代表性示例。一旦预选，该代表性子集可以有效地替换完整的培训数据，从而提高效率，同时保持ICL的可比性。此外，我们的预选子集还有益于微调LLM，我们引入了双层优化方法，该方法可以提高训练效率而不牺牲性能。 LLMS的实验范围从300m到8B参数表明，馈线可以将训练数据大小降低20％以上，同时保持性能并与ICL中的各种下游演示选择策略无缝集成。

Title: MATP-BENCH: Can MLLM Be a Good Automated Theorem Prover for Multimodal Problems?

• Authors: Zhitao He, Zongwei Lyu, Dazhong Chen, Dadi Guo, Yi R. Fung
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06034
• Pdf URL: https://arxiv.org/pdf/2506.06034
• Copy Paste: [[2506.06034]] MATP-BENCH: Can MLLM Be a Good Automated Theorem Prover for Multimodal Problems?(https://arxiv.org/abs/2506.06034)
• Keywords: language model, llm
• Abstract: Numerous theorems, such as those in geometry, are often presented in multimodal forms (e.g., diagrams). Humans benefit from visual reasoning in such settings, using diagrams to gain intuition and guide the proof process. Modern Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in solving a wide range of mathematical problems. However, the potential of MLLMs as Automated Theorem Provers (ATPs), specifically in the multimodal domain, remains underexplored. In this paper, we introduce the Multimodal Automated Theorem Proving benchmark (MATP-BENCH), a new Multimodal, Multi-level, and Multi-language benchmark designed to evaluate MLLMs in this role as multimodal automated theorem provers. MATP-BENCH consists of 1056 multimodal theorems drawn from high school, university, and competition-level mathematics. All these multimodal problems are accompanied by formalizations in Lean 4, Coq and Isabelle, thus making the benchmark compatible with a wide range of theorem-proving frameworks. MATP-BENCH requires models to integrate sophisticated visual understanding with mastery of a broad spectrum of mathematical knowledge and rigorous symbolic reasoning to generate formal proofs. We use MATP-BENCH to evaluate a variety of advanced multimodal language models. Existing methods can only solve a limited number of the MATP-BENCH problems, indicating that this benchmark poses an open challenge for research on automated theorem proving.
• 摘要：许多定理（例如几何形状）通常以多模式形式（例如图表）呈现。在这种情况下，人类受益于视觉推理，使用图表获得直觉并指导证明过程。现代多模式大型语言模型（MLLM）在解决广泛的数学问题方面表现出了显着的功能。但是，MLLM作为自动定理抛弃（ATP）的潜力，特别是在多模式域中，仍然没有被忽视。在本文中，我们介绍了多模式的自动定理证明基准（MATP-Bench），这是一种新的多模式，多层次和多语言基准，旨在评估MLLM的多模式自动定理provers。 MATP板凳由1056个来自高中，大学和竞争级数学的多模式定理组成。所有这些多模式问题都伴随着精益4，Coq和Isabelle中的形式化，从而使基准测试与广泛的定理框架兼容。 MATP BENC需要模型将复杂的视觉理解与广泛的数学知识和严格的符号推理的精通整合在一起，以产生正式的证据。我们使用MATP基础来评估各种高级多模式模型。现有方法只能解决有限数量的MATP基础问题，这表明该基准对自动定理证明的研究构成了一个开放的挑战。

Title: Hey, That's My Data! Label-Only Dataset Inference in Large Language Models

• Authors: Chen Xiong, Zihao Wang, Rui Zhu, Tsung-Yi Ho, Pin-Yu Chen, Jingwei Xiong, Haixu Tang, Lucila Ohno-Machado
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.06057
• Pdf URL: https://arxiv.org/pdf/2506.06057
• Copy Paste: [[2506.06057]] Hey, That's My Data! Label-Only Dataset Inference in Large Language Models(https://arxiv.org/abs/2506.06057)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have revolutionized Natural Language Processing by excelling at interpreting, reasoning about, and generating human language. However, their reliance on large-scale, often proprietary datasets poses a critical challenge: unauthorized usage of such data can lead to copyright infringement and significant financial harm. Existing dataset-inference methods typically depend on log probabilities to detect suspicious training material, yet many leading LLMs have begun withholding or obfuscating these signals. This reality underscores the pressing need for label-only approaches capable of identifying dataset membership without relying on internal model logits. We address this gap by introducing CatShift, a label-only dataset-inference framework that capitalizes on catastrophic forgetting: the tendency of an LLM to overwrite previously learned knowledge when exposed to new data. If a suspicious dataset was previously seen by the model, fine-tuning on a portion of it triggers a pronounced post-tuning shift in the model's outputs; conversely, truly novel data elicits more modest changes. By comparing the model's output shifts for a suspicious dataset against those for a known non-member validation set, we statistically determine whether the suspicious set is likely to have been part of the model's original training corpus. Extensive experiments on both open-source and API-based LLMs validate CatShift's effectiveness in logit-inaccessible settings, offering a robust and practical solution for safeguarding proprietary data.
• 摘要：大型语言模型（LLM）通过出色地解释，推理和产生人类语言来彻底改变自然语言处理。但是，他们对大规模，通常专有数据集的依赖提出了一个关键的挑战：未经授权使用此类数据可能会导致版权侵权和严重的财务危害。现有的数据集 - 推断方法通常取决于对数概率检测可疑培训材料，但是许多领先的LLM已开始扣留或混淆这些信号。这个现实强调了能够识别数据集成员资格而无需依赖内部模型逻辑的仅标签方法的紧迫需求。我们通过引入CatShift来解决这一差距，这是一个仅标签的数据集推进框架，它利用了灾难性的遗忘：LLM倾向于在暴露于新数据时覆盖先前学到的知识的趋势。如果该模型以前看到一个可疑数据集，则对其部分进行微调触发了模型输出的明显后调整偏移。相反，真正新颖的数据引起了更多适度的变化。通过将可疑数据集的模型的输出变化与已知的非成员验证集的模型变化，我们可以从统计上确定可疑集是否可能是该模型原始培训语料库的一部分。开源和基于API的LLMS的广泛实验验证了CatShift在Logit-Inable-coss-coss-cers访问设置中的有效性，为保护专有数据提供了强大而实用的解决方案。

Title: Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models

• Authors: Yingqi Hu, Zhuo Zhang, Jingyuan Zhang, Lizhen Qu, Zenglin Xu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.06060
• Pdf URL: https://arxiv.org/pdf/2506.06060
• Copy Paste: [[2506.06060]] Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models(https://arxiv.org/abs/2506.06060)
• Keywords: language model, llm
• Abstract: Federated fine-tuning of large language models (FedLLMs) presents a promising approach for achieving strong model performance while preserving data privacy in sensitive domains. However, the inherent memorization ability of LLMs makes them vulnerable to training data extraction attacks. To investigate this risk, we introduce simple yet effective extraction attack algorithms specifically designed for FedLLMs. In contrast to prior "verbatim" extraction attacks, which assume access to fragments from all training data, our approach operates under a more realistic threat model, where the attacker only has access to a single client's data and aims to extract previously unseen personally identifiable information (PII) from other clients. This requires leveraging contextual prefixes held by the attacker to generalize across clients. To evaluate the effectiveness of our approaches, we propose two rigorous metrics-coverage rate and efficiency-and extend a real-world legal dataset with PII annotations aligned with CPIS, GDPR, and CCPA standards, achieving 89.9% human-verified precision. Experimental results show that our method can extract up to 56.57% of victim-exclusive PII, with "Address," "Birthday," and "Name" being the most vulnerable categories. Our findings underscore the pressing need for robust defense strategies and contribute a new benchmark and evaluation framework for future research in privacy-preserving federated learning.
• 摘要：大型语言模型（FedLlms）的联合微调为实现强大的模型性能提供了一种有希望的方法，同时保留了敏感域中的数据隐私。但是，LLMS的固有记忆能力使它们容易受到培训数据提取攻击的影响。为了调查这种风险，我们引入了专门为FedLlms设计的简单但有效的提取攻击算法。与先前的“逐字”提取攻击相反，该攻击假设从所有培训数据中访问片段，我们的方法在更现实的威胁模型下运行，攻击者只能访问单个客户的数据，并旨在从其他客户中提取以前看不见的个人身份信息（PII）。这就需要利用攻击者持有的上下文前缀来跨客户概括。为了评估我们的方法的有效性，我们提出了两个严格的指标覆盖率和效率，并扩展了一个现实世界中的法律数据集，其中PII注释与CPIS，GDPR和CCPA标准一致，可实现89.9％的人类验证精度。实验结果表明，我们的方法最多可以提取56.57％的受害者PII，其中“地址”，“生日”和“名称”是最脆弱的类别。我们的发现强调了对强大的防御策略的迫切需求，并为未来保护隐私的联合学习研究做出了新的基准和评估框架。

Title: Zero-Shot Detection of LLM-Generated Code via Approximated Task Conditioning

• Authors: Maor Ashkenazi, Ofir Brenner, Tal Furman Shohet, Eran Treister
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.06069
• Pdf URL: https://arxiv.org/pdf/2506.06069
• Copy Paste: [[2506.06069]] Zero-Shot Detection of LLM-Generated Code via Approximated Task Conditioning(https://arxiv.org/abs/2506.06069)
• Keywords: language model, llm, prompt
• Abstract: Detecting Large Language Model (LLM)-generated code is a growing challenge with implications for security, intellectual property, and academic integrity. We investigate the role of conditional probability distributions in improving zero-shot LLM-generated code detection, when considering both the code and the corresponding task prompt that generated it. Our key insight is that when evaluating the probability distribution of code tokens using an LLM, there is little difference between LLM-generated and human-written code. However, conditioning on the task reveals notable differences. This contrasts with natural language text, where differences exist even in the unconditional distributions. Leveraging this, we propose a novel zero-shot detection approach that approximates the original task used to generate a given code snippet and then evaluates token-level entropy under the approximated task conditioning (ATC). We further provide a mathematical intuition, contextualizing our method relative to previous approaches. ATC requires neither access to the generator LLM nor the original task prompts, making it practical for real-world applications. To the best of our knowledge, it achieves state-of-the-art results across benchmarks and generalizes across programming languages, including Python, CPP, and Java. Our findings highlight the importance of task-level conditioning for LLM-generated code detection. The supplementary materials and code are available at this https URL, including the dataset gathering implementation, to foster further research in this area.
• 摘要：检测大型语言模型（LLM）生成的代码是一个日益严重的挑战，对安全，知识产权和学术完整性产生了影响。在考虑代码和相应的任务提示提示时，我们研究条件概率分布在改善零击LM生成的代码检测中的作用。我们的关键见解是，在使用LLM评估代码令牌的概率分布时，LLM生成的代码和人工编写的代码之间几乎没有区别。但是，对任务的条件揭示了显着的差异。这与自然语言文本形成鲜明对比，即使在无条件分布中也存在差异。利用这一点，我们提出了一种新颖的零摄像检测方法，该方法近似于用于生成给定代码段的原始任务，然后在近似任务条件（ATC）下评估令牌级熵。我们进一步提供了数学直觉，相对于以前的方法将我们的方法背景而出。 ATC既不需要访问发电机LLM也不需要原始任务提示，从而使其适用于现实世界的应用程序。据我们所知，它可以在基准之间取得最新的结果，并在包括Python，CPP和Java在内的编程语言中概括。我们的发现突出了任务级调节对LLM生成的代码检测的重要性。此HTTPS URL（包括数据集收集实施）可在此领域提供进一步的研究，可在此HTTPS URL（包括数据集收集实施）中获得补充材料和代码。

Title: MIRIAD: Augmenting LLMs with millions of medical query-response pairs

• Authors: Qinyue Zheng, Salman Abdullah, Sam Rawal, Cyril Zakka, Sophie Ostmeier, Maximilian Purk, Eduardo Reis, Eric J. Topol, Jure Leskovec, Michael Moor
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06091
• Pdf URL: https://arxiv.org/pdf/2506.06091
• Copy Paste: [[2506.06091]] MIRIAD: Augmenting LLMs with millions of medical query-response pairs(https://arxiv.org/abs/2506.06091)
• Keywords: llm, hallucination, chat
• Abstract: LLMs are bound to transform healthcare with advanced decision support and flexible chat assistants. However, LLMs are prone to generate inaccurate medical content. To ground LLMs in high-quality medical knowledge, LLMs have been equipped with external knowledge via RAG, where unstructured medical knowledge is split into small text chunks that can be selectively retrieved and integrated into the LLMs context. Yet, existing RAG pipelines rely on raw, unstructured medical text, which can be noisy, uncurated and difficult for LLMs to effectively leverage. Systematic approaches to organize medical knowledge to best surface it to LLMs are generally lacking. To address these challenges, we introduce MIRIAD, a large-scale, curated corpus of 5,821,948 medical QA pairs, each rephrased from and grounded in a passage from peer-reviewed medical literature using a semi-automated pipeline combining LLM generation, filtering, grounding, and human annotation. Unlike prior medical corpora, which rely on unstructured text, MIRIAD encapsulates web-scale medical knowledge in an operationalized query-response format, which enables more targeted retrieval. Experiments on challenging medical QA benchmarks show that augmenting LLMs with MIRIAD improves accuracy up to 6.7% compared to unstructured RAG baselines with the same source corpus and with the same amount of retrieved text. Moreover, MIRIAD improved the ability of LLMs to detect medical hallucinations by 22.5 to 37% (increase in F1 score). We further introduce MIRIAD-Atlas, an interactive map of MIRIAD spanning 56 medical disciplines, enabling clinical users to visually explore, search, and refine medical knowledge. MIRIAD promises to unlock a wealth of down-stream applications, including medical information retrievers, enhanced RAG applications, and knowledge-grounded chat interfaces, which ultimately enables more reliable LLM applications in healthcare.
• 摘要：LLM必须通过高级决策支持和灵活的聊天助理来改变医疗保健。但是，LLM容易产生不准确的医疗内容。在高质量的医学知识中，LLMS通过抹布配备了外部知识，在那里，非结构化的医学知识被分为小文本块，可以选择性地检索并集成到LLMS上下文中。但是，现有的破布管道依赖于原始的，非结构化的医学文本，这可能是嘈杂的，未经许理的和难以有效利用的。通常缺乏组织医学知识以最好地表现为LLM的系统方法。为了应对这些挑战，我们介绍了Miriad，这是一个5,821,948个医学质量质量检查的大规模，精选的语料库，每个质量质量质量检查QA对，每个QA对，并在同行评审的医学文献中使用半自我访问的医学文献，使用半自我访问的管道结合LLM，过滤，接地和人类注释。与依靠非结构化文本的先前医学公司不同，Miriad以操作的查询响应格式封装了网络规模的医学知识，这可以实现更多的目标检索。关于挑战性医疗质量检查基准测试的实验表明，与具有相同源源的非结构化抹布基线相比，使用MiriAD的LLM提高了准确性高达6.7％，并且具有相同数量的文本。此外，Miriad提高了LLMS检测医学幻觉的能力22.5％至37％（增加了F1分数）。我们进一步介绍了Miriad-Atlas，这是一张跨越56个医疗学科的Miriad的互动图，使临床用户能够在视觉上探索，搜索和完善医学知识。 Miriad承诺将解锁大量下流应用程序，包括医疗信息检索器，增强的破布应用程序和知识接地的聊天接口，最终使医疗保健中更可靠的LLM应用程序。

Title: Reinforcing Code Generation: Improving Text-to-SQL with Execution-Based Learning

• Authors: Atharv Kulkarni, Vivek Srikumar
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06093
• Pdf URL: https://arxiv.org/pdf/2506.06093
• Copy Paste: [[2506.06093]] Reinforcing Code Generation: Improving Text-to-SQL with Execution-Based Learning(https://arxiv.org/abs/2506.06093)
• Keywords: language model, llm
• Abstract: In this work, we study the problem of code generation with a large language model (LLM), with a focus on generating SQL queries from natural language questions. We ask: Instead of using supervised fine tuning with text-code pairs, can we tune a model by having it interact with a database engine? We frame this problem as a reinforcement learning problem where the model receives execution-based feedback from the environment in the form of scalar rewards. These rewards penalize execution failures and assign positive values when a query returns a correct answer. We use the rewards within the Group Relative Policy Optimization (GRPO) framework. We use a tabular reasoning benchmark to test and evaluate our findings. We find that with only weak supervision in the form of question-answer pairs, RL-tuning improves the accuracy of model generated SQL code from 31.49 to 49.83 while reducing error percentage from 25.43% to 14.71%. This improvement allowed the model nearly match the performance performance to the larger SQLCoder-70B model. Our work demonstrates the potential of using execution-based feedback to improve symbolic reasoning capabilities of LLMs.
• 摘要：在这项工作中，我们研究了具有大语言模型（LLM）的代码生成问题，重点是从自然语言问题中产生SQL查询。我们问：我们可以通过与数据库引擎进行交互，而不是使用文本代码对的监督微调，而是可以调整模型吗？我们将这个问题框起来是一个强化学习问题，其中该模型以标量奖励的形式从环境中接收基于执行的反馈。这些奖励会惩罚执行失败并分配正值时，当查询返回正确的答案时。我们在组相对策略优化（GRPO）框架中使用奖励。我们使用表格推理基准测试和评估我们的发现。我们发现，只有以问答对的形式进行的弱监督，RL调整将模型生成的SQL代码的准确性从31.49提高到49.83，而将错误百分比从25.43％则减少到14.71％。该改进使该模型几乎将性能与较大的SQLCoder-70B模型匹配。我们的工作证明了使用基于执行的反馈来提高LLM的符号推理功能的潜力。

Title: Bridging the Gap: In-Context Learning for Modeling Human Disagreement

• Authors: Benedetta Muscato, Yue Li, Gizem Gezici, Zhixue Zhao, Fosca Giannotti
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06113
• Pdf URL: https://arxiv.org/pdf/2506.06113
• Copy Paste: [[2506.06113]] Bridging the Gap: In-Context Learning for Modeling Human Disagreement(https://arxiv.org/abs/2506.06113)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) have shown strong performance on NLP classification tasks. However, they typically rely on aggregated labels-often via majority voting-which can obscure the human disagreement inherent in subjective annotations. This study examines whether LLMs can capture multiple perspectives and reflect annotator disagreement in subjective tasks such as hate speech and offensive language detection. We use in-context learning (ICL) in zero-shot and few-shot settings, evaluating four open-source LLMs across three label modeling strategies: aggregated hard labels, and disaggregated hard and soft labels. In few-shot prompting, we assess demonstration selection methods based on textual similarity (BM25, PLM-based), annotation disagreement (entropy), a combined ranking, and example ordering strategies (random vs. curriculum-based). Results show that multi-perspective generation is viable in zero-shot settings, while few-shot setups often fail to capture the full spectrum of human judgments. Prompt design and demonstration selection notably affect performance, though example ordering has limited impact. These findings highlight the challenges of modeling subjectivity with LLMs and the importance of building more perspective-aware, socially intelligent models.
• 摘要：大型语言模型（LLMS）在NLP分类任务上表现出很强的性能。但是，他们通常依靠总数通过多数投票的汇总标签 - 这可能掩盖了主观注释中固有的人类分歧。这项研究检查了LLM是否可以捕获多种观点并反映了主观任务中的注释者分歧，例如仇恨言论和进攻性语言检测。我们在零射击和少量设置中使用秘密学习（ICL），评估了三种标签建模策略中的四个开源LLM：汇总的硬标签，并分解了硬和软标签。在少数发动的提示中，我们根据文本相似性（BM25，基于PLM），注释分歧（熵），联合排名和示例订购策略（基于随机与课程）评估演示选择方法。结果表明，在零拍设置中，多观点的生成是可行的，而很少有射击设置通常无法捕获人类的全部判断。迅速的设计和演示选择显着影响性能，尽管示例订购的影响有限。这些发现突出了用LLM建模主观性的挑战，以及建立更多观点，具有社会智能模型的重要性。

Title: Let's CONFER: A Dataset for Evaluating Natural Language Inference Models on CONditional InFERence and Presupposition

• Authors: Tara Azin, Daniel Dumitrescu, Diana Inkpen, Raj Singh
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06133
• Pdf URL: https://arxiv.org/pdf/2506.06133
• Copy Paste: [[2506.06133]] Let's CONFER: A Dataset for Evaluating Natural Language Inference Models on CONditional InFERence and Presupposition(https://arxiv.org/abs/2506.06133)
• Keywords: language model, gpt, llm, prompt
• Abstract: Natural Language Inference (NLI) is the task of determining whether a sentence pair represents entailment, contradiction, or a neutral relationship. While NLI models perform well on many inference tasks, their ability to handle fine-grained pragmatic inferences, particularly presupposition in conditionals, remains underexplored. In this study, we introduce CONFER, a novel dataset designed to evaluate how NLI models process inference in conditional sentences. We assess the performance of four NLI models, including two pre-trained models, to examine their generalization to conditional reasoning. Additionally, we evaluate Large Language Models (LLMs), including GPT-4o, LLaMA, Gemma, and DeepSeek-R1, in zero-shot and few-shot prompting settings to analyze their ability to infer presuppositions with and without prior context. Our findings indicate that NLI models struggle with presuppositional reasoning in conditionals, and fine-tuning on existing NLI datasets does not necessarily improve their performance.
• 摘要：自然语言推论（NLI）是确定句子对是否代表需要，矛盾或中立关系的任务。尽管NLI模型在许多推理任务上表现良好，但它们处理精细元素务实的推论，尤其是条件性的预设的能力仍然没有得到充实的态度。在这项研究中，我们介绍了Condry，这是一个新型数据集，旨在评估NLI模型如何处理条件句子中的推断。我们评估了四个NLI模型的性能，包括两个预训练模型，以检查其对条件推理的概括。此外，我们以零射门和少量促使设置来评估包括GPT-4O，Llama，Gemma和DeepSeek-R1在内的大型语言模型（LLMS），以分析其在有或没有先验上下文的情况下推断预设的能力。我们的发现表明，NLI模型在有条件的预设推理方面遇到了困难，并且对现有的NLI数据集进行微调并不一定会提高其性能。

Title: semantic-features: A User-Friendly Tool for Studying Contextual Word Embeddings in Interpretable Semantic Spaces

• Authors: Jwalanthi Ranganathan, Rohan Jha, Kanishka Misra, Kyle Mahowald
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.06169
• Pdf URL: https://arxiv.org/pdf/2506.06169
• Copy Paste: [[2506.06169]] semantic-features: A User-Friendly Tool for Studying Contextual Word Embeddings in Interpretable Semantic Spaces(https://arxiv.org/abs/2506.06169)
• Keywords: language model
• Abstract: We introduce semantic-features, an extensible, easy-to-use library based on Chronis et al. (2023) for studying contextualized word embeddings of LMs by projecting them into interpretable spaces. We apply this tool in an experiment where we measure the contextual effect of the choice of dative construction (prepositional or double object) on the semantic interpretation of utterances (Bresnan, 2007). Specifically, we test whether "London" in "I sent London the letter." is more likely to be interpreted as an animate referent (e.g., as the name of a person) than in "I sent the letter to London." To this end, we devise a dataset of 450 sentence pairs, one in each dative construction, with recipients being ambiguous with respect to person-hood vs. place-hood. By applying semantic-features, we show that the contextualized word embeddings of three masked language models show the expected sensitivities. This leaves us optimistic about the usefulness of our tool.
• 摘要：我们介绍了语义功能，这是一个基于Chronis等人的可扩展，易于使用的库。 （2023）用于研究LMS的上下文化单词嵌入，通过将其投影到可解释的空间中。我们在一个实验中应用了此工具，在该实验中，我们可以测量选择性结构（介词或双对象）对语音的语义解释的上下文效应（Bresnan，2007）。具体来说，我们在“我给伦敦寄来的信”中测试了“伦敦”。比“我给伦敦的信”中，更有可能被解释为动画指称人（例如，一个人的名字）。为此，我们设计了一个450个句子对的数据集，每个句子中有一个，接收者在人事与地方hood方方面是模棱两可的。通过应用语义功能，我们表明三种蒙版语言模型的上下文化词嵌入显示了预期的敏感性。这使我们对工具的实用性感到乐观。

Title: Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach

• Authors: James Ford, Anthony Rios
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06175
• Pdf URL: https://arxiv.org/pdf/2506.06175
• Copy Paste: [[2506.06175]] Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach(https://arxiv.org/abs/2506.06175)
• Keywords: language model, gpt, llm, hallucination, prompt, agent
• Abstract: Large language models can translate natural-language chart descriptions into runnable code, yet approximately 15\% of the generated scripts still fail to execute, even after supervised fine-tuning and reinforcement learning. We investigate whether this persistent error rate stems from model limitations or from reliance on a single-prompt design. To explore this, we propose a lightweight multi-agent pipeline that separates drafting, execution, repair, and judgment, using only an off-the-shelf GPT-4o-mini model. On the \textsc{Text2Chart31} benchmark, our system reduces execution errors to 4.5\% within three repair iterations, outperforming the strongest fine-tuned baseline by nearly 5 percentage points while requiring significantly less compute. Similar performance is observed on the \textsc{ChartX} benchmark, with an error rate of 4.6\%, demonstrating strong generalization. Under current benchmarks, execution success appears largely solved. However, manual review reveals that 6 out of 100 sampled charts contain hallucinations, and an LLM-based accessibility audit shows that only 33.3\% (\textsc{Text2Chart31}) and 7.2\% (\textsc{ChartX}) of generated charts satisfy basic colorblindness guidelines. These findings suggest that future work should shift focus from execution reliability toward improving chart aesthetics, semantic fidelity, and accessibility.
• 摘要：大型语言模型可以将自然语言图表描述转化为可运行的代码，但是即使经过监督的微调和强化学习，大约15％的生成脚本仍无法执行。我们调查了这种持续错误率是源于模型限制还是依赖单个prompt设计的依赖。为了探讨这一点，我们提出了一种轻巧的多机管道，该管道仅使用现成的GPT-4O-Mini模型将起草，执行，修复和判断分开。在\ textsc {text2chart31}基准上，我们的系统在三个维修迭代中将执行错误降低到4.5 \％，表现优于最强的微调基线，同时需要较小的计算。在\ textsc {ChartX}基准上观察到类似的性能，错误率为4.6 \％，显示出强烈的概括。在当前的基准下，执行成功似乎在很大程度上得到了解决。但是，手动审查显示，100个样品图表中有6个包含幻觉，基于LLM的可访问性审计显示，只有33.3 \％（\ textsc {text2chart31}）和7.2 \％（\ textsc {chartx}）的生成图表满足基本色素dugilelines。这些发现表明，未来的工作应将重点从执行可靠性转变为改善图表美学，语义保真度和可访问性。

Title: Detecting Voice Phishing with Precision: Fine-Tuning Small Language Models

• Authors: Ju Yong Sim, Seong Hwan Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06180
• Pdf URL: https://arxiv.org/pdf/2506.06180
• Copy Paste: [[2506.06180]] Detecting Voice Phishing with Precision: Fine-Tuning Small Language Models(https://arxiv.org/abs/2506.06180)
• Keywords: language model, gpt, prompt, chain-of-thought
• Abstract: We develop a voice phishing (VP) detector by fine-tuning Llama3, a representative open-source, small language model (LM). In the prompt, we provide carefully-designed VP evaluation criteria and apply the Chain-of-Thought (CoT) technique. To evaluate the robustness of LMs and highlight differences in their performance, we construct an adversarial test dataset that places the models under challenging conditions. Moreover, to address the lack of VP transcripts, we create transcripts by referencing existing or new types of VP techniques. We compare cases where evaluation criteria are included, the CoT technique is applied, or both are used together. In the experiment, our results show that the Llama3-8B model, fine-tuned with a dataset that includes a prompt with VP evaluation criteria, yields the best performance among small LMs and is comparable to that of a GPT-4-based VP detector. These findings indicate that incorporating human expert knowledge into the prompt is more effective than using the CoT technique for small LMs in VP detection.
• 摘要：我们通过微调Llama3（一种代表性的开源，小语言模型（LM）来开发语音网络钓鱼（VP）检测器。在提示中，我们提供了精心设计的VP评估标准，并运用了经过思考链（COT）技术。为了评估LMS的鲁棒性并突出其性能差异，我们构建了一个对抗性测试数据集，该数据集将模型置于具有挑战性的条件下。此外，为了解决缺乏副总裁成绩单，我们通过引用现有或新类型的VP技术来创建成绩单。我们比较包括评估标准，应用COT技术或一起使用的情况。在实验中，我们的结果表明，Llama3-8B模型的数据集进行了微调，该数据集包含带有VP评估标准的提示，可在小LMS中产生最佳性能，并且与GPT-4基于GPT-4的VP检测器的性能相当。这些发现表明，将人类专家知识纳入提示比在VP检测中使用COT技术更有效。

Title: Building Models of Neurological Language

• Authors: Henry Watkins
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.06208
• Pdf URL: https://arxiv.org/pdf/2506.06208
• Copy Paste: [[2506.06208]] Building Models of Neurological Language(https://arxiv.org/abs/2506.06208)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: This report documents the development and evaluation of domain-specific language models for neurology. Initially focused on building a bespoke model, the project adapted to rapid advances in open-source and commercial medical LLMs, shifting toward leveraging retrieval-augmented generation (RAG) and representational models for secure, local deployment. Key contributions include the creation of neurology-specific datasets (case reports, QA sets, textbook-derived data), tools for multi-word expression extraction, and graph-based analyses of medical terminology. The project also produced scripts and Docker containers for local hosting. Performance metrics and graph community results are reported, with future possible work open for multimodal models using open-source architectures like phi-4.
• 摘要：本报告记录了神经病学特定域语言模型的开发和评估。该项目最初专注于建立定制模型，适应开源和商业医疗LLM的快速进步，转向利用检索效果的生成（RAG）和代表性模型，以确保安全的本地部署。关键贡献包括创建特定于神经病学的数据集（案例报告，质量保证集，教科书衍生的数据），用于多词表达式提取的工具以及基于图形的医学术语分析。该项目还生产了用于本地托管的脚本和码头容器。报告了性能指标和图形社区结果，使用PHI-4等开源架构为多模式模型开放未来可能的工作。

Title: Can Theoretical Physics Research Benefit from Language Agents?

• Authors: Sirui Lu, Zhijing Jin, Terry Jingchen Zhang, Pavel Kos, J. Ignacio Cirac, Bernhard Schölkopf
• Subjects: cs.CL, cs.AI, math-ph, quant-ph
• Abstract URL: https://arxiv.org/abs/2506.06214
• Pdf URL: https://arxiv.org/pdf/2506.06214
• Copy Paste: [[2506.06214]] Can Theoretical Physics Research Benefit from Language Agents?(https://arxiv.org/abs/2506.06214)
• Keywords: language model, llm, agent
• Abstract: Large Language Models (LLMs) are rapidly advancing across diverse domains, yet their application in theoretical physics research is not yet mature. This position paper argues that LLM agents can potentially help accelerate theoretical, computational, and applied physics when properly integrated with domain knowledge and toolbox. We analyze current LLM capabilities for physics -- from mathematical reasoning to code generation -- identifying critical gaps in physical intuition, constraint satisfaction, and reliable reasoning. We envision future physics-specialized LLMs that could handle multimodal data, propose testable hypotheses, and design experiments. Realizing this vision requires addressing fundamental challenges: ensuring physical consistency, and developing robust verification methods. We call for collaborative efforts between physics and AI communities to help advance scientific discovery in physics.
• 摘要：大型语言模型（LLM）正在跨不同领域迅速发展，但它们在理论物理研究中的应用尚未成熟。该立场论文认为，当与域知识和工具箱正确整合时，LLM代理可以潜在地帮助加速理论，计算和应用物理。我们分析了当前的LLM物理功能 - 从数学推理到代码生成 - 确定物理直觉，约束满意度和可靠推理的关键差距。我们设想可以处理多模式数据，提出可检验的假设和设计实验的未来物理学专用LLM。意识到这一愿景需要应对基本挑战：确保身体一致性，并开发强大的验证方法。我们呼吁物理和人工智能社区之间的合作努力，以帮助进步物理学中的科学发现。

Title: Explaining Matters: Leveraging Definitions and Semantic Expansion for Sexism Detection

• Authors: Sahrish Khan, Arshad Jhumka, Gabriele Pergola
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06238
• Pdf URL: https://arxiv.org/pdf/2506.06238
• Copy Paste: [[2506.06238]] Explaining Matters: Leveraging Definitions and Semantic Expansion for Sexism Detection(https://arxiv.org/abs/2506.06238)
• Keywords: language model, prompt
• Abstract: The detection of sexism in online content remains an open problem, as harmful language disproportionately affects women and marginalized groups. While automated systems for sexism detection have been developed, they still face two key challenges: data sparsity and the nuanced nature of sexist language. Even in large, well-curated datasets like the Explainable Detection of Online Sexism (EDOS), severe class imbalance hinders model generalization. Additionally, the overlapping and ambiguous boundaries of fine-grained categories introduce substantial annotator disagreement, reflecting the difficulty of interpreting nuanced expressions of sexism. To address these challenges, we propose two prompt-based data augmentation techniques: Definition-based Data Augmentation (DDA), which leverages category-specific definitions to generate semantically-aligned synthetic examples, and Contextual Semantic Expansion (CSE), which targets systematic model errors by enriching examples with task-specific semantic features. To further improve reliability in fine-grained classification, we introduce an ensemble strategy that resolves prediction ties by aggregating complementary perspectives from multiple language models. Our experimental evaluation on the EDOS dataset demonstrates state-of-the-art performance across all tasks, with notable improvements of macro F1 by 1.5 points for binary classification (Task A) and 4.1 points for fine-grained classification (Task C).
• 摘要：在线内容中对性别歧视的发现仍然是一个空旷的问题，因为有害语言不成比例地影响妇女和边缘化群体。尽管已经开发了自动化的性别歧视系统，但它们仍然面临两个主要挑战：数据稀疏性和性别歧视语言的细微挑战。即使在大型，经过精心策划的数据集中，例如可解释的在线性别歧视（EDOS），严重的类不平衡阻碍了模型的概括。此外，细颗粒类别的重叠和模棱两可的界限引入了大量注释者分歧，反映了解释性别歧视的细微表情的困难。为了应对这些挑战，我们提出了两种基于及时的数据增强技术：基于定义的数据扩展（DDA），该技术利用特定于类别的定义生成语义分配的合成示例，以及上下文语义语义扩展（CSE），该示例（CSE）通过具有任务特定语义特征的系统模型来实现系统模型误差。为了进一步提高细粒分类的可靠性，我们引入了一种整体策略，该策略通过汇总来自多种语言模型的互补观点来解决预测关系。我们对EDOS数据集的实验评估展示了所有任务中最先进的性能，用于二进制分类的宏F1显着提高了1.5分（任务A）和4.1分，用于细粒度分类（任务C）。

Title: Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge

• Authors: Yi Sui, Chaozhuo Li, Chen Zhang, Dawei song, Qiuchi Li
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06240
• Pdf URL: https://arxiv.org/pdf/2506.06240
• Copy Paste: [[2506.06240]] Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge(https://arxiv.org/abs/2506.06240)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Retrieval-augmented generation (RAG) is a cost-effective approach to mitigate the hallucination of Large Language Models (LLMs) by incorporating the retrieved external knowledge into the generation process. However, external knowledge may conflict with the parametric knowledge of LLMs. Furthermore, current LLMs lack inherent mechanisms for resolving such knowledge conflicts, making traditional RAG methods suffer from degraded performance and stability. Thus, we propose a Dual-Stream Knowledge-Augmented Framework for Shared-Private Semantic Synergy (DSSP-RAG). Central to the framework is a novel approach that refines self-attention into a mixed-attention, distinguishing shared and private semantics for a controlled internal-external knowledge integration. To effectively facilitate DSSP in RAG, we further introduce an unsupervised hallucination detection method based on cognitive uncertainty, ensuring the necessity of introducing knowledge, and an Energy Quotient (EQ) based on attention difference matrices to reduce noise in the retrieved external knowledge. Extensive experiments on benchmark datasets show that DSSP-RAG can effectively resolve conflicts and enhance the complementarity of dual-stream knowledge, leading to superior performance over strong baselines.
• 摘要：检索增强的生成（RAG）是一种经济高效的方法，可以通过将检索到的外部知识纳入生成过程来减轻大语模型（LLMS）的幻觉。但是，外部知识可能与LLM的参数知识相抵触。此外，当前的LLM缺乏解决此类知识冲突的固有机制，使传统的抹布方法遭受了降解的性能和稳定性。因此，我们为共享私人语义协同作用（DSSP-rag）提出了一个双流知识调节框架。该框架的核心是一种新颖的方法，它将自我注意力完善为混合意见，区分共享和私人语义，以进行受控的内部知识整合。为了有效地促进抹布中的DSSP，我们进一步基于认知不确定性引入了一种无监督的幻觉检测方法，确保了基于注意力差矩阵的能量商（EQ），以减少检索到的外部知识中的噪声。基准数据集上的广泛实验表明，DSSP-rag可以有效地解决冲突并增强双流知识的互补性，从而导致优越的性能超过了强质基线。

Title: Cartridges: Lightweight and general-purpose long context representations via self-study

• Authors: Sabri Eyuboglu, Ryan Ehrlich, Simran Arora, Neel Guha, Dylan Zinsley, Emily Liu, Will Tennien, Atri Rudra, James Zou, Azalia Mirhoseini, Christopher Re
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.06266
• Pdf URL: https://arxiv.org/pdf/2506.06266
• Copy Paste: [[2506.06266]] Cartridges: Lightweight and general-purpose long context representations via self-study(https://arxiv.org/abs/2506.06266)
• Keywords: language model, long context, chat
• Abstract: Large language models are often used to answer queries grounded in large text corpora (e.g. codebases, legal documents, or chat histories) by placing the entire corpus in the context window and leveraging in-context learning (ICL). Although current models support contexts of 100K-1M tokens, this setup is costly to serve because the memory consumption of the KV cache scales with input length. We explore an alternative: training a smaller KV cache offline on each corpus. At inference time, we load this trained KV cache, which we call a Cartridge, and decode a response. Critically, the cost of training a Cartridge can be amortized across all the queries referencing the same corpus. However, we find that the naive approach of training the Cartridge with next-token prediction on the corpus is not competitive with ICL. Instead, we propose self-study, a training recipe in which we generate synthetic conversations about the corpus and train the Cartridge with a context-distillation objective. We find that Cartridges trained with self-study replicate the functionality of ICL, while being significantly cheaper to serve. On challenging long-context benchmarks, Cartridges trained with self-study match ICL performance while using 38.6x less memory and enabling 26.4x higher throughput. Self-study also extends the model's effective context length (e.g. from 128k to 484k tokens on MTOB) and surprisingly, leads to Cartridges that can be composed at inference time without retraining.
• 摘要：大型语言模型通常用于回答基于大型文本语料库（例如代码库，法律文档或聊天历史记录）的查询，并将整个语料库放置在上下文窗口中并利用封闭式学习（ICL）。尽管当前的模型支持100K-1M令牌的上下文，但由于内存消耗输入长度，此设置为昂贵。我们探索一种替代方案：在每个语料库上训练一个较小的KV缓存。在推理时，我们加载了这个训练有素的KV缓存，我们称之为墨盒，并解码了响应。至关重要的是，培训的成本可以在引用同一语料库的所有查询中摊销。但是，我们发现，对弹药筒的幼稚方法对语料库的下一步预测与ICL没有竞争力。取而代之的是，我们提出了自学，这是一种培训食谱，在该食谱中，我们在该食谱中进行了有关语料库的合成对话，并以上下文放弃目标训练墨盒。我们发现，经过自学训练的墨盒复制了ICL的功能，同时便宜得多。在挑战长篇小写基准测试方面，经过自学匹配ICL性能训练的墨盒，同时减少38.6倍的记忆力，并使吞吐量更高26.4倍。自我研究还将模型的有效上下文长度（例如128K上升到MTOB上的484K令牌），令人惊讶地导致可以在推理时间内组成而无需重新培训的墨盒。

Title: AdvSumm: Adversarial Training for Bias Mitigation in Text Summarization

• Authors: Mukur Gupta, Nikhil Reddy Varimalla, Nicholas Deas, Melanie Subbiah, Kathleen McKeown
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2506.06273
• Pdf URL: https://arxiv.org/pdf/2506.06273
• Copy Paste: [[2506.06273]] AdvSumm: Adversarial Training for Bias Mitigation in Text Summarization(https://arxiv.org/abs/2506.06273)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have achieved impressive performance in text summarization and are increasingly deployed in real-world applications. However, these systems often inherit associative and framing biases from pre-training data, leading to inappropriate or unfair outputs in downstream tasks. In this work, we present AdvSumm (Adversarial Summarization), a domain-agnostic training framework designed to mitigate bias in text summarization through improved generalization. Inspired by adversarial robustness, AdvSumm introduces a novel Perturber component that applies gradient-guided perturbations at the embedding level of Sequence-to-Sequence models, enhancing the model's robustness to input variations. We empirically demonstrate that AdvSumm effectively reduces different types of bias in summarization-specifically, name-nationality bias and political framing bias-without compromising summarization quality. Compared to standard transformers and data augmentation techniques like back-translation, AdvSumm achieves stronger bias mitigation performance across benchmark datasets.
• 摘要：大型语言模型（LLMS）在文本摘要中取得了令人印象深刻的性能，并越来越多地部署在现实世界应用程序中。但是，这些系统通常会从训练前数据中继承关联和框架偏见，从而导致下游任务中的不适当或不公平输出。在这项工作中，我们介绍了Advsumm（对抗性摘要），这是一个域形不足的培训框架，旨在通过改进的概括来减轻文本摘要的偏见。受对抗鲁棒性的启发，Advsumm引入了一种新型的Perturber组件，该组件在序列到序列模型的嵌入水平上应用梯度引导的扰动，从而增强了模型对输入变化的鲁棒性。我们从经验上证明，Advsumm在特定于摘要中有效地降低了不同类型的偏见，姓名 - 民族性偏见和政治框架偏见而没有损害摘要的质量。与标准的变压器和数据增强技术（如反向翻译）相比，Advsumm在基准数据集中实现了更强的偏置缓解性能。