2026-02-23

Title: Neural Synchrony Between Socially Interacting Language Models

• Authors: Zhining Zhang, Wentao Zhu, Chi Han, Yizhou Wang, Heng Ji
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.17815
• Pdf URL: https://arxiv.org/pdf/2602.17815
• Copy Paste: [[2602.17815]] Neural Synchrony Between Socially Interacting Language Models(https://arxiv.org/abs/2602.17815)
• Keywords: language model, llm
• Abstract: Neuroscience has uncovered a fundamental mechanism of our social nature: human brain activity becomes synchronized with others in many social contexts involving interaction. Traditionally, social minds have been regarded as an exclusive property of living beings. Although large language models (LLMs) are widely accepted as powerful approximations of human behavior, with multi-LLM system being extensively explored to enhance their capabilities, it remains controversial whether they can be meaningfully compared to human social minds. In this work, we explore neural synchrony between socially interacting LLMs as an empirical evidence for this debate. Specifically, we introduce neural synchrony during social simulations as a novel proxy for analyzing the sociality of LLMs at the representational level. Through carefully designed experiments, we demonstrate that it reliably reflects both social engagement and temporal alignment in their interactions. Our findings indicate that neural synchrony between LLMs is strongly correlated with their social performance, highlighting an important link between neural synchrony and the social behaviors of LLMs. Our work offers a new perspective to examine the "social minds" of LLMs, highlighting surprising parallels in the internal dynamics that underlie human and LLM social interaction.
• 摘要：神经科学揭示了我们社会本质的一个基本机制：在许多涉及互动的社会环境中，人脑活动与其他人的大脑活动变得同步。传统上，社会思想被认为是生物的专有财产。尽管大语言模型（LLM）被广泛认为是人类行为的强大近似，并且多 LLM 系统被广泛探索以增强其能力，但它们是否可以与人类社会思维进行有意义的比较仍然存在争议。在这项工作中，我们探索了社会互动的法学硕士之间的神经同步性，作为这场辩论的经验证据。具体来说，我们在社会模拟过程中引入神经同步，作为在表征层面分析法学硕士社会性的新代理。通过精心设计的实验，我们证明它可靠地反映了互动中的社会参与度和时间一致性。我们的研究结果表明，法学硕士之间的神经同步性与其社会表现密切相关，凸显了神经同步性与法学硕士的社会行为之间的重要联系。我们的工作提供了一个新的视角来审视法学硕士的“社交思维”，强调了人类和法学硕士社交互动背后的内部动力惊人的相似之处。

Title: On the scaling relationship between cloze probabilities and language model next-token prediction

• Authors: Cassandra L. Jacobs, Morgan Grobol
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.17848
• Pdf URL: https://arxiv.org/pdf/2602.17848
• Copy Paste: [[2602.17848]] On the scaling relationship between cloze probabilities and language model next-token prediction(https://arxiv.org/abs/2602.17848)
• Keywords: language model
• Abstract: Recent work has shown that larger language models have better predictive power for eye movement and reading time data. While even the best models under-allocate probability mass to human responses, larger models assign higher-quality estimates of next tokens and their likelihood of production in cloze data because they are less sensitive to lexical co-occurrence statistics while being better aligned semantically to human cloze responses. The results provide support for the claim that the greater memorization capacity of larger models helps them guess more semantically appropriate words, but makes them less sensitive to low-level information that is relevant for word recognition.
• 摘要：最近的研究表明，更大的语言模型对眼球运动和阅读时间数据具有更好的预测能力。虽然即使是最好的模型也不会充分分配人类反应的概率质量，但较大的模型会对下一个标记及其在完形填空数据中产生的可能性分配更高质量的估计，因为它们对词汇共现统计不太敏感，同时在语义上与人类完形填空反应更好地对齐。结果支持了这样一种说法，即较大模型的记忆能力更强，有助于它们猜测语义上更合适的单词，但使它们对与单词识别相关的低级信息不太敏感。

Title: Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

• Authors: Joschka Braun
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.17881
• Pdf URL: https://arxiv.org/pdf/2602.17881
• Copy Paste: [[2602.17881]] Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations(https://arxiv.org/abs/2602.17881)
• Keywords: language model, prompt
• Abstract: Steering vectors are a lightweight method for controlling language model behavior by adding a learned bias to the activations at inference time. Although effective on average, steering effect sizes vary across samples and are unreliable for many target behaviors. In my thesis, I investigate why steering reliability differs across behaviors and how it is impacted by steering vector training data. First, I find that higher cosine similarity between training activation differences predicts more reliable steering. Second, I observe that behavior datasets where positive and negative activations are better separated along the steering direction are more reliably steerable. Finally, steering vectors trained on different prompt variations are directionally distinct, yet perform similarly well and exhibit correlated efficacy across datasets. My findings suggest that steering vectors are unreliable when the latent target behavior representation is not effectively approximated by the linear steering direction. Taken together, these insights offer a practical diagnostic for steering unreliability and motivate the development of more robust steering methods that explicitly account for non-linear latent behavior representations.
• 摘要：引导向量是一种轻量级方法，用于通过在推理时向激活添加学习偏差来控制语言模型行为。尽管平均而言有效，但引导效果大小因样本而异，并且对于许多目标行为来说并不可靠。在我的论文中，我研究了为什么转向可靠性因行为而异，以及转向向量训练数据如何影响转向可靠性。首先，我发现训练激活差异之间的余弦相似度越高，预测转向就越可靠。其次，我观察到，积极和消极激活沿着转向方向更好地分离的行为数据集更可靠地可转向。最后，在不同提示变化上训练的转向向量在方向上是不同的，但表现相似且在数据集上表现出相关的功效。我的研究结果表明，当潜在目标行为表示不能通过线性转向方向有效近似时，转向向量是不可靠的。总而言之，这些见解为转向不可靠性提供了实用的诊断，并促进了更鲁棒的转向方法的开发，这些方法明确地考虑了非线性潜在行为表示。

Title: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions

• Authors: Raymond Li, Amirhossein Abaskohi, Chuyuan Li, Gabriel Murray, Giuseppe Carenini
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.17907
• Pdf URL: https://arxiv.org/pdf/2602.17907
• Copy Paste: [[2602.17907]] Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions(https://arxiv.org/abs/2602.17907)
• Keywords: language model, prompt
• Abstract: Traditional neural topic models are typically optimized by reconstructing the document's Bag-of-Words (BoW) representations, overlooking contextual information and struggling with data sparsity. In this work, we propose a novel approach to construct semantically-grounded soft label targets using Language Models (LMs) by projecting the next token probabilities, conditioned on a specialized prompt, onto a pre-defined vocabulary to obtain contextually enriched supervision signals. By training the topic models to reconstruct the soft labels using the LM hidden states, our method produces higher-quality topics that are more closely aligned with the underlying thematic structure of the corpus. Experiments on three datasets show that our method achieves substantial improvements in topic coherence, purity over existing baselines. Additionally, we also introduce a retrieval-based metric, which shows that our approach significantly outperforms existing methods in identifying semantically similar documents, highlighting its effectiveness for retrieval-oriented applications.
• 摘要：传统的神经主题模型通常通过重建文档的词袋（BoW）表示、忽略上下文信息并努力解决数据稀疏性来进行优化。在这项工作中，我们提出了一种使用语言模型（LM）构建基于语义的软标签目标的新颖方法，通过将基于专门提示的下一个标记概率投影到预定义的词汇上，以获得上下文丰富的监督信号。通过训练主题模型以使用 LM 隐藏状态重建软标签，我们的方法产生了与语料库的底层主题结构更紧密结合的更高质量的主题。对三个数据集的实验表明，我们的方法在主题连贯性和纯度方面比现有基线实现了实质性改进。此外，我们还引入了基于检索的指标，这表明我们的方法在识别语义相似文档方面显着优于现有方法，突出了其对于面向检索的应用程序的有效性。

Title: Analyzing LLM Instruction Optimization for Tabular Fact Verification

• Authors: Xiaotang Du, Giwon Hong, Wai-Chung Kwan, Rohit Saxena, Ivan Titov, Pasquale Minervini, Emily Allaway
• Subjects: cs.CL, cs.PL
• Abstract URL: https://arxiv.org/abs/2602.17937
• Pdf URL: https://arxiv.org/pdf/2602.17937
• Copy Paste: [[2602.17937]] Analyzing LLM Instruction Optimization for Tabular Fact Verification(https://arxiv.org/abs/2602.17937)
• Keywords: language model, llm, prompt, chain-of-thought, agent
• Abstract: Instruction optimization provides a lightweight, model-agnostic approach to enhancing the reasoning performance of large language models (LLMs). This paper presents the first systematic comparison of instruction optimization, based on the DSPy optimization framework, for tabular fact verification. We evaluate four out-of-the-box prompting techniques that cover both text-only prompting and code use: direct prediction, Chain-of-Thought (CoT), ReAct with SQL tools, and CodeAct with Python execution. We study three optimizers from the DSPy framework -- COPRO, MiPROv2, and SIMBA -- across four benchmarks and three model families. We find that instruction optimization consistently improves verification accuracy, with MiPROv2 yielding the most stable gains for CoT, and SIMBA providing the largest benefits for ReAct agents, particularly at larger model scales. Behavioral analyses reveal that SIMBA encourages more direct reasoning paths by applying heuristics, thereby improving numerical comparison abilities in CoT reasoning and helping avoid unnecessary tool calls in ReAct agents. Across different prompting techniques, CoT remains effective for tabular fact checking, especially with smaller models. Although ReAct agents built with larger models can achieve competitive performance, they require careful instruction optimization.
• 摘要：指令优化提供了一种轻量级、与模型无关的方法来增强大型语言模型 (LLM) 的推理性能。本文提出了基于 DSPy 优化框架的指令优化的首次系统比较，以进行表格事实验证。我们评估了四种开箱即用的提示技术，涵盖纯文本提示和代码使用：直接预测、思想链 (CoT)、使用 SQL 工具的 ReAct 以及使用 Python 执行的 CodeAct。我们研究了 DSPy 框架中的三个优化器——COPRO、MiPROv2 和 SIMBA——跨越四个基准和三个模型系列。我们发现指令优化持续提高了验证准确性，MiPROv2 为 CoT 带来了最稳定的增益，而 SIMBA 为 ReAct 代理提供了最大的好处，特别是在较大的模型规模下。行为分析表明，SIMBA 通过应用启发式方法鼓励更直接的推理路径，从而提高 CoT 推理中的数值比较能力，并有助于避免 ReAct 代理中不必要的工具调用。在不同的提示技术中，CoT 对于表格事实检查仍然有效，尤其是对于较小的模型。尽管用更大的模型构建的 ReAct 代理可以实现有竞争力的性能，但它们需要仔细的指令优化。

Title: CUICurate: A GraphRAG-based Framework for Automated Clinical Concept Curation for NLP applications

• Authors: Victoria Blake, Mathew Miller, Jamie Novak, Sze-yuan Ooi, Blanca Gallego
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.17949
• Pdf URL: https://arxiv.org/pdf/2602.17949
• Copy Paste: [[2602.17949]] CUICurate: A GraphRAG-based Framework for Automated Clinical Concept Curation for NLP applications(https://arxiv.org/abs/2602.17949)
• Keywords: language model, gpt, llm, retrieval-augmented generation
• Abstract: Background: Clinical named entity recognition tools commonly map free text to Unified Medical Language System (UMLS) Concept Unique Identifiers (CUIs). For many downstream tasks, however, the clinically meaningful unit is not a single CUI but a concept set comprising related synonyms, subtypes, and supertypes. Constructing such concept sets is labour-intensive, inconsistently performed, and poorly supported by existing tools, particularly for NLP pipelines that operate directly on UMLS CUIs. Methods We present CUICurate, a Graph-based retrieval-augmented generation (GraphRAG) framework for automated UMLS concept set curation. A UMLS knowledge graph (KG) was constructed and embedded for semantic retrieval. For each target concept, candidate CUIs were retrieved from the KG, followed by large language model (LLM) filtering and classification steps comparing two LLMs (GPT-5 and GPT-5-mini). The framework was evaluated on five lexically heterogeneous clinical concepts against a manually curated benchmark and gold-standard concept sets. Results Across all concepts, CUICurate produced substantially larger and more complete concept sets than the manual benchmarks whilst matching human precision. Comparisons between the two LLMs found that GPT-5-mini achieved higher recall during filtering, while GPT-5 produced classifications that more closely aligned with clinician judgements. Outputs were stable across repeated runs and computationally inexpensive. Conclusions CUICurate offers a scalable and reproducible approach to support UMLS concept set curation that substantially reduces manual effort. By integrating graph-based retrieval with LLM reasoning, the framework produces focused candidate concept sets that can be adapted to clinical NLP pipelines for different phenotyping and analytic requirements.
• 摘要：背景：临床命名实体识别工具通常将自由文本映射到统一医学语言系统 (UMLS) 概念唯一标识符 (CUI)。然而，对于许多下游任务，具有临床意义的单元不是单个 CUI，而是由相关同义词、子类型和超类型组成的概念集。构建此类概念集需要大量劳动力，执行不一致，并且现有工具的支持很差，特别是对于直接在 UMLS CUI 上运行的 NLP 管道。方法 我们提出了 CUICurate，一个基于图的检索增强生成 (GraphRAG) 框架，用于自动化 UMLS 概念集管理。构建并嵌入了 UMLS 知识图（KG）以进行语义检索。对于每个目标概念，从 KG 中检索候选 CUI，然后进行大型语言模型 (LLM) 过滤和分类步骤，比较两个 LLM（GPT-5 和 GPT-5-mini）。该框架根据手动策划的基准和黄金标准概念集对五个词汇异质临床概念进行了评估。结果 在所有概念中，CUICurate 生成了比手动基准更大、更完整的概念集，同时符合人类精度。两个法学硕士之间的比较发现，GPT-5-mini 在过滤过程中实现了更高的召回率，而 GPT-5 产生的分类更符合临床医生的判断。重复运行后输出稳定且计算成本低廉。结论 CUICurate 提供了一种可扩展且可重复的方法来支持 UMLS 概念集管理，从而大大减少了手动工作量。通过将基于图的检索与法学硕士推理相结合，该框架生成了有针对性的候选概念集，这些概念集可以适应临床 NLP 流程，以满足不同的表型和分析要求。

Title: Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering

• Authors: Amine Kobeissi, Philippe Langlais
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2602.17981
• Pdf URL: https://arxiv.org/pdf/2602.17981
• Copy Paste: [[2602.17981]] Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering(https://arxiv.org/abs/2602.17981)
• Keywords: retrieval-augmented generation
• Abstract: Retrieval-augmented generation is increasingly used for financial question answering over long regulatory filings, yet reliability depends on retrieving the exact context needed to justify answers in high stakes settings. We study a frequent failure mode in which the correct document is retrieved but the page or chunk that contains the answer is missed, leading the generator to extrapolate from incomplete context. Despite its practical significance, this within-document retrieval failure mode has received limited systematic attention in the Financial Question Answering (QA) literature. We evaluate retrieval at multiple levels of granularity, document, page, and chunk level, and introduce an oracle based analysis to provide empirical upper bounds on retrieval and generative performance. On a 150 question subset of FinanceBench, we reproduce and compare diverse retrieval strategies including dense, sparse, hybrid, and hierarchical methods with reranking and query reformulation. Across methods, gains in document discovery tend to translate into stronger page recall, yet oracle performance still suggests headroom for page and chunk level retrieval. To target this gap, we introduce a domain fine-tuned page scorer that treats pages as an intermediate retrieval unit between documents and chunks. Unlike prior passage-based hierarchical retrieval, we fine-tune a bi-encoder specifically for page-level relevance on financial filings, exploiting the semantic coherence of pages. Overall, our results demonstrate a significant improvement in page recall and chunk retrieval.
• 摘要：检索增强生成越来越多地用于回答冗长的监管文件中的财务问题，但可靠性取决于检索在高风险环境中证明答案合理所需的确切上下文。我们研究了一种常见的失败模式，其中检索到了正确的文档，但包含答案的页面或块被遗漏，导致生成器从不完整的上下文中进行推断。尽管具有实际意义，但这种文档内检索失败模式在金融问答 (QA) 文献中受到的系统关注有限。我们在粒度、文档、页面和块级别的多个级别上评估检索，并引入基于预言机的分析来提供检索和生成性能的经验上限。在 FinanceBench 的 150 个问题子集上，我们重现并比较了不同的检索策略，包括密集、稀疏、混合和分层方法以及重新排序和查询重构。在各种方法中，文档发现的收益往往会转化为更强的页面回忆，但预言机性能仍然表明页面和块级检索有空间。为了解决这一差距，我们引入了一个域微调页面评分器，它将页面视为文档和块之间的中间检索单元。与之前基于段落的分层检索不同，我们专门针对财务文件的页面级相关性微调双编码器，利用页面的语义一致性。总的来说，我们的结果表明页面召回和块检索方面有了显着的改进。

Title: Towards More Standardized AI Evaluation: From Models to Agents

• Authors: Ali El Filali, Inès Bedar
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.18029
• Pdf URL: https://arxiv.org/pdf/2602.18029
• Copy Paste: [[2602.18029]] Towards More Standardized AI Evaluation: From Models to Agents(https://arxiv.org/abs/2602.18029)
• Keywords: agent
• Abstract: Evaluation is no longer a final checkpoint in the machine learning lifecycle. As AI systems evolve from static models to compound, tool-using agents, evaluation becomes a core control function. The question is no longer "How good is the model?" but "Can we trust the system to behave as intended, under change, at scale?". Yet most evaluation practices remain anchored in assumptions inherited from the model-centric era: static benchmarks, aggregate scores, and one-off success criteria. This paper argues that such approaches are increasingly obscure rather than illuminating system behavior. We examine how evaluation pipelines themselves introduce silent failure modes, why high benchmark scores routinely mislead teams, and how agentic systems fundamentally alter the meaning of performance measurement. Rather than proposing new metrics or harder benchmarks, we aim to clarify the role of evaluation in the AI era, and especially for agents: not as performance theater, but as a measurement discipline that conditions trust, iteration, and governance in non-deterministic systems.
• 摘要：评估不再是机器学习生命周期中的最终检查点。随着人工智能系统从静态模型发展到复合的、使用工具的代理，评估成为核心控制功能。问题不再是“模型有多好？”而是“我们能否相信系统在变化的情况下、大规模地按预期运行？”。然而，大多数评估实践仍然植根于从以模型为中心的时代继承下来的假设：静态基准、总分和一次性成功标准。本文认为，此类方法越来越晦涩难懂，而不是阐明系统行为。我们研究评估管道本身如何引入无声故障模式，为什么高基准分数经常误导团队，以及代理系统如何从根本上改变绩效衡量的含义。我们的目标不是提出新的指标或更难的基准，而是澄清评估在人工智能时代的作用，特别是对于智能体而言：不是作为表演剧场，而是作为一种衡量非确定性系统中的信任、迭代和治理的规范。

Title: Perceived Political Bias in LLMs Reduces Persuasive Abilities

• Authors: Matthew DiGiuseppe, Joshua Robison
• Subjects: cs.CL, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2602.18092
• Pdf URL: https://arxiv.org/pdf/2602.18092
• Copy Paste: [[2602.18092]] Perceived Political Bias in LLMs Reduces Persuasive Abilities(https://arxiv.org/abs/2602.18092)
• Keywords: gpt, llm, chat
• Abstract: Conversational AI has been proposed as a scalable way to correct public misconceptions and spread misinformation. Yet its effectiveness may depend on perceptions of its political neutrality. As LLMs enter partisan conflict, elites increasingly portray them as ideologically aligned. We test whether these credibility attacks reduce LLM-based persuasion. In a preregistered U.S. survey experiment (N=2144), participants completed a three-round conversation with ChatGPT about a personally held economic policy misconception. Compared to a neutral control, a short message indicating that the LLM was biased against the respondent's party attenuated persuasion by 28%. Transcript analysis indicates that the warnings alter the interaction: respondents push back more and engage less receptively. These findings suggest that the persuasive impact of conversational AI is politically contingent, constrained by perceptions of partisan alignment.
• 摘要：对话式人工智能被提议作为一种可扩展的方式来纠正公众的误解和传播错误信息。然而，其有效性可能取决于对其政治中立性的看法。随着法学硕士陷入党派冲突，精英们越来越将他们描绘成意识形态一致的人。我们测试这些可信度攻击是否会降低基于 LLM 的说服力。在一项预先注册的美国调查实验（N=2144）中，参与者与 ChatGPT 完成了三轮对话，讨论了个人对经济政策的误解。与中立对照相比，一条表明 LLM 对受访者一方有偏见的短信使说服力减弱了 28%。文字记录分析表明，警告改变了互动：受访者更多地抵制，更少地接受。这些发现表明，对话式人工智能的说服力影响在政治上是偶然的，受到党派联盟看法的限制。

Title: Agentic Adversarial QA for Improving Domain-Specific LLMs

• Authors: Vincent Grari, Ciprian Tomoiaga, Sylvain Lamprier, Tatsunori Hashimoto, Marcin Detyniecki
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.18137
• Pdf URL: https://arxiv.org/pdf/2602.18137
• Copy Paste: [[2602.18137]] Agentic Adversarial QA for Improving Domain-Specific LLMs(https://arxiv.org/abs/2602.18137)
• Keywords: language model, llm, agent
• Abstract: Large Language Models (LLMs), despite extensive pretraining on broad internet corpora, often struggle to adapt effectively to specialized domains. There is growing interest in fine-tuning these models for such domains; however, progress is constrained by the scarcity and limited coverage of high-quality, task-relevant data. To address this, synthetic data generation methods such as paraphrasing or knowledge extraction are commonly applied. Although these approaches excel at factual recall and conceptual knowledge, they suffer from two critical shortcomings: (i) they provide minimal support for interpretive reasoning capabilities in these specialized domains, and (ii) they often produce synthetic corpora that are excessively large and redundant, resulting in poor sample efficiency. To overcome these gaps, we propose an adversarial question-generation framework that produces a compact set of semantically challenging questions. These questions are constructed by comparing the outputs of the model to be adapted and a robust expert model grounded in reference documents, using an iterative, feedback-driven process designed to reveal and address comprehension gaps. Evaluation on specialized subsets of the LegalBench corpus demonstrates that our method achieves greater accuracy with substantially fewer synthetic samples.
• 摘要：尽管在广泛的互联网语料库上进行了广泛的预训练，大型语言模型（LLM）通常难以有效地适应专业领域。人们越来越有兴趣针对这些领域微调这些模型；然而，与任务相关的高质量数据稀缺且覆盖面有限，进展受到限制。为了解决这个问题，通常应用合成数据生成方法，例如释义或知识提取。尽管这些方法在事实回忆和概念知识方面表现出色，但它们存在两个关键缺点：（i）它们对这些专业领域的解释推理能力提供的支持很少，（ii）它们经常产生过大且冗余的合成语料库，导致样本效率低下。为了克服这些差距，我们提出了一个对抗性问题生成框架，该框架可以生成一组紧凑的语义上具有挑战性的问题。这些问题是通过比较要调整的模型的输出和基于参考文档的强大专家模型的输出而构建的，使用旨在揭示和解决理解差距的迭代、反馈驱动的过程。对 LegalBench 语料库的专门子集的评估表明，我们的方法可以用更少的合成样本实现更高的准确性。

Title: Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention

• Authors: Siya Qi, Yudong Chen, Runcong Zhao, Qinglin Zhu, Zhanghao Hu, Wei Liu, Yulan He, Zheng Yuan, Lin Gui
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.18145
• Pdf URL: https://arxiv.org/pdf/2602.18145
• Copy Paste: [[2602.18145]] Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention(https://arxiv.org/abs/2602.18145)
• Keywords: language model, llm, hallucination
• Abstract: Hallucination detection is critical for ensuring the reliability of large language models (LLMs) in context-based generation. Prior work has explored intrinsic signals available during generation, among which attention offers a direct view of grounding behavior. However, existing approaches typically rely on coarse summaries that fail to capture fine-grained instabilities in attention. Inspired by signal processing, we introduce a frequency-aware perspective on attention by analyzing its variation during generation. We model attention distributions as discrete signals and extract high-frequency components that reflect rapid local changes in attention. Our analysis reveals that hallucinated tokens are associated with high-frequency attention energy, reflecting fragmented and unstable grounding behavior. Based on this insight, we develop a lightweight hallucination detector using high-frequency attention features. Experiments on the RAGTruth and HalluRAG benchmarks show that our approach achieves performance gains over verification-based, internal-representation-based, and attention-based methods across models and tasks.
• 摘要：幻觉检测对于确保基于上下文的生成中大型语言模型 (LLM) 的可靠性至关重要。先前的工作已经探索了生成过程中可用的内在信号，其中注意力提供了接地行为的直接视图。然而，现有的方法通常依赖于粗略的摘要，无法捕获注意力中细粒度的不稳定性。受信号处理的启发，我们通过分析注意力在生成过程中的变化，引入了注意力的频率感知视角。我们将注意力分布建模为离散信号，并提取反映注意力快速局部变化的高频分量。我们的分析表明，幻觉令牌与高频注意力能量相关，反映了碎片化和不稳定的接地行为。基于这一见解，我们开发了一种使用高频注意力特征的轻量级幻觉检测器。 RAGTruth 和 HalluRAG 基准测试表明，我们的方法在模型和任务中比基于验证、基于内部表示和基于注意力的方法实现了性能提升。

Title: The Statistical Signature of LLMs

• Authors: Ortal Hadad, Edoardo Loru, Jacopo Nudo, Niccolò Di Marco, Matteo Cinelli, Walter Quattrociocchi
• Subjects: cs.CL, cs.CY, physics.soc-ph
• Abstract URL: https://arxiv.org/abs/2602.18152
• Pdf URL: https://arxiv.org/pdf/2602.18152
• Copy Paste: [[2602.18152]] The Statistical Signature of LLMs(https://arxiv.org/abs/2602.18152)
• Keywords: language model, llm
• Abstract: Large language models generate text through probabilistic sampling from high-dimensional distributions, yet how this process reshapes the structural statistical organization of language remains incompletely characterized. Here we show that lossless compression provides a simple, model-agnostic measure of statistical regularity that differentiates generative regimes directly from surface text. We analyze compression behavior across three progressively more complex information ecosystems: controlled human-LLM continuations, generative mediation of a knowledge infrastructure (Wikipedia vs. Grokipedia), and fully synthetic social interaction environments (Moltbook vs. Reddit). Across settings, compression reveals a persistent structural signature of probabilistic generation. In controlled and mediated contexts, LLM-produced language exhibits higher structural regularity and compressibility than human-written text, consistent with a concentration of output within highly recurrent statistical patterns. However, this signature shows scale dependence: in fragmented interaction environments the separation attenuates, suggesting a fundamental limit to surface-level distinguishability at small scales. This compressibility-based separation emerges consistently across models, tasks, and domains and can be observed directly from surface text without relying on model internals or semantic evaluation. Overall, our findings introduce a simple and robust framework for quantifying how generative systems reshape textual production, offering a structural perspective on the evolving complexity of communication.
• 摘要：大型语言模型通过高维分布的概率采样生成文本，但这个过程如何重塑语言的结构统计组织仍然不完全表征。在这里，我们展示了无损压缩提供了一种简单的、与模型无关的统计规律性度量，可以直接将生成机制与表面文本区分开来。我们分析了三个逐渐复杂的信息生态系统的压缩行为：受控的人类法学硕士延续、知识基础设施的生成中介（维基百科与 Grokipedia）以及完全合成的社交互动环境（Moltbook 与 Reddit）。在不同的设置中，压缩揭示了概率生成的持久结构特征。在受控和中介环境中，法学硕士生成的语言比人类书写的文本表现出更高的结构规律性和可压缩性，这与高度重复的统计模式中的输出集中一致。然而，这个特征显示了尺度依赖性：在分散的交互环境中，分离减弱，这表明小尺度下表面水平可区分性的根本限制。这种基于可压缩性的分离在模型、任务和领域中一致出现，并且可以直接从表面文本中观察到，而无需依赖模型内部或语义评估。总的来说，我们的研究结果引入了一个简单而强大的框架，用于量化生成系统如何重塑文本生产，为不断发展的通信复杂性提供了结构视角。

Title: FENCE: A Financial and Multimodal Jailbreak Detection Dataset

• Authors: Mirae Kim, Seonghun Jeong, Youngjun Kwak
• Subjects: cs.CL, cs.AI, cs.DB
• Abstract URL: https://arxiv.org/abs/2602.18154
• Pdf URL: https://arxiv.org/pdf/2602.18154
• Copy Paste: [[2602.18154]] FENCE: A Financial and Multimodal Jailbreak Detection Dataset(https://arxiv.org/abs/2602.18154)
• Keywords: language model, gpt, llm
• Abstract: Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs). VLMs are particularly vulnerable because they process both text and images, creating broader attack surfaces. However, available resources for jailbreak detection are scarce, particularly in finance. To address this gap, we present FENCE, a bilingual (Korean-English) multimodal dataset for training and evaluating jailbreak detectors in financial applications. FENCE emphasizes domain realism through finance-relevant queries paired with image-grounded threats. Experiments with commercial and open-source VLMs reveal consistent vulnerabilities, with GPT-4o showing measurable attack success rates and open-source models displaying greater exposure. A baseline detector trained on FENCE achieves 99 percent in-distribution accuracy and maintains strong performance on external benchmarks, underscoring the dataset's robustness for training reliable detection models. FENCE provides a focused resource for advancing multimodal jailbreak detection in finance and for supporting safer, more reliable AI systems in sensitive domains. Warning: This paper includes example data that may be offensive.
• 摘要：越狱对大型语言模型 (LLM) 和视觉语言模型 (VLM) 的部署构成重大风险。 VLM 特别容易受到攻击，因为它们同时处理文本和图像，从而产生更广泛的攻击面。然而，用于越狱检测的可用资源很少，特别是在金融领域。为了解决这一差距，我们提出了 FENCE，这是一个双语（韩语-英语）多模式数据集，用于训练和评估金融应用中的越狱检测器。 FENCE 通过与金融相关的查询与基于图像的威胁相结合来强调领域现实性。商业和开源 VLM 的实验揭示了一致的漏洞，GPT-4o 显示了可测量的攻击成功率，开源模型显示出更大的暴露程度。在 FENCE 上训练的基线检测器可实现 99% 的分布内准确率，并在外部基准测试中保持强劲的性能，强调了数据集在训练可靠检测模型方面的稳健性。 FENCE 提供了集中资源，用于推进金融领域的多模式越狱检测，并支持敏感领域中更安全、更可靠的人工智能系统。警告：本文包含可能令人反感的示例数据。

Title: Click it or Leave it: Detecting and Spoiling Clickbait with Informativeness Measures and Large Language Models

• Authors: Wojciech Michaluk, Tymoteusz Urban, Mateusz Kubita, Soveatin Kuntur, Anna Wroblewska
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.18171
• Pdf URL: https://arxiv.org/pdf/2602.18171
• Copy Paste: [[2602.18171]] Click it or Leave it: Detecting and Spoiling Clickbait with Informativeness Measures and Large Language Models(https://arxiv.org/abs/2602.18171)
• Keywords: language model, llm, prompt
• Abstract: Clickbait headlines degrade the quality of online information and undermine user trust. We present a hybrid approach to clickbait detection that combines transformer-based text embeddings with linguistically motivated informativeness features. Using natural language processing techniques, we evaluate classical vectorizers, word embedding baselines, and large language model embeddings paired with tree-based classifiers. Our best-performing model, XGBoost over embeddings augmented with 15 explicit features, achieves an F1-score of 91\%, outperforming TF-IDF, Word2Vec, GloVe, LLM prompt based classification, and feature-only baselines. The proposed feature set enhances interpretability by highlighting salient linguistic cues such as second-person pronouns, superlatives, numerals, and attention-oriented punctuation, enabling transparent and well-calibrated clickbait predictions. We release code and trained models to support reproducible research.
• 摘要：点击诱饵头条新闻会降低在线信息的质量并破坏用户的信任。我们提出了一种点击诱饵检测的混合方法，它将基于变压器的文本嵌入与语言驱动的信息特征相结合。使用自然语言处理技术，我们评估经典向量化器、词嵌入基线以及与基于树的分类器配对的大型语言模型嵌入。我们性能最佳的模型，XGBoost 嵌入增强了 15 个显式特征，实现了 91% 的 F1 分数，优于 TF-IDF、Word2Vec、GloVe、基于 LLM 提示的分类和纯特征基线。所提出的功能集通过突出显示显着的语言线索（例如第二人称代词、最高级、数字和以注意力为导向的标点符号）来增强可解释性，从而实现透明且经过良好校准的标题诱饵预测。我们发布代码和训练模型来支持可重复的研究。

Title: Information-Theoretic Storage Cost in Sentence Comprehension

• Authors: Kohei Kajikawa, Shinnosuke Isono, Ethan Gotlieb Wilcox
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.18217
• Pdf URL: https://arxiv.org/pdf/2602.18217
• Copy Paste: [[2602.18217]] Information-Theoretic Storage Cost in Sentence Comprehension(https://arxiv.org/abs/2602.18217)
• Keywords: language model
• Abstract: Real-time sentence comprehension imposes a significant load on working memory, as comprehenders must maintain contextual information to anticipate future input. While measures of such load have played an important role in psycholinguistic theories, they have been formalized, largely, using symbolic grammars, which assign discrete, uniform costs to syntactic predictions. This study proposes a measure of processing storage cost based on an information-theoretic formalization, as the amount of information previous words carry about future context, under uncertainty. Unlike previous discrete, grammar-based metrics, this measure is continuous, theory-neutral, and can be estimated from pre-trained neural language models. The validity of this approach is demonstrated through three analyses in English: our measure (i) recovers well-known processing asymmetries in center embeddings and relative clauses, (ii) correlates with a grammar-based storage cost in a syntactically-annotated corpus, and (iii) predicts reading-time variance in two large-scale naturalistic datasets over and above baseline models with traditional information-based predictors.
• 摘要：实时句子理解会给工作记忆带来很大的负担，因为理解者必须维护上下文信息以预测未来的输入。虽然这种负载的测量在心理语言学理论中发挥了重要作用，但它们很大程度上已经使用符号语法进行了形式化，符号语法为句法预测分配了离散的、统一的成本。本研究提出了一种基于信息论形式化的处理存储成本的衡量方法，即在不确定性下，先前单词携带的有关未来上下文的信息量。与之前离散的、基于语法的度量不同，这种度量是连续的、理论中立的，并且可以根据预先训练的神经语言模型进行估计。这种方法的有效性通过英语的三项分析得到证明：我们的措施（i）恢复中心嵌入和相关子句中众所周知的处理不对称性，（ii）与句法注释语料库中基于语法的存储成本相关，以及（iii）使用传统的基于信息的预测器预测两个大规模自然数据集中的阅读时间方差。

Title: Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning

• Authors: Lexiang Tang, Weihao Gao, Bingchen Zhao, Lu Ma, Qiao jin, Bang Yang, Yuexian Zou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.18232
• Pdf URL: https://arxiv.org/pdf/2602.18232
• Copy Paste: [[2602.18232]] Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning(https://arxiv.org/abs/2602.18232)
• Keywords: language model, llm
• Abstract: Recent work on test-time scaling for large language model (LLM) reasoning typically assumes that allocating more inference-time computation uniformly improves correctness. However, prior studies show that reasoning uncertainty is highly localized: a small subset of low-confidence tokens disproportionately contributes to reasoning errors and unnecessary output expansion. Motivated by this observation, we propose Thinking by Subtraction, a confidence-driven contrastive decoding approach that improves reasoning reliability through targeted token-level intervention. Our method, Confidence-Driven Contrastive Decoding, detects low-confidence tokens during decoding and intervenes selectively at these positions. It constructs a contrastive reference by replacing high-confidence tokens with minimal placeholders, and refines predictions by subtracting this reference distribution at low-confidence locations. Experiments show that CCD significantly improves accuracy across mathematical reasoning benchmarks while substantially reducing output length, with minimal KV-cache overhead. As a training-free method, CCD enhances reasoning reliability through targeted low-confidence intervention without computational redundancy. Our code will be made available at: this https URL.
• 摘要：最近关于大型语言模型 (LLM) 推理的测试时间扩展的工作通常假设分配更多的推理时间计算可以统一提高正确性。然而，先前的研究表明，推理不确定性是高度局部化的：一小部分低置信度标记会不成比例地导致推理错误和不必要的输出扩展。受这一观察的启发，我们提出了减法思考，这是一种置信驱动的对比解码方法，通过有针对性的标记级干预来提高推理可靠性。我们的方法“置信度驱动对比解码”在解码过程中检测低置信度标记，并有选择地在这些位置进行干预。它通过用最少的占位符替换高置信度标记来构建对比参考，并通过在低置信度位置减去此参考分布来完善预测。实验表明，CCD 显着提高了数学推理基准的准确性，同时大幅减少了输出长度，并且 KV 缓存开销最小。作为一种免训练方法，CCD通过有针对性的低置信度干预而无需计算冗余，从而增强了推理可靠性。我们的代码将在以下位置提供：此 https URL。

Title: Simplifying Outcomes of Language Model Component Analyses with ELIA

• Authors: Aaron Louis Eidt, Nils Feldhus
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2602.18262
• Pdf URL: https://arxiv.org/pdf/2602.18262
• Copy Paste: [[2602.18262]] Simplifying Outcomes of Language Model Component Analyses with ELIA(https://arxiv.org/abs/2602.18262)
• Keywords: language model, llm
• Abstract: While mechanistic interpretability has developed powerful tools to analyze the internal workings of Large Language Models (LLMs), their complexity has created an accessibility gap, limiting their use to specialists. We address this challenge by designing, building, and evaluating ELIA (Explainable Language Interpretability Analysis), an interactive web application that simplifies the outcomes of various language model component analyses for a broader audience. The system integrates three key techniques -- Attribution Analysis, Function Vector Analysis, and Circuit Tracing -- and introduces a novel methodology: using a vision-language model to automatically generate natural language explanations (NLEs) for the complex visualizations produced by these methods. The effectiveness of this approach was empirically validated through a mixed-methods user study, which revealed a clear preference for interactive, explorable interfaces over simpler, static visualizations. A key finding was that the AI-powered explanations helped bridge the knowledge gap for non-experts; a statistical analysis showed no significant correlation between a user's prior LLM experience and their comprehension scores, suggesting that the system reduced barriers to comprehension across experience levels. We conclude that an AI system can indeed simplify complex model analyses, but its true power is unlocked when paired with thoughtful, user-centered design that prioritizes interactivity, specificity, and narrative guidance.
• 摘要：虽然机械可解释性已经开发出强大的工具来分析大型语言模型（LLM）的内部运作，但它们的复杂性造成了可访问性差距，限制了专家的使用。我们通过设计、构建和评估 ELIA（可解释语言可解释性分析）来应对这一挑战，这是一个交互式 Web 应用程序，可以为更广泛的受众简化各种语言模型组件分析的结果。该系统集成了三种关键技术——归因分析、函数向量分析和电路追踪——并引入了一种新颖的方法：使用视觉语言模型为这些方法产生的复杂可视化自动生成自然语言解释（NLE）。这种方法的有效性通过混合方法的用户研究得到了实证验证，该研究揭示了人们对交互式、可探索界面的明显偏好，而不是更简单的静态可视化。一个重要的发现是，人工智能驱动的解释有助于弥合非专家的知识差距；统计分析显示，用户之前的法学硕士经验与其理解分数之间没有显着相关性，这表明该系统减少了不同经验水平的理解障碍。我们的结论是，人工智能系统确实可以简化复杂的模型分析，但当与优先考虑交互性、特异性和叙述指导的深思熟虑、以用户为中心的设计结合使用时，它的真正力量就会被释放。

Title: Vichara: Appellate Judgment Prediction and Explanation for the Indian Judicial System

• Authors: Pavithra PM Nair, Preethu Rose Anish
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.18346
• Pdf URL: https://arxiv.org/pdf/2602.18346
• Copy Paste: [[2602.18346]] Vichara: Appellate Judgment Prediction and Explanation for the Indian Judicial System(https://arxiv.org/abs/2602.18346)
• Keywords: language model, gpt
• Abstract: In jurisdictions like India, where courts face an extensive backlog of cases, artificial intelligence offers transformative potential for legal judgment prediction. A critical subset of this backlog comprises appellate cases, which are formal decisions issued by higher courts reviewing the rulings of lower courts. To this end, we present Vichara, a novel framework tailored to the Indian judicial system that predicts and explains appellate judgments. Vichara processes English-language appellate case proceeding documents and decomposes them into decision points. Decision points are discrete legal determinations that encapsulate the legal issue, deciding authority, outcome, reasoning, and temporal context. The structured representation isolates the core determinations and their context, enabling accurate predictions and interpretable explanations. Vichara's explanations follow a structured format inspired by the IRAC (Issue-Rule-Application-Conclusion) framework and adapted for Indian legal reasoning. This enhances interpretability, allowing legal professionals to assess the soundness of predictions efficiently. We evaluate Vichara on two datasets, PredEx and the expert-annotated subset of the Indian Legal Documents Corpus (ILDC_expert), using four large language models: GPT-4o mini, Llama-3.1-8B, Mistral-7B, and Qwen2.5-7B. Vichara surpasses existing judgment prediction benchmarks on both datasets, with GPT-4o mini achieving the highest performance (F1: 81.5 on PredEx, 80.3 on ILDC_expert), followed by Llama-3.1-8B. Human evaluation of the generated explanations across Clarity, Linking, and Usefulness metrics highlights GPT-4o mini's superior interpretability.
• 摘要：在印度等法院面临大量积压案件的司法管辖区，人工智能为法律判决预测提供了变革潜力。这些积压案件的一个关键部分包括上诉案件，这些案件是上级法院审查下级法院裁决后发布的正式决定。为此，我们提出了 Vichara，这是一个针对印度司法系统量身定制的新颖框架，可以预测和解释上诉判决。 Vichara 处理英语上诉案件程序文件并将其分解为决策点。决策点是离散的法律决定，概括了法律问题、决策权威、结果、推理和时间背景。结构化表示将核心决策及其上下文隔离开来，从而实现准确的预测和可解释的解释。 Vichara 的解释遵循受 IRAC（问题-规则-应用-结论）框架启发并针对印度法律推理进行调整的结构化格式。这增强了可解释性，使法律专业人士能够有效评估预测的合理性。我们使用四种大型语言模型：GPT-4o mini、Llama-3.1-8B、Mistral-7B 和 Qwen2.5-7B，在两个数据集（PredEx 和印度法律文件语料库专家注释子集 (ILDC_expert)）上评估 Vichara。 Vichara 在两个数据集上都超越了现有的判断预测基准，其中 GPT-4o mini 实现了最高性能（F1：在 PredEx 上为 81.5，在 ILDC_expert 上为 80.3），其次是 Llama-3.1-8B。对生成的解释的清晰度、链接和有用性指标的人工评估凸显了 GPT-4o mini 卓越的可解释性。

Title: Validating Political Position Predictions of Arguments

• Authors: Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2602.18351
• Pdf URL: https://arxiv.org/pdf/2602.18351
• Copy Paste: [[2602.18351]] Validating Political Position Predictions of Arguments(https://arxiv.org/abs/2602.18351)
• Keywords: language model, retrieval-augmented generation
• Abstract: Real-world knowledge representation often requires capturing subjective, continuous attributes -- such as political positions -- that conflict with pairwise validation, the widely accepted gold standard for human evaluation. We address this challenge through a dual-scale validation framework applied to political stance prediction in argumentative discourse, combining pointwise and pairwise human annotation. Using 22 language models, we construct a large-scale knowledge base of political position predictions for 23,228 arguments drawn from 30 debates that appeared on the UK politicial television programme \textit{Question Time}. Pointwise evaluation shows moderate human-model agreement (Krippendorff's $\alpha=0.578$), reflecting intrinsic subjectivity, while pairwise validation reveals substantially stronger alignment between human- and model-derived rankings ($\alpha=0.86$ for the best model). This work contributes: (i) a practical validation methodology for subjective continuous knowledge that balances scalability with reliability; (ii) a validated structured argumentation knowledge base enabling graph-based reasoning and retrieval-augmented generation in political domains; and (iii) evidence that ordinal structure can be extracted from pointwise language models predictions from inherently subjective real-world discourse, advancing knowledge representation capabilities for domains where traditional symbolic or categorical approaches are insufficient.
• 摘要：现实世界的知识表示通常需要捕获主观的、连续的属性（例如政治立场），这与成对验证（人类评估的广泛接受的黄金标准）相冲突。我们通过应用于争论性话语中的政治立场预测的双尺度验证框架来应对这一挑战，结合逐点和成对的人类注释。使用 22 种语言模型，我们构建了一个大规模的政治立场预测知识库，其中包含来自英国政治电视节目 \textit{Question Time} 中出现的 30 场辩论的 23,228 个论点。逐点评估显示了适度的人类模型一致性（Krippendorff 的 $\alpha=0.578$），反映了内在的主观性，而成对验证则揭示了人类和模型派生排名之间更强的一致性（最佳模型的 $\alpha=0.86$）。这项工作有助于：（i）一种平衡可扩展性与可靠性的主观连续知识的实用验证方法； (ii) 经过验证的结构化论证知识库，可在政治领域实现基​​于图形的推理和检索增强生成； (iii) 有证据表明序数结构可以从本质上主观的现实世界话语的逐点语言模型预测中提取，从而提高传统符号或分类方法不足的领域的知识表示能力。

Title: SPQ: An Ensemble Technique for Large Language Model Compression

• Authors: Jiamin Yao, Eren Gultepe
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2602.18420
• Pdf URL: https://arxiv.org/pdf/2602.18420
• Copy Paste: [[2602.18420]] SPQ: An Ensemble Technique for Large Language Model Compression(https://arxiv.org/abs/2602.18420)
• Keywords: language model, gpt, llm
• Abstract: This study presents an ensemble technique, SPQ (SVD-Pruning-Quantization), for large language model (LLM) compression that combines variance-retained singular value decomposition (SVD), activation-based pruning, and post-training linear quantization. Each component targets a different source of inefficiency: i) pruning removes redundant neurons in MLP layers, ii) SVD reduces attention projections into compact low-rank factors, iii) and 8-bit quantization uniformly compresses all linear layers. At matched compression ratios, SPQ outperforms individual methods (SVD-only, pruning-only, or quantization-only) in perplexity, demonstrating the benefit of combining complementary techniques. Applied to LLaMA-2-7B, SPQ achieves up to 75% memory reduction while maintaining or improving perplexity (e.g., WikiText-2 5.47 to 4.91) and preserving accuracy on downstream benchmarks such as C4, TruthfulQA, and GSM8K. Compared to strong baselines like GPTQ and SparseGPT, SPQ offers competitive perplexity and accuracy while using less memory (6.86 GB vs. 7.16 GB for GPTQ). Moreover, SPQ improves inference throughput over GPTQ, achieving up to a 1.9x speedup, which further enhances its practicality for real-world deployment. The effectiveness of SPQ's robust compression through layer-aware and complementary compression techniques may provide practical deployment of LLMs in memory-constrained environments. Code is available at: this https URL
• 摘要：本研究提出了一种用于大型语言模型 (LLM) 压缩的集成技术 SPQ（SVD-剪枝-量化），该技术结合了方差保留奇异值分解 (SVD)、基于激活的剪枝和训练后线性量化。每个组件针对不同的低效率来源：i) 剪枝去除 MLP 层中的冗余神经元，ii) SVD 将注意力投影减少为紧凑的低秩因子，iii) 和 8 位量化统一压缩所有线性层。在匹配的压缩比下，SPQ 在困惑度方面优于单个方法（仅 SVD、仅剪枝或仅量化），证明了结合互补技术的好处。应用于 LLaMA-2-7B 时，SPQ 可实现高达 75% 的内存减少，同时保持或改善复杂性（例如 WikiText-2 5.47 至 4.91）并保持下游基准（如 C4、TruthfulQA 和 GSM8K）的准确性。与 GPTQ 和 SparseGPT 等强大的基线相比，SPQ 提供了有竞争力的复杂性和准确性，同时使用更少的内存（GPTQ 为 6.86 GB 与 7.16 GB）。此外，SPQ 相对于 GPTQ 提高了推理吞吐量，实现了高达 1.9 倍的加速，这进一步增强了其实际部署的实用性。 SPQ 通过分层感知和互补压缩技术实现的稳健压缩的有效性可以在内存受限的环境中提供 LLM 的实际部署。代码位于：此 https URL

Title: RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering

• Authors: Deniz Qian, Hung-Ting Chen, Eunsol Choi
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2602.18425
• Pdf URL: https://arxiv.org/pdf/2602.18425
• Copy Paste: [[2602.18425]] RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering(https://arxiv.org/abs/2602.18425)
• Keywords: agent
• Abstract: Comprehensively retrieving diverse documents is crucial to address queries that admit a wide range of valid answers. We introduce retrieve-verify-retrieve (RVR), a multi-round retrieval framework designed to maximize answer coverage. Initially, a retriever takes the original query and returns a candidate document set, followed by a verifier that identifies a high-quality subset. For subsequent rounds, the query is augmented with previously verified documents to uncover answers that are not yet covered in previous rounds. RVR is effective even with off-the-shelf retrievers, and fine-tuning retrievers for our inference procedure brings further gains. Our method outperforms baselines, including agentic search approaches, achieving at least 10% relative and 3% absolute gain in complete recall percentage on a multi-answer retrieval dataset (QAMPARI). We also see consistent gains on two out-of-domain datasets (QUEST and WebQuestionsSP) across different base retrievers. Our work presents a promising iterative approach for comprehensive answer recall leveraging a verifier and adapting retrievers to a new inference scenario.
• 摘要：全面检索不同的文档对于解决允许广泛有效答案的查询至关重要。我们引入检索-验证-检索（RVR），这是一种旨在最大化答案覆盖率的多轮检索框架。最初，检索器获取原始查询并返回候选文档集，然后由验证器识别高质量子集。对于后续轮次，查询将使用先前验证的文档进行扩充，以发现前轮次中尚未涵盖的答案。即使对于现成的检索器，RVR 也是有效的，并且针对我们的推理过程微调检索器会带来进一步的收益。我们的方法优于基线，包括代理搜索方法，在多答案检索数据集 (QAMPARI) 上的完全召回率方面实现了至少 10% 的相对增益和 3% 的绝对增益。我们还看到不同的基础检索器在两个域外数据集（QUEST 和 WebQuestionsSP）上取得了一致的收益。我们的工作提出了一种有前途的迭代方法，利用验证器并使检索器适应新的推理场景来进行全面的答案回忆。

Title: VIRAASAT: Traversing Novel Paths for Indian Cultural Reasoning

• Authors: Harshul Raj Surana, Arijit Maji, Aryan Vats, Akash Ghosh, Sriparna Saha, Amit Sheth
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2602.18429
• Pdf URL: https://arxiv.org/pdf/2602.18429
• Copy Paste: [[2602.18429]] VIRAASAT: Traversing Novel Paths for Indian Cultural Reasoning(https://arxiv.org/abs/2602.18429)
• Keywords: language model, llm, chain-of-thought
• Abstract: Large Language Models (LLMs) have made significant progress in reasoning tasks across various domains such as mathematics and coding. However, their performance deteriorates in tasks requiring rich socio-cultural knowledge and diverse local contexts, particularly those involving Indian Culture. Existing Cultural benchmarks are (i) Manually crafted, (ii) contain single-hop questions testing factual recall, and (iii) prohibitively costly to scale, leaving this deficiency largely unmeasured. To address this, we introduce VIRAASAT, a novel, semi-automated multi-hop approach for generating cultural specific multi-hop Question-Answering dataset for Indian culture. VIRAASAT leverages a Knowledge Graph comprising more than 700 expert-curated cultural artifacts, covering 13 key attributes of Indian culture (history, festivals, etc). VIRAASAT spans all 28 states and 8 Union Territories, yielding more than 3,200 multi-hop questions that necessitate chained cultural reasoning. We evaluate current State-of-the-Art (SOTA) LLMs on VIRAASAT and identify key limitations in reasoning wherein fine-tuning on Chain-of-Thought(CoT) traces fails to ground and synthesize low-probability facts. To bridge this gap, we propose a novel framework named Symbolic Chain-of-Manipulation (SCoM). Adapting the Chain-of-Manipulation paradigm, we train the model to simulate atomic Knowledge Graph manipulations internally. SCoM teaches the model to reliably traverse the topological structure of the graph. Experiments on Supervised Fine-Tuning (SFT) demonstrate that SCoM outperforms standard CoT baselines by up to 20%. We release the VIRAASAT dataset along with our findings, laying a strong foundation towards building Culturally Aware Reasoning Models.
• 摘要：大型语言模型 (LLM) 在数学和编码等各个领域的推理任务中取得了重大进展。然而，在需要丰富的社会文化知识和不同的当地背景的任务中，尤其是涉及印度文化的任务中，他们的表现却下降了。现有的文化基准是（i）手动制定的，（ii）包含测试事实回忆的单跳问题，以及（iii）扩展成本过高，导致这种缺陷在很大程度上无法衡量。为了解决这个问题，我们引入了 VIRAASAT，一种新颖的半自动多跳方法，用于生成印度文化特定文化的多跳问答数据集。 VIRAASAT 利用知识图谱，其中包含 700 多个专家策划的文化文物，涵盖印度文化的 13 个关键属性（历史、节日等）。 VIRAASAT 横跨所有 28 个州和 8 个联邦直辖区，产生超过 3,200 个需要链式文化推理的多跳问题。我们评估了 VIRAASAT 上当前最先进的 (SOTA) 法学硕士，并确定了推理中的关键局限性，其中思想链 (CoT) 轨迹的微调无法基础和合成低概率事实。为了弥补这一差距，我们提出了一种名为符号操作链（SCoM）的新颖框架。采用操作链范式，我们训练模型在内部模拟原子知识图操作。 SCoM 教导模型可靠地遍历图的拓扑结构。监督微调 (SFT) 实验表明，SCoM 的性能比标准 CoT 基线高出 20%。我们发布了 VIRAASAT 数据集以及我们的研究结果，为构建文化感知推理模型奠定了坚实的基础。