2024-11-14

Title: Large Language Models Can Self-Improve in Long-context Reasoning

• Authors: Siheng Li, Cheng Yang, Zesen Cheng, Lemao Liu, Mo Yu, Yujiu Yang, Wai Lam
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08147
• Pdf URL: https://arxiv.org/pdf/2411.08147
• Copy Paste: [[2411.08147]] Large Language Models Can Self-Improve in Long-context Reasoning(https://arxiv.org/abs/2411.08147)
• Keywords: language model, gpt, llm, long context
• Abstract: Large language models (LLMs) have achieved substantial progress in processing long contexts but still struggle with long-context reasoning. Existing approaches typically involve fine-tuning LLMs with synthetic data, which depends on annotations from human experts or advanced models like GPT-4, thus restricting further advancements. To address this issue, we investigate the potential for LLMs to self-improve in long-context reasoning and propose \ours, an approach specifically designed for this purpose. This approach is straightforward: we sample multiple outputs for each question, score them with Minimum Bayes Risk, and then apply supervised fine-tuning or preference optimization based on these outputs. Extensive experiments on several leading LLMs demonstrate the effectiveness of \ours, with an absolute improvement of $4.2$ points for Llama-3.1-8B-Instruct. Furthermore, \ours achieves superior performance compared to prior approaches that depend on data produced by human experts or advanced models. We anticipate that this work will open new avenues for self-improvement techniques in long-context scenarios, which are essential for the continual advancement of LLMs.
• 摘要：大型语言模型 (LLM) 在处理长上下文方面取得了实质性进展，但在长上下文推理方面仍然存在困难。现有方法通常涉及使用合成数据对 LLM 进行微调，这取决于人类专家或 GPT-4 等高级模型的注释，从而限制了进一步的发展。为了解决这个问题，我们研究了 LLM 在长上下文推理中自我改进的潜力，并提出了专门为此设计的方法 \ours。这种方法很简单：我们对每个问题抽取多个输出，用最小贝叶斯风险对它们进行评分，然后根据这些输出应用监督微调或偏好优化。在几个领先的 LLM 上进行的大量实验证明了 \ours 的有效性，Llama-3.1-8B-Instruct 的绝对改进为 $4.2$ 分。此外，与依赖人类专家或高级模型生成的数据的先前方法相比，\ours 实现了卓越的性能。我们期望这项工作将为长期情景中的自我完善技术开辟新的途径，这对于法学硕士的持续进步至关重要。

Title: Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset

• Authors: Khaoula Chehbouni, Jonathan Colaço-Carr, Yash More, Jackie CK Cheung, Golnoosh Farnadi
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2411.08243
• Pdf URL: https://arxiv.org/pdf/2411.08243
• Copy Paste: [[2411.08243]] Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset(https://arxiv.org/abs/2411.08243)
• Keywords: language model, llm
• Abstract: In an effort to mitigate the harms of large language models (LLMs), learning from human feedback (LHF) has been used to steer LLMs towards outputs that are intended to be both less harmful and more helpful. Despite the widespread adoption of LHF in practice, the quality of this feedback and its effectiveness as a safety mitigation technique remain unclear. This study addresses these issues by auditing the widely-used Helpful and Harmless (HH) dataset by Anthropic. Our work includes: (1) a thorough investigation of the dataset's content through both manual and automated evaluation; (2) experiments demonstrating the dataset's impact on models' safety; and (3) an analysis of the 100 most influential papers citing this dataset. Through our audit, we showcase how conceptualization failures and quality issues identified in the HH dataset can create additional harms by leading to disparate safety behaviors across demographic groups. Our findings highlight the need for more nuanced, context-sensitive approaches to safety mitigation in LLMs.
• 摘要：为了减轻大型语言模型 (LLM) 的危害，人们已使用从人类反馈中学习 (LHF) 来引导 LLM 实现危害更小、更有帮助的输出。尽管 LHF 在实践中被广泛采用，但这种反馈的质量及其作为安全缓解技术的有效性仍不清楚。本研究通过审核 Anthropic 广泛使用的“有用且无害” (HH) 数据集来解决这些问题。我们的工作包括：(1) 通过手动和自动评估彻底调查数据集的内容；(2) 实验证明数据集对模型安全性的影响；(3) 分析引用该数据集的 100 篇最具影响力的论文。通过我们的审核，我们展示了 HH 数据集中发现的概念化失败和质量问题如何导致不同人口群体的安全行为不同，从而造成额外的危害。我们的研究结果强调了 LLM 中需要更细致入微、更注重情境的安全性缓解方法。

Title: Knowledge Bases in Support of Large Language Models for Processing Web News

• Authors: Yihe Zhang, Nabin Pakka, Nian-feng Tzeng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08278
• Pdf URL: https://arxiv.org/pdf/2411.08278
• Copy Paste: [[2411.08278]] Knowledge Bases in Support of Large Language Models for Processing Web News(https://arxiv.org/abs/2411.08278)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have received considerable interest in wide applications lately. During pre-training via massive datasets, such a model implicitly memorizes the factual knowledge of trained datasets in its hidden parameters. However, knowledge held implicitly in parameters often makes its use by downstream applications ineffective due to the lack of common-sense reasoning. In this article, we introduce a general framework that permits to build knowledge bases with an aid of LLMs, tailored for processing Web news. The framework applies a rule-based News Information Extractor (NewsIE) to news items for extracting their relational tuples, referred to as knowledge bases, which are then graph-convoluted with the implicit knowledge facts of news items obtained by LLMs, for their classification. It involves two lightweight components: 1) NewsIE: for extracting the structural information of every news item, in the form of relational tuples; 2) BERTGraph: for graph convoluting the implicit knowledge facts with relational tuples extracted by NewsIE. We have evaluated our framework under different news-related datasets for news category classification, with promising experimental results.
• 摘要：大型语言模型 (LLM) 近来在广泛应用中引起了广泛关注。在通过海量数据集进行预训练时，这种模型会在其隐藏参数中隐式地记忆训练数据集的事实知识。然而，由于缺乏常识推理，参数中隐含的知识通常会使下游应用程序无法有效使用。在本文中，我们介绍了一个通用框架，该框架允许借助 LLM 构建专门用于处理网络新闻的知识库。该框架将基于规则的新闻信息提取器 (NewsIE) 应用于新闻项目以提取其关系元组（称为知识库），然后将其与 LLM 获得的新闻项目的隐性知识事实进行图卷积，以对其进行分类。它涉及两个轻量级组件：1) NewsIE：用于以关系元组的形式提取每个新闻项目的结构信息；2) BERTGraph：用于将隐性知识事实与 NewsIE 提取的关系元组进行图卷积。我们已经在不同的新闻相关数据集下评估了我们的框架以进行新闻类别分类，并得到了令人满意的实验结果。

Title: R3HF: Reward Redistribution for Enhancing Reinforcement Learning from Human Feedback

• Authors: Jiahui Li, Tai-wei Chang, Fengda Zhang, Kun Kuang, Long Chen
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08302
• Pdf URL: https://arxiv.org/pdf/2411.08302
• Copy Paste: [[2411.08302]] R3HF: Reward Redistribution for Enhancing Reinforcement Learning from Human Feedback(https://arxiv.org/abs/2411.08302)
• Keywords: language model, llm
• Abstract: Reinforcement learning from human feedback (RLHF) provides a paradigm for aligning large language models (LLMs) with human preferences. This involves the initial training of a reward model based on pairwise human feedback. The reward model is subsequently utilized in reinforcement learning to assess the scores of each generated sentence as a whole, further guiding the optimization of LLMs. However, current approaches have a significant shortcoming: \emph{They allocate a single, sparse, and delayed reward to an entire sequence of output}. This may overlook some significant individual contributions of each token towards the desired outcome. To overcome this limitation, our paper proposes a novel reward redistribution method called R3HF, which facilitates a more fine-grained, token-level reward allocation. Specifically, our method treats the reward prediction task of the reward model as a regression problem. As a result, the redistributed rewards are computed by evaluating the specific contribution of each token to the reward model's output. This detailed approach improves the model's understanding of language nuances, leading to more precise enhancements in its performance. Our method is crafted to integrate seamlessly with most current techniques while incurring minimal computational costs. Through comprehensive experiments across diverse datasets and tasks, we have verified the effectiveness and superiority of our approach.
• 摘要：基于人类反馈的强化学习 (RLHF) 为将大型语言模型 (LLM) 与人类偏好相结合提供了一种范例。这涉及基于成对人类反馈的奖励模型的初始训练。随后，奖励模型用于强化学习，以评估每个生成的句子的整体得分，从而进一步指导 LLM 的优化。然而，当前的方法有一个显著的缺点：\emph{它们将单一、稀疏且延迟的奖励分配给整个输出序列}。这可能会忽略每个 token 对期望结果的一些重要个人贡献。为了克服这一限制，我们的论文提出了一种名为 R3HF 的新型奖励重新分配方法，该方法有助于实现更细粒度的 token 级奖励分配。具体而言，我们的方法将奖励模型的奖励预测任务视为回归问题。因此，通过评估每个 token 对奖励模型输出的具体贡献来计算重新分配的奖励。这种详细的方法提高了模型对语言细微差别的理解，从而更精确地提高了其性能。我们的方法旨在与大多数现有技术无缝集成，同时将计算成本降至最低。通过对各种数据集和任务的全面实验，我们验证了我们方法的有效性和优越性。

Title: Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oracle

• Authors: Hui Dai, Ryan Teehan, Mengye Ren
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.08324
• Pdf URL: https://arxiv.org/pdf/2411.08324
• Copy Paste: [[2411.08324]] Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oracle(https://arxiv.org/abs/2411.08324)
• Keywords: language model, llm, retrieval augmented generation
• Abstract: Many existing evaluation benchmarks for Large Language Models (LLMs) quickly become outdated due to the emergence of new models and training data. These benchmarks also fall short in assessing how LLM performance changes over time, as they consist of static questions without a temporal dimension. To address these limitations, we propose using future event prediction as a continuous evaluation method to assess LLMs' temporal generalization and forecasting abilities. Our benchmark, Daily Oracle, automatically generates question-answer (QA) pairs from daily news, challenging LLMs to predict "future" event outcomes. Our findings reveal that as pre-training data becomes outdated, LLM performance degrades over time. While Retrieval Augmented Generation (RAG) has the potential to enhance prediction accuracy, the performance degradation pattern persists, highlighting the need for continuous model updates.
• 摘要：由于新模型和训练数据的出现，许多现有的大型语言模型 (LLM) 评估基准很快就过时了。这些基准还无法评估 LLM 性能随时间的变化，因为它们由没有时间维度的静态问题组成。为了解决这些限制，我们建议使用未来事件预测作为一种持续评估方法来评估 LLM 的时间泛化和预测能力。我们的基准 Daily Oracle 会自动从每日新闻中生成问答 (QA) 对，挑战 LLM 预测“未来”事件结果。我们的研究结果表明，随着预训练数据过时，LLM 性能会随着时间的推移而下降。虽然检索增强生成 (RAG) 有可能提高预测准确性，但性能下降模式仍然存在，这凸显了持续更新模型的必要性。

Title: Refining Translations with LLMs: A Constraint-Aware Iterative Prompting Approach

• Authors: Shangfeng Chen, Xiayang Shi, Pu Li, Yinlin Li, Jingjing Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.08348
• Pdf URL: https://arxiv.org/pdf/2411.08348
• Copy Paste: [[2411.08348]] Refining Translations with LLMs: A Constraint-Aware Iterative Prompting Approach(https://arxiv.org/abs/2411.08348)
• Keywords: language model, llm, hallucination, prompt, retrieval-augmented generation
• Abstract: Large language models (LLMs) have demonstrated remarkable proficiency in machine translation (MT), even without specific training on the languages in question. However, translating rare words in low-resource or domain-specific contexts remains challenging for LLMs. To address this issue, we propose a multi-step prompt chain that enhances translation faithfulness by prioritizing key terms crucial for semantic accuracy. Our method first identifies these keywords and retrieves their translations from a bilingual dictionary, integrating them into the LLM's context using Retrieval-Augmented Generation (RAG). We further mitigate potential output hallucinations caused by long prompts through an iterative self-checking mechanism, where the LLM refines its translations based on lexical and semantic constraints. Experiments using Llama and Qwen as base models on the FLORES-200 and WMT datasets demonstrate significant improvements over baselines, highlighting the effectiveness of our approach in enhancing translation faithfulness and robustness, particularly in low-resource scenarios.
• 摘要：大型语言模型 (LLM) 已在机器翻译 (MT) 方面表现出色，即使没有针对相关语言进行专门训练也是如此。然而，在资源匮乏或特定领域的背景下翻译罕见词对 LLM 来说仍然具有挑战性。为了解决这个问题，我们提出了一个多步骤提示链，通过优先考虑对语义准确性至关重要的关键术语来提高翻译的忠实度。我们的方法首先识别这些关键词并从双语词典中检索它们的翻译，然后使用检索增强生成 (RAG) 将它们集成到 LLM 的上下文中。我们通过迭代自检机制进一步减轻了由长提示引起的潜在输出幻觉，其中 LLM 根据词汇和语义约束改进其翻译。在 FLORES-200 和 WMT 数据集上使用 Llama 和 Qwen 作为基础模型进行的实验表明，与基线相比有显着改进，突出了我们的方法在提高翻译忠实度和稳健性方面的有效性，尤其是在资源匮乏的情况下。

Title: CLaSP: Learning Concepts for Time-Series Signals from Natural Language Supervision

• Authors: Aoi Ito, Kota Dohi, Yohei Kawaguchi
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.08397
• Pdf URL: https://arxiv.org/pdf/2411.08397
• Copy Paste: [[2411.08397]] CLaSP: Learning Concepts for Time-Series Signals from Natural Language Supervision(https://arxiv.org/abs/2411.08397)
• Keywords: language model, llm
• Abstract: This paper proposes a foundation model called "CLaSP" that can search time series signals using natural language that describes the characteristics of the signals as queries. Previous efforts to represent time series signal data in natural language have had challenges in designing a conventional class of time series signal characteristics, formulating their quantification, and creating a dictionary of synonyms. To overcome these limitations, the proposed method introduces a neural network based on contrastive learning. This network is first trained using the datasets TRUCE and SUSHI, which consist of time series signals and their corresponding natural language descriptions. Previous studies have proposed vocabularies that data analysts use to describe signal characteristics, and SUSHI was designed to cover these terms. We believe that a neural network trained on these datasets will enable data analysts to search using natural language vocabulary. Furthermore, our method does not require a dictionary of predefined synonyms, and it leverages common sense knowledge embedded in a large-scale language model (LLM). Experimental results demonstrate that CLaSP enables natural language search of time series signal data and can accurately learn the points at which signal data changes.
• 摘要：本文提出了一种名为“CLaSP”的基础模型，该模型可以使用自然语言搜索时间序列信号，将信号的特征描述为查询。以前用自然语言表示时间序列信号数据的努力在设计传统的时间序列信号特征类、制定量化方法和创建同义词词典方面遇到了挑战。为了克服这些限制，所提出的方法引入了基于对比学习的神经网络。该网络首先使用数据集 TRUCE 和 SUSHI 进行训练，这两个数据集由时间序列信号及其相应的自然语言描述组成。先前的研究提出了数据分析师用来描述信号特征的词汇表，而 SUSHI 的设计就是为了涵盖这些术语。我们相信，在这些数据集上训练的神经网络将使数据分析师能够使用自然语言词汇进行搜索。此外，我们的方法不需要预定义的同义词词典，它利用嵌入在大规模语言模型 (LLM) 中的常识知识。实验结果表明，CLaSP 可以实现时间序列信号数据的自然语言搜索，并且可以准确地学习信号数据变化的点。

Title: One STEP at a time: Language Agents are Stepwise Planners

• Authors: Minh Nguyen, Ehsan Shareghi
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.08432
• Pdf URL: https://arxiv.org/pdf/2411.08432
• Copy Paste: [[2411.08432]] One STEP at a time: Language Agents are Stepwise Planners(https://arxiv.org/abs/2411.08432)
• Keywords: language model, agent
• Abstract: Language agents have shown promising adaptability in dynamic environments to perform complex tasks. However, despite the versatile knowledge embedded in large language models, these agents still fall short when it comes to tasks that require planning. We introduce STEP, a novel framework designed to efficiently learn from previous experiences to enhance the planning capabilities of language agents in future steps. Concretely, STEP functions through four interconnected components. First, the Planner takes on the task, breaks it down into subtasks and provides relevant insights. Then the Executor generates action candidates, while the Evaluator ensures the actions align with learned rules from previous experiences. Lastly, Memory stores experiences to inform future decisions. In the ScienceWorld benchmark, our results show that STEP consistently outperforms state-of-the-art models, achieving an overall score of 67.4 and successfully completing 12 out of 18 tasks. These findings highlight STEP's potential as a framework for enhancing planning capabilities in language agents, paving the way for more sophisticated task-solving in dynamic environments.
• 摘要：语言代理在动态环境中表现出良好的适应性，可以执行复杂的任务。然而，尽管大型语言模型中嵌入了多种知识，但这些代理在需要规划的任务方面仍然不足。我们引入了 STEP，这是一个新颖的框架，旨在有效地从以前的经验中学习，以增强语言代理在未来步骤中的规划能力。具体来说，STEP 通过四个相互关联的组件发挥作用。首先，规划器承担任务，将其分解为子任务并提供相关见解。然后，执行器生成操作候选，而评估器确保操作与从以前的经验中学习到的规则一致。最后，内存存储经验以指导未来的决策。在 ScienceWorld 基准测试中，我们的结果表明 STEP 始终优于最先进的模型，总分为 67.4，成功完成了 18 项任务中的 12 项。这些发现凸显了 STEP 作为增强语言代理规划能力的框架的潜力，为在动态环境中更复杂的任务解决铺平了道路。

Title: Towards Objective and Unbiased Decision Assessments with LLM-Enhanced Hierarchical Attention Networks

• Authors: Junhua Liu, Kwan Hui Lim, Roy Ka-Wei Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08504
• Pdf URL: https://arxiv.org/pdf/2411.08504
• Copy Paste: [[2411.08504]] Towards Objective and Unbiased Decision Assessments with LLM-Enhanced Hierarchical Attention Networks(https://arxiv.org/abs/2411.08504)
• Keywords: llm, agent
• Abstract: How objective and unbiased are we while making decisions? This work investigates cognitive bias identification in high-stake decision making process by human experts, questioning its effectiveness in real-world settings, such as candidates assessments for university admission. We begin with a statistical analysis assessing correlations among different decision points among in the current process, which discovers discrepancies that imply cognitive bias and inconsistency in decisions. This motivates our exploration of bias-aware AI-augmented workflow that surpass human judgment. We propose BGM-HAN, a hierarchical attention network enhanced by byte-pair encoding, multi-head attention and gated residual connection. Using it as backbone model, we further propose a Shortlist-Analyse-Recommend (SAR) agentic workflow, which simulate real-world decision-making. In our experiments, both the proposed model and the agentic workflow significantly improves on both human judgment and alternative models, validated with real-world data.
• 摘要：我们在做决策时有多客观和公正？这项工作研究了人类专家在高风险决策过程中对认知偏见的识别，质疑其在现实环境中的有效性，例如大学录取的候选人评估。我们首先进行统计分析，评估当前流程中不同决策点之间的相关性，发现暗示认知偏见和决策不一致的差异。这促使我们探索超越人类判断的偏见感知人工智能增强工作流程。我们提出了 BGM-HAN，这是一个通过字节对编码、多头注意力和门控残差连接增强的分层注意力网络。使用它作为骨干模型，我们进一步提出了一个入围名单-分析-推荐 (SAR) 代理工作流程，模拟现实世界的决策。在我们的实验中，所提出的模型和代理工作流程都显著提高了人类判断和替代模型，并通过现实世界数据进行了验证。

Title: Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understanding

• Authors: Deyi Ji, Lanyun Zhu, Siqi Gao, Peng Xu, Hongtao Lu, Jieping Ye, Feng Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.08516
• Pdf URL: https://arxiv.org/pdf/2411.08516
• Copy Paste: [[2411.08516]] Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understanding(https://arxiv.org/abs/2411.08516)
• Keywords: language model, llm
• Abstract: The ubiquity and value of tables as semi-structured data across various domains necessitate advanced methods for understanding their complexity and vast amounts of information. Despite the impressive capabilities of large language models (LLMs) in advancing the natural language understanding frontier, their application to large-scale tabular data presents significant challenges, specifically regarding table size and complex intricate relationships. Existing works have shown promise with small-scale tables but often flounder when tasked with the complex reasoning required by larger, interconnected tables found in real-world scenarios. To address this gap, we introduce "Tree-of-Table", a novel approach designed to enhance LLMs' reasoning capabilities over large and complex tables. Our method employs Table Condensation and Decomposition to distill and reorganize relevant data into a manageable format, followed by the construction of a hierarchical Table-Tree that facilitates tree-structured reasoning. Through a meticulous Table-Tree Execution process, we systematically unravel the tree-structured reasoning chain to derive the solutions. Experiments across diverse datasets, including WikiTQ, TableFact, FeTaQA, and BIRD, demonstrate that Tree-of-Table sets a new benchmark with superior performance, showcasing remarkable efficiency and generalization capabilities in large-scale table reasoning.
• 摘要：表格作为半结构化数据在各个领域中的普遍性和价值，需要先进的方法来理解其复杂性和大量信息。尽管大型语言模型 (LLM) 在推进自然语言理解前沿方面具有令人印象深刻的能力，但它们在大规模表格数据中的应用仍面临重大挑战，特别是在表格大小和复杂错综复杂的关系方面。现有的研究已经显示出对小规模表格的前景，但在处理现实世界场景中较大、相互关联的表格所需的复杂推理时，它们往往会陷入困境。为了解决这一差距，我们引入了“表树”，这是一种旨在增强 LLM 对大型复杂表格的推理能力的新方法。我们的方法采用表格压缩和分解来将相关数据提炼和重新组织为可管理的格式，然后构建一个层次化的表树，以促进树结构推理。通过细致的表树执行过程，我们系统地解开树结构推理链以得出解决方案。在 WikiTQ、TableFact、FeTaQA 和 BIRD 等不同数据集上的实验表明，Tree-of-Table 以卓越的性能树立了新的基准，在大规模表格推理中展现出卓越的效率和泛化能力。

Title: Neural Topic Modeling with Large Language Models in the Loop

• Authors: Xiaohao Yang, He Zhao, Weijie Xu, Yuanyuan Qi, Jueqing Lu, Dinh Phung, Lan Du
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.08534
• Pdf URL: https://arxiv.org/pdf/2411.08534
• Copy Paste: [[2411.08534]] Neural Topic Modeling with Large Language Models in the Loop(https://arxiv.org/abs/2411.08534)
• Keywords: language model, llm
• Abstract: Topic modeling is a fundamental task in natural language processing, allowing the discovery of latent thematic structures in text corpora. While Large Language Models (LLMs) have demonstrated promising capabilities in topic discovery, their direct application to topic modeling suffers from issues such as incomplete topic coverage, misalignment of topics, and inefficiency. To address these limitations, we propose LLM-ITL, a novel LLM-in-the-loop framework that integrates LLMs with many existing Neural Topic Models (NTMs). In LLM-ITL, global topics and document representations are learned through the NTM, while an LLM refines the topics via a confidence-weighted Optimal Transport (OT)-based alignment objective. This process enhances the interpretability and coherence of the learned topics, while maintaining the efficiency of NTMs. Extensive experiments demonstrate that LLM-ITL can help NTMs significantly improve their topic interpretability while maintaining the quality of document representation.
• 摘要：主题建模是自然语言处理中的一项基本任务，可用于发现文本语料库中的潜在主题结构。虽然大型语言模型 (LLM) 在主题发现方面表现出良好的能力，但它们直接应用于主题建模存在诸如主题覆盖不完整、主题错位和效率低下等问题。为了解决这些限制，我们提出了 LLM-ITL，这是一种新颖的 LLM-in-the-loop 框架，将 LLM 与许多现有的神经主题模型 (NTM) 集成在一起。在 LLM-ITL 中，全局主题和文档表示是通过 NTM 学习的，而 LLM 通过基于置信度加权的最佳传输 (OT) 的对齐目标来细化主题。此过程增强了所学习主题的可解释性和连贯性，同时保持了 NTM 的效率。大量实验表明，LLM-ITL 可以帮助 NTM 显著提高其主题可解释性，同时保持文档表示的质量。

Title: CorrSynth -- A Correlated Sampling Method for Diverse Dataset Generation from LLMs

• Authors: Suhas S Kowshik, Abhishek Divekar, Vijit Malik
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.08553
• Pdf URL: https://arxiv.org/pdf/2411.08553
• Copy Paste: [[2411.08553]] CorrSynth -- A Correlated Sampling Method for Diverse Dataset Generation from LLMs(https://arxiv.org/abs/2411.08553)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) have demonstrated remarkable performance in diverse tasks using zero-shot and few-shot prompting. Even though their capabilities of data synthesis have been studied well in recent years, the generated data suffers from a lack of diversity, less adherence to the prompt, and potential biases that creep into the data from the generator model. In this work, we tackle the challenge of generating datasets with high diversity, upon which a student model is trained for downstream tasks. Taking the route of decoding-time guidance-based approaches, we propose CorrSynth, which generates data that is more diverse and faithful to the input prompt using a correlated sampling strategy. Further, our method overcomes the complexity drawbacks of some other guidance-based techniques like classifier-based guidance. With extensive experiments, we show the effectiveness of our approach and substantiate our claims. In particular, we perform intrinsic evaluation to show the improvements in diversity. Our experiments show that CorrSynth improves both student metrics and intrinsic metrics upon competitive baselines across four datasets, showing the innate advantage of our method.
• 摘要：大型语言模型 (LLM) 已在使用零样本和少样本提示的各种任务中表现出色。尽管近年来它们的数据合成能力得到了很好的研究，但生成的数据仍存在缺乏多样性、对提示的遵守程度较低以及生成器模型数据中可能存在偏差的问题。在这项工作中，我们解决了生成具有高多样性的数据集的挑战，在此数据集上训练学生模型以完成下游任务。我们采用了基于解码时间指导的方法，提出了 CorrSynth，它使用相关采样策略生成更加多样化且忠实于输入提示的数据。此外，我们的方法克服了其他一些基于指导的技术（如基于分类器的指导）的复杂性缺陷。通过大量实验，我们展示了我们方法的有效性并证实了我们的说法。特别是，我们进行了内在评估以显示多样性的改进。我们的实验表明，CorrSynth 在四个数据集的竞争基线上同时提高了学生指标和内在指标，显示了我们方法的先天优势。

Title: Dynamic Subset Tuning: Expanding the Operational Range of Parameter-Efficient Training for Large Language Models

• Authors: Felix Stahlberg, Jared Lichtarge, Shankar Kumar
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.08610
• Pdf URL: https://arxiv.org/pdf/2411.08610
• Copy Paste: [[2411.08610]] Dynamic Subset Tuning: Expanding the Operational Range of Parameter-Efficient Training for Large Language Models(https://arxiv.org/abs/2411.08610)
• Keywords: language model, prompt
• Abstract: We propose a novel parameter-efficient training (PET) method for large language models that adapts models to downstream tasks by optimizing a small subset of the existing model parameters. Unlike prior methods, this subset is not fixed in location but rather which parameters are modified evolves over the course of training. This dynamic parameter selection can yield good performance with many fewer parameters than extant methods. Our method enables a seamless scaling of the subset size across an arbitrary proportion of the total model size, while popular PET approaches like prompt tuning and LoRA cover only a small part of this spectrum. We match or outperform prompt tuning and LoRA in most cases on a variety of NLP tasks (MT, QA, GSM8K, SuperGLUE) for a given parameter budget across different model families and sizes.
• 摘要：我们针对大型语言模型提出了一种新颖的参数高效训练 (PET) 方法，通过优化现有模型参数的一小部分，使模型适应下游任务。与之前的方法不同，此子集的位置并不固定，而是在训练过程中会不断修改哪些参数。这种动态参数选择可以用比现有方法少得多的参数获得良好的性能。我们的方法能够在模型总大小的任意比例上无缝缩放子集大小，而流行的 PET 方法（如即时调整和 LoRA）仅覆盖了这一范围的一小部分。在大多数情况下，我们在各种 NLP 任务（MT、QA、GSM8K、SuperGLUE）上的表现与即时调整和 LoRA 相当或优于它们，适用于不同模型系列和大小的给定参数预算。

Title: Are Triggers Needed for Document-Level Event Extraction?

• Authors: Shaden Shaar, Wayne Chen, Maitreyi Chatterjee, Barry Wang, Wenting Zhao, Claire Cardie
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.08708
• Pdf URL: https://arxiv.org/pdf/2411.08708
• Copy Paste: [[2411.08708]] Are Triggers Needed for Document-Level Event Extraction?(https://arxiv.org/abs/2411.08708)
• Keywords: llm, prompt
• Abstract: Most existing work on event extraction has focused on sentence-level texts and presumes the identification of a trigger-span -- a word or phrase in the input that evokes the occurrence of an event of interest. Event arguments are then extracted with respect to the trigger. Indeed, triggers are treated as integral to, and trigger detection as an essential component of, event extraction. In this paper, we provide the first investigation of the role of triggers for the more difficult and much less studied task of document-level event extraction. We analyze their usefulness in multiple end-to-end and pipelined neural event extraction models for three document-level event extraction datasets, measuring performance using triggers of varying quality (human-annotated, LLM-generated, keyword-based, and random). Our research shows that trigger effectiveness varies based on the extraction task's characteristics and data quality, with basic, automatically-generated triggers serving as a viable alternative to human-annotated ones. Furthermore, providing detailed event descriptions to the extraction model helps maintain robust performance even when trigger quality degrades. Perhaps surprisingly, we also find that the mere existence of trigger input, even random ones, is important for prompt-based LLM approaches to the task.
• 摘要：现有的大多数事件提取研究都集中在句子级文本上，并假设识别触发范围——输入中引起感兴趣事件发生的单词或短语。然后根据触发器提取事件参数。事实上，触发器被视为事件提取不可或缺的一部分，触发器检测是事件提取的重要组成部分。在本文中，我们首次研究了触发器在更困难且研究较少的文档级事件提取任务中的作用。我们分析了它们在三个文档级事件提取数据集的多个端到端和流水线神经事件提取模型中的实用性，使用不同质量的触发器（人工注释、LLM 生成、基于关键字和随机）来测量性能。我们的研究表明，触发器的有效性因提取任务的特征和数据质量而异，基本、自动生成的触发器是人工注释触发器的可行替代方案。此外，向提取模型提供详细的事件描述有助于在触发器质量下降时保持强大的性能。也许令人惊讶的是，我们还发现，触发输入的存在，即使是随机的输入，对于基于提示的 LLM 方法来说也很重要。

Title: QCG-Rerank: Chunks Graph Rerank with Query Expansion in Retrieval-Augmented LLMs for Tourism Domain

• Authors: Qikai Wei, Mingzhi Yang, Chunlong Han, Jingfu Wei, Minghao Zhang, Feifei Shi, Huansheng Ning
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08724
• Pdf URL: https://arxiv.org/pdf/2411.08724
• Copy Paste: [[2411.08724]] QCG-Rerank: Chunks Graph Rerank with Query Expansion in Retrieval-Augmented LLMs for Tourism Domain(https://arxiv.org/abs/2411.08724)
• Keywords: language model, llm, hallucination, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) mitigates the issue of hallucination in Large Language Models (LLMs) by integrating information retrieval techniques. However, in the tourism domain, since the query is usually brief and the content in the database is diverse, existing RAG may contain a significant amount of irrelevant or contradictory information contents after retrieval. To address this challenge, we propose the QCG-Rerank model. This model first performs an initial retrieval to obtain candidate chunks and then enhances semantics by extracting critical information to expand the original query. Next, we utilize the expanded query and candidate chunks to calculate similarity scores as the initial transition probability and construct the chunks graph. Subsequently, We iteratively compute the transition probabilities based on an initial estimate until convergence. The chunks with the highest score are selected and input into the LLMs to generate responses. We evaluate the model on Cultour, IIRC, StrategyQA, HotpotQA, SQuAD, and MuSiQue datasets. The experimental results demonstrate the effectiveness and superiority of the QCG-Rerank method.
• 摘要：检索增强生成 (RAG) 通过集成信息检索技术缓解了大型语言模型 (LLM) 中的幻觉问题。然而，在旅游领域，由于查询通常很简短且数据库中的内容多样，现有的 RAG 可能在检索后包含大量不相关或矛盾的信息内容。为了解决这一挑战，我们提出了 QCG-Rerank 模型。该模型首先执行初始检索以获取候选块，然后通过提取关键信息来扩展原始查询以增强语义。接下来，我们利用扩展的查询和候选块来计算相似度得分作为初始转移概率并构建块图。随后，我们根据初始估计迭代计算转移概率，直到收敛。选择得分最高的块并将其输入到 LLM 中以生成响应。我们在 Cultour、IIRC、StrategyQA、HotpotQA、SQuAD 和 MuSiQue 数据集上评估了该模型。实验结果证明了QCG-Rerank方法的有效性和优越性。

Title: Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models

• Authors: Somanshu Singla, Zhen Wang, Tianyang Liu, Abdullah Ashfaq, Zhiting Hu, Eric P. Xing
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.08733
• Pdf URL: https://arxiv.org/pdf/2411.08733
• Copy Paste: [[2411.08733]] Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models(https://arxiv.org/abs/2411.08733)
• Keywords: language model, llm, prompt
• Abstract: Aligning Large Language Models (LLMs) traditionally relies on costly training and human preference annotations. Self-alignment seeks to reduce these expenses by enabling models to align themselves. To further lower costs and achieve alignment without any expensive tuning or annotations, we introduce a new tuning-free approach for self-alignment, Dynamic Rewarding with Prompt Optimization (\ours). Our approach leverages a search-based optimization framework that allows LLMs to iteratively self-improve and craft the optimal alignment instructions, all without additional training or human intervention. The core of \ours is a dynamic rewarding mechanism, which identifies and rectifies model-specific alignment weaknesses, allowing LLMs to adapt efficiently to diverse alignment challenges. Empirical evaluations on eight recent LLMs, both open- and closed-sourced, demonstrate that \ours significantly enhances alignment performance, with base models outperforming their SFT/RLHF-tuned counterparts. Moreover, the prompts automatically optimized by \ours surpass those curated by human experts, further validating the effectiveness of our approach. Our findings highlight the great potential of current LLMs to achieve adaptive self-alignment through inference-time optimization, complementing tuning-based alignment methods.
• 摘要：对齐大型语言模型 (LLM) 传统上依赖于昂贵的训练和人类偏好注释。自对齐旨在通过使模型能够自我对齐来减少这些费用。为了进一步降低成本并在没有任何昂贵的调整或注释的情况下实现对齐，我们引入了一种新的无需调整的自对齐方法，即动态奖励和提示优化 (\ours)。我们的方法利用基于搜索的优化框架，允许 LLM 迭代地自我改进并制定最佳对齐指令，所有这些都无需额外的训练或人工干预。\ours 的核心是一种动态奖励机制，它可以识别和纠正特定于模型的对齐弱点，使 LLM 能够有效地适应各种对齐挑战。对八个最近的 LLM（开源和闭源）的实证评估表明，\ours 显著提高了对齐性能，基础模型的表现优于 SFT/RLHF 调整后的模型。此外，\ours 自动优化的提示超过了人类专家策划的提示，进一步验证了我们方法的有效性。我们的研究结果凸显了当前 LLM 通过推理时间优化实现自适应自对齐的巨大潜力，补充了基于调整的对齐方法。

Title: A Comparative Study of Discrete Speech Tokens for Semantic-Related Tasks with Large Language Models

• Authors: Dingdong Wang, Mingyu Cui, Dongchao Yang, Xueyuan Chen, Helen Meng
• Subjects: cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2411.08742
• Pdf URL: https://arxiv.org/pdf/2411.08742
• Copy Paste: [[2411.08742]] A Comparative Study of Discrete Speech Tokens for Semantic-Related Tasks with Large Language Models(https://arxiv.org/abs/2411.08742)
• Keywords: language model, llm
• Abstract: With the rise of Speech Large Language Models (Speech LLMs), there has been growing interest in discrete speech tokens for their ability to integrate with text-based tokens seamlessly. Compared to most studies that focus on continuous speech features, although discrete-token based LLMs have shown promising results on certain tasks, the performance gap between these two paradigms is rarely explored. In this paper, we present a fair and thorough comparison between discrete and continuous features across a variety of semantic-related tasks using a light-weight LLM (Qwen1.5-0.5B). Our findings reveal that continuous features generally outperform discrete tokens, particularly in tasks requiring fine-grained semantic understanding. Moreover, this study goes beyond surface-level comparison by identifying key factors behind the under-performance of discrete tokens, such as limited token granularity and inefficient information retention. To enhance the performance of discrete tokens, we explore potential aspects based on our analysis. We hope our results can offer new insights into the opportunities for advancing discrete speech tokens in Speech LLMs.
• 摘要：随着语音大型语言模型 (Speech LLM) 的兴起，人们对离散语音标记的兴趣日益浓厚，因为它们能够与基于文本的标记无缝集成。与大多数专注于连续语音特征的研究相比，尽管基于离散标记的 LLM 在某些任务上表现出了良好的效果，但这两种范式之间的性能差距却很少被探究。在本文中，我们使用轻量级 LLM (Qwen1.5-0.5B) 对各种语义相关任务中的离散特征和连续特征进行了公平而彻底的比较。我们的研究结果表明，连续特征通常优于离散标记，特别是在需要细粒度语义理解的任务中。此外，这项研究超越了表面级别的比较，通过确定离散标记表现不佳的关键因素，例如有限的标记粒度和低效的信息保留。为了提高离散标记的性能，我们根据分析探索了潜在的方面。我们希望我们的结果能够为语音 LLM 中离散语音标记的改进提供新的见解。

Title: Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers

• Authors: Clément Dumas, Chris Wendler, Veniamin Veselovsky, Giovanni Monea, Robert West
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08745
• Pdf URL: https://arxiv.org/pdf/2411.08745
• Copy Paste: [[2411.08745]] Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers(https://arxiv.org/abs/2411.08745)
• Keywords: language model, llm, prompt
• Abstract: A central question in multilingual language modeling is whether large language models (LLMs) develop a universal concept representation, disentangled from specific languages. In this paper, we address this question by analyzing latent representations (latents) during a word translation task in transformer-based LLMs. We strategically extract latents from a source translation prompt and insert them into the forward pass on a target translation prompt. By doing so, we find that the output language is encoded in the latent at an earlier layer than the concept to be translated. Building on this insight, we conduct two key experiments. First, we demonstrate that we can change the concept without changing the language and vice versa through activation patching alone. Second, we show that patching with the mean over latents across different languages does not impair and instead improves the models' performance in translating the concept. Our results provide evidence for the existence of language-agnostic concept representations within the investigated models.
• 摘要：多语言建模的一个核心问题是大型语言模型 (LLM) 是否能开发出一种通用的概念表征，与特定语言区分开来。在本文中，我们通过分析基于 Transformer 的 LLM 中单词翻译任务中的潜在表征 (latents) 来解决这个问题。我们从源翻译提示中策略性地提取潜在表征，并将其插入到目标翻译提示的前向传递中。通过这样做，我们发现输出语言在潜在表征中的编码比要翻译的概念更早。基于这一见解，我们进行了两个关键实验。首先，我们证明我们可以在不改变语言的情况下改变概念，反之亦然，这仅通过激活修补即可。其次，我们表明，使用不同语言的潜在表征的平均值进行修补不会损害模型在翻译概念方面的表现，反而会提高模型的表现。我们的结果为所研究模型中存在与语言无关的概念表征提供了证据。

Title: CamemBERT 2.0: A Smarter French Language Model Aged to Perfection

• Authors: Wissam Antoun, Francis Kulumba, Rian Touchent, Éric de la Clergerie, Benoît Sagot, Djamé Seddah
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.08868
• Pdf URL: https://arxiv.org/pdf/2411.08868
• Copy Paste: [[2411.08868]] CamemBERT 2.0: A Smarter French Language Model Aged to Perfection(https://arxiv.org/abs/2411.08868)
• Keywords: language model
• Abstract: French language models, such as CamemBERT, have been widely adopted across industries for natural language processing (NLP) tasks, with models like CamemBERT seeing over 4 million downloads per month. However, these models face challenges due to temporal concept drift, where outdated training data leads to a decline in performance, especially when encountering new topics and terminology. This issue emphasizes the need for updated models that reflect current linguistic trends. In this paper, we introduce two new versions of the CamemBERT base model-CamemBERTav2 and CamemBERTv2-designed to address these challenges. CamemBERTav2 is based on the DeBERTaV3 architecture and makes use of the Replaced Token Detection (RTD) objective for better contextual understanding, while CamemBERTv2 is built on RoBERTa, which uses the Masked Language Modeling (MLM) objective. Both models are trained on a significantly larger and more recent dataset with longer context length and an updated tokenizer that enhances tokenization performance for French. We evaluate the performance of these models on both general-domain NLP tasks and domain-specific applications, such as medical field tasks, demonstrating their versatility and effectiveness across a range of use cases. Our results show that these updated models vastly outperform their predecessors, making them valuable tools for modern NLP systems. All our new models, as well as intermediate checkpoints, are made openly available on Huggingface.
• 摘要：法语模型（例如 CamemBERT）已广泛应用于各行各业的自然语言处理 (NLP) 任务，CamemBERT 等模型每月的下载量超过 400 万次。然而，这些模型面临着时间概念漂移带来的挑战，过时的训练数据会导致性能下降，尤其是在遇到新主题和术语时。这个问题强调了更新模型以反映当前语言趋势的必要性。在本文中，我们介绍了 CamemBERT 基础模型的两个新版本 - CamemBERTav2 和 CamemBERTv2 - 旨在应对这些挑战。CamemBERTav2 基于 DeBERTaV3 架构，利用替换标记检测 (RTD) 目标来更好地理解上下文，而 CamemBERTv2 则基于 RoBERTa 构建，后者使用掩码语言建模 (MLM) 目标。这两个模型都是在更大、更新的数据集上训练的，具有更长的上下文长度和更新的标记器，可增强法语的标记化性能。我们评估了这些模型在通用领域的 NLP 任务和特定领域的应用（例如医疗领域的任务）上的性能，展示了它们在一系列用例中的多功能性和有效性。我们的结果表明，这些更新后的模型远远优于其前身，使其成为现代 NLP 系统的宝贵工具。我们所有的新模型以及中间检查点都在 Huggingface 上公开提供。

Title: The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models

• Authors: Daniel P. Jeong, Pranav Mani, Saurabh Garg, Zachary C. Lipton, Michael Oberst
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.08870
• Pdf URL: https://arxiv.org/pdf/2411.08870
• Copy Paste: [[2411.08870]] The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models(https://arxiv.org/abs/2411.08870)
• Keywords: language model, llm, prompt
• Abstract: Several recent works seek to develop foundation models specifically for medical applications, adapting general-purpose large language models (LLMs) and vision-language models (VLMs) via continued pretraining on publicly available biomedical corpora. These works typically claim that such domain-adaptive pretraining (DAPT) improves performance on downstream medical tasks, such as answering medical licensing exam questions. In this paper, we compare ten public "medical" LLMs and two VLMs against their corresponding base models, arriving at a different conclusion: all medical VLMs and nearly all medical LLMs fail to consistently improve over their base models in the zero-/few-shot prompting and supervised fine-tuning regimes for medical question-answering (QA). For instance, across all tasks and model pairs we consider in the 3-shot setting, medical LLMs only outperform their base models in 22.7% of cases, reach a (statistical) tie in 36.8% of cases, and are significantly worse than their base models in the remaining 40.5% of cases. Our conclusions are based on (i) comparing each medical model head-to-head, directly against the corresponding base model; (ii) optimizing the prompts for each model separately in zero-/few-shot prompting; and (iii) accounting for statistical uncertainty in comparisons. While these basic practices are not consistently adopted in the literature, our ablations show that they substantially impact conclusions. Meanwhile, we find that after fine-tuning on specific QA tasks, medical LLMs can show performance improvements, but the benefits do not carry over to tasks based on clinical notes. Our findings suggest that state-of-the-art general-domain models may already exhibit strong medical knowledge and reasoning capabilities, and offer recommendations to strengthen the conclusions of future studies.
• 摘要：最近有几篇论文致力于开发专门用于医学应用的基础模型，通过在公开的生物医学语料库上进行持续预训练来调整通用大型语言模型 (LLM) 和视觉语言模型 (VLM)。这些论文通常声称，这种领域自适应预训练 (DAPT) 可以提高下游医学任务（例如回答医学执照考试问题）的性能。在本文中，我们将十个公共“医学”LLM 和两个 VLM 与它们相应的基础模型进行比较，得出了一个不同的结论：所有医学 VLM 和几乎所有医学 LLM 在医学问答 (QA) 的零次/少量提示和监督微调机制中都无法持续改进其基础模型。例如，在我们考虑的 3 次设置的所有任务和模型对中，医学 LLM 仅在 22.7% 的情况下优于其基本模型，在 36.8% 的情况下达到（统计）平局，在其余 40.5% 的情况下明显差于其基本模型。我们的结论基于 (i) 将每个医学模型直接与相应的基本模型进行正面比较；(ii) 在零次/少次提示中分别优化每个模型的提示；(iii) 在比较中考虑统计不确定性。虽然这些基本做法在文献中并未得到一致采用，但我们的修正表明它们对结论有重大影响。同时，我们发现在对特定的 QA 任务进行微调后，医学 LLM 可以显示出性能改进，但这些好处不会延续到基于临床记录的任务中。我们的研究结果表明，最先进的通用领域模型可能已经表现出强大的医学知识和推理能力，并提出了加强未来研究结论的建议。