2024-11-15

Title: CoCoP: Enhancing Text Classification with LLM through Code Completion Prompt

• Authors: Mohammad Mahdi Mohajeri, Mohammad Javad Dousti, Majid Nili Ahmadabadi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.08979
• Pdf URL: https://arxiv.org/pdf/2411.08979
• Copy Paste: [[2411.08979]] CoCoP: Enhancing Text Classification with LLM through Code Completion Prompt(https://arxiv.org/abs/2411.08979)
• Keywords: language model, llm, prompt
• Abstract: Text classification is a fundamental task in natural language processing (NLP), and large language models (LLMs) have demonstrated their capability to perform this task across various domains. However, the performance of LLMs heavily depends on the quality of their input prompts. Recent studies have also shown that LLMs exhibit remarkable results in code-related tasks. To leverage the capabilities of LLMs in text classification, we propose the Code Completion Prompt (CoCoP) method, which transforms the text classification problem into a code completion task. CoCoP significantly improves text classification performance across diverse datasets by utilizing LLMs' code-completion capability. For instance, CoCoP enhances the accuracy of the SST2 dataset by more than 20%. Moreover, when CoCoP integrated with LLMs specifically designed for code-related tasks (code models), such as CodeLLaMA, this method demonstrates better or comparable performance to few-shot learning techniques while using only one-tenth of the model size. The source code of our proposed method will be available to the public upon the acceptance of the paper.
• 摘要：文本分类是自然语言处理 (NLP) 中的一项基本任务，大型​​语言模型 (LLM) 已证明其能够在各个领域执行此任务。然而，LLM 的性能在很大程度上取决于其输入提示的质量。最近的研究还表明，LLM 在代码相关任务中表现出色。为了利用 LLM 在文本分类中的能力，我们提出了代码完成提示 (CoCoP) 方法，该方法将文本分类问题转化为代码完成任务。CoCoP 利用 LLM 的代码完成功能显著提高了不同数据集的文本分类性能。例如，CoCoP 将 SST2 数据集的准确率提高了 20% 以上。此外，当 CoCoP 与专为代码相关任务设计的 LLM（代码模型）集成时，例如 CodeLLaMA，该方法仅使用十分之一的模型大小，就表现出比小样本学习技术更好或相当的性能。我们提出的方法的源代码将在论文被接受后向公众开放。

Title: Code-mixed LLM: Improve Large Language Models' Capability to Handle Code-Mixing through Reinforcement Learning from AI Feedback

• Authors: Wenbo Zhang, Aditya Majumdar, Amulya Yadav
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.09073
• Pdf URL: https://arxiv.org/pdf/2411.09073
• Copy Paste: [[2411.09073]] Code-mixed LLM: Improve Large Language Models' Capability to Handle Code-Mixing through Reinforcement Learning from AI Feedback(https://arxiv.org/abs/2411.09073)
• Keywords: language model, llm
• Abstract: Code-mixing(CM) or code-switching(CSW) refers to the juxtaposition of linguistic units from two or more languages during the conversation or sometimes even a single utterance. Code-mixing introduces unique challenges in daily life, such as syntactic mismatches and semantic blending, that are rarely encountered in monolingual settings. Large language models (LLMs) have revolutionized the field of natural language processing (NLP) by offering unprecedented capabilities in understanding human languages. However, the effectiveness of current state-of-the-art multilingual LLMs has not yet been fully explored in the CM scenario. To fill this gap, we first benchmark the performance of multilingual LLMs on various code-mixing NLP tasks. Then we propose to improve the multilingual LLMs' ability to understand code-mixing through reinforcement learning from human feedback (RLHF) and code-mixed machine translation tasks. Given the high-cost and time-consuming preference labeling procedure, we improve this by utilizing LLMs as annotators to perform the reinforcement learning from AI feedback (RLAIF). The experiments show the effectiveness of the proposed method.
• 摘要：代码混合 (CM) 或代码转换 (CSW) 是指在对话过程中，甚至有时在单个话语中，将两种或多种语言的语言单位并置。代码混合在日常生活中带来了独特的挑战，例如句法不匹配和语义混合，而这些挑战在单语环境中很少遇到。大型语言模型 (LLM) 通过提供前所未有的理解人类语言的能力，彻底改变了自然语言处理 (NLP) 领域。然而，目前最先进的多语言 LLM 的有效性尚未在 CM 场景中得到充分探索。为了填补这一空白，我们首先在各种代码混合 NLP 任务上对多语言 LLM 的性能进行基准测试。然后，我们建议通过从人类反馈 (RLHF) 和代码混合机器翻译任务中进行强化学习来提高多语言 LLM 理解代码混合的能力。考虑到偏好标记程序成本高且耗时，我们通过使用 LLM 作为注释器来执行从 AI 反馈 (RLAIF) 进行的强化学习来改进这一点。实验证明了所提方法的有效性。

Title: Personalized Help for Optimizing Low-Skilled Users' Strategy

• Authors: Feng Gu, Wichayaporn Wongkamjan, Jordan Lee Boyd-Graber, Jonathan K. Kummerfeld, Denis Peskoff, Jonathan May
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09109
• Pdf URL: https://arxiv.org/pdf/2411.09109
• Copy Paste: [[2411.09109]] Personalized Help for Optimizing Low-Skilled Users' Strategy(https://arxiv.org/abs/2411.09109)
• Keywords: agent
• Abstract: AIs can beat humans in game environments; however, how helpful those agents are to human remains understudied. We augment CICERO, a natural language agent that demonstrates superhuman performance in Diplomacy, to generate both move and message advice based on player intentions. A dozen Diplomacy games with novice and experienced players, with varying advice settings, show that some of the generated advice is beneficial. It helps novices compete with experienced players and in some instances even surpass them. The mere presence of advice can be advantageous, even if players do not follow it.
• 摘要：人工智能可以在游戏环境中击败人类；然而，这些代理对人类的帮助程度仍未得到充分研究。我们增强了 CICERO，这是一种在外交方面表现出超人表现的自然语言代理，可以根据玩家意图生成移动和消息建议。十几场外交游戏，新手和有经验的玩家参与，建议设置各不相同，结果表明，生成的一些建议是有益的。它可以帮助新手与有经验的玩家竞争，在某些情况下甚至超越他们。即使玩家不遵循建议，仅仅存在建议也是有利的。

Title: P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs

• Authors: Yidan Zhang, Boyi Deng, Yu Wan, Baosong Yang, Haoran Wei, Fei Huang, Bowen Yu, Junyang Lin, Fei Huang, Jingren Zhou
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09116
• Pdf URL: https://arxiv.org/pdf/2411.09116
• Copy Paste: [[2411.09116]] P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs(https://arxiv.org/abs/2411.09116)
• Keywords: language model, llm, prompt
• Abstract: Recent advancements in large language models (LLMs) showcase varied multilingual capabilities across tasks like translation, code generation, and reasoning. Previous assessments often limited their scope to fundamental natural language processing (NLP) or isolated capability-specific tasks. To alleviate this drawback, we aim to present a comprehensive multilingual multitask benchmark. First, we present a pipeline for selecting available and reasonable benchmarks from massive ones, addressing the oversight in previous work regarding the utility of these benchmarks, i.e., their ability to differentiate between models being evaluated. Leveraging this pipeline, we introduce P-MMEval, a large-scale benchmark covering effective fundamental and capability-specialized datasets. Furthermore, P-MMEval delivers consistent language coverage across various datasets and provides parallel samples. Finally, we conduct extensive experiments on representative multilingual model series to compare performances across models, analyze dataset effectiveness, examine prompt impacts on model performances, and explore the relationship between multilingual performances and factors such as tasks, model sizes, and languages. These insights offer valuable guidance for future research. The dataset is available at this https URL.
• 摘要：大型语言模型 (LLM) 的最新进展展示了在翻译、代码生成和推理等任务中多种多样的多语言能力。以前的评估通常将其范围限制在基本的自然语言处理 (NLP) 或孤立的特定能力任务上。为了缓解这一缺点，我们旨在提出一个全面的多语言多任务基准。首先，我们提出了一个从大量基准中选择可用且合理的基准的流程，解决了以前工作中对这些基准的实用性（即它们区分被评估模型的能力）的疏忽。利用这个流程，我们引入了 P-MMEval，这是一个涵盖有效基础和能力专业化数据集的大规模基准。此外，P-MMEval 可在各种数据集中提供一致的语言覆盖范围并提供并行样本。最后，我们对代表性多语言模型系列进行了广泛的实验，以比较不同模型的性能，分析数据集的有效性，检查对模型性能的直接影响，并探索多语言性能与任务、模型大小和语言等因素之间的关系。这些见解为未来的研究提供了宝贵的指导。数据集可在此 https URL 上获得。

Title: DROJ: A Prompt-Driven Attack against Large Language Models

• Authors: Leyang Hu, Boran Wang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.09125
• Pdf URL: https://arxiv.org/pdf/2411.09125
• Copy Paste: [[2411.09125]] DROJ: A Prompt-Driven Attack against Large Language Models(https://arxiv.org/abs/2411.09125)
• Keywords: language model, llm, prompt, chat
• Abstract: Large Language Models (LLMs) have demonstrated exceptional capabilities across various natural language processing tasks. Due to their training on internet-sourced datasets, LLMs can sometimes generate objectionable content, necessitating extensive alignment with human feedback to avoid such outputs. Despite massive alignment efforts, LLMs remain susceptible to adversarial jailbreak attacks, which usually are manipulated prompts designed to circumvent safety mechanisms and elicit harmful responses. Here, we introduce a novel approach, Directed Rrepresentation Optimization Jailbreak (DROJ), which optimizes jailbreak prompts at the embedding level to shift the hidden representations of harmful queries towards directions that are more likely to elicit affirmative responses from the model. Our evaluations on LLaMA-2-7b-chat model show that DROJ achieves a 100\% keyword-based Attack Success Rate (ASR), effectively preventing direct refusals. However, the model occasionally produces repetitive and non-informative responses. To mitigate this, we introduce a helpfulness system prompt that enhances the utility of the model's responses. Our code is available at this https URL.
• 摘要：大型语言模型 (LLM) 在各种自然语言处理任务中都表现出了卓越的能力。由于 LLM 是在互联网数据集上进行训练的，因此有时会生成令人反感的内容，因此需要与人工反馈进行广泛的协调以避免此类输出。尽管进行了大量的协调工作，但 LLM 仍然容易受到对抗性越狱攻击，这些攻击通常是操纵提示，旨在绕过安全机制并引发有害响应。在这里，我们介绍了一种新方法，即定向表示优化越狱 (DROJ)，它在嵌入级别优化越狱提示，将有害查询的隐藏表示转移到更有可能从模型中引发肯定响应的方向。我们对 LLaMA-2-7b-chat 模型的评估表明，DROJ 实现了 100% 基于关键字的攻击成功率 (ASR)，有效地防止了直接拒绝。但是，该模型偶尔会产生重复且无信息的响应。为了缓解这种情况，我们引入了一个有用系统提示，以增强模型响应的实用性。我们的代码可在此 https URL 上获取。

Title: Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering

• Authors: Nghia Trung Ngo, Chien Van Nguyen, Franck Dernoncourt, Thien Huu Nguyen
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2411.09213
• Pdf URL: https://arxiv.org/pdf/2411.09213
• Copy Paste: [[2411.09213]] Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering(https://arxiv.org/abs/2411.09213)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-augmented generation (RAG) has emerged as a promising approach to enhance the performance of large language models (LLMs) in knowledge-intensive tasks such as those from medical domain. However, the sensitive nature of the medical domain necessitates a completely accurate and trustworthy system. While existing RAG benchmarks primarily focus on the standard retrieve-answer setting, they overlook many practical scenarios that measure crucial aspects of a reliable medical system. This paper addresses this gap by providing a comprehensive evaluation framework for medical question-answering (QA) systems in a RAG setting for these situations, including sufficiency, integration, and robustness. We introduce Medical Retrieval-Augmented Generation Benchmark (MedRGB) that provides various supplementary elements to four medical QA datasets for testing LLMs' ability to handle these specific scenarios. Utilizing MedRGB, we conduct extensive evaluations of both state-of-the-art commercial LLMs and open-source models across multiple retrieval conditions. Our experimental results reveals current models' limited ability to handle noise and misinformation in the retrieved documents. We further analyze the LLMs' reasoning processes to provides valuable insights and future directions for developing RAG systems in this critical medical domain.
• 摘要：检索增强生成 (RAG) 已成为一种有前途的方法，可提高大型语言模型 (LLM) 在知识密集型任务（例如医学领域的任务）中的性能。然而，医学领域的敏感性要求系统完全准确且值得信赖。虽然现有的 RAG 基准主要关注标准的检索答案设置，但它们忽略了许多实际场景，这些场景衡量了可靠医疗系统的关键方面。本文通过为这些情况下的 RAG 环境中的医学问答 (QA) 系统提供全面的评估框架来解决这一差距，包括充分性、集成性和稳健性。我们引入了医学检索增强生成基准 (MedRGB)，它为四个医学 QA 数据集提供了各种补充元素，以测试 LLM 处理这些特定场景的能力。利用 MedRGB，我们对多种检索条件下最先进的商业 LLM 和开源模型进行了广泛的评估。我们的实验结果揭示了当前模型处理检索到的文档中的噪声和错误信息的能力有限。我们进一步分析了 LLM 的推理过程，为在这个关键的医学领域开发 RAG 系统提供了宝贵的见解和未来方向。

Title: HateGPT: Unleashing GPT-3.5 Turbo to Combat Hate Speech on X

• Authors: Aniket Deroy, Subhankar Maity
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09214
• Pdf URL: https://arxiv.org/pdf/2411.09214
• Copy Paste: [[2411.09214]] HateGPT: Unleashing GPT-3.5 Turbo to Combat Hate Speech on X(https://arxiv.org/abs/2411.09214)
• Keywords: language model, gpt, prompt
• Abstract: The widespread use of social media platforms like Twitter and Facebook has enabled people of all ages to share their thoughts and experiences, leading to an immense accumulation of user-generated content. However, alongside the benefits, these platforms also face the challenge of managing hate speech and offensive content, which can undermine rational discourse and threaten democratic values. As a result, there is a growing need for automated methods to detect and mitigate such content, especially given the complexity of conversations that may require contextual analysis across multiple languages, including code-mixed languages like Hinglish, German-English, and Bangla. We participated in the English task where we have to classify English tweets into two categories namely Hate and Offensive and Non Hate-Offensive. In this work, we experiment with state-of-the-art large language models like GPT-3.5 Turbo via prompting to classify tweets into Hate and Offensive or Non Hate-Offensive. In this study, we evaluate the performance of a classification model using Macro-F1 scores across three distinct runs. The Macro-F1 score, which balances precision and recall across all classes, is used as the primary metric for model evaluation. The scores obtained are 0.756 for run 1, 0.751 for run 2, and 0.754 for run 3, indicating a high level of performance with minimal variance among the runs. The results suggest that the model consistently performs well in terms of precision and recall, with run 1 showing the highest performance. These findings highlight the robustness and reliability of the model across different runs.
• 摘要：Twitter 和 Facebook 等社交媒体平台的广泛使用使各个年龄段的人都可以分享他们的想法和经历，从而导致用户生成内容的大量积累。然而，除了这些好处之外，这些平台还面临着管理仇恨言论和冒犯性内容的挑战，这些内容可能会破坏理性的讨论并威胁民主价值观。因此，人们越来越需要自动化方法来检测和缓解此类内容，尤其是考虑到对话的复杂性，可能需要跨多种语言进行上下文分析，包括印式英语、德语-英语和孟加拉语等代码混合语言。我们参加了英语任务，其中我们必须将英语推文分为两类，即仇恨和冒犯性和非仇恨-冒犯性。在这项工作中，我们通过提示将推文分为仇恨和冒犯性或非仇恨-冒犯性，尝试使用最先进的大型语言模型（如 GPT-3.5 Turbo）。在这项研究中，我们使用 Macro-F1 分数在三次不同的运行中评估分类模型的性能。 Macro-F1 分数平衡了所有类别的准确率和召回率，是模型评估的主要指标。第一次运行获得的分数为 0.756，第二次运行获得的分数为 0.751，第三次运行获得的分数为 0.754，表明性能水平较高，且运行之间的差异最小。结果表明，该模型在准确率和召回率方面始终表现良好，其中第一次运行表现出最高的性能。这些发现凸显了模型在不同运行中的稳健性和可靠性。

Title: Enhancing Financial Domain Adaptation of Language Models via Model Augmentation

• Authors: Kota Tanabe, Masanori Hirano, Kazuki Matoya, Kentaro Imajo, Hiroki Sakaji, Itsuki Noda
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.09249
• Pdf URL: https://arxiv.org/pdf/2411.09249
• Copy Paste: [[2411.09249]] Enhancing Financial Domain Adaptation of Language Models via Model Augmentation(https://arxiv.org/abs/2411.09249)
• Keywords: language model, llm
• Abstract: The domain adaptation of language models, including large language models (LLMs), has become increasingly important as the use of such models continues to expand. This study demonstrates the effectiveness of Composition to Augment Language Models (CALM) in adapting to the financial domain. CALM is a model to extend the capabilities of existing models by introducing cross-attention between two LLMs with different functions. In our experiments, we developed a CALM to enhance the financial performance of an LLM with strong response capabilities by leveraging a financial-specialized LLM. Notably, the CALM was trained using a financial dataset different from the one used to train the financial-specialized LLM, confirming CALM's ability to adapt to various datasets. The models were evaluated through quantitative Japanese financial benchmarks and qualitative response comparisons, demonstrating that CALM enables superior responses with higher scores than the original models and baselines. Additionally, comparative experiments on connection points revealed that connecting the middle layers of the models is most effective in facilitating adaptation to the financial domain. These findings confirm that CALM is a practical approach for adapting LLMs to the financial domain.
• 摘要：随着语言模型（包括大型语言模型 (LLM)）的使用范围不断扩大，语言模型的领域适应性变得越来越重要。本研究证明了组合增强语言模型 (CALM) 在适应金融领域的有效性。CALM 是一种通过在两个具有不同功能的 LLM 之间引入交叉注意力来扩展现有模型功能的模型。在我们的实验中，我们开发了一个 CALM，通过利用金融专业 LLM 来增强具有强大响应能力的 LLM 的金融性能。值得注意的是，CALM 是使用与训练金融专业 LLM 所用的数据集不同的金融数据集进行训练的，这证实了 CALM 适应各种数据集的能力。通过定量的日本金融基准和定性的响应比较对模型进行了评估，表明 CALM 能够比原始模型和基线获得更高的分数和更优的响应。此外，对连接点的比较实验表明，连接模型的中间层最能有效地促进对金融领域的适应。这些发现证实了 CALM 是将 LLM 应用于金融领域的一种实用方法。

Title: DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicine

• Authors: Jean Seo, Jongwon Lim, Dongjun Jang, Hyopil Shin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09255
• Pdf URL: https://arxiv.org/pdf/2411.09255
• Copy Paste: [[2411.09255]] DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicine(https://arxiv.org/abs/2411.09255)
• Keywords: language model, llm, hallucination
• Abstract: We introduce DAHL, a benchmark dataset and automated evaluation system designed to assess hallucination in long-form text generation, specifically within the biomedical domain. Our benchmark dataset, meticulously curated from biomedical research papers, consists of 8,573 questions across 29 categories. DAHL evaluates fact-conflicting hallucinations in Large Language Models (LLMs) by deconstructing responses into atomic units, each representing a single piece of information. The accuracy of these responses is averaged to produce the DAHL Score, offering a more in-depth evaluation of hallucinations compared to previous methods that rely on multiple-choice tasks. We conduct experiments with 8 different models, finding that larger models tend to hallucinate less; however, beyond a model size of 7 to 8 billion parameters, further scaling does not significantly improve factual accuracy. The DAHL Score holds potential as an efficient alternative to human-annotated preference labels, being able to be expanded to other specialized domains. We release the dataset and code in public.
• 摘要：我们引入了 DAHL，这是一个基准数据集和自动评估系统，旨在评估长文本生成中的幻觉，特别是在生物医学领域。我们的基准数据集是从生物医学研究论文中精心挑选出来的，包含 29 个类别的 8,573 个问题。DAHL 通过将响应分解为原子单元来评估大型语言模型 (LLM) 中与事实相冲突的幻觉，每个原子单元代表一条信息。这些响应的准确性被平均以产生 DAHL 分数，与以前依赖多项选择任务的方法相比，它提供了更深入的幻觉评估。我们对 8 种不同的模型进行了实验，发现较大的模型往往幻觉较少；然而，超过 70 到 80 亿个参数的模型大小，进一步扩展并不能显着提高事实准确性。DAHL 分数具有成为人类注释偏好标签的有效替代方案的潜力，能够扩展到其他专业领域。我们公开发布了数据集和代码。

Title: Cross-Modal Consistency in Multimodal Large Language Models

• Authors: Xiang Zhang, Senyu Li, Ning Shi, Bradley Hauer, Zijun Wu, Grzegorz Kondrak, Muhammad Abdul-Mageed, Laks V.S. Lakshmanan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.09273
• Pdf URL: https://arxiv.org/pdf/2411.09273
• Copy Paste: [[2411.09273]] Cross-Modal Consistency in Multimodal Large Language Models(https://arxiv.org/abs/2411.09273)
• Keywords: language model, gpt, llm
• Abstract: Recent developments in multimodal methodologies have marked the beginning of an exciting era for models adept at processing diverse data types, encompassing text, audio, and visual content. Models like GPT-4V, which merge computer vision with advanced language processing, exhibit extraordinary proficiency in handling intricate tasks that require a simultaneous understanding of both textual and visual information. Prior research efforts have meticulously evaluated the efficacy of these Vision Large Language Models (VLLMs) in various domains, including object detection, image captioning, and other related fields. However, existing analyses have often suffered from limitations, primarily centering on the isolated evaluation of each modality's performance while neglecting to explore their intricate cross-modal interactions. Specifically, the question of whether these models achieve the same level of accuracy when confronted with identical task instances across different modalities remains unanswered. In this study, we take the initiative to delve into the interaction and comparison among these modalities of interest by introducing a novel concept termed cross-modal consistency. Furthermore, we propose a quantitative evaluation framework founded on this concept. Our experimental findings, drawn from a curated collection of parallel vision-language datasets developed by us, unveil a pronounced inconsistency between the vision and language modalities within GPT-4V, despite its portrayal as a unified multimodal model. Our research yields insights into the appropriate utilization of such models and hints at potential avenues for enhancing their design.
• 摘要：多模态方法的最新发展标志着一个激动人心的时代的开始，这些模型擅长处理各种数据类型，包括文本、音频和视觉内容。像 GPT-4V 这样的模型将计算机视觉与高级语言处理相结合，在处理需要同时理解文本和视觉信息的复杂任务方面表现出非凡的能力。先前的研究工作已经仔细评估了这些视觉大型语言模型 (VLLM) 在各个领域的有效性，包括对象检测、图像字幕和其他相关领域。然而，现有的分析往往存在局限性，主要集中在对每种模态性能的孤立评估上，而忽略了探索它们复杂的跨模态相互作用。具体来说，当面对不同模态中的相同任务实例时，这些模型是否达到相同的准确度水平的问题仍未得到解答。在本研究中，我们主动深入研究了这些感兴趣的模态之间的相互作用和比较，引入了一个称为跨模态一致性的新概念。此外，我们提出了一个基于这一概念的定量评估框架。我们的实验结果来自我们开发的一系列并行视觉-语言数据集，揭示了 GPT-4V 中的视觉和语言模式之间存在明显的不一致性，尽管它被描绘成一个统一的多模式模型。我们的研究为如何恰当地利用这些模型提供了见解，并暗示了改进其设计的潜在途径。

Title: StreamAdapter: Efficient Test Time Adaptation from Contextual Streams

• Authors: Dilxat Muhtar, Yelong Shen, Yaming Yang, Xiaodong Liu, Yadong Lu, Jianfeng Liu, Yuefeng Zhan, Hao Sun, Weiwei Deng, Feng Sun, Xueliang Zhang, Jianfeng Gao, Weizhu Chen, Qi Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.09289
• Pdf URL: https://arxiv.org/pdf/2411.09289
• Copy Paste: [[2411.09289]] StreamAdapter: Efficient Test Time Adaptation from Contextual Streams(https://arxiv.org/abs/2411.09289)
• Keywords: language model, llm
• Abstract: In-context learning (ICL) allows large language models (LLMs) to adapt to new tasks directly from the given demonstrations without requiring gradient updates. While recent advances have expanded context windows to accommodate more demonstrations, this approach increases inference costs without necessarily improving performance. To mitigate these issues, We propose StreamAdapter, a novel approach that directly updates model parameters from context at test time, eliminating the need for explicit in-context demonstrations. StreamAdapter employs context mapping and weight absorption mechanisms to dynamically transform ICL demonstrations into parameter updates with minimal additional parameters. By reducing reliance on numerous in-context examples, StreamAdapter significantly reduce inference costs and allows for efficient inference with constant time complexity, regardless of demonstration count. Extensive experiments across diverse tasks and model architectures demonstrate that StreamAdapter achieves comparable or superior adaptation capability to ICL while requiring significantly fewer demonstrations. The superior task adaptation and context encoding capabilities of StreamAdapter on both language understanding and generation tasks provides a new perspective for adapting LLMs at test time using context, allowing for more efficient adaptation across scenarios and more cost-effective inference
• 摘要：上下文学习 (ICL) 允许大型语言模型 (LLM) 直接从给定的演示中适应新任务，而无需梯度更新。虽然最近的进展已经扩大了上下文窗口以容纳更多的演示，但这种方法增加了推理成本，而不一定能提高性能。为了缓解这些问题，我们提出了 StreamAdapter，这是一种新颖的方法，它直接从测试时的上下文更新模型参数，从而无需显式的上下文演示。StreamAdapter 采用上下文映射和权重吸收机制，以最少的附加参数将 ICL 演示动态转换为参数更新。通过减少对大量上下文示例的依赖，StreamAdapter 显着降低了推理成本，并允许以恒定的时间复杂度进行高效推理，而不管演示数量如何。在不同任务和模型架构中进行的大量实验表明，StreamAdapter 实现了与 ICL 相当或更优越的适应能力，同时需要更少的演示。StreamAdapter 在语言理解和生成任务上的卓越任务适应和上下文编码能力为使用上下文在测试时适应 LLM 提供了一个新的视角，允许在场景之间进行更高效的适应和更具成本效益的推理

Title: DTELS: Towards Dynamic Granularity of Timeline Summarization

• Authors: Chenlong Zhang, Tong Zhou, Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, Jun Zhao
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09297
• Pdf URL: https://arxiv.org/pdf/2411.09297
• Copy Paste: [[2411.09297]] DTELS: Towards Dynamic Granularity of Timeline Summarization(https://arxiv.org/abs/2411.09297)
• Keywords: language model, llm
• Abstract: The rapid proliferation of online news has posed significant challenges in tracking the continuous development of news topics. Traditional timeline summarization constructs a chronological summary of the events but often lacks the flexibility to meet the diverse granularity needs. To overcome this limitation, we introduce a new paradigm, Dynamic-granularity TimELine Summarization, (DTELS), which aims to construct adaptive timelines based on user instructions or requirements. This paper establishes a comprehensive benchmark for DTLES that includes: (1) an evaluation framework grounded in journalistic standards to assess the timeline quality across four dimensions: Informativeness, Granular Consistency, Factuality, and Coherence; (2) a large-scale, multi-source dataset with multiple granularity timeline annotations based on a consensus process to facilitate authority; (3) extensive experiments and analysis with two proposed solutions based on Large Language Models (LLMs) and existing state-of-the-art TLS methods. The experimental results demonstrate the effectiveness of LLM-based solutions. However, even the most advanced LLMs struggle to consistently generate timelines that are both informative and granularly consistent, highlighting the challenges of the DTELS task.
• 摘要：在线新闻的快速增长为跟踪新闻主题的持续发展带来了重大挑战。传统的时间线摘要按时间顺序构建事件摘要，但通常缺乏满足不同粒度需求的灵活性。为了克服这一限制，我们引入了一种新范式，即动态粒度时间线摘要 (DTELS)，旨在根据用户指令或要求构建自适应时间线。本文为 DTLES 建立了一个全面的基准，包括：（1）基于新闻标准的评估框架，用于评估四个维度的时间线质量：信息量、粒度一致性、事实性和连贯性；（2）基于共识过程的具有多种粒度时间线注释的大规模多源数据集，以促进权威性；（3）对基于大型语言模型 (LLM) 和现有的最先进 TLS 方法的两种拟议解决方案进行了广泛的实验和分析。实验结果证明了基于 LLM 的解决方案的有效性。然而，即使是最先进的 LLM 也难以持续生成信息丰富且细节一致的时间表，这凸显了 DTELS 任务的挑战。

Title: DriveThru: a Document Extraction Platform and Benchmark Datasets for Indonesian Local Language Archives

• Authors: MohammadRifqi Farhansyah, Muhammad Zuhdi Fikri Johari, Afinzaki Amiral, Ayu Purwarianti, Kumara Ari Yuana, Derry Tanti Wijaya
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09318
• Pdf URL: https://arxiv.org/pdf/2411.09318
• Copy Paste: [[2411.09318]] DriveThru: a Document Extraction Platform and Benchmark Datasets for Indonesian Local Language Archives(https://arxiv.org/abs/2411.09318)
• Keywords: llm
• Abstract: Indonesia is one of the most diverse countries linguistically. However, despite this linguistic diversity, Indonesian languages remain underrepresented in Natural Language Processing (NLP) research and technologies. In the past two years, several efforts have been conducted to construct NLP resources for Indonesian languages. However, most of these efforts have been focused on creating manual resources thus difficult to scale to more languages. Although many Indonesian languages do not have a web presence, locally there are resources that document these languages well in printed forms such as books, magazines, and newspapers. Digitizing these existing resources will enable scaling of Indonesian language resource construction to many more languages. In this paper, we propose an alternative method of creating datasets by digitizing documents, which have not previously been used to build digital language resources in Indonesia. DriveThru is a platform for extracting document content utilizing Optical Character Recognition (OCR) techniques in its system to provide language resource building with less manual effort and cost. This paper also studies the utility of current state-of-the-art LLM for post-OCR correction to show the capability of increasing the character accuracy rate (CAR) and word accuracy rate (WAR) compared to off-the-shelf OCR.
• 摘要：印度尼西亚是语言最多样化的国家之一。然而，尽管语言多样性如此，印尼语在自然语言处理 (NLP) 研究和技术中仍然代表性不足。在过去的两年中，已经开展了多项努力来构建印尼语的 NLP 资源。然而，这些努力大部分都集中在创建手动资源上，因此难以扩展到更多语言。虽然许多印尼语没有网络存在，但当地有一些资源以印刷形式很好地记录了这些语言，如书籍、杂志和报纸。将这些现有资源数字化将使印尼语资源建设能够扩展到更多语言。在本文中，我们提出了一种通过数字化文档来创建数据集的替代方法，这种方法以前从未用于在印度尼西亚构建数字语言资源。DriveThru 是一个利用系统中的光学字符识别 (OCR) 技术提取文档内容的平台，以更少的人工和成本提供语言资源建设。本文还研究了当前最先进的 LLM 在 OCR 后校正中的实用性，以显示与现成的 OCR 相比提高字符准确率 (CAR) 和词语准确率 (WAR) 的能力。

Title: MM-Eval: A Hierarchical Benchmark for Modern Mongolian Evaluation in LLMs

• Authors: Mengyuan Zhang, Ruihui Wang, Bo Xia, Yuan Sun, Xiaobing Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.09492
• Pdf URL: https://arxiv.org/pdf/2411.09492
• Copy Paste: [[2411.09492]] MM-Eval: A Hierarchical Benchmark for Modern Mongolian Evaluation in LLMs(https://arxiv.org/abs/2411.09492)
• Keywords: language model, gpt, llm, chat
• Abstract: Large language models (LLMs) excel in high-resource languages but face notable challenges in low-resource languages like Mongolian. This paper addresses these challenges by categorizing capabilities into language abilities (syntax and semantics) and cognitive abilities (knowledge and reasoning). To systematically evaluate these areas, we developed MM-Eval, a specialized dataset based on Modern Mongolian Language Textbook I and enriched with WebQSP and MGSM datasets. Preliminary experiments on models including Qwen2-7B-Instruct, GLM4-9b-chat, Llama3.1-8B-Instruct, GPT-4, and DeepseekV2.5 revealed that: 1) all models performed better on syntactic tasks than semantic tasks, highlighting a gap in deeper language understanding; and 2) knowledge tasks showed a moderate decline, suggesting that models can transfer general knowledge from high-resource to low-resource contexts. The release of MM-Eval, comprising 569 syntax, 677 semantics, 344 knowledge, and 250 reasoning tasks, offers valuable insights for advancing NLP and LLMs in low-resource languages like Mongolian. The dataset is available at this https URL.
• 摘要：大型语言模型 (LLM) 在资源丰富的语言中表现出色，但在蒙古语等资源匮乏的语言中面临显著挑战。本文通过将能力分为语言能力（语法和语义）和认知能力（知识和推理）来解决这些挑战。为了系统地评估这些领域，我们开发了 MM-Eval，这是一个基于《现代蒙古语教科书 I》并丰富了 WebQSP 和 MGSM 数据集的专门数据集。对包括 Qwen2-7B-Instruct、GLM4-9b-chat、Llama3.1-8B-Instruct、GPT-4 和 DeepseekV2.5 在内的模型进行的初步实验表明：1) 所有模型在句法任务上的表现都优于语义任务，突显了更深层次的语言理解方面的差距；2) 知识任务表现出适度下降，表明模型可以将一般知识从资源丰富的环境转移到资源匮乏的环境。 MM-Eval 的发布包括 569 个语法、677 个语义、344 个知识和 250 个推理任务，为推进蒙古语等资源匮乏语言的 NLP 和 LLM 提供了宝贵的见解。数据集可在此 https URL 上获取。

Title: A Practical Guide to Fine-tuning Language Models with Limited Data

• Authors: Márton Szép, Daniel Rueckert, Rüdiger von Eisenhart-Rothe, Florian Hinterwimmer
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.09539
• Pdf URL: https://arxiv.org/pdf/2411.09539
• Copy Paste: [[2411.09539]] A Practical Guide to Fine-tuning Language Models with Limited Data(https://arxiv.org/abs/2411.09539)
• Keywords: language model, llm
• Abstract: Employing pre-trained Large Language Models (LLMs) has become the de facto standard in Natural Language Processing (NLP) despite their extensive data requirements. Motivated by the recent surge in research focused on training LLMs with limited data, particularly in low-resource domains and languages, this paper surveys recent transfer learning approaches to optimize model performance in downstream tasks where data is scarce. We first address initial and continued pre-training strategies to better leverage prior knowledge in unseen domains and languages. We then examine how to maximize the utility of limited data during fine-tuning and few-shot learning. The final section takes a task-specific perspective, reviewing models and methods suited for different levels of data scarcity. Our goal is to provide practitioners with practical guidelines for overcoming the challenges posed by constrained data while also highlighting promising directions for future research.
• 摘要：尽管预先训练的大型语言模型 (LLM) 需要大量数据，但使用它们已成为自然语言处理 (NLP) 的事实标准。近期，使用有限数据训练 LLM 的研究激增，尤其是在资源匮乏的领域和语言中，受此推动，本文调查了最近的迁移学习方法，以优化模型在数据稀缺的下游任务中的性能。我们首先讨论初始和持续的预训练策略，以更好地利用未知领域和语言中的先验知识。然后，我们研究如何在微调和小样本学习期间最大限度地发挥有限数据的效用。最后一部分从特定任务的视角出发，回顾适合不同数据稀缺程度的模型和方法。我们的目标是为从业者提供克服受限数据带来的挑战的实用指南，同时也强调未来研究的有希望的方向。

Title: Piecing It All Together: Verifying Multi-Hop Multimodal Claims

• Authors: Haoran Wang, Aman Rangapur, Xiongxiao Xu, Yueqing Liang, Haroon Gharwi, Carl Yang, Kai Shu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.09547
• Pdf URL: https://arxiv.org/pdf/2411.09547
• Copy Paste: [[2411.09547]] Piecing It All Together: Verifying Multi-Hop Multimodal Claims(https://arxiv.org/abs/2411.09547)
• Keywords: language model
• Abstract: Existing claim verification datasets often do not require systems to perform complex reasoning or effectively interpret multimodal evidence. To address this, we introduce a new task: multi-hop multimodal claim verification. This task challenges models to reason over multiple pieces of evidence from diverse sources, including text, images, and tables, and determine whether the combined multimodal evidence supports or refutes a given claim. To study this task, we construct MMCV, a large-scale dataset comprising 16k multi-hop claims paired with multimodal evidence, generated and refined using large language models, with additional input from human feedback. We show that MMCV is challenging even for the latest state-of-the-art multimodal large language models, especially as the number of reasoning hops increases. Additionally, we establish a human performance benchmark on a subset of MMCV. We hope this dataset and its evaluation task will encourage future research in multimodal multi-hop claim verification.
• 摘要：现有的索赔验证数据集通常不需要系统执行复杂的推理或有效地解释多模态证据。为了解决这个问题，我们引入了一项新任务：多跳多模态索赔验证。这项任务要求模型对来自不同来源（包括文本、图像和表格）的多种证据进行推理，并确定组合的多模态证据是支持还是反驳给定的索赔。为了研究这项任务，我们构建了 MMCV，这是一个大型数据集，包含 16k 个多跳索赔与多模态证据配对，使用大型语言模型生成和完善，并附加来自人工反馈的输入。我们表明，即使对于最新的最先进的多模态大型语言模型来说，MMCV 也具有挑战性，尤其是随着推理跳数的增加。此外，我们在 MMCV 的一个子集上建立了人类性能基准。我们希望这个数据集及其评估任务能够鼓励未来对多模态多跳索赔验证的研究。

Title: BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency

• Authors: Akari Haga, Akiyo Fukatsu, Miyu Oba, Arianna Bisazza, Yohei Oseki
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09587
• Pdf URL: https://arxiv.org/pdf/2411.09587
• Copy Paste: [[2411.09587]] BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency(https://arxiv.org/abs/2411.09587)
• Keywords: language model, gpt
• Abstract: While current large language models have achieved a remarkable success, their data efficiency remains a challenge to overcome. Recently it has been suggested that child-directed speech (CDS) can improve training data efficiency of modern language models based on Transformer neural networks. However, it is not yet understood which specific properties of CDS are effective for training these models. In the context of the BabyLM Challenge, we focus on Variation Sets (VSs), sets of consecutive utterances expressing a similar intent with slightly different words and structures, which are ubiquitous in CDS. To assess the impact of VSs on training data efficiency, we augment CDS data with different proportions of artificial VSs and use these datasets to train an auto-regressive model, GPT-2. We find that the best proportion of VSs depends on the evaluation benchmark: BLiMP and GLUE scores benefit from the presence of VSs, but EWOK scores do not. Additionally, the results vary depending on multiple factors such as the number of epochs and the order of utterance presentation. Taken together, these findings suggest that VSs can have a beneficial influence on language models, while leaving room for further investigation.
• 摘要：尽管当前的大型语言模型取得了显著的成功，但它们的数据效率仍然是一个有待克服的挑战。最近有研究表明，儿童导向语音 (CDS) 可以提高基于 Transformer 神经网络的现代语言模型的训练数据效率。然而，目前尚不清楚 CDS 的哪些特定属性对于训练这些模型有效。在 BabyLM 挑战赛中，我们专注于变体集 (VS)，即表达相似意图但单词和结构略有不同的连续话语集，它们在 CDS 中无处不在。为了评估 VS 对训练数据效率的影响，我们用不同比例的人工 VS 增强 CDS 数据，并使用这些数据集训练自回归模型 GPT-2。我们发现 VS 的最佳比例取决于评估基准：BLiMP 和 GLUE 分数受益于 VS 的存在，但 EWOK 分数则不然。此外，结果会因多种因素而异，例如 epoch 的数量和话语呈现的顺序。总的来说，这些发现表明 VS 可以对语言模型产生有益的影响，同时也为进一步研究留下了空间。

Title: The Moral Foundations Weibo Corpus

• Authors: Renjie Cao, Miaoyan Hu, Jiahan Wei, Baha Ihnaini
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2411.09612
• Pdf URL: https://arxiv.org/pdf/2411.09612
• Copy Paste: [[2411.09612]] The Moral Foundations Weibo Corpus(https://arxiv.org/abs/2411.09612)
• Keywords: language model
• Abstract: Moral sentiments expressed in natural language significantly influence both online and offline environments, shaping behavioral styles and interaction patterns, including social media selfpresentation, cyberbullying, adherence to social norms, and ethical decision-making. To effectively measure moral sentiments in natural language processing texts, it is crucial to utilize large, annotated datasets that provide nuanced understanding for accurate analysis and modeltraining. However, existing corpora, while valuable, often face linguistic limitations. To address this gap in the Chinese language domain,we introduce the Moral Foundation Weibo Corpus. This corpus consists of 25,671 Chinese comments on Weibo, encompassing six diverse topic areas. Each comment is manually annotated by at least three systematically trained annotators based on ten moral categories derived from a grounded theory of morality. To assess annotator reliability, we present the kappa testresults, a gold standard for measuring consistency. Additionally, we apply several the latest large language models to supplement the manual annotations, conducting analytical experiments to compare their performance and report baseline results for moral sentiment classification.
• 摘要：自然语言中表达的道德情感对线上和线下环境都有重大影响，塑造了行为风格和互动模式，包括社交媒体自我表现、网络欺凌、遵守社会规范和道德决策。为了有效地衡量自然语言处理文本中的道德情感，利用大型注释数据集至关重要，这些数据集为准确的分析和模型训练提供了细致入微的理解。然而，现有的语料库虽然很有价值，但往往面临语言限制。为了解决中文领域的这一空白，我们推出了道德基础微博语料库。该语料库包含 25,671 条微博中文评论，涵盖六个不同的主题领域。每条评论都由至少三名经过系统训练的注释者根据从道德扎根理论中得出的十个道德类别手动注释。为了评估注释者的可靠性，我们展示了 kappa 测试结果，这是衡量一致性的黄金标准。此外，我们应用了几种最新的大型语言模型来补充手动注释，进行分析实验以比较它们的性能并报告道德情感分类的基线结果。

Title: PTR: Precision-Driven Tool Recommendation for Large Language Models

• Authors: Hang Gao, Yongfeng Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2411.09613
• Pdf URL: https://arxiv.org/pdf/2411.09613
• Copy Paste: [[2411.09613]] PTR: Precision-Driven Tool Recommendation for Large Language Models(https://arxiv.org/abs/2411.09613)
• Keywords: language model, llm, prompt
• Abstract: By augmenting Large Language Models (LLMs) with external tools, their capacity to solve complex problems has been significantly enhanced. However, despite ongoing advancements in the parsing capabilities of LLMs, incorporating all available tools simultaneously in the prompt remains impractical due to the vast number of external tools. Consequently, it is essential to provide LLMs with a precise set of tools tailored to the specific task, considering both quantity and quality. Current tool retrieval methods primarily focus on refining the ranking list of tools and directly packaging a fixed number of top-ranked tools as the tool set. However, these approaches often fail to equip LLMs with the optimal set of tools prior to execution, since the optimal number of tools for different tasks could be different, resulting in inefficiencies such as redundant or unsuitable tools, which impede immediate access to the most relevant tools. This paper addresses the challenge of recommending precise toolsets for LLMs. We introduce the problem of tool recommendation, define its scope, and propose a novel Precision-driven Tool Recommendation (PTR) approach. PTR captures an initial, concise set of tools by leveraging historical tool bundle usage and dynamically adjusts the tool set by performing tool matching, culminating in a multi-view-based tool addition. Additionally, we present a new dataset, RecTools, and a metric, TRACC, designed to evaluate the effectiveness of tool recommendation for LLMs. We further validate our design choices through comprehensive experiments, demonstrating promising accuracy across two open benchmarks and our RecTools dataset.
• 摘要：通过为大型语言模型 (LLM) 增加外部工具，它们解决复杂问题的能力得到了显著增强。然而，尽管 LLM 的解析能力不断进步，但由于外部工具数量庞大，将所有可用工具同时纳入提示中仍然不切实际。因此，必须为 LLM 提供一套针对特定任务量身定制的精确工具，同时兼顾数量和质量。当前的工具检索方法主要侧重于完善工具排名列表，并直接将固定数量的排名靠前的工具打包为工具集。然而，这些方法通常无法在执行之前为 LLM 配备最佳工具集，因为不同任务的最佳工具数量可能不同，从而导致效率低下，例如工具冗余或不合适，从而阻碍立即访问最相关的工具。本文解决了为 LLM 推荐精确工具集的挑战。我们介绍了工具推荐问题，定义了其范围，并提出了一种新颖的精确驱动工具推荐 (PTR) 方法。 PTR 利用历史工具包使用情况捕获初始的简洁工具集，并通过执行工具匹配来动态调整工具集，最终实现基于多视图的工具添加。此外，我们还提供了一个新数据集 RecTools 和一个指标 TRACC，旨在评估 LLM 工具推荐的有效性。我们通过全面的实验进一步验证了我们的设计选择，证明了在两个开放基准和我们的 RecTools 数据集上具有良好的准确性。

Title: Adaptive Decoding via Latent Preference Optimization

• Authors: Shehzaad Dhuliawala, Ilia Kulikov, Ping Yu, Asli Celikyilmaz, Jason Weston, Sainbayar Sukhbaatar, Jack Lanchantin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09661
• Pdf URL: https://arxiv.org/pdf/2411.09661
• Copy Paste: [[2411.09661]] Adaptive Decoding via Latent Preference Optimization(https://arxiv.org/abs/2411.09661)
• Keywords: language model
• Abstract: During language model decoding, it is known that using higher temperature sampling gives more creative responses, while lower temperatures are more factually accurate. However, such models are commonly applied to general instruction following, which involves both creative and fact seeking tasks, using a single fixed temperature across all examples and tokens. In this work, we introduce Adaptive Decoding, a layer added to the model to select the sampling temperature dynamically at inference time, at either the token or example level, in order to optimize performance. To learn its parameters we introduce Latent Preference Optimization (LPO) a general approach to train discrete latent variables such as choices of temperature. Our method outperforms all fixed decoding temperatures across a range of tasks that require different temperatures, including UltraFeedback, Creative Story Writing, and GSM8K.
• 摘要：在语言模型解码过程中，众所周知，使用较高温度采样可以产生更具创造性的响应，而较低的温度则更符合事实。然而，这种模型通常应用于一般指令遵循，这涉及创造性和事实寻求任务，在所有示例和标记中使用单一固定温度。在这项工作中，我们引入了自适应解码，这是添加到模型中的一层，用于在推理时在标记或示例级别动态选择采样温度，以优化性能。为了学习它的参数，我们引入了潜在偏好优化 (LPO)，这是一种训练离散潜在变量（例如温度选择）的通用方法。我们的方法在一系列需要不同温度的任务中优于所有固定解码温度，包括 UltraFeedback、Creative Story Writing 和 GSM8K。

Title: Squeezed Attention: Accelerating Long Context Length LLM Inference

• Authors: Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Monishwaran Maheswaran, June Paik, Michael W. Mahoney, Kurt Keutzer, Amir Gholami
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2411.09688
• Pdf URL: https://arxiv.org/pdf/2411.09688
• Copy Paste: [[2411.09688]] Squeezed Attention: Accelerating Long Context Length LLM Inference(https://arxiv.org/abs/2411.09688)
• Keywords: language model, llm, long context, prompt
• Abstract: Emerging Large Language Model (LLM) applications require long input prompts to perform complex downstream tasks like document analysis and code generation. For these long context length applications, the length of the input prompt poses a significant challenge in terms of inference efficiency since the inference costs increase linearly with sequence length. However, for many of these applications, much of the context in the prompt is fixed across different user inputs, thereby providing the opportunity to perform offline optimizations to process user inputs quickly, as they are received. In this work, we propose Squeezed Attention as a mechanism to accelerate LLM applications where a large portion of the input prompt is fixed. We first leverage K-means clustering offline to group the keys for the fixed context based on semantic similarity and represent each cluster with a single centroid value. During inference, we compare query tokens from the user input with the centroids to predict which of the keys from the fixed context are semantically relevant and need to be loaded during inference. We then compute exact attention using only these important keys from the fixed context, thereby reducing bandwidth and computational costs. We also extend our method to use a hierarchical centroid lookup to identify important keys, which can reduce the complexity of attention from linear to logarithmic with respect to the context length. We implement optimized Triton kernels for centroid comparison and sparse FlashAttention with important keys, achieving more than 4x speedups during both the prefill and generation phases for long-context inference. Furthermore, we have extensively evaluated our method on various long-context benchmarks including LongBench, where it achieves a 3x reduction in KV cache budget without accuracy loss and up to an 8x reduction with <0.5 point accuracy gap for various models.
• 摘要：新兴的大型语言模型 (LLM) 应用程序需要较长的输入提示来执行复杂的下游任务，如文档分析和代码生成。对于这些长上下文长度的应用程序，输入提示的长度对推理效率提出了重大挑战，因为推理成本随序列长度线性增加。然而，对于许多此类应用程序，提示中的大部分上下文在不同的用户输入中是固定的，从而提供了执行离线优化的机会，以便在收到用户输入时快速处理它们。在这项工作中，我们提出了压缩注意力机制来加速大部分输入提示固定的 LLM 应用程序。我们首先利用离线 K 均值聚类根据语义相似性对固定上下文的键进行分组，并用单个质心值表示每个聚类。在推理过程中，我们将用户输入中的查询标记与质心进行比较，以预测固定上下文中的哪些键在语义上相关，需要在推理期间加载。然后，我们仅使用固定上下文中的这些重要键来计算精确的注意力，从而减少带宽和计算成本。我们还扩展了我们的方法，使用分层质心查找来识别重要键，这可以将注意力的复杂性从线性降低到对上下文长度的对数。我们实施了优化的 Triton 内核用于质心比较，并使用重要键对 FlashAttention 进行稀疏处理，在长上下文推理的预填充和生成阶段实现了 4 倍以上的加速。此外，我们在包括 LongBench 在内的各种长上下文基准上对我们的方法进行了广泛的评估，其中它将 KV 缓存预算减少了 3 倍，而准确度没有损失，并且将各种模型的准确度差距降至 8 倍，而准确度差距不到 0.5 点。