2024-10-14

Title: Increasing the Difficulty of Automatically Generated Questions via Reinforcement Learning with Synthetic Preference

• Authors: William Thorne, Ambrose Robinson, Bohua Peng, Chenghua Lin, Diana Maynard
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08289
• Pdf URL: https://arxiv.org/pdf/2410.08289
• Copy Paste: [[2410.08289]] Increasing the Difficulty of Automatically Generated Questions via Reinforcement Learning with Synthetic Preference(https://arxiv.org/abs/2410.08289)
• Keywords: chat, retrieval-augmented generation
• Abstract: As the cultural heritage sector increasingly adopts technologies like Retrieval-Augmented Generation (RAG) to provide more personalised search experiences and enable conversations with collections data, the demand for specialised evaluation datasets has grown. While end-to-end system testing is essential, it's equally important to assess individual components. We target the final, answering task, which is well-suited to Machine Reading Comprehension (MRC). Although existing MRC datasets address general domains, they lack the specificity needed for cultural heritage information. Unfortunately, the manual creation of such datasets is prohibitively expensive for most heritage institutions. This paper presents a cost-effective approach for generating domain-specific MRC datasets with increased difficulty using Reinforcement Learning from Human Feedback (RLHF) from synthetic preference data. Our method leverages the performance of existing question-answering models on a subset of SQuAD to create a difficulty metric, assuming that more challenging questions are answered correctly less frequently. This research contributes: (1) A methodology for increasing question difficulty using PPO and synthetic data; (2) Empirical evidence of the method's effectiveness, including human evaluation; (3) An in-depth error analysis and study of emergent phenomena; and (4) An open-source codebase and set of three llama-2-chat adapters for reproducibility and adaptation.
• 摘要：随着文化遗产部门越来越多地采用检索增强生成 (RAG) 等技术来提供更加个性化的搜索体验并实现与馆藏数据的对话，对专门评估数据集的需求也随之增长。虽然端到端系统测试至关重要，但评估各个组件也同样重要。我们的目标是最终的答题任务，这非常适合机器阅读理解 (MRC)。虽然现有的 MRC 数据集涉及一般领域，但它们缺乏文化遗产信息所需的特异性。不幸的是，对于大多数遗产机构来说，手动创建此类数据集的成本过高。本文介绍了一种经济有效的方法，用于使用从合成偏好数据中进行的从人类反馈强化学习 (RLHF) 生成难度更大的领域特定 MRC 数据集。我们的方法利用现有问答模型在 SQuAD 子集上的性能来创建难度指标，假设更具挑战性的问题正确回答的频率较低。这项研究贡献了：(1) 一种使用 PPO 和合成数据增加问题难度的方法； (2) 该方法有效性的经验证据，包括人工评估；(3) 对突发现象进行深入的错误分析和研究；(4) 一个开源代码库和一组三个 llama-2-chat 适配器，用于可重复性和适应性。

Title: Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation

• Authors: Zhuohang Li, Jiaxin Zhang, Chao Yan, Kamalika Das, Sricharan Kumar, Murat Kantarcioglu, Bradley A. Malin
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2410.08320
• Pdf URL: https://arxiv.org/pdf/2410.08320
• Copy Paste: [[2410.08320]] Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation(https://arxiv.org/abs/2410.08320)
• Keywords: language model, hallucination, retrieval augmented generation
• Abstract: Language models (LMs) are known to suffer from hallucinations and misinformation. Retrieval augmented generation (RAG) that retrieves verifiable information from an external knowledge corpus to complement the parametric knowledge in LMs provides a tangible solution to these problems. However, the generation quality of RAG is highly dependent on the relevance between a user's query and the retrieved documents. Inaccurate responses may be generated when the query is outside of the scope of knowledge represented in the external knowledge corpus or if the information in the corpus is out-of-date. In this work, we establish a statistical framework that assesses how well a query can be answered by an RAG system by capturing the relevance of knowledge. We introduce an online testing procedure that employs goodness-of-fit (GoF) tests to inspect the relevance of each user query to detect out-of-knowledge queries with low knowledge relevance. Additionally, we develop an offline testing framework that examines a collection of user queries, aiming to detect significant shifts in the query distribution which indicates the knowledge corpus is no longer sufficiently capable of supporting the interests of the users. We demonstrate the capabilities of these strategies through a systematic evaluation on eight question-answering (QA) datasets, the results of which indicate that the new testing framework is an efficient solution to enhance the reliability of existing RAG systems.
• 摘要：众所周知，语言模型 (LM) 容易产生幻觉和错误信息。检索增强生成 (RAG) 从外部知识语料库中检索可验证信息来补充 LM 中的参数知识，为这些问题提供了切实可行的解决方案。然而，RAG 的生成质量高度依赖于用户查询与检索到的文档之间的相关性。当查询超出外部知识语料库所代表的知识范围或语料库中的信息过时时，可能会生成不准确的响应。在这项工作中，我们建立了一个统计框架，通过捕获知识的相关性来评估 RAG 系统对查询的回答效果。我们引入了一种在线测试程序，该程序采用拟合优度 (GoF) 测试来检查每个用户查询的相关性，以检测知识相关性低的知识范围之外的查询。此外，我们还开发了一个离线测试框架，用于检查一系列用户查询，旨在检测查询分布中的重大变化，这表明知识库不再能够充分支持用户的兴趣。我们通过对八个问答 (QA) 数据集进行系统评估来展示这些策略的能力，结果表明，新的测试框架是提高现有 RAG 系统可靠性的有效解决方案。

Title: Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains

• Authors: Krithika Ramesh, Nupoor Gandhi, Pulkit Madaan, Lisa Bauer, Charith Peris, Anjalie Field
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08327
• Pdf URL: https://arxiv.org/pdf/2410.08327
• Copy Paste: [[2410.08327]] Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains(https://arxiv.org/abs/2410.08327)
• Keywords: language model
• Abstract: The difficulty of anonymizing text data hinders the development and deployment of NLP in high-stakes domains that involve private data, such as healthcare and social services. Poorly anonymized sensitive data cannot be easily shared with annotators or external researchers, nor can it be used to train public models. In this work, we explore the feasibility of using synthetic data generated from differentially private language models in place of real data to facilitate the development of NLP in these domains without compromising privacy. In contrast to prior work, we generate synthetic data for real high-stakes domains, and we propose and conduct use-inspired evaluations to assess data quality. Our results show that prior simplistic evaluations have failed to highlight utility, privacy, and fairness issues in the synthetic data. Overall, our work underscores the need for further improvements to synthetic data generation for it to be a viable way to enable privacy-preserving data sharing.
• 摘要：文本数据匿名化的困难阻碍了 NLP 在涉及私人数据的高风险领域（例如医疗保健和社会服务）的开发和部署。匿名性较差的敏感数据无法轻易与注释者或外部研究人员共享，也无法用于训练公共模型。在这项工作中，我们探索了使用差异化隐私语言模型生成的合成数据代替真实数据的可行性，以促进这些领域的 NLP 开发而不损害隐私。与之前的工作相比，我们为真实的高风险领域生成合成数据，并提出并进行使用启发式评估来评估数据质量。我们的结果表明，之前的简单评估未能突出合成数据中的实用性、隐私和公平性问题。总体而言，我们的工作强调需要进一步改进合成数据生成，以使其成为实现隐私保护数据共享的可行方法。

Title: Exploring Natural Language-Based Strategies for Efficient Number Learning in Children through Reinforcement Learning

• Authors: Tirthankar Mittra
• Subjects: cs.CL, cs.AI, cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2410.08334
• Pdf URL: https://arxiv.org/pdf/2410.08334
• Copy Paste: [[2410.08334]] Exploring Natural Language-Based Strategies for Efficient Number Learning in Children through Reinforcement Learning(https://arxiv.org/abs/2410.08334)
• Keywords: agent
• Abstract: This paper investigates how children learn numbers using the framework of reinforcement learning (RL), with a focus on the impact of language instructions. The motivation for using reinforcement learning stems from its parallels with psychological learning theories in controlled environments. By using state of the art deep reinforcement learning models, we simulate and analyze the effects of various forms of language instructions on number acquisition. Our findings indicate that certain linguistic structures more effectively improve numerical comprehension in RL agents. Additionally, our model predicts optimal sequences for presenting numbers to RL agents which enhance their speed of learning. This research provides valuable insights into the interplay between language and numerical cognition, with implications for both educational strategies and the development of artificial intelligence systems designed to support early childhood learning.
• 摘要：本文研究了儿童如何使用强化学习 (RL) 框架学习数字，重点关注语言指导的影响。使用强化学习的动机源于其与受控环境中的心理学习理论的相似性。通过使用最先进的深度强化学习模型，我们模拟和分析了各种形式的语言指导对数字习得的影响。我们的研究结果表明，某些语言结构可以更有效地提高 RL 代理的数字理解能力。此外，我们的模型可以预测向 RL 代理呈现数字的最佳序列，从而提高他们的学习速度。这项研究为语言和数字认知之间的相互作用提供了宝贵的见解，对教育策略和旨在支持早期儿童学习的人工智能系统的开发都有重要意义。

Title: Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation

• Authors: Thomas Gauthier-Caron, Shamane Siriwardhana, Elliot Stein, Malikeh Ehghaghi, Charles Goddard, Mark McQuade, Jacob Solawetz, Maxime Labonne
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2410.08371
• Pdf URL: https://arxiv.org/pdf/2410.08371
• Copy Paste: [[2410.08371]] Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation(https://arxiv.org/abs/2410.08371)
• Keywords: language model
• Abstract: By merging models, AI systems can combine the distinct strengths of separate language models, achieving a balance between multiple capabilities without requiring substantial retraining. However, the integration process can be intricate due to differences in training methods and fine-tuning, typically necessitating specialized knowledge and repeated refinement. This paper explores model merging techniques across a spectrum of complexity, examining where automated methods like evolutionary strategies stand compared to hyperparameter-driven approaches such as DARE, TIES-Merging and simpler methods like Model Soups. In addition, we introduce Differentiable Adaptive Merging (DAM), an efficient, adaptive merging approach as an alternative to evolutionary merging that optimizes model integration through scaling coefficients, minimizing computational demands. Our findings reveal that even simple averaging methods, like Model Soups, perform competitively when model similarity is high, underscoring each technique's unique strengths and limitations. We open-sourced DAM, including the implementation code and experiment pipeline, on GitHub: this https URL.
• 摘要：通过合并模型，AI 系统可以结合不同语言模型的不同优势，实现多种能力之间的平衡，而无需进行大量的再训练。然而，由于训练方法和微调的差异，集成过程可能很复杂，通常需要专业知识和反复改进。本文探讨了各种复杂程度的模型合并技术，研究了自动化方法（如进化策略）与超参数驱动方法（如 DARE、TIES-Merging）和更简单的方法（如 Model Soups）相比的优势。此外，我们引入了可微分自适应合并 (DAM)，这是一种高效的自适应合并方法，可替代进化合并，通过缩放系数优化模型集成，最大限度地减少计算需求。我们的研究结果表明，即使是简单的平均方法（如 Model Soups），在模型相似度高时也能表现出色，这突显了每种技术独特的优势和局限性。我们在 GitHub 上开源了 DAM，包括实现代码和实验流程：此 https URL。

Title: Evaluating Transformer Models for Suicide Risk Detection on Social Media

• Authors: Jakub Pokrywka, Jeremi I. Kaczmarek, Edward J. Gorzelańczyk
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08375
• Pdf URL: https://arxiv.org/pdf/2410.08375
• Copy Paste: [[2410.08375]] Evaluating Transformer Models for Suicide Risk Detection on Social Media(https://arxiv.org/abs/2410.08375)
• Keywords: gpt, prompt
• Abstract: The detection of suicide risk in social media is a critical task with potential life-saving implications. This paper presents a study on leveraging state-of-the-art natural language processing solutions for identifying suicide risk in social media posts as a submission for the "IEEE BigData 2024 Cup: Detection of Suicide Risk on Social Media" conducted by the kubapok team. We experimented with the following configurations of transformer-based models: fine-tuned DeBERTa, GPT-4o with CoT and few-shot prompting, and fine-tuned GPT-4o. The task setup was to classify social media posts into four categories: indicator, ideation, behavior, and attempt. Our findings demonstrate that the fine-tuned GPT-4o model outperforms two other configurations, achieving high accuracy in identifying suicide risk. Notably, our model achieved second place in the competition. By demonstrating that straightforward, general-purpose models can achieve state-of-the-art results, we propose that these models, combined with minimal tuning, may have the potential to be effective solutions for automated suicide risk detection on social media.
• 摘要：在社交媒体中检测自杀风险是一项关键任务，具有潜在的挽救生命的意义。本文介绍了一项利用最先进的自然语言处理解决方案识别社交媒体帖子中自杀风险的研究，该研究是 kubapok 团队开展的“IEEE BigData 2024 Cup：检测社交媒体上的自杀风险”的提交。我们尝试了以下基于 Transformer 的模型配置：微调的 DeBERTa、带有 CoT 和少样本提示的 GPT-4o 以及微调的 GPT-4o。任务设置是将社交媒体帖子分为四类：指标、想法、行为和尝试。我们的研究结果表明，微调的 GPT-4o 模型优于其他两种配置，在识别自杀风险方面实现了高精度。值得注意的是，我们的模型在比赛中获得了第二名。通过证明简单的通用模型可以获得最先进的结果，我们认为这些模型加上最少的调整，有可能成为社交媒体上自动自杀风险检测的有效解决方案。

Title: GUS-Net: Social Bias Classification in Text with Generalizations, Unfairness, and Stereotypes

• Authors: Maximus Powers, Hua Wei, Umang Mavani, Harshitha Reddy Jonala, Ansh Tiwari
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08388
• Pdf URL: https://arxiv.org/pdf/2410.08388
• Copy Paste: [[2410.08388]] GUS-Net: Social Bias Classification in Text with Generalizations, Unfairness, and Stereotypes(https://arxiv.org/abs/2410.08388)
• Keywords: language model, llm, agent
• Abstract: The detection of bias in natural language processing (NLP) is a critical challenge, particularly with the increasing use of large language models (LLMs) in various domains. This paper introduces GUS-Net, an innovative approach to bias detection that focuses on three key types of biases: (G)eneralizations, (U)nfairness, and (S)tereotypes. GUS-Net leverages generative AI and automated agents to create a comprehensive synthetic dataset, enabling robust multi-label token classification. Our methodology enhances traditional bias detection methods by incorporating the contextual encodings of pre-trained models, resulting in improved accuracy and depth in identifying biased entities. Through extensive experiments, we demonstrate that GUS-Net outperforms state-of-the-art techniques, achieving superior performance in terms of accuracy, F1-score, and Hamming Loss. The findings highlight GUS-Net's effectiveness in capturing a wide range of biases across diverse contexts, making it a valuable tool for social bias detection in text. This study contributes to the ongoing efforts in NLP to address implicit bias, providing a pathway for future research and applications in various fields. The Jupyter notebooks used to create the dataset and model are available at: this https URL. Warning: This paper contains examples of harmful language, and reader discretion is recommended.
• 摘要：自然语言处理 (NLP) 中的偏见检测是一项关键挑战，尤其是随着大型语言模型 (LLM) 在各个领域的使用越来越多。本文介绍了 GUS-Net，这是一种创新的偏见检测方法，侧重于三种主要偏见类型：（G）概括、（U）不公平和（S）刻板印象。GUS-Net 利用生成式 AI 和自动化代理来创建全面的合成数据集，从而实现强大的多标签标记分类。我们的方法通过结合预训练模型的上下文编码来增强传统的偏见检测方法，从而提高了识别有偏见实体的准确性和深度。通过大量实验，我们证明 GUS-Net 优于最先进的技术，在准确性、F1 分数和汉明损失方面取得了卓越的表现。研究结果强调了 GUS-Net 在捕捉不同背景下的各种偏见方面的有效性，使其成为文本中社会偏见检测的宝贵工具。本研究有助于 NLP 领域持续努力解决隐性偏见，为未来各个领域的研究和应用提供了途径。用于创建数据集和模型的 Jupyter 笔记本可从以下网址获取：此 https URL。警告：本文包含有害语言示例，建议读者谨慎阅读。

Title: KV Prediction for Improved Time to First Token

• Authors: Maxwell Horton, Qingqing Cao, Chenfan Sun, Yanzi Jin, Sachin Mehta, Mohammad Rastegari, Moin Nabi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08391
• Pdf URL: https://arxiv.org/pdf/2410.08391
• Copy Paste: [[2410.08391]] KV Prediction for Improved Time to First Token(https://arxiv.org/abs/2410.08391)
• Keywords: language model, prompt
• Abstract: Inference with transformer-based language models begins with a prompt processing step. In this step, the model generates the first output token and stores the KV cache needed for future generation steps. This prompt processing step can be computationally expensive, taking 10s of seconds or more for billion-parameter models on edge devices when prompt lengths or batch sizes rise. This degrades user experience by introducing significant latency into the model's outputs. To reduce the time spent producing the first output (known as the ``time to first token'', or TTFT) of a pretrained model, we introduce a novel method called KV Prediction. In our method, a small auxiliary model is used to process the prompt and produce an approximation of the KV cache used by a base model. This approximated KV cache is then used with the base model for autoregressive generation without the need to query the auxiliary model again. We demonstrate that our method produces a pareto-optimal efficiency-accuracy trade-off when compared to baselines. On TriviaQA, we demonstrate relative accuracy improvements in the range of $15\%-50\%$ across a range of TTFT FLOPs budgets. We also demonstrate accuracy improvements of up to $30\%$ on HumanEval python code completion at fixed TTFT FLOPs budgets. Additionally, we benchmark models on an Apple M2 Pro CPU and demonstrate that our improvement in FLOPs translates to a TTFT speedup on hardware. We release our code at this https URL .
• 摘要：使用基于转换器的语言模型进行推理始于一个提示处理步骤。在此步骤中，模型生成第一个输出标记并存储未来生成步骤所需的 KV 缓存。此提示处理步骤的计算成本可能很高，当提示长度或批量大小增加时，边缘设备上的十亿参数模型需要花费数十秒或更长时间。这会在模型的输出中引入显着的延迟，从而降低用户体验。为了减少生成预训练模型的第一个输出（称为“第一个标记时间”或 TTFT）所花费的时间，我们引入了一种称为 KV 预测的新方法。在我们的方法中，使用一个小型辅助模型来处理提示并生成基本模型使用的 KV 缓存的近似值。然后，将此近似的 KV 缓存与基本模型一起用于自回归生成，而无需再次查询辅助模型。我们证明，与基线相比，我们的方法产生了帕累托最优的效率-准确性权衡。在 TriviaQA 上，我们展示了在一系列 TTFT FLOPs 预算范围内相对准确度的提升，提升幅度在 15%-50%。我们还展示了在固定 TTFT FLOPs 预算下 HumanEval Python 代码补全中准确度提升高达 30%。此外，我们在 Apple M2 Pro CPU 上对模型进行了基准测试，并展示了我们在 FLOPs 方面的改进转化为硬件上的 TTFT 加速。我们在此 https URL 上发布了我们的代码。

Title: The Effects of Hallucinations in Synthetic Training Data for Relation Extraction

• Authors: Steven Rogulsky, Nicholas Popovic, Michael Färber
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2410.08393
• Pdf URL: https://arxiv.org/pdf/2410.08393
• Copy Paste: [[2410.08393]] The Effects of Hallucinations in Synthetic Training Data for Relation Extraction(https://arxiv.org/abs/2410.08393)
• Keywords: hallucination
• Abstract: Relation extraction is crucial for constructing knowledge graphs, with large high-quality datasets serving as the foundation for training, fine-tuning, and evaluating models. Generative data augmentation (GDA) is a common approach to expand such datasets. However, this approach often introduces hallucinations, such as spurious facts, whose impact on relation extraction remains underexplored. In this paper, we examine the effects of hallucinations on the performance of relation extraction on the document and sentence levels. Our empirical study reveals that hallucinations considerably compromise the ability of models to extract relations from text, with recall reductions between 19.1% and 39.2%. We identify that relevant hallucinations impair the model's performance, while irrelevant hallucinations have a minimal impact. Additionally, we develop methods for the detection of hallucinations to improve data quality and model performance. Our approaches successfully classify texts as either 'hallucinated' or 'clean,' achieving high F1-scores of 83.8% and 92.2%. These methods not only assist in removing hallucinations but also help in estimating their prevalence within datasets, which is crucial for selecting high-quality data. Overall, our work confirms the profound impact of relevant hallucinations on the effectiveness of relation extraction models.
• 摘要：关系提取对于构建知识图谱至关重要，大量高质量数据集是训练、微调和评估模型的基础。生成数据增强 (GDA) 是扩展此类数据集的常用方法。然而，这种方法通常会引入幻觉，例如虚假事实，其对关系提取的影响仍未得到充分探索。在本文中，我们研究了幻觉对文档和句子级别关系提取性能的影响。我们的实证研究表明，幻觉会严重损害模型从文本中提取关系的能力，召回率会降低 19.1% 至 39.2%。我们发现相关的幻觉会损害模型的性能，而不相关的幻觉的影响微乎其微。此外，我们还开发了检测幻觉的方法，以提高数据质量和模型性能。我们的方法成功地将文本分类为“幻觉”或“干净”，实现了 83.8% 和 92.2% 的高 F1 分数。这些方法不仅有助于消除幻觉，还有助于估计其在数据集中的普遍性，这对于选择高质量数据至关重要。总的来说，我们的工作证实了相关幻觉对关系提取模型有效性的深远影响。

Title: Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models

• Authors: Sitao Cheng, Liangming Pan, Xunjian Yin, Xinyi Wang, William Yang Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08414
• Pdf URL: https://arxiv.org/pdf/2410.08414
• Copy Paste: [[2410.08414]] Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models(https://arxiv.org/abs/2410.08414)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) encode vast amounts of knowledge during pre-training (parametric knowledge, or PK) and can further be enhanced by incorporating contextual knowledge (CK). Can LLMs effectively integrate their internal PK with external CK to solve complex problems? In this paper, we investigate the dynamic interaction between PK and CK, categorizing their relationships into four types: Supportive, Complementary, Conflicting, and Irrelevant. To support this investigation, we introduce ECHOQA, a benchmark spanning scientific, factual, and commonsense knowledge. Our results show that LLMs tend to suppress their PK when contextual information is available, even when it is complementary or irrelevant. While tailored instructions can encourage LLMs to rely more on their PK, they still struggle to fully leverage it. These findings reveal a key vulnerability in LLMs, raising concerns about their reliability in knowledge-intensive tasks. Resources are available at this https URL Interplay.
• 摘要：大型语言模型 (LLM) 在预训练期间编码大量知识（参数知识或 PK），并且可以通过合并上下文知识 (CK) 进一步增强。LLM 能否有效地将其内部 PK 与外部 CK 相结合以解决复杂问题？在本文中，我们研究了 PK 和 CK 之间的动态交互，将它们的关系分为四种类型：支持性、互补性、冲突性和无关性。为了支持这项调查，我们引入了 ECHOQA，这是一个涵盖科学、事实和常识知识的基准。我们的结果表明，当上下文信息可用时，LLM 倾向于抑制其 PK，即使它是互补的或不相关的。虽然量身定制的说明可以鼓励 LLM 更多地依赖其 PK，但它们仍然难以充分利用它。这些发现揭示了 LLM 的一个关键弱点，引发了人们对其在知识密集型任务中的可靠性的担忧。资源可在此 https URL Interplay 上找到。

Title: oRetrieval Augmented Generation for 10 Large Language Models and its Generalizability in Assessing Medical Fitness

• Authors: Yu He Ke, Liyuan Jin, Kabilan Elangovan, Hairil Rizal Abdullah, Nan Liu, Alex Tiong Heng Sia, Chai Rick Soh, Joshua Yi Min Tung, Jasmine Chiat Ling Ong, Chang-Fu Kuo, Shao-Chun Wu, Vesela P. Kovacheva, Daniel Shu Wei Ting
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08431
• Pdf URL: https://arxiv.org/pdf/2410.08431
• Copy Paste: [[2410.08431]] oRetrieval Augmented Generation for 10 Large Language Models and its Generalizability in Assessing Medical Fitness(https://arxiv.org/abs/2410.08431)
• Keywords: language model, gpt, llm, hallucination, retrieval augmented generation
• Abstract: Large Language Models (LLMs) show potential for medical applications but often lack specialized clinical knowledge. Retrieval Augmented Generation (RAG) allows customization with domain-specific information, making it suitable for healthcare. This study evaluates the accuracy, consistency, and safety of RAG models in determining fitness for surgery and providing preoperative instructions. We developed LLM-RAG models using 35 local and 23 international preoperative guidelines and tested them against human-generated responses. A total of 3,682 responses were evaluated. Clinical documents were processed using Llamaindex, and 10 LLMs, including GPT3.5, GPT4, and Claude-3, were assessed. Fourteen clinical scenarios were analyzed, focusing on seven aspects of preoperative instructions. Established guidelines and expert judgment were used to determine correct responses, with human-generated answers serving as comparisons. The LLM-RAG models generated responses within 20 seconds, significantly faster than clinicians (10 minutes). The GPT4 LLM-RAG model achieved the highest accuracy (96.4% vs. 86.6%, p=0.016), with no hallucinations and producing correct instructions comparable to clinicians. Results were consistent across both local and international guidelines. This study demonstrates the potential of LLM-RAG models for preoperative healthcare tasks, highlighting their efficiency, scalability, and reliability.
• 摘要：大型语言模型 (LLM) 显示出医疗应用的潜力，但通常缺乏专业的临床知识。检索增强生成 (RAG) 允许使用特定领域的信息进行定制，使其适用于医疗保健。本研究评估了 RAG 模型在确定手术适合性和提供术前指导方面的准确性、一致性和安全性。我们使用 35 个地方和 23 个国际术前指南开发了 LLM-RAG 模型，并根据人工生成的响应对其进行了测试。总共评估了 3,682 个响应。使用 Llamaindex 处理临床文件，并评估了 10 个 LLM，包括 GPT3.5、GPT4 和 Claude-3。分析了 14 种临床场景，重点关注术前指导的七个方面。使用既定的指南和专家判断来确定正确的反应，并以人工生成的答案作为比较。LLM-RAG 模型在 20 秒内生成响应，明显快于临床医生（10 分钟）。 GPT4 LLM-RAG 模型的准确率最高（96.4% vs. 86.6%，p=0.016），没有幻觉，并且给出的正确指令与临床医生相当。结果与当地和国际指南一致。这项研究展示了 LLM-RAG 模型在术前医疗保健任务中的潜力，突出了它们的效率、可扩展性和可靠性。

Title: Exploring the Role of Reasoning Structures for Constructing Proofs in Multi-Step Natural Language Reasoning with Large Language Models

• Authors: Zi'ou Zheng, Christopher Malon, Martin Renqiang Min, Xiaodan Zhu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08436
• Pdf URL: https://arxiv.org/pdf/2410.08436
• Copy Paste: [[2410.08436]] Exploring the Role of Reasoning Structures for Constructing Proofs in Multi-Step Natural Language Reasoning with Large Language Models(https://arxiv.org/abs/2410.08436)
• Keywords: language model, llm
• Abstract: When performing complex multi-step reasoning tasks, the ability of Large Language Models (LLMs) to derive structured intermediate proof steps is important for ensuring that the models truly perform the desired reasoning and for improving models' explainability. This paper is centred around a focused study: whether the current state-of-the-art generalist LLMs can leverage the structures in a few examples to better construct the proof structures with \textit{in-context learning}. Our study specifically focuses on structure-aware demonstration and structure-aware pruning. We demonstrate that they both help improve performance. A detailed analysis is provided to help understand the results.
• 摘要：在执行复杂的多步骤推理任务时，大型语言模型 (LLM) 得出结构化中间证明步骤的能力对于确保模型真正执行所需的推理和提高模型的可解释性非常重要。本文围绕一项重点研究展开：当前最先进的通用 LLM 是否可以利用几个示例中的结构，通过 \textit{上下文学习} 更好地构建证明结构。我们的研究特别关注结构感知演示和结构感知修剪。我们证明它们都有助于提高性能。提供了详细的分析以帮助理解结果。

Title: Generation with Dynamic Vocabulary

• Authors: Yanting Liu, Tao Ji, Changzhi Sun, Yuanbin Wu, Xiaoling Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08481
• Pdf URL: https://arxiv.org/pdf/2410.08481
• Copy Paste: [[2410.08481]] Generation with Dynamic Vocabulary(https://arxiv.org/abs/2410.08481)
• Keywords: language model
• Abstract: We introduce a new dynamic vocabulary for language models. It can involve arbitrary text spans during generation. These text spans act as basic generation bricks, akin to tokens in the traditional static vocabularies. We show that, the ability to generate multi-tokens atomically improve both generation quality and efficiency (compared to the standard language model, the MAUVE metric is increased by 25%, the latency is decreased by 20%). The dynamic vocabulary can be deployed in a plug-and-play way, thus is attractive for various downstream applications. For example, we demonstrate that dynamic vocabulary can be applied to different domains in a training-free manner. It also helps to generate reliable citations in question answering tasks (substantially enhancing citation results without compromising answer accuracy).
• 摘要：我们为语言模型引入了一种新的动态词汇表。它可以在生成过程中涉及任意文本跨度。这些文本跨度充当基本生成块，类似于传统静态词汇表中的标记。我们表明，自动生成多个标记的能力可以提高生成质量和效率（与标准语言模型相比，MAUVE 指标提高了 25%，延迟降低了 20%）。动态词汇表可以即插即用的方式部署，因此对各种下游应用具有吸引力。例如，我们证明动态词汇表可以以无需训练的方式应用于不同的领域。它还有助于在问答任务中生成可靠的引用（在不影响答案准确性的情况下显着增强引用结果）。

Title: Scaling Laws for Predicting Downstream Performance in LLMs

• Authors: Yangyi Chen, Binxuan Huang, Yifan Gao, Zhengyang Wang, Jingfeng Yang, Heng Ji
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2410.08527
• Pdf URL: https://arxiv.org/pdf/2410.08527
• Copy Paste: [[2410.08527]] Scaling Laws for Predicting Downstream Performance in LLMs(https://arxiv.org/abs/2410.08527)
• Keywords: language model, llm
• Abstract: Precise estimation of downstream performance in large language models (LLMs) prior to training is essential for guiding their development process. Scaling laws analysis utilizes the statistics of a series of significantly smaller sampling language models (LMs) to predict the performance of the target LLM. For downstream performance prediction, the critical challenge lies in the emergent abilities in LLMs that occur beyond task-specific computational thresholds. In this work, we focus on the pre-training loss as a more computation-efficient metric for performance estimation. Our two-stage approach consists of first estimating a function that maps computational resources (e.g., FLOPs) to the pre-training Loss using a series of sampling models, followed by mapping the pre-training loss to downstream task Performance after the critical "emergent phase". In preliminary experiments, this FLP solution accurately predicts the performance of LLMs with 7B and 13B parameters using a series of sampling LMs up to 3B, achieving error margins of 5% and 10%, respectively, and significantly outperforming the FLOPs-to-Performance approach. This motivates FLP-M, a fundamental approach for performance prediction that addresses the practical need to integrate datasets from multiple sources during pre-training, specifically blending general corpora with code data to accurately represent the common necessity. FLP-M extends the power law analytical function to predict domain-specific pre-training loss based on FLOPs across data sources, and employs a two-layer neural network to model the non-linear relationship between multiple domain-specific loss and downstream performance. By utilizing a 3B LLM trained on a specific ratio and a series of smaller sampling LMs, FLP-M can effectively forecast the performance of 3B and 7B LLMs across various data mixtures for most benchmarks within 10% error margins.
• 摘要：在训练之前精确估计大型语言模型 (LLM) 中的下游性能对于指导其开发过程至关重要。缩放定律分析利用一系列明显较小的采样语言模型 (LM) 的统计数据来预测目标 LLM 的性能。对于下游性能预测，关键挑战在于 LLM 中超出任务特定计算阈值的新兴能力。在这项工作中，我们专注于预训练损失，将其作为性能评估的计算效率更高的指标。我们的两阶段方法包括首先使用一系列采样模型估计将计算资源（例如 FLOP）映射到预训练损失的函数，然后在关键的“新兴阶段”之后将预训练损失映射到下游任务性能。在初步实验中，该 FLP 解决方案使用一系列高达 3B 的采样 LM 准确预测了具有 7B 和 13B 参数的 LLM 的性能，分别实现了 5% 和 10% 的误差幅度，并且明显优于 FLOPs-to-Performance 方法。这激发了 FLP-M 的诞生，这是一种性能预测的基本方法，它解决了在预训练期间集成来自多个来源的数据集的实际需求，具体来说，将一般语料与代码数据混合以准确表示常见的需求。FLP-M 扩展了幂律分析函数，以根据跨数据源的 FLOPs 预测特定领域的预训练损失，并采用两层神经网络来模拟多个特定领域的损失与下游性能之间的非线性关系。通过利用按特定比率训练的 3B LLM 和一系列较小的采样 LM，FLP-M 可以有效地预测 3B 和 7B LLM 在各种数据混合中的性能，对于大多数基准测试，误差幅度在 10% 以内。

Title: Humanity in AI: Detecting the Personality of Large Language Models

• Authors: Baohua Zhan, Yongyi Huang, Wenyao Cui, Huaping Zhang, Jianyun Shang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08545
• Pdf URL: https://arxiv.org/pdf/2410.08545
• Copy Paste: [[2410.08545]] Humanity in AI: Detecting the Personality of Large Language Models(https://arxiv.org/abs/2410.08545)
• Keywords: language model, gpt, llm, hallucination, chat
• Abstract: Questionnaires are a common method for detecting the personality of Large Language Models (LLMs). However, their reliability is often compromised by two main issues: hallucinations (where LLMs produce inaccurate or irrelevant responses) and the sensitivity of responses to the order of the presented options. To address these issues, we propose combining text mining with questionnaires method. Text mining can extract psychological features from the LLMs' responses without being affected by the order of options. Furthermore, because this method does not rely on specific answers, it reduces the influence of hallucinations. By normalizing the scores from both methods and calculating the root mean square error, our experiment results confirm the effectiveness of this approach. To further investigate the origins of personality traits in LLMs, we conduct experiments on both pre-trained language models (PLMs), such as BERT and GPT, as well as conversational models (ChatLLMs), such as ChatGPT. The results show that LLMs do contain certain personalities, for example, ChatGPT and ChatGLM exhibit the personality traits of 'Conscientiousness'. Additionally, we find that the personalities of LLMs are derived from their pre-trained data. The instruction data used to train ChatLLMs can enhance the generation of data containing personalities and expose their hidden personality. We compare the results with the human average personality score, and we find that the personality of FLAN-T5 in PLMs and ChatGPT in ChatLLMs is more similar to that of a human, with score differences of 0.34 and 0.22, respectively.
• 摘要：问卷调查是检测大型语言模型 (LLM) 性格的常用方法。然而，它们的可靠性往往受到两个主要问题的影响：幻觉（LLM 产生不准确或不相关的回答）和回答对呈现选项顺序的敏感性。为了解决这些问题，我们建议将文本挖掘与问卷调查方法相结合。文本挖掘可以从 LLM 的回答中提取心理特征，而不受选项顺序的影响。此外，由于这种方法不依赖于特定的答案，因此它减少了幻觉的影响。通过对两种方法的分数进行归一化并计算均方根误差，我们的实验结果证实了该方法的有效性。为了进一步研究 LLM 中性格特征的起源，我们对预训练语言模型 (PLM)（例如 BERT 和 GPT）以及对话模型（ChatLLM）（例如 ChatGPT）进行了实验。结果表明，LLM 确实包含某些性格特征，例如 ChatGPT 和 ChatGLM 表现出“尽责性”的性格特征。此外，我们发现 LLM 的性格特征源自其预训练数据。用于训练 ChatLLM 的指导数据可以增强包含性格的数据的生成并揭示其隐藏的性格。我们将结果与人类平均性格得分进行比较，发现 PLM 中的 FLAN-T5 和 ChatLLM 中的 ChatGPT 的性格与人类更相似，得分差异分别为 0.34 和 0.22。

Title: Parameter-Efficient Fine-Tuning of Large Language Models using Semantic Knowledge Tuning

• Authors: Nusrat Jahan Prottasha, Asif Mahmud, Md. Shohanur Islam Sobuj, Prakash Bhat, Md Kowsher, Niloofar Yousefi, Ozlem Ozmen Garibay
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08598
• Pdf URL: https://arxiv.org/pdf/2410.08598
• Copy Paste: [[2410.08598]] Parameter-Efficient Fine-Tuning of Large Language Models using Semantic Knowledge Tuning(https://arxiv.org/abs/2410.08598)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) are gaining significant popularity in recent years for specialized tasks using prompts due to their low computational cost. Standard methods like prefix tuning utilize special, modifiable tokens that lack semantic meaning and require extensive training for best performance, often falling short. In this context, we propose a novel method called Semantic Knowledge Tuning (SK-Tuning) for prompt and prefix tuning that employs meaningful words instead of random tokens. This method involves using a fixed LLM to understand and process the semantic content of the prompt through zero-shot capabilities. Following this, it integrates the processed prompt with the input text to improve the model's performance on particular tasks. Our experimental results show that SK-Tuning exhibits faster training times, fewer parameters, and superior performance on tasks such as text classification and understanding compared to other tuning methods. This approach offers a promising method for optimizing the efficiency and effectiveness of LLMs in processing language tasks.
• 摘要：近年来，大型语言模型 (LLM) 因其计算成本低而越来越受到使用提示的专门任务的欢迎。前缀调整等标准方法使用特殊的可修改标记，这些标记缺乏语义含义，需要大量训练才能获得最佳性能，但往往达不到要求。在这种情况下，我们提出了一种称为语义知识调整 (SK-Tuning) 的新方法，用于提示和前缀调整，该方法使用有意义的单词而不是随机标记。此方法涉及使用固定的 LLM 通过零样本功能来理解和处理提示的语义内容。随后，它将处理后的提示与输入文本集成，以提高模型在特定任务上的性能。我们的实验结果表明，与其他调整方法相比，SK-Tuning 在文本分类和理解等任务上表现出更快的训练时间、更少的参数和更出色的性能。这种方法为优化 LLM 在处理语言任务中的效率和有效性提供了一种有前途的方法。

Title: StraGo: Harnessing Strategic Guidance for Prompt Optimization

• Authors: Yurong Wu, Yan Gao, Bin Benjamin Zhu, Zineng Zhou, Xiaodi Sun, Sheng Yang, Jian-Guang Lou, Zhiming Ding, Linjun Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08601
• Pdf URL: https://arxiv.org/pdf/2410.08601
• Copy Paste: [[2410.08601]] StraGo: Harnessing Strategic Guidance for Prompt Optimization(https://arxiv.org/abs/2410.08601)
• Keywords: language model, llm, prompt
• Abstract: Prompt engineering is pivotal for harnessing the capabilities of large language models (LLMs) across diverse applications. While existing prompt optimization methods improve prompt effectiveness, they often lead to prompt drifting, where newly generated prompts can adversely impact previously successful cases while addressing failures. Furthermore, these methods tend to rely heavily on LLMs' intrinsic capabilities for prompt optimization tasks. In this paper, we introduce StraGo (Strategic-Guided Optimization), a novel approach designed to mitigate prompt drifting by leveraging insights from both successful and failed cases to identify critical factors for achieving optimization objectives. StraGo employs a how-to-do methodology, integrating in-context learning to formulate specific, actionable strategies that provide detailed, step-by-step guidance for prompt optimization. Extensive experiments conducted across a range of tasks, including reasoning, natural language understanding, domain-specific knowledge, and industrial applications, demonstrate StraGo's superior performance. It establishes a new state-of-the-art in prompt optimization, showcasing its ability to deliver stable and effective prompt improvements.
• 摘要：提示工程对于充分利用大型语言模型 (LLM) 在不同应用中的功能至关重要。虽然现有的提示优化方法可以提高提示的有效性，但它们往往会导致提示漂移，即新生成的提示可能会对以前成功的案例产生不利影响，同时解决失败的问题。此外，这些方法往往严重依赖 LLM 的内在功能来完成提示优化任务。在本文中，我们介绍了 StraGo（战略引导优化），这是一种新颖的方法，旨在通过利用成功和失败案例的见解来确定实现优化目标的关键因素，从而减轻提示漂移。StraGo 采用一种操作方法，整合了情境学习来制定具体的可操作策略，为提示优化提供详细的分步指导。在推理、自然语言理解、领域特定知识和工业应用等一系列任务中进行的大量实验证明了 StraGo 的卓越性能。它建立了提示优化领域的新领先水平，展示了其提供稳定有效的提示改进的能力。

Title: QEFT: Quantization for Efficient Fine-Tuning of LLMs

• Authors: Changhun Lee, Jun-gyu Jin, Younghyun Cho, Eunhyeok Park
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2410.08661
• Pdf URL: https://arxiv.org/pdf/2410.08661
• Copy Paste: [[2410.08661]] QEFT: Quantization for Efficient Fine-Tuning of LLMs(https://arxiv.org/abs/2410.08661)
• Keywords: language model, llm
• Abstract: With the rapid growth in the use of fine-tuning for large language models (LLMs), optimizing fine-tuning while keeping inference efficient has become highly important. However, this is a challenging task as it requires improvements in all aspects, including inference speed, fine-tuning speed, memory consumption, and, most importantly, model quality. Previous studies have attempted to achieve this by combining quantization with fine-tuning, but they have failed to enhance all four aspects simultaneously. In this study, we propose a new lightweight technique called Quantization for Efficient Fine-Tuning (QEFT). QEFT accelerates both inference and fine-tuning, is supported by robust theoretical foundations, offers high flexibility, and maintains good hardware compatibility. Our extensive experiments demonstrate that QEFT matches the quality and versatility of full-precision parameter-efficient fine-tuning, while using fewer resources. Our code is available at this https URL.
• 摘要：随着大型语言模型 (LLM) 中微调的使用快速增长，优化微调同时保持推理效率变得非常重要。然而，这是一项具有挑战性的任务，因为它需要改进所有方面，包括推理速度、微调速度、内存消耗，以及最重要的模型质量。先前的研究试图通过将量化与微调相结合来实现这一点，但它们未能同时增强所有四个方面。在本研究中，我们提出了一种称为量化高效微调 (QEFT) 的新型轻量级技术。QEFT 加速了推理和微调，有强大的理论基础支持，具有高灵活性，并保持了良好的硬件兼容性。我们大量的实验表明，QEFT 具有与全精度参数高效微调相当的质量和多功能性，同时使用更少的资源。我们的代码可在此 https URL 上找到。

Title: AMPO: Automatic Multi-Branched Prompt Optimization

• Authors: Sheng Yang, Yurong Wu, Yan Gao, Zineng Zhou, Bin Benjamin Zhu, Xiaodi Sun, Jian-Guang Lou, Zhiming Ding, Anbang Hu, Yuan Fang, Yunsong Li, Junyan Chen, Linjun Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08696
• Pdf URL: https://arxiv.org/pdf/2410.08696
• Copy Paste: [[2410.08696]] AMPO: Automatic Multi-Branched Prompt Optimization(https://arxiv.org/abs/2410.08696)
• Keywords: language model, llm, prompt
• Abstract: Prompt engineering is very important to enhance the performance of large language models (LLMs). When dealing with complex issues, prompt engineers tend to distill multiple patterns from examples and inject relevant solutions to optimize the prompts, achieving satisfying results. However, existing automatic prompt optimization techniques are only limited to producing single flow instructions, struggling with handling diverse patterns. In this paper, we present AMPO, an automatic prompt optimization method that can iteratively develop a multi-branched prompt using failure cases as feedback. Our goal is to explore a novel way of structuring prompts with multi-branches to better handle multiple patterns in complex tasks, for which we introduce three modules: Pattern Recognition, Branch Adjustment, and Branch Pruning. In experiments across five tasks, AMPO consistently achieves the best results. Additionally, our approach demonstrates significant optimization efficiency due to our adoption of a minimal search strategy.
• 摘要：提示工程对于提升大型语言模型 (LLM) 的性能非常重要。在处理复杂问题时，提示工程师倾向于从示例中提炼多种模式并注入相关解决方案来优化提示，从而获得令人满意的结果。然而，现有的自动提示优化技术仅限于生成单一流程指令，难以处理多种模式。在本文中，我们提出了 AMPO，这是一种自动提示优化方法，可以使用失败案例作为反馈迭代开发多分支提示。我们的目标是探索一种构造具有多分支的提示的新方法，以更好地处理复杂任务中的多种模式，为此我们引入了三个模块：模式识别、分支调整和分支修剪。在五个任务的实验中，AMPO 始终取得最佳结果。此外，由于我们采用了最小搜索策略，我们的方法表现出显著的优化效率。

Title: SocialGaze: Improving the Integration of Human Social Norms in Large Language Models

• Authors: Anvesh Rao Vijjini, Rakesh R. Menon, Jiayi Fu, Shashank Srivastava, Snigdha Chaturvedi
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2410.08698
• Pdf URL: https://arxiv.org/pdf/2410.08698
• Copy Paste: [[2410.08698]] SocialGaze: Improving the Integration of Human Social Norms in Large Language Models(https://arxiv.org/abs/2410.08698)
• Keywords: language model, gpt, llm, prompt
• Abstract: While much research has explored enhancing the reasoning capabilities of large language models (LLMs) in the last few years, there is a gap in understanding the alignment of these models with social values and norms. We introduce the task of judging social acceptance. Social acceptance requires models to judge and rationalize the acceptability of people's actions in social situations. For example, is it socially acceptable for a neighbor to ask others in the community to keep their pets indoors at night? We find that LLMs' understanding of social acceptance is often misaligned with human consensus. To alleviate this, we introduce SocialGaze, a multi-step prompting framework, in which a language model verbalizes a social situation from multiple perspectives before forming a judgment. Our experiments demonstrate that the SocialGaze approach improves the alignment with human judgments by up to 11 F1 points with the GPT-3.5 model. We also identify biases and correlations in LLMs in assigning blame that is related to features such as the gender (males are significantly more likely to be judged unfairly) and age (LLMs are more aligned with humans for older narrators).
• 摘要：尽管近几年来，许多研究都在探索如何增强大型语言模型 (LLM) 的推理能力，但在理解这些模型与社会价值观和规范的一致性方面仍然存在差距。我们引入了判断社会接受度的任务。社会接受度要求模型判断和合理化人们在社交场合中行为的可接受性。例如，邻居要求社区中的其他人晚上将宠物留在室内，这在社会上是否可以接受？我们发现 LLM 对社会接受度的理解往往与人类共识不一致。为了缓解这种情况，我们引入了 SocialGaze，这是一个多步骤提示框架，其中语言模型从多个角度对社交场合进行语言描述，然后形成判断。我们的实验表明，SocialGaze 方法将 GPT-3.5 模型与人类判断的一致性提高了多达 11 个 F1 点。我们还发现，法学硕士在追究责任时存在偏见和相关性，这些偏见和相关性与性别（男性更有可能受到不公平的评判）和年龄（对于年龄较大的叙述者，法学硕士更接近人类）等特征有关。

Title: On the token distance modeling ability of higher RoPE attention dimension

• Authors: Xiangyu Hong, Che Jiang, Biqing Qi, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08703
• Pdf URL: https://arxiv.org/pdf/2410.08703
• Copy Paste: [[2410.08703]] On the token distance modeling ability of higher RoPE attention dimension(https://arxiv.org/abs/2410.08703)
• Keywords: language model
• Abstract: Length extrapolation algorithms based on Rotary position embedding (RoPE) have shown promising results in extending the context length of language models. However, understanding how position embedding can capture longer-range contextual information remains elusive. Based on the intuition that different dimensions correspond to different frequency of changes in RoPE encoding, we conducted a dimension-level analysis to investigate the correlation between a hidden dimension of an attention head and its contribution to capturing long-distance dependencies. Using our correlation metric, we identified a particular type of attention heads, which we named Positional Heads, from various length-extrapolated models. These heads exhibit a strong focus on long-range information interaction and play a pivotal role in long input processing, as evidence by our ablation. We further demonstrate the correlation between the efficiency of length extrapolation and the extension of the high-dimensional attention allocation of these heads. The identification of Positional Heads provides insights for future research in long-text comprehension.
• 摘要：基于旋转位置嵌入 (RoPE) 的长度外推算法在扩展语言模型的上下文长度方面表现出良好的效果。然而，了解位置嵌入如何捕获更长距离的上下文信息仍然难以捉摸。基于不同维度对应 RoPE 编码中不同变化频率的直觉，我们进行了维度级分析，以研究注意力头的隐藏维度与其对捕获长距离依赖关系的贡献之间的相关性。使用我们的相关性指标，我们从各种长度外推模型中确定了一种特殊类型的注意力头，我们将其命名为位置头。这些头表现出对长距离信息交互的强烈关注，并在长输入处理中发挥关键作用，这是我们的消融所证明的。我们进一步证明了长度外推的效率与这些头的高维注意力分配扩展之间的相关性。位置头的识别为未来的长文本理解研究提供了见解。

Title: Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models

• Authors: Yeeun Kim, Young Rok Choi, Eunkyung Choi, Jinhwan Choi, Hai Jin Park, Wonseok Hwang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08731
• Pdf URL: https://arxiv.org/pdf/2410.08731
• Copy Paste: [[2410.08731]] Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models(https://arxiv.org/abs/2410.08731)
• Keywords: language model, gpt, llm, retrieval-augmented generation
• Abstract: Large language models (LLMs) have demonstrated remarkable performance in the legal domain, with GPT-4 even passing the Uniform Bar Exam in the U.S. However their efficacy remains limited for non-standardized tasks and tasks in languages other than English. This underscores the need for careful evaluation of LLMs within each legal system before application. Here, we introduce KBL, a benchmark for assessing the Korean legal language understanding of LLMs, consisting of (1) 7 legal knowledge tasks (510 examples), (2) 4 legal reasoning tasks (288 examples), and (3) the Korean bar exam (4 domains, 53 tasks, 2,510 examples). First two datasets were developed in close collaboration with lawyers to evaluate LLMs in practical scenarios in a certified manner. Furthermore, considering legal practitioners' frequent use of extensive legal documents for research, we assess LLMs in both a closed book setting, where they rely solely on internal knowledge, and a retrieval-augmented generation (RAG) setting, using a corpus of Korean statutes and precedents. The results indicate substantial room and opportunities for improvement.
• 摘要：大型语言模型 (LLM) 在法律领域表现出色，GPT-4 甚至通过了美国的统一律师资格考试。然而，它们对于非标准化任务和英语以外语言的任务的效力仍然有限。这强调了在应用之前需要仔细评估每个法律体系内的 LLM。在这里，我们介绍了 KBL，这是评估韩国法律语言对 LLM 理解的基准，包括 (1) 7 个法律知识任务（510 个示例）、(2) 4 个法律推理任务（288 个示例）和 (3) 韩国律师资格考试（4 个领域、53 个任务、2,510 个示例）。前两个数据集是与律师密切合作开发的，以认证的方式评估实际场景中的 LLM。此外，考虑到法律从业者经常使用大量法律文件进行研究，我们在闭卷式环境（仅依赖内部知识）和检索增强生成 (RAG) 环境中评估法学硕士，使用韩国法规和判例语料库。结果表明有很大的改进空间和机会。

Title: Measuring the Groundedness of Legal Question-Answering Systems

• Authors: Dietrich Trautmann, Natalia Ostapuk, Quentin Grail, Adrian Alan Pol, Guglielmo Bonifazi, Shang Gao, Martin Gajek
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08764
• Pdf URL: https://arxiv.org/pdf/2410.08764
• Copy Paste: [[2410.08764]] Measuring the Groundedness of Legal Question-Answering Systems(https://arxiv.org/abs/2410.08764)
• Keywords: language model, prompt
• Abstract: In high-stakes domains like legal question-answering, the accuracy and trustworthiness of generative AI systems are of paramount importance. This work presents a comprehensive benchmark of various methods to assess the groundedness of AI-generated responses, aiming to significantly enhance their reliability. Our experiments include similarity-based metrics and natural language inference models to evaluate whether responses are well-founded in the given contexts. We also explore different prompting strategies for large language models to improve the detection of ungrounded responses. We validated the effectiveness of these methods using a newly created grounding classification corpus, designed specifically for legal queries and corresponding responses from retrieval-augmented prompting, focusing on their alignment with source material. Our results indicate potential in groundedness classification of generated responses, with the best method achieving a macro-F1 score of 0.8. Additionally, we evaluated the methods in terms of their latency to determine their suitability for real-world applications, as this step typically follows the generation process. This capability is essential for processes that may trigger additional manual verification or automated response regeneration. In summary, this study demonstrates the potential of various detection methods to improve the trustworthiness of generative AI in legal settings.
• 摘要：在法律问答等高风险领域，生成式人工智能系统的准确性和可信度至关重要。这项工作提出了各种方法的综合基准，以评估人工智能生成的响应的扎实性，旨在显著提高其可靠性。我们的实验包括基于相似性的指标和自然语言推理模型，以评估响应在给定上下文中是否有充分依据。我们还探索了大型语言模型的不同提示策略，以提高对无依据响应的检测。我们使用新创建的扎实分类语料库验证了这些方法的有效性，该语料库专门为法律查询和检索增强提示的相应响应而设计，重点关注它们与源材料的一致性。我们的结果表明，生成的响应在扎实性分类方面具有潜力，最佳方法的宏观 F1 得分为 0.8。此外，我们根据延迟对这些方法进行了评估，以确定它们是否适用于实际应用，因为此步骤通常在生成过程之后进行。此功能对于可能触发额外手动验证或自动响应再生的过程至关重要。总之，这项研究展示了各种检测方法在法律环境中提高生成式人工智能的可信度的潜力。

Title: Data Processing for the OpenGPT-X Model Family

• Authors: Nicolo' Brandizzi, Hammam Abdelwahab, Anirban Bhowmick, Lennard Helmer, Benny Jörg Stein, Pavel Denisov, Qasid Saleem, Michael Fromm, Mehdi Ali, Richard Rutmann, Farzad Naderi, Mohamad Saif Agy, Alexander Schwirjow, Fabian Küch, Luzian Hahn, Malte Ostendorff, Pedro Ortiz Suarez, Georg Rehm, Dennis Wegener, Nicolas Flores-Herr, Joachim Köhler, Johannes Leveling
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08800
• Pdf URL: https://arxiv.org/pdf/2410.08800
• Copy Paste: [[2410.08800]] Data Processing for the OpenGPT-X Model Family(https://arxiv.org/abs/2410.08800)
• Keywords: language model, gpt, llm
• Abstract: This paper presents a comprehensive overview of the data preparation pipeline developed for the OpenGPT-X project, a large-scale initiative aimed at creating open and high-performance multilingual large language models (LLMs). The project goal is to deliver models that cover all major European languages, with a particular focus on real-world applications within the European Union. We explain all data processing steps, starting with the data selection and requirement definition to the preparation of the final datasets for model training. We distinguish between curated data and web data, as each of these categories is handled by distinct pipelines, with curated data undergoing minimal filtering and web data requiring extensive filtering and deduplication. This distinction guided the development of specialized algorithmic solutions for both pipelines. In addition to describing the processing methodologies, we provide an in-depth analysis of the datasets, increasing transparency and alignment with European data regulations. Finally, we share key insights and challenges faced during the project, offering recommendations for future endeavors in large-scale multilingual data preparation for LLMs.
• 摘要：本文全面概述了为 OpenGPT-X 项目开发的数据准备流程，OpenGPT-X 项目是一项大型计划，旨在创建开放且高性能的多语言大型语言模型 (LLM)。该项目的目标是提供涵盖所有主要欧洲语言的模型，特别关注欧盟内的实际应用。我们解释了所有数据处理步骤，从数据选择和需求定义开始，到准备用于模型训练的最终数据集。我们区分了精选数据和网络数据，因为每个类别都由不同的管道处理，精选数据经过最少的过滤，而网络数据需要大量过滤和重复数据删除。这种区别指导了这两个管道的专门算法解决方案的开发。除了描述处理方法外，我们还对数据集进行了深入分析，提高了透明度并与欧洲数据法规保持一致。最后，我们分享了项目期间面临的关键见解和挑战，为 LLM 大规模多语言数据准备的未来努力提供了建议。

Title: StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization

• Authors: Zhuoqun Li, Xuanang Chen, Haiyang Yu, Hongyu Lin, Yaojie Lu, Qiaoyu Tang, Fei Huang, Xianpei Han, Le Sun, Yongbin Li
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08815
• Pdf URL: https://arxiv.org/pdf/2410.08815
• Copy Paste: [[2410.08815]] StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization(https://arxiv.org/abs/2410.08815)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) in many knowledge-based tasks. However, existing RAG methods struggle with knowledge-intensive reasoning tasks, because useful information required to these tasks are badly scattered. This characteristic makes it difficult for existing RAG methods to accurately identify key information and perform global reasoning with such noisy augmentation. In this paper, motivated by the cognitive theories that humans convert raw information into various structured knowledge when tackling knowledge-intensive reasoning, we proposes a new framework, StructRAG, which can identify the optimal structure type for the task at hand, reconstruct original documents into this structured format, and infer answers based on the resulting structure. Extensive experiments across various knowledge-intensive tasks show that StructRAG achieves state-of-the-art performance, particularly excelling in challenging scenarios, demonstrating its potential as an effective solution for enhancing LLMs in complex real-world applications.
• 摘要：检索增强生成 (RAG) 是有效增强许多知识型任务中的大型语言模型 (LLM) 的关键手段。然而，现有的 RAG 方法在知识密集型推理任务中表现不佳，因为这些任务所需的有用信息非常分散。这一特性使得现有的 RAG 方法难以准确识别关键信息并在噪声增强的情况下进行全局推理。本文受人类在处理知识密集型推理时将原始信息转换为各种结构化知识的认知理论的启发，我们提出了一个新框架 StructRAG，它可以识别当前任务的最佳结构类型，将原始文档重建为这种结构化格式，并根据生成的结构推断答案。在各种知识密集型任务中进行的大量实验表明，StructRAG 实现了最先进的性能，尤其是在具有挑战性的场景中表现出色，展示了其作为在复杂的实际应用中增强 LLM 的有效解决方案的潜力。

Title: Which Demographics do LLMs Default to During Annotation?

• Authors: Christopher Bagdon, Aidan Combs, Lynn Greschner, Roman Klinger, Jiahui Li, Sean Papay, Nadine Probol, Yarik Menchaca Resendiz, Johannes Schäfer, Aswathy Velutharambath, Sabine Weber, Amelie Wührl
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08820
• Pdf URL: https://arxiv.org/pdf/2410.08820
• Copy Paste: [[2410.08820]] Which Demographics do LLMs Default to During Annotation?(https://arxiv.org/abs/2410.08820)
• Keywords: language model, llm, prompt
• Abstract: Demographics and cultural background of annotators influence the labels they assign in text annotation -- for instance, an elderly woman might find it offensive to read a message addressed to a "bro", but a male teenager might find it appropriate. It is therefore important to acknowledge label variations to not under-represent members of a society. Two research directions developed out of this observation in the context of using large language models (LLM) for data annotations, namely (1) studying biases and inherent knowledge of LLMs and (2) injecting diversity in the output by manipulating the prompt with demographic information. We combine these two strands of research and ask the question to which demographics an LLM resorts to when no demographics is given. To answer this question, we evaluate which attributes of human annotators LLMs inherently mimic. Furthermore, we compare non-demographic conditioned prompts and placebo-conditioned prompts (e.g., "you are an annotator who lives in house number 5") to demographics-conditioned prompts ("You are a 45 year old man and an expert on politeness annotation. How do you rate {instance}"). We study these questions for politeness and offensiveness annotations on the POPQUORN data set, a corpus created in a controlled manner to investigate human label variations based on demographics which has not been used for LLM-based analyses so far. We observe notable influences related to gender, race, and age in demographic prompting, which contrasts with previous studies that found no such effects.
• 摘要：注释者的人口统计和文化背景会影响他们在文本注释中分配的标签——例如，一位老年妇女可能会觉得阅读写给“兄弟”的消息是一种冒犯，但一位男性青少年可能会觉得这是合适的。因此，重要的是要承认标签的变化，以免低估社会成员。在使用大型语言模型 (LLM) 进行数据注释的背景下，从这一观察中发展出两个研究方向，即 (1) 研究 LLM 的偏见和固有知识和 (2) 通过使用人口统计信息操纵提示来为输出注入多样性。我们将这两条研究线索结合起来，并提出一个问题：当没有给出人口统计数据时，LLM 会诉诸于哪些人口统计数据。为了回答这个问题，我们评估了 LLM 固有模仿人类注释者的哪些属性。此外，我们将非人口统计学条件提示和安慰剂条件提示（例如，“您是住在 5 号房子的注释者”）与人口统计学条件提示（“您是 45 岁的男人，是礼貌注释方面的专家。您如何评价 {instance}”）。我们在 POPQUORN 数据集上研究这些问题的礼貌性和冒犯性注释，POPQUORN 数据集是一个以受控方式创建的语料库，用于调查基于人口统计学的人类标签变化，迄今为止尚未用于基于 LLM 的分析。我们观察到人口统计学提示中与性别、种族和年龄相关的显着影响，这与之前没有发现此类影响的研究形成鲜明对比。

Title: Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation

• Authors: Ruobing Wang, Daren Zha, Shi Yu, Qingfei Zhao, Yuxuan Chen, Yixuan Wang, Shuo Wang, Yukun Yan, Zhenghao Liu, Xu Han, Zhiyuan Liu, Maosong Sun
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08821
• Pdf URL: https://arxiv.org/pdf/2410.08821
• Copy Paste: [[2410.08821]] Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation(https://arxiv.org/abs/2410.08821)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-Augmented Generation (RAG) mitigates issues of the factual errors and hallucinated outputs generated by Large Language Models (LLMs) in open-domain question-answering tasks (OpenQA) via introducing external knowledge. For complex QA, however, existing RAG methods use LLMs to actively predict retrieval timing and directly use the retrieved information for generation, regardless of whether the retrieval timing accurately reflects the actual information needs, or sufficiently considers prior retrieved knowledge, which may result in insufficient information gathering and interaction, yielding low-quality answers. To address these, we propose a generic RAG approach called Adaptive Note-Enhanced RAG (Adaptive-Note) for complex QA tasks, which includes the iterative information collector, adaptive memory reviewer, and task-oriented generator, while following a new Retriever-and-Memory paradigm. Specifically, Adaptive-Note introduces an overarching view of knowledge growth, iteratively gathering new information in the form of notes and updating them into the existing optimal knowledge structure, enhancing high-quality knowledge interactions. In addition, we employ an adaptive, note-based stop-exploration strategy to decide "what to retrieve and when to stop" to encourage sufficient knowledge exploration. We conduct extensive experiments on five complex QA datasets, and the results demonstrate the superiority and effectiveness of our method and its components. The code and data are at this https URL.
• 摘要：检索增强生成 (RAG) 通过引入外部知识缓解了大型语言模型 (LLM) 在开放域问答任务 (OpenQA) 中生成的事实错误和幻觉输出的问题。但对于复杂的 QA，现有的 RAG 方法使用 LLM 主动预测检索时间并直接使用检索到的信息进行生成，而不管检索时间是否准确反映了实际的信息需求，或者是否充分考虑了先前检索到的知识，这可能导致信息收集和交互不足，从而产生低质量的答案。针对这些问题，我们提出了一种通用的 RAG 方法，称为自适应注释增强 RAG (Adaptive-Note)，用于复杂 QA 任务，其中包括迭代信息收集器、自适应记忆审阅器和面向任务的生成器，同时遵循新的检索器和记忆范式。具体来说，Adaptive-Note 引入了知识增长的总体视图，以笔记的形式迭代收集新信息并将其更新到现有的最佳知识结构中，从而增强高质量的知识交互。此外，我们采用基于笔记的自适应停止探索策略来决定“检索什么以及何时停止”，以鼓励充分的知识探索。我们在五个复杂的 QA 数据集上进行了广泛的实验，结果证明了我们的方法及其组件的优越性和有效性。代码和数据位于此 https URL。

Title: Measuring the Inconsistency of Large Language Models in Preferential Ranking

• Authors: Xiutian Zhao, Ke Wang, Wei Peng
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08851
• Pdf URL: https://arxiv.org/pdf/2410.08851
• Copy Paste: [[2410.08851]] Measuring the Inconsistency of Large Language Models in Preferential Ranking(https://arxiv.org/abs/2410.08851)
• Keywords: language model, llm, hallucination
• Abstract: Despite large language models' (LLMs) recent advancements, their bias and hallucination issues persist, and their ability to offer consistent preferential rankings remains underexplored. This study investigates the capacity of LLMs to provide consistent ordinal preferences, a crucial aspect in scenarios with dense decision space or lacking absolute answers. We introduce a formalization of consistency based on order theory, outlining criteria such as transitivity, asymmetry, reversibility, and independence from irrelevant alternatives. Our diagnostic experiments on selected state-of-the-art LLMs reveal their inability to meet these criteria, indicating a strong positional bias and poor transitivity, with preferences easily swayed by irrelevant alternatives. These findings highlight a significant inconsistency in LLM-generated preferential rankings, underscoring the need for further research to address these limitations.
• 摘要：尽管大型语言模型 (LLM) 最近取得了进展，但其偏见和幻觉问题仍然存在，而且其提供一致优先排名的能力仍未得到充分探索。本研究调查了 LLM 提供一致序数偏好的能力，这在决策空间密集或缺乏绝对答案的场景中是一个关键方面。我们引入了基于顺序理论的一致性形式化，概述了传递性、不对称性、可逆性和与无关替代方案的独立性等标准。我们对选定的最先进的 LLM 进行的诊断实验表明，它们无法满足这些标准，表明存在强烈的位置偏见和较差的传递性，偏好很容易受到无关替代方案的影响。这些发现凸显了 LLM 生成的优先排名存在显著的不一致性，强调需要进一步研究来解决这些局限性。

Title: Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies

• Authors: Yingqiang Gao, Lukas Fischer, Alexa Lintner, Sarah Ebling
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2410.08860
• Pdf URL: https://arxiv.org/pdf/2410.08860
• Copy Paste: [[2410.08860]] Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies(https://arxiv.org/abs/2410.08860)
• Keywords: language model, llm
• Abstract: Audio descriptions (ADs) function as acoustic commentaries designed to assist blind persons and persons with visual impairments in accessing digital media content on television and in movies, among other settings. As an accessibility service typically provided by trained AD professionals, the generation of ADs demands significant human effort, making the process both time-consuming and costly. Recent advancements in natural language processing (NLP) and computer vision (CV), particularly in large language models (LLMs) and vision-language models (VLMs), have allowed for getting a step closer to automatic AD generation. This paper reviews the technologies pertinent to AD generation in the era of LLMs and VLMs: we discuss how state-of-the-art NLP and CV technologies can be applied to generate ADs and identify essential research directions for the future.
• 摘要：音频描述 (AD) 可用作声音评论，旨在帮助盲人和视障人士在电视、电影等场景中访问数字媒体内容。作为一项通常由经过培训的 AD 专业人员提供的无障碍服务，生成 AD 需要大量人力，因此该过程既耗时又昂贵。自然语言处理 (NLP) 和计算机视觉 (CV) 的最新进展，尤其是大型语言模型 (LLM) 和视觉语言模型 (VLM) 的进展，让我们距离自动生成 AD 又近了一步。本文回顾了 LLM 和 VLM 时代与 AD 生成相关的技术：我们讨论了如何应用最先进的 NLP 和 CV 技术来生成 AD，并确定了未来的重要研究方向。

Title: RoRA-VLM: Robust Retrieval-Augmented Vision Language Models

• Authors: Jingyuan Qi, Zhiyang Xu, Rulin Shao, Yang Chen, Jing Di, Yu Cheng, Qifan Wang, Lifu Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08876
• Pdf URL: https://arxiv.org/pdf/2410.08876
• Copy Paste: [[2410.08876]] RoRA-VLM: Robust Retrieval-Augmented Vision Language Models(https://arxiv.org/abs/2410.08876)
• Keywords: language model
• Abstract: Current vision-language models (VLMs) still exhibit inferior performance on knowledge-intensive tasks, primarily due to the challenge of accurately encoding all the associations between visual objects and scenes to their corresponding entities and background knowledge. While retrieval augmentation methods offer an efficient way to integrate external knowledge, extending them to vision-language domain presents unique challenges in (1) precisely retrieving relevant information from external sources due to the inherent discrepancy within the multimodal queries, and (2) being resilient to the irrelevant, extraneous and noisy information contained in the retrieved multimodal knowledge snippets. In this work, we introduce RORA-VLM, a novel and robust retrieval augmentation framework specifically tailored for VLMs, with two key innovations: (1) a 2-stage retrieval process with image-anchored textual-query expansion to synergistically combine the visual and textual information in the query and retrieve the most relevant multimodal knowledge snippets; and (2) a robust retrieval augmentation method that strengthens the resilience of VLMs against irrelevant information in the retrieved multimodal knowledge by injecting adversarial noises into the retrieval-augmented training process, and filters out extraneous visual information, such as unrelated entities presented in images, via a query-oriented visual token refinement strategy. We conduct extensive experiments to validate the effectiveness and robustness of our proposed methods on three widely adopted benchmark datasets. Our results demonstrate that with a minimal amount of training instance, RORA-VLM enables the base model to achieve significant performance improvement and constantly outperform state-of-the-art retrieval-augmented VLMs on all benchmarks while also exhibiting a novel zero-shot domain transfer capability.
• 摘要：当前的视觉语言模型 (VLM) 在知识密集型任务上仍然表现不佳，这主要是因为难以准确地将视觉对象和场景之间的所有关联编码到相应的实体和背景知识。虽然检索增强方法提供了一种整合外部知识的有效方式，但将其扩展到视觉语言领域面临着独特的挑战：(1) 由于多模态查询中固有的差异，需要从外部来源精确检索相关信息，以及 (2) 对检索到的多模态知识片段中包含的不相关、无关和噪声信息具有弹性。在本文中，我们介绍了 RORA-VLM，这是一个专为 VLM 量身定制的新颖且强大的检索增强框架，具有两项关键创新：(1) 采用以图像锚定的文本查询扩展的 2 阶段检索过程，协同结合查询中的视觉和文本信息并检索最相关的多模态知识片段； (2) 一种强大的检索增强方法，通过在检索增强训练过程中注入对抗性噪声来增强 VLM 对检索到的多模态知识中不相关信息的弹性，并通过面向查询的视觉标记细化策略过滤掉无关的视觉信息，例如图像中呈现的不相关实体。我们在三个广泛采用的基准数据集上进行了大量实验来验证我们提出的方法的有效性和鲁棒性。我们的结果表明，只需最少的训练实例，RORA-VLM 就能使基础模型实现显着的性能提升，并在所有基准测试中不断超越最先进的检索增强 VLM，同时还表现出新颖的零样本域传输能力。

Title: A Benchmark for Cross-Domain Argumentative Stance Classification on Social Media

• Authors: Jiaqing Yuan, Ruijie Xi, Munindar P. Singh
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08900
• Pdf URL: https://arxiv.org/pdf/2410.08900
• Copy Paste: [[2410.08900]] A Benchmark for Cross-Domain Argumentative Stance Classification on Social Media(https://arxiv.org/abs/2410.08900)
• Keywords: language model
• Abstract: Argumentative stance classification plays a key role in identifying authors' viewpoints on specific topics. However, generating diverse pairs of argumentative sentences across various domains is challenging. Existing benchmarks often come from a single domain or focus on a limited set of topics. Additionally, manual annotation for accurate labeling is time-consuming and labor-intensive. To address these challenges, we propose leveraging platform rules, readily available expert-curated content, and large language models to bypass the need for human annotation. Our approach produces a multidomain benchmark comprising 4,498 topical claims and 30,961 arguments from three sources, spanning 21 domains. We benchmark the dataset in fully supervised, zero-shot, and few-shot settings, shedding light on the strengths and limitations of different methodologies. We release the dataset and code in this study at hidden for anonymity.
• 摘要：论证立场分类在识别作者对特定主题的观点方面起着关键作用。然而，在不同的领域生成不同的论证句子对是一项挑战。现有的基准通常来自单个领域或专注于有限的一组主题。此外，手动注释以进行准确标记既费时又费力。为了应对这些挑战，我们建议利用平台规则、现成的专家策划内容和大型语言模型来绕过人工注释的需要。我们的方法产生了一个多领域基准，包括来自三个来源的 4,498 个主题主张和 30,961 个论点，涵盖 21 个领域。我们在完全监督、零样本和少量样本设置下对数据集进行了基准测试，揭示了不同方法的优势和局限性。我们以匿名方式发布了本研究中的数据集和代码。

Title: Lifelong Event Detection via Optimal Transport

• Authors: Viet Dao, Van-Cuong Pham, Quyen Tran, Thanh-Thien Le, Linh Ngo Van, Thien Huu Nguyen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08905
• Pdf URL: https://arxiv.org/pdf/2410.08905
• Copy Paste: [[2410.08905]] Lifelong Event Detection via Optimal Transport(https://arxiv.org/abs/2410.08905)
• Keywords: language model
• Abstract: Continual Event Detection (CED) poses a formidable challenge due to the catastrophic forgetting phenomenon, where learning new tasks (with new coming event types) hampers performance on previous ones. In this paper, we introduce a novel approach, Lifelong Event Detection via Optimal Transport (LEDOT), that leverages optimal transport principles to align the optimization of our classification module with the intrinsic nature of each class, as defined by their pre-trained language modeling. Our method integrates replay sets, prototype latent representations, and an innovative Optimal Transport component. Extensive experiments on MAVEN and ACE datasets demonstrate LEDOT's superior performance, consistently outperforming state-of-the-art baselines. The results underscore LEDOT as a pioneering solution in continual event detection, offering a more effective and nuanced approach to addressing catastrophic forgetting in evolving environments.
• 摘要：由于灾难性遗忘现象，持续事件检测 (CED) 面临着巨大的挑战，即学习新任务（具有新出现的事件类型）会妨碍先前任务的表现。在本文中，我们介绍了一种新方法，即通过最佳传输进行终身事件检测 (LEDOT)，该方法利用最佳传输原理将分类模块的优化与每个类的内在性质相一致，这些内在性质由其预训练语言模型定义。我们的方法集成了重放集、原型潜在表示和创新的最佳传输组件。在 MAVEN 和 ACE 数据集上进行的大量实验证明了 LEDOT 的卓越性能，始终优于最先进的基线。结果强调 LEDOT 是持续事件检测的先驱解决方案，为解决不断变化的环境中的灾难性遗忘问题提供了一种更有效、更细致入微的方法。

Title: Towards Cross-Lingual LLM Evaluation for European Languages

• Authors: Klaudia Thellmann, Bernhard Stadler, Michael Fromm, Jasper Schulze Buschhoff, Alex Jude, Fabio Barth, Johannes Leveling, Nicolas Flores-Herr, Joachim Köhler, René Jäkel, Mehdi Ali
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2410.08928
• Pdf URL: https://arxiv.org/pdf/2410.08928
• Copy Paste: [[2410.08928]] Towards Cross-Lingual LLM Evaluation for European Languages(https://arxiv.org/abs/2410.08928)
• Keywords: language model, llm
• Abstract: The rise of Large Language Models (LLMs) has revolutionized natural language processing across numerous languages and tasks. However, evaluating LLM performance in a consistent and meaningful way across multiple European languages remains challenging, especially due to the scarcity of multilingual benchmarks. We introduce a cross-lingual evaluation approach tailored for European languages. We employ translated versions of five widely-used benchmarks to assess the capabilities of 40 LLMs across 21 European languages. Our contributions include examining the effectiveness of translated benchmarks, assessing the impact of different translation services, and offering a multilingual evaluation framework for LLMs that includes newly created datasets: EU20-MMLU, EU20-HellaSwag, EU20-ARC, EU20-TruthfulQA, and EU20-GSM8K. The benchmarks and results are made publicly available to encourage further research in multilingual LLM evaluation.
• 摘要：大型语言模型 (LLM) 的兴起彻底改变了众多语言和任务的自然语言处理。然而，以一致且有意义的方式评估多种欧洲语言的 LLM 性能仍然具有挑战性，尤其是由于多语言基准的稀缺。我们引入了一种针对欧洲语言量身定制的跨语言评估方法。我们使用五个广泛使用的基准的翻译版本来评估 21 种欧洲语言的 40 个 LLM 的能力。我们的贡献包括检查翻译基准的有效性、评估不同翻译服务的影响，并为 LLM 提供一个多语言评估框架，其中包括新创建的数据集：EU20-MMLU、EU20-HellaSwag、EU20-ARC、EU20-TruthfulQA 和 EU20-GSM8K。基准和结果已公开，以鼓励对多语言 LLM 评估的进一步研究。

Title: Language Imbalance Driven Rewarding for Multilingual Self-improving

• Authors: Wen Yang, Junhong Wu, Chen Wang, Chengqing Zong, Jiajun Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08964
• Pdf URL: https://arxiv.org/pdf/2410.08964
• Copy Paste: [[2410.08964]] Language Imbalance Driven Rewarding for Multilingual Self-improving(https://arxiv.org/abs/2410.08964)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have achieved state-of-the-art performance across numerous tasks. However, these advancements have predominantly benefited "first-class" languages such as English and Chinese, leaving many other languages underrepresented. This imbalance, while limiting broader applications, generates a natural preference ranking between languages, offering an opportunity to bootstrap the multilingual capabilities of LLM in a self-improving manner. Thus, we propose $\textit{Language Imbalance Driven Rewarding}$, where the inherent imbalance between dominant and non-dominant languages within LLMs is leveraged as a reward signal. Iterative DPO training demonstrates that this approach not only enhances LLM performance in non-dominant languages but also improves the dominant language's capacity, thereby yielding an iterative reward signal. Fine-tuning Meta-Llama-3-8B-Instruct over two iterations of this approach results in continuous improvements in multilingual performance across instruction-following and arithmetic reasoning tasks, evidenced by an average improvement of 7.46% win rate on the X-AlpacaEval leaderboard and 13.9% accuracy on the MGSM benchmark. This work serves as an initial exploration, paving the way for multilingual self-improvement of LLMs.
• 摘要：大型语言模型 (LLM) 已在众多任务中取得了最佳表现。然而，这些进步主要惠及英语和中文等“一流”语言，而许多其他语言却没有得到充分重视。这种不平衡虽然限制了更广泛的应用，但却在语言之间产生了自然的偏好排名，为以自我改进的方式引导 LLM 的多语言能力提供了机会。因此，我们提出了“语言不平衡驱动奖励”，其中 LLM 中主导语言和非主导语言之间固有的不平衡被用作奖励信号。迭代 DPO 训练表明，这种方法不仅可以提高非主导语言的 LLM 性能，还可以提高主导语言的能力，从而产生迭代奖励信号。通过对 Meta-Llama-3-8B-Instruct 进行两次迭代微调，在指令遵循和算术推理任务中，多语言性能不断提高，X-AlpacaEval 排行榜上的胜率平均提高了 7.46%，MGSM 基准上的准确率提高了 13.9%。这项工作是初步探索，为 LLM 的多语言自我完善铺平了道路。

Title: Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements

• Authors: Jingyu Zhang, Ahmed Elgohary, Ahmed Magooda, Daniel Khashabi, Benjamin Van Durme
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08968
• Pdf URL: https://arxiv.org/pdf/2410.08968
• Copy Paste: [[2410.08968]] Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements(https://arxiv.org/abs/2410.08968)
• Keywords: language model, llm, prompt
• Abstract: The current paradigm for safety alignment of large language models (LLMs) follows a one-size-fits-all approach: the model refuses to interact with any content deemed unsafe by the model provider. This approach lacks flexibility in the face of varying social norms across cultures and regions. In addition, users may have diverse safety needs, making a model with static safety standards too restrictive to be useful, as well as too costly to be re-aligned. We propose Controllable Safety Alignment (CoSA), a framework designed to adapt models to diverse safety requirements without re-training. Instead of aligning a fixed model, we align models to follow safety configs -- free-form natural language descriptions of the desired safety behaviors -- that are provided as part of the system prompt. To adjust model safety behavior, authorized users only need to modify such safety configs at inference time. To enable that, we propose CoSAlign, a data-centric method for aligning LLMs to easily adapt to diverse safety configs. Furthermore, we devise a novel controllability evaluation protocol that considers both helpfulness and configured safety, summarizing them into CoSA-Score, and construct CoSApien, a human-authored benchmark that consists of real-world LLM use cases with diverse safety requirements and corresponding evaluation prompts. We show that CoSAlign leads to substantial gains of controllability over strong baselines including in-context alignment. Our framework encourages better representation and adaptation to pluralistic human values in LLMs, and thereby increasing their practicality.
• 摘要：目前，大型语言模型 (LLM) 的安全对齐范式遵循一刀切的方法：模型拒绝与模型提供者认为不安全的任何内容进行交互。面对不同文化和地区的不同社会规范，这种方法缺乏灵活性。此外，用户可能有不同的安全需求，这使得具有静态安全标准的模型过于严格而无法使用，并且重新对齐的成本太高。我们提出了可控安全对齐 (CoSA)，这是一个旨在使模型适应各种安全要求而无需重新训练的框架。我们不是对齐固定模型，而是对齐模型以遵循安全配置（所需安全行为的自由形式自然语言描述），这些配置是系统提示的一部分。要调整模型安全行为，授权用户只需在推理时修改此类安全配置。为了实现这一点，我们提出了 CoSAlign，这是一种以数据为中心的方法，用于对齐 LLM 以轻松适应不同的安全配置。此外，我们设计了一种新颖的可控性评估协议，该协议同时考虑了有用性和配置的安全性，并将它们总结为 CoSA-Score，并构建了 CoSApien，这是一个由人类编写的基准，由具有各种安全要求和相应评估提示的真实 LLM 用例组成。我们表明，CoSAlign 比包括上下文对齐在内的强基线可显著提高可控性。我们的框架鼓励在 LLM 中更好地表示和适应多元化的人类价值观，从而提高其实用性。

Title: NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models

• Authors: Zheng Yi Ho, Siyuan Liang, Sen Zhang, Yibing Zhan, Dacheng Tao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.08970
• Pdf URL: https://arxiv.org/pdf/2410.08970
• Copy Paste: [[2410.08970]] NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models(https://arxiv.org/abs/2410.08970)
• Keywords: language model, llm, hallucination
• Abstract: Hallucinations in Large Language Models (LLMs) remain a major obstacle, particularly in high-stakes applications where factual accuracy is critical. While representation editing and reading methods have made strides in reducing hallucinations, their heavy reliance on specialised tools and training on in-domain samples, makes them difficult to scale and prone to overfitting. This limits their accuracy gains and generalizability to diverse datasets. This paper presents a lightweight method, Norm Voting (NoVo), which harnesses the untapped potential of attention head norms to dramatically enhance factual accuracy in zero-shot multiple-choice questions (MCQs). NoVo begins by automatically selecting truth-correlated head norms with an efficient, inference-only algorithm using only 30 random samples, allowing NoVo to effortlessly scale to diverse datasets. Afterwards, selected head norms are employed in a simple voting algorithm, which yields significant gains in prediction accuracy. On TruthfulQA MC1, NoVo surpasses the current state-of-the-art and all previous methods by an astounding margin -- at least 19 accuracy points. NoVo demonstrates exceptional generalization to 20 diverse datasets, with significant gains in over 90\% of them, far exceeding all current representation editing and reading methods. NoVo also reveals promising gains to finetuning strategies and building textual adversarial defence. NoVo's effectiveness with head norms opens new frontiers in LLM interpretability, robustness and reliability.
• 摘要：大型语言模型 (LLM) 中的幻觉仍然是一个主要障碍，特别是在事实准确性至关重要的高风险应用中。虽然表征编辑和阅读方法在减少幻觉方面取得了长足进步，但它们严重依赖专门的工具和对领域内样本的训练，这使得它们难以扩展且容易过度拟合。这限制了它们的准确性提升和对不同数据集的普遍性。本文提出了一种轻量级方法，即规范投票 (NoVo)，它利用注意力头部规范的未开发潜力来显着提高零样本多项选择题 (MCQ) 的事实准确性。NoVo 首先使用高效的推理算法自动选择与真相相关的头部规范，仅使用 30 个随机样本，从而使 NoVo 能够毫不费力地扩展到不同的数据集。之后，在简单的投票算法中使用选定的头部规范，这可以显着提高预测准确性。在 TruthfulQA MC1 上，NoVo 以惊人的优势超越了当前最先进的方法和所有以前的方法——至少 19 个准确度点。NoVo 展示了对 20 个不同数据集的出色泛化能力，其中 90% 以上取得了显著的进步，远远超过了所有当前的表示编辑和阅读方法。NoVo 还展示了微调策略和构建文本对抗防御的可喜进步。NoVo 对头部规范的有效性为 LLM 的可解释性、稳健性和可靠性开辟了新领域。

Title: Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory

• Authors: Rebecca M. M. Hicke, Ross Deans Kristensen-McLachlan
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2410.08991
• Pdf URL: https://arxiv.org/pdf/2410.08991
• Copy Paste: [[2410.08991]] Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory(https://arxiv.org/abs/2410.08991)
• Keywords: language model, llm, prompt
• Abstract: Metaphors are everywhere. They appear extensively across all domains of natural language, from the most sophisticated poetry to seemingly dry academic prose. A significant body of research in the cognitive science of language argues for the existence of conceptual metaphors, the systematic structuring of one domain of experience in the language of another. Conceptual metaphors are not simply rhetorical flourishes but are crucial evidence of the role of analogical reasoning in human cognition. In this paper, we ask whether Large Language Models (LLMs) can accurately identify and explain the presence of such conceptual metaphors in natural language data. Using a novel prompting technique based on metaphor annotation guidelines, we demonstrate that LLMs are a promising tool for large-scale computational research on conceptual metaphors. Further, we show that LLMs are able to apply procedural guidelines designed for human annotators, displaying a surprising depth of linguistic knowledge.
• 摘要：隐喻无处不在。它们广泛出现在自然语言的所有领域，从最复杂的诗歌到看似枯燥的学术散文。语言认知科学的大量研究都主张概念隐喻的存在，即一个经验领域在另一个经验领域的语言中的系统结构。概念隐喻不仅仅是修辞手法，而且是类比推理在人类认知中的作用的关键证据。在本文中，我们探讨大型语言模型 (LLM) 是否能够准确识别和解释自然语言数据中此类概念隐喻的存在。使用基于隐喻注释指南的新型提示技术，我们证明了 LLM 是用于大规模概念隐喻计算研究的有前途的工具。此外，我们表明 LLM 能够应用为人类注释者设计的程序指南，显示出令人惊讶的语言知识深度。

Title: Hypothesis-only Biases in Large Language Model-Elicited Natural Language Inference

• Authors: Grace Proebsting, Adam Poliak
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.08996
• Pdf URL: https://arxiv.org/pdf/2410.08996
• Copy Paste: [[2410.08996]] Hypothesis-only Biases in Large Language Model-Elicited Natural Language Inference(https://arxiv.org/abs/2410.08996)
• Keywords: language model, gpt, llm
• Abstract: We test whether replacing crowdsource workers with LLMs to write Natural Language Inference (NLI) hypotheses similarly results in annotation artifacts. We recreate a portion of the Stanford NLI corpus using GPT-4, Llama-2 and Mistral 7b, and train hypothesis-only classifiers to determine whether LLM-elicited hypotheses contain annotation artifacts. On our LLM-elicited NLI datasets, BERT-based hypothesis-only classifiers achieve between 86-96% accuracy, indicating these datasets contain hypothesis-only artifacts. We also find frequent "give-aways" in LLM-generated hypotheses, e.g. the phrase "swimming in a pool" appears in more than 10,000 contradictions generated by GPT-4. Our analysis provides empirical evidence that well-attested biases in NLI can persist in LLM-generated data.
• 摘要：我们测试用 LLM 代替众包工作者编写自然语言推理 (NLI) 假设是否同样会产生注释伪影。我们使用 GPT-4、Llama-2 和 Mistral 7b 重建了斯坦福 NLI 语料库的一部分，并训练仅假设分类器以确定 LLM 引发的假设是否包含注释伪影。在我们 LLM 引发的 NLI 数据集上，基于 BERT 的仅假设分类器的准确率达到 86-96%，这表明这些数据集包含仅假设伪影。我们还发现 LLM 生成的假设中经常出现“泄露”，例如短语“在游泳池中游泳”出现在 GPT-4 生成的 10,000 多个矛盾中。我们的分析提供了经验证据，表明 NLI 中经过充分证明的偏见可能会在 LLM 生成的数据中持续存在。

Title: SuperCorrect: Supervising and Correcting Language Models with Error-Driven Insights

• Authors: Ling Yang, Zhaochen Yu, Tianjun Zhang, Minkai Xu, Joseph E. Gonzalez, Bin Cui, Shuicheng Yan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.09008
• Pdf URL: https://arxiv.org/pdf/2410.09008
• Copy Paste: [[2410.09008]] SuperCorrect: Supervising and Correcting Language Models with Error-Driven Insights(https://arxiv.org/abs/2410.09008)
• Keywords: language model, gpt, llm
• Abstract: Large language models (LLMs) like GPT-4, PaLM, and LLaMA have shown significant improvements in various reasoning tasks. However, smaller models such as Llama-3-8B and DeepSeekMath-Base still struggle with complex mathematical reasoning because they fail to effectively identify and correct reasoning errors. Recent reflection-based methods aim to address these issues by enabling self-reflection and self-correction, but they still face challenges in independently detecting errors in their reasoning steps. To overcome these limitations, we propose SuperCorrect, a novel two-stage framework that uses a large teacher model to supervise and correct both the reasoning and reflection processes of a smaller student model. In the first stage, we extract hierarchical high-level and detailed thought templates from the teacher model to guide the student model in eliciting more fine-grained reasoning thoughts. In the second stage, we introduce cross-model collaborative direct preference optimization (DPO) to enhance the self-correction abilities of the student model by following the teacher's correction traces during training. This cross-model DPO approach teaches the student model to effectively locate and resolve erroneous thoughts with error-driven insights from the teacher model, breaking the bottleneck of its thoughts and acquiring new skills and knowledge to tackle challenging problems. Extensive experiments consistently demonstrate our superiority over previous methods. Notably, our SuperCorrect-7B model significantly surpasses powerful DeepSeekMath-7B by 7.8%/5.3% and Qwen2.5-Math-7B by 15.1%/6.3% on MATH/GSM8K benchmarks, achieving new SOTA performance among all 7B models. Code: this https URL
• 摘要：GPT-4、PaLM 和 LLaMA 等大型语言模型 (LLM) 在各种推理任务中都表现出了显著的进步。然而，较小的模型（如 Llama-3-8B 和 DeepSeekMath-Base）仍然难以应对复杂的数学推理，因为它们无法有效地识别和纠正推理错误。最近的基于反思的方法旨在通过实现自我反思和自我纠正来解决这些问题，但它们在独立检测推理步骤中的错误方面仍然面临挑战。为了克服这些限制，我们提出了 SuperCorrect，这是一个新颖的两阶段框架，它使用大型教师模型来监督和纠正较小的学生模型的推理和反思过程。在第一阶段，我们从教师模型中提取分层的高级和详细思维模板，以指导学生模型引出更细粒度的推理思维。在第二阶段，我们引入了跨模型协作直接偏好优化 (DPO)，通过在训练期间跟踪教师的纠正轨迹来增强学生模型的自我纠正能力。这种跨模型 DPO 方法教会学生模型利用教师模型的错误驱动洞察有效地定位和解决错误的想法，打破其思维瓶颈并获得新技能和知识来应对具有挑战性的问题。大量实验始终证明我们优于以前的方法。值得注意的是，我们的 SuperCorrect-7B 模型在 MATH/GSM8K 基准上显著超越强大的 DeepSeekMath-7B 7.8%/5.3%，超越 Qwen2.5-Math-7B 15.1%/6.3%，在所有 7B 模型中取得了新的 SOTA 性能。代码：这个 https URL

Title: The Impact of Visual Information in Chinese Characters: Evaluating Large Models' Ability to Recognize and Utilize Radicals

• Authors: Xiaofeng Wu, Karl Stratos, Wei Xu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.09013
• Pdf URL: https://arxiv.org/pdf/2410.09013
• Copy Paste: [[2410.09013]] The Impact of Visual Information in Chinese Characters: Evaluating Large Models' Ability to Recognize and Utilize Radicals(https://arxiv.org/abs/2410.09013)
• Keywords: language model, llm, prompt
• Abstract: The glyphic writing system of Chinese incorporates information-rich visual features in each character, such as radicals that provide hints about meaning or pronunciation. However, there has been no investigation into whether contemporary Large Language Models (LLMs) and Vision-Language Models (VLMs) can harness these sub-character features in Chinese through prompting. In this study, we establish a benchmark to evaluate LLMs' and VLMs' understanding of visual elements in Chinese characters, including radicals, composition structures, strokes, and stroke counts. Our results reveal that models surprisingly exhibit some, but still limited, knowledge of the visual information, regardless of whether images of characters are provided. To incite models' ability to use radicals, we further experiment with incorporating radicals into the prompts for Chinese language understanding tasks. We observe consistent improvement in Part-Of-Speech tagging when providing additional information about radicals, suggesting the potential to enhance CLP by integrating sub-character information.
• 摘要：中文的字形书写系统在每个字符中都包含信息丰富的视觉特征，例如提供有关含义或发音的提示的部首。然而，目前还没有研究当代大型语言模型 (LLM) 和视觉语言模型 (VLM) 是否可以通过提示利用中文中的这些子字符特征。在本研究中，我们建立了一个基准来评估 LLM 和 VLM 对汉字中视觉元素的理解，包括部首、构图结构、笔画和笔画数。我们的结果表明，无论是否提供字符图像，模型都令人惊讶地表现出一些但仍然有限的视觉信息知识。为了激发模型使用部首的能力，我们进一步尝试将部首纳入中文语言理解任务的提示中。我们观察到，在提供有关部首的额外信息时，词性标注得到了持续改善，这表明通过整合子字符信息可以增强 CLP。

Title: MedMobile: A mobile-sized language model with expert-level clinical capabilities

• Authors: Krithik Vishwanath, Jaden Stryker, Anton Alaykin, Daniel Alexander Alber, Eric Karl Oermann
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.09019
• Pdf URL: https://arxiv.org/pdf/2410.09019
• Copy Paste: [[2410.09019]] MedMobile: A mobile-sized language model with expert-level clinical capabilities(https://arxiv.org/abs/2410.09019)
• Keywords: language model, retrieval augmented generation
• Abstract: Language models (LMs) have demonstrated expert-level reasoning and recall abilities in medicine. However, computational costs and privacy concerns are mounting barriers to wide-scale implementation. We introduce a parsimonious adaptation of phi-3-mini, MedMobile, a 3.8 billion parameter LM capable of running on a mobile device, for medical applications. We demonstrate that MedMobile scores 75.7% on the MedQA (USMLE), surpassing the passing mark for physicians (~60%), and approaching the scores of models 100 times its size. We subsequently perform a careful set of ablations, and demonstrate that chain of thought, ensembling, and fine-tuning lead to the greatest performance gains, while unexpectedly retrieval augmented generation fails to demonstrate significant improvements
• 摘要：语言模型 (LM) 已在医学领域展现出专家级的推理和回忆能力。然而，计算成本和隐私问题日益成为大规模实施的障碍。我们引入了 phi-3-mini 的简约改编版 MedMobile，这是一个能够在移动设备上运行的 38 亿参数 LM，适用于医疗应用。我们证明 MedMobile 在 MedQA (USMLE) 上的得分为 75.7%，超过了医生的及格分数（约 60%），并接近其 100 倍大小的模型的得分。随后，我们进行了一组仔细的消融，并证明思路链、集成和微调可带来最大的性能提升，而意外的检索增强生成未能展示出显着的改进

Title: Mentor-KD: Making Small Language Models Better Multi-step Reasoners

• Authors: Hojae Lee, Junho Kim, SangKeun Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.09037
• Pdf URL: https://arxiv.org/pdf/2410.09037
• Copy Paste: [[2410.09037]] Mentor-KD: Making Small Language Models Better Multi-step Reasoners(https://arxiv.org/abs/2410.09037)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Large Language Models (LLMs) have displayed remarkable performances across various complex tasks by leveraging Chain-of-Thought (CoT) prompting. Recently, studies have proposed a Knowledge Distillation (KD) approach, reasoning distillation, which transfers such reasoning ability of LLMs through fine-tuning language models of multi-step rationales generated by LLM teachers. However, they have inadequately considered two challenges regarding insufficient distillation sets from the LLM teacher model, in terms of 1) data quality and 2) soft label provision. In this paper, we propose Mentor-KD, which effectively distills the multi-step reasoning capability of LLMs to smaller LMs while addressing the aforementioned challenges. Specifically, we exploit a mentor, intermediate-sized task-specific fine-tuned model, to augment additional CoT annotations and provide soft labels for the student model during reasoning distillation. We conduct extensive experiments and confirm Mentor-KD's effectiveness across various models and complex reasoning tasks.
• 摘要：大型语言模型 (LLM) 通过利用思路链 (CoT) 提示在各种复杂任务中表现出色。最近，研究提出了一种知识蒸馏 (KD) 方法，即推理蒸馏，它通过对 LLM 教师生成的多步骤原理的语言模型进行微调来转移 LLM 的这种推理能力。然而，他们没有充分考虑 LLM 教师模型中蒸馏集不足的两个挑战，即 1) 数据质量和 2) 软标签提供。在本文中，我们提出了 Mentor-KD，它在解决上述挑战的同时有效地将 LLM 的多步推理能力蒸馏到较小的 LM。具体来说，我们利用导师、中型任务特定的微调模型，在推理蒸馏过程中增强额外的 CoT 注释并为学生模型提供软标签。我们进行了广泛的实验并确认了 Mentor-KD 在各种模型和复杂推理任务中的有效性。

Title: SimpleStrat: Diversifying Language Model Generation with Stratification

• Authors: Justin Wong, Yury Orlovskiy, Michael Luo, Sanjit A. Seshia, Joseph E. Gonzalez
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2410.09038
• Pdf URL: https://arxiv.org/pdf/2410.09038
• Copy Paste: [[2410.09038]] SimpleStrat: Diversifying Language Model Generation with Stratification(https://arxiv.org/abs/2410.09038)
• Keywords: language model, gpt, llm
• Abstract: Generating diverse responses from large language models (LLMs) is crucial for applications such as planning/search and synthetic data generation, where diversity provides distinct answers across generations. Prior approaches rely on increasing temperature to increase diversity. However, contrary to popular belief, we show not only does this approach produce lower quality individual generations as temperature increases, but it depends on model's next-token probabilities being similar to the true distribution of answers. We propose \method{}, an alternative approach that uses the language model itself to partition the space into strata. At inference, a random stratum is selected and a sample drawn from within the strata. To measure diversity, we introduce CoverageQA, a dataset of underspecified questions with multiple equally plausible answers, and assess diversity by measuring KL Divergence between the output distribution and uniform distribution over valid ground truth answers. As computing probability per response/solution for proprietary models is infeasible, we measure recall on ground truth solutions. Our evaluation show using SimpleStrat achieves higher recall by 0.05 compared to GPT-4o and 0.36 average reduction in KL Divergence compared to Llama 3.
• 摘要：从大型语言模型 (LLM) 生成多样化的响应对于规划/搜索和合成数据生成等应用至关重要，在这些应用中，多样性会在各代之间提供不同的答案。先前的方法依赖于提高温度来增加多样性。然而，与普遍看法相反，我们表明，这种方法不仅会随着温度升高而产生质量较低的单个代，而且还取决于模型的下一个标记概率是否与答案的真实分布相似。我们提出了 \method{}，这是一种使用语言模型本身将空间划分为层的替代方法。在推理时，选择一个随机层并从层中抽取样本。为了衡量多样性，我们引入了 CoverageQA，这是一个具有多个同样合理答案的未指定问题的数据集，并通过测量输出分布与有效基本事实答案的均匀分布之间的 KL 散度来评估多样性。由于无法为专有模型计算每个响应/解决方案的概率，因此我们测量基本事实解决方案的召回率。我们的评估表明，与 GPT-4o 相比，使用 SimpleStrat 的召回率提高了 0.05，与 Llama 3 相比，KL 散度平均降低了 0.36。

Title: AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation

• Authors: Zijun Wang, Haoqin Tu, Jieru Mei, Bingchen Zhao, Yisen Wang, Cihang Xie
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2410.09040
• Pdf URL: https://arxiv.org/pdf/2410.09040
• Copy Paste: [[2410.09040]] AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation(https://arxiv.org/abs/2410.09040)
• Keywords: language model, gpt, llm, prompt
• Abstract: This paper studies the vulnerabilities of transformer-based Large Language Models (LLMs) to jailbreaking attacks, focusing specifically on the optimization-based Greedy Coordinate Gradient (GCG) strategy. We first observe a positive correlation between the effectiveness of attacks and the internal behaviors of the models. For instance, attacks tend to be less effective when models pay more attention to system prompts designed to ensure LLM safety alignment. Building on this discovery, we introduce an enhanced method that manipulates models' attention scores to facilitate LLM jailbreaking, which we term AttnGCG. Empirically, AttnGCG shows consistent improvements in attack efficacy across diverse LLMs, achieving an average increase of ~7% in the Llama-2 series and ~10% in the Gemma series. Our strategy also demonstrates robust attack transferability against both unseen harmful goals and black-box LLMs like GPT-3.5 and GPT-4. Moreover, we note our attention-score visualization is more interpretable, allowing us to gain better insights into how our targeted attention manipulation facilitates more effective jailbreaking. We release the code at this https URL.
• 摘要：本文研究了基于 Transformer 的大型语言模型 (LLM) 易受越狱攻击的弱点，特别关注基于优化的贪婪坐标梯度 (GCG) 策略。我们首先观察到攻击的有效性与模型的内部行为之间存在正相关性。例如，当模型更加关注旨在确保 LLM 安全对齐的系统提示时，攻击往往不那么有效。基于这一发现，我们引入了一种增强方法，可以操纵模型的注意力分数以促进 LLM 越狱，我们称之为 AttnGCG。从经验上看，AttnGCG 在不同的 LLM 中表现出攻击效力的持续提高，在 Llama-2 系列中平均增加了约 7%，在 Gemma 系列中平均增加了约 10%。我们的策略还展示了针对看不见的有害目标和黑盒 LLM（如 GPT-3.5 和 GPT-4）的强大攻击可转移性。此外，我们注意到我们的注意力分数可视化更具可解释性，使我们能够更好地了解我们的有针对性的注意力操纵如何促进更有效的越狱。我们在这个 https URL 上发布了代码。

Title: Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models

• Authors: Qin Liu, Chao Shang, Ling Liu, Nikolaos Pappas, Jie Ma, Neha Anna John, Srikanth Doss, Lluis Marquez, Miguel Ballesteros, Yassine Benajiba
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2410.09047
• Pdf URL: https://arxiv.org/pdf/2410.09047
• Copy Paste: [[2410.09047]] Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models(https://arxiv.org/abs/2410.09047)
• Keywords: language model, llm
• Abstract: The safety alignment ability of Vision-Language Models (VLMs) is prone to be degraded by the integration of the vision module compared to its LLM backbone. We investigate this phenomenon, dubbed as ''safety alignment degradation'' in this paper, and show that the challenge arises from the representation gap that emerges when introducing vision modality to VLMs. In particular, we show that the representations of multi-modal inputs shift away from that of text-only inputs which represent the distribution that the LLM backbone is optimized for. At the same time, the safety alignment capabilities, initially developed within the textual embedding space, do not successfully transfer to this new multi-modal representation space. To reduce safety alignment degradation, we introduce Cross-Modality Representation Manipulation (CMRM), an inference time representation intervention method for recovering the safety alignment ability that is inherent in the LLM backbone of VLMs, while simultaneously preserving the functional capabilities of VLMs. The empirical results show that our framework significantly recovers the alignment ability that is inherited from the LLM backbone with minimal impact on the fluency and linguistic capabilities of pre-trained VLMs even without additional training. Specifically, the unsafe rate of LLaVA-7B on multi-modal input can be reduced from 61.53% to as low as 3.15% with only inference-time intervention. WARNING: This paper contains examples of toxic or harmful language.
• 摘要：视觉语言模型 (VLM) 的安全对齐能力容易因视觉模块与其 LLM 主干的集成而降低。我们研究了这种现象，本文将其称为“安全对齐退化”，并表明挑战源于将视觉模态引入 VLM 时出现的表示差距。具体而言，我们表明多模态输入的表示偏离了纯文本输入的表示，后者代表了 LLM 主干优化的分布。同时，最初在文本嵌入空间中开发的安全对齐能力无法成功转移到这个新的多模态表示空间。为了减少安全对齐退化，我们引入了跨模态表示操纵 (CMRM)，这是一种推理时间表示干预方法，用于恢复 VLM 的 LLM 主干中固有的安全对齐能力，同时保留 VLM 的功能能力。实证结果表明，我们的框架显著恢复了从 LLM 主干继承的对齐能力，即使没有额外的训练，对预训练 VLM 的流畅性和语言能力的影响也很小。具体来说，仅通过推理时间干预，LLaVA-7B 对多模态输入的不安全率就可以从 61.53% 降低到 3.15%。警告：本文包含有毒或有害语言的例子。