2024-03-29

Title: Measuring Political Bias in Large Language Models: What Is Said and How It Is Said

• Authors: Yejin Bang, Delong Chen, Nayeon Lee, Pascale Fung
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.18932
• Pdf URL: https://arxiv.org/pdf/2403.18932
• Copy Paste: [[2403.18932]] Measuring Political Bias in Large Language Models: What Is Said and How It Is Said(https://arxiv.org/abs/2403.18932)
• Keywords: language model, llm
• Abstract: We propose to measure political bias in LLMs by analyzing both the content and style of their generated content regarding political issues. Existing benchmarks and measures focus on gender and racial biases. However, political bias exists in LLMs and can lead to polarization and other harms in downstream applications. In order to provide transparency to users, we advocate that there should be fine-grained and explainable measures of political biases generated by LLMs. Our proposed measure looks at different political issues such as reproductive rights and climate change, at both the content (the substance of the generation) and the style (the lexical polarity) of such bias. We measured the political bias in eleven open-sourced LLMs and showed that our proposed framework is easily scalable to other topics and is explainable.
• 摘要：我们建议通过分析法学硕士生成的有关政治问题的内容的内容和风格来衡量法学硕士的政治偏见。现有的基准和措施侧重于性别和种族偏见。然而，法学硕士存在政治偏见，可能导致下游申请中的两极分化和其他危害。为了向用户提供透明度，我们主张应该对法学硕士产生的政治偏见采取细粒度且可解释的措施。我们提出的措施着眼于不同的政治问题，例如生殖权利和气候变化，以及这种偏见的内容（一代人的实质内容）和风格（词汇极性）。我们测量了十一个开源法学硕士的政治偏见，并表明我们提出的框架可以轻松扩展到其他主题并且是可解释的。

Title: Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformers

• Authors: Laura Bergomi, Tommaso M. Buonocore, Paolo Antonazzo, Lorenzo Alberghi, Riccardo Bellazzi, Lorenzo Preda, Chandra Bortolotto, Enea Parimbelli
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.18938
• Pdf URL: https://arxiv.org/pdf/2403.18938
• Copy Paste: [[2403.18938]] Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformers(https://arxiv.org/abs/2403.18938)
• Keywords: language model, gpt, llm
• Abstract: BACKGROUND: Radiology reports are typically written in a free-text format, making clinical information difficult to extract and use. Recently the adoption of structured reporting (SR) has been recommended by various medical societies thanks to the advantages it offers, e.g. standardization, completeness and information retrieval. We propose a pipeline to extract information from free-text radiology reports, that fits with the items of the reference SR registry proposed by a national society of interventional and medical radiology, focusing on CT staging of patients with lymphoma. METHODS: Our work aims to leverage the potential of Natural Language Processing (NLP) and Transformer-based models to deal with automatic SR registry filling. With the availability of 174 radiology reports, we investigate a rule-free generative Question Answering approach based on a domain-specific version of T5 (IT5). Two strategies (batch-truncation and ex-post combination) are implemented to comply with the model's context length limitations. Performance is evaluated in terms of strict accuracy, F1, and format accuracy, and compared with the widely used GPT-3.5 Large Language Model. A 5-point Likert scale questionnaire is used to collect human-expert feedback on the similarity between medical annotations and generated answers. RESULTS: The combination of fine-tuning and batch splitting allows IT5 to achieve notable results; it performs on par with GPT-3.5 albeit its size being a thousand times smaller in terms of parameters. Human-based assessment scores show a high correlation (Spearman's correlation coefficients>0.88, p-values<0.001) with AI performance metrics (F1) and confirm the superior ability of LLMs (i.e., GPT-3.5, 175B of parameters) in generating plausible human-like statements.
• 摘要：背景：放射学报告通常以自由文本格式编写，使得临床信息难以提取和使用。最近，由于结构化报告 (SR) 所提供的优点，各个医学协会都推荐采用它，例如：标准化、完整性和信息检索。我们提出了一个从自由文本放射学报告中提取信息的管道，该管道与国家介入和医学放射学协会提出的参考 SR 注册表项相符，重点关注淋巴瘤患者的 CT 分期。方法：我们的工作旨在利用自然语言处理 (NLP) 和基于 Transformer 的模型的潜力来处理自动 SR 注册表填充问题。凭借 174 份放射学报告的可用性，我们研究了一种基于特定领域版本的 T5 (IT5) 的无规则生成问答方法。实施两种策略（批量截断和事后组合）以符合模型的上下文长度限制。性能从严格准确性、F1 和格式准确性方面进行评估，并与广泛使用的 GPT-3.5 大语言模型进行比较。使用 5 点李克特量表调查问卷来收集人类专家对医学注释和生成答案之间相似性的反馈。结果：微调和批量分割的结合使IT5取得了显着的效果；它的性能与 GPT-3.5 相当，尽管其尺寸在参数方面要小一千倍。基于人的评估分数显示出与 AI 性能指标 (F1) 的高度相关性（Spearman 相关系数>0.88，p 值<0.001），并证实了 LLM（即 GPT-3.5，参数 175B）在生成合理结果方面的卓越能力类似人类的陈述。

Title: A Survey on Large Language Models from Concept to Implementation

• Authors: Chen Wang, Jin Zhao, Jiaqi Gong
• Subjects: cs.CL, cs.AI, cs.IT, cs.LG
• Abstract URL: https://arxiv.org/abs/2403.18969
• Pdf URL: https://arxiv.org/pdf/2403.18969
• Copy Paste: [[2403.18969]] A Survey on Large Language Models from Concept to Implementation(https://arxiv.org/abs/2403.18969)
• Keywords: language model, gpt, llm, chat
• Abstract: Recent advancements in Large Language Models (LLMs), particularly those built on Transformer architectures, have significantly broadened the scope of natural language processing (NLP) applications, transcending their initial use in chatbot technology. This paper investigates the multifaceted applications of these models, with an emphasis on the GPT series. This exploration focuses on the transformative impact of artificial intelligence (AI) driven tools in revolutionizing traditional tasks like coding and problem-solving, while also paving new paths in research and development across diverse industries. From code interpretation and image captioning to facilitating the construction of interactive systems and advancing computational domains, Transformer models exemplify a synergy of deep learning, data analysis, and neural network design. This survey provides an in-depth look at the latest research in Transformer models, highlighting their versatility and the potential they hold for transforming diverse application sectors, thereby offering readers a comprehensive understanding of the current and future landscape of Transformer-based LLMs in practical applications.
• 摘要：大型语言模型 (LLM) 的最新进展，特别是基于 Transformer 架构构建的模型，显着拓宽了自然语言处理 (NLP) 应用程序的范围，超越了它们最初在聊天机器人技术中的应用。本文研究了这些模型的多方面应用，重点是 GPT 系列。这项探索的重点是人工智能 (AI) 驱动工具对编码和解决问题等传统任务的革命性影响，同时也为不同行业的研发铺平了新的道路。从代码解释和图像字幕到促进交互式系统的构建和推进计算领域，Transformer 模型体现了深度学习、数据分析和神经网络设计的协同作用。这项调查深入探讨了 Transformer 模型的最新研究，强调了它们的多功能性以及它们在改变不同应用领域的潜力，从而让读者全面了解基于 Transformer 的法学硕士在实际应用中的当前和未来前景。

Title: Conformal Intent Classification and Clarification for Fast and Accurate Intent Recognition

• Authors: Floris den Hengst, Ralf Wolter, Patrick Altmeyer, Arda Kaygan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.18973
• Pdf URL: https://arxiv.org/pdf/2403.18973
• Copy Paste: [[2403.18973]] Conformal Intent Classification and Clarification for Fast and Accurate Intent Recognition(https://arxiv.org/abs/2403.18973)
• Keywords: agent
• Abstract: We present Conformal Intent Classification and Clarification (CICC), a framework for fast and accurate intent classification for task-oriented dialogue systems. The framework turns heuristic uncertainty scores of any intent classifier into a clarification question that is guaranteed to contain the true intent at a pre-defined confidence level. By disambiguating between a small number of likely intents, the user query can be resolved quickly and accurately. Additionally, we propose to augment the framework for out-of-scope detection. In a comparative evaluation using seven intent recognition datasets we find that CICC generates small clarification questions and is capable of out-of-scope detection. CICC can help practitioners and researchers substantially in improving the user experience of dialogue agents with specific clarification questions.
• 摘要：我们提出了保形意图分类和澄清（CICC），这是一个面向任务的对话系统快速准确的意图分类的框架。该框架将任何意图分类器的启发式不确定性分数转变为澄清问题，保证在预定义的置信水平上包含真实意图。通过消除少量可能意图的歧义，可以快速准确地解决用户查询。此外，我们建议增强范围外检测的框架。在使用七个意图识别数据集的比较评估中，我们发现 CICC 会生成小的澄清问题，并且能够进行范围外检测。 CICC 可以帮助从业者和研究人员通过具体的澄清问题大幅改善对话代理的用户体验。

Title: A Novel Corpus of Annotated Medical Imaging Reports and Information Extraction Results Using BERT-based Language Models

• Authors: Namu Park, Kevin Lybarger, Giridhar Kaushik Ramachandran, Spencer Lewis, Aashka Damani, Ozlem Uzuner, Martin Gunn, Meliha Yetisgen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.18975
• Pdf URL: https://arxiv.org/pdf/2403.18975
• Copy Paste: [[2403.18975]] A Novel Corpus of Annotated Medical Imaging Reports and Information Extraction Results Using BERT-based Language Models(https://arxiv.org/abs/2403.18975)
• Keywords: language model
• Abstract: Medical imaging is critical to the diagnosis, surveillance, and treatment of many health conditions, including oncological, neurological, cardiovascular, and musculoskeletal disorders, among others. Radiologists interpret these complex, unstructured images and articulate their assessments through narrative reports that remain largely unstructured. This unstructured narrative must be converted into a structured semantic representation to facilitate secondary applications such as retrospective analyses or clinical decision support. Here, we introduce the Corpus of Annotated Medical Imaging Reports (CAMIR), which includes 609 annotated radiology reports from three imaging modality types: Computed Tomography, Magnetic Resonance Imaging, and Positron Emission Tomography-Computed Tomography. Reports were annotated using an event-based schema that captures clinical indications, lesions, and medical problems. Each event consists of a trigger and multiple arguments, and a majority of the argument types, including anatomy, normalize the spans to pre-defined concepts to facilitate secondary use. CAMIR uniquely combines a granular event structure and concept normalization. To extract CAMIR events, we explored two BERT (Bi-directional Encoder Representation from Transformers)-based architectures, including an existing architecture (mSpERT) that jointly extracts all event information and a multi-step approach (PL-Marker++) that we augmented for the CAMIR schema.
• 摘要：医学成像对于许多健康状况的诊断、监测和治疗至关重要，包括肿瘤、神经、心血管和肌肉骨骼疾病等。放射科医生解释这些复杂的、非结构化的图像，并通过基本上非结构化的叙述性报告阐明他们的评估。这种非结构化的叙述必须转换为结构化的语义表示，以促进二次应用，例如回顾性​​分析或临床决策支持。在这里，我们介绍注释医学影像报告语料库 (CAMIR)，其中包括来自三种成像模式类型的 609 份注释放射学报告：计算机断层扫描、磁共振成像和正电子发射断层扫描-计算机断层扫描。报告使用基于事件的模式进行注释，捕获临床适应症、病变和医疗问题。每个事件由一个触发器和多个参数组成，大多数参数类型（包括解剖学）将跨度标准化为预定义的概念，以方便二次使用。 CAMIR 独特地结合了精细的事件结构和概念规范化。为了提取 CAMIR 事件，我们探索了两种基于 BERT（来自 Transformers 的双向编码器表示）的架构，包括联合提取所有事件信息的现有架构 (mSpERT) 和我们增强的多步骤方法 (PL-Marker++) CAMIR 模式。

Title: "Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing

• Authors: Vipula Rawte, S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Prachi Priya, Aman Chadha, Amit P. Sheth, Amitava Das
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.18976
• Pdf URL: https://arxiv.org/pdf/2403.18976
• Copy Paste: [[2403.18976]] "Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing(https://arxiv.org/abs/2403.18976)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Hallucination has emerged as the most vulnerable aspect of contemporary Large Language Models (LLMs). In this paper, we introduce the Sorry, Come Again (SCA) prompting, aimed to avoid LLM hallucinations by enhancing comprehension through: (i) optimal paraphrasing and (ii) injecting [PAUSE] tokens to delay LLM generation. First, we provide an in-depth analysis of linguistic nuances: formality, readability, and concreteness of prompts for 21 LLMs, and elucidate how these nuances contribute to hallucinated generation. Prompts with lower readability, formality, or concreteness pose comprehension challenges for LLMs, similar to those faced by humans. In such scenarios, an LLM tends to speculate and generate content based on its imagination (associative memory) to fill these information gaps. Although these speculations may occasionally align with factual information, their accuracy is not assured, often resulting in hallucination. Recent studies reveal that an LLM often neglects the middle sections of extended prompts, a phenomenon termed as lost in the middle. While a specific paraphrase may suit one LLM, the same paraphrased version may elicit a different response from another LLM. Therefore, we propose an optimal paraphrasing technique to identify the most comprehensible paraphrase of a given prompt, evaluated using Integrated Gradient (and its variations) to guarantee that the LLM accurately processes all words. While reading lengthy sentences, humans often pause at various points to better comprehend the meaning read thus far. We have fine-tuned an LLM with injected [PAUSE] tokens, allowing the LLM to pause while reading lengthier prompts. This has brought several key contributions: (i) determining the optimal position to inject [PAUSE], (ii) determining the number of [PAUSE] tokens to be inserted, and (iii) introducing reverse proxy tuning to fine-tune the LLM for [PAUSE] insertion.
• 摘要：幻觉已成为当代大型语言模型（LLM）中最脆弱的方面。在本文中，我们介绍了“抱歉，再来一次”(SCA) 提示，旨在通过以下方式增强理解力，从而避免 LLM 幻觉：(i) 最佳释义和 (ii) 注入 [PAUSE] 标记以延迟 LLM 生成。首先，我们对语言上的细微差别进行了深入分析：21 名法学硕士的提示的形式性、可读性和具体性，并阐明这些细微差别如何导致幻觉的产生。可读性、形式性或具体性较低的提示给法学硕士带来了理解挑战，类似于人类所面临的挑战。在这种情况下，法学硕士倾向于根据其想象力（联想记忆）推测和生成内容，以填补这些信息空白。尽管这些推测有时可能与事实信息相符，但其准确性并不确定，常常会导致幻觉。最近的研究表明，法学硕士经常忽略扩展提示的中间部分，这种现象被称为迷失在中间。虽然特定的释义可能适合一个法学硕士，但相同的释义版本可能会引起另一个法学硕士的不同反应。因此，我们提出了一种最佳释义技术来识别给定提示的最容易理解的释义，并使用集成梯度（及其变体）进行评估，以保证法学硕士准确地处理所有单词。在阅读长句子时，人们经常会在不同的地方停顿，以更好地理解到目前为止所读到的含义。我们通过注入 [PAUSE] 标记对 LLM 进行了微调，允许 LLM 在阅读较长的提示时暂停。这带来了几个关键贡献：(i) 确定注入 [PAUSE] 的最佳位置，(ii) 确定要插入的 [PAUSE] 代币的数量，以及 (iii) 引入反向代理调整来微调 LLM [暂停]插入。

Title: Evaluating Large Language Models for Health-Related Text Classification Tasks with Public Social Media Data

• Authors: Yuting Guo, Anthony Ovadje, Mohammed Ali Al-Garadi, Abeed Sarker
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2403.19031
• Pdf URL: https://arxiv.org/pdf/2403.19031
• Copy Paste: [[2403.19031]] Evaluating Large Language Models for Health-Related Text Classification Tasks with Public Social Media Data(https://arxiv.org/abs/2403.19031)
• Keywords: language model, gpt, llm
• Abstract: Large language models (LLMs) have demonstrated remarkable success in NLP tasks. However, there is a paucity of studies that attempt to evaluate their performances on social media-based health-related natural language processing tasks, which have traditionally been difficult to achieve high scores in. We benchmarked one supervised classic machine learning model based on Support Vector Machines (SVMs), three supervised pretrained language models (PLMs) based on RoBERTa, BERTweet, and SocBERT, and two LLM based classifiers (GPT3.5 and GPT4), across 6 text classification tasks. We developed three approaches for leveraging LLMs for text classification: employing LLMs as zero-shot classifiers, us-ing LLMs as annotators to annotate training data for supervised classifiers, and utilizing LLMs with few-shot examples for augmentation of manually annotated data. Our comprehensive experiments demonstrate that employ-ing data augmentation using LLMs (GPT-4) with relatively small human-annotated data to train lightweight supervised classification models achieves superior results compared to training with human-annotated data alone. Supervised learners also outperform GPT-4 and GPT-3.5 in zero-shot settings. By leveraging this data augmentation strategy, we can harness the power of LLMs to develop smaller, more effective domain-specific NLP models. LLM-annotated data without human guidance for training light-weight supervised classification models is an ineffective strategy. However, LLM, as a zero-shot classifier, shows promise in excluding false negatives and potentially reducing the human effort required for data annotation. Future investigations are imperative to explore optimal training data sizes and the optimal amounts of augmented data.
• 摘要：大型语言模型 (LLM) 在 NLP 任务中取得了显着的成功。然而，很少有研究试图评估他们在基于社交媒体的健康相关自然语言处理任务上的表现，而这些任务传统上很难取得高分。我们对一种基于支持向量的有监督经典机器学习模型进行了基准测试机器 (SVM)、三个基于 RoBERTa、BERTweet 和 SocBERT 的有监督预训练语言模型 (PLM)，以及两个基于 LLM 的分类器（GPT3.5 和 GPT4），涵盖 6 个文本分类任务。我们开发了三种利用 LLM 进行文本分类的方法：使用 LLM 作为零样本分类器，使用 LLM 作为注释器来注释监督分类器的训练数据，以及利用 LLM 和少量样本来增强手动注释数据。我们的综合实验表明，与单独使用人工注释数据进行训练相比，使用 LLM (GPT-4) 和相对较小的人工注释数据进行数据增强来训练轻量级监督分类模型可以获得更好的结果。监督学习器在零样本设置中也优于 GPT-4 和 GPT-3.5。通过利用这种数据增强策略，我们可以利用法学硕士的力量来开发更小、更有效的特定领域 NLP 模型。在没有人工指导的情况下使用法学硕士注释数据来训练轻量级监督分类模型是一种无效的策略。然而，LLM 作为一种零样本分类器，在排除漏报并可能减少数据注释所需的人力方面表现出了希望。未来的研究必须探索最佳的训练数据大小和最佳的增强数据量。

Title: CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems

• Authors: Amin Abolghasemi, Zhaochun Ren, Arian Askari, Mohammad Aliannejadi, Maarten de Rijke, Suzan Verberne
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19056
• Pdf URL: https://arxiv.org/pdf/2403.19056
• Copy Paste: [[2403.19056]] CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems(https://arxiv.org/abs/2403.19056)
• Keywords: language model, llm
• Abstract: An important unexplored aspect in previous work on user satisfaction estimation for Task-Oriented Dialogue (TOD) systems is their evaluation in terms of robustness for the identification of user dissatisfaction: current benchmarks for user satisfaction estimation in TOD systems are highly skewed towards dialogues for which the user is satisfied. The effect of having a more balanced set of satisfaction labels on performance is unknown. However, balancing the data with more dissatisfactory dialogue samples requires further data collection and human annotation, which is costly and time-consuming. In this work, we leverage large language models (LLMs) and unlock their ability to generate satisfaction-aware counterfactual dialogues to augment the set of original dialogues of a test collection. We gather human annotations to ensure the reliability of the generated samples. We evaluate two open-source LLMs as user satisfaction estimators on our augmented collection against state-of-the-art fine-tuned models. Our experiments show that when used as few-shot user satisfaction estimators, open-source LLMs show higher robustness to the increase in the number of dissatisfaction labels in the test collection than the fine-tuned state-of-the-art models. Our results shed light on the need for data augmentation approaches for user satisfaction estimation in TOD systems. We release our aligned counterfactual dialogues, which are curated by human annotation, to facilitate further research on this topic.
• 摘要：之前关于面向任务的对话（TOD）系统的用户满意度估计的一个重要的未探索的方面是它们在识别用户不满意方面的稳健性方面的评估：当前 TOD 系统中用户满意度估计的基准高度偏向于以下对话：用户满意。拥有一套更平衡的满意度标签对绩效的影响尚不清楚。然而，用更多不满意的对话样本来平衡数据需要进一步的数据收集和人工注释，这是昂贵且耗时的。在这项工作中，我们利用大型语言模型 (LLM) 并解锁其生成满意度感知反事实对话的能力，以增强测试集合的原始对话集。我们收集人工注释以确保生成样本的可靠性。我们根据最先进的微调模型评估了两个开源法学硕士作为我们的增强集合的用户满意度评估器。我们的实验表明，当用作小样本用户满意度估计器时，开源法学硕士对测试集合中不​​满意标签数量的增加表现出比经过微调的最先进模型更高的鲁棒性。我们的结果揭示了 TOD 系统中用户满意度评估的数据增强方法的需求。我们发布了由人工注释整理的一致的反事实对话，以促进对该主题的进一步研究。

Title: Learning From Correctness Without Prompting Makes LLM Efficient Reasoner

• Authors: Yuxuan Yao, Han Wu, Zhijiang Guo, Biyan Zhou, Jiahui Gao, Sichun Luo, Hanxu Hou, Xiaojin Fu, Linqi Song
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19094
• Pdf URL: https://arxiv.org/pdf/2403.19094
• Copy Paste: [[2403.19094]] Learning From Correctness Without Prompting Makes LLM Efficient Reasoner(https://arxiv.org/abs/2403.19094)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Large language models (LLMs) have demonstrated outstanding performance across various tasks, yet they still exhibit limitations such as hallucination, unfaithful reasoning, and toxic content. One potential approach to mitigate these issues is learning from human or external feedback (e.g. tools). In this paper, we introduce an intrinsic self-correct reasoning framework for LLMs that eliminates the need for human feedback, external tools, and handcraft prompts. The proposed framework, based on a multi-step reasoning paradigm \textbf{Le}arning from \textbf{Co}rrectness (\textsc{LeCo}), improves reasoning performance without needing to learn from errors. This paradigm prioritizes learning from correct reasoning steps, and a unique method to measure confidence for each reasoning step based on generation logits. Experimental results across various multi-step reasoning tasks demonstrate the effectiveness of the framework in improving reasoning performance with reduced token consumption.
• 摘要：大型语言模型（LLM）在各种任务中表现出了出色的性能，但它们仍然表现出幻觉、不忠实推理和有毒内容等局限性。缓解这些问题的一种潜在方法是从人类或外部反馈（例如工具）中学习。在本文中，我们为法学硕士介绍了一种内在的自我正确推理框架，消除了对人类反馈、外部工具和手工提示的需要。所提出的框架基于从\textbf{Co}正确性（\textsc{LeCo}）学习的多步推理范式\textbf{Le}，无需从错误中学习即可提高推理性能。该范例优先考虑从正确的推理步骤中学习，以及基于生成逻辑来衡量每个推理步骤的置信度的独特方法。各种多步推理任务的实验结果证明了该框架在提高推理性能和减少令牌消耗方面的有效性。

Title: FACTOID: FACtual enTailment fOr hallucInation Detection

• Authors: Vipula Rawte, S.M Towhidul Islam Tonmoy, Krishnav Rajbangshi, Shravani Nag, Aman Chadha, Amit P. Sheth, Amitava Das
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19113
• Pdf URL: https://arxiv.org/pdf/2403.19113
• Copy Paste: [[2403.19113]] FACTOID: FACtual enTailment fOr hallucInation Detection(https://arxiv.org/abs/2403.19113)
• Keywords: language model, gpt, llm, hallucination, prompt, retrieval augmented generation
• Abstract: The widespread adoption of Large Language Models (LLMs) has facilitated numerous benefits. However, hallucination is a significant concern. In response, Retrieval Augmented Generation (RAG) has emerged as a highly promising paradigm to improve LLM outputs by grounding them in factual information. RAG relies on textual entailment (TE) or similar methods to check if the text produced by LLMs is supported or contradicted, compared to retrieved documents. This paper argues that conventional TE methods are inadequate for spotting hallucinations in content generated by LLMs. For instance, consider a prompt about the 'USA's stance on the Ukraine war''. The AI-generated text states, ...U.S. President Barack Obama says the U.S. will not put troops in Ukraine...'' However, during the war the U.S. president is Joe Biden which contradicts factual reality. Moreover, current TE systems are unable to accurately annotate the given text and identify the exact portion that is contradicted. To address this, we introduces a new type of TE called ``Factual Entailment (FE).'', aims to detect factual inaccuracies in content generated by LLMs while also highlighting the specific text segment that contradicts reality. We present FACTOID (FACTual enTAILment for hallucInation Detection), a benchmark dataset for FE. We propose a multi-task learning (MTL) framework for FE, incorporating state-of-the-art (SoTA) long text embeddings such as e5-mistral-7b-instruct, along with GPT-3, SpanBERT, and RoFormer. The proposed MTL architecture for FE achieves an avg. 40\% improvement in accuracy on the FACTOID benchmark compared to SoTA TE methods. As FE automatically detects hallucinations, we assessed 15 modern LLMs and ranked them using our proposed Auto Hallucination Vulnerability Index (HVI_auto). This index quantifies and offers a comparative scale to evaluate and rank LLMs according to their hallucinations.
• 摘要：大型语言模型 (LLM) 的广泛采用带来了许多好处。然而，幻觉是一个重大问题。作为回应，检索增强生成（RAG）已成为一种非常有前途的范式，通过以事实信息为基础来提高法学硕士的输出。 RAG 依靠文本蕴含 (TE) 或类似方法来检查法学硕士生成的文本与检索到的文档相比是否得到支持或矛盾。本文认为，传统的 TE 方法不足以发现法学硕士生成的内容中的幻觉。例如，考虑有关“美国对乌克兰战争的立场”的提示。人工智能生成的文本指出，……美国奥巴马总统表示美国不会在乌克兰驻军……”然而，战争期间的美国总统是乔·拜登，这与事实相矛盾。此外，当前的 TE 系统无法准确注释给定文本并识别出矛盾的确切部分。为了解决这个问题，我们引入了一种名为“事实蕴涵（FE）”的新型 TE，旨在检测法学硕士生成的内容中的事实不准确之处，同时突出显示与现实相矛盾的特定文本片段。我们提出了 FACTOID（幻觉检测的 FACTual enTAILment），这是 FE 的基准数据集。我们提出了一个 FE 多任务学习 (MTL) 框架，结合了最先进的 (SoTA) 长文本嵌入，例如 e5-mistral-7b-instruct，以及 GPT-3、SpanBERT 和 RoFormer。所提出的 FE MTL 架构实现了平均。与 SoTA TE 方法相比，FACTOID 基准的准确性提高了 40%。由于 FE 自动检测幻觉，我们评估了 15 个现代法学硕士，并使用我们提出的自动幻觉脆弱性指数 (HVI_auto) 对它们进行排名。该指数量化并提供了一个比较尺度，以根据法学硕士的幻觉对其进行评估和排名。

Title: MFORT-QA: Multi-hop Few-shot Open Rich Table Question Answering

• Authors: Che Guan, Mengyu Huang, Peng Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19116
• Pdf URL: https://arxiv.org/pdf/2403.19116
• Copy Paste: [[2403.19116]] MFORT-QA: Multi-hop Few-shot Open Rich Table Question Answering(https://arxiv.org/abs/2403.19116)
• Keywords: language model, gpt, llm, prompt, chat, retrieval-augmented generation, chain-of-thought
• Abstract: In today's fast-paced industry, professionals face the challenge of summarizing a large number of documents and extracting vital information from them on a daily basis. These metrics are frequently hidden away in tables and/or their nested hyperlinks. To address this challenge, the approach of Table Question Answering (QA) has been developed to extract the relevant information. However, traditional Table QA training tasks that provide a table and an answer(s) from a gold cell coordinate(s) for a question may not always ensure extracting the accurate answer(s). Recent advancements in Large Language Models (LLMs) have opened up new possibilities for extracting information from tabular data using prompts. In this paper, we introduce the Multi-hop Few-shot Open Rich Table QA (MFORT-QA) approach, which consists of two major steps. The first step involves Few-Shot Learning (FSL), where relevant tables and associated contexts of hyperlinks are retrieved based on a given question. The retrieved content is then used to construct few-shot prompts as inputs to an LLM, such as ChatGPT. To tackle the challenge of answering complex questions, the second step leverages Chain-of-thought (CoT) prompting to decompose the complex question into a sequential chain of questions and reasoning thoughts in a multi-hop manner. Retrieval-Augmented Generation (RAG) enhances this process by retrieving relevant tables and contexts of hyperlinks that are relevant to the resulting reasoning thoughts and questions. These additional contexts are then used to supplement the prompt used in the first step, resulting in more accurate answers from an LLM. Empirical results from OTT-QA demonstrate that our abstractive QA approach significantly improves the accuracy of extractive Table QA methods.
• 摘要：在当今快节奏的行业中，专业人士每天面临着总结大量文档并从中提取重要信息的挑战。这些指标经常隐藏在表格和/或其嵌套超链接中。为了应对这一挑战，开发了表格问答（QA）方法来提取相关信息。然而，为问题提供表格和黄金单元坐标答案的传统表格 QA 训练任务可能并不总是确保提取准确的答案。大型语言模型 (LLM) 的最新进展为使用提示从表格数据中提取信息开辟了新的可能性。在本文中，我们介绍了多跳少样本开放式丰富表 QA (MFORT-QA) 方法，该方法由两个主要步骤组成。第一步涉及少样本学习 (FSL)，其中根据给定问题检索相关表格和超链接的关联上下文。然后，检索到的内容用于构建少量提示，作为 LLM（例如 ChatGPT）的输入。为了应对回答复杂问题的挑战，第二步利用思想链 (CoT) 提示，以多跳方式将复杂问题分解为一系列问题和推理思想的顺序链。检索增强生成 (RAG) 通过检索与所得推理想法和问题相关的相关表格和超链接上下文来增强此过程。然后，这些额外的上下文将用于补充第一步中使用的提示，从而使法学硕士得到更准确的答案。 OTT-QA 的经验结果表明，我们的抽象 QA 方法显着提高了提取表 QA 方法的准确性。

Title: Code Comparison Tuning for Code Large Language Models

• Authors: Yufan Jiang, Qiaozhi He, Xiaomin Zhuang, Zhihua Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19121
• Pdf URL: https://arxiv.org/pdf/2403.19121
• Copy Paste: [[2403.19121]] Code Comparison Tuning for Code Large Language Models(https://arxiv.org/abs/2403.19121)
• Keywords: language model, llm
• Abstract: We present Code Comparison Tuning (CCT), a simple and effective tuning method for code large language models (Code LLMs) to better handle subtle code errors. Specifically, we integrate the concept of comparison into instruction tuning, both at the token and sequence levels, enabling the model to discern even the slightest deviations in code. To compare the original code with an erroneous version containing manually added code errors, we use token-level preference loss for detailed token-level comparisons. Additionally, we combine code segments to create a new instruction tuning sample for sequence-level comparisons, enhancing the model's bug-fixing capability. Experimental results on the HumanEvalFix benchmark show that CCT surpasses instruction tuning in pass@1 scores by up to 4 points across diverse code LLMs, and extensive analysis demonstrates the effectiveness of our method.
• 摘要：我们提出了代码比较调优 (CCT)，这是一种简单有效的代码大语言模型 (Code LLM) 调优方法，可以更好地处理细微的代码错误。具体来说，我们在标记和序列级别将比较的概念集成到指令调整中，使模型能够识别代码中最轻微的偏差。为了将原始代码与包含手动添加代码错误的错误版本进行比较，我们使用令牌级别偏好损失来进行详细的令牌级别比较。此外，我们结合代码段来创建新的指令调优样本，用于序列级比较，增强模型的错误修复能力。 HumanEvalFix 基准测试的实验结果表明，在不同的代码 LLM 中，CCT 在 pass@1 分数中超过了指令调整高达 4 分，并且广泛的分析证明了我们方法的有效性。

Title: Compressing Large Language Models by Streamlining the Unimportant Layer

• Authors: Xiaodong Chen, Yuxuan Hu, Jing Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19135
• Pdf URL: https://arxiv.org/pdf/2403.19135
• Copy Paste: [[2403.19135]] Compressing Large Language Models by Streamlining the Unimportant Layer(https://arxiv.org/abs/2403.19135)
• Keywords: language model, llm
• Abstract: Large language models (LLM) have been extensively applied in various natural language tasks and domains, but their applicability is constrained by the large number of parameters of the models. Consequently, there is an increasing emphasis on compact models that exhibit high performance. In this study, we observe that different layers in LLM have varying degrees of perturbation on the hidden states, which allows us to identify less important layers. Based on this phenomenon, we propose LLM-Streamline, which consists of two parts: layer pruning, where we remove a set of consecutive layers with the lowest importance in the model according to the target sparsity; and layer replacement, where we train a lightweight model to substitute the pruned layers, thereby mitigating the performance degradation caused by pruning. In our experiments, we utilize structures such as a multi-layer perceptron (MLP) and a transformer layer as lightweight models and ultimately demonstrate that a single MLP can effectively fit the pruned layers. Comprehensive experiments show that our proposed method, LLM-Streamline, outperforms previous state-of-the-art (SOTA) model pruning methods.
• 摘要：大型语言模型（LLM）已广泛应用于各种自然语言任务和领域，但其适用性受到模型参数数量过多的限制。因此，人们越来越重视具有高性能的紧凑型模型。在这项研究中，我们观察到 LLM 中的不同层对隐藏状态有不同程度的扰动，这使我们能够识别不太重要的层。基于这种现象，我们提出了LLM-Streamline，它由两部分组成：层剪枝，根据目标稀疏度删除模型中重要性最低的一组连续层；和层替换，我们训练一个轻量级模型来替换修剪后的层，从而减轻修剪引起的性能下降。在我们的实验中，我们利用多层感知器（MLP）和变换层等结构作为轻量级模型，并最终证明单个 MLP 可以有效地拟合剪枝层。综合实验表明，我们提出的方法 LLM-Streamline 优于以前最先进的 (SOTA) 模型修剪方法。

Title: STaR-GATE: Teaching Language Models to Ask Clarifying Questions

• Authors: Chinmaya Andukuri, Jan-Philipp Fränken, Tobias Gerstenberg, Noah D. Goodman
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19154
• Pdf URL: https://arxiv.org/pdf/2403.19154
• Copy Paste: [[2403.19154]] STaR-GATE: Teaching Language Models to Ask Clarifying Questions(https://arxiv.org/abs/2403.19154)
• Keywords: language model, prompt
• Abstract: When prompting language models to complete a task, users often leave important aspects unsaid. While asking questions could resolve this ambiguity \citep[GATE;][]{li2023eliciting}, models often struggle to ask good questions. We explore a language model's ability to self-improve \citep[STaR;][]{zelikman2022star} by rewarding the model for generating useful questions -- a simple method we dub STaR-GATE. We generate a synthetic dataset of 25,500 unique persona-task prompts to simulate conversations between a pretrained language model -- the \texttt{Questioner} -- and a \texttt{Roleplayer} whose preferences are unknown to the \texttt{Questioner}. By asking questions, the \texttt{Questioner} elicits preferences from the \texttt{Roleplayer}. The \texttt{Questioner} is iteratively finetuned on questions that increase the probability of high-quality responses to the task, which are generated by an \texttt{Oracle} with access to the \texttt{Roleplayer}'s latent preferences. After two iterations of self-improvement, the \texttt{Questioner} asks better questions, allowing it to generate responses that are preferred over responses from the initial model on \highlightpink{\textbf{72\%}} of tasks. Our results indicate that teaching a language model to ask better questions leads to better personalized responses.
• 摘要：当提示语言模型完成任务时，用户通常会忽略重要的方面。虽然提出问题可以解决这种歧义 \citep[GATE;][]{li2023eliciting}，但模型通常很难提出好的问题。我们通过奖励模型生成有用的问题来探索语言模型自我改进的能力，我们将这种简单方法称为 STaR-GATE。我们生成了一个包含 25,500 个独特角色任务提示的综合数据集，以模拟预训练语言模型（\texttt{提问者}）和 \texttt{提问者} 未知其偏好的 \texttt{Roleplayer} 之间的对话。通过提问，\texttt{提问者} 引出\texttt{角色玩家} 的偏好。 \texttt{Questioner} 对问题进行迭代微调，以提高对任务做出高质量响应的可能性，这些问题是由能够访问 \texttt{Roleplayer} 潜在偏好的 \texttt{Oracle} 生成的。经过两次自我改进迭代后，\texttt{Questioner} 会提出更好的问题，从而使其能够在 \highlightpink{\textbf{72\%}} 任务上生成优于初始模型响应的响应。我们的结果表明，教授语言模型提出更好的问题会带来更好的个性化回答。

Title: Disentangling Length from Quality in Direct Preference Optimization

• Authors: Ryan Park, Rafael Rafailov, Stefano Ermon, Chelsea Finn
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2403.19159
• Pdf URL: https://arxiv.org/pdf/2403.19159
• Copy Paste: [[2403.19159]] Disentangling Length from Quality in Direct Preference Optimization(https://arxiv.org/abs/2403.19159)
• Keywords: language model, gpt
• Abstract: Reinforcement Learning from Human Feedback (RLHF) has been a crucial component in the recent success of Large Language Models. However, RLHF is know to exploit biases in human preferences, such as verbosity. A well-formatted and eloquent answer is often more highly rated by users, even when it is less helpful and objective. A number of approaches have been developed to control those biases in the classical RLHF literature, but the problem remains relatively under-explored for Direct Alignment Algorithms such as Direct Preference Optimization (DPO). Unlike classical RLHF, DPO does not train a separate reward model or use reinforcement learning directly, so previous approaches developed to control verbosity cannot be directly applied to this setting. Our work makes several contributions. For the first time, we study the length problem in the DPO setting, showing significant exploitation in DPO and linking it to out-of-distribution bootstrapping. We then develop a principled but simple regularization strategy that prevents length exploitation, while still maintaining improvements in model quality. We demonstrate these effects across datasets on summarization and dialogue, where we achieve up to 20\% improvement in win rates when controlling for length, despite the GPT4 judge's well-known verbosity bias.
• 摘要：来自人类反馈的强化学习（RLHF）一直是大型语言模型最近取得成功的关键组成部分。然而，众所周知，RLHF 会利用人类偏好的偏差，例如冗长程度。格式良好且雄辩的答案通常会受到用户的更高评价，即使它不太有帮助和客观。经典 RLHF 文献中已经开发了许多方法来控制这些偏差，但对于直接偏好优化 (DPO) 等直接对齐算法，该问题仍然相对未得到充分研究。与经典的 RLHF 不同，DPO 不训练单独的奖励模型或直接使用强化学习，因此之前开发的控制冗长的方法不能直接应用于此设置。我们的工作做出了多项贡献。我们首次研究了 DPO 设置中的长度问题，显示了 DPO 中的重大利用，并将其与分布外引导联系起来。然后，我们开发了一种有原则但简单的正则化策略，可以防止长度利用，同时仍然保持模型质量的改进。我们在摘要和对话的数据集上展示了这些影响，尽管 GPT4 法官存在众所周知的冗长偏见，但在控制长度时，我们的获胜率提高了 20%。

Title: Improving Vietnamese-English Medical Machine Translation

• Authors: Nhu Vo, Dat Quoc Nguyen, Dung D. Le, Massimo Piccardi, Wray Buntine
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19161
• Pdf URL: https://arxiv.org/pdf/2403.19161
• Copy Paste: [[2403.19161]] Improving Vietnamese-English Medical Machine Translation(https://arxiv.org/abs/2403.19161)
• Keywords: gpt, chat
• Abstract: Machine translation for Vietnamese-English in the medical domain is still an under-explored research area. In this paper, we introduce MedEV -- a high-quality Vietnamese-English parallel dataset constructed specifically for the medical domain, comprising approximately 360K sentence pairs. We conduct extensive experiments comparing Google Translate, ChatGPT (gpt-3.5-turbo), state-of-the-art Vietnamese-English neural machine translation models and pre-trained bilingual/multilingual sequence-to-sequence models on our new MedEV dataset. Experimental results show that the best performance is achieved by fine-tuning "vinai-translate" for each translation direction. We publicly release our dataset to promote further research.
• 摘要：医学领域的越南语-英语机器翻译仍然是一个尚未开发的研究领域。在本文中，我们介绍了 MedEV——一个专为医学领域构建的高质量越南语-英语并行数据集，包含大约 36 万个句子对。我们在新的 MedEV 数据集上进行了大量实验，比较了 Google Translate、ChatGPT (gpt-3.5-turbo)、最先进的越南语-英语神经机器翻译模型和预训练的双语/多语言序列到序列模型。实验结果表明，通过对每个翻译方向微调“vinai-translate”可以获得最佳性能。我们公开发布我们的数据集以促进进一步的研究。

Title: Mitigating Misleading Chain-of-Thought Reasoning with Selective Filtering

• Authors: Yexin Wu, Zhuosheng Zhang, Hai Zhao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19167
• Pdf URL: https://arxiv.org/pdf/2403.19167
• Copy Paste: [[2403.19167]] Mitigating Misleading Chain-of-Thought Reasoning with Selective Filtering(https://arxiv.org/abs/2403.19167)
• Keywords: language model, chain-of-thought
• Abstract: Large language models have manifested remarkable capabilities by leveraging chain-of-thought (CoT) reasoning techniques to solve intricate questions through step-by-step reasoning chains. Despite its success, the efficacy of such reasoning is inherently contingent upon the quality of CoT. However, flawless CoT reasoning cannot be guaranteed due to the presence of indecomposable questions and the potential for erroneous reasoning chains, particularly in the case of small-scale language models. To tackle this challenge, we propose a novel approach called the selective filtering reasoner (SelF-Reasoner) that assesses the entailment relationship between the question and the candidate reasoning chain. Then, we proceed with CoT reasoning when the reasoning chain demonstrates confidence; otherwise, we opt to predict the answer directly. SelF-Reasoner improves the fine-tuned T5 baseline consistently over the ScienceQA, ECQA, and LastLetter tasks. Code is available at \texttt{https://github.com/LibroWu/SelF-Reasoner}.
• 摘要：大型语言模型通过利用思想链（CoT）推理技术通过逐步推理链解决复杂的问题，表现出了非凡的能力。尽管取得了成功，但这种推理的有效性本质上取决于 CoT 的质量。然而，由于存在不可分解的问题和可能出现错误推理链，尤其是在小规模语言模型的情况下，无法保证完美的 CoT 推理。为了应对这一挑战，我们提出了一种称为选择性过滤推理器（SeF-Reasoner）的新颖方法，用于评估问题与候选推理链之间的蕴涵关系。然后，当推理链表现出置信度时，我们继续进行 CoT 推理；否则，我们选择直接预测答案。 Self-Reasoner 相对于 ScienceQA、ECQA 和 LastLetter 任务，一致地改进了微调的 T5 基线。代码可在 \texttt{https://github.com/LibroWu/SelF-Reasoner} 获取。

Title: MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs

• Authors: Xianhao Yu, Jiaqi Fu, Renjia Deng, Wenjuan Han
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19267
• Pdf URL: https://arxiv.org/pdf/2403.19267
• Copy Paste: [[2403.19267]] MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs(https://arxiv.org/abs/2403.19267)
• Keywords: agent
• Abstract: Conventional multi-agent simulators often assume perfect information and limitless capabilities, hindering the ecological validity of social interactions. We propose a multi-agent Minecraft simulator, MineLand, that bridges this gap by introducing limited multimodal senses and physical needs. Our simulator supports up to 48 agents with limited visual, auditory, and environmental awareness, forcing them to actively communicate and collaborate to fulfill physical needs like food and resources. This fosters dynamic and valid multi-agent interactions. We further introduce an AI agent framework, Alex, inspired by multitasking theory, enabling agents to handle intricate coordination and scheduling. Our experiments demonstrate that the simulator, the corresponding benchmark, and the AI agent framework contribute to more ecological and nuanced collective behavior. The source code of MineLand and Alex is openly available at https://github.com/cocacola-lab/MineLand.
• 摘要：传统的多智能体模拟器通常假设完美的信息和无限的能力，阻碍了社会互动的生态有效性。我们提出了一个多智能体 Minecraft 模拟器 MineLand，它通过引入有限的多模态感官和物理需求来弥补这一差距。我们的模拟器支持多达 48 名视觉、听觉和环境意识有限的特工，迫使他们积极沟通和协作，以满足食物和资源等身体需求。这促进了动态且有效的多代理交互。我们进一步引入了人工智能代理框架 Alex，其灵感来自多任务理论，使代理能够处理复杂的协调和调度。我们的实验表明，模拟器、相应的基准测试和人工智能代理框架有助于更加生态和细致的集体行为。 MineLand 和 Alex 的源代码可在 https://github.com/cocacola-lab/MineLand 上公开获取。

Title: sDPO: Don't Use Your Data All at Once

• Authors: Dahyun Kim, Yungi Kim, Wonho Song, Hyeonwoo Kim, Yunsu Kim, Sanghoon Kim, Chanjun Park
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19270
• Pdf URL: https://arxiv.org/pdf/2403.19270
• Copy Paste: [[2403.19270]] sDPO: Don't Use Your Data All at Once(https://arxiv.org/abs/2403.19270)
• Keywords: language model, llm
• Abstract: As development of large language models (LLM) progresses, aligning them with human preferences has become increasingly important. We propose stepwise DPO (sDPO), an extension of the recently popularized direct preference optimization (DPO) for alignment tuning. This approach involves dividing the available preference datasets and utilizing them in a stepwise manner, rather than employing it all at once. We demonstrate that this method facilitates the use of more precisely aligned reference models within the DPO training framework. Furthermore, sDPO trains the final model to be more performant, even outperforming other popular LLMs with more parameters.
• 摘要：随着大型语言模型（LLM）的发展，使其与人类偏好保持一致变得越来越重要。我们提出了逐步 DPO (sDPO)，这是最近流行的用于对齐调整的直接偏好优化 (DPO) 的扩展。这种方法涉及划分可用的偏好数据集并逐步使用它们，而不是一次性使用它们。我们证明，这种方法有助于在 DPO 培训框架内使用更精确对齐的参考模型。此外，sDPO 训练的最终模型性能更高，甚至优于其他具有更多参数的流行法学硕士。

Title: Knowledge Boundary and Persona Dynamic Shape A Better Social Media Agent

• Authors: Junkai Zhou, Liang Pang, Ya Jing, Jia Gu, Huawei Shen, Xueqi Cheng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19275
• Pdf URL: https://arxiv.org/pdf/2403.19275
• Copy Paste: [[2403.19275]] Knowledge Boundary and Persona Dynamic Shape A Better Social Media Agent(https://arxiv.org/abs/2403.19275)
• Keywords: agent
• Abstract: Constructing personalized and anthropomorphic agents holds significant importance in the simulation of social networks. However, there are still two key problems in existing works: the agent possesses world knowledge that does not belong to its personas, and it cannot eliminate the interference of diverse persona information on current actions, which reduces the personalization and anthropomorphism of the agent. To solve the above problems, we construct the social media agent based on personalized knowledge and dynamic persona information. For personalized knowledge, we add external knowledge sources and match them with the persona information of agents, thereby giving the agent personalized world knowledge. For dynamic persona information, we use current action information to internally retrieve the persona information of the agent, thereby reducing the interference of diverse persona information on the current action. To make the agent suitable for social media, we design five basic modules for it: persona, planning, action, memory and reflection. To provide an interaction and verification environment for the agent, we build a social media simulation sandbox. In the experimental verification, automatic and human evaluations demonstrated the effectiveness of the agent we constructed.
• 摘要：构建个性化和拟人化的代理在社交网络的模拟中具有重要意义。然而，现有作品仍然存在两个关键问题：智能体拥有不属于其角色的世界知识，并且无法消除多样化的角色信息对当前行为的干扰，从而降低了智能体的个性化和拟人化。为了解决上述问题，我们构建了基于个性化知识和动态人物信息的社交媒体代理。对于个性化知识，我们添加外部知识源并将其与智能体的角色信息相匹配，从而赋予智能体个性化的世界知识。对于动态角色信息，我们使用当前动作信息来内部检索代理的角色信息，从而减少不同角色信息对当前动作的干扰。为了使智能体适合社交媒体，我们为其设计了五个基本模块：角色、计划、行动、记忆和反思。为了给代理提供交互和验证环境，我们构建了一个社交媒体模拟沙箱。在实验验证中，自动评估和人工评估证明了我们构建的代理的有效性。

Title: Fine-Tuning Language Models with Reward Learning on Policy

• Authors: Hao Lang, Fei Huang, Yongbin Li
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2403.19279
• Pdf URL: https://arxiv.org/pdf/2403.19279
• Copy Paste: [[2403.19279]] Fine-Tuning Language Models with Reward Learning on Policy(https://arxiv.org/abs/2403.19279)
• Keywords: language model, llm
• Abstract: Reinforcement learning from human feedback (RLHF) has emerged as an effective approach to aligning large language models (LLMs) to human preferences. RLHF contains three steps, i.e., human preference collecting, reward learning, and policy optimization, which are usually performed serially. Despite its popularity, however, (fixed) reward models may suffer from inaccurate off-distribution, since policy optimization continuously shifts LLMs' data distribution. Repeatedly collecting new preference data from the latest LLMs may alleviate this issue, which unfortunately makes the resulting system more complicated and difficult to optimize. In this paper, we propose reward learning on policy (RLP), an unsupervised framework that refines a reward model using policy samples to keep it on-distribution. Specifically, an unsupervised multi-view learning method is introduced to learn robust representations of policy samples. Meanwhile, a synthetic preference generation approach is developed to simulate high-quality preference data with policy outputs. Extensive experiments on three benchmark datasets show that RLP consistently outperforms the state-of-the-art. Our code is available at \url{https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/rlp}.
• 摘要：基于人类反馈的强化学习 (RLHF) 已成为使大型语言模型 (LLM) 与人类偏好保持一致的有效方法。 RLHF 包含三个步骤，即人类偏好收集、奖励学习和策略优化，通常是串行执行的。然而，尽管它很受欢迎，但（固定）奖励模型可能会受到不准确的偏离分布的影响，因为政策优化不断改变法学硕士的数据分布。从最新的法学硕士中反复收集新的偏好数据可能会缓解这个问题，但不幸的是，这使得最终的系统更加复杂且难以优化。在本文中，我们提出了策略奖励学习（RLP），这是一种无监督框架，它使用策略样本来完善奖励模型以保持其分布。具体来说，引入无监督多视图学习方法来学习策略样本的鲁棒表示。同时，开发了一种综合偏好生成方法来模拟具有政策输出的高质量偏好数据。对三个基准数据集的大量实验表明，RLP 始终优于最先进的技术。我们的代码位于 \url{https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/rlp}。

Title: Ungrammatical-syntax-based In-context Example Selection for Grammatical Error Correction

• Authors: Chenming Tang, Fanyi Qu, Yunfang Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19283
• Pdf URL: https://arxiv.org/pdf/2403.19283
• Copy Paste: [[2403.19283]] Ungrammatical-syntax-based In-context Example Selection for Grammatical Error Correction(https://arxiv.org/abs/2403.19283)
• Keywords: language model, llm, prompt
• Abstract: In the era of large language models (LLMs), in-context learning (ICL) stands out as an effective prompting strategy that explores LLMs' potency across various tasks. However, applying LLMs to grammatical error correction (GEC) is still a challenging task. In this paper, we propose a novel ungrammatical-syntax-based in-context example selection strategy for GEC. Specifically, we measure similarity of sentences based on their syntactic structures with diverse algorithms, and identify optimal ICL examples sharing the most similar ill-formed syntax to the test input. Additionally, we carry out a two-stage process to further improve the quality of selection results. On benchmark English GEC datasets, empirical results show that our proposed ungrammatical-syntax-based strategies outperform commonly-used word-matching or semantics-based methods with multiple LLMs. This indicates that for a syntax-oriented task like GEC, paying more attention to syntactic information can effectively boost LLMs' performance. Our code will be publicly available after the publication of this paper.
• 摘要：在大型语言模型 (LLM) 时代，情境学习 (ICL) 作为一种有效的激励策略脱颖而出，可以探索 LLM 在各种任务中的潜力。然而，将法学硕士应用于语法错误纠正（GEC）仍然是一项具有挑战性的任务。在本文中，我们提出了一种新颖的基于非语法语法的 GEC 上下文示例选择策略。具体来说，我们使用不同的算法根据句子的句法结构来测量句子的相似性，并识别与测试输入共享最相似的格式错误语法的最佳 ICL 示例。此外，我们还实施了两阶段流程，以进一步提高选择结果的质量。在基准英语 GEC 数据集上，实证结果表明，我们提出的基于非语法语法的策略优于具有多个 LLM 的常用单词匹配或基于语义的方法。这表明对于像GEC这样面向语法的任务，更多地关注语法信息可以有效提高LLM的性能。我们的代码将在本文发表后公开。

Title: Going Beyond Word Matching: Syntax Improves In-context Example Selection for Machine Translation

• Authors: Chenming Tang, Zhixiang Wang, Yunfang Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19285
• Pdf URL: https://arxiv.org/pdf/2403.19285
• Copy Paste: [[2403.19285]] Going Beyond Word Matching: Syntax Improves In-context Example Selection for Machine Translation(https://arxiv.org/abs/2403.19285)
• Keywords: language model, llm, prompt
• Abstract: In-context learning (ICL) is the trending prompting strategy in the era of large language models (LLMs), where a few examples are demonstrated to evoke LLMs' power for a given task. How to select informative examples remains an open issue. Previous works on in-context example selection for machine translation (MT) focus on superficial word-level features while ignoring deep syntax-level knowledge. In this paper, we propose a syntax-based in-context example selection method for MT, by computing the syntactic similarity between dependency trees using Polynomial Distance. In addition, we propose an ensemble strategy combining examples selected by both word-level and syntax-level criteria. Experimental results between English and 6 common languages indicate that syntax can effectively enhancing ICL for MT, obtaining the highest COMET scores on 11 out of 12 translation directions.
• 摘要：上下文学习（ICL）是大语言模型（LLM）时代的趋势提示策略，其中展示了一些例子来激发LLM对给定任务的能力。如何选择信息丰富的示例仍然是一个悬而未决的问题。之前关于机器翻译（MT）的上下文示例选择的工作侧重于表面的单词级特征，而忽略了深层的语法级知识。在本文中，我们提出了一种基于语法的机器翻译上下文示例选择方法，通过使用多项式距离计算依存树之间的语法相似性。此外，我们提出了一种结合按单词级和语法级标准选择的示例的集成策略。英语与 6 种常见语言之间的实验结果表明，语法可以有效增强 MT 的 ICL，在 12 个翻译方向中的 11 个上获得最高的 COMET 分数。

Title: MATEval: A Multi-Agent Discussion Framework for Advancing Open-Ended Text Evaluation

• Authors: Yu Li, Shenyu Zhang, Rui Wu, Xiutian Huang, Yongrui Chen, Wenhao Xu, Guilin Qi, Dehai Min
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19305
• Pdf URL: https://arxiv.org/pdf/2403.19305
• Copy Paste: [[2403.19305]] MATEval: A Multi-Agent Discussion Framework for Advancing Open-Ended Text Evaluation(https://arxiv.org/abs/2403.19305)
• Keywords: language model, gpt, llm, chain-of-thought, agent
• Abstract: Recent advancements in generative Large Language Models(LLMs) have been remarkable, however, the quality of the text generated by these models often reveals persistent issues. Evaluating the quality of text generated by these models, especially in open-ended text, has consistently presented a significant challenge. Addressing this, recent work has explored the possibility of using LLMs as evaluators. While using a single LLM as an evaluation agent shows potential, it is filled with significant uncertainty and instability. To address these issues, we propose the MATEval: A "Multi-Agent Text Evaluation framework" where all agents are played by LLMs like GPT-4. The MATEval framework emulates human collaborative discussion methods, integrating multiple agents' interactions to evaluate open-ended text. Our framework incorporates self-reflection and Chain-of-Thought (CoT) strategies, along with feedback mechanisms, enhancing the depth and breadth of the evaluation process and guiding discussions towards consensus, while the framework generates comprehensive evaluation reports, including error localization, error types and scoring. Experimental results show that our framework outperforms existing open-ended text evaluation methods and achieves the highest correlation with human evaluation, which confirms the effectiveness and advancement of our framework in addressing the uncertainties and instabilities in evaluating LLMs-generated text. Furthermore, our framework significantly improves the efficiency of text evaluation and model iteration in industrial scenarios.
• 摘要：生成式大型语言模型 (LLM) 的最新进展非常显着，然而，这些模型生成的文本质量往往揭示出长期存在的问题。评估这些模型生成的文本的质量，尤其是开放式文本的质量，一直是一个重大挑战。为了解决这个问题，最近的工作探索了使用法学硕士作为评估者的可能性。虽然使用单一法学硕士作为评估代理显示出潜力，但它充满了巨大的不确定性和不稳定性。为了解决这些问题，我们提出了 MATEval：一种“多智能体文本评估框架”，其中所有智能体都由 GPT-4 等 LLM 扮演。 MATEval 框架模拟人类协作讨论方法，集成多个代理的交互来评估开放式文本。我们的框架结合了自我反思和思想链（CoT）策略以及反馈机制，增强了评估过程的深度和广度，引导讨论达成共识，同时该框架生成全面的评估报告，包括错误定位、错误类型和评分。实验结果表明，我们的框架优于现有的开放式文本评估方法，并与人类评估实现了最高的相关性，这证实了我们的框架在解决评估法学硕士生成文本的不确定性和不稳定性方面的有效性和先进性。此外，我们的框架显着提高了工业场景中文本评估和模型迭代的效率。

Title: TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios

• Authors: Xiaokang Zhang, Jing Zhang, Zeyao Ma, Yang Li, Bohan Zhang, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, Jifan Yu, Shu Zhao, Juanzi Li, Jie Tang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19318
• Pdf URL: https://arxiv.org/pdf/2403.19318
• Copy Paste: [[2403.19318]] TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios(https://arxiv.org/abs/2403.19318)
• Keywords: language model, llm
• Abstract: We introduce TableLLM, a robust large language model (LLM) with 13 billion parameters, purpose-built for proficiently handling tabular data manipulation tasks, whether they are embedded within documents or spreadsheets, catering to real-world office scenarios. We propose a distant supervision method for training, which comprises a reasoning process extension strategy, aiding in training LLMs to understand reasoning patterns more effectively as well as a cross-way validation strategy, ensuring the quality of the automatically generated data. To evaluate the performance of TableLLM, we have crafted a benchmark tailored to address both document and spreadsheet formats as well as constructed a well-organized evaluation pipeline capable of handling both scenarios. Thorough evaluations underscore the advantages of TableLLM when compared to various existing general-purpose and tabular data-focused LLMs. We have publicly released the model checkpoint, source code, benchmarks, and a web application for user interaction.
• 摘要：我们推出 TableLLM，这是一种强大的大型语言模型 (LLM)，拥有 130 亿个参数，专门用于熟练处理表格数据操作任务，无论它们是嵌入文档还是电子表格中，以满足现实办公场景的需要。我们提出了一种远程监督训练方法，其中包括推理过程扩展策略，帮助训练法学硕士更有效地理解推理模式，以及交叉验证策略，确保自动生成数据的质量。为了评估 TableLLM 的性能，我们精心设计了一个针对文档和电子表格格式的基准测试，并构建了一个能够处理这两种情况的组织良好的评估管道。与各种现有的通用和以表格数据为中心的法学硕士相比，全面的评估强调了 TableLLM 的优势。我们公开发布了模型检查点、源代码、基准测试和用于用户交互的 Web 应用程序。

Title: Dataverse: Open-Source ETL (Extract, Transform, Load) Pipeline for Large Language Models

• Authors: Hyunbyung Park, Sukyung Lee, Gyoungjin Gim, Yungi Kim, Dahyun Kim, Chanjun Park
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2403.19340
• Pdf URL: https://arxiv.org/pdf/2403.19340
• Copy Paste: [[2403.19340]] Dataverse: Open-Source ETL (Extract, Transform, Load) Pipeline for Large Language Models(https://arxiv.org/abs/2403.19340)
• Keywords: language model, llm
• Abstract: To address the challenges associated with data processing at scale, we propose Dataverse, a unified open-source Extract-Transform-Load (ETL) pipeline for large language models (LLMs) with a user-friendly design at its core. Easy addition of custom processors with block-based interface in Dataverse allows users to readily and efficiently use Dataverse to build their own ETL pipeline. We hope that Dataverse will serve as a vital tool for LLM development and open source the entire library to welcome community contribution. Additionally, we provide a concise, two-minute video demonstration of our system, illustrating its capabilities and implementation.
• 摘要：为了解决与大规模数据处理相关的挑战，我们提出了 Dataverse，这是一个适用于大型语言模型 (LLM) 的统一开源提取-转换-加载 (ETL) 管道，其核心是用户友好的设计。在 Dataverse 中轻松添加具有基于块的接口的自定义处理器，使用户能够轻松高效地使用 Dataverse 构建自己的 ETL 管道。我们希望 Dataverse 能够成为 LLM 开发的重要工具，并开源整个库以欢迎社区贡献。此外，我们还提供了系统的简明两分钟视频演示，说明了其功能和实现。

Title: Large Language Models Are Unconscious of Unreasonability in Math Problems

• Authors: Jingyuan Ma, Damai Dai, Zhifang Sui
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19346
• Pdf URL: https://arxiv.org/pdf/2403.19346
• Copy Paste: [[2403.19346]] Large Language Models Are Unconscious of Unreasonability in Math Problems(https://arxiv.org/abs/2403.19346)
• Keywords: language model, llm, hallucination, prompt
• Abstract: Large language models (LLMs) demonstrate substantial capabilities in solving math problems. However, they tend to produce hallucinations when given questions containing unreasonable errors. In this paper, we study the behavior of LLMs when faced with unreasonable math problems and further explore their potential to address these problems. First, we construct the Unreasonable Math Problem (UMP) benchmark to examine the error detection ability of LLMs. Experiments show that LLMs are able to detect unreasonable errors, but still fail in generating non-hallucinatory content. In order to improve their ability of error detection and correction, we further design a strategic prompt template called Critical Calculation and Conclusion(CCC). With CCC, LLMs can better self-evaluate and detect unreasonable errors in math questions, making them more reliable and safe in practical application scenarios.
• 摘要：大型语言模型 (LLM) 展示了解决数学问题的强大能力。然而，当提出包含不合理错误的问题时，他们往往会产生幻觉。在本文中，我们研究了法学硕士在面对不合理的数学问题时的行为，并进一步探索他们解决这些问题的潜力。首先，我们构建了不合理数学问题（UMP）基准来检验法学硕士的错误检测能力。实验表明，法学硕士能够检测到不合理的错误，但仍然无法生成非幻觉内容。为了提高他们发现和纠正错误的能力，我们进一步设计了一个策略提示模板，称为关键计算和结论（CCC）。通过CCC，法学硕士可以更好地进行自我评估并发现数学问题中的不合理错误，使其在实际应用场景中更加可靠和安全。

Title: A diverse Multilingual News Headlines Dataset from around the World

• Authors: Felix Leeb, Bernhard Schölkopf
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19352
• Pdf URL: https://arxiv.org/pdf/2403.19352
• Copy Paste: [[2403.19352]] A diverse Multilingual News Headlines Dataset from around the World(https://arxiv.org/abs/2403.19352)
• Keywords: language model
• Abstract: Babel Briefings is a novel dataset featuring 4.7 million news headlines from August 2020 to November 2021, across 30 languages and 54 locations worldwide with English translations of all articles included. Designed for natural language processing and media studies, it serves as a high-quality dataset for training or evaluating language models as well as offering a simple, accessible collection of articles, for example, to analyze global news coverage and cultural narratives. As a simple demonstration of the analyses facilitated by this dataset, we use a basic procedure using a TF-IDF weighted similarity metric to group articles into clusters about the same event. We then visualize the \emph{event signatures} of the event showing articles of which languages appear over time, revealing intuitive features based on the proximity of the event and unexpectedness of the event. The dataset is available on \href{https://www.kaggle.com/datasets/felixludos/babel-briefings}{Kaggle} and \href{https://huggingface.co/datasets/felixludos/babel-briefings}{HuggingFace} with accompanying \href{https://github.com/felixludos/babel-briefings}{GitHub} code.
• 摘要：Babel Briefings 是一个新颖的数据集，包含 2020 年 8 月至 2021 年 11 月期间的 470 万条新闻标题，涵盖 30 种语言和全球 54 个地点，并包含所有文章的英文翻译。它专为自然语言处理和媒体研究而设计，可作为高质量的数据集用于训练或评估语言模型，并提供简单、易于访问的文章集，例如用于分析全球新闻报道和文化叙事。作为该数据集促进的分析的简单演示，我们使用一个基本程序，使用 TF-IDF 加权相似性度量将有关同一事件的文章分组为集群。然后，我们将事件的 \emph{事件签名} 可视化，显示随时间推移出现的语言的文章，根据事件的邻近性和事件的意外性揭示直观特征。该数据集可在 \href{https://www.kaggle.com/datasets/felixludos/babel-briefings}{Kaggle} 和 \href{https://huggingface.co/datasets/felixludos/babel-briefings}{ HuggingFace} 以及随附的 \href{https://github.com/felixludos/babel-briefings}{GitHub} 代码。

Title: Checkpoint Merging via Bayesian Optimization in LLM Pretraining

• Authors: Deyuan Liu, Zecheng Wang, Bingning Wang, Weipeng Chen, Chunshan Li, Zhiying Tu, Dianhui Chu, Bo Li, Dianbo Sui
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19390
• Pdf URL: https://arxiv.org/pdf/2403.19390
• Copy Paste: [[2403.19390]] Checkpoint Merging via Bayesian Optimization in LLM Pretraining(https://arxiv.org/abs/2403.19390)
• Keywords: language model, gpt, llm
• Abstract: The rapid proliferation of large language models (LLMs) such as GPT-4 and Gemini underscores the intense demand for resources during their training processes, posing significant challenges due to substantial computational and environmental costs. To alleviate this issue, we propose checkpoint merging in pretraining LLM. This method utilizes LLM checkpoints with shared training trajectories, and is rooted in an extensive search space exploration for the best merging weight via Bayesian optimization. Through various experiments, we demonstrate that: (1) Our proposed methodology exhibits the capacity to augment pretraining, presenting an opportunity akin to obtaining substantial benefits at minimal cost; (2) Our proposed methodology, despite requiring a given held-out dataset, still demonstrates robust generalization capabilities across diverse domains, a pivotal aspect in pretraining.
• 摘要：GPT-4 和 Gemini 等大型语言模型 (LLM) 的快速增长凸显了其训练过程中对资源的强烈需求，由于大量的计算和环境成本，带来了重大挑战。为了缓解这个问题，我们建议在预训练 LLM 中进行检查点合并。该方法利用具有共享训练轨迹的 LLM 检查点，并植根于通过贝叶斯优化对最佳合并权重进行广泛的搜索空间探索。通过各种实验，我们证明：（1）我们提出的方法展示了增强预训练的能力，提供了类似于以最小成本获得实质性收益的机会； (2) 尽管需要给定的保留数据集，我们提出的方法仍然展示了跨不同领域的强大泛化能力，这是预训练的关键方面。

Title: BP4ER: Bootstrap Prompting for Explicit Reasoning in Medical Dialogue Generation

• Authors: Yuhong He, Yongqi Zhang, Shizhu He, Jun Wan
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19414
• Pdf URL: https://arxiv.org/pdf/2403.19414
• Copy Paste: [[2403.19414]] BP4ER: Bootstrap Prompting for Explicit Reasoning in Medical Dialogue Generation(https://arxiv.org/abs/2403.19414)
• Keywords: language model, llm, prompt
• Abstract: Medical dialogue generation (MDG) has gained increasing attention due to its substantial practical value. Previous works typically employ a sequence-to-sequence framework to generate medical responses by modeling dialogue context as sequential text with annotated medical entities. While these methods have been successful in generating fluent responses, they fail to provide process explanations of reasoning and require extensive entity annotation. To address these limitations, we propose the method Bootstrap Prompting for Explicit Reasoning in MDG (BP4ER), which explicitly model MDG's multi-step reasoning process and iteratively enhance this reasoning process. We employ a least-to-most prompting strategy to guide a large language model (LLM) in explicit reasoning, breaking down MDG into simpler sub-questions. These sub-questions build on answers from previous ones. Additionally, we also introduce two distinct bootstrapping techniques for prompting, which autonomously correct errors and facilitate the LLM's explicit reasoning. This approach eliminates the need for entity annotation and increases the transparency of the MDG process by explicitly generating the intermediate reasoning chain. The experimental findings on the two public datasets indicate that BP4ER outperforms state-of-the-art methods in terms of both objective and subjective evaluation metrics.
• 摘要：医学对话生成（MDG）因其巨大的实用价值而受到越来越多的关注。以前的工作通常采用序列到序列框架，通过将对话上下文建模为带有注释医疗实体的序列文本来生成医疗响应。虽然这些方法成功地生成了流畅的响应，但它们无法提供推理的过程解释并且需要大量的实体注释。为了解决这些限制，我们提出了 MDG 中显式推理的 Bootstrap Prompting（BP4ER）方法，该方法显式地建模 MDG 的多步骤推理过程并迭代增强该推理过程。我们采用从最少到最多的提示策略来指导大型语言模型（LLM）进行显式推理，将 MDG 分解为更简单的子问题。这些子问题建立在之前问题的答案之上。此外，我们还引入了两种不同的引导引导技术进行提示，这些技术可以自动纠正错误并促进法学硕士的明确推理。这种方法消除了对实体注释的需要，并通过显式生成中间推理链来增加 MDG 过程的透明度。两个公共数据集的实验结果表明，BP4ER 在客观和主观评估指标方面都优于最先进的方法。

Title: Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model

• Authors: Qi Gou, Cam-Tu Nguyen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19443
• Pdf URL: https://arxiv.org/pdf/2403.19443
• Copy Paste: [[2403.19443]] Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model(https://arxiv.org/abs/2403.19443)
• Keywords: language model, gpt, llm
• Abstract: Large Language Models (LLMs) have become increasingly popular due to their ability to process and generate natural language. However, as they are trained on massive datasets of text, LLMs can inherit harmful biases and produce outputs that are not aligned with human values. This paper studies two main approaches to LLM alignment: Reinforcement Learning with Human Feedback (RLHF) and contrastive learning-based methods like Direct Preference Optimization (DPO). By analyzing the stability and robustness of RLHF and DPO, we propose MPO (Mixed Preference Optimization), a novel method that mitigates the weaknesses of both approaches. Specifically, we propose a two-stage training procedure: first train DPO on an easy dataset, and then perform RLHF on a difficult set with DPO model being the reference model. Here, the easy and difficult sets are constructed by a well-trained reward model that splits response pairs into those with large gaps of reward (easy), and those with small gaps (difficult). The first stage allows us to obtain a relatively optimal policy (LLM) model quickly, whereas the second stage refines LLM with online RLHF, thus mitigating the distribution shift issue associated with DPO. Experiments are conducted on two public alignment datasets, namely HH-RLHF and TLDR, demonstrating the effectiveness of MPO, both in terms of GPT4 and human evaluation.
• 摘要：大型语言模型 (LLM) 由于其处理和生成自然语言的能力而变得越来越受欢迎。然而，当法学硕士接受海量文本数据集的训练时，他们可能会继承有害的偏见，并产生与人类价值观不符的输出。本文研究了 LLM 调整的两种主要方法：人类反馈强化学习 (RLHF) 和基于对比学习的方法，例如直接偏好优化 (DPO)。通过分析 RLHF 和 DPO 的稳定性和鲁棒性，我们提出了 MPO（混合偏好优化），这是一种弥补这两种方法缺点的新方法。具体来说，我们提出了一个两阶段的训练过程：首先在简单的数据集上训练 DPO，然后以 DPO 模型作为参考模型在困难的数据集上执行 RLHF。在这里，简单和困难的集合是由训练有素的奖励模型构建的，该模型将响应对分为奖励差距较大的响应对（简单）和奖励差距较小的响应对（困难）。第一阶段使我们能够快速获得相对最优的策略（LLM）模型，而第二阶段通过在线RLHF完善LLM，从而减轻与DPO相关的分配转移问题。在两个公共比对数据集 HH-RLHF 和 TLDR 上进行了实验，证明了 MPO 在 GPT4 和人类评估方面的有效性。

Title: JDocQA: Japanese Document Question Answering Dataset for Generative Language Models

• Authors: Eri Onami, Shuhei Kurita, Taiki Miyanishi, Taro Watanabe
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19454
• Pdf URL: https://arxiv.org/pdf/2403.19454
• Copy Paste: [[2403.19454]] JDocQA: Japanese Document Question Answering Dataset for Generative Language Models(https://arxiv.org/abs/2403.19454)
• Keywords: language model, llm, hallucination
• Abstract: Document question answering is a task of question answering on given documents such as reports, slides, pamphlets, and websites, and it is a truly demanding task as paper and electronic forms of documents are so common in our society. This is known as a quite challenging task because it requires not only text understanding but also understanding of figures and tables, and hence visual question answering (VQA) methods are often examined in addition to textual approaches. We introduce Japanese Document Question Answering (JDocQA), a large-scale document-based QA dataset, essentially requiring both visual and textual information to answer questions, which comprises 5,504 documents in PDF format and annotated 11,600 question-and-answer instances in Japanese. Each QA instance includes references to the document pages and bounding boxes for the answer clues. We incorporate multiple categories of questions and unanswerable questions from the document for realistic question-answering applications. We empirically evaluate the effectiveness of our dataset with text-based large language models (LLMs) and multimodal models. Incorporating unanswerable questions in finetuning may contribute to harnessing the so-called hallucination generation.
• 摘要：文档问答是对给定文档（例如报告、幻灯片、小册子和网站）进行问答的任务，由于纸质和电子形式的文档在我们的社会中非常普遍，因此这是一项真正艰巨的任务。这被认为是一项相当具有挑战性的任务，因为它不仅需要文本理解，还需要理解图形和表格，因此除了文本方法之外，还经常检查视觉问答（VQA）方法。我们引入了日语文档问答（JDocQA），这是一个基于文档的大规模问答数据集，本质上需要视觉和文本信息来回答问题，其中包含 5,504 个 PDF 格式的文档和 11,600 个带注释的日语问答实例。每个 QA 实例都包含对文档页面的引用和答案线索的边界框。我们将文档中的多类问题和无法回答的问题合并起来，以实现实际的问答应用。我们使用基于文本的大语言模型（LLM）和多模态模型来实证评估数据集的有效性。在微调中纳入无法回答的问题可能有助于利用所谓的幻觉生成。

Title: Improving Clinical NLP Performance through Language Model-Generated Synthetic Clinical Data

• Authors: Shan Chen, Jack Gallifant, Marco Guevara, Yanjun Gao, Majid Afshar, Timothy Miller, Dmitriy Dligach, Danielle S. Bitterman
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19511
• Pdf URL: https://arxiv.org/pdf/2403.19511
• Copy Paste: [[2403.19511]] Improving Clinical NLP Performance through Language Model-Generated Synthetic Clinical Data(https://arxiv.org/abs/2403.19511)
• Keywords: language model
• Abstract: Generative models have been showing potential for producing data in mass. This study explores the enhancement of clinical natural language processing performance by utilizing synthetic data generated from advanced language models. Promising results show feasible applications in such a high-stakes domain.
• 摘要：生成模型已经显示出大规模生成数据的潜力。本研究探索利用高级语言模型生成的合成数据来增强临床自然语言处理性能。有希望的结果表明在如此高风险的领域中具有可行的应用。

Title: Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models

• Authors: Ang Lv, Kaiyi Zhang, Yuhan Chen, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie, Rui Yan
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2403.19521
• Pdf URL: https://arxiv.org/pdf/2403.19521
• Copy Paste: [[2403.19521]] Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models(https://arxiv.org/abs/2403.19521)
• Keywords: language model, gpt, prompt
• Abstract: In this paper, we deeply explore the mechanisms employed by Transformer-based language models in factual recall tasks. In zero-shot scenarios, given a prompt like "The capital of France is," task-specific attention heads extract the topic entity, such as "France," from the context and pass it to subsequent MLPs to recall the required answer such as "Paris." We introduce a novel analysis method aimed at decomposing the outputs of the MLP into components understandable by humans. Through this method, we quantify the function of the MLP layer following these task-specific heads. In the residual stream, it either erases or amplifies the information originating from individual heads. Moreover, it generates a component that redirects the residual stream towards the direction of its expected answer. These zero-shot mechanisms are also employed in few-shot scenarios. Additionally, we observed a widely existent anti-overconfidence mechanism in the final layer of models, which suppresses correct predictions. We mitigate this suppression by leveraging our interpretation to improve factual recall performance. Our interpretations have been evaluated across various language models, from the GPT-2 families to 1.3B OPT, and across tasks covering different domains of factual knowledge.
• 摘要：在本文中，我们深入探讨了基于 Transformer 的语言模型在事实回忆任务中采用的机制。在零样本场景中，给出像“法国的首都是”这样的提示，特定于任务的注意力头从上下文中提取主题实体，例如“法国”，并将其传递给后续的 MLP 以回忆所需的答案，例如“巴黎。”我们引入了一种新颖的分析方法，旨在将 MLP 的输出分解为人类可以理解的组件。通过这种方法，我们量化了这些特定任务头之后的 MLP 层的功能。在残余流中，它会擦除​​或放大来自各个磁头的信息。此外，它还生成一个组件，将剩余流重定向到其预期答案的方向。这些零样本机制也适用于少样本场景。此外，我们在模型的最后一层观察到广泛存在的反过度自信机制，它抑制了正确的预测。我们通过利用我们的解释来提高事实回忆性能来减轻这种抑制。我们的解释已经在从 GPT-2 系列到 1.3B OPT 的各种语言模型以及涵盖不同事实知识领域的任务中进行了评估。

Title: WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models

• Authors: Piotr Molenda, Adian Liusie, Mark J. F. Gales
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2403.19548
• Pdf URL: https://arxiv.org/pdf/2403.19548
• Copy Paste: [[2403.19548]] WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models(https://arxiv.org/abs/2403.19548)
• Keywords: llm
• Abstract: Watermarking generative-AI systems, such as LLMs, has gained considerable interest, driven by their enhanced capabilities across a wide range of tasks. Although current approaches have demonstrated that small, context-dependent shifts in the word distributions can be used to apply and detect watermarks, there has been little work in analyzing the impact that these perturbations have on the quality of generated texts. Balancing high detectability with minimal performance degradation is crucial in terms of selecting the appropriate watermarking setting; therefore this paper proposes a simple analysis framework where comparative assessment, a flexible NLG evaluation framework, is used to assess the quality degradation caused by a particular watermark setting. We demonstrate that our framework provides easy visualization of the quality-detection trade-off of watermark settings, enabling a simple solution to find an LLM watermark operating point that provides a well-balanced performance. This approach is applied to two different summarization systems and a translation system, enabling cross-model analysis for a task, and cross-task analysis.
• 摘要：法学硕士等水印生成人工智能系统因其在广泛任务中增强的功能而引起了人们极大的兴趣。尽管当前的方法已经证明，单词分布中依赖于上下文的微小变化可用于应用和检测水印，但在分析这些扰动对生成文本的质量的影响方面几乎没有开展工作。在选择适当的水印设置时，平衡高可检测性和最小的性能下降至关重要；因此，本文提出了一个简单的分析框架，其中使用比较评估（一种灵活的NLG评估框架）来评估特定水印设置引起的质量下降。我们证明，我们的框架提供了水印设置的质量检测权衡的简单可视化，使简单的解决方案能够找到提供均衡性能的 LLM 水印操作点。该方法应用于两个不同的摘要系统和翻译系统，实现任务的跨模型分析和跨任务分析。

Title: Retrieval-Enhanced Knowledge Editing for Multi-Hop Question Answering in Language Models

• Authors: Yucheng Shi, Qiaoyu Tan, Xuansheng Wu, Shaochen Zhong, Kaixiong Zhou, Ninghao Liu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2403.19631
• Pdf URL: https://arxiv.org/pdf/2403.19631
• Copy Paste: [[2403.19631]] Retrieval-Enhanced Knowledge Editing for Multi-Hop Question Answering in Language Models(https://arxiv.org/abs/2403.19631)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) have shown proficiency in question-answering tasks but often struggle to integrate real-time knowledge updates, leading to potentially outdated or inaccurate responses. This problem becomes even more challenging when dealing with multi-hop questions since they require LLMs to update and integrate multiple knowledge pieces relevant to the questions. To tackle the problem, we propose the Retrieval-Augmented model Editing (RAE) framework tailored for multi-hop question answering. RAE first retrieves edited facts and then refines the language model through in-context learning. Specifically, our retrieval approach, based on mutual information maximization, leverages the reasoning abilities of LLMs to identify chain facts that na\"ive similarity-based searches might miss. Additionally, our framework incorporates a pruning strategy to eliminate redundant information from the retrieved facts, which enhances the editing accuracy and mitigates the hallucination problem. Our framework is supported by theoretical justification for its fact retrieval efficacy. Finally, comprehensive evaluation across various LLMs validates RAE's ability in providing accurate answers with updated knowledge.
• 摘要：大型语言模型 (LLM) 在问答任务方面表现出熟练程度，但常常难以集成实时知识更新，从而导致可能过时或不准确的响应。在处理多跳问题时，这个问题变得更具挑战性，因为它们需要法学硕士更新和整合与问题相关的多个知识片段。为了解决这个问题，我们提出了专为多跳问答而设计的检索增强模型编辑（RAE）框架。 RAE 首先检索经过编辑的事实，然后通过上下文学习来完善语言模型。具体来说，我们的检索方法基于互信息最大化，利用法学硕士的推理能力来识别基于相似性的搜索可能会错过的链事实。此外，我们的框架采用了修剪策略，以消除检索到的事实中的冗余信息，这提高了编辑准确性并减轻了幻觉问题。我们的框架得到了其事实检索功效的理论论证的支持。最后，对各个法学硕士的综合评估验证了 RAE 通过更新的知识提供准确答案的能力。