2024-08-30

Title: Using large language models to estimate features of multi-word expressions: Concreteness, valence, arousal

• Authors: Gonzalo Martínez, Juan Diego Molero, Sandra González, Javier Conde, Marc Brysbaert, Pedro Reviriego
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16012
• Pdf URL: https://arxiv.org/pdf/2408.16012
• Copy Paste: [[2408.16012]] Using large language models to estimate features of multi-word expressions: Concreteness, valence, arousal(https://arxiv.org/abs/2408.16012)
• Keywords: language model, gpt, llm, chat
• Abstract: This study investigates the potential of large language models (LLMs) to provide accurate estimates of concreteness, valence and arousal for multi-word expressions. Unlike previous artificial intelligence (AI) methods, LLMs can capture the nuanced meanings of multi-word expressions. We systematically evaluated ChatGPT-4o's ability to predict concreteness, valence and arousal. In Study 1, ChatGPT-4o showed strong correlations with human concreteness ratings (r = .8) for multi-word expressions. In Study 2, these findings were repeated for valence and arousal ratings of individual words, matching or outperforming previous AI models. Study 3 extended the prevalence and arousal analysis to multi-word expressions and showed promising results despite the lack of large-scale human benchmarks. These findings highlight the potential of LLMs for generating valuable psycholinguistic data related to multiword expressions. To help researchers with stimulus selection, we provide datasets with AI norms of concreteness, valence and arousal for 126,397 English single words and 63,680 multi-word expressions
• 摘要：本研究探讨了大型语言模型 (LLM) 为多词表达提供具体性、价位和唤醒度的准确估计的潜力。与以前的人工智能 (AI) 方法不同，LLM 可以捕捉多词表达的细微含义。我们系统地评估了 ChatGPT-4o 预测具体性、价位和唤醒度的能力。在研究 1 中，ChatGPT-4o 与人类对多词表达的具体性评分 (r = .8) 表现出很强的相关性。在研究 2 中，这些发现在单个单词的价位和唤醒度评分中重复出现，与以前的 AI 模型相当或优于后者。研究 3 将普遍性和唤醒度分析扩展到多词表达，尽管缺乏大规模的人类基准，但仍显示出令人鼓舞的结果。这些发现凸显了 LLM 在生成与多词表达相关的有价值的心理语言学数据的潜力。为了帮助研究人员进行刺激选择，我们提供了包含 126,397 个英文单词和 63,680 个多词表达的具体性、效价和唤醒度的 AI 规范数据集

Title: Using Large Language Models to Create AI Personas for Replication and Prediction of Media Effects: An Empirical Test of 133 Published Experimental Research Findings

• Authors: Leo Yeykelis, Kaavya Pichai, James J. Cummings, Byron Reeves
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2408.16073
• Pdf URL: https://arxiv.org/pdf/2408.16073
• Copy Paste: [[2408.16073]] Using Large Language Models to Create AI Personas for Replication and Prediction of Media Effects: An Empirical Test of 133 Published Experimental Research Findings(https://arxiv.org/abs/2408.16073)
• Keywords: language model, llm, prompt
• Abstract: This report analyzes the potential for large language models (LLMs) to expedite accurate replication of published message effects studies. We tested LLM-powered participants (personas) by replicating 133 experimental findings from 14 papers containing 45 recent studies in the Journal of Marketing (January 2023-May 2024). We used a new software tool, Viewpoints AI (this https URL), that takes study designs, stimuli, and measures as input, automatically generates prompts for LLMs to act as a specified sample of unique personas, and collects their responses to produce a final output in the form of a complete dataset and statistical analysis. The underlying LLM used was Anthropic's Claude Sonnet 3.5. We generated 19,447 AI personas to replicate these studies with the exact same sample attributes, study designs, stimuli, and measures reported in the original human research. Our LLM replications successfully reproduced 76% of the original main effects (84 out of 111), demonstrating strong potential for AI-assisted replication of studies in which people respond to media stimuli. When including interaction effects, the overall replication rate was 68% (90 out of 133). The use of LLMs to replicate and accelerate marketing research on media effects is discussed with respect to the replication crisis in social science, potential solutions to generalizability problems in sampling subjects and experimental conditions, and the ability to rapidly test consumer responses to various media stimuli. We also address the limitations of this approach, particularly in replicating complex interaction effects in media response studies, and suggest areas for future research and improvement in AI-assisted experimental replication of media effects.
• 摘要：本报告分析了大型语言模型 (LLM) 加快准确复制已发表的信息效果研究的潜力。我们通过复制《市场营销杂志》（2023 年 1 月至 2024 年 5 月）中 14 篇论文（包含 45 项最新研究）中的 133 项实验结果，测试了由 LLM 驱动的参与者（角色）。我们使用了一种新的软件工具 Viewpoints AI（此 https URL），它将研究设计、刺激和测量作为输入，自动生成提示，让 LLM 充当特定角色的指定样本，并收集他们的反应以完整数据集和统计分析的形式生成最终输出。使用的底层 LLM 是 Anthropic 的 Claude Sonnet 3.5。我们生成了 19,447 个 AI 角色来复制这些研究，其样本属性、研究设计、刺激和测量与原始人类研究中报告的完全相同。我们的 LLM 复制成功重现了 76% 的原始主效应（111 个中的 84 个），表明人工智能辅助复制人们对媒体刺激做出反应的研究具有巨大潜力。当包括交互效应时，总体复制率为 68%（133 个中的 90 个）。本文讨论了使用 LLM 复制和加速媒体效应营销研究的问题，涉及社会科学中的复制危机、抽样对象和实验条件中的普遍性问题的潜在解决方案以及快速测试消费者对各种媒体刺激的反应的能力。我们还讨论了这种方法的局限性，特别是在复制媒体反应研究中的复杂交互效应方面，并提出了人工智能辅助实验复制媒体效应未来研究和改进的领域。

Title: Structured Event Reasoning with Large Language Models

• Authors: Li Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16098
• Pdf URL: https://arxiv.org/pdf/2408.16098
• Copy Paste: [[2408.16098]] Structured Event Reasoning with Large Language Models(https://arxiv.org/abs/2408.16098)
• Keywords: language model, llm, prompt
• Abstract: Reasoning about real-life events is a unifying challenge in AI and NLP that has profound utility in a variety of domains, while fallacy in high-stake applications could be catastrophic. Able to work with diverse text in these domains, large language models (LLMs) have proven capable of answering questions and solving problems. However, I show that end-to-end LLMs still systematically fail to reason about complex events, and they lack interpretability due to their black-box nature. To address these issues, I propose three general approaches to use LLMs in conjunction with a structured representation of events. The first is a language-based representation involving relations of sub-events that can be learned by LLMs via fine-tuning. The second is a semi-symbolic representation involving states of entities that can be predicted and leveraged by LLMs via few-shot prompting. The third is a fully symbolic representation that can be predicted by LLMs trained with structured data and be executed by symbolic solvers. On a suite of event reasoning tasks spanning common-sense inference and planning, I show that each approach greatly outperforms end-to-end LLMs with more interpretability. These results suggest manners of synergy between LLMs and structured representations for event reasoning and beyond.
• 摘要：推理现实生活中的事件是 AI 和 NLP 中一个统一的挑战，在各种领域都具有深远的实用性，而在高风险应用中的谬误可能是灾难性的。大型语言模型 (LLM) 能够处理这些领域中的各种文本，已被证明能够回答问题和解决问题。然而，我表明端到端 LLM 仍然系统地无法推理复杂事件，并且由于其黑箱性质而缺乏可解释性。为了解决这些问题，我提出了三种将 LLM 与事件的结构化表示结合使用的通用方法。第一种是基于语言的表示，涉及子事件的关系，LLM 可以通过微调来学习。第二种是半符号表示，涉及实体的状态，LLM 可以通过少量提示来预测和利用这些状态。第三种是完全符号表示，可以由使用结构化数据训练的 LLM 预测并由符号求解器执行。在一系列涵盖常识推理和规划的事件推理任务中，我表明每种方法都大大优于端到端 LLM，并且具有更高的可解释性。这些结果表明 LLM 和结构化表示在事件推理及其他方面存在协同作用。

Title: Evaluating Computational Representations of Character: An Austen Character Similarity Benchmark

• Authors: Funing Yang, Carolyn Jane Anderson
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16131
• Pdf URL: https://arxiv.org/pdf/2408.16131
• Copy Paste: [[2408.16131]] Evaluating Computational Representations of Character: An Austen Character Similarity Benchmark(https://arxiv.org/abs/2408.16131)
• Keywords: gpt
• Abstract: Several systems have been developed to extract information about characters to aid computational analysis of English literature. We propose character similarity grouping as a holistic evaluation task for these pipelines. We present AustenAlike, a benchmark suite of character similarities in Jane Austen's novels. Our benchmark draws on three notions of character similarity: a structurally defined notion of similarity; a socially defined notion of similarity; and an expert defined set extracted from literary criticism. We use AustenAlike to evaluate character features extracted using two pipelines, BookNLP and FanfictionNLP. We build character representations from four kinds of features and compare them to the three AustenAlike benchmarks and to GPT-4 similarity rankings. We find that though computational representations capture some broad similarities based on shared social and narrative roles, the expert pairings in our third benchmark are challenging for all systems, highlighting the subtler aspects of similarity noted by human readers.
• 摘要：已经开发了几种系统来提取有关人物的信息，以辅助对英语文学进行计算分析。我们提出将人物相似性分组作为这些流程的整体评估任务。我们提出了 AustenAlike，这是简·奥斯汀小说中人物相似性的基准套件。我们的基准借鉴了人物相似性的三个概念：结构定义的相似性概念；社会定义的相似性概念；以及从文学批评中提取的专家定义集。我们使用 AustenAlike 来评估使用两个流程 BookNLP 和 FanfictionNLP 提取的人物特征。我们从四种特征构建人物表征，并将它们与三个 AustenAlike 基准和 GPT-4 相似性排名进行比较。我们发现，尽管计算表征基于共同的社交和叙事角色捕捉到了一些广泛的相似性，但我们第三个基准中的专家配对对所有系统来说都具有挑战性，突出了人类读者注意到的相似性的微妙方面。

Title: FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses over SORRY-Bench

• Authors: Aman Priyanshu, Supriti Vijay
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2408.16163
• Pdf URL: https://arxiv.org/pdf/2408.16163
• Copy Paste: [[2408.16163]] FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses over SORRY-Bench(https://arxiv.org/abs/2408.16163)
• Keywords: language model, gpt, llm, prompt
• Abstract: This paper introduces FRACTURED-SORRY-Bench, a framework for evaluating the safety of Large Language Models (LLMs) against multi-turn conversational attacks. Building upon the SORRY-Bench dataset, we propose a simple yet effective method for generating adversarial prompts by breaking down harmful queries into seemingly innocuous sub-questions. Our approach achieves a maximum increase of +46.22\% in Attack Success Rates (ASRs) across GPT-4, GPT-4o, GPT-4o-mini, and GPT-3.5-Turbo models compared to baseline methods. We demonstrate that this technique poses a challenge to current LLM safety measures and highlights the need for more robust defenses against subtle, multi-turn attacks.
• 摘要：本文介绍了 FRACTURED-SORRY-Bench，这是一个用于评估大型语言模型 (LLM) 抵御多轮对话攻击的安全性的框架。基于 SORRY-Bench 数据集，我们提出了一种简单而有效的方法，通过将有害查询分解为看似无害的子问题来生成对抗性提示。与基线方法相比，我们的方法在 GPT-4、GPT-4o、GPT-4o-mini 和 GPT-3.5-Turbo 模型中的攻击成功率 (ASR) 最高提高了 +46.22\%。我们证明这种技术对当前的 LLM 安全措施构成了挑战，并强调需要更强大的防御措施来抵御微妙的多轮攻击。

Title: Making the Most of your Model: Methods for Finetuning and Applying Pretrained Transformers

• Authors: Davis Yoshida
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16241
• Pdf URL: https://arxiv.org/pdf/2408.16241
• Copy Paste: [[2408.16241]] Making the Most of your Model: Methods for Finetuning and Applying Pretrained Transformers(https://arxiv.org/abs/2408.16241)
• Keywords: language model
• Abstract: This thesis provides methods and analysis of models which make progress on this goal. The techniques outlined are task agnostic, and should provide benefit when used with nearly any transformer LM. We introduce two new finetuning methods which add new capabilities to the models they are used on. The first adds a recurrence mechanism, which removes the fixed-window sized constraint and improves the efficiency of a transformer decoder. The second allows masked language models (MLMs) to be used for initialization of both the encoder and decoder of a non-autoregressive sequence-to-sequence transformer, opening up generative applications of models which were previously only used for natural language understanding tasks. We also introduce two new techniques for improving the quality of predictions of any transformer decoder without additional finetuning. One, hidden state optimization, can be applied to any transformer decoder to improve the quality of predictions at inference time, especially for few-shot classification. The other, conditional beam search, allows practitioners to search for natural language generation (NLG) model outputs with high likelihood while conditioning on the event that the output is not degenerate (e.g. empty, repetitive, etc.). Finally, we provide theoretical and empirical insights on the divergence of model-likelihood and output quality which has widely been observed in prior work. These insights apply to any model which represents a distribution over text, and apply to language models which are not transformers or even autoregressive. We argue that the NLP community has, to some extent, misunderstood the implications of these findings, and encourage a point of view which has more nuance.
• 摘要：本论文提供了在此目标上取得进展的方法和模型分析。所概述的技术与任务无关，与几乎任何 Transformer 语言模型一起使用时都应该能带来好处。我们引入了两种新的微调方法，它们为所使用的模型增加了新功能。第一种方法增加了一个递归机制，消除了固定窗口大小的限制并提高了 Transformer 解码器的效率。第二种方法允许使用掩码语言模型 (MLM) 来初始化非自回归序列到序列 Transformer 的编码器和解码器，从而开辟了以前仅用于自然语言理解任务的模型的生成应用。我们还介绍了两种新技术，用于提高任何 Transformer 解码器的预测质量，而无需额外的微调。一种是隐藏状态优化，可以应用于任何 Transformer 解码器，以提高推理时的预测质量，尤其是对于小样本分类。另一种方法是条件定向搜索，它允许从业者搜索具有高可能性的自然语言生成 (NLG) 模型输出，同时以输出不退化（例如空、重复等）为条件。最后，我们提供了关于模型可能性和输出质量差异的理论和实证见解，这在以前的工作中已经得到广泛观察。这些见解适用于任何表示文本分布的模型，也适用于不是转换器甚至不是自回归的语言模型。我们认为，NLP 社区在某种程度上误解了这些发现的含义，并鼓励一种更微妙的观点。

Title: LoraMap: Harnessing the Power of LoRA Connections

• Authors: Hyeryun Park, Jeongwon Kwak, Dongsuk Jang, Sumin Park, Jinwook Choi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2408.16264
• Pdf URL: https://arxiv.org/pdf/2408.16264
• Copy Paste: [[2408.16264]] LoraMap: Harnessing the Power of LoRA Connections(https://arxiv.org/abs/2408.16264)
• Keywords: language model, llm, hallucination
• Abstract: Large Language Models (LLMs) can benefit from mitigating hallucinations through fact-checking and overcoming substantial computational overhead with parameter-efficient techniques such as Low-Rank Adaptation (LoRA). While some studies have explored the parallel integration of multiple LoRAs, these approaches need attention to the connections between them. This paper investigates methods to establish connections among multiple LoRAs. We create three reasoning datasets tailored to fact-checking and fine-tune individual LoRAs, allowing them to view and reason from diverse perspectives. Then, we explore strategies for allocating these reasoning LoRAs and introduce LoraMap, an approach to map connections between them. The results on the fact-checking task demonstrate that the performance of LoraMap is superior to LoraHub, an existing LoRA composition method. LoraMap also outperforms with significantly fewer parameters than LoraConcat, which concatenates LoRAs and further fine-tunes them.
• 摘要：大型语言模型 (LLM) 可以通过事实核查减轻幻觉，并使用低秩自适应 (LoRA) 等参数高效技术克服大量计算开销。虽然一些研究已经探索了多个 LoRA 的并行集成，但这些方法需要注意它们之间的联系。本文研究了在多个 LoRA 之间建立联系的方法。我们创建了三个针对事实核查的推理数据集，并对各个 LoRA 进行了微调，使它们能够从不同的角度进行查看和推理。然后，我们探索了分配这些推理 LoRA 的策略，并介绍了一种映射它们之间联系的方法 LoraMap。事实核查任务的结果表明，LoraMap 的性能优于现有的 LoRA 组合方法 LoraHub。LoraMap 的性能也优于 LoraConcat，后者将 LoRA 连接起来并进一步对其进行微调。

Title: Enhancing AI-Driven Psychological Consultation: Layered Prompts with Large Language Models

• Authors: Rafael Souza, Jia-Hao Lim, Alexander Davis
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16276
• Pdf URL: https://arxiv.org/pdf/2408.16276
• Copy Paste: [[2408.16276]] Enhancing AI-Driven Psychological Consultation: Layered Prompts with Large Language Models(https://arxiv.org/abs/2408.16276)
• Keywords: language model, gpt, llm, prompt
• Abstract: Psychological consultation is essential for improving mental health and well-being, yet challenges such as the shortage of qualified professionals and scalability issues limit its accessibility. To address these challenges, we explore the use of large language models (LLMs) like GPT-4 to augment psychological consultation services. Our approach introduces a novel layered prompting system that dynamically adapts to user input, enabling comprehensive and relevant information gathering. We also develop empathy-driven and scenario-based prompts to enhance the LLM's emotional intelligence and contextual understanding in therapeutic settings. We validated our approach through experiments using a newly collected dataset of psychological consultation dialogues, demonstrating significant improvements in response quality. The results highlight the potential of our prompt engineering techniques to enhance AI-driven psychological consultation, offering a scalable and accessible solution to meet the growing demand for mental health support.
• 摘要：心理咨询对于改善心理健康和幸福感至关重要，但合格专业人员短缺和可扩展性问题等挑战限制了其可及性。为了应对这些挑战，我们探索使用 GPT-4 等大型语言模型 (LLM) 来增强心理咨询服务。我们的方法引入了一种新颖的分层提示系统，可以动态适应用户输入，从而实现全面和相关的信息收集。我们还开发了同理心驱动和基于场景的提示，以增强 LLM 在治疗环境中的情商和情境理解。我们通过使用新收集的心理咨询对话数据集进行实验验证了我们的方法，结果表明响应质量显着提高。结果凸显了我们的提示工程技术在增强 AI 驱动的心理咨询方面的潜力，提供了一种可扩展且可访问的解决方案，以满足日益增长的心理健康支持需求。

Title: Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems

• Authors: Tian Ye, Zicheng Xu, Yuanzhi Li, Zeyuan Allen-Zhu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2408.16293
• Pdf URL: https://arxiv.org/pdf/2408.16293
• Copy Paste: [[2408.16293]] Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems(https://arxiv.org/abs/2408.16293)
• Keywords: language model, prompt
• Abstract: Language models have demonstrated remarkable performance in solving reasoning tasks; however, even the strongest models still occasionally make reasoning mistakes. Recently, there has been active research aimed at improving reasoning accuracy, particularly by using pretrained language models to "self-correct" their mistakes via multi-round prompting. In this paper, we follow this line of work but focus on understanding the usefulness of incorporating "error-correction" data directly into the pretraining stage. This data consists of erroneous solution steps immediately followed by their corrections. Using a synthetic math dataset, we show promising results: this type of pretrain data can help language models achieve higher reasoning accuracy directly (i.e., through simple auto-regression, without multi-round prompting) compared to pretraining on the same amount of error-free data. We also delve into many details, such as (1) how this approach differs from beam search, (2) how such data can be prepared, (3) whether masking is needed on the erroneous tokens, (4) the amount of error required, (5) whether such data can be deferred to the fine-tuning stage, and many others.
• 摘要：语言模型在解决推理任务方面表现出色；然而，即使是最强大的模型也偶尔会犯推理错误。最近，有积极的研究旨在提高推理准确性，特别是通过使用预训练语言模型通过多轮提示“自我纠正”错误。在本文中，我们遵循这条工作路线，但重点是了解将“错误纠正”数据直接纳入预训练阶段的实用性。这些数据包括错误的解决步骤及其后续纠正。使用合成数学数据集，我们展示了有希望的结果：与在相同数量的无错误数据上进行预训练相比，这种类型的预训练数据可以帮助语言模型直接实现更高的推理准确性（即通过简单的自回归，无需多轮提示）。我们还深入研究了许多细节，例如（1）这种方法与集束搜索有何不同，（2）如何准备此类数据，（3）是否需要对错误的标记进行屏蔽，（4）所需的错误量，（5）是否可以将此类数据推迟到微调阶段，等等。

Title: Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic

• Authors: Xin Zheng, Jie Lou, Boxi Cao, Xueru Wen, Yuqiu Ji, Hongyu Lin, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16326
• Pdf URL: https://arxiv.org/pdf/2408.16326
• Copy Paste: [[2408.16326]] Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic(https://arxiv.org/abs/2408.16326)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: Self-critic has become an important mechanism for enhancing the reasoning performance of LLMs. However, current approaches mainly involve basic prompts without further training, which tend to be over-simplified, leading to limited accuracy.Moreover, there is a lack of in-depth investigation of the relationship between LLM's ability to criticism and its task-solving this http URL address these issues, we propose Critic-CoT, a novel framework that pushes LLMs toward System-2-like critic capability, via step-wise CoT reasoning format and distant-supervision data construction, without the need for human annotation. Experiments on GSM8K and MATH show that via filtering out invalid solutions or iterative refinement, our enhanced model boosts task-solving performance, which demonstrates the effectiveness of our method. Further, we find that training on critique and refinement alone improves the generation. We hope our work could shed light on future research on improving the reasoning and critic ability of LLMs.
• 摘要：自我批评已成为提升 LLM 推理性能的重要机制。然而，目前的方法主要涉及基本提示而没有进一步的训练，这些提示往往过于简单，导致准确性有限。此外，缺乏对 LLM 批评能力与其任务解决之间关系的深入研究。为了解决这些问题，我们提出了 Critic-CoT，这是一个新颖的框架，它通过逐步 CoT 推理格式和远程监督数据构建，将 LLM 推向类似 System-2 的批评能力，而无需人工注释。在 GSM8K 和 MATH 上的实验表明，通过过滤无效解决方案或迭代细化，我们的增强模型提高了任务解决性能，这证明了我们方法的有效性。此外，我们发现仅对批评和细化进行训练就可以提高生成率。我们希望我们的工作可以为未来提高 LLM 推理和批评能力的研究提供启示。

Title: The Unreasonable Ineffectiveness of Nucleus Sampling on Mitigating Text Memorization

• Authors: Luka Borec, Philipp Sadler, David Schlangen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16345
• Pdf URL: https://arxiv.org/pdf/2408.16345
• Copy Paste: [[2408.16345]] The Unreasonable Ineffectiveness of Nucleus Sampling on Mitigating Text Memorization(https://arxiv.org/abs/2408.16345)
• Keywords: language model, gpt, llm
• Abstract: This work analyses the text memorization behavior of large language models (LLMs) when subjected to nucleus sampling. Stochastic decoding methods like nucleus sampling are typically applied to overcome issues such as monotonous and repetitive text generation, which are often observed with maximization-based decoding techniques. We hypothesize that nucleus sampling might also reduce the occurrence of memorization patterns, because it could lead to the selection of tokens outside the memorized sequence. To test this hypothesis we create a diagnostic dataset with a known distribution of duplicates that gives us some control over the likelihood of memorization of certain parts of the training data. Our analysis of two GPT-Neo models fine-tuned on this dataset interestingly shows that (i) an increase of the nucleus size reduces memorization only modestly, and (ii) even when models do not engage in "hard" memorization -- a verbatim reproduction of training samples -- they may still display "soft" memorization whereby they generate outputs that echo the training data but without a complete one-by-one resemblance.
• 摘要：这项研究分析了大型语言模型 (LLM) 在进行核心采样时的文本记忆行为。像核心采样这样的随机解码方法通常用于克服诸如单调和重复的文本生成等问题，这些问题通常在基于最大化的解码技术中观察到。我们假设核心采样也可能减少记忆模式的发生，因为它可能导致选择记忆序列之外的标记。为了检验这一假设，我们创建了一个诊断数据集，其中重复项的分布已知，这使我们能够控制记忆某些部分训练数据的可能性。我们对两个在此数据集上进行微调的 GPT-Neo 模型的分析有趣地表明：(i) 核心大小的增加只会略微减少记忆，(ii) 即使模型不进行“硬”记忆——逐字复制训练样本——它们仍可能显示“软”记忆，即它们生成的输出与训练数据相呼应，但没有完全一一相似。

Title: MQM-Chat: Multidimensional Quality Metrics for Chat Translation

• Authors: Yunmeng Li, Jun Suzuki, Makoto Morishita, Kaori Abe, Kentaro Inui
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16390
• Pdf URL: https://arxiv.org/pdf/2408.16390
• Copy Paste: [[2408.16390]] MQM-Chat: Multidimensional Quality Metrics for Chat Translation(https://arxiv.org/abs/2408.16390)
• Keywords: chat
• Abstract: The complexities of chats pose significant challenges for machine translation models. Recognizing the need for a precise evaluation metric to address the issues of chat translation, this study introduces Multidimensional Quality Metrics for Chat Translation (MQM-Chat). Through the experiments of five models using MQM-Chat, we observed that all models generated certain fundamental errors, while each of them has different shortcomings, such as omission, overly correcting ambiguous source content, and buzzword issues, resulting in the loss of stylized information. Our findings underscore the effectiveness of MQM-Chat in evaluating chat translation, emphasizing the importance of stylized content and dialogue consistency for future studies.
• 摘要：聊天的复杂性对机器翻译模型提出了重大挑战。认识到需要一个精确的评估指标来解决聊天翻译问题，本研究引入了聊天翻译多维质量指标 (MQM-Chat)。通过使用 MQM-Chat 对五个模型进行的实验，我们观察到所有模型都产生了某些基本错误，而每个模型都有不同的缺点，例如遗漏、过度纠正模棱两可的源内容和流行语问题，导致风格化信息的丢失。我们的研究结果强调了 MQM-Chat 在评估聊天翻译方面的有效性，强调了风格化内容和对话一致性对未来研究的重要性。

Title: Instruction-tuned Large Language Models for Machine Translation in the Medical Domain

• Authors: Miguel Rios
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16440
• Pdf URL: https://arxiv.org/pdf/2408.16440
• Copy Paste: [[2408.16440]] Instruction-tuned Large Language Models for Machine Translation in the Medical Domain(https://arxiv.org/abs/2408.16440)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have shown promising results on machine translation for high resource language pairs and domains. However, in specialised domains (e.g. medical) LLMs have shown lower performance compared to standard neural machine translation models. The consistency in the machine translation of terminology is crucial for users, researchers, and translators in specialised domains. In this study, we compare the performance between baseline LLMs and instruction-tuned LLMs in the medical domain. In addition, we introduce terminology from specialised medical dictionaries into the instruction formatted datasets for fine-tuning LLMs. The instruction-tuned LLMs significantly outperform the baseline models with automatic metrics.
• 摘要：大型语言模型 (LLM) 在高资源语言对和领域的机器翻译方面表现出了良好的效果。然而，在专业领域（例如医学），LLM 的性能低于标准神经机器翻译模型。术语机器翻译的一致性对于专业领域的用户、研究人员和翻译人员至关重要。在本研究中，我们比较了医学领域中基线 LLM 和指令调整 LLM 之间的性能。此外，我们将专业医学词典中的术语引入指令格式的数据集中，以对 LLM 进行微调。指令调整的 LLM 在自动指标方面明显优于基线模型。

Title: Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning

• Authors: Rochelle Choenni, Ekaterina Shutova
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16482
• Pdf URL: https://arxiv.org/pdf/2408.16482
• Copy Paste: [[2408.16482]] Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning(https://arxiv.org/abs/2408.16482)
• Keywords: language model, llm
• Abstract: Improving the alignment of Large Language Models (LLMs) with respect to the cultural values that they encode has become an increasingly important topic. In this work, we study whether we can exploit existing knowledge about cultural values at inference time to adjust model responses to cultural value probes. We present a simple and inexpensive method that uses a combination of in-context learning (ICL) and human survey data, and show that we can improve the alignment to cultural values across 5 models that include both English-centric and multilingual LLMs. Importantly, we show that our method could prove useful in test languages other than English and can improve alignment to the cultural values that correspond to a range of culturally diverse countries.
• 摘要：改进大型语言模型 (LLM) 与其所编码的文化价值观的一致性已成为一个日益重要的课题。在这项工作中，我们研究是否可以在推理时利用有关文化价值观的现有知识来调整模型对文化价值观探测的响应。我们提出了一种简单且廉价的方法，该方法结合了上下文学习 (ICL) 和人工调查数据，并表明我们可以改进 5 个模型（包括以英语为中心和多语言 LLM）与文化价值观的一致性。重要的是，我们表明我们的方法在英语以外的测试语言中也很有用，并且可以改进与一系列文化多元化国家/地区相对应的文化价值观的一致性。

Title: LLMs vs Established Text Augmentation Techniques for Classification: When do the Benefits Outweight the Costs?

• Authors: Jan Cegin, Jakub Simko, Peter Brusilovsky
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16502
• Pdf URL: https://arxiv.org/pdf/2408.16502
• Copy Paste: [[2408.16502]] LLMs vs Established Text Augmentation Techniques for Classification: When do the Benefits Outweight the Costs?(https://arxiv.org/abs/2408.16502)
• Keywords: language model, llm
• Abstract: The generative large language models (LLMs) are increasingly being used for data augmentation tasks, where text samples are LLM-paraphrased and then used for classifier fine-tuning. However, a research that would confirm a clear cost-benefit advantage of LLMs over more established augmentation methods is largely missing. To study if (and when) is the LLM-based augmentation advantageous, we compared the effects of recent LLM augmentation methods with established ones on 6 datasets, 3 classifiers and 2 fine-tuning methods. We also varied the number of seeds and collected samples to better explore the downstream model accuracy space. Finally, we performed a cost-benefit analysis and show that LLM-based methods are worthy of deployment only when very small number of seeds is used. Moreover, in many cases, established methods lead to similar or better model accuracies.
• 摘要：生成式大型语言模型 (LLM) 越来越多地用于数据增强任务，其中文本样本经过 LLM 释义，然后用于分类器微调。然而，一项能够证实 LLM 相对于更成熟的增强方法具有明显成本效益优势的研究在很大程度上是缺失的。为了研究基于 LLM 的增强是否（以及何时）有利，我们在 6 个数据集、3 个分类器和 2 种微调方法上比较了最近的 LLM 增强方法与成熟方法的效果。我们还改变了种子的数量并收集了样本，以更好地探索下游模型精度空间。最后，我们进行了成本效益分析，并表明只有在使用非常少量的种子时，基于 LLM 的方法才值得部署。此外，在许多情况下，成熟的方法可以实现相似或更好的模型精度。

Title: CNIMA: A Universal Evaluation Framework and Automated Approach for Assessing Second Language Dialogues

• Authors: Rena Gao, Jingxuan Wu, Carsten Roever, Xuetong Wu, Jing Wu, Long Lv, Jey Han Lau
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16518
• Pdf URL: https://arxiv.org/pdf/2408.16518
• Copy Paste: [[2408.16518]] CNIMA: A Universal Evaluation Framework and Automated Approach for Assessing Second Language Dialogues(https://arxiv.org/abs/2408.16518)
• Keywords: language model
• Abstract: We develop CNIMA (Chinese Non-Native Interactivity Measurement and Automation), a Chinese-as-a-second-language labelled dataset with 10K dialogues. We annotate CNIMA using an evaluation framework -- originally introduced for English-as-a-second-language dialogues -- that assesses micro-level features (e.g.\ backchannels) and macro-level interactivity labels (e.g.\ topic management) and test the framework's transferability from English to Chinese. We found the framework robust across languages and revealed universal and language-specific relationships between micro-level and macro-level features. Next, we propose an approach to automate the evaluation and find strong performance, creating a new tool for automated second language assessment. Our system can be adapted to other languages easily as it uses large language models and as such does not require large-scale annotated training data.
• 摘要：我们开发了 CNIMA（中文非母语互动测量与自动化），这是一个带有 10K 对话的以中文为第二语言的数据集。我们使用一个评估框架（最初是为英语为第二语言的对话而引入的）对 CNIMA 进行注释，该框架评估微观特征（例如反向通道）和宏观互动标签（例如主题管理），并测试该框架从英语到中文的可转移性。我们发现该框架在各种语言中都很稳定，并揭示了微观和宏观特征之间的普遍和特定语言关系。接下来，我们提出了一种自动化评估的方法，并获得了良好的效果，从而为自动化第二语言评估创建了一种新工具。我们的系统可以轻松适应其他语言，因为它使用大型语言模型，因此不需要大规模带注释的训练数据。

Title: SALSA: Speedy ASR-LLM Synchronous Aggregation

• Authors: Ashish Mittal, Darshan Prabhu, Sunita Sarawagi, Preethi Jyothi
• Subjects: cs.CL, cs.LG, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2408.16542
• Pdf URL: https://arxiv.org/pdf/2408.16542
• Copy Paste: [[2408.16542]] SALSA: Speedy ASR-LLM Synchronous Aggregation(https://arxiv.org/abs/2408.16542)
• Keywords: llm
• Abstract: Harnessing pre-trained LLMs to improve ASR systems, particularly for low-resource languages, is now an emerging area of research. Existing methods range from using LLMs for ASR error correction to tightly coupled systems that replace the ASR decoder with the LLM. These approaches either increase decoding time or require expensive training of the cross-attention layers. We propose SALSA, which couples the decoder layers of the ASR to the LLM decoder, while synchronously advancing both decoders. Such coupling is performed with a simple projection of the last decoder state, and is thus significantly more training efficient than earlier approaches. A challenge of our proposed coupling is handling the mismatch between the tokenizers of the LLM and ASR systems. We handle this mismatch using cascading tokenization with respect to the LLM and ASR vocabularies. We evaluate SALSA on 8 low-resource languages in the FLEURS benchmark, yielding substantial WER reductions of up to 38%.
• 摘要：利用预训练的 LLM 来改进 ASR 系统（尤其是针对低资源语言的 ASR 系统）现在是一个新兴的研究领域。现有方法包括使用 LLM 进行 ASR 纠错，以及使用 LLM 替换 ASR 解码器的紧密耦合系统。这些方法要么增加解码时间，要么需要昂贵的交叉注意层训练。我们提出了 SALSA，它将 ASR 的解码器层与 LLM 解码器耦合，同时同步推进两个解码器。这种耦合是通过对最后一个解码器状态的简单投影来执行的，因此比以前的方法训练效率要高得多。我们提出的耦合的一个挑战是处理 LLM 和 ASR 系统的标记器之间的不匹配。我们使用与 LLM 和 ASR 词汇表相关的级联标记来处理这种不匹配。我们在 FLEURS 基准中对 8 种低资源语言的 SALSA 进行了评估，结果 WER 大幅降低，最高可达 38%。

Title: Enhancing Dialogue Generation in Werewolf Game Through Situation Analysis and Persuasion Strategies

• Authors: Zhiyang Qi, Michimasa Inaba
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2408.16586
• Pdf URL: https://arxiv.org/pdf/2408.16586
• Copy Paste: [[2408.16586]] Enhancing Dialogue Generation in Werewolf Game Through Situation Analysis and Persuasion Strategies(https://arxiv.org/abs/2408.16586)
• Keywords: language model, gpt, llm, hallucination
• Abstract: Recent advancements in natural language processing, particularly with large language models (LLMs) like GPT-4, have significantly enhanced dialogue systems, enabling them to generate more natural and fluent conversations. Despite these improvements, challenges persist, such as managing continuous dialogues, memory retention, and minimizing hallucinations. The AIWolfDial2024 addresses these challenges by employing the Werewolf Game, an incomplete information game, to test the capabilities of LLMs in complex interactive environments. This paper introduces a LLM-based Werewolf Game AI, where each role is supported by situation analysis to aid response generation. Additionally, for the werewolf role, various persuasion strategies, including logical appeal, credibility appeal, and emotional appeal, are employed to effectively persuade other players to align with its actions.
• 摘要：自然语言处理领域的最新进展，尤其是 GPT-4 等大型语言模型 (LLM)，大大增强了对话系统，使其能够生成更自然、更流畅的对话。尽管取得了这些进步，但挑战仍然存在，例如管理连续对话、记忆保留和尽量减少幻觉。AIWolfDial2024 通过使用不完全信息游戏狼人游戏来测试 LLM 在复杂交互环境中的能力，以应对这些挑战。本文介绍了一种基于 LLM 的狼人游戏 AI，其中每个角色都由情境分析支持，以帮助生成响应。此外，对于狼人角色，采用了各种说服策略，包括逻辑诉求、可信度诉求和情感诉求，以有效地说服其他玩家与其行动保持一致。

Title: Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling

• Authors: Hritik Bansal, Arian Hosseini, Rishabh Agarwal, Vinh Q. Tran, Mehran Kazemi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2408.16737
• Pdf URL: https://arxiv.org/pdf/2408.16737
• Copy Paste: [[2408.16737]] Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling(https://arxiv.org/abs/2408.16737)
• Keywords: language model, llm
• Abstract: Training on high-quality synthetic data from strong language models (LMs) is a common strategy to improve the reasoning performance of LMs. In this work, we revisit whether this strategy is compute-optimal under a fixed inference budget (e.g., FLOPs). To do so, we investigate the trade-offs between generating synthetic data using a stronger but more expensive (SE) model versus a weaker but cheaper (WC) model. We evaluate the generated data across three key metrics: coverage, diversity, and false positive rate, and show that the data from WC models may have higher coverage and diversity, but also exhibit higher false positive rates. We then finetune LMs on data from SE and WC models in different settings: knowledge distillation, self-improvement, and a novel weak-to-strong improvement setup where a weaker LM teaches reasoning to a stronger LM. Our findings reveal that models finetuned on WC-generated data consistently outperform those trained on SE-generated data across multiple benchmarks and multiple choices of WC and SE models. These results challenge the prevailing practice of relying on SE models for synthetic data generation, suggesting that WC may be the compute-optimal approach for training advanced LM reasoners.
• 摘要：使用来自强语言模型 (LM) 的高质量合成数据进行训练是提高 LM 推理性能的常用策略。在这项工作中，我们重新审视了这种策略在固定推理预算（例如 FLOP）下是否是计算最优的。为此，我们研究了使用更强大但更昂贵的 (SE) 模型与更弱但更便宜的 (WC) 模型生成合成数据之间的权衡。我们通过三个关键指标评估生成的数据：覆盖率、多样性和假阳性率，并表明来自 WC 模型的数据可能具有更高的覆盖率和多样性，但也表现出更高的假阳性率。然后，我们在不同的设置下对来自 SE 和 WC 模型的数据微调 LM：知识提炼、自我改进和一种新颖的从弱到强的改进设置，其中较弱的 LM 向较强的 LM 教授推理。我们的研究结果表明，在多个基准和多种 WC 和 SE 模型选择中，在 WC 生成的数据上微调的模型始终优于在 SE 生成的数据上训练的模型。这些结果对依赖 SE 模型进行合成数据生成的现行做法提出了挑战，表明 WC 可能是训练高级 LM 推理器的计算最优方法。

Title: Theoretical and Methodological Framework for Studying Texts Produced by Large Language Models

• Authors: Jiří Milička
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16740
• Pdf URL: https://arxiv.org/pdf/2408.16740
• Copy Paste: [[2408.16740]] Theoretical and Methodological Framework for Studying Texts Produced by Large Language Models(https://arxiv.org/abs/2408.16740)
• Keywords: language model, llm
• Abstract: This paper addresses the conceptual, methodological and technical challenges in studying large language models (LLMs) and the texts they produce from a quantitative linguistics perspective. It builds on a theoretical framework that distinguishes between the LLM as a substrate and the entities the model simulates. The paper advocates for a strictly non-anthropomorphic approach to models while cautiously applying methodologies used in studying human linguistic behavior to the simulated entities. While natural language processing researchers focus on the models themselves, their architecture, evaluation, and methods for improving performance, we as quantitative linguists should strive to build a robust theory concerning the characteristics of texts produced by LLMs, how they differ from human-produced texts, and the properties of simulated entities. Additionally, we should explore the potential of LLMs as an instrument for studying human culture, of which language is an integral part.
• 摘要：本文从定量语言学的角度探讨了研究大型语言模型 (LLM) 及其生成的文本时面临的概念、方法和技术挑战。它建立在一个理论框架之上，该框架区分了作为基础的 LLM 和模型模拟的实体。本文主张对模型采取严格的非拟人化方法，同时谨慎地将研究人类语言行为的方法应用于模拟实体。虽然自然语言处理研究人员专注于模型本身、模型架构、评估和提高性能的方法，但作为定量语言学家，我们应该努力建立一个关于 LLM 生成的文本特征、它们与人类生成的文本的区别以及模拟实体的属性的强大理论。此外，我们应该探索 LLM 作为研究人类文化工具的潜力，语言是人类文化不可或缺的一部分。

Title: Assessing Large Language Models for Online Extremism Research: Identification, Explanation, and New Knowledge

• Authors: Beidi Dong, Jin R. Lee, Ziwei Zhu, Balassubramanian Srinivasan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2408.16749
• Pdf URL: https://arxiv.org/pdf/2408.16749
• Copy Paste: [[2408.16749]] Assessing Large Language Models for Online Extremism Research: Identification, Explanation, and New Knowledge(https://arxiv.org/abs/2408.16749)
• Keywords: language model, gpt, prompt
• Abstract: The United States has experienced a significant increase in violent extremism, prompting the need for automated tools to detect and limit the spread of extremist ideology online. This study evaluates the performance of Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-Trained Transformers (GPT) in detecting and classifying online domestic extremist posts. We collected social media posts containing "far-right" and "far-left" ideological keywords and manually labeled them as extremist or non-extremist. Extremist posts were further classified into one or more of five contributing elements of extremism based on a working definitional framework. The BERT model's performance was evaluated based on training data size and knowledge transfer between categories. We also compared the performance of GPT 3.5 and GPT 4 models using different prompts: naïve, layperson-definition, role-playing, and professional-definition. Results showed that the best performing GPT models outperformed the best performing BERT models, with more detailed prompts generally yielding better results. However, overly complex prompts may impair performance. Different versions of GPT have unique sensitives to what they consider extremist. GPT 3.5 performed better at classifying far-left extremist posts, while GPT 4 performed better at classifying far-right extremist posts. Large language models, represented by GPT models, hold significant potential for online extremism classification tasks, surpassing traditional BERT models in a zero-shot setting. Future research should explore human-computer interactions in optimizing GPT models for extremist detection and classification tasks to develop more efficient (e.g., quicker, less effort) and effective (e.g., fewer errors or mistakes) methods for identifying extremist content.
• 摘要：美国暴力极端主义活动显著增加，这促使人们需要自动化工具来检测和限制极端主义思想在网上的传播。本研究评估了 Transformers 的双向编码器表示 (BERT) 和生成式预训练 Transformers (GPT) 在检测和分类在线国内极端主义帖子方面的性能。我们收集了包含“极右”和“极左”意识形态关键词的社交媒体帖子，并手动将其标记为极端主义或非极端主义。根据工作定义框架，极端主义帖子进一步分为五个极端主义因素中的一个或多个。BERT 模型的性能是根据训练数据大小和类别之间的知识转移来评估的。我们还使用不同的提示比较了 GPT 3.5 和 GPT 4 模型的性能：天真、外行定义、角色扮演和专业定义。结果表明，表现最佳的 GPT 模型优于表现最佳的 BERT 模型，更详细的提示通常会产生更好的结果。然而，过于复杂的提示可能会影响性能。不同版本的 GPT 对它们所认为的极端主义内容有独特的敏感度。GPT 3.5 在对极左极端主义帖子进行分类方面表现更好，而 GPT 4 在对极右极端主义帖子进行分类方面表现更好。以 GPT 模型为代表的大型语言模型在在线极端主义分类任务中具有巨大潜力，在零样本设置下超越了传统的 BERT 模型。未来的研究应该探索人机交互，以优化 GPT 模型以用于极端主义检测和分类任务，以开发更高效（例如更快、更省力）和更有效（例如更少错误或失误）的极端主义内容识别方法。

Title: A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Models

• Authors: Yi-Lin Tuan, William Yang Wang
• Subjects: cs.CL, cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2408.16751
• Pdf URL: https://arxiv.org/pdf/2408.16751
• Copy Paste: [[2408.16751]] A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Models(https://arxiv.org/abs/2408.16751)
• Keywords: language model
• Abstract: Beyond maximum likelihood estimation (MLE), the standard objective of a language model (LM) that optimizes good examples probabilities, many studies have explored ways that also penalize bad examples for enhancing the quality of output distribution, including unlikelihood training, exponential maximizing average treatment effect (ExMATE), and direct preference optimization (DPO). To systematically compare these methods and further provide a unified recipe for LM optimization, in this paper, we present a unique angle of gradient analysis of loss functions that simultaneously reward good examples and penalize bad ones in LMs. Through both mathematical results and experiments on CausalDialogue and Anthropic HH-RLHF datasets, we identify distinct functional characteristics among these methods. We find that ExMATE serves as a superior surrogate for MLE, and that combining DPO with ExMATE instead of MLE further enhances both the statistical (5-7%) and generative (+18% win rate) performance.
• 摘要：除了最大似然估计 (MLE)（优化好示例概率的语言模型 (LM) 的标准目标）之外，许多研究还探索了惩罚坏示例以提高输出分布质量的方法，包括可能性训练、指数最大化平均处理效果 (ExMATE) 和直接偏好优化 (DPO)。为了系统地比较这些方法并进一步提供统一的 LM 优化方法，在本文中，我们提出了一个独特的梯度分析损失函数角度，该函数同时奖励 LM 中的好示例并惩罚坏示例。通过数学结果和对 CausalDialogue 和 Anthropic HH-RLHF 数据集的实验，我们发现这些方法之间存在不同的功能特征。我们发现 ExMATE 是 MLE 的更好替代品，并且将 DPO 与 ExMATE 结合使用而不是 MLE 可以进一步提高统计（5-7%）和生成（+18% 胜率）性能。

Title: Reinforcement Learning without Human Feedback for Last Mile Fine-Tuning of Large Language Models

• Authors: Alec Solway
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2408.16753
• Pdf URL: https://arxiv.org/pdf/2408.16753
• Copy Paste: [[2408.16753]] Reinforcement Learning without Human Feedback for Last Mile Fine-Tuning of Large Language Models(https://arxiv.org/abs/2408.16753)
• Keywords: language model, hallucination
• Abstract: Reinforcement learning is used to align language models with human preference signals after first pre-training the model to predict the next token of text within a large corpus using likelihood maximization. Before being deployed in a specific domain, models are often further fine-tuned on task specific data. Since human preferences are often unavailable for the last step, it is performed using likelihood maximization as that is the typical default method. However, reinforcement learning has other advantages besides facilitating alignment to a human derived reward function. For one, whereas likelihood maximization is a form of imitation learning in which the model is trained on what to do under ideal conditions, reinforcement learning is not limited to demonstrating actions just for optimally reached states and trains a model what to do under a range of scenarios as it explores the policy space. In addition, it also trains a model what not to do, suppressing competitive but poor actions. This work develops a framework for last-mile fine-tuning using reinforcement learning and tests whether it garners performance gains. The experiments center on abstractive summarization, but the framework is general and broadly applicable. Use of the procedure produced significantly better results than likelihood maximization when comparing raw predictions. For the specific data tested, the gap could be bridged by employing post-processing of the maximum likelihood outputs. Nonetheless, the framework offers a new avenue for model optimization in situations where post-processing may be less straightforward or effective, and it can be extended to include more complex classes of undesirable outputs to penalize and train against, such as hallucinations.
• 摘要：强化学习用于将语言模型与人类偏好信号对齐，首先使用似然最大化对模型进行预训练，以预测大型语料库中的下一个文本标记。在部署到特定领域之前，模型通常会根据特定于任务的数据进行进一步微调。由于最后一步通常无法获得人类偏好，因此使用似然最大化来执行，因为这是典型的默认方法。然而，除了促进与人类衍生的奖励函数对齐之外，强化学习还有其他优势。首先，似然最大化是一种模仿学习，其中模型在理想条件下接受训练以了解应该做什么，而强化学习并不局限于仅为达到最佳状态展示动作，而是在探索策略空间时训练模型在一系列场景下应该做什么。此外，它还训练模型不该做什么，抑制竞争性但糟糕的行为。这项工作开发了一个使用强化学习进行最后一英里微调的框架，并测试它是否能获得性能提升。实验以抽象概括为中心，但该框架具有普遍性和广泛适用性。在比较原始预测时，使用该程序产生的结果明显优于似然最大化。对于测试的特定数据，可以通过对最大似然输出进行后处理来弥补差距。尽管如此，该框架在后处理可能不那么直接或有效的情况下为模型优化提供了一种新途径，并且可以扩展到包括更复杂的不良输出类别，以进行惩罚和训练，例如幻觉。

Title: How Far Can Cantonese NLP Go? Benchmarking Cantonese Capabilities of Large Language Models

• Authors: Jiyue Jiang, Liheng Chen, Pengan Chen, Sheng Wang, Qinghang Bao, Lingpeng Kong, Yu Li, Chuan Wu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2408.16756
• Pdf URL: https://arxiv.org/pdf/2408.16756
• Copy Paste: [[2408.16756]] How Far Can Cantonese NLP Go? Benchmarking Cantonese Capabilities of Large Language Models(https://arxiv.org/abs/2408.16756)
• Keywords: language model, llm
• Abstract: The rapid evolution of large language models (LLMs) has transformed the competitive landscape in natural language processing (NLP), particularly for English and other data-rich languages. However, underrepresented languages like Cantonese, spoken by over 85 million people, face significant development gaps, which is particularly concerning given the economic significance of the Guangdong-Hong Kong-Macau Greater Bay Area, and in substantial Cantonese-speaking populations in places like Singapore and North America. Despite its wide use, Cantonese has scant representation in NLP research, especially compared to other languages from similarly developed regions. To bridge these gaps, we outline current Cantonese NLP methods and introduce new benchmarks designed to evaluate LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonese, which aim to advance open-source Cantonese LLM technology. We also propose future research directions and recommended models to enhance Cantonese LLM development.
• 摘要：大型语言模型 (LLM) 的快速发展改变了自然语言处理 (NLP) 领域的竞争格局，尤其是英语和其他数据丰富的语言。然而，像粤语这样代表性不足的语言（使用人数超过 8500 万）面临着巨大的发展差距，鉴于粤港澳大湾区的经济重要性以及新加坡和北美等地大量使用粤语的人口，这一点尤其令人担忧。尽管粤语被广泛使用，但在 NLP 研究中却很少见，尤其是与来自类似发达地区的其他语言相比。为了弥补这些差距，我们概述了当前的粤语 NLP 方法，并介绍了旨在评估 LLM 在事实生成、数理逻辑、复杂推理和粤语常识方面表现的新基准，旨在推进开源粤语 LLM 技术。我们还提出了未来的研究方向和推荐模型，以加强粤语 LLM 的发展。