2024-07-26

Title: The #Somos600M Project: Generating NLP resources that represent the diversity of the languages from LATAM, the Caribbean, and Spain

• Authors: María Grandury
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17479
• Pdf URL: https://arxiv.org/pdf/2407.17479
• Copy Paste: [[2407.17479]] The #Somos600M Project: Generating NLP resources that represent the diversity of the languages from LATAM, the Caribbean, and Spain(https://arxiv.org/abs/2407.17479)
• Keywords: language model, llm
• Abstract: We are 600 million Spanish speakers. We launched the #Somos600M Project because the diversity of the languages from LATAM, the Caribbean and Spain needs to be represented in Artificial Intelligence (AI) systems. Despite being the 7.5% of the world population, there is no open dataset to instruction-tune large language models (LLMs), nor a leaderboard to evaluate and compare them. In this paper, we present how we have created as an international open-source community the first versions of the instruction and evaluation datasets, indispensable resources for the advancement of Natural Language Processing (NLP) in our languages.
• 摘要：我们有 6 亿西班牙语使用者。我们启动了 #Somos600M 项目，因为拉丁美洲、加勒比地区和西班牙的语言多样性需要在人工智能 (AI) 系统中体现出来。尽管占世界人口的 7.5%，但没有开放数据集来对大型语言模型 (LLM) 进行指令调整，也没有排行榜来评估和比较它们。在本文中，我们介绍了我们作为一个国际开源社区如何创建指令和评估数据集的第一个版本，这些是推动我们语言的自然语言处理 (NLP) 不可或缺的资源。

Title: Generative artificial intelligence in dentistry: Current approaches and future challenges

• Authors: Fabián Villena, Claudia Véliz, Rosario García-Huidobro, Sebastián Aguayo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17532
• Pdf URL: https://arxiv.org/pdf/2407.17532
• Copy Paste: [[2407.17532]] Generative artificial intelligence in dentistry: Current approaches and future challenges(https://arxiv.org/abs/2407.17532)
• Keywords: prompt, chat
• Abstract: Artificial intelligence (AI) has become a commodity for people because of the advent of generative AI (GenAI) models that bridge the usability gap of AI by providing a natural language interface to interact with complex models. These GenAI models range from text generation - such as two-way chat systems - to the generation of image or video from textual descriptions input by a user. These advancements in AI have impacted Dentistry in multiple aspects. In dental education, the student now has the opportunity to solve a plethora of questions by only prompting a GenAI model and have the answer in a matter of seconds. GenAI models can help us deliver better patient healthcare by helping practitioners gather knowledge quickly and efficiently. Finally, GenAI can also be used in dental research, where the applications range from new drug discovery to assistance in academic writing. In this review, we first define GenAI models and describe their multiple generation modalities; then, we explain and discuss their current and potential applications in Dentistry; and finally, we describe the challenges these new technologies impose in our area.
• 摘要：人工智能 (AI) 已成为人们的商品，因为生成式人工智能 (GenAI) 模型的出现，它通过提供自然语言界面与复杂模型交互来弥补人工智能的可用性差距。这些 GenAI 模型包括文本生成（例如双向聊天系统）以及根据用户输入的文本描述生成图像或视频。人工智能的这些进步对牙科产生了多方面的影响。在牙科教育中，学生现在有机会通过仅提示 GenAI 模型来解决大量问题，并在几秒钟内得到答案。GenAI 模型可以帮助从业者快速有效地收集知识，从而帮助我们提供更好的患者医疗保健。最后，GenAI 还可以用于牙科研究，其应用范围从新药发现到学术写作协助。在这篇评论中，我们首先定义 GenAI 模型并描述它们的多种生成模式；然后，我们解释和讨论它们在牙科中的当前和潜在应用；最后，我们描述了这些新技术给我们领域带来的挑战。

Title: Coupling Speech Encoders with Downstream Text Models

• Authors: Ciprian Chelba, Johan Schalkwyk
• Subjects: cs.CL, cs.LG, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2407.17605
• Pdf URL: https://arxiv.org/pdf/2407.17605
• Copy Paste: [[2407.17605]] Coupling Speech Encoders with Downstream Text Models(https://arxiv.org/abs/2407.17605)
• Keywords: language model, llm
• Abstract: We present a modular approach to building cascade speech translation (AST) models that guarantees that the resulting model performs no worse than the 1-best cascade baseline while preserving state-of-the-art speech recognition (ASR) and text translation (MT) performance for a given task. Our novel contribution is the use of an ``exporter'' layer that is trained under L2-loss to ensure a strong match between ASR embeddings and the MT token embeddings for the 1-best sequence. The ``exporter'' output embeddings are fed directly to the MT model in lieu of 1-best token embeddings, thus guaranteeing that the resulting model performs no worse than the 1-best cascade baseline, while allowing back-propagation gradient to flow from the MT model into the ASR components. The matched-embeddings cascade architecture provide a significant improvement over its 1-best counterpart in scenarios where incremental training of the MT model is not an option and yet we seek to improve quality by leveraging (speech, transcription, translated transcription) data provided with the AST task. The gain disappears when the MT model is incrementally trained on the parallel text data available with the AST task. The approach holds promise for other scenarios that seek to couple ASR encoders and immutable text models, such at large language models (LLM).
• 摘要：我们提出了一种模块化方法来构建级联语音翻译 (AST) 模型，该方法可保证生成的模型的性能不低于 1-best 级联基线，同时为给定任务保留最先进的语音识别 (ASR) 和文本翻译 (MT) 性能。我们新颖的贡献是使用在 L2 损失下训练的“导出器”层来确保 ASR 嵌入和 1-best 序列的 MT 标记嵌入之间的强匹配。“导出器”输出嵌入代替 1-best 标记嵌入直接输入到 MT 模型，从而保证生成的模型的性能不低于 1-best 级联基线，同时允许反向传播梯度从 MT 模型流入 ASR 组件。在无法增量训练 MT 模型但我们希望通过利用 AST 任务提供的（语音、转录、翻译转录）数据来提高质量的情况下，匹配嵌入级联架构比其 1-best 对应架构提供了显著的改进。当使用 AST 任务提供的并行文本数据增量训练 MT 模型时，这种改进就会消失。该方法有望用于寻求结合 ASR 编码器和不可变文本模型的其他场景，例如大型语言模型 (LLM)。

Title: IgnitionInnovators at "Discharge Me!": Chain-of-Thought Instruction Finetuning Large Language Models for Discharge Summaries

• Authors: An Quang Tang, Xiuzhen Zhang, Minh Ngoc Dinh
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17636
• Pdf URL: https://arxiv.org/pdf/2407.17636
• Copy Paste: [[2407.17636]] IgnitionInnovators at "Discharge Me!": Chain-of-Thought Instruction Finetuning Large Language Models for Discharge Summaries(https://arxiv.org/abs/2407.17636)
• Keywords: language model, llm, prompt, chain-of-thought
• Abstract: This paper presents our proposed approach to the Discharge Me! shared task, collocated with the 23th Workshop on Biomedical Natural Language Processing (BioNLP). In this work, we develop an LLM-based framework for solving the Discharge Summary Documentation (DSD) task, i.e., generating the two critical target sections `Brief Hospital Course' and `Discharge Instructions' in the discharge summary. By streamlining the recent instruction-finetuning process on LLMs, we explore several prompting strategies for optimally adapting LLMs to specific generation task of DSD. Experimental results show that providing a clear output structure, complimented by a set of comprehensive Chain-of-Thoughts (CoT) questions, effectively improves the model's reasoning capability, and thereby, enhancing the structural correctness and faithfulness of clinical information in the generated text. Source code is available at: this https URL
• 摘要：本文介绍了我们提出的 Discharge Me! 共享任务方法，与第 23 届生物医学自然语言处理 (BioNLP) 研讨会同期举行。在这项工作中，我们开发了一个基于 LLM 的框架来解决出院总结文档 (DSD) 任务，即生成出院总结中的两个关键目标部分“简要医院病程”和“出院说明”。通过简化最近对 LLM 的指令微调过程，我们探索了几种提示策略，以使 LLM 最佳地适应 DSD 的特定生成任务。实验结果表明，提供清晰的输出结构，辅以一组全面的思路 (CoT) 问题，可以有效提高模型的推理能力，从而提高生成文本中临床信息的结构正确性和真实性。源代码可在以下位置获得：此 https URL

Title: Time Matters: Examine Temporal Effects on Biomedical Language Models

• Authors: Weisi Liu, Zhe He, Xiaolei Huang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17638
• Pdf URL: https://arxiv.org/pdf/2407.17638
• Copy Paste: [[2407.17638]] Time Matters: Examine Temporal Effects on Biomedical Language Models(https://arxiv.org/abs/2407.17638)
• Keywords: language model
• Abstract: Time roots in applying language models for biomedical applications: models are trained on historical data and will be deployed for new or future data, which may vary from training data. While increasing biomedical tasks have employed state-of-the-art language models, there are very few studies have examined temporal effects on biomedical models when data usually shifts across development and deployment. This study fills the gap by statistically probing relations between language model performance and data shifts across three biomedical tasks. We deploy diverse metrics to evaluate model performance, distance methods to measure data drifts, and statistical methods to quantify temporal effects on biomedical language models. Our study shows that time matters for deploying biomedical language models, while the degree of performance degradation varies by biomedical tasks and statistical quantification approaches. We believe this study can establish a solid benchmark to evaluate and assess temporal effects on deploying biomedical language models.
• 摘要：将语言模型应用于生物医学应用时，时间是根源：模型是在历史数据上训练的，并将部署用于新数据或未来的数据，这些数据可能与训练数据不同。虽然越来越多的生物医学任务采用了最先进的语言模型，但很少有研究检查过数据在开发和部署过程中通常会发生变化时对生物医学模型的时间影响。这项研究通过统计探究语言模型性能与三项生物医学任务中数据变化之间的关系，填补了这一空白。我们部署了不同的指标来评估模型性能，部署了距离方法来测量数据漂移，部署了统计方法来量化对生物医学语言模型的时间影响。我们的研究表明，时间对于部署生物医学语言模型很重要，而性能下降的程度因生物医学任务和统计量化方法而异。我们相信这项研究可以建立一个可靠的基准来评估和评估时间对部署生物医学语言模型的影响。

Title: Efficient LLM Training and Serving with Heterogeneous Context Sharding among Attention Heads

• Authors: Xihui Lin, Yunan Zhang, Suyu Ge, Barun Patra, Vishrav Chaudhary, Xia Song
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17678
• Pdf URL: https://arxiv.org/pdf/2407.17678
• Copy Paste: [[2407.17678]] Efficient LLM Training and Serving with Heterogeneous Context Sharding among Attention Heads(https://arxiv.org/abs/2407.17678)
• Keywords: llm
• Abstract: Existing LLM training and inference frameworks struggle in boosting efficiency with sparsity while maintaining the integrity of context and model architecture. Inspired by the sharding concept in database and the fact that attention parallelizes over heads on accelerators, we propose Sparsely-Sharded (S2) Attention, an attention algorithm that allocates heterogeneous context partitions for different attention heads to divide and conquer. S2-Attention enforces each attention head to only attend to a partition of contexts following a strided sparsity pattern, while the full context is preserved as the union of all the shards. As attention heads are processed in separate thread blocks, the context reduction for each head can thus produce end-to-end speed-up and memory reduction. At inference, LLMs trained with S2-Attention can then take the KV cache reduction as free meals with guaranteed model quality preserve. In experiments, we show S2-Attentioncan provide as much as (1) 25.3X wall-clock attention speed-up over FlashAttention-2, resulting in 6X reduction in end-to-end training time and 10X inference latency, (2) on-par model training quality compared to default attention, (3)perfect needle retrieval accuracy over 32K context window. On top of the algorithm, we build DKernel, an LLM training and inference kernel library that allows users to customize sparsity patterns for their own models. We open-sourced DKerneland make it compatible with Megatron, Pytorch, and vLLM.
• 摘要：现有的 LLM 训练和推理框架难以在保持上下文和模型架构完整性的同时提高稀疏性效率。受数据库中分片概念以及注意力在加速器上并行化的事实的启发，我们提出了稀疏分片 (S2) 注意力机制，这是一种注意力算法，它为不同的注意力头分配异构上下文分区以分而治之。S2-Attention 强制每个注意力头仅关注遵循步幅稀疏模式的上下文分区，而完整上下文则保留为所有分片的并集。由于注意力头在单独的线程块中处理，因此每个头的上下文减少可以产生端到端的加速和内存减少。在推理时，使用 S2-Attention 训练的 LLM 可以将 KV 缓存减少视为免费餐，同时保证模型质量。在实验中，我们表明 S2-Attention 可以提供高达 (1) 25.3 倍的时钟注意力加速，比 FlashAttention-2 快 6 倍，从而将端到端训练时间缩短 6 倍，推理延迟缩短 10 倍，(2) 与默认注意力相比，模型训练质量相当，(3) 在 32K 上下文窗口上具有完美的针检索准确度。在算法的基础上，我们构建了 DKernel，这是一个 LLM 训练和推理内核库，允许用户为自己的模型自定义稀疏模式。我们开源了 DKerne，并使其与 Megatron、Pytorch 和 vLLM 兼容。

Title: Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification

• Authors: Lynnette Hui Xian Ng, Iain Cruickshank, Roy Ka-Wei Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17688
• Pdf URL: https://arxiv.org/pdf/2407.17688
• Copy Paste: [[2407.17688]] Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification(https://arxiv.org/abs/2407.17688)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in executing tasks based on natural language queries. However, these models, trained on curated datasets, inherently embody biases ranging from racial to national and gender biases. It remains uncertain whether these biases impact the performance of LLMs for certain tasks. In this study, we investigate the political biases of LLMs within the stance classification task, specifically examining whether these models exhibit a tendency to more accurately classify politically-charged stances. Utilizing three datasets, seven LLMs, and four distinct prompting schemes, we analyze the performance of LLMs on politically oriented statements and targets. Our findings reveal a statistically significant difference in the performance of LLMs across various politically oriented stance classification tasks. Furthermore, we observe that this difference primarily manifests at the dataset level, with models and prompting schemes showing statistically similar performances across different stance classification datasets. Lastly, we observe that when there is greater ambiguity in the target the statement is directed towards, LLMs have poorer stance classification accuracy.
• 摘要：大型语言模型 (LLM) 在执行基于自然语言查询的任务方面表现出了卓越的能力。然而，这些在精选数据集上训练的模型本身就体现了从种族到国家和性别偏见等各种偏见。目前尚不确定这些偏见是否会影响 LLM 在某些任务上的表现。在本研究中，我们调查了 LLM 在立场分类任务中的政治偏见，具体检查了这些模型是否表现出更准确地对政治立场进行分类的倾向。利用三个数据集、七个 LLM 和四种不同的提示方案，我们分析了 LLM 在政治导向的陈述和目标上的表现。我们的研究结果显示，在各种政治导向的立场分类任务中，LLM 的表现存在统计学上的显著差异。此外，我们观察到这种差异主要体现在数据集级别，模型和提示方案在不同的立场分类数据集上表现出统计上相似的表现。最后，我们观察到，当陈述所针对的目标存在较大模糊性时，LLM 的立场分类准确性会较差。

Title: Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy?

• Authors: Hao Shen, Zihan Li, Minqiang Yang, Minghui Ni, Yongfeng Tao, Zhengyang Yu, Weihao Zheng, Chen Xu, Bin Hu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17730
• Pdf URL: https://arxiv.org/pdf/2407.17730
• Copy Paste: [[2407.17730]] Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy?(https://arxiv.org/abs/2407.17730)
• Keywords: language model, llm
• Abstract: In contemporary society, the issue of psychological health has become increasingly prominent, characterized by the diversification, complexity, and universality of mental disorders. Cognitive Behavioral Therapy (CBT), currently the most influential and clinically effective psychological treatment method with no side effects, has limited coverage and poor quality in most countries. In recent years, researches on the recognition and intervention of emotional disorders using large language models (LLMs) have been validated, providing new possibilities for psychological assistance therapy. However, are LLMs truly possible to conduct cognitive behavioral therapy? Many concerns have been raised by mental health experts regarding the use of LLMs for therapy. Seeking to answer this question, we collected real CBT corpus from online video websites, designed and conducted a targeted automatic evaluation framework involving the evaluation of emotion tendency of generated text, structured dialogue pattern and proactive inquiry ability. For emotion tendency, we calculate the emotion tendency score of the CBT dialogue text generated by each model. For structured dialogue pattern, we use a diverse range of automatic evaluation metrics to compare speaking style, the ability to maintain consistency of topic and the use of technology in CBT between different models . As for inquiring to guide the patient, we utilize PQA (Proactive Questioning Ability) metric. We also evaluated the CBT ability of the LLM after integrating a CBT knowledge base to explore the help of introducing additional knowledge to enhance the model's CBT counseling ability. Four LLM variants with excellent performance on natural language processing are evaluated, and the experimental result shows the great potential of LLMs in psychological counseling realm, especially after combining with other technological means.
• 摘要：当代社会，心理健康问题日益突出，心理障碍呈现多样化、复杂性、普遍性特点。认知行为疗法（CBT）是目前最具影响力、临床效果显著且无副作用的心理治疗方法，但在大多数国家覆盖面有限、质量较差。近年来，利用大型语言模型（LLM）对情绪障碍进行识别和干预的研究得到验证，为心理援助治疗提供了新的可能性。然而，LLM真的可以进行认知行为治疗吗？许多心理健康专家对使用LLM进行治疗提出了担忧。为了回答这个问题，我们从在线视频网站收集了真实的CBT语料，设计并进行了有针对性的自动评估框架，涉及生成文本的情绪倾向、结构化对话模式和主动探究能力的评估。对于情绪倾向，我们计算每个模型生成的CBT对话文本的情绪倾向得分。对于结构化对话模式，我们使用多种自动评估指标来比较不同模型在CBT中的说话风格、保持话题一致性的能力和技术使用情况。对于询问引导患者，我们采用了 PQA（主动提问能力）指标。我们还在集成 CBT 知识库后评估了 LLM 的 CBT 能力，以探索引入额外知识对增强模型的 CBT 咨询能力的帮助。对在自然语言处理上表现优异的四个 LLM 变体进行了评估，实验结果显示了 LLM 在心理咨询领域的巨大潜力，尤其是与其他技术手段相结合之后。

Title: BotEval: Facilitating Interactive Human Evaluation

• Authors: Hyundong Cho, Thamme Gowda, Yuyang Huang, Zixun Lu, Tianli Tong, Jonathan May
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17770
• Pdf URL: https://arxiv.org/pdf/2407.17770
• Copy Paste: [[2407.17770]] BotEval: Facilitating Interactive Human Evaluation(https://arxiv.org/abs/2407.17770)
• Keywords: language model, chat
• Abstract: Following the rapid progress in natural language processing (NLP) models, language models are applied to increasingly more complex interactive tasks such as negotiations and conversation moderations. Having human evaluators directly interact with these NLP models is essential for adequately evaluating the performance on such interactive tasks. We develop BotEval, an easily customizable, open-source, evaluation toolkit that focuses on enabling human-bot interactions as part of the evaluation process, as opposed to human evaluators making judgements for a static input. BotEval balances flexibility for customization and user-friendliness by providing templates for common use cases that span various degrees of complexity and built-in compatibility with popular crowdsourcing platforms. We showcase the numerous useful features of BotEval through a study that evaluates the performance of various chatbots on their effectiveness for conversational moderation and discuss how BotEval differs from other annotation tools.
• 摘要：随着自然语言处理 (NLP) 模型的快速发展，语言模型被应用于越来越复杂的交互任务，例如谈判和对话主持。让人类评估者直接与这些 NLP 模型交互对于充分评估此类交互任务的性能至关重要。我们开发了 BotEval，这是一个易于定制的开源评估工具包，专注于将人机交互作为评估过程的一部分，而不是人类评估者对静态输入做出判断。BotEval 通过提供涵盖不同复杂程度的常见用例模板和与流行众包平台的内置兼容性，平衡了定制灵活性和用户友好性。我们通过一项评估各种聊天机器人在对话主持方面的有效性的研究展示了 BotEval 的众多有用功能，并讨论了 BotEval 与其他注释工具的不同之处。

Title: Demystifying Verbatim Memorization in Large Language Models

• Authors: Jing Huang, Diyi Yang, Christopher Potts
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.17817
• Pdf URL: https://arxiv.org/pdf/2407.17817
• Copy Paste: [[2407.17817]] Demystifying Verbatim Memorization in Large Language Models(https://arxiv.org/abs/2407.17817)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) frequently memorize long sequences verbatim, often with serious legal and privacy implications. Much prior work has studied such verbatim memorization using observational data. To complement such work, we develop a framework to study verbatim memorization in a controlled setting by continuing pre-training from Pythia checkpoints with injected sequences. We find that (1) non-trivial amounts of repetition are necessary for verbatim memorization to happen; (2) later (and presumably better) checkpoints are more likely to verbatim memorize sequences, even for out-of-distribution sequences; (3) the generation of memorized sequences is triggered by distributed model states that encode high-level features and makes important use of general language modeling capabilities. Guided by these insights, we develop stress tests to evaluate unlearning methods and find they often fail to remove the verbatim memorized information, while also degrading the LM. Overall, these findings challenge the hypothesis that verbatim memorization stems from specific model weights or mechanisms. Rather, verbatim memorization is intertwined with the LM's general capabilities and thus will be very difficult to isolate and suppress without degrading model quality.
• 摘要：大型语言模型 (LLM) 经常逐字记忆长序列，这通常会带来严重的法律和隐私问题。许多先前的研究已经使用观察数据研究了这种逐字记忆。为了补充这项工作，我们开发了一个框架，通过继续从注入序列的 Pythia 检查点进行预训练，在受控环境中研究逐字记忆。我们发现 (1) 逐字记忆需要大量的重复；(2) 较晚（可能更好）的检查点更有可能逐字记忆序列，即使对于分布外的序列也是如此；(3) 记忆序列的生成是由分布式模型状态触发的，这些状态编码高级特征并充分利用了通用语言建模功能。在这些见解的指导下，我们开发了压力测试来评估反学习方法，发现它们通常无法删除逐字记忆的信息，同时还会降低 LM 的性能。总体而言，这些发现挑战了逐字记忆源于特定模型权重或机制的假设。相反，逐字记忆与 LM 的一般功能交织在一起，因此在不降低模型质量的情况下很难隔离和抑制它。

Title: factgenie: A Framework for Span-based Evaluation of Generated Texts

• Authors: Zdeněk Kasner, Ondřej Plátek, Patrícia Schmidtová, Simone Balloccu, Ondřej Dušek
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17863
• Pdf URL: https://arxiv.org/pdf/2407.17863
• Copy Paste: [[2407.17863]] factgenie: A Framework for Span-based Evaluation of Generated Texts(https://arxiv.org/abs/2407.17863)
• Keywords: language model
• Abstract: We present factgenie: a framework for annotating and visualizing word spans in textual model outputs. Annotations can capture various span-based phenomena such as semantic inaccuracies or irrelevant text. With factgenie, the annotations can be collected both from human crowdworkers and large language models. Our framework consists of a web interface for data visualization and gathering text annotations, powered by an easily extensible codebase.
• 摘要：我们推出了 factgenie：一个用于注释和可视化文本模型输出中的单词跨度的框架。注释可以捕获各种基于跨度的现象，例如语义不准确或不相关的文本。使用 factgenie，可以从人工众包工作者和大型语言模型中收集注释。我们的框架由一个用于数据可视化和收集文本注释的 Web 界面组成，由易于扩展的代码库提供支持。

Title: Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions

• Authors: Jiwon Suh, Injae Na, Woohwan Jung
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17874
• Pdf URL: https://arxiv.org/pdf/2407.17874
• Copy Paste: [[2407.17874]] Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions(https://arxiv.org/abs/2407.17874)
• Keywords: language model, llm
• Abstract: End-to-end automatic speech recognition (E2E ASR) systems have significantly improved speech recognition through training on extensive datasets. Despite these advancements, they still struggle to accurately recognize domain specific words, such as proper nouns and technical terminologies. To address this problem, we propose a method to utilize the state-of-the-art Whisper without modifying its architecture, preserving its generalization performance while enabling it to leverage descriptions effectively. Moreover, we propose two additional training techniques to improve the domain specific ASR: decoder fine-tuning, and context perturbation. We also propose a method to use a Large Language Model (LLM) to generate descriptions with simple metadata, when descriptions are unavailable. Our experiments demonstrate that proposed methods notably enhance domain-specific ASR accuracy on real-life datasets, with LLM-generated descriptions outperforming human-crafted ones in effectiveness.
• 摘要：端到端自动语音识别 (E2E ASR) 系统通过对大量数据集进行训练，显著提高了语音识别能力。尽管取得了这些进步，但它们仍然难以准确识别特定领域的词汇，例如专有名词和技术术语。为了解决这个问题，我们提出了一种方法，利用最先进的 Whisper，而无需修改其架构，在保留其泛化性能的同时，使其能够有效地利用描述。此外，我们提出了两种额外的训练技术来改进特定领域的 ASR：解码器微调和上下文扰动。我们还提出了一种在没有描述时使用大型语言模型 (LLM) 生成具有简单元数据的描述的方法。我们的实验表明，所提出的方法显著提高了现实数据集上特定领域的 ASR 准确性，其中 LLM 生成的描述在有效性上优于人工编写的描述。

Title: The Power of Combining Data and Knowledge: GPT-4o is an Effective Interpreter of Machine Learning Models in Predicting Lymph Node Metastasis of Lung Cancer

• Authors: Danqing Hu, Bing Liu, Xiaofeng Zhu, Nan Wu
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.17900
• Pdf URL: https://arxiv.org/pdf/2407.17900
• Copy Paste: [[2407.17900]] The Power of Combining Data and Knowledge: GPT-4o is an Effective Interpreter of Machine Learning Models in Predicting Lymph Node Metastasis of Lung Cancer(https://arxiv.org/abs/2407.17900)
• Keywords: language model, gpt, llm, prompt
• Abstract: Lymph node metastasis (LNM) is a crucial factor in determining the initial treatment for patients with lung cancer, yet accurate preoperative diagnosis of LNM remains challenging. Recently, large language models (LLMs) have garnered significant attention due to their remarkable text generation capabilities. Leveraging the extensive medical knowledge learned from vast corpora, LLMs can estimate probabilities for clinical problems, though their performance has historically been inferior to data-driven machine learning models. In this paper, we propose a novel ensemble method that combines the medical knowledge acquired by LLMs with the latent patterns identified by machine learning models to enhance LNM prediction performance. Initially, we developed machine learning models using patient data. We then designed a prompt template to integrate the patient data with the predicted probability from the machine learning model. Subsequently, we instructed GPT-4o, the most advanced LLM developed by OpenAI, to estimate the likelihood of LNM based on patient data and then adjust the estimate using the machine learning output. Finally, we collected three outputs from the GPT-4o using the same prompt and ensembled these results as the final prediction. Using the proposed method, our models achieved an AUC value of 0.765 and an AP value of 0.415 for LNM prediction, significantly improving predictive performance compared to baseline machine learning models. The experimental results indicate that GPT-4o can effectively leverage its medical knowledge and the probabilities predicted by machine learning models to achieve more accurate LNM predictions. These findings demonstrate that LLMs can perform well in clinical risk prediction tasks, offering a new paradigm for integrating medical knowledge and patient data in clinical predictions.
• 摘要：淋巴结转移 (LNM) 是决定肺癌患者初始治疗的关键因素，但术前准确诊断 LNM 仍然具有挑战性。最近，大型语言模型 (LLM) 因其出色的文本生成能力而备受关注。利用从海量语料库中学习到的广泛医学知识，LLM 可以估计临床问题的概率，尽管它们的性能历来不如数据驱动的机器学习模型。在本文中，我们提出了一种新颖的集成方法，将 LLM 获得的医学知识与机器学习模型识别的潜在模式相结合，以增强 LNM 预测性能。最初，我们使用患者数据开发了机器学习模型。然后，我们设计了一个提示模板，将患者数据与机器学习模型的预测概率相结合。随后，我们指示 OpenAI 开发的最先进的 LLM GPT-4o 根据患者数据估计 LNM 的可能性，然后使用机器学习输出调整估计值。最后，我们使用相同的提示从 GPT-4o 收集了三个输出，并将这些结果集成为最终预测。使用所提出的方法，我们的模型在 LNM 预测中实现了 0.765 的 AUC 值和 0.415 的 AP 值，与基线机器学习模型相比，预测性能显著提高。实验结果表明，GPT-4o 可以有效利用其医学知识和机器学习模型预测的概率来实现更准确的 LNM 预测。这些发现表明 LLM 可以在临床风险预测任务中表现出色，为将医学知识和患者数据整合到临床预测中提供了一种新范式。

Title: Modelling Multimodal Integration in Human Concept Processing with Vision-and-Language Models

• Authors: Anna Bavaresco, Marianne de Heer Kloots, Sandro Pezzelle, Raquel Fernández
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17914
• Pdf URL: https://arxiv.org/pdf/2407.17914
• Copy Paste: [[2407.17914]] Modelling Multimodal Integration in Human Concept Processing with Vision-and-Language Models(https://arxiv.org/abs/2407.17914)
• Keywords: language model
• Abstract: Representations from deep neural networks (DNNs) have proven remarkably predictive of neural activity involved in both visual and linguistic processing. Despite these successes, most studies to date concern unimodal DNNs, encoding either visual or textual input but not both. Yet, there is growing evidence that human meaning representations integrate linguistic and sensory-motor information. Here we investigate whether the integration of multimodal information operated by current vision-and-language DNN models (VLMs) leads to representations that are more aligned with human brain activity than those obtained by language-only and vision-only DNNs. We focus on fMRI responses recorded while participants read concept words in the context of either a full sentence or an accompanying picture. Our results reveal that VLM representations correlate more strongly than language- and vision-only DNNs with activations in brain areas functionally related to language processing. A comparison between different types of visuo-linguistic architectures shows that recent generative VLMs tend to be less brain-aligned than previous architectures with lower performance on downstream applications. Moreover, through an additional analysis comparing brain vs. behavioural alignment across multiple VLMs, we show that -- with one remarkable exception -- representations that strongly align with behavioural judgments do not correlate highly with brain responses. This indicates that brain similarity does not go hand in hand with behavioural similarity, and vice versa.
• 摘要：深度神经网络 (DNN) 的表征已被证明能够出色地预测涉及视觉和语言处理的神经活动。尽管取得了这些成功，但迄今为止的大多数研究都涉及单模态 DNN，即编码视觉或文本输入，但不是同时编码两者。然而，越来越多的证据表明，人类意义表征整合了语言和感觉运动信息。在这里，我们研究了当前视觉和语言 DNN 模型 (VLM) 操作的多模态信息整合是否会导致比仅语言和仅视觉 DNN 获得的表征更符合人类大脑活动的表征。我们关注的是参与者在完整句子或附带图片的上下文中阅读概念词时记录的 fMRI 反应。我们的结果表明，与仅语言和仅视觉的 DNN 相比，VLM 表征与与语言处理功能相关的大脑区域的激活具有更强的相关性。对不同类型的视觉语言架构的比较表明，最近的生成式 VLM 往往比以前的架构更不符合大脑，下游应用程序的性能较低。此外，通过对多个 VLM 中的大脑与行为一致性进行比较的额外分析，我们发现——除了一个显著的例外——与行为判断高度一致的表征与大脑反应并不高度相关。这表明大脑相似性并不与行为相似性齐头并进，反之亦然。

Title: Positive Text Reframing under Multi-strategy Optimization

• Authors: Shutong Jia, Biwei Cao, Qingqing Gao, Jiuxin Cao, Bo Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17940
• Pdf URL: https://arxiv.org/pdf/2407.17940
• Copy Paste: [[2407.17940]] Positive Text Reframing under Multi-strategy Optimization(https://arxiv.org/abs/2407.17940)
• Keywords: language model
• Abstract: Differing from sentiment transfer, positive reframing seeks to substitute negative perspectives with positive expressions while preserving the original meaning. With the emergence of pre-trained language models (PLMs), it is possible to achieve acceptable results by fine-tuning PLMs. Nevertheless, generating fluent, diverse and task-constrained reframing text remains a significant challenge. To tackle this issue, a \textbf{m}ulti-\textbf{s}trategy \textbf{o}ptimization \textbf{f}ramework (MSOF) is proposed in this paper. Starting from the objective of positive reframing, we first design positive sentiment reward and content preservation reward to encourage the model to transform the negative expressions of the original text while ensuring the integrity and consistency of the semantics. Then, different decoding optimization approaches are introduced to improve the quality of text generation. Finally, based on the modeling formula of positive reframing, we propose a multi-dimensional re-ranking method that further selects candidate sentences from three dimensions: strategy consistency, text similarity and fluency. Extensive experiments on two Seq2Seq PLMs, BART and T5, demonstrate our framework achieves significant improvements on unconstrained and controlled positive reframing tasks.
• 摘要：与情绪迁移不同，积极重构寻求用积极表达取代消极观点，同时保留原始含义。随着预训练语言模型 (PLM) 的出现，通过微调 PLM 可以获得可接受的结果。尽管如此，生成流畅、多样且任务受限的重构文本仍然是一项重大挑战。为了解决这个问题，本文提出了一个多策略优化框架 (MSOF)。从积极重构的目标出发，我们首先设计积极情绪奖励和内容保存奖励，以鼓励模型在确保语义完整性和一致性的同时转换原始文本的消极表达。然后，引入不同的解码优化方法来提高文本生成的质量。最后，基于正向重构的建模公式，我们提出了一种多维重排序方法，进一步从策略一致性、文本相似性和流畅性三个维度选择候选句子。在两个 Seq2Seq PLM、BART 和 T5 上进行的大量实验表明，我们的框架在无约束和受控的正向重构任务上取得了显着的改进。

Title: The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communication

• Authors: Tom Kouwenhoven, Max Peeperkorn, Bram van Dijk, Tessa Verhoef
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17960
• Pdf URL: https://arxiv.org/pdf/2407.17960
• Copy Paste: [[2407.17960]] The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communication(https://arxiv.org/abs/2407.17960)
• Keywords: agent
• Abstract: Natural language has the universal properties of being compositional and grounded in reality. The emergence of linguistic properties is often investigated through simulations of emergent communication in referential games. However, these experiments have yielded mixed results compared to similar experiments addressing linguistic properties of human language. Here we address representational alignment as a potential contributing factor to these results. Specifically, we assess the representational alignment between agent image representations and between agent representations and input images. Doing so, we confirm that the emergent language does not appear to encode human-like conceptual visual features, since agent image representations drift away from inputs whilst inter-agent alignment increases. We moreover identify a strong relationship between inter-agent alignment and topographic similarity, a common metric for compositionality, and address its consequences. To address these issues, we introduce an alignment penalty that prevents representational drift but interestingly does not improve performance on a compositional discrimination task. Together, our findings emphasise the key role representational alignment plays in simulations of language emergence.
• 摘要：自然语言具有普遍的属性，即具有组合性和扎根于现实。语言属性的出现通常通过模拟指涉游戏中出现的交流来研究。然而，与解决人类语言语言属性的类似实验相比，这些实验产生了好坏参半的结果。在这里，我们将表征对齐作为这些结果的潜在贡献因素。具体来说，我们评估了代理图像表示之间以及代理表示与输入图像之间的表征对齐。这样做，我们确认新兴语言似乎没有编码类似人类的概念视觉特征，因为代理图像表示会偏离输入，而代理间对齐会增加。此外，我们还确定了代理间对齐与地形相似性之间的强关系，这是组合性的常用指标，并解决了其后果。为了解决这些问题，我们引入了一种对齐惩罚，它可以防止表征漂移，但有趣的是，它不会提高组合辨别任务的性能。总之，我们的研究结果强调了表征对齐在语言出现模拟中的关键作用。

Title: What does Kiki look like? Cross-modal associations between speech sounds and visual shapes in vision-and-language models

• Authors: Tessa Verhoef, Kiana Shahrasbi, Tom Kouwenhoven
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17974
• Pdf URL: https://arxiv.org/pdf/2407.17974
• Copy Paste: [[2407.17974]] What does Kiki look like? Cross-modal associations between speech sounds and visual shapes in vision-and-language models(https://arxiv.org/abs/2407.17974)
• Keywords: language model
• Abstract: Humans have clear cross-modal preferences when matching certain novel words to visual shapes. Evidence suggests that these preferences play a prominent role in our linguistic processing, language learning, and the origins of signal-meaning mappings. With the rise of multimodal models in AI, such as vision- and-language (VLM) models, it becomes increasingly important to uncover the kinds of visio-linguistic associations these models encode and whether they align with human representations. Informed by experiments with humans, we probe and compare four VLMs for a well-known human cross-modal preference, the bouba-kiki effect. We do not find conclusive evidence for this effect but suggest that results may depend on features of the models, such as architecture design, model size, and training details. Our findings inform discussions on the origins of the bouba-kiki effect in human cognition and future developments of VLMs that align well with human cross-modal associations.
• 摘要：在将某些新词与视觉形状匹配时，人类具有明显的跨模态偏好。有证据表明，这些偏好在我们的语言处理、语言学习以及信号意义映射的起源中发挥着重要作用。随着人工智能中多模态模型（如视觉和语言 (VLM) 模型）的兴起，揭示这些模型编码的视觉语言关联类型以及它们是否与人类表征一致变得越来越重要。根据对人类的实验，我们探究并比较了四个 VLM，以获得众所周知的人类跨模态偏好，即 bouba-kiki 效应。我们没有找到这种影响的确凿证据，但表明结果可能取决于模型的特征，例如架构设计、模型大小和训练细节。我们的研究结果为人类认知中 bouba-kiki 效应的起源以及与人类跨模态关联高度一致的 VLM 的未来发展提供了讨论。

Title: Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption

• Authors: Shi Luohe, Zhang Hongyi, Yao Yao, Li Zuchao, Zhao Hai
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.18003
• Pdf URL: https://arxiv.org/pdf/2407.18003
• Copy Paste: [[2407.18003]] Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption(https://arxiv.org/abs/2407.18003)
• Keywords: language model, gpt, llm, chat
• Abstract: Large Language Models (LLMs), epitomized by ChatGPT' s release in late 2022, have revolutionized various industries with their advanced language comprehension. However, their efficiency is challenged by the Transformer architecture' s struggle with handling long texts. KV-Cache has emerged as a pivotal solution to this issue, converting the time complexity of token generation from quadratic to linear, albeit with increased GPU memory overhead proportional to conversation length. With the development of the LLM community and academia, various KV-Cache compression methods have been proposed. In this review, we dissect the various properties of KV-Cache and elaborate on various methods currently used to optimize the KV-Cache space usage of LLMs. These methods span the pre-training phase, deployment phase, and inference phase, and we summarize the commonalities and differences among these methods. Additionally, we list some metrics for evaluating the long-text capabilities of large language models, from both efficiency and capability perspectives. Our review thus sheds light on the evolving landscape of LLM optimization, offering insights into future advancements in this dynamic field.
• 摘要：大型语言模型 (LLM) 以 2022 年底发布的 ChatGPT 为代表，以其先进的语言理解能力彻底改变了各个行业。然而，Transformer 架构在处理长文本方面存在困难，这给它们的效率带来了挑战。KV-Cache 已成为解决此问题的关键解决方案，它将 token 生成的时间复杂度从二次变为线性，尽管 GPU 内存开销会随着对话长度而增加。随着 LLM 社区和学术界的发展，人们提出了各种 KV-Cache 压缩方法。在本综述中，我们剖析了 KV-Cache 的各种属性，并详细阐述了当前用于优化 LLM 的 KV-Cache 空间使用的各种方法。这些方法涵盖了预训练阶段、部署阶段和推理阶段，我们总结了这些方法之间的共同点和差异。此外，我们从效率和能力的角度列出了一些用于评估大型语言模型长文本能力的指标。因此，我们的评论揭示了 LLM 优化的不断发展的前景，为这一充满活力的领域的未来发展提供了见解。

Title: Difficulty Estimation and Simplification of French Text Using LLMs

• Authors: Henri Jamet, Yash Raj Shrestha, Michalis Vlachos
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.18061
• Pdf URL: https://arxiv.org/pdf/2407.18061
• Copy Paste: [[2407.18061]] Difficulty Estimation and Simplification of French Text Using LLMs(https://arxiv.org/abs/2407.18061)
• Keywords: language model, llm
• Abstract: We leverage generative large language models for language learning applications, focusing on estimating the difficulty of foreign language texts and simplifying them to lower difficulty levels. We frame both tasks as prediction problems and develop a difficulty classification model using labeled examples, transfer learning, and large language models, demonstrating superior accuracy compared to previous approaches. For simplification, we evaluate the trade-off between simplification quality and meaning preservation, comparing zero-shot and fine-tuned performances of large language models. We show that meaningful text simplifications can be obtained with limited fine-tuning. Our experiments are conducted on French texts, but our methods are language-agnostic and directly applicable to other foreign languages.
• 摘要：我们利用生成式大型语言模型进行语言学习应用，重点是估计外语文本的难度并将其简化到较低的难度级别。我们将这两项任务都定义为预测问题，并使用标记示例、迁移学习和大型语言模型开发难度分类模型，与以前的方法相比，该模型表现出更高的准确性。对于简化，我们评估了简化质量和含义保留之间的权衡，比较了大型语言模型的零样本和微调性能。我们表明，只需进行有限的微调即可获得有意义的文本简化。我们的实验是在法语文本上进行的，但我们的方法与语言无关，可直接应用于其他外语。

Title: PEFT-U: Parameter-Efficient Fine-Tuning for User Personalization

• Authors: Christopher Clarke, Yuzhao Heng, Lingjia Tang, Jason Mars
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.18078
• Pdf URL: https://arxiv.org/pdf/2407.18078
• Copy Paste: [[2407.18078]] PEFT-U: Parameter-Efficient Fine-Tuning for User Personalization(https://arxiv.org/abs/2407.18078)
• Keywords: language model, gpt, llm, chat
• Abstract: The recent emergence of Large Language Models (LLMs) has heralded a new era of human-AI interaction. These sophisticated models, exemplified by Chat-GPT and its successors, have exhibited remarkable capabilities in language understanding. However, as these LLMs have undergone exponential growth, a crucial dimension that remains understudied is the personalization of these models. Large foundation models such as GPT-3 etc. focus on creating a universal model that serves a broad range of tasks and users. This approach emphasizes the model's generalization capabilities, treating users as a collective rather than as distinct individuals. While practical for many common applications, this one-size-fits-all approach often fails to address the rich tapestry of human diversity and individual needs. To explore this issue we introduce the PEFT-U Benchmark: a new dataset for building and evaluating NLP models for user personalization. \datasetname{} consists of a series of user-centered tasks containing diverse and individualized expressions where the preferences of users can potentially differ for the same input. Using PEFT-U, we explore the challenge of efficiently personalizing LLMs to accommodate user-specific preferences in the context of diverse user-centered tasks.
• 摘要：大型语言模型 (LLM) 的出现预示着人机交互新时代的到来。这些复杂的模型，以 Chat-GPT 及其后继者为代表，在语言理解方面表现出了非凡的能力。然而，随着这些 LLM 呈指数级增长，一个尚未得到充分研究的关键维度是这些模型的个性化。大型基础模型（如 GPT-3 等）专注于创建一个服务于广泛任务和用户的通用模型。这种方法强调模型的泛化能力，将用户视为一个集体，而不是不同的个体。虽然这种一刀切的方法适用于许多常见的应用，但往往无法解决人类多样性和个人需求的丰富问题。为了探讨这个问题，我们引入了 PEFT-U 基准：一个用于构建和评估用户个性化 NLP 模型的新数据集。\datasetname{} 由一系列以用户为中心的任务组成，其中包含多样化和个性化的表达，其中用户对相同输入的偏好可能会有所不同。使用 PEFT-U，我们探索了在多样化的以用户为中心的任务环境中有效个性化 LLM 以适应用户特定偏好的挑战。

Title: Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic

• Authors: Fakhraddin Alwajih, Gagan Bhatia, Muhammad Abdul-Mageed
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.18129
• Pdf URL: https://arxiv.org/pdf/2407.18129
• Copy Paste: [[2407.18129]] Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic(https://arxiv.org/abs/2407.18129)
• Keywords: language model, llm
• Abstract: Recent advancements have significantly enhanced the capabilities of Multimodal Large Language Models (MLLMs) in generating and understanding image-to-text content. Despite these successes, progress is predominantly limited to English due to the scarcity of high quality multimodal resources in other languages. This limitation impedes the development of competitive models in languages such as Arabic. To alleviate this situation, we introduce an efficient Arabic multimodal assistant, dubbed Dallah, that utilizes an advanced language model based on LLaMA-2 to facilitate multimodal interactions. Dallah demonstrates state-of-the-art performance in Arabic MLLMs. Through fine-tuning six Arabic dialects, Dallah showcases its capability to handle complex dialectal interactions incorporating both textual and visual elements. The model excels in two benchmark tests: one evaluating its performance on Modern Standard Arabic (MSA) and another specifically designed to assess dialectal responses. Beyond its robust performance in multimodal interaction tasks, Dallah has the potential to pave the way for further development of dialect-aware Arabic MLLMs.
• 摘要：最近的进展显著增强了多模态大型语言模型 (MLLM) 在生成和理解图像到文本内容方面的能力。尽管取得了这些成功，但由于其他语言中缺乏高质量的多模态资源，进展主要局限于英语。这种限制阻碍了阿拉伯语等语言中竞争模型的开发。为了缓解这种情况，我们推出了一款高效的阿拉伯语多模态助手，称为 Dallah，它利用基于 LLaMA-2 的高级语言模型来促进多模态交互。Dallah 在阿拉伯语 MLLM 中展示了最先进的性能。通过对六种阿拉伯语方言进行微调，Dallah 展示了其处理结合文本和视觉元素的复杂方言交互的能力。该模型在两项基准测试中表现出色：一项评估其在现代标准阿拉伯语 (MSA) 上的表现，另一项专门用于评估方言响应。除了在多模态交互任务中的强大性能外，Dallah 还有潜力为进一步开发方言感知阿拉伯语 MLLM 铺平道路。

Title: Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning

• Authors: Tianduo Wang, Shichen Li, Wei Lu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.18248
• Pdf URL: https://arxiv.org/pdf/2407.18248
• Copy Paste: [[2407.18248]] Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning(https://arxiv.org/abs/2407.18248)
• Keywords: language model, gpt, chain-of-thought
• Abstract: Effective training of language models (LMs) for mathematical reasoning tasks demands high-quality supervised fine-tuning data. Besides obtaining annotations from human experts, a common alternative is sampling from larger and more powerful LMs. However, this knowledge distillation approach can be costly and unstable, particularly when relying on closed-source, proprietary LMs like GPT-4, whose behaviors are often unpredictable. In this work, we demonstrate that the reasoning abilities of small-scale LMs can be enhanced through self-training, a process where models learn from their own outputs. We also show that the conventional self-training can be further augmented by a preference learning algorithm called Direct Preference Optimization (DPO). By integrating DPO into self-training, we leverage preference data to guide LMs towards more accurate and diverse chain-of-thought reasoning. We evaluate our method across various mathematical reasoning tasks using different base models. Our experiments show that this approach not only improves LMs' reasoning performance but also offers a more cost-effective and scalable solution compared to relying on large proprietary LMs.
• 摘要：有效训练用于数学推理任务的语言模型 (LM) 需要高质量的监督微调数据。除了从人类专家那里获得注释外，一种常见的替代方法是从更大、更强大的 LM 中采样。然而，这种知识提炼方法成本高昂且不稳定，尤其是在依赖 GPT-4 等闭源专有 LM 时，其行为往往不可预测。在这项工作中，我们证明了小规模 LM 的推理能力可以通过自我训练来增强，这是一个模型从自己的输出中学习的过程。我们还表明，传统的自我训练可以通过一种称为直接偏好优化 (DPO) 的偏好学习算法进一步增强。通过将 DPO 集成到自我训练中，我们利用偏好数据来引导 LM 实现更准确、更多样化的思路链推理。我们使用不同的基础模型在各种数学推理任务中评估我们的方法。我们的实验表明，与依赖大型专有 LM 相比，这种方法不仅可以提高 LM 的推理性能，而且还可以提供一种更具成本效益和可扩展的解决方案。