2024-07-25

Title: Educating LLMs like Human Students: Structure-aware Injection of Domain Knowledge

• Authors: Kai Liu, Ze Chen, Zhihang Fu, Rongxin Jiang, Fan Zhou, Yaowu Chen, Yue Wu, Jieping Ye
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.16724
• Pdf URL: https://arxiv.org/pdf/2407.16724
• Copy Paste: [[2407.16724]] Educating LLMs like Human Students: Structure-aware Injection of Domain Knowledge(https://arxiv.org/abs/2407.16724)
• Keywords: language model, llm, prompt
• Abstract: This paper presents a pioneering methodology, termed StructTuning, to efficiently transform foundation Large Language Models (LLMs) into domain specialists. It significantly minimizes the training corpus requirement to a mere 0.3% while achieving an impressive 50% of traditional knowledge injection performance. Our method is inspired by the educational processes for human students, particularly how structured domain knowledge from textbooks is absorbed and then applied to tackle real-world challenges through specific exercises. Based on this, we propose a novel two-stage knowledge injection strategy: Structure-aware Continual Pre-Training (SCPT) and Structure-aware Supervised Fine-Tuning (SSFT). In the SCPT phase, we organize the training data into an auto-generated taxonomy of domain knowledge, enabling LLMs to effectively memorize textual segments linked to specific expertise within the taxonomy's architecture. Subsequently, in the SSFT phase, we explicitly prompt models to reveal the underlying knowledge structure in their outputs, leveraging this structured domain insight to address practical problems adeptly. Our ultimate method has undergone extensive evaluations across model architectures and scales, using closed-book question-answering tasks on LongBench and MMedBench datasets. Remarkably, our method matches 50% of the improvement displayed by the state-of-the-art MMedLM2 on MMedBench, but with only 0.3% quantity of the training corpus. This breakthrough showcases the potential to scale up our StructTuning for stronger domain-specific LLMs. Code will be made public soon.
• 摘要：本文提出了一种开创性的方法，称为 StructTuning，可有效地将基础大型语言模型 (LLM) 转变为领域专家。它显著将训练语料库需求降至仅 0.3%，同时实现了传统知识注入性能的 50% 的惊人水平。我们的方法受到人类学生教育过程的启发，特别是如何吸收教科书中的结构化领域知识，然后通过特定练习应用于解决现实世界的挑战。基于此，我们提出了一种新颖的两阶段知识注入策略：结构感知持续预训练 (SCPT) 和结构感知监督微调 (SSFT)。在 SCPT 阶段，我们将训练数据组织成自动生成的领域知识分类法，使 LLM 能够有效地记忆与分类法架构内特定专业知识相关的文本段。随后，在 SSFT 阶段，我们明确提示模型在其输出中揭示底层知识结构，利用这种结构化领域洞察力巧妙地解决实际问题。我们的最终方法已在模型架构和规模上进行了广泛的评估，使用 LongBench 和 MMedBench 数据集上的闭卷问答任务。值得注意的是，我们的方法与最先进的 MMedLM2 在 MMedBench 上显示的改进相差 50%，但训练语料库的数量仅为 0.3%。这一突破展示了扩展我们的 StructTuning 以构建更强大的特定领​​域 LLM 的潜力。代码将很快公开。

Title: Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach

• Authors: Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.16833
• Pdf URL: https://arxiv.org/pdf/2407.16833
• Copy Paste: [[2407.16833]] Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach(https://arxiv.org/abs/2407.16833)
• Keywords: language model, gpt, llm, long context, retrieval augmented generation
• Abstract: Retrieval Augmented Generation (RAG) has been a powerful tool for Large Language Models (LLMs) to efficiently process overly lengthy contexts. However, recent LLMs like Gemini-1.5 and GPT-4 show exceptional capabilities to understand long contexts directly. We conduct a comprehensive comparison between RAG and long-context (LC) LLMs, aiming to leverage the strengths of both. We benchmark RAG and LC across various public datasets using three latest LLMs. Results reveal that when resourced sufficiently, LC consistently outperforms RAG in terms of average performance. However, RAG's significantly lower cost remains a distinct advantage. Based on this observation, we propose Self-Route, a simple yet effective method that routes queries to RAG or LC based on model self-reflection. Self-Route significantly reduces the computation cost while maintaining a comparable performance to LC. Our findings provide a guideline for long-context applications of LLMs using RAG and LC.
• 摘要：检索增强生成 (RAG) 一直是大型语言模型 (LLM) 有效处理过长上下文的强大工具。然而，最近的 LLM 如 Gemini-1.5 和 GPT-4 表现出直接理解长上下文的卓越能力。我们对 RAG 和长上下文 (LC) LLM 进行了全面比较，旨在充分利用两者的优势。我们使用三个最新的 LLM 在各种公共数据集上对 RAG 和 LC 进行基准测试。结果表明，在资源充足的情况下，LC 在平均性能方面始终优于 RAG。然而，RAG 显著降低的成本仍然是一个明显的优势。基于这一观察，我们提出了 Self-Route，这是一种简单而有效的方法，它基于模型自我反思将查询路由到 RAG 或 LC。Self-Route 显着降低了计算成本，同时保持了与 LC 相当的性能。我们的研究结果为使用 RAG 和 LC 的 LLM 的长上下文应用提供了指导。

Title: Generation Constraint Scaling Can Mitigate Hallucination

• Authors: Georgios Kollias, Payel Das, Subhajit Chaudhury
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.16908
• Pdf URL: https://arxiv.org/pdf/2407.16908
• Copy Paste: [[2407.16908]] Generation Constraint Scaling Can Mitigate Hallucination(https://arxiv.org/abs/2407.16908)
• Keywords: language model, llm, hallucination
• Abstract: Addressing the issue of hallucinations in large language models (LLMs) is a critical challenge. As the cognitive mechanisms of hallucination have been related to memory, here we explore hallucination for LLM that is enabled with explicit memory mechanisms. We empirically demonstrate that by simply scaling the readout vector that constrains generation in a memory-augmented LLM decoder, hallucination mitigation can be achieved in a training-free manner. Our method is geometry-inspired and outperforms a state-of-the-art LLM editing method on the task of generation of Wikipedia-like biography entries both in terms of generation quality and runtime complexity.
• 摘要：解决大型语言模型 (LLM) 中的幻觉问题是一项关键挑战。由于幻觉的认知机制与记忆有关，我们在此探索使用显性记忆机制实现的 LLM 幻觉。我们通过经验证明，只需缩放限制记忆增强 LLM 解码器中生成的读出向量，就可以以无需训练的方式实现幻觉缓解。我们的方法受几何启发，在生成类似维基百科的传记条目的任务上，无论是在生成质量还是运行时复杂度方面，都优于最先进的 LLM 编辑方法。

Title: Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning

• Authors: Yeongbin Seo, Dongha Lee, Jinyoung Yeo
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.16920
• Pdf URL: https://arxiv.org/pdf/2407.16920
• Copy Paste: [[2407.16920]] Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning(https://arxiv.org/abs/2407.16920)
• Keywords: language model, llm
• Abstract: Previous studies on continual knowledge learning (CKL) in large language models (LLMs) have predominantly focused on approaches such as regularization, architectural modifications, and rehearsal techniques to mitigate catastrophic forgetting. However, these methods naively inherit the inefficiencies of standard training procedures, indiscriminately applying uniform weight across all tokens, which can lead to unnecessary parameter updates and increased forgetting. To address these shortcomings, we propose a novel CKL approach termed Train-Attention-Augmented Language Model (TAALM), which enhances learning efficiency by dynamically predicting and applying weights to tokens based on their usefulness. This method employs a meta-learning framework that optimizes token importance predictions, facilitating targeted knowledge updates and minimizing forgetting. Also, we observe that existing benchmarks do not clearly exhibit the trade-off between learning and retaining, therefore we propose a new benchmark, \textsc{LAMA-ckl}, to address this issue. Through experiments conducted on both newly introduced and established CKL benchmarks, TAALM proves the state-of-the-art performance upon the baselines, and also shows synergistic compatibility when integrated with previous CKL approaches.
• 摘要：先前关于大型语言模型 (LLM) 中的持续知识学习 (CKL) 的研究主要集中在诸如正则化、架构修改和排练技术等方法上，以减轻灾难性遗忘。然而，这些方法天真地继承了标准训练程序的低效率，不加区分地对所有标记应用统一的权重，这可能导致不必要的参数更新和遗忘增加。为了解决这些缺点，我们提出了一种称为训练注意力增强语言模型 (TAALM) 的新型 CKL 方法，该方法通过根据标记的有用性动态预测和应用权重来提高学习效率。该方法采用元学习框架来优化标记重要性预测，促进有针对性的知识更新并最大限度地减少遗忘。此外，我们观察到现有基准没有清楚地展示学习和保留之间的权衡，因此我们提出了一个新的基准 \textsc{LAMA-ckl} 来解决这个问题。通过对新引入和已建立的 CKL 基准进行的实验，TAALM 证明了基线之上的最先进性能，并且与以前的 CKL 方法相结合时也表现出协同兼容性。

Title: ScholarChemQA: Unveiling the Power of Language Models in Chemical Research Question Answering

• Authors: Xiuying Chen, Tairan Wang, Taicheng Guo, Kehan Guo, Juexiao Zhou, Haoyang Li, Mingchen Zhuge, Jürgen Schmidhuber, Xin Gao, Xiangliang Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.16931
• Pdf URL: https://arxiv.org/pdf/2407.16931
• Copy Paste: [[2407.16931]] ScholarChemQA: Unveiling the Power of Language Models in Chemical Research Question Answering(https://arxiv.org/abs/2407.16931)
• Keywords: language model, llm
• Abstract: Question Answering (QA) effectively evaluates language models' reasoning and knowledge depth. While QA datasets are plentiful in areas like general domain and biomedicine, academic chemistry is less explored. Chemical QA plays a crucial role in both education and research by effectively translating complex chemical information into readily understandable format. Addressing this gap, we introduce ScholarChemQA, a large-scale QA dataset constructed from chemical papers. This dataset reflects typical real-world challenges, including an imbalanced data distribution and a substantial amount of unlabeled data that can be potentially useful. Correspondingly, we introduce a QAMatch model, specifically designed to effectively answer chemical questions by fully leveraging our collected data. We first address the issue of imbalanced label distribution by re-weighting the instance-wise loss based on the inverse frequency of each class, ensuring minority classes are not dominated by majority ones during optimization. Next, we utilize the unlabeled data to enrich the learning process, generating a variety of augmentations based on a SoftMix operation and ensuring their predictions align with the same target, i.e., pseudo-labels. To ensure the quality of the pseudo-labels, we propose a calibration procedure aimed at closely aligning the pseudo-label estimates of individual samples with a desired ground truth distribution. Experiments show that our QAMatch significantly outperforms the recent similar-scale baselines and Large Language Models (LLMs) not only on our ScholarChemQA dataset but also on four benchmark datasets. We hope our benchmark and model can facilitate and promote more research on chemical QA.
• 摘要：问答 (QA) 可以有效评估语言模型的推理和知识深度。虽然 QA 数据集在通用领域和生物医学等领域非常丰富，但学术化学领域的探索较少。化学 QA 可以有效地将复杂的化学信息转化为易于理解的格式，在教育和研究中发挥着至关重要的作用。为了解决这一差距，我们推出了 ScholarChemQA，这是一个由化学论文构建的大规模 QA 数据集。该数据集反映了典型的现实世界挑战，包括不平衡的数据分布和大量可能有用的未标记数据。相应地，我们引入了 QAMatch 模型，该模型专门设计用于通过充分利用我们收集的数据来有效回答化学问题。我们首先通过根据每个类别的逆频率重新加权实例损失来解决标签分布不平衡的问题，确保在优化过程中少数类别不会被多数类别所主导。接下来，我们利用未标记的数据来丰富学习过程，基于 SoftMix 操作生成各种增强，并确保它们的预测与同一目标（即伪标签）一致。为了确保伪标签的质量，我们提出了一种校准程序，旨在将单个样本的伪标签估计值与所需的基本事实分布紧密对齐。实验表明，我们的 QAMatch 不仅在我们的 ScholarChemQA 数据集上，而且在四个基准数据集上都明显优于最近的类似规模基线和大型语言模型 (LLM)。我们希望我们的基准和模型能够促进和推动对化学 QA 的更多研究。

Title: Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model

• Authors: Jaewoong Choi, Janghyeok Yoon, Changyong Lee
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.16939
• Pdf URL: https://arxiv.org/pdf/2407.16939
• Copy Paste: [[2407.16939]] Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model(https://arxiv.org/abs/2407.16939)
• Keywords: language model
• Abstract: Despite the usefulness of machine learning approaches for the early screening of potential breakthrough technologies, their practicality is often hindered by opaque models. To address this, we propose an interpretable machine learning approach to predicting future citation counts from patent texts using a patent-specific hierarchical attention network (PatentHAN) model. Central to this approach are (1) a patent-specific pre-trained language model, capturing the meanings of technical words in patent claims, (2) a hierarchical network structure, enabling detailed analysis at the claim level, and (3) a claim-wise self-attention mechanism, revealing pivotal claims during the screening process. A case study of 35,376 pharmaceutical patents demonstrates the effectiveness of our approach in early screening of potential breakthrough technologies while ensuring interpretability. Furthermore, we conduct additional analyses using different language models and claim types to examine the robustness of the approach. It is expected that the proposed approach will enhance expert-machine collaboration in identifying breakthrough technologies, providing new insight derived from text mining into technological value.
• 摘要：尽管机器学习方法对于潜在突破性技术的早期筛选非常有用，但其实用性往往受到不透明模型的阻碍。为了解决这个问题，我们提出了一种可解释的机器学习方法，使用专利特定的分层注意力网络 (PatentHAN) 模型从专利文本中预测未来的引用计数。该方法的核心是 (1) 专利特定的预训练语言模型，捕捉专利权利要求中技术词汇的含义，(2) 分层网络结构，能够在权利要求层面进行详细分析，以及 (3) 权利要求自注意力机制，在筛选过程中揭示关键权利要求。对 35,376 项制药专利的案例研究证明了我们的方法在早期筛选潜在突破性技术方面的有效性，同时确保了可解释性。此外，我们使用不同的语言模型和权利要求类型进行了额外的分析，以检验该方法的稳健性。预计所提出的方法将加强专家与机器在识别突破性技术方面的协作，为从文本挖掘中获得的技术价值新见解提供新见解。

Title: Towards Transfer Unlearning: Empirical Evidence of Cross-Domain Bias Mitigation

• Authors: Huimin Lu, Masaru Isonuma, Junichiro Mori, Ichiro Sakata
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.16951
• Pdf URL: https://arxiv.org/pdf/2407.16951
• Copy Paste: [[2407.16951]] Towards Transfer Unlearning: Empirical Evidence of Cross-Domain Bias Mitigation(https://arxiv.org/abs/2407.16951)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) often inherit biases from vast amounts of training corpora. Traditional debiasing methods, while effective to some extent, do not completely eliminate memorized biases and toxicity in LLMs. In this paper, we study an unlearning-based approach to debiasing in LLMs by performing gradient ascent on hate speech against minority groups, i.e., minimizing the likelihood of biased or toxic content. Specifically, we propose a mask language modeling unlearning technique, which unlearns the harmful part of the text. This method enables LLMs to selectively forget and disassociate from biased and harmful content. Experimental results demonstrate the effectiveness of our approach in diminishing bias while maintaining the language modeling abilities. Surprisingly, the results also unveil an unexpected potential for cross-domain transfer unlearning: debiasing in one bias form (e.g. gender) may contribute to mitigating others (e.g. race and religion).
• 摘要：大型语言模型 (LLM) 通常会从大量训练语料库中继承偏见。传统的去偏见方法虽然在一定程度上有效，但并不能完全消除 LLM 中记忆的偏见和毒性。在本文中，我们研究了一种基于反学习的 LLM 去偏见方法，即对针对少数群体的仇恨言论进行梯度上升，即最大限度地降低有偏见或有害内容的可能性。具体来说，我们提出了一种掩码语言建模反学习技术，它可以反学习文本中的有害部分。这种方法使 LLM 能够有选择地忘记和脱离有偏见和有害的内容。实验结果证明了我们的方法在减少偏见的同时保持语言建模能力的有效性。令人惊讶的是，结果还揭示了跨领域转移反学习的意外潜力：消除一种偏见形式（例如性别）的偏见可能有助于减轻其他偏见（例如种族和宗教）。

Title: Towards Aligning Language Models with Textual Feedback

• Authors: Saüc Abadal Lloret, Shehzaad Dhuliawala, Keerthiram Murugesan, Mrinmaya Sachan
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.16970
• Pdf URL: https://arxiv.org/pdf/2407.16970
• Copy Paste: [[2407.16970]] Towards Aligning Language Models with Textual Feedback(https://arxiv.org/abs/2407.16970)
• Keywords: language model, llm
• Abstract: We present ALT (ALignment with Textual feedback), an approach that aligns language models with user preferences expressed in text. We argue that text offers greater expressiveness, enabling users to provide richer feedback than simple comparative preferences and this richer feedback can lead to more efficient and effective alignment. ALT aligns the model by conditioning its generation on the textual feedback. Our method relies solely on language modeling techniques and requires minimal hyper-parameter tuning, though it still presents the main benefits of RL-based alignment algorithms and can effectively learn from textual feedback. We explore the efficacy and efficiency of textual feedback across different tasks such as toxicity reduction, summarization, and dialog response generation. We find that ALT outperforms PPO for the task of toxicity reduction while being able to match its performance on summarization with only 20% of the samples. We also explore how ALT can be used with feedback provided by an existing LLM where we explore an LLM providing constrained and unconstrained textual feedback. We also outline future directions to align models with natural language feedback.
• 摘要：我们提出了 ALT（与文本反馈对齐），这是一种将语言模型与文本中表达的用户偏好对齐的方法。我们认为文本提供了更大的表现力，使用户能够提供比简单的比较偏好更丰富的反馈，这种更丰富的反馈可以带来更高效、更有效的对齐。ALT 通过根据文本反馈调整模型的生成来对齐模型。我们的方法完全依赖于语言建模技术，并且需要最少的超参数调整，但它仍然具有基于 RL 的对齐算法的主要优点，并且可以有效地从文本反馈中学习。我们探索了文本反馈在不同任务（例如毒性降低、总结和对话响应生成）中的有效性和效率。我们发现 ALT 在毒性降低任务中的表现优于 PPO，同时仅用 20% 的样本就能匹配其总结性能。我们还探讨了如何将 ALT 与现有 LLM 提供的反馈一起使用，其中我们探索了提供受约束和不受约束的文本反馈的 LLM。我们还概述了将模型与自然语言反馈对齐的未来方向。

Title: Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspective

• Authors: Yujian Liu, Yang Zhang, Tommi Jaakkola, Shiyu Chang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.16997
• Pdf URL: https://arxiv.org/pdf/2407.16997
• Copy Paste: [[2407.16997]] Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspective(https://arxiv.org/abs/2407.16997)
• Keywords: llm
• Abstract: This paper investigates Who's Harry Potter (WHP), a pioneering yet insufficiently understood method for LLM unlearning. We explore it in two steps. First, we introduce a new task of LLM targeted unlearning, where given an unlearning target (e.g., a person) and some unlearning documents, we aim to unlearn only the information about the target, rather than everything in the unlearning documents. We further argue that a successful unlearning should satisfy criteria such as not outputting gibberish, not fabricating facts about the unlearning target, and not releasing factual information under jailbreak attacks. Second, we construct a causal intervention framework for targeted unlearning, where the knowledge of the unlearning target is modeled as a confounder between LLM input and output, and the unlearning process as a deconfounding process. This framework justifies and extends WHP, deriving a simple unlearning algorithm that includes WHP as a special case. Experiments on existing and new datasets show that our approach, without explicitly optimizing for the aforementioned criteria, achieves competitive performance in all of them. Our code is available at this https URL.
• 摘要：本文探讨了《谁是哈利波特》（WHP），这是一种开创性的但尚未得到充分理解的 LLM 反学习方法。我们分两个步骤进行探索。首先，我们引入了一个新的 LLM 目标反学习任务，给定一个反学习目标（例如，一个人）和一些反学习文档，我们的目标是只反学习有关目标的信息，而不是反学习文档中的所有内容。我们进一步认为，成功的反学习应该满足一些标准，例如不输出胡言乱语、不捏造关于反学习目标的事实、不在越狱攻击下泄露事实信息。其次，我们为目标反学习构建了一个因果干预框架，其中反学习目标的知识被建模为 LLM 输入和输出之间的混杂因素，反学习过程被建模为去混杂过程。该框架证明并扩展了 WHP，推导出一个简单的反学习算法，其中包括 WHP 作为特例。在现有和新数据集上进行的实验表明，我们的方法无需针对上述标准进行明确优化，在所有数据集上都取得了具有竞争力的性能。我们的代码可在此 https URL 上找到。

Title: Unveiling In-Context Learning: A Coordinate System to Understand Its Working Mechanism

• Authors: Anhao Zhao, Fanghua Ye, Jinlan Fu, Xiaoyu Shen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17011
• Pdf URL: https://arxiv.org/pdf/2407.17011
• Copy Paste: [[2407.17011]] Unveiling In-Context Learning: A Coordinate System to Understand Its Working Mechanism(https://arxiv.org/abs/2407.17011)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) exhibit remarkable in-context learning (ICL) capabilities. However, the underlying working mechanism of ICL remains poorly understood. Recent research presents two conflicting views on ICL: One attributes it to LLMs' inherent ability of task recognition, deeming label correctness and shot numbers of demonstrations as not crucial; the other emphasizes the impact of similar examples in the demonstrations, stressing the need for label correctness and more shots. In this work, we provide a Two-Dimensional Coordinate System that unifies both views into a systematic framework. The framework explains the behavior of ICL through two orthogonal variables: whether LLMs can recognize the task and whether similar examples are presented in the demonstrations. We propose the peak inverse rank metric to detect the task recognition ability of LLMs and study LLMs' reactions to different definitions of similarity. Based on these, we conduct extensive experiments to elucidate how ICL functions across each quadrant on multiple representative classification tasks. Finally, we extend our analyses to generation tasks, showing that our coordinate system can also be used to interpret ICL for generation tasks effectively.
• 摘要：大型语言模型 (LLM) 表现出卓越的上下文学习 (ICL) 能力。然而，ICL 的底层工作机制仍然不太清楚。最近的研究对 ICL 提出了两种相互矛盾的观点：一种观点将其归因于 LLM 固有的任务识别能力，认为标签正确性和演示的镜头数量并不重要；另一种观点强调演示中类似示例的影响，强调标签正确性和更多镜头的必要性。在这项工作中，我们提供了一个二维坐标系，将这两种观点统一为一个系统框架。该框架通过两个正交变量来解释 ICL 的行为：LLM 是否可以识别任务以及演示中是否呈现了类似的示例。我们提出了峰值逆秩度量来检测 LLM 的任务识别能力，并研究 LLM 对不同相似性定义的反应。基于这些，我们进行了广泛的实验，以阐明 ICL 在多个代表性分类任务中在每个象限中如何发挥作用。最后，我们将分析扩展到生成任务，表明我们的坐标系也可以用于有效地解释生成任务的 ICL。

Title: Can Language Models Evaluate Human Written Text? Case Study on Korean Student Writing for Education

• Authors: Seungyoon Kim, Seungone Kim
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17022
• Pdf URL: https://arxiv.org/pdf/2407.17022
• Copy Paste: [[2407.17022]] Can Language Models Evaluate Human Written Text? Case Study on Korean Student Writing for Education(https://arxiv.org/abs/2407.17022)
• Keywords: language model, gpt, llm
• Abstract: Large language model (LLM)-based evaluation pipelines have demonstrated their capability to robustly evaluate machine-generated text. Extending this methodology to assess human-written text could significantly benefit educational settings by providing direct feedback to enhance writing skills, although this application is not straightforward. In this paper, we investigate whether LLMs can effectively assess human-written text for educational purposes. We collected 100 texts from 32 Korean students across 15 types of writing and employed GPT-4-Turbo to evaluate them using grammaticality, fluency, coherence, consistency, and relevance as criteria. Our analyses indicate that LLM evaluators can reliably assess grammaticality and fluency, as well as more objective types of writing, though they struggle with other criteria and types of writing. We publicly release our dataset and feedback.
• 摘要：基于大型语言模型 (LLM) 的评估流程已证明其能够稳健地评估机器生成的文本。将这种方法扩展到评估人工编写的文本可以极大地造福教育环境，因为它可以提供直接反馈来提高写作技能，尽管这种应用并不简单。在本文中，我们研究了 LLM 是否可以有效地评估人工编写的文本以用于教育目的。我们从 32 名韩国学生那里收集了 15 种写作类型的 100 篇文本，并使用 GPT-4-Turbo 以语法性、流畅性、连贯性、一致性和相关性为标准对其进行评估。我们的分析表明，LLM 评估人员可以可靠地评估语法性和流畅性，以及更客观的写作类型，尽管他们在其他标准和写作类型方面有困难。我们公开发布了我们的数据集和反馈。

Title: From Internal Conflict to Contextual Adaptation of Language Models

• Authors: Sara Vera Marjanović, Haeun Yu, Pepa Atanasova, Maria Maistro, Christina Lioma, Isabelle Augenstein
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17023
• Pdf URL: https://arxiv.org/pdf/2407.17023
• Copy Paste: [[2407.17023]] From Internal Conflict to Contextual Adaptation of Language Models(https://arxiv.org/abs/2407.17023)
• Keywords: language model
• Abstract: Knowledge-intensive language understanding tasks require Language Models (LMs) to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated knowledge. Nevertheless, studies indicate that LMs often ignore the provided context as it can conflict with the pre-existing LM's memory learned during pre-training. Moreover, conflicting knowledge can already be present in the LM's parameters, termed intra-memory conflict. Existing works have studied the two types of knowledge conflicts only in isolation. We conjecture that the (degree of) intra-memory conflicts can in turn affect LM's handling of context-memory conflicts. To study this, we introduce the DYNAMICQA dataset, which includes facts with a temporal dynamic nature where a fact can change with a varying time frequency and disputable dynamic facts, which can change depending on the viewpoint. DYNAMICQA is the first to include real-world knowledge conflicts and provide context to study the link between the different types of knowledge conflicts. With the proposed dataset, we assess the use of uncertainty for measuring the intra-memory conflict and introduce a novel Coherent Persuasion (CP) score to evaluate the context's ability to sway LM's semantic output. Our extensive experiments reveal that static facts, which are unlikely to change, are more easily updated with additional context, relative to temporal and disputable facts.
• 摘要：知识密集型语言理解任务需要语言模型 (LM) 整合相关上下文，以减轻其固有的弱点，例如不完整或过时的知识。然而，研究表明，LM 通常会忽略提供的上下文，因为它可能与预训练期间学习的现有 LM 记忆相冲突。此外，冲突的知识可能已经存在于 LM 的参数中，称为内存内冲突。现有研究仅孤立地研究了这两种类型的知识冲突。我们推测内存内冲突的程度反过来会影响 LM 对上下文内存冲突的处理。为了研究这一点，我们引入了 DYNAMICQA 数据集，其中包括具有时间动态性质的事实，其中事实可以随着时间频率的变化而变化，以及有争议的动态事实，这些事实可能会根据观点而变化。DYNAMICQA 是第一个包含现实世界知识冲突并提供背景来研究不同类型知识冲突之间联系的数据集。利用建议的数据集，我们评估了使用不确定性来衡量记忆内冲突的效果，并引入了新的连贯说服 (CP) 分数来评估上下文影响 LM 语义输出的能力。我们进行了广泛的实验，结果表明，相对于暂时性和有争议的事实，不太可能发生变化的静态事实更容易通过附加上下文进行更新。

Title: SAFETY-J: Evaluating Safety with Critique

• Authors: Yixiu Liu, Yuxiang Zheng, Shijie Xia, Yuan Guo, Jiajun Li, Yi Tu, Chaoling Song, Pengfei Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17075
• Pdf URL: https://arxiv.org/pdf/2407.17075
• Copy Paste: [[2407.17075]] SAFETY-J: Evaluating Safety with Critique(https://arxiv.org/abs/2407.17075)
• Keywords: language model, llm
• Abstract: The deployment of Large Language Models (LLMs) in content generation raises significant safety concerns, particularly regarding the transparency and interpretability of content evaluations. Current methods, primarily focused on binary safety classifications, lack mechanisms for detailed critique, limiting their utility for model improvement and user trust. To address these limitations, we introduce SAFETY-J, a bilingual generative safety evaluator for English and Chinese with critique-based judgment. SAFETY-J utilizes a robust training dataset that includes diverse dialogues and augmented query-response pairs to assess safety across various scenarios comprehensively. We establish an automated meta-evaluation benchmark that objectively assesses the quality of critiques with minimal human intervention, facilitating scalable and continuous improvement. Additionally, SAFETY-J employs an iterative preference learning technique to dynamically refine safety assessments based on meta-evaluations and critiques. Our evaluations demonstrate that SAFETY-J provides more nuanced and accurate safety evaluations, thereby enhancing both critique quality and predictive reliability in complex content scenarios. To facilitate further research and application, we will open-source SAFETY-J's training protocols, datasets, and code.
• 摘要：在内容生成中部署大型语言模型 (LLM) 引发了重大的安全问题，尤其是关于内容评估的透明度和可解释性。当前的方法主要侧重于二元安全分类，缺乏详细批评的机制，限制了它们在模型改进和用户信任方面的效用。为了解决这些限制，我们推出了 SAFETY-J，这是一种基于批评判断的英语和中文双语生成安全评估器。SAFETY-J 利用包含各种对话和增强的查询-响应对的强大训练数据集来全面评估各种场景的安全性。我们建立了一个自动化的元评估基准，以最少的人为干预客观地评估批评的质量，促进可扩展和持续的改进。此外，SAFETY-J 采用迭代偏好学习技术，根据元评估和批评动态改进安全评估。我们的评估表明，SAFETY-J 提供了更细致入微、更准确的安全评估，从而提高了复杂内容场景中的批评质量和预测可靠性。为了促进进一步的研究和应用，我们将开源SAFETY-J的训练协议、数据集和代码。

Title: Behavioral Testing: Can Large Language Models Implicitly Resolve Ambiguous Entities?

• Authors: Anastasiia Sedova, Robert Litschko, Diego Frassinelli, Benjamin Roth, Barbara Plank
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.17125
• Pdf URL: https://arxiv.org/pdf/2407.17125
• Copy Paste: [[2407.17125]] Behavioral Testing: Can Large Language Models Implicitly Resolve Ambiguous Entities?(https://arxiv.org/abs/2407.17125)
• Keywords: language model, llm, prompt
• Abstract: One of the major aspects contributing to the striking performance of large language models (LLMs) is the vast amount of factual knowledge accumulated during pre-training. Yet, many LLMs suffer from self-inconsistency, which raises doubts about their trustworthiness and reliability. In this paper, we focus on entity type ambiguity and analyze current state-of-the-art LLMs for their proficiency and consistency in applying their factual knowledge when prompted for entities under ambiguity. To do so, we propose an evaluation protocol that disentangles knowing from applying knowledge, and test state-of-the-art LLMs on 49 entities. Our experiments reveal that LLMs perform poorly with ambiguous prompts, achieving only 80% accuracy. Our results further demonstrate systematic discrepancies in LLM behavior and their failure to consistently apply information, indicating that the models can exhibit knowledge without being able to utilize it, significant biases for preferred readings, as well as self inconsistencies. Our study highlights the importance of handling entity ambiguity in future for more trustworthy LLMs
• 摘要：大型语言模型 (LLM) 表现出色的主要因素之一是预训练期间积累的大量事实知识。然而，许多 LLM 存在自我不一致的问题，这引发了人们对其可信度和可靠性的怀疑。在本文中，我们重点关注实体类型歧义，并分析当前最先进的 LLM 在提示歧义实体时应用事实知识的熟练程度和一致性。为此，我们提出了一种将知识与应用知识区分开来的评估协议，并在 49 个实体上测试了最先进的 LLM。我们的实验表明，LLM 在歧义提示下表现不佳，准确率仅为 80%。我们的结果进一步证明了 LLM 行为的系统性差异及其无法一致地应用信息，表明模型可以展示知识而不能利用它，对偏好阅读存在显著偏见，以及自我不一致。我们的研究强调了未来处理实体歧义问题对于提高法学硕士可信度的重要性

Title: SDoH-GPT: Using Large Language Models to Extract Social Determinants of Health (SDoH)

• Authors: Bernardo Consoli, Xizhi Wu, Song Wang, Xinyu Zhao, Yanshan Wang, Justin Rousseau, Tom Hartvigsen, Li Shen, Huanmei Wu, Yifan Peng, Qi Long, Tianlong Chen, Ying Ding
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17126
• Pdf URL: https://arxiv.org/pdf/2407.17126
• Copy Paste: [[2407.17126]] SDoH-GPT: Using Large Language Models to Extract Social Determinants of Health (SDoH)(https://arxiv.org/abs/2407.17126)
• Keywords: language model, gpt, llm
• Abstract: Extracting social determinants of health (SDoH) from unstructured medical notes depends heavily on labor-intensive annotations, which are typically task-specific, hampering reusability and limiting sharing. In this study we introduced SDoH-GPT, a simple and effective few-shot Large Language Model (LLM) method leveraging contrastive examples and concise instructions to extract SDoH without relying on extensive medical annotations or costly human intervention. It achieved tenfold and twentyfold reductions in time and cost respectively, and superior consistency with human annotators measured by Cohen's kappa of up to 0.92. The innovative combination of SDoH-GPT and XGBoost leverages the strengths of both, ensuring high accuracy and computational efficiency while consistently maintaining 0.90+ AUROC scores. Testing across three distinct datasets has confirmed its robustness and accuracy. This study highlights the potential of leveraging LLMs to revolutionize medical note classification, demonstrating their capability to achieve highly accurate classifications with significantly reduced time and cost.
• 摘要：从非结构化医疗笔记中提取健康的社会决定因素 (SDoH) 在很大程度上依赖于劳动密集型的注释，这些注释通常是特定于任务的，妨碍了可重用性并限制了共享。在本研究中，我们引入了 SDoH-GPT，这是一种简单有效的少样本大型语言模型 (LLM) 方法，利用对比示例和简洁的说明来提取 SDoH，而无需依赖广泛的医疗注释或昂贵的人工干预。它分别将时间和成本减少了 10 倍和 20 倍，并且与人类注释者的一致性更高，Cohen's kappa 测量结果高达 0.92。SDoH-GPT 和 XGBoost 的创新组合利用了两者的优势，确保了高精度和计算效率，同时始终保持 0.90+ AUROC 分数。在三个不同的数据集上进行的测试证实了它的稳健性和准确性。这项研究强调了利用 LLM 彻底改变医疗笔记分类的潜力，展示了它们能够以显着减少的时间和成本实现高度准确的分类。

Title: SimCT: A Simple Consistency Test Protocol in LLMs Development Lifecycle

• Authors: Fufangchen Zhao, Guoqiang Jin, Rui Zhao, Jiangheng Huang, Fei Tan
• Subjects: cs.CL, cs.SE
• Abstract URL: https://arxiv.org/abs/2407.17150
• Pdf URL: https://arxiv.org/pdf/2407.17150
• Copy Paste: [[2407.17150]] SimCT: A Simple Consistency Test Protocol in LLMs Development Lifecycle(https://arxiv.org/abs/2407.17150)
• Keywords: language model, llm
• Abstract: In this work, we report our efforts to advance the standard operation procedure of developing Large Language Models (LLMs) or LLMs-based systems or services in industry. We introduce the concept of Large Language Model Development Lifecycle (LDLC) and then highlight the importance of consistency test in ensuring the delivery quality. The principled solution of consistency test, however, is usually overlooked by industrial practitioners and not urgent in academia, and current practical solutions are insufficiently rigours and labor-intensive. We thus propose a simple yet effective consistency test protocol, named SimCT. SimCT is mainly to proactively check the consistency across different development stages of "bare metal" LLMs or associated services without accessing the model artifacts, in an attempt to expedite the delivery by reducing the back-and-forth alignment communications among multiple teams involved in different development stages. Specifically, SimCT encompasses response-wise and model-wise tests. We implement the protocol with LightGBM and Student's t-test for two components respectively, and perform extensive experiments to substantiate the effectiveness of SimCT and the involved components.
• 摘要：在本文中，我们报告了我们为推进行业中开发大型语言模型 (LLM) 或基于 LLM 的系统或服务的标准操作流程所做的努力。我们引入了大型语言模型开发生命周期 (LDLC) 的概念，然后强调了一致性测试在确保交付质量方面的重要性。然而，一致性测试的原则性解决方案通常被行业从业者忽视，在学术界并不紧迫，目前的实用解决方案不够严格且劳动密集。因此，我们提出了一个简单而有效的一致性测试协议，称为 SimCT。SimCT 主要是在不访问模型工件的情况下主动检查“裸机”LLM 或相关服务的不同开发阶段之间的一致性，试图通过减少参与不同开发阶段的多个团队之间的来回协调沟通来加快交付速度。具体来说，SimCT 包括响应和模型测试。我们分别对两个组件使用 LightGBM 和学生 t 检验实现该协议，并进行了大量实验来证实 SimCT 和所涉及组件的有效性。

Title: Label Alignment and Reassignment with Generalist Large Language Model for Enhanced Cross-Domain Named Entity Recognition

• Authors: Ke Bao, Chonghuan Yang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17344
• Pdf URL: https://arxiv.org/pdf/2407.17344
• Copy Paste: [[2407.17344]] Label Alignment and Reassignment with Generalist Large Language Model for Enhanced Cross-Domain Named Entity Recognition(https://arxiv.org/abs/2407.17344)
• Keywords: language model, gpt, chat
• Abstract: Named entity recognition on the in-domain supervised and few-shot settings have been extensively discussed in the NLP community and made significant progress. However, cross-domain NER, a more common task in practical scenarios, still poses a challenge for most NER methods. Previous research efforts in that area primarily focus on knowledge transfer such as correlate label information from source to target domains but few works pay attention to the problem of label conflict. In this study, we introduce a label alignment and reassignment approach, namely LAR, to address this issue for enhanced cross-domain named entity recognition, which includes two core procedures: label alignment between source and target domains and label reassignment for type inference. The process of label reassignment can significantly be enhanced by integrating with an advanced large-scale language model such as ChatGPT. We conduct an extensive range of experiments on NER datasets involving both supervised and zero-shot scenarios. Empirical experimental results demonstrate the validation of our method with remarkable performance under the supervised and zero-shot out-of-domain settings compared to SOTA methods.
• 摘要：在 NLP 社区中，域内监督和少样本设置的命名实体识别已经得到了广泛的讨论，并取得了重大进展。然而，跨域 NER 是实际场景中更常见的任务，对大多数 NER 方法来说仍然是一个挑战。该领域的先前研究主要集中于知识迁移，例如将标签信息从源域关联到目标域，但很少有研究关注标签冲突问题。在本研究中，我们引入了一种标签对齐和重新分配方法，即 LAR，来解决此问题以增强跨域命名实体识别，它包括两个核心过程：源域和目标域之间的标签对齐以及用于类型推断的标签重新分配。通过与 ChatGPT 等先进的大规模语言模型集成，可以显著增强标签重新分配过程。我们在涉及监督和零样本场景的 NER 数据集上进行了广泛的实验。实证实验结果证明了我们的方法的有效性，与 SOTA 方法相比，在监督和零样本域外设置下具有卓越的性能。

Title: Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching

• Authors: Yuyang Ding, Hanglei Hu, Jie Zhou, Qin Chen, Bo Jiang, Liang He
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17349
• Pdf URL: https://arxiv.org/pdf/2407.17349
• Copy Paste: [[2407.17349]] Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching(https://arxiv.org/abs/2407.17349)
• Keywords: language model, llm, chain-of-thought
• Abstract: With the introduction of large language models (LLMs), automatic math reasoning has seen tremendous success. However, current methods primarily focus on providing solutions or using techniques like Chain-of-Thought to enhance problem-solving accuracy. In this paper, we focus on improving the capability of mathematics teaching via a Socratic teaching-based LLM (\texttt{SocraticLLM}), which guides learners toward profound thinking with clarity and self-discovery via conversation. We collect and release a high-quality mathematical teaching dataset, named \texttt{SocraticMATH}, which provides Socratic-style conversations of problems with extra knowledge. Also, we propose a knowledge-enhanced LLM as a strong baseline to generate reliable responses with review, guidance/heuristic, rectification, and summarization. Experimental results show the great advantages of \texttt{SocraticLLM} by comparing it with several strong generative models. The codes and datasets are available on \url{this https URL}.
• 摘要：随着大型语言模型 (LLM) 的引入，自动数学推理取得了巨大的成功。然而，当前的方法主要侧重于提供解决方案或使用诸如思维链之类的技术来提高解决问题的准确性。在本文中，我们专注于通过基于苏格拉底教学的 LLM (\texttt{SocraticLLM}) 提高数学教学能力，该模型引导学习者通过对话清晰地进行深刻思考和自我发现。我们收集并发布了一个名为 \texttt{SocraticMATH} 的高质量数学教学数据集，它提供了带有额外知识的苏格拉底式问题对话。此外，我们提出了一个知识增强型 LLM 作为强大的基线，以生成具有复习、指导/启发式、纠正和总结的可靠响应。通过将 \texttt{SocraticLLM} 与几个强大的生成模型进行比较，实验结果显示了 \texttt{SocraticLLM} 的巨大优势。代码和数据集可在 \url{此 https URL} 上找到。

Title: A Comprehensive Approach to Misspelling Correction with BERT and Levenshtein Distance

• Authors: Amirreza Naziri, Hossein Zeinali
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.17383
• Pdf URL: https://arxiv.org/pdf/2407.17383
• Copy Paste: [[2407.17383]] A Comprehensive Approach to Misspelling Correction with BERT and Levenshtein Distance(https://arxiv.org/abs/2407.17383)
• Keywords: language model
• Abstract: Writing, as an omnipresent form of human communication, permeates nearly every aspect of contemporary life. Consequently, inaccuracies or errors in written communication can lead to profound consequences, ranging from financial losses to potentially life-threatening situations. Spelling mistakes, among the most prevalent writing errors, are frequently encountered due to various factors. This research aims to identify and rectify diverse spelling errors in text using neural networks, specifically leveraging the Bidirectional Encoder Representations from Transformers (BERT) masked language model. To achieve this goal, we compiled a comprehensive dataset encompassing both non-real-word and real-word errors after categorizing different types of spelling mistakes. Subsequently, multiple pre-trained BERT models were employed. To ensure optimal performance in correcting misspelling errors, we propose a combined approach utilizing the BERT masked language model and Levenshtein distance. The results from our evaluation data demonstrate that the system presented herein exhibits remarkable capabilities in identifying and rectifying spelling mistakes, often surpassing existing systems tailored for the Persian language.
• 摘要：写作是人类交流的一种无处不在的形式，几乎渗透到当代生活的方方面面。因此，书面交流中的不准确或错误会导致严重后果，从经济损失到可能危及生命的情况。拼写错误是最常见的写作错误之一，由于各种因素而经常发生。这项研究旨在使用神经网络识别和纠正文本中的各种拼写错误，特别是利用 Transformers 的双向编码器表示 (BERT) 掩码语言模型。为了实现这一目标，我们在对不同类型的拼写错误进行分类后，编制了一个包含非真实单词和真实单词错误的综合数据集。随后，采用了多个预先训练的 BERT 模型。为了确保在纠正拼写错误方面的最佳性能，我们提出了一种利用 BERT 掩码语言模型和 Levenshtein 距离的组合方法。我们的评估数据结果表明，本文介绍的系统在识别和纠正拼写错误方面表现出非凡的能力，通常超越了现有的针对波斯语定制的系统。

Title: PERSONA: A Reproducible Testbed for Pluralistic Alignment

• Authors: Louis Castricato, Nathan Lile, Rafael Rafailov, Jan-Philipp Fränken, Chelsea Finn
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17387
• Pdf URL: https://arxiv.org/pdf/2407.17387
• Copy Paste: [[2407.17387]] PERSONA: A Reproducible Testbed for Pluralistic Alignment(https://arxiv.org/abs/2407.17387)
• Keywords: language model, prompt
• Abstract: The rapid advancement of language models (LMs) necessitates robust alignment with diverse user values. However, current preference optimization approaches often fail to capture the plurality of user opinions, instead reinforcing majority viewpoints and marginalizing minority perspectives. We introduce PERSONA, a reproducible test bed designed to evaluate and improve pluralistic alignment of LMs. We procedurally generate diverse user profiles from US census data, resulting in 1,586 synthetic personas with varied demographic and idiosyncratic attributes. We then generate a large-scale evaluation dataset containing 3,868 prompts and 317,200 feedback pairs obtained from our synthetic personas. Leveraging this dataset, we systematically evaluate LM capabilities in role-playing diverse users, verified through human judges, and the establishment of both a benchmark, PERSONA Bench, for pluralistic alignment approaches as well as an extensive dataset to create new and future benchmarks. The full dataset and benchmarks are available here: this https URL.
• 摘要：语言模型 (LM) 的快速发展需要与多样化的用户价值观进行稳健的对齐。然而，当前的偏好优化方法往往无法捕捉到多种用户意见，反而会强化多数观点并边缘化少数观点。我们引入了 PERSONA，这是一个可重复的测试平台，旨在评估和改进 LM 的多元化对齐。我们从美国人口普查数据中程序化地生成了多样化的用户资料，从而产生了 1,586 个具有不同人口统计和特殊属性的合成人物。然后，我们生成了一个大规模评估数据集，其中包含从我们的合成人物中获得的 3,868 个提示和 317,200 个反馈对。利用这个数据集，我们系统地评估了 LM 在扮演不同用户角色方面的能力，并通过人类评判者进行了验证，并建立了多元化对齐方法的基准 PERSONA Bench 以及用于创建新基准和未来基准的广泛数据集。完整的数据集和基准可在此处获得：此 https URL。

Title: CovScore: Evaluation of Multi-Document Abstractive Title Set Generation

• Authors: Itamar Trainin, Omri Abend
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17390
• Pdf URL: https://arxiv.org/pdf/2407.17390
• Copy Paste: [[2407.17390]] CovScore: Evaluation of Multi-Document Abstractive Title Set Generation(https://arxiv.org/abs/2407.17390)
• Keywords: llm
• Abstract: This paper introduces CovScore, an automatic reference-less methodology for evaluating thematic title sets, extracted from a corpus of documents. While such extraction methods are widely used, evaluating their effectiveness remains an open question. Moreover, some existing practices heavily rely on slow and laborious human annotation procedures. Inspired by recently introduced LLM-based judge methods, we propose a novel methodology that decomposes quality into five main metrics along different aspects of evaluation. This framing simplifies and expedites the manual evaluation process and enables automatic and independent LLM-based evaluation. As a test case, we apply our approach to a corpus of Holocaust survivor testimonies, motivated both by its relevance to title set extraction and by the moral significance of this pursuit. We validate the methodology by experimenting with naturalistic and synthetic title set generation systems and compare their performance with the methodology.
• 摘要：本文介绍了 CovScore，这是一种自动无参考方法，用于评估从文档语料库中提取的主题标题集。虽然这种提取方法被广泛使用，但评估它们的有效性仍是一个悬而未决的问题。此外，一些现有的实践严重依赖缓慢而费力的人工注释程序。受最近推出的基于 LLM 的评判方法的启发，我们提出了一种新颖的方法，将质量分解为五个主要指标，并根据不同的评估方面进行评估。这种框架简化并加快了手动评估过程，并实现了自动和独立的基于 LLM 的评估。作为一个测试案例，我们将我们的方法应用于大屠杀幸存者证词语料库，其动机既在于它与标题集提取的相关性，也在于这一追求的道德意义。我们通过试验自然主义和合成标题集生成系统来验证该方法，并将它们的性能与该方法进行比较。

Title: Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models

• Authors: Yida Zhao, Chao Lou, Kewei Tu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17406
• Pdf URL: https://arxiv.org/pdf/2407.17406
• Copy Paste: [[2407.17406]] Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models(https://arxiv.org/abs/2407.17406)
• Keywords: language model
• Abstract: Syntactic Transformer language models aim to achieve better generalization through simultaneously modeling syntax trees and sentences. While prior work has been focusing on adding constituency-based structures to Transformers, we introduce Dependency Transformer Grammars (DTGs), a new class of Transformer language model with explicit dependency-based inductive bias. DTGs simulate dependency transition systems with constrained attention patterns by modifying attention masks, incorporate the stack information through relative positional encoding, and augment dependency arc representation with a combination of token embeddings and operation embeddings. When trained on a dataset of sentences annotated with dependency trees, DTGs achieve better generalization while maintaining comparable perplexity with Transformer language model baselines. DTGs also outperform recent constituency-based models, showing that dependency can better guide Transformer language models. Our code is released at this https URL.
• 摘要：句法 Transformer 语言模型旨在通过同时对语法树和句子进行建模来实现更好的泛化。虽然之前的工作一直专注于为 Transformer 添加基于成分的结构，但我们引入了依赖 Transformer 语法 (DTG)，这是一类具有显式基于依赖的归纳偏差的新型 Transformer 语言模型。DTG 通过修改注意力掩码来模拟具有受限注意力模式的依赖转换系统，通过相对位置编码合并堆栈信息，并通过标记嵌入和操作嵌入的组合来增强依赖弧表示。当在带有依赖树注释的句子数据集上进行训练时，DTG 实现了更好的泛化，同时保持与 Transformer 语言模型基线相当的困惑度。DTG 的表现也优于最近的基于成分的模型，表明依赖关系可以更好地指导 Transformer 语言模型。我们的代码发布在此 https URL 上。

Title: Fluent Student-Teacher Redteaming

• Authors: T. Ben Thompson, Michael Sklar (Confirm Labs)
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2407.17447
• Pdf URL: https://arxiv.org/pdf/2407.17447
• Copy Paste: [[2407.17447]] Fluent Student-Teacher Redteaming(https://arxiv.org/abs/2407.17447)
• Keywords: language model, prompt
• Abstract: Many publicly available language models have been safety tuned to reduce the likelihood of toxic or liability-inducing text. Users or security analysts attempt to jailbreak or redteam these models with adversarial prompts which cause compliance with requests. One attack method is to apply discrete optimization techniques to the prompt. However, the resulting attack strings are often gibberish text, easily filtered by defenders due to high measured perplexity, and may fail for unseen tasks and/or well-tuned models. In this work, we improve existing algorithms (primarily GCG and BEAST) to develop powerful and fluent attacks on safety-tuned models like Llama-2 and Phi-3. Our technique centers around a new distillation-based approach that encourages the victim model to emulate a toxified finetune, either in terms of output probabilities or internal activations. To encourage human-fluent attacks, we add a multi-model perplexity penalty and a repetition penalty to the objective. We also enhance optimizer strength by allowing token insertions, token swaps, and token deletions and by using longer attack sequences. The resulting process is able to reliably jailbreak the most difficult target models with prompts that appear similar to human-written prompts. On Advbench we achieve attack success rates $>93$% for Llama-2-7B, Llama-3-8B, and Vicuna-7B, while maintaining model-measured perplexity $<33$; we achieve $95$% attack success for Phi-3, though with higher perplexity. We also find a universally-optimized single fluent prompt that induces $>88$% compliance on previously unseen tasks across Llama-2-7B, Phi-3-mini and Vicuna-7B and transfers to other black-box models.
• 摘要：许多公开可用的语言模型都经过了安全调整，以降低出现有毒或引起责任的文本的可能性。用户或安全分析师试图使用对抗性提示来越狱或红队这些模型，从而导致遵守请求。一种攻击方法是将离散优化技术应用于提示。然而，产生的攻击字符串通常是乱码文本，由于测量的困惑度高，很容易被防御者过滤，并且可能对看不见的任务和/或经过良好调整的模型失败。在这项工作中，我们改进了现有算法（主要是 GCG 和 BEAST），以开发对 Llama-2 和 Phi-3 等安全调整模型的强大而流畅的攻击。我们的技术以一种新的基于蒸馏的方法为中心，该方法鼓励受害者模型模拟有毒的微调，无论是在输出概率还是内部激活方面。为了鼓励人类流畅的攻击，我们在目标中添加了多模型困惑度惩罚和重复惩罚。我们还通过允许插入令牌、交换令牌和删除令牌以及使用更长的攻击序列来增强优化器强度。由此产生的过程能够可靠地越狱最困难的目标模型，其提示与人工编写的提示类似。在 Advbench 上，我们对 Llama-2-7B、Llama-3-8B 和 Vicuna-7B 的攻击成功率达到 $>93$%，同时保持模型测量的困惑度 $<33$；我们对 Phi-3 的攻击成功率达到 $95$%，尽管困惑度更高。我们还发现了一个通用优化的单一流畅提示，它在 Llama-2-7B、Phi-3-mini 和 Vicuna-7B 上对以前未见过的任务产生了 $>88$% 的顺从性，并转移到其他黑盒模型。

Title: CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models

• Authors: Jiawei Gu, Zacc Yang, Chuanghao Ding, Rui Zhao, Fei Tan
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2407.17467
• Pdf URL: https://arxiv.org/pdf/2407.17467
• Copy Paste: [[2407.17467]] CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models(https://arxiv.org/abs/2407.17467)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) excel in diverse tasks but often underperform in specialized fields due to limited domain-specific or proprietary corpus. Continual pre-training (CPT) enhances LLM capabilities by imbuing new domain-specific or proprietary knowledge while replaying general corpus to prevent catastrophic forgetting. The data mixture ratio of general corpus and domain-specific corpus, however, has been chosen heuristically, leading to sub-optimal training efficiency in practice. In this context, we attempt to re-visit the scaling behavior of LLMs under the hood of CPT, and discover a power-law relationship between loss, mixture ratio, and training tokens scale. We formalize the trade-off between general and domain-specific capabilities, leading to a well-defined Critical Mixture Ratio (CMR) of general and domain data. By striking the balance, CMR maintains the model's general ability and achieves the desired domain transfer, ensuring the highest utilization of available resources. Therefore, if we value the balance between efficiency and effectiveness, CMR can be consider as the optimal mixture ratio.Through extensive experiments, we ascertain the predictability of CMR, and propose CMR scaling law and have substantiated its generalization. These findings offer practical guidelines for optimizing LLM training in specialized domains, ensuring both general and domain-specific performance while efficiently managing training resources.
• 摘要：大型语言模型 (LLM) 在许多任务上表现出色，但由于领域特定或专有语料库有限，在专业领域中往往表现不佳。持续预训练 (CPT) 通过在重放一般语料库的同时注入新的领域特定或专有知识来增强 LLM 功能，以防止灾难性遗忘。然而，一般语料库和领域特定语料库的数据混合比率是启发式选择的，导致实践中的训练效率不佳。在这种情况下，我们尝试重新审视 CPT 下的 LLM 的扩展行为，并发现损失、混合比率和训练标记规模之间的幂律关系。我们形式化了一般能力和领域特定能力之间的权衡，从而得到了一般数据和领域数据的明确定义的临界混合比 (CMR)。通过达到平衡，CMR 保持了模型的一般能力并实现了所需的领域转移，确保最大程度地利用可用资源。因此，如果我们重视效率和效果之间的平衡，CMR 可以被视为最佳混合比例。通过大量实验，我们确定了 CMR 的可预测性，并提出了 CMR 缩放定律并证实了其泛化性。这些发现为优化专业领域的 LLM 培训提供了实用指南，确保在有效管理培训资源的同时兼顾通用和特定领域的表现。

Title: WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queries

• Authors: Wenting Zhao, Tanya Goyal, Yu Ying Chiu, Liwei Jiang, Benjamin Newman, Abhilasha Ravichander, Khyathi Chandu, Ronan Le Bras, Claire Cardie, Yuntian Deng, Yejin Choi
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17468
• Pdf URL: https://arxiv.org/pdf/2407.17468
• Copy Paste: [[2407.17468]] WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queries(https://arxiv.org/abs/2407.17468)
• Keywords: language model, llm, hallucination, prompt, chat
• Abstract: While hallucinations of large language models (LLMs) prevail as a major challenge, existing evaluation benchmarks on factuality do not cover the diverse domains of knowledge that the real-world users of LLMs seek information about. To bridge this gap, we introduce WildHallucinations, a benchmark that evaluates factuality. It does so by prompting LLMs to generate information about entities mined from user-chatbot conversations in the wild. These generations are then automatically fact-checked against a systematically curated knowledge source collected from web search. Notably, half of these real-world entities do not have associated Wikipedia pages. We evaluate 118,785 generations from 15 LLMs on 7,919 entities. We find that LLMs consistently hallucinate more on entities without Wikipedia pages and exhibit varying hallucination rates across different domains. Finally, given the same base models, adding a retrieval component only slightly reduces hallucinations but does not eliminate hallucinations.
• 摘要：虽然大型语言模型 (LLM) 的幻觉是一个主要挑战，但现有的事实性评估基准并未涵盖 LLM 的真实用户寻求信息的各种知识领域。为了弥补这一差距，我们引入了 WildHallucinations，这是一个评估事实性的基准。它通过提示 LLM 生成从野外用户聊天机器人对话中挖掘出的实体信息来实现这一点。然后，这些生成的内容会根据从网络搜索中收集的系统整理的知识源自动进行事实核查。值得注意的是，这些现实世界实体中有一半没有关联的维基百科页面。我们对 15 个 LLM 中的 118,785 个生成内容进行了评估，涉及 7,919 个实体。我们发现 LLM 在没有维基百科页面的实体上始终产生更多的幻觉，并且在不同领域表现出不同的幻觉率。最后，给定相同的基础模型，添加检索组件只能稍微减少幻觉，但并不能消除幻觉。

Title: I Could've Asked That: Reformulating Unanswerable Questions

• Authors: Wenting Zhao, Ge Gao, Claire Cardie, Alexander M. Rush
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2407.17469
• Pdf URL: https://arxiv.org/pdf/2407.17469
• Copy Paste: [[2407.17469]] I Could've Asked That: Reformulating Unanswerable Questions(https://arxiv.org/abs/2407.17469)
• Keywords: language model, gpt, llm
• Abstract: When seeking information from unfamiliar documents, users frequently pose questions that cannot be answered by the documents. While existing large language models (LLMs) identify these unanswerable questions, they do not assist users in reformulating their questions, thereby reducing their overall utility. We curate CouldAsk, an evaluation benchmark composed of existing and new datasets for document-grounded question answering, specifically designed to study reformulating unanswerable questions. We evaluate state-of-the-art open-source and proprietary LLMs on CouldAsk. The results demonstrate the limited capabilities of these models in reformulating questions. Specifically, GPT-4 and Llama2-7B successfully reformulate questions only 26% and 12% of the time, respectively. Error analysis shows that 62% of the unsuccessful reformulations stem from the models merely rephrasing the questions or even generating identical questions. We publicly release the benchmark and the code to reproduce the experiments.
• 摘要：在从不熟悉的文档中查找信息时，用户经常会提出文档无法回答的问题。虽然现有的大型语言模型 (LLM) 可以识别这些无法回答的问题，但它们无法帮助用户重新表述问题，从而降低了它们的整体效用。我们策划了 CouldAsk，这是一个由现有和新的基于文档的问答数据集组成的评估基准，专门用于研究重新表述无法回答的问题。我们在 CouldAsk 上评估了最先进的开源和专有 LLM。结果表明这些模型在重新表述问题方面的能力有限。具体来说，GPT-4 和 Llama2-7B 分别只有 26% 和 12% 的时间成功重新表述问题。错误分析表明，62% 的不成功重新表述源于模型仅仅重新表述问题甚至生成相同的问题。我们公开发布了基准和代码以重现实验。