2025-01-31

Title: InnerThoughts: Disentangling Representations and Predictions in Large Language Models

• Authors: Didier Chételat, Joseph Cotnareanu, Rylee Thompson, Yingxue Zhang, Mark Coates
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.17994
• Pdf URL: https://arxiv.org/pdf/2501.17994
• Copy Paste: [[2501.17994]] InnerThoughts: Disentangling Representations and Predictions in Large Language Models(https://arxiv.org/abs/2501.17994)
• Keywords: language model, llm, prompt
• Abstract: Large language models (LLMs) contain substantial factual knowledge which is commonly elicited by multiple-choice question-answering prompts. Internally, such models process the prompt through multiple transformer layers, building varying representations of the problem within its hidden states. Ultimately, however, only the hidden state corresponding to the final layer and token position are used to predict the answer label. In this work, we propose instead to learn a small separate neural network predictor module on a collection of training questions, that take the hidden states from all the layers at the last temporal position as input and outputs predictions. In effect, such a framework disentangles the representational abilities of LLMs from their predictive abilities. On a collection of hard benchmarks, our method achieves considerable improvements in performance, sometimes comparable to supervised fine-tuning procedures, but at a fraction of the computational cost.
• 摘要：大型语言模型 (LLM) 包含大量事实知识，这些知识通常由多项选择题回答提示引出。在内部，此类模型通过多个 Transformer 层处理提示，在其隐藏状态中构建问题的不同表示。但最终只有与最后一层和 token 位置相对应的隐藏状态才会用于预测答案标签。在这项工作中，我们建议在一系列训练问题上学习一个单独的小型神经网络预测器模块，该模块将最后时间位置的所有层的隐藏状态作为输入并输出预测。实际上，这样的框架将 LLM 的表示能力与其预测能力区分开来。在一系列硬基准测试中，我们的方法实现了性能的显着提升，有时可与监督微调程序相媲美，但计算成本仅为其一小部分。

Title: Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation

• Authors: Yibo Wang, Tiansheng Huang, Li Shen, Huanjin Yao, Haotian Luo, Rui Liu, Naiqiang Tan, Jiaxing Huang, Dacheng Tao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18100
• Pdf URL: https://arxiv.org/pdf/2501.18100
• Copy Paste: [[2501.18100]] Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation(https://arxiv.org/abs/2501.18100)
• Keywords: language model, llm
• Abstract: Harmful fine-tuning attack introduces significant security risks to the fine-tuning services. Mainstream defenses aim to vaccinate the model such that the later harmful fine-tuning attack is less effective. However, our evaluation results show that such defenses are fragile -- with a few fine-tuning steps, the model still can learn the harmful knowledge. To this end, we do further experiment and find that an embarrassingly simple solution -- adding purely random perturbations to the fine-tuned model, can recover the model from harmful behavior, though it leads to a degradation in the model's fine-tuning performance. To address the degradation of fine-tuning performance, we further propose Panacea, which optimizes an adaptive perturbation that will be applied to the model after fine-tuning. Panacea maintains model's safety alignment performance without compromising downstream fine-tuning performance. Comprehensive experiments are conducted on different harmful ratios, fine-tuning tasks and mainstream LLMs, where the average harmful scores are reduced by up-to 21.5%, while maintaining fine-tuning performance. As a by-product, we analyze the optimized perturbation and show that different layers in various LLMs have distinct safety coefficients. Source code available at this https URL
• 摘要：有害的微调攻击给微调服务带来了重大的安全风险。主流防御措施旨在为模型接种疫苗，使后续有害的微调攻击效果降低。然而，我们的评估结果表明，这种防御措施十分脆弱——只需几个微调步骤，模型仍然可以学习到有害的知识。为此，我们做了进一步的实验，发现一个非常简单的解决方案——在微调模型中添加纯随机扰动，可以使模型从有害行为中恢复，尽管这会导致模型微调性能下降。为了解决微调性能下降的问题，我们进一步提出了 Panacea，它优化了微调后应用于模型的自适应扰动。Panacea 在不损害下游微调性能的情况下保持了模型的安全对齐性能。对不同的有害率、微调任务和主流 LLM 进行了全面的实验，平均有害分数降低了 21.5%，同时保持了微调性能。作为副产品，我们分析了优化后的扰动，并表明不同 LLM 中的不同层具有不同的安全系数。源代码可在此 https URL 上获取

Title: Diverse Preference Optimization

• Authors: Jack Lanchantin, Angelica Chen, Shehzaad Dhuliawala, Ping Yu, Jason Weston, Sainbayar Sukhbaatar, Ilia Kulikov
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18101
• Pdf URL: https://arxiv.org/pdf/2501.18101
• Copy Paste: [[2501.18101]] Diverse Preference Optimization(https://arxiv.org/abs/2501.18101)
• Keywords: language model
• Abstract: Post-training of language models, either through reinforcement learning, preference optimization or supervised finetuning, tends to sharpen the output probability distribution and reduce the diversity of generated responses. This is particularly a problem for creative generative tasks where varied responses are desired. %This impacts the ability to generate high quality synthetic data which is becoming a vital component of model training. In this work we introduce Diverse Preference Optimization (DivPO), an online optimization method which learns to generate much more diverse responses than standard pipelines, while maintaining the quality of the generations. In DivPO, preference pairs are selected by first considering a pool of responses, and a measure of diversity among them, and selecting chosen examples as being more rare but high quality, while rejected examples are more common, but low quality. DivPO results in generating 45.6% more diverse persona attributes, and an 74.6% increase in story diversity, while maintaining similar win rates as standard baselines.
• 摘要：语言模型的后期训练，无论是通过强化学习、偏好优化还是监督微调，都倾向于锐化输出概率分布并降低生成响应的多样性。这对于需要多样化响应的创造性生成任务来说尤其成问题。%这会影响生成高质量合成数据的能力，而这些数据正成为模型训练的重要组成部分。在这项工作中，我们引入了多样化偏好优化 (DivPO)，这是一种在线优化方法，它学习生成比标准管道更加多样化的响应，同时保持生成的质量。在 DivPO 中，选择偏好对时，首先考虑一组响应及其多样性度量，并选择更稀有但质量更高的选定示例，而拒绝的示例更常见但质量较低。DivPO 可生成 45.6% 的多样化角色属性，故事多样性增加 74.6%，同时保持与标准基线相似的胜率。

Title: Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models

• Authors: Qika Lin, Tianzhe Zhao, Kai He, Zhen Peng, Fangzhi Xu, Ling Huang, Jingying Ma, Mengling Feng
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18119
• Pdf URL: https://arxiv.org/pdf/2501.18119
• Copy Paste: [[2501.18119]] Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models(https://arxiv.org/abs/2501.18119)
• Keywords: language model, llm, prompt
• Abstract: Due to the presence of the natural gap between Knowledge Graph (KG) structures and the natural language, the effective integration of holistic structural information of KGs with Large Language Models (LLMs) has emerged as a significant question. To this end, we propose a two-stage framework to learn and apply quantized codes for each entity, aiming for the seamless integration of KGs with LLMs. Firstly, a self-supervised quantized representation (SSQR) method is proposed to compress both KG structural and semantic knowledge into discrete codes (\ie, tokens) that align the format of language sentences. We further design KG instruction-following data by viewing these learned codes as features to directly input to LLMs, thereby achieving seamless integration. The experiment results demonstrate that SSQR outperforms existing unsupervised quantized methods, producing more distinguishable codes. Further, the fine-tuned LLaMA2 and LLaMA3.1 also have superior performance on KG link prediction and triple classification tasks, utilizing only 16 tokens per entity instead of thousands in conventional prompting methods.
• 摘要：由于知识图谱 (KG) 结构与自然语言之间存在天然差距，将 KG 的整体结构信息与大型语言模型 (LLM) 有效集成成为一个重要问题。为此，我们提出了一个两阶段框架来学习和应用每个实体的量化代码，旨在将 KG 与 LLM 无缝集成。首先，提出一种自监督量化表示 (SSQR) 方法，将 KG 结构和语义知识压缩为与语言句子格式一致的离散代码（即 token）。我们进一步设计 KG 指令跟踪数据，将这些学习到的代码视为特征直接输入到 LLM，从而实现无缝集成。实验结果表明，SSQR 优于现有的无监督量化方法，可以产生更具区分性的代码。此外，经过微调的 LLaMA2 和 LLaMA3.1 在 KG 链接预测和三重分类任务上也具有出色的性能，每个实体仅使用 16 个 token，而不是传统提示方法中的数千个 token。

Title: Unraveling the Capabilities of Language Models in News Summarization

• Authors: Abdurrahman Odabaşı, Göksel Biricik
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2501.18128
• Pdf URL: https://arxiv.org/pdf/2501.18128
• Copy Paste: [[2501.18128]] Unraveling the Capabilities of Language Models in News Summarization(https://arxiv.org/abs/2501.18128)
• Keywords: language model, gpt, llm
• Abstract: Given the recent introduction of multiple language models and the ongoing demand for improved Natural Language Processing tasks, particularly summarization, this work provides a comprehensive benchmarking of 20 recent language models, focusing on smaller ones for the news summarization task. In this work, we systematically test the capabilities and effectiveness of these models in summarizing news article texts which are written in different styles and presented in three distinct datasets. Specifically, we focus in this study on zero-shot and few-shot learning settings and we apply a robust evaluation methodology that combines different evaluation concepts including automatic metrics, human evaluation, and LLM-as-a-judge. Interestingly, including demonstration examples in the few-shot learning setting did not enhance models' performance and, in some cases, even led to worse quality of the generated summaries. This issue arises mainly due to the poor quality of the gold summaries that have been used as reference summaries, which negatively impacts the models' performance. Furthermore, our study's results highlight the exceptional performance of GPT-3.5-Turbo and GPT-4, which generally dominate due to their advanced capabilities. However, among the public models evaluated, certain models such as Qwen1.5-7B, SOLAR-10.7B-Instruct-v1.0, Meta-Llama-3-8B and Zephyr-7B-Beta demonstrated promising results. These models showed significant potential, positioning them as competitive alternatives to large models for the task of news summarization.
• 摘要：鉴于最近引入了多种语言模型，以及对改进自然语言处理任务（尤其是摘要）的持续需求，本研究对 20 种近期语言模型进行了全面的基准测试，重点关注用于新闻摘要任务的较小模型。在这项工作中，我们系统地测试了这些模型在摘要新闻文章文本方面的能力和有效性，这些文本以不同的风格编写并呈现在三个不同的数据集中。具体来说，我们在本研究中关注零样本和少样本学习设置，并应用了一种强大的评估方法，该方法结合了不同的评估概念，包括自动指标、人工评估和 LLM-as-a-judge。有趣的是，在少样本学习设置中包含演示示例并没有提高模型的性能，在某些情况下甚至导致生成的摘要质量更差。这个问题主要是因为用作参考摘要的黄金摘要质量差，这对模型的性能产生了负面影响。此外，我们的研究结果突出了 GPT-3.5-Turbo 和 GPT-4 的卓越性能，它们通常因其先进的功能而占据主导地位。然而，在评估的公共模型中，某些模型（例如 Qwen1.5-7B、SOLAR-10.7B-Instruct-v1.0、Meta-Llama-3-8B 和 Zephyr-7B-Beta）表现出了令人鼓舞的结果。这些模型显示出巨大的潜力，使它们成为新闻摘要任务中大型模型的有竞争力的替代品。

Title: Mixed-Precision Graph Neural Quantization for Low Bit Large Language Models

• Authors: Wanlong Liu, Yichen Xiao, Dingyi Zeng, Hongyang Zhao, Wenyu Chen, Malu Zhang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18154
• Pdf URL: https://arxiv.org/pdf/2501.18154
• Copy Paste: [[2501.18154]] Mixed-Precision Graph Neural Quantization for Low Bit Large Language Models(https://arxiv.org/abs/2501.18154)
• Keywords: language model, gpt, llm
• Abstract: Post-Training Quantization (PTQ) is pivotal for deploying large language models (LLMs) within resource-limited settings by significantly reducing resource demands. However, existing PTQ strategies underperform at low bit levels < 3 bits due to the significant difference between the quantized and original weights. To enhance the quantization performance at low bit widths, we introduce a Mixed-precision Graph Neural PTQ (MG-PTQ) approach, employing a graph neural network (GNN) module to capture dependencies among weights and adaptively assign quantization bit-widths. Through the information propagation of the GNN module, our method more effectively captures dependencies among target weights, leading to a more accurate assessment of weight importance and optimized allocation of quantization strategies. Extensive experiments on the WikiText2 and C4 datasets demonstrate that our MG-PTQ method outperforms previous state-of-the-art PTQ method GPTQ, setting new benchmarks for quantization performance under low-bit conditions.
• 摘要：训练后量化 (PTQ) 是部署资源有限环境下大型语言模型 (LLM) 的关键，因为它可以显著减少资源需求。然而，现有的 PTQ 策略在低位元级别 (< 3 位元) 下表现不佳，这是因为量化权重和原始权重之间存在显著差异。为了提高低位元宽度下的量化性能，我们引入了一种混合精度图神经 PTQ (MG-PTQ) 方法，采用图神经网络 (GNN) 模块来捕获权重之间的依赖关系并自适应地分配量化位元宽度。通过 GNN 模块的信息传播，我们的方法可以更有效地捕获目标权重之间的依赖关系，从而更准确地评估权重重要性并优化量化策略的分配。在 WikiText2 和 C4 数据集上进行的大量实验表明，我们的 MG-PTQ 方法优于之前最先进的 PTQ 方法 GPTQ，为低位元条件下的量化性能树立了新的标杆。

Title: Contextually Structured Token Dependency Encoding for Large Language Models

• Authors: James Blades, Frederick Somerfield, William Langley, Susan Everingham, Maurice Witherington
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18205
• Pdf URL: https://arxiv.org/pdf/2501.18205
• Copy Paste: [[2501.18205]] Contextually Structured Token Dependency Encoding for Large Language Models(https://arxiv.org/abs/2501.18205)
• Keywords: language model
• Abstract: Token representation strategies within large-scale neural architectures often rely on contextually refined embeddings, yet conventional approaches seldom encode structured relationships explicitly within token interactions. Self-attention mechanisms effectively capture dynamic contextual dependencies, but their reliance on learned weight distributions limits the preservation of long-range hierarchical structures in generated sequences. Dependency-aware token encoding introduces a structured approach to embedding initialization, ensuring that relational constraints are embedded within token representations rather than inferred solely through attention dynamics. The proposed encoding mechanism refines token interactions through dependency-weighted attention computations, ensuring that syntactic and semantic dependencies are retained across multiple processing layers. Empirical evaluations indicate reductions in perplexity across diverse linguistic benchmarks, suggesting improvements in contextual coherence and predictive consistency in autoregressive text generation. Computational efficiency assessments reveal a moderate increase in memory consumption and training time, attributed to additional matrix computations within the encoding module, yet scalability remains feasible within conventional transformer architectures. Structured encoding enhances lexical variation and dependency retention, reinforcing linguistic coherence without requiring external syntactic annotations or auxiliary training objectives. Statistical comparisons highlight improvements in dependency alignment, particularly in longer sequences where conventional self-attention models exhibit degradation in hierarchical consistency. Sentence length distributions indicate a reduction in abrupt phrase transitions, further supporting the hypothesis that explicit dependency encoding facilitates more structured phrase generation.
• 摘要：大规模神经架构中的标记表示策略通常依赖于上下文细化的嵌入，但传统方法很少在标记交互中明确编码结构化关系。自注意力机制有效地捕获动态上下文依赖关系，但它们对学习到的权重分布的依赖限制了生成序列中长距离层次结构的保留。依赖感知标记编码引入了一种结构化的嵌入初始化方法，确保关系约束嵌入在标记表示中，而不是仅通过注意力动态推断出来。所提出的编码机制通过依赖加权注意力计算来细化标记交互，确保句法和语义依赖关系在多个处理层中得到保留。实证评估表明，不同语言基准的困惑度有所降低，这表明自回归文本生成中的上下文连贯性和预测一致性有所改善。计算效率评估显示，内存消耗和训练时间略有增加，这归因于编码模块中的额外矩阵计算，但在传统的变压器架构中仍然具有可扩展性。结构化编码增强了词汇变化和依存关系保留，增强了语言连贯性，而无需外部句法注释或辅助训练目标。统计比较突出了依存关系对齐方面的改进，特别是在较长的序列中，传统的自注意模型表现出层次一致性的下降。句子长度分布表明突然的短语转换减少，进一步支持了显式依存关系编码有助于更结构化的短语生成的假设。

Title: Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models

• Authors: Haoyu Liang, Youran Sun, Yunfeng Cai, Jun Zhu, Bo Zhang
• Subjects: cs.CL, cs.AI, cs.LG, cs.NE
• Abstract URL: https://arxiv.org/abs/2501.18280
• Pdf URL: https://arxiv.org/pdf/2501.18280
• Copy Paste: [[2501.18280]] Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models(https://arxiv.org/abs/2501.18280)
• Keywords: language model, llm, prompt
• Abstract: The security issue of large language models (LLMs) has gained significant attention recently, with various defense mechanisms developed to prevent harmful outputs, among which safeguards based on text embedding models serve as a fundamental defense. Through testing, we discover that the distribution of text embedding model outputs is significantly biased with a large mean. Inspired by this observation, we propose novel efficient methods to search for universal magic words that can attack text embedding models. The universal magic words as suffixes can move the embedding of any text towards the bias direction, therefore manipulate the similarity of any text pair and mislead safeguards. By appending magic words to user prompts and requiring LLMs to end answers with magic words, attackers can jailbreak the safeguard. To eradicate this security risk, we also propose defense mechanisms against such attacks, which can correct the biased distribution of text embeddings in a train-free manner.
• 摘要：大型语言模型 (LLM) 的安全问题最近引起了广泛关注，各种防御机制被开发出来以防止有害输出，其中基于文本嵌入模型的保护措施是根本的防御措施。通过测试，我们发现文本嵌入模型输出的分布存在显著偏差，且平均值较大。受此观察的启发，我们提出了新颖有效的方法来搜索可以攻击文本嵌入模型的通用魔法词。作为后缀的通用魔法词可以将任何文本的嵌入移向偏差方向，从而操纵任何文本对的相似性并误导保护措施。通过在用户提示后附加魔法词并要求 LLM 以魔法词结尾答案，攻击者可以越狱保护措施。为了消除这种安全风险，我们还提出了针对此类攻击的防御机制，这些机制可以以无需训练的方式纠正文本嵌入的偏差分布。

Title: Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models

• Authors: Jennifer D'Souza, Zachary Laubach, Tarek Al Mustafa, Sina Zarrieß, Robert Frühstückl, Phyllis Illari
• Subjects: cs.CL, cs.AI, cs.DL
• Abstract URL: https://arxiv.org/abs/2501.18287
• Pdf URL: https://arxiv.org/pdf/2501.18287
• Copy Paste: [[2501.18287]] Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models(https://arxiv.org/abs/2501.18287)
• Keywords: language model, llm
• Abstract: This paper presents an exploratory study that harnesses the capabilities of large language models (LLMs) to mine key ecological entities from invasion biology literature. Specifically, we focus on extracting species names, their locations, associated habitats, and ecosystems, information that is critical for understanding species spread, predicting future invasions, and informing conservation efforts. Traditional text mining approaches often struggle with the complexity of ecological terminology and the subtle linguistic patterns found in these texts. By applying general-purpose LLMs without domain-specific fine-tuning, we uncover both the promise and limitations of using these models for ecological entity extraction. In doing so, this study lays the groundwork for more advanced, automated knowledge extraction tools that can aid researchers and practitioners in understanding and managing biological invasions.
• 摘要：本文介绍了一项探索性研究，利用大型语言模型 (LLM) 的功能从入侵生物学文献中挖掘关键生态实体。具体来说，我们专注于提取物种名称、它们的位置、相关栖息地和生态系统，这些信息对于了解物种传播、预测未来入侵和指导保护工作至关重要。传统的文本挖掘方法通常难以应对生态术语的复杂性以及这些文本中发现的微妙语言模式。通过应用没有领域特定微调的通用 LLM，我们发现了使用这些模型进行生态实体提取的前景和局限性。通过这样做，这项研究为更先进的自动化知识提取工具奠定了基础，这些工具可以帮助研究人员和从业者了解和管理生物入侵。

Title: RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects

• Authors: Yiteng Tu, Weihang Su, Yujia Zhou, Yiqun Liu, Qingyao Ai
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.18365
• Pdf URL: https://arxiv.org/pdf/2501.18365
• Copy Paste: [[2501.18365]] RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects(https://arxiv.org/abs/2501.18365)
• Keywords: language model, llm, retrieval-augmented generation
• Abstract: Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieved from a knowledge base. However, its effectiveness is fundamentally constrained by the reliability of both the retriever and the knowledge base. In real-world scenarios, imperfections in these components often lead to the retrieval of noisy, irrelevant, or misleading counterfactual information, ultimately undermining the trustworthiness of RAG systems. To address this challenge, we propose Robust Fine-Tuning (RbFT), a method designed to enhance the resilience of LLMs against retrieval defects through two targeted fine-tuning tasks. Experimental results demonstrate that RbFT significantly improves the robustness of RAG systems across diverse retrieval conditions, surpassing existing methods while maintaining high inference efficiency and compatibility with other robustness techniques.
• 摘要：检索增强生成 (RAG) 通过集成从知识库中检索到的外部知识来增强大型语言模型 (LLM)。然而，它的有效性从根本上受到检索器和知识库可靠性的制约。在现实世界中，这些组件的缺陷往往会导致检索到嘈杂、不相关或误导性的反事实信息，最终破坏 RAG 系统的可信度。为了应对这一挑战，我们提出了稳健微调 (RbFT)，这种方法旨在通过两个有针对性的微调任务增强 LLM 对检索缺陷的弹性。实验结果表明，RbFT 显著提高了 RAG 系统在不同检索条件下的稳健性，超越了现有方法，同时保持了高推理效率和与其他稳健性技术的兼容性。

Title: GENIE: Generative Note Information Extraction model for structuring EHR data

• Authors: Huaiyuan Ying, Hongyi Yuan, Jinsen Lu, Zitian Qu, Yang Zhao, Zhengyun Zhao, Isaac Kohane, Tianxi Cai, Sheng Yu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18435
• Pdf URL: https://arxiv.org/pdf/2501.18435
• Copy Paste: [[2501.18435]] GENIE: Generative Note Information Extraction model for structuring EHR data(https://arxiv.org/abs/2501.18435)
• Keywords: language model, gpt, llm
• Abstract: Electronic Health Records (EHRs) hold immense potential for advancing healthcare, offering rich, longitudinal data that combines structured information with valuable insights from unstructured clinical notes. However, the unstructured nature of clinical text poses significant challenges for secondary applications. Traditional methods for structuring EHR free-text data, such as rule-based systems and multi-stage pipelines, are often limited by their time-consuming configurations and inability to adapt across clinical notes from diverse healthcare settings. Few systems provide a comprehensive attribute extraction for terminologies. While giant large language models (LLMs) like GPT-4 and LLaMA 405B excel at structuring tasks, they are slow, costly, and impractical for large-scale use. To overcome these limitations, we introduce GENIE, a Generative Note Information Extraction system that leverages LLMs to streamline the structuring of unstructured clinical text into usable data with standardized format. GENIE processes entire paragraphs in a single pass, extracting entities, assertion statuses, locations, modifiers, values, and purposes with high accuracy. Its unified, end-to-end approach simplifies workflows, reduces errors, and eliminates the need for extensive manual intervention. Using a robust data preparation pipeline and fine-tuned small scale LLMs, GENIE achieves competitive performance across multiple information extraction tasks, outperforming traditional tools like cTAKES and MetaMap and can handle extra attributes to be extracted. GENIE strongly enhances real-world applicability and scalability in healthcare systems. By open-sourcing the model and test data, we aim to encourage collaboration and drive further advancements in EHR structurization.
• 摘要：电子健康记录 (EHR) 具有巨大的潜力，可以推动医疗保健的发展，提供丰富的纵向数据，将结构化信息与非结构化临床笔记中的宝贵见解相结合。然而，临床文本的非结构化性质对二次应用提出了重大挑战。传统的构建 EHR 自由文本数据的方法，例如基于规则的系统和多阶段管道，通常受到其耗时的配置和无法适应来自不同医疗环境的临床笔记的限制。很少有系统提供术语的全面属性提取。虽然 GPT-4 和 LLaMA 405B 等大型语言模型 (LLM) 在结构化任务方面表现出色，但它们速度慢、成本高，不适合大规模使用。为了克服这些限制，我们推出了 GENIE，这是一个生成性笔记信息提取系统，它利用 LLM 将非结构化临床文本的结构简化为具有标准化格式的可用数据。 GENIE 一次性处理整个段落，以高精度提取实体、断言状态、位置、修饰符、值和目的。其统一的端到端方法简化了工作流程，减少了错误，并消除了大量人工干预的需要。使用强大的数据准备管道和经过微调的小规模 LLM，GENIE 在多个信息提取任务中实现了具有竞争力的性能，优于 cTAKES 和 MetaMap 等传统工具，并且可以处理要提取的额外属性。GENIE 大大增强了医疗保健系统的实际适用性和可扩展性。通过开源模型和测试数据，我们旨在鼓励协作并推动 EHR 结构化的进一步发展。

Title: CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering

• Authors: Yumeng Wang, Zhiyuan Fan, Qingyun Wang, May Fung, Heng Ji
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18457
• Pdf URL: https://arxiv.org/pdf/2501.18457
• Copy Paste: [[2501.18457]] CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering(https://arxiv.org/abs/2501.18457)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are pretrained on extensive multilingual corpora to acquire both language-specific cultural knowledge and general knowledge. Ideally, while LLMs should provide consistent responses to culture-independent questions across languages, we observe significant performance disparities. To address this, we explore the Cross-Lingual Self-Aligning ability of Language Models (CALM) to align knowledge across languages. Specifically, for a given question, we sample multiple responses across different languages, and select the most self-consistent response as the target, leaving the remaining responses as negative examples. We then employ direct preference optimization (DPO) to align the model's knowledge across different languages. Evaluations on the MEDQA and X-CSQA datasets demonstrate CALM's effectiveness in enhancing cross-lingual knowledge question answering, both in zero-shot and retrieval augmented settings. We also found that increasing the number of languages involved in CALM training leads to even higher accuracy and consistency. We offer a qualitative analysis of how cross-lingual consistency can enhance knowledge alignment and explore the method's generalizability. The source code and data of this paper are available on GitHub.
• 摘要：大型语言模型 (LLM) 在广泛的多语言语料库上进行预训练，以获取特定语言的文化知识和一般知识。理想情况下，虽然 LLM 应该为跨语言的文化独立问题提供一致的答案，但我们观察到显著的性能差异。为了解决这个问题，我们探索了语言模型 (CALM) 的跨语言自对齐能力，以跨语言对齐知识。具体来说，对于给定的问题，我们从不同语言中抽取多个答案，并选择最自洽的答案作为目标，其余答案作为反面例子。然后，我们采用直接偏好优化 (DPO) 来对齐不同语言之间的模型知识。对 MEDQA 和 X-CSQA 数据集的评估证明了 CALM 在增强跨语言知识问答方面的有效性，无论是在零样本设置还是检索增强设置中。我们还发现，增加 CALM 训练中涉及的语言数量可以提高准确性和一致性。我们对跨语言一致性如何增强知识对齐进行了定性分析，并探索了该方法的通用性。本文的源代码和数据可在GitHub上找到。

Title: Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch

• Authors: Arthur Douillard, Yanislav Donchev, Keith Rush, Satyen Kale, Zachary Charles, Zachary Garrett, Gabriel Teston, Dave Lacey, Ross McIlroy, Jiajun Shen, Alexandre Ramé, Arthur Szlam, Marc'Aurelio Ranzato, Paul Barham
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18512
• Pdf URL: https://arxiv.org/pdf/2501.18512
• Copy Paste: [[2501.18512]] Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch(https://arxiv.org/abs/2501.18512)
• Keywords: language model, llm
• Abstract: Training of large language models (LLMs) is typically distributed across a large number of accelerators to reduce training time. Since internal states and parameter gradients need to be exchanged at each and every single gradient step, all devices need to be co-located using low-latency high-bandwidth communication links to support the required high volume of exchanged bits. Recently, distributed algorithms like DiLoCo have relaxed such co-location constraint: accelerators can be grouped into ``workers'', where synchronizations between workers only occur infrequently. This in turn means that workers can afford being connected by lower bandwidth communication links without affecting learning quality. However, in these methods, communication across workers still requires the same peak bandwidth as before, as the synchronizations require all parameters to be exchanged across all workers. In this paper, we improve DiLoCo in three ways. First, we synchronize only subsets of parameters in sequence, rather than all at once, which greatly reduces peak bandwidth. Second, we allow workers to continue training while synchronizing, which decreases wall clock time. Third, we quantize the data exchanged by workers, which further reduces bandwidth across workers. By properly combining these modifications, we show experimentally that we can distribute training of billion-scale parameters and reach similar quality as before, but reducing required bandwidth by two orders of magnitude.
• 摘要：大型语言模型 (LLM) 的训练通常分布在大量加速器上，以减少训练时间。由于内部状态和参数梯度需要在每个梯度步骤中交换，因此所有设备都需要使用低延迟高带宽通信链路共置，以支持所需的大量交换位。最近，像 DiLoCo 这样的分布式算法放宽了这种共置限制：加速器可以分组为“工作者”，其中工作者之间的同步很少发生。这反过来意味着工作者可以通过较低带宽的通信链路连接而不会影响学习质量。然而，在这些方法中，工作者之间的通信仍然需要与以前相同的峰值带宽，因为同步需要所有工作者之间交换所有参数。在本文中，我们从三个方面改进了 DiLoCo。首先，我们只按顺序同步参数子集，而不是一次同步所有参数，这大大降低了峰值带宽。其次，我们允许工作者在同步的同时继续训练，从而减少了挂钟时间。第三，我们量化了 Worker 之间交换的数据，从而进一步减少了 Worker 之间的带宽。通过适当组合这些修改，我们通过实验证明，我们可以分布式训练十亿级参数，并达到与以前类似的质量，但所需带宽减少了两个数量级。

Title: Differentially Private Steering for Large Language Model Alignment

• Authors: Anmol Goel, Yaxi Hu, Iryna Gurevych, Amartya Sanyal
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18532
• Pdf URL: https://arxiv.org/pdf/2501.18532
• Copy Paste: [[2501.18532]] Differentially Private Steering for Large Language Model Alignment(https://arxiv.org/abs/2501.18532)
• Keywords: language model, llm, hallucination
• Abstract: Aligning Large Language Models (LLMs) with human values and away from undesirable behaviors (such as hallucination) has become increasingly important. Recently, steering LLMs towards a desired behavior via activation editing has emerged as an effective method to mitigate harmful generations at inference-time. Activation editing modifies LLM representations by preserving information from positive demonstrations (e.g., truthful) and minimising information from negative demonstrations (e.g., hallucinations). When these demonstrations come from a private dataset, the aligned LLM may leak private information contained in those private samples. In this work, we present the first study of aligning LLM behavior with private datasets. Our work proposes the \textit{\underline{P}rivate \underline{S}teering for LLM \underline{A}lignment (PSA)} algorithm to edit LLM activations with differential privacy (DP) guarantees. We conduct extensive experiments on seven different benchmarks with open-source LLMs of different sizes (0.5B to 7B) and model families (LlaMa, Qwen, Mistral and Gemma). Our results show that PSA achieves DP guarantees for LLM alignment with minimal loss in performance, including alignment metrics, open-ended text generation quality, and general-purpose reasoning. We also develop the first Membership Inference Attack (MIA) for evaluating and auditing the empirical privacy for the problem of LLM steering via activation editing. Our attack is tailored for activation editing and relies solely on the generated texts without their associated probabilities. Our experiments support the theoretical guarantees by showing improved guarantees for our \textit{PSA} algorithm compared to several existing non-private techniques.
• 摘要：将大型语言模型 (LLM) 与人类价值观保持一致并远离不良行为（例如幻觉）变得越来越重要。最近，通过激活编辑将 LLM 引导至期望行为已成为一种在推理时减轻有害生成的有效方法。激活编辑通过保留来自正面演示（例如真实）的信息并最小化来自负面演示（例如幻觉）的信息来修改 LLM 表示。当这些演示来自私有数据集时，对齐的 LLM 可能会泄露这些私有样本中包含的私有信息。在这项工作中，我们提出了第一项将 LLM 行为与私有数据集对齐的研究。我们的工作提出了 \textit{\underline{P}rivate \underline{S}steering for LLM \underline{A}lignment (PSA)} 算法来编辑具有差分隐私 (DP) 保证的 LLM 激活。我们对七个不同的基准进行了广泛的实验，这些基准使用了不同大小（0.5B 到 7B）和模型系列（LlaMa、Qwen、Mistral 和 Gemma）的开源 LLM。结果表明，PSA 实现了 LLM 对齐的 DP 保证，同时性能损失最小，包括对齐指标、开放式文本生成质量和通用推理。我们还开发了第一个成员推理攻击 (MIA)，用于评估和审核通过激活编辑进行 LLM 转向问题的经验隐私。我们的攻击是针对激活编辑量身定制的，仅依赖于生成的文本而不依赖于其相关概率。与现有的几种非私有技术相比，我们的实验通过展示我们的 \textit{PSA} 算法的改进保证来支持理论保证。

Title: Can we Retrieve Everything All at Once? ARM: An Alignment-Oriented LLM-based Retrieval Method

• Authors: Peter Baile Chen, Yi Zhang, Michael Cafarella, Dan Roth
• Subjects: cs.CL, cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2501.18539
• Pdf URL: https://arxiv.org/pdf/2501.18539
• Copy Paste: [[2501.18539]] Can we Retrieve Everything All at Once? ARM: An Alignment-Oriented LLM-based Retrieval Method(https://arxiv.org/abs/2501.18539)
• Keywords: llm, agent
• Abstract: Real-world open-domain questions can be complicated, particularly when answering them involves information from multiple information sources. LLMs have demonstrated impressive performance in decomposing complex tasks into simpler steps, and previous work has used it for better retrieval in support of complex questions. However, LLM's decomposition of questions is unaware of what data is available and how data is organized, often leading to a sub-optimal retrieval performance. Recent effort in agentic RAG proposes to perform retrieval in an iterative fashion, where a followup query is derived as an action based on previous rounds of retrieval. While this provides one way of interacting with the data collection, agentic RAG's exploration of data is inefficient because successive queries depend on previous results rather than being guided by the organization of available data in the collection. To address this problem, we propose an LLM-based retrieval method -- ARM, that aims to better align the question with the organization of the data collection by exploring relationships among data objects beyond matching the utterance of the query, thus leading to a retrieve-all-at-once solution for complex queries. We evaluated ARM on two datasets, Bird and OTT-QA. On Bird, it outperforms standard RAG with query decomposition by up to 5.2 pt in execution accuracy and agentic RAG (ReAct) by up to 15.9 pt. On OTT-QA, it achieves up to 5.5 pt and 19.3 pt higher F1 match scores compared to these approaches.
• 摘要：现实世界中的开放域问题可能很复杂，特别是当回答这些问题涉及来自多个信息源的信息时。LLM 在将复杂任务分解为更简单的步骤方面表现出色，之前的研究已经使用它来更好地检索以支持复杂问题。然而，LLM 对问题的分解并不知道有哪些数据可用以及数据是如何组织的，这通常会导致检索性能不佳。Agentic RAG 的最新努力提出以迭代方式执行检索，其中后续查询是基于前几轮检索得出的操作。虽然这提供了一种与数据集合交互的方式，但 agentic RAG 对数据的探索效率低下，因为连续查询依赖于先前的结果，而不是由集合中可用数据的组织来指导。为了解决这个问题，我们提出了一种基于 LLM 的检索方法 ARM，该方法旨在通过探索数据对象之间的关系（而不仅仅是匹配查询语句）来更好地将问题与数据集合的组织联系起来，从而为复杂查询提供一次性检索的解决方案。我们在 Bird 和 OTT-QA 两个数据集上评估了 ARM。在 Bird 上，它在执行准确度方面比标准 RAG 查询分解高出 5.2 分，比 agentic RAG (ReAct) 高出 15.9 分。在 OTT-QA 上，与这些方法相比，它的 F1 匹配分数分别高出 5.5 分和 19.3 分。

Title: R.I.P.: Better Models by Survival of the Fittest Prompts

• Authors: Ping Yu, Weizhe Yuan, Olga Golovneva, Tianhao Wu, Sainbayar Sukhbaatar, Jason Weston, Jing Xu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2501.18578
• Pdf URL: https://arxiv.org/pdf/2501.18578
• Copy Paste: [[2501.18578]] R.I.P.: Better Models by Survival of the Fittest Prompts(https://arxiv.org/abs/2501.18578)
• Keywords: prompt
• Abstract: Training data quality is one of the most important drivers of final model quality. In this work, we introduce a method for evaluating data integrity based on the assumption that low-quality input prompts result in high variance and low quality responses. This is achieved by measuring the rejected response quality and the reward gap between the chosen and rejected preference pair. Our method, Rejecting Instruction Preferences (RIP) can be used to filter prompts from existing training sets, or to make high quality synthetic datasets, yielding large performance gains across various benchmarks compared to unfiltered data. Using Llama 3.1-8B-Instruct, RIP improves AlpacaEval2 LC Win Rate by 9.4%, Arena-Hard by 8.7%, and WildBench by 9.9%. Using Llama 3.3-70B-Instruct, RIP improves Arena-Hard from 67.5 to 82.9, which is from 18th place to 6th overall in the leaderboard.
• 摘要：训练数据质量是最终模型质量的最重要驱动因素之一。在这项工作中，我们引入了一种评估数据完整性的方法，该方法基于低质量输入提示会导致高方差和低质量响应的假设。这是通过测量被拒绝的响应质量以及所选和被拒绝的偏好对之间的奖励差距来实现的。我们的方法，拒绝指令偏好 (RIP)，可用于过滤现有训练集中的提示，或制作高质量的合成数据集，与未过滤的数据相比，在各种基准测试中产生巨大的性能提升。使用 Llama 3.1-8B-Instruct，RIP 将 AlpacaEval2 LC 胜率提高了 9.4%，Arena-Hard 提高了 8.7%，WildBench 提高了 9.9%。使用 Llama 3.3-70B-Instruct，RIP 将 Arena-Hard 从 67.5 提高到 82.9，即在排行榜上从第 18 位上升到第 6 位。

Title: Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs

• Authors: Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, Dong Yu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2501.18585
• Pdf URL: https://arxiv.org/pdf/2501.18585
• Copy Paste: [[2501.18585]] Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs(https://arxiv.org/abs/2501.18585)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) such as OpenAI's o1 have demonstrated remarkable abilities in complex reasoning tasks by scaling test-time compute and exhibiting human-like deep thinking. However, we identify a phenomenon we term underthinking, where o1-like LLMs frequently switch between different reasoning thoughts without sufficiently exploring promising paths to reach a correct solution. This behavior leads to inadequate depth of reasoning and decreased performance, particularly on challenging mathematical problems. To systematically analyze this issue, we conduct experiments on three challenging test sets and two representative open-source o1-like models, revealing that frequent thought switching correlates with incorrect responses. We introduce a novel metric to quantify underthinking by measuring token efficiency in incorrect answers. To address underthinking, we propose a decoding strategy with thought switching penalty TIP that discourages premature transitions between thoughts, encouraging deeper exploration of each reasoning path. Experimental results demonstrate that our approach improves accuracy across challenging datasets without requiring model fine-tuning. Our findings contribute to understanding reasoning inefficiencies in o1-like LLMs and offer a practical solution to enhance their problem-solving capabilities.
• 摘要：大型语言模型 (LLM)（例如 OpenAI 的 o1）通过扩展测试时间计算并展现出类似人类的深度思考，在复杂的推理任务中展现出了非凡的能力。然而，我们发现了一种现象，我们称之为“思考不足”，其中 o1 类 LLM 经常在不同的推理思维之间切换，而没有充分探索有希望的路径以找到正确的解决方案。这种行为导致推理深度不足和性能下降，尤其是在具有挑战性的数学问题上。为了系统地分析这个问题，我们对三个具有挑战性的测试集和两个具有代表性的开源 o1 类模型进行了实验，结果表明频繁的思维切换与错误回答相关。我们引入了一种新颖的指标，通过测量错误答案中的标记效率来量化思考不足。为了解决思考不足的问题，我们提出了一种带有思维切换惩罚 TIP 的解码策略，该策略阻止了思维之间的过早转换，鼓励更深入地探索每条推理路径。实验结果表明，我们的方法提高了具有挑战性的数据集的准确性，而无需模型微调。我们的研究有助于理解 o1 类 LLM 中的推理效率低下问题，并为提高其解决问题的能力提供实用的解决方案。