2024-02-08

Title: PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks

• Authors: Junwei Su, Difan Zou, Chuan Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04284
• Pdf URL: https://arxiv.org/pdf/2402.04284
• Copy Paste: [[2402.04284]] PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks(https://arxiv.org/abs/2402.04284)
• Keywords: rag
• Abstract: Memory-based Dynamic Graph Neural Networks (MDGNNs) are a family of dynamic graph neural networks that leverage a memory module to extract, distill, and memorize long-term temporal dependencies, leading to superior performance compared to memory-less counterparts. However, training MDGNNs faces the challenge of handling entangled temporal and structural dependencies, requiring sequential and chronological processing of data sequences to capture accurate temporal patterns. During the batch training, the temporal data points within the same batch will be processed in parallel, while their temporal dependencies are neglected. This issue is referred to as temporal discontinuity and restricts the effective temporal batch size, limiting data parallelism and reducing MDGNNs' flexibility in industrial applications. This paper studies the efficient training of MDGNNs at scale, focusing on the temporal discontinuity in training MDGNNs with large temporal batch sizes. We first conduct a theoretical study on the impact of temporal batch size on the convergence of MDGNN training. Based on the analysis, we propose PRES, an iterative prediction-correction scheme combined with a memory coherence learning objective to mitigate the effect of temporal discontinuity, enabling MDGNNs to be trained with significantly larger temporal batches without sacrificing generalization performance. Experimental results demonstrate that our approach enables up to a 4x larger temporal batch (3.4x speed-up) during MDGNN training.
• 摘要：基于内存的动态图神经网络 (MDGNN) 是一系列动态图神经网络，它利用内存模块来提取、提取和记忆长期时间依赖性，与无内存网络相比，具有卓越的性能。然而，训练 MDGNN 面临着处理纠缠的时间和结构依赖性的挑战，需要对数据序列进行顺序和时间顺序处理以捕获准确的时间模式。在批量训练期间，同一批次内的时间数据点将被并行处理，而它们的时间依赖性被忽略。这个问题被称为时间不连续性，它限制了有效的时间批量大小，限制了数据并行性并降低了 MDGNN 在工业应用中的灵活性。本文研究了大规模 MDGNN 的有效训练，重点关注训练大时间批量大小的 MDGNN 时的时间不连续性。我们首先对时间批量大小对 MDGNN 训练收敛性的影响进行了理论研究。基于分析，我们提出了 PRES，一种迭代预测校正方案，与记忆一致性学习目标相结合，以减轻时间不连续性的影响，使 MDGNN 能够在不牺牲泛化性能的情况下使用更大的时间批次进行训练。实验结果表明，我们的方法在 MDGNN 训练期间可实现高达 4 倍大的时间批量（3.4 倍加速）。

Title: BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

• Authors: Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, Xiaojuan Qi
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04291
• Pdf URL: https://arxiv.org/pdf/2402.04291
• Copy Paste: [[2402.04291]] BiLLM: Pushing the Limit of Post-Training Quantization for LLMs(https://arxiv.org/abs/2402.04291)
• Keywords: language model, llm
• Abstract: Pretrained large language models (LLMs) exhibit exceptional general language processing capabilities but come with significant demands on memory and computational resources. As a powerful compression technology, binarization can extremely reduce model weights to a mere 1 bit, lowering the expensive computation and memory requirements. However, existing quantization techniques fall short of maintaining LLM performance under ultra-low bit-widths. In response to this challenge, we present BiLLM, a groundbreaking 1-bit post-training quantization scheme tailored for pretrained LLMs. Based on the weight distribution of LLMs, BiLLM first identifies and structurally selects salient weights, and minimizes the compression loss through an effective binary residual approximation strategy. Moreover, considering the bell-shaped distribution of the non-salient weights, we propose an optimal splitting search to group and binarize them accurately. BiLLM achieving for the first time high-accuracy inference (e.g. 8.41 perplexity on LLaMA2-70B) with only 1.08-bit weights across various LLMs families and evaluation metrics, outperforms SOTA quantization methods of LLM by significant margins. Moreover, BiLLM enables the binarization process of the LLM with 7 billion weights within 0.5 hours on a single GPU, demonstrating satisfactory time efficiency.
• 摘要：预训练的大型语言模型 (LLM) 表现出卓越的通用语言处理能力，但对内存和计算资源有很高的要求。作为一种强大的压缩技术，二值化可以将模型权重极大地降低至仅 1 位，从而降低了昂贵的计算和内存需求。然而，现有的量化技术无法在超低位宽下保持 LLM 性能。为了应对这一挑战，我们提出了 BiLLM，这是一种专为预训练的 LLM 量身定制的突破性 1 位训练后量化方案。基于LLM的权重分布，BiLLM首先识别并结构性地选择显着权重，并通过有效的二元残差逼近策略最小化压缩损失。此外，考虑到非显着权重的钟形分布，我们提出了一种最佳分割搜​​索来准确地对它们进行分组和二值化。 BiLLM 首次在各种 LLM 系列和评估指标中以 1.08 位权重实现高精度推理（例如 LLaMA2-70B 上的困惑度为 8.41），显着优于 LLM 的 SOTA 量化方法。此外，BiLLM在单GPU上能够在0.5小时内完成70亿个权重的LLM二值化过程，表现出令人满意的时间效率。

Title: AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies

• Authors: Xixi Hu, Bo Liu, Xingchao Liu, Qiang Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04292
• Pdf URL: https://arxiv.org/pdf/2402.04292
• Copy Paste: [[2402.04292]] AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies(https://arxiv.org/abs/2402.04292)
• Keywords: code
• Abstract: Diffusion-based imitation learning improves Behavioral Cloning (BC) on multi-modal decision-making, but comes at the cost of significantly slower inference due to the recursion in the diffusion process. It urges us to design efficient policy generators while keeping the ability to generate diverse actions. To address this challenge, we propose AdaFlow, an imitation learning framework based on flow-based generative modeling. AdaFlow represents the policy with state-conditioned ordinary differential equations (ODEs), which are known as probability flows. We reveal an intriguing connection between the conditional variance of their training loss and the discretization error of the ODEs. With this insight, we propose a variance-adaptive ODE solver that can adjust its step size in the inference stage, making AdaFlow an adaptive decision-maker, offering rapid inference without sacrificing diversity. Interestingly, it automatically reduces to a one-step generator when the action distribution is uni-modal. Our comprehensive empirical evaluation shows that AdaFlow achieves high performance across all dimensions, including success rate, behavioral diversity, and inference speed. The code is available at https://github.com/hxixixh/AdaFlow
• 摘要：基于扩散的模仿学习改进了多模态决策的行为克隆（BC），但由于扩散过程中的递归，其代价是推理速度明显减慢。它敦促我们设计高效的政策生成器，同时保持制定多样化行动的能力。为了应对这一挑战，我们提出了 AdaFlow，一种基于流生成建模的模仿学习框架。 AdaFlow 表示具有状态条件常微分方程 (ODE) 的策略，称为概率流。我们揭示了训练损失的条件方差与 ODE 的离散化误差之间的有趣联系。有了这种见解，我们提出了一种方差自适应 ODE 求解器，可以在推理阶段调整其步长，使 AdaFlow 成为自适应决策者，在不牺牲多样性的情况下提供快速推理。有趣的是，当动作分布是单峰时，它会自动简化为单步生成器。我们全面的实证评估表明，AdaFlow 在所有维度上都实现了高性能，包括成功率、行为多样性和推理速度。代码可在 https://github.com/hxixixh/AdaFlow 获取

Title: LightHGNN: Distilling Hypergraph Neural Networks into MLPs for $100\times$ Faster Inference

• Authors: Yifan Feng, Yihe Luo, Shihui Ying, Yue Gao
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04296
• Pdf URL: https://arxiv.org/pdf/2402.04296
• Copy Paste: [[2402.04296]] LightHGNN: Distilling Hypergraph Neural Networks into MLPs for $100\times$ Faster Inference(https://arxiv.org/abs/2402.04296)
• Keywords: rag
• Abstract: Hypergraph Neural Networks (HGNNs) have recently attracted much attention and exhibited satisfactory performance due to their superiority in high-order correlation modeling. However, it is noticed that the high-order modeling capability of hypergraph also brings increased computation complexity, which hinders its practical industrial deployment. In practice, we find that one key barrier to the efficient deployment of HGNNs is the high-order structural dependencies during inference. In this paper, we propose to bridge the gap between the HGNNs and inference-efficient Multi-Layer Perceptron (MLPs) to eliminate the hypergraph dependency of HGNNs and thus reduce computational complexity as well as improve inference speed. Specifically, we introduce LightHGNN and LightHGNN$^+$ for fast inference with low complexity. LightHGNN directly distills the knowledge from teacher HGNNs to student MLPs via soft labels, and LightHGNN$^+$ further explicitly injects reliable high-order correlations into the student MLPs to achieve topology-aware distillation and resistance to over-smoothing. Experiments on eight hypergraph datasets demonstrate that even without hypergraph dependency, the proposed LightHGNNs can still achieve competitive or even better performance than HGNNs and outperform vanilla MLPs by $16.3$ on average. Extensive experiments on three graph datasets further show the average best performance of our LightHGNNs compared with all other methods. Experiments on synthetic hypergraphs with 5.5w vertices indicate LightHGNNs can run $100\times$ faster than HGNNs, showcasing their ability for latency-sensitive deployments.
• 摘要：超图神经网络（HGNN）由于其在高阶相关建模方面的优越性，最近引起了广泛的关注并表现出了令人满意的性能。然而，人们注意到超图的高阶建模能力也带来了计算复杂度的增加，这阻碍了其实际的工业部署。在实践中，我们发现高效部署 HGNN 的一个关键障碍是推理过程中的高阶结构依赖性。在本文中，我们建议弥合 HGNN 和推理高效的多层感知器（MLP）之间的差距，以消除 HGNN 的超图依赖性，从而降低计算复杂性并提高推理速度。具体来说，我们引入了 LightHGNN 和 LightHGNN$^+$ 以实现低复杂度的快速推理。 LightHGNN 通过软标签直接将教师 HGNN 的知识提炼到学生 MLP，并且 LightHGNN$^+$ 进一步明确地将可靠的高阶相关性注入到学生 MLP 中，以实现拓扑感知的提炼和抗过度平滑。对八个超图数据集的实验表明，即使没有超图依赖性，所提出的 LightHGNN 仍然可以实现比 HGNN 有竞争力甚至更好的性能，并且平均比普通 MLP 多 16.3 美元。对三个图数据集的广泛实验进一步表明，与所有其他方法相比，我们的 LightHGNN 具有平均最佳性能。对具有 5.5w 个顶点的合成超图进行的实验表明，LightHGNN 的运行速度比 HGNN 快 100 倍，展示了其对延迟敏感的部署的能力。

Title: Training Language Models to Generate Text with Citations via Fine-grained Rewards

• Authors: Chengyu Huang, Zeqiu Wu, Yushi Hu, Wenya Wang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04315
• Pdf URL: https://arxiv.org/pdf/2402.04315
• Copy Paste: [[2402.04315]] Training Language Models to Generate Text with Citations via Fine-grained Rewards(https://arxiv.org/abs/2402.04315)
• Keywords: language model, gpt, llm, hallucination, prompt
• Abstract: While recent Large Language Models (LLMs) have proven useful in answering user queries, they are prone to hallucination, and their responses often lack credibility due to missing references to reliable sources. An intuitive solution to these issues would be to include in-text citations referring to external documents as evidence. While previous works have directly prompted LLMs to generate in-text citations, their performances are far from satisfactory, especially when it comes to smaller LLMs. In this work, we propose an effective training framework using fine-grained rewards to teach LLMs to generate highly supportive and relevant citations, while ensuring the correctness of their responses. We also conduct a systematic analysis of applying these fine-grained rewards to common LLM training strategies, demonstrating its advantage over conventional practices. We conduct extensive experiments on Question Answering (QA) datasets taken from the ALCE benchmark and validate the model's generalizability using EXPERTQA. On LLaMA-2-7B, the incorporation of fine-grained rewards achieves the best performance among the baselines, even surpassing that of GPT-3.5-turbo.
• 摘要：虽然最近的大型语言模型（LLM）已被证明在回答用户查询方面很有用，但它们很容易产生幻觉，并且由于缺少对可靠来源的引用，它们的响应往往缺乏可信度。这些问题的直观解决方案是在文本中引用外部文档作为证据。虽然之前的工作直接促使法学硕士生成文本引用，但它们的表现远不能令人满意，尤其是对于规模较小的法学硕士。在这项工作中，我们提出了一个有效的培训框架，使用细粒度的奖励来教导法学硕士生成高度支持性和相关的引文，同时确保他们的回答的正确性。我们还对将这些细粒度奖励应用于常见的 LLM 培训策略进行了系统分析，证明了其相对于传统做法的优势。我们对取自 ALCE 基准的问答 (QA) 数据集进行了广泛的实验，并使用 EXPERTQA 验证了模型的通用性。在 LLaMA-2-7B 上，细粒度奖励的结合实现了基线中最好的性能，甚至超过了 GPT-3.5-turbo。

Title: LESS: Selecting Influential Data for Targeted Instruction Tuning

• Authors: Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, Danqi Chen
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04333
• Pdf URL: https://arxiv.org/pdf/2402.04333
• Copy Paste: [[2402.04333]] LESS: Selecting Influential Data for Targeted Instruction Tuning(https://arxiv.org/abs/2402.04333)
• Keywords: language model, llm, chat, rag
• Abstract: Instruction tuning has unlocked powerful capabilities in large language models (LLMs), effectively using combined datasets to develop generalpurpose chatbots. However, real-world applications often require a specialized suite of skills (e.g., reasoning). The challenge lies in identifying the most relevant data from these extensive datasets to effectively develop specific capabilities, a setting we frame as targeted instruction tuning. We propose LESS, an optimizer-aware and practically efficient algorithm to effectively estimate data influences and perform Low-rank gradiEnt Similarity Search for instruction data selection. Crucially, LESS adapts existing influence formulations to work with the Adam optimizer and variable-length instruction data. LESS first constructs a highly reusable and transferable gradient datastore with low-dimensional gradient features and then selects examples based on their similarity to few-shot examples embodying a specific capability. Experiments show that training on a LESS-selected 5% of the data can often outperform training on the full dataset across diverse downstream tasks. Furthermore, the selected data is highly transferable: smaller models can be leveraged to select useful data for larger models and models from different families. Our qualitative analysis shows that our method goes beyond surface form cues to identify data that exemplifies the necessary reasoning skills for the intended downstream application.
• 摘要：指令调优释放了大型语言模型 (LLM) 的强大功能，有效地使用组合数据集来开发通用聊天机器人。然而，现实世界的应用程序通常需要一套专门的技能（例如推理）。挑战在于从这些广泛的数据集中识别最相关的数据，以有效地开发特定的功能，我们将这种设置称为有针对性的指令调整。我们提出了 LESS，一种优化器感知且实用有效的算法，可有效估计数据影响并执行低秩梯度相似性搜索以进行指令数据选择。至关重要的是，LESS 适应了现有的影响公式，以与 Adam 优化器和可变长度指令数据一起使用。 LESS 首先构建具有低维梯度特征的高度可重用和可转移的梯度数据存储，然后根据示例与体现特定功能的少数样本的相似性来选择示例。实验表明，在不同下游任务中，对较少选择的 5% 数据进行训练通常可以优于对完整数据集进行训练。此外，所选数据具有高度可转移性：可以利用较小的模型为较大的模型和来自不同系列的模型选择有用的数据。我们的定性分析表明，我们的方法超越了表面形式线索，可以识别能够体现预期下游应用所需推理技能的数据。

Title: LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Text

• Authors: Dor Bernsohn, Gil Semo, Yaron Vazana, Gila Hayat, Ben Hagag, Joel Niklaus, Rohit Saha, Kyryl Truskovskyi
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04335
• Pdf URL: https://arxiv.org/pdf/2402.04335
• Copy Paste: [[2402.04335]] LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Text(https://arxiv.org/abs/2402.04335)
• Keywords: language model, llm, code, rag
• Abstract: In this study, we focus on two main tasks, the first for detecting legal violations within unstructured textual data, and the second for associating these violations with potentially affected individuals. We constructed two datasets using Large Language Models (LLMs) which were subsequently validated by domain expert annotators. Both tasks were designed specifically for the context of class-action cases. The experimental design incorporated fine-tuning models from the BERT family and open-source LLMs, and conducting few-shot experiments using closed-source LLMs. Our results, with an F1-score of 62.69\% (violation identification) and 81.02\% (associating victims), show that our datasets and setups can be used for both tasks. Finally, we publicly release the datasets and the code used for the experiments in order to advance further research in the area of legal natural language processing (NLP).
• 摘要：在本研究中，我们重点关注两项主要任务，第一个任务是检测非结构化文本数据中的违法行为，第二个任务是将这些违法行为与潜在受影响的个人相关联。我们使用大型语言模型（LLM）构建了两个数据集，随后由领域专家注释者进行了验证。这两项任务都是专门针对集体诉讼案件而设计的。实验设计结合了 BERT 系列和开源 LLM 的微调模型，并使用闭源 LLM 进行了几次实验。我们的结果，F1 分数为 62.69%（违规识别）和 81.02%（关联受害者），表明我们的数据集和设置可用于这两项任务。最后，我们公开发布实验所用的数据集和代码，以推进法律自然语言处理（NLP）领域的进一步研究。

Title: Does Confidence Calibration Help Conformal Prediction?

• Authors: Huajun Xi, Jianguo Huang, Lei Feng, Hongxin Wei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04344
• Pdf URL: https://arxiv.org/pdf/2402.04344
• Copy Paste: [[2402.04344]] Does Confidence Calibration Help Conformal Prediction?(https://arxiv.org/abs/2402.04344)
• Keywords: rag
• Abstract: Conformal prediction, as an emerging uncertainty qualification technique, constructs prediction sets that are guaranteed to contain the true label with high probability. Previous works usually employ temperature scaling to calibrate the classifier, assuming that confidence calibration can benefit conformal prediction. In this work, we first show that post-hoc calibration methods surprisingly lead to larger prediction sets with improved calibration, while over-confidence with small temperatures benefits the conformal prediction performance instead. Theoretically, we prove that high confidence reduces the probability of appending a new class in the prediction set. Inspired by the analysis, we propose a novel method, $\textbf{Conformal Temperature Scaling}$ (ConfTS), which rectifies the objective through the gap between the threshold and the non-conformity score of the ground-truth label. In this way, the new objective of ConfTS will optimize the temperature value toward an optimal set that satisfies the $\textit{marginal coverage}$. Experiments demonstrate that our method can effectively improve widely-used conformal prediction methods.
• 摘要：保形预测作为一种新兴的不确定性限定技术，构建保证以高概率包含真实标签的预测集。以前的工作通常采用温度缩放来校准分类器，假设置信校准可以有利于保形预测。在这项工作中，我们首先表明，事后校准方法通过改进的校准令人惊讶地导致更大的预测集，而对小温度的过度自信反而有利于共形预测性能。从理论上讲，我们证明高置信度会降低在预测集中添加新类的概率。受分析的启发，我们提出了一种新方法，$\textbf{Conformal Thermal Scaling}$ (ConfTS)，它通过阈值与地面真实标签的不合格分数之间的差距来纠正目标。这样，ConfTS的新目标将优化温度值，使其达到满足$\textit{边缘覆盖}$的最佳集合。实验表明，我们的方法可以有效改进广泛使用的共形预测方法。

Title: The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry

• Authors: Michael Zhang, Kush Bhatia, Hermann Kumbong, Christopher Ré
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04347
• Pdf URL: https://arxiv.org/pdf/2402.04347
• Copy Paste: [[2402.04347]] The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry(https://arxiv.org/abs/2402.04347)
• Keywords: language model, gpt, code
• Abstract: Linear attentions have shown potential for improving Transformer efficiency, reducing attention's quadratic complexity to linear in sequence length. This holds exciting promise for (1) training linear Transformers from scratch, (2) "finetuned-conversion" of task-specific Transformers into linear versions that recover task performance, and (3) "pretrained-conversion" of Transformers such as large language models into linear versions finetunable on downstream tasks. However, linear attentions often underperform standard softmax attention in quality. To close this performance gap, we find prior linear attentions lack key properties of softmax attention tied to good performance: low-entropy (or "spiky") weights and dot-product monotonicity. We further observe surprisingly simple feature maps that retain these properties and match softmax performance, but are inefficient to compute in linear attention. We thus propose Hedgehog, a learnable linear attention that retains the spiky and monotonic properties of softmax attention while maintaining linear complexity. Hedgehog uses simple trainable MLPs to produce attention weights mimicking softmax attention. Experiments show Hedgehog recovers over 99% of standard Transformer quality in train-from-scratch and finetuned-conversion settings, outperforming prior linear attentions up to 6 perplexity points on WikiText-103 with causal GPTs, and up to 8.7 GLUE score points on finetuned bidirectional BERTs. Hedgehog also enables pretrained-conversion. Converting a pretrained GPT-2 into a linear attention variant achieves state-of-the-art 16.7 perplexity on WikiText-103 for 125M subquadratic decoder models. We finally turn a pretrained Llama-2 7B into a viable linear attention Llama. With low-rank adaptation, Hedgehog-Llama2 7B achieves 28.1 higher ROUGE-1 points over the base standard attention model, where prior linear attentions lead to 16.5 point drops.
• 摘要：线性注意力已经显示出提高 Transformer 效率的潜力，将注意力的二次复杂度降低到序列长度的线性。这为以下方面带来了令人兴奋的前景：(1) 从头开始​​训练线性 Transformer，(2) 将特定于任务的 Transformer“微调转换”为可恢复任务性能的线性版本，以及 (3) Transformer（例如大型语言）的“预训练转换”将模型转换为可在下游任务上微调的线性版本。然而，线性注意力在质量上常常低于标准的 softmax 注意力。为了缩小这种性能差距，我们发现先前的线性注意力缺乏与良好性能相关的 softmax 注意力的关键属性：低熵（或“尖峰”）权重和点积单调性。我们进一步观察到令人惊讶的简单特征图，它们保留了这些属性并匹配 softmax 性能，但在线性注意力中计算效率低下。因此，我们提出了 Hedgehog，一种可学习的线性注意力，它保留了 softmax 注意力的尖峰和单调特性，同时保持了线性复杂性。 Hedgehog 使用简单的可训练 MLP 来生成模仿 softmax 注意力的注意力权重。实验表明，Hedgehog 在从头开始训练和微调转换设置中恢复了 99% 以上的标准 Transformer 质量，在具有因果 GPT 的 WikiText-103 上优于先前的线性注意力，最多 6 个困惑点，在微调双向上最多 8.7 个 GLUE 得分点BERT。 Hedgehog 还支持预训练转换。将预训练的 GPT-2 转换为线性注意变体，可​​在 WikiText-103 上针对 125M 次二次解码器模型实现最先进的 16.7 困惑度。我们最终将预训练的 Llama-2 7B 转变为可行的线性注意力 Llama。通过低秩适应，Hedgehog-Llama2 7B 比基本标准注意力模型获得了 28.1 点更高的 ROUGE-1 点，而先前的线性注意力导致 16.5 点下降。

Title: Neural Networks Learn Statistics of Increasing Complexity

• Authors: Nora Belrose, Quintin Pope, Lucia Quirke, Alex Mallen, Xiaoli Fern
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04362
• Pdf URL: https://arxiv.org/pdf/2402.04362
• Copy Paste: [[2402.04362]] Neural Networks Learn Statistics of Increasing Complexity(https://arxiv.org/abs/2402.04362)
• Keywords: llm, code
• Abstract: The distributional simplicity bias (DSB) posits that neural networks learn low-order moments of the data distribution first, before moving on to higher-order correlations. In this work, we present compelling new evidence for the DSB by showing that networks automatically learn to perform well on maximum-entropy distributions whose low-order statistics match those of the training set early in training, then lose this ability later. We also extend the DSB to discrete domains by proving an equivalence between token $n$-gram frequencies and the moments of embedding vectors, and by finding empirical evidence for the bias in LLMs. Finally we use optimal transport methods to surgically edit the low-order statistics of one class to match those of another, and show that early-training networks treat the edited samples as if they were drawn from the target class. Code is available at https://github.com/EleutherAI/features-across-time.
• 摘要：分布简单性偏差 (DSB) 假设神经网络首先学习数据分布的低阶矩，然后再学习高阶相关性。在这项工作中，我们为 DSB 提供了令人信服的新证据，表明网络会自动学习在最大熵分布上表现良好，其低阶统计数据在训练早期与训练集的统计数据相匹配，然后失去这种能力。我们还通过证明 token $n$-gram 频率和嵌入向量矩之间的等价性，并通过寻找 LLM 偏差的经验证据，将 DSB 扩展到离散域。最后，我们使用最佳传输方法对一个类别的低阶统计数据进行外科手术编辑，以匹配另一类别的低阶统计数据，并表明早期训练网络将编辑后的样本视为从目标类别中提取的样本。代码可在 https://github.com/EleutherAI/features-across-time 获取。

Title: Pedestrian crossing decisions can be explained by bounded optimal decision-making under noisy visual perception

• Authors: Yueyang Wang, Aravinda Ramakrishnan Srinivasan, Jussi P.P. Jokinen, Antti Oulasvirta, Gustav Markkula
• Subjects: cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04370
• Pdf URL: https://arxiv.org/pdf/2402.04370
• Copy Paste: [[2402.04370]] Pedestrian crossing decisions can be explained by bounded optimal decision-making under noisy visual perception(https://arxiv.org/abs/2402.04370)
• Keywords: rag
• Abstract: This paper presents a model of pedestrian crossing decisions, based on the theory of computational rationality. It is assumed that crossing decisions are boundedly optimal, with bounds on optimality arising from human cognitive limitations. While previous models of pedestrian behaviour have been either 'black-box' machine learning models or mechanistic models with explicit assumptions about cognitive factors, we combine both approaches. Specifically, we model mechanistically noisy human visual perception and assumed rewards in crossing, but we use reinforcement learning to learn bounded optimal behaviour policy. The model reproduces a larger number of known empirical phenomena than previous models, in particular: (1) the effect of the time to arrival of an approaching vehicle on whether the pedestrian accepts the gap, the effect of the vehicle's speed on both (2) gap acceptance and (3) pedestrian timing of crossing in front of yielding vehicles, and (4) the effect on this crossing timing of the stopping distance of the yielding vehicle. Notably, our findings suggest that behaviours previously framed as 'biases' in decision-making, such as speed-dependent gap acceptance, might instead be a product of rational adaptation to the constraints of visual perception. Our approach also permits fitting the parameters of cognitive constraints and rewards per individual, to better account for individual differences. To conclude, by leveraging both RL and mechanistic modelling, our model offers novel insights about pedestrian behaviour, and may provide a useful foundation for more accurate and scalable pedestrian models.
• 摘要：本文提出了一种基于计算理性理论的行人过路决策模型。假设交叉决策是有界最优的，最优性的界限源于人类认知的局限性。虽然以前的行人行为模型要么是“黑匣子”机器学习模型，要么是对认知因素有明确假设的机械模型，但我们将这两种方法结合起来。具体来说，我们对人类视觉感知的机械噪声进行建模，并假定交叉时的奖励，但我们使用强化学习来学习有界最优行为策略。该模型比以前的模型重现了更多的已知经验现象，特别是：（1）接近车辆的到达时间对行人是否接受间隙的影响，车辆速度对两者的影响（2）间隙接受度和（3）行人在让行车辆前面过马路的时间，以及（4）让行车辆的停车距离对过马路时间的影响。值得注意的是，我们的研究结果表明，以前被视为决策“偏见”的行为，例如速度相关的间隙接受，可能是对视觉感知限制的理性适应的产物。我们的方法还允许拟合每个人的认知约束和奖励参数，以更好地解释个体差异。总而言之，通过利用强化学习和机械建模，我们的模型提供了有关行人行为的新颖见解，并可能为更准确和可扩展的行人模型提供有用的基础。

Title: $\texttt{NeRCC}$: Nested-Regression Coded Computing for Resilient Distributed Prediction Serving Systems

• Authors: Parsa Moradi, Mohammad Ali Maddah-Ali
• Subjects: cs.LG, cs.DC, cs.IT
• Abstract URL: https://arxiv.org/abs/2402.04377
• Pdf URL: https://arxiv.org/pdf/2402.04377
• Copy Paste: [[2402.04377]] $\texttt{NeRCC}$: Nested-Regression Coded Computing for Resilient Distributed Prediction Serving Systems(https://arxiv.org/abs/2402.04377)
• Keywords: code, rag
• Abstract: Resilience against stragglers is a critical element of prediction serving systems, tasked with executing inferences on input data for a pre-trained machine-learning model. In this paper, we propose NeRCC, as a general straggler-resistant framework for approximate coded computing. NeRCC includes three layers: (1) encoding regression and sampling, which generates coded data points, as a combination of original data points, (2) computing, in which a cluster of workers run inference on the coded data points, (3) decoding regression and sampling, which approximately recovers the predictions of the original data points from the available predictions on the coded data points. We argue that the overall objective of the framework reveals an underlying interconnection between two regression models in the encoding and decoding layers. We propose a solution to the nested regressions problem by summarizing their dependence on two regularization terms that are jointly optimized. Our extensive experiments on different datasets and various machine learning models, including LeNet5, RepVGG, and Vision Transformer (ViT), demonstrate that NeRCC accurately approximates the original predictions in a wide range of stragglers, outperforming the state-of-the-art by up to 23%.
• 摘要：针对落后者的弹性是预测服务系统的关键要素，其任务是对预先训练的机器学习模型的输入数据执行推理。在本文中，我们提出 NeRCC，作为近似编码计算的通用抗掉队框架。 NeRCC 包括三层：(1) 编码回归和采样，生成编码数据点，作为原始数据点的组合；(2) 计算，其中一组工作人员对编码数据点进行推理；(3) 解码回归和采样，从编码数据点的可用预测中近似恢复原始数据点的预测。我们认为该框架的总体目标揭示了编码层和解码层中两个回归模型之间的潜在互连。我们通过总结嵌套回归问题对联合优化的两个正则化项的依赖性，提出了嵌套回归问题的解决方案。我们对不同数据集和各种机器学习模型（包括 LeNet5、RepVGG 和 Vision Transformer (ViT)）进行的广泛实验表明，NeRCC 在各种落后者中准确地逼近原始预测，比最先进的模型高出至 23%。

Title: Fine-Tuned Language Models Generate Stable Inorganic Materials as Text

• Authors: Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, C. Lawrence Zitnick, Zachary Ulissi
• Subjects: cs.LG, cond-mat.mtrl-sci
• Abstract URL: https://arxiv.org/abs/2402.04379
• Pdf URL: https://arxiv.org/pdf/2402.04379
• Copy Paste: [[2402.04379]] Fine-Tuned Language Models Generate Stable Inorganic Materials as Text(https://arxiv.org/abs/2402.04379)
• Keywords: language model, llm, prompt, code
• Abstract: We propose fine-tuning large language models for generation of stable materials. While unorthodox, fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable, with around 90% of sampled structures obeying physical constraints on atom positions and charges. Using energy above hull calculations from both learned ML potentials and gold-standard DFT calculations, we show that our strongest model (fine-tuned LLaMA-2 70B) can generate materials predicted to be metastable at about twice the rate (49% vs 28%) of CDVAE, a competing diffusion model. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material, infilling of partial structures and text-conditional generation. Finally, we show that language models' ability to capture key symmetries of crystal structures improves with model scale, suggesting that the biases of pretrained LLMs are surprisingly well-suited for atomistic data.
• 摘要：我们建议微调大型语言模型以生成稳定的材料。虽然非正统，但在文本编码的原子数据上微调大型语言模型实现起来很简单而且可靠，大约 90% 的采样结构遵守原子位置和电荷的物理约束。使用来自学习的 ML 势和黄金标准 DFT 计算的船体能量计算，我们表明我们最强大的模型（微调的 LLaMA-2 70B）可以以大约两倍的速度生成预计亚稳态的材料（49% vs 28%） ）CDVAE，一种竞争扩散模型。由于文本提示固有的灵活性，我们的模型可以同时用于稳定材料的无条件生成、部分结构的填充和文本条件生成。最后，我们表明语言模型捕获晶体结构关键对称性的能力随着模型规模的增加而提高，这表明预训练的 LLM 的偏差非常适合原子数据。

Title: FairWire: Fair Graph Generation

• Authors: O. Deniz Kose, Yanning Shen
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2402.04383
• Pdf URL: https://arxiv.org/pdf/2402.04383
• Copy Paste: [[2402.04383]] FairWire: Fair Graph Generation(https://arxiv.org/abs/2402.04383)
• Keywords: rag
• Abstract: Machine learning over graphs has recently attracted growing attention due to its ability to analyze and learn complex relations within critical interconnected systems. However, the disparate impact that is amplified by the use of biased graph structures in these algorithms has raised significant concerns for the deployment of them in real-world decision systems. In addition, while synthetic graph generation has become pivotal for privacy and scalability considerations, the impact of generative learning algorithms on the structural bias has not yet been investigated. Motivated by this, this work focuses on the analysis and mitigation of structural bias for both real and synthetic graphs. Specifically, we first theoretically analyze the sources of structural bias that result in disparity for the predictions of dyadic relations. To alleviate the identified bias factors, we design a novel fairness regularizer that offers a versatile use. Faced with the bias amplification in graph generation models that is brought to light in this work, we further propose a fair graph generation framework, FairWire, by leveraging our fair regularizer design in a generative model. Experimental results on real-world networks validate that the proposed tools herein deliver effective structural bias mitigation for both real and synthetic graphs.
• 摘要：基于图的机器学习最近引起了越来越多的关注，因为它能够分析和学习关键互连系统中的复杂关系。然而，在这些算法中使用有偏差的图结构会放大不同的影响，这引起了人们对在现实世界的决策系统中部署它们的严重担忧。此外，虽然合成图生成已成为隐私和可扩展性考虑的关键，但生成学习算法对结构偏差的影响尚未得到研究。受此启发，这项工作的重点是分析和减轻真实图和合成图的结构偏差。具体来说，我们首先从理论上分析导致二元关系预测不一致的结构偏差的来源。为了减轻已确定的偏差因素，我们设计了一种新颖的公平正则化器，它具有多种用途。面对这项工作中揭示的图生成模型中的偏差放大，我们通过在生成模型中利用我们的公平正则化器设计，进一步提出了一个公平的图生成框架 FairWire。现实世界网络的实验结果验证了本文提出的工具可以为真实图和合成图提供有效的结构偏差缓解。

Title: QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks

• Authors: Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, Christopher De Sa
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04396
• Pdf URL: https://arxiv.org/pdf/2402.04396
• Copy Paste: [[2402.04396]] QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks(https://arxiv.org/abs/2402.04396)
• Keywords: llm, code
• Abstract: Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing their weights to low-precision. In this work, we introduce QuIP#, a weight-only PTQ method that achieves state-of-the-art results in extreme compression regimes ($\le$ 4 bits per weight) using three novel techniques. First, QuIP# improves the incoherence processing from QuIP by using the randomized Hadamard transform, which is faster and has better theoretical properties. Second, QuIP# uses vector quantization techniques to take advantage of the ball-shaped sub-Gaussian distribution that incoherent weights possess: specifically, we introduce a set of hardware-efficient codebooks based on the highly symmetric $E_8$ lattice, which achieves the optimal 8-dimension unit ball packing. Third, QuIP# uses fine-tuning to improve fidelity to the original model. Our experiments show that QuIP# outperforms existing PTQ methods, enables new behaviors in PTQ scaling, and supports fast inference.
• 摘要：训练后量化 (PTQ) 通过将权重量化为低精度来减少 LLM 的内存占用。在这项工作中，我们介绍了 QuIP#，这是一种仅权重 PTQ 方法，它使用三种新技术在极端压缩机制（每个权重 4 位）中实现了最先进的结果。首先，QuIP# 通过使用随机 Hadamard 变换改进了 QuIP 的不相干处理，该变换速度更快且具有更好的理论特性。其次，QuIP#使用矢量量化技术来利用不相干权重所具有的球形亚高斯分布：具体来说，我们引入了一组基于高度对称$E_8$晶格的硬件高效码本，它实现了最优8维单元球封装。第三，QuIP# 使用微调来提高原始模型的保真度。我们的实验表明，QuIP# 优于现有的 PTQ 方法，支持 PTQ 扩展中的新行为，并支持快速推理。

Title: CEHR-GPT: Generating Electronic Health Records with Chronological Patient Timelines

• Authors: Chao Pang, Xinzhuo Jiang, Nishanth Parameshwar Pavinkurve, Krishna S. Kalluri, Elise L. Minto, Jason Patterson, Linying Zhang, George Hripcsak, Noémie Elhadad, Karthik Natarajan
• Subjects: cs.LG, cs.AI, cs.CY
• Abstract URL: https://arxiv.org/abs/2402.04400
• Pdf URL: https://arxiv.org/pdf/2402.04400
• Copy Paste: [[2402.04400]] CEHR-GPT: Generating Electronic Health Records with Chronological Patient Timelines(https://arxiv.org/abs/2402.04400)
• Keywords: gpt, rag
• Abstract: Synthetic Electronic Health Records (EHR) have emerged as a pivotal tool in advancing healthcare applications and machine learning models, particularly for researchers without direct access to healthcare data. Although existing methods, like rule-based approaches and generative adversarial networks (GANs), generate synthetic data that resembles real-world EHR data, these methods often use a tabular format, disregarding temporal dependencies in patient histories and limiting data replication. Recently, there has been a growing interest in leveraging Generative Pre-trained Transformers (GPT) for EHR data. This enables applications like disease progression analysis, population estimation, counterfactual reasoning, and synthetic data generation. In this work, we focus on synthetic data generation and demonstrate the capability of training a GPT model using a particular patient representation derived from CEHR-BERT, enabling us to generate patient sequences that can be seamlessly converted to the Observational Medical Outcomes Partnership (OMOP) data format.
• 摘要：综合电子健康记录 (EHR) 已成为推进医疗保健应用和机器学习模型的关键工具，特别是对于无法直接访问医疗保健数据的研究人员而言。尽管基于规则的方法和生成对抗网络 (GAN) 等现有方法可以生成类似于现实世界 EHR 数据的合成数据，但这些方法通常使用表格格式，忽略患者病史中的时间依赖性并限制数据复制。最近，人们对利用生成式预训练 Transformer (GPT) 处理 EHR 数据越来越感兴趣。这使得疾病进展分析、人口估计、反事实推理和合成数据生成等应用成为可能。在这项工作中，我们专注于合成数据生成，并展示了使用源自 CEHR-BERT 的特定患者表示来训练 GPT 模型的能力，使我们能够生成可以无缝转换为观察性医疗结果合作伙伴关系 (OMOP) 的患者序列数据格式。

Title: Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning

• Authors: Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, Meng Jiang
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04401
• Pdf URL: https://arxiv.org/pdf/2402.04401
• Copy Paste: [[2402.04401]] Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning(https://arxiv.org/abs/2402.04401)
• Keywords: language model, llm, prompt
• Abstract: Personalization in large language models (LLMs) is increasingly important, aiming to align LLM's interactions, content, and recommendations with individual user preferences. Recent advances in LLM personalization have spotlighted effective prompt design, by enriching user queries with non-parametric knowledge through behavior history retrieval and textual profiles. However, these approaches were limited due to a lack of model ownership, resulting in constrained customization and privacy issues. Moreover, they often failed to accurately capture user behavior patterns, especially in cases where user data were complex and dynamic. To address these shortcomings, we introduce One PEFT Per User (OPPU), which employs personalized parameter-efficient fine-tuning (PEFT) modules, to store user-specific behavior patterns and preferences. By plugging in users' personal PEFT parameters, they can own and use their LLMs personally. OPPU integrates parametric user knowledge in the personal PEFT parameters with the non-parametric knowledge acquired through retrieval and profile. This integration adapts individual LLMs to user behavior shifts. Experimental results demonstrate that OPPU significantly outperforms existing prompt-based methods across seven diverse tasks in the LaMP benchmark. Further in-depth studies reveal OPPU's enhanced capabilities in handling user behavior shifts, modeling users at different active levels, maintaining robustness across various user history formats, and displaying versatility with different PEFT methods.
• 摘要：大语言模型 (LLM) 中的个性化变得越来越重要，旨在使 LLM 的交互、内容和推荐与个人用户偏好保持一致。 LLM个性化的最新进展突出了有效的提示设计，通过行为历史检索和文本配置文件用非参数知识丰富用户查询。然而，由于缺乏模型所有权，这些方法受到限制，导致定制受限和隐私问题。此外，它们常常无法准确捕获用户行为模式，特别是在用户数据复杂且动态的情况下。为了解决这些缺点，我们引入了每个用户一个 PEFT (OPPU)，它采用个性化参数高效微调 (PEFT) 模块来存储用户特定的行为模式和偏好。通过插入用户的个人 PEFT 参数，他们可以个人拥有和使用他们的 LLM。 OPPU 将个人 PEFT 参数中的参数化用户知识与通过检索和配置文件获取的非参数化知识相结合。这种集成使各个法学硕士适应用户行为的变化。实验结果表明，OPPU 在 LaMP 基准测试中的七个不同任务中显着优于现有的基于提示的方法。进一步的深入研究揭示了 OPPU 在处理用户行为变化、对不同活跃级别的用户进行建模、在各种用户历史格式中保持鲁棒性以及通过不同 PEFT 方法显示多功能性方面的增强能力。

Title: Chatbot Meets Pipeline: Augment Large Language Model with Definite Finite Automaton

• Authors: Yiyou Sun, Junjie Hu, Wei Cheng, Haifeng Chen
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04411
• Pdf URL: https://arxiv.org/pdf/2402.04411
• Copy Paste: [[2402.04411]] Chatbot Meets Pipeline: Augment Large Language Model with Definite Finite Automaton(https://arxiv.org/abs/2402.04411)
• Keywords: language model, llm, chat, agent
• Abstract: This paper introduces the Definite Finite Automaton augmented large language model (DFA-LLM), a novel framework designed to enhance the capabilities of conversational agents using large language models (LLMs). Traditional LLMs face challenges in generating regulated and compliant responses in special scenarios with predetermined response guidelines, like emotional support and customer service. Our framework addresses these challenges by embedding a Definite Finite Automaton (DFA), learned from training dialogues, within the LLM. This structured approach enables the LLM to adhere to a deterministic response pathway, guided by the DFA. The advantages of DFA-LLM include an interpretable structure through human-readable DFA, context-aware retrieval for responses in conversations, and plug-and-play compatibility with existing LLMs. Extensive benchmarks validate DFA-LLM's effectiveness, indicating its potential as a valuable contribution to the conversational agent.
• 摘要：本文介绍了定有限自动机增强大语言模型 (DFA-LLM)，这是一种新颖的框架，旨在增强使用大语言模型 (LLM) 的会话代理的功能。传统的法学硕士面临着在特殊情况下通过预定的响应指南（例如情感支持和客户服务）生成规范且合规的响应的挑战。我们的框架通过在法学硕士中嵌入从培训对话中学到的定有限自动机（DFA）来解决这些挑战。这种结构化方法使法学硕士能够遵循由 DFA 指导的确定性响应路径。 DFA-LLM 的优点包括通过人类可读的 DFA 实现的可解释结构、对话中响应的上下文感知检索以及与现有 LLM 的即插即用兼容性。广泛的基准验证了 DFA-LLM 的有效性，表明其对对话代理做出宝贵贡献的潜力。

Title: The VampPrior Mixture Model

• Authors: Andrew Stirn, David A. Knowles
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2402.04412
• Pdf URL: https://arxiv.org/pdf/2402.04412
• Copy Paste: [[2402.04412]] The VampPrior Mixture Model(https://arxiv.org/abs/2402.04412)
• Keywords: code
• Abstract: Current clustering priors for deep latent variable models (DLVMs) require defining the number of clusters a-priori and are susceptible to poor initializations. Addressing these deficiencies could greatly benefit deep learning-based scRNA-seq analysis by performing integration and clustering simultaneously. We adapt the VampPrior (Tomczak & Welling, 2018) into a Dirichlet process Gaussian mixture model, resulting in the VampPrior Mixture Model (VMM), a novel prior for DLVMs. We propose an inference procedure that alternates between variational inference and Empirical Bayes to cleanly distinguish variational and prior parameters. Using the VMM in a Variational Autoencoder attains highly competitive clustering performance on benchmark datasets. Augmenting scVI (Lopez et al., 2018), a popular scRNA-seq integration method, with the VMM significantly improves its performance and automatically arranges cells into biologically meaningful clusters.
• 摘要：当前深度潜变量模型 (DLVM) 的聚类先验需要先验定义聚类数量，并且容易受到初始化不良的影响。通过同时执行集成和聚类，解决这些缺陷可以极大地有益于基于深度学习的 scRNA-seq 分析。我们将 VampPrior (Tomczak & Welling, 2018) 改编为狄利克雷过程高斯混合模型，产生了 VampPrior 混合模型 (VMM)，这是 DLVM 的一种新颖先验。我们提出了一种在变分推理和经验贝叶斯之间交替的推理过程，以清楚地区分变分参数和先验参数。在变分自动编码器中使用 VMM 可在基准数据集上获得极具竞争力的聚类性能。使用 VMM 增强 scVI（Lopez 等人，2018）是一种流行的 scRNA-seq 整合方法，可显着提高其性能并自动将细胞排列成具有生物学意义的簇。

Title: Decentralized Blockchain-based Robust Multi-agent Multi-armed Bandit

• Authors: Mengfan Xu, Diego Klabjan
• Subjects: cs.LG, cs.MA
• Abstract URL: https://arxiv.org/abs/2402.04417
• Pdf URL: https://arxiv.org/pdf/2402.04417
• Copy Paste: [[2402.04417]] Decentralized Blockchain-based Robust Multi-agent Multi-armed Bandit(https://arxiv.org/abs/2402.04417)
• Keywords: rag, agent
• Abstract: We study a robust multi-agent multi-armed bandit problem where multiple clients or participants are distributed on a fully decentralized blockchain, with the possibility of some being malicious. The rewards of arms are homogeneous among the clients, following time-invariant stochastic distributions that are revealed to the participants only when the system is secure enough. The system's objective is to efficiently ensure the cumulative rewards gained by the honest participants. To this end and to the best of our knowledge, we are the first to incorporate advanced techniques from blockchains, as well as novel mechanisms, into the system to design optimal strategies for honest participants. This allows various malicious behaviors and the maintenance of participant privacy. More specifically, we randomly select a pool of validators who have access to all participants, design a brand-new consensus mechanism based on digital signatures for these validators, invent a UCB-based strategy that requires less information from participants through secure multi-party computation, and design the chain-participant interaction and an incentive mechanism to encourage participants' participation. Notably, we are the first to prove the theoretical guarantee of the proposed algorithms by regret analyses in the context of optimality in blockchains. Unlike existing work that integrates blockchains with learning problems such as federated learning which mainly focuses on numerical optimality, we demonstrate that the regret of honest participants is upper bounded by $log{T}$. This is consistent with the multi-agent multi-armed bandit problem without malicious participants and the robust multi-agent multi-armed bandit problem with purely Byzantine attacks.
• 摘要：我们研究了一个强大的多代理多臂强盗问题，其中多个客户端或参与者分布在完全去中心化的区块链上，其中一些可能是恶意的。武器的奖励在客户之间是同质的，遵循时不变的随机分布，只有当系统足够安全时才会向参与者透露。系统的目标是有效保证诚实参与者获得的累积奖励。为此，据我们所知，我们是第一个将区块链的先进技术以及新颖的机制纳入系统中的公司，为诚实的参与者设计最佳策略。这允许各种恶意行为并维护参与者的隐私。更具体地说，我们随机选择一个可以访问所有参与者的验证者池，为这些验证者设计一个基于数字签名的全新共识机制，发明一种基于 UCB 的策略，通过安全的多方计算需要更少的参与者信息，设计链-参与者互动和激励机制，鼓励参与者参与。值得注意的是，我们是第一个在区块链最优性背景下通过遗憾分析证明所提出算法的理论保证的人。与将区块链与主要关注数值最优性的联邦学习等学习问题相结合的现有工作不同，我们证明诚实参与者的遗憾上限为 $log{T}$。这与没有恶意参与者的多智能体多臂老虎机问题和具有纯拜占庭攻击的鲁棒多智能体多臂老虎机问题是一致的。

Title: PreGIP: Watermarking the Pretraining of Graph Neural Networks for Deep Intellectual Property Protection

• Authors: Enyan Dai, Minhua Lin, Suhang Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04435
• Pdf URL: https://arxiv.org/pdf/2402.04435
• Copy Paste: [[2402.04435]] PreGIP: Watermarking the Pretraining of Graph Neural Networks for Deep Intellectual Property Protection(https://arxiv.org/abs/2402.04435)
• Keywords: code
• Abstract: Pretraining on Graph Neural Networks (GNNs) has shown great power in facilitating various downstream tasks. As pretraining generally requires huge amount of data and computational resources, the pretrained GNNs are high-value Intellectual Properties (IP) of the legitimate owner. However, adversaries may illegally copy and deploy the pretrained GNN models for their downstream tasks. Though initial efforts have been made to watermark GNN classifiers for IP protection, these methods require the target classification task for watermarking, and thus are not applicable to self-supervised pretraining of GNN models. Hence, in this work, we propose a novel framework named PreGIP to watermark the pretraining of GNN encoder for IP protection while maintain the high-quality of the embedding space. PreGIP incorporates a task-free watermarking loss to watermark the embedding space of pretrained GNN encoder. A finetuning-resistant watermark injection is further deployed. Theoretical analysis and extensive experiments show the effectiveness of {\method} in IP protection and maintaining high-performance for downstream tasks.
• 摘要：图神经网络（GNN）的预训练在促进各种下游任务方面表现出了强大的力量。由于预训练通常需要大量的数据和计算资源，因此预训练的 GNN 是合法所有者的高价值知识产权（IP）。然而，攻击者可能会非法复制和部署预训练的 GNN 模型以用于其下游任务。尽管已经初步尝试对 GNN 分类器进行水印以进行 IP 保护，但这些方法需要水印的目标分类任务，因此不适用于 GNN 模型的自监督预训练。因此，在这项工作中，我们提出了一种名为 PreGIP 的新颖框架，对 GNN 编码器的预训练进行水印以保护 IP，同时保持嵌入空间的高质量。 PreGIP 结合了无任务水印损失来对预训练 GNN 编码器的嵌入空间进行水印。进一步部署了抗微调水印注入。理论分析和大量实验表明了{\method}在IP保护和保持下游任务高性能方面的有效性。

Title: Structured Entity Extraction Using Large Language Models

• Authors: Haolun Wu, Ye Yuan, Liana Mikaelyan, Alexander Meulemans, Xue Liu, James Hensman, Bhaskar Mitra
• Subjects: cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04437
• Pdf URL: https://arxiv.org/pdf/2402.04437
• Copy Paste: [[2402.04437]] Structured Entity Extraction Using Large Language Models(https://arxiv.org/abs/2402.04437)
• Keywords: language model, llm
• Abstract: Recent advances in machine learning have significantly impacted the field of information extraction, with Large Language Models (LLMs) playing a pivotal role in extracting structured information from unstructured text. This paper explores the challenges and limitations of current methodologies in structured entity extraction and introduces a novel approach to address these issues. We contribute to the field by first introducing and formalizing the task of Structured Entity Extraction (SEE), followed by proposing Approximate Entity Set OverlaP (AESOP) Metric designed to appropriately assess model performance on this task. Later, we propose a new model that harnesses the power of LLMs for enhanced effectiveness and efficiency through decomposing the entire extraction task into multiple stages. Quantitative evaluation and human side-by-side evaluation confirm that our model outperforms baselines, offering promising directions for future advancements in structured entity extraction.
• 摘要：机器学习的最新进展对信息提取领域产生了重大影响，其中大型语言模型 (LLM) 在从非结构化文本中提取结构化信息方面发挥着关键作用。本文探讨了结构化实体提取中当前方法的挑战和局限性，并介绍了一种解决这些问题的新方法。我们首先引入并形式化结构化实体提取 (SEE) 任务，然后提出近似实体集重叠 (AESOP) 指标，旨在适当评估该任务的模型性能，从而为该领域做出贡献。后来，我们提出了一种新模型，通过将整个提取任务分解为多个阶段，利用法学硕士的力量来提高有效性和效率。定量评估和人类并行评估证实我们的模型优于基线，为结构化实体提取的未来进步提供了有希望的方向。

Title: Evaluating Embeddings for One-Shot Classification of Doctor-AI Consultations

• Authors: Olumide Ebenezer Ojo, Olaronke Oluwayemisi Adebanji, Alexander Gelbukh, Hiram Calvo, Anna Feldman
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04442
• Pdf URL: https://arxiv.org/pdf/2402.04442
• Copy Paste: [[2402.04442]] Evaluating Embeddings for One-Shot Classification of Doctor-AI Consultations(https://arxiv.org/abs/2402.04442)
• Keywords: gpt
• Abstract: Effective communication between healthcare providers and patients is crucial to providing high-quality patient care. In this work, we investigate how Doctor-written and AI-generated texts in healthcare consultations can be classified using state-of-the-art embeddings and one-shot classification systems. By analyzing embeddings such as bag-of-words, character n-grams, Word2Vec, GloVe, fastText, and GPT2 embeddings, we examine how well our one-shot classification systems capture semantic information within medical consultations. Results show that the embeddings are capable of capturing semantic features from text in a reliable and adaptable manner. Overall, Word2Vec, GloVe and Character n-grams embeddings performed well, indicating their suitability for modeling targeted to this task. GPT2 embedding also shows notable performance, indicating its suitability for models tailored to this task as well. Our machine learning architectures significantly improved the quality of health conversations when training data are scarce, improving communication between patients and healthcare providers.
• 摘要：医疗保健提供者和患者之间的有效沟通对于提供高质量的患者护理至关重要。在这项工作中，我们研究了如何使用最先进的嵌入和一次性分类系统对医疗保健咨询中医生编写和人工智能生成的文本进行分类。通过分析词袋、字符 n-gram、Word2Vec、GloVe、fastText 和 GPT2 嵌入等嵌入，我们检查了一次性分类系统在医疗咨询中捕获语义信息的效果。结果表明，嵌入能够以可靠且适应性强的方式从文本中捕获语义特征。总体而言，Word2Vec、GloVe 和 Character n-gram 嵌入表现良好，表明它们适合针对此任务进行建模。 GPT2 嵌入也显示出显着的性能，表明它也适用于针对此任务定制的模型。当训练数据稀缺时，我们的机器学习架构显着提高了健康对话的质量，改善了患者和医疗保健提供者之间的沟通。

Title: DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems

• Authors: Yair Schiff, Zhong Yi Wan, Jeffrey B. Parker, Stephan Hoyer, Volodymyr Kuleshov, Fei Sha, Leonardo Zepeda-Núñez
• Subjects: cs.LG, math.DS
• Abstract URL: https://arxiv.org/abs/2402.04467
• Pdf URL: https://arxiv.org/pdf/2402.04467
• Copy Paste: [[2402.04467]] DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems(https://arxiv.org/abs/2402.04467)
• Keywords: rag
• Abstract: Learning dynamics from dissipative chaotic systems is notoriously difficult due to their inherent instability, as formalized by their positive Lyapunov exponents, which exponentially amplify errors in the learned dynamics. However, many of these systems exhibit ergodicity and an attractor: a compact and highly complex manifold, to which trajectories converge in finite-time, that supports an invariant measure, i.e., a probability distribution that is invariant under the action of the dynamics, which dictates the long-term statistical behavior of the system. In this work, we leverage this structure to propose a new framework that targets learning the invariant measure as well as the dynamics, in contrast with typical methods that only target the misfit between trajectories, which often leads to divergence as the trajectories' length increases. We use our framework to propose a tractable and sample efficient objective that can be used with any existing learning objectives. Our Dynamics Stable Learning by Invariant Measures (DySLIM) objective enables model training that achieves better point-wise tracking and long-term statistical accuracy relative to other learning objectives. By targeting the distribution with a scalable regularization term, we hope that this approach can be extended to more complex systems exhibiting slowly-variant distributions, such as weather and climate models.
• 摘要：从耗散混沌系统中学习动力学是出了名的困难，因为它们固有的不稳定性，正如其正李雅普诺夫指数所形式化的那样，它会指数级地放大学习动力学中的误差。然而，这些系统中的许多都表现出遍历性和吸引子：一个紧凑且高度复杂的流形，轨迹在有限时间内收敛到该流形，支持不变测度，即在动力学作用下不变的概率分布，决定了系统的长期统计行为。在这项工作中，我们利用这种结构提出了一个新的框架，该框架的目标是学习不变度量和动态，这与仅针对轨迹之间的失配的典型方法形成鲜明对比，随着轨迹长度的增加，这通常会导致发散。我们使用我们的框架提出一个易于处理且样本高效的目标，可以与任何现有的学习目标一起使用。我们的不变测量动态稳定学习 (DySLIM) 目标使模型训练能够实现相对于其他学习目标更好的逐点跟踪和长期统计准确性。通过使用可扩展的正则化项来定位分布，我们希望这种方法可以扩展到表现出缓慢变化分布的更复杂的系统，例如天气和气候模型。

Title: Detecting Mode Collapse in Language Models via Narration

• Authors: Sil Hamilton
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04477
• Pdf URL: https://arxiv.org/pdf/2402.04477
• Copy Paste: [[2402.04477]] Detecting Mode Collapse in Language Models via Narration(https://arxiv.org/abs/2402.04477)
• Keywords: language model, gpt
• Abstract: No two authors write alike. Personal flourishes invoked in written narratives, from lexicon to rhetorical devices, imply a particular author--what literary theorists label the implied or virtual author; distinct from the real author or narrator of a text. Early large language models trained on unfiltered training sets drawn from a variety of discordant sources yielded incoherent personalities, problematic for conversational tasks but proving useful for sampling literature from multiple perspectives. Successes in alignment research in recent years have allowed researchers to impose subjectively consistent personae on language models via instruction tuning and reinforcement learning from human feedback (RLHF), but whether aligned models retain the ability to model an arbitrary virtual author has received little scrutiny. By studying 4,374 stories sampled from three OpenAI language models, we show successive versions of GPT-3 suffer from increasing degrees of "mode collapse" whereby overfitting the model during alignment constrains it from generalizing over authorship: models suffering from mode collapse become unable to assume a multiplicity of perspectives. Our method and results are significant for researchers seeking to employ language models in sociological simulations.
• 摘要：没有两个作者写得一样。从词汇到修辞手段，书面叙述中引用的个人华丽辞藻都暗示着某个特定的作者——文学理论家将其称为隐含的或虚拟的作者；与文本的真实作者或叙述者不同。早期的大型语言模型在来自各种不一致来源的未经过滤的训练集上进行训练，产生了不连贯的个性，这对于对话任务来说是有问题的，但事实证明对于从多个角度采样文献是有用的。近年来对齐研究的成功使研究人员能够通过指令调整和人类反馈的强化学习（RLHF）将主观一致的角色强加于语言模型，但对齐模型是否保留对任意虚拟作者进行建模的能力却很少受到关注。通过研究从三种 OpenAI 语言模型中采样的 4,374 个故事，我们发现 GPT-3 的连续版本遭受了越来越严重的“模式崩溃”，从而在对齐过程中过度拟合模型限制了它对作者身份的泛化：遭受模式崩溃的模型变得无法假设多种观点。我们的方法和结果对于寻求在社会学模拟中使用语言模型的研究人员来说具有重要意义。

Title: De-amplifying Bias from Differential Privacy in Language Model Fine-tuning

• Authors: Sanjari Srivastava, Piotr Mardziel, Zhikhun Zhang, Archana Ahlawat, Anupam Datta, John C Mitchell
• Subjects: cs.LG, cs.CR, cs.CY, stat.ME
• Abstract URL: https://arxiv.org/abs/2402.04489
• Pdf URL: https://arxiv.org/pdf/2402.04489
• Copy Paste: [[2402.04489]] De-amplifying Bias from Differential Privacy in Language Model Fine-tuning(https://arxiv.org/abs/2402.04489)
• Keywords: language model, llm
• Abstract: Fairness and privacy are two important values machine learning (ML) practitioners often seek to operationalize in models. Fairness aims to reduce model bias for social/demographic sub-groups. Privacy via differential privacy (DP) mechanisms, on the other hand, limits the impact of any individual's training data on the resulting model. The trade-offs between privacy and fairness goals of trustworthy ML pose a challenge to those wishing to address both. We show that DP amplifies gender, racial, and religious bias when fine-tuning large language models (LLMs), producing models more biased than ones fine-tuned without DP. We find the cause of the amplification to be a disparity in convergence of gradients across sub-groups. Through the case of binary gender bias, we demonstrate that Counterfactual Data Augmentation (CDA), a known method for addressing bias, also mitigates bias amplification by DP. As a consequence, DP and CDA together can be used to fine-tune models while maintaining both fairness and privacy.
• 摘要：公平和隐私是机器学习 (ML) 从业者经常寻求在模型中实现的两个重要价值观。公平旨在减少社会/人口亚群体的模型偏差。另一方面，通过差分隐私 (DP) 机制实现的隐私限制了任何个人的训练数据对结果模型的影响。值得信赖的机器学习的隐私和公平目标之间的权衡对那些希望同时解决这两个问题的人提出了挑战。我们表明，在微调大型语言模型 (LLM) 时，DP 会放大性别、种族和宗教偏见，产生的模型比没有 DP 微调的模型更具偏见。我们发现放大的原因是子组间梯度收敛的差异。通过二元性别偏见的案例，我们证明了反事实数据增强（CDA）这种解决偏见的已知方法也可以减轻 DP 造成的偏见放大。因此，DP 和 CDA 一起可以用来微调模型，同时保持公平性和隐私性。

Title: Grandmaster-Level Chess Without Search

• Authors: Anian Ruoss, Grégoire Delétang, Sourabh Medapati, Jordi Grau-Moya, Li Kevin Wenliang, Elliot Catt, John Reid, Tim Genewein
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2402.04494
• Pdf URL: https://arxiv.org/pdf/2402.04494
• Copy Paste: [[2402.04494]] Grandmaster-Level Chess Without Search(https://arxiv.org/abs/2402.04494)
• Keywords: gpt
• Abstract: The recent breakthrough successes in machine learning are mainly attributed to scale: namely large-scale attention-based architectures and datasets of unprecedented scale. This paper investigates the impact of training at scale for chess. Unlike traditional chess engines that rely on complex heuristics, explicit search, or a combination of both, we train a 270M parameter transformer model with supervised learning on a dataset of 10 million chess games. We annotate each board in the dataset with action-values provided by the powerful Stockfish 16 engine, leading to roughly 15 billion data points. Our largest model reaches a Lichess blitz Elo of 2895 against humans, and successfully solves a series of challenging chess puzzles, without any domain-specific tweaks or explicit search algorithms. We also show that our model outperforms AlphaZero's policy and value networks (without MCTS) and GPT-3.5-turbo-instruct. A systematic investigation of model and dataset size shows that strong chess performance only arises at sufficient scale. To validate our results, we perform an extensive series of ablations of design choices and hyperparameters.
• 摘要：最近机器学习领域的突破性成功主要归功于规模：即大规模的基于注意力的架构和前所未有的规模的数据集。本文研究了大规模训练对国际象棋的影响。与依赖复杂启发式、显式搜索或两者结合的传统国际象棋引擎不同，我们在 1000 万场国际象棋比赛的数据集上通过监督学习训练 270M 参数转换器模型。我们使用强大的 Stockfish 16 引擎提供的动作值对数据集中的每个板进行注释，从而产生大约 150 亿个数据点。我们最大的模型对人类的 Lichess blitz Elo 达到了 2895，并成功解决了一系列具有挑战性的国际象棋难题，无需任何特定领域的调整或显式搜索算法。我们还表明，我们的模型优于 AlphaZero 的策略和价值网络（没有 MCTS）和 GPT-3.5-turbo-instruct。对模型和数据集大小的系统研究表明，强大的国际象棋表现只有在足够规模的情况下才会出现。为了验证我们的结果，我们对设计选择和超参数进行了一系列广泛的消融。

Title: The Fine-Grained Complexity of Gradient Computation for Training Large Language Models

• Authors: Josh Alman, Zhao Song
• Subjects: cs.LG, cs.CC, cs.CL, cs.DS
• Abstract URL: https://arxiv.org/abs/2402.04497
• Pdf URL: https://arxiv.org/pdf/2402.04497
• Copy Paste: [[2402.04497]] The Fine-Grained Complexity of Gradient Computation for Training Large Language Models(https://arxiv.org/abs/2402.04497)
• Keywords: language model, llm
• Abstract: Large language models (LLMs) have made fundamental contributions over the last a few years. To train an LLM, one needs to alternatingly run `forward' computations and `backward' computations. The forward computation can be viewed as attention function evaluation, and the backward computation can be viewed as a gradient computation. In previous work by [Alman and Song, NeurIPS 2023], it was proved that the forward step can be performed in almost-linear time in certain parameter regimes, but that there is no truly sub-quadratic time algorithm in the remaining parameter regimes unless the popular hypothesis SETH is false. In this work, we show nearly identical results for the harder-seeming problem of computing the gradient of loss function of one layer attention network, and thus for the entire process of LLM training. This completely characterizes the fine-grained complexity of every step of LLM training.
• 摘要：大型语言模型（LLM）在过去几年中做出了基础性贡献。为了训练法学硕士，需要交替运行“前向”计算和“后向”计算。前向计算可以看作是注意力函数评估，后向计算可以看作是梯度计算。在[Alman and Song, NeurIPS 2023]之前的工作中，证明了前向步骤可以在某些参数范围内以几乎线性的时间执行，但在其余参数范围内不存在真正的次二次时间算法，除非流行的假设 SETH 是错误的。在这项工作中，我们对于计算一层注意力网络损失函数梯度的看似困难的问题以及 LLM 训练的整个过程显示了几乎相同的结果。这完全体现了LLM培训每一步的细粒度复杂性。

Title: Online Cascade Learning for Efficient Inference over Streams

• Authors: Lunyiu Nie, Zhimin Ding, Erdong Hu, Christopher Jermaine, Swarat Chaudhuri
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04513
• Pdf URL: https://arxiv.org/pdf/2402.04513
• Copy Paste: [[2402.04513]] Online Cascade Learning for Efficient Inference over Streams(https://arxiv.org/abs/2402.04513)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have a natural role in answering complex queries about data streams, but the high computational cost of LLM inference makes them infeasible in many such tasks. We propose online cascade learning, the first approach to addressing this challenge. The objective here is to learn a "cascade" of models, starting with lower-capacity models (such as logistic regressors) and ending with a powerful LLM, along with a deferral policy that determines the model that is used on a given input. We formulate the task of learning cascades online as an imitation-learning problem and give a no-regret algorithm for the problem. Experimental results across four benchmarks show that our method parallels LLMs in accuracy while cutting down inference costs by as much as 90%, underscoring its efficacy and adaptability in stream processing.
• 摘要：大型语言模型 (LLM) 在回答有关数据流的复杂查询方面具有天然的作用，但 LLM 推理的高计算成本使其在许多此类任务中不可行。我们提出在线级联学习，这是应对这一挑战的第一种方法。这里的目标是学习模型的“级联”，从较低容量的模型（例如逻辑回归）开始，到强大的法学硕士结束，以及确定在给定输入上使用的模型的延期策略。我们将在线学习级联的任务表述为模仿学习问题，并给出了该问题的无悔算法。四个基准测试的实验结果表明，我们的方法在准确性方面与法学硕士相当，同时将推理成本降低了 90%，强调了其在流处理中的有效性和适应性。

Title: SumRec: A Framework for Recommendation using Open-Domain Dialogue

• Authors: Ryutaro Asahara, Masaki Takahashi, Chiho Iwahashi, Michimasa Inaba
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04523
• Pdf URL: https://arxiv.org/pdf/2402.04523
• Copy Paste: [[2402.04523]] SumRec: A Framework for Recommendation using Open-Domain Dialogue(https://arxiv.org/abs/2402.04523)
• Keywords: language model, llm, code, chat
• Abstract: Chat dialogues contain considerable useful information about a speaker's interests, preferences, and experiences.Thus, knowledge from open-domain chat dialogue can be used to personalize various systems and offer recommendations for advanced information.This study proposed a novel framework SumRec for recommending information from open-domain chat dialogue.The study also examined the framework using ChatRec, a newly constructed dataset for training and evaluation. To extract the speaker and item characteristics, the SumRec framework employs a large language model (LLM) to generate a summary of the speaker information from a dialogue and to recommend information about an item according to the type of user.The speaker and item information are then input into a score estimation model, generating a recommendation score.Experimental results show that the SumRec framework provides better recommendations than the baseline method of using dialogues and item descriptions in their original form. Our dataset and code is publicly available at https://github.com/Ryutaro-A/SumRec
• 摘要：聊天对话包含有关说话者的兴趣、偏好和经验的大量有用信息。因此，开放域聊天对话中的知识可用于个性化各种系统并提供高级信息的推荐。本研究提出了一种新颖的框架 SumRec，用于推荐来自开放域聊天对话的信息。该研究还使用 ChatRec（一个新构建的用于训练和评估的数据集）检查了该框架。为了提取说话者和项目特征，SumRec 框架采用大型语言模型 (LLM) 从对话中生成说话者信息的摘要，并根据用户类型推荐有关项目的信息。说话者和项目信息是然后输入到分数估计模型中，生成推荐分数。实验结果表明，SumRec 框架提供了比使用原始形式的对话和项目描述的基线方法更好的推荐。我们的数据集和代码可在 https://github.com/Ryutaro-A/SumRec 上公开获取

Title: Learning Diverse Policies with Soft Self-Generated Guidance

• Authors: Guojian Wang, Faguo Wu, Xiao Zhang, Jianxiang Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04539
• Pdf URL: https://arxiv.org/pdf/2402.04539
• Copy Paste: [[2402.04539]] Learning Diverse Policies with Soft Self-Generated Guidance(https://arxiv.org/abs/2402.04539)
• Keywords: lora, agent
• Abstract: Reinforcement learning (RL) with sparse and deceptive rewards is challenging because non-zero rewards are rarely obtained. Hence, the gradient calculated by the agent can be stochastic and without valid information. Recent studies that utilize memory buffers of previous experiences can lead to a more efficient learning process. However, existing methods often require these experiences to be successful and may overly exploit them, which can cause the agent to adopt suboptimal behaviors. This paper develops an approach that uses diverse past trajectories for faster and more efficient online RL, even if these trajectories are suboptimal or not highly rewarded. The proposed algorithm combines a policy improvement step with an additional exploration step using offline demonstration data. The main contribution of this paper is that by regarding diverse past trajectories as guidance, instead of imitating them, our method directs its policy to follow and expand past trajectories while still being able to learn without rewards and approach optimality. Furthermore, a novel diversity measurement is introduced to maintain the team's diversity and regulate exploration. The proposed algorithm is evaluated on discrete and continuous control tasks with sparse and deceptive rewards. Compared with the existing RL methods, the experimental results indicate that our proposed algorithm is significantly better than the baseline methods regarding diverse exploration and avoiding local optima.
• 摘要：具有稀疏和欺骗性奖励的强化学习（RL）具有挑战性，因为很少获得非零奖励。因此，代理计算的梯度可能是随机的并且没有有效信息。最近的研究利用了以前经验的记忆缓冲区，可以带来更有效的学习过程。然而，现有的方法通常需要这些经验才能成功，并且可能过度利用它们，这可能导致代理采取次优行为。本文开发了一种方法，使用不同的过去轨迹来实现更快、更高效的在线强化学习，即使这些轨迹不是最优的或回报率​​不高。所提出的算法将策略改进步骤与使用离线演示数据的附加探索步骤相结合。本文的主要贡献在于，通过将不同的过去轨迹作为指导，而不是模仿它们，我们的方法指导其策略遵循和扩展过去的轨迹，同时仍然能够在没有奖励的情况下学习并接近最优。此外，引入了一种新颖的多样性测量方法来保持团队的多样性并规范探索。所提出的算法在具有稀疏和欺骗性奖励的离散和连续控制任务上进行评估。与现有的强化学习方法相比，实验结果表明，我们提出的算法在多样化探索和避免局部最优方面明显优于基线方法。

Title: Share What You Already Know: Cross-Language-Script Transfer and Alignment for Sentiment Detection in Code-Mixed Data

• Authors: Niraj Pahari, Kazutaka Shimada
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04542
• Pdf URL: https://arxiv.org/pdf/2402.04542
• Copy Paste: [[2402.04542]] Share What You Already Know: Cross-Language-Script Transfer and Alignment for Sentiment Detection in Code-Mixed Data(https://arxiv.org/abs/2402.04542)
• Keywords: code
• Abstract: Code-switching entails mixing multiple languages. It is an increasingly occurring phenomenon in social media texts. Usually, code-mixed texts are written in a single script, even though the languages involved have different scripts. Pre-trained multilingual models primarily utilize the data in the native script of the language. In existing studies, the code-switched texts are utilized as they are. However, using the native script for each language can generate better representations of the text owing to the pre-trained knowledge. Therefore, a cross-language-script knowledge sharing architecture utilizing the cross attention and alignment of the representations of text in individual language scripts was proposed in this study. Experimental results on two different datasets containing Nepali-English and Hindi-English code-switched texts, demonstrate the effectiveness of the proposed method. The interpretation of the model using model explainability technique illustrates the sharing of language-specific knowledge between language-specific representations.
• 摘要：语码转换需要混合多种语言。这是社交媒体文本中越来越多出现的现象。通常，代码混合文本是用单个脚本编写的，即使所涉及的语言具有不同的脚本。预训练的多语言模型主要利用该语言的本机脚本中的数据。在现有的研究中，语码转换文本按原样使用。然而，由于预先训练的知识，使用每种语言的本机脚本可以生成更好的文本表示。因此，本研究提出了一种利用各个语言脚本中文本表示的交叉关注和对齐的跨语言脚本知识共享架构。在包含尼泊尔语-英语和印地语-英语语码转换文本的两个不同数据集上的实验结果证明了该方法的有效性。使用模型可解释性技术对模型的解释说明了特定语言表示之间特定语言知识的共享。

Title: Curvature-Informed SGD via General Purpose Lie-Group Preconditioners

• Authors: Omead Pooladzandi, Xi-Lin Li
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04553
• Pdf URL: https://arxiv.org/pdf/2402.04553
• Copy Paste: [[2402.04553]] Curvature-Informed SGD via General Purpose Lie-Group Preconditioners(https://arxiv.org/abs/2402.04553)
• Keywords: code
• Abstract: We present a novel approach to accelerate stochastic gradient descent (SGD) by utilizing curvature information obtained from Hessian-vector products or finite differences of parameters and gradients, similar to the BFGS algorithm. Our approach involves two preconditioners: a matrix-free preconditioner and a low-rank approximation preconditioner. We update both preconditioners online using a criterion that is robust to stochastic gradient noise and does not require line search or damping. To preserve the corresponding symmetry or invariance, our preconditioners are constrained to certain connected Lie groups. The Lie group's equivariance property simplifies the preconditioner fitting process, while its invariance property eliminates the need for damping, which is commonly required in second-order optimizers. As a result, the learning rate for parameter updating and the step size for preconditioner fitting are naturally normalized, and their default values work well in most scenarios. Our proposed approach offers a promising direction for improving the convergence of SGD with low computational overhead. We demonstrate that Preconditioned SGD (PSGD) outperforms SoTA on Vision, NLP, and RL tasks across multiple modern deep-learning architectures. We have provided code for reproducing toy and large scale experiments in this paper.
• 摘要：我们提出了一种利用从 Hessian 向量乘积或参数和梯度的有限差分获得的曲率信息来加速随机梯度下降 (SGD) 的新方法，类似于 BFGS 算法。我们的方法涉及两个预处理器：无矩阵预处理器和低秩近似预处理器。我们使用对随机梯度噪声具有鲁棒性且不需要线搜索或阻尼的标准在线更新两个预处理器。为了保持相应的对称性或不变性，我们的预处理器被限制为某些连接的李群。李群的等方差性质简化了预处理器拟合过程，而其不变性质消除了二阶优化器中通常需要的阻尼的需要。因此，参数更新的学习率和预处理器拟合的步长自然被归一化，并且它们的默认值在大多数情况下都适用。我们提出的方法为以低计算开销提高 SGD 的收敛性提供了一个有前途的方向。我们证明，在多种现代深度学习架构中，预处理 SGD (PSGD) 在视觉、NLP 和 RL 任务上的表现优于 SoTA。我们在本文中提供了复制玩具和大规模实验的代码。

Title: Can Large Language Model Agents Simulate Human Trust Behaviors?

• Authors: Chengxing Xie, Canyu Chen, Feiran Jia, Ziyu Ye, Kai Shu, Adel Bibi, Ziniu Hu, Philip Torr, Bernard Ghanem, Guohao Li
• Subjects: cs.AI, cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2402.04559
• Pdf URL: https://arxiv.org/pdf/2402.04559
• Copy Paste: [[2402.04559]] Can Large Language Model Agents Simulate Human Trust Behaviors?(https://arxiv.org/abs/2402.04559)
• Keywords: language model, llm, agent
• Abstract: Large Language Model (LLM) agents have been increasingly adopted as simulation tools to model humans in applications such as social science. However, one fundamental question remains: can LLM agents really simulate human behaviors? In this paper, we focus on one of the most critical behaviors in human interactions, trust, and aim to investigate whether or not LLM agents can simulate human trust behaviors. We first find that LLM agents generally exhibit trust behaviors, referred to as agent trust, under the framework of Trust Games, which are widely recognized in behavioral economics. Then, we discover that LLM agents can have high behavioral alignment with humans regarding trust behaviors, indicating the feasibility to simulate human trust behaviors with LLM agents. In addition, we probe into the biases in agent trust and the differences in agent trust towards agents and humans. We also explore the intrinsic properties of agent trust under conditions including advanced reasoning strategies and external manipulations. We further offer important implications for various scenarios where trust is paramount. Our study represents a significant step in understanding the behaviors of LLM agents and the LLM-human analogy.
• 摘要：大型语言模型（LLM）代理已越来越多地被用作模拟工具，在社会科学等应用中对人类进行建模。然而，一个基本问题仍然存在：LLM 代理真的可以模拟人类行为吗？在本文中，我们关注人类互动中最关键的行为之一——信任，并旨在研究LLM代理是否可以模拟人类信任行为。我们首先发现LLM代理人在行为经济学广泛认可的信任博弈框架下普遍表现出信任行为，称为代理人信任。然后，我们发现LLM代理可以在信任行为方面与人类具有高度的行为一致性，这表明用LLM代理模拟人类信任行为的可行性。此外，我们还探讨了代理信任的偏差以及代理对代理和人类的信任的差异。我们还探讨了在高级推理策略和外部操作等条件下代理信任的内在属性。我们进一步为信任至关重要的各种场景提供了重要的启示。我们的研究代表了理解法学硕士代理人的行为和法学硕士与人类类比的重要一步。

Title: OIL-AD: An Anomaly Detection Framework for Sequential Decision Sequences

• Authors: Chen Wang, Sarah Erfani, Tansu Alpcan, Christopher Leckie
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04567
• Pdf URL: https://arxiv.org/pdf/2402.04567
• Copy Paste: [[2402.04567]] OIL-AD: An Anomaly Detection Framework for Sequential Decision Sequences(https://arxiv.org/abs/2402.04567)
• Keywords: agent
• Abstract: Anomaly detection in decision-making sequences is a challenging problem due to the complexity of normality representation learning and the sequential nature of the task. Most existing methods based on Reinforcement Learning (RL) are difficult to implement in the real world due to unrealistic assumptions, such as having access to environment dynamics, reward signals, and online interactions with the environment. To address these limitations, we propose an unsupervised method named Offline Imitation Learning based Anomaly Detection (OIL-AD), which detects anomalies in decision-making sequences using two extracted behaviour features: action optimality and sequential association. Our offline learning model is an adaptation of behavioural cloning with a transformer policy network, where we modify the training process to learn a Q function and a state value function from normal trajectories. We propose that the Q function and the state value function can provide sufficient information about agents' behavioural data, from which we derive two features for anomaly detection. The intuition behind our method is that the action optimality feature derived from the Q function can differentiate the optimal action from others at each local state, and the sequential association feature derived from the state value function has the potential to maintain the temporal correlations between decisions (state-action pairs). Our experiments show that OIL-AD can achieve outstanding online anomaly detection performance with up to 34.8% improvement in F1 score over comparable baselines.
• 摘要：由于正态表示学习的复杂性和任务的顺序性质，决策序列中的异常检测是一个具有挑战性的问题。大多数基于强化学习（RL）的现有方法由于不切实际的假设（例如获取环境动态、奖励信号以及与环境的在线交互）而难以在现实世界中实施。为了解决这些限制，我们提出了一种名为基于离线模仿学习的异常检测（OIL-AD）的无监督方法，该方法使用两个提取的行为特征：动作最优性和顺序关联来检测决策序列中的异常。我们的离线学习模型是通过变压器策略网络对行为克隆进行的改编，我们修改训练过程以从正常轨迹学习 Q 函数和状态值函数。我们提出Q函数和状态值函数可以提供有关代理行为数据的足够信息，从中我们得出用于异常检测的两个特征。我们方法背后的直觉是，从 Q 函数导出的动作最优特征可以在每个局部状态下将最优动作与其他动作区分开来，并且从状态值函数导出的顺序关联特征有可能维持决策之间的时间相关性（状态-动作对）。我们的实验表明，OIL-AD 可以实现出色的在线异常检测性能，F1 分数比可比基线提高高达 34.8%。

Title: S-Agents: self-organizing agents in open-ended environment

• Authors: Jiaqi Chen, Yuxian Jiang, Jiachen Lu, Li Zhang
• Subjects: cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2402.04578
• Pdf URL: https://arxiv.org/pdf/2402.04578
• Copy Paste: [[2402.04578]] S-Agents: self-organizing agents in open-ended environment(https://arxiv.org/abs/2402.04578)
• Keywords: language model, llm, rag, agent
• Abstract: Leveraging large language models (LLMs), autonomous agents have significantly improved, gaining the ability to handle a variety of tasks. In open-ended settings, optimizing collaboration for efficiency and effectiveness demands flexible adjustments. Despite this, current research mainly emphasizes fixed, task-oriented workflows and overlooks agent-centric organizational structures. Drawing inspiration from human organizational behavior, we introduce a self-organizing agent system (S-Agents) with a "tree of agents" structure for dynamic workflow, an "hourglass agent architecture" for balancing information priorities, and a "non-obstructive collaboration" method to allow asynchronous task execution among agents. This structure can autonomously coordinate a group of agents, efficiently addressing the challenges of an open and dynamic environment without human intervention. Our experiments demonstrate that S-Agents proficiently execute collaborative building tasks and resource collection in the Minecraft environment, validating their effectiveness.
• 摘要：利用大型语言模型 (LLM)，自主代理得到了显着改进，获得了处理各种任务的能力。在开放式环境中，优化协作以提高效率和效果需要灵活的调整。尽管如此，当前的研究主要强调固定的、以任务为导向的工作流程，而忽视了以代理为中心的组织结构。从人类组织行为中汲取灵感，我们引入了一种自组织代理系统（S-Agents），该系统具有用于动态工作流程的“代理树”结构、用于平衡信息优先级的“沙漏代理架构”以及“无阻碍协作” “方法允许代理之间异步执行任务。这种结构可以自主协调一组智能体，无需人工干预即可有效应对开放动态环境的挑战。我们的实验表明，S-Agents 在 Minecraft 环境中熟练地执行协作构建任务和资源收集，验证了其有效性。

Title: Collective Counterfactual Explanations via Optimal Transport

• Authors: Ahmad-Reza Ehyaei, Ali Shirali, Samira Samadi
• Subjects: cs.LG, stat.ME
• Abstract URL: https://arxiv.org/abs/2402.04579
• Pdf URL: https://arxiv.org/pdf/2402.04579
• Copy Paste: [[2402.04579]] Collective Counterfactual Explanations via Optimal Transport(https://arxiv.org/abs/2402.04579)
• Keywords: rag
• Abstract: Counterfactual explanations provide individuals with cost-optimal actions that can alter their labels to desired classes. However, if substantial instances seek state modification, such individual-centric methods can lead to new competitions and unanticipated costs. Furthermore, these recommendations, disregarding the underlying data distribution, may suggest actions that users perceive as outliers. To address these issues, our work proposes a collective approach for formulating counterfactual explanations, with an emphasis on utilizing the current density of the individuals to inform the recommended actions. Our problem naturally casts as an optimal transport problem. Leveraging the extensive literature on optimal transport, we illustrate how this collective method improves upon the desiderata of classical counterfactual explanations. We support our proposal with numerical simulations, illustrating the effectiveness of the proposed approach and its relation to classic methods.
• 摘要：反事实解释为个人提供了成本最优的行动，可以将他们的标签更改为所需的类别。然而，如果大量实例寻求国家修改，这种以个人为中心的方法可能会导致新的竞争和意想不到的成本。此外，这些建议在不考虑底层数据分布的情况下，可能会建议用户视为异常值的操作。为了解决这些问题，我们的工作提出了一种制定反事实解释的集体方法，重点是利用个人的当前密度来为建议的行动提供信息。我们的问题自然地转化为最优运输问题。利用有关最佳运输的大量文献，我们说明了这种集体方法如何改进经典反事实解释的需求。我们通过数值模拟支持我们的建议，说明所提出方法的有效性及其与经典方法的关系。

Title: UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset

• Authors: Haoyu Wang, Shuo Wang, Yukun Yan, Xujia Wang, Zhiyu Yang, Yuzhuang Xu, Zhenghao Liu, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04588
• Pdf URL: https://arxiv.org/pdf/2402.04588
• Copy Paste: [[2402.04588]] UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset(https://arxiv.org/abs/2402.04588)
• Keywords: language model, llm, lora
• Abstract: Open-source large language models (LLMs) have gained significant strength across diverse fields. Nevertheless, the majority of studies primarily concentrate on English, with only limited exploration into the realm of multilingual supervised fine-tuning. In this work, we therefore construct an open-source multilingual supervised fine-tuning dataset. Different from previous works that simply translate English instructions, we consider both the language-specific and language-agnostic abilities of LLMs. For language-specific abilities, we introduce a knowledge-grounded data augmentation approach to elicit more culture-specific knowledge of LLMs, improving their ability to serve users from different countries. For language-agnostic abilities, we find through experiments that modern LLMs exhibit strong cross-lingual transfer capabilities, thus repeatedly learning identical content in various languages is not necessary. Consequently, we can substantially prune the language-agnostic SFT data without any performance degradation, making the SFT process more efficient. The resulting UltraLink dataset comprises approximately 1 million samples across five languages, and the proposed data construction method can also be easily extended to other languages. UltraLink-LM, which is trained on UltraLink, outperforms several representative baselines across many tasks.
• 摘要：开源大语言模型 (LLM) 在各个领域都获得了显着的优势。然而，大多数研究主要集中在英语上，对多语言监督微调领域的探索有限。因此，在这项工作中，我们构建了一个开源多语言监督微调数据集。与之前简单翻译英语说明的作品不同，我们同时考虑了法学硕士的特定语言和与语言无关的能力。对于特定语言的能力，我们引入了一种基于知识的数据增强方法，以引出法学硕士更多的特定文化知识，提高他们为不同国家的用户服务的能力。对于与语言无关的能力，我们通过实验发现现代法学硕士表现出很强的跨语言迁移能力，因此不需要重复学习各种语言的相同内容。因此，我们可以大幅修剪与语言无关的 SFT 数据，而不会降低任何性能，从而使 SFT 过程更加高效。由此产生的 UltraLink 数据集包含五种语言的大约 100 万个样本，并且所提出的数据构建方法也可以轻松扩展到其他语言。 UltraLink-LM 在 UltraLink 上进行训练，在许多任务中优于几个代表性基线。

Title: CMSA algorithm for solving the prioritized pairwise test data generation problem in software product lines

• Authors: Javier Ferrer, Francisco Chicano, José Antonio Ortega Toro
• Subjects: cs.AI, cs.SE
• Abstract URL: https://arxiv.org/abs/2402.04597
• Pdf URL: https://arxiv.org/pdf/2402.04597
• Copy Paste: [[2402.04597]] CMSA algorithm for solving the prioritized pairwise test data generation problem in software product lines(https://arxiv.org/abs/2402.04597)
• Keywords: rag
• Abstract: In Software Product Lines (SPLs) it may be difficult or even impossible to test all the products of the family because of the large number of valid feature combinations that may exist. Thus, we want to find a minimal subset of the product family that allows us to test all these possible combinations (pairwise). Furthermore, when testing a single product is a great effort, it is desirable to first test products composed of a set of priority features. This problem is called Prioritized Pairwise Test Data Generation Problem. State-of-the-art algorithms based on Integer Linear Programming for this problema are faster enough for small and medium instances. However, there exists some real instances that are too large to be computed with these algorithms in a reasonable time because of the exponential growth of the number of candidate solutions. Also, these heuristics not always lead us to the best solutions. In this work we propose a new approach based on a hybrid metaheuristic algorithm called Construct, Merge, Solve & Adapt. We compare this matheuristic with four algorithms: a Hybrid algorithm based on Integer Linear Programming ((HILP), a Hybrid algorithm based on Integer Nonlinear Programming (HINLP), the Parallel Prioritized Genetic Solver (PPGS), and a greedy algorithm called prioritized-ICPL. The analysis reveals that CMSA results in statistically significantly better quality solutions in most instances and for most levels of weighted coverage, although it requires more execution time.
• 摘要：在软件产品线 (SPL) 中，由于可能存在大量有效的功能组合，因此测试该系列的所有产品可能很困难甚至不可能。因此，我们希望找到产品系列的最小子集，使我们能够测试所有这些可能的组合（成对）。此外，当测试单个产品需要付出很大的努力时，最好首先测试由一组优先功能组成的产品。该问题称为优先化成对测试数据生成问题。针对此问题的基于整数线性规划的最先进算法对于中小型实例来说足够快。然而，由于候选解的数量呈指数增长，存在一些太大的实际实例，无法在合理的时间内使用这些算法进行计算。此外，这些启发法并不总能引导我们找到最佳解决方案。在这项工作中，我们提出了一种基于混合元启发式算法的新方法，称为“构造、合并、求解和适应”。我们将此数学方法与四种算法进行比较：基于整数线性规划 (HILP) 的混合算法、基于整数非线性规划 (HINLP) 的混合算法、并行优先级遗传求解器 (PPGS) 以及称为优先级-ICPL 的贪婪算法分析表明，在大多数情况下和大多数加权覆盖级别下，CMSA 会产生统计上显着更高质量的解决方案，尽管它需要更多的执行时间。

Title: Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector

• Authors: Haihui Yang, Xiaojun Quan
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04601
• Pdf URL: https://arxiv.org/pdf/2402.04601
• Copy Paste: [[2402.04601]] Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector(https://arxiv.org/abs/2402.04601)
• Keywords: language model, llm, code
• Abstract: Chinese grammatical error correction (CGEC) faces serious overcorrection challenges when employing autoregressive generative models such as sequence-to-sequence (Seq2Seq) models and decoder-only large language models (LLMs). While previous methods aim to address overcorrection in Seq2Seq models, they are difficult to adapt to decoder-only LLMs. In this paper, we propose an alignment-enhanced corrector for the overcorrection problem that applies to both Seq2Seq models and decoder-only LLMs. Our method first trains a correction model to generate an initial correction of the source sentence. Then, we combine the source sentence with the initial correction and feed it through an alignment model for another round of correction, aiming to enforce the alignment model to focus on potential overcorrection. Moreover, to enhance the model's ability to identify nuances, we further explore the reverse alignment of the source sentence and the initial correction. Finally, we transfer the alignment knowledge from two alignment models to the correction model, instructing it on how to avoid overcorrection. Experimental results on three CGEC datasets demonstrate the effectiveness of our approach in alleviating overcorrection and improving overall performance.
• 摘要：当使用序列到序列（Seq2Seq）模型和仅解码器的大型语言模型（LLM）等自回归生成模型时，中文语法错误校正（CGEC）面临严重的过度校正挑战。虽然之前的方法旨在解决 Seq2Seq 模型中的过度校正问题，但它们很难适应仅解码器的 LLM。在本文中，我们针对过度校正问题提出了一种对齐增强校正器，该校正器适用于 Seq2Seq 模型和仅解码器的 LLM。我们的方法首先训练一个校正模型来生成源句子的初始校正。然后，我们将源句子与初始校正结合起来，并通过对齐模型进行另一轮校正，旨在强制对齐模型关注潜在的过度校正。此外，为了增强模型识别细微差别的能力，我们进一步探索源句子和初始校正的反向对齐。最后，我们将两个对齐模型的对齐知识转移到校正模型，指导其如何避免过度校正。三个 CGEC 数据集上的实验结果证明了我们的方法在减轻过度校正和提高整体性能方面的有效性。

Title: Improving Cross-Domain Low-Resource Text Generation through LLM Post-Editing: A Programmer-Interpreter Approach

• Authors: Zhuang Li, Levon Haroutunian, Raj Tumuluri, Philip Cohen, Gholamreza Haffari
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04609
• Pdf URL: https://arxiv.org/pdf/2402.04609
• Copy Paste: [[2402.04609]] Improving Cross-Domain Low-Resource Text Generation through LLM Post-Editing: A Programmer-Interpreter Approach(https://arxiv.org/abs/2402.04609)
• Keywords: language model, gpt, llm
• Abstract: Post-editing has proven effective in improving the quality of text generated by large language models (LLMs) such as GPT-3.5 or GPT-4, particularly when direct updating of their parameters to enhance text quality is infeasible or expensive. However, relying solely on smaller language models for post-editing can limit the LLMs' ability to generalize across domains. Moreover, the editing strategies in these methods are not optimally designed for text-generation tasks. To address these limitations, we propose a neural programmer-interpreter approach that preserves the domain generalization ability of LLMs when editing their output. The editing actions in this framework are specifically devised for text generation. Extensive experiments demonstrate that the programmer-interpreter significantly enhances GPT-3.5's performance in logical form-to-text conversion and low-resource machine translation, surpassing other state-of-the-art (SOTA) LLM post-editing methods in cross-domain settings.
• 摘要：事实证明，后期编辑可以有效提高 GPT-3.5 或 GPT-4 等大型语言模型 (LLM) 生成的文本质量，特别是当直接更新其参数以提高文本质量不可行或成本高昂时。然而，仅仅依靠较小的语言模型进行后期编辑可能会限制法学硕士跨领域泛化的能力。此外，这些方法中的编辑策略并不是针对文本生成任务进行最佳设计的。为了解决这些限制，我们提出了一种神经程序员解释器方法，该方法在编辑法学硕士的输出时保留了法学硕士的领域泛化能力。该框架中的编辑操作是专门为文本生成而设计的。大量实验表明，程序员解释器显着增强了 GPT-3.5 在逻辑形式到文本转换和低资源机器翻译方面的性能，在交叉方面超越了其他最先进的 (SOTA) LLM 译后编辑方法。 - 域设置。

Title: Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models

• Authors: Chirag Agarwal, Sree Harsha Tanneru, Himabindu Lakkaraju
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04614
• Pdf URL: https://arxiv.org/pdf/2402.04614
• Copy Paste: [[2402.04614]] Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models(https://arxiv.org/abs/2402.04614)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) are deployed as powerful tools for several natural language processing (NLP) applications. Recent works show that modern LLMs can generate self-explanations (SEs), which elicit their intermediate reasoning steps for explaining their behavior. Self-explanations have seen widespread adoption owing to their conversational and plausible nature. However, there is little to no understanding of their faithfulness. In this work, we discuss the dichotomy between faithfulness and plausibility in SEs generated by LLMs. We argue that while LLMs are adept at generating plausible explanations -- seemingly logical and coherent to human users -- these explanations do not necessarily align with the reasoning processes of the LLMs, raising concerns about their faithfulness. We highlight that the current trend towards increasing the plausibility of explanations, primarily driven by the demand for user-friendly interfaces, may come at the cost of diminishing their faithfulness. We assert that the faithfulness of explanations is critical in LLMs employed for high-stakes decision-making. Moreover, we urge the community to identify the faithfulness requirements of real-world applications and ensure explanations meet those needs. Finally, we propose some directions for future work, emphasizing the need for novel methodologies and frameworks that can enhance the faithfulness of self-explanations without compromising their plausibility, essential for the transparent deployment of LLMs in diverse high-stakes domains.
• 摘要：大型语言模型 (LLM) 被部署为多种自然语言处理 (NLP) 应用程序的强大工具。最近的研究表明，现代法学硕士可以生成自我解释（SE），从而引出他们的中间推理步骤来解释他们的行为。自我解释因其对话性和合理性而被广泛采用。然而，人们对他们的忠诚却知之甚少。在这项工作中，我们讨论了法学硕士生成的 SE 中的忠实性和合理性之间的二分法。我们认为，虽然法学硕士善于生成看似合理的解释——对人类用户来说似乎是合乎逻辑且连贯的——但这些解释不一定与法学硕士的推理过程一致，引发了人们对其忠诚度的担忧。我们强调，当前增加解释合理性的趋势主要是由对用户友好界面的需求驱动的，但可能会以降低解释的可信度为代价。我们断言，解释的忠实度对于从事高风险决策的法学硕士来说至关重要。此外，我们敦促社区确定现实世界应用程序的忠实性要求，并确保解释满足这些需求。最后，我们提出了未来工作的一些方向，强调需要新的方法和框架，以增强自我解释的真实性，同时又不损害其合理性，这对于法学硕士在不同高风险领域的透明部署至关重要。

Title: TinyLLM: Learning a Small Student from Multiple Large Language Models

• Authors: Yijun Tian, Yikun Han, Xiusi Chen, Wei Wang, Nitesh V. Chawla
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04616
• Pdf URL: https://arxiv.org/pdf/2402.04616
• Copy Paste: [[2402.04616]] TinyLLM: Learning a Small Student from Multiple Large Language Models(https://arxiv.org/abs/2402.04616)
• Keywords: language model, llm, rag, chain-of-thought
• Abstract: Transferring the reasoning capability from stronger large language models (LLMs) to smaller ones has been quite appealing, as smaller LLMs are more flexible to deploy with less expense. Among the existing solutions, knowledge distillation stands out due to its outstanding efficiency and generalization. However, existing methods suffer from several drawbacks, including limited knowledge diversity and the lack of rich contextual information. To solve the problems and facilitate the learning of compact language models, we propose TinyLLM, a novel knowledge distillation paradigm to learn a small student LLM from multiple large teacher LLMs. In particular, we encourage the student LLM to not only generate the correct answers but also understand the rationales behind these answers. Given that different LLMs possess diverse reasoning skills, we guide the student model to assimilate knowledge from various teacher LLMs. We further introduce an in-context example generator and a teacher-forcing Chain-of-Thought strategy to ensure that the rationales are accurate and grounded in contextually appropriate scenarios. Extensive experiments on six datasets across two reasoning tasks demonstrate the superiority of our method. Results show that TinyLLM can outperform large teacher LLMs significantly, despite having a considerably smaller model size.
• 摘要：将推理能力从更强的大型语言模型 (LLM) 转移到较小的模型非常有吸引力，因为较小的 LLM 部署起来更灵活，成本也更低。在现有的解决方案中，知识蒸馏因其出色的效率和泛化性而脱颖而出。然而，现有方法存在一些缺点，包括知识多样性有限和缺乏丰富的上下文信息。为了解决这些问题并促进紧凑语言模型的学习，我们提出了 TinyLLM，一种新颖的知识蒸馏范式，用于从多个大型教师 LLM 中学习小型学生 LLM。特别是，我们鼓励法学硕士学生不仅要生成正确的答案，还要理解这些答案背后的基本原理。鉴于不同的法学硕士拥有不同的推理技能，我们引导学生模型吸收不同教师法学硕士的知识。我们进一步引入了上下文示例生成器和教师强制思想链策略，以确保基本原理准确并基于上下文适当的场景。对两个推理任务的六个数据集进行的广泛实验证明了我们方法的优越性。结果表明，尽管模型规模要小得多，但 TinyLLM 的表现仍显着优于大型教师 LLM。

Title: InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory

• Authors: Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, Song Han, Maosong Sun
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04617
• Pdf URL: https://arxiv.org/pdf/2402.04617
• Copy Paste: [[2402.04617]] InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory(https://arxiv.org/abs/2402.04617)
• Keywords: language model, llm, agent
• Abstract: Large language models (LLMs) have emerged as a cornerstone in real-world applications with lengthy streaming inputs, such as LLM-driven agents. However, existing LLMs, pre-trained on sequences with restricted maximum length, cannot generalize to longer sequences due to the out-of-domain and distraction issues. To alleviate these issues, existing efforts employ sliding attention windows and discard distant tokens to achieve the processing of extremely long sequences. Unfortunately, these approaches inevitably fail to capture long-distance dependencies within sequences to deeply understand semantics. This paper introduces a training-free memory-based method, InfLLM, to unveil the intrinsic ability of LLMs to process streaming long sequences. Specifically, InfLLM stores distant contexts into additional memory units and employs an efficient mechanism to lookup token-relevant units for attention computation. Thereby, InfLLM allows LLMs to efficiently process long sequences while maintaining the ability to capture long-distance dependencies. Without any training, InfLLM enables LLMs pre-trained on sequences of a few thousand tokens to achieve superior performance than competitive baselines continually training these LLMs on long sequences. Even when the sequence length is scaled to $1,024$K, InfLLM still effectively captures long-distance dependencies.
• 摘要：大型语言模型 (LLM) 已成为具有冗长流输入的现实应用程序的基石，例如 LLM 驱动的代理。然而，由于域外和干扰问题，现有的法学硕士在最大长度受限的序列上进行预训练，无法推广到更长的序列。为了缓解这些问题，现有的努力采用滑动注意窗口并丢弃远处的标记来实现极长序列的处理。不幸的是，这些方法不可避免地无法捕获序列内的长距离依赖关系以深入理解语义。本文介绍了一种免训练的基于内存的方法 InfLLM，以揭示 LLM 处理流式长序列的内在能力。具体来说，InfLLM 将遥远的上下文存储到额外的内存单元中，并采用有效的机制来查找与令牌相关的单元以进行注意力计算。因此，InfLLM 允许 LLM 有效地处理长序列，同时保持捕获长距离依赖性的能力。无需任何训练，InfLLM 就可以对数千个 token 的序列进行预训练，从而实现比在长序列上持续训练这些 LLM 的竞争基线更出色的性能。即使序列长度缩放至 $1,024$K，InfLLM 仍然可以有效捕获长距离依赖关系。

Title: MEMORYLLM: Towards Self-Updatable Large Language Models

• Authors: Yu Wang, Xiusi Chen, Jingbo Shang, Julian McAuley
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04624
• Pdf URL: https://arxiv.org/pdf/2402.04624
• Copy Paste: [[2402.04624]] MEMORYLLM: Towards Self-Updatable Large Language Models(https://arxiv.org/abs/2402.04624)
• Keywords: language model, llm
• Abstract: Existing Large Language Models (LLMs) usually remain static after deployment, which might make it hard to inject new knowledge into the model. We aim to build models containing a considerable portion of self-updatable parameters, enabling the model to integrate new knowledge effectively and efficiently. To this end, we introduce MEMORYLLM, a model that comprises a transformer and a fixed-size memory pool within the latent space of the transformer. MEMORYLLM can self-update with text knowledge and memorize the knowledge injected earlier. Our evaluations demonstrate the ability of MEMORYLLM to effectively incorporate new knowledge, as evidenced by its performance on model editing benchmarks. Meanwhile, the model exhibits long-term information retention capacity, which is validated through our custom-designed evaluations and long-context benchmarks. MEMORYLLM also shows operational integrity without any sign of performance degradation even after nearly a million memory updates.
• 摘要：现有的大型语言模型 (LLM) 在部署后通常保持静态，这可能导致难以将新知识注入到模型中。我们的目标是构建包含相当一部分可自我更新参数的模型，使模型能够有效且高效地集成新知识。为此，我们引入了 MEMORYLLM，一个由变压器和变压器潜在空间内的固定大小内存池组成的模型。 MEMORYLLM可以利用文本知识进行自我更新，并记住之前注入的知识。我们的评估证明了 MEMORYLLM 有效整合新知识的能力，其在模型编辑基准上的表现就证明了这一点。同时，该模型表现出长期信息保留能力，这一点通过我们定制设计的评估和长上下文基准进行了验证。 MEMORYLLM 还显示出操作完整性，即使在近一百万次内存更新后也没有任何性能下降的迹象。

Title: SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graph

• Authors: Julio C. Rangel, Tarcisio Mendes de Farias, Ana Claudia Sima, Norio Kobayashi
• Subjects: cs.AI, cs.CL, cs.DB, cs.IR
• Abstract URL: https://arxiv.org/abs/2402.04627
• Pdf URL: https://arxiv.org/pdf/2402.04627
• Copy Paste: [[2402.04627]] SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graph(https://arxiv.org/abs/2402.04627)
• Keywords: language model, llm, rag
• Abstract: The recent success of Large Language Models (LLM) in a wide range of Natural Language Processing applications opens the path towards novel Question Answering Systems over Knowledge Graphs leveraging LLMs. However, one of the main obstacles preventing their implementation is the scarcity of training data for the task of translating questions into corresponding SPARQL queries, particularly in the case of domain-specific KGs. To overcome this challenge, in this study, we evaluate several strategies for fine-tuning the OpenLlama LLM for question answering over life science knowledge graphs. In particular, we propose an end-to-end data augmentation approach for extending a set of existing queries over a given knowledge graph towards a larger dataset of semantically enriched question-to-SPARQL query pairs, enabling fine-tuning even for datasets where these pairs are scarce. In this context, we also investigate the role of semantic "clues" in the queries, such as meaningful variable names and inline comments. Finally, we evaluate our approach over the real-world Bgee gene expression knowledge graph and we show that semantic clues can improve model performance by up to 33% compared to a baseline with random variable names and no comments included.
• 摘要：最近，大型语言模型 (LLM) 在各种自然语言处理应用中取得的成功，为利用 LLM 的基于知识图谱的新型问答系统开辟了道路。然而，阻碍其实施的主要障碍之一是缺乏将问题转换为相应 SPARQL 查询的训练数据，特别是在特定于领域的知识图谱的情况下。为了克服这一挑战，在本研究中，我们评估了几种微调 OpenLlama LLM 生命科学知识图问答的策略。特别是，我们提出了一种端到端数据增强方法，用于将给定知识图上的一组现有查询扩展到语义丰富的问题​​到 SPARQL 查询对的更大数据集，甚至可以对这些数据集进行微调对是稀缺的。在这种情况下，我们还研究了查询中语义“线索”的作用，例如有意义的变量名称和内联注释。最后，我们在现实世界的 Bgee 基因表达知识图谱上评估了我们的方法，结果表明，与具有随机变量名称且不包含注释的基线相比，语义线索可以将模型性能提高高达 33%。

Title: The Future of Cognitive Strategy-enhanced Persuasive Dialogue Agents: New Perspectives and Trends

• Authors: Mengqi Chen, Bin Guo, Hao Wang, Haoyu Li, Qian Zhao, Jingqi Liu, Yasan Ding, Yan Pan, Zhiwen Yu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04631
• Pdf URL: https://arxiv.org/pdf/2402.04631
• Copy Paste: [[2402.04631]] The Future of Cognitive Strategy-enhanced Persuasive Dialogue Agents: New Perspectives and Trends(https://arxiv.org/abs/2402.04631)
• Keywords: language model, llm, agent
• Abstract: Persuasion, as one of the crucial abilities in human communication, has garnered extensive attention from researchers within the field of intelligent dialogue systems. We humans tend to persuade others to change their viewpoints, attitudes or behaviors through conversations in various scenarios (e.g., persuasion for social good, arguing in online platforms). Developing dialogue agents that can persuade others to accept certain standpoints is essential to achieving truly intelligent and anthropomorphic dialogue system. Benefiting from the substantial progress of Large Language Models (LLMs), dialogue agents have acquired an exceptional capability in context understanding and response generation. However, as a typical and complicated cognitive psychological system, persuasive dialogue agents also require knowledge from the domain of cognitive psychology to attain a level of human-like persuasion. Consequently, the cognitive strategy-enhanced persuasive dialogue agent (defined as CogAgent), which incorporates cognitive strategies to achieve persuasive targets through conversation, has become a predominant research paradigm. To depict the research trends of CogAgent, in this paper, we first present several fundamental cognitive psychology theories and give the formalized definition of three typical cognitive strategies, including the persuasion strategy, the topic path planning strategy, and the argument structure prediction strategy. Then we propose a new system architecture by incorporating the formalized definition to lay the foundation of CogAgent. Representative works are detailed and investigated according to the combined cognitive strategy, followed by the summary of authoritative benchmarks and evaluation metrics. Finally, we summarize our insights on open issues and future directions of CogAgent for upcoming researchers.
• 摘要：说服作为人类交流的关键能力之一，引起了智能对话系统领域研究人员的广泛关注。我们人类倾向于通过各种场景的对话来说服他人改变他们的观点、态度或行为（例如，为了社会公益而劝说、在网络平台上争论）。开发能够说服他人接受某些观点的对话代理对于实现真正智能和拟人化的对话系统至关重要。受益于大型语言模型（LLM）的实质性进展，对话代理在上下文理解和响应生成方面获得了卓越的能力。然而，作为一种典型且复杂的认知心理系统，说服性对话代理也需要认知心理学领域的知识才能达到类似人类的说服水平。因此，认知策略增强的说服性对话代理（定义为 CogAgent），结合认知策略通过对话实现说服目标，已成为主要的研究范式。为了描绘CogAgent的研究趋势，本文首先提出了几种基本的认知心理学理论，并给出了三​​种典型认知策略的形式化定义，包括说服策略、主题路径规划策略和论证结构预测策略。然后我们结合形式化定义提出了一个新的系统架构，为CogAgent奠定了基础。根据组合认知策略对代表性作品进行详细研究，并总结权威基准和评价指标。最后，我们为即将到来的研究人员总结了对 CogAgent 的未解决问题和未来方向的见解。

Title: TransLLaMa: LLM-based Simultaneous Translation System

• Authors: Roman Koshkin, Katsuhito Sudoh, Satoshi Nakamura
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04636
• Pdf URL: https://arxiv.org/pdf/2402.04636
• Copy Paste: [[2402.04636]] TransLLaMa: LLM-based Simultaneous Translation System(https://arxiv.org/abs/2402.04636)
• Keywords: language model, gpt, llm, code, rag
• Abstract: Decoder-only large language models (LLMs) have recently demonstrated impressive capabilities in text generation and reasoning. Nonetheless, they have limited applications in simultaneous machine translation (SiMT), currently dominated by encoder-decoder transformers. This study demonstrates that, after fine-tuning on a small dataset comprising causally aligned source and target sentence pairs, a pre-trained open-source LLM can control input segmentation directly by generating a special "wait" token. This obviates the need for a separate policy and enables the LLM to perform English-German and English-Russian SiMT tasks with BLEU scores that are comparable to those of specific state-of-the-art baselines. We also evaluated closed-source models such as GPT-4, which displayed encouraging results in performing the SiMT task without prior training (zero-shot), indicating a promising avenue for enhancing future SiMT systems.
• 摘要：仅解码器的大型语言模型 (LLM) 最近在文本生成和推理方面展示了令人印象深刻的功能。尽管如此，它们在同步机器翻译（SiMT）中的应用有限，目前由编码器-解码器变压器主导。这项研究表明，在对包含因果对齐的源句和目标句对的小数据集进行微调后，预训练的开源 LLM 可以通过生成特殊的“等待”标记来直接控制输入分段。这消除了对单独政策的需要，并使法学硕士能够执行英语-德语和英语-俄语 SiMT 任务，其 BLEU 分数可与特定的最先进基线相媲美。我们还评估了 GPT-4 等闭源模型，该模型在无需事先训练（零样本）的情况下执行 SiMT 任务时显示出令人鼓舞的结果，这表明了增强未来 SiMT 系统的有希望的途径。

Title: Domain Bridge: Generative model-based domain forensic for black-box models

• Authors: Jiyi Zhang, Han Fang, Ee-Chien Chang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04640
• Pdf URL: https://arxiv.org/pdf/2402.04640
• Copy Paste: [[2402.04640]] Domain Bridge: Generative model-based domain forensic for black-box models(https://arxiv.org/abs/2402.04640)
• Keywords: code, rag
• Abstract: In forensic investigations of machine learning models, techniques that determine a model's data domain play an essential role, with prior work relying on large-scale corpora like ImageNet to approximate the target model's domain. Although such methods are effective in finding broad domains, they often struggle in identifying finer-grained classes within those domains. In this paper, we introduce an enhanced approach to determine not just the general data domain (e.g., human face) but also its specific attributes (e.g., wearing glasses). Our approach uses an image embedding model as the encoder and a generative model as the decoder. Beginning with a coarse-grained description, the decoder generates a set of images, which are then presented to the unknown target model. Successful classifications by the model guide the encoder to refine the description, which in turn, are used to produce a more specific set of images in the subsequent iteration. This iterative refinement narrows down the exact class of interest. A key strength of our approach lies in leveraging the expansive dataset, LAION-5B, on which the generative model Stable Diffusion is trained. This enlarges our search space beyond traditional corpora, such as ImageNet. Empirical results showcase our method's performance in identifying specific attributes of a model's input domain, paving the way for more detailed forensic analyses of deep learning models.
• 摘要：在机器学习模型的取证调查中，确定模型数据域的技术起着至关重要的作用，之前的工作依赖于 ImageNet 等大规模语料库来近似目标模型的域。尽管此类方法可以有效地查找广泛的领域，但它们通常难以识别这些领域中更细粒度的类。在本文中，我们引入了一种增强方法，不仅可以确定一般数据域（例如人脸），还可以确定其特定属性（例如戴眼镜）。我们的方法使用图像嵌入模型作为编码器，使用生成模型作为解码器。从粗粒度描述开始，解码器生成一组图像，然后将其呈现给未知目标模型。模型的成功分类引导编码器完善描述，进而在后续迭代中用于生成更具体的图像集。这种迭代细化缩小了感兴趣的确切类别的范围。我们方法的关键优势在于利用广泛的数据集 LAION-5B，在该数据集上训练生成模型稳定扩散。这将我们的搜索空间扩大到传统语料库（例如 ImageNet）之外。实证结果展示了我们的方法在识别模型输入域的特定属性方面的性能，为深度学习模型的更详细的取证分析铺平了道路。

Title: LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views

• Authors: Yuji Roh, Qingyun Liu, Huan Gui, Zhe Yuan, Yujin Tang, Steven Euijong Whang, Liang Liu, Shuchao Bi, Lichan Hong, Ed H. Chi, Zhe Zhao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04644
• Pdf URL: https://arxiv.org/pdf/2402.04644
• Copy Paste: [[2402.04644]] LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views(https://arxiv.org/abs/2402.04644)
• Keywords: rag
• Abstract: Fine-tuning is becoming widely used for leveraging the power of pre-trained foundation models in new downstream tasks. While there are many successes of fine-tuning on various tasks, recent studies have observed challenges in the generalization of fine-tuned models to unseen distributions (i.e., out-of-distribution; OOD). To improve OOD generalization, some previous studies identify the limitations of fine-tuning data and regulate fine-tuning to preserve the general representation learned from pre-training data. However, potential limitations in the pre-training data and models are often ignored. In this paper, we contend that overly relying on the pre-trained representation may hinder fine-tuning from learning essential representations for downstream tasks and thus hurt its OOD generalization. It can be especially catastrophic when new tasks are from different (sub)domains compared to pre-training data. To address the issues in both pre-training and fine-tuning data, we propose a novel generalizable fine-tuning method LEVI, where the pre-trained model is adaptively ensembled layer-wise with a small task-specific model, while preserving training and inference efficiencies. By combining two complementing models, LEVI effectively suppresses problematic features in both the fine-tuning data and pre-trained model and preserves useful features for new tasks. Broad experiments with large language and vision models show that LEVI greatly improves fine-tuning generalization via emphasizing different views from fine-tuning data and pre-trained features.
• 摘要：微调正在广泛用于在新的下游任务中利用预先训练的基础模型的力量。尽管在各种任务上的微调取得了许多成功，但最近的研究发现，将微调模型推广到未见过的分布（即分布外；OOD）方面存在挑战。为了提高 OOD 泛化能力，之前的一些研究确定了微调数据的局限性，并调节微调以保留从预训练数据中学到的一般表示。然而，预训练数据和模型的潜在局限性常常被忽视。在本文中，我们认为过度依赖预训练的表示可能会阻碍学习下游任务的基本表示的微调，从而损害其 OOD 泛化性。与预训练数据相比，当新任务来自不同（子）域时，这可能尤其是灾难性的。为了解决预训练和微调数据中的问题，我们提出了一种新颖的可泛化微调方法 LEVI，其中预训练模型与小型任务特定模型自适应分层集成，同时保留训练和推理效率。通过结合两个互补模型，LEVI 有效地抑制了微调数据和预训练模型中的有问题的特征，并为新任务保留了有用的特征。对大型语言和视觉模型的广泛实验表明，LEVI 通过强调微调数据和预训练特征的不同视图，极大地提高了微调泛化能力。

Title: Latent Plan Transformer: Planning as Latent Variable Inference

• Authors: Deqian Kong, Dehong Xu, Minglu Zhao, Bo Pang, Jianwen Xie, Andrew Lizarraga, Yuhao Huang, Sirui Xie, Ying Nian Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04647
• Pdf URL: https://arxiv.org/pdf/2402.04647
• Copy Paste: [[2402.04647]] Latent Plan Transformer: Planning as Latent Variable Inference(https://arxiv.org/abs/2402.04647)
• Keywords: prompt, rag
• Abstract: In tasks aiming for long-term returns, planning becomes necessary. We study generative modeling for planning with datasets repurposed from offline reinforcement learning. Specifically, we identify temporal consistency in the absence of step-wise rewards as one key technical challenge. We introduce the Latent Plan Transformer (LPT), a novel model that leverages a latent space to connect a Transformer-based trajectory generator and the final return. LPT can be learned with maximum likelihood estimation on trajectory-return pairs. In learning, posterior sampling of the latent variable naturally gathers sub-trajectories to form a consistent abstraction despite the finite context. During test time, the latent variable is inferred from an expected return before policy execution, realizing the idea of planning as inference. It then guides the autoregressive policy throughout the episode, functioning as a plan. Our experiments demonstrate that LPT can discover improved decisions from suboptimal trajectories. It achieves competitive performance across several benchmarks, including Gym-Mujoco, Maze2D, and Connect Four, exhibiting capabilities of nuanced credit assignments, trajectory stitching, and adaptation to environmental contingencies. These results validate that latent variable inference can be a strong alternative to step-wise reward prompting.
• 摘要：在以长期回报为目标的任务中，规划是必要的。我们研究利用离线强化学习重新利用的数据集进行规划的生成模型。具体来说，我们将缺乏逐步奖励的情况下的时间一致性视为一项关键的技术挑战。我们引入了 Latent Plan Transformer (LPT)，这是一种利用潜在空间连接基于 Transformer 的轨迹生成器和最终返回的新颖模型。 LPT 可以通过轨迹-返回对的最大似然估计来学习。在学习中，尽管上下文有限，但潜在变量的后验采样自然会收集子轨迹以形成一致的抽象。在测试期间，潜变量是根据策略执行前的预期收益推断出来的，实现了规划即推理的思想。然后它在整个事件中指导自回归政策，作为计划发挥作用。我们的实验表明，LPT 可以从次优轨迹中发现改进的决策。它在 Gym-Mujoco、Maze2D 和 Connect Four 等多个基准测试中实现了具有竞争力的性能，展示了细致入微的信用分配、轨迹拼接和适应环境突发事件的能力。这些结果验证了潜变量推理可以成为逐步奖励提示的有力替代方案。

Title: Open-Vocabulary Calibration for Vision-Language Models

• Authors: Shuoyuan Wang, Jindong Wang, Guoqing Wang, Bob Zhang, Kaiyang Zhou, Hongxin Wei
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04655
• Pdf URL: https://arxiv.org/pdf/2402.04655
• Copy Paste: [[2402.04655]] Open-Vocabulary Calibration for Vision-Language Models(https://arxiv.org/abs/2402.04655)
• Keywords: language model, prompt, chat
• Abstract: Vision-language models (VLMs) have emerged as formidable tools, showing their strong capability in handling various open-vocabulary tasks in image recognition, text-driven visual content generation, and visual chatbots, to name a few. In recent years, considerable efforts and resources have been devoted to adaptation methods for improving downstream performance of VLMs, particularly on parameter-efficient fine-tuning methods like prompt learning. However, a crucial aspect that has been largely overlooked is the confidence calibration problem in fine-tuned VLMs, which could greatly reduce reliability when deploying such models in the real world. This paper bridges the gap by systematically investigating the confidence calibration problem in the context of prompt learning and reveals that existing calibration methods are insufficient to address the problem, especially in the open-vocabulary setting. To solve the problem, we present a simple and effective approach called Distance-Aware Calibration (DAC), which is based on scaling the temperature using as guidance the distance between predicted text labels and base classes. The experiments with 7 distinct prompt learning methods applied across 11 diverse downstream datasets demonstrate the effectiveness of DAC, which achieves high efficacy without sacrificing the inference speed.
• 摘要：视觉语言模型（VLM）已经成为强大的工具，显示出它们在处理图像识别、文本驱动的视觉内容生成和视觉聊天机器人等各种开放词汇任务方面的强大能力。近年来，人们在提高 VLM 下游性能的适应方法上投入了大量的精力和资源，特别是在快速学习等参数高效的微调方法上。然而，一个在很大程度上被忽视的关键方面是微调 VLM 中的置信度校准问题，这可能会大大降低在现实世界中部署此类模型时的可靠性。本文通过系统地研究即时学习背景下的置信度校准问题来弥补这一差距，并揭示现有的校准方法不足以解决该问题，特别是在开放词汇环境中。为了解决这个问题，我们提出了一种简单而有效的方法，称为距离感知校准（DAC），该方法基于使用预测文本标签和基类之间的距离作为指导来缩放温度。在 11 个不同的下游数据集上应用 7 种不同的提示学习方法的实验证明了 DAC 的有效性，它在不牺牲推理速度的情况下实现了高效率。

Title: A Perspective on Individualized Treatment Effects Estimation from Time-series Health Data

• Authors: Ghadeer O. Ghosheh, Moritz Gögl, Tingting Zhu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04668
• Pdf URL: https://arxiv.org/pdf/2402.04668
• Copy Paste: [[2402.04668]] A Perspective on Individualized Treatment Effects Estimation from Time-series Health Data(https://arxiv.org/abs/2402.04668)
• Keywords: rag
• Abstract: The burden of diseases is rising worldwide, with unequal treatment efficacy for patient populations that are underrepresented in clinical trials. Healthcare, however, is driven by the average population effect of medical treatments and, therefore, operates in a "one-size-fits-all" approach, not necessarily what best fits each patient. These facts suggest a pressing need for methodologies to study individualized treatment effects (ITE) to drive personalized treatment. Despite the increased interest in machine-learning-driven ITE estimation models, the vast majority focus on tabular data with limited review and understanding of methodologies proposed for time-series electronic health records (EHRs). To this end, this work provides an overview of ITE works for time-series data and insights into future research. The work summarizes the latest work in the literature and reviews it in light of theoretical assumptions, types of treatment settings, and computational frameworks. Furthermore, this work discusses challenges and future research directions for ITEs in a time-series setting. We hope this work opens new directions and serves as a resource for understanding one of the exciting yet under-studied research areas.
• 摘要：全球范围内的疾病负担正在增加，临床试验中代表性不足的患者群体的治疗效果不平等。然而，医疗保健是由医疗治疗的平均人口效应驱动的，因此，以“一刀切”的方式运作，不一定最适合每个患者。这些事实表明迫切需要研究个体化治疗效果（ITE）的方法来推动个性化治疗。尽管人们对机器学习驱动的 ITE 估计模型越来越感兴趣，但绝大多数模型都关注表格数据，而对时间序列电子健康记录 (EHR) 提出的方法的审查和理解有限。为此，这项工作概述了时间序列数据的 ITE 工作以及对未来研究的见解。该工作总结了文献中的最新工作，并根据理论假设、治疗设置类型和计算框架对其进行了回顾。此外，这项工作讨论了时间序列环境中 ITE 的挑战和未来研究方向。我们希望这项工作能够开辟新的方向，并成为了解令人兴奋但尚未充分研究的研究领域之一的资源。

Title: Group Distributionally Robust Dataset Distillation with Risk Minimization

• Authors: Saeed Vahidian, Mingyu Wang, Jianyang Gu, Vyacheslav Kungurtsev, Wei Jiang, Yiran Chen
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2402.04676
• Pdf URL: https://arxiv.org/pdf/2402.04676
• Copy Paste: [[2402.04676]] Group Distributionally Robust Dataset Distillation with Risk Minimization(https://arxiv.org/abs/2402.04676)
• Keywords: rag
• Abstract: Dataset distillation (DD) has emerged as a widely adopted technique for crafting a synthetic dataset that captures the essential information of a training dataset, facilitating the training of accurate neural models. Its applications span various domains, including transfer learning, federated learning, and neural architecture search. The most popular methods for constructing the synthetic data rely on matching the convergence properties of training the model with the synthetic dataset and the training dataset. However, targeting the training dataset must be thought of as auxiliary in the same sense that the training set is an approximate substitute for the population distribution, and the latter is the data of interest. Yet despite its popularity, an aspect that remains unexplored is the relationship of DD to its generalization, particularly across uncommon subgroups. That is, how can we ensure that a model trained on the synthetic dataset performs well when faced with samples from regions with low population density? Here, the representativeness and coverage of the dataset become salient over the guaranteed training error at inference. Drawing inspiration from distributionally robust optimization, we introduce an algorithm that combines clustering with the minimization of a risk measure on the loss to conduct DD. We provide a theoretical rationale for our approach and demonstrate its effective generalization and robustness across subgroups through numerical experiments.
• 摘要：数据集蒸馏（DD）已成为一种广泛采用的技术，用于制作合成数据集，捕获训练数据集的基本信息，促进准确神经模型的训练。其应用跨越各个领域，包括迁移学习、联邦学习和神经架构搜索。构建合成数据的最流行的方法依赖于将训练模型与合成数据集和训练数据集的收敛特性相匹配。然而，以训练数据集为目标必须被视为辅助，就像训练集是总体分布的近似替代品一样，后者是感兴趣的数据。然而，尽管 DD 很受欢迎，但仍有待探索的一个方面是 DD 与其泛化的关系，特别是在不常见的亚组中。也就是说，我们如何确保在合成数据集上训练的模型在面对来自人口密度低的地区的样本时表现良好？在这里，数据集的代表性和覆盖范围比推理时保证的训练误差变得更加突出。从分布式鲁棒优化中汲取灵感，我们引入了一种将聚类与损失风险度量最小化相结合的算法，以进行 DD。我们为我们的方法提供了理论依据，并通过数值实验证明了其跨子组的有效泛化和鲁棒性。

Title: Source Identification in Abstractive Summarization

• Authors: Yoshi Suhara, Dimitris Alikaniotis
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04677
• Pdf URL: https://arxiv.org/pdf/2402.04677
• Copy Paste: [[2402.04677]] Source Identification in Abstractive Summarization(https://arxiv.org/abs/2402.04677)
• Keywords: code
• Abstract: Neural abstractive summarization models make summaries in an end-to-end manner, and little is known about how the source information is actually converted into summaries. In this paper, we define input sentences that contain essential information in the generated summary as $\textit{source sentences}$ and study how abstractive summaries are made by analyzing the source sentences. To this end, we annotate source sentences for reference summaries and system summaries generated by PEGASUS on document-summary pairs sampled from the CNN/DailyMail and XSum datasets. We also formulate automatic source sentence detection and compare multiple methods to establish a strong baseline for the task. Experimental results show that the perplexity-based method performs well in highly abstractive settings, while similarity-based methods perform robustly in relatively extractive settings. Our code and data are available at https://github.com/suhara/sourcesum.
• 摘要：神经抽象摘要模型以端到端的方式进行摘要，但人们对源信息实际上如何转换为摘要知之甚少。在本文中，我们将生成的摘要中包含基本信息的输入句子定义为$\textit{源句子}$，并通过分析源句子来研究如何进行抽象摘要。为此，我们在从 CNN/DailyMail 和 XSum 数据集采样的文档摘要对上注释 PEGASUS 生成的参考摘要和系统摘要的源句子。我们还制定了自动源句检测并比较多种方法，为任务建立强大的基线。实验结果表明，基于困惑度的方法在高度抽象的设置中表现良好，而基于相似性的方法在相对抽象的设置中表现良好。我们的代码和数据可在 https://github.com/suhara/sourcesum 获取。

Title: Large Language Models As Faithful Explainers

• Authors: Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Ruixiang Tang, Fan Yang, Mengnan Du, Xuanting Cai, Xia Hu
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04678
• Pdf URL: https://arxiv.org/pdf/2402.04678
• Copy Paste: [[2402.04678]] Large Language Models As Faithful Explainers(https://arxiv.org/abs/2402.04678)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have recently become proficient in addressing complex tasks by utilizing their rich internal knowledge and reasoning ability. Consequently, this complexity hinders traditional input-focused explanation algorithms for explaining the complex decision-making processes of LLMs. Recent advancements have thus emerged for self-explaining their predictions through a single feed-forward inference in a natural language format. However, natural language explanations are often criticized for lack of faithfulness since these explanations may not accurately reflect the decision-making behaviors of the LLMs. In this work, we introduce a generative explanation framework, xLLM, to improve the faithfulness of the explanations provided in natural language formats for LLMs. Specifically, we propose an evaluator to quantify the faithfulness of natural language explanation and enhance the faithfulness by an iterative optimization process of xLLM, with the goal of maximizing the faithfulness scores. Experiments conducted on three NLU datasets demonstrate that xLLM can significantly improve the faithfulness of generated explanations, which are in alignment with the behaviors of LLMs.
• 摘要：大型语言模型（LLM）最近已经能够利用其丰富的内部知识和推理能力来熟练地解决复杂的任务。因此，这种复杂性阻碍了传统的以输入为中心的解释算法来解释法学硕士的复杂决策过程。因此，最近出现了通过自然语言格式的单一前馈推理来自我解释其预测的进展。然而，自然语言解释常常被批评缺乏忠实性，因为这些解释可能无法准确反映法学硕士的决策行为。在这项工作中，我们引入了一个生成解释框架 xLLM，以提高以自然语言格式为法学硕士提供的解释的忠实度。具体来说，我们提出了一个评估器来量化自然语言解释的可信度，并通过 xLLM 的迭代优化过程来增强可信度，以最大化可信度得分为目标。在三个 NLU 数据集上进行的实验表明，xLLM 可以显着提高生成解释的可信度，这与 LLM 的行为一致。

Title: Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers

• Authors: Abhimanyu Rajeshkumar Bambhaniya, Amir Yazdanbakhsh, Suvinay Subramanian, Sheng-Chun Kao, Shivani Agrawal, Utku Evci, Tushar Krishna
• Subjects: cs.LG, cs.AR
• Abstract URL: https://arxiv.org/abs/2402.04744
• Pdf URL: https://arxiv.org/pdf/2402.04744
• Copy Paste: [[2402.04744]] Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers(https://arxiv.org/abs/2402.04744)
• Keywords: language model, code
• Abstract: N:M Structured sparsity has garnered significant interest as a result of relatively modest overhead and improved efficiency. Additionally, this form of sparsity holds considerable appeal for reducing the memory footprint owing to their modest representation overhead. There have been efforts to develop training recipes for N:M structured sparsity, they primarily focus on low-sparsity regions ($\sim$50\%). Nonetheless, performance of models trained using these approaches tends to decline when confronted with high-sparsity regions ($>$80\%). In this work, we study the effectiveness of existing sparse training recipes at \textit{high-sparsity regions} and argue that these methods fail to sustain the model quality on par with low-sparsity regions. We demonstrate that the significant factor contributing to this disparity is the presence of elevated levels of induced noise in the gradient magnitudes. To mitigate this undesirable effect, we employ decay mechanisms to progressively restrict the flow of gradients towards pruned elements. Our approach improves the model quality by up to 2$\%$ and 5$\%$ in vision and language models at high sparsity regime, respectively. We also evaluate the trade-off between model accuracy and training compute cost in terms of FLOPs. At iso-training FLOPs, our method yields better performance compared to conventional sparse training recipes, exhibiting an accuracy improvement of up to 2$\%$. The source code is available at https://github.com/abhibambhaniya/progressive_gradient_flow_nm_sparsity.
• 摘要：N:M 结构化稀疏性由于相对适度的开销和提高的效率而引起了人们的极大兴趣。此外，这种形式的稀疏性由于其适度的表示开销而对于减少内存占用具有很大的吸引力。人们一直在努力开发 N:M 结构稀疏性的训练方法，他们主要关注低稀疏性区域 ($\sim$50\%)。尽管如此，当遇到高稀疏区域（$>$80\%）时，使用这些方法训练的模型的性能往往会下降。在这项工作中，我们研究了 \textit{高稀疏区域} 上现有稀疏训练方法的有效性，并认为这些方法无法维持与低稀疏区域相同的模型质量。我们证明，造成这种差异的重要因素是梯度幅度中感应噪声水平升高。为了减轻这种不良影响，我们采用衰减机制来逐步限制梯度流向修剪元素。我们的方法在高稀疏度情况下将视觉和语言模型的模型质量分别提高了 2$\%$ 和 5$\%$。我们还根据 FLOP 评估模型准确性和训练计算成本之间的权衡。在 iso-training FLOP 中，与传统的稀疏训练方法相比，我们的方法产生了更好的性能，准确率提高了高达 2$\%$。源代码可在 https://github.com/abhibambhaniya/progressive_gradient_flow_nm_sparsity 获取。

Title: Code as Reward: Empowering Reinforcement Learning with VLMs

• Authors: David Venuto, Sami Nur Islam, Martin Klissarov, Doina Precup, Sherry Yang, Ankit Anand
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04764
• Pdf URL: https://arxiv.org/pdf/2402.04764
• Copy Paste: [[2402.04764]] Code as Reward: Empowering Reinforcement Learning with VLMs(https://arxiv.org/abs/2402.04764)
• Keywords: language model, code, rag, agent
• Abstract: Pre-trained Vision-Language Models (VLMs) are able to understand visual concepts, describe and decompose complex tasks into sub-tasks, and provide feedback on task completion. In this paper, we aim to leverage these capabilities to support the training of reinforcement learning (RL) agents. In principle, VLMs are well suited for this purpose, as they can naturally analyze image-based observations and provide feedback (reward) on learning progress. However, inference in VLMs is computationally expensive, so querying them frequently to compute rewards would significantly slowdown the training of an RL agent. To address this challenge, we propose a framework named Code as Reward (VLM-CaR). VLM-CaR produces dense reward functions from VLMs through code generation, thereby significantly reducing the computational burden of querying the VLM directly. We show that the dense rewards generated through our approach are very accurate across a diverse set of discrete and continuous environments, and can be more effective in training RL policies than the original sparse environment rewards.
• 摘要：预训练的视觉语言模型 (VLM) 能够理解视觉概念、描述复杂任务并将其分解为子任务，并提供任务完成情况的反馈。在本文中，我们的目标是利用这些功能来支持强化学习（RL）代理的训练。原则上，VLM 非常适合此目的，因为它们可以自然地分析基于图像的观察结果并提供有关学习进度的反馈（奖励）。然而，VLM 中的推理计算成本很高，因此频繁查询它们来计算奖励会显着减慢 RL 代理的训练速度。为了应对这一挑战，我们提出了一个名为“代码即奖励”（VLM-CaR）的框架。 VLM-CaR 通过代码生成从 VLM 产生密集的奖励函数，从而显着减少直接查询 VLM 的计算负担。我们证明，通过我们的方法生成的密集奖励在不同的离散和连续环境中都非常准确，并且在训练 RL 策略方面比原始的稀疏环境奖励更有效。

Title: StableMask: Refining Causal Masking in Decoder-only Transformer

• Authors: Qingyu Yin, Xuzheng He, Xiang Zhuang, Yu Zhao, Jianhua Yao, Xiaoyu Shen, Qiang Zhang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04779
• Pdf URL: https://arxiv.org/pdf/2402.04779
• Copy Paste: [[2402.04779]] StableMask: Refining Causal Masking in Decoder-only Transformer(https://arxiv.org/abs/2402.04779)
• Keywords: language model, llm, code
• Abstract: The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling. Despite its exceptional performance across various tasks, we have identified two limitations: First, it requires all attention scores to be non-zero and sum up to 1, even if the current embedding has sufficient self-contained information. This compels the model to assign disproportional excessive attention to specific tokens. Second, RPE-based Transformers are not universal approximators due to their limited capacity at encoding absolute positional information, which limits their application in position-critical tasks. In this work, we propose StableMask: a parameter-free method to address both limitations by refining the causal mask. It introduces pseudo-attention values to balance attention distributions and encodes absolute positional information via a progressively decreasing mask ratio. StableMask's effectiveness is validated both theoretically and empirically, showing significant enhancements in language models with parameter sizes ranging from 71M to 1.4B across diverse datasets and encoding methods. We further show that it naturally supports (1) efficient extrapolation without special tricks such as StreamingLLM and (2) easy integration with existing attention optimization techniques.
• 摘要：具有因果屏蔽和相对位置编码（RPE）的纯解码器 Transformer 架构已成为语言建模中事实上的选择。尽管它在各种任务中表现出色，但我们发现了两个局限性：首先，它要求所有注意力分数不为零且总和为 1，即使当前嵌入具有足够的独立信息。这迫使模型对特定标记分配不成比例的过度关注。其次，基于 RPE 的 Transformer 不是通用逼近器，因为它们编码绝对位置信息的能力有限，这限制了它们在位置关键任务中的应用。在这项工作中，我们提出了 StableMask：一种无参数方法，通过细化因果掩码来解决这两个限制。它引入伪注意力值来平衡注意力分布，并通过逐渐减小的掩码比率对绝对位置信息进行编码。 StableMask 的有效性得到了理论和经验的验证，在不同的数据集和编码方法中，参数大小从 71M 到 1.4B 的语言模型都得到了显着的增强。我们进一步表明，它自然支持（1）高效外推，无需 StreamingLLM 等特殊技巧；（2）与现有注意力优化技术轻松集成。

Title: A Hypothesis-Driven Framework for the Analysis of Self-Rationalising Models

• Authors: Marc Braun, Jenny Kunz
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04787
• Pdf URL: https://arxiv.org/pdf/2402.04787
• Copy Paste: [[2402.04787]] A Hypothesis-Driven Framework for the Analysis of Self-Rationalising Models(https://arxiv.org/abs/2402.04787)
• Keywords: gpt, llm
• Abstract: The self-rationalising capabilities of LLMs are appealing because the generated explanations can give insights into the plausibility of the predictions. However, how faithful the explanations are to the predictions is questionable, raising the need to explore the patterns behind them further. To this end, we propose a hypothesis-driven statistical framework. We use a Bayesian network to implement a hypothesis about how a task (in our example, natural language inference) is solved, and its internal states are translated into natural language with templates. Those explanations are then compared to LLM-generated free-text explanations using automatic and human evaluations. This allows us to judge how similar the LLM's and the Bayesian network's decision processes are. We demonstrate the usage of our framework with an example hypothesis and two realisations in Bayesian networks. The resulting models do not exhibit a strong similarity to GPT-3.5. We discuss the implications of this as well as the framework's potential to approximate LLM decisions better in future work.
• 摘要：法学硕士的自我合理化能力很有吸引力，因为生成的解释可以深入了解预测的合理性。然而，这些解释对预测的忠实程度值得怀疑，因此需要进一步探索其背后的模式。为此，我们提出了一个假设驱动的统计框架。我们使用贝叶斯网络来实现关于如何解决任务（在我们的示例中为自然语言推理）的假设，并将其内部状态通过模板翻译为自然语言。然后使用自动和人工评估将这些解释与法学硕士生成的自由文本解释进行比较。这使我们能够判断法学硕士和贝叶斯网络的决策过程有多么相似。我们通过一个示例假设和贝叶斯网络中的两个实现来演示我们的框架的用法。生成的模型与 GPT-3.5 没有表现出很强的相似性。我们讨论了这一点的含义以及该框架在未来工作中更好地近似法学硕士决策的潜力。

Title: MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark

• Authors: Dongping Chen, Ruoxi Chen, Shilin Zhang, Yinuo Liu, Yaochen Wang, Huichi Zhou, Qihui Zhang, Pan Zhou, Yao Wan, Lichao Sun
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2402.04788
• Pdf URL: https://arxiv.org/pdf/2402.04788
• Copy Paste: [[2402.04788]] MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark(https://arxiv.org/abs/2402.04788)
• Keywords: language model, gpt, llm, code
• Abstract: Multimodal Large Language Models (MLLMs) have gained significant attention recently, showing remarkable potential in artificial general intelligence. However, assessing the utility of MLLMs presents considerable challenges, primarily due to the absence multimodal benchmarks that align with human preferences. Inspired by LLM-as-a-Judge in LLMs, this paper introduces a novel benchmark, termed MLLM-as-a-Judge, to assess the ability of MLLMs in assisting judges including three distinct tasks: Scoring Evaluation, Pair Comparison, and Batch Ranking. Our study reveals that, while MLLMs demonstrate remarkable human-like discernment in Pair Comparisons, there is a significant divergence from human preferences in Scoring Evaluation and Batch Ranking tasks. Furthermore, MLLMs still face challenges in judgment, including diverse biases, hallucinatory responses, and inconsistencies, even for advanced models such as GPT-4V. These findings emphasize the pressing need for enhancements and further research efforts regarding MLLMs as fully reliable evaluators. Code and dataset are available at https://github.com/Dongping-Chen/MLLM-as-a-Judge.
• 摘要：多模态大语言模型（MLLM）最近受到了广泛关注，在通用人工智能领域显示出巨大的潜力。然而，评估 MLLM 的效用面临着相当大的挑战，这主要是由于缺乏符合人类偏好的多模式基准。受法学硕士中的“LLM-as-a-Judge”的启发，本文引入了一种新颖的基准，称为“MLLM-as-a-Judge”，用于评估 MLLM 协助法官的能力，包括三个不同的任务：评分评估、配对比较和批量排行。我们的研究表明，虽然 MLLM 在配对比较中表现出显着的类似人类的辨别能力，但在评分评估和批次排名任务中与人类偏好存在显着差异。此外，即使对于 GPT-4V 等先进模型，MLLM 仍然面临判断挑战，包括多样化的偏见、幻觉反应和不一致。这些发现强调了将 MLLM 作为完全可靠的评估器进行增强和进一步研究工作的迫切需要。代码和数据集可在 https://github.com/Dongping-Chen/MLLM-as-a-Judge 获取。

Title: Direct Language Model Alignment from Online AI Feedback

• Authors: Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, Mathieu Blondel
• Subjects: cs.AI, cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2402.04792
• Pdf URL: https://arxiv.org/pdf/2402.04792
• Copy Paste: [[2402.04792]] Direct Language Model Alignment from Online AI Feedback(https://arxiv.org/abs/2402.04792)
• Keywords: language model, llm, prompt, rag
• Abstract: Direct alignment from preferences (DAP) methods, such as DPO, have recently emerged as efficient alternatives to reinforcement learning from human feedback (RLHF), that do not require a separate reward model. However, the preference datasets used in DAP methods are usually collected ahead of training and never updated, thus the feedback is purely offline. Moreover, responses in these datasets are often sampled from a language model distinct from the one being aligned, and since the model evolves over training, the alignment phase is inevitably off-policy. In this study, we posit that online feedback is key and improves DAP methods. Our method, online AI feedback (OAIF), uses an LLM as annotator: on each training iteration, we sample two responses from the current model and prompt the LLM annotator to choose which one is preferred, thus providing online feedback. Despite its simplicity, we demonstrate via human evaluation in several tasks that OAIF outperforms both offline DAP and RLHF methods. We further show that the feedback leveraged in OAIF is easily controllable, via instruction prompts to the LLM annotator.
• 摘要：偏好直接调整 (DAP) 方法（例如 DPO）最近已成为人类反馈强化学习 (RLHF) 的有效替代方案，不需要单独的奖励模型。然而，DAP 方法中使用的偏好数据集通常是在训练之前收集的，并且从未更新，因此反馈纯粹是离线的。此外，这些数据集中的响应通常是从与正在对齐的语言模型不同的语言模型中采样的，并且由于模型在训练过程中不断发展，因此对齐阶段不可避免地会偏离策略。在这项研究中，我们认为在线反馈是关键并改进了 DAP 方法。我们的方法，在线人工智能反馈（OAIF），使用 LLM 作为注释器：在每次训练迭代中，我们从当前模型中采样两个响应，并提示 LLM 注释器选择首选哪一个，从而提供在线反馈。尽管它很简单，但我们通过在多项任务中的人工评估证明 OAIF 优于离线 DAP 和 RLHF 方法。我们进一步表明，通过 LLM 注释器的指令提示，OAIF 中利用的反馈很容易控制。

Title: Scalable Multi-view Clustering via Explicit Kernel Features Maps

• Authors: Chakib Fettal, Lazhar Labiod, Mohamed Nadif
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04794
• Pdf URL: https://arxiv.org/pdf/2402.04794
• Copy Paste: [[2402.04794]] Scalable Multi-view Clustering via Explicit Kernel Features Maps(https://arxiv.org/abs/2402.04794)
• Keywords: rag
• Abstract: A growing awareness of multi-view learning as an important component in data science and machine learning is a consequence of the increasing prevalence of multiple views in real-world applications, especially in the context of networks. In this paper we introduce a new scalability framework for multi-view subspace clustering. An efficient optimization strategy is proposed, leveraging kernel feature maps to reduce the computational burden while maintaining good clustering performance. The scalability of the algorithm means that it can be applied to large-scale datasets, including those with millions of data points, using a standard machine, in a few minutes. We conduct extensive experiments on real-world benchmark networks of various sizes in order to evaluate the performance of our algorithm against state-of-the-art multi-view subspace clustering methods and attributed-network multi-view approaches.
• 摘要：人们越来越意识到多视图学习是数据科学和机器学习的重要组成部分，这是多视图在现实世界应用中日益流行的结果，尤其是在网络环境中。在本文中，我们介绍了一种用于多视图子空间聚类的新的可扩展性框架。提出了一种有效的优化策略，利用内核特征图来减少计算负担，同时保持良好的聚类性能。该算法的可扩展性意味着它可以使用标准机器在几分钟内应用于大规模数据集，包括具有数百万个数据点的数据集。我们对各种规模的现实世界基准网络进行了广泛的实验，以便根据最先进的多视图子空间聚类方法和属性网络多视图方法来评估我们的算法的性能。

Title: Aspect-Based Sentiment Analysis for Open-Ended HR Survey Responses

• Authors: Lois Rink, Job Meijdam, David Graus
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04812
• Pdf URL: https://arxiv.org/pdf/2402.04812
• Copy Paste: [[2402.04812]] Aspect-Based Sentiment Analysis for Open-Ended HR Survey Responses(https://arxiv.org/abs/2402.04812)
• Keywords: language model
• Abstract: Understanding preferences, opinions, and sentiment of the workforce is paramount for effective employee lifecycle management. Open-ended survey responses serve as a valuable source of information. This paper proposes a machine learning approach for aspect-based sentiment analysis (ABSA) of Dutch open-ended responses in employee satisfaction surveys. Our approach aims to overcome the inherent noise and variability in these responses, enabling a comprehensive analysis of sentiments that can support employee lifecycle management. Through response clustering we identify six key aspects (salary, schedule, contact, communication, personal attention, agreements), which we validate by domain experts. We compile a dataset of 1,458 Dutch survey responses, revealing label imbalance in aspects and sentiments. We propose few-shot approaches for ABSA based on Dutch BERT models, and compare them against bag-of-words and zero-shot baselines. Our work significantly contributes to the field of ABSA by demonstrating the first successful application of Dutch pre-trained language models to aspect-based sentiment analysis in the domain of human resources (HR).
• 摘要：了解员工的偏好、意见和情绪对于有效的员工生命周期管理至关重要。开放式调查回复是宝贵的信息来源。本文提出了一种机器学习方法，用于对荷兰员工满意度调查中的开放式回答进行基于方面的情感分析 (ABSA)。我们的方法旨在克服这些响应中固有的噪音和可变性，从而能够对支持员工生命周期管理的情绪进行全面分析。通过响应聚类，我们确定了六个关键方面（薪资、日程安排、联系方式、沟通、个人关注、协议），并由领域专家进行验证。我们编制了包含 1,458 份荷兰调查回复的数据集，揭示了方面和情绪方面的标签不平衡。我们提出了基于荷兰 BERT 模型的 ABSA 的少样本方法，并将它们与词袋和零样本基线进行比较。我们的工作通过展示荷兰预训练语言模型在人力资源 (HR) 领域基于方面的情感分析的首次成功应用，为 ABSA 领域做出了重大贡献。

Title: BOWLL: A Deceptively Simple Open World Lifelong Learner

• Authors: Roshni Kamath, Rupert Mitchell, Subarnaduti Paul, Kristian Kersting, Martin Mundt
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04814
• Pdf URL: https://arxiv.org/pdf/2402.04814
• Copy Paste: [[2402.04814]] BOWLL: A Deceptively Simple Open World Lifelong Learner(https://arxiv.org/abs/2402.04814)
• Keywords: lora, rag
• Abstract: The quest to improve scalar performance numbers on predetermined benchmarks seems to be deeply engraved in deep learning. However, the real world is seldom carefully curated and applications are seldom limited to excelling on test sets. A practical system is generally required to recognize novel concepts, refrain from actively including uninformative data, and retain previously acquired knowledge throughout its lifetime. Despite these key elements being rigorously researched individually, the study of their conjunction, open world lifelong learning, is only a recent trend. To accelerate this multifaceted field's exploration, we introduce its first monolithic and much-needed baseline. Leveraging the ubiquitous use of batch normalization across deep neural networks, we propose a deceptively simple yet highly effective way to repurpose standard models for open world lifelong learning. Through extensive empirical evaluation, we highlight why our approach should serve as a future standard for models that are able to effectively maintain their knowledge, selectively focus on informative data, and accelerate future learning.
• 摘要：在预定基准上提高标量性能数字的追求似乎深深地铭刻在深度学习中。然而，现实世界很少经过精心策划，应用程序也很少局限于在测试集上表现出色。实用的系统通常需要识别新颖的概念，避免主动包含无信息的数据，并在其整个生命周期中保留以前获得的知识。尽管对这些关键要素进行了单独的严格研究，但对它们的结合（开放世界终身学习）的研究只是最近的趋势。为了加速这个多方面领域的探索，我们引入了它的第一个整体且急需的基线。利用深度神经网络中批量归一化的普遍使用，我们提出了一种看似简单但高效的方法来重新利用标准模型以实现开放世界的终身学习。通过广泛的实证评估，我们强调了为什么我们的方法应该成为能够有效维护知识、有选择地关注信息数据并加速未来学习的模型的未来标准。

Title: How Realistic Is Your Synthetic Data? Constraining Deep Generative Models for Tabular Data

• Authors: Mihaela Cătălina Stoian, Salijona Dyrmishi, Maxime Cordy, Thomas Lukasiewicz, Eleonora Giunchiglia
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04823
• Pdf URL: https://arxiv.org/pdf/2402.04823
• Copy Paste: [[2402.04823]] How Realistic Is Your Synthetic Data? Constraining Deep Generative Models for Tabular Data(https://arxiv.org/abs/2402.04823)
• Keywords: code
• Abstract: Deep Generative Models (DGMs) have been shown to be powerful tools for generating tabular data, as they have been increasingly able to capture the complex distributions that characterize them. However, to generate realistic synthetic data, it is often not enough to have a good approximation of their distribution, as it also requires compliance with constraints that encode essential background knowledge on the problem at hand. In this paper, we address this limitation and show how DGMs for tabular data can be transformed into Constrained Deep Generative Models (C-DGMs), whose generated samples are guaranteed to be compliant with the given constraints. This is achieved by automatically parsing the constraints and transforming them into a Constraint Layer (CL) seamlessly integrated with the DGM. Our extensive experimental analysis with various DGMs and tasks reveals that standard DGMs often violate constraints, some exceeding $95\%$ non-compliance, while their corresponding C-DGMs are never non-compliant. Then, we quantitatively demonstrate that, at training time, C-DGMs are able to exploit the background knowledge expressed by the constraints to outperform their standard counterparts with up to $6.5\%$ improvement in utility and detection. Further, we show how our CL does not necessarily need to be integrated at training time, as it can be also used as a guardrail at inference time, still producing some improvements in the overall performance of the models. Finally, we show that our CL does not hinder the sample generation time of the models.
• 摘要：深度生成模型 (DGM) 已被证明是生成表格数据的强大工具，因为它们越来越能够捕获表征数据的复杂分布。然而，为了生成真实的合成数据，仅仅对其分布有一个很好的近似值通常是不够的，因为它还需要遵守对当前问题的基本背景知识进行编码的约束。在本文中，我们解决了这一限制，并展示了如何将表格数据的 DGM 转换为约束深度生成模型 (C-DGM)，其生成的样本保证符合给定的约束。这是通过自动解析约束并将其转换为与 DGM 无缝集成的约束层 (CL) 来实现的。我们对各种 DGM 和任务进行的广泛实验分析表明，标准 DGM 经常违反约束条件，有些不合规性超过 95\%$，而其相应的 C-DGM 绝不会不合规。然后，我们定量地证明，在训练时，C-DGM 能够利用约束所表达的背景知识来超越其标准对手，在效用和检测方面提高高达 6.5\%$。此外，我们还展示了我们的 CL 不一定需要在训练时集成，因为它也可以在推理时用作护栏，仍然对模型的整体性能产生一些改进。最后，我们证明我们的 CL 不会阻碍模型的样本生成时间。

Title: Learning Communication Policies for Different Follower Behaviors in a Collaborative Reference Game

• Authors: Philipp Sadler, Sherzod Hakimov, David Schlangen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04824
• Pdf URL: https://arxiv.org/pdf/2402.04824
• Copy Paste: [[2402.04824]] Learning Communication Policies for Different Follower Behaviors in a Collaborative Reference Game(https://arxiv.org/abs/2402.04824)
• Keywords: agent
• Abstract: Albrecht and Stone (2018) state that modeling of changing behaviors remains an open problem "due to the essentially unconstrained nature of what other agents may do". In this work we evaluate the adaptability of neural artificial agents towards assumed partner behaviors in a collaborative reference game. In this game success is achieved when a knowledgeable Guide can verbally lead a Follower to the selection of a specific puzzle piece among several distractors. We frame this language grounding and coordination task as a reinforcement learning problem and measure to which extent a common reinforcement training algorithm (PPO) is able to produce neural agents (the Guides) that perform well with various heuristic Follower behaviors that vary along the dimensions of confidence and autonomy. We experiment with a learning signal that in addition to the goal condition also respects an assumed communicative effort. Our results indicate that this novel ingredient leads to communicative strategies that are less verbose (staying silent in some of the steps) and that with respect to that the Guide's strategies indeed adapt to the partner's level of confidence and autonomy.
• 摘要：Albrecht 和 Stone（2018）指出，“由于其他智能体可能做的事情本质上不受限制”，行为变化的建模仍然是一个悬而未决的问题。在这项工作中，我们评估了神经人工代理对协作参考游戏中假设的伙伴行为的适应性。在这个游戏中，当知识渊博的向导能够口头引导追随者从多个干扰物中选择特定的拼图时，就获得了成功。我们将这种语言基础和协调任务构建为强化学习问题，并衡量常见的强化训练算法（PPO）能够在多大程度上产生神经代理（指南），这些代理在各种启发式追随者行为中表现良好，这些行为沿着维度变化信心和自主权。我们用一个学习信号进行实验，除了目标条件之外，还尊重假设的沟通努力。我们的结果表明，这种新颖的成分导致了更简洁的沟通策略（在某些步骤中保持沉默），并且就这一点而言，指南的策略确实适应了合作伙伴的自信和自主水平。

Title: Closing the Gap Between SGP4 and High-Precision Propagation via Differentiable Programming

• Authors: Giacomo Acciarini, Atılım Güneş Baydin, Dario Izzo
• Subjects: cs.LG, astro-ph.EP
• Abstract URL: https://arxiv.org/abs/2402.04830
• Pdf URL: https://arxiv.org/pdf/2402.04830
• Copy Paste: [[2402.04830]] Closing the Gap Between SGP4 and High-Precision Propagation via Differentiable Programming(https://arxiv.org/abs/2402.04830)
• Keywords: rag
• Abstract: The Simplified General Perturbations 4 (SGP4) orbital propagation method is widely used for predicting the positions and velocities of Earth-orbiting objects rapidly and reliably. Despite continuous refinement, SGP models still lack the precision of numerical propagators, which offer significantly smaller errors. This study presents dSGP4, a novel differentiable version of SGP4 implemented using PyTorch. By making SGP4 differentiable, dSGP4 facilitates various space-related applications, including spacecraft orbit determination, state conversion, covariance transformation, state transition matrix computation, and covariance propagation. Additionally, dSGP4's PyTorch implementation allows for embarrassingly parallel orbital propagation across batches of Two-Line Element Sets (TLEs), leveraging the computational power of CPUs, GPUs, and advanced hardware for distributed prediction of satellite positions at future times. Furthermore, dSGP4's differentiability enables integration with modern machine learning techniques. Thus, we propose a novel orbital propagation paradigm, ML-dSGP4, where neural networks are integrated into the orbital propagator. Through stochastic gradient descent, this combined model's inputs, outputs, and parameters can be iteratively refined, surpassing SGP4's precision. Neural networks act as identity operators by default, adhering to SGP4's behavior. However, dSGP4's differentiability allows fine-tuning with ephemeris data, enhancing precision while maintaining computational speed. This empowers satellite operators and researchers to train the model using specific ephemeris or high-precision numerical propagation data, significantly advancing orbital prediction capabilities.
• 摘要：简化一般扰动 4 (SGP4) 轨道传播方法广泛用于快速可靠地预测地球轨道物体的位置和速度。尽管不断完善，SGP 模型仍然缺乏数值传播器的精度，而数值传播器提供的误差要小得多。本研究提出了 dSGP4，这是一种使用 PyTorch 实现的 SGP4 的新型可微分版本。通过使 SGP4 可微，dSGP4 促进了各种与空间相关的应用，包括航天器轨道确定、状态转换、协方差变换、状态转移矩阵计算和协方差传播。此外，dSGP4 的 PyTorch 实现允许跨批次两线元素集 (TLE) 进行令人尴尬的并行轨道传播，利用 CPU、GPU 和先进硬件的计算能力来分布式预测未来的卫星位置。此外，dSGP4 的可微性使得能够与现代机器学习技术集成。因此，我们提出了一种新的轨道传播范例 ML-dSGP4，其中神经网络集成到轨道传播器中。通过随机梯度下降，该组合模型的输入、输出和参数可以迭代细化，超越了 SGP4 的精度。默认情况下，神经网络充当身份运营商，遵循 SGP4 的行为。然而，dSGP4 的可微性允许使用星历数据进行微调，从而在保持计算速度的同时提高精度。这使得卫星运营商和研究人员能够使用特定的星历或高精度数值传播数据来训练模型，从而显着提高轨道预测能力。

Title: Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning

• Authors: Hao Zhao, Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04833
• Pdf URL: https://arxiv.org/pdf/2402.04833
• Copy Paste: [[2402.04833]] Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning(https://arxiv.org/abs/2402.04833)
• Keywords: gpt, llm
• Abstract: There is a consensus that instruction fine-tuning of LLMs requires high-quality data, but what are they? LIMA (NeurIPS 2023) and AlpaGasus (ICLR 2024) are state-of-the-art methods for selecting such high-quality examples, either via manual curation or using GPT-3.5-Turbo as a quality scorer. We show that the extremely simple baseline of selecting the 1,000 instructions with longest responses from standard datasets can consistently outperform these sophisticated methods according to GPT-4 and PaLM-2 as judges, while remaining competitive on the OpenLLM benchmarks that test factual knowledge. We demonstrate this for several state-of-the-art LLMs (Llama-2-7B, Llama-2-13B, and Mistral-7B) and datasets (Alpaca-52k and Evol-Instruct-70k). In addition, a lightweight refinement of such long instructions can further improve the abilities of the fine-tuned LLMs, and allows us to obtain the 2nd highest-ranked Llama-2-7B-based model on AlpacaEval 2.0 while training on only 1,000 examples and no extra preference data. We also conduct a thorough analysis of our models to ensure that their enhanced performance is not simply due to GPT-4's preference for longer responses, thus ruling out any artificial improvement. In conclusion, our findings suggest that fine-tuning on the longest instructions should be the default baseline for any research on instruction fine-tuning.
• 摘要：人们一致认为法学硕士的指令微调需要高质量的数据，但高质量的数据是什么？ LIMA (NeurIPS 2023) 和 AlpaGasus (ICLR 2024) 是选择此类高质量示例的最先进方法，可以通过手动管理或使用 GPT-3.5-Turbo 作为质量评分器。我们证明，从标准数据集中选择具有最长响应的 1,000 条指令的极其简单的基线可以始终优于根据 GPT-4 和 PaLM-2 作为判断的这些复杂方法，同时在测试事实知识的 OpenLLM 基准上保持竞争力。我们用几个最先进的 Llama-2-7B、Llama-2-13B 和 Mistral-7B）和数据集（Alpaca-52k 和 Evol-Instruct-70k）演示了这一点。此外，对如此长的指令进行轻量级细化可以进一步提高微调后的 LLM 的能力，并使我们能够在 AlpacaEval 2.0 上获得排名第二的基于 Llama-2-7B 的模型，同时仅训练 1,000 个示例和没有额外的偏好数据。我们还对我们的模型进行了彻底的分析，以确保其性能的增强不仅仅是由于 GPT-4 偏好较长的响应，从而排除了任何人为改进。总之，我们的研究结果表明，对最长指令的微调应该成为任何指令微调研究的默认基线。

Title: On the Completeness of Invariant Geometric Deep Learning Models

• Authors: Zian Li, Xiyuan Wang, Shijia Kang, Muhan Zhang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04836
• Pdf URL: https://arxiv.org/pdf/2402.04836
• Copy Paste: [[2402.04836]] On the Completeness of Invariant Geometric Deep Learning Models(https://arxiv.org/abs/2402.04836)
• Keywords: rag
• Abstract: Invariant models, one important class of geometric deep learning models, are capable of generating meaningful geometric representations by leveraging informative geometric features. These models are characterized by their simplicity, good experimental results and computational efficiency. However, their theoretical expressive power still remains unclear, restricting a deeper understanding of the potential of such models. In this work, we concentrate on characterizing the theoretical expressiveness of invariant models. We first rigorously bound the expressiveness of the most classical invariant model, Vanilla DisGNN (message passing neural networks incorporating distance), restricting its unidentifiable cases to be only those highly symmetric geometric graphs. To break these corner cases' symmetry, we introduce a simple yet E(3)-complete invariant design by nesting Vanilla DisGNN, named GeoNGNN. Leveraging GeoNGNN as a theoretical tool, we for the first time prove the E(3)-completeness of three well-established geometric models: DimeNet, GemNet and SphereNet. Our results fill the gap in the theoretical power of invariant models, contributing to a rigorous and comprehensive understanding of their capabilities. Experimentally, GeoNGNN exhibits good inductive bias in capturing local environments, and achieves competitive results w.r.t. complicated models relying on high-order invariant/equivariant representations while exhibiting significantly faster computational speed.
• 摘要：不变模型是一类重要的几何深度学习模型，能够通过利用信息丰富的几何特征生成有意义的几何表示。这些模型具有简单、实验效果好、计算效率高等特点。然而，它们的理论表达能力仍不清楚，限制了对此类模型潜力的更深入理解。在这项工作中，我们专注于表征不变模型的理论表达能力。我们首先严格限制了最经典的不变模型 Vanilla DisGNN（包含距离的消息传递神经网络）的表达能力，将其无法识别的情况限制为仅那些高度对称的几何图。为了打破这些极端情况的对称性，我们通过嵌套 Vanilla DisGNN 引入了一种简单但 E(3) 完整的不变设计，名为 GeoNGNN。利用 GeoNGNN 作为理论工具，我们首次证明了三个成熟的几何模型的 E(3) 完备性：DimeNet、GemNet 和 SphereNet。我们的结果填补了不变模型理论能力的空白，有助于对其能力进行严格和全面的理解。实验上，GeoNGNN 在捕获局部环境方面表现出良好的归纳偏差，并取得了有竞争力的结果。依赖高阶不变/等变表示的复杂模型，同时表现出更快的计算速度。

Title: PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition

• Authors: Jinghui Lu, Ziwei Yang, Yanjie Wang, Xuejing Liu, Can Huang
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04838
• Pdf URL: https://arxiv.org/pdf/2402.04838
• Copy Paste: [[2402.04838]] PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition(https://arxiv.org/abs/2402.04838)
• Keywords: language model, llm
• Abstract: In this study, we aim to reduce generation latency for Named Entity Recognition (NER) with Large Language Models (LLMs). The main cause of high latency in LLMs is the sequential decoding process, which autoregressively generates all labels and mentions for NER, significantly increase the sequence length. To this end, we introduce Parallel Decoding in LLM for NE} (PaDeLLM-NER), a approach that integrates seamlessly into existing generative model frameworks without necessitating additional modules or architectural modifications. PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency. Experiments reveal that PaDeLLM-NER significantly increases inference speed that is 1.76 to 10.22 times faster than the autoregressive approach for both English and Chinese. Simultaneously it maintains the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets.
• 摘要：在本研究中，我们的目标是减少使用大型语言模型 (LLM) 的命名实体识别 (NER) 的生成延迟。 LLM 中高延迟的主要原因是顺序解码过程，该过程会自回归生成 NER 的所有标签和提及，显着增加序列长度。为此，我们在 LLM for NE (PaDeLLM-NER) 中引入并行解码，这是一种无缝集成到现有生成模型框架中的方法，无需额外的模块或架构修改。 PaDeLLM-NER 允许同时解码所有提及，从而减少生成延迟。实验表明，PaDeLLM-NER 显着提高了英语和中文的推理速度，比自回归方法快 1.76 至 10.22 倍。同时，它保持了预测的质量，其性能与各种数据集的最新技术水平相当。

Title: Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learning

• Authors: Yuxuan Bian, Xuan Ju, Jiangtong Li, Zhijian Xu, Dawei Cheng, Qiang Xu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04852
• Pdf URL: https://arxiv.org/pdf/2402.04852
• Copy Paste: [[2402.04852]] Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learning(https://arxiv.org/abs/2402.04852)
• Keywords: language model, llm
• Abstract: In this study, we present aLLM4TS, an innovative framework that adapts Large Language Models (LLMs) for time-series representation learning. Central to our approach is that we reconceive time-series forecasting as a self-supervised, multi-patch prediction task, which, compared to traditional mask-and-reconstruction methods, captures temporal dynamics in patch representations more effectively. Our strategy encompasses two-stage training: (i). a causal continual pre-training phase on various time-series datasets, anchored on next patch prediction, effectively syncing LLM capabilities with the intricacies of time-series data; (ii). fine-tuning for multi-patch prediction in the targeted time-series context. A distinctive element of our framework is the patch-wise decoding layer, which departs from previous methods reliant on sequence-level decoding. Such a design directly transposes individual patches into temporal sequences, thereby significantly bolstering the model's proficiency in mastering temporal patch-based representations. aLLM4TS demonstrates superior performance in several downstream tasks, proving its effectiveness in deriving temporal representations with enhanced transferability and marking a pivotal advancement in the adaptation of LLMs for time-series analysis.
• 摘要：在这项研究中，我们提出了 aLLM4TS，这是一种创新框架，适用于时间序列表示学习的大型语言模型 (LLM)。我们方法的核心是，我们将时间序列预测重新视为一种自我监督的多块预测任务，与传统的掩模和重建方法相比，它可以更有效地捕获块表示中的时间动态。我们的策略包括两个阶段的培训：(i)。对各种时间序列数据集进行因果连续预训练阶段，锚定于下一个补丁预测，有效地将 LLM 功能与复杂的时间序列数据同步； (二).在目标时间序列上下文中微调多块预测。我们框架的一个独特元素是补丁式解码层，它不同于以前依赖于序列级解码的方法。这种设计直接将各个补丁转置为时间序列，从而显着增强模型掌握基于时间补丁的表示的能力。 aLLM4TS 在多个下游任务中展示了卓越的性能，证明了其在导出具有增强可转移性的时间表示方面的有效性，并标志着 LLM 适应时间序列分析的关键进步。

Title: CodeIt: Self-Improving Language Models with Prioritized Hindsight Replay

• Authors: Natasha Butt, Blazej Manczak, Auke Wiggers, Corrado Rainone, David Zhang, Michaël Defferrard, Taco Cohen
• Subjects: cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04858
• Pdf URL: https://arxiv.org/pdf/2402.04858
• Copy Paste: [[2402.04858]] CodeIt: Self-Improving Language Models with Prioritized Hindsight Replay(https://arxiv.org/abs/2402.04858)
• Keywords: language model, code
• Abstract: Large language models are increasingly solving tasks that are commonly believed to require human-level reasoning ability. However, these models still perform very poorly on benchmarks of general intelligence such as the Abstraction and Reasoning Corpus (ARC). In this paper, we approach ARC as a programming-by-examples problem, and introduce a novel and scalable method for language model self-improvement called Code Iteration (CodeIt). Our method iterates between 1) program sampling and hindsight relabeling, and 2) learning from prioritized experience replay. By relabeling the goal of an episode (i.e., the target program output given input) to the realized output produced by the sampled program, our method effectively deals with the extreme sparsity of rewards in program synthesis. Applying CodeIt to the ARC dataset, we demonstrate that prioritized hindsight replay, along with pre-training and data-augmentation, leads to successful inter-task generalization. CodeIt is the first neuro-symbolic approach that scales to the full ARC evaluation dataset. Our method solves 15% of ARC evaluation tasks, achieving state-of-the-art performance and outperforming existing neural and symbolic baselines.
• 摘要：大型语言模型越来越多地解决人们普遍认为需要人类推理能力的任务。然而，这些模型在抽象与推理语料库（ARC）等通用智能基准上的表现仍然很差。在本文中，我们将 ARC 作为示例编程问题来处理，并介绍了一种新颖且可扩展的语言模型自我改进方法，称为代码迭代 (CodeIt)。我们的方法在 1）程序采样和事后重新标记以及 2）从优先经验重放中学习之间进行迭代。通过将一个片段的目标（即给定输入的目标程序输出）重新标记为采样程序产生的实现输出，我们的方法有效地处理了程序合成中奖励的极端稀疏性。将 CodeIt 应用于 ARC 数据集，我们证明了优先的事后重放以及预训练和数据增强可以成功实现任务间泛化。 CodeIt 是第一个可扩展到完整 ARC 评估数据集的神经符号方法。我们的方法解决了 15% 的 ARC 评估任务，实现了最先进的性能，并超越了现有的神经和符号基线。

Title: Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy

• Authors: Ruichu Cai, Siyang Huang, Jie Qiao, Wei Chen, Yan Zeng, Keli Zhang, Fuchun Sun, Yang Yu, Zhifeng Hao
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04869
• Pdf URL: https://arxiv.org/pdf/2402.04869
• Copy Paste: [[2402.04869]] Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy(https://arxiv.org/abs/2402.04869)
• Keywords: lora, agent
• Abstract: As a key component to intuitive cognition and reasoning solutions in human intelligence, causal knowledge provides great potential for reinforcement learning (RL) agents' interpretability towards decision-making by helping reduce the searching space. However, there is still a considerable gap in discovering and incorporating causality into RL, which hinders the rapid development of causal RL. In this paper, we consider explicitly modeling the generation process of states with the causal graphical model, based on which we augment the policy. We formulate the causal structure updating into the RL interaction process with active intervention learning of the environment. To optimize the derived objective, we propose a framework with theoretical performance guarantees that alternates between two steps: using interventions for causal structure learning during exploration and using the learned causal structure for policy guidance during exploitation. Due to the lack of public benchmarks that allow direct intervention in the state space, we design the root cause localization task in our simulated fault alarm environment and then empirically show the effectiveness and robustness of the proposed method against state-of-the-art baselines. Theoretical analysis shows that our performance improvement attributes to the virtuous cycle of causal-guided policy learning and causal structure learning, which aligns with our experimental results.
• 摘要：作为人类智能中直觉认知和推理解决方案的关键组成部分，因果知识通过帮助减少搜索空间，为强化学习 (RL) 代理的决策可解释性提供了巨大的潜力。然而，在发现因果关系并将其纳入强化学习方面仍存在相当大的差距，这阻碍了因果强化学习的快速发展。在本文中，我们考虑使用因果图模型对状态的生成过程进行显式建模，并在此基础上增强策略。我们通过对环境的主动干预学习，将因果结构更新表述为强化学习交互过程。为了优化导出的目标，我们提出了一个具有理论性能保证的框架，该框架在两个步骤之间交替：在探索过程中使用干预措施进行因果结构学习，并在开发过程中使用学习到的因果结构进行政策指导。由于缺乏允许直接干预状态空间的公共基准，我们在模拟故障警报环境中设计了根本原因定位任务，然后根据最先进的基准凭经验证明所提出方法的有效性和鲁棒性。理论分析表明，我们的绩效改进归因于因果引导的政策学习和因果结构学习的良性循环，这与我们的实验结果一致。

Title: L4Q: Parameter Efficient Quantization-Aware Training on Large Language Models via LoRA-wise LSQ

• Authors: Hyesung Jeon, Yulhwa Kim, Jae-joon Kim
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04902
• Pdf URL: https://arxiv.org/pdf/2402.04902
• Copy Paste: [[2402.04902]] L4Q: Parameter Efficient Quantization-Aware Training on Large Language Models via LoRA-wise LSQ(https://arxiv.org/abs/2402.04902)
• Keywords: language model, llm, lora, rag
• Abstract: Post-training quantization (PTQ) and quantization-aware training (QAT) methods are gaining popularity in mitigating the high memory and computational costs associated with Large Language Models (LLMs). In resource-constrained scenarios, PTQ, with its reduced training overhead, is often preferred over QAT, despite the latter's potential for higher accuracy. Meanwhile, parameter-efficient fine-tuning (PEFT) methods like low-rank adaptation (LoRA) have been introduced, and recent efforts have explored quantization-aware PEFT techniques. However, these approaches may lack generality due to their reliance on the pre-quantized model's configuration. Their effectiveness may be compromised by non-linearly quantized or mixed-precision weights, and the retraining of specific quantization parameters might impede optimal performance. To address these challenges, we propose L4Q, an algorithm for parameter-efficient quantization-aware training. L4Q leverages LoRA-wise learned quantization step size for LLMs, aiming to enhance generality. The simultaneous quantization-and-fine-tuning process of L4Q is applicable to high-precision models, yielding linearly quantized weights with superior accuracy. Our experiments, conducted on the LLaMA and LLaMA2 model families using an instructional dataset, showcase L4Q's capabilities in language comprehension and few-shot in-context learning, achieving sub-4-bit precision while maintaining comparable training times to applying PEFT on a quantized model.
• 摘要：训练后量化 (PTQ) 和量化感知训练 (QAT) 方法在减轻与大型语言模型 (LLM) 相关的高内存和计算成本方面越来越受欢迎。在资源受限的情况下，PTQ 由于训练开销减少，通常比 QAT 更受青睐，尽管后者具有更高的准确性。同时，引入了低秩自适应（LoRA）等参数高效微调（PEFT）方法，并且最近的工作已经探索了量化感知PEFT技术。然而，这些方法可能缺乏通用性，因为它们依赖于预量化模型的配置。它们的有效性可能会受到非线性量化或混合精度权重的影响，并且特定量化参数的重新训练可能会妨碍最佳性能。为了应对这些挑战，我们提出了 L4Q，一种用于参数高效量化感知训练的算法。 L4Q 利用 LLM 的 LoRA 学习量化步长，旨在增强通用性。 L4Q的同时量化和微调过程适用于高精度模型，产生具有卓越精度的线性量化权重。我们使用教学数据集在 LLaMA 和 LLaMA2 模型系列上进行的实验展示了 L4Q 在语言理解和少量上下文学习方面的能力，实现了低于 4 位的精度，同时保持了与在量化模型上应用 PEFT 相当的训练时间。

Title: Conformal Monte Carlo Meta-learners for Predictive Inference of Individual Treatment Effects

• Authors: Jef Jonkers, Jarne Verhaeghe, Glenn Van Wallendael, Luc Duchateau, Sofie Van Hoecke
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2402.04906
• Pdf URL: https://arxiv.org/pdf/2402.04906
• Copy Paste: [[2402.04906]] Conformal Monte Carlo Meta-learners for Predictive Inference of Individual Treatment Effects(https://arxiv.org/abs/2402.04906)
• Keywords: rag
• Abstract: Knowledge of the effect of interventions, called the treatment effect, is paramount for decision-making. Approaches to estimating this treatment effect, e.g. by using Conditional Average Treatment Effect (CATE) estimators, often only provide a point estimate of this treatment effect, while additional uncertainty quantification is frequently desired instead. Therefore, we present a novel method, the Conformal Monte Carlo (CMC) meta-learners, leveraging conformal predictive systems, Monte Carlo sampling, and CATE meta-learners, to instead produce a predictive distribution usable in individualized decision-making. Furthermore, we show how specific assumptions on the noise distribution of the outcome heavily affect these uncertainty predictions. Nonetheless, the CMC framework shows strong experimental coverage while retaining small interval widths to provide estimates of the true individual treatment effect.
• 摘要：了解干预措施的效果（称为治疗效果）对于决策至关重要。估计这种治疗效果的方法，例如通过使用条件平均治疗效果（CATE）估计器，通常只能提供治疗效果的点估计，而通常需要额外的不确定性量化。因此，我们提出了一种新颖的方法，即保形蒙特卡罗 (CMC) 元学习器，利用保形预测系统、蒙特卡罗采样和 CATE 元学习器，来生成可用于个性化决策的预测分布。此外，我们还展示了对结果噪声分布的具体假设如何严重影响这些不确定性预测。尽管如此，CMC 框架显示了强大的实验覆盖范围，同时保留了较小的间隔宽度，以提供对真实个体治疗效果的估计。

Title: Personalized Text Generation with Fine-Grained Linguistic Control

• Authors: Bashar Alhafni, Vivek Kulkarni, Dhruv Kumar, Vipul Raheja
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04914
• Pdf URL: https://arxiv.org/pdf/2402.04914
• Copy Paste: [[2402.04914]] Personalized Text Generation with Fine-Grained Linguistic Control(https://arxiv.org/abs/2402.04914)
• Keywords: language model, code
• Abstract: As the text generation capabilities of large language models become increasingly prominent, recent studies have focused on controlling particular aspects of the generated text to make it more personalized. However, most research on controllable text generation focuses on controlling the content or modeling specific high-level/coarse-grained attributes that reflect authors' writing styles, such as formality, domain, or sentiment. In this paper, we focus on controlling fine-grained attributes spanning multiple linguistic dimensions, such as lexical and syntactic attributes. We introduce a novel benchmark to train generative models and evaluate their ability to generate personalized text based on multiple fine-grained linguistic attributes. We systematically investigate the performance of various large language models on our benchmark and draw insights from the factors that impact their performance. We make our code, data, and pretrained models publicly available.
• 摘要：随着大型语言模型的文本生成能力变得越来越突出，最近的研究集中在控制生成文本的特定方面以使其更加个性化。然而，大多数关于可控文本生成的研究都集中在控制内容或建模反映作者写作风格的特定高级/粗粒度属性，例如形式、领域或情感。在本文中，我们专注于控制跨越多个语言维度的细粒度属性，例如词汇和句法属性。我们引入了一种新颖的基准来训练生成模型并评估它们基于多种细粒度语言属性生成个性化文本的能力。我们系统地研究了各种大型语言模型在基准测试中的性能，并从影响其性能的因素中得出见解。我们公开我们的代码、数据和预训练模型。

Title: Prompting Implicit Discourse Relation Annotation

• Authors: Frances Yung, Mansoor Ahmad, Merel Scholman, Vera Demberg
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04918
• Pdf URL: https://arxiv.org/pdf/2402.04918
• Copy Paste: [[2402.04918]] Prompting Implicit Discourse Relation Annotation(https://arxiv.org/abs/2402.04918)
• Keywords: language model, gpt, prompt, chat
• Abstract: Pre-trained large language models, such as ChatGPT, archive outstanding performance in various reasoning tasks without supervised training and were found to have outperformed crowdsourcing workers. Nonetheless, ChatGPT's performance in the task of implicit discourse relation classification, prompted by a standard multiple-choice question, is still far from satisfactory and considerably inferior to state-of-the-art supervised approaches. This work investigates several proven prompting techniques to improve ChatGPT's recognition of discourse relations. In particular, we experimented with breaking down the classification task that involves numerous abstract labels into smaller subtasks. Nonetheless, experiment results show that the inference accuracy hardly changes even with sophisticated prompt engineering, suggesting that implicit discourse relation classification is not yet resolvable under zero-shot or few-shot settings.
• 摘要：预训练的大型语言模型，例如 ChatGPT，在没有监督训练的情况下在各种推理任务中表现出色，并且被发现优于众包工作者。尽管如此，ChatGPT 在标准多项选择题引发的隐式话语关系分类任务中的表现仍然远不能令人满意，并且远远低于最先进的监督方法。这项工作研究了几种经过验证的提示技术，以提高 ChatGPT 对话语关系的识别。特别是，我们尝试将涉及大量抽象标签的分类任务分解为更小的子任务。尽管如此，实验结果表明，即使采用复杂的提示工程，推理精度也几乎没有变化，这表明隐式话语关系分类在零样本或少样本设置下尚无法解决。

Title: Two Trades is not Baffled: Condense Graph via Crafting Rational Gradient Matching

• Authors: Tianle Zhang, Yuchen Zhang, Kun Wang, Kai Wang, Beining Yang, Kaipeng Zhang, Wenqi Shao, Ping Liu, Joey Tianyi Zhou, Yang You
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.04924
• Pdf URL: https://arxiv.org/pdf/2402.04924
• Copy Paste: [[2402.04924]] Two Trades is not Baffled: Condense Graph via Crafting Rational Gradient Matching(https://arxiv.org/abs/2402.04924)
• Keywords: code, rag
• Abstract: Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have raised growing concerns. As one of the most promising directions, graph condensation methods address these issues by employing gradient matching, aiming to condense the full graph into a more concise yet information-rich synthetic set. Though encouraging, these strategies primarily emphasize matching directions of the gradients, which leads to deviations in the training trajectories. Such deviations are further magnified by the differences between the condensation and evaluation phases, culminating in accumulated errors, which detrimentally affect the performance of the condensed graphs. In light of this, we propose a novel graph condensation method named \textbf{C}raf\textbf{T}ing \textbf{R}ationa\textbf{L} trajectory (\textbf{CTRL}), which offers an optimized starting point closer to the original dataset's feature distribution and a more refined strategy for gradient matching. Theoretically, CTRL can effectively neutralize the impact of accumulated errors on the performance of condensed graphs. We provide extensive experiments on various graph datasets and downstream tasks to support the effectiveness of CTRL. Code is released at https://github.com/NUS-HPC-AI-Lab/CTRL.
• 摘要：大规模图的训练在图表示学习方面取得了显着的成果，但其成本和存储引起了越来越多的关注。作为最有前途的方向之一，图压缩方法通过采用梯度匹配来解决这些问题，旨在将完整图压缩为更简洁但信息丰富的合成集。尽管令人鼓舞，但这些策略主要强调梯度的匹配方向，这会导致训练轨迹出现偏差。这种偏差会因压缩阶段和评估阶段之间的差异而进一步放大，最终导致累积误差，从而对压缩图的性能产生不利影响。鉴于此，我们提出了一种新颖的图压缩方法，名为 \textbf{C}raf\textbf{T}ing \textbf{R}ationa\textbf{L} 轨迹（\textbf{CTRL}），它提供了一种优化的起始更接近原始数据集的特征分布和更精细的梯度匹配策略。理论上，CTRL可以有效抵消累积误差对压缩图性能的影响。我们对各种图数据集和下游任务提供了广泛的实验，以支持 CTRL 的有效性。代码发布于 https://github.com/NUS-HPC-AI-Lab/CTRL。

Title: A Bayesian Approach to Online Learning for Contextual Restless Bandits with Applications to Public Health

• Authors: Biyonka Liang, Lily Xu, Aparna Taneja, Milind Tambe, Lucas Janson
• Subjects: cs.LG, stat.AP
• Abstract URL: https://arxiv.org/abs/2402.04933
• Pdf URL: https://arxiv.org/pdf/2402.04933
• Copy Paste: [[2402.04933]] A Bayesian Approach to Online Learning for Contextual Restless Bandits with Applications to Public Health(https://arxiv.org/abs/2402.04933)
• Keywords: rag
• Abstract: Restless multi-armed bandits (RMABs) are used to model sequential resource allocation in public health intervention programs. In these settings, the underlying transition dynamics are often unknown a priori, requiring online reinforcement learning (RL). However, existing methods in online RL for RMABs cannot incorporate properties often present in real-world public health applications, such as contextual information and non-stationarity. We present Bayesian Learning for Contextual RMABs (BCoR), an online RL approach for RMABs that novelly combines techniques in Bayesian modeling with Thompson sampling to flexibly model a wide range of complex RMAB settings, such as contextual and non-stationary RMABs. A key contribution of our approach is its ability to leverage shared information within and between arms to learn unknown RMAB transition dynamics quickly in budget-constrained settings with relatively short time horizons. Empirically, we show that BCoR achieves substantially higher finite-sample performance than existing approaches over a range of experimental settings, including one constructed from a real-world public health campaign in India.
• 摘要：不安的多臂老虎机 (RMAB) 用于对公共卫生干预计划中的顺序资源分配进行建模。在这些设置中，潜在的转变动态通常是先验未知的，需要在线强化学习（RL）。然而，RMAB 的在线强化学习中的现有方法无法纳入现实世界公共卫生应用中常见的属性，例如上下文信息和非平稳性。我们提出了上下文 RMAB 的贝叶斯学习 (BCoR)，这是一种用于 RMAB 的在线 RL 方法，它新颖地将贝叶斯建模技术与汤普森采样相结合，以灵活地对各种复杂的 RMAB 设置进行建模，例如上下文和非平稳 RMAB。我们方法的一个关键贡献是它能够利用臂内部和臂之间的共享信息，在预算有限的环境中、在相对较短的时间范围内快速了解未知的 RMAB 过渡动态。根据经验，我们表明，在一系列实验设置（包括根据印度真实公共卫生活动构建的实验设置）中，BCoR 比现有方法实现了更高的有限样本性能。

Title: Reconfidencing LLMs from the Grouping Loss Perspective

• Authors: Lihu Chen, Alexandre Perez-Lebel, Fabian M. Suchanek, Gaël Varoquaux
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.04957
• Pdf URL: https://arxiv.org/pdf/2402.04957
• Copy Paste: [[2402.04957]] Reconfidencing LLMs from the Grouping Loss Perspective(https://arxiv.org/abs/2402.04957)
• Keywords: language model, gpt, llm, chat
• Abstract: Large Language Models (LLMs), including ChatGPT and LLaMA, are susceptible to generating hallucinated answers in a confident tone. While efforts to elicit and calibrate confidence scores have proven useful, recent findings show that controlling uncertainty must go beyond calibration: predicted scores may deviate significantly from the actual posterior probabilities due to the impact of grouping loss. In this work, we construct a new evaluation dataset derived from a knowledge base to assess confidence scores given to answers of Mistral and LLaMA. Experiments show that they tend to be overconfident. Further, we show that they are more overconfident on some answers than others, \emph{eg} depending on the nationality of the person in the query. In uncertainty-quantification theory, this is grouping loss. To address this, we propose a solution to reconfidence LLMs, canceling not only calibration but also grouping loss. The LLMs, after the reconfidencing process, indicate improved confidence alignment with the accuracy of their responses.
• 摘要：大型语言模型 (LLM)，包括 ChatGPT 和 LLaMA，很容易以自信的语气生成幻觉答案。虽然引出和校准置信度分数的努力已被证明是有用的，但最近的研究结果表明，控制不确定性必须超越校准：由于分组损失的影响，预测分数可能会显着偏离实际后验概率。在这项工作中，我们构建了一个从知识库派生的新评估数据集，以评估 Mistral 和 LLaMA 答案的置信度得分。实验表明他们往往过于自信。此外，我们表明他们对某些答案比其他答案更加过度自信，\emph{eg} 取决于查询中人的国籍。在不确定性量化理论中，这就是分组损失。为了解决这个问题，我们提出了一种重新信任法学硕士的解决方案，不仅取消校准，还取消分组损失。经过重新信任过程后，法学硕士表明他们的回答准确性与置信度有所提高。

Title: Text or Image? What is More Important in Cross-Domain Generalization Capabilities of Hate Meme Detection Models?

• Authors: Piush Aggarwal, Jawar Mehrabanian, Weigang Huang, Özge Alacam, Torsten Zesch
• Subjects: cs.CL, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2402.04967
• Pdf URL: https://arxiv.org/pdf/2402.04967
• Copy Paste: [[2402.04967]] Text or Image? What is More Important in Cross-Domain Generalization Capabilities of Hate Meme Detection Models?(https://arxiv.org/abs/2402.04967)
• Keywords: rag
• Abstract: This paper delves into the formidable challenge of cross-domain generalization in multimodal hate meme detection, presenting compelling findings. We provide enough pieces of evidence supporting the hypothesis that only the textual component of hateful memes enables the existing multimodal classifier to generalize across different domains, while the image component proves highly sensitive to a specific training dataset. The evidence includes demonstrations showing that hate-text classifiers perform similarly to hate-meme classifiers in a zero-shot setting. Simultaneously, the introduction of captions generated from images of memes to the hate-meme classifier worsens performance by an average F1 of 0.02. Through blackbox explanations, we identify a substantial contribution of the text modality (average of 83%), which diminishes with the introduction of meme's image captions (52%). Additionally, our evaluation on a newly created confounder dataset reveals higher performance on text confounders as compared to image confounders with an average $\Delta$F1 of 0.18.
• 摘要：本文深入探讨了多模式仇恨模因检测中跨领域泛化的艰巨挑战，提出了令人信服的发现。我们提供了足够的证据支持这一假设，即只有可恶模因的文本成分才能使现有的多模态分类器能够跨不同领域进行泛化，而图像成分被证明对特定的训练数据集高度敏感。证据包括演示，表明仇恨文本分类器在零样本设置中的表现与仇恨模因分类器类似。同时，将模因图像生成的字幕引入仇恨模因分类器会使性能恶化，平均 F1 为 0.02。通过黑盒解释，我们确定了文本模式的重大贡献（平均为 83%），随着模因图像标题的引入（52%），这种贡献逐渐减弱。此外，我们对新创建的混杂因素数据集的评估显示，与平均 $\Delta$F1 为 0.18 的图像混杂因素相比，文本混杂因素的性能更高。

Title: Multi-Sender Persuasion -- A Computational Perspective

• Authors: Safwan Hossain, Tonghan Wang, Tao Lin, Yiling Chen, David C. Parkes, Haifeng Xu
• Subjects: cs.AI, cs.GT
• Abstract URL: https://arxiv.org/abs/2402.04971
• Pdf URL: https://arxiv.org/pdf/2402.04971
• Copy Paste: [[2402.04971]] Multi-Sender Persuasion -- A Computational Perspective(https://arxiv.org/abs/2402.04971)
• Keywords: agent
• Abstract: We consider multiple senders with informational advantage signaling to convince a single self-interested actor towards certain actions. Generalizing the seminal Bayesian Persuasion framework, such settings are ubiquitous in computational economics, multi-agent learning, and machine learning with multiple objectives. The core solution concept here is the Nash equilibrium of senders' signaling policies. Theoretically, we prove that finding an equilibrium in general is PPAD-Hard; in fact, even computing a sender's best response is NP-Hard. Given these intrinsic difficulties, we turn to finding local Nash equilibria. We propose a novel differentiable neural network to approximate this game's non-linear and discontinuous utilities. Complementing this with the extra-gradient algorithm, we discover local equilibria that Pareto dominates full-revelation equilibria and those found by existing neural networks. Broadly, our theoretical and empirical contributions are of interest to a large class of economic problems.
• 摘要：我们考虑使用信息优势信号的多个发送者来说服单个自利行为者采取某些行动。概括开创性的贝叶斯说服框架，这种设置在计算经济学、多智能体学习和多目标机器学习中无处不在。这里的核心解决方案概念是发送者信令策略的纳什均衡。理论上，我们证明找到一般均衡是 PPAD-Hard；事实上，即使计算发送者的最佳响应也是 NP 困难的。考虑到这些内在的困难，我们转向寻找局部纳什均衡。我们提出了一种新颖的可微神经网络来近似该游戏的非线性和不连续效用。用超梯度算法对此进行补充，我们发现帕累托主导的局部均衡以及现有神经网络发现的局部均衡。广泛而言，我们的理论和实证贡献对一大类经济问题很有意义。

Title: An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration

• Authors: Yihao Li, Ru Zhang, Jianyi Liu, Gongshen Liu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.04978
• Pdf URL: https://arxiv.org/pdf/2402.04978
• Copy Paste: [[2402.04978]] An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration(https://arxiv.org/abs/2402.04978)
• Keywords: language model, llm, hallucination, prompt
• Abstract: While Large Language Models (LLMs) demonstrate exceptional performance in a multitude of Natural Language Processing (NLP) tasks, they encounter challenges in practical applications, including issues with hallucinations, inadequate knowledge updating, and limited transparency in the reasoning process. To overcome these limitations, this study innovatively proposes a collaborative training-free reasoning scheme involving tight cooperation between Knowledge Graph (KG) and LLMs. This scheme first involves using LLMs to iteratively explore KG, selectively retrieving a task-relevant knowledge subgraph to support reasoning. The LLMs are then guided to further combine inherent implicit knowledge to reason on the subgraph while explicitly elucidating the reasoning process. Through such a cooperative approach, our scheme achieves more reliable knowledge-based reasoning and facilitates the tracing of the reasoning results. Experimental results show that our scheme significantly progressed across multiple datasets, notably achieving over a 10% improvement on the QALD10 dataset compared to the best baseline and the fine-tuned state-of-the-art (SOTA) work. Building on this success, this study hopes to offer a valuable reference for future research in the fusion of KG and LLMs, thereby enhancing LLMs' proficiency in solving complex issues.
• 摘要：虽然大型语言模型 (LLM) 在众多自然语言处理 (NLP) 任务中表现出卓越的性能，但它们在实际应用中遇到了挑战，包括幻觉问题、知识更新不足以及推理过程透明度有限等问题。为了克服这些限制，本研究创新性地提出了一种协作免训练推理方案，涉及知识图谱（KG）和法学硕士之间的紧密合作。该方案首先涉及使用LLM迭代探索知识图谱，选择性地检索与任务相关的知识子图以支持推理。然后引导法学硕士进一步结合固有的隐性知识对子图进行推理，同时明确阐明推理过程。通过这种合作方式，我们的方案实现了更可靠的基于知识的推理，并有利于推理结果的追踪。实验结果表明，我们的方案在多个数据集上取得了显着进展，特别是与最佳基线和微调的最先进 (SOTA) 工作相比，在 QALD10 数据集上实现了 10% 以上的改进。在此成功的基础上，本研究希望为未来KG与LLM融合的研究提供有价值的参考，从而提高LLM解决复杂问题的能力。

Title: Beyond explaining: XAI-based Adaptive Learning with SHAP Clustering for Energy Consumption Prediction

• Authors: Tobias Clement, Hung Truong Thanh Nguyen, Nils Kemmerzell, Mohamed Abdelaal, Davor Stjelja
• Subjects: cs.LG, cs.DB
• Abstract URL: https://arxiv.org/abs/2402.04982
• Pdf URL: https://arxiv.org/pdf/2402.04982
• Copy Paste: [[2402.04982]] Beyond explaining: XAI-based Adaptive Learning with SHAP Clustering for Energy Consumption Prediction(https://arxiv.org/abs/2402.04982)
• Keywords: rag
• Abstract: This paper presents an approach integrating explainable artificial intelligence (XAI) techniques with adaptive learning to enhance energy consumption prediction models, with a focus on handling data distribution shifts. Leveraging SHAP clustering, our method provides interpretable explanations for model predictions and uses these insights to adaptively refine the model, balancing model complexity with predictive performance. We introduce a three-stage process: (1) obtaining SHAP values to explain model predictions, (2) clustering SHAP values to identify distinct patterns and outliers, and (3) refining the model based on the derived SHAP clustering characteristics. Our approach mitigates overfitting and ensures robustness in handling data distribution shifts. We evaluate our method on a comprehensive dataset comprising energy consumption records of buildings, as well as two additional datasets to assess the transferability of our approach to other domains, regression, and classification problems. Our experiments demonstrate the effectiveness of our approach in both task types, resulting in improved predictive performance and interpretable model explanations.
• 摘要：本文提出了一种将可解释人工智能（XAI）技术与自适应学习相结合的方法，以增强能耗预测模型，重点是处理数据分布变化。利用 SHAP 聚类，我们的方法为模型预测提供了可解释的解释，并使用这些见解来自适应地细化模型，平衡模型复杂性与预测性能。我们引入了一个三阶段过程：（1）获取 SHAP 值来解释模型预测，（2）对 SHAP 值进行聚类以识别不同的模式和异常值，以及（3）根据导出的 SHAP 聚类特征完善模型。我们的方法可以减轻过度拟合并确保处理数据分布变化的稳健性。我们在一个综合数据集上评估我们的方法，该数据集包括建筑物的能耗记录，以及两个附加数据集，以评估我们的方法到其他领域、回归和分类问题的可转移性。我们的实验证明了我们的方法在两种任务类型中的有效性，从而提高了预测性能和可解释的模型解释。

Title: Pedagogical Alignment of Large Language Models

• Authors: Shashank Sonkar, Kangqi Ni, Sapana Chaudhary, Richard G. Baraniuk
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.05000
• Pdf URL: https://arxiv.org/pdf/2402.05000
• Copy Paste: [[2402.05000]] Pedagogical Alignment of Large Language Models(https://arxiv.org/abs/2402.05000)
• Keywords: language model, llm
• Abstract: In this paper, we introduce the novel concept of pedagogically aligned Large Language Models (LLMs) that signifies a transformative shift in the application of LLMs within educational contexts. Rather than providing direct responses to user queries, pedagogically-aligned LLMs function as scaffolding tools, breaking complex problems into manageable subproblems and guiding students towards the final answer through constructive feedback and hints. The objective is to equip learners with problem-solving strategies that deepen their understanding and internalization of the subject matter. Previous research in this field has primarily applied the supervised finetuning approach without framing the objective as an alignment problem, hence not employing reinforcement learning through human feedback (RLHF) methods. This study reinterprets the narrative by viewing the task through the lens of alignment and demonstrates how RLHF methods emerge naturally as a superior alternative for aligning LLM behaviour. Building on this perspective, we propose a novel approach for constructing a reward dataset specifically designed for the pedagogical alignment of LLMs. We apply three state-of-the-art RLHF algorithms and find that they outperform SFT significantly. Our qualitative analyses across model differences and hyperparameter sensitivity further validate the superiority of RLHF over SFT. Also, our study sheds light on the potential of online feedback for enhancing the performance of pedagogically-aligned LLMs, thus providing valuable insights for the advancement of these models in educational settings.
• 摘要：在本文中，我们介绍了教学上一致的大语言模型（LLM）的新概念，这标志着 LLM 在教育环境中应用的变革性转变。与教学相一致的法学硕士不是对用户的查询提供直接答复，而是充当脚手架工具，将复杂的问题分解为可管理的子问题，并通过建设性的反馈和提示引导学生获得最终答案。目的是为学习者提供解决问题的策略，加深他们对主题的理解和内化。该领域的先前研究主要应用监督微调方法，而没有将目标视为对齐问题，因此没有采用通过人类反馈（RLHF）方法进行强化学习。这项研究通过从对齐的角度看待任务来重新解释叙述，并展示了 RLHF 方法如何自然地成为对齐 LLM 行为的优越替代方案。基于这个观点，我们提出了一种新的方法来构建专门为法学硕士教学协调而设计的奖励数据集。我们应用了三种最先进的 RLHF 算法，发现它们的性能显着优于 SFT。我们对模型差异和超参数敏感性的定性分析进一步验证了 RLHF 相对于 SFT 的优越性。此外，我们的研究揭示了在线反馈在提高教学法学硕士表现方面的潜力，从而为这些模型在教育环境中的进步提供了宝贵的见解。

Title: Randomized Confidence Bounds for Stochastic Partial Monitoring

• Authors: Maxime Heuillet, Ola Ahmad, Audrey Durand
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05002
• Pdf URL: https://arxiv.org/pdf/2402.05002
• Copy Paste: [[2402.05002]] Randomized Confidence Bounds for Stochastic Partial Monitoring(https://arxiv.org/abs/2402.05002)
• Keywords: rag, agent
• Abstract: The partial monitoring (PM) framework provides a theoretical formulation of sequential learning problems with incomplete feedback. On each round, a learning agent plays an action while the environment simultaneously chooses an outcome. The agent then observes a feedback signal that is only partially informative about the (unobserved) outcome. The agent leverages the received feedback signals to select actions that minimize the (unobserved) cumulative loss. In contextual PM, the outcomes depend on some side information that is observable by the agent before selecting the action on each round. In this paper, we consider the contextual and non-contextual PM settings with stochastic outcomes. We introduce a new class of strategies based on the randomization of deterministic confidence bounds, that extend regret guarantees to settings where existing stochastic strategies are not applicable. Our experiments show that the proposed RandCBP and RandCBPside* strategies improve state-of-the-art baselines in PM games. To encourage the adoption of the PM framework, we design a use case on the real-world problem of monitoring the error rate of any deployed classification system.
• 摘要：部分监控（PM）框架提供了具有不完全反馈的顺序学习问题的理论表述。在每一轮中，学习代理都会执行一个动作，而环境同时选择一个结果。然后，代理观察反馈信号，该信号仅提供有关（未观察到的）结果的部分信息。代理利用收到的反馈信号来选择最小化（未观察到的）累积损失的操作。在上下文 PM 中，结果取决于代理在每轮选择操作之前可以观察到的一些辅助信息。在本文中，我们考虑具有随机结果的上下文和非上下文 PM 设置。我们引入了一类基于确定性置信区间随机化的新策略，将后悔保证扩展到现有随机策略不适用的环境。我们的实验表明，所提出的 RandCBP 和 RandCBPside* 策略改善了 PM 游戏中最先进的基线。为了鼓励采用 PM 框架，我们针对监控任何已部署分类系统的错误率的现实问题设计了一个用例。

Title: Example-based Explanations for Random Forests using Machine Unlearning

• Authors: Tanmay Surve, Romila Pradhan
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.05007
• Pdf URL: https://arxiv.org/pdf/2402.05007
• Copy Paste: [[2402.05007]] Example-based Explanations for Random Forests using Machine Unlearning(https://arxiv.org/abs/2402.05007)
• Keywords: rag
• Abstract: Tree-based machine learning models, such as decision trees and random forests, have been hugely successful in classification tasks primarily because of their predictive power in supervised learning tasks and ease of interpretation. Despite their popularity and power, these models have been found to produce unexpected or discriminatory outcomes. Given their overwhelming success for most tasks, it is of interest to identify sources of their unexpected and discriminatory behavior. However, there has not been much work on understanding and debugging tree-based classifiers in the context of fairness. We introduce FairDebugger, a system that utilizes recent advances in machine unlearning research to identify training data subsets responsible for instances of fairness violations in the outcomes of a random forest classifier. FairDebugger generates top-$k$ explanations (in the form of coherent training data subsets) for model unfairness. Toward this goal, FairDebugger first utilizes machine unlearning to estimate the change in the tree structures of the random forest when parts of the underlying training data are removed, and then leverages the Apriori algorithm from frequent itemset mining to reduce the subset search space. We empirically evaluate our approach on three real-world datasets, and demonstrate that the explanations generated by FairDebugger are consistent with insights from prior studies on these datasets.
• 摘要：基于树的机器学习模型（例如决策树和随机森林）在分类任务中取得了巨大成功，主要是因为它们在监督学习任务中具有预测能力并且易于解释。尽管这些模型很受欢迎并且很有影响力，但人们发现这些模型会产生意想不到的或歧视性的结果。鉴于他们在大多数任务上取得了压倒性的成功，找出他们意外和歧视行为的根源是很有意义的。然而，在公平性的背景下理解和调试基于树的分类器还没有太多的工作。我们引入了 FairDebugger，该系统利用机器去学习研究的最新进展来识别导致随机森林分类器结果中公平性违规实例的训练数据子集。 FairDebugger 为模型不公平性生成 top-$k$ 解释（以一致的训练数据子集的形式）。为了实现这一目标，FairDebugger 首先利用机器去学习来估计删除部分底层训练数据时随机森林树结构的变化，然后利用频繁项集挖掘的 Apriori 算法来减少子集搜索空间。我们在三个真实数据集上对我们的方法进行了实证评估，并证明 FairDebugger 生成的解释与先前对这些数据集的研究的见解是一致的。

Title: Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching

• Authors: Yuchen Zhang, Tianle Zhang, Kai Wang, Ziyao Guo, Yuxuan Liang, Xavier Bresson, Wei Jin, Yang You
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05011
• Pdf URL: https://arxiv.org/pdf/2402.05011
• Copy Paste: [[2402.05011]] Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching(https://arxiv.org/abs/2402.05011)
• Keywords: code
• Abstract: Graph condensation aims to reduce the size of a large-scale graph dataset by synthesizing a compact counterpart without sacrificing the performance of Graph Neural Networks (GNNs) trained on it, which has shed light on reducing the computational cost for training GNNs. Nevertheless, existing methods often fall short of accurately replicating the original graph for certain datasets, thereby failing to achieve the objective of lossless condensation. To understand this phenomenon, we investigate the potential reasons and reveal that the previous state-of-the-art trajectory matching method provides biased and restricted supervision signals from the original graph when optimizing the condensed one. This significantly limits both the scale and efficacy of the condensed graph. In this paper, we make the first attempt toward \textit{lossless graph condensation} by bridging the previously neglected supervision signals. Specifically, we employ a curriculum learning strategy to train expert trajectories with more diverse supervision signals from the original graph, and then effectively transfer the information into the condensed graph with expanding window matching. Moreover, we design a loss function to further extract knowledge from the expert trajectories. Theoretical analysis justifies the design of our method and extensive experiments verify its superiority across different datasets. Code is released at https://github.com/NUS-HPC-AI-Lab/GEOM.
• 摘要：图压缩旨在通过合成紧凑的对应数据集来减小大规模图数据集的大小，而不牺牲在其上训练的图神经网络（GNN）的性能，这为降低训练 GNN 的计算成本提供了线索。然而，现有方法往往无法准确复制某些数据集的原始图，从而无法达到无损压缩的目的。为了理解这一现象，我们调查了潜在的原因，并揭示了以前最先进的轨迹匹配方法在优化压缩图时从原始图中提供了有偏差和受限的监督信号。这极大地限制了压缩图的规模和功效。在本文中，我们通过桥接先前被忽略的监督信号，首次尝试 \textit{无损图压缩}。具体来说，我们采用课程学习策略，从原始图中训练具有更多样化监督信号的专家轨迹，然后有效地将信息转移到具有扩展窗口匹配的压缩图中。此外，我们设计了一个损失函数来进一步从专家轨迹中提取知识。理论分析证明了我们的方法设计的合理性，大量的实验验证了它在不同数据集上的优越性。代码发布于 https://github.com/NUS-HPC-AI-Lab/GEOM。

Title: Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth

• Authors: Kevin Kögler, Alexander Shevchenko, Hamed Hassani, Marco Mondelli
• Subjects: cs.LG, cs.IT, stat.ML
• Abstract URL: https://arxiv.org/abs/2402.05013
• Pdf URL: https://arxiv.org/pdf/2402.05013
• Copy Paste: [[2402.05013]] Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth(https://arxiv.org/abs/2402.05013)
• Keywords: code
• Abstract: Autoencoders are a prominent model in many empirical branches of machine learning and lossy data compression. However, basic theoretical questions remain unanswered even in a shallow two-layer setting. In particular, to what degree does a shallow autoencoder capture the structure of the underlying data distribution? For the prototypical case of the 1-bit compression of sparse Gaussian data, we prove that gradient descent converges to a solution that completely disregards the sparse structure of the input. Namely, the performance of the algorithm is the same as if it was compressing a Gaussian source - with no sparsity. For general data distributions, we give evidence of a phase transition phenomenon in the shape of the gradient descent minimizer, as a function of the data sparsity: below the critical sparsity level, the minimizer is a rotation taken uniformly at random (just like in the compression of non-sparse data); above the critical sparsity, the minimizer is the identity (up to a permutation). Finally, by exploiting a connection with approximate message passing algorithms, we show how to improve upon Gaussian performance for the compression of sparse data: adding a denoising function to a shallow architecture already reduces the loss provably, and a suitable multi-layer decoder leads to a further improvement. We validate our findings on image datasets, such as CIFAR-10 and MNIST.
• 摘要：自动编码器是机器学习和有损数据压缩的许多经验分支中的重要模型。然而，即使在浅层两层环境中，基本的理论问题仍然没有得到解答。特别是，浅层自动编码器在多大程度上捕获底层数据分布的结构？对于稀疏高斯数据的 1 位压缩的典型情况，我们证明梯度下降收敛到完全忽略输入的稀疏结构的解决方案。也就是说，该算法的性能与压缩高斯源相同 - 没有稀疏性。对于一般数据分布，我们以梯度下降最小化器的形式给出相变现象的证据，作为数据稀疏性的函数：低于临界稀疏性水平，最小化器是随机均匀旋转的（就像在非稀疏数据的压缩）；在临界稀疏度之上，最小化者是恒等式（直到排列）。最后，通过利用与近似消息传递算法的联系，我们展示了如何改进稀疏数据压缩的高斯性能：向浅层架构添加去噪函数已经证明可以减少损失，并且合适的多层解码器可以导致进一步改进。我们在 CIFAR-10 和 MNIST 等图像数据集上验证了我们的发现。

Title: A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?

• Authors: Agustinus Kristiadi, Felix Strieth-Kalthoff, Marta Skreta, Pascal Poupart, Alán Aspuru-Guzik, Geoff Pleiss
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05015
• Pdf URL: https://arxiv.org/pdf/2402.05015
• Copy Paste: [[2402.05015]] A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?(https://arxiv.org/abs/2402.05015)
• Keywords: language model, llm, lora, rag
• Abstract: Automation is one of the cornerstones of contemporary material discovery. Bayesian optimization (BO) is an essential part of such workflows, enabling scientists to leverage prior domain knowledge into efficient exploration of a large molecular space. While such prior knowledge can take many forms, there has been significant fanfare around the ancillary scientific knowledge encapsulated in large language models (LLMs). However, existing work thus far has only explored LLMs for heuristic materials searches. Indeed, recent work obtains the uncertainty estimate -- an integral part of BO -- from point-estimated, non-Bayesian LLMs. In this work, we study the question of whether LLMs are actually useful to accelerate principled Bayesian optimization in the molecular space. We take a sober, dispassionate stance in answering this question. This is done by carefully (i) viewing LLMs as fixed feature extractors for standard but principled BO surrogate models and by (ii) leveraging parameter-efficient finetuning methods and Bayesian neural networks to obtain the posterior of the LLM surrogate. Our extensive experiments with real-world chemistry problems show that LLMs can be useful for BO over molecules, but only if they have been pretrained or finetuned with domain-specific data.
• 摘要：自动化是当代材料发现的基石之一。贝叶斯优化 (BO) 是此类工作流程的重要组成部分，使科学家能够利用先前的领域知识来有效探索大型分子空间。虽然此类先验知识可以采取多种形式，但人们对封装在大型语言模型 (LLM) 中的辅助科学知识进行了大肆宣传。然而，迄今为止，现有的工作仅探索了法学硕士的启发式材料搜索。事实上，最近的工作从点估计的非贝叶斯法学硕士获得了不确定性估计——BO 的一个组成部分。在这项工作中，我们研究了法学硕士是否真的有助于加速分子空间中的原则性贝叶斯优化。我们以冷静、冷静的态度来回答这个问题。这是通过仔细（i）将LLM视为标准但有原则的BO代理模型的固定特征提取器以及（ii）利用参数高效的微调方法和贝叶斯神经网络来获得LLM代理的后验来完成的。我们对现实世界化学问题进行的广泛实验表明，法学硕士对于分子上的 BO 很有用，但前提是它们已经过特定领域数据的预训练或微调。

Title: How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language Models

• Authors: Miriam Cuscito, Alfio Ferrara, Martin Ruskov
• Subjects: cs.CL, cs.CY
• Abstract URL: https://arxiv.org/abs/2402.05034
• Pdf URL: https://arxiv.org/pdf/2402.05034
• Copy Paste: [[2402.05034]] How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language Models(https://arxiv.org/abs/2402.05034)
• Keywords: language model
• Abstract: In this paper, we explore the idea of analysing the historical bias of contextual language models based on BERT by measuring their adequacy with respect to Early Modern (EME) and Modern (ME) English. In our preliminary experiments, we perform fill-in-the-blank tests with 60 masked sentences (20 EME-specific, 20 ME-specific and 20 generic) and three different models (i.e., BERT Base, MacBERTh, English HLM). We then rate the model predictions according to a 5-point bipolar scale between the two language varieties and derive a weighted score to measure the adequacy of each model to EME and ME varieties of English.
• 摘要：在本文中，我们探讨了通过衡量早期现代（EME）和现代（ME）英语的充分性来分析基于 BERT 的上下文语言模型的历史偏差的想法。在我们的初步实验中，我们使用 60 个屏蔽句子（20 个 EME 特定的、20 个 ME 特定的和 20 个通用的）和三种不同的模型（即 BERT Base、MacBERTh、English HLM）进行填空测试。然后，我们根据两种语言变体之间的 5 分双极量表对模型预测进行评分，并得出加权分数来衡量每个模型对 EME 和 ME 英语变体的充分性。

Title: PAC Learnability under Explanation-Preserving Graph Perturbations

• Authors: Xu Zheng, Farhad Shirani, Tianchun Wang, Shouwei Gao, Wenqian Dong, Wei Cheng, Dongsheng Luo
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05039
• Pdf URL: https://arxiv.org/pdf/2402.05039
• Copy Paste: [[2402.05039]] PAC Learnability under Explanation-Preserving Graph Perturbations(https://arxiv.org/abs/2402.05039)
• Keywords: rag
• Abstract: Graphical models capture relations between entities in a wide range of applications including social networks, biology, and natural language processing, among others. Graph neural networks (GNN) are neural models that operate over graphs, enabling the model to leverage the complex relationships and dependencies in graph-structured data. A graph explanation is a subgraph which is an `almost sufficient' statistic of the input graph with respect to its classification label. Consequently, the classification label is invariant, with high probability, to perturbations of graph edges not belonging to its explanation subgraph. This work considers two methods for leveraging such perturbation invariances in the design and training of GNNs. First, explanation-assisted learning rules are considered. It is shown that the sample complexity of explanation-assisted learning can be arbitrarily smaller than explanation-agnostic learning. Next, explanation-assisted data augmentation is considered, where the training set is enlarged by artificially producing new training samples via perturbation of the non-explanation edges in the original training set. It is shown that such data augmentation methods may improve performance if the augmented data is in-distribution, however, it may also lead to worse sample complexity compared to explanation-agnostic learning rules if the augmented data is out-of-distribution. Extensive empirical evaluations are provided to verify the theoretical analysis.
• 摘要：图形模型捕获广泛应用中实体之间的关系，包括社交网络、生物学和自然语言处理等。图神经网络 (GNN) 是在图上运行的神经模型，使模型能够利用图结构数据中的复杂关系和依赖关系。图解释是一个子图，它是输入图相对于其分类标签的“几乎足够”的统计量。因此，分类标签对于不属于其解释子图的图边的扰动很有可能是不变的。这项工作考虑了两种在 GNN 的设计和训练中利用这种扰动不变性的方法。首先，考虑解释辅助学习规则。研究表明，解释辅助学习的样本复杂度可以任意小于解释不可知学习的样本复杂度。接下来，考虑解释辅助数据增强，其中通过扰动原始训练集中的非解释边缘来人为地产生新的训练样本来扩大训练集。结果表明，如果增强数据分布均匀，则此类数据增强方法可以提高性能，但是，如果增强数据分布不均匀，则与解释不可知的学习规则相比，这种数据增强方法也可能导致更差的样本复杂性。提供了广泛的实证评估来验证理论分析。

Title: SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

• Authors: Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, Jing Shao
• Subjects: cs.CL, cs.AI, cs.CR, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05044
• Pdf URL: https://arxiv.org/pdf/2402.05044
• Copy Paste: [[2402.05044]] SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models(https://arxiv.org/abs/2402.05044)
• Keywords: language model, llm
• Abstract: In the rapidly evolving landscape of Large Language Models (LLMs), ensuring robust safety measures is paramount. To meet this crucial need, we propose \emph{SALAD-Bench}, a safety benchmark specifically designed for evaluating LLMs, attack, and defense methods. Distinguished by its breadth, SALAD-Bench transcends conventional benchmarks through its large scale, rich diversity, intricate taxonomy spanning three levels, and versatile functionalities.SALAD-Bench is crafted with a meticulous array of questions, from standard queries to complex ones enriched with attack, defense modifications and multiple-choice. To effectively manage the inherent complexity, we introduce an innovative evaluators: the LLM-based MD-Judge for QA pairs with a particular focus on attack-enhanced queries, ensuring a seamless, and reliable evaluation. Above components extend SALAD-Bench from standard LLM safety evaluation to both LLM attack and defense methods evaluation, ensuring the joint-purpose utility. Our extensive experiments shed light on the resilience of LLMs against emerging threats and the efficacy of contemporary defense tactics. Data and evaluator are released under \url{https://github.com/OpenSafetyLab/SALAD-BENCH}. Warning: this paper includes examples that may be offensive or harmful.
• 摘要：在快速发展的大型语言模型 (LLM) 领域，确保稳健的安全措施至关重要。为了满足这一关键需求，我们提出了 \emph{SALAD-Bench}，这是一个专门为评估 LLM、攻击和防御方法而设计的安全基准。 SALAD-Bench 以其广度而著称，其大规模、丰富的多样性、跨越三个级别的复杂分类以及多功能功能超越了传统基准。SALAD-Bench 精心设计了一系列问题，从标准查询到富含攻击的复杂问题、防御修改和多项选择。为了有效管理固有的复杂性，我们引入了一种创新的评估器：基于 LLM 的 MD-Judge，用于 QA 对，特别关注攻击增强查询，确保无缝、可靠的评估。上述组件将SALAD-Bench从标准的LLM安全评估扩展到LLM攻击和防御方法评估，确保了联合目的的实用性。我们的广泛实验揭示了法学硕士抵御新威胁的能力以及当代防御策略的有效性。数据和评估器在 \url{https://github.com/OpenSafetyLab/SALAD-BENCH} 下发布。警告：本文包含可能令人反感或有害的示例。

Title: How VADER is your AI? Towards a definition of artificial intelligence systems appropriate for regulation

• Authors: Leonardo C. T. Bezerra, Alexander E. I. Brownlee, Luana Ferraz Alvarenga, Renan Cipriano Moioli, Thais Vasconcelos Batista
• Subjects: cs.AI
• Abstract URL: https://arxiv.org/abs/2402.05048
• Pdf URL: https://arxiv.org/pdf/2402.05048
• Copy Paste: [[2402.05048]] How VADER is your AI? Towards a definition of artificial intelligence systems appropriate for regulation(https://arxiv.org/abs/2402.05048)
• Keywords: rag
• Abstract: Artificial intelligence (AI) has driven many information and communication technology (ICT) breakthroughs. Nonetheless, the scope of ICT systems has expanded far beyond AI since the Turing test proposal. Critically, recent AI regulation proposals adopt AI definitions affecting ICT techniques, approaches, and systems that are not AI. In some cases, even works from mathematics, statistics, and engineering would be affected. Worryingly, AI misdefinitions are observed from Western societies to the Global South. In this paper, we propose a framework to score how \textit{validated as appropriately-defined for regulation} (VADER) an AI definition is. Our online, publicly-available VADER framework scores the coverage of premises that should underlie AI definitions for regulation, which aim to (i) reproduce principles observed in other successful technology regulations, and (ii) include all AI techniques and approaches while excluding non-AI works. Regarding the latter, our score is based on a dataset of representative AI, non-AI ICT, and non-ICT examples. We demonstrate our contribution by reviewing the AI regulation proposals of key players, namely the United States, United Kingdom, European Union, and Brazil. Importantly, none of the proposals assessed achieve the appropriateness score, ranging from a revision need to a concrete risk to ICT systems and works from other fields.
• 摘要：人工智能 (AI) 推动了许多信息和通信技术 (ICT) 的突破。尽管如此，自图灵测试提案以来，ICT 系统的范围已经远远超出了人工智能的范围。至关重要的是，最近的人工智能监管提案采用了影响非人工智能的 ICT 技术、方法和系统的人工智能定义。在某些情况下，甚至数学、统计学和工程学的作品也会受到影响。令人担忧的是，从西方社会到南半球，人工智能的错误定义随处可见。在本文中，我们提出了一个框架来对 AI 定义的 \textit{验证为适当定义的监管} (VADER) 进行评分。我们的在线、公开可用的 VADER 框架对作为监管人工智能定义基础的前提覆盖范围进行了评分，其目的是（i）重现其他成功技术监管中观察到的原则，以及（ii）包括所有人工智能技术和方法，同时排除非人工智能技术和方法。人工智能有效。对于后者，我们的分数基于代表性人工智能、非人工智能 ICT 和非 ICT 示例的数据集。我们通过审查主要参与者（即美国、英国、欧盟和巴西）的人工智能监管提案来展示我们的贡献。重要的是，评估的提案均未达到适当性评分，从修订需求到对 ICT 系统和其他领域工作的具体风险。

Title: A Roadmap to Pluralistic Alignment

• Authors: Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, Yejin Choi
• Subjects: cs.AI, cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2402.05070
• Pdf URL: https://arxiv.org/pdf/2402.05070
• Copy Paste: [[2402.05070]] A Roadmap to Pluralistic Alignment(https://arxiv.org/abs/2402.05070)
• Keywords: language model
• Abstract: With increased power and prevalence of AI systems, it is ever more critical that AI systems are designed to serve all, i.e., people with diverse values and perspectives. However, aligning models to serve pluralistic human values remains an open research question. In this piece, we propose a roadmap to pluralistic alignment, specifically using language models as a test bed. We identify and formalize three possible ways to define and operationalize pluralism in AI systems: 1) Overton pluralistic models that present a spectrum of reasonable responses; 2) Steerably pluralistic models that can steer to reflect certain perspectives; and 3) Distributionally pluralistic models that are well-calibrated to a given population in distribution. We also propose and formalize three possible classes of pluralistic benchmarks: 1) Multi-objective benchmarks, 2) Trade-off steerable benchmarks, which incentivize models to steer to arbitrary trade-offs, and 3) Jury-pluralistic benchmarks which explicitly model diverse human ratings. We use this framework to argue that current alignment techniques may be fundamentally limited for pluralistic AI; indeed, we highlight empirical evidence, both from our own experiments and from other work, that standard alignment procedures might reduce distributional pluralism in models, motivating the need for further research on pluralistic alignment.
• 摘要：随着人工智能系统的功能不断增强和普及，人工智能系统的设计目的是为所有人（即具有不同价值观和观点的人）提供服务，这一点变得越来越重要。然而，调整模型以服务多元人类价值观仍然是一个开放的研究问题。在这篇文章中，我们提出了多元对齐的路线图，特别是使用语言模型作为测试平台。我们确定并形式化了三种可能的方法来定义和实施人工智能系统中的多元化：1）奥弗顿多元化模型，提出一系列合理的反应； 2）可引导的多元化模型，可以引导反映某些观点； 3) 分布多元模型，针对分布中的给定群体进行了良好校准。我们还提出并形式化了三类可能的多元基准：1）多目标基准，2）权衡可引导基准，激励模型转向任意权衡，以及3）陪审团多元基准，明确模拟不同的人类评级。我们使用这个框架来论证当前的对齐技术可能从根本上限制了多元人工智能；事实上，我们强调了来自我们自己的实验和其他工作的经验证据，即标准对齐程序可能会减少模型中的分布多元化，从而激发了对多元对齐进行进一步研究的需要。

Title: On diffusion models for amortized inference: Benchmarking and improving stochastic control and sampling

• Authors: Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, Nikolay Malkin
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2402.05098
• Pdf URL: https://arxiv.org/pdf/2402.05098
• Copy Paste: [[2402.05098]] On diffusion models for amortized inference: Benchmarking and improving stochastic control and sampling(https://arxiv.org/abs/2402.05098)
• Keywords: lora, code
• Abstract: We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at https://github.com/GFNOrg/gfn-diffusion as a base for future work on diffusion models for amortized inference.
• 摘要：我们研究训练扩散模型以从具有给定非标准化密度或能量函数的分布中进行采样的问题。我们对几种扩散结构推理方法进行了基准测试，包括基于模拟的变分方法和离策略方法（连续生成流网络）。我们的结果揭示了现有算法的相对优势，同时对过去工作中的一些主张提出了质疑。我们还提出了一种新颖的离策略方法探索策略，基于目标空间中的本地搜索并使用重播缓冲区，并表明它提高了各种目标分布上的样本质量。我们研究的采样方法和基准的代码在 https://github.com/GFNOrg/gfn-diffusion 上公开，作为未来摊销推理扩散模型工作的基础。

Title: Hydragen: High-Throughput LLM Inference with Shared Prefixes

• Authors: Jordan Juravsky, Bradley Brown, Ryan Ehrlich, Daniel Y. Fu, Christopher Ré, Azalia Mirhoseini
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05099
• Pdf URL: https://arxiv.org/pdf/2402.05099
• Copy Paste: [[2402.05099]] Hydragen: High-Throughput LLM Inference with Shared Prefixes(https://arxiv.org/abs/2402.05099)
• Keywords: language model, llm, prompt, chat, rag
• Abstract: Transformer-based large language models (LLMs) are now deployed to hundreds of millions of users. LLM inference is commonly performed on batches of sequences that share a prefix, such as few-shot examples or a chatbot system prompt. Decoding in this large-batch setting can be bottlenecked by the attention operation, which reads large key-value (KV) caches from memory and computes inefficient matrix-vector products for every sequence in the batch. In this work, we introduce Hydragen, a hardware-aware exact implementation of attention with shared prefixes. Hydragen computes attention over the shared prefix and unique suffixes separately. This decomposition enables efficient prefix attention by batching queries together across sequences, reducing redundant memory reads and enabling the use of hardware-friendly matrix multiplications. Our method can improve end-to-end LLM throughput by up to 32x against competitive baselines, with speedup growing with the batch size and shared prefix length. Hydragen also enables the use of very long shared contexts: with a high batch size, increasing the prefix length from 1K to 16K tokens decreases Hydragen throughput by less than 15%, while the throughput of baselines drops by over 90%. Hydragen generalizes beyond simple prefix-suffix decomposition and can be applied to tree-based prompt sharing patterns, allowing us to further reduce inference time on competitive programming problems by 55%.
• 摘要：基于 Transformer 的大型语言模型 (LLM) 现已部署到数亿用户。 LLM 推理通常在共享前缀的批量序列上执行，例如少数样本示例或聊天机器人系统提示。这种大批量设置中的解码可能会受到注意力操作的瓶颈，注意力操作会从内存中读取大型键值 (KV) 缓存，并为批次中的每个序列计算低效的矩阵向量乘积。在这项工作中，我们引入了 Hydragen，这是一种具有共享前缀的硬件感知的注意力精确实现。 Hydragen 分别计算对共享前缀和唯一后缀的注意力。这种分解通过跨序列批量查询来实现高效的前缀注意力，减少冗余内存读取并支持使用硬件友好的矩阵乘法。与竞争基线相比，我们的方法可以将端到端 LLM 吞吐量提高高达 32 倍，并且加速随着批量大小和共享前缀长度而增长。 Hydragen 还支持使用非常长的共享上下文：在批量大小较高的情况下，将前缀长度从 1K 令牌增加到 16K 令牌会使 Hydragen 吞吐量降低不到 15%，而基线吞吐量则下降超过 90%。 Hydragen 的推广超越了简单的前缀后缀分解，并且可以应用于基于树的提示共享模式，使我们能够将竞争性编程问题的推理时间进一步减少 55%。

Title: Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding

• Authors: Zachary Ankner, Rishab Parthasarathy, Aniruddha Nrusimha, Christopher Rinard, Jonathan Ragan-Kelley, William Brandon
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05109
• Pdf URL: https://arxiv.org/pdf/2402.05109
• Copy Paste: [[2402.05109]] Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding(https://arxiv.org/abs/2402.05109)
• Keywords: llm
• Abstract: To combat the memory bandwidth-bound nature of autoregressive LLM inference, previous research has proposed the speculative decoding framework. To perform speculative decoding, a small draft model proposes candidate continuations of the input sequence, that are then verified in parallel by the base model. One way to specify the draft model, as used in the recent Medusa decoding framework, is as a collection of light-weight heads, called draft heads, that operate on the base model's hidden states. To date, all existing draft heads have been sequentially independent, meaning that they speculate tokens in the candidate continuation independently of any preceding tokens in the candidate continuation. In this work, we propose Hydra heads, a sequentially dependent, drop-in replacement for standard draft heads that significantly improves speculation accuracy. Decoding with Hydra heads improves throughput compared to Medusa decoding with standard draft heads. We further explore the design space of Hydra head training objectives and architectures, and propose a carefully-tuned Hydra head recipe, which we call Hydra++, that improves decoding throughput by 1.31x and 2.71x compared to Medusa decoding and autoregressive decoding, respectively. Overall, Hydra heads are a simple intervention on standard draft heads that significantly improve the end-to-end speed of draft head based speculative decoding.
• 摘要：为了克服自回归 LLM 推理的内存带宽限制性质，之前的研究提出了推测性解码框架。为了执行推测解码，小型草稿模型提出输入序列的候选延续，然后由基本模型并行验证。指定草稿模型的一种方法（如最近的 Medusa 解码框架中所使用的）是作为轻量级头（称为草稿头）的集合，它们对基本模型的隐藏状态进行操作。迄今为止，所有现有的草稿头都是顺序独立的，这意味着它们独立于候选延续中的任何先前令牌来推测候选延续中的令牌。在这项工作中，我们提出了 Hydra 头，这是标准草案头的顺序依赖的直接替代品，可显着提高推测准确性。与使用标准草案头的 Medusa 解码相比，使用 Hydra 头进行解码可提高吞吐量。我们进一步探索了 Hydra 头部训练目标和架构的设计空间，并提出了经过仔细调整的 Hydra 头部配方，我们称之为 Hydra++，与 Medusa 解码和自回归解码相比，其解码吞吐量分别提高了 1.31 倍和 2.71 倍。总体而言，Hydra 头是对标准草案头的简单干预，可显着提高基于草案头的推测解码的端到端速度。

Title: Opening the AI black box: program synthesis via mechanistic interpretability

• Authors: Eric J. Michaud, Isaac Liao, Vedang Lad, Ziming Liu, Anish Mudide, Chloe Loughridge, Zifan Carl Guo, Tara Rezaei Kheirkhah, Mateja Vukelić, Max Tegmark
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05110
• Pdf URL: https://arxiv.org/pdf/2402.05110
• Copy Paste: [[2402.05110]] Opening the AI black box: program synthesis via mechanistic interpretability(https://arxiv.org/abs/2402.05110)
• Keywords: language model, gpt, code
• Abstract: We present MIPS, a novel method for program synthesis based on automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the learned algorithm into Python code. We test MIPS on a benchmark of 62 algorithmic tasks that can be learned by an RNN and find it highly complementary to GPT-4: MIPS solves 32 of them, including 13 that are not solved by GPT-4 (which also solves 30). MIPS uses an integer autoencoder to convert the RNN into a finite state machine, then applies Boolean or integer symbolic regression to capture the learned algorithm. As opposed to large language models, this program synthesis technique makes no use of (and is therefore not limited by) human training data such as algorithms and code from GitHub. We discuss opportunities and challenges for scaling up this approach to make machine-learned models more interpretable and trustworthy.
• 摘要：我们提出了 MIPS，一种新的程序合成方法，它基于经过训练来执行所需任务的神经网络的自动机械解释性，将学习到的算法自动提炼为 Python 代码。我们在 RNN 可以学习的 62 个算法任务的基准上测试了 MIPS，发现它与 GPT-4 高度互补：MIPS 解决了其中 32 个任务，其中包括 GPT-4 无法解决的 13 个任务（GPT-4 也解决了 30 个任务）。 MIPS 使用整数自动编码器将 RNN 转换为有限状态机，然后应用布尔或整数符号回归来捕获学习算法。与大型语言模型相反，这种程序综合技术不使用（因此不受限制）人类训练数据，例如来自 GitHub 的算法和代码。我们讨论了扩大这种方法的机遇和挑战，以使机器学习模型更具可解释性和可信度。