2024-02-12

Title: Cooperative Knowledge Distillation: A Learner Agnostic Approach

• Authors: Michael Livanos, Ian Davidson, Stephen Wong
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.05942
• Pdf URL: https://arxiv.org/pdf/2402.05942
• Copy Paste: [[2402.05942]] Cooperative Knowledge Distillation: A Learner Agnostic Approach(https://arxiv.org/abs/2402.05942)
• Keywords: code
• Abstract: Knowledge distillation is a simple but powerful way to transfer knowledge between a teacher model to a student model. Existing work suffers from at least one of the following key limitations in terms of direction and scope of transfer which restrict its use: all knowledge is transferred from teacher to student regardless of whether or not that knowledge is useful, the student is the only one learning in this exchange, and typically distillation transfers knowledge only from a single teacher to a single student. We formulate a novel form of knowledge distillation in which many models can act as both students and teachers which we call cooperative distillation. The models cooperate as follows: a model (the student) identifies specific deficiencies in it's performance and searches for another model (the teacher) who encodes learned knowledge into instructional virtual instances via counterfactual instance generation. Because different models may have different strengths and weaknesses, all models can act as either students or teachers (cooperation) when appropriate and only distill knowledge in areas specific to their strengths (focus). Since counterfactuals as a paradigm are not tied to any specific algorithm, we can use this method to distill knowledge between learners of different architectures, algorithms, and even feature spaces. We demonstrate that our approach not only outperforms baselines such as transfer learning, self-supervised learning, and multiple knowledge distillation algorithms on several datasets, but it can also be used in settings where the aforementioned techniques cannot.
• 摘要：知识蒸馏是一种在教师模型和学生模型之间转移知识的简单但有效的方法。现有工作在转移方向和范围方面至少存在以下一项关键限制，限制了其使用：所有知识都从教师转移到学生，无论该知识是否有用，学生是唯一的学习者在这种交换中，通常蒸馏仅将知识从单个教师转移到单个学生。我们制定了一种新颖的知识蒸馏形式，其中许多模型可以充当学生和教师，我们称之为合作蒸馏。这些模型的协作方式如下：一个模型（学生）识别其性能中的具体缺陷，并搜索另一个模型（教师），该模型通过反事实实例生成将学到的知识编码到教学虚拟实例中。由于不同的模型可能有不同的优势和劣势，因此所有模型都可以在适当的时候充当学生或教师（合作），并且仅在其优势（重点）特定领域提取知识。由于反事实作为一种范式不依赖于任何特定的算法，因此我们可以使用这种方法在不同架构、算法甚至特征空间的学习者之间提取知识。我们证明，我们的方法不仅在多个数据集上优于迁移学习、自监督学习和多种知识蒸馏算法等基线，而且还可以用于上述技术无法使用的环境中。

Title: A hybrid IndRNNLSTM approach for real-time anomaly detection in software-defined networks

• Authors: Sajjad Salem, Salman Asoudeh
• Subjects: cs.LG, cs.AI, cs.NI
• Abstract URL: https://arxiv.org/abs/2402.05943
• Pdf URL: https://arxiv.org/pdf/2402.05943
• Copy Paste: [[2402.05943]] A hybrid IndRNNLSTM approach for real-time anomaly detection in software-defined networks(https://arxiv.org/abs/2402.05943)
• Keywords: code
• Abstract: Anomaly detection in SDN using data flow prediction is a difficult task. This problem is included in the category of time series and regression problems. Machine learning approaches are challenging in this field due to the manual selection of features. On the other hand, deep learning approaches have important features due to the automatic selection of features. Meanwhile, RNN-based approaches have been used the most. The LSTM and GRU approaches learn dependent entities well; on the other hand, the IndRNN approach learns non-dependent entities in time series. The proposed approach tried to use a combination of IndRNN and LSTM approaches to learn dependent and non-dependent features. Feature selection approaches also provide a suitable view of features for the models; for this purpose, four feature selection models, Filter, Wrapper, Embedded, and Autoencoder were used. The proposed IndRNNLSTM algorithm, in combination with Embedded, was able to achieve MAE=1.22 and RMSE=9.92 on NSL-KDD data.
• 摘要：使用数据流预测进行 SDN 中的异常检测是一项艰巨的任务。该问题属于时间序列和回归问题的范畴。由于手动选择特征，机器学习方法在该领域具有挑战性。另一方面，由于特征的自动选择，深度学习方法具有重要的特征。同时，基于 RNN 的方法使用得最多。 LSTM 和 GRU 方法可以很好地学习依赖实体；另一方面，IndRNN 方法学习时间序列中的非依赖实体。所提出的方法尝试结合使用 IndRNN 和 LSTM 方法来学习相关和非相关特征。特征选择方法还为模型提供了合适的特征视图；为此，使用了四种特征选择模型：过滤器、包装器、嵌入式和自动编码器。所提出的IndRNNLSTM算法与Embedded相结合，能够在NSL-KDD数据上实现MAE=1.22和RMSE=9.92。

Title: Todyformer: Towards Holistic Dynamic Graph Transformers with Structure-Aware Tokenization

• Authors: Mahdi Biparva, Raika Karimi, Faezeh Faez, Yingxue Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05944
• Pdf URL: https://arxiv.org/pdf/2402.05944
• Copy Paste: [[2402.05944]] Todyformer: Towards Holistic Dynamic Graph Transformers with Structure-Aware Tokenization(https://arxiv.org/abs/2402.05944)
• Keywords: rag
• Abstract: Temporal Graph Neural Networks have garnered substantial attention for their capacity to model evolving structural and temporal patterns while exhibiting impressive performance. However, it is known that these architectures are encumbered by issues that constrain their performance, such as over-squashing and over-smoothing. Meanwhile, Transformers have demonstrated exceptional computational capacity to effectively address challenges related to long-range dependencies. Consequently, we introduce Todyformer-a novel Transformer-based neural network tailored for dynamic graphs. It unifies the local encoding capacity of Message-Passing Neural Networks (MPNNs) with the global encoding of Transformers through i) a novel patchifying paradigm for dynamic graphs to improve over-squashing, ii) a structure-aware parametric tokenization strategy leveraging MPNNs, iii) a Transformer with temporal positional-encoding to capture long-range dependencies, and iv) an encoding architecture that alternates between local and global contextualization, mitigating over-smoothing in MPNNs. Experimental evaluations on public benchmark datasets demonstrate that Todyformer consistently outperforms the state-of-the-art methods for downstream tasks. Furthermore, we illustrate the underlying aspects of the proposed model in effectively capturing extensive temporal dependencies in dynamic graphs.
• 摘要：时态图神经网络因其对不断演变的结构和时间模式进行建模的能力而受到广泛关注，同时表现出令人印象深刻的性能。然而，众所周知，这些架构受到限制其性能的问题的困扰，例如过度挤压和过度平滑。与此同时，Transformers 展示了卓越的计算能力，可以有效解决与远程依赖相关的挑战。因此，我们引入了 Todyformer——一种为动态图量身定制的新型基于 Transformer 的神经网络。它将消息传递神经网络 (MPNN) 的本地编码能力与 Transformer 的全局编码统一起来，通过 i) 一种新颖的动态图修补范例来改善过度挤压，ii) 利用 MPNN 的结构感知参数标记化策略，iii ) 具有时间位置编码的 Transformer，用于捕获远程依赖性，以及 iv) 在局部和全局上下文之间交替的编码架构，减轻 MPNN 中的过度平滑。对公共基准数据集的实验评估表明，Todyformer 在下游任务方面始终优于最先进的方法。此外，我们还说明了所提出的模型在有效捕获动态图中广泛的时间依赖性方面的基本方面。

Title: DE$^3$-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks

• Authors: Jianing He, Qi Zhang, Weiping Ding, Duoqian Miao, Jun Zhao, Liang Hu, Longbing Cao
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.05948
• Pdf URL: https://arxiv.org/pdf/2402.05948
• Copy Paste: [[2402.05948]] DE$^3$-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks(https://arxiv.org/abs/2402.05948)
• Keywords: language model, rag
• Abstract: Early exiting has demonstrated its effectiveness in accelerating the inference of pre-trained language models like BERT by dynamically adjusting the number of layers executed. However, most existing early exiting methods only consider local information from an individual test sample to determine their exiting indicators, failing to leverage the global information offered by sample population. This leads to suboptimal estimation of prediction correctness, resulting in erroneous exiting decisions. To bridge the gap, we explore the necessity of effectively combining both local and global information to ensure reliable early exiting during inference. Purposefully, we leverage prototypical networks to learn class prototypes and devise a distance metric between samples and class prototypes. This enables us to utilize global information for estimating the correctness of early predictions. On this basis, we propose a novel Distance-Enhanced Early Exiting framework for BERT (DE$^3$-BERT). DE$^3$-BERT implements a hybrid exiting strategy that supplements classic entropy-based local information with distance-based global information to enhance the estimation of prediction correctness for more reliable early exiting decisions. Extensive experiments on the GLUE benchmark demonstrate that DE$^3$-BERT consistently outperforms state-of-the-art models under different speed-up ratios with minimal storage or computational overhead, yielding a better trade-off between model performance and inference efficiency. Additionally, an in-depth analysis further validates the generality and interpretability of our method.
• 摘要：早期退出已经证明了其通过动态调整执行层数来加速 BERT 等预训练语言模型推理的有效性。然而，大多数现有的早期现有方法仅考虑单个测试样本的局部信息来确定其现有指标，未能利用样本群体提供的全局信息。这会导致对预测正确性的估计不理想，从而导致错误的现有决策。为了弥补这一差距，我们探讨了有效结合本地和全局信息的必要性，以确保推理过程中可靠的早期退出。我们有目的地利用原型网络来学习类原型，并设计样本和类原型之间的距离度量。这使我们能够利用全球信息来估计早期预测的正确性。在此基础上，我们提出了一种新颖的 BERT 距离增强早期退出框架（DE$^3$-BERT）。 DE$^3$-BERT 实现了一种混合退出策略，用基于距离的全局信息补充经典的基于熵的局部信息，以增强预测正确性的估计，从而获得更可靠的早期退出决策。 GLUE 基准上的大量实验表明，DE$^3$-BERT 在不同加速比下始终优于最先进的模型，且存储或计算开销最小，从而在模型性能和推理效率之间实现更好的权衡。此外，深入的分析进一步验证了我们方法的通用性和可解释性。

Title: \textit{SQT} -- \textit{std} $Q$-target

• Authors: Nitsan Soffair, Dotan Di-Castro, Orly Avner, Shie Mannor
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.05950
• Pdf URL: https://arxiv.org/pdf/2402.05950
• Copy Paste: [[2402.05950]] \textit{SQT} -- \textit{std} $Q$-target(https://arxiv.org/abs/2402.05950)
• Keywords: code
• Abstract: \textit{Std} $Q$-target is a \textit{conservative}, actor-critic, ensemble, $Q$-learning-based algorithm, which is based on a single key $Q$-formula: $Q$-networks standard deviation, which is an "uncertainty penalty", and, serves as a minimalistic solution to the problem of \textit{overestimation} bias. We implement \textit{SQT} on top of TD3/TD7 code and test it against the state-of-the-art (SOTA) actor-critic algorithms, DDPG, TD3 and TD7 on seven popular MuJoCo and Bullet tasks. Our results demonstrate \textit{SQT}'s $Q$-target formula superiority over \textit{TD3}'s $Q$-target formula as a \textit{conservative} solution to overestimation bias in RL, while \textit{SQT} shows a clear performance advantage on a wide margin over DDPG, TD3, and TD7 on all tasks.
• 摘要：\textit{Std} $Q$-target 是一个 \textit{conservative}、actor-critic、ensemble、基于 $Q$-learning 的算法，它基于单键 $Q$-formula：$Q$-网络标准差，这是一种“不确定性惩罚”，并且可以作为 \textit{overestimation} 偏差问题的简约解决方案。我们在 TD3/TD7 代码之上实现 \textit{SQT}，并在七个流行的 MuJoCo 和 Bullet 任务上针对最先进的 (SOTA) actor-critic 算法、DDPG、TD3 和 TD7 对其进行测试。我们的结果证明，作为 RL 中高估偏差的保守解决方案，\textit{SQT} 的 $Q$-target 公式优于 \textit{TD3} 的 $Q$-target 公式，而 \textit{SQT在所有任务上，与 DDPG、TD3 和 TD7 相比，表现出明显的性能优势。

Title: Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques

• Authors: Qiheng Mao, Zemin Liu, Chenghao Liu, Zhuo Li, Jianling Sun
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.05952
• Pdf URL: https://arxiv.org/pdf/2402.05952
• Copy Paste: [[2402.05952]] Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques(https://arxiv.org/abs/2402.05952)
• Keywords: language model, llm
• Abstract: The integration of Large Language Models (LLMs) with Graph Representation Learning (GRL) marks a significant evolution in analyzing complex data structures. This collaboration harnesses the sophisticated linguistic capabilities of LLMs to improve the contextual understanding and adaptability of graph models, thereby broadening the scope and potential of GRL. Despite a growing body of research dedicated to integrating LLMs into the graph domain, a comprehensive review that deeply analyzes the core components and operations within these models is notably lacking. Our survey fills this gap by proposing a novel taxonomy that breaks down these models into primary components and operation techniques from a novel technical perspective. We further dissect recent literature into two primary components including knowledge extractors and organizers, and two operation techniques including integration and training stratigies, shedding light on effective model design and training strategies. Additionally, we identify and explore potential future research avenues in this nascent yet underexplored field, proposing paths for continued progress.
• 摘要：大型语言模型 (LLM) 与图表示学习 (GRL) 的集成标志着复杂数据结构分析的重大发展。此次合作利用法学硕士复杂的语言能力来提高图模型的上下文理解和适应性，从而扩大 GRL 的范围和潜力。尽管越来越多的研究致力于将法学硕士整合到图领域，但深入分析这些模型中的核心组件和操作的全面审查却明显缺乏。我们的调查通过提出一种新颖的分类法来填补这一空白，该分类法从新颖的技术角度将这些模型分解为主要组件和操作技术。我们进一步将最近的文献剖析为两个主要组成部分，包括知识提取器和组织者，以及包括集成和培训策略在内的两种操作技术，阐明有效的模型设计和培训策略。此外，我们还确定并探索这个新兴但尚未充分探索的领域未来潜在的研究途径，并提出持续进展的路径。

Title: Phase-driven Domain Generalizable Learning for Nonstationary Time Series

• Authors: Payal Mohapatra, Lixu Wang, Qi Zhu
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2402.05960
• Pdf URL: https://arxiv.org/pdf/2402.05960
• Copy Paste: [[2402.05960]] Phase-driven Domain Generalizable Learning for Nonstationary Time Series(https://arxiv.org/abs/2402.05960)
• Keywords: rag
• Abstract: Monitoring and recognizing patterns in continuous sensing data is crucial for many practical applications. These real-world time-series data are often nonstationary, characterized by varying statistical and spectral properties over time. This poses a significant challenge in developing learning models that can effectively generalize across different distributions. In this work, based on our observation that nonstationary statistics are intrinsically linked to the phase information, we propose a time-series learning framework, PhASER. It consists of three novel elements: 1) phase augmentation that diversifies non-stationarity while preserving discriminatory semantics, 2) separate feature encoding by viewing time-varying magnitude and phase as independent modalities, and 3) feature broadcasting by incorporating phase with a novel residual connection for inherent regularization to enhance distribution invariant learning. Upon extensive evaluation on 5 datasets from human activity recognition, sleep-stage classification, and gesture recognition against 10 state-of-the-art baseline methods, we demonstrate that PhASER consistently outperforms the best baselines by an average of 5% and up to 13% in some cases. Moreover, PhASER's principles can be applied broadly to boost the generalization ability of existing time series classification models.
• 摘要：监测和识别连续传感数据的模式对于许多实际应用至关重要。这些现实世界的时间序列数据通常是非平稳的，其特征是统计和频谱特性随时间变化。这对开发能够有效泛化不同分布的学习模型提出了重大挑战。在这项工作中，根据我们对非平稳统计与相位信息本质上联系的观察，我们提出了一个时间序列学习框架 PhASER。它由三个新颖的元素组成：1）相位增强，使非平稳性多样化，同时保留歧视性语义；2）通过将时变幅度和相位视为独立模态来进行单独的特征编码；3）通过将相位与新颖的残差结合起来进行特征广播连接固有正则化以增强分布不变学习。根据 10 种最先进的基线方法对来自人类活动识别、睡眠阶段分类和手势识别的 5 个数据集进行了广泛评估，我们证明 PhASER 始终优于最佳基线，平均优于 5%，最高可达 13% % 在某些情况下。此外，PhASER 的原理可以广泛应用，以提高现有时间序列分类模型的泛化能力。

Title: EXGC: Bridging Efficiency and Explainability in Graph Condensation

• Authors: Junfeng Fang, Xinglin Li, Yongduo Sui, Yuan Gao, Guibin Zhang, Kun Wang, Xiang Wang, Xiangnan He
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05962
• Pdf URL: https://arxiv.org/pdf/2402.05962
• Copy Paste: [[2402.05962]] EXGC: Bridging Efficiency and Explainability in Graph Condensation(https://arxiv.org/abs/2402.05962)
• Keywords: code, rag
• Abstract: Graph representation learning on vast datasets, like web data, has made significant strides. However, the associated computational and storage overheads raise concerns. In sight of this, Graph condensation (GCond) has been introduced to distill these large real datasets into a more concise yet information-rich synthetic graph. Despite acceleration efforts, existing GCond methods mainly grapple with efficiency, especially on expansive web data graphs. Hence, in this work, we pinpoint two major inefficiencies of current paradigms: (1) the concurrent updating of a vast parameter set, and (2) pronounced parameter redundancy. To counteract these two limitations correspondingly, we first (1) employ the Mean-Field variational approximation for convergence acceleration, and then (2) propose the objective of Gradient Information Bottleneck (GDIB) to prune redundancy. By incorporating the leading explanation techniques (e.g., GNNExplainer and GSAT) to instantiate the GDIB, our EXGC, the Efficient and eXplainable Graph Condensation method is proposed, which can markedly boost efficiency and inject explainability. Our extensive evaluations across eight datasets underscore EXGC's superiority and relevance. Code is available at https://github.com/MangoKiller/EXGC.
• 摘要：针对网络数据等海量数据集的图表示学习已经取得了重大进展。然而，相关的计算和存储开销引起了人们的担忧。鉴于此，引入了图压缩（GCond）来将这些大型真实数据集提炼成更简洁但信息丰富的合成图。尽管做出了加速努力，现有的 GCond 方法主要还是在解决效率问题，尤其是在广泛的网络数据图上。因此，在这项工作中，我们指出了当前范例的两个主要低效之处：（1）大量参数集的并发更新，以及（2）明显的参数冗余。为了相应地抵消这两个限制，我们首先（1）采用平均场变分近似来加速收敛，然后（2）提出梯度信息瓶颈（GDIB）的目标来修剪冗余。通过结合领先的解释技术（例如 GNNExplainer 和 GSAT）来实例化 GDIB，我们的 EXGC，提出了高效且可解释的图压缩方法，该方法可以显着提高效率并注入可解释性。我们对八个数据集的广泛评估强调了 EXGC 的优越性和相关性。代码可在 https://github.com/MangoKiller/EXGC 获取。

Title: Frugal Actor-Critic: Sample Efficient Off-Policy Deep Reinforcement Learning Using Unique Experiences

• Authors: Nikhil Kumar Singh, Indranil Saha
• Subjects: cs.LG, cs.AI, cs.RO, eess.SY
• Abstract URL: https://arxiv.org/abs/2402.05963
• Pdf URL: https://arxiv.org/pdf/2402.05963
• Copy Paste: [[2402.05963]] Frugal Actor-Critic: Sample Efficient Off-Policy Deep Reinforcement Learning Using Unique Experiences(https://arxiv.org/abs/2402.05963)
• Keywords: lora
• Abstract: Efficient utilization of the replay buffer plays a significant role in the off-policy actor-critic reinforcement learning (RL) algorithms used for model-free control policy synthesis for complex dynamical systems. We propose a method for achieving sample efficiency, which focuses on selecting unique samples and adding them to the replay buffer during the exploration with the goal of reducing the buffer size and maintaining the independent and identically distributed (IID) nature of the samples. Our method is based on selecting an important subset of the set of state variables from the experiences encountered during the initial phase of random exploration, partitioning the state space into a set of abstract states based on the selected important state variables, and finally selecting the experiences with unique state-reward combination by using a kernel density estimator. We formally prove that the off-policy actor-critic algorithm incorporating the proposed method for unique experience accumulation converges faster than the vanilla off-policy actor-critic algorithm. Furthermore, we evaluate our method by comparing it with two state-of-the-art actor-critic RL algorithms on several continuous control benchmarks available in the Gym environment. Experimental results demonstrate that our method achieves a significant reduction in the size of the replay buffer for all the benchmarks while achieving either faster convergent or better reward accumulation compared to the baseline algorithms.
• 摘要：重放缓冲区的有效利用在用于复杂动态系统的无模型控制策略合成的离策略行为者批评家强化学习 (RL) 算法中发挥着重要作用。我们提出了一种实现样本效率的方法，该方法的重点是在探索过程中选择独特的样本并将它们添加到重播缓冲区中，目的是减少缓冲区大小并保持样本的独立同分布（IID）性质。我们的方法基于从随机探索初始阶段遇到的经验中选择状态变量集的重要子集，根据所选的重要状态变量将状态空间划分为一组抽象状态，最后选择经验通过使用核密度估计器具有独特的状态奖励组合。我们正式证明，结合所提出的独特经验积累方法的离策略演员批评家算法比普通的离政策演员批评家算法收敛得更快。此外，我们通过在 Gym 环境中可用的几个连续控制基准上将我们的方法与两种最先进的 actor-critic RL 算法进行比较来评估我们的方法。实验结果表明，与基线算法相比，我们的方法显着减少了所有基准的重放缓冲区的大小，同时实现了更快的收敛或更好的奖励累积。

Title: Rethink Model Re-Basin and the Linear Mode Connectivity

• Authors: Xingyu Qu, Samuel Horvath
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.05966
• Pdf URL: https://arxiv.org/pdf/2402.05966
• Copy Paste: [[2402.05966]] Rethink Model Re-Basin and the Linear Mode Connectivity(https://arxiv.org/abs/2402.05966)
• Keywords: rag
• Abstract: Recent studies suggest that with sufficiently wide models, most SGD solutions can, up to permutation, converge into the same basin. This phenomenon, known as the model re-basin regime, has significant implications for model averaging. However, current re-basin strategies are limited in effectiveness due to a lack of comprehensive understanding of underlying mechanisms. Addressing this gap, our work revisits standard practices and uncovers the frequent inadequacies of existing matching algorithms, which we show can be mitigated through proper re-normalization. By introducing a more direct analytical approach, we expose the interaction between matching algorithms and re-normalization processes. This perspective not only clarifies and refines previous findings but also facilitates novel insights. For instance, it connects the linear mode connectivity to pruning, motivating a lightweight yet effective post-pruning plug-in that can be directly merged with any existing pruning techniques. Our implementation is available at https://github.com/XingyuQu/rethink-re-basin.
• 摘要：最近的研究表明，只要模型足够宽，大多数 SGD 解决方案在排列之前都可以收敛到同一个盆地。这种现象被称为模型重新流域机制，对模型平均具有重要意义。然而，由于缺乏对潜在机制的全面了解，当前的流域重建策略的有效性有限。为了解决这一差距，我们的工作重新审视了标准实践，并揭示了现有匹配算法经常存在的缺陷，我们表明可以通过适当的重新标准化来缓解这些缺陷。通过引入更直接的分析方法，我们揭示了匹配算法和重归一化过程之间的相互作用。这种观点不仅澄清和完善了以前的发现，而且还促进了新的见解。例如，它将线性模式连接与修剪连接起来，激发了轻量级但有效的后修剪插件，可以直接与任何现有的修剪技术合并。我们的实现可在 https://github.com/XingyuQu/rethink-re-basin 获取。

Title: Breaking Symmetry When Training Transformers

• Authors: Chunsheng Zuo, Michael Guerzhoy
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.05969
• Pdf URL: https://arxiv.org/pdf/2402.05969
• Copy Paste: [[2402.05969]] Breaking Symmetry When Training Transformers(https://arxiv.org/abs/2402.05969)
• Keywords: rag
• Abstract: As we show in this paper, the prediction for output token $n+1$ of Transformer architectures without one of the mechanisms of positional encodings and causal attention is invariant to permutations of input tokens $1, 2, ..., n-1$. Usually, both mechanisms are employed and the symmetry with respect to the input tokens is broken. Recently, it has been shown that one can train Transformers without positional encodings. This must be enabled by the causal attention mechanism. In this paper, we elaborate on the argument that the causal connection mechanism must be responsible for the fact that Transformers are able to model input sequences where the order is important. Vertical "slices" of Transformers are all encouraged to represent the same location $k$ in the input sequence. We hypothesize that residual connections contribute to this phenomenon, and demonstrate evidence for this.
• 摘要：正如我们在本文中所示，没有位置编码和因果注意机制之一的 Transformer 架构的输出标记 $n+1$ 的预测对于输入标记 $1, 2, ..., n-1$ 的排列是不变的。通常，两种机制都会被采用，并且输入令牌的对称性会被打破。最近，研究表明无需位置编码即可训练 Transformer。这必须通过因果注意机制来实现。在本文中，我们详细阐述了因果连接机制必须对 Transformer 能够对顺序很重要的输入序列进行建模这一事实负责的论点。鼓励 Transformer 的垂直“切片”代表输入序列中的相同位置 $k$。我们假设残余连接导致了这种现象，并证明了这一点的证据。

Title: Exploring the Impact of In-Browser Deep Learning Inference on Quality of User Experience and Performance

• Authors: Qipeng Wang, Shiqi Jiang, Zhenpeng Chen, Xu Cao, Yuanchun Li, Aoyu Li, Ying Zhang, Yun Ma, Ting Cao, Xuanzhe Liu
• Subjects: cs.LG, cs.PF
• Abstract URL: https://arxiv.org/abs/2402.05981
• Pdf URL: https://arxiv.org/pdf/2402.05981
• Copy Paste: [[2402.05981]] Exploring the Impact of In-Browser Deep Learning Inference on Quality of User Experience and Performance(https://arxiv.org/abs/2402.05981)
• Keywords: rag
• Abstract: Deep Learning (DL) is increasingly being integrated into Web applications through a method known as "in-browser inference", where the DL processes occur directly within Web browsers. However, the actual performance of this method and its effect on user experience quality (QoE) is not well-understood. This gap in knowledge necessitates new forms of QoE measurement, going beyond traditional metrics such as page load time. To address this, we conducted the first extensive performance evaluation of in-browser inference. We introduced new metrics for this purpose: responsiveness, smoothness, and inference accuracy. Our thorough study included 9 widely-used DL models and tested them across 50 popular PC Web browsers. The findings show a significant latency issue with in-browser inference: it's on average 16.9 times slower on CPU and 4.9 times slower on GPU than native inference methods. Several factors contribute to this latency, including underused hardware instruction sets, inherent delays in the runtime environment, resource competition within the browser, and inefficiencies in software libraries and GPU abstractions. Moreover, in-browser inference demands a lot of memory, sometimes up to 334.6 times more than the size of the DL models themselves. This excessive memory usage is partly due to suboptimal memory management. Additionally, we noticed that in-browser inference increases the time it takes for graphical user interface (GUI) components to load in web browsers by a significant 67.2\%, which severely impacts the overall QoE for users of web applications that depend on this technology.
• 摘要：深度学习 (DL) 越来越多地通过一种称为“浏览器内推理”的方法集成到 Web 应用程序中，其中 DL 过程直接发生在 Web 浏览器中。然而，这种方法的实际性能及其对用户体验质量（QoE）的影响尚不清楚。这种知识差距需要新形式的 QoE 测量，超越页面加载时间等传统指标。为了解决这个问题，我们对浏览器内推理进行了首次广泛的性能评估。为此，我们引入了新的指标：响应度、平滑度和推理准确性。我们的全面研究包括 9 个广泛使用的 DL 模型，并在 50 种流行的 PC Web 浏览器中测试了它们。研究结果表明，浏览器内推理存在显着的延迟问题：与本机推理方法相比，在 CPU 上平均慢 16.9 倍，在 GPU 上慢 4.9 倍。导致这种延迟的因素有很多，包括未充分利用的硬件指令集、运行时环境中的固有延迟、浏览器内的资源竞争以及软件库和 GPU 抽象的低效率。此外，浏览器内推理需要大量内存，有时高达 DL 模型本身大小的 334.6 倍。内存使用过多的部分原因是内存管理不理想。此外，我们注意到浏览器内推理将图形用户界面 (GUI) 组件在 Web 浏览器中加载所需的时间增加了 67.2%，这严重影响了依赖该技术的 Web 应用程序用户的整体 QoE 。

Title: Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing

• Authors: Yong Cao, Wenyan Li, Jiaang Li, Yifei Yuan, Daniel Hershcovich
• Subjects: cs.CL, cs.CV
• Abstract URL: https://arxiv.org/abs/2402.06015
• Pdf URL: https://arxiv.org/pdf/2402.06015
• Copy Paste: [[2402.06015]] Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing(https://arxiv.org/abs/2402.06015)
• Keywords: language model, gpt
• Abstract: Pretrained large Vision-Language models have drawn considerable interest in recent years due to their remarkable performance. Despite considerable efforts to assess these models from diverse perspectives, the extent of visual cultural awareness in the state-of-the-art GPT-4V model remains unexplored. To tackle this gap, we extensively probed GPT-4V using the MaRVL benchmark dataset, aiming to investigate its capabilities and limitations in visual understanding with a focus on cultural aspects. Specifically, we introduced three visual related tasks, i.e. caption classification, pairwise captioning, and culture tag selection, to systematically delve into fine-grained visual cultural evaluation. Experimental results indicate that GPT-4V excels at identifying cultural concepts but still exhibits weaker performance in low-resource languages, such as Tamil and Swahili. Notably, through human evaluation, GPT-4V proves to be more culturally relevant in image captioning tasks than the original MaRVL human annotations, suggesting a promising solution for future visual cultural benchmark construction.
• 摘要：近年来，预训练的大型视觉语言模型因其卓越的性能而引起了人们的极大兴趣。尽管从不同角度评估这些模型付出了相当大的努力，但最先进的 GPT-4V 模型中视觉文化意识的程度仍未得到探索。为了解决这一差距，我们使用 MaRVL 基准数据集广泛探讨了 GPT-4V，旨在研究其在视觉理解方面的能力和局限性，重点关注文化方面。具体来说，我们引入了三个视觉相关任务，即字幕分类、成对字幕和文化标签选择，以系统地深入研究细粒度的视觉文化评估。实验结果表明，GPT-4V 在识别文化概念方面表现出色，但在泰米尔语和斯瓦希里语等低资源语言中表现仍然较弱。值得注意的是，通过人类评估，GPT-4V 被证明在图像字幕任务中比原始 MaRVL 人类注释更具文化相关性，这为未来视觉文化基准构建提供了一个有前途的解决方案。

Title: Checking the Sufficiently Scattered Condition using a Global Non-Convex Optimization Software

• Authors: Nicolas Gillis, Robert Luce
• Subjects: cs.LG, eess.SP, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2402.06019
• Pdf URL: https://arxiv.org/pdf/2402.06019
• Copy Paste: [[2402.06019]] Checking the Sufficiently Scattered Condition using a Global Non-Convex Optimization Software(https://arxiv.org/abs/2402.06019)
• Keywords: code
• Abstract: The sufficiently scattered condition (SSC) is a key condition in the study of identifiability of various matrix factorization problems, including nonnegative, minimum-volume, symmetric, simplex-structured, and polytopic matrix factorizations. The SSC allows one to guarantee that the computed matrix factorization is unique/identifiable, up to trivial ambiguities. However, this condition is NP-hard to check in general. In this paper, we show that it can however be checked in a reasonable amount of time in realistic scenarios, when the factorization rank is not too large. This is achieved by formulating the problem as a non-convex quadratic optimization problem over a bounded set. We use the global non-convex optimization software Gurobi, and showcase the usefulness of this code on synthetic data sets and on real-world hyperspectral images.
• 摘要：充分分散条件（SSC）是研究各种矩阵分解问题可辨识性的关键条件，包括非负、最小体积、对称、单纯形结构和多面矩阵分解问题。 SSC 可以保证计算出的矩阵分解是唯一的/可识别的，甚至可以忽略不计的歧义。然而，这种情况一般来说是 NP 难检查的。在本文中，我们表明，当分解秩不太大时，可以在现实场景中在合理的时间内对其进行检查。这是通过将问题表述为有界集上的非凸二次优化问题来实现的。我们使用全局非凸优化软件 Gurobi，并展示了该代码在合成数据集和真实世界高光谱图像上的有用性。

Title: Decision Theory-Guided Deep Reinforcement Learning for Fast Learning

• Authors: Zelin Wan, Jin-Hee Cho, Mu Zhu, Ahmed H. Anwar, Charles Kamhoua, Munindar P. Singh
• Subjects: cs.LG, cs.AI, cs.GT
• Abstract URL: https://arxiv.org/abs/2402.06023
• Pdf URL: https://arxiv.org/pdf/2402.06023
• Copy Paste: [[2402.06023]] Decision Theory-Guided Deep Reinforcement Learning for Fast Learning(https://arxiv.org/abs/2402.06023)
• Keywords: lora, rag, agent
• Abstract: This paper introduces a novel approach, Decision Theory-guided Deep Reinforcement Learning (DT-guided DRL), to address the inherent cold start problem in DRL. By integrating decision theory principles, DT-guided DRL enhances agents' initial performance and robustness in complex environments, enabling more efficient and reliable convergence during learning. Our investigation encompasses two primary problem contexts: the cart pole and maze navigation challenges. Experimental results demonstrate that the integration of decision theory not only facilitates effective initial guidance for DRL agents but also promotes a more structured and informed exploration strategy, particularly in environments characterized by large and intricate state spaces. The results of experiment demonstrate that DT-guided DRL can provide significantly higher rewards compared to regular DRL. Specifically, during the initial phase of training, the DT-guided DRL yields up to an 184% increase in accumulated reward. Moreover, even after reaching convergence, it maintains a superior performance, ending with up to 53% more reward than standard DRL in large maze problems. DT-guided DRL represents an advancement in mitigating a fundamental challenge of DRL by leveraging functions informed by human (designer) knowledge, setting a foundation for further research in this promising interdisciplinary domain.
• 摘要：本文介绍了一种新方法——决策理论引导的深度强化学习（DT-guided DRL），来解决 DRL 中固有的冷启动问题。通过集成决策理论原理，DT 引导的 DRL 增强了智能体在复杂环境中的初始性能和鲁棒性，从而在学习过程中实现更高效、更可靠的收敛。我们的调查涵盖两个主要问题背景：车杆和迷宫导航挑战。实验结果表明，决策理论的集成不仅有助于对 DRL 智能体进行有效的初始指导，而且还促进了更加结构化和知情的探索策略，特别是在以大而复杂的状态空间为特征的环境中。实验结果表明，与常规 DRL 相比，DT 引导的 DRL 可以提供显着更高的奖励。具体来说，在训练的初始阶段，DT 引导的 DRL 使累积奖励增加了 184%。此外，即使在达到收敛后，它仍保持卓越的性能，在大型迷宫问题中，其奖励比标准 DRL 高出 53%。 DT 引导的 DRL 代表了通过利用人类（设计者）知识提供的功能来缓解 DRL 基本挑战的进步，为这个有前景的跨学科领域的进一步研究奠定了基础。

Title: Optimizing Predictive AI in Physical Design Flows with Mini Pixel Batch Gradient Descent

• Authors: Haoyu Yang, Anthony Agnesina, Haoxing Ren
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06034
• Pdf URL: https://arxiv.org/pdf/2402.06034
• Copy Paste: [[2402.06034]] Optimizing Predictive AI in Physical Design Flows with Mini Pixel Batch Gradient Descent(https://arxiv.org/abs/2402.06034)
• Keywords: rag
• Abstract: Exploding predictive AI has enabled fast yet effective evaluation and decision-making in modern chip physical design flows. State-of-the-art frameworks typically include the objective of minimizing the mean square error (MSE) between the prediction and the ground truth. We argue the averaging effect of MSE induces limitations in both model training and deployment, and good MSE behavior does not guarantee the capability of these models to assist physical design flows which are likely sabotaged due to a small portion of prediction error. To address this, we propose mini-pixel batch gradient descent (MPGD), a plug-and-play optimization algorithm that takes the most informative entries into consideration, offering probably faster and better convergence. Experiments on representative benchmark suits show the significant benefits of MPGD on various physical design prediction tasks using CNN or Graph-based models.
• 摘要：爆炸式的预测人工智能使得现代芯片物理设计流程能够快速而有效地进行评估和决策。最先进的框架通常包括最小化预测与真实情况之间的均方误差（MSE）的目标。我们认为 MSE 的平均效应会导致模型训练和部署方面的限制，并且良好的 MSE 行为并不能保证这些模型能够协助物理设计流程，而物理设计流程可能会由于一小部分预测误差而被破坏。为了解决这个问题，我们提出了迷你像素批量梯度下降（MPGD），这是一种即插即用的优化算法，它考虑了信息最丰富的条目，可能提供更快更好的收敛。对代表性基准套件的实验表明，MPGD 在使用 CNN 或基于图的模型的各种物理设计预测任务中具有显着优势。

Title: Contrastive Approach to Prior Free Positive Unlabeled Learning

• Authors: Anish Acharya, Sujay Sanghavi
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2402.06038
• Pdf URL: https://arxiv.org/pdf/2402.06038
• Copy Paste: [[2402.06038]] Contrastive Approach to Prior Free Positive Unlabeled Learning(https://arxiv.org/abs/2402.06038)
• Keywords: rag
• Abstract: Positive Unlabeled (PU) learning refers to the task of learning a binary classifier given a few labeled positive samples, and a set of unlabeled samples (which could be positive or negative). In this paper, we propose a novel PU learning framework, that starts by learning a feature space through pretext-invariant representation learning and then applies pseudo-labeling to the unlabeled examples, leveraging the concentration property of the embeddings. Overall, our proposed approach handily outperforms state-of-the-art PU learning methods across several standard PU benchmark datasets, while not requiring a-priori knowledge or estimate of class prior. Remarkably, our method remains effective even when labeled data is scant, where most PU learning algorithms falter. We also provide simple theoretical analysis motivating our proposed algorithms and establish generalization guarantee for our approach.
• 摘要：正无标记（PU）学习是指在给定一些标记的正样本和一组未标记样本（可以是正样本或负样本）的情况下学习二元分类器的任务。在本文中，我们提出了一种新颖的 PU 学习框架，首先通过借口不变表示学习来学习特征空间，然后利用嵌入的集中特性将伪标记应用于未标记的示例。总体而言，我们提出的方法在多个标准 PU 基准数据集上轻松优于最先进的 PU 学习方法，同时不需要先验知识或类先验估计。值得注意的是，即使标记数据很少（大多数 PU 学习算法都会出现问题），我们的方法仍然有效。我们还提供简单的理论分析来激发我们提出的算法，并为我们的方法建立泛化保证。

Title: A Prompt Response to the Demand for Automatic Gender-Neutral Translation

• Authors: Beatrice Savoldi, Andrea Piergentili, Dennis Fucci, Matteo Negri, Luisa Bentivogli
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06041
• Pdf URL: https://arxiv.org/pdf/2402.06041
• Copy Paste: [[2402.06041]] A Prompt Response to the Demand for Automatic Gender-Neutral Translation(https://arxiv.org/abs/2402.06041)
• Keywords: language model, gpt, prompt
• Abstract: Gender-neutral translation (GNT) that avoids biased and undue binary assumptions is a pivotal challenge for the creation of more inclusive translation technologies. Advancements for this task in Machine Translation (MT), however, are hindered by the lack of dedicated parallel data, which are necessary to adapt MT systems to satisfy neutral constraints. For such a scenario, large language models offer hitherto unforeseen possibilities, as they come with the distinct advantage of being versatile in various (sub)tasks when provided with explicit instructions. In this paper, we explore this potential to automate GNT by comparing MT with the popular GPT-4 model. Through extensive manual analyses, our study empirically reveals the inherent limitations of current MT systems in generating GNTs and provides valuable insights into the potential and challenges associated with prompting for neutrality.
• 摘要：避免偏见和不适当的二元假设的性别中立翻译（GNT）是创建更具包容性的翻译技术的关键挑战。然而，机器翻译 (MT) 中这项任务的进展因缺乏专用并行数据而受到阻碍，而专用并行数据是调整 MT 系统以满足中性约束所必需的。对于这种情况，大型语言模型提供了迄今为止不可预见的可能性，因为它们具有在提供明确指令时在各种（子）任务中具有通用性的独特优势。在本文中，我们通过将 MT 与流行的 GPT-4 模型进行比较，探索自动化 GNT 的潜力。通过广泛的手动分析，我们的研究凭经验揭示了当前 MT 系统在生成 GNT 方面的固有局限性，并为促进中立性相关的潜力和挑战提供了宝贵的见解。

Title: OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models

• Authors: Hainiu Xu, Runcong Zhao, Lixing Zhu, Jinhua Du, Yulan He
• Subjects: cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06044
• Pdf URL: https://arxiv.org/pdf/2402.06044
• Copy Paste: [[2402.06044]] OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models(https://arxiv.org/abs/2402.06044)
• Keywords: language model, llm, agent
• Abstract: Neural Theory-of-Mind (N-ToM), machine's ability to understand and keep track of the mental states of others, is pivotal in developing socially intelligent agents. However, prevalent N-ToM benchmarks have several shortcomings, including the presence of ambiguous and artificial narratives, absence of personality traits and preferences, a lack of questions addressing characters' psychological mental states, and limited diversity in the questions posed. In response to these issues, we construct OpenToM, a new benchmark for assessing N-ToM with (1) longer and clearer narrative stories, (2) characters with explicit personality traits, (3) actions that are triggered by character intentions, and (4) questions designed to challenge LLMs' capabilities of modeling characters' mental states of both the physical and psychological world. Using OpenToM, we reveal that state-of-the-art LLMs thrive at modeling certain aspects of mental states in the physical world but fall short when tracking characters' mental states in the psychological world.
• 摘要：神经心理理论 (N-ToM) 是机器理解和跟踪他人心理状态的能力，对于开发社交智能体至关重要。然而，流行的 N-ToM 基准有几个缺点，包括存在模糊和人为的叙述、缺乏人格特征和偏好、缺乏针对人物心理状态的问题以及所提出问题的多样性有限。针对这些问题，我们构建了 OpenToM，一个评估 N-ToM 的新基准，具有（1）更长且更清晰的叙事故事，（2）具有明确个性特征的角色，（3）由角色意图触发的动作，以及（ 4) 旨在挑战法学硕士模拟人物生理和心理世界心理状态的能力的问题。使用 OpenToM，我们发现最先进的法学硕士在模拟物理世界中心理状态的某些方面方面表现出色，但在跟踪心理世界中角色的心理状态时却表现不佳。

Title: Limits of Large Language Models in Debating Humans

• Authors: James Flamino, Mohammed Shahid Modi, Boleslaw K. Szymanski, Brendan Cross, Colton Mikolajczyk
• Subjects: cs.AI, cs.CL, cs.HC, stat.AP
• Abstract URL: https://arxiv.org/abs/2402.06049
• Pdf URL: https://arxiv.org/pdf/2402.06049
• Copy Paste: [[2402.06049]] Limits of Large Language Models in Debating Humans(https://arxiv.org/abs/2402.06049)
• Keywords: language model, llm, agent
• Abstract: Large Language Models (LLMs) have shown remarkable promise in their ability to interact proficiently with humans. Subsequently, their potential use as artificial confederates and surrogates in sociological experiments involving conversation is an exciting prospect. But how viable is this idea? This paper endeavors to test the limits of current-day LLMs with a pre-registered study integrating real people with LLM agents acting as people. The study focuses on debate-based opinion consensus formation in three environments: humans only, agents and humans, and agents only. Our goal is to understand how LLM agents influence humans, and how capable they are in debating like humans. We find that LLMs can blend in and facilitate human productivity but are less convincing in debate, with their behavior ultimately deviating from human's. We elucidate these primary failings and anticipate that LLMs must evolve further before being viable debaters.
• 摘要：大型语言模型 (LLM) 在与人类熟练交互的能力方面表现出了非凡的前景。随后，它们在涉及对话的社会学实验中作为人工同盟者和代理人的潜在用途是一个令人兴奋的前景。但这个想法的可行性如何？本文试图通过一项预先注册的研究来测试当前法学硕士的局限性，该研究将真实的人和充当人的法学硕士代理人结合起来。该研究重点关注三种环境中基于辩论的意见共识形成：仅人类、代理人与人类、以及仅代理人。我们的目标是了解 LLM 代理人如何影响人类，以及他们像人类一样进行辩论的能力如何。我们发现法学硕士可以融入并促进人类生产力，但在争论中缺乏说服力，他们的行为最终偏离了人类的行为。我们阐明了这些主要缺陷，并预计法学硕士必须进一步发展才能成为可行的辩论者。

Title: ActiveDP: Bridging Active Learning and Data Programming

• Authors: Naiqing Guan, Nick Koudas
• Subjects: cs.LG, cs.DB
• Abstract URL: https://arxiv.org/abs/2402.06056
• Pdf URL: https://arxiv.org/pdf/2402.06056
• Copy Paste: [[2402.06056]] ActiveDP: Bridging Active Learning and Data Programming(https://arxiv.org/abs/2402.06056)
• Keywords: rag
• Abstract: Modern machine learning models require large labelled datasets to achieve good performance, but manually labelling large datasets is expensive and time-consuming. The data programming paradigm enables users to label large datasets efficiently but produces noisy labels, which deteriorates the downstream model's performance. The active learning paradigm, on the other hand, can acquire accurate labels but only for a small fraction of instances. In this paper, we propose ActiveDP, an interactive framework bridging active learning and data programming together to generate labels with both high accuracy and coverage, combining the strengths of both paradigms. Experiments show that ActiveDP outperforms previous weak supervision and active learning approaches and consistently performs well under different labelling budgets.
• 摘要：现代机器学习模型需要大型标记数据集才能获得良好的性能，但手动标记大型数据集既昂贵又耗时。数据编程范例使用户能够有效地标记大型数据集，但会产生嘈杂的标签，从而降低下游模型的性能。另一方面，主动学习范式可以获取准确的标签，但仅限于一小部分实例。在本文中，我们提出了 ActiveDP，一种将主动学习和数据编程结合在一起的交互式框架，以生成具有高精度和覆盖率的标签，结合了两种范式的优点。实验表明，ActiveDP 优于之前的弱监督和主动学习方法，并且在不同的标签预算下始终表现良好。

Title: Scaling Artificial Intelligence for Digital Wargaming in Support of Decision-Making

• Authors: Scotty Black, Christian Darken
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06075
• Pdf URL: https://arxiv.org/pdf/2402.06075
• Copy Paste: [[2402.06075]] Scaling Artificial Intelligence for Digital Wargaming in Support of Decision-Making(https://arxiv.org/abs/2402.06075)
• Keywords: agent
• Abstract: In this unprecedented era of technology-driven transformation, it becomes more critical than ever that we aggressively invest in developing robust artificial intelligence (AI) for wargaming in support of decision-making. By advancing AI-enabled systems and pairing these with human judgment, we will be able to enhance all-domain awareness, improve the speed and quality of our decision cycles, offer recommendations for novel courses of action, and more rapidly counter our adversary's actions. It therefore becomes imperative that we accelerate the development of AI to help us better address the complexity of modern challenges and dilemmas that currently requires human intelligence and, if possible, attempt to surpass human intelligence--not to replace humans, but to augment and better inform human decision-making at machine speed. Although deep reinforcement learning continues to show promising results in intelligent agent behavior development for the long-horizon, complex tasks typically found in combat modeling and simulation, further research is needed to enable the scaling of AI to deal with these intricate and expansive state-spaces characteristic of wargaming for either concept development, education, or analysis. To help address this challenge, in our research, we are developing and implementing a hierarchical reinforcement learning framework that includes a multi-model approach and dimension-invariant observation abstractions.
• 摘要：在这个前所未有的技术驱动转型时代，我们比以往任何时候都更需要积极投资开发强大的人工智能（AI），用于兵棋推演以支持决策。通过推进人工智能系统并将其与人类判断相结合，我们将能够增强全领域意识，提高决策周期的速度和质量，为新颖的行动方案提供建议，并更快地反击对手的行动。因此，我们必须加快人工智能的发展，以帮助我们更好地解决当前需要人类智能的现代挑战和困境的复杂性，并在可能的情况下尝试超越人类智能——不是取代人类，而是增强和更好地解决人类智能问题。以机器速度告知人类决策。尽管深度强化学习在战斗建模和模拟中常见的长期、复杂任务的智能代理行为开发方面继续显示出有希望的结果，但仍需要进一步的研究来扩展人工智能以处理这些复杂而广阔的状态空间用于概念开发、教育或分析的兵棋推演的特征。为了帮助应对这一挑战，在我们的研究中，我们正在开发和实施一个分层强化学习框架，其中包括多模型方法和维度不变的观察抽象。

Title: SubGen: Token Generation in Sublinear Time and Memory

• Authors: Amir Zandieh, Insu Han, Vahab Mirrokni, Amin Karbasi
• Subjects: cs.LG, cs.AI, cs.DS
• Abstract URL: https://arxiv.org/abs/2402.06082
• Pdf URL: https://arxiv.org/pdf/2402.06082
• Copy Paste: [[2402.06082]] SubGen: Token Generation in Sublinear Time and Memory(https://arxiv.org/abs/2402.06082)
• Keywords: language model, llm, code
• Abstract: Despite the significant success of large language models (LLMs), their extensive memory requirements pose challenges for deploying them in long-context token generation. The substantial memory footprint of LLM decoders arises from the necessity to store all previous tokens in the attention module, a requirement imposed by key-value (KV) caching. In this work, our focus is on developing an efficient compression technique for the KV cache. Empirical evidence indicates a significant clustering tendency within key embeddings in the attention module. Building on this key insight, we have devised a novel caching method with sublinear complexity, employing online clustering on key tokens and online $\ell_2$ sampling on values. The result is a provably accurate and efficient attention decoding algorithm, termed SubGen. Not only does this algorithm ensure a sublinear memory footprint and sublinear time complexity, but we also establish a tight error bound for our approach. Empirical evaluations on long-context question-answering tasks demonstrate that SubGen significantly outperforms existing and state-of-the-art KV cache compression methods in terms of performance and efficiency.
• 摘要：尽管大型语言模型 (LLM) 取得了巨大成功，但其大量的内存需求给在长上下文令牌生成中部署它们带来了挑战。 LLM 解码器的大量内存占用是由于需要将所有先前的标记存储在注意力模块中，这是键值 (KV) 缓存提出的要求。在这项工作中，我们的重点是为 KV 缓存开发一种高效的压缩技术。经验证据表明注意力模块中的关键嵌入存在显着的聚类趋势。基于这一关键见解，我们设计了一种具有亚线性复杂度的新颖缓存方法，对关键标记采用在线聚类并对值进行在线 $\ell_2$ 采样。结果是一种可证明准确且高效的注意力解码算法，称为 SubGen。该算法不仅确保了次线性内存占用和次线性时间复杂度，而且我们还为我们的方法建立了严格的误差界限。对长上下文问答任务的实证评估表明，SubGen 在性能和效率方面显着优于现有的和最先进的 KV 缓存压缩方法。

Title: Rethinking Data Selection for Supervised Fine-Tuning

• Authors: Ming Shen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06094
• Pdf URL: https://arxiv.org/pdf/2402.06094
• Copy Paste: [[2402.06094]] Rethinking Data Selection for Supervised Fine-Tuning(https://arxiv.org/abs/2402.06094)
• Keywords: language model
• Abstract: Although supervised finetuning (SFT) has emerged as an essential technique to align large language models with humans, it is considered superficial, with style learning being its nature. At the same time, recent works indicate the importance of data selection for SFT, showing that finetuning with high-quality and diverse subsets of the original dataset leads to superior downstream performance. In this work, we rethink the intuition behind data selection for SFT. Considering SFT is superficial, we propose that essential demonstrations for SFT should focus on reflecting human-like interactions instead of data quality or diversity. However, it is not straightforward to directly assess to what extent a demonstration reflects human styles. Towards an initial attempt in this direction, we find selecting instances with long responses is surprisingly more effective for SFT than utilizing full datasets or instances selected based on quality and diversity. We hypothesize that such a simple heuristic implicitly mimics a crucial aspect of human-style conversation: detailed responses are usually more helpful.
• 摘要：尽管监督微调（SFT）已成为使大型语言模型与人类保持一致的基本技术，但它被认为是肤浅的，风格学习才是其本质。与此同时，最近的工作表明了 SFT 数据选择的重要性，表明对原始数据集的高质量和多样化子集进行微调可以带来卓越的下游性能。在这项工作中，我们重新思考 SFT 数据选择背后的直觉。考虑到 SFT 很肤浅，我们建议 SFT 的基本演示应侧重于反映类人交互，而不是数据质量或多样性。然而，直接评估示威在多大程度上反映了人类风格并不容易。在这个方向上的初步尝试中，我们发现选择具有长响应的实例对于 SFT 来说比利用完整数据集或基于质量和多样性选择的实例更有效。我们假设这种简单的启发式方法隐含地模仿了人类对话的一个关键方面：详细的回答通常更有帮助。

Title: Function Aligned Regression: A Method Explicitly Learns Functional Derivatives from Data

• Authors: Dixian Zhu, Livnat Jerby-Arnon
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06104
• Pdf URL: https://arxiv.org/pdf/2402.06104
• Copy Paste: [[2402.06104]] Function Aligned Regression: A Method Explicitly Learns Functional Derivatives from Data(https://arxiv.org/abs/2402.06104)
• Keywords: code
• Abstract: Regression is a fundamental task in machine learning that has garnered extensive attention over the past decades. The conventional approach for regression involves employing loss functions that primarily concentrate on aligning model prediction with the ground truth for each individual data sample, which, as we show, can result in sub-optimal prediction of the relationships between the different samples. Recent research endeavors have introduced novel perspectives by incorporating label similarity information to regression. However, a notable gap persists in these approaches when it comes to fully capturing the intricacies of the underlying ground truth function. In this work, we propose FAR (Function Aligned Regression) as a arguably better and more efficient solution to fit the underlying function of ground truth by capturing functional derivatives. We demonstrate the effectiveness of the proposed method practically on 2 synthetic datasets and on 8 extensive real-world tasks from 6 benchmark datasets with other 8 competitive baselines. The code is open-sourced at \url{https://github.com/DixianZhu/FAR}.
• 摘要：回归是机器学习中的一项基本任务，在过去几十年中引起了广泛的关注。传统的回归方法涉及使用损失函数，该函数主要集中于将模型预测与每个单独数据样本的基本事实相匹配，正如我们所展示的，这可能会导致不同样本之间关系的次优预测。最近的研究工作通过将标签相似性信息纳入回归引入了新颖的观点。然而，在充分捕获底层真实函数的复杂性时，这些方法仍然存在显着的差距。在这项工作中，我们提出 FAR（函数对齐回归）作为一种可以说是更好、更有效的解决方案，通过捕获函数导数来拟合地面实况的底层函数。我们在 2 个合成数据集和来自 6 个基准数据集以及其他 8 个竞争基线的 8 个广泛的现实世界任务上实际证明了所提出的方法的有效性。该代码在 \url{https://github.com/DixianZhu/FAR} 上开源。

Title: AI enhanced data assimilation and uncertainty quantification applied to Geological Carbon Storage

• Authors: G. S. Seabra (1, 2), N. T. Mücke (3, 4), V. L. S. Silva (2, 5), D. Voskov (1, 6), F. Vossepoel (1) ((1) TU Delft, Netherlands, (2) Petrobras, Brazil, (3) Centrum Wiskunde & Informatica, Netherlands, (4) Utrecht University, Netherlands, (5) Imperial College London, United Kingdom, (6) Stanford University, USA)
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06110
• Pdf URL: https://arxiv.org/pdf/2402.06110
• Copy Paste: [[2402.06110]] AI enhanced data assimilation and uncertainty quantification applied to Geological Carbon Storage(https://arxiv.org/abs/2402.06110)
• Keywords: rag
• Abstract: This study investigates the integration of machine learning (ML) and data assimilation (DA) techniques, focusing on implementing surrogate models for Geological Carbon Storage (GCS) projects while maintaining high fidelity physical results in posterior states. Initially, we evaluate the surrogate modeling capability of two distinct machine learning models, Fourier Neural Operators (FNOs) and Transformer UNet (T-UNet), in the context of CO$_2$ injection simulations within channelized reservoirs. We introduce the Surrogate-based hybrid ESMDA (SH-ESMDA), an adaptation of the traditional Ensemble Smoother with Multiple Data Assimilation (ESMDA). This method uses FNOs and T-UNet as surrogate models and has the potential to make the standard ESMDA process at least 50% faster or more, depending on the number of assimilation steps. Additionally, we introduce Surrogate-based Hybrid RML (SH-RML), a variational data assimilation approach that relies on the randomized maximum likelihood (RML) where both the FNO and the T-UNet enable the computation of gradients for the optimization of the objective function, and a high-fidelity model is employed for the computation of the posterior states. Our comparative analyses show that SH-RML offers better uncertainty quantification compared to conventional ESMDA for the case study.
• 摘要：本研究研究了机器学习 (ML) 和数据同化 (DA) 技术的集成，重点是实施地质碳封存 (GCS) 项目的替代模型，同时保持后验状态的高保真度物理结果。最初，我们在渠道化储层内 CO$_2$ 注入模拟的背景下评估了两种不同的机器学习模型：傅里叶神经算子 (FNO) 和 Transformer UNet (T-UNet) 的代理建模能力。我们引入了基于代理的混合 ESMDA (SH-ESMDA)，它是传统的多重数据同化集成平滑器 (ESMDA) 的改编。该方法使用 FNO 和 T-UNet 作为替代模型，有可能使标准 ESMDA 过程至少快 50% 或更多，具体取决于同化步骤的数量。此外，我们还引入了基于代理的混合 RML (SH-RML)，这是一种依赖于随机最大似然 (RML) 的变分数据同化方法，其中 FNO 和 T-UNet 都可以计算梯度以优化目标函数，并采用高保真模型来计算后验状态。我们的比较分析表明，与案例研究中的传统 ESMDA 相比，SH-RML 提供了更好的不确定性量化。

Title: Exploring Group and Symmetry Principles in Large Language Models

• Authors: Shima Imani, Hamid Palangi
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06120
• Pdf URL: https://arxiv.org/pdf/2402.06120
• Copy Paste: [[2402.06120]] Exploring Group and Symmetry Principles in Large Language Models(https://arxiv.org/abs/2402.06120)
• Keywords: language model, llm
• Abstract: Large Language Models (LLMs) have demonstrated impressive performance across a wide range of applications; however, assessing their reasoning capabilities remains a significant challenge. In this paper, we introduce a framework grounded in group and symmetry principles, which have played a crucial role in fields such as physics and mathematics, and offer another way to evaluate their capabilities. While the proposed framework is general, to showcase the benefits of employing these properties, we focus on arithmetic reasoning and investigate the performance of these models on four group properties: closure, identity, inverse, and associativity. Our findings reveal that LLMs studied in this work struggle to preserve group properties across different test regimes. In the closure test, we observe biases towards specific outputs and an abrupt degradation in their performance from 100% to 0% after a specific sequence length. They also perform poorly in the identity test, which represents adding irrelevant information in the context, and show sensitivity when subjected to inverse test, which examines the robustness of the model with respect to negation. In addition, we demonstrate that breaking down problems into smaller steps helps LLMs in the associativity test that we have conducted. To support these tests we have developed a synthetic dataset which will be released.
• 摘要：大型语言模型 (LLM) 在广泛的应用程序中展示了令人印象深刻的性能；然而，评估他们的推理能力仍然是一个重大挑战。在本文中，我们介绍了一个基于群原理和对称原理的框架，这些原理在物理和数学等领域发挥了至关重要的作用，并提供了另一种评估其能力的方法。虽然提出的框架是通用的，但为了展示使用这些属性的好处，我们专注于算术推理并研究这些模型在四个组属性上的性能：闭包、恒等、逆和关联性。我们的研究结果表明，在这项工作中研究的法学硕士很难在不同的测试制度中保留群体属性。在封闭测试中，我们观察到特定输出的偏差以及在特定序列长度后其性能从 100% 突然下降到 0%。它们在身份测试中也表现不佳，身份测试代表在上下文中添加不相关的信息，并且在进行逆向测试时表现出敏感性，逆向测试检查模型相对于否定的稳健性。此外，我们还证明，将问题分解为更小的步骤有助于法学硕士进行我们进行的关联性测试。为了支持这些测试，我们开发了一个即将发布的综合数据集。

Title: Iterated Denoising Energy Matching for Sampling from Boltzmann Densities

• Authors: Tara Akhound-Sadegh, Jarrid Rector-Brooks, Avishek Joey Bose, Sarthak Mittal, Pablo Lemos, Cheng-Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, Alexander Tong
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2402.06121
• Pdf URL: https://arxiv.org/pdf/2402.06121
• Copy Paste: [[2402.06121]] Iterated Denoising Energy Matching for Sampling from Boltzmann Densities(https://arxiv.org/abs/2402.06121)
• Keywords: lora, rag
• Abstract: Efficiently generating statistically independent samples from an unnormalized probability distribution, such as equilibrium samples of many-body systems, is a foundational problem in science. In this paper, we propose Iterated Denoising Energy Matching (iDEM), an iterative algorithm that uses a novel stochastic score matching objective leveraging solely the energy function and its gradient -- and no data samples -- to train a diffusion-based sampler. Specifically, iDEM alternates between (I) sampling regions of high model density from a diffusion-based sampler and (II) using these samples in our stochastic matching objective to further improve the sampler. iDEM is scalable to high dimensions as the inner matching objective, is simulation-free, and requires no MCMC samples. Moreover, by leveraging the fast mode mixing behavior of diffusion, iDEM smooths out the energy landscape enabling efficient exploration and learning of an amortized sampler. We evaluate iDEM on a suite of tasks ranging from standard synthetic energy functions to invariant $n$-body particle systems. We show that the proposed approach achieves state-of-the-art performance on all metrics and trains $2-5\times$ faster, which allows it to be the first method to train using energy on the challenging $55$-particle Lennard-Jones system.
• 摘要：从非标准化概率分布中有效地生成统计上独立的样本（例如多体系统的平衡样本）是科学中的一个基本问题。在本文中，我们提出了迭代去噪能量匹配（iDEM），这是一种迭代算法，它使用一种新颖的随机分数匹配目标，仅利用能量函数及其梯度（没有数据样本）来训练基于扩散的采样器。具体来说，iDEM 在 (I) 来自基于扩散的采样器的高模型密度采样区域和 (II) 在我们的随机匹配目标中使用这些样本以进一步改进采样器之间进行交替。 iDEM 作为内部匹配目标可扩展到高维度，无需模拟，并且不需要 MCMC 样本。此外，通过利用扩散的快速模式混合行为，iDEM 平滑了能量景观，从而实现了摊销采样器的高效探索和学习。我们在一系列任务上评估 iDEM，从标准合成能量函数到不变的 $n$ 体粒子系统。我们表明，所提出的方法在所有指标上都实现了最先进的性能，并且训练速度提高了 2-5 倍，这使其成为第一个在具有挑战性的 55 美元粒子 Lennard-Jones 上使用能量进行训练的方法系统。

Title: Language Model Sentence Completion with a Parser-Driven Rhetorical Control Method

• Authors: Joshua Zingale, Jugal Kalita
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06125
• Pdf URL: https://arxiv.org/pdf/2402.06125
• Copy Paste: [[2402.06125]] Language Model Sentence Completion with a Parser-Driven Rhetorical Control Method(https://arxiv.org/abs/2402.06125)
• Keywords: language model, llm, code
• Abstract: Controlled text generation (CTG) seeks to guide large language model (LLM) output to produce text that conforms to desired criteria. The current study presents a novel CTG algorithm that enforces adherence toward specific rhetorical relations in an LLM sentence-completion context by a parser-driven decoding scheme that requires no model fine-tuning. The method is validated both with automatic and human evaluation. The code is accessible on GitHub.
• 摘要：受控文本生成 (CTG) 旨在指导大语言模型 (LLM) 输出以生成符合所需标准的文本。当前的研究提出了一种新颖的 CTG 算法，该算法通过解析器驱动的解码方案强制遵守 LLM 句子完成上下文中的特定修辞关系，无需模型微调。该方法通过自动评估和人工评估进行了验证。该代码可在 GitHub 上访问。

Title: Learn To be Efficient: Build Structured Sparsity in Large Language Models

• Authors: Haizhong Zheng, Xiaoyan Bai, Beidi Chen, Fan Lai, Atul Prakash
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06126
• Pdf URL: https://arxiv.org/pdf/2402.06126
• Copy Paste: [[2402.06126]] Learn To be Efficient: Build Structured Sparsity in Large Language Models(https://arxiv.org/abs/2402.06126)
• Keywords: language model, gpt, llm
• Abstract: Large Language Models (LLMs) have achieved remarkable success with their billion-level parameters, yet they incur high inference overheads. The emergence of activation sparsity in LLMs provides a natural approach to reduce this cost by involving only parts of the parameters for inference. Existing methods only focus on utilizing this naturally formed activation sparsity, overlooking the potential for further amplifying this inherent sparsity. In this paper, we hypothesize that LLMs can learn to be efficient by achieving more structured activation sparsity.To achieve this, we introduce a novel algorithm, Learn-To-be-Efficient (LTE), designed to train efficiency-aware LLMs to learn to activate fewer neurons and achieve a better trade-off between sparsity and performance. Furthermore, unlike SOTA MoEfication methods, which mainly focus on ReLU-based models, LTE can also be applied to LLMs like GPT and LLaMA with soft activation functions. We evaluate LTE on four models and eleven datasets. The experiments show that LTE achieves a better trade-off between sparsity and task performance. For instance, LTE with LLaMA provides a 1.83x-2.59x FLOPs speed-up on language generation tasks, outperforming the state-of-the-art methods.
• 摘要：大型语言模型 (LLM) 凭借其十亿级参数取得了显着的成功，但它们会产生很高的推理开销。 LLM 中激活稀疏性的出现提供了一种自然的方法，通过仅涉及部分参数进行推理来降低这种成本。现有方法仅专注于利用这种自然形成的激活稀疏性，而忽视了进一步放大这种固有稀疏性的潜力。在本文中，我们假设 LLM 可以通过实现更结构化的激活稀疏性来学习高效。为了实现这一目标，我们引入了一种新颖的算法，Learn-To-be-Efficient (LTE)，旨在训练具有效率意识的 LLM 学习激活更少的神经元并在稀疏性和性能之间实现更好的权衡。此外，与主要关注基于ReLU的模型的SOTA MoEfication方法不同，LTE还可以应用于具有软激活函数的GPT和LLaMA等LLM。我们在四个模型和 11 个数据集上评估 LTE。实验表明，LTE 在稀疏性和任务性能之间实现了更好的权衡。例如，带有 LLaMA 的 LTE 将语言生成任务的 FLOP 速度提高了 1.83 倍至 2.59 倍，优于最先进的方法。

Title: Jointly Learning Representations for Map Entities via Heterogeneous Graph Contrastive Learning

• Authors: Jiawei Jiang, Yifan Yang, Jingyuan Wang, Junjie Wu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06135
• Pdf URL: https://arxiv.org/pdf/2402.06135
• Copy Paste: [[2402.06135]] Jointly Learning Representations for Map Entities via Heterogeneous Graph Contrastive Learning(https://arxiv.org/abs/2402.06135)
• Keywords: code
• Abstract: The electronic map plays a crucial role in geographic information systems, serving various urban managerial scenarios and daily life services. Developing effective Map Entity Representation Learning (MERL) methods is crucial to extracting embedding information from electronic maps and converting map entities into representation vectors for downstream applications. However, existing MERL methods typically focus on one specific category of map entities, such as POIs, road segments, or land parcels, which is insufficient for real-world diverse map-based applications and might lose latent structural and semantic information interacting between entities of different types. Moreover, using representations generated by separate models for different map entities can introduce inconsistencies. Motivated by this, we propose a novel method named HOME-GCL for learning representations of multiple categories of map entities. Our approach utilizes a heterogeneous map entity graph (HOME graph) that integrates both road segments and land parcels into a unified framework. A HOME encoder with parcel-segment joint feature encoding and heterogeneous graph transformer is then deliberately designed to convert segments and parcels into representation vectors. Moreover, we introduce two types of contrastive learning tasks, namely intra-entity and inter-entity tasks, to train the encoder in a self-supervised manner. Extensive experiments on three large-scale datasets covering road segment-based, land parcel-based, and trajectory-based tasks demonstrate the superiority of our approach. To the best of our knowledge, HOME-GCL is the first attempt to jointly learn representations for road segments and land parcels using a unified model.
• 摘要：电子地图在地理信息系统中发挥着至关重要的作用，服务于各类城市管理场景和日常生活服务。开发有效的地图实体表示学习（MERL）方法对于从电子地图中提取嵌入信息并将地图实体转换为下游应用程序的表示向量至关重要。然而，现有的 MERL 方法通常专注于某一特定类别的地图实体，例如 POI、路段或地块，这不足以满足现实世界中基于地图的多样化应用，并且可能会丢失实体之间交互的潜在结构和语义信息。不同种类。此外，使用由不同模型为不同地图实体生成的表示可能会导致不一致。受此启发，我们提出了一种名为 HOME-GCL 的新方法，用于学习多类别地图实体的表示。我们的方法利用异构地图实体图（HOME 图），将路段和地块集成到统一的框架中。然后特意设计了具有地块-分段联合特征编码和异构图转换器的 HOME 编码器，以将分段和地块转换为表示向量。此外，我们引入了两种类型的对比学习任务，即实体内和实体间任务，以自我监督的方式训练编码器。对三个大型数据集（涵盖基于路段、基于地块和基于轨迹的任务）的广泛实验证明了我们方法的优越性。据我们所知，HOME-GCL 是首次尝试使用统一模型联合学习路段和地块的表示。

Title: DeAL: Decoding-time Alignment for Large Language Models

• Authors: James Y. Huang, Sailik Sengupta, Daniele Bonadiman, Yi-an Lai, Arshit Gupta, Nikolaos Pappas, Saab Mansour, Katrin Kirchoff, Dan Roth
• Subjects: cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06147
• Pdf URL: https://arxiv.org/pdf/2402.06147
• Copy Paste: [[2402.06147]] DeAL: Decoding-time Alignment for Large Language Models(https://arxiv.org/abs/2402.06147)
• Keywords: language model, llm, prompt
• Abstract: Large Language Models (LLMs) are nowadays expected to generate content aligned with human preferences. Current work focuses on alignment at model training time, through techniques such as Reinforcement Learning with Human Feedback (RLHF). However, it is unclear if such methods are an effective choice to teach alignment objectives to the model. First, the inability to incorporate multiple, custom rewards and reliance on a model developer's view of universal and static principles are key limitations. Second, the residual gaps in model training and the reliability of such approaches are also questionable (e.g. susceptibility to jail-breaking even after safety training). To address these, we propose DeAL, a framework that allows the user to customize reward functions and enables Decoding-time Alignment of LLMs (DeAL). At its core, we view decoding as a heuristic-guided search process and facilitate the use of a wide variety of alignment objectives. Our experiments with programmatic constraints such as keyword and length constraints (studied widely in the pre-LLM era) and abstract objectives such as harmlessness and helpfulness (proposed in the post-LLM era) show that we can DeAL with fine-grained trade-offs, improve adherence to alignment objectives, and address residual gaps in LLMs. Lastly, while DeAL can be effectively paired with RLHF and prompting techniques, its generality makes decoding slower, an optimization we leave for future work.
• 摘要：如今，大型语言模型（LLM）有望生成符合人类偏好的内容。目前的工作重点是通过人类反馈强化学习（RLHF）等技术在模型训练时进行对齐。然而，尚不清楚此类方法是否是向模型教授对齐目标的有效选择。首先，无法整合多种自定义奖励以及依赖模型开发人员对通用和静态原则的看法是主要限制。其次，模型训练中的剩余差距和此类方法的可靠性也值得怀疑（例如，即使在安全训练之后也容易越狱）。为了解决这些问题，我们提出了 DeAL，这是一个允许用户自定义奖励函数并启用 LLM 解码时间对齐 (DeAL) 的框架。从本质上讲，我们将解码视为启发式引导的搜索过程，并促进使用各种对齐目标。我们对关键字和长度约束等程序性约束（在 LLM 时代之前广泛研究）和无害性和有用性（在 LLM 时代后提出）等抽象目标的实验表明，我们可以通过细粒度的权衡来进行 DeAL ，提高对调整目标的遵守，并解决法学硕士的剩余差距。最后，虽然 DeAL 可以有效地与 RLHF 和提示技术配合使用，但它的通用性使得解码速度变慢，这是我们留给未来工作的优化。

Title: Domain Generalization with Small Data

• Authors: Kecheng Chen, Elena Gal, Hong Yan, Haoliang Li
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2402.06150
• Pdf URL: https://arxiv.org/pdf/2402.06150
• Copy Paste: [[2402.06150]] Domain Generalization with Small Data(https://arxiv.org/abs/2402.06150)
• Keywords: rag
• Abstract: In this work, we propose to tackle the problem of domain generalization in the context of \textit{insufficient samples}. Instead of extracting latent feature embeddings based on deterministic models, we propose to learn a domain-invariant representation based on the probabilistic framework by mapping each data point into probabilistic embeddings. Specifically, we first extend empirical maximum mean discrepancy (MMD) to a novel probabilistic MMD that can measure the discrepancy between mixture distributions (i.e., source domains) consisting of a series of latent distributions rather than latent points. Moreover, instead of imposing the contrastive semantic alignment (CSA) loss based on pairs of latent points, a novel probabilistic CSA loss encourages positive probabilistic embedding pairs to be closer while pulling other negative ones apart. Benefiting from the learned representation captured by probabilistic models, our proposed method can marriage the measurement on the \textit{distribution over distributions} (i.e., the global perspective alignment) and the distribution-based contrastive semantic alignment (i.e., the local perspective alignment). Extensive experimental results on three challenging medical datasets show the effectiveness of our proposed method in the context of insufficient data compared with state-of-the-art methods.
• 摘要：在这项工作中，我们建议在 \textit{样本不足} 的背景下解决域泛化问题。我们建议通过将每个数据点映射到概率嵌入来学习基于概率框架的域不变表示，而不是基于确定性模型提取潜在特征嵌入。具体来说，我们首先将经验最大平均差异（MMD）扩展到一种新的概率MMD，它可以测量由一系列潜在分布而不是潜在点组成的混合分布（即源域）之间的差异。此外，一种新颖的概率 CSA 损失不是基于潜在点对强加对比语义对齐（CSA）损失，而是鼓励正概率嵌入对更接近，同时将其他负概率嵌入对拉开。受益于概率模型捕获的学习表示，我们提出的方法可以将 \textit{分布上的分布} 的测量（即全局视角对齐）和基于分布的对比语义对齐（即局部视角对齐）结合起来。对三个具有挑战性的医学数据集的广泛实验结果表明，与最先进的方法相比，我们提出的方法在数据不足的情况下是有效的。

Title: Model Editing with Canonical Examples

• Authors: John Hewitt, Sarah Chen, Lanruo Lora Xie, Edward Adams, Percy Liang, Christopher D. Manning
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06155
• Pdf URL: https://arxiv.org/pdf/2402.06155
• Copy Paste: [[2402.06155]] Model Editing with Canonical Examples(https://arxiv.org/abs/2402.06155)
• Keywords: language model, gpt, lora, rag
• Abstract: We introduce model editing with canonical examples, a setting in which (1) a single learning example is provided per desired behavior, (2) evaluation is performed exclusively out-of-distribution, and (3) deviation from an initial model is strictly limited. A canonical example is a simple instance of good behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g., An aspect of researchers is coldhearted). The evaluation set contains more complex examples of each behavior (like a paragraph in which the capital of Mauritius is called for.) We create three datasets and modify three more for model editing with canonical examples, covering knowledge-intensive improvements, social bias mitigation, and syntactic edge cases. In our experiments on Pythia language models, we find that LoRA outperforms full finetuning and MEMIT. We then turn to the Backpack language model architecture because it is intended to enable targeted improvement. The Backpack defines a large bank of sense vectors--a decomposition of the different uses of each word--which are weighted and summed to form the output logits of the model. We propose sense finetuning, which selects and finetunes a few ($\approx$ 10) sense vectors for each canonical example, and find that it outperforms other finetuning methods, e.g., 4.8% improvement vs 0.3%. Finally, we improve GPT-J-6B by an inference-time ensemble with just the changes from sense finetuning of a 35x smaller Backpack, in one setting outperforming editing GPT-J itself (4.1% vs 1.0%).
• 摘要：我们引入了带有规范示例的模型编辑，在这种设置中（1）根据所需行为提供单个学习示例，（2）仅在分布外执行评估，以及（3）严格限制与初始模型的偏差。一个典型的例子是良好行为的简单实例，例如，毛里求斯的首都是路易港）或不良行为，例如，研究人员的一个方面是冷酷的）。评估集包含每种行为的更复杂的示例（例如要求毛里求斯首都的段落）。我们创建三个数据集并修改另外三个数据集，以使用规范示例进行模型编辑，涵盖知识密集型改进、社会偏见缓解、和句法边缘情况。在 Pythia 语言模型的实验中，我们发现 LoRA 的性能优于完全微调和 MEMIT。然后我们转向 Backpack 语言模型架构，因为它的目的是实现有针对性的改进。 Backpack 定义了大量意义向量（每个单词不同用途的分解），对它们进行加权和求和以形成模型的输出逻辑。我们提出了意义微调，它为每个典型示例选择并微调一些（$\approx$10）意义向量，并发现它优于其他微调方法，例如 4.8% 的改进 vs 0.3%。最后，我们通过推理时间集成改进了 GPT-J-6B，仅对小 35 倍的 Backpack 进行感知微调，在一种设置中优于编辑 GPT-J 本身（4.1% vs 1.0%）。

Title: Premier-TACO: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss

• Authors: Ruijie Zheng, Yongyuan Liang, Xiyao Wang, Shuang Ma, Hal Daumé III, Huazhe Xu, John Langford, Praveen Palanisamy, Kalyan Shankar Basu, Furong Huang
• Subjects: cs.LG, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2402.06187
• Pdf URL: https://arxiv.org/pdf/2402.06187
• Copy Paste: [[2402.06187]] Premier-TACO: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss(https://arxiv.org/abs/2402.06187)
• Keywords: code, rag
• Abstract: We present Premier-TACO, a multitask feature representation learning approach designed to improve few-shot policy learning efficiency in sequential decision-making tasks. Premier-TACO leverages a subset of multitask offline datasets for pretraining a general feature representation, which captures critical environmental dynamics and is fine-tuned using minimal expert demonstrations. It advances the temporal action contrastive learning (TACO) objective, known for state-of-the-art results in visual control tasks, by incorporating a novel negative example sampling strategy. This strategy is crucial in significantly boosting TACO's computational efficiency, making large-scale multitask offline pretraining feasible. Our extensive empirical evaluation in a diverse set of continuous control benchmarks including Deepmind Control Suite, MetaWorld, and LIBERO demonstrate Premier-TACO's effectiveness in pretraining visual representations, significantly enhancing few-shot imitation learning of novel tasks. Our code, pretraining data, as well as pretrained model checkpoints will be released at https://github.com/PremierTACO/premier-taco.
• 摘要：我们提出了 Premier-TACO，这是一种多任务特征表示学习方法，旨在提高顺序决策任务中的小样本策略学习效率。 Premier-TACO 利用多任务离线数据集的子集来预训练一般特征表示，该表示捕获关键的环境动态，并使用最少的专家演示进行微调。它通过结合新颖的负例采样策略，推进了时间动作对比学习（TACO）目标，该目标以视觉控制任务中最先进的结果而闻名。该策略对于显着提高 TACO 的计算效率至关重要，使大规模多任务离线预训练变得可行。我们对包括 Deepmind Control Suite、MetaWorld 和 LIBERO 在内的各种连续控制基准进行了广泛的实证评估，证明了 Premier-TACO 在预训练视觉表示方面的有效性，显着增强了新任务的小样本模仿学习。我们的代码、预训练数据以及预训练模型检查点将在 https://github.com/PremierTACO/premier-taco 发布。

Title: Large Language Models: A Survey

• Authors: Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, Jianfeng Gao
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06196
• Pdf URL: https://arxiv.org/pdf/2402.06196
• Copy Paste: [[2402.06196]] Large Language Models: A Survey(https://arxiv.org/abs/2402.06196)
• Keywords: language model, gpt, llm, chat
• Abstract: Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks, since the release of ChatGPT in November 2022. LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data, as predicted by scaling laws \cite{kaplan2020scaling,hoffmann2022training}. The research area of LLMs, while very recent, is evolving rapidly in many different ways. In this paper, we review some of the most prominent LLMs, including three popular LLM families (GPT, LLaMA, PaLM), and discuss their characteristics, contributions and limitations. We also give an overview of techniques developed to build, and augment LLMs. We then survey popular datasets prepared for LLM training, fine-tuning, and evaluation, review widely used LLM evaluation metrics, and compare the performance of several popular LLMs on a set of representative benchmarks. Finally, we conclude the paper by discussing open challenges and future research directions.
• 摘要：自2022年11月ChatGPT发布以来，大型语言模型（LLM）因其在广泛的自然语言任务上的强劲表现而受到广泛关注。LLM的通用语言理解和生成能力是通过训练获得的正如缩放定律所预测的那样，在大量文本数据上有数十亿个模型参数 \cite{kaplan2020scaling,hoffmann2022training}。法学硕士的研究领域虽然最近，但正在以许多不同的方式迅速发展。在本文中，我们回顾了一些最著名的法学硕士，包括三个流行的法学硕士系列（GPT、LLaMA、PaLM），并讨论了它们的特点、贡献和局限性。我们还概述了为构建和增强法学硕士而开发的技术。然后，我们调查为 LLM 培训、微调和评估准备的流行数据集，审查广泛使用的 LLM 评估指标，并比较几个流行的 LLM 在一组代表性基准上的表现。最后，我们通过讨论开放的挑战和未来的研究方向来总结本文。

Title: The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluate

• Authors: Juhyun Oh, Eunsu Kim, Inha Cha, Alice Oh
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06204
• Pdf URL: https://arxiv.org/pdf/2402.06204
• Copy Paste: [[2402.06204]] The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluate(https://arxiv.org/abs/2402.06204)
• Keywords: language model, llm
• Abstract: This paper explores the assumption that Large Language Models (LLMs) skilled in generation tasks are equally adept as evaluators. We assess the performance of three LLMs and one open-source LM in Question-Answering (QA) and evaluation tasks using the TriviaQA (Joshi et al., 2017) dataset. Results indicate a significant disparity, with LLMs exhibiting lower performance in evaluation tasks compared to generation tasks. Intriguingly, we discover instances of unfaithful evaluation where models accurately evaluate answers in areas where they lack competence, underscoring the need to examine the faithfulness and trustworthiness of LLMs as evaluators. This study contributes to the understanding of "the Generative AI Paradox" (West et al., 2023), highlighting a need to explore the correlation between generative excellence and evaluation proficiency, and the necessity to scrutinize the faithfulness aspect in model evaluations.
• 摘要：本文探讨了这样一个假设：擅长生成任务的大型语言模型（LLM）同样擅长评估器。我们使用 TriviaQA (Joshi et al., 2017) 数据集评估了三个法学硕士和一个开源语言模型在问答 (QA) 和评估任务中的表现。结果表明存在显着差异，与生成任务相比，法学硕士在评估任务中表现出较低的表现。有趣的是，我们发现了不忠实评估的例子，其中模型准确地评估了他们缺乏能力的领域的答案，这强调了有必要检查法学硕士作为评估者的忠诚度和可信度。这项研究有助于理解“生成人工智能悖论”（West et al., 2023），强调需要探索生成卓越性和评估熟练程度之间的相关性，以及审查模型评估中的忠实度方面的必要性。

Title: ResumeFlow: An LLM-facilitated Pipeline for Personalized Resume Generation and Refinement

• Authors: Saurabh Bhausaheb Zinjad, Amrita Bhattacharjee, Amey Bhilegaonkar, Huan Liu
• Subjects: cs.CL, cs.IR
• Abstract URL: https://arxiv.org/abs/2402.06221
• Pdf URL: https://arxiv.org/pdf/2402.06221
• Copy Paste: [[2402.06221]] ResumeFlow: An LLM-facilitated Pipeline for Personalized Resume Generation and Refinement(https://arxiv.org/abs/2402.06221)
• Keywords: language model, gpt, llm, hallucination, rag
• Abstract: Crafting the ideal, job-specific resume is a challenging task for many job applicants, especially for early-career applicants. While it is highly recommended that applicants tailor their resume to the specific role they are applying for, manually tailoring resumes to job descriptions and role-specific requirements is often (1) extremely time-consuming, and (2) prone to human errors. Furthermore, performing such a tailoring step at scale while applying to several roles may result in a lack of quality of the edited resumes. To tackle this problem, in this demo paper, we propose ResumeFlow: a Large Language Model (LLM) aided tool that enables an end user to simply provide their detailed resume and the desired job posting, and obtain a personalized resume specifically tailored to that specific job posting in the matter of a few seconds. Our proposed pipeline leverages the language understanding and information extraction capabilities of state-of-the-art LLMs such as OpenAI's GPT-4 and Google's Gemini, in order to (1) extract details from a job description, (2) extract role-specific details from the user-provided resume, and then (3) use these to refine and generate a role-specific resume for the user. Our easy-to-use tool leverages the user-chosen LLM in a completely off-the-shelf manner, thus requiring no fine-tuning. We demonstrate the effectiveness of our tool via a video demo and propose novel task-specific evaluation metrics to control for alignment and hallucination. Our tool is available at https://job-aligned-resume.streamlit.app.
• 摘要：对于许多求职者来说，特别是对于职业生涯早期的求职者来说，制作理想的、针对具体工作的简历是一项具有挑战性的任务。虽然强烈建议申请人根据其申请的特定职位定制简历，但根据职位描述和特定职位要求手动定制简历通常 (1) 极其耗时，(2) 容易出现人为错误。此外，在申请多个职位时大规模执行此类定制步骤可能会导致编辑后的简历质量下降。为了解决这个问题，在这篇演示论文中，我们提出了 ResumeFlow：一种大型语言模型 (LLM) 辅助工具，使最终用户能够简单地提供其详细简历和所需的职位发布，并获得专门针对该特定职位量身定制的个性化简历几秒钟内即可发布招聘信息。我们提出的管道利用了最先进的 LLM 的语言理解和信息提取功能，例如 OpenAI 的 GPT-4 和 Google 的 Gemini，以便 (1) 从职位描述中提取详细信息，(2) 提取特定于角色的信息从用户提供的简历中获取详细信息，然后 (3) 使用这些信息为用户完善并生成特定于角色的简历。我们易于使用的工具以完全现成的方式利用用户选择的 LLM，因此无需进行微调。我们通过视频演示展示了我们工具的有效性，并提出了新颖的特定于任务的评估指标来控制对齐和幻觉。我们的工具可在 https://job-aligned-resume.streamlit.app 上获取。

Title: Studious Bob Fight Back Against Jailbreaking via Prompt Adversarial Tuning

• Authors: Yichuan Mo, Yuji Wang, Zeming Wei, Yisen Wang
• Subjects: cs.LG, cs.AI, cs.CL, cs.CR
• Abstract URL: https://arxiv.org/abs/2402.06255
• Pdf URL: https://arxiv.org/pdf/2402.06255
• Copy Paste: [[2402.06255]] Studious Bob Fight Back Against Jailbreaking via Prompt Adversarial Tuning(https://arxiv.org/abs/2402.06255)
• Keywords: language model, llm, lora, prompt
• Abstract: Although Large Language Models (LLMs) have achieved tremendous success in various applications, they are also susceptible to certain prompts that can induce them to bypass built-in safety measures and provide dangerous or illegal content, a phenomenon known as jailbreak. To protect LLMs from producing harmful information, various defense strategies are proposed, with most focusing on content filtering or adversarial training of models. In this paper, we propose an approach named Prompt Adversarial Tuning (PAT) to train a defense control mechanism, which is then embedded as a prefix to user prompts to implement our defense strategy. We design a training process similar to adversarial training to achieve our optimized goal, alternating between updating attack and defense controls. To our knowledge, we are the first to implement defense from the perspective of prompt tuning. Once employed, our method will hardly impact the operational efficiency of LLMs. Experiments show that our method is effective in both black-box and white-box settings, reducing the success rate of advanced attacks to nearly 0 while maintaining the benign answer rate of 80% to simple benign questions. Our work might potentially chart a new perspective for future explorations in LLM security.
• 摘要：尽管大型语言模型 (LLM) 在各种应用中取得了巨大成功，但它们也容易受到某些提示的影响，这些提示可能会诱使它们绕过内置安全措施并提供危险或非法内容，这种现象称为越狱。为了保护法学硕士免于产生有害信息，人们提出了各种防御策略，其中最关注的是内容过滤或模型的对抗性训练。在本文中，我们提出了一种名为提示对抗调整（PAT）的方法来训练防御控制机制，然后将其作为前缀嵌入到用户提示中以实施我们的防御策略。我们设计了一个类似于对抗性训练的训练过程，以实现我们的优化目标，在更新攻击和防御控制之间交替。据我们所知，我们是第一个从即时调优角度实现防御的。一旦采用，我们的方法几乎不会影响法学硕士的运营效率。实验表明，我们的方法在黑盒和白盒设置中都是有效的，将高级攻击的成功率降低到接近 0，同时保持简单良性问题 80% 的良性回答率。我们的工作可能会为法学硕士安全性的未来探索提供新的视角。

Title: On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inference

• Authors: Siyu Ren, Kenny Q. Zhu
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06262
• Pdf URL: https://arxiv.org/pdf/2402.06262
• Copy Paste: [[2402.06262]] On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inference(https://arxiv.org/abs/2402.06262)
• Keywords: language model, llm, code
• Abstract: Despite the recent success associated with Large Language Models~(LLMs), they are notably cost-prohibitive to deploy in resource-constrained environments due to their excessive memory and computational demands. In addition to model parameters, the key-value cache is also stored in GPU memory, growing linearly with batch size and sequence length. As a remedy, recent works have proposed various eviction policies for maintaining the overhead of key-value cache under a given budget. This paper embarks on the efficacy of existing eviction policies in terms of \textit{importance score calculation} and \textit{eviction scope construction}. We identify the deficiency of prior policies in these two aspects and introduce RoCo, a \underline{r}\underline{o}bust \underline{c}ache \underline{o}mission policy based on temporal attention scores and robustness measures. Extensive experimentation spanning prefilling and auto-regressive decoding stages validates the superiority of RoCo. Finally, we release EasyKV, a versatile software package dedicated to user-friendly key-value constrained generative inference. Code available at \url{https://github.com/DRSY/EasyKV}.
• 摘要：尽管最近与大型语言模型（LLM）相关的成功，但由于其过多的内存和计算需求，在资源有限的环境中部署它们的成本非常高昂。除了模型参数之外，键值缓存也存储在 GPU 内存中，随着批量大小和序列长度线性增长。作为一种补救措施，最近的工作提出了各种驱逐策略，以在给定的预算下维持键值缓存的开销。本文从 \textit{重要性得分计算} 和 \textit{驱逐范围构建} 方面着手探讨现有驱逐政策的有效性。我们发现了先前策略在这两个方面的不足，并引入了 RoCo，一种基于时间注意力分数和鲁棒性度量的 \underline{r}\underline{o}bust \underline{c}ache \underline{o}mission 策略。涵盖预填充和自回归解码阶段的广泛实验验证了 RoCo 的优越性。最后，我们发布了 EasyKV，这是一个多功能软件包，专用于用户友好的键值约束生成推理。代码可在 \url{https://github.com/DRSY/EasyKV} 获取。

Title: LLaVA-Docent: Instruction Tuning with Multimodal Large Language Model to Support Art Appreciation Education

• Authors: Unggi Lee, Minji Jeon, Yunseo Lee, Gyuri Byun, Yoorim Son, Jaeyoon Shin, Hongkyu Ko, Hyeoncheol Kim
• Subjects: cs.AI, cs.SI
• Abstract URL: https://arxiv.org/abs/2402.06264
• Pdf URL: https://arxiv.org/pdf/2402.06264
• Copy Paste: [[2402.06264]] LLaVA-Docent: Instruction Tuning with Multimodal Large Language Model to Support Art Appreciation Education(https://arxiv.org/abs/2402.06264)
• Keywords: language model, gpt, llm, rag
• Abstract: Art appreciation is vital in nurturing critical thinking and emotional intelligence among learners. However, traditional art appreciation education has often been hindered by limited access to art resources, especially for disadvantaged students, and an imbalanced emphasis on STEM subjects in mainstream education. In response to these challenges, recent technological advancements have paved the way for innovative solutions. This study explores the application of multi-modal large language models (MLLMs) in art appreciation education, focusing on developing LLaVA-Docent, a model that leverages these advancements. Our approach involved a comprehensive literature review and consultations with experts in the field, leading to developing a robust data framework. Utilizing this framework, we generated a virtual dialogue dataset that was leveraged by GPT-4. This dataset was instrumental in training the MLLM, named LLaVA-Docent. Six researchers conducted quantitative and qualitative evaluations of LLaVA-Docent to assess its effectiveness, benchmarking it against the GPT-4 model in a few-shot setting. The evaluation process revealed distinct strengths and weaknesses of the LLaVA-Docent model. Our findings highlight the efficacy of LLaVA-Docent in enhancing the accessibility and engagement of art appreciation education. By harnessing the potential of MLLMs, this study makes a significant contribution to the field of art education, proposing a novel methodology that reimagines the way art appreciation is taught and experienced.
• 摘要：艺术欣赏对于培养学习者的批判性思维和情商至关重要。然而，传统的艺术欣赏教育往往受到艺术资源有限的阻碍，尤其是对于弱势学生而言，以及主流教育对STEM科目的重视不平衡。为了应对这些挑战，最近的技术进步为创新解决方案铺平了道路。本研究探讨了多模态大语言模型 (MLLM) 在艺术欣赏教育中的应用，重点开发 LLaVA-Docent，这是一个利用这些进步的模型。我们的方法包括全面的文献综述以及与该领域专家的咨询，从而开发出一个强大的数据框架。利用这个框架，我们生成了 GPT-4 使用的虚拟对话数据集。该数据集在训练 MLLM（名为 LLaVA-Docent）方面发挥了重要作用。六名研究人员对 LLaVA-Docent 进行了定量和定性评估，以评估其有效性，并在几次设置中将其与 GPT-4 模型进行基准测试。评估过程揭示了 LLaVA-Docent 模型的明显优点和缺点。我们的研究结果强调了 LLaVA-Docent 在提高艺术欣赏教育的可及性和参与度方面的功效。通过利用 MLLM 的潜力，这项研究为艺术教育领域做出了重大贡献，提出了一种新颖的方法，重新构想了艺术欣赏的教学和体验方式。

Title: Value function interference and greedy action selection in value-based multi-objective reinforcement learning

• Authors: Peter Vamplew, Cameron Foale, Richard Dazeley
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06266
• Pdf URL: https://arxiv.org/pdf/2402.06266
• Copy Paste: [[2402.06266]] Value function interference and greedy action selection in value-based multi-objective reinforcement learning(https://arxiv.org/abs/2402.06266)
• Keywords: agent
• Abstract: Multi-objective reinforcement learning (MORL) algorithms extend conventional reinforcement learning (RL) to the more general case of problems with multiple, conflicting objectives, represented by vector-valued rewards. Widely-used scalar RL methods such as Q-learning can be modified to handle multiple objectives by (1) learning vector-valued value functions, and (2) performing action selection using a scalarisation or ordering operator which reflects the user's utility with respect to the different objectives. However, as we demonstrate here, if the user's utility function maps widely varying vector-values to similar levels of utility, this can lead to interference in the value-function learned by the agent, leading to convergence to sub-optimal policies. This will be most prevalent in stochastic environments when optimising for the Expected Scalarised Return criterion, but we present a simple example showing that interference can also arise in deterministic environments. We demonstrate empirically that avoiding the use of random tie-breaking when identifying greedy actions can ameliorate, but not fully overcome, the problems caused by value function interference.
• 摘要：多目标强化学习 (MORL) 算法将传统的强化学习 (RL) 扩展到更一般的具有多个相互冲突的目标的问题，以向量值奖励为代表。可以修改广泛使用的标量强化学习方法（例如 Q 学习）来处理多个目标，方法是：（1）学习向量值函数，以及（2）使用反映用户效用的标量或排序运算符执行动作选择不同的目标。然而，正如我们在这里演示的，如果用户的效用函数将广泛变化的向量值映射到相似的效用水平，这可能会导致对代理学习的价值函数的干扰，从而导致收敛到次优策略。当针对预期阶梯回报标准进行优化时，这在随机环境中最为普遍，但我们提供了一个简单的示例，表明在确定性环境中也可能出现干扰。我们凭经验证明，在识别贪婪行为时避免使用随机平局打破可以改善但不能完全克服由价值函数干扰引起的问题。

Title: Evaluating Membership Inference Attacks and Defenses in Federated Learning

• Authors: Gongxi Zhu, Donghao Li, Hanlin Gu, Yuxing Han, Yuan Yao, Lixin Fan, Qiang Yang
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2402.06289
• Pdf URL: https://arxiv.org/pdf/2402.06289
• Copy Paste: [[2402.06289]] Evaluating Membership Inference Attacks and Defenses in Federated Learning(https://arxiv.org/abs/2402.06289)
• Keywords: code
• Abstract: Membership Inference Attacks (MIAs) pose a growing threat to privacy preservation in federated learning. The semi-honest attacker, e.g., the server, may determine whether a particular sample belongs to a target client according to the observed model information. This paper conducts an evaluation of existing MIAs and corresponding defense strategies. Our evaluation on MIAs reveals two important findings about the trend of MIAs. Firstly, combining model information from multiple communication rounds (Multi-temporal) enhances the overall effectiveness of MIAs compared to utilizing model information from a single epoch. Secondly, incorporating models from non-target clients (Multi-spatial) significantly improves the effectiveness of MIAs, particularly when the clients' data is homogeneous. This highlights the importance of considering the temporal and spatial model information in MIAs. Next, we assess the effectiveness via privacy-utility tradeoff for two type defense mechanisms against MIAs: Gradient Perturbation and Data Replacement. Our results demonstrate that Data Replacement mechanisms achieve a more optimal balance between preserving privacy and maintaining model utility. Therefore, we recommend the adoption of Data Replacement methods as a defense strategy against MIAs. Our code is available in https://github.com/Liar-Mask/FedMIA.
• 摘要：成员推理攻击 (MIA) 对联邦学习中的隐私保护构成了越来越大的威胁。半诚实攻击者，例如服务器，可以根据观察到的模型信息来确定特定样本是否属于目标客户端。本文对现有的 MIA 和相应的防御策略进行了评估。我们对 MIA 的评估揭示了有关 MIA 趋势的两个重要发现。首先，与利用单个 epoch 的模型信息相比，组合来自多轮通信（多时间）的模型信息可以增强 MIA 的整体有效性。其次，合并来自非目标客户（多空间）的模型可以显着提高 MIA 的有效性，特别是当客户数据同质时。这凸显了在 MIA 中考虑时间和空间模型信息的重要性。接下来，我们通过隐私与效用权衡来评估两种针对 MIA 的防御机制的有效性：梯度扰动和数据替换。我们的结果表明，数据替换机制在保护隐私和维护模型效用之间实现了更佳的平衡。因此，我们建议采用数据替换方法作为针对 MIA 的防御策略。我们的代码可在 https://github.com/Liar-Mask/FedMIA 中获取。

Title: Multimodal Interpretable Data-Driven Models for Early Prediction of Antimicrobial Multidrug Resistance Using Multivariate Time-Series

• Authors: Sergio Martínez-Agüero, Antonio G. Marques, Inmaculada Mora-Jiménez, Joaquín Alvárez-Rodríguez, Cristina Soguero-Ruiza
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2402.06295
• Pdf URL: https://arxiv.org/pdf/2402.06295
• Copy Paste: [[2402.06295]] Multimodal Interpretable Data-Driven Models for Early Prediction of Antimicrobial Multidrug Resistance Using Multivariate Time-Series(https://arxiv.org/abs/2402.06295)
• Keywords: rag
• Abstract: Electronic health records (EHR) is an inherently multimodal register of the patient's health status characterized by static data and multivariate time series (MTS). While MTS are a valuable tool for clinical prediction, their fusion with other data modalities can possibly result in more thorough insights and more accurate results. Deep neural networks (DNNs) have emerged as fundamental tools for identifying and defining underlying patterns in the healthcare domain. However, fundamental improvements in interpretability are needed for DNN models to be widely used in the clinical setting. In this study, we present an approach built on a collection of interpretable multimodal data-driven models that may anticipate and understand the emergence of antimicrobial multidrug resistance (AMR) germs in the intensive care unit (ICU) of the University Hospital of Fuenlabrada (Madrid, Spain). The profile and initial health status of the patient are modeled using static variables, while the evolution of the patient's health status during the ICU stay is modeled using several MTS, including mechanical ventilation and antibiotics intake. The multimodal DNNs models proposed in this paper include interpretable principles in addition to being effective at predicting AMR and providing an explainable prediction support system for AMR in the ICU. Furthermore, our proposed methodology based on multimodal models and interpretability schemes can be leveraged in additional clinical problems dealing with EHR data, broadening the impact and applicability of our results.
• 摘要：电子健康记录 (EHR) 本质上是患者健康状况的多模式登记，以静态数据和多元时间序列 (MTS) 为特征。虽然 MTS 是临床预测的宝贵工具，但它们与其他数据模式的融合可能会带来更全面的见解和更准确的结果。深度神经网络 (DNN) 已成为识别和定义医疗保健领域潜在模式的基本工具。然而，要在临床环境中广泛使用 DNN 模型，需要在可解释性方面进行根本性改进。在这项研究中，我们提出了一种基于可解释的多模式数据驱动模型集合的方法，该模型可以预测和理解富恩拉夫拉达大学医院（马德里）重症监护室（ICU）中抗菌药物多药耐药性（AMR）细菌的出现， 西班牙）。使用静态变量对患者的概况和初始健康状况进行建模，而在 ICU 住院期间患者健康状况的演变则使用多个 MTS 进行建模，包括机械通气和抗生素摄入量。本文提出的多模态 DNN 模型除了能够有效预测 AMR 并为 ICU 中的 AMR 提供可解释的预测支持系统之外，还包括可解释的原理。此外，我们提出的基于多模式模型和可解释性方案的方法可以用于处理 EHR 数据的其他临床问题，扩大我们结果的影响和适用性。

Title: Prompt Learning on Temporal Interaction Graphs

• Authors: Xi Chen, Siwei Zhang, Yun Xiong, Xixi Wu, Jiawei Zhang, Xiangguo Sun, Yao Zhang, Yinglong Zhao, Yulin Kang
• Subjects: cs.AI, cs.LG, cs.SI
• Abstract URL: https://arxiv.org/abs/2402.06326
• Pdf URL: https://arxiv.org/pdf/2402.06326
• Copy Paste: [[2402.06326]] Prompt Learning on Temporal Interaction Graphs(https://arxiv.org/abs/2402.06326)
• Keywords: prompt
• Abstract: Temporal Interaction Graphs (TIGs) are widely utilized to represent real-world systems. To facilitate representation learning on TIGs, researchers have proposed a series of TIG models. However, these models are still facing two tough gaps between the pre-training and downstream predictions in their ``pre-train, predict'' training paradigm. First, the temporal discrepancy between the pre-training and inference data severely undermines the models' applicability in distant future predictions on the dynamically evolving data. Second, the semantic divergence between pretext and downstream tasks hinders their practical applications, as they struggle to align with their learning and prediction capabilities across application scenarios. Recently, the ``pre-train, prompt'' paradigm has emerged as a lightweight mechanism for model generalization. Applying this paradigm is a potential solution to solve the aforementioned challenges. However, the adaptation of this paradigm to TIGs is not straightforward. The application of prompting in static graph contexts falls short in temporal settings due to a lack of consideration for time-sensitive dynamics and a deficiency in expressive power. To address this issue, we introduce Temporal Interaction Graph Prompting (TIGPrompt), a versatile framework that seamlessly integrates with TIG models, bridging both the temporal and semantic gaps. In detail, we propose a temporal prompt generator to offer temporally-aware prompts for different tasks. These prompts stand out for their minimalistic design, relying solely on the tuning of the prompt generator with very little supervision data. To cater to varying computational resource demands, we propose an extended ``pre-train, prompt-based fine-tune'' paradigm, offering greater flexibility. Through extensive experiments, the TIGPrompt demonstrates the SOTA performance and remarkable efficiency advantages.
• 摘要：时间交互图（TIG）被广泛用于表示现实世界的系统。为了促进 TIG 的表征学习，研究人员提出了一系列 TIG 模型。然而，这些模型在“预训练，预测”训练范式中仍然面临着预训练和下游预测之间的两个巨大差距。首先，预训练和推理数据之间的时间差异严重损害了模型在动态演变数据的遥远未来预测中的适用性。其次，借口和下游任务之间的语义分歧阻碍了它们的实际应用，因为它们很难在跨应用场景中保持学习和预测能力。最近，“预训练、提示”范式已经成为模型泛化的轻量级机制。应用这种范式是解决上述挑战的潜在解决方案。然而，将此范例应用于 TIG 并不简单。由于缺乏对时间敏感动态的考虑和表达能力的缺乏，静态图上下文中的提示应用在时间设置上存在不足。为了解决这个问题，我们引入了时间交互图提示（TIGPrompt），这是一个与 TIG 模型无缝集成的多功能框架，弥合了时间和语义差距。详细地说，我们提出了一个时间提示生成器，为不同的任务提供时间感知的提示。这些提示因其简约的设计而脱颖而出，仅依靠提示生成器的调整而很少有监督数据。为了满足不同的计算资源需求，我们提出了一种扩展的“预训练、基于提示的微调”范例，提供了更大的灵活性。通过大量的实验，TIGPrompt 展示了 SOTA 性能和显着的效率优势。

Title: InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning

• Authors: Huaiyuan Ying, Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, Yudong Wang, Zijian Wu, Shuaibin Li, Fengzhe Zhou, Hongwei Liu, Songyang Zhang, Wenwei Zhang, Hang Yan, Xipeng Qiu, Jiayu Wang, Kai Chen, Dahua Lin
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06332
• Pdf URL: https://arxiv.org/pdf/2402.06332
• Copy Paste: [[2402.06332]] InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning(https://arxiv.org/abs/2402.06332)
• Keywords: language model, llm, code, chain-of-thought
• Abstract: The math abilities of large language models can represent their abstract reasoning ability. In this paper, we introduce and open-source our math reasoning LLMs InternLM-Math which is continue pre-trained from InternLM2. We unify chain-of-thought reasoning, reward modeling, formal reasoning, data augmentation, and code interpreter in a unified seq2seq format and supervise our model to be a versatile math reasoner, verifier, prover, and augmenter. These abilities can be used to develop the next math LLMs or self-iteration. InternLM-Math obtains open-sourced state-of-the-art performance under the setting of in-context learning, supervised fine-tuning, and code-assisted reasoning in various informal and formal benchmarks including GSM8K, MATH, Hungary math exam, MathBench-ZH, and MiniF2F. Our pre-trained model achieves 30.3 on the MiniF2F test set without fine-tuning. We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math. Our models, codes, and data are released at \url{https://github.com/InternLM/InternLM-Math}.
• 摘要：大语言模型的数学能力可以代表其抽象推理能力。在本文中，我们介绍并开源了我们的数学推理法学硕士 InternLM-Math，它是从 InternLM2 继续进行预训练的。我们以统一的 seq2seq 格式统一思想链推理、奖励建模、形式推理、数据增强和代码解释器，并监督我们的模型成为多功能数学推理器、验证器、证明器和增强器。这些能力可用于开发下一个数学法学硕士或自我迭代。 InternLM-Math 在各种非正式和正式基准测试（包括 GSM8K、MATH、匈牙利数学考试、MathBench）中，在上下文学习、监督微调和代码辅助推理的设置下获得了开源的最先进性能-ZH 和 MiniF2F。我们的预训练模型在 MiniF2F 测试集上无需微调即可达到 30.3。我们进一步探讨了如何使用 LEAN 来解决数学问题，并研究其在多任务学习设置下的性能，这表明使用 LEAN 作为数学求解和证明的统一平台的可能性。我们的模型、代码和数据发布在 \url{https://github.com/InternLM/InternLM-Math}。

Title: RareBench: Can LLMs Serve as Rare Diseases Specialists?

• Authors: Xuanzhong Chen, Xiaohao Mao, Qihan Guo, Lun Wang, Shuyang Zhang, Ting Chen
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06341
• Pdf URL: https://arxiv.org/pdf/2402.06341
• Copy Paste: [[2402.06341]] RareBench: Can LLMs Serve as Rare Diseases Specialists?(https://arxiv.org/abs/2402.06341)
• Keywords: language model, gpt, llm, prompt, chat, rag
• Abstract: Generalist Large Language Models (LLMs), such as GPT-4, have shown considerable promise in various domains, including medical diagnosis. Rare diseases, affecting approximately 300 million people worldwide, often have unsatisfactory clinical diagnosis rates primarily due to a lack of experienced physicians and the complexity of differentiating among many rare diseases. In this context, recent news such as "ChatGPT correctly diagnosed a 4-year-old's rare disease after 17 doctors failed" underscore LLMs' potential, yet underexplored, role in clinically diagnosing rare diseases. To bridge this research gap, we introduce RareBench, a pioneering benchmark designed to systematically evaluate the capabilities of LLMs on 4 critical dimensions within the realm of rare diseases. Meanwhile, we have compiled the largest open-source dataset on rare disease patients, establishing a benchmark for future studies in this domain. To facilitate differential diagnosis of rare diseases, we develop a dynamic few-shot prompt methodology, leveraging a comprehensive rare disease knowledge graph synthesized from multiple knowledge bases, significantly enhancing LLMs' diagnostic performance. Moreover, we present an exhaustive comparative study of GPT-4's diagnostic capabilities against those of specialist physicians. Our experimental findings underscore the promising potential of integrating LLMs into the clinical diagnostic process for rare diseases. This paves the way for exciting possibilities in future advancements in this field.
• 摘要：通用大型语言模型 (LLM)，例如 GPT-4，在包括医学诊断在内的各个领域都显示出了巨大的前景。罕见病影响着全球约3亿人，其临床诊断率往往不理想，主要是由于缺乏经验丰富的医生以及区分许多罕见病的复杂性。在这种背景下，最近的新闻，如“在 17 名医生失败后，ChatGPT 正确诊断了一名 4 岁儿童的罕见病”，强调了法学硕士在临床诊断罕见病方面的潜力，但尚未得到充分探索。为了弥补这一研究差距，我们引入了 RareBench，这是一个开创性的基准，旨在系统地评估法学硕士在罕见疾病领域 4 个关键维度上的能力。同时，我们编制了关于罕见病患者的最大的开源数据集，为该领域的未来研究建立了基准。为了促进罕见疾病的鉴别诊断，我们开发了一种动态的小样本提示方法，利用从多个知识库合成的全面的罕见疾病知识图谱，显着提高了法学硕士的诊断性能。此外，我们还对 GPT-4 的诊断能力与专科医生的诊断能力进行了详尽的比较研究。我们的实验结果强调了将法学硕士纳入罕见疾病临床诊断过程的巨大潜力。这为该领域未来的进步铺平了道路。

Title: Modelling Human Values for AI Reasoning

• Authors: Nardine Osman, Mark d'Inverno
• Subjects: cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2402.06359
• Pdf URL: https://arxiv.org/pdf/2402.06359
• Copy Paste: [[2402.06359]] Modelling Human Values for AI Reasoning(https://arxiv.org/abs/2402.06359)
• Keywords: agent
• Abstract: One of today's most significant societal challenges is building AI systems whose behaviour, or the behaviour it enables within communities of interacting agents (human and artificial), aligns with human values. To address this challenge, we detail a formal model of human values for their explicit computational representation. To our knowledge, this has not been attempted as yet, which is surprising given the growing volume of research integrating values within AI. Taking as our starting point the wealth of research investigating the nature of human values from social psychology over the last few decades, we set out to provide such a formal model. We show how this model can provide the foundational apparatus for AI-based reasoning over values, and demonstrate its applicability in real-world use cases. We illustrate how our model captures the key ideas from social psychology research and propose a roadmap for future integrated, and interdisciplinary, research into human values in AI. The ability to automatically reason over values not only helps address the value alignment problem but also facilitates the design of AI systems that can support individuals and communities in making more informed, value-aligned decisions. More and more, individuals and organisations are motivated to understand their values more explicitly and explore whether their behaviours and attitudes properly reflect them. Our work on modelling human values will enable AI systems to be designed and deployed to meet this growing need.
• 摘要：当今最重大的社会挑战之一是构建人工智能系统，其行为或它在交互主体（人类和人工）社区内实现的行为符合人类价值观。为了应对这一挑战，我们详细介绍了人类价值观的正式模型，以实现其明确的计算表示。据我们所知，目前还没有尝试过这种做法，考虑到人工智能中整合价值的研究数量不断增加，这一点令人惊讶。以过去几十年来从社会心理学研究人类价值观本质的大量研究为出发点，我们着手提供这样一个正式的模型。我们展示了该模型如何为基于人工智能的值推理提供基础工具，并证明其在实际用例中的适用性。我们说明了我们的模型如何捕捉社会心理学研究的关键思想，并为未来人工智能中人类价值的综合、跨学科研究提出了路线图。自动推理价值观的能力不仅有助于解决价值观一致问题，而且有助于人工智能系统的设计，支持个人和社区做出更明智、价值观一致的决策。越来越多的个人和组织有动力更明确地理解他们的价值观，并探索他们的行为和态度是否正确反映了这些价值观。我们在人类价值观建模方面的工作将使人工智能系统的设计和部署能够满足这一不断增长的需求。

Title: TEE4EHR: Transformer Event Encoder for Better Representation Learning in Electronic Health Records

• Authors: Hojjat Karami, David Atienza, Anisoara Ionescu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06367
• Pdf URL: https://arxiv.org/pdf/2402.06367
• Copy Paste: [[2402.06367]] TEE4EHR: Transformer Event Encoder for Better Representation Learning in Electronic Health Records(https://arxiv.org/abs/2402.06367)
• Keywords: code
• Abstract: Irregular sampling of time series in electronic health records (EHRs) is one of the main challenges for developing machine learning models. Additionally, the pattern of missing data in certain clinical variables is not at random but depends on the decisions of clinicians and the state of the patient. Point process is a mathematical framework for analyzing event sequence data that is consistent with irregular sampling patterns. Our model, TEE4EHR, is a transformer event encoder (TEE) with point process loss that encodes the pattern of laboratory tests in EHRs. The utility of our TEE has been investigated in a variety of benchmark event sequence datasets. Additionally, we conduct experiments on two real-world EHR databases to provide a more comprehensive evaluation of our model. Firstly, in a self-supervised learning approach, the TEE is jointly learned with an existing attention-based deep neural network which gives superior performance in negative log-likelihood and future event prediction. Besides, we propose an algorithm for aggregating attention weights that can reveal the interaction between the events. Secondly, we transfer and freeze the learned TEE to the downstream task for the outcome prediction, where it outperforms state-of-the-art models for handling irregularly sampled time series. Furthermore, our results demonstrate that our approach can improve representation learning in EHRs and can be useful for clinical prediction tasks.
• 摘要：电子健康记录 (EHR) 中时间序列的不规则采样是开发机器学习模型的主要挑战之一。此外，某些临床变量中缺失数据的模式不是随机的，而是取决于临床医生的决定和患者的状态。点过程是一种用于分析与不规则采样模式一致的事件序列数据的数学框架。我们的模型 TEE4EHR 是一种具有点过程丢失的变压器事件编码器 (TEE)，可对 EHR 中的实验室测试模式进行编码。我们的 TEE 的实用性已在各种基准事件序列数据集中进行了研究。此外，我们在两个现实世界的 EHR 数据库上进行了实验，以便对我们的模型进行更全面的评估。首先，在自监督学习方法中，TEE 与现有的基于注意力的深度神经网络联合学习，该网络在负对数似然和未来事件预测方面具有优越的性能。此外，我们提出了一种聚合注意力权重的算法，可以揭示事件之间的相互作用。其次，我们将学习到的 TEE 转移并冻结到下游任务以进行结果预测，在处理不规则采样时间序列方面，它的性能优于最先进的模型。此外，我们的结果表明，我们的方法可以改善电子病历中的表征学习，并且可用于临床预测任务。

Title: Human Aesthetic Preference-Based Large Text-to-Image Model Personalization: Kandinsky Generation as an Example

• Authors: Aven-Le Zhou, Yu-Ao Wang, Wei Wu, Kang Zhang
• Subjects: cs.AI, cs.HC, cs.MM
• Abstract URL: https://arxiv.org/abs/2402.06389
• Pdf URL: https://arxiv.org/pdf/2402.06389
• Copy Paste: [[2402.06389]] Human Aesthetic Preference-Based Large Text-to-Image Model Personalization: Kandinsky Generation as an Example(https://arxiv.org/abs/2402.06389)
• Keywords: prompt, rag
• Abstract: With the advancement of neural generative capabilities, the art community has actively embraced GenAI (generative artificial intelligence) for creating painterly content. Large text-to-image models can quickly generate aesthetically pleasing outcomes. However, the process can be non-deterministic and often involves tedious trial-and-error, as users struggle with formulating effective prompts to achieve their desired results. This paper introduces a prompting-free generative approach that empowers users to automatically generate personalized painterly content that incorporates their aesthetic preferences in a customized artistic style. This approach involves utilizing ``semantic injection'' to customize an artist model in a specific artistic style, and further leveraging a genetic algorithm to optimize the prompt generation process through real-time iterative human feedback. By solely relying on the user's aesthetic evaluation and preference for the artist model-generated images, this approach creates the user a personalized model that encompasses their aesthetic preferences and the customized artistic style.
• 摘要：随着神经生成能力的进步，艺术界积极采用 GenAI（生成人工智能）来创作绘画内容。大型文本到图像模型可以快速生成美观的结果。然而，这个过程可能是不确定的，并且经常涉及繁琐的试错，因为用户很难制定有效的提示来实现他们想要的结果。本文介绍了一种无提示生成方法，使用户能够自动生成个性化的绘画内容，将他们的审美偏好融入定制的艺术风格中。这种方法涉及利用“语义注入”来定制特定艺术风格的艺术家模型，并进一步利用遗传算法通过实时迭代的人类反馈来优化提示生成过程。通过仅仅依靠用户的审美评价和对艺术家模型生成的图像的偏好，这种方法为用户创建了包含他们的审美偏好和定制的艺术风格的个性化模型。

Title: Hierarchical Transformers are Efficient Meta-Reinforcement Learners

• Authors: Gresa Shala, André Biedenkapp, Josif Grabocka
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06402
• Pdf URL: https://arxiv.org/pdf/2402.06402
• Copy Paste: [[2402.06402]] Hierarchical Transformers are Efficient Meta-Reinforcement Learners(https://arxiv.org/abs/2402.06402)
• Keywords: agent
• Abstract: We introduce Hierarchical Transformers for Meta-Reinforcement Learning (HTrMRL), a powerful online meta-reinforcement learning approach. HTrMRL aims to address the challenge of enabling reinforcement learning agents to perform effectively in previously unseen tasks. We demonstrate how past episodes serve as a rich source of information, which our model effectively distills and applies to new contexts. Our learned algorithm is capable of outperforming the previous state-of-the-art and provides more efficient meta-training while significantly improving generalization capabilities. Experimental results, obtained across various simulated tasks of the Meta-World Benchmark, indicate a significant improvement in learning efficiency and adaptability compared to the state-of-the-art on a variety of tasks. Our approach not only enhances the agent's ability to generalize from limited data but also paves the way for more robust and versatile AI systems.
• 摘要：我们介绍用于元强化学习的分层变压器（HTrMRL），这是一种强大的在线元强化学习方法。 HTrMRL 旨在解决使强化学习代理能够有效执行以前未见过的任务的挑战。我们展示了过去的事件如何作为丰富的信息源，我们的模型有效地提取这些信息并将其应用于新的环境。我们学习的算法能够超越以前的最先进算法，并提供更有效的元训练，同时显着提高泛化能力。在元世界基准的各种模拟任务中获得的实验结果表明，与各种任务的最新技术相比，学习效率和适应性有了显着提高。我们的方法不仅增强了智能体从有限数据中进行概括的能力，而且还为更强大和更通用的人工智能系统铺平了道路。

Title: Trust the Process: Zero-Knowledge Machine Learning to Enhance Trust in Generative AI Interactions

• Authors: Bianca-Mihaela Ganescu, Jonathan Passerat-Palmbach
• Subjects: cs.LG, cs.CR
• Abstract URL: https://arxiv.org/abs/2402.06414
• Pdf URL: https://arxiv.org/pdf/2402.06414
• Copy Paste: [[2402.06414]] Trust the Process: Zero-Knowledge Machine Learning to Enhance Trust in Generative AI Interactions(https://arxiv.org/abs/2402.06414)
• Keywords: gpt
• Abstract: Generative AI, exemplified by models like transformers, has opened up new possibilities in various domains but also raised concerns about fairness, transparency and reliability, especially in fields like medicine and law. This paper emphasizes the urgency of ensuring fairness and quality in these domains through generative AI. It explores using cryptographic techniques, particularly Zero-Knowledge Proofs (ZKPs), to address concerns regarding performance fairness and accuracy while protecting model privacy. Applying ZKPs to Machine Learning models, known as ZKML (Zero-Knowledge Machine Learning), enables independent validation of AI-generated content without revealing sensitive model information, promoting transparency and trust. ZKML enhances AI fairness by providing cryptographic audit trails for model predictions and ensuring uniform performance across users. We introduce snarkGPT, a practical ZKML implementation for transformers, to empower users to verify output accuracy and quality while preserving model privacy. We present a series of empirical results studying snarkGPT's scalability and performance to assess the feasibility and challenges of adopting a ZKML-powered approach to capture quality and performance fairness problems in generative AI models.
• 摘要：以 Transformer 等模型为代表的生成式人工智能在各个领域开辟了新的可能性，但也引起了人们对公平、透明度和可靠性的担忧，特别是在医学和法律等领域。本文强调了通过生成人工智能确保这些领域的公平和质量的紧迫性。它探索使用加密技术，特别是零知识证明（ZKP），来解决有关性能公平性和准确性的问题，同时保护模型隐私。将 ZKP 应用于机器学习模型，称为 ZKML（零知识机器学习），可以独立验证 AI 生成的内容，而不会泄露敏感的模型信息，从而提高透明度和信任。 ZKML 通过为模型预测提供加密审计跟踪并确保跨用户的统一性能来增强 AI 公平性。我们引入 snarkGPT，这是一种实用的 Transformer ZKML 实现，使用户能够验证输出的准确性和质量，同时保护模型隐私。我们提出了一系列研究 snarkGPT 的可扩展性和性能的实证结果，以评估采用 ZKML 支持的方法来捕获生成 AI 模型中的质量和性能公平问题的可行性和挑战。

Title: Findings of the First Workshop on Simulating Conversational Intelligence in Chat

• Authors: Yvette Graham, Mohammed Rameez Qureshi, Haider Khalid, Gerasimos Lampouras, Ignacio Iacobacci, Qun Liu
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06420
• Pdf URL: https://arxiv.org/pdf/2402.06420
• Copy Paste: [[2402.06420]] Findings of the First Workshop on Simulating Conversational Intelligence in Chat(https://arxiv.org/abs/2402.06420)
• Keywords: chat
• Abstract: The aim of this workshop is to bring together experts working on open-domain dialogue research. In this speedily advancing research area many challenges still exist, such as learning information from conversations, engaging in realistic and convincing simulation of human intelligence and reasoning. SCI-CHAT follows previous workshops on open domain dialogue but with a focus on the simulation of intelligent conversation as judged in a live human evaluation. Models aim to include the ability to follow a challenging topic over a multi-turn conversation, while positing, refuting and reasoning over arguments. The workshop included both a research track and shared task. The main goal of this paper is to provide an overview of the shared task and a link to an additional paper that will include an in depth analysis of the shared task results following presentation at the workshop.
• 摘要：本次研讨会的目的是汇集从事开放领域对话研究的专家。在这个快速发展的研究领域仍然存在许多挑战，例如从对话中学习信息，对人类智能和推理进行现实且令人信服的模拟。 SCI-CHAT 沿袭了之前关于开放领域对话的研讨会，但重点是根据现场人类评估来模拟智能对话。模型旨在包括在多轮对话中跟踪具有挑战性的主题的能力，同时对论点进行假设、反驳和推理。研讨会包括研究轨道和共享任务。本文的主要目标是提供共享任务的概述以及另一篇论文的链接，其中将包括对研讨会上演示后的共享任务结果的深入分析。

Title: An Algorithmic Framework for Constructing Multiple Decision Trees by Evaluating Their Combination Performance Throughout the Construction Process

• Authors: Keito Tajima, Naoki Ichijo, Yuta Nakahara, Toshiyasu Matsushima
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06452
• Pdf URL: https://arxiv.org/pdf/2402.06452
• Copy Paste: [[2402.06452]] An Algorithmic Framework for Constructing Multiple Decision Trees by Evaluating Their Combination Performance Throughout the Construction Process(https://arxiv.org/abs/2402.06452)
• Keywords: rag
• Abstract: Predictions using a combination of decision trees are known to be effective in machine learning. Typical ideas for constructing a combination of decision trees for prediction are bagging and boosting. Bagging independently constructs decision trees without evaluating their combination performance and averages them afterward. Boosting constructs decision trees sequentially, only evaluating a combination performance of a new decision tree and the fixed past decision trees at each step. Therefore, neither method directly constructs nor evaluates a combination of decision trees for the final prediction. When the final prediction is based on a combination of decision trees, it is natural to evaluate the appropriateness of the combination when constructing them. In this study, we propose a new algorithmic framework that constructs decision trees simultaneously and evaluates their combination performance throughout the construction process. Our framework repeats two procedures. In the first procedure, we construct new candidates of combinations of decision trees to find a proper combination of decision trees. In the second procedure, we evaluate each combination performance of decision trees under some criteria and select a better combination. To confirm the performance of the proposed framework, we perform experiments on synthetic and benchmark data.
• 摘要：众所周知，使用决策树组合进行预测在机器学习中是有效的。构建用于预测的决策树组合的典型想法是 bagging 和 boosting。 Bagging 独立构建决策树，无需评估其组合性能，然后对其进行平均。 Boosting按顺序构造决策树，仅在每一步评估新决策树和固定的过去决策树的组合性能。因此，两种方法都不会直接构建或评估最终预测的决策树组合。当最终的预测基于决策树的组合时，在构建决策树时很自然地要评估组合的适当性。在本研究中，我们提出了一种新的算法框架，可以同时构建决策树并在整个构建过程中评估它们的组合性能。我们的框架重复两个过程。在第一个过程中，我们构造新的决策树组合候选，以找到合适的决策树组合。在第二个过程中，我们在某些标准下评估决策树的每个组合性能并选择更好的组合。为了确认所提出框架的性能，我们对合成数据和基准数据进行了实验。

Title: V-STaR: Training Verifiers for Self-Taught Reasoners

• Authors: Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, Rishabh Agarwal
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06457
• Pdf URL: https://arxiv.org/pdf/2402.06457
• Copy Paste: [[2402.06457]] V-STaR: Training Verifiers for Self-Taught Reasoners(https://arxiv.org/abs/2402.06457)
• Keywords: language model, llm, code
• Abstract: Common self-improvement approaches for large language models (LLMs), such as STaR (Zelikman et al., 2022), iteratively fine-tune LLMs on self-generated solutions to improve their problem-solving ability. However, these approaches discard the large amounts of incorrect solutions generated during this process, potentially neglecting valuable information in such solutions. To address this shortcoming, we propose V-STaR that utilizes both the correct and incorrect solutions generated during the self-improvement process to train a verifier using DPO that judges correctness of model-generated solutions. This verifier is used at inference time to select one solution among many candidate solutions. Running V-STaR for multiple iterations results in progressively better reasoners and verifiers, delivering a 4% to 17% test accuracy improvement over existing self-improvement and verification approaches on common code generation and math reasoning benchmarks with LLaMA2 models.
• 摘要：大型语言模型 (LLM) 的常见自我改进方法，例如 STaR (Zelikman et al., 2022)，会在自我生成的解决方案上迭代微调 LLM，以提高其解决问题的能力。然而，这些方法丢弃了在此过程中生成的大量不正确的解决方案，可能忽略了此类解决方案中的有价值的信息。为了解决这个缺点，我们提出了 V-STaR，它利用自我改进过程中生成的正确和错误的解决方案来训练使用 DPO 的验证器，以判断模型生成的解决方案的正确性。该验证器用于在推理时从许多候选解决方案中选择一个解决方案。运行 V-STaR 进行多次迭代会产生逐渐更好的推理器和验证器，与使用 LLaMA2 模型的常见代码生成和数学推理基准的现有自我改进和验证方法相比，测试精度提高了 4% 到 17%。

Title: Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings

• Authors: Yichen Jiang, Xiang Zhou, Mohit Bansal
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06492
• Pdf URL: https://arxiv.org/pdf/2402.06492
• Copy Paste: [[2402.06492]] Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings(https://arxiv.org/abs/2402.06492)
• Keywords: code, rag
• Abstract: Transformers generalize to novel compositions of structures and entities after being trained on a complex dataset, but easily overfit on datasets of insufficient complexity. We observe that when the training set is sufficiently complex, the model encodes sentences that have a common syntactic structure using a systematic attention pattern. Inspired by this observation, we propose SQ-Transformer (Structurally Quantized) that explicitly encourages systematicity in the embeddings and attention layers, even with a training set of low complexity. At the embedding level, we introduce Structure-oriented Vector Quantization (SoVQ) to cluster word embeddings into several classes of structurally equivalent entities. At the attention level, we devise the Systematic Attention Layer (SAL) and an alternative, Systematically Regularized Layer (SRL) that operate on the quantized word embeddings so that sentences of the same structure are encoded with invariant or similar attention patterns. Empirically, we show that SQ-Transformer achieves stronger compositional generalization than the vanilla Transformer on multiple low-complexity semantic parsing and machine translation datasets. In our analysis, we show that SoVQ indeed learns a syntactically clustered embedding space and SAL/SRL induces generalizable attention patterns, which lead to improved systematicity.
• 摘要：在复杂数据集上进行训练后，Transformer 可以推广到结构和实体的新颖组合，但很容易在复杂性不足的数据集上过度拟合。我们观察到，当训练集足够复杂时，模型使用系统注意模式对具有共同句法结构的句子进行编码。受这一观察的启发，我们提出了 SQ-Transformer（结构量化），它明确鼓励嵌入和注意力层的系统性，即使训练集复杂度较低。在嵌入层面，我们引入了面向结构的矢量量化（SoVQ），将词嵌入聚类为几类结构等效的实体。在注意力层面，我们设计了系统注意力层（SAL）和替代的系统正则化层（SRL），它们对量化的词嵌入进行操作，以便使用不变或相似的注意力模式对相同结构的句子进行编码。根据经验，我们表明 SQ-Transformer 在多个低复杂度语义解析和机器翻译数据集上比普通 Transformer 实现了更强的组合泛化。在我们的分析中，我们表明 SoVQ 确实学习了句法聚类的嵌入空间，并且 SAL/SRL 诱导了可泛化的注意力模式，从而提高了系统性。

Title: On the Fly Detection of Root Causes from Observed Data with Application to IT Systems

• Authors: Lei Zan, Charles K. Assaad, Emilie Devijver, Eric Gaussier
• Subjects: cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06500
• Pdf URL: https://arxiv.org/pdf/2402.06500
• Copy Paste: [[2402.06500]] On the Fly Detection of Root Causes from Observed Data with Application to IT Systems(https://arxiv.org/abs/2402.06500)
• Keywords: rag, agent
• Abstract: This paper introduces a new structural causal model tailored for representing threshold-based IT systems and presents a new algorithm designed to rapidly detect root causes of anomalies in such systems. When root causes are not causally related, the method is proven to be correct; while an extension is proposed based on the intervention of an agent to relax this assumption. Our algorithm and its agent-based extension leverage causal discovery from offline data and engage in subgraph traversal when encountering new anomalies in online data. Our extensive experiments demonstrate the superior performance of our methods, even when applied to data generated from alternative structural causal models or real IT monitoring data.
• 摘要：本文介绍了一种专为表示基于阈值的 IT 系统而定制的新结构因果模型，并提出了一种旨在快速检测此类系统中异常的根本原因的新算法。当根本原因不存在因果关系时，该方法被证明是正确的；而基于代理人的干预提出了扩展以放宽这一假设。我们的算法及其基于代理的扩展利用离线数据的因果发现，并在遇到在线数据中的新异常时进行子图遍历。我们广泛的实验证明了我们的方法的卓越性能，即使应用于从替代结构因果模型或真实 IT 监控数据生成的数据。

Title: Scalable Interactive Machine Learning for Future Command and Control

• Authors: Anna Madison, Ellen Novoseller, Vinicius G. Goecks, Benjamin T. Files, Nicholas Waytowich, Alfred Yu, Vernon J. Lawhern, Steven Thurman, Christopher Kelshaw, Kaleb McDowell
• Subjects: cs.LG, cs.AI, cs.CL, cs.HC
• Abstract URL: https://arxiv.org/abs/2402.06501
• Pdf URL: https://arxiv.org/pdf/2402.06501
• Copy Paste: [[2402.06501]] Scalable Interactive Machine Learning for Future Command and Control(https://arxiv.org/abs/2402.06501)
• Keywords: rag
• Abstract: Future warfare will require Command and Control (C2) personnel to make decisions at shrinking timescales in complex and potentially ill-defined situations. Given the need for robust decision-making processes and decision-support tools, integration of artificial and human intelligence holds the potential to revolutionize the C2 operations process to ensure adaptability and efficiency in rapidly changing operational environments. We propose to leverage recent promising breakthroughs in interactive machine learning, in which humans can cooperate with machine learning algorithms to guide machine learning algorithm behavior. This paper identifies several gaps in state-of-the-art science and technology that future work should address to extend these approaches to function in complex C2 contexts. In particular, we describe three research focus areas that together, aim to enable scalable interactive machine learning (SIML): 1) developing human-AI interaction algorithms to enable planning in complex, dynamic situations; 2) fostering resilient human-AI teams through optimizing roles, configurations, and trust; and 3) scaling algorithms and human-AI teams for flexibility across a range of potential contexts and situations.
• 摘要：未来的战争将要求指挥和控制（C2）人员在复杂且可能不明确的情况下在缩短的时间内做出决策。鉴于对稳健决策流程和决策支持工具的需求，人工智能和人类智能的集成有可能彻底改变 C2 操作流程，以确保快速变化的操作环境中的适应性和效率。我们建议利用交互式机器学习领域最近有希望的突破，人类可以与机器学习算法合作来指导机器学习算法的行为。本文指出了最先进科学技术中的几个差距，未来的工作应该解决这些差距，以扩展这些方法在复杂的 C2 环境中发挥作用。我们特别描述了三个研究重点领域，这些领域共同致力于实现可扩展的交互式机器学习（SIML）：1）开发人机交互算法以实现复杂动态情况下的规划； 2）通过优化角色、配置和信任来培养有弹性的人类人工智能团队； 3）扩展算法和人类人工智能团队，以在各种潜在的环境和情况下实现灵活性。

Title: Multimodal Clinical Trial Outcome Prediction with Large Language Models

• Authors: Wenhao Zheng, Dongsheng Peng, Hongxia Xu, Hongtu Zhu, Tianfan Fu, Huaxiu Yao
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06512
• Pdf URL: https://arxiv.org/pdf/2402.06512
• Copy Paste: [[2402.06512]] Multimodal Clinical Trial Outcome Prediction with Large Language Models(https://arxiv.org/abs/2402.06512)
• Keywords: language model, code, rag
• Abstract: The clinical trial is a pivotal and costly process, often spanning multiple years and requiring substantial financial resources. Therefore, the development of clinical trial outcome prediction models aims to exclude drugs likely to fail and holds the potential for significant cost savings. Recent data-driven attempts leverage deep learning methods to integrate multimodal data for predicting clinical trial outcomes. However, these approaches rely on manually designed modal-specific encoders, which limits both the extensibility to adapt new modalities and the ability to discern similar information patterns across different modalities. To address these issues, we propose a multimodal mixture-of-experts (LIFTED) approach for clinical trial outcome prediction. Specifically, LIFTED unifies different modality data by transforming them into natural language descriptions. Then, LIFTED constructs unified noise-resilient encoders to extract information from modal-specific language descriptions. Subsequently, a sparse Mixture-of-Experts framework is employed to further refine the representations, enabling LIFTED to identify similar information patterns across different modalities and extract more consistent representations from those patterns using the same expert model. Finally, a mixture-of-experts module is further employed to dynamically integrate different modality representations for prediction, which gives LIFTED the ability to automatically weigh different modalities and pay more attention to critical information. The experiments demonstrate that LIFTED significantly enhances performance in predicting clinical trial outcomes across all three phases compared to the best baseline, showcasing the effectiveness of our proposed key components.
• 摘要：临床试验是一个关键且成本高昂的过程，通常跨越多年，并且需要大量的财政资源。因此，临床试验结果预测模型的开发旨在排除可能失败的药物，并具有显着节省成本的潜力。最近的数据驱动尝试利用深度学习方法来整合多模态数据来预测临床试验结果。然而，这些方法依赖于手动设计的特定于模态的编码器，这限制了适应新模态的可扩展性以及跨不同模态识别相似信息模式的能力。为了解决这些问题，我们提出了一种用于临床试验结果预测的多模式专家混合 (LIFTED) 方法。具体来说，LIFTED 通过将不同的模态数据转换为自然语言描述来统一它们。然后，LIFTED 构建统一的抗噪声编码器，以从特定于模态的语言描述中提取信息。随后，采用稀疏专家混合框架来进一步细化表示，使 LIFTED 能够识别不同模态的相似信息模式，并使用相同的专家模型从这些模式中提取更一致的表示。最后，进一步采用混合专家模块来动态集成不同模态表示进行预测，这使得 LIFTED 能够自动权衡不同模态并更加关注关键信息。实验表明，与最佳基线相比，LIFTED 显着提高了预测所有三个阶段的临床试验结果的性能，展示了我们提出的关键组件的有效性。

Title: Introspective Planning: Guiding Language-Enabled Agents to Refine Their Own Uncertainty

• Authors: Kaiqu Liang, Zixu Zhang, Jaime Fernández Fisac
• Subjects: cs.AI, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06529
• Pdf URL: https://arxiv.org/pdf/2402.06529
• Copy Paste: [[2402.06529]] Introspective Planning: Guiding Language-Enabled Agents to Refine Their Own Uncertainty(https://arxiv.org/abs/2402.06529)
• Keywords: language model, llm, hallucination, agent
• Abstract: Large language models (LLMs) exhibit advanced reasoning skills, enabling robots to comprehend natural language instructions and strategically plan high-level actions through proper grounding. However, LLM hallucination may result in robots confidently executing plans that are misaligned with user goals or, in extreme cases, unsafe. Additionally, inherent ambiguity in natural language instructions can induce task uncertainty, particularly in situations where multiple valid options exist. To address this issue, LLMs must identify such uncertainty and proactively seek clarification. This paper explores the concept of introspective planning as a systematic method for guiding LLMs in forming uncertainty--aware plans for robotic task execution without the need for fine-tuning. We investigate uncertainty quantification in task-level robot planning and demonstrate that introspection significantly improves both success rates and safety compared to state-of-the-art LLM-based planning approaches. Furthermore, we assess the effectiveness of introspective planning in conjunction with conformal prediction, revealing that this combination yields tighter confidence bounds, thereby maintaining statistical success guarantees with fewer superfluous user clarification queries.
• 摘要：大型语言模型 (LLM) 展现出先进的推理技能，使机器人能够理解自然语言指令，并通过适当的基础来战略性地规划高级行动。然而，LLM 幻觉可能会导致机器人自信地执行与用户目标不一致的计划，或者在极端情况下是不安全的。此外，自然语言指令中固有的歧义可能会导致任务的不确定性，特别是在存在多个有效选项的情况下。为了解决这个问题，法学硕士必须识别这种不确定性并主动寻求澄清。本文探讨了内省计划的概念，作为一种系统方法，用于指导法学硕士形成机器人任务执行的不确定性感知计划，而无需进行微调。我们研究了任务级机器人规划中的不确定性量化，并证明与最先进的基于 LLM 的规划方法相比，内省显着提高了成功率和安全性。此外，我们评估了内省规划与保形预测相结合的有效性，表明这种组合产生更严格的置信界限，从而通过更少的多余用户澄清查询来维持统计成功保证。

Title: Generative Adversarial Bayesian Optimization for Surrogate Objectives

• Authors: Michael S. Yao, Yimeng Zeng, Hamsa Bastani, Jacob Gardner, James C. Gee, Osbert Bastani
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06532
• Pdf URL: https://arxiv.org/pdf/2402.06532
• Copy Paste: [[2402.06532]] Generative Adversarial Bayesian Optimization for Surrogate Objectives(https://arxiv.org/abs/2402.06532)
• Keywords: code
• Abstract: Offline model-based policy optimization seeks to optimize a learned surrogate objective function without querying the true oracle objective during optimization. However, inaccurate surrogate model predictions are frequently encountered along the optimization trajectory. To address this limitation, we propose generative adversarial Bayesian optimization (GABO) using adaptive source critic regularization, a task-agnostic framework for Bayesian optimization that employs a Lipschitz-bounded source critic model to constrain the optimization trajectory to regions where the surrogate function is reliable. We show that under certain assumptions for the continuous input space prior, our algorithm dynamically adjusts the strength of the source critic regularization. GABO outperforms existing baselines on a number of different offline optimization tasks across a variety of scientific domains. Our code is available at https://github.com/michael-s-yao/gabo
• 摘要：基于离线模型的策略优化旨在优化学习的代理目标函数，而无需在优化过程中查询真实的预言机目标。然而，在优化轨迹上经常会遇到不准确的替代模型预测。为了解决这个限制，我们提出了使用自适应源批评正则化的生成对抗贝叶斯优化（GABO），这是一种与任务无关的贝叶斯优化框架，它采用 Lipschitz 有界源批评模型将优化轨迹限制在代理函数可靠的区域。我们表明，在连续输入空间先验的某些假设下，我们的算法动态调整源批评正则化的强度。 GABO 在跨多个科学领域的许多不同离线优化任务上均优于现有基线。我们的代码位于 https://github.com/michael-s-yao/gabo

Title: Calibrating Long-form Generations from Large Language Models

• Authors: Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru, Arman Cohan, Bhuwan Dhingra
• Subjects: cs.CL, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06544
• Pdf URL: https://arxiv.org/pdf/2402.06544
• Copy Paste: [[2402.06544]] Calibrating Long-form Generations from Large Language Models(https://arxiv.org/abs/2402.06544)
• Keywords: language model, gpt, llm, chat
• Abstract: To enhance Large Language Models' (LLMs) reliability, calibration is essential -- the model's assessed confidence scores should align with the actual likelihood of its responses being correct. However, current confidence elicitation methods and calibration metrics typically rely on a binary true/false assessment of response correctness. This approach does not apply to long-form generation, where an answer can be partially correct. Addressing this gap, we introduce a unified calibration framework, in which both the correctness of the LLMs' responses and their associated confidence levels are treated as distributions across a range of scores. Within this framework, we develop three metrics to precisely evaluate LLM calibration and further propose two confidence elicitation methods based on self-consistency and self-evaluation. Our experiments, which include long-form QA and summarization tasks, demonstrate that larger models don't necessarily guarantee better calibration, that calibration performance is found to be metric-dependent, and that self-consistency methods excel in factoid datasets. We also find that calibration can be enhanced through techniques such as fine-tuning, integrating relevant source documents, scaling the temperature, and combining self-consistency with self-evaluation. Lastly, we showcase a practical application of our system: selecting and cascading open-source models and ChatGPT to optimize correctness given a limited API budget. This research not only challenges existing notions of LLM calibration but also offers practical methodologies for improving trustworthiness in long-form generation.
• 摘要：为了提高大型语言模型 (LLM) 的可靠性，校准至关重要——模型评估的置信度分数应与其响应正确的实际可能性保持一致。然而，当前的置信度启发方法和校准指标通常依赖于响应正确性的二元真/假评估。此方法不适用于长格式生成，其中答案可能部分正确。为了解决这一差距，我们引入了一个统一的校准框架，其中法学硕士回答的正确性及其相关的置信水平都被视为一系列分数的分布。在此框架内，我们开发了三个指标来精确评估LLM校准，并进一步提出了两种基于自我一致性和自我评估的置信度启发方法。我们的实验（包括长格式的 QA 和总结任务）表明，较大的模型不一定能保证更好的校准，校准性能被发现与度量相关，并且自洽方法在事实数据集中表现出色。我们还发现，可以通过微调、整合相关源文档、缩放温度以及将自我一致性与自我评估相结合等技术来增强校准。最后，我们展示了我们系统的实际应用：选择并级联开源模型和 ChatGPT，以在 API 预算有限的情况下优化正确性。这项研究不仅挑战了 LLM 校准的现有概念，而且还提供了提高长格式生成可信度的实用方法。

Title: Bryndza at ClimateActivism 2024: Stance, Target and Hate Event Detection via Retrieval-Augmented GPT-4 and LLaMA

• Authors: Marek Šuppa, Daniel Skala, Daniela Jašš, Samuel Sučík, Andrej Švec, Peter Hraška
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06549
• Pdf URL: https://arxiv.org/pdf/2402.06549
• Copy Paste: [[2402.06549]] Bryndza at ClimateActivism 2024: Stance, Target and Hate Event Detection via Retrieval-Augmented GPT-4 and LLaMA(https://arxiv.org/abs/2402.06549)
• Keywords: language model, gpt, llm, code
• Abstract: This study details our approach for the CASE 2024 Shared Task on Climate Activism Stance and Hate Event Detection, focusing on Hate Speech Detection, Hate Speech Target Identification, and Stance Detection as classification challenges. We explored the capability of Large Language Models (LLMs), particularly GPT-4, in zero- or few-shot settings enhanced by retrieval augmentation and re-ranking for Tweet classification. Our goal was to determine if LLMs could match or surpass traditional methods in this context. We conducted an ablation study with LLaMA for comparison, and our results indicate that our models significantly outperformed the baselines, securing second place in the Target Detection task. The code for our submission is available at https://github.com/NaiveNeuron/bryndza-case-2024
• 摘要：本研究详细介绍了我们针对气候激进主义立场和仇恨事件检测的 CASE 2024 共享任务的方法，重点关注仇恨言论检测、仇恨言论目标识别和立场检测作为分类挑战。我们探索了大型语言模型 (LLM)（特别是 GPT-4）在零次或少量设置中的功能，通过检索增强和推文分类重新排名来增强。我们的目标是确定法学硕士是否可以在这方面匹配或超越传统方法。我们与 LLaMA 进行了一项消融研究进行比较，结果表明我们的模型显着优于基线，在目标检测任务中获得第二名。我们提交的代码位于 https://github.com/NaiveNeuron/bryndza-case-2024

Title: Deceptive Path Planning via Reinforcement Learning with Graph Neural Networks

• Authors: Michael Y. Fatemi, Wesley A. Suttle, Brian M. Sadler
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06552
• Pdf URL: https://arxiv.org/pdf/2402.06552
• Copy Paste: [[2402.06552]] Deceptive Path Planning via Reinforcement Learning with Graph Neural Networks(https://arxiv.org/abs/2402.06552)
• Keywords: agent
• Abstract: Deceptive path planning (DPP) is the problem of designing a path that hides its true goal from an outside observer. Existing methods for DPP rely on unrealistic assumptions, such as global state observability and perfect model knowledge, and are typically problem-specific, meaning that even minor changes to a previously solved problem can force expensive computation of an entirely new solution. Given these drawbacks, such methods do not generalize to unseen problem instances, lack scalability to realistic problem sizes, and preclude both on-the-fly tunability of deception levels and real-time adaptivity to changing environments. In this paper, we propose a reinforcement learning (RL)-based scheme for training policies to perform DPP over arbitrary weighted graphs that overcomes these issues. The core of our approach is the introduction of a local perception model for the agent, a new state space representation distilling the key components of the DPP problem, the use of graph neural network-based policies to facilitate generalization and scaling, and the introduction of new deception bonuses that translate the deception objectives of classical methods to the RL setting. Through extensive experimentation we show that, without additional fine-tuning, at test time the resulting policies successfully generalize, scale, enjoy tunable levels of deception, and adapt in real-time to changes in the environment.
• 摘要：欺骗性路径规划（DPP）是设计一条向外部观察者隐藏其真实目标的路径的问题。现有的 DPP 方法依赖于不切实际的假设，例如全局状态可观测性和完美的模型知识，并且通常是特定于问题的，这意味着即使对先前解决的问题进行微小的更改，也可能会迫使对全新的解决方案进行昂贵的计算。鉴于这些缺点，此类方法不能推广到未见过的问题实例，缺乏对实际问题规模的可扩展性，并且排除了欺骗级别的动态可调性和对不断变化的环境的实时适应性。在本文中，我们提出了一种基于强化学习 (RL) 的训练策略方案，以在任意加权图上执行 DPP，从而克服这些问题。我们方法的核心是引入代理的局部感知模型、提取 DPP 问题关键组成部分的新状态空间表示、使用基于图神经网络的策略来促进泛化和扩展，以及引入新的欺骗奖励将经典方法的欺骗目标转化为强化学习设置。通过广泛的实验，我们表明，在没有额外微调的情况下，在测试时产生的策略成功地概括、扩展、享受可调节的欺骗水平，并实时适应环境的变化。

Title: The Quantified Boolean Bayesian Network: Theory and Experiments with a Logical Graphical Model

• Authors: Gregory Coppola
• Subjects: cs.AI, cs.IR
• Abstract URL: https://arxiv.org/abs/2402.06557
• Pdf URL: https://arxiv.org/pdf/2402.06557
• Copy Paste: [[2402.06557]] The Quantified Boolean Bayesian Network: Theory and Experiments with a Logical Graphical Model(https://arxiv.org/abs/2402.06557)
• Keywords: language model, llm
• Abstract: This paper introduces the Quantified Boolean Bayesian Network (QBBN), which provides a unified view of logical and probabilistic reasoning. The QBBN is meant to address a central problem with the Large Language Model (LLM), which has become extremely popular in Information Retrieval, which is that the LLM hallucinates. A Bayesian Network, by construction, cannot hallucinate, because it can only return answers that it can explain. We show how a Bayesian Network over an unbounded number of boolean variables can be configured to represent the logical reasoning underlying human language. We do this by creating a key-value version of the First-Order Calculus, for which we can prove consistency and completeness. We show that the model is trivially trained over fully observed data, but that inference is non-trivial. Exact inference in a Bayesian Network is intractable (i.e. $\Omega(2^N)$ for $N$ variables). For inference, we investigate the use of Loopy Belief Propagation (LBP), which is not guaranteed to converge, but which has been shown to often converge in practice. Our experiments show that LBP indeed does converge very reliably, and our analysis shows that a round of LBP takes time $O(N2^n)$, where $N$ bounds the number of variables considered, and $n$ bounds the number of incoming connections to any factor, and further improvements may be possible. Our network is specifically designed to alternate between AND and OR gates in a Boolean Algebra, which connects more closely to logical reasoning, allowing a completeness proof for an expanded version of our network, and also allows inference to follow specific but adequate pathways, that turn out to be fast.
• 摘要：本文介绍了量化布尔贝叶斯网络（QBBN），它提供了逻辑和概率推理的统一视图。 QBBN 旨在解决大型语言模型 (LLM) 的一个核心问题，该模型在信息检索中非常流行，即 LLM 的幻觉。从结构上看，贝叶斯网络不会产生幻觉，因为它只能返回它可以解释的答案。我们展示了如何配置无限数量布尔变量上的贝叶斯网络来表示人类语言背后的逻辑推理。我们通过创建一阶微积分的键值版本来做到这一点，我们可以证明它的一致性和完整性。我们表明，该模型是在完全观察到的数据上进行的简单训练，但推理并非简单。贝叶斯网络中的精确推理很棘手（即 $\Omega(2^N)$ 对于 $N$ 变量）。为了进行推理，我们研究了循环置信传播（LBP）的使用，它不能保证收敛，但在实践中已被证明经常收敛。我们的实验表明，LBP 确实非常可靠地收敛，并且我们的分析表明，一轮 LBP 需要时间 $O(N2^n)$，其中 $N$ 限制了考虑的变量数量，$n$ 限制了所考虑变量的数量。任何因素的传入连接，并且可能有进一步的改进。我们的网络专门设计用于在布尔代数中的“与”门和“或”门之间交替，这与逻辑推理联系更紧密，允许为我们网络的扩展版本提供完整性证明，并且还允许推理遵循特定但充分的路径，从而将出去要快。

Title: Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following

• Authors: Brian Yang, Huangyuan Su, Nikolaos Gkanatsios, Tsung-Wei Ke, Ayush Jain, Jeff Schneider, Katerina Fragkiadaki
• Subjects: cs.LG, cs.AI, cs.CL, cs.RO
• Abstract URL: https://arxiv.org/abs/2402.06559
• Pdf URL: https://arxiv.org/pdf/2402.06559
• Copy Paste: [[2402.06559]] Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following(https://arxiv.org/abs/2402.06559)
• Keywords: llm, lora, prompt
• Abstract: Diffusion models excel at modeling complex and multimodal trajectory distributions for decision-making and control. Reward-gradient guided denoising has been recently proposed to generate trajectories that maximize both a differentiable reward function and the likelihood under the data distribution captured by a diffusion model. Reward-gradient guided denoising requires a differentiable reward function fitted to both clean and noised samples, limiting its applicability as a general trajectory optimizer. In this paper, we propose DiffusionES, a method that combines gradient-free optimization with trajectory denoising to optimize black-box non-differentiable objectives while staying in the data manifold. Diffusion-ES samples trajectories during evolutionary search from a diffusion model and scores them using a black-box reward function. It mutates high-scoring trajectories using a truncated diffusion process that applies a small number of noising and denoising steps, allowing for much more efficient exploration of the solution space. We show that DiffusionES achieves state-of-the-art performance on nuPlan, an established closed-loop planning benchmark for autonomous driving. Diffusion-ES outperforms existing sampling-based planners, reactive deterministic or diffusion-based policies, and reward-gradient guidance. Additionally, we show that unlike prior guidance methods, our method can optimize non-differentiable language-shaped reward functions generated by few-shot LLM prompting. When guided by a human teacher that issues instructions to follow, our method can generate novel, highly complex behaviors, such as aggressive lane weaving, which are not present in the training data. This allows us to solve the hardest nuPlan scenarios which are beyond the capabilities of existing trajectory optimization methods and driving policies.
• 摘要：扩散模型擅长对复杂的多模态轨迹分布进行建模，以进行决策和控制。最近提出了奖励梯度引导去噪，以生成最大化可微奖励函数和扩散模型捕获的数据分布下的可能性的轨迹。奖励梯度引导去噪需要一个适合干净样本和噪声样本的可微奖励函数，限制了其作为通用轨迹优化器的适用性。在本文中，我们提出了 DiffusionES，一种将无梯度优化与轨迹去噪相结合的方法，以优化黑盒不可微目标，同时保持在数据流形中。 Diffusion-ES 在进化搜索过程中从扩散模型中采样轨迹，并使用黑盒奖励函数对其进行评分。它使用截断的扩散过程来改变高分轨迹，该过程应用少量的噪声和去噪步骤，从而可以更有效地探索解决方案空间。我们展示了 DiffusionES 在 nuPlan 上实现了最先进的性能，nuPlan 是一个既定的自动驾驶闭环规划基准。 Diffusion-ES 优于现有的基于采样的规划器、反应性确定性或基于扩散的策略以及奖励梯度指导。此外，我们表明，与之前的指导方法不同，我们的方法可以优化由少样本 LLM 提示生成的不可微分的语言形状奖励函数。当由人类老师发出指令进行指导时，我们的方法可以生成新颖的、高度复杂的行为，例如训练数据中不存在的攻击性车道编织。这使我们能够解决最困难的 nuPlan 场景，这些场景超出了现有轨迹优化方法和驾驶策略的能力。

Title: SAE: Single Architecture Ensemble Neural Networks

• Authors: Martin Ferianc, Hongxiang Fan, Miguel Rodrigues
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2402.06580
• Pdf URL: https://arxiv.org/pdf/2402.06580
• Copy Paste: [[2402.06580]] SAE: Single Architecture Ensemble Neural Networks(https://arxiv.org/abs/2402.06580)
• Keywords: rag
• Abstract: Ensembles of separate neural networks (NNs) have shown superior accuracy and confidence calibration over single NN across tasks. Recent methods compress ensembles within a single network via early exits or multi-input multi-output frameworks. However, the landscape of these methods is fragmented thus far, making it difficult to choose the right approach for a given task. Furthermore, the algorithmic performance of these methods is behind the ensemble of separate NNs and requires extensive architecture tuning. We propose a novel methodology unifying these approaches into a Single Architecture Ensemble (SAE). Our method learns the optimal number and depth of exits per ensemble input in a single NN. This enables the SAE framework to flexibly tailor its configuration for a given architecture or application. We evaluate SAEs on image classification and regression across various network architecture types and sizes. We demonstrate competitive accuracy or confidence calibration to baselines while reducing the compute operations or parameter count by up to $1.5{\sim}3.7\times$.
• 摘要：独立神经网络 (NN) 的集成在跨任务中表现出比单个神经网络更高的准确性和置信度校准。最近的方法通过早期退出或多输入多输出框架来压缩单个网络内的集成。然而，到目前为止，这些方法的情况还很分散，因此很难为给定的任务选择正确的方法。此外，这些方法的算法性能落后于单独的神经网络的集合，并且需要大量的架构调整。我们提出了一种新颖的方法，将这些方法统一到单一架构集成（SAE）中。我们的方法学习单个神经网络中每个集合输入的最佳出口数量和深度。这使得 SAE 框架能够针对给定的架构或应用程序灵活地定制其配置。我们评估各种网络架构类型和规模的图像分类和回归的 SAE。我们展示了对基线的有竞争力的准确性或置信度校准，同时将计算操作或参数数量减少了高达 $1.5{\sim}3.7\times$。

Title: G-SciEdBERT: A Contextualized LLM for Science Assessment Tasks in German

• Authors: Ehsan Latif, Gyeong-Geon Lee, Knut Neuman, Tamara Kastorff, Xiaoming Zhai
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06584
• Pdf URL: https://arxiv.org/pdf/2402.06584
• Copy Paste: [[2402.06584]] G-SciEdBERT: A Contextualized LLM for Science Assessment Tasks in German(https://arxiv.org/abs/2402.06584)
• Keywords: language model, llm
• Abstract: The advancement of natural language processing has paved the way for automated scoring systems in various languages, such as German (e.g., German BERT [G-BERT]). Automatically scoring written responses to science questions in German is a complex task and challenging for standard G-BERT as they lack contextual knowledge in the science domain and may be unaligned with student writing styles. This paper developed a contextualized German Science Education BERT (G-SciEdBERT), an innovative large language model tailored for scoring German-written responses to science tasks. Using G-BERT, we pre-trained G-SciEdBERT on a corpus of 50K German written science responses with 5M tokens to the Programme for International Student Assessment (PISA) 2015. We fine-tuned G-SciEdBERT on 59 assessment items and examined the scoring accuracy. We then compared its performance with G-BERT. Our findings reveal a substantial improvement in scoring accuracy with G-SciEdBERT, demonstrating a 10% increase of quadratic weighted kappa compared to G-BERT (mean accuracy difference = 0.096, SD = 0.024). These insights underline the significance of specialized language models like G-SciEdBERT, which is trained to enhance the accuracy of automated scoring, offering a substantial contribution to the field of AI in education.
• 摘要：自然语言处理的进步为各种语言的自动评分系统铺平了道路，例如德语（例如德语 BERT [G-BERT]）。自动对德语科学问题的书面回答进行评分对于标准 G-BERT 来说是一项复杂的任务和挑战，因为它们缺乏科学领域的上下文知识，并且可能与学生的写作风格不一致。本文开发了一种情境化的德国科学教育 BERT (G-SciEdBERT)，这是一种创新的大型语言模型，专为对德国人对科学任务的书面回答进行评分而量身定制。使用 G-BERT，我们在 50K 个德国书面科学回答语料库上对 G-SciEdBERT 进行了预训练，并为 2015 年国际学生评估计划 (PISA) 提供了 500 万个代币。我们在 59 个评估项目上对 G-SciEdBERT 进行了微调，并检查了评分准确度。然后我们将其性能与 G-BERT 进行了比较。我们的研究结果表明，G-SciEdBERT 的评分准确性有了显着提高，与 G-BERT 相比，二次加权 kappa 增加了 10%（平均准确性差异 = 0.096，SD = 0.024）。这些见解强调了 G-SciEdBERT 等专门语言模型的重要性，该模型经过训练可以提高自动评分的准确性，为教育人工智能领域做出重大贡献。

Title: Self-consistent context aware conformer transducer for speech recognition

• Authors: Konstantin Kolokolov, Pavel Pekichev, Karthik Raghunathan
• Subjects: cs.CL, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2402.06592
• Pdf URL: https://arxiv.org/pdf/2402.06592
• Copy Paste: [[2402.06592]] Self-consistent context aware conformer transducer for speech recognition(https://arxiv.org/abs/2402.06592)
• Keywords: language model
• Abstract: We propose a novel neural network architecture based on conformer transducer that adds contextual information flow to the ASR systems. Our method improves the accuracy of recognizing uncommon words while not harming the word error rate of regular words. We explore the uncommon words accuracy improvement when we use the new model and/or shallow fusion with context language model. We found that combination of both provides cumulative gain in uncommon words recognition accuracy.
• 摘要：我们提出了一种基于构象传感器的新型神经网络架构，为 ASR 系统添加了上下文信息流。我们的方法提高了识别不常见单词的准确性，同时不损害常规单词的错误率。当我们使用新模型和/或与上下文语言模型浅层融合时，我们探索不常见单词准确性的提高。我们发现两者的结合可以提供不常见单词识别准确性的累积增益。

Title: Understanding the Weakness of Large Language Model Agents within a Complex Android Environment

• Authors: Mingzhe Xing, Rongkai Zhang, Hui Xue, Qi Chen, Fan Yang, Zhen Xiao
• Subjects: cs.AI, cs.HC, cs.SE
• Abstract URL: https://arxiv.org/abs/2402.06596
• Pdf URL: https://arxiv.org/pdf/2402.06596
• Copy Paste: [[2402.06596]] Understanding the Weakness of Large Language Model Agents within a Complex Android Environment(https://arxiv.org/abs/2402.06596)
• Keywords: language model, llm, lora, code, agent
• Abstract: Large language models (LLMs) have empowered intelligent agents to execute intricate tasks within domain-specific software such as browsers and games. However, when applied to general-purpose software systems like operating systems, LLM agents face three primary challenges. Firstly, the action space is vast and dynamic, posing difficulties for LLM agents to maintain an up-to-date understanding and deliver accurate responses. Secondly, real-world tasks often require inter-application cooperation}, demanding farsighted planning from LLM agents. Thirdly, agents need to identify optimal solutions aligning with user constraints, such as security concerns and preferences. These challenges motivate AndroidArena, an environment and benchmark designed to evaluate LLM agents on a modern operating system. To address high-cost of manpower, we design a scalable and semi-automated method to construct the benchmark. In the task evaluation, AndroidArena incorporates accurate and adaptive metrics to address the issue of non-unique solutions. Our findings reveal that even state-of-the-art LLM agents struggle in cross-APP scenarios and adhering to specific constraints. Additionally, we identify a lack of four key capabilities, i.e., understanding, reasoning, exploration, and reflection, as primary reasons for the failure of LLM agents. Furthermore, we provide empirical analysis on the failure of reflection, and improve the success rate by 27% with our proposed exploration strategy. This work is the first to present valuable insights in understanding fine-grained weakness of LLM agents, and offers a path forward for future research in this area. Environment, benchmark, and evaluation code for AndroidArena are released at https://github.com/AndroidArenaAgent/AndroidArena.
• 摘要：大型语言模型 (LLM) 使智能代理能够在浏览器和游戏等特定领域的软件中执行复杂的任务。然而，当应用于操作系统等通用软件系统时，LLM 代理面临三个主要挑战。首先，行动空间广阔且动态，给法学硕士代理人保持最新的理解和提供准确的反应带来了困难。其次，现实世界的任务通常需要应用程序之间的合作}，需要LLM代理进行有远见的规划。第三，代理需要确定符合用户约束的最佳解决方案，例如安全问题和偏好。这些挑战激发了 AndroidArena 的发展，这是一个旨在评估现代操作系统上的 LLM 代理的环境和基准。为了解决人力成本高的问题，我们设计了一种可扩展的半自动化方法来构建基准。在任务评估中，AndroidArena 结合了准确和自适应的指标来解决非唯一解决方案的问题。我们的研究结果表明，即使是最先进的法学硕士代理人也会在跨 APP 场景中陷入困境并遵守特定的约束。此外，我们认为缺乏四种关键能力，即理解、推理、探索和反思，是LLM代理失败的主要原因。此外，我们对反射失败进行了实证分析，并通过我们提出的探索策略将成功率提高了 27%。这项工作首次在理解法学硕士代理人的细粒度弱点方面提出了宝贵的见解，并为该领域的未来研究提供了前进的道路。 AndroidArena 的环境、基准测试和评估代码发布于 https://github.com/AndroidArenaAgent/AndroidArena。

Title: RQP-SGD: Differential Private Machine Learning through Noisy SGD and Randomized Quantization

• Authors: Ce Feng, Parv Venkitasubramaniam
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2402.06606
• Pdf URL: https://arxiv.org/pdf/2402.06606
• Copy Paste: [[2402.06606]] RQP-SGD: Differential Private Machine Learning through Noisy SGD and Randomized Quantization(https://arxiv.org/abs/2402.06606)
• Keywords: prompt
• Abstract: The rise of IoT devices has prompted the demand for deploying machine learning at-the-edge with real-time, efficient, and secure data processing. In this context, implementing machine learning (ML) models with real-valued weight parameters can prove to be impractical particularly for large models, and there is a need to train models with quantized discrete weights. At the same time, these low-dimensional models also need to preserve privacy of the underlying dataset. In this work, we present RQP-SGD, a new approach for privacy-preserving quantization to train machine learning models for low-memory ML-at-the-edge. This approach combines differentially private stochastic gradient descent (DP-SGD) with randomized quantization, providing a measurable privacy guarantee in machine learning. In particular, we study the utility convergence of implementing RQP-SGD on ML tasks with convex objectives and quantization constraints and demonstrate its efficacy over deterministic quantization. Through experiments conducted on two datasets, we show the practical effectiveness of RQP-SGD.
• 摘要：物联网设备的兴起促使人们需要在边缘部署机器学习，进行实时、高效、安全的数据处理。在这种情况下，使用实值权重参数实现机器学习（ML）模型可能被证明是不切实际的，特别是对于大型模型，并且需要使用量化的离散权重来训练模型。同时，这些低维模型还需要保护底层数据集的隐私。在这项工作中，我们提出了 RQP-SGD，这是一种隐私保护量化的新方法，用于训练低内存边缘 ML 的机器学习模型。这种方法将差分隐私随机梯度下降（DP-SGD）与随机量化相结合，为机器学习提供了可测量的隐私保证。特别是，我们研究了在具有凸目标和量化约束的 ML 任务上实现 RQP-SGD 的效用收敛性，并证明了其相对于确定性量化的有效性。通过在两个数据集上进行的实验，我们展示了 RQP-SGD 的实际有效性。

Title: TIC: Translate-Infer-Compile for accurate 'text to plan' using LLMs and logical intermediate representations

• Authors: Sudhir Agarwal, Anu Sreepathy
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06608
• Pdf URL: https://arxiv.org/pdf/2402.06608
• Copy Paste: [[2402.06608]] TIC: Translate-Infer-Compile for accurate 'text to plan' using LLMs and logical intermediate representations(https://arxiv.org/abs/2402.06608)
• Keywords: llm, rag
• Abstract: We study the problem of generating plans for given natural language planning task requests. On one hand, LLMs excel at natural language processing but do not perform well on planning. On the other hand, classical planning tools excel at planning tasks but require input in a structured language such as the Planning Domain Definition Language (PDDL). We leverage the strengths of both the techniques by using an LLM for generating the PDDL representation (task PDDL) of planning task requests followed by using a classical planner for computing a plan. Unlike previous approaches that use LLMs for generating task PDDLs directly, our approach comprises of (a) translate: using an LLM only for generating a logically interpretable intermediate representation of natural language task descriptions, (b) infer: deriving additional logically dependent information from the intermediate representation using a logic reasoner (currently, Answer Set Programming solver), and (c) compile: generating the target task PDDL from the base and inferred information. We observe that using an LLM to only output the intermediate representation significantly reduces LLM errors. Consequently, TIC approach achieves, for at least one LLM, high accuracy on task PDDL generation for all seven domains of our evaluation dataset.
• 摘要：我们研究为给定的自然语言规划任务请求生成计划的问题。一方面，法学硕士擅长自然语言处理，但在规划方面表现不佳。另一方面，经典规划工具擅长规划任务，但需要使用结构化语言（例如规划域定义语言 (PDDL)）进行输入。我们利用 LLM 生成规划任务请求的 PDDL 表示（任务 PDDL），然后使用经典规划器来计算计划，从而充分利用这两种技术的优势。与之前使用 LLM 直接生成任务 PDDL 的方法不同，我们的方法包括 (a) 翻译：仅使用 LLM 生成自然语言任务描述的逻辑上可解释的中间表示，(b) 推断：从使用逻辑推理器（当前为答案集编程求解器）的中间表示，以及 (c) 编译：根据基本信息和推断信息生成目标任务 PDDL。我们观察到，使用 LLM 仅输出中间表示可以显着减少 LLM 错误。因此，对于至少一个法学硕士，TIC 方法在我们评估数据集的所有七个领域的任务 PDDL 生成上实现了高精度。

Title: FaBERT: Pre-training BERT on Persian Blogs

• Authors: Mostafa Masumi, Seyed Soroush Majd, Mehrnoush Shamsfard, Hamid Beigy
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06617
• Pdf URL: https://arxiv.org/pdf/2402.06617
• Copy Paste: [[2402.06617]] FaBERT: Pre-training BERT on Persian Blogs(https://arxiv.org/abs/2402.06617)
• Keywords: language model
• Abstract: We introduce FaBERT, a Persian BERT-base model pre-trained on the HmBlogs corpus, encompassing both informal and formal Persian texts. FaBERT is designed to excel in traditional Natural Language Understanding (NLU) tasks, addressing the intricacies of diverse sentence structures and linguistic styles prevalent in the Persian language. In our comprehensive evaluation of FaBERT on 12 datasets in various downstream tasks, encompassing Sentiment Analysis (SA), Named Entity Recognition (NER), Natural Language Inference (NLI), Question Answering (QA), and Question Paraphrasing (QP), it consistently demonstrated improved performance, all achieved within a compact model size. The findings highlight the importance of utilizing diverse and cleaned corpora, such as HmBlogs, to enhance the performance of language models like BERT in Persian Natural Language Processing (NLP) applications. FaBERT is openly accessible at https://huggingface.co/sbunlp/fabert
• 摘要：我们引入了 FaBERT，这是一种基于波斯语 BERT 的模型，在 HmBlogs 语料库上进行了预训练，涵盖非正式和正式的波斯语文本。 FaBERT 旨在在传统自然语言理解 (NLU) 任务中表现出色，解决波斯语中流行的多种句子结构和语言风格的复杂性。我们在各种下游任务（包括情感分析（SA）、命名实体识别（NER）、自然语言推理（NLI）、问答（QA）和问题释义（QP））中对 FaBERT 的 12 个数据集进行综合评估，结果一致展示了改进的性能，所有这些都是在紧凑的模型尺寸内实现的。研究结果强调了利用 HmBlogs 等多样化且干净的语料库来增强波斯自然语言处理 (NLP) 应用程序中 BERT 等语言模型性能的重要性。 FaBERT 可在 https://huggingface.co/sbunlp/fabert 上公开访问

Title: Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

• Authors: Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergün, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Minh Chien, Sebastian Ruder, Surya Guthikonda, Emad A. Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, Sara Hooker
• Subjects: cs.CL, cs.AI
• Abstract URL: https://arxiv.org/abs/2402.06619
• Pdf URL: https://arxiv.org/pdf/2402.06619
• Copy Paste: [[2402.06619]] Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning(https://arxiv.org/abs/2402.06619)
• Keywords: language model, llm
• Abstract: Datasets are foundational to many breakthroughs in modern artificial intelligence. Many recent achievements in the space of natural language processing (NLP) can be attributed to the finetuning of pre-trained models on a diverse set of tasks that enables a large language model (LLM) to respond to instructions. Instruction fine-tuning (IFT) requires specifically constructed and annotated datasets. However, existing datasets are almost all in the English language. In this work, our primary goal is to bridge the language gap by building a human-curated instruction-following dataset spanning 65 languages. We worked with fluent speakers of languages from around the world to collect natural instances of instructions and completions. Furthermore, we create the most extensive multilingual collection to date, comprising 513 million instances through templating and translating existing datasets across 114 languages. In total, we contribute four key resources: we develop and open-source the Aya Annotation Platform, the Aya Dataset, the Aya Collection, and the Aya Evaluation Suite. The Aya initiative also serves as a valuable case study in participatory research, involving collaborators from 119 countries. We see this as a valuable framework for future research collaborations that aim to bridge gaps in resources.
• 摘要：数据集是现代人工智能许多突破的基础。自然语言处理 (NLP) 领域的许多最新成就可归因于对各种任务集的预训练模型的微调，使大型语言模型 (LLM) 能够响应指令。指令微调（IFT）需要专门构建和注释的数据集。然而，现有的数据集几乎都是英文的。在这项工作中，我们的主要目标是通过构建涵盖 65 种语言的人工管理的指令跟踪数据集来弥合语言差距。我们与来自世界各地的语言流利的人合作，收集指令和完成的自然实例。此外，我们通过模板化和翻译 114 种语言的现有数据集，创建了迄今为止最广泛的多语言集合，其中包含 5.13 亿个实例。总的来说，我们贡献了四种关键资源：我们开发并开源了 Aya 注释平台、Aya 数据集、Aya 集合和 Aya 评估套件。 Aya 倡议也是参与式研究的一个有价值的案例研究，涉及来自 119 个国家的合作者。我们认为这是未来旨在弥合资源差距的研究合作的一个有价值的框架。

Title: Understanding the Effects of Iterative Prompting on Truthfulness

• Authors: Satyapriya Krishna, Chirag Agarwal, Himabindu Lakkaraju
• Subjects: cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06625
• Pdf URL: https://arxiv.org/pdf/2402.06625
• Copy Paste: [[2402.06625]] Understanding the Effects of Iterative Prompting on Truthfulness(https://arxiv.org/abs/2402.06625)
• Keywords: language model, llm, prompt
• Abstract: The development of Large Language Models (LLMs) has notably transformed numerous sectors, offering impressive text generation capabilities. Yet, the reliability and truthfulness of these models remain pressing concerns. To this end, we investigate iterative prompting, a strategy hypothesized to refine LLM responses, assessing its impact on LLM truthfulness, an area which has not been thoroughly explored. Our extensive experiments delve into the intricacies of iterative prompting variants, examining their influence on the accuracy and calibration of model responses. Our findings reveal that naive prompting methods significantly undermine truthfulness, leading to exacerbated calibration errors. In response to these challenges, we introduce several prompting variants designed to address the identified issues. These variants demonstrate marked improvements over existing baselines, signaling a promising direction for future research. Our work provides a nuanced understanding of iterative prompting and introduces novel approaches to enhance the truthfulness of LLMs, thereby contributing to the development of more accurate and trustworthy AI systems.
• 摘要：大型语言模型 (LLM) 的发展显着改变了许多领域，提供了令人印象深刻的文本生成功能。然而，这些模型的可靠性和真实性仍然令人担忧。为此，我们研究了迭代提示，这是一种假设用于完善法学硕士反应的策略，评估其对法学硕士真实性的影响，这是一个尚未得到彻底探索的领域。我们广泛的实验深入研究了迭代提示变量的复杂性，检查它们对模型响应的准确性和校准的影响。我们的研究结果表明，幼稚的提示方法会显着破坏真实性，导致校准错误加剧。为了应对这些挑战，我们引入了几种旨在解决已识别问题的提示变体。这些变体显示出对现有基线的显着改进，标志着未来研究的有希望的方向。我们的工作提供了对迭代提示的细致入微的理解，并引入了提高法学硕士真实性的新方法，从而有助于开发更准确、更值得信赖的人工智能系统。

Title: Feedback Loops With Language Models Drive In-Context Reward Hacking

• Authors: Alexander Pan, Erik Jones, Meena Jagadeesan, Jacob Steinhardt
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2402.06627
• Pdf URL: https://arxiv.org/pdf/2402.06627
• Copy Paste: [[2402.06627]] Feedback Loops With Language Models Drive In-Context Reward Hacking(https://arxiv.org/abs/2402.06627)
• Keywords: language model, llm, agent
• Abstract: Language models influence the external world: they query APIs that read and write to web pages, generate content that shapes human behavior, and run system commands as autonomous agents. These interactions form feedback loops: LLM outputs affect the world, which in turn affect subsequent LLM outputs. In this work, we show that feedback loops can cause in-context reward hacking (ICRH), where the LLM at test-time optimizes a (potentially implicit) objective but creates negative side effects in the process. For example, consider an LLM agent deployed to increase Twitter engagement; the LLM may retrieve its previous tweets into the context window and make them more controversial, increasing engagement but also toxicity. We identify and study two processes that lead to ICRH: output-refinement and policy-refinement. For these processes, evaluations on static datasets are insufficient -- they miss the feedback effects and thus cannot capture the most harmful behavior. In response, we provide three recommendations for evaluation to capture more instances of ICRH. As AI development accelerates, the effects of feedback loops will proliferate, increasing the need to understand their role in shaping LLM behavior.
• 摘要：语言模型影响外部世界：它们查询读取和写入网页的 API，生成塑造人类行为的内容，并作为自主代理运行系统命令。这些相互作用形成反馈循环：LLM输出影响世界，进而影响后续的LLM输出。在这项工作中，我们表明反馈循环可能会导致上下文奖励黑客（ICRH），其中法学硕士在测试时优化了（可能是隐含的）目标，但在此过程中产生了负面影响。例如，考虑部署一个 LLM 代理来提高 Twitter 参与度；法学硕士可能会将其之前的推文检索到上下文窗口中，并使它们更具争议性，从而增加参与度，但也会产生毒性。我们确定并研究了导致 ICRH 的两个过程：产出细化和政策细化。对于这些过程，对静态数据集的评估是不够的——它们错过了反馈效果，因此无法捕获最有害的行为。作为回应，我们提供了三项评估建议，以捕获更多 ICRH 实例。随着人工智能发展的加速，反馈循环的影响将会激增，越来越需要了解它们在塑造法学硕士行为中的作用。