2025-02-14

Title: COutfitGAN: Learning to Synthesize Compatible Outfits Supervised by Silhouette Masks and Fashion Styles

• Authors: Dongliang Zhou, Haijun Zhang, Qun Li, Jianghong Ma, Xiaofei Xu
• Subjects: cs.CV, cs.GR, cs.MM
• Abstract URL: https://arxiv.org/abs/2502.08674
• Pdf URL: https://arxiv.org/pdf/2502.08674
• Copy Paste: [[2502.08674]] COutfitGAN: Learning to Synthesize Compatible Outfits Supervised by Silhouette Masks and Fashion Styles(https://arxiv.org/abs/2502.08674)
• Keywords: generation
• Abstract: How to recommend outfits has gained considerable attention in both academia and industry in recent years. Many studies have been carried out regarding fashion compatibility learning, to determine whether the fashion items in an outfit are compatible or not. These methods mainly focus on evaluating the compatibility of existing outfits and rarely consider applying such knowledge to 'design' new fashion items. We propose the new task of generating complementary and compatible fashion items based on an arbitrary number of given fashion items. In particular, given some fashion items that can make up an outfit, the aim of this paper is to synthesize photo-realistic images of other, complementary, fashion items that are compatible with the given ones. To achieve this, we propose an outfit generation framework, referred to as COutfitGAN, which includes a pyramid style extractor, an outfit generator, a UNet-based real/fake discriminator, and a collocation discriminator. To train and evaluate this framework, we collected a large-scale fashion outfit dataset with over 200K outfits and 800K fashion items from the Internet. Extensive experiments show that COutfitGAN outperforms other baselines in terms of similarity, authenticity, and compatibility measurements.
• 摘要：近年来，如何推荐服装在学术界和行业都引起了人们的关注。已经对时尚兼容性学习进行了许多研究，以确定服装中的时尚项目是否兼容。这些方法主要集中于评估现有服装的兼容性，而很少考虑将这种知识应用于“设计”新时尚项目。我们建议根据任意数量的给定时尚项目生成互补和兼容的时尚项目的新任务。特别是，鉴于一些可以构成服装的时尚项目，本文的目的是合成与给定的其他，互补的，互补的时尚物品的照片真实图像。为了实现这一目标，我们提出了一个服装生成框架，称为Coutfitgan，其中包括金字塔样式提取器，装备生成器，基于UNET的真实/假歧视器和搭配歧视器。为了培训和评估此框架，我们从互联网上收集了一个大型时尚服装数据集，其中包含超过200k的服装和80万种时尚物品。广泛的实验表明，在相似性，真实性和兼容性测量方面，Coutfitgan优于其他基准。

Title: Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient Text-to-Image Generation

• Authors: Hoigi Seo, Wongi Jeong, Jae-sun Seo, Se Young Chun
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.08690
• Pdf URL: https://arxiv.org/pdf/2502.08690
• Copy Paste: [[2502.08690]] Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient Text-to-Image Generation(https://arxiv.org/abs/2502.08690)
• Keywords: generation
• Abstract: Large-scale text encoders in text-to-image (T2I) diffusion models have demonstrated exceptional performance in generating high-quality images from textual prompts. Unlike denoising modules that rely on multiple iterative steps, text encoders require only a single forward pass to produce text embeddings. However, despite their minimal contribution to total inference time and floating-point operations (FLOPs), text encoders demand significantly higher memory usage, up to eight times more than denoising modules. To address this inefficiency, we propose Skip and Re-use layers (Skrr), a simple yet effective pruning strategy specifically designed for text encoders in T2I diffusion models. Skrr exploits the inherent redundancy in transformer blocks by selectively skipping or reusing certain layers in a manner tailored for T2I tasks, thereby reducing memory consumption without compromising performance. Extensive experiments demonstrate that Skrr maintains image quality comparable to the original model even under high sparsity levels, outperforming existing blockwise pruning methods. Furthermore, Skrr achieves state-of-the-art memory efficiency while preserving performance across multiple evaluation metrics, including the FID, CLIP, DreamSim, and GenEval scores.
• 摘要：文本到图像（T2I）扩散模型中的大规模文本编码在从文本提示中生成高质量的图像时表现出了出色的性能。与依赖多个迭代步骤的DeNORING模块不同，文本编码器仅需要单个正向通行证来产生文本嵌入。但是，尽管对总推断时间和浮点操作（FLOPS）的贡献很小，但文本编码的需求明显更高的内存使用情况，最多是降级模块的八倍。为了解决此效率低下，我们提出了跳过和重复使用层（SKRR），这是一种简单而有效的修剪策略，专为T2I扩散模型中的文本编码设计而设计。 SKRR通过针对T2I任务量身定制的方式选择性跳过或重复某些层来利用变压器块中固有的冗余性，从而在不损害性能的情况下减少内存消耗。广泛的实验表明，即使在高稀疏度下，SKRR也保持与原始模型相当的图像质量，表现优于现有的块状修剪方法。此外，SKRR可实现最新的记忆效率，同时在多个评估指标（包括FID，剪辑，DreamsIM和Geneval分数）中保持性能。

Title: Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics

• Authors: Sebastian Sanokowski, Wilhelm Berghammer, Martin Ennemoser, Haoyu Peter Wang, Sepp Hochreiter, Sebastian Lehner
• Subjects: cs.LG, cond-mat.stat-mech, cs.AI, physics.comp-ph, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.08696
• Pdf URL: https://arxiv.org/pdf/2502.08696
• Copy Paste: [[2502.08696]] Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics(https://arxiv.org/abs/2502.08696)
• Keywords: generative
• Abstract: Learning to sample from complex unnormalized distributions over discrete domains emerged as a promising research direction with applications in statistical physics, variational inference, and combinatorial optimization. Recent work has demonstrated the potential of diffusion models in this domain. However, existing methods face limitations in memory scaling and thus the number of attainable diffusion steps since they require backpropagation through the entire generative process. To overcome these limitations we introduce two novel training methods for discrete diffusion samplers, one grounded in the policy gradient theorem and the other one leveraging Self-Normalized Neural Importance Sampling (SN-NIS). These methods yield memory-efficient training and achieve state-of-the-art results in unsupervised combinatorial optimization. Numerous scientific applications additionally require the ability of unbiased sampling. We introduce adaptations of SN-NIS and Neural Markov Chain Monte Carlo that enable for the first time the application of discrete diffusion models to this problem. We validate our methods on Ising model benchmarks and find that they outperform popular autoregressive approaches. Our work opens new avenues for applying diffusion models to a wide range of scientific applications in discrete domains that were hitherto restricted to exact likelihood models.
• 摘要：学会从复杂的非平均分布中采样离散域的分布，作为有前途的研究方向，并在统计物理学，变异推断和组合优化中应用。最近的工作证明了在该域中扩散模型的潜力。但是，现有方法在内存缩放中面临局限性，因此可以通过整个生成过程进行反向传播，因此可达到的扩散步骤的数量。为了克服这些局限性，我们引入了两种新型的培训方法，用于离散扩散采样器，一种基于策略梯度定理，另一个基于一个利用自称的神经重要性采样（SN-nis）。这些方法产生了记忆有效的训练并实现最新的训练，从而实现了无监督的组合优化。许多科学应用还需要无偏抽样的能力。我们介绍了SN-NIS和神经马尔可夫链蒙特卡洛的适应性，这是首次将离散扩散模型应用于此问题。我们验证了关于ISING模型基准测试的方法，并发现它们表现优于流行的自动回归方法。我们的工作开辟了新的途径，用于将扩散模型应用于迄今仅限于精确可能模型的离散域中的广泛科学应用。

Title: HistoSmith: Single-Stage Histology Image-Label Generation via Conditional Latent Diffusion for Enhanced Cell Segmentation and Classification

• Authors: Valentina Vadori, Jean-Marie Graïc, Antonella Peruffo, Livio Finos, Ujwala Kiran Chaudhari, Enrico Grisan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08754
• Pdf URL: https://arxiv.org/pdf/2502.08754
• Copy Paste: [[2502.08754]] HistoSmith: Single-Stage Histology Image-Label Generation via Conditional Latent Diffusion for Enhanced Cell Segmentation and Classification(https://arxiv.org/abs/2502.08754)
• Keywords: generation
• Abstract: Precise segmentation and classification of cell instances are vital for analyzing the tissue microenvironment in histology images, supporting medical diagnosis, prognosis, treatment planning, and studies of brain cytoarchitecture. However, the creation of high-quality annotated datasets for training remains a major challenge. This study introduces a novel single-stage approach (HistoSmith) for generating image-label pairs to augment histology datasets. Unlike state-of-the-art methods that utilize diffusion models with separate components for label and image generation, our approach employs a latent diffusion model to learn the joint distribution of cellular layouts, classification masks, and histology images. This model enables tailored data generation by conditioning on user-defined parameters such as cell types, quantities, and tissue types. Trained on the Conic H&E histopathology dataset and the Nissl-stained CytoDArk0 dataset, the model generates realistic and diverse labeled samples. Experimental results demonstrate improvements in cell instance segmentation and classification, particularly for underrepresented cell types like neutrophils in the Conic dataset. These findings underscore the potential of our approach to address data scarcity challenges.
• 摘要：细胞实例的精确分割和分类对于分析组织学图像中的组织微环境至关重要，支持医学诊断，预后，治疗计划和脑细胞结构结构研究。但是，为培训创建高质量注释的数据集仍然是一个重大挑战。这项研究介绍了一种新型的单阶段方法（HIMESMITH），用于生成图像标签对增强组织学数据集。与使用带有单独组件的扩散模型的最新方法不同，我们的方法采用了潜在扩散模型来学习细胞布局，分类掩码和组织学图像的联合分布。该模型通过根据用户定义的参数（例如细胞类型，数量和组织类型）进行调节来实现量身定制的数据生成。该模型在Conic H＆E组织病理学数据集和NISSL染色的Cytodark0数据集中受过培训，生成了现实且具有标记的样品。实验结果证明了细胞实例分割和分类的改善，尤其是对于代表性不足的细胞类型（如圆锥数据集中的中性粒细胞）。这些发现强调了我们解决数据稀缺挑战的方法的潜力。

Title: SB-Bench: Stereotype Bias Benchmark for Large Multimodal Models

• Authors: Vishal Narnaware, Ashmal Vayani, Rohit Gupta, Swetha Sirnam, Mubarak Shah
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.08779
• Pdf URL: https://arxiv.org/pdf/2502.08779
• Copy Paste: [[2502.08779]] SB-Bench: Stereotype Bias Benchmark for Large Multimodal Models(https://arxiv.org/abs/2502.08779)
• Keywords: generation
• Abstract: Stereotype biases in Large Multimodal Models (LMMs) perpetuate harmful societal prejudices, undermining the fairness and equity of AI applications. As LMMs grow increasingly influential, addressing and mitigating inherent biases related to stereotypes, harmful generations, and ambiguous assumptions in real-world scenarios has become essential. However, existing datasets evaluating stereotype biases in LMMs often lack diversity and rely on synthetic images, leaving a gap in bias evaluation for real-world visual contexts. To address this, we introduce the Stereotype Bias Benchmark (SB-bench), the most comprehensive framework to date for assessing stereotype biases across nine diverse categories with non-synthetic images. SB-bench rigorously evaluates LMMs through carefully curated, visually grounded scenarios, challenging them to reason accurately about visual stereotypes. It offers a robust evaluation framework featuring real-world visual samples, image variations, and multiple-choice question formats. By introducing visually grounded queries that isolate visual biases from textual ones, SB-bench enables a precise and nuanced assessment of a model's reasoning capabilities across varying levels of difficulty. Through rigorous testing of state-of-the-art open-source and closed-source LMMs, SB-bench provides a systematic approach to assessing stereotype biases in LMMs across key social dimensions. This benchmark represents a significant step toward fostering fairness in AI systems and reducing harmful biases, laying the groundwork for more equitable and socially responsible LMMs. Our code and dataset are publicly available.
• 摘要：大型多模型（LMM）中的刻板印象偏见使有害的社会偏见永存，破坏了AI应用的公平性和平等性。随着LMM越来越有影响力，在现实世界情景中，与刻板印象，有害世代和模棱两可的假设有关的固有偏见已经变得至关重要。但是，评估LMM中刻板印象偏见的现有数据集通常缺乏多样性并依赖合成图像，从而在现实世界中的视觉环境中给予偏见评估差距。为了解决这个问题，我们介绍了刻板印象的偏置基准（SB基础），这是迄今为止最全面的框架，用于评估具有非合成图像的九种不同类别的刻板印象偏见。 SB板凳通过精心策划的，视觉接地的场景严格评估LMM，挑战它们准确地推理了视觉刻板印象。它提供了一个可靠的评估框架，其中包含现实世界的视觉样本，图像变化和多项选择问题格式。通过引入视觉扎根的查询，将视觉偏见与文本偏差分离出来，SB板台上可以对模型的推理能力进行精确而细微的评估。通过严格测试最先进的开源和封闭源LMM，SB板凳提供了一种系统的方法来评估关键社会维度的LMMS中的刻板印象偏见。该基准是迈向促进AI系统中公平性并减少有害偏见的重要一步，为更公平且对社会负责的LMM奠定了基础。我们的代码和数据集公开可用。

Title: Deep EEG Super-Resolution: Upsampling EEG Spatial Resolution with Generative Adversarial Networks

• Authors: Isaac Corley, Yufei Huang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08803
• Pdf URL: https://arxiv.org/pdf/2502.08803
• Copy Paste: [[2502.08803]] Deep EEG Super-Resolution: Upsampling EEG Spatial Resolution with Generative Adversarial Networks(https://arxiv.org/abs/2502.08803)
• Keywords: super-resolution, generative
• Abstract: Electroencephalography (EEG) activity contains a wealth of information about what is happening within the human brain. Recording more of this data has the potential to unlock endless future applications. However, the cost of EEG hardware is increasingly expensive based upon the number of EEG channels being recorded simultaneously. We combat this problem in this paper by proposing a novel deep EEG super-resolution (SR) approach based on Generative Adversarial Networks (GANs). This approach can produce high spatial resolution EEG data from low resolution samples, by generating channel-wise upsampled data to effectively interpolate numerous missing channels, thus reducing the need for expensive EEG equipment. We tested the performance using an EEG dataset from a mental imagery task. Our proposed GAN model provided 10^4 fold and 10^2 fold reduction in mean-squared error (MSE) and mean-absolute error (MAE), respectively, over the baseline bicubic interpolation method. We further validate our method by training a classifier on the original classification task, which displayed minimal loss in accuracy while using the super-resolved data. The proposed SR EEG by GAN is a promising approach to improve the spatial resolution of low density EEG headsets.
• 摘要：脑电图（EEG）活动包含有关人脑中正在发生的事情的大量信息。记录更多此数据有可能解锁无限的未来应用程序。但是，根据同时记录的脑电图渠道的数量，EEG硬件的成本越来越昂贵。我们在本文中提出了基于生成对抗网络（GAN）的新型深度脑电图超分辨率（SR）方法来解决这个问题。这种方法可以通过生成频道上采样的数据以有效插值众多缺失的通道，从而减少对昂贵的脑电图设备的需求，从而从低分辨率样本产生高空间分辨率的EEG数据。我们使用心理图像任务中的EEG数据集测试了性能。我们提出的GAN模型分别在基线双学的插值方法上分别提供了于点误差（MSE）和均值误差（MAE）的10^4倍和10^2倍的减少。我们通过培训分类器的原始分类任务来进一步验证我们的方法，该分类器在使用超级分辨数据的同时，准确性损失最小。 GAN提出的SR EEG是改善低密度脑电图的空间分辨率的一种有希望的方法。

Title: A First-order Generative Bilevel Optimization Framework for Diffusion Models

• Authors: Quan Xiao, Hui Yuan, A F M Saif, Gaowen Liu, Ramana Kompella, Mengdi Wang, Tianyi Chen
• Subjects: cs.LG, math.OC, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.08808
• Pdf URL: https://arxiv.org/pdf/2502.08808
• Copy Paste: [[2502.08808]] A First-order Generative Bilevel Optimization Framework for Diffusion Models(https://arxiv.org/abs/2502.08808)
• Keywords: generative
• Abstract: Diffusion models, which iteratively denoise data samples to synthesize high-quality outputs, have achieved empirical success across domains. However, optimizing these models for downstream tasks often involves nested bilevel structures, such as tuning hyperparameters for fine-tuning tasks or noise schedules in training dynamics, where traditional bilevel methods fail due to the infinite-dimensional probability space and prohibitive sampling costs. We formalize this challenge as a generative bilevel optimization problem and address two key scenarios: (1) fine-tuning pre-trained models via an inference-only lower-level solver paired with a sample-efficient gradient estimator for the upper level, and (2) training diffusion models from scratch with noise schedule optimization by reparameterizing the lower-level problem and designing a computationally tractable gradient estimator. Our first-order bilevel framework overcomes the incompatibility of conventional bilevel methods with diffusion processes, offering theoretical grounding and computational practicality. Experiments demonstrate that our method outperforms existing fine-tuning and hyperparameter search baselines.
• 摘要：扩散模型（迭代地降解数据样本）可以合成高质量的输出，从而在范围内取得了经验成功。但是，针对下游任务进行优化这些模型通常涉及嵌套的双层结构，例如在训练动力学中调整超参数以进行微调任务或噪声时间表，在这种动力学中，由于无限二维概率概率空间和过度的采样成本，传统的双光线方法失败了。我们将这一挑战正式化为生成性双重优化问题，并解决了两个关键方案：（1）通过仅推理的仅较低级别求解器进行微调预训练的模型，并与上层的样品有效梯度估计器配对，以及（ 2）通过重新聚集较低级别的问题并设计计算可拖动的梯度估计器，从头开始训练扩散模型，并优化噪声时间表。我们的一阶二线框架克服了传统的二线方法与扩散过程的不相容性，提供了理论基础和计算实用性。实验表明，我们的方法优于现有的微调和超参数搜索基准。

Title: DejAIvu: Identifying and Explaining AI Art on the Web in Real-Time with Saliency Maps

• Authors: Jocelyn Dzuong
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08821
• Pdf URL: https://arxiv.org/pdf/2502.08821
• Copy Paste: [[2502.08821]] DejAIvu: Identifying and Explaining AI Art on the Web in Real-Time with Saliency Maps(https://arxiv.org/abs/2502.08821)
• Keywords: generative
• Abstract: The recent surge in advanced generative models, such as diffusion models and generative adversarial networks (GANs), has led to an alarming rise in AI-generated images across various domains on the web. While such technologies offer benefits such as democratizing artistic creation, they also pose challenges in misinformation, digital forgery, and authenticity verification. Additionally, the uncredited use of AI-generated images in media and marketing has sparked significant backlash from online communities. In response to this, we introduce DejAIvu, a Chrome Web extension that combines real-time AI-generated image detection with saliency-based explainability while users browse the web. Using an ONNX-optimized deep learning model, DejAIvu automatically analyzes images on websites such as Google Images, identifies AI-generated content using model inference, and overlays a saliency heatmap to highlight AI-related artifacts. Our approach integrates efficient in-browser inference, gradient-based saliency analysis, and a seamless user experience, ensuring that AI detection is both transparent and interpretable. We also evaluate DejAIvu across multiple pretrained architectures and benchmark datasets, demonstrating high accuracy and low latency, making it a practical and deployable tool for enhancing AI image accountability. The code for this system can be found at this https URL.
• 摘要：高级生成模型的最新激增，例如扩散模型和生成对抗网络（GAN），导致AI生成的图像在Web上的各个域上产生了令人震惊的上升。尽管此类技术提供了使艺术创作民主化的好处，但它们在错误信息，数字伪造和真实性验证方面也构成了挑战。此外，在媒体和营销中未经信用的AI生成图像的使用引发了在线社区的强烈反对。为此，我们介绍了Dejaivu，这是一种Chrome Web扩展名，将实时AI生成的图像检测与用户浏览网络时的显着解释性结合在一起。 Dejaivu使用ONNX优化的深度学习模型，自动分析了Google Images等网站上的图像，使用模型推断识别AI生成的内容，并覆盖显着热图以突出显示与AI相关的工件。我们的方法集成了有效的浏览器推理，基于梯度的显着性分析和无缝的用户体验，以确保AI检测既透明又可解释。我们还评估了跨多个预处理的架构和基准数据集的Dejaivu，表明了高准确性和低潜伏期，使其成为增强AI图像问责制的实用且可部署的工具。该系统的代码可以在此HTTPS URL上找到。

Title: A Survey on Data-Centric AI: Tabular Learning from Reinforcement Learning and Generative AI Perspective

• Authors: Wangyang Ying, Cong Wei, Nanxu Gong, Xinyuan Wang, Haoyue Bai, Arun Vignesh Malarkkan, Sixun Dong, Dongjie Wang, Denghui Zhang, Yanjie Fu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08828
• Pdf URL: https://arxiv.org/pdf/2502.08828
• Copy Paste: [[2502.08828]] A Survey on Data-Centric AI: Tabular Learning from Reinforcement Learning and Generative AI Perspective(https://arxiv.org/abs/2502.08828)
• Keywords: generation, generative
• Abstract: Tabular data is one of the most widely used data formats across various domains such as bioinformatics, healthcare, and marketing. As artificial intelligence moves towards a data-centric perspective, improving data quality is essential for enhancing model performance in tabular data-driven applications. This survey focuses on data-driven tabular data optimization, specifically exploring reinforcement learning (RL) and generative approaches for feature selection and feature generation as fundamental techniques for refining data spaces. Feature selection aims to identify and retain the most informative attributes, while feature generation constructs new features to better capture complex data patterns. We systematically review existing generative methods for tabular data engineering, analyzing their latest advancements, real-world applications, and respective strengths and limitations. This survey emphasizes how RL-based and generative techniques contribute to the automation and intelligence of feature engineering. Finally, we summarize the existing challenges and discuss future research directions, aiming to provide insights that drive continued innovation in this field.
• 摘要：表格数据是在生物信息学，医疗保健和营销等各个领域中使用最广泛的数据格式之一。随着人工智能朝着以数据为中心的角度发展，提高数据质量对于在表格数据驱动的应用程序中提高模型性能至关重要。这项调查重点是数据驱动的表格数据优化，特别探索了增强学习（RL）和特征选择和特征生成的生成方法，作为精炼数据空间的基本技术。功能选择旨在识别和保留最有用的属性，而功能生成构建新功能以更好地捕获复杂的数据模式。我们系统地回顾了用于表格数据工程的现有生成方法，分析其最新进步，现实应用程序以及各自的优势和局限性。这项调查强调了基于RL的和生成技术如何促进功能工程的自动化和智能。最后，我们总结了现有的挑战并讨论未来的研究指示，旨在提供洞察力，以推动该领域的持续创新。

Title: A Reversible Solver for Diffusion SDEs

• Authors: Zander W. Blasingame, Chen Liu
• Subjects: cs.LG, cs.AI, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.08834
• Pdf URL: https://arxiv.org/pdf/2502.08834
• Copy Paste: [[2502.08834]] A Reversible Solver for Diffusion SDEs(https://arxiv.org/abs/2502.08834)
• Keywords: generation
• Abstract: Diffusion models have quickly become the state-of-the-art for generation tasks across many different data modalities. An important ability of diffusion models is the ability to encode samples from the data distribution back into the sampling prior distribution. This is useful for performing alterations to real data samples along with guided generation via the continuous adjoint equations. We propose an algebraically reversible solver for diffusion SDEs that can exactly invert real data samples into the prior distribution.
• 摘要：扩散模型已迅速成为许多不同数据模式的生成任务的最新任务。扩散模型的重要能力是能够从数据分布中编码样本回到先验分布中。这对于通过连续的伴随方程进行对真实数据样本的更改以及指导生成很有用。我们为扩散SDE提出了一个代数可逆的求解器，该求解器可以准确地将实际数据样本倒入先前的分布中。

Title: A Systematic Evaluation of Generative Models on Tabular Transportation Data

• Authors: Chengen Wang, Alvaro Cardenas, Gurcan Comert, Murat Kantarcioglu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.08856
• Pdf URL: https://arxiv.org/pdf/2502.08856
• Copy Paste: [[2502.08856]] A Systematic Evaluation of Generative Models on Tabular Transportation Data(https://arxiv.org/abs/2502.08856)
• Keywords: generation, generative
• Abstract: The sharing of large-scale transportation data is beneficial for transportation planning and policymaking. However, it also raises significant security and privacy concerns, as the data may include identifiable personal information, such as individuals' home locations. To address these concerns, synthetic data generation based on real transportation data offers a promising solution that allows privacy protection while potentially preserving data utility. Although there are various synthetic data generation techniques, they are often not tailored to the unique characteristics of transportation data, such as the inherent structure of transportation networks formed by all trips in the datasets. In this paper, we use New York City taxi data as a case study to conduct a systematic evaluation of the performance of widely used tabular data generative models. In addition to traditional metrics such as distribution similarity, coverage, and privacy preservation, we propose a novel graph-based metric tailored specifically for transportation data. This metric evaluates the similarity between real and synthetic transportation networks, providing potentially deeper insights into their structural and functional alignment. We also introduced an improved privacy metric to address the limitations of the commonly-used one. Our experimental results reveal that existing tabular data generative models often fail to perform as consistently as claimed in the literature, particularly when applied to transportation data use cases. Furthermore, our novel graph metric reveals a significant gap between synthetic and real data. This work underscores the potential need to develop generative models specifically tailored to take advantage of the unique characteristics of emerging domains, such as transportation.
• 摘要：大规模运输数据的共享对运输计划和决策有益。但是，这也引起了严重的安全性和隐私问题，因为数据可能包括可识别的个人信息，例如个人的家庭位置。为了解决这些问题，基于实际运输数据的合成数据生成提供了有希望的解决方案，可以保护隐私，同时潜在地保留数据实用程序。尽管有各种综合数据生成技术，但通常并不是针对传输数据的独特特征量身定制的，例如数据集中所有旅行形成的运输网络的固有结构。在本文中，我们使用纽约市出租车数据作为案例研究，对广泛使用的表格数据生成模型的性能进行系统评估。除了传统的指标，例如分布相似性，覆盖范围和隐私保护外，我们还提出了一种专门针对运输数据量身定制的基于图形的新型度量。该度量标准评估了真实和合成运输网络之间的相似性，从而为它们的结构和功能一致性提供了更深入的见解。我们还提出了一个改进的隐私指标，以解决常用限制的局限性。我们的实验结果表明，现有的表格数据生成模型通常无法像文献中所主张的那样一致，尤其是应用于运输数据用例时。此外，我们的新图形指标揭示了合成数据和真实数据之间的显着差距。这项工作强调了开发专门为利用新兴域（例如运输）的独特特征而定制的生成模型的潜在需求。

Title: Diffusion Models Through a Global Lens: Are They Culturally Inclusive?

• Authors: Zahra Bayramli, Ayhan Suleymanzade, Na Min An, Huzama Ahmad, Eunsu Kim, Junyeong Park, James Thorne, Alice Oh
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.08914
• Pdf URL: https://arxiv.org/pdf/2502.08914
• Copy Paste: [[2502.08914]] Diffusion Models Through a Global Lens: Are They Culturally Inclusive?(https://arxiv.org/abs/2502.08914)
• Keywords: generative
• Abstract: Text-to-image diffusion models have recently enabled the creation of visually compelling, detailed images from textual prompts. However, their ability to accurately represent various cultural nuances remains an open question. In our work, we introduce CultDiff benchmark, evaluating state-of-the-art diffusion models whether they can generate culturally specific images spanning ten countries. We show that these models often fail to generate cultural artifacts in architecture, clothing, and food, especially for underrepresented country regions, by conducting a fine-grained analysis of different similarity aspects, revealing significant disparities in cultural relevance, description fidelity, and realism compared to real-world reference images. With the collected human evaluations, we develop a neural-based image-image similarity metric, namely, CultDiff-S, to predict human judgment on real and generated images with cultural artifacts. Our work highlights the need for more inclusive generative AI systems and equitable dataset representation over a wide range of cultures.
• 摘要：文本到图像扩散模型最近已从文本提示中创建了视觉引人入胜的详细图像。但是，它们准确代表各种文化细微差别的能力仍然是一个悬而未决的问题。在我们的工作中，我们介绍了邪教基准测试，评估了最先进的扩散模型，它们是否可以产生跨越十个国家的文化特定图像。我们表明，这些模型通常无法通过对不同相似性方面的精细分析进行精细分析，从而揭示了文化相关性，描述忠诚度和现实主义比较，从而产生建筑，服装和食物（尤其是代表人数不足的国家地区）中的文化文物，尤其是对于代表性不足的国家地区。到现实世界参考图像。通过收集的人类评估，我们开发了一种基于神经的图像图像相似性度量，即邪教-S，以预测人类对具有文化伪像的真实图像的判断。我们的工作强调了在广泛的文化中需要更具包容性的生成AI系统和公平的数据集表示。

Title: Dynamic watermarks in images generated by diffusion models

• Authors: Yunzhuo Chen, Naveed Akhtar, Nur Al Hasan Haldar, Ajmal Mian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.08927
• Pdf URL: https://arxiv.org/pdf/2502.08927
• Copy Paste: [[2502.08927]] Dynamic watermarks in images generated by diffusion models(https://arxiv.org/abs/2502.08927)
• Keywords: generation
• Abstract: High-fidelity text-to-image diffusion models have revolutionized visual content generation, but their widespread use raises significant ethical concerns, including intellectual property protection and the misuse of synthetic media. To address these challenges, we propose a novel multi-stage watermarking framework for diffusion models, designed to establish copyright and trace generated images back to their source. Our multi-stage watermarking technique involves embedding: (i) a fixed watermark that is localized in the diffusion model's learned noise distribution and, (ii) a human-imperceptible, dynamic watermark in generates images, leveraging a fine-tuned decoder. By leveraging the Structural Similarity Index Measure (SSIM) and cosine similarity, we adapt the watermark's shape and color to the generated content while maintaining robustness. We demonstrate that our method enables reliable source verification through watermark classification, even when the dynamic watermark is adjusted for content-specific variations. Source model verification is enabled through watermark classification. o support further research, we generate a dataset of watermarked images and introduce a methodology to evaluate the statistical impact of watermarking on generated this http URL, we rigorously test our framework against various attack scenarios, demonstrating its robustness and minimal impact on image quality. Our work advances the field of AI-generated content security by providing a scalable solution for model ownership verification and misuse prevention.
• 摘要：高保真的文本到图像扩散模型已彻底改变了视觉内容的产生，但是它们的广泛使用引起了重大的道德问题，包括知识产权保护和滥用合成媒体。为了应对这些挑战，我们为扩散模型提出了一个新型的多阶段水印框架，旨在建立版权和痕量生成的图像回到其来源。我们的多阶段水印技术涉及嵌入：（i）固定的水印，该水印位于扩散模型的学习噪声分布中，（ii）在生成图像中的人类侵蚀，动态水印，利用微调解码器。通过利用结构相似性指数度量（SSIM）和余弦相似性，我们将水印的形状和颜色适应生成的内容，同时保持健壮性。我们证明我们的方法可以通过水印分类来实现可靠的源验证，即使调整了特定于内容的变化的动态水印。源模型验证是通过水印分类启用的。 o支持进一步的研究，我们生成了一个水印图像的数据集，并引入了一种评估水印对生成的HTTP URL的统计影响的方法，我们严格地测试了我们的框架，以针对各种攻击场景，证明其稳健性和对图像质量的最小影响。我们的工作通过为模型所有权验证和滥用预防提供可扩展的解决方案来推动AI生成的内容安全性的领域。

Title: Text-driven 3D Human Generation via Contrastive Preference Optimization

• Authors: Pengfei Zhou, Xukun Shen, Yong Hu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.08977
• Pdf URL: https://arxiv.org/pdf/2502.08977
• Copy Paste: [[2502.08977]] Text-driven 3D Human Generation via Contrastive Preference Optimization(https://arxiv.org/abs/2502.08977)
• Keywords: generation
• Abstract: Recent advances in Score Distillation Sampling (SDS) have improved 3D human generation from textual descriptions. However, existing methods still face challenges in accurately aligning 3D models with long and complex textual inputs. To address this challenge, we propose a novel framework that introduces contrastive preferences, where human-level preference models, guided by both positive and negative prompts, assist SDS for improved alignment. Specifically, we design a preference optimization module that integrates multiple models to comprehensively capture the full range of textual features. Furthermore, we introduce a negation preference module to mitigate over-optimization of irrelevant details by leveraging static-dynamic negation prompts, effectively preventing ``reward hacking". Extensive experiments demonstrate that our method achieves state-of-the-art results, significantly enhancing texture realism and visual alignment with textual descriptions, particularly for long and complex inputs.
• 摘要：评分蒸馏采样（SDS）的最新进展已从文本描述中改善了3D人类一代。但是，现有方法仍然面临挑战，可以将3D模型与长期和复杂的文本输入保持一致。为了应对这一挑战，我们提出了一个新的框架，该框架引入了对比偏好，在该挑战中，在正面和负面提示的指导下，人类水平的偏好模型有助于SDS改善对齐。具体来说，我们设计了一个偏好优化模块，该模块集成了多个模型，以全面捕获文本功能的全部范围。此外，我们引入了一个否定偏好模块，以通过利用静态动态否定提示来减轻无关紧要的细节过度避免细节，从而有效防止``奖励黑客攻击''。广泛的实验表明，我们的方法可实现最新的结果，从而显着增强结果，从纹理现实主义和视觉对齐文字描述，尤其是对于长而复杂的输入。

Title: StyleBlend: Enhancing Style-Specific Content Creation in Text-to-Image Diffusion Models

• Authors: Zichong Chen, Shijin Wang, Yang Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09064
• Pdf URL: https://arxiv.org/pdf/2502.09064
• Copy Paste: [[2502.09064]] StyleBlend: Enhancing Style-Specific Content Creation in Text-to-Image Diffusion Models(https://arxiv.org/abs/2502.09064)
• Keywords: generation
• Abstract: Synthesizing visually impressive images that seamlessly align both text prompts and specific artistic styles remains a significant challenge in Text-to-Image (T2I) diffusion models. This paper introduces StyleBlend, a method designed to learn and apply style representations from a limited set of reference images, enabling content synthesis of both text-aligned and stylistically coherent. Our approach uniquely decomposes style into two components, composition and texture, each learned through different strategies. We then leverage two synthesis branches, each focusing on a corresponding style component, to facilitate effective style blending through shared features without affecting content generation. StyleBlend addresses the common issues of text misalignment and weak style representation that previous methods have struggled with. Extensive qualitative and quantitative comparisons demonstrate the superiority of our approach.
• 摘要：综合视觉上令人印象深刻的图像无缝地对齐文本提示和特定的艺术风格仍然是文本对图像（T2I）扩散模型的重大挑战。本文介绍了StyleBlend，该方法旨在从有限的参考图像中学习和应用样式表示形式，从而可以综合文本一致性和风格相干的内容。我们的方法独特地将样式分解为两个组成和纹理的组成部分，每种组成和纹理都是通过不同的策略学习的。然后，我们利用两个综合分支，每个分支都集中在相应的样式组件上，以促进通过共享功能融合的有效样式而不会影响内容的生成。 StyleBlend解决了以前方法所努力的文本未对准和弱样式表示的常见问题。广泛的定性和定量比较证明了我们方法的优势。

Title: Finite-Time Analysis of Discrete-Time Stochastic Interpolants

• Authors: Yuhao Liu, Yu Chen, Rui Hu, Longbo Huang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09130
• Pdf URL: https://arxiv.org/pdf/2502.09130
• Copy Paste: [[2502.09130]] Finite-Time Analysis of Discrete-Time Stochastic Interpolants(https://arxiv.org/abs/2502.09130)
• Keywords: generative
• Abstract: The stochastic interpolant framework offers a powerful approach for constructing generative models based on ordinary differential equations (ODEs) or stochastic differential equations (SDEs) to transform arbitrary data distributions. However, prior analyses of this framework have primarily focused on the continuous-time setting, assuming a perfect solution of the underlying equations. In this work, we present the first discrete-time analysis of the stochastic interpolant framework, where we introduce an innovative discrete-time sampler and derive a finite-time upper bound on its distribution estimation error. Our result provides a novel quantification of how different factors, including the distance between source and target distributions and estimation accuracy, affect the convergence rate and also offers a new principled way to design efficient schedules for convergence acceleration. Finally, numerical experiments are conducted on the discrete-time sampler to corroborate our theoretical findings.
• 摘要：随机插值框架为基于普通微分方程（ODE）或随机微分方程（SDE）构建生成模型提供了一种强大的方法，以转换任意数据分布。但是，假设基础方程的完美解决方案，对该框架的先前分析主要集中在连续时间设置上。在这项工作中，我们介绍了随机插入式框架的第一个离散时间分析，在该框架中我们引入了创新的离散时间采样器，并在其分布估计误差上得出了有限的时间上限。我们的结果提供了一个新的量化，对不同因素（包括源和目标分布之间的距离以及估计准确性之间的距离）如何影响收敛率，并提供了一种新的原则方法，以设计有效的时间表以进行收敛加速。最后，在离散时间采样器上进行了数值实验，以证实我们的理论发现。

Title: Interpreting and Steering Protein Language Models through Sparse Autoencoders

• Authors: Edith Natalia Villegas Garcia, Alessio Ansuini
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2502.09135
• Pdf URL: https://arxiv.org/pdf/2502.09135
• Copy Paste: [[2502.09135]] Interpreting and Steering Protein Language Models through Sparse Autoencoders(https://arxiv.org/abs/2502.09135)
• Keywords: generation
• Abstract: The rapid advancements in transformer-based language models have revolutionized natural language processing, yet understanding the internal mechanisms of these models remains a significant challenge. This paper explores the application of sparse autoencoders (SAE) to interpret the internal representations of protein language models, specifically focusing on the ESM-2 8M parameter model. By performing a statistical analysis on each latent component's relevance to distinct protein annotations, we identify potential interpretations linked to various protein characteristics, including transmembrane regions, binding sites, and specialized motifs. We then leverage these insights to guide sequence generation, shortlisting the relevant latent components that can steer the model towards desired targets such as zinc finger domains. This work contributes to the emerging field of mechanistic interpretability in biological sequence models, offering new perspectives on model steering for sequence design.
• 摘要：基于变压器的语言模型的快速进步彻底改变了自然语言处理，但是了解这些模型的内部机制仍然是一个重大挑战。本文探讨了稀疏自动编码器（SAE）在解释蛋白质语言模型的内部表示方面的应用，特别是针对ESM-2 8M参数模型。通过对每个潜在成分与不同蛋白质注释相关的相关性进行统计分析，我们确定了与各种蛋白质特征相关的潜在解释，包括跨膜区域，结合位点和专门的基序。然后，我们利用这些见解来指导序列的产生，将相关的潜在组件入围，这些组件可以将模型引导到所需的靶标，例如锌指域。这项工作有助于生物序列模型中机械性可解释性的新兴领域，从而提供了有关序列设计模型转向的新观点。

Title: Regularization can make diffusion models more efficient

• Authors: Mahsa Taheri, Johannes Lederer
• Subjects: cs.LG, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09151
• Pdf URL: https://arxiv.org/pdf/2502.09151
• Copy Paste: [[2502.09151]] Regularization can make diffusion models more efficient(https://arxiv.org/abs/2502.09151)
• Keywords: generative
• Abstract: Diffusion models are one of the key architectures of generative AI. Their main drawback, however, is the computational costs. This study indicates that the concept of sparsity, well known especially in statistics, can provide a pathway to more efficient diffusion pipelines. Our mathematical guarantees prove that sparsity can reduce the input dimension's influence on the computational complexity to that of a much smaller intrinsic dimension of the data. Our empirical findings confirm that inducing sparsity can indeed lead to better samples at a lower cost.
• 摘要：扩散模型是生成AI的关键体系结构之一。但是，他们的主要缺点是计算成本。这项研究表明，稀疏性的概念，尤其是在统计中，可以为更有效的扩散管道提供途径。我们的数学保证证明，稀疏性可以减少输入维度对计算复杂性的影响到数据的固有维度较小。我们的经验发现证实，诱导稀疏性确实可以以较低的成本导致更好的样本。

Title: LOB-Bench: Benchmarking Generative AI for Finance - an Application to Limit Order Book Data

• Authors: Peer Nagy, Sascha Frey, Kang Li, Bidipta Sarkar, Svitlana Vyetrenko, Stefan Zohren, Ani Calinescu, Jakob Foerster
• Subjects: cs.LG, cs.CE, q-fin.CP, q-fin.TR
• Abstract URL: https://arxiv.org/abs/2502.09172
• Pdf URL: https://arxiv.org/pdf/2502.09172
• Copy Paste: [[2502.09172]] LOB-Bench: Benchmarking Generative AI for Finance - an Application to Limit Order Book Data(https://arxiv.org/abs/2502.09172)
• Keywords: generative
• Abstract: While financial data presents one of the most challenging and interesting sequence modelling tasks due to high noise, heavy tails, and strategic interactions, progress in this area has been hindered by the lack of consensus on quantitative evaluation paradigms. To address this, we present LOB-Bench, a benchmark, implemented in python, designed to evaluate the quality and realism of generative message-by-order data for limit order books (LOB) in the LOBSTER format. Our framework measures distributional differences in conditional and unconditional statistics between generated and real LOB data, supporting flexible multivariate statistical evaluation. The benchmark also includes features commonly used LOB statistics such as spread, order book volumes, order imbalance, and message inter-arrival times, along with scores from a trained discriminator network. Lastly, LOB-Bench contains "market impact metrics", i.e. the cross-correlations and price response functions for specific events in the data. We benchmark generative autoregressive state-space models, a (C)GAN, as well as a parametric LOB model and find that the autoregressive GenAI approach beats traditional model classes.
• 摘要：尽管由于高噪音，沉重的尾巴和战略性相互作用，财务数据列出了最具挑战性，最有趣的序列建模任务之一，但由于对定量评估范式的共识缺乏共识而阻碍了这一领域的进展。为了解决这个问题，我们介绍了在Python中实现的基准Lob Bench，旨在评估以龙虾格式评估限制订单书籍（LOB）的生成消息的质量和现实主义。我们的框架衡量了生成和真实LOB数据之间条件和无条件统计的分布差异，从而支持灵活的多元统计评估。该基准还包括常用的LOB统计信息，例如差异，订单簿卷，订单不平衡和消息间隔时间以及训练有素的歧视网络的得分。最后，Lob Bench包含“市场影响指标”，即数据中特定事件的互相关和价格响应功能。我们基于生成自回旋的状态空间模型A（c）GAN以及参数LOB模型，发现自动回归的Genai方法击败了传统的模型类。

Title: ConsistentDreamer: View-Consistent Meshes Through Balanced Multi-View Gaussian Optimization

• Authors: Onat Şahin, Mohammad Altillawi, George Eskandar, Carlos Carbone, Ziyuan Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09278
• Pdf URL: https://arxiv.org/pdf/2502.09278
• Copy Paste: [[2502.09278]] ConsistentDreamer: View-Consistent Meshes Through Balanced Multi-View Gaussian Optimization(https://arxiv.org/abs/2502.09278)
• Keywords: generation
• Abstract: Recent advances in diffusion models have significantly improved 3D generation, enabling the use of assets generated from an image for embodied AI simulations. However, the one-to-many nature of the image-to-3D problem limits their use due to inconsistent content and quality across views. Previous models optimize a 3D model by sampling views from a view-conditioned diffusion prior, but diffusion models cannot guarantee view consistency. Instead, we present ConsistentDreamer, where we first generate a set of fixed multi-view prior images and sample random views between them with another diffusion model through a score distillation sampling (SDS) loss. Thereby, we limit the discrepancies between the views guided by the SDS loss and ensure a consistent rough shape. In each iteration, we also use our generated multi-view prior images for fine-detail reconstruction. To balance between the rough shape and the fine-detail optimizations, we introduce dynamic task-dependent weights based on homoscedastic uncertainty, updated automatically in each iteration. Additionally, we employ opacity, depth distortion, and normal alignment losses to refine the surface for mesh extraction. Our method ensures better view consistency and visual quality compared to the state-of-the-art.
• 摘要：扩散模型的最新进展已显着改善了3D生成，从而实现了从图像产生的资产进行体现的AI模拟。但是，图像到3D问题的一对一性质由于跨视图的内容和质量不一致而限制了它们的使用。先前的模型通过从视图条件扩散之前对3D模型进行了优化3D模型，但是扩散模型无法保证视图一致性。取而代之的是，我们提出一致的Dreamer，在其中首先通过分数蒸馏采样（SDS）损失，首先生成一组固定的多视图先验图像，并在它们之间使用另一个扩散模型进行示例随机视图。因此，我们限制了由SDS损失引导的视图之间的差异，并确保一致的粗糙形状。在每次迭代中，我们还将生成的多视图先验图像用于细节重建。为了在粗糙的形状和细节优化之间取得平衡，我们基于同质的不确定性引入动态任务依赖性权重，在每次迭代中会自动更新。此外，我们采用不透明度，深度失真和正常的比对损失来完善表面以提取网格。与最先进的方法相比，我们的方法可确保更好的视图一致性和视觉质量。

Title: FE-LWS: Refined Image-Text Representations via Decoder Stacking and Fused Encodings for Remote Sensing Image Captioning

• Authors: Swadhin Das, Raksha Sharma
• Subjects: cs.CV, cs.HC, cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09282
• Pdf URL: https://arxiv.org/pdf/2502.09282
• Copy Paste: [[2502.09282]] FE-LWS: Refined Image-Text Representations via Decoder Stacking and Fused Encodings for Remote Sensing Image Captioning(https://arxiv.org/abs/2502.09282)
• Keywords: generation
• Abstract: Remote sensing image captioning aims to generate descriptive text from remote sensing images, typically employing an encoder-decoder framework. In this setup, a convolutional neural network (CNN) extracts feature representations from the input image, which then guide the decoder in a sequence-to-sequence caption generation process. Although much research has focused on refining the decoder, the quality of image representations from the encoder remains crucial for accurate captioning. This paper introduces a novel approach that integrates features from two distinct CNN based encoders, capturing complementary information to enhance caption generation. Additionally, we propose a weighted averaging technique to combine the outputs of all GRUs in the stacked decoder. Furthermore, a comparison-based beam search strategy is incorporated to refine caption selection. The results demonstrate that our fusion-based approach, along with the enhanced stacked decoder, significantly outperforms both the transformer-based state-of-the-art model and other LSTM-based baselines.
• 摘要：遥感图像字幕旨在从遥感图像中生成描述性文本，通常采用编码器框架。在此设置中，卷积神经网络（CNN）提取了输入图像的特征表示，然后在序列到序列字幕生成过程中引导解码器。尽管许多研究集中在精炼解码器上，但编码器的图像表示质量对于准确的字幕至关重要。本文介绍了一种新颖的方法，该方法整合了两个不同基于CNN的编码器的特征，从而捕获补充信息以增强字幕生成。此外，我们提出了一种加权平均技术，以结合堆叠解码器中所有GRU的输出。此外，还制定了基于比较的梁搜索策略以完善标题选择。结果表明，我们的基于融合的方法以及增强的堆叠解码器大大优于基于变压器的最先进模型和其他基于LSTM的基线。

Title: When do neural networks learn world models?

• Authors: Tianren Zhang, Guanyu Chen, Feng Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09297
• Pdf URL: https://arxiv.org/pdf/2502.09297
• Copy Paste: [[2502.09297]] When do neural networks learn world models?(https://arxiv.org/abs/2502.09297)
• Keywords: generation
• Abstract: Humans develop world models that capture the underlying generation process of data. Whether neural networks can learn similar world models remains an open problem. In this work, we provide the first theoretical results for this problem, showing that in a multi-task setting, models with a low-degree bias provably recover latent data-generating variables under mild assumptions -- even if proxy tasks involve complex, non-linear functions of the latents. However, such recovery is also sensitive to model architecture. Our analysis leverages Boolean models of task solutions via the Fourier-Walsh transform and introduces new techniques for analyzing invertible Boolean transforms, which may be of independent interest. We illustrate the algorithmic implications of our results and connect them to related research areas, including self-supervised learning, out-of-distribution generalization, and the linear representation hypothesis in large language models.
• 摘要：人类开发世界模型，以捕获数据的潜在生成过程。神经网络是否可以学习类似的世界模型仍然是一个开放的问题。在这项工作中，我们为这个问题提供了第一个理论结果，表明在多任务设置中，具有低度偏差的模型可证明在轻度假设下恢复潜在的数据生成变量 - 即使代理任务涉及复杂的，非复杂的，非 - 潜伏期的线性功能。但是，这种恢复也对模型体系结构敏感。我们的分析通过Fourier-Walsh转换利用了任务解决方案的布尔模型，并引入了用于分析可逆布尔变换的新技术，这可能具有独立的兴趣。我们说明了结果的算法含义，并将其连接到相关的研究领域，包括自我监督的学习，分布式概括以及大语言模型中的线性表示假设。

Title: Machine learning for modelling unstructured grid data in computational physics: a review

• Authors: Sibo Cheng, Marc Bocquet, Weiping Ding, Tobias Sebastian Finn, Rui Fu, Jinlong Fu, Yike Guo, Eleda Johnson, Siyi Li, Che Liu, Eric Newton Moro, Jie Pan, Matthew Piggott, Cesar Quilodran, Prakhar Sharma, Kun Wang, Dunhui Xiao, Xiao Xue, Yong Zeng, Mingrui Zhang, Hao Zhou, Kewei Zhu, Rossella Arcucci
• Subjects: cs.LG, cs.CE, physics.data-an, physics.flu-dyn
• Abstract URL: https://arxiv.org/abs/2502.09346
• Pdf URL: https://arxiv.org/pdf/2502.09346
• Copy Paste: [[2502.09346]] Machine learning for modelling unstructured grid data in computational physics: a review(https://arxiv.org/abs/2502.09346)
• Keywords: generation, generative
• Abstract: Unstructured grid data are essential for modelling complex geometries and dynamics in computational physics. Yet, their inherent irregularity presents significant challenges for conventional machine learning (ML) techniques. This paper provides a comprehensive review of advanced ML methodologies designed to handle unstructured grid data in high-dimensional dynamical systems. Key approaches discussed include graph neural networks, transformer models with spatial attention mechanisms, interpolation-integrated ML methods, and meshless techniques such as physics-informed neural networks. These methodologies have proven effective across diverse fields, including fluid dynamics and environmental simulations. This review is intended as a guidebook for computational scientists seeking to apply ML approaches to unstructured grid data in their domains, as well as for ML researchers looking to address challenges in computational physics. It places special focus on how ML methods can overcome the inherent limitations of traditional numerical techniques and, conversely, how insights from computational physics can inform ML development. To support benchmarking, this review also provides a summary of open-access datasets of unstructured grid data in computational physics. Finally, emerging directions such as generative models with unstructured data, reinforcement learning for mesh generation, and hybrid physics-data-driven paradigms are discussed to inspire future advancements in this evolving field.
• 摘要：非结构化的网格数据对于对计算物理中的复杂几何形状和动力学进行建模至关重要。然而，它们固有的不规则性对传统机器学习（ML）技术提出了重大挑战。本文对旨在处理高维动力学系统中非结构化的网格数据的高级ML方法进行了全面评论。讨论的关键方法包括图形神经网络，具有空间注意力机制的变压器模型，插值集成的ML方法以及无网络技术（例如物理知识的神经网络）。这些方法已被证明在各种领域，包括流体动力学和环境模拟。这篇综述旨在作为寻求将ML方法应用于其领域中的非结构化网格数据的计算科学家的指南，以及寻求应对计算物理挑战的ML研究人员。它特别关注ML方法如何克服传统数值技术的固有局限性，相反，来自计算物理学的洞察力如何为ML开发提供了信息。为了支持基准测试，本综述还提供了计算物理中非结构化网格数据的开放访问数据集的摘要。最后，讨论了新兴方向，例如具有非结构化数据的生成模型，网格生成的增强学习以及混合物理-DATA驱动的范式，以激发这个不断发展的领域的未来进步。

Title: ImageRAG: Dynamic Image Retrieval for Reference-Guided Image Generation

• Authors: Rotem Shalev-Arkushin, Rinon Gal, Amit H. Bermano, Ohad Fried
• Subjects: cs.CV, cs.GR
• Abstract URL: https://arxiv.org/abs/2502.09411
• Pdf URL: https://arxiv.org/pdf/2502.09411
• Copy Paste: [[2502.09411]] ImageRAG: Dynamic Image Retrieval for Reference-Guided Image Generation(https://arxiv.org/abs/2502.09411)
• Keywords: generation
• Abstract: Diffusion models enable high-quality and diverse visual content synthesis. However, they struggle to generate rare or unseen concepts. To address this challenge, we explore the usage of Retrieval-Augmented Generation (RAG) with image generation models. We propose ImageRAG, a method that dynamically retrieves relevant images based on a given text prompt, and uses them as context to guide the generation process. Prior approaches that used retrieved images to improve generation, trained models specifically for retrieval-based generation. In contrast, ImageRAG leverages the capabilities of existing image conditioning models, and does not require RAG-specific training. Our approach is highly adaptable and can be applied across different model types, showing significant improvement in generating rare and fine-grained concepts using different base models. Our project page is available at: this https URL
• 摘要：扩散模型可实现高质量和不同的视觉内容综合。但是，他们难以产生稀有或看不见的概念。为了应对这一挑战，我们探讨了使用图像生成模型的检索效果生成（RAG）的用法。我们提出了Imagerag，该方法是一种基于给定文本提示的动态检索相关图像的方法，并将其用作指导生成过程的上下文。先前使用检索图像来改善发电的方法，专门用于基于检索的生成的训练模型。相反，Imagerag利用现有图像调节模型的功能，并且不需要特定于破布的训练。我们的方法是高度适应的，可以在不同的模型类型上应用，在使用不同的基本模型生成稀有和细粒度的概念方面显示出显着改善。我们的项目页面可用：此HTTPS URL

Title: Redistribute Ensemble Training for Mitigating Memorization in Diffusion Models

• Authors: Xiaoliu Guan, Yu Wu, Huayang Huang, Xiao Liu, Jiaxu Miao, Yi Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09434
• Pdf URL: https://arxiv.org/pdf/2502.09434
• Copy Paste: [[2502.09434]] Redistribute Ensemble Training for Mitigating Memorization in Diffusion Models(https://arxiv.org/abs/2502.09434)
• Keywords: generation
• Abstract: Diffusion models, known for their tremendous ability to generate high-quality samples, have recently raised concerns due to their data memorization behavior, which poses privacy risks. Recent methods for memory mitigation have primarily addressed the issue within the context of the text modality in cross-modal generation tasks, restricting their applicability to specific conditions. In this paper, we propose a novel method for diffusion models from the perspective of visual modality, which is more generic and fundamental for mitigating memorization. Directly exposing visual data to the model increases memorization risk, so we design a framework where models learn through proxy model parameters instead. Specially, the training dataset is divided into multiple shards, with each shard training a proxy model, then aggregated to form the final model. Additionally, practical analysis of training losses illustrates that the losses for easily memorable images tend to be obviously lower. Thus, we skip the samples with abnormally low loss values from the current mini-batch to avoid memorizing. However, balancing the need to skip memorization-prone samples while maintaining sufficient training data for high-quality image generation presents a key challenge. Thus, we propose IET-AGC+, which redistributes highly memorizable samples between shards, to mitigate these samples from over-skipping. Furthermore, we dynamically augment samples based on their loss values to further reduce memorization. Extensive experiments and analysis on four datasets show that our method successfully reduces memory capacity while maintaining performance. Moreover, we fine-tune the pre-trained diffusion models, e.g., Stable Diffusion, and decrease the memorization score by 46.7\%, demonstrating the effectiveness of our method. Code is available in: this https URL.
• 摘要：扩散模型以其产生高质量样本的巨大能力而闻名，最近由于其数据记忆行为而引起了人们的关注，这带来了隐私风险。最新的缓解记忆方法主要解决了跨模式生成任务中文本模式的上下文中的问题，从而将其适用性限制在特定条件下。在本文中，我们从视觉模态的角度提出了一种扩散模型的新方法，这对于缓解记忆更为通用和基础。直接将视觉数据暴露于模型会增加记忆风险，因此我们设计了一个框架，模型通过代理模型参数学习。特别是，训练数据集分为多个碎片，每个碎片训练一个代理模型，然后汇总以形成最终模型。此外，对训练损失的实际分析表明，易于记忆的图像的损失显然较低。因此，我们从当前的迷你批次中跳过异常低损耗值的样品，以避免记忆。但是，平衡需要跳过易于记忆的样本的需求，同时维持足够的培训数据以实现高质量图像生成带来的关键挑战。因此，我们提出了IET-AGC+，该IET-AGC+在碎片之间重新分配了高度可记住的样品，以减轻这些样本过度的样本。此外，我们根据样品的损失值动态增强样本，以进一步降低记忆。在四个数据集上进行的广泛实验和分析表明，我们的方法在保持性能的同时成功降低了内存能力。此外，我们微调了预训练的扩散模型，例如稳定的扩散，并将记忆评分降低了46.7 \％，证明了我们方法的有效性。代码可在：此HTTPS URL中可用。

Title: EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling

• Authors: Theodoros Kouzelis, Ioannis Kakogeorgiou, Spyros Gidaris, Nikos Komodakis
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2502.09509
• Pdf URL: https://arxiv.org/pdf/2502.09509
• Copy Paste: [[2502.09509]] EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling(https://arxiv.org/abs/2502.09509)
• Keywords: generative
• Abstract: Latent generative models have emerged as a leading approach for high-quality image synthesis. These models rely on an autoencoder to compress images into a latent space, followed by a generative model to learn the latent distribution. We identify that existing autoencoders lack equivariance to semantic-preserving transformations like scaling and rotation, resulting in complex latent spaces that hinder generative performance. To address this, we propose EQ-VAE, a simple regularization approach that enforces equivariance in the latent space, reducing its complexity without degrading reconstruction quality. By finetuning pre-trained autoencoders with EQ-VAE, we enhance the performance of several state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT, achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning. EQ-VAE is compatible with both continuous and discrete autoencoders, thus offering a versatile enhancement for a wide range of latent generative models. Project page and code: this https URL.
• 摘要：潜在生成模型已成为高质量图像合成的领先方法。这些模型依靠自动编码器将图像压缩到潜在空间中，然后是生成模型来学习潜在分布。我们确定现有的自动编码器缺乏对语义保护转换（如缩放和旋转）的等值线，从而导致复杂的潜在空间阻碍了生成性能。为了解决这个问题，我们提出了EQ-VAE，这是一种简单的正则化方法，可以在潜在空间中实现符号，从而降低了其复杂性而不会降低重建质量。通过使用EQ-VAE进行预先训练的自动编码器，我们提高了几种最先进的生成模型的性能，包括DIT，SIT，REPA和MASKGIT，仅在DIT-XL/2上实现了7个速度SD-VAE微调。 EQ-VAE与连续和离散的自动编码器均兼容，因此为广泛的潜在生成模型提供了多功能增强功能。项目页面和代码：此HTTPS URL。

Title: Diffusion Models for Molecules: A Survey of Methods and Tasks

• Authors: Liang Wang, Chao Song, Zhiyuan Liu, Yu Rong, Qiang Liu, Shu Wu, Liang Wang
• Subjects: cs.LG, cs.AI, cs.CE
• Abstract URL: https://arxiv.org/abs/2502.09511
• Pdf URL: https://arxiv.org/pdf/2502.09511
• Copy Paste: [[2502.09511]] Diffusion Models for Molecules: A Survey of Methods and Tasks(https://arxiv.org/abs/2502.09511)
• Keywords: generation, generative
• Abstract: Generative tasks about molecules, including but not limited to molecule generation, are crucial for drug discovery and material design, and have consistently attracted significant attention. In recent years, diffusion models have emerged as an impressive class of deep generative models, sparking extensive research and leading to numerous studies on their application to molecular generative tasks. Despite the proliferation of related work, there remains a notable lack of up-to-date and systematic surveys in this area. Particularly, due to the diversity of diffusion model formulations, molecular data modalities, and generative task types, the research landscape is challenging to navigate, hindering understanding and limiting the area's growth. To address this, this paper conducts a comprehensive survey of diffusion model-based molecular generative methods. We systematically review the research from the perspectives of methodological formulations, data modalities, and task types, offering a novel taxonomy. This survey aims to facilitate understanding and further flourishing development in this area. The relevant papers are summarized at: this https URL.
• 摘要：关于分子的生成任务，包括但不限于分子的产生，对于药物发现和材料设计至关重要，并且一直引起大幅关注。近年来，扩散模型已成为一类令人印象深刻的深层生成模型，引发了广泛的研究，并导致了许​​多研究其在分子生成任务中的应用。尽管相关工作激增，但在该领域仍缺乏最新和系统的调查。尤其是，由于扩散模型公式，分子数据模式和生成任务类型的多样性，研究格局在导航，阻碍理解和限制该地区的增长方面具有挑战性。为了解决这个问题，本文对基于扩散模型的分子生成方法进行了综合调查。我们从方法论表述，数据方式和任务类型的角度系统地回顾了研究，并提供了一种新颖的分类学。这项调查旨在促进该领域的理解和进一步发展。相关论文总结为：此HTTPS URL。

Title: SQ-GAN: Semantic Image Communications Using Masked Vector Quantization

• Authors: Francesco Pezone, Sergio Barbarossa, Giuseppe Caire
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2502.09520
• Pdf URL: https://arxiv.org/pdf/2502.09520
• Copy Paste: [[2502.09520]] SQ-GAN: Semantic Image Communications Using Masked Vector Quantization(https://arxiv.org/abs/2502.09520)
• Keywords: generative
• Abstract: This work introduces Semantically Masked VQ-GAN (SQ-GAN), a novel approach integrating generative models to optimize image compression for semantic/task-oriented communications. SQ-GAN employs off-the-shelf semantic semantic segmentation and a new specifically developed semantic-conditioned adaptive mask module (SAMM) to selectively encode semantically significant features of the images. SQ-GAN outperforms state-of-the-art image compression schemes such as JPEG2000 and BPG across multiple metrics, including perceptual quality and semantic segmentation accuracy on the post-decoding reconstructed image, at extreme low compression rates expressed in bits per pixel.
• 摘要：这项工作引入了语义掩盖的VQ-GAN（SQ-GAN），这是一种新颖的方法，该方法集成了生成模型，以优化语义/面向任务的通信的图像压缩。 SQ-GAN采用现成的语义语义分割，并采用了专门开发的语义调节自适应掩码模块（SAMM）来选择性编码图像的语义上具有重要意义的特征。 SQ-GAN优于多个度量标准的最先进的图像压缩方案，例如JPEG2000和BPG，包括对后编码后重建的图像的感知质量和语义分割精度，以每个像素的位表达的极低压缩率。

Title: Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model

• Authors: Fei Shen, Cong Wang, Junyao Gao, Qin Guo, Jisheng Dang, Jinhui Tang, Tat-Seng Chua
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09533
• Pdf URL: https://arxiv.org/pdf/2502.09533
• Copy Paste: [[2502.09533]] Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model(https://arxiv.org/abs/2502.09533)
• Keywords: generation
• Abstract: Recent advances in conditional diffusion models have shown promise for generating realistic TalkingFace videos, yet challenges persist in achieving consistent head movement, synchronized facial expressions, and accurate lip synchronization over extended generations. To address these, we introduce the \textbf{M}otion-priors \textbf{C}onditional \textbf{D}iffusion \textbf{M}odel (\textbf{MCDM}), which utilizes both archived and current clip motion priors to enhance motion prediction and ensure temporal consistency. The model consists of three key elements: (1) an archived-clip motion-prior that incorporates historical frames and a reference frame to preserve identity and context; (2) a present-clip motion-prior diffusion model that captures multimodal causality for accurate predictions of head movements, lip sync, and expressions; and (3) a memory-efficient temporal attention mechanism that mitigates error accumulation by dynamically storing and updating motion features. We also release the \textbf{TalkingFace-Wild} dataset, a multilingual collection of over 200 hours of footage across 10 languages. Experimental results demonstrate the effectiveness of MCDM in maintaining identity and motion continuity for long-term TalkingFace generation. Code, models, and datasets will be publicly available.
• 摘要：有条件扩散模型的最新进展显示出了产生逼真的说话视频的希望，但挑战在实现一致的头部移动，同步的面部表情以及在延长世代内准确的唇部同步。要解决这些问题，我们介绍了\ textbf {m} otion-priors \ textbf {c} onditional \ textbf {d} iffusion \ textbf {m} odel（\ textbf {mcdm}）增强运动预测并确保时间一致性。该模型由三个关键要素组成：（1）一个存档的CLIP MOTHIOT-PRIOR，其中包含历史框架和一个保留身份和背景的参考框架； （2）捕获多模式因果关系的当前旋转运动 - 扩散模型，以准确预测头部移动，唇部同步和表达； （3）一种记忆有效的时间注意机制，该机制通过动态存储和更新运动功能来减轻误差的积累。我们还发布了\ textbf {TalkingFace-wild}数据集，这是一个超过200个小时的10个语言镜头的多语言集合。实验结果证明了MCDM在长期说话表面生成的身份和运动连续性方面的有效性。代码，模型和数据集将公开可用。

Title: DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra

• Authors: Montgomery Bohde, Mrunali Manjrekar, Runzhong Wang, Shuiwang Ji, Connor W. Coley
• Subjects: cs.LG, q-bio.QM
• Abstract URL: https://arxiv.org/abs/2502.09571
• Pdf URL: https://arxiv.org/pdf/2502.09571
• Copy Paste: [[2502.09571]] DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra(https://arxiv.org/abs/2502.09571)
• Keywords: generation, generative
• Abstract: Mass spectrometry plays a fundamental role in elucidating the structures of unknown molecules and subsequent scientific discoveries. One formulation of the structure elucidation task is the conditional $\textit{de novo}$ generation of molecular structure given a mass spectrum. Toward a more accurate and efficient scientific discovery pipeline for small molecules, we present DiffMS, a formula-restricted encoder-decoder generative network that achieves state-of-the-art performance on this task. The encoder utilizes a transformer architecture and models mass spectra domain knowledge such as peak formulae and neutral losses, and the decoder is a discrete graph diffusion model restricted by the heavy-atom composition of a known chemical formula. To develop a robust decoder that bridges latent embeddings and molecular structures, we pretrain the diffusion decoder with fingerprint-structure pairs, which are available in virtually infinite quantities, compared to structure-spectrum pairs that number in the tens of thousands. Extensive experiments on established benchmarks show that DiffMS outperforms existing models on $\textit{de novo}$ molecule generation. We provide several ablations to demonstrate the effectiveness of our diffusion and pretraining approaches and show consistent performance scaling with increasing pretraining dataset size. DiffMS code is publicly available at this https URL.
• 摘要：质谱法在阐明未知分子的结构和随后的科学发现方面起着基本作用。结构阐明任务的一种表述是条件$ \ textit {de novo} $分子结构的产生给定质量谱。为了建立针对小分子的更准确，更有效的科学发现管道，我们提出了Diffms，这是一种公式限制的编码编码生成网络，可在此任务上实现最新的性能。编码器利用变压器结构，并模拟质谱域知识，例如峰值公式和中性损失，而解码器是由已知化学公式的重原子组成限制的离散图扩散模型。为了开发一个可靠的解码器，它可以弥合潜在的嵌入和分子结构，我们将扩散解码器与指纹结构对进行了预先解码器，与结构 - 光谱对相比，它们几乎以无限的数量获得，这些数字是无限的。在既定基准上进行的广泛实验表明，DIFFM在$ \ textit {de Novo} $分子生成上的现有模型优于现有模型。我们提供了几种消融，以证明我们扩散和预训练方法的有效性，并随着预处理的数据集大小显示一致的性能缩放。 DIFFMS代码可在此HTTPS URL上公开可用。

Title: Rolling Ahead Diffusion for Traffic Scene Simulation

• Authors: Yunpeng Liu, Matthew Niedoba, William Harvey, Adam Scibior, Berend Zwartsenberg, Frank Wood
• Subjects: cs.LG, cs.RO
• Abstract URL: https://arxiv.org/abs/2502.09587
• Pdf URL: https://arxiv.org/pdf/2502.09587
• Copy Paste: [[2502.09587]] Rolling Ahead Diffusion for Traffic Scene Simulation(https://arxiv.org/abs/2502.09587)
• Keywords: generation
• Abstract: Realistic driving simulation requires that NPCs not only mimic natural driving behaviors but also react to the behavior of other simulated agents. Recent developments in diffusion-based scenario generation focus on creating diverse and realistic traffic scenarios by jointly modelling the motion of all the agents in the scene. However, these traffic scenarios do not react when the motion of agents deviates from their modelled trajectories. For example, the ego-agent can be controlled by a stand along motion planner. To produce reactive scenarios with joint scenario models, the model must regenerate the scenario at each timestep based on new observations in a Model Predictive Control (MPC) fashion. Although reactive, this method is time-consuming, as one complete possible future for all NPCs is generated per simulation step. Alternatively, one can utilize an autoregressive model (AR) to predict only the immediate next-step future for all NPCs. Although faster, this method lacks the capability for advanced planning. We present a rolling diffusion based traffic scene generation model which mixes the benefits of both methods by predicting the next step future and simultaneously predicting partially noised further future steps at the same time. We show that such model is efficient compared to diffusion model based AR, achieving a beneficial compromise between reactivity and computational efficiency.
• 摘要：现实的驾驶模拟要求NPC不仅模仿自然驾驶行为，而且还对其他模拟药物的行为做出反应。基于扩散的方案生成的最新发展集中在共同建模现场所有代理的运动中，以创建各种各样的交通情况。但是，当代理人偏离其建模轨迹时，这些交通情况不会做出反应。例如，自我代理可以由运动计划者沿着支架控制。为了通过联合场景模型产生反应性场景，该模型必须基于模型预测控制（MPC）方式以新的观察结果来重新生成每个时间段的场景。尽管反应性，但这种方法是耗时的，因为每个NPC的一个完整的未来都是每个模拟步骤生成的。另外，可以利用自回归模型（AR）仅预测所有NPC的立即下一步未来。尽管更快，但这种方法缺乏高级计划的能力。我们提出了一个基于滚动扩散的交通场景生成模型，该模型通过预测下一步的未来并同时预测未来的未来步骤，从而混合了这两种方法的好处。我们表明，与基于扩散模型的AR相比，这种模型是有效的，从而实现了反应性和计算效率之间的有益折衷。

Title: Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs

• Authors: Siyan Zhao, Mingyi Hong, Yang Liu, Devamanyu Hazarika, Kaixiang Lin
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2502.09597
• Pdf URL: https://arxiv.org/pdf/2502.09597
• Copy Paste: [[2502.09597]] Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs(https://arxiv.org/abs/2502.09597)
• Keywords: generation
• Abstract: Large Language Models (LLMs) are increasingly used as chatbots, yet their ability to personalize responses to user preferences remains limited. We introduce PrefEval, a benchmark for evaluating LLMs' ability to infer, memorize and adhere to user preferences in a long-context conversational setting. PrefEval comprises 3,000 manually curated user preference and query pairs spanning 20 topics. PrefEval contains user personalization or preference information in both explicit and implicit forms, and evaluates LLM performance using a generation and a classification task. With PrefEval, we evaluated the aforementioned preference following capabilities of 10 open-source and proprietary LLMs in multi-session conversations with varying context lengths up to 100k tokens. We benchmark with various prompting, iterative feedback, and retrieval-augmented generation methods. Our benchmarking effort reveals that state-of-the-art LLMs face significant challenges in proactively following users' preferences during conversations. In particular, in zero-shot settings, preference following accuracy falls below 10% at merely 10 turns (~3k tokens) across most evaluated models. Even with advanced prompting and retrieval methods, preference following still deteriorates in long-context conversations. Furthermore, we show that fine-tuning on PrefEval significantly improves performance. We believe PrefEval serves as a valuable resource for measuring, understanding, and enhancing LLMs' preference following abilities, paving the way for personalized conversational agents. Our code and dataset are available at this https URL.
• 摘要：大型语言模型（LLMS）越来越多地用作聊天机器人，但它们对用户偏好的响应的个性化能力仍然有限。我们介绍了Prefefal，这是一种评估LLMS在长篇小说对话设置中推断，记忆和遵守用户偏好的能力的基准。前传说包括3,000个手动策划的用户偏好和查询对，涵盖20个主题。前传说包含用户的个性化或偏好信息，以显式和隐式形式，并使用一代和分类任务评估LLM性能。借助前期，我们评估了上述偏好，遵循10个开源和专有LLMS的多主题对话的功能，其上下文长度最高为100K令牌。我们通过各种提示，迭代反馈和检索增强的生成方法进行基准测试。我们的基准测试工作表明，最先进的LLM在主动遵循用户在对话中的偏好方面面临重大挑战。特别是，在零拍设置中，在大多数评估的模型中，精度之后的偏好仅在10圈（〜3k令牌）下降至10％以下。即使采用先进的提示和检索方法，遵循的偏好仍会在长篇小说对话中恶化。此外，我们表明，对前幕进行微调可显着提高性能。我们认为，前传说是衡量，理解和增强LLMS偏好遵循能力的宝贵资源，为个性化的对话代理铺平了道路。我们的代码和数据集可在此HTTPS URL上找到。

Title: GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis

• Authors: Angelos Zavras, Dimitrios Michail, Xiao Xiang Zhu, Begüm Demir, Ioannis Papoutsis
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09598
• Pdf URL: https://arxiv.org/pdf/2502.09598
• Copy Paste: [[2502.09598]] GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis(https://arxiv.org/abs/2502.09598)
• Keywords: generation
• Abstract: The continuous operation of Earth-orbiting satellites generates vast and ever-growing archives of Remote Sensing (RS) images. Natural language presents an intuitive interface for accessing, querying, and interpreting the data from such archives. However, existing Vision-Language Models (VLMs) are predominantly trained on web-scraped, noisy image-text data, exhibiting limited exposure to the specialized domain of RS. This deficiency results in poor performance on RS-specific tasks, as commonly used datasets often lack detailed, scientifically accurate textual descriptions and instead emphasize solely on attributes like date and location. To bridge this critical gap, we introduce GAIA, a novel dataset designed for multi-scale, multi-sensor, and multi-modal RS image analysis. GAIA comprises of 205,150 meticulously curated RS image-text pairs, representing a diverse range of RS modalities associated to different spatial resolutions. Unlike existing vision-language datasets in RS, GAIA specifically focuses on capturing a diverse range of RS applications, providing unique information about environmental changes, natural disasters, and various other dynamic phenomena. The dataset provides a spatially and temporally balanced distribution, spanning across the globe, covering the last 25 years with a balanced temporal distribution of observations. GAIA's construction involved a two-stage process: (1) targeted web-scraping of images and accompanying text from reputable RS-related sources, and (2) generation of five high-quality, scientifically grounded synthetic captions for each image using carefully crafted prompts that leverage the advanced vision-language capabilities of GPT-4o. Our extensive experiments, including fine-tuning of CLIP and BLIP2 models, demonstrate that GAIA significantly improves performance on RS image classification, cross-modal retrieval and image captioning tasks.
• 摘要：地球卫星的连续操作产生了遥不可及的遥感（RS）图像档案。自然语言提出了一个直观的界面，用于访问，查询和解释此类档案的数据。然而，现有的视觉模型（VLM）主要是在网络结束的，嘈杂的图像文本数据上训练的，表现出对RS专业领域的有限接触。这种缺陷会导致RS特定任务的性能不佳，因为常用的数据集通常缺乏详细的，科学的准确的文本描述，而仅强调日期和位置等属性。为了弥合这个关键的差距，我们介绍了Gaia，Gaia是一种专为多尺度，多传感器和多模式RS图像分析而设计的新型数据集。盖亚（Gaia）由205,150个精心策划的RS图像文本对组成，代表与不同空间分辨率相关的各种RS模态。与RS中的现有视觉语言数据集不同，Gaia专门专注于捕获各种RS应用程序，提供有关环境变化，自然灾害和其他各种动态现象的独特信息。该数据集提供了一个空间和时间平衡的分布，遍布全球，涵盖了过去25年，观察值的时间分布平衡。 Gaia的构造涉及一个两个阶段的过程：（1）来自知名RS相关资源的图像和随附文本的针对性网络剪贴，以及（2）使用精心制作的提示为每个图像创造了五个高质量的，科学的基于科学的合成字幕这利用了GPT-4O的先进视力语言功能。我们的广泛实验（包括夹子和BLIP2模型的微调）表明，Gaia显着提高了RS图像分类，跨模式检索和图像字幕任务的性能。

Title: Score-of-Mixture Training: Training One-Step Generative Models Made Simple

• Authors: Tejas Jayashankar, J. Jon Ryu, Gregory Wornell
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2502.09609
• Pdf URL: https://arxiv.org/pdf/2502.09609
• Copy Paste: [[2502.09609]] Score-of-Mixture Training: Training One-Step Generative Models Made Simple(https://arxiv.org/abs/2502.09609)
• Keywords: generative
• Abstract: We propose Score-of-Mixture Training (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the $\alpha$-skew Jensen-Shannon divergence. At its core, SMT estimates the score of mixture distributions between real and fake samples across multiple noise levels. Similar to consistency models, our approach supports both training from scratch (SMT) and distillation using a pretrained diffusion model, which we call Score-of-Mixture Distillation (SMD). It is simple to implement, requires minimal hyperparameter tuning, and ensures stable training. Experiments on CIFAR-10 and ImageNet 64x64 show that SMT/SMD are competitive with and can even outperform existing methods.
• 摘要：我们提出了分数训练（SMT），这是一个新型训练单步生成模型的框架，方法是最大程度地减少称为$ \ alpha $ -skew Jensen-Shannon Divergence的分歧。 SMT以多种噪声水平估计了真实样品和假样品之间的混合分布的分数。与一致性模型相似，我们的方法使用验证的扩散模型支持从头开始的训练（SMT）和蒸馏，我们称之为混合分数蒸馏（SMD）。实施非常简单，需要最小的高参数调整并确保稳定的培训。 CIFAR-10和Imagenet 64x64上的实验表明，SMT/SMD具有竞争力，甚至可以超过现有方法。

Title: Designing a Conditional Prior Distribution for Flow-Based Generative Models

• Authors: Noam Issachar, Mohammad Salama, Raanan Fattal, Sagie Benaim
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09611
• Pdf URL: https://arxiv.org/pdf/2502.09611
• Copy Paste: [[2502.09611]] Designing a Conditional Prior Distribution for Flow-Based Generative Models(https://arxiv.org/abs/2502.09611)
• Keywords: generation, generative
• Abstract: Flow-based generative models have recently shown impressive performance for conditional generation tasks, such as text-to-image generation. However, current methods transform a general unimodal noise distribution to a specific mode of the target data distribution. As such, every point in the initial source distribution can be mapped to every point in the target distribution, resulting in long average paths. To this end, in this work, we tap into a non-utilized property of conditional flow-based models: the ability to design a non-trivial prior distribution. Given an input condition, such as a text prompt, we first map it to a point lying in data space, representing an ``average" data point with the minimal average distance to all data points of the same conditional mode (e.g., class). We then utilize the flow matching formulation to map samples from a parametric distribution centered around this point to the conditional target distribution. Experimentally, our method significantly improves training times and generation efficiency (FID, KID and CLIP alignment scores) compared to baselines, producing high quality samples using fewer sampling steps.
• 摘要：基于流量的生成模型最近显示了有条件生成任务的令人印象深刻的性能，例如文本对图像生成。但是，当前方法将一般的单峰噪声分布转换为目标数据分布的特定模式。因此，可以将初始源分布中的每个点映射到目标分布中的每个点，从而导致较长的平均路径。为此，在这项工作中，我们利用了基于条件流的模型的非利用属性：设计非平凡的先验分布的能力。给定输入条件，例如文本提示然后，我们利用流量匹配的公式来绘制围绕有条件的目标分布的参数分布。使用更少的采样步骤的高质量样品。

Title: Latent Radiance Fields with 3D-aware 2D Representations

• Authors: Chaoyi Zhou, Xi Liu, Feng Luo, Siyu Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2502.09613
• Pdf URL: https://arxiv.org/pdf/2502.09613
• Copy Paste: [[2502.09613]] Latent Radiance Fields with 3D-aware 2D Representations(https://arxiv.org/abs/2502.09613)
• Keywords: generation
• Abstract: Latent 3D reconstruction has shown great promise in empowering 3D semantic understanding and 3D generation by distilling 2D features into the 3D space. However, existing approaches struggle with the domain gap between 2D feature space and 3D representations, resulting in degraded rendering performance. To address this challenge, we propose a novel framework that integrates 3D awareness into the 2D latent space. The framework consists of three stages: (1) a correspondence-aware autoencoding method that enhances the 3D consistency of 2D latent representations, (2) a latent radiance field (LRF) that lifts these 3D-aware 2D representations into 3D space, and (3) a VAE-Radiance Field (VAE-RF) alignment strategy that improves image decoding from the rendered 2D representations. Extensive experiments demonstrate that our method outperforms the state-of-the-art latent 3D reconstruction approaches in terms of synthesis performance and cross-dataset generalizability across diverse indoor and outdoor scenes. To our knowledge, this is the first work showing the radiance field representations constructed from 2D latent representations can yield photorealistic 3D reconstruction performance.
• 摘要：潜在的3D重建在赋予3D语义理解和3D生成方面表现出巨大的希望，这是通过将2D特征提炼到3D空间中的。但是，现有的方法与2D特征空间和3D表示之间的域间隙困难，从而导致渲染性能下降。为了应对这一挑战，我们提出了一个新颖的框架，将3D意识集成到2D潜在空间中。该框架由三个阶段组成：（1）一种对应感知的自动编码方法，可以增强2D潜在表示的3D一致性，（2）将这些3D-Aware 2D表示为3D空间，以及（ 3）VAE-RADIANCE场（VAE-RF）对齐策略，可改善从渲染的2D表示形式中解码的图像。广泛的实验表明，我们的方法优于最先进的潜在3D重建方法，在综合性能和跨室外场景的综合性能和跨数据集的概括方面。据我们所知，这是展示由2D潜在表示构建的辐射场表示的第一部作品，可以产生逼真的3D重建性能。

Title: Theoretical Benefit and Limitation of Diffusion Language Model

• Authors: Guhao Feng, Yihan Geng, Jian Guan, Wei Wu, Liwei Wang, Di He
• Subjects: cs.LG, cs.AI, cs.CL, stat.ML
• Abstract URL: https://arxiv.org/abs/2502.09622
• Pdf URL: https://arxiv.org/pdf/2502.09622
• Copy Paste: [[2502.09622]] Theoretical Benefit and Limitation of Diffusion Language Model(https://arxiv.org/abs/2502.09622)
• Keywords: generation
• Abstract: Diffusion language models have emerged as a promising approach for text generation. One would naturally expect this method to be an efficient replacement for autoregressive models since multiple tokens can be sampled in parallel during each diffusion step. However, its efficiency-accuracy trade-off is not yet well understood. In this paper, we present a rigorous theoretical analysis of a widely used type of diffusion language model, the Masked Diffusion Model (MDM), and find that its effectiveness heavily depends on the target evaluation metric. Under mild conditions, we prove that when using perplexity as the metric, MDMs can achieve near-optimal perplexity in sampling steps regardless of sequence length, demonstrating that efficiency can be achieved without sacrificing performance. However, when using the sequence error rate--which is important for understanding the "correctness" of a sequence, such as a reasoning chain--we show that the required sampling steps must scale linearly with sequence length to obtain "correct" sequences, thereby eliminating MDM's efficiency advantage over autoregressive models. Our analysis establishes the first theoretical foundation for understanding the benefits and limitations of MDMs. All theoretical findings are supported by empirical studies.
• 摘要：扩散语言模型已成为文本生成的有前途的方法。人们自然会期望这种方法是自回归模型的有效替换，因为在每个扩散步骤中可以并行采样多个令牌。但是，其效率准确的权衡尚未得到充分理解。在本文中，我们对广泛使用类型的扩散语言模型（掩盖扩散模型（MDM））进行了严格的理论分析，并发现其有效性在很大程度上取决于目标评估指标。在轻度条件下，我们证明，当使用困惑作为度量标准时，MDMS在抽样步骤中都可以达到近乎最佳的困惑，无论序列长度如何，都表明可以在不牺牲性能的情况下实现效率。但是，当使用序列错误率（对于理解序列的“正确性”（例如推理链）非常重要）时，我们表明所需的采样步骤必须以序列长度线性扩展，以获得“正确的”序列，从而消除了MDM比自回归模型的效率优势。我们的分析建立了理解MDM的收益和局限性的第一个理论基础。所有理论发现都得到经验研究的支持。