2025-04-28

Title: Spectral Dictionary Learning for Generative Image Modeling

• Authors: Andrew Kiruluta
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.17804
• Pdf URL: https://arxiv.org/pdf/2504.17804
• Copy Paste: [[2504.17804]] Spectral Dictionary Learning for Generative Image Modeling(https://arxiv.org/abs/2504.17804)
• Keywords: generation, generative
• Abstract: We propose a novel spectral generative model for image synthesis that departs radically from the common variational, adversarial, and diffusion paradigms. In our approach, images, after being flattened into one-dimensional signals, are reconstructed as linear combinations of a set of learned spectral basis functions, where each basis is explicitly parameterized in terms of frequency, phase, and amplitude. The model jointly learns a global spectral dictionary with time-varying modulations and per-image mixing coefficients that quantify the contributions of each spectral component. Subsequently, a simple probabilistic model is fitted to these mixing coefficients, enabling the deterministic generation of new images by sampling from the latent space. This framework leverages deterministic dictionary learning, offering a highly interpretable and physically meaningful representation compared to methods relying on stochastic inference or adversarial training. Moreover, the incorporation of frequency-domain loss functions, computed via the short-time Fourier transform (STFT), ensures that the synthesized images capture both global structure and fine-grained spectral details, such as texture and edge information. Experimental evaluations on the CIFAR-10 benchmark demonstrate that our approach not only achieves competitive performance in terms of reconstruction quality and perceptual fidelity but also offers improved training stability and computational efficiency. This new type of generative model opens up promising avenues for controlled synthesis, as the learned spectral dictionary affords a direct handle on the intrinsic frequency content of the images, thus providing enhanced interpretability and potential for novel applications in image manipulation and analysis.
• 摘要：我们提出了一个新型的光谱生成模型，用于图像合成，该模型从根本上偏离了常见的变异，对抗和扩散范式。在我们的方法中，将图像被扁平化为一维信号后，被重建为一组学习的光谱基函数的线性组合，其中每个基础都以频率，相位和振幅的方式明确参数化。该模型共同学习了一个全球光谱词典，并通过时变调制和每图像混合系数来量化每个光谱成分的贡献。随后，将一个简单的概率模型拟合到这些混合系数中，从而通过从潜在空间中取样来确定性生成新图像。该框架利用确定性的词典学习，与依靠随机推理或对抗性训练的方法相比，提供了高度可解释且物理上有意义的表示。此外，通过短时傅立叶变换（STFT）计算的频域损耗函数的结合可确保合成的图像既捕获全局结构和细粒光谱细节，例如纹理和边缘信息。对CIFAR-10基准的实验评估表明，我们的方法不仅在重建质量和感知忠诚度方面取得了竞争性能，而且还提供了提高的训练稳定性和计算效率。这种新型的生成模型为受控综合开辟了有希望的途径，因为学习的光谱词典为图像的固有频率内容提供了直接处理，从而为图像操纵和分析提供了新颖的解释性和潜力。

Title: Subject-driven Video Generation via Disentangled Identity and Motion

• Authors: Daneul Kim, Jingxu Zhang, Wonjoon Jin, Sunghyun Cho, Qi Dai, Jaesik Park, Chong Luo
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2504.17816
• Pdf URL: https://arxiv.org/pdf/2504.17816
• Copy Paste: [[2504.17816]] Subject-driven Video Generation via Disentangled Identity and Motion(https://arxiv.org/abs/2504.17816)
• Keywords: generation
• Abstract: We propose to train a subject-driven customized video generation model through decoupling the subject-specific learning from temporal dynamics in zero-shot without additional tuning. A traditional method for video customization that is tuning-free often relies on large, annotated video datasets, which are computationally expensive and require extensive annotation. In contrast to the previous approach, we introduce the use of an image customization dataset directly on training video customization models, factorizing the video customization into two folds: (1) identity injection through image customization dataset and (2) temporal modeling preservation with a small set of unannotated videos through the image-to-video training method. Additionally, we employ random image token dropping with randomized image initialization during image-to-video fine-tuning to mitigate the copy-and-paste issue. To further enhance learning, we introduce stochastic switching during joint optimization of subject-specific and temporal features, mitigating catastrophic forgetting. Our method achieves strong subject consistency and scalability, outperforming existing video customization models in zero-shot settings, demonstrating the effectiveness of our framework.
• 摘要：我们建议通过在零射击中从时间动力学中解除特定于主题的学习，而无需进行其他调整，以训练主题驱动的自定义视频生成模型。无调的传统视频定制方法通常依赖于大型注释的视频数据集，这些数据集在计算上昂贵，需要广泛的注释。与以前的方法相反，我们直接在训练视频自定义模型上介绍了图像自定义数据集的使用，将视频自定义分为两个折叠：（1）通过图像自定义数据集注入身份，以​​及（2）通过图像训练方法通过一小部分未经通知的视频进行时间建模保存。此外，我们在图像到视频微调过程中使用随机图像初始化采用随机图像令牌下降，以减轻复制和贴止问题。为了进一步增强学习，我们在主题特定和时间特征的关节优化期间引入了随机切换，从而减轻了灾难性的遗忘。我们的方法实现了强大的主题一致性和可扩展性，在零摄影设置中优于现有的视频自定义模型，证明了我们框架的有效性。

Title: Dual Prompting Image Restoration with Diffusion Transformers

• Authors: Dehong Kong, Fan Li, Zhixin Wang, Jiaqi Xu, Renjing Pei, Wenbo Li, WenQi Ren
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17825
• Pdf URL: https://arxiv.org/pdf/2504.17825
• Copy Paste: [[2504.17825]] Dual Prompting Image Restoration with Diffusion Transformers(https://arxiv.org/abs/2504.17825)
• Keywords: restoration
• Abstract: Recent state-of-the-art image restoration methods mostly adopt latent diffusion models with U-Net backbones, yet still facing challenges in achieving high-quality restoration due to their limited capabilities. Diffusion transformers (DiTs), like SD3, are emerging as a promising alternative because of their better quality with scalability. In this paper, we introduce DPIR (Dual Prompting Image Restoration), a novel image restoration method that effectivly extracts conditional information of low-quality images from multiple perspectives. Specifically, DPIR consits of two branches: a low-quality image conditioning branch and a dual prompting control branch. The first branch utilizes a lightweight module to incorporate image priors into the DiT with high efficiency. More importantly, we believe that in image restoration, textual description alone cannot fully capture its rich visual characteristics. Therefore, a dual prompting module is designed to provide DiT with additional visual cues, capturing both global context and local appearance. The extracted global-local visual prompts as extra conditional control, alongside textual prompts to form dual prompts, greatly enhance the quality of the restoration. Extensive experimental results demonstrate that DPIR delivers superior image restoration performance.
• 摘要：最近的最新图像恢复方法主要采用带有U-NET骨架的潜在扩散模型，但由于功能有限，在实现高质量恢复方面仍面临挑战。扩散变压器（DITS），例如SD3，由于其质量更好，并具有可扩展性，因此成为了有希望的选择。在本文中，我们介绍了DPIR（双重提示图像恢复），这是一种新型的图像恢复方法，可从多个角度从多个角度提取低质量图像的条件信息。具体而言，两个分支的DPIR提示：一个低质量的图像调节分支和双重提示控制分支。第一个分支利用轻量级模块以高效率将图像先验纳入DIT。更重要的是，我们认为，在图像恢复中，仅文本描述不能完全捕获其丰富的视觉特征。因此，双重提示模块旨在为DIT提供其他视觉提示，从而捕获全球上下文和本地外观。提取的全部本地视觉提示作为额外的条件控制，以及文本提示形成双重提示，极大地提高了修复的质量。广泛的实验结果表明，DPIR可以提供出色的图像恢复性能。

Title: FashionM3: Multimodal, Multitask, and Multiround Fashion Assistant based on Unified Vision-Language Model

• Authors: Kaicheng Pang, Xingxing Zou, Waikeung Wong
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.17826
• Pdf URL: https://arxiv.org/pdf/2504.17826
• Copy Paste: [[2504.17826]] FashionM3: Multimodal, Multitask, and Multiround Fashion Assistant based on Unified Vision-Language Model(https://arxiv.org/abs/2504.17826)
• Keywords: generation
• Abstract: Fashion styling and personalized recommendations are pivotal in modern retail, contributing substantial economic value in the fashion industry. With the advent of vision-language models (VLM), new opportunities have emerged to enhance retailing through natural language and visual interactions. This work proposes FashionM3, a multimodal, multitask, and multiround fashion assistant, built upon a VLM fine-tuned for fashion-specific tasks. It helps users discover satisfying outfits by offering multiple capabilities including personalized recommendation, alternative suggestion, product image generation, and virtual try-on simulation. Fine-tuned on the novel FashionRec dataset, comprising 331,124 multimodal dialogue samples across basic, personalized, and alternative recommendation tasks, FashionM3 delivers contextually personalized suggestions with iterative refinement through multiround interactions. Quantitative and qualitative evaluations, alongside user studies, demonstrate FashionM3's superior performance in recommendation effectiveness and practical value as a fashion assistant.
• 摘要：时尚风格和个性化建议在现代零售业中至关重要，在时装行业中贡献了实质性的经济价值。随着视觉模型（VLM）的出现，出现了新的机会，可以通过自然语言和视觉互动来增强零售。这项工作提出了FashionM3，这是一种多式联运，多任务和多个时尚助理，建立在VLM微调的，用于特定于时尚的任务。它通过提供多种功能，包括个性化建议，替代建议，产品图像生成和虚拟的尝试模拟，帮助用户发现满足服装的满足服装。 FashionM3对新型FashionRec数据集进行了微调，其中包括基本，个性化和替代性建议任务的331,124个多模式对话样本，通过多个互动，通过迭代改进提供了上下文个性化建议。定量和定性评估，以及用户研究，证明了时尚助理的推荐有效性和实践价值的出色表现。

Title: Self-Balancing, Memory Efficient, Dynamic Metric Space Data Maintenance, for Rapid Multi-Kernel Estimation

• Authors: Aditya S Ellendula, Chandrajit Bajaj
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.18003
• Pdf URL: https://arxiv.org/pdf/2504.18003
• Copy Paste: [[2504.18003]] Self-Balancing, Memory Efficient, Dynamic Metric Space Data Maintenance, for Rapid Multi-Kernel Estimation(https://arxiv.org/abs/2504.18003)
• Keywords: generation, generative
• Abstract: We present a dynamic self-balancing octree data structure that enables efficient neighborhood maintenance in evolving metric spaces, a key challenge in modern machine learning systems. Many learning and generative models operate as dynamical systems whose representations evolve during training, requiring fast, adaptive spatial organization. Our two-parameter octree supports logarithmic-time updates and queries, eliminating the need for costly full rebuilds as data distributions shift. We demonstrate its effectiveness in four areas: (1) accelerating Stein variational gradient descent by supporting more particles with lower overhead; (2) enabling real-time, incremental KNN classification with logarithmic complexity; (3) facilitating efficient, dynamic indexing and retrieval for retrieval-augmented generation; and (4) improving sample efficiency by jointly optimizing input and latent spaces. Across all applications, our approach yields exponential speedups while preserving accuracy, particularly in high-dimensional spaces where maintaining adaptive spatial structure is critical.
• 摘要：我们提出了动态的自我平衡OCTREE数据结构，该结构能够在不断发展的度量空间中有效的邻里维护，这是现代机器学习系统中的主要挑战。许多学习和生成模型都是动态系统，其表示在培训期间的表示，需要快速，适应性的空间组织。我们的两参数OCTREE支持对数时间更新和查询，无需随着数据分布的转移而需要昂贵的全面重建。我们在四个领域中证明了它的有效性：（1）通过支撑更多较低开销的颗粒来加速Stein变异梯度下降； （2）实现实时的，增量的KNN分类，并具有对数复杂性； （3）促进有效的，动态的索引和检索，以获取检索发电机； （4）通过共同优化输入和潜在空间来提高样品效率。在所有应用中，我们的方法都会产生指数加速的同时保持准确性，尤其是在维持适应性空间结构的高维空间中，至关重要。

Title: Cabbage: A Differential Growth Framework for Open Surfaces

• Authors: Xiaoyi Liu, Hao Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18040
• Pdf URL: https://arxiv.org/pdf/2504.18040
• Copy Paste: [[2504.18040]] Cabbage: A Differential Growth Framework for Open Surfaces(https://arxiv.org/abs/2504.18040)
• Keywords: generation
• Abstract: We propose Cabbage, a differential growth framework to model buckling behavior in 3D open surfaces found in nature-like the curling of flower petals. Cabbage creates high-quality triangular meshes free of self-intersection. Cabbage-Shell is driven by edge subdivision which differentially increases discretization resolution. Shell forces expands the surface, generating buckling over time. Feature-aware smoothing and remeshing ensures mesh quality. Corrective collision effectively prevents self-collision even in tight spaces. We additionally provide Cabbage-Collision, and approximate alternative, followed by CAD-ready surface generation. Cabbage is the first open-source effort with this calibre and robustness, outperforming SOTA methods in its morphological expressiveness, mesh quality, and stably generates large, complex patterns over hundreds of simulation steps. It is a source not only of computational modeling, digital fabrication, education, but also high-quality, annotated data for geometry processing and shape analysis.
• 摘要：我们提出了白菜，这是一个差异生长框架，以模拟在花瓣的卷发中发现的3D开放表面中的屈曲行为。卷心菜创建了没有自我交流的高质量三角形网格。白菜壳是由边缘细分驱动的，该细分差异增加了离散化的分辨率。壳力扩大表面，随着时间的推移产生屈曲。功能感知的平滑和重新融合可确保网状质量。即使在狭窄的空间中，纠正碰撞也可以有效防止自我碰撞。我们还提供卷心菜胶水和近似替代方案，然后提供可加入CAD的表面产生。卷心菜是这种能力和鲁棒性的第一个开源工作，其形态表达性，网状质量优于SOTA方法，并且在数百个模拟步骤中稳定地产生了大型，复杂的模式。它不仅是计算建模，数字制造，教育的来源，而且是用于几何处理和形状分析的高质量的注释数据。

Title: Privacy-Preserving Personalized Federated Learning for Distributed Photovoltaic Disaggregation under Statistical Heterogeneity

• Authors: Xiaolu Chen, Chenghao Huang, Yanru Zhang, Hao Wang
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.18078
• Pdf URL: https://arxiv.org/pdf/2504.18078
• Copy Paste: [[2504.18078]] Privacy-Preserving Personalized Federated Learning for Distributed Photovoltaic Disaggregation under Statistical Heterogeneity(https://arxiv.org/abs/2504.18078)
• Keywords: generation
• Abstract: The rapid expansion of distributed photovoltaic (PV) installations worldwide, many being behind-the-meter systems, has significantly challenged energy management and grid operations, as unobservable PV generation further complicates the supply-demand balance. Therefore, estimating this generation from net load, known as PV disaggregation, is critical. Given privacy concerns and the need for large training datasets, federated learning becomes a promising approach, but statistical heterogeneity, arising from geographical and behavioral variations among prosumers, poses new challenges to PV disaggregation. To overcome these challenges, a privacy-preserving distributed PV disaggregation framework is proposed using Personalized Federated Learning (PFL). The proposed method employs a two-level framework that combines local and global modeling. At the local level, a transformer-based PV disaggregation model is designed to generate solar irradiance embeddings for representing local PV conditions. A novel adaptive local aggregation mechanism is adopted to mitigate the impact of statistical heterogeneity on the local model, extracting a portion of global information that benefits the local model. At the global level, a central server aggregates information uploaded from multiple data centers, preserving privacy while enabling cross-center knowledge sharing. Experiments on real-world data demonstrate the effectiveness of this proposed framework, showing improved accuracy and robustness compared to benchmark methods.
• 摘要：全球分布式光伏（PV）装置的迅速扩展，许多是幕后系统，都大大挑战了能源管理和电网操作，因为不可观察的PV生成使供应需求的平衡变得更加复杂。因此，从净负荷（称为PV分解）中估算这一代至关重要。鉴于隐私问题和大型培训数据集的需求，联邦学习成为一种有前途的方法，但是统计异质性是由生产者之间的地理和行为变化引起的，对PV分类构成了新的挑战。为了克服这些挑战，使用个性化联合学习（PFL）提出了保护隐私的分布式PV分解框架。所提出的方法采用了将本地和全球建模结合在一起的两级框架。在本地一级，基于变压器的PV分解模型旨在生成用于表示本地PV条件的太阳辐照度嵌入。采用了一种新型的自适应局部聚合机制来减轻统计异质性对本地模型的影响，从而提取了有益于本地模型的一部分全球信息。在全球层面，中央服务器汇总了从多个数据中心上传的信息，在启用跨中心知识共享的同时保留隐私。对现实世界数据的实验证明了该提出的框架的有效性，与基准方法相比，其准确性和鲁棒性提高了。

Title: Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation

• Authors: Weipeng Tan, Chuming Lin, Chengming Xu, FeiFan Xu, Xiaobin Hu, Xiaozhong Ji, Junwei Zhu, Chengjie Wang, Yanwei Fu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18087
• Pdf URL: https://arxiv.org/pdf/2504.18087
• Copy Paste: [[2504.18087]] Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation(https://arxiv.org/abs/2504.18087)
• Keywords: generation
• Abstract: Recent advances in Talking Head Generation (THG) have achieved impressive lip synchronization and visual quality through diffusion models; yet existing methods struggle to generate emotionally expressive portraits while preserving speaker identity. We identify three critical limitations in current emotional talking head generation: insufficient utilization of audio's inherent emotional cues, identity leakage in emotion representations, and isolated learning of emotion correlations. To address these challenges, we propose a novel framework dubbed as DICE-Talk, following the idea of disentangling identity with emotion, and then cooperating emotions with similar characteristics. First, we develop a disentangled emotion embedder that jointly models audio-visual emotional cues through cross-modal attention, representing emotions as identity-agnostic Gaussian distributions. Second, we introduce a correlation-enhanced emotion conditioning module with learnable Emotion Banks that explicitly capture inter-emotion relationships through vector quantization and attention-based feature aggregation. Third, we design an emotion discrimination objective that enforces affective consistency during the diffusion process through latent-space classification. Extensive experiments on MEAD and HDTF datasets demonstrate our method's superiority, outperforming state-of-the-art approaches in emotion accuracy while maintaining competitive lip-sync performance. Qualitative results and user studies further confirm our method's ability to generate identity-preserving portraits with rich, correlated emotional expressions that naturally adapt to unseen identities.
• 摘要：通过扩散模型，谈话头部发电（THG）的最新进展（THG）取得了令人印象深刻的同步和视觉质量。然而，现有的方法难以在保持演讲者身份的同时产生情感表达的肖像。我们确定了当前情感上的话题的三个临界局限性：对音频固有的情绪提示的利用不足，情绪表征中的身份泄漏以及情绪相关性的孤立学习。为了应对这些挑战，我们提出了一个新颖的框架，称为骰子谈话，遵循将身份解散为情感，然后与具有相似特征的情绪合作的想法。首先，我们开发出一个分离的情感嵌入器，该嵌入者通过交叉模式的注意力共同对视听情感提示进行建模，将情绪表示为身份敏捷的高斯分布。其次，我们通过可学习的情感库介绍了一个相关增强的情绪调节模块，该模块通过向量量化和基于注意力的特征聚合明确捕获情感关系。第三，我们设计了一个情感歧视目标，该目标通过潜在空间分类在扩散过程中实现情感一致性。关于Mead和HDTF数据集的广泛实验证明了我们的方法的优势，在保持竞争性的LIP同步性能的同时，在情感准确性方面表现优于最先进的方法。定性结果和用户研究进一步证实了我们方法具有自然适应看不见的身份的丰富，相关的情感表达的丰富，相关的情绪表达的能力。

Title: Salient Region-Guided Spacecraft Image Arbitrary-Scale Super-Resolution Network

• Authors: Jingfan Yang, Hu Gao, Ying Zhang, Depeng Dang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18127
• Pdf URL: https://arxiv.org/pdf/2504.18127
• Copy Paste: [[2504.18127]] Salient Region-Guided Spacecraft Image Arbitrary-Scale Super-Resolution Network(https://arxiv.org/abs/2504.18127)
• Keywords: super-resolution
• Abstract: Spacecraft image super-resolution seeks to enhance low-resolution spacecraft images into high-resolution ones. Although existing arbitrary-scale super-resolution methods perform well on general images, they tend to overlook the difference in features between the spacecraft core region and the large black space background, introducing irrelevant noise. In this paper, we propose a salient region-guided spacecraft image arbitrary-scale super-resolution network (SGSASR), which uses features from the spacecraft core salient regions to guide latent modulation and achieve arbitrary-scale super-resolution. Specifically, we design a spacecraft core region recognition block (SCRRB) that identifies the core salient regions in spacecraft images using a pre-trained saliency detection model. Furthermore, we present an adaptive-weighted feature fusion enhancement mechanism (AFFEM) to selectively aggregate the spacecraft core region features with general image features by dynamic weight parameter to enhance the response of the core salient regions. Experimental results demonstrate that the proposed SGSASR outperforms state-of-the-art approaches.
• 摘要：航天器图像超分辨率旨在将低分辨率航天器图像增强到高分辨率图像中。尽管现有的任意规模的超分辨率方法在一般图像上表现良好，但它们倾向于忽略航天器核心区域和大型黑色空间背景之间特征的差异，从而引入了无关的噪声。在本文中，我们提出了一个显着的区域引导的航天器图像任意规模的超分辨率网络（SGSASR），该网络使用航天器核心明显区域的特征来指导潜在调制并实现任意规模的超级分辨率。具体而言，我们设计了一个航天器核心区域识别块（SCRRB），该块使用预训练的显着检测模型来识别航天器图像中的核心显着区域。此外，我们提出了一种自适应加权特征融合增强机制（FAFFEM），以选择性地汇总具有动态重量参数具有一般图像特征的航天器核心区域特征，以增强核心良好区域的响应。实验结果表明，所提出的SGSASR优于最先进的方法。

Title: Score-Based Deterministic Density Sampling

• Authors: Vasily Ilin, Bamdad Hosseini, Jingwei Hu
• Subjects: cs.LG, math.PR, math.ST
• Abstract URL: https://arxiv.org/abs/2504.18130
• Pdf URL: https://arxiv.org/pdf/2504.18130
• Copy Paste: [[2504.18130]] Score-Based Deterministic Density Sampling(https://arxiv.org/abs/2504.18130)
• Keywords: generative
• Abstract: We propose and analyze a deterministic sampling framework using Score-Based Transport Modeling (SBTM) for sampling an unnormalized target density $\pi$. While diffusion generative modeling relies on pre-training the score function $\nabla \log f_t$ using samples from $\pi$, SBTM addresses the more general and challenging setting where only $\nabla \log\pi$ is known. SBTM approximates the Wasserstein gradient flow on KL$(f_t\|\pi)$ by learning the time-varying score $\nabla \log f_t$ on the fly using score matching. The learned score gives immediate access to relative Fisher information, providing a built-in convergence diagnostic. The deterministic trajectories are smooth, interpretable, and free of Brownian-motion noise, while having the same distribution as ULA. We prove that SBTM dissipates relative entropy at the same rate as the exact gradient flow, provided sufficient training. We further extend our framework to annealed dynamics, to handle non log-concave targets. Numerical experiments validate our theoretical findings: SBTM converges at the optimal rate, has smooth trajectories, and is easily integrated with annealed dynamics. We compare to the baselines of ULA and annealed ULA.
• 摘要：我们建议使用基于得分的传输建模（SBTM）来采样非标准化目标密度$ \ pi $的确定性采样框架。虽然扩散生成建模依赖于预训练得分函数$ \ nabla \ log f_t $使用$ \ pi $的样本，但SBTM地址是仅$ \ nabla \ log \ pi $的更通用和具有挑战性的设置。 SBTM通过学习使用得分匹配的时间变化的分数$ \ nabla \ log f_t $来近似Kl $（f_t \ | \ pi）$上的Wasserstein梯度流。学到的分数可立即访问相对的Fisher信息，从而提供内置的收敛诊断。确定性的轨迹平稳，可解释且没有布朗运动噪声，同时具有与ULA相同的分布。我们证明，SBTM以与确切的梯度流相同的速率消散相对熵，提供了足够的训练。我们进一步将框架扩展到退火动力学，以处理非对数凸目标的目标。数值实验验证了我们的理论发现：SBTM以最佳速率收敛，具有平滑的轨迹，并且很容易与退火动力学集成。我们与Ula和退火Ula的基线进行比较。

Title: A Generative Graph Contrastive Learning Model with Global Signal

• Authors: Xiaofan Wei, Binyan Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.18148
• Pdf URL: https://arxiv.org/pdf/2504.18148
• Copy Paste: [[2504.18148]] A Generative Graph Contrastive Learning Model with Global Signal(https://arxiv.org/abs/2504.18148)
• Keywords: generative
• Abstract: Graph contrastive learning (GCL) has garnered significant attention recently since it learns complex structural information from graphs through self-supervised learning manner. However, prevalent GCL models may suffer from performance degradation due to inappropriate contrastive signals. Concretely, they commonly generate augmented views based on random perturbation, which leads to biased essential structures due to the introduction of noise. In addition, they assign equal weight to both hard and easy sample pairs, thereby ignoring the difference in importance of the sample pairs. To address these issues, this study proposes a novel Contrastive Signal Generative Framework for Accurate Graph Learning (CSG2L) with the following two-fold ideas: a) building a singular value decomposition (SVD)-directed augmented module (SVD-aug) to obtain the global interactions as well as avoiding the random noise perturbation; b) designing a local-global dependency learning module (LGDL) with an adaptive reweighting strategy which can differentiate the effects of hard and easy sample pairs. Extensive experiments on benchmark datasets demonstrate that the proposed CSG2L outperforms the state-of-art baselines. Moreover, CSG2L is compatible with a variety of GNNs.
• 摘要：图形对比学习（GCL）最近引起了人们的重大关注，因为它通过自我监督的学习方式从图形中学习了复杂的结构信息。但是，由于不适当的对比信号，普遍的GCL模型可能会遭受性能降解。具体而言，它们通常会基于随机扰动产生增强视图，这导致由于引入噪声而导致基本结构有偏见。此外，它们为硬和简单的样品对分配了相同的重量，从而忽略了样本对的重要性差异。为了解决这些问题，本研究提出了一个新型的对比度信号生成框架，以进行准确的图形学习（CSG2L），其两个构想：a）构建一个奇异的值分解（SVD）指导的增强模块（SVD-aug），以获得全球相互作用，并避免随机噪声扰动； b）设计具有自适应重新加权策略的局部全球依赖性学习模块（LGDL），该模块可以区分硬和简单的样品对的影响。基准数据集的广泛实验表明，所提出的CSG2L的表现优于最先进的基准。此外，CSG2L与各种GNN兼容。

Title: Offline Learning of Controllable Diverse Behaviors

• Authors: Mathieu Petitbois, Rémy Portelas, Sylvain Lamprier, Ludovic Denoyer
• Subjects: cs.LG, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2504.18160
• Pdf URL: https://arxiv.org/pdf/2504.18160
• Copy Paste: [[2504.18160]] Offline Learning of Controllable Diverse Behaviors(https://arxiv.org/abs/2504.18160)
• Keywords: generation
• Abstract: Imitation Learning (IL) techniques aim to replicate human behaviors in specific tasks. While IL has gained prominence due to its effectiveness and efficiency, traditional methods often focus on datasets collected from experts to produce a single efficient policy. Recently, extensions have been proposed to handle datasets of diverse behaviors by mainly focusing on learning transition-level diverse policies or on performing entropy maximization at the trajectory level. While these methods may lead to diverse behaviors, they may not be sufficient to reproduce the actual diversity of demonstrations or to allow controlled trajectory generation. To overcome these drawbacks, we propose a different method based on two key features: a) Temporal Consistency that ensures consistent behaviors across entire episodes and not just at the transition level as well as b) Controllability obtained by constructing a latent space of behaviors that allows users to selectively activate specific behaviors based on their requirements. We compare our approach to state-of-the-art methods over a diverse set of tasks and environments. Project page: this https URL
• 摘要：模仿学习（IL）技术旨在复制特定任务中的人类行为。尽管IL由于其有效性和效率而获得了突出性，但传统方法通常集中在专家收集的数据集上，以制定一种有效的政策。最近，已经提出了扩展来处理各种行为数据集，主要是专注于学习过渡级别的不同策略或在轨迹级别上执行熵最大化。尽管这些方法可能导致各种行为，但它们可能不足以再现示范的实际多样性或允许受控的轨迹产生。为了克服这些缺点，我们基于两个关键特征提出了一种不同的方法：a）时间一致性，该方法确保整个事件的一致行为，而不仅仅是在过渡水平上以及b）b）通过构建潜在行为空间来获得可控性，从而使用户可以根据要求选择特定的特定行为。我们比较了各种任务和环境集的最先进方法的方法。项目页面：此HTTPS URL

Title: Optimizing Multi-Round Enhanced Training in Diffusion Models for Improved Preference Understanding

• Authors: Kun Li, Jianhui Wang, Yangfan He, Xinyuan Song, Ruoyu Wang, Hongyang He, Wenxin Zhang, Jiaqi Chen, Keqin Li, Sida Li, Miao Zhang, Tianyu Shi, Xueqian Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18204
• Pdf URL: https://arxiv.org/pdf/2504.18204
• Copy Paste: [[2504.18204]] Optimizing Multi-Round Enhanced Training in Diffusion Models for Improved Preference Understanding(https://arxiv.org/abs/2504.18204)
• Keywords: generation, generative
• Abstract: Generative AI has significantly changed industries by enabling text-driven image generation, yet challenges remain in achieving high-resolution outputs that align with fine-grained user preferences. Consequently, multi-round interactions are necessary to ensure the generated images meet expectations. Previous methods enhanced prompts via reward feedback but did not optimize over a multi-round dialogue dataset. In this work, we present a Visual Co-Adaptation (VCA) framework incorporating human-in-the-loop feedback, leveraging a well-trained reward model aligned with human preferences. Using a diverse multi-turn dialogue dataset, our framework applies multiple reward functions, such as diversity, consistency, and preference feedback, while fine-tuning the diffusion model through LoRA, thus optimizing image generation based on user input. We also construct multi-round dialogue datasets of prompts and image pairs aligned with user intent. Experiments demonstrate that our method outperforms state-of-the-art baselines, significantly improving image consistency and alignment with user intent. Our approach consistently surpasses competing models in user satisfaction, especially in multi-turn dialogue scenarios.
• 摘要：生成的AI通过实现文本驱动的图像生成而显着改变了行业，但是在实现与细粒用户偏好相符的高分辨率输出方面仍然存在挑战。因此，必须进行多轮交互，以确保生成的图像满足期望。以前的方法通过奖励反馈增强了提示，但没有在多轮对话数据集上进行优化。在这项工作中，我们提出了一个视觉共同适应（VCA）框架，该框架结合了人类的反馈，利用了训练有素的奖励模型，该模型与人类的偏好相符。我们使用多样化的多转化对话数据集，我们的框架应用了多种奖励功能，例如多样性，一致性和偏好反馈，同时通过LORA微调扩散模型，从而根据用户输入来优化图像生成。我们还构建了与用户意图对齐的提示和图像对的多轮对话数据集。实验表明，我们的方法的表现优于最先进的基线，从而显着提高了图像一致性和与用户意图的一致性。我们的方法一致地超过了用户满意度的竞争模型，尤其是在多转向对话方案中。

Title: Seeing Soundscapes: Audio-Visual Generation and Separation from Soundscapes Using Audio-Visual Separator

• Authors: Minjae Kang, Martim Brandão
• Subjects: cs.CV, cs.AI, cs.MM, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2504.18283
• Pdf URL: https://arxiv.org/pdf/2504.18283
• Copy Paste: [[2504.18283]] Seeing Soundscapes: Audio-Visual Generation and Separation from Soundscapes Using Audio-Visual Separator(https://arxiv.org/abs/2504.18283)
• Keywords: generation, generative
• Abstract: Recent audio-visual generative models have made substantial progress in generating images from audio. However, existing approaches focus on generating images from single-class audio and fail to generate images from mixed audio. To address this, we propose an Audio-Visual Generation and Separation model (AV-GAS) for generating images from soundscapes (mixed audio containing multiple classes). Our contribution is threefold: First, we propose a new challenge in the audio-visual generation task, which is to generate an image given a multi-class audio input, and we propose a method that solves this task using an audio-visual separator. Second, we introduce a new audio-visual separation task, which involves generating separate images for each class present in a mixed audio input. Lastly, we propose new evaluation metrics for the audio-visual generation task: Class Representation Score (CRS) and a modified R@K. Our model is trained and evaluated on the VGGSound dataset. We show that our method outperforms the state-of-the-art, achieving 7% higher CRS and 4% higher R@2* in generating plausible images with mixed audio.
• 摘要：最近的视听生成模型在产生音频的图像方面取得了重大进展。但是，现有方法专注于从单级音频生成图像，而无法从混合音频中生成图像。为了解决这个问题，我们提出了一个视听生成和分离模型（AV-GAS），用于生成来自Soundscapes的图像（包含多个类的混合音频）。我们的贡献是三倍：首先，我们在视听生成任务中提出了一个新的挑战，即给定给定多级音频输入的图像生成图像，我们提出了一种使用音频 - 视觉分离器来解决此任务的方法。其次，我们引入了一个新的视听分离任务，该任务涉及为混合音频输入中的每个类生成单独的图像。最后，我们为视听生成任务提出了新的评估指标：类表示分数（CRS）和修改后的R@K。我们的模型在VGGSOUND数据集上进行了训练和评估。我们表明，我们的方法的表现优于最先进的方法，在用混合音频产生合理的图像时，CRS高7％，R@2*提高了4％。

Title: STP4D: Spatio-Temporal-Prompt Consistent Modeling for Text-to-4D Gaussian Splatting

• Authors: Yunze Deng, Haijun Xiong, Bin Feng, Xinggang Wang, Wenyu Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18318
• Pdf URL: https://arxiv.org/pdf/2504.18318
• Copy Paste: [[2504.18318]] STP4D: Spatio-Temporal-Prompt Consistent Modeling for Text-to-4D Gaussian Splatting(https://arxiv.org/abs/2504.18318)
• Keywords: generation
• Abstract: Text-to-4D generation is rapidly developing and widely applied in various scenarios. However, existing methods often fail to incorporate adequate spatio-temporal modeling and prompt alignment within a unified framework, resulting in temporal inconsistencies, geometric distortions, or low-quality 4D content that deviates from the provided texts. Therefore, we propose STP4D, a novel approach that aims to integrate comprehensive spatio-temporal-prompt consistency modeling for high-quality text-to-4D generation. Specifically, STP4D employs three carefully designed modules: Time-varying Prompt Embedding, Geometric Information Enhancement, and Temporal Extension Deformation, which collaborate to accomplish this goal. Furthermore, STP4D is among the first methods to exploit the Diffusion model to generate 4D Gaussians, combining the fine-grained modeling capabilities and the real-time rendering process of 4DGS with the rapid inference speed of the Diffusion model. Extensive experiments demonstrate that STP4D excels in generating high-fidelity 4D content with exceptional efficiency (approximately 4.6s per asset), surpassing existing methods in both quality and speed.
• 摘要：文本到4D的生成正在迅速发展，并在各种情况下广泛应用。但是，现有方法通常无法在统一的框架中纳入足够的时空建模和迅速对齐，从而导致时间不一致，几何扭曲或低质量的4D内容，从而偏离了所提供的文本。因此，我们提出了STP4D，这是一种新型方法，旨在为高质量的文本到4D生成整合全面的时空推出一致性建模。具体而言，STP4D采用了三个精心设计的模块：随时间变化的及时嵌入，几何信息增强和时间扩展变形，可以协作以实现此目标。此外，STP4D是利用扩散模型生成4D高斯，将4DGS的实时渲染过程与扩散模型的快速推断速度相结合的第一个方法之一。广泛的实验表明，STP4D在产生具有出色效率的高保真4D内容方面表现出色（每资产约4.6秒），超过了质量和速度的现有方法。

Title: SSD-Poser: Avatar Pose Estimation with State Space Duality from Sparse Observations

• Authors: Shuting Zhao, Linxin Bai, Liangjing Shao, Ye Zhang, Xinrong Chen
• Subjects: cs.CV, cs.HC
• Abstract URL: https://arxiv.org/abs/2504.18332
• Pdf URL: https://arxiv.org/pdf/2504.18332
• Copy Paste: [[2504.18332]] SSD-Poser: Avatar Pose Estimation with State Space Duality from Sparse Observations(https://arxiv.org/abs/2504.18332)
• Keywords: generative
• Abstract: The growing applications of AR/VR increase the demand for real-time full-body pose estimation from Head-Mounted Displays (HMDs). Although HMDs provide joint signals from the head and hands, reconstructing a full-body pose remains challenging due to the unconstrained lower body. Recent advancements often rely on conventional neural networks and generative models to improve performance in this task, such as Transformers and diffusion models. However, these approaches struggle to strike a balance between achieving precise pose reconstruction and maintaining fast inference speed. To overcome these challenges, a lightweight and efficient model, SSD-Poser, is designed for robust full-body motion estimation from sparse observations. SSD-Poser incorporates a well-designed hybrid encoder, State Space Attention Encoders, to adapt the state space duality to complex motion poses and enable real-time realistic pose reconstruction. Moreover, a Frequency-Aware Decoder is introduced to mitigate jitter caused by variable-frequency motion signals, remarkably enhancing the motion smoothness. Comprehensive experiments on the AMASS dataset demonstrate that SSD-Poser achieves exceptional accuracy and computational efficiency, showing outstanding inference efficiency compared to state-of-the-art methods.
• 摘要：AR/VR的应用不断增长，增加了对头部安装显示（HMD）实时全身姿势估算的需求。尽管HMD从头部和手提供联合信号，但由于不受限制的下半身，重建全身姿势仍然具有挑战性。最近的进步通常依赖于常规的神经网络和生成模型来改善此任务中的性能，例如变形金刚和扩散模型。但是，这些方法难以在实现精确的姿势重建和保持快速推理速度之间取得平衡。为了克服这些挑战，一种轻巧有效的模型SSD-Poser设计用于从稀疏观测值中进行强大的全身运动估计。 SSD-Poser结合了精心设计的混合编码器，即状态空间注意编码器，以使状态空间二元性适应复杂运动姿势并实现实时逼真的姿势重建。此外，引入了频感知解码器，以减轻由可变频率运动信号引起的抖动，从而显着增强了运动平滑度。在Amass数据集上进行的全面实验表明，SSD Poser具有出色的准确性和计算效率，与最先进的方法相比显示出出色的推理效率。

Title: COCO-Inpaint: A Benchmark for Image Inpainting Detection and Manipulation Localization

• Authors: Haozhen Yan, Yan Hong, Jiahui Zhan, Yikun Ji, Jun Lan, Huijia Zhu, Weiqiang Wang, Jianfu Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.18361
• Pdf URL: https://arxiv.org/pdf/2504.18361
• Copy Paste: [[2504.18361]] COCO-Inpaint: A Benchmark for Image Inpainting Detection and Manipulation Localization(https://arxiv.org/abs/2504.18361)
• Keywords: generation
• Abstract: Recent advancements in image manipulation have achieved unprecedented progress in generating photorealistic content, but also simultaneously eliminating barriers to arbitrary manipulation and editing, raising concerns about multimedia authenticity and cybersecurity. However, existing Image Manipulation Detection and Localization (IMDL) methodologies predominantly focus on splicing or copy-move forgeries, lacking dedicated benchmarks for inpainting-based manipulations. To bridge this gap, we present COCOInpaint, a comprehensive benchmark specifically designed for inpainting detection, with three key contributions: 1) High-quality inpainting samples generated by six state-of-the-art inpainting models, 2) Diverse generation scenarios enabled by four mask generation strategies with optional text guidance, and 3) Large-scale coverage with 258,266 inpainted images with rich semantic diversity. Our benchmark is constructed to emphasize intrinsic inconsistencies between inpainted and authentic regions, rather than superficial semantic artifacts such as object shapes. We establish a rigorous evaluation protocol using three standard metrics to assess existing IMDL approaches. The dataset will be made publicly available to facilitate future research in this area.
• 摘要：图像操纵方面的最新进展在产生影像含量方面取得了前所未有的进展，同时消除了任意操纵和编辑的障碍，从而引起了对多媒体真实性和网络安全的关注。但是，现有的图像操纵检测和本地化（IMDL）方法论主要集中在剪接或复制移动伪造上，缺乏用于基于介入的操作的专用基准。 To bridge this gap, we present COCOInpaint, a comprehensive benchmark specifically designed for inpainting detection, with three key contributions: 1) High-quality inpainting samples generated by six state-of-the-art inpainting models, 2) Diverse generation scenarios enabled by four mask generation strategies with optional text guidance, and 3) Large-scale coverage with 258,266 inpainted images with rich semantic diversity.我们的基准是为了强调覆盖区域和真实区域之间的内在矛盾，而不是浅表语义伪影，例如对象形状。我们使用三个标准指标建立了严格的评估协议来评估现有的IMDL方法。该数据集将公开使用，以促进该领域的未来研究。

Title: Fast Autoregressive Models for Continuous Latent Generation

• Authors: Tiankai Hang, Jianmin Bao, Fangyun Wei, Dong Chen
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.18391
• Pdf URL: https://arxiv.org/pdf/2504.18391
• Copy Paste: [[2504.18391]] Fast Autoregressive Models for Continuous Latent Generation(https://arxiv.org/abs/2504.18391)
• Keywords: generation
• Abstract: Autoregressive models have demonstrated remarkable success in sequential data generation, particularly in NLP, but their extension to continuous-domain image generation presents significant challenges. Recent work, the masked autoregressive model (MAR), bypasses quantization by modeling per-token distributions in continuous spaces using a diffusion head but suffers from slow inference due to the high computational cost of the iterative denoising process. To address this, we propose the Fast AutoRegressive model (FAR), a novel framework that replaces MAR's diffusion head with a lightweight shortcut head, enabling efficient few-step sampling while preserving autoregressive principles. Additionally, FAR seamlessly integrates with causal Transformers, extending them from discrete to continuous token generation without requiring architectural modifications. Experiments demonstrate that FAR achieves $2.3\times$ faster inference than MAR while maintaining competitive FID and IS scores. This work establishes the first efficient autoregressive paradigm for high-fidelity continuous-space image generation, bridging the critical gap between quality and scalability in visual autoregressive modeling.
• 摘要：自回归模型在连续数据生成中表现出了很大的成功，尤其是在NLP中，但它们扩展到连续域图像生成带来了重大挑战。最近的工作，即掩盖的自回旋模型（MAR），通过使用扩散头对连续空间中的双向分布进行建模来绕过量化，但由于迭代denoisis过程的高计算成本而导致推理缓慢。为了解决这个问题，我们提出了快速自回旋模型（FAR），这是一个新颖的框架，用轻质的快捷方式取代了MAR的扩散头，从而实现了有效的少量步骤采样，同时保留了自动回归原理。此外，与因果变压器无缝集成在一起，将它们从离散到连续的代币生成而扩展，而无需进行体系结构修改。实验表明，在保持有竞争力的FID和分数的同时，远比MAR的推断范围比MAR快$ 2.3 \ times $ $。这项工作建立了第一个有效的自回归范式，用于高保真连续空间图像产生，从而弥合了视觉自回旋建模中质量和可扩展性之间的临界差距。

Title: Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization

• Authors: Kesen Zhao, Beier Zhu, Qianru Sun, Hanwang Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18397
• Pdf URL: https://arxiv.org/pdf/2504.18397
• Copy Paste: [[2504.18397]] Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization(https://arxiv.org/abs/2504.18397)
• Keywords: generation
• Abstract: Chain-of-thought (CoT) reasoning greatly improves the interpretability and problem-solving abilities of multimodal large language models (MLLMs). However, existing approaches are focused on text CoT, limiting their ability to leverage visual cues. Visual CoT remains underexplored, and the only work is based on supervised fine-tuning (SFT) that relies on extensive labeled bounding-box data and is hard to generalize to unseen cases. In this paper, we introduce Unsupervised Visual CoT (UV-CoT), a novel framework for image-level CoT reasoning via preference optimization. UV-CoT performs preference comparisons between model-generated bounding boxes (one is preferred and the other is dis-preferred), eliminating the need for bounding-box annotations. We get such preference data by introducing an automatic data generation pipeline. Given an image, our target MLLM (e.g., LLaVA-1.5-7B) generates seed bounding boxes using a template prompt and then answers the question using each bounded region as input. An evaluator MLLM (e.g., OmniLLM-12B) ranks the responses, and these rankings serve as supervision to train the target MLLM with UV-CoT by minimizing negative log-likelihood losses. By emulating human perception--identifying key regions and reasoning based on them--UV-CoT can improve visual comprehension, particularly in spatial reasoning tasks where textual descriptions alone fall short. Our experiments on six datasets demonstrate the superiority of UV-CoT, compared to the state-of-the-art textual and visual CoT methods. Our zero-shot testing on four unseen datasets shows the strong generalization of UV-CoT. The code is available in this https URL.
• 摘要：经过思考链（COT）的推理大大提高了多模式大语模型（MLLM）的解释性和解决问题的能力。但是，现有的方法集中在文本COT上，限制了它们利用视觉提示的能力。 Visual Cot仍然没有被忽视，唯一的工作是基于依赖广泛标记的边界盒数据的监督微调（SFT），并且很难推广到看不见的情况。在本文中，我们介绍了无监督的视觉婴儿床（UV-COT），这是通过偏好优化的图像级婴儿级推理的新型框架。 UV-COT执行模型生成的边界框之间的偏好比较（一个是首选，另一个是偏爱的），从而消除了对边界框注释的需求。我们通过引入自动数据生成管道来获得此类偏好数据。给定图像，我们的目标MLLM（例如LLAVA-1.5-7B）使用模板提示生成种子边界框，然后使用每个有界区域作为输入来回答问题。评估符MLLM（例如Omnillm-12b）对响应进行排名，这些排名是通过最大程度地减少负模具可能的损失来训练目标MLLM的监督。通过模拟人类的看法（识别基于它们的关键区域及其推理）可以改善视觉理解，尤其是在单独文本描述短缺的空间推理任务中。与最先进的文本和视觉COT方法相比，我们在六个数据集上进行的实验证明了紫外线-COT的优势。我们在四个看不见的数据集上进行的零拍测试显示了UV-COT的强烈概括。该代码可在此HTTPS URL中使用。

Title: LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoning

• Authors: Rui Li, Biao Zhang, Zhenyu Li, Federico Tombari, Peter Wonka
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18424
• Pdf URL: https://arxiv.org/pdf/2504.18424
• Copy Paste: [[2504.18424]] LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoning(https://arxiv.org/abs/2504.18424)
• Keywords: generation, generative
• Abstract: We present layered ray intersections (LaRI), a new method for unseen geometry reasoning from a single image. Unlike conventional depth estimation that is limited to the visible surface, LaRI models multiple surfaces intersected by the camera rays using layered point maps. Benefiting from the compact and layered representation, LaRI enables complete, efficient, and view-aligned geometric reasoning to unify object- and scene-level tasks. We further propose to predict the ray stopping index, which identifies valid intersecting pixels and layers from LaRI's output. We build a complete training data generation pipeline for synthetic and real-world data, including 3D objects and scenes, with necessary data cleaning steps and coordination between rendering engines. As a generic method, LaRI's performance is validated in two scenarios: It yields comparable object-level results to the recent large generative model using 4% of its training data and 17% of its parameters. Meanwhile, it achieves scene-level occluded geometry reasoning in only one feed-forward.
• 摘要：我们提出了分层的射线相交（LARI），这是一种从单个图像中未见几何推理的新方法。与传统的深度估计仅限于可见的表面不同，Lari使用分层映射与相机射线相交的多个表面模型。 Lari受益于紧凑和分层的表示，可以使完整，高效且视野一致的几何推理能够统一对象和场景级任务。我们进一步建议预测射线停止索引，该索引可以从Lari的输出中标识有效的相交像素和层。我们为合成和现实世界数据（包括3D对象和场景）建立了完整的培训数据生成管道，并通过必要的数据清洁步骤和渲染引擎之间的协调。作为一种通用方法，在两种情况下，Lari的性能得到了验证：它使用其4％的训练数据和17％的参数将可比的对象级结果与最近的大生成模型产生可比的结果。同时，它仅在一个馈送方面就可以实现场景级的遮挡几何形状推理。

Title: NoiseController: Towards Consistent Multi-view Video Generation via Noise Decomposition and Collaboration

• Authors: Haotian Dong, Xin Wang, Di Lin, Yipeng Wu, Qin Chen, Ruonan Liu, Kairui Yang, Ping Li, Qing Guo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18448
• Pdf URL: https://arxiv.org/pdf/2504.18448
• Copy Paste: [[2504.18448]] NoiseController: Towards Consistent Multi-view Video Generation via Noise Decomposition and Collaboration(https://arxiv.org/abs/2504.18448)
• Keywords: generation
• Abstract: High-quality video generation is crucial for many fields, including the film industry and autonomous driving. However, generating videos with spatiotemporal consistencies remains challenging. Current methods typically utilize attention mechanisms or modify noise to achieve consistent videos, neglecting global spatiotemporal information that could help ensure spatial and temporal consistency during video generation. In this paper, we propose the NoiseController, consisting of Multi-Level Noise Decomposition, Multi-Frame Noise Collaboration, and Joint Denoising, to enhance spatiotemporal consistencies in video generation. In multi-level noise decomposition, we first decompose initial noises into scene-level foreground/background noises, capturing distinct motion properties to model multi-view foreground/background variations. Furthermore, each scene-level noise is further decomposed into individual-level shared and residual components. The shared noise preserves consistency, while the residual component maintains diversity. In multi-frame noise collaboration, we introduce an inter-view spatiotemporal collaboration matrix and an intra-view impact collaboration matrix , which captures mutual cross-view effects and historical cross-frame impacts to enhance video quality. The joint denoising contains two parallel denoising U-Nets to remove each scene-level noise, mutually enhancing video generation. We evaluate our NoiseController on public datasets focusing on video generation and downstream tasks, demonstrating its state-of-the-art performance.
• 摘要：高质量的视频生成对于包括电影业和自动驾驶在内的许多领域至关重要。但是，生成具有时空一致性的视频仍然具有挑战性。当前的方法通常会使用注意机制或修改噪声来获得一致的视频，从而忽略了全球时空信息，从而有助于确保视频生成过程中的空间和时间一致性。在本文中，我们提出了由多级噪声分解，多帧噪声协作和联合DeNoing的噪声控制器，以增强视频生成中的时空一致性。在多级噪声分解中，我们首先将初始噪声分解为场景级别的前景/背景噪声，从而捕获不同的运动属性，以模拟多视图前景/背景变化。此外，每个场景级别的噪声进一步分解为个人级别的共享和残留组件。共享的噪声可以保留一致性，而残差组件保持多样性。在多帧噪声协作中，我们引入了视图时空的合作矩阵和视图Inpect Consergoration Matrix，该矩阵捕获了相互的跨视图效果和历史跨框架的影响，以提高视频质量。联合DeNoising包含两个平行的Denoing U-NET，以消除每个场景级别的噪声，并相互增强视频的生成。我们在公共数据集上评估了我们的NoiseController，重点是视频生成和下游任务，以证明其最先进的性能。

Title: Action-Minimization Meets Generative Modeling: Efficient Transition Path Sampling with the Onsager-Machlup Functional

• Authors: Sanjeev Raja, Martin Šípka, Michael Psenka, Tobias Kreiman, Michal Pavelka, Aditi S. Krishnapriyan
• Subjects: cs.LG, cond-mat.mtrl-sci, physics.chem-ph, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2504.18506
• Pdf URL: https://arxiv.org/pdf/2504.18506
• Copy Paste: [[2504.18506]] Action-Minimization Meets Generative Modeling: Efficient Transition Path Sampling with the Onsager-Machlup Functional(https://arxiv.org/abs/2504.18506)
• Keywords: generative
• Abstract: Transition path sampling (TPS), which involves finding probable paths connecting two points on an energy landscape, remains a challenge due to the complexity of real-world atomistic systems. Current machine learning approaches use expensive, task-specific, and data-free training procedures, limiting their ability to benefit from recent advances in atomistic machine learning, such as high-quality datasets and large-scale pre-trained models. In this work, we address TPS by interpreting candidate paths as trajectories sampled from stochastic dynamics induced by the learned score function of pre-trained generative models, specifically denoising diffusion and flow matching. Under these dynamics, finding high-likelihood transition paths becomes equivalent to minimizing the Onsager-Machlup (OM) action functional. This enables us to repurpose pre-trained generative models for TPS in a zero-shot manner, in contrast with bespoke, task-specific TPS models trained in previous work. We demonstrate our approach on varied molecular systems, obtaining diverse, physically realistic transition pathways and generalizing beyond the pre-trained model's original training dataset. Our method can be easily incorporated into new generative models, making it practically relevant as models continue to scale and improve with increased data availability.
• 摘要：过渡路径采样（TPS）涉及找到可能在能量景观上连接两个点的可能路径，由于现实世界原子系统的复杂性，这仍然是一个挑战。当前的机器学习方法采用昂贵的，特定于任务的和无数据的培训程序，限制了它们从原子机器学习最新进展中受益的能力，例如高质量的数据集和大规模的预训练模型。在这项工作中，我们通过将候选路径解释为从预先训练的生成模型的学习分数函数引起的随机动力学中采样的轨迹来解决TP，特别是降级扩散和流动匹配。在这些动力学下，找到高样的过渡路径变得等同于最小化onsager-machlup（OM）动作功能。与定制的，特定于任务的TPS模型相比，我们能够以零拍的方式重新利用TPS的预训练的生成模型。我们展示了我们在各种分子系统上的方法，获得了多样化的，物理现实的过渡途径，并推广了预先训练的模型的原始训练数据集。我们的方法可以很容易地纳入新的生成模型中，因此随着模型继续扩展和改进的数据可用性，它实际上具有相关性。

Title: Eval3D: Interpretable and Fine-grained Evaluation for 3D Generation

• Authors: Shivam Duggal, Yushi Hu, Oscar Michel, Aniruddha Kembhavi, William T. Freeman, Noah A. Smith, Ranjay Krishna, Antonio Torralba, Ali Farhadi, Wei-Chiu Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18509
• Pdf URL: https://arxiv.org/pdf/2504.18509
• Copy Paste: [[2504.18509]] Eval3D: Interpretable and Fine-grained Evaluation for 3D Generation(https://arxiv.org/abs/2504.18509)
• Keywords: generation
• Abstract: Despite the unprecedented progress in the field of 3D generation, current systems still often fail to produce high-quality 3D assets that are visually appealing and geometrically and semantically consistent across multiple viewpoints. To effectively assess the quality of the generated 3D data, there is a need for a reliable 3D evaluation tool. Unfortunately, existing 3D evaluation metrics often overlook the geometric quality of generated assets or merely rely on black-box multimodal large language models for coarse assessment. In this paper, we introduce Eval3D, a fine-grained, interpretable evaluation tool that can faithfully evaluate the quality of generated 3D assets based on various distinct yet complementary criteria. Our key observation is that many desired properties of 3D generation, such as semantic and geometric consistency, can be effectively captured by measuring the consistency among various foundation models and tools. We thus leverage a diverse set of models and tools as probes to evaluate the inconsistency of generated 3D assets across different aspects. Compared to prior work, Eval3D provides pixel-wise measurement, enables accurate 3D spatial feedback, and aligns more closely with human judgments. We comprehensively evaluate existing 3D generation models using Eval3D and highlight the limitations and challenges of current models.
• 摘要：尽管在3D一代领域取得了前所未有的进展，但当前系统通常仍未产生高质量的3D资产，这些资产在视觉上具有吸引力，几何和语义在多种观点上具有一致性。为了有效评估生成的3D数据的质量，需要使用可靠的3D评估工具。不幸的是，现有的3D评估指标通常会忽略生成的资产的几何质量，或仅依靠黑盒多模式大语言模型进行粗略评估。在本文中，我们介绍了一种精细的，可解释的评估工具Read3D，可以根据各种不同但互补的标准忠实地评估生成的3D资产的质量。我们的主要观察结果是，可以通过测量各种基础模型和工具之间的一致性来有效地捕获3D代的许多期望特性，例如语义和几何一致性。因此，我们利用各种模型和工具作为探针来评估各个方面生成的3D资产的不一致。与先前的工作相比，Eval3D提供了按像素的测量值，可以实现准确的3D空间反馈，并与人类判断更紧密地保持一致。我们使用este3D全面评估了现有的3D生成模型，并突出了当前模型的局限性和挑战。

Title: Intelligent Attacks and Defense Methods in Federated Learning-enabled Energy-Efficient Wireless Networks

• Authors: Han Zhang, Hao Zhou, Medhat Elsayed, Majid Bavand, Raimundas Gaigalas, Yigit Ozcan, Melike Erol-Kantarci
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.18519
• Pdf URL: https://arxiv.org/pdf/2504.18519
• Copy Paste: [[2504.18519]] Intelligent Attacks and Defense Methods in Federated Learning-enabled Energy-Efficient Wireless Networks(https://arxiv.org/abs/2504.18519)
• Keywords: generative
• Abstract: Federated learning (FL) is a promising technique for learning-based functions in wireless networks, thanks to its distributed implementation capability. On the other hand, distributed learning may increase the risk of exposure to malicious attacks where attacks on a local model may spread to other models by parameter exchange. Meanwhile, such attacks can be hard to detect due to the dynamic wireless environment, especially considering local models can be heterogeneous with non-independent and identically distributed (non-IID) data. Therefore, it is critical to evaluate the effect of malicious attacks and develop advanced defense techniques for FL-enabled wireless networks. In this work, we introduce a federated deep reinforcement learning-based cell sleep control scenario that enhances the energy efficiency of the network. We propose multiple intelligent attacks targeting the learning-based approach and we propose defense methods to mitigate such attacks. In particular, we have designed two attack models, generative adversarial network (GAN)-enhanced model poisoning attack and regularization-based model poisoning attack. As a counteraction, we have proposed two defense schemes, autoencoder-based defense, and knowledge distillation (KD)-enabled defense. The autoencoder-based defense method leverages an autoencoder to identify the malicious participants and only aggregate the parameters of benign local models during the global aggregation, while KD-based defense protects the model from attacks by controlling the knowledge transferred between the global model and local models.
• 摘要：Federated学习（FL）是一项有前途的技术，用于无线网络中基于学习的功能，这要归功于其分布式实现能力。另一方面，分布式学习可能会增加暴露于恶意攻击的风险，因为对本地模型的攻击可能通过参数交换扩散到其他模型。同时，由于动态无线环境，可能很难检测到此类攻击，尤其是考虑到本地模型可能与非独立且分布相同的（非IID）数据是异质的。因此，评估恶意攻击的影响并为实现FL的无线网络开发高级防御技术至关重要。在这项工作中，我们介绍了一个联合的基于深层学习的基于学习的细胞睡眠控制方案，从而提高了网络的能源效率。我们提出了针对基于学习的方法的多次智能攻击，并提出了防御方法来减轻此类攻击。特别是，我们设计了两种攻击模型，即生成对抗网络（GAN）增强模型中毒攻击和基于正则化的模型中毒攻击。作为反击，我们提出了两种防御计划，基于自动编码器的防御和知识蒸馏（KD）支持的防御。基于自动编码器的防御方法利用自动编码器来识别恶意参与者，并且仅在全球聚合过程中汇总了良性本地模型的参数，而基于KD的防御方法通过控制全球模型和本地模型之间传递的知识来保护模型免受攻击。

Title: Augmenting Perceptual Super-Resolution via Image Quality Predictors

• Authors: Fengjia Zhang, Samrudhdhi B. Rangrej, Tristan Aumentado-Armstrong, Afsaneh Fazly, Alex Levinshtein
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.18524
• Pdf URL: https://arxiv.org/pdf/2504.18524
• Copy Paste: [[2504.18524]] Augmenting Perceptual Super-Resolution via Image Quality Predictors(https://arxiv.org/abs/2504.18524)
• Keywords: super-resolution, quality assessment
• Abstract: Super-resolution (SR), a classical inverse problem in computer vision, is inherently ill-posed, inducing a distribution of plausible solutions for every input. However, the desired result is not simply the expectation of this distribution, which is the blurry image obtained by minimizing pixelwise error, but rather the sample with the highest image quality. A variety of techniques, from perceptual metrics to adversarial losses, are employed to this end. In this work, we explore an alternative: utilizing powerful non-reference image quality assessment (NR-IQA) models in the SR context. We begin with a comprehensive analysis of NR-IQA metrics on human-derived SR data, identifying both the accuracy (human alignment) and complementarity of different metrics. Then, we explore two methods of applying NR-IQA models to SR learning: (i) altering data sampling, by building on an existing multi-ground-truth SR framework, and (ii) directly optimizing a differentiable quality score. Our results demonstrate a more human-centric perception-distortion tradeoff, focusing less on non-perceptual pixel-wise distortion, instead improving the balance between perceptual fidelity and human-tuned NR-IQA measures.
• 摘要：超分辨率（SR）是计算机视觉中的经典逆问题，它本质上是不良的，可以为每个输入引起合理解决方案的分布。但是，所需的结果不仅是对此分布的期望，即通过最小化像素方误差而获得的模糊图像，而是具有最高图像质量的样本。从感知指标到对抗性损失，采用了各种技术。在这项工作中，我们探讨了一种替代方法：在SR上下文中利用强大的非参考图像质量评估（NR-IQA）模型。我们从对人类衍生的SR数据的NR-IQA指标进行全面分析，从而确定了不同指标的准确性（人类对齐）和互补性。然后，我们探讨了将NR-IQA模型应用于SR学习的两种方法：（i）通过在现有的多个地面真实SR框架上构建来更改数据采样，以及（ii）直接优化可区分的质量分数。我们的结果表明，以人为中心的感知分数的权衡表明，较少的重点是非知觉像素的扭曲，而是改善了感知忠诚度和人为调节的NR-IQA测量之间的平衡。