2025-05-19

Title: Aquarius: A Family of Industry-Level Video Generation Models for Marketing Scenarios

• Authors: Huafeng Shi, Jianzhong Liang, Rongchang Xie, Xian Wu, Cheng Chen, Chang Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.10584
• Pdf URL: https://arxiv.org/pdf/2505.10584
• Copy Paste: [[2505.10584]] Aquarius: A Family of Industry-Level Video Generation Models for Marketing Scenarios(https://arxiv.org/abs/2505.10584)
• Keywords: generation, generative
• Abstract: This report introduces Aquarius, a family of industry-level video generation models for marketing scenarios designed for thousands-xPU clusters and models with hundreds of billions of parameters. Leveraging efficient engineering architecture and algorithmic innovation, Aquarius demonstrates exceptional performance in high-fidelity, multi-aspect-ratio, and long-duration video synthesis. By disclosing the framework's design details, we aim to demystify industrial-scale video generation systems and catalyze advancements in the generative video community. The Aquarius framework consists of five components: Distributed Graph and Video Data Processing Pipeline: Manages tens of thousands of CPUs and thousands of xPUs via automated task distribution, enabling efficient video data processing. Additionally, we are about to open-source the entire data processing framework named "Aquarius-Datapipe". Model Architectures for Different Scales: Include a Single-DiT architecture for 2B models and a Multimodal-DiT architecture for 13.4B models, supporting multi-aspect ratios, multi-resolution, and multi-duration video generation. High-Performance infrastructure designed for video generation model training: Incorporating hybrid parallelism and fine-grained memory optimization strategies, this infrastructure achieves 36% MFU at large scale. Multi-xPU Parallel Inference Acceleration: Utilizes diffusion cache and attention optimization to achieve a 2.35x inference speedup. Multiple marketing-scenarios applications: Including image-to-video, text-to-video (avatar), video inpainting and video personalization, among others. More downstream applications and multi-dimensional evaluation metrics will be added in the upcoming version updates.
• 摘要：该报告介绍了Aquarius，Aquarius是一个由行业级别的视频生成模型的家庭，用于营销场景，专为具有数百亿参数的数千个XPU群集和模型设计。在利用高效的工程体系结构和算法创新时，水瓶座表现出高保真性，多光谱比率和长期视频综合方面的出色表现。通过披露框架的设计细节，我们旨在揭开工业规模的视频生成系统的神秘面纱，并在生成视频社区中催化进步。水瓶座框架由五个组件组成：分布式图和视频数据处理管道：通过自动任务分发管理数以万计的CPU和数千个XPU，从而实现有效的视频数据处理。此外，我们即将开放名为“ Aquarius-Datapipe”的整个数据处理框架。不同尺度的模型体系结构：包括用于2B型号的单位体系结构，以及13.4B模型的多模式架构，支持多主比率，多分辨率和多持续视频视频生成。该基础架构的高性能基础设施是为视频生成模型培训而设计的：结合混合并行性存储优化策略，该基础架构在大规模上达到了36％的MFU。多XPU平行推理加速度：利用扩散缓存和注意力优化以实现2.35倍的推理加速。多个营销筛选应用程序：包括图像到视频，文本对视频（Avatar），视频介绍和视频个性化等。即将到来的版本更新将添加更多下游应用程序和多维评估指标。

Title: Efficient Malicious UAV Detection Using Autoencoder-TSMamba Integration

• Authors: Azim Akhtarshenas, Ramin Toosi, David López-Pérez, Tohid Alizadeh, Alireza Hosseini
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2505.10585
• Pdf URL: https://arxiv.org/pdf/2505.10585
• Copy Paste: [[2505.10585]] Efficient Malicious UAV Detection Using Autoencoder-TSMamba Integration(https://arxiv.org/abs/2505.10585)
• Keywords: generation
• Abstract: Malicious Unmanned Aerial Vehicles (UAVs) present a significant threat to next-generation networks (NGNs), posing risks such as unauthorized surveillance, data theft, and the delivery of hazardous materials. This paper proposes an integrated (AE)-classifier system to detect malicious UAVs. The proposed AE, based on a 4-layer Tri-orientated Spatial Mamba (TSMamba) architecture, effectively captures complex spatial relationships crucial for identifying malicious UAV activities. The first phase involves generating residual values through the AE, which are subsequently processed by a ResNet-based classifier. This classifier leverages the residual values to achieve lower complexity and higher accuracy. Our experiments demonstrate significant improvements in both binary and multi-class classification scenarios, achieving up to 99.8 % recall compared to 96.7 % in the benchmark. Additionally, our method reduces computational complexity, making it more suitable for large-scale deployment. These results highlight the robustness and scalability of our approach, offering an effective solution for malicious UAV detection in NGN environments.
• 摘要：恶意无人驾驶汽车（UAVS）对下一代网络（NGN）构成了重大威胁，构成了未经授权的监视，数据盗窃和危险材料提供的风险。本文提出了一个集成（AE） - 分类器系统来检测恶意无人机。提出的AE基于4层三层为导向的空间Mamba（TSMAMBA）架构，有效地捕获了复杂的空间关系对于识别恶意无人机活动至关重要。第一阶段涉及通过AE生成残差值，后来由基于RESNET的分类器处理。该分类器利用残差值来达到较低的复杂性和较高的准确性。我们的实验表明，二进制和多类分类方案的显着改善，达到99.8％的召回率，而基准为96.7％。此外，我们的方法降低了计算复杂性，使其更适合大规模部署。这些结果突出了我们方法的鲁棒性和可扩展性，为NGN环境中的恶意无人机检测提供了有效的解决方案。

Title: Super-Resolution Generative Adversarial Networks based Video Enhancement

• Authors: Kağan ÇETİN
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.10589
• Pdf URL: https://arxiv.org/pdf/2505.10589
• Copy Paste: [[2505.10589]] Super-Resolution Generative Adversarial Networks based Video Enhancement(https://arxiv.org/abs/2505.10589)
• Keywords: restoration, super-resolution, generative
• Abstract: This study introduces an enhanced approach to video super-resolution by extending ordinary Single-Image Super-Resolution (SISR) Super-Resolution Generative Adversarial Network (SRGAN) structure to handle spatio-temporal data. While SRGAN has proven effective for single-image enhancement, its design does not account for the temporal continuity required in video processing. To address this, a modified framework that incorporates 3D Non-Local Blocks is proposed, which is enabling the model to capture relationships across both spatial and temporal dimensions. An experimental training pipeline is developed, based on patch-wise learning and advanced data degradation techniques, to simulate real-world video conditions and learn from both local and global structures and details. This helps the model generalize better and maintain stability across varying video content while maintaining the general structure besides the pixel-wise correctness. Two model variants-one larger and one more lightweight-are presented to explore the trade-offs between performance and efficiency. The results demonstrate improved temporal coherence, sharper textures, and fewer visual artifacts compared to traditional single-image methods. This work contributes to the development of practical, learning-based solutions for video enhancement tasks, with potential applications in streaming, gaming, and digital restoration.
• 摘要：这项研究通过扩展普通的单图像超分辨率（SISR）超分辨率生成对抗网络（SRGAN）结构来介绍一种增强的视频超分辨率方法，以处理时空数据。尽管SRGAN已被证明对单像增强有效，但其设计并不能说明视频处理所需的时间连续性。为了解决这个问题，提出了一个合并3D非本地块的修改框架，这使该模型能够捕获跨空间和时间维度的关系。基于贴剂学习和高级数据退化技术开发了实验培训管道，以模拟现实世界的视频条件，并从本地和全球结构和细节中学习。这有助于模型更好地推广并保持各种视频内容的稳定性，同时除了像素的正确性外保持一般结构。两种模型变体 - 更大，还有一个轻量级 - 呈现，以探索性能和效率之间的权衡。结果表明，与传统的单图像方法相比，时间相干性的提高，更清晰的纹理和更少的视觉伪像。这项工作有助于开发用于视频增强任务的实用，基于学习的解决方案，并在流，游戏和数字恢复中使用潜在的应用。

Title: SRMamba: Mamba for Super-Resolution of LiDAR Point Clouds

• Authors: Chuang Chen, Wenyi Ge
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.10601
• Pdf URL: https://arxiv.org/pdf/2505.10601
• Copy Paste: [[2505.10601]] SRMamba: Mamba for Super-Resolution of LiDAR Point Clouds(https://arxiv.org/abs/2505.10601)
• Keywords: super-resolution
• Abstract: In recent years, range-view-based LiDAR point cloud super-resolution techniques attract significant attention as a low-cost method for generating higher-resolution point cloud data. However, due to the sparsity and irregular structure of LiDAR point clouds, the point cloud super-resolution problem remains a challenging topic, especially for point cloud upsampling under novel views. In this paper, we propose SRMamba, a novel method for super-resolution of LiDAR point clouds in sparse scenes, addressing the key challenge of recovering the 3D spatial structure of point clouds from novel views. Specifically, we implement projection technique based on Hough Voting and Hole Compensation strategy to eliminate horizontally linear holes in range image. To improve the establishment of long-distance dependencies and to focus on potential geometric features in vertical 3D space, we employ Visual State Space model and Multi-Directional Scanning mechanism to mitigate the loss of 3D spatial structural information due to the range image. Additionally, an asymmetric U-Net network adapts to the input characteristics of LiDARs with different beam counts, enabling super-resolution reconstruction for multi-beam point clouds. We conduct a series of experiments on multiple challenging public LiDAR datasets (SemanticKITTI and nuScenes), and SRMamba demonstrates significant superiority over other algorithms in both qualitative and quantitative evaluations.
• 摘要：近年来，基于范围视图的LiDar点云超分辨率技术吸引了引起高成本方法的大幅关注，用于生成更高分辨率的点云数据。但是，由于激光点云的稀疏性和不规则结构，点云超分辨率问题仍然是一个具有挑战性的话题，尤其是在新型观点下的点云上提取采样。在本文中，我们提出了Srmamba，这是一种稀疏场景中激光点云的超级分辨率的新方法，以解决从新视图中恢复点云的3D空间结构的关键挑战。具体而言，我们基于Hough投票和孔补偿策略实施投影技术，以消除范围图像中的水平线性孔。为了改善长距离依赖性的建立并专注于垂直3D空间中的潜在几何特征，我们采用了视觉状态空间模型和多方向扫描机制来减轻由于范围图像而导致的3D空间结构信息的丢失。此外，一个不对称的U-NET网络适应具有不同光束计数的激光雷达的输入特性，从而实现了多光点云的超分辨率重建。我们对多个挑战性的公共激光雷达数据集（Semantickitti和nuscenes）进行了一系列实验，而Srmamba在定性和定量评估中都表现出比其他算法相比的显着优势。

Title: MONAQ: Multi-Objective Neural Architecture Querying for Time-Series Analysis on Resource-Constrained Devices

• Authors: Patara Trirat, Jae-Gil Lee
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.10607
• Pdf URL: https://arxiv.org/pdf/2505.10607
• Copy Paste: [[2505.10607]] MONAQ: Multi-Objective Neural Architecture Querying for Time-Series Analysis on Resource-Constrained Devices(https://arxiv.org/abs/2505.10607)
• Keywords: generation
• Abstract: The growing use of smartphones and IoT devices necessitates efficient time-series analysis on resource-constrained hardware, which is critical for sensing applications such as human activity recognition and air quality prediction. Recent efforts in hardware-aware neural architecture search (NAS) automate architecture discovery for specific platforms; however, none focus on general time-series analysis with edge deployment. Leveraging the problem-solving and reasoning capabilities of large language models (LLM), we propose MONAQ, a novel framework that reformulates NAS into Multi-Objective Neural Architecture Querying tasks. MONAQ is equipped with multimodal query generation for processing multimodal time-series inputs and hardware constraints, alongside an LLM agent-based multi-objective search to achieve deployment-ready models via code generation. By integrating numerical data, time-series images, and textual descriptions, MONAQ improves an LLM's understanding of time-series data. Experiments on fifteen datasets demonstrate that MONAQ-discovered models outperform both handcrafted models and NAS baselines while being more efficient.
• 摘要：智能手机和物联网设备的日益增长的使用需要对资源受限的硬件进行有效的时间序列分析，这对于传感应用程序（例如人类活动识别和空气质量预测）至关重要。硬件感知神经体系结构搜索（NAS）的最新努力自动化特定平台的架构发现；但是，没有一个专注于Edge部署的一般时间序列分析。利用大语言模型（LLM）的解决问题和推理能力，我们提出了Monaq，这是一个新颖的框架，将NAS重新制定为多目标神经体系结构查询任务。 Monaq配备了多模式查询生成，用于处理多模式的时间序列输入和硬件约束，以及基于LLM代理的多目标搜索，以通过代码生成实现部署准备模型。通过集成数值数据，时间序列图像和文本描述，MONAQ可以提高LLM对时间序列数据的理解。 15个数据集的实验表明，莫纳克（Monaq）发现的模型的表现优于手工制作的模型和NAS基准，同时更有效。

Title: MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly

• Authors: Zhaowei Wang, Wenhao Yu, Xiyu Ren, Jipeng Zhang, Yu Zhao, Rohit Saxena, Liang Cheng, Ginny Wong, Simon See, Pasquale Minervini, Yangqiu Song, Mark Steedman
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.10610
• Pdf URL: https://arxiv.org/pdf/2505.10610
• Copy Paste: [[2505.10610]] MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly(https://arxiv.org/abs/2505.10610)
• Keywords: generation
• Abstract: The rapid extension of context windows in large vision-language models has given rise to long-context vision-language models (LCVLMs), which are capable of handling hundreds of images with interleaved text tokens in a single forward pass. In this work, we introduce MMLongBench, the first benchmark covering a diverse set of long-context vision-language tasks, to evaluate LCVLMs effectively and thoroughly. MMLongBench is composed of 13,331 examples spanning five different categories of downstream tasks, such as Visual RAG and Many-Shot ICL. It also provides broad coverage of image types, including various natural and synthetic images. To assess the robustness of the models to different input lengths, all examples are delivered at five standardized input lengths (8K-128K tokens) via a cross-modal tokenization scheme that combines vision patches and text tokens. Through a thorough benchmarking of 46 closed-source and open-source LCVLMs, we provide a comprehensive analysis of the current models' vision-language long-context ability. Our results show that: i) performance on a single task is a weak proxy for overall long-context capability; ii) both closed-source and open-source models face challenges in long-context vision-language tasks, indicating substantial room for future improvement; iii) models with stronger reasoning ability tend to exhibit better long-context performance. By offering wide task coverage, various image types, and rigorous length control, MMLongBench provides the missing foundation for diagnosing and advancing the next generation of LCVLMs.
• 摘要：在大型视觉语言模型中，上下文窗口的快速扩展导致了长篇小说视觉模型（LCVLMS），这些模型能够在单个正向通行证中处理数百张带有交错文本令牌的图像。在这项工作中，我们介绍了Mmlongbench，这是第一个涵盖一套长篇文化视觉语言任务的基准，以有效而彻底地评估LCVLMS。 Mmlongbench由13,331个示例组成，涵盖了五个不同类别的下游任务，例如Visual抹布和许多弹药ICL。它还提供了图像类型的广泛覆盖范围，包括各种自然图像和合成图像。为了评估模型对不同输入长度的鲁棒性，所有示例均以五个标准化的输入长度（8k-128k令牌）通过跨模式的令牌化方案进行交付，该方案结合了视觉斑块和文本令牌。通过对46个封闭源和开源LCVLM的彻底基准测试，我们对当前模型的视觉语言长篇下说能力进行了全面分析。我们的结果表明：i）单个任务的性能是整体长篇文化功能的弱代理； ii）封闭源和开源模型在长篇文化视觉任务中面临挑战，这表明将来有很大的改进空间； iii）具有更强推理能力的模型倾向于表现出更好的长期性能。通过提供广泛的任务覆盖范围，各种图像类型和严格的长度控制，Mmlongbench为诊断和推进下一代LCVLM的基础提供了缺失的基础。

Title: How many measurements are enough? Bayesian recovery in inverse problems with general distributions

• Authors: Ben Adcock, Nick Huang
• Subjects: cs.LG, math.ST
• Abstract URL: https://arxiv.org/abs/2505.10630
• Pdf URL: https://arxiv.org/pdf/2505.10630
• Copy Paste: [[2505.10630]] How many measurements are enough? Bayesian recovery in inverse problems with general distributions(https://arxiv.org/abs/2505.10630)
• Keywords: generative
• Abstract: We study the sample complexity of Bayesian recovery for solving inverse problems with general prior, forward operator and noise distributions. We consider posterior sampling according to an approximate prior $\mathcal{P}$, and establish sufficient conditions for stable and accurate recovery with high probability. Our main result is a non-asymptotic bound that shows that the sample complexity depends on (i) the intrinsic complexity of $\mathcal{P}$, quantified by its so-called approximate covering number, and (ii) concentration bounds for the forward operator and noise distributions. As a key application, we specialize to generative priors, where $\mathcal{P}$ is the pushforward of a latent distribution via a Deep Neural Network (DNN). We show that the sample complexity scales log-linearly with the latent dimension $k$, thus establishing the efficacy of DNN-based priors. Generalizing existing results on deterministic (i.e., non-Bayesian) recovery for the important problem of random sampling with an orthogonal matrix $U$, we show how the sample complexity is determined by the coherence of $U$ with respect to the support of $\mathcal{P}$. Hence, we establish that coherence plays a fundamental role in Bayesian recovery as well. Overall, our framework unifies and extends prior work, providing rigorous guarantees for the sample complexity of solving Bayesian inverse problems with arbitrary distributions.
• 摘要：我们研究了贝叶斯恢复的样本复杂性，以解决一般先验，前向操作员和噪声分布的逆问题。我们根据大约先前的$ \ Mathcal {p} $考虑后验采样，并建立足够的条件，以高概率稳定而准确的恢复。我们的主要结果是一种非质合结构，表明样品复杂性取决于（i）$ \ natercal {p} $的内在复杂性，该复杂性是由其所谓的近似覆盖率量化的，以及（ii）向前操作员和噪声分布的浓度界限。作为关键应用程序，我们专门研究生成先验，其中$ \ Mathcal {p} $是通过深神经网络（DNN）的潜在分布的推动力。我们表明，样本复杂性与潜在尺寸$ k $线性缩放，从而确立了基于DNN的先验的功效。将现有结果推广到确定性的（即非乘式）恢复中，以与正交矩阵$ u $进行随机抽样的重要问题，我们展示了如何通过$ u $的连贯性来确定样本复杂性，以确定$ \ Mathcal {p} $的支持。因此，我们确定连贯性在贝叶斯恢复中也起着基本作用。总体而言，我们的框架统一并扩展了先前的工作，为解决贝叶斯逆问题的样本复杂性提供了严格的保证。

Title: CLIP Embeddings for AI-Generated Image Detection: A Few-Shot Study with Lightweight Classifier

• Authors: Ziyang Ou
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.10664
• Pdf URL: https://arxiv.org/pdf/2505.10664
• Copy Paste: [[2505.10664]] CLIP Embeddings for AI-Generated Image Detection: A Few-Shot Study with Lightweight Classifier(https://arxiv.org/abs/2505.10664)
• Keywords: generation
• Abstract: Verifying the authenticity of AI-generated images presents a growing challenge on social media platforms these days. While vision-language models (VLMs) like CLIP outdo in multimodal representation, their capacity for AI-generated image classification is underexplored due to the absence of such labels during the pre-training process. This work investigates whether CLIP embeddings inherently contain information indicative of AI generation. A proposed pipeline extracts visual embeddings using a frozen CLIP model, feeds its embeddings to lightweight networks, and fine-tunes only the final classifier. Experiments on the public CIFAKE benchmark show the performance reaches 95% accuracy without language reasoning. Few-shot adaptation to curated custom with 20% of the data results in performance to 85%. A closed-source baseline (Gemini-2.0) has the best zero-shot accuracy yet fails on specific styles. Notably, some specific image types, such as wide-angle photographs and oil paintings, pose significant challenges to classification. These results indicate previously unexplored difficulties in classifying certain types of AI-generated images, revealing new and more specific questions in this domain that are worth further investigation.
• 摘要：如今，验证AI生成图像的真实性在社交媒体平台上提出了越来越多的挑战。尽管视觉模型（VLM）（例如剪辑）在多模式表示中淘汰了，但由于在预训练过程中没有此类标签，因此没有忽略AI生成的图像分类的能力。这项工作调查了剪辑嵌入是否固有地包含指示AI生成的信息。提出的管道使用冷冻夹模型提取视觉嵌入，将其嵌入到轻质网络中，而仅微调最终分类器。公共CIFAKE基准的实验表明，在没有语言推理的情况下，性能达到了95％的精度。几乎没有针对20％数据的策划策划的速度适应，从而使性能达到85％。封闭源基线（Gemini-2.0）具有最佳的零照片精度，但在特定样式上失败了。值得注意的是，某些特定的图像类型（例如广角照片和油画）对分类构成了重大挑战。这些结果表明，在对某些类型的AI生成的图像进行分类时，以前未开发的困难，在该领域中揭示了值得进一步研究的新的和更具体的问题。

Title: Clustering Rooftop PV Systems via Probabilistic Embeddings

• Authors: Kutay Bölat, Tarek Alskaif, Peter Palensky, Simon Tindemans
• Subjects: cs.LG, eess.SP
• Abstract URL: https://arxiv.org/abs/2505.10699
• Pdf URL: https://arxiv.org/pdf/2505.10699
• Copy Paste: [[2505.10699]] Clustering Rooftop PV Systems via Probabilistic Embeddings(https://arxiv.org/abs/2505.10699)
• Keywords: generation
• Abstract: As the number of rooftop photovoltaic (PV) installations increases, aggregators and system operators are required to monitor and analyze these systems, raising the challenge of integration and management of large, spatially distributed time-series data that are both high-dimensional and affected by missing values. In this work, a probabilistic entity embedding-based clustering framework is proposed to address these problems. This method encodes each PV system's characteristic power generation patterns and uncertainty as a probability distribution, then groups systems by their statistical distances and agglomerative clustering. Applied to a multi-year residential PV dataset, it produces concise, uncertainty-aware cluster profiles that outperform a physics-based baseline in representativeness and robustness, and support reliable missing-value imputation. A systematic hyperparameter study further offers practical guidance for balancing model performance and robustness.
• 摘要：随着屋顶光伏（PV）安装的数量的增加，需要聚合器和系统运算符来监视和分析这些系统，从而增加了集成和管理大型，空间分布的时间序列数据的挑战，这些数据既具有高维度又受缺失值影响。在这项工作中，提出了一个基于概率的实体嵌入聚类框架来解决这些问题。此方法将每个PV系统的特征发电模式和不确定性作为概率分布进行编码，然后按系统的统计距离和集聚聚类对系统进行分组。应用于多年住宅PV数据集，它产生简洁的，不确定性的集群剖面，在代表性和鲁棒性方面优于基于物理的基线，并支持可靠的缺失值插补。一项系统的超参数研究进一步提供了平衡模型性能和鲁棒性的实用指导。

Title: IMAGE-ALCHEMY: Advancing subject fidelity in personalised text-to-image generation

• Authors: Amritanshu Tiwari, Cherish Puniani, Kaustubh Sharma, Ojasva Nema
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.10743
• Pdf URL: https://arxiv.org/pdf/2505.10743
• Copy Paste: [[2505.10743]] IMAGE-ALCHEMY: Advancing subject fidelity in personalised text-to-image generation(https://arxiv.org/abs/2505.10743)
• Keywords: generation, generative
• Abstract: Recent advances in text-to-image diffusion models, particularly Stable Diffusion, have enabled the generation of highly detailed and semantically rich images. However, personalizing these models to represent novel subjects based on a few reference images remains challenging. This often leads to catastrophic forgetting, overfitting, or large computational this http URL propose a two-stage pipeline that addresses these limitations by leveraging LoRA-based fine-tuning on the attention weights within the U-Net of the Stable Diffusion XL (SDXL) model. First, we use the unmodified SDXL to generate a generic scene by replacing the subject with its class label. Then, we selectively insert the personalized subject through a segmentation-driven image-to-image (Img2Img) pipeline that uses the trained LoRA this http URL framework isolates the subject encoding from the overall composition, thus preserving SDXL's broader generative capabilities while integrating the new subject in a high-fidelity manner. Our method achieves a DINO similarity score of 0.789 on SDXL, outperforming existing personalized text-to-image approaches.
• 摘要：文本到图像扩散模型的最新进展，尤其是稳定的扩散，使得能够产生高度详细且语义丰富的图像。但是，将这些模型个性化以根据一些参考图像表示新主题仍然具有挑战性。这通常会导致灾难性的遗忘，过度拟合或大型计算，此HTTP URL提出了一条两阶段的管道，该管道通过利用基于Lora的微调来解决这些限制，以对稳定扩散XL（SDXL）模型的U-NET的注意力重量进行基于Lora的微调。首先，我们使用未修改的SDXL通过用其类标签替换主题来生成通用场景。然后，我们通过细分驱动的图像对象（IMG2IMG）管道选择性地插入个性化主题，该管道使用训练有素的LORA这个HTTP URL框架将编码的主题与整体组成的编码隔离，从而保留SDXL更广泛的生成能力，同时将新主题集成到高足限的方式中。我们的方法在SDXL上达到了0.789的恐龙相似性评分，表现优于现有的个性化文本对图像方法。

Title: Deep Symbolic Optimization: Reinforcement Learning for Symbolic Mathematics

• Authors: Conor F. Hayes, Felipe Leno Da Silva, Jiachen Yang, T. Nathan Mundhenk, Chak Shing Lee, Jacob F. Pettit, Claudio Santiago, Sookyung Kim, Joanne T. Kim, Ignacio Aravena Solis, Ruben Glatt, Andre R. Goncalves, Alexander Ladd, Ahmet Can Solak, Thomas Desautels, Daniel Faissol, Brenden K. Petersen, Mikel Landajuela
• Subjects: cs.LG, cs.NE, cs.SC
• Abstract URL: https://arxiv.org/abs/2505.10762
• Pdf URL: https://arxiv.org/pdf/2505.10762
• Copy Paste: [[2505.10762]] Deep Symbolic Optimization: Reinforcement Learning for Symbolic Mathematics(https://arxiv.org/abs/2505.10762)
• Keywords: generative
• Abstract: Deep Symbolic Optimization (DSO) is a novel computational framework that enables symbolic optimization for scientific discovery, particularly in applications involving the search for intricate symbolic structures. One notable example is equation discovery, which aims to automatically derive mathematical models expressed in symbolic form. In DSO, the discovery process is formulated as a sequential decision-making task. A generative neural network learns a probabilistic model over a vast space of candidate symbolic expressions, while reinforcement learning strategies guide the search toward the most promising regions. This approach integrates gradient-based optimization with evolutionary and local search techniques, and it incorporates in-situ constraints, domain-specific priors, and advanced policy optimization methods. The result is a robust framework capable of efficiently exploring extensive search spaces to identify interpretable and physically meaningful models. Extensive evaluations on benchmark problems have demonstrated that DSO achieves state-of-the-art performance in both accuracy and interpretability. In this chapter, we provide a comprehensive overview of the DSO framework and illustrate its transformative potential for automating symbolic optimization in scientific discovery.
• 摘要：深符号优化（DSO）是一个新颖的计算框架，可以为科学发现提供符号优化，尤其是在涉及搜索复杂符号结构的应用中。一个值得注意的例子是方程发现，该方程式旨在自动得出以符号形式表达的数学模型。在DSO中，发现过程被称为一项顺序决策任务。生成的神经网络在大量的候选象征表达式上学习了概率模型，同时增强学习策略指导搜索最有前途的地区。这种方法将基于梯度的优化与进化和本地搜索技术集成在一起，并结合了原位约束，特定领域的先验和高级策略优化方法。结果是一个强大的框架，能够有效地探索广泛的搜索空间，以识别可解释且具有物理意义的模型。对基准问题的广泛评估表明，DSO在准确性和可解释性方面都能达到最先进的表现。在本章中，我们提供了DSO框架的全面概述，并说明了其在科学发现中自动化符号优化的变革潜力。

Title: Context-Aware Probabilistic Modeling with LLM for Multimodal Time Series Forecasting

• Authors: Yueyang Yao, Jiajun Li, Xingyuan Dai, MengMeng Zhang, Xiaoyan Gong, Fei-Yue Wang, Yisheng Lv
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.10774
• Pdf URL: https://arxiv.org/pdf/2505.10774
• Copy Paste: [[2505.10774]] Context-Aware Probabilistic Modeling with LLM for Multimodal Time Series Forecasting(https://arxiv.org/abs/2505.10774)
• Keywords: generation
• Abstract: Time series forecasting is important for applications spanning energy markets, climate analysis, and traffic management. However, existing methods struggle to effectively integrate exogenous texts and align them with the probabilistic nature of large language models (LLMs). Current approaches either employ shallow text-time series fusion via basic prompts or rely on deterministic numerical decoding that conflict with LLMs' token-generation paradigm, which limits contextual awareness and distribution modeling. To address these limitations, we propose CAPTime, a context-aware probabilistic multimodal time series forecasting method that leverages text-informed abstraction and autoregressive LLM decoding. Our method first encodes temporal patterns using a pretrained time series encoder, then aligns them with textual contexts via learnable interactions to produce joint multimodal representations. By combining a mixture of distribution experts with frozen LLMs, we enable context-aware probabilistic forecasting while preserving LLMs' inherent distribution modeling capabilities. Experiments on diverse time series forecasting tasks demonstrate the superior accuracy and generalization of CAPTime, particularly in multimodal scenarios. Additional analysis highlights its robustness in data-scarce scenarios through hybrid probabilistic decoding.
• 摘要：时间序列预测对于涵盖能源市场，气候分析和交通管理的应用很重要。但是，现有方法难以有效地整合外源文本，并与大语言模型（LLM）的概率性质保持一致。当前的方法要么通过基本提示采用浅文本时间系列融合，要么依靠确定性的数值解码，即与LLMS的代币生成范式冲突，这限制了上下文意识和分布建模。为了解决这些局限性，我们提出了Captime，Captime是一种上下文感知的概率多模式时间序列预测方法，该方法利用文本信息的抽象和自动回归的LLM解码。我们的方法首先使用验证的时间序列编码器编码时间模式，然后通过可学习的交互与文本上下文对齐，以产生联合多模式表示。通过将分销专家的混合物与冷冻LLM相结合，我们可以启用上下文感知的概率预测，同时保留LLMS固有的分配建模功能。各种时间序列预测任务的实验证明了CAPTIME的卓越准确性和概括，尤其是在多模式场景中。其他分析突出了其通过混合概率解码在数据筛选方案中的鲁棒性。

Title: MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generation

• Authors: Gabriel Maldonado, Armin Danesh Pazho, Ghazal Alinezhad Noghre, Vinit Katariya, Hamed Tabkhi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.10810
• Pdf URL: https://arxiv.org/pdf/2505.10810
• Copy Paste: [[2505.10810]] MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generation(https://arxiv.org/abs/2505.10810)
• Keywords: generation
• Abstract: Human motion generation is essential for fields such as animation, robotics, and virtual reality, requiring models that effectively capture motion dynamics from text descriptions. Existing approaches often rely on Contrastive Language-Image Pretraining (CLIP)-based text encoders, but their training on text-image pairs constrains their ability to understand temporal and kinematic structures inherent in motion and motion generation. This work introduces MoCLIP, a fine-tuned CLIP model with an additional motion encoding head, trained on motion sequences using contrastive learning and tethering loss. By explicitly incorporating motion-aware representations, MoCLIP enhances motion fidelity while remaining compatible with existing CLIP-based pipelines and seamlessly integrating into various CLIP-based methods. Experiments demonstrate that MoCLIP improves Top-1, Top-2, and Top-3 accuracy while maintaining competitive FID, leading to improved text-to-motion alignment results. These results highlight MoCLIP's versatility and effectiveness, establishing it as a robust framework for enhancing motion generation.
• 摘要：人类运动产生对于诸如动画，机器人技术和虚拟现实等领域至关重要，需要从文本描述中有效捕获运动动态的模型。现有的方法通常依赖于对比的语言图像预处理（基于剪辑）的文本编码器，但是他们对文本图像对的培训限制了他们理解运动和运动产生固有的时间和运动结构的能力。这项工作引入了Moclip，这是一种带有额外运动编码头的微调夹模型，使用对比度学习和束缚损失对运动序列进行了训练。通过明确合并运动感知表示，Moclip增强了运动保真度，同时与现有的基于夹的管道兼容，并无缝集成到基于夹的各种方法中。实验表明，MOCLIP在保持竞争性FID的同时提高了TOP-1，TOP-2和TOP-3精度，从而改善了文本到动作对准结果。这些结果突出了Moclip的多功能性和有效性，将其确立为增强运动产生的强大框架。

Title: Approximation and Generalization Abilities of Score-based Neural Network Generative Models for Sub-Gaussian Distributions

• Authors: Guoji Fu, Wee Sun Lee
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.10880
• Pdf URL: https://arxiv.org/pdf/2505.10880
• Copy Paste: [[2505.10880]] Approximation and Generalization Abilities of Score-based Neural Network Generative Models for Sub-Gaussian Distributions(https://arxiv.org/abs/2505.10880)
• Keywords: generative
• Abstract: This paper studies the approximation and generalization abilities of score-based neural network generative models (SGMs) in estimating an unknown distribution $P_0$ from $n$ i.i.d. observations in $d$ dimensions. Assuming merely that $P_0$ is $\alpha$-sub-Gaussian, we prove that for any time step $t \in [t_0, n^{O(1)}]$, where $t_0 \geq O(\alpha^2n^{-2/d}\log n)$, there exists a deep ReLU neural network with width $\leq O(\log^3n)$ and depth $\leq O(n^{3/d}\log_2n)$ that can approximate the scores with $\tilde{O}(n^{-1})$ mean square error and achieve a nearly optimal rate of $\tilde{O}(n^{-1}t_0^{-d/2})$ for score estimation, as measured by the score matching loss. Our framework is universal and can be used to establish convergence rates for SGMs under milder assumptions than previous work. For example, assuming further that the target density function $p_0$ lies in Sobolev or Besov classes, with an appropriately early stopping strategy, we demonstrate that neural network-based SGMs can attain nearly minimax convergence rates up to logarithmic factors. Our analysis removes several crucial assumptions, such as Lipschitz continuity of the score function or a strictly positive lower bound on the target density.
• 摘要：本文研究了基于得分的神经网络生成模型（SGM）的近似和概括能力，从而估算了$ n $ i.i.d.的未知分布$ p_0 $。 $ d $尺寸的观察。假设$ p_0 $是$ \ alpha $ -sub-gaussian，我们证明，在[t_0，n^{o（1）] $中的任何时间步骤$ t \中o（\ log^3n）$和深度$ \ leq o（n^{3/d} \ log_2n）$，可以用$ \ tilde {o}（n^{ - 1}）$均值近似得分近似，并达到平方误差，并达到几乎最佳的速率$ \ tilde {通过得分匹配损失来衡量。我们的框架是通用的，可用于在较温和的假设下建立SGM的收敛率，而不是以前的工作。例如，假设目标密度函数$ P_0 $在于Sobolev或BESOV类，则具有适当的早期停止策略，我们证明了基于神经网络的SGM可以达到几乎最小的收敛率，直到对数因素。我们的分析消除了几个关键假设，例如分数函数的Lipschitz连续性或目标密度严格的正下限。

Title: Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models

• Authors: Akhil Agnihotri, Rahul Jain, Deepak Ramachandran, Zheng Wen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.10892
• Pdf URL: https://arxiv.org/pdf/2505.10892
• Copy Paste: [[2505.10892]] Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models(https://arxiv.org/abs/2505.10892)
• Keywords: generative
• Abstract: Post-training of LLMs with RLHF, and subsequently preference optimization algorithms such as DPO, IPO, etc., made a big difference in improving human alignment. However, all such techniques can only work with a single (human) objective. In practice, human users have multiple objectives, such as helpfulness and harmlessness, and there is no natural way to aggregate them into a single objective. In this paper, we address the multi-objective preference-alignment problem, where a policy must optimize several, potentially conflicting, objectives. We introduce the Multi-Objective Preference Optimization (MOPO) algorithm, which frames alignment as a constrained KL-regularized optimization: the primary objective is maximized while secondary objectives are lower-bounded by tunable safety thresholds. Unlike prior work, MOPO operates directly on pairwise preference data, requires no point-wise reward assumption, and avoids heuristic prompt-context engineering. The method recovers policies on the Pareto front whenever the front is attainable; practically, it reduces to simple closed-form iterative updates suitable for large-scale training. On synthetic benchmarks with diverse canonical preference structures, we show that MOPO approximates the Pareto front. When fine-tuning a 1.3B-parameter language model on real-world human-preference datasets, MOPO attains higher rewards and yields policies that Pareto-dominate baselines; ablation studies confirm optimization stability and robustness to hyperparameters.
• 摘要：使用RLHF的LLM培训以及随后的偏好优化算法，例如DPO，IPO等，在改善人类对齐方面具有很大的不同。但是，所有这些技术只能与一个（人类）目标一起使用。在实践中，人类用户具有多种目标，例如帮助和无害性，并且没有自然的方法将其汇总为一个目标。在本文中，我们解决了多目标的首选项问题，其中策略必须优化几个潜在的相互矛盾的目标。我们介绍了多目标优先优化（MOPO）算法，该算法将对齐作为受约束的KL调节优化：主要目标是最大化的，而次级目标则通过可调安全性阈值较低。与先前的工作不同，MOPO直接在成对的偏好数据上运行，不需要明智的奖励假设，并且避免了启发式及时及其启发式文化工程。每当可以实现前部时，该方法就会在帕累托方面恢复政策。实际上，它减少了简单的封闭式迭代更新，适合大规模培训。在具有不同规范偏好结构的合成基准上，我们表明MOPO近似帕累托前部。在对现实世界中的人类质疑数据集中微调1.3B参数语言模型时，MOPO获得了更高的奖励，并产生了占主导地位的基线的政策。消融研究证实了对超参数的优化稳定性和鲁棒性。

Title: Shackled Dancing: A Bit-Locked Diffusion Algorithm for Lossless and Controllable Image Steganography

• Authors: Tianshuo Zhang, Gao Jia, Wenzhe Zhai, Rui Yann, Xianglei Xing
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.10950
• Pdf URL: https://arxiv.org/pdf/2505.10950
• Copy Paste: [[2505.10950]] Shackled Dancing: A Bit-Locked Diffusion Algorithm for Lossless and Controllable Image Steganography(https://arxiv.org/abs/2505.10950)
• Keywords: generation, generative
• Abstract: Data steganography aims to conceal information within visual content, yet existing spatial- and frequency-domain approaches suffer from trade-offs between security, capacity, and perceptual quality. Recent advances in generative models, particularly diffusion models, offer new avenues for adaptive image synthesis, but integrating precise information embedding into the generative process remains challenging. We introduce Shackled Dancing Diffusion, or SD$^2$, a plug-and-play generative steganography method that combines bit-position locking with diffusion sampling injection to enable controllable information embedding within the generative trajectory. SD$^2$ leverages the expressive power of diffusion models to synthesize diverse carrier images while maintaining full message recovery with $100\%$ accuracy. Our method achieves a favorable balance between randomness and constraint, enhancing robustness against steganalysis without compromising image fidelity. Extensive experiments show that SD$^2$ substantially outperforms prior methods in security, embedding capacity, and stability. This algorithm offers new insights into controllable generation and opens promising directions for secure visual communication.
• 摘要：数据隐志旨在将信息隐藏在视觉内容中，但现有的空间和频域方法在安全性，容量和感知质量之间取舍。生成模型，尤其是扩散模型的最新进展为自适应图像综合提供了新的途径，但是将精确的信息嵌入到生成过程中仍然具有挑战性。我们介绍了shak缩的舞蹈扩散，即SD $^2 $，一种即插即用的生成隐志方法，将位位点锁定与扩散采样注入结合在一起，以在生成轨迹中启用可控的信息嵌入。 SD $^2 $利用扩散模型的表达能力来综合不同的载体图像，同时以$ 100 \％$的精度维持完整的消息恢复。我们的方法在随机性和约束之间取得了有利的平衡，增强了对稳定分析的鲁棒性，而不会损害图像保真度。广泛的实验表明，SD $^2 $在安全性，嵌入能力和稳定性方面的先验方法大大优于先前的方法。该算法为可控生成提供了新的见解，并为安全的视觉交流打开了有希望的方向。

Title: SubGCache: Accelerating Graph-based RAG with Subgraph-level KV Cache

• Authors: Qiuyu Zhu, Liang Zhang, Qianxiong Xu, Cheng Long, Jie Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.10951
• Pdf URL: https://arxiv.org/pdf/2505.10951
• Copy Paste: [[2505.10951]] SubGCache: Accelerating Graph-based RAG with Subgraph-level KV Cache(https://arxiv.org/abs/2505.10951)
• Keywords: generation
• Abstract: Graph-based retrieval-augmented generation (RAG) enables large language models (LLMs) to incorporate structured knowledge via graph retrieval as contextual input, enhancing more accurate and context-aware reasoning. We observe that for different queries, it could retrieve similar subgraphs as prompts, and thus we propose SubGCache, which aims to reduce inference latency by reusing computation across queries with similar structural prompts (i.e., subgraphs). Specifically, SubGCache clusters queries based on subgraph embeddings, constructs a representative subgraph for each cluster, and pre-computes the key-value (KV) cache of the representative subgraph. For each query with its retrieved subgraph within a cluster, it reuses the pre-computed KV cache of the representative subgraph of the cluster without computing the KV tensors again for saving computation. Experiments on two new datasets across multiple LLM backbones and graph-based RAG frameworks demonstrate that SubGCache consistently reduces inference latency with comparable and even improved generation quality, achieving up to 6.68$\times$ reduction in time-to-first-token (TTFT).
• 摘要：基于图的检索效果生成（RAG）使大型语言模型（LLMS）通过图检索将结构化知识合并为上下文输入，增强了更准确和上下文感知的推理。我们观察到，对于不同的查询，它可以检索与提示相似的子图，因此我们提出了subgcache，旨在通过重复使用具有相似结构提示的查询（即子图）的查询来减少推理潜伏期。具体而言，基于子图嵌入的subgcache群集查询，为每个群集构造一个代表性子图，并预先计算代表性子图的键值（KV）缓存。对于每个查询，均在集群中检索到的子图，它重复了群集代表子图的预先计算的KV缓存，而无需再次计算KV张量以节省计算。在多个LLM骨架和基于图的抹布框架上进行两个新数据集的实验表明，子gcache始终通过可比甚至改善的发电质量来降低推理潜伏期，达到6.68 $ \ times $降低，减少了第一位tost-toke（TTTFT）的时间。

Title: GenoArmory: A Unified Evaluation Framework for Adversarial Attacks on Genomic Foundation Models

• Authors: Haozheng Luo, Chenghao Qiu, Yimin Wang, Shang Wu, Jiahao Yu, Han Liu, Binghui Wang, Yan Chen
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2505.10983
• Pdf URL: https://arxiv.org/pdf/2505.10983
• Copy Paste: [[2505.10983]] GenoArmory: A Unified Evaluation Framework for Adversarial Attacks on Genomic Foundation Models(https://arxiv.org/abs/2505.10983)
• Keywords: generative
• Abstract: We propose the first unified adversarial attack benchmark for Genomic Foundation Models (GFMs), named GenoArmory. Unlike existing GFM benchmarks, GenoArmory offers the first comprehensive evaluation framework to systematically assess the vulnerability of GFMs to adversarial attacks. Methodologically, we evaluate the adversarial robustness of five state-of-the-art GFMs using four widely adopted attack algorithms and three defense strategies. Importantly, our benchmark provides an accessible and comprehensive framework to analyze GFM vulnerabilities with respect to model architecture, quantization schemes, and training datasets. Additionally, we introduce GenoAdv, a new adversarial sample dataset designed to improve GFM safety. Empirically, classification models exhibit greater robustness to adversarial perturbations compared to generative models, highlighting the impact of task type on model vulnerability. Moreover, adversarial attacks frequently target biologically significant genomic regions, suggesting that these models effectively capture meaningful sequence features.
• 摘要：我们提出了第一个名为Genoarmory的基因组基础模型（GFM）的统一对抗性攻击基准。与现有的GFM基准不同，GenoarMory提供了第一个全面的评估框架，以系统地评估GFMS对对抗攻击的脆弱性。从方法上讲，我们使用四种广泛采用的攻击算法和三种防御策略评估了五个最先进的GFM的对抗性鲁棒性。重要的是，我们的基准提供了一个可访问且全面的框架，可以在模型架构，量化方案和培训数据集方面分析GFM漏洞。此外，我们介绍了GenoAdv，这是一种旨在提高GFM安全性的新型对抗样品数据集。从经验上讲，与生成模型相比，分类模型对对抗性扰动表现出更大的鲁棒性，强调了任务类型对模型脆弱性的影响。此外，对抗性攻击通常针对具有生物学意义的基因组区域，这表明这些模型有效地捕获了有意义的序列特征。

Title: DDAE++: Enhancing Diffusion Models Towards Unified Generative and Discriminative Learning

• Authors: Weilai Xiang, Hongyu Yang, Di Huang, Yunhong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.10999
• Pdf URL: https://arxiv.org/pdf/2505.10999
• Copy Paste: [[2505.10999]] DDAE++: Enhancing Diffusion Models Towards Unified Generative and Discriminative Learning(https://arxiv.org/abs/2505.10999)
• Keywords: generation, generative
• Abstract: While diffusion models have gained prominence in image synthesis, their generative pre-training has been shown to yield discriminative representations, paving the way towards unified visual generation and understanding. However, two key questions remain: 1) Can these representations be leveraged to improve the training of diffusion models themselves, rather than solely benefiting downstream tasks? 2) Can the feature quality be enhanced to rival or even surpass modern self-supervised learners, without compromising generative capability? This work addresses these questions by introducing self-conditioning, a straightforward yet effective mechanism that internally leverages the rich semantics inherent in denoising network to guide its own decoding layers, forming a tighter bottleneck that condenses high-level semantics to improve generation. Results are compelling: our method boosts both generation FID and recognition accuracy with 1% computational overhead and generalizes across diverse diffusion architectures. Crucially, self-conditioning facilitates an effective integration of discriminative techniques, such as contrastive self-distillation, directly into diffusion models without sacrificing generation quality. Extensive experiments on pixel-space and latent-space datasets show that in linear evaluations, our enhanced diffusion models, particularly UViT and DiT, serve as strong representation learners, surpassing various self-supervised models.
• 摘要：尽管扩散模型在图像综合中已获得突出性，但它们的生成预训练已被证明产生了歧视性表示，为统一的视觉产生和理解铺平了道路。但是，仍然存在两个关键问题：1）可以利用这些表示形式来改善扩散模型本身的训练，而不是仅仅使下游任务受益？ 2）在不损害生成能力的情况下，是否可以提高功能质量到竞争对手，甚至超过现代的自我监督学习者？这项工作通过引入自我调节来解决这些问题，这是一种直接而有效的机制，它在内部利用了DeNoing网络固有的丰富语义来指导其自己的解码层，形成了更紧密的瓶颈，从而凝结了高级语义，以改善发电。结果令人信服：我们的方法可以提高生成FID和识别精度，并具有1％的计算开销，并在各种扩散体系结构之间进行了概括。至关重要的是，自我条件有助于有效地整合歧视技术，例如对比度自我鉴定，直接进入扩散模型，而无需牺牲产生质量。关于像素空间和潜在空间数据集的广泛实验表明，在线性评估中，我们增强的扩散模型，尤其是紫外线和DIT，是强大的代表学习者，超过了各种自我监督模型。

Title: Towards Robust and Controllable Text-to-Motion via Masked Autoregressive Diffusion

• Authors: Zongye Zhang, Bohan Kong, Qingjie Liu, Yunhong Wang
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2505.11013
• Pdf URL: https://arxiv.org/pdf/2505.11013
• Copy Paste: [[2505.11013]] Towards Robust and Controllable Text-to-Motion via Masked Autoregressive Diffusion(https://arxiv.org/abs/2505.11013)
• Keywords: generation
• Abstract: Generating 3D human motion from text descriptions remains challenging due to the diverse and complex nature of human motion. While existing methods excel within the training distribution, they often struggle with out-of-distribution motions, limiting their applicability in real-world scenarios. Existing VQVAE-based methods often fail to represent novel motions faithfully using discrete tokens, which hampers their ability to generalize beyond seen data. Meanwhile, diffusion-based methods operating on continuous representations often lack fine-grained control over individual frames. To address these challenges, we propose a robust motion generation framework MoMADiff, which combines masked modeling with diffusion processes to generate motion using frame-level continuous representations. Our model supports flexible user-provided keyframe specification, enabling precise control over both spatial and temporal aspects of motion synthesis. MoMADiff demonstrates strong generalization capability on novel text-to-motion datasets with sparse keyframes as motion prompts. Extensive experiments on two held-out datasets and two standard benchmarks show that our method consistently outperforms state-of-the-art models in motion quality, instruction fidelity, and keyframe adherence.
• 摘要：由于人类运动的多样化和复杂的性质，从文本描述中产生3D人类运动仍然具有挑战性。尽管现有的方法在培训分布中表现出色，但他们经常在分发动作方面挣扎，从而限制了其在现实情况下的适用性。现有的基于VQVAE的方法通常无法使用离散令牌忠实地代表新的动作，从而妨碍了它们超越可见数据的能力。同时，基于连续表示的基于扩散的方法通常缺乏对单个帧的细粒度控制。为了应对这些挑战，我们提出了一个强大的运动生成框架Momadiff，该框架结合了蒙版的建模与扩散过程，以使用框架级连续表示来生成运动。我们的模型支持灵活的用户提供的密钥帧规范，可以精确控制运动合成的空间和时间方面。 Momadiff在运动提示中以稀疏的钥匙帧在新颖的文本到动作数据集上表现出强大的概括能力。在两个持有数据集和两个标准基准上进行的广泛实验表明，我们的方法在运动质量，指导忠诚度和密钥框架遵守方面始终优于最先进的模型。

Title: HSRMamba: Efficient Wavelet Stripe State Space Model for Hyperspectral Image Super-Resolution

• Authors: Baisong Li, Xingwang Wang, Haixiao Xu
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2505.11062
• Pdf URL: https://arxiv.org/pdf/2505.11062
• Copy Paste: [[2505.11062]] HSRMamba: Efficient Wavelet Stripe State Space Model for Hyperspectral Image Super-Resolution(https://arxiv.org/abs/2505.11062)
• Keywords: super-resolution, generation
• Abstract: Single hyperspectral image super-resolution (SHSR) aims to restore high-resolution images from low-resolution hyperspectral images. Recently, the Visual Mamba model has achieved an impressive balance between performance and computational efficiency. However, due to its 1D scanning paradigm, the model may suffer from potential artifacts during image generation. To address this issue, we propose HSRMamba. While maintaining the computational efficiency of Visual Mamba, we introduce a strip-based scanning scheme to effectively reduce artifacts from global unidirectional scanning. Additionally, HSRMamba uses wavelet decomposition to alleviate modal conflicts between high-frequency spatial features and low-frequency spectral features, further improving super-resolution performance. Extensive experiments show that HSRMamba not only excels in reducing computational load and model size but also outperforms existing methods, achieving state-of-the-art results.
• 摘要：单光谱图像超分辨率（SHSR）旨在恢复低分辨率高光谱图像的高分辨率图像。最近，Visual Mamba模型在性能和计算效率之间取得了令人印象深刻的平衡。但是，由于其1D扫描范式，该模型可能会在图像生成过程中遭受潜在的伪影。为了解决这个问题，我们提出了HSRMAMBA。在维持视觉MAMBA的计算效率的同时，我们引入了一种基于剥离的扫描方案，以有效地减少全球单向扫描的伪像。此外，HSRMAMBA使用小波分解来减轻高频空间特征和低频光谱特征之间的模态冲突，从而进一步改善了超分辨率的性能。广泛的实验表明，HSRMAMBA不仅在减少计算负载和模型大小方面表现出色，而且还优于现有方法，从而实现了最新的结果。

Title: Assessing the Performance of Analog Training for Transfer Learning

• Authors: Omobayode Fagbohungbe, Corey Lammie, Malte J. Rasch, Takashi Ando, Tayfun Gokmen, Vijay Narayanan
• Subjects: cs.LG, cs.AI, cs.AR, cs.CV, cs.DC, cs.NE
• Abstract URL: https://arxiv.org/abs/2505.11067
• Pdf URL: https://arxiv.org/pdf/2505.11067
• Copy Paste: [[2505.11067]] Assessing the Performance of Analog Training for Transfer Learning(https://arxiv.org/abs/2505.11067)
• Keywords: generation
• Abstract: Analog in-memory computing is a next-generation computing paradigm that promises fast, parallel, and energy-efficient deep learning training and transfer learning (TL). However, achieving this promise has remained elusive due to a lack of suitable training algorithms. Analog memory devices exhibit asymmetric and non-linear switching behavior in addition to device-to-device variation, meaning that most, if not all, of the current off-the-shelf training algorithms cannot achieve good training outcomes. Also, recently introduced algorithms have enjoyed limited attention, as they require bi-directionally switching devices of unrealistically high symmetry and precision and are highly sensitive. A new algorithm chopped TTv2 (c-TTv2), has been introduced, which leverages the chopped technique to address many of the challenges mentioned above. In this paper, we assess the performance of the c-TTv2 algorithm for analog TL using a Swin-ViT model on a subset of the CIFAR100 dataset. We also investigate the robustness of our algorithm to changes in some device specifications, including weight transfer noise, symmetry point skew, and symmetry point variability
• 摘要：模拟内存计算是一种下一代计算范式，可以快速，平行和节能的深度学习训练和转移学习（TL）。但是，由于缺乏合适的培训算法，实现这一诺言仍然难以捉摸。模拟记忆设备除了设备到设备的变化外，还表现出不对称和非线性切换行为，这意味着大多数（如果不是全部）当前的现成的训练算法都无法实现良好的训练成果。同样，最近引入的算法受到了有限的关注，因为它们需要双向交换不切实际的高对称性和精度的设备，并且非常敏感。已经引入了一种新的算法切碎的TTV2（C-TTV2），该算法利用切碎的技术来应对上述许多挑战。在本文中，我们使用CIFAR100数据集子集上的SWIN-VIT模型评估了模拟TL的C-TTV2算法的性能。我们还研究了算法对某些设备规范的变化的鲁棒性，包括重量传递噪声，对称点偏斜和对称点可变性

Title: Towards Self-Improvement of Diffusion Models via Group Preference Optimization

• Authors: Renjie Chen, Wenfeng Lin, Yichen Zhang, Jiangchuan Wei, Boyuan Liu, Chao Feng, Jiao Ran, Mingyu Guo
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.11070
• Pdf URL: https://arxiv.org/pdf/2505.11070
• Copy Paste: [[2505.11070]] Towards Self-Improvement of Diffusion Models via Group Preference Optimization(https://arxiv.org/abs/2505.11070)
• Keywords: generation
• Abstract: Aligning text-to-image (T2I) diffusion models with Direct Preference Optimization (DPO) has shown notable improvements in generation quality. However, applying DPO to T2I faces two challenges: the sensitivity of DPO to preference pairs and the labor-intensive process of collecting and annotating high-quality data. In this work, we demonstrate that preference pairs with marginal differences can degrade DPO performance. Since DPO relies exclusively on relative ranking while disregarding the absolute difference of pairs, it may misclassify losing samples as wins, or vice versa. We empirically show that extending the DPO from pairwise to groupwise and incorporating reward standardization for reweighting leads to performance gains without explicit data selection. Furthermore, we propose Group Preference Optimization (GPO), an effective self-improvement method that enhances performance by leveraging the model's own capabilities without requiring external data. Extensive experiments demonstrate that GPO is effective across various diffusion models and tasks. Specifically, combining with widely used computer vision models, such as YOLO and OCR, the GPO improves the accurate counting and text rendering capabilities of the Stable Diffusion 3.5 Medium by 20 percentage points. Notably, as a plug-and-play method, no extra overhead is introduced during inference.
• 摘要：具有直接偏好优化（DPO）的对齐文本对图像（T2I）扩散模型已显示出显着改善生成质量。但是，将DPO应用于T2I面临两个挑战：DPO对偏好对的敏感性以及收集和注释高质量数据的劳动密集型过程。在这项工作中，我们证明了与边际差异的偏好对会降低DPO性能。由于DPO完全依赖相对排名，而无视对的绝对差异，因此它可能将丢失样本的分类为胜利，反之亦然。我们从经验上表明，将DPO从成对延伸到GroupWise，并将奖励标准化纳入重新加权导致绩效提高而无需明确的数据选择。此外，我们提出了小组偏好优化（GPO），这是一种有效的自我改进方法，通过在不需要外部数据的情况下利用模型自己的能力来增强性能。广泛的实验表明，GPO在各种扩散模型和任务中都是有效的。具体而言，GPO与广泛使用的计算机视觉模型（例如Yolo和OCR）结合，可以提高稳定的扩散3.5培养基的准确计数和文本渲染能力，提高20个百分点。值得注意的是，作为一种插件方法，推断期间没有额外的开销。

Title: Fault Diagnosis across Heterogeneous Domains via Self-Adaptive Temporal-Spatial Attention and Sample Generation

• Authors: Guangqiang Li, M. Amine Atoui, Xiangshun Li
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.11083
• Pdf URL: https://arxiv.org/pdf/2505.11083
• Copy Paste: [[2505.11083]] Fault Diagnosis across Heterogeneous Domains via Self-Adaptive Temporal-Spatial Attention and Sample Generation(https://arxiv.org/abs/2505.11083)
• Keywords: generation
• Abstract: Deep learning methods have shown promising performance in fault diagnosis for multimode process. Most existing studies assume that the collected health state categories from different operating modes are identical. However, in real industrial scenarios, these categories typically exhibit only partial overlap. The incompleteness of the available data and the large distributional differences between the operating modes pose a significant challenge to existing fault diagnosis methods. To address this problem, a novel fault diagnosis model named self-adaptive temporal-spatial attention network (TSA-SAN) is proposed. First, inter-mode mappings are constructed using healthy category data to generate multimode samples. To enrich the diversity of the fault data, interpolation is performed between healthy and fault samples. Subsequently, the fault diagnosis model is trained using real and generated data. The self-adaptive instance normalization is established to suppress irrelevant information while retaining essential statistical features for diagnosis. In addition, a temporal-spatial attention mechanism is constructed to focus on the key features, thus enhancing the generalization ability of the model. The extensive experiments demonstrate that the proposed model significantly outperforms the state-of-the-art methods. The code will be available on Github at this https URL.
• 摘要：深度学习方法显示了多模过程中故障诊断的有希望的表现。大多数现有研究都认为，来自不同操作模式的收集的健康状态类别是相同的。但是，在实际的工业场景中，这些类别通常仅显示部分重叠。可用数据的不完整性和操作模式之间的巨大分布差异对现有的故障诊断方法构成了重大挑战。为了解决这个问题，提出了一个新型的故障诊断模型，称为自适应暂时空间注意网络（TSA-SAN）。首先，使用健康类别数据构建模式间映射以生成多模样本。为了丰富故障数据的多样性，在健康和断层样品之间进行了插值。随后，使用真实和生成的数据训练故障诊断模型。建立了自适应实例归一化，以抑制无关紧要的信息，同时保留诊断的基本统计特征。此外，构建了时间空间注意机制，以关注关键特征，从而增强模型的概括能力。广泛的实验表明，所提出的模型显着胜过最新方法。该代码将在此HTTPS URL的GitHub上可用。

Title: MAVOS-DD: Multilingual Audio-Video Open-Set Deepfake Detection Benchmark

• Authors: Florinel-Alin Croitoru, Vlad Hondru, Marius Popescu, Radu Tudor Ionescu, Fahad Shahbaz Khan, Mubarak Shah
• Subjects: cs.CV, cs.AI, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2505.11109
• Pdf URL: https://arxiv.org/pdf/2505.11109
• Copy Paste: [[2505.11109]] MAVOS-DD: Multilingual Audio-Video Open-Set Deepfake Detection Benchmark(https://arxiv.org/abs/2505.11109)
• Keywords: generation, generative
• Abstract: We present the first large-scale open-set benchmark for multilingual audio-video deepfake detection. Our dataset comprises over 250 hours of real and fake videos across eight languages, with 60% of data being generated. For each language, the fake videos are generated with seven distinct deepfake generation models, selected based on the quality of the generated content. We organize the training, validation and test splits such that only a subset of the chosen generative models and languages are available during training, thus creating several challenging open-set evaluation setups. We perform experiments with various pre-trained and fine-tuned deepfake detectors proposed in recent literature. Our results show that state-of-the-art detectors are not currently able to maintain their performance levels when tested in our open-set scenarios. We publicly release our data and code at: this https URL.
• 摘要：我们介绍了第一个大型开放设定基准测试，用于多语言音频 - 视频胶合胶检测。我们的数据集包括跨八种语言的250个小时的真实和虚假视频，其中60％的数据正在生成。对于每种语言，伪造的视频都是由七个不同的DeepFake生成模型生成的，这些模型是根据生成内容的质量选择的。我们组织了培训，验证和测试拆分，以便在培训期间只能使用所选生成模型和语言的一个子集，从而创建了一些具有挑战性的开放式评估设置。我们对最近的文献中提出的各种预训练和微调的深层探测器进行了实验。我们的结果表明，在我们的开放式场景中进行测试时，最新的检测器目前无法保持其性能水平。我们将数据和代码公开发布：此HTTPS URL。

Title: Deepfake Forensic Analysis: Source Dataset Attribution and Legal Implications of Synthetic Media Manipulation

• Authors: Massimiliano Cassia, Luca Guarnera, Mirko Casu, Ignazio Zangara, Sebastiano Battiato
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.11110
• Pdf URL: https://arxiv.org/pdf/2505.11110
• Copy Paste: [[2505.11110]] Deepfake Forensic Analysis: Source Dataset Attribution and Legal Implications of Synthetic Media Manipulation(https://arxiv.org/abs/2505.11110)
• Keywords: generative
• Abstract: Synthetic media generated by Generative Adversarial Networks (GANs) pose significant challenges in verifying authenticity and tracing dataset origins, raising critical concerns in copyright enforcement, privacy protection, and legal compliance. This paper introduces a novel forensic framework for identifying the training dataset (e.g., CelebA or FFHQ) of GAN-generated images through interpretable feature analysis. By integrating spectral transforms (Fourier/DCT), color distribution metrics, and local feature descriptors (SIFT), our pipeline extracts discriminative statistical signatures embedded in synthetic outputs. Supervised classifiers (Random Forest, SVM, XGBoost) achieve 98-99% accuracy in binary classification (real vs. synthetic) and multi-class dataset attribution across diverse GAN architectures (StyleGAN, AttGAN, GDWCT, StarGAN, and StyleGAN2). Experimental results highlight the dominance of frequency-domain features (DCT/FFT) in capturing dataset-specific artifacts, such as upsampling patterns and spectral irregularities, while color histograms reveal implicit regularization strategies in GAN training. We further examine legal and ethical implications, showing how dataset attribution can address copyright infringement, unauthorized use of personal data, and regulatory compliance under frameworks like GDPR and California's AB 602. Our framework advances accountability and governance in generative modeling, with applications in digital forensics, content moderation, and intellectual property litigation.
• 摘要：生成对抗网络（GAN）生成的合成媒体在验证真实性和追踪数据集起源时提出了重大挑战，在版权执行，隐私保护和法律遵守方面引起了关键的关注。本文介绍了一个新颖的法医框架，用于通过可解释的特征分析来识别GAN生成图像的训练数据集（例如Celeba或FFHQ）。通过整合光谱变换（傅立叶/DCT），颜色分布指标和局部特征描述符（SIFT），我们的管道提取了嵌入在合成输出中的区分统计特征。监督分类器（随机森林，SVM，XGBOOST）在二进制分类（实际与合成）和多级数据集属性方面的精度达到98-99％，跨不同的GAN架构（stylegan，attgan，attgan，gdwct，stargan，stargan和stylegan2）。实验结果凸显了频率域特征（DCT/FFT）在捕获数据集特异性伪像（例如上采样模式和光谱不规则之类的）中的优势，而颜色直方图在GAN训练中揭示了隐式正则化策略。我们进一步研究了法律和道德意义，展示了数据集归因如何解决版权侵权，未经授权使用个人数据以及在GDPR和加利福尼亚AB 602等框架下的法规合规性。我们的框架在生成型建模中提高了责任性和治理性，并在数字辅助工具，内容，内容，知识阶段和知识财产中的应用中应用。

Title: One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework

• Authors: Feiran Li, Qianqian Xu, Shilong Bao, Zhiyong Yang, Xiaochun Cao, Qingming Huang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.11131
• Pdf URL: https://arxiv.org/pdf/2505.11131
• Copy Paste: [[2505.11131]] One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework(https://arxiv.org/abs/2505.11131)
• Keywords: generation
• Abstract: Concept erasing has recently emerged as an effective paradigm to prevent text-to-image diffusion models from generating visually undesirable or even harmful content. However, current removal methods heavily rely on manually crafted text prompts, making it challenging to achieve a high erasure (efficacy) while minimizing the impact on other benign concepts (usability). In this paper, we attribute the limitations to the inherent gap between the text and image modalities, which makes it hard to transfer the intricately entangled concept knowledge from text prompts to the image generation process. To address this, we propose a novel solution by directly integrating visual supervision into the erasure process, introducing the first text-image Collaborative Concept Erasing (Co-Erasing) framework. Specifically, Co-Erasing describes the concept jointly by text prompts and the corresponding undesirable images induced by the prompts, and then reduces the generating probability of the target concept through negative guidance. This approach effectively bypasses the knowledge gap between text and image, significantly enhancing erasure efficacy. Additionally, we design a text-guided image concept refinement strategy that directs the model to focus on visual features most relevant to the specified text concept, minimizing disruption to other benign concepts. Finally, comprehensive experiments suggest that Co-Erasing outperforms state-of-the-art erasure approaches significantly with a better trade-off between efficacy and usability. Codes are available at this https URL.
• 摘要：概念擦除最近已成为一种有效的范式，以防止文本对图像扩散模型产生视觉上不受欢迎甚至有害内容。但是，当前的删除方法在很大程度上依赖手动制作的文本提示，这使得实现高擦除（功效）的挑战，同时最大程度地减少对其他良性概念（可用性）的影响。在本文中，我们将局限性归因于文本和图像方式之间的固有差距，这使得很难将复杂的纠缠概念知识从文本提示转移到图像生成过程。为了解决这个问题，我们通过将视觉监督直接整合到擦除过程中，提出了一个新的解决方案，并介绍了第一个文本图像协作概念擦除（配置）框架。具体而言，共同处理通过文本提示和提示引起的相应的不良图像共同描述了该概念，然后通过阴性指导降低了目标概念的生成概率。这种方法有效地绕开了文本和图像之间的知识差距，从而显着增强了擦除功效。此外，我们设计了一种文本引导的图像概念改进策略，该策略指导该模型专注于与指定文本概念最相关的视觉特征，从而最大程度地减少对其他良性概念的破坏。最后，全面的实验表明，共同处理优于最先进的擦除方法，在疗效和可用性之间取消了更好的权衡。代码可在此HTTPS URL上找到。

Title: Human-Aligned Bench: Fine-Grained Assessment of Reasoning Ability in MLLMs vs. Humans

• Authors: Yansheng Qiu, Li Xiao, Zhaopan Xu, Pengfei Zhou, Zheng Wang, Kaipeng Zhang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.11141
• Pdf URL: https://arxiv.org/pdf/2505.11141
• Copy Paste: [[2505.11141]] Human-Aligned Bench: Fine-Grained Assessment of Reasoning Ability in MLLMs vs. Humans(https://arxiv.org/abs/2505.11141)
• Keywords: generation
• Abstract: The goal of achieving Artificial General Intelligence (AGI) is to imitate humans and surpass them. Models such as OpenAI's o1, o3, and DeepSeek's R1 have demonstrated that large language models (LLMs) with human-like reasoning capabilities exhibit exceptional performance and are being gradually integrated into multimodal large language models (MLLMs). However, whether these models possess capabilities comparable to humans in handling reasoning tasks remains unclear at present. In this paper, we propose Human-Aligned Bench, a benchmark for fine-grained alignment of multimodal reasoning with human performance. Specifically, we collected 9,794 multimodal questions that solely rely on contextual reasoning, including bilingual (Chinese and English) multimodal questions and pure text-based questions, encompassing four question types: visual reasoning, definition judgment, analogical reasoning, and logical judgment. More importantly, each question is accompanied by human success rates and options that humans are prone to choosing incorrectly. Extensive experiments on the Human-Aligned Bench reveal notable differences between the performance of current MLLMs in multimodal reasoning and human performance. The findings on our benchmark provide insights into the development of the next-generation models.
• 摘要：实现人工智能（AGI）的目标是模仿人类并超越它们。诸如OpenAI的O1，O3和DeepSeek的R1之类的模型表明，具有类似人类推理能力的大型语言模型（LLM）表现出非凡的性能，并逐渐将其整合到多模式的大语言模型（MLLMS）中。但是，目前尚不清楚这些模型是否具有与人类处理推理任务相当的能力。在本文中，我们提出了与人统一的长凳，这是一种与人类绩效相处的细粒度对齐的基准。具体而言，我们收集了9,794个多模式问题，这些问题仅依赖上下文推理，包括双语（中文和英语）多模式问题和纯粹的基于文本的问题，包括四种问题类型：视觉推理，定义判断，类似的推理和逻辑判断。更重要的是，每个问题都伴随着人类的成功率和选择，而人类容易选择不正确。对人吻合的台式的广泛实验揭示了当前MLLM在多模式推理和人类绩效中的表现之间的显着差异。我们的基准测试结果提供了有关下一代模型发展的见解。

Title: CompAlign: Improving Compositional Text-to-Image Generation with a Complex Benchmark and Fine-Grained Feedback

• Authors: Yixin Wan, Kai-Wei Chang
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.11178
• Pdf URL: https://arxiv.org/pdf/2505.11178
• Copy Paste: [[2505.11178]] CompAlign: Improving Compositional Text-to-Image Generation with a Complex Benchmark and Fine-Grained Feedback(https://arxiv.org/abs/2505.11178)
• Keywords: generation
• Abstract: State-of-the-art T2I models are capable of generating high-resolution images given textual prompts. However, they still struggle with accurately depicting compositional scenes that specify multiple objects, attributes, and spatial relations. We present CompAlign, a challenging benchmark with an emphasis on assessing the depiction of 3D-spatial relationships, for evaluating and improving models on compositional image generation. CompAlign consists of 900 complex multi-subject image generation prompts that combine numerical and 3D-spatial relationships with varied attribute bindings. Our benchmark is remarkably challenging, incorporating generation tasks with 3+ generation subjects with complex 3D-spatial relationships. Additionally, we propose CompQuest, an interpretable and accurate evaluation framework that decomposes complex prompts into atomic sub-questions, then utilizes a MLLM to provide fine-grained binary feedback on the correctness of each aspect of generation elements in model-generated images. This enables precise quantification of alignment between generated images and compositional prompts. Furthermore, we propose an alignment framework that uses CompQuest's feedback as preference signals to improve diffusion models' compositional image generation abilities. Using adjustable per-image preferences, our method is easily scalable and flexible for different tasks. Evaluation of 9 T2I models reveals that: (1) models remarkable struggle more with compositional tasks with more complex 3D-spatial configurations, and (2) a noticeable performance gap exists between open-source accessible models and closed-source commercial models. Further empirical study on using CompAlign for model alignment yield promising results: post-alignment diffusion models achieve remarkable improvements in compositional accuracy, especially on complex generation tasks, outperforming previous approaches.
• 摘要：最先进的T2I模型能够生成给定文本提示的高分辨率图像。但是，他们仍然在准确地描绘了指定多个对象，属性和空间关系的构图场景方面挣扎。我们提出了Compalign，这是一个具有挑战性的基准，重点是评估3D空间关系的描述，用于评估和改进组成图像生成的模型。 compalign由900个复杂的多主体图像生成提示组成，这些提示将数值和3D空间关系与各种属性绑定组合在一起。我们的基准非常具有挑战性，将生成任务与具有复杂3D空间关系的3个以上的生成主题结合在一起。此外，我们提出了一个可解释且准确的评估框架，将复杂提示分解为原子亚问题，然后利用MLLM来提供有关模型生成图像中发电元素各个方面的正确性的细粒二进制反馈。这可以精确地量化生成的图像和组成提示之间的对齐。此外，我们提出了一个使用Compquest的反馈作为偏好信号来提高扩散模型的组成图像生成能力的偏好信号。使用可调节的每个图像偏好，我们的方法易于扩展且适用于不同的任务。对9个T2I模型的评估表明：（1）模型在具有更复杂的3D空间配置的组成任务中进行了巨大的斗争，并且（2）开源的可访问模型与封闭式商业模型之间存在明显的性能差距。进一步的实证研究对模型对准的统计结果的进一步研究产生了令人鼓舞的结果：对齐后扩散模型可在组成准确性方面取得显着提高，尤其是在复杂的生成任务上，表现优于先前的方法。

Title: DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modeling

• Authors: Yuang Ai, Qihang Fan, Xuefeng Hu, Zhenheng Yang, Ran He, Huaibo Huang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.11196
• Pdf URL: https://arxiv.org/pdf/2505.11196
• Copy Paste: [[2505.11196]] DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modeling(https://arxiv.org/abs/2505.11196)
• Keywords: generation, generative
• Abstract: Diffusion Transformer (DiT), a promising diffusion model for visual generation, demonstrates impressive performance but incurs significant computational overhead. Intriguingly, analysis of pre-trained DiT models reveals that global self-attention is often redundant, predominantly capturing local patterns-highlighting the potential for more efficient alternatives. In this paper, we revisit convolution as an alternative building block for constructing efficient and expressive diffusion models. However, naively replacing self-attention with convolution typically results in degraded performance. Our investigations attribute this performance gap to the higher channel redundancy in ConvNets compared to Transformers. To resolve this, we introduce a compact channel attention mechanism that promotes the activation of more diverse channels, thereby enhancing feature diversity. This leads to Diffusion ConvNet (DiCo), a family of diffusion models built entirely from standard ConvNet modules, offering strong generative performance with significant efficiency gains. On class-conditional ImageNet benchmarks, DiCo outperforms previous diffusion models in both image quality and generation speed. Notably, DiCo-XL achieves an FID of 2.05 at 256x256 resolution and 2.53 at 512x512, with a 2.7x and 3.1x speedup over DiT-XL/2, respectively. Furthermore, our largest model, DiCo-H, scaled to 1B parameters, reaches an FID of 1.90 on ImageNet 256x256-without any additional supervision during training. Code: this https URL.
• 摘要：扩散变压器（DIT）是一种有希望的视觉生成扩散模型，表现出令人印象深刻的性能，但造成了大量的计算开销。有趣的是，对预训练的DIT模型的分析表明，全球自我注意通常是冗余的，主要捕获局部模式，从而高出了更有效的替代方案的可能性。在本文中，我们将卷积作为构建高效和表达扩散模型的替代构建块。但是，天真地用卷积替换自我注意力通常会导致性能降解。与变压器相比，我们的调查将这一性能差距归因于Convnets中较高的通道冗余。为了解决这一问题，我们引入了一种紧凑的通道注意机制，该机制促进了更多不同的渠道的激活，从而增强了特征多样性。这导致扩散Convnet（DICO），这是一个完全由标准Convnet模块构建的扩散模型家族，提供了具有效率显着提高的强生成性能。在类条件成像网基准上，DICO在图像质量和发电速度方面都优于先前的扩散模型。值得注意的是，DICO-XL在256x256分辨率下的FID为2.05，在512x512分别达到2.53，分别在DIT-XL/2上具有2.7倍和3.1倍的速度。此外，我们最大的模型DICO-H缩放到1B参数，在Imagenet 256x256上达到1.90的FID，而在训练过程中，没有任何其他监督。代码：此HTTPS URL。

Title: Minimizing False-Positive Attributions in Explanations of Non-Linear Models

• Authors: Anders Gjølbye, Stefan Haufe, Lars Kai Hansen
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.11210
• Pdf URL: https://arxiv.org/pdf/2505.11210
• Copy Paste: [[2505.11210]] Minimizing False-Positive Attributions in Explanations of Non-Linear Models(https://arxiv.org/abs/2505.11210)
• Keywords: generative
• Abstract: Suppressor variables can influence model predictions without being dependent on the target outcome and they pose a significant challenge for Explainable AI (XAI) methods. These variables may cause false-positive feature attributions, undermining the utility of explanations. Although effective remedies exist for linear models, their extension to non-linear models and to instance-based explanations has remained limited. We introduce PatternLocal, a novel XAI technique that addresses this gap. PatternLocal begins with a locally linear surrogate, e.g. LIME, KernelSHAP, or gradient-based methods, and transforms the resulting discriminative model weights into a generative representation, thereby suppressing the influence of suppressor variables while preserving local fidelity. In extensive hyperparameter optimization on the XAI-TRIS benchmark, PatternLocal consistently outperformed other XAI methods and reduced false-positive attributions when explaining non-linear tasks, thereby enabling more reliable and actionable insights.
• 摘要：抑制变量可以影响模型预测，而不会依赖目标结果，并且它们对可解释的AI（XAI）方法构成了重大挑战。这些变量可能会导致虚假阳性特征归因，从而破坏解释的效用。尽管线性模型存在有效的补救措施，但它们扩展到非线性模型和基于实例的解释仍然有限。我们介绍了PatternLocal，这是一种解决这一差距的新型XAI技术。 patternlocal始于局部线性替代物，例如石灰，内核变形或基于梯度的方法，并将所得的判别模型权重转换为生成表示，从而抑制了抑制变量的影响，同时保留了局部保真度。在对XAI-Tris基准测试的广泛超参数优化中，模式插座在解释非线性任务时始终优于其他XAI方法，并减少了假阳性属性，从而实现了更可靠且可行的见解。

Title: Massive-STEPS: Massive Semantic Trajectories for Understanding POI Check-ins -- Dataset and Benchmarks

• Authors: Wilson Wongso, Hao Xue, Flora D. Salim
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.11239
• Pdf URL: https://arxiv.org/pdf/2505.11239
• Copy Paste: [[2505.11239]] Massive-STEPS: Massive Semantic Trajectories for Understanding POI Check-ins -- Dataset and Benchmarks(https://arxiv.org/abs/2505.11239)
• Keywords: generative
• Abstract: Understanding human mobility through Point-of-Interest (POI) recommendation is increasingly important for applications such as urban planning, personalized services, and generative agent simulation. However, progress in this field is hindered by two key challenges: the over-reliance on older datasets from 2012-2013 and the lack of reproducible, city-level check-in datasets that reflect diverse global regions. To address these gaps, we present Massive-STEPS (Massive Semantic Trajectories for Understanding POI Check-ins), a large-scale, publicly available benchmark dataset built upon the Semantic Trails dataset and enriched with semantic POI metadata. Massive-STEPS spans 12 geographically and culturally diverse cities and features more recent (2017-2018) and longer-duration (24 months) check-in data than prior datasets. We benchmarked a wide range of POI recommendation models on Massive-STEPS using both supervised and zero-shot approaches, and evaluated their performance across multiple urban contexts. By releasing Massive-STEPS, we aim to facilitate reproducible and equitable research in human mobility and POI recommendation. The dataset and benchmarking code are available at: this https URL
• 摘要：通过利益点（POI）推荐了解人类流动性对于城市规划，个性化服务和生成代理模拟等应用的越来越重要。但是，这一领域的进展受到了两个主要挑战的阻碍：2012 - 2013年对较旧数据集的过度依赖和缺乏可重现的城市级入住数据集，反映了各种全球地区。为了解决这些差距，我们介绍了巨大的步骤（用于理解POI检查的大规模语义轨迹），这是一种建立在语义跟踪数据集的大规模公开的基准数据集，并具有语义上的poi metadata。与先前的数据集中相比，大型步骤在地理和文化上不同的城市跨越12个城市，并具有更新的（2017-2018）和更长的（24个月）登记数据。我们使用监督和零拍方法对大规模步骤进行了广泛的POI推荐模型，并评估了它们在多个城市环境中的性能。通过释放大规模的步骤，我们旨在促进人类流动性和POI建议中可再现和公平的研究。数据集和基准代码可在以下网址提供：此HTTPS URL

Title: Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models

• Authors: Fu-Yun Wang, Yunhao Shui, Jingtan Piao, Keqiang Sun, Hongsheng Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.11245
• Pdf URL: https://arxiv.org/pdf/2505.11245
• Copy Paste: [[2505.11245]] Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models(https://arxiv.org/abs/2505.11245)
• Keywords: generation
• Abstract: Diffusion models have made substantial advances in image generation, yet models trained on large, unfiltered datasets often yield outputs misaligned with human preferences. Numerous methods have been proposed to fine-tune pre-trained diffusion models, achieving notable improvements in aligning generated outputs with human preferences. However, we argue that existing preference alignment methods neglect the critical role of handling unconditional/negative-conditional outputs, leading to a diminished capacity to avoid generating undesirable outcomes. This oversight limits the efficacy of classifier-free guidance~(CFG), which relies on the contrast between conditional generation and unconditional/negative-conditional generation to optimize output quality. In response, we propose a straightforward but versatile effective approach that involves training a model specifically attuned to negative preferences. This method does not require new training strategies or datasets but rather involves minor modifications to existing techniques. Our approach integrates seamlessly with models such as SD1.5, SDXL, video diffusion models and models that have undergone preference optimization, consistently enhancing their alignment with human preferences.
• 摘要：扩散模型在图像产生方面取得了重大进展，但是在大型未经过滤数据集上训练的模型通常会产生与人类偏好未对准的产量。已经提出了许多方法来微调预训练的扩散模型，从而在与人类偏好的对齐产量的对齐产量方面取得了显着改善。但是，我们认为现有的偏好一致性方法忽略了处理无条件/负有条件输出的关键作用，从而导致避免产生不良结果的能力降低。这种疏忽限制了无分类器指导〜（CFG）的功效，该指导〜（CFG）依赖于条件生成与无条件/无条件/负有生成之间的对比来优化输出质量。作为回应，我们提出了一种直接但多才多艺的有效方法，涉及训练专门针对负面偏好的模型。此方法不需要新的培训策略或数据集，而是涉及对现有技术的微小修改。我们的方法与SD1.5，SDXL，视频扩散模型和具有偏好优化的模型等模型无缝集成，并始终增强其与人类偏好的一致性。

Title: DRAGON: A Large-Scale Dataset of Realistic Images Generated by Diffusion Models

• Authors: Giulia Bertazzini, Daniele Baracchi, Dasara Shullani, Isao Echizen, Alessandro Piva
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2505.11257
• Pdf URL: https://arxiv.org/pdf/2505.11257
• Copy Paste: [[2505.11257]] DRAGON: A Large-Scale Dataset of Realistic Images Generated by Diffusion Models(https://arxiv.org/abs/2505.11257)
• Keywords: generation
• Abstract: The remarkable ease of use of diffusion models for image generation has led to a proliferation of synthetic content online. While these models are often employed for legitimate purposes, they are also used to generate fake images that support misinformation and hate speech. Consequently, it is crucial to develop robust tools capable of detecting whether an image has been generated by such models. Many current detection methods, however, require large volumes of sample images for training. Unfortunately, due to the rapid evolution of the field, existing datasets often cover only a limited range of models and quickly become outdated. In this work, we introduce DRAGON, a comprehensive dataset comprising images from 25 diffusion models, spanning both recent advancements and older, well-established architectures. The dataset contains a broad variety of images representing diverse subjects. To enhance image realism, we propose a simple yet effective pipeline that leverages a large language model to expand input prompts, thereby generating more diverse and higher-quality outputs, as evidenced by improvements in standard quality metrics. The dataset is provided in multiple sizes (ranging from extra-small to extra-large) to accomodate different research scenarios. DRAGON is designed to support the forensic community in developing and evaluating detection and attribution techniques for synthetic content. Additionally, the dataset is accompanied by a dedicated test set, intended to serve as a benchmark for assessing the performance of newly developed methods.
• 摘要：扩散模型的显着易于使用，导致在线合成内容的扩散。尽管这些模型通常用于合法目的，但它们也用于生成支持错误信息和仇恨言论的假图像。因此，开发能够检测出这种模型是否已生成图像的强大工具至关重要。但是，许多当前的检测方法需要大量的样本图像进行训练。不幸的是，由于该领域的快速发展，现有数据集通常仅涵盖有限的模型，并迅速过时。在这项工作中，我们介绍了Dragon，这是一个全面的数据集，其中包括来自25种扩散模型的图像，涵盖了最近的进步和较旧的，建立良好的体系结构。该数据集包含代表不同主题的各种图像。为了增强图像现实主义，我们提出了一条简单而有效的管道，该管道利用大型语言模型扩展输入提示，从而产生更多多样化和更高质量的输出，这可以改善标准质量指标。该数据集以多种尺寸（从超小型到超大型）提供多种尺寸，以适应不同的研究方案。 Dragon旨在支持法医社区开发和评估合成内容的检测和归因技术。此外，数据集伴随着专用的测试集，旨在作为评估新开发方法的性能的基准。

Title: Multi-view dense image matching with similarity learning and geometry priors

• Authors: Mohamed Ali Chebbi, Ewelina Rupnik, Paul Lopes, Marc Pierrot-Deseilligny
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.11264
• Pdf URL: https://arxiv.org/pdf/2505.11264
• Copy Paste: [[2505.11264]] Multi-view dense image matching with similarity learning and geometry priors(https://arxiv.org/abs/2505.11264)
• Keywords: generation
• Abstract: We introduce MV-DeepSimNets, a comprehensive suite of deep neural networks designed for multi-view similarity learning, leveraging epipolar geometry for training. Our approach incorporates an online geometry prior to characterize pixel relationships, either along the epipolar line or through homography rectification. This enables the generation of geometry-aware features from native images, which are then projected across candidate depth hypotheses using plane sweeping. Our method geometric preconditioning effectively adapts epipolar-based features for enhanced multi-view reconstruction, without requiring the laborious multi-view training dataset creation. By aggregating learned similarities, we construct and regularize the cost volume, leading to improved multi-view surface reconstruction over traditional dense matching approaches. MV-DeepSimNets demonstrates superior performance against leading similarity learning networks and end-to-end regression models, especially in terms of generalization capabilities across both aerial and satellite imagery with varied ground sampling distances. Our pipeline is integrated into MicMac software and can be readily adopted in standard multi-resolution image matching pipelines.
• 摘要：我们介绍了MV-Deepsimnets，这是一套全面的深神经网络套件，旨在多视图相似性学习，利用Epolaral几何进行训练。我们的方法在表征像素关系之前，沿着外星线或通过同构晶整流，在线几何形状结合了在线几何形状。这使本机图像产生了几何感知特征，然后使用平面扫地将其投影到候选深度假设上。我们的方法几何预处理有效地调整了基于表现的特征，以增强多视图重建，而无需努力的多视图训练数据集创建。通过汇总学习相似之处，我们构建和正规化成本量，从而改善了传统密集匹配方法的多视图表面​​重建。 MV-Deepsimnets在领先的相似性学习网络和端到端回归模型上表现出了出色的性能，尤其是在空中和卫星图像的概括能力方面，地面采样距离各不相同。我们的管道已集成到MICMAC软件中，并且可以在标准的多分辨率图像匹配管道中很容易采用。

Title: Equal is Not Always Fair: A New Perspective on Hyperspectral Representation Non-Uniformity

• Authors: Wuzhou Quan, Mingqiang Wei, Jinhui Tang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.11267
• Pdf URL: https://arxiv.org/pdf/2505.11267
• Copy Paste: [[2505.11267]] Equal is Not Always Fair: A New Perspective on Hyperspectral Representation Non-Uniformity(https://arxiv.org/abs/2505.11267)
• Keywords: super-resolution
• Abstract: Hyperspectral image (HSI) representation is fundamentally challenged by pervasive non-uniformity, where spectral dependencies, spatial continuity, and feature efficiency exhibit complex and often conflicting behaviors. Most existing models rely on a unified processing paradigm that assumes homogeneity across dimensions, leading to suboptimal performance and biased representations. To address this, we propose FairHyp, a fairness-directed framework that explicitly disentangles and resolves the threefold non-uniformity through cooperative yet specialized modules. We introduce a Runge-Kutta-inspired spatial variability adapter to restore spatial coherence under resolution discrepancies, a multi-receptive field convolution module with sparse-aware refinement to enhance discriminative features while respecting inherent sparsity, and a spectral-context state space model that captures stable and long-range spectral dependencies via bidirectional Mamba scanning and statistical aggregation. Unlike one-size-fits-all solutions, FairHyp achieves dimension-specific adaptation while preserving global consistency and mutual reinforcement. This design is grounded in the view that non-uniformity arises from the intrinsic structure of HSI representations, rather than any particular task setting. To validate this, we apply FairHyp across four representative tasks including classification, denoising, super-resolution, and inpaintin, demonstrating its effectiveness in modeling a shared structural flaw. Extensive experiments show that FairHyp consistently outperforms state-of-the-art methods under varied imaging conditions. Our findings redefine fairness as a structural necessity in HSI modeling and offer a new paradigm for balancing adaptability, efficiency, and fidelity in high-dimensional vision tasks.
• 摘要：高光谱图像（HSI）表示从根本上面临着普遍的非均匀性的挑战，那里的光谱依赖性，空间连续性和特征效率表现出复杂且通常是冲突的行为。大多数现有模型都依赖于统一的处理范式，该范式在跨维度上假定同质性，从而导致次优性能和偏见表示。为了解决这个问题，我们提出了Fairhyp，这是一个公平指导的框架，该框架通过合作但专业的模块明确地解开并解决了三倍的非均匀性。 We introduce a Runge-Kutta-inspired spatial variability adapter to restore spatial coherence under resolution discrepancies, a multi-receptive field convolution module with sparse-aware refinement to enhance discriminative features while respecting inherent sparsity, and a spectral-context state space model that captures stable and long-range spectral dependencies via bidirectional Mamba scanning and statistical聚合。与单一适合的解决方案不同，FairHYP可以实现特定于维度的适应性，同时保留了全球一致性和相互的增强。这种设计的基础是，不均匀源于HSI表示的内在结构，而不是任何特定的任务设置。为了验证这一点，我们在四个代表性任务中采用FairHyp，包括分类，去核，超分辨率和paintin，证明了其在建模共享结构缺陷中的有效性。广泛的实验表明，在各种成像条件下，FairHyp始终优于最先进的方法。我们的发现将公平性重新定义为HSI建模中的结构性必需品，并为在高维视觉任务中平衡适应性，效率和忠诚度提供了新的范式。

Title: CROC: Evaluating and Training T2I Metrics with Pseudo- and Human-Labeled Contrastive Robustness Checks

• Authors: Christoph Leiter, Yuki M. Asano, Margret Keuper, Steffen Eger
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2505.11314
• Pdf URL: https://arxiv.org/pdf/2505.11314
• Copy Paste: [[2505.11314]] CROC: Evaluating and Training T2I Metrics with Pseudo- and Human-Labeled Contrastive Robustness Checks(https://arxiv.org/abs/2505.11314)
• Keywords: generation
• Abstract: The assessment of evaluation metrics (meta-evaluation) is crucial for determining the suitability of existing metrics in text-to-image (T2I) generation tasks. Human-based meta-evaluation is costly and time-intensive, and automated alternatives are scarce. We address this gap and propose CROC: a scalable framework for automated Contrastive Robustness Checks that systematically probes and quantifies metric robustness by synthesizing contrastive test cases across a comprehensive taxonomy of image properties. With CROC, we generate a pseudo-labeled dataset (CROC$^{syn}$) of over one million contrastive prompt-image pairs to enable a fine-grained comparison of evaluation metrics. We also use the dataset to train CROCScore, a new metric that achieves state-of-the-art performance among open-source methods, demonstrating an additional key application of our framework. To complement this dataset, we introduce a human-supervised benchmark (CROC$^{hum}$) targeting especially challenging categories. Our results highlight robustness issues in existing metrics: for example, many fail on prompts involving negation, and all tested open-source metrics fail on at least 25% of cases involving correct identification of body parts.
• 摘要：评估指标的评估（元评估）对于确定现有指标在文本到图像（T2I）生成任务中的适用性至关重要。基于人类的元评估是昂贵且耗时的，并且自动化替代方案稀缺。我们解决了这一差距并提出CROC：自动对比鲁棒性检查的可扩展框架，该框架通过在图像属性的全面分类法中综合对比测试用例来系统地探测和量化度量鲁棒性。使用CROC，我们生成一个超过一百万个对比度及时图像对的伪标记的数据集（Croc $^{Syn} $），以实现评估指标的细粒度比较。我们还使用数据集来训练Crocscore，这是一种新的指标，可以在开源方法之间实现最先进的性能，从而证明了我们框架的附加关键应用程序。为了补充该数据集，我们介绍了针对特别具有挑战性的类别的人类监督基准（CROC $^{HUM} $）。我们的结果突出了现有指标中的鲁棒性问题：例如，许多涉及否定的提示失败，并且所有经过测试的开源指标在至少25％的案件中失败了，涉及正确识别身体部位的情况。

Title: Temporally-Grounded Language Generation: A Benchmark for Real-Time Vision-Language Models

• Authors: Keunwoo Peter Yu, Joyce Chai
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2505.11326
• Pdf URL: https://arxiv.org/pdf/2505.11326
• Copy Paste: [[2505.11326]] Temporally-Grounded Language Generation: A Benchmark for Real-Time Vision-Language Models(https://arxiv.org/abs/2505.11326)
• Keywords: generation
• Abstract: Vision-language models (VLMs) have shown remarkable progress in offline tasks such as image captioning and video question answering. However, real-time interactive environments impose new demands on VLMs, requiring them to generate utterances that are not only semantically accurate but also precisely timed. We identify two core capabilities necessary for such settings -- $\textit{perceptual updating}$ and $\textit{contingency awareness}$ -- and propose a new benchmark task, $\textbf{Temporally-Grounded Language Generation (TGLG)}$, to evaluate them. TGLG requires models to generate utterances in response to streaming video such that both content and timing align with dynamic visual input. To support this benchmark, we curate evaluation datasets from sports broadcasting and egocentric human interaction domains, and introduce a new metric, $\textbf{TRACE}$, to evaluate TGLG by jointly measuring semantic similarity and temporal alignment. Finally, we present $\textbf{Vision-Language Model with Time-Synchronized Interleaving (VLM-TSI)}$, a model that interleaves visual and linguistic tokens in a time-synchronized manner, enabling real-time language generation without relying on turn-based assumptions. Experimental results show that VLM-TSI significantly outperforms a strong baseline, yet overall performance remains modest -- highlighting the difficulty of TGLG and motivating further research in real-time VLMs. Code and data available $\href{this https URL}{here}$.
• 摘要：视觉语言模型（VLM）在离线任务（例如图像字幕和视频问题回答）中显示出了显着的进展。但是，实时交互式环境对VLM施加了新的需求，要求它们产生不仅在语义上精确而且还准确的话语。我们确定了此类设置所需的两个核心功能 - $ \ textIt {感知更新} $和$ \ textit {artingency Interenty} $ - 并提出了一个新的基准任务，$ \ textbf {permutally-formatenly-grounded语言生成（tglg）} $，以评估它们。 TGLG需要模型来响应流视频来生成话语，以使内容和正时与动态视觉输入保持一致。为了支持此基准，我们从体育广播和以人类的人类交互域进行策划评估数据集，并引入新的度量标准，$ \ textbf {trace} $，通过共同测量语义相似性和时间对齐来评估TGLG。最后，我们提出了$ \ textbf {视觉语言模型，具有时间同步交错（VLM-TSI）} $，该模型以时间同步的方式交织了视觉和语言令牌，以实时的语言生成而无需依靠基于转弯的假设。实验结果表明，VLM-TSI明显胜过强大的基线，但总体表现仍然适中 - 强调了TGLG的难度并激发了实时VLM的进一步研究。代码和数据可用$ \ href {this HTTPS url} {there} $。

Title: MARRS: Masked Autoregressive Unit-based Reaction Synthesis

• Authors: Y.B. Wang, S Wang, J.N. Zhang, J.F. Wu, Q.D. He, C.C. Fu, C.J. Wang, Y. Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.11334
• Pdf URL: https://arxiv.org/pdf/2505.11334
• Copy Paste: [[2505.11334]] MARRS: Masked Autoregressive Unit-based Reaction Synthesis(https://arxiv.org/abs/2505.11334)
• Keywords: generation
• Abstract: This work aims at a challenging task: human action-reaction synthesis, i.e., generating human reactions based on the action sequence of the other as conditions. Currently, autoregressive modeling approaches have achieved remarkable performance in motion generation tasks, e.g. text-to-motion. However, vector quantization (VQ) accompanying autoregressive generation has inherent disadvantages, including loss of quantization information, low codebook utilization, etc. Moreover, unlike text-to-motion, which focuses solely on the movement of body joints, human action-reaction synthesis also encompasses fine-grained hand movements. In this work, we propose MARRS, a novel framework designed to generate coordinated and fine-grained reaction motions in continuous representations. Initially, we present the Unit-distinguished Motion Variational AutoEncoder (UD-VAE), which segments the entire body into distinct body and hand units, encoding them independently. Subsequently, we propose Action-Conditioned Fusion (ACF), which involves randomly masking a subset of reactive tokens and extracting specific information about the body and hands from the active tokens. Furthermore, we introduce Adaptive Unit Modulation (AUM) to facilitate interaction between body and hand units by using the information from one unit to adaptively modulate the other. Finally, for the diffusion model, we employ a compact MLP as a noise predictor for each distinct body unit and incorporate the diffusion loss to model the probability distribution of each token. Quantitative and qualitative results demonstrate that our method achieves superior performance. The code will be released upon acceptance.
• 摘要：这项工作的目的是涉及一项具有挑战性的任务：人类的行动反应综合，即基于对方作为条件的动作顺序产生人类反应。当前，自回归建模方法在运动生成任务中取得了显着的性能，例如文本到动作。然而，随附自回归产生的矢量量化（VQ）具有固有的缺点，包括量化信息的丢失，低密码书的利用等。此外，与文本到动作不同，这仅关注人体关节的运动，人类的动作反应合成还包括精细的手工运动。在这项工作中，我们提出了Marrs，这是一个新颖的框架，旨在在连续表示中产生协调和细粒的反应运动。最初，我们介绍了单位分化的运动变异自动编码器（UD-VAE），该动态将整个身体分为不同的身体和手部单位，并独立编码它们。随后，我们提出了动作条件的融合（ACF），涉及随机掩盖反应性令牌的子集并从活动令牌中提取有关身体和手的特定信息。此外，我们通过使用一个单元的信息自适应调节另一个单位的信息来介绍自适应单元调制（AUM），以促进身体和手部单位之间的相互作用。最后，对于扩散模型，我们使用紧凑的MLP作为每个不同身体单位的噪声预测指标，并结合了扩散损失，以模拟每个令牌的概率分布。定量和定性结果表明，我们的方法达到了卓越的性能。该代码将在接受后发布。

Title: LGBQPC: Local Granular-Ball Quality Peaks Clustering

• Authors: Zihang Jia, Zhen Zhang, Witold Pedrycz
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.11359
• Pdf URL: https://arxiv.org/pdf/2505.11359
• Copy Paste: [[2505.11359]] LGBQPC: Local Granular-Ball Quality Peaks Clustering(https://arxiv.org/abs/2505.11359)
• Keywords: generation
• Abstract: The density peaks clustering (DPC) algorithm has attracted considerable attention for its ability to detect arbitrarily shaped clusters based on a simple yet effective assumption. Recent advancements integrating granular-ball (GB) computing with DPC have led to the GB-based DPC (GBDPC) algorithm, which improves computational efficiency. However, GBDPC demonstrates limitations when handling complex clustering tasks, particularly those involving data with complex manifold structures or non-uniform density distributions. To overcome these challenges, this paper proposes the local GB quality peaks clustering (LGBQPC) algorithm, which offers comprehensive improvements to GBDPC in both GB generation and clustering processes based on the principle of justifiable granularity (POJG). Firstly, an improved GB generation method, termed GB-POJG+, is developed, which systematically refines the original GB-POJG in four key aspects: the objective function, termination criterion for GB division, definition of abnormal GB, and granularity level adaptation strategy. GB-POJG+ simplifies parameter configuration by requiring only a single penalty coefficient and ensures high-quality GB generation while maintaining the number of generated GBs within an acceptable range. In the clustering phase, two key innovations are introduced based on the GB k-nearest neighbor graph: relative GB quality for density estimation and geodesic distance for GB distance metric. These modifications substantially improve the performance of GBDPC on datasets with complex manifold structures or non-uniform density distributions. Extensive numerical experiments on 40 benchmark datasets, including both synthetic and publicly available datasets, validate the superior performance of the proposed LGBQPC algorithm.
• 摘要：密度峰聚类（DPC）算法因其基于简单但有效的假设检测任意形状的簇而引起了极大的关注。将粒状球（GB）计算与DPC相结合的最新进展已导致基于GB的DPC（GBDPC）算法，从而提高了计算效率。但是，GBDPC在处理复杂的聚类任务时表明了局限性，尤其是那些涉及具有复杂歧管结构或非均匀密度分布的数据的任务。为了克服这些挑战，本文提出了本地GB质量峰值聚类（LGBQPC）算法，该算法基于合理的粒度（POJG）的原理，在GB生成和聚类过程中对GBDPC进行了全面改进。首先，开发了一种改进的GB生成方法，称为GB-POJG+，该方法在四个关键方面系统地完善了原始的GB-POJG：目标函数，GB分裂的终止标准，异常GB的定义和粒度级别适应策略。 GB-POJG+仅需要一个惩罚系数来简化参数配置，并确保高质量的GB生成，同时将生成的GB的数量保持在可接受的范围内。在聚类阶段，基于GB K-Neartible图形：GB密度估计的相对GB质量和GB距离指标的地球距离引入了两个关键创新。这些修改基本上改善了具有复杂的歧管结构或非均匀密度分布的数据集上GBDPC的性能。在40个基准数据集上进行的大量数值实验，包括合成数据集和公开数据集，验证了所提出的LGBQPC算法的出色性能。

Title: Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach

• Authors: Rares Cristian, Pavithra Harsha, Georgia Perakis, Brian Quanz
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2505.11360
• Pdf URL: https://arxiv.org/pdf/2505.11360
• Copy Paste: [[2505.11360]] Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach(https://arxiv.org/abs/2505.11360)
• Keywords: generation
• Abstract: End-to-end learning has become a widely applicable and studied problem in training predictive ML models to be aware of their impact on downstream decision-making tasks. These end-to-end models often outperform traditional methods that separate training from the optimization and only myopically focus on prediction error. However, the computational complexity of end-to-end frameworks poses a significant challenge, particularly for large-scale problems. While training an ML model using gradient descent, each time we need to compute a gradient we must solve an expensive optimization problem. We present a meta-optimization method that learns efficient algorithms to approximate optimization problems, dramatically reducing computational overhead of solving the decision problem in general, an aspect we leverage in the training within the end-to-end framework. Our approach introduces a neural network architecture that near-optimally solves optimization problems while ensuring feasibility constraints through alternate projections. We prove exponential convergence, approximation guarantees, and generalization bounds for our learning method. This method offers superior computational efficiency, producing high-quality approximations faster and scaling better with problem size compared to existing techniques. Our approach applies to a wide range of optimization problems including deterministic, single-stage as well as two-stage stochastic optimization problems. We illustrate how our proposed method applies to (1) an electricity generation problem using real data from an electricity routing company coordinating the movement of electricity throughout 13 states, (2) a shortest path problem with a computer vision task of predicting edge costs from terrain maps, (3) a two-stage multi-warehouse cross-fulfillment newsvendor problem, as well as a variety of other newsvendor-like problems.
• 摘要：端到端的学习已成为培训预测ML模型中的广泛适用和研究的问题，以了解其对下游决策任务的影响。这些端到端模型通常优于传统方法，这些方法将训练与优化分开，只有近距离专注于预测错误。但是，端到端框架的计算复杂性提出了重大挑战，尤其是对于大规模问题。在使用梯度下降训练ML模型时，每次我们需要计算梯度时，我们都必须解决昂贵的优化问题。我们提出了一种元优化方法，该方法将学习有效的算法以近似优化问题，从而大大降低了解决决策问题的计算开销，这是我们在端到端框架内的培训中利用这一方面的方面。我们的方法介绍了一种神经网络体系结构，该架构几乎可以很好地解决优化问题，同时通过替代预测确保可行性约束。我们证明了我们的学习方法的指数收敛，近似保证和概括界限。与现有技术相比，该方法具有卓越的计算效率，可以更快地产生高质量的近似值，并且随着问题的规模而缩放得更好。我们的方法适用于广泛的优化问题，包括确定性，单阶段和两阶段随机优化问题。我们说明了我们提出的方法如何适用于（1）使用电力路由公司的真实数据协调电力在整个13个州的流动，（2）最短的路径问题与计算机视觉任务的最短路径问题，（3）两级多工具屋交叉填充新闻报道问题，以及其他新闻杂货，以及其他问题。

Title: Face Consistency Benchmark for GenAI Video

• Authors: Michal Podstawski, Malgorzata Kudelska, Haohong Wang
• Subjects: cs.CV, cs.MM
• Abstract URL: https://arxiv.org/abs/2505.11425
• Pdf URL: https://arxiv.org/pdf/2505.11425
• Copy Paste: [[2505.11425]] Face Consistency Benchmark for GenAI Video(https://arxiv.org/abs/2505.11425)
• Keywords: generation
• Abstract: Video generation driven by artificial intelligence has advanced significantly, enabling the creation of dynamic and realistic content. However, maintaining character consistency across video sequences remains a major challenge, with current models struggling to ensure coherence in appearance and attributes. This paper introduces the Face Consistency Benchmark (FCB), a framework for evaluating and comparing the consistency of characters in AI-generated videos. By providing standardized metrics, the benchmark highlights gaps in existing solutions and promotes the development of more reliable approaches. This work represents a crucial step toward improving character consistency in AI video generation technologies.
• 摘要：由人工智能驱动的视频生成已经大大提高，从而创建了动态和现实的内容。但是，在视频序列中保持角色一致性仍然是一个重大挑战，当前模型正在努力确保外观和属性连贯性。本文介绍了面部一致性基准（FCB），该基准是评估和比较AI生成视频中字符一致性的框架。通过提供标准化的指标，该基准强调了现有解决方案中的差距，并促进了更可靠的方法的发展。这项工作代表了提高AI视频生成技术中角色一致性的关键步骤。

Title: A Generative Framework for Causal Estimation via Importance-Weighted Diffusion Distillation

• Authors: Xinran Song, Tianyu Chen, Mingyuan Zhou
• Subjects: cs.LG, stat.AP, stat.ME, stat.ML
• Abstract URL: https://arxiv.org/abs/2505.11444
• Pdf URL: https://arxiv.org/pdf/2505.11444
• Copy Paste: [[2505.11444]] A Generative Framework for Causal Estimation via Importance-Weighted Diffusion Distillation(https://arxiv.org/abs/2505.11444)
• Keywords: generative
• Abstract: Estimating individualized treatment effects from observational data is a central challenge in causal inference, largely due to covariate imbalance and confounding bias from non-randomized treatment assignment. While inverse probability weighting (IPW) is a well-established solution to this problem, its integration into modern deep learning frameworks remains limited. In this work, we propose Importance-Weighted Diffusion Distillation (IWDD), a novel generative framework that combines the pretraining of diffusion models with importance-weighted score distillation to enable accurate and fast causal estimation-including potential outcome prediction and treatment effect estimation. We demonstrate how IPW can be naturally incorporated into the distillation of pretrained diffusion models, and further introduce a randomization-based adjustment that eliminates the need to compute IPW explicitly-thereby simplifying computation and, more importantly, provably reducing the variance of gradient estimates. Empirical results show that IWDD achieves state-of-the-art out-of-sample prediction performance, with the highest win rates compared to other baselines, significantly improving causal estimation and supporting the development of individualized treatment strategies. We will release our PyTorch code for reproducibility and future research.
• 摘要：从观察数据中估算个性化的治疗效果是因果推断的核心挑战，这在很大程度上是由于协变量失衡和非随机治疗分配的混淆偏见。尽管反概率加权（IPW）是解决此问题的一个完善的解决方案，但其整合到现代深度学习框架中仍然有限。在这项工作中，我们提出了重要的加权扩散蒸馏（IWDD），这是一种新型的生成框架，将扩散模型的预处理与重要性加权分数蒸馏相结合，以实现准确且快速的因果估计，包括潜在的预测和治疗效果估计。我们证明了如何将IPW自然纳入预验证的扩散模型的蒸馏中，并进一步引入了基于随机的调整，以显式地计算IPW的需求，从而简化计算，更重要的是，更重要的是，可以降低毕业估计的方差。经验结果表明，IWDD实现了最新的样本外预测性能，与其他基准相比，获胜率最高，从而显着改善了因果估计并支持个性化治疗策略的发展。我们将发布我们的Pytorch代码，以供可重复性和未来的研究。

Title: PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignment

• Authors: Dingbang Huang, Wenbo Li, Yifei Zhao, Xinyu Pan, Yanhong Zeng, Bo Dai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.11468
• Pdf URL: https://arxiv.org/pdf/2505.11468
• Copy Paste: [[2505.11468]] PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignment(https://arxiv.org/abs/2505.11468)
• Keywords: generation
• Abstract: Diffusion models have made remarkable advancements in generating high-quality images from textual descriptions. Recent works like LayerDiffuse have extended the previous single-layer, unified image generation paradigm to transparent image layer generation. However, existing multi-layer generation methods fail to handle the interactions among multiple layers such as rational global layout, physics-plausible contacts and visual effects like shadows and reflections while maintaining high alpha quality. To solve this problem, we propose PSDiffusion, a unified diffusion framework for simultaneous multi-layer text-to-image generation. Our model can automatically generate multi-layer images with one RGB background and multiple RGBA foregrounds through a single feed-forward process. Unlike existing methods that combine multiple tools for post-decomposition or generate layers sequentially and separately, our method introduces a global-layer interactive mechanism that generates layered-images concurrently and collaboratively, ensuring not only high quality and completeness for each layer, but also spatial and visual interactions among layers for global coherence.
• 摘要：扩散模型在从文本描述中生成高质量的图像方面取得了显着的进步。诸如LayerDiffuse之类的最新作品已将以前的单层图像生成范式扩展到透明的图像层生成。但是，现有的多层生成方法无法处理多个层之间的相互作用，例如理性的全球布局，物理学知识触点以及视觉效果，例如阴影和反射，同时保持高α质量。为了解决这个问题，我们提出了PSDiffusion，这是同时多层文本对图像生成的统一扩散框架。我们的模型可以通过单个馈送过程自动生成具有一个RGB背景和多个RGBA前景的多层图像。与现有的方法结合了多种工具以依次和单独地生成层，我们的方法引入了一种全球层交互式机制，该机制同时且协作生成分层图像，不仅确保每层的高质量和完整性，而且还确保了全球一致性的层之间的空间和视觉交互。

Title: QVGen: Pushing the Limit of Quantized Video Generative Models

• Authors: Yushi Huang, Ruihao Gong, Jing Liu, Yifu Ding, Chengtao Lv, Haotong Qin, Jun Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2505.11497
• Pdf URL: https://arxiv.org/pdf/2505.11497
• Copy Paste: [[2505.11497]] QVGen: Pushing the Limit of Quantized Video Generative Models(https://arxiv.org/abs/2505.11497)
• Keywords: generative
• Abstract: Video diffusion models (DMs) have enabled high-quality video synthesis. Yet, their substantial computational and memory demands pose serious challenges to real-world deployment, even on high-end GPUs. As a commonly adopted solution, quantization has proven notable success in reducing cost for image DMs, while its direct application to video DMs remains ineffective. In this paper, we present QVGen, a novel quantization-aware training (QAT) framework tailored for high-performance and inference-efficient video DMs under extremely low-bit quantization (e.g., 4-bit or below). We begin with a theoretical analysis demonstrating that reducing the gradient norm is essential to facilitate convergence for QAT. To this end, we introduce auxiliary modules ($\Phi$) to mitigate large quantization errors, leading to significantly enhanced convergence. To eliminate the inference overhead of $\Phi$, we propose a rank-decay strategy that progressively eliminates $\Phi$. Specifically, we repeatedly employ singular value decomposition (SVD) and a proposed rank-based regularization $\mathbf{\gamma}$ to identify and decay low-contributing components. This strategy retains performance while zeroing out inference overhead. Extensive experiments across $4$ state-of-the-art (SOTA) video DMs, with parameter sizes ranging from $1.3$B $\sim14$B, show that QVGen is the first to reach full-precision comparable quality under 4-bit settings. Moreover, it significantly outperforms existing methods. For instance, our 3-bit CogVideoX-2B achieves improvements of $+25.28$ in Dynamic Degree and $+8.43$ in Scene Consistency on VBench.
• 摘要：视频扩散模型（DMS）已启用了高质量的视频综合。然而，即使在高端GPU上，他们的大量计算和记忆都需要对现实部署构成严重的挑战。作为一种普遍采用的解决方案，量化在降低图像DM的成本方面已被证明取得了显着的成功，而将视频DMS的直接应用仍然无效。在本文中，我们介绍了QVGEN，这是一种针对极低位量化的新型量化感知训练（QAT）框架（例如，4位或以下）。我们从理论分析开始，表明减少梯度规范对于促进QAT收敛至关重要。为此，我们引入了辅助模块（$ \ phi $）来减轻大量量化错误，从而显着增强了收敛性。为了消除$ \ phi $的推理间接费用，我们提出了一种排名dec的策略，该策略逐渐消除了$ \ phi $。具体来说，我们反复采用单数值分解（SVD）和提议的基于等级的正规化$ \ Mathbf {\ gamma} $来识别和衰减低量化组件。该策略在零推理开销时保留性能。 $ 4 $最先进（SOTA）视频DMS的大量实验，参数尺寸从$ 1.3 $ b $ \ sim14 $ b不等，表明QVGEN是第一个在4位设置下达到全精度可比质量的QVGEN。此外，它极大地胜过现有方法。例如，我们的3位COGVIDEOX-2B在VBENCH上实现了$+25.28美元的动态度和$+8.43 $的场景一致性的改善。