2025-06-04

Title: TaskVAE: Task-Specific Variational Autoencoders for Exemplar Generation in Continual Learning for Human Activity Recognition

• Authors: Bonpagna Kann, Sandra Castellanos-Paez, Romain Rombourg, Philippe Lalanda
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.01965
• Pdf URL: https://arxiv.org/pdf/2506.01965
• Copy Paste: [[2506.01965]] TaskVAE: Task-Specific Variational Autoencoders for Exemplar Generation in Continual Learning for Human Activity Recognition(https://arxiv.org/abs/2506.01965)
• Keywords: generation
• Abstract: As machine learning based systems become more integrated into daily life, they unlock new opportunities but face the challenge of adapting to dynamic data environments. Various forms of data shift-gradual, abrupt, or cyclic-threaten model accuracy, making continual adaptation essential. Continual Learning (CL) enables models to learn from evolving data streams while minimizing forgetting of prior knowledge. Among CL strategies, replay-based methods have proven effective, but their success relies on balancing memory constraints and retaining old class accuracy while learning new classes. This paper presents TaskVAE, a framework for replay-based CL in class-incremental settings. TaskVAE employs task-specific Variational Autoencoders (VAEs) to generate synthetic exemplars from previous tasks, which are then used to train the classifier alongside new task data. In contrast to traditional methods that require prior knowledge of the total class count or rely on a single VAE for all tasks, TaskVAE adapts flexibly to increasing tasks without such constraints. We focus on Human Activity Recognition (HAR) using IMU sensor-equipped devices. Unlike previous HAR studies that combine data across all users, our approach focuses on individual user data, better reflecting real-world scenarios where a person progressively learns new activities. Extensive experiments on 5 different HAR datasets show that TaskVAE outperforms experience replay methods, particularly with limited data, and exhibits robust performance as dataset size increases. Additionally, memory footprint of TaskVAE is minimal, being equivalent to only 60 samples per task, while still being able to generate an unlimited number of synthetic samples. The contributions lie in balancing memory constraints, task-specific generation, and long-term stability, making it a reliable solution for real-world applications in domains like HAR.
• 摘要：随着基于机器学习的系统越来越融合到日常生活中，它们将释放新的机会，但面临适应动态数据环境的挑战。各种形式的数据移位范围，突然或环保模型的准确性，使得持续适应至关重要。持续学习（CL）使模型能够从不断发展的数据流中学习，同时最大程度地减少忘记先验知识。在策略中，基于重播的方法已被证明有效，但它们的成功依赖于平衡记忆约束并在学习新课程的同时保持旧班级准确性。本文介绍了TaskVae，这是一个在课堂开发设置中基于基于重播的CL的框架。 TaskVae采用特定于任务的变分自动编码器（VAE）来生成先前任务的合成示例，然后将其与新任务数据一起训练分类器。与需要先验了解总班级或依靠所有任务的传统方法相反，TaskVae灵活地适应了没有这种约束的任务。我们使用配备IMU传感器的设备专注于人类活动识别（HAR）。与以前的HAR研究结合了所有用户的数据，我们的方法专注于单个用户数据，更好地反映了一个人逐步学习新活动的现实情况。在5个不同的HAR数据集上进行的广泛实验表明，TaskVae的表现优于重播方法，尤其是数据有限，并且随着数据集尺寸的增加而表现出强大的性能。此外，任务录的内存足迹很小，相当于每个任务的60个样本，同时仍然能够生成无限数量的合成样本。贡献在于平衡记忆约束，特定于任务的生成和长期稳定性，这使其成为HAR等领域中现实世界应用的可靠解决方案。

Title: Crack Path Prediction with Operator Learning using Discrete Particle System data Generation

• Authors: Elham Kiyani, Venkatesh Ananchaperumal, Ahmad Peyvan, Mahendaran Uchimali, Gang Li, George Em Karniadakis
• Subjects: cs.LG, cond-mat.mtrl-sci, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.01976
• Pdf URL: https://arxiv.org/pdf/2506.01976
• Copy Paste: [[2506.01976]] Crack Path Prediction with Operator Learning using Discrete Particle System data Generation(https://arxiv.org/abs/2506.01976)
• Keywords: generation
• Abstract: Accurately modeling crack propagation is critical for predicting failure in engineering materials and structures, where small cracks can rapidly evolve and cause catastrophic damage. The interaction of cracks with discontinuities, such as holes, significantly affects crack deflection and arrest. Recent developments in discrete particle systems with multibody interactions based on constitutive behavior have demonstrated the ability to capture crack nucleation and evolution without relying on continuum assumptions. In this work, we use data from Constitutively Informed Particle Dynamics (CPD) simulations to train operator learning models, specifically Deep Operator Networks (DeepONets), which learn mappings between function spaces instead of finite-dimensional vectors. We explore two DeepONet variants: vanilla and Fusion DeepONet, for predicting time-evolving crack propagation in specimens with varying geometries. Three representative cases are studied: (i) varying notch height without active fracture; and (ii) and (iii) combinations of notch height and hole radius where dynamic fracture occurs on irregular discrete meshes. The models are trained on 32 to 45 samples, using geometric inputs in the branch network and spatial-temporal coordinates in the trunk network. Results show that Fusion DeepONet consistently outperforms the vanilla variant, with more accurate predictions especially in non-fracturing cases. Fracture-driven scenarios involving displacement and crack evolution remain more challenging. These findings highlight the potential of Fusion DeepONet to generalize across complex, geometry-varying, and time-dependent crack propagation phenomena.
• 摘要：准确地对裂纹传播进行建模对于预测工程材料和结构的故障至关重要，在该工程材料和结构中，小裂缝可以快速发展并造成灾难性损害。裂缝与不连续性（例如孔）的相互作用显着影响裂纹挠度和停滞。具有基于本构行为的多体相互作用的离散粒子系统中的最新发展已经证明了捕获裂纹成核和进化的能力，而无需依赖连续假设。在这项工作中，我们使用构成知情的粒子动力学（CPD）模拟的数据来培训操作员学习模型，特别是深层操作员网络（DeepOnets），这些网络（DeepOnets）在功能空间之间学习映射，而不是有限维矢量。我们探索了两个deponet变体：香草和融合deponet，用于预测具有不同几何形状的标本中的随时间变化的裂纹繁殖。研究了三个代表性的案例：（i）如果没有主动骨折，则变化不高； （ii）和（iii）Notch高度和孔半径的组合，在不规则离散网格上发生动态断裂。使用分支网络中的几何输入和中继网络中的空间坐标，对模型进行了32至45个样品的训练。结果表明，Fusion Deeponet始终优于香草变体，具有更准确的预测，尤其是在非压裂情况下。涉及流离失所和裂纹进化的裂缝驱动的场景仍然更具挑战性。这些发现突出了融合层融合跨复合物，几何变化和时间依赖性裂纹传播现象的潜力。

Title: Towards Unsupervised Training of Matching-based Graph Edit Distance Solver via Preference-aware GAN

• Authors: Wei Huang, Hanchen Wang, Dong Wen, Shaozhen Ma, Wenjie Zhang, Xuemin Lin
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.01977
• Pdf URL: https://arxiv.org/pdf/2506.01977
• Copy Paste: [[2506.01977]] Towards Unsupervised Training of Matching-based Graph Edit Distance Solver via Preference-aware GAN(https://arxiv.org/abs/2506.01977)
• Keywords: generative
• Abstract: Graph Edit Distance (GED) is a fundamental graph similarity metric widely used in various applications. However, computing GED is an NP-hard problem. Recent state-of-the-art hybrid GED solver has shown promising performance by formulating GED as a bipartite graph matching problem, then leveraging a generative diffusion model to predict node matching between two graphs, from which both the GED and its corresponding edit path can be extracted using a traditional algorithm. However, such methods typically rely heavily on ground-truth supervision, where the ground-truth labels are often costly to obtain in real-world scenarios. In this paper, we propose GEDRanker, a novel unsupervised GAN-based framework for GED computation. Specifically, GEDRanker consists of a matching-based GED solver and introduces an interpretable preference-aware discriminator with an effective training strategy to guide the matching-based GED solver toward generating high-quality node matching without the need for ground-truth labels. Extensive experiments on benchmark datasets demonstrate that our GEDRanker enables the matching-based GED solver to achieve near-optimal solution quality without any ground-truth supervision.
• 摘要：图编辑距离（GED）是在各种应用中广泛使用的基本图形相似性。但是，计算GED是一个NP硬性问题。最新的最新混合GED求解器通过将GED作为两部分图匹配问题，然后利用生成扩散模型来预测两个图之间的节点匹配，从而从中可以使用传统算法从中提取GED及其相应的编辑路径。但是，这种方法通常很大程度上依赖于地面监督，在现实世界中，地面真相标签通常是昂贵的。在本文中，我们提出了Gedranker，这是一种基于GED计算的基于GAN的新型框架。具体而言，Gedranker由一个基于匹配的GED求解器组成，并通过有效的训练策略引入了可解释的偏好感知歧视器，可指导基于匹配的GED求解器，以生成高质量的节点匹配，而无需对地面真相标记。基准数据集上的广泛实验表明，我们的Gedranker使基于匹配的GED求解器能够在没有任何地面实际监督的情况下实现近乎最佳的解决方案质量。

Title: Improvement of AMPs Identification with Generative Adversarial Network and Ensemble Classification

• Authors: Reyhaneh Keshavarzpour, Eghbal Mansoori
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.01983
• Pdf URL: https://arxiv.org/pdf/2506.01983
• Copy Paste: [[2506.01983]] Improvement of AMPs Identification with Generative Adversarial Network and Ensemble Classification(https://arxiv.org/abs/2506.01983)
• Keywords: generative
• Abstract: Identification of antimicrobial peptides is an important and necessary issue in today's era. Antimicrobial peptides are essential as an alternative to antibiotics for biomedical applications and many other practical applications. These oligopeptides are useful in drug design and cause innate immunity against microorganisms. Artificial intelligence algorithms have played a significant role in the ease of identifying these this http URL research is improved by improving proposed method in the field of antimicrobial peptides prediction. Suggested method is improved by combining the best coding method from different perspectives, In the following a deep neural network to balance the imbalanced combined datasets. The results of this research show that the proposed method have a significant improvement in the accuracy and efficiency of the prediction of antimicrobial peptides and are able to provide the best results compared to the existing methods. These development in the field of prediction and classification of antimicrobial peptides, basically in the fields of medicine and pharmaceutical industries, have high effectiveness and application.
• 摘要：在当今时代，鉴定抗菌肽是一个重要且必要的问题。抗菌肽是生物医学应用和许多其他实际应用的抗生素的替代品。这些寡肽在药物设计中很有用，并引起对微生物的先天免疫力。人工智能算法通过改进抗菌肽预测领域的提议方法来改善该HTTP URL研究的易于确定这些HTTP URL研究。通过从不同角度组合最佳的编码方法，在以下深度神经网络中以平衡不平衡的组合数据集，可以改善建议的方法。这项研究的结果表明，所提出的方法在预测抗菌肽的准确性和效率方面具有显着提高，并且与现有方法相比，能够提供最佳结果。这些在医学和制药行业领域的预测和抗菌肽的分类领域的发展具有很高的有效性和应用。

Title: SpecMemo: Speculative Decoding is in Your Pocket

• Authors: Selin Yildirim, Deming Chen
• Subjects: cs.LG, cs.AI, cs.DC
• Abstract URL: https://arxiv.org/abs/2506.01986
• Pdf URL: https://arxiv.org/pdf/2506.01986
• Copy Paste: [[2506.01986]] SpecMemo: Speculative Decoding is in Your Pocket(https://arxiv.org/abs/2506.01986)
• Keywords: generation
• Abstract: Recent advancements in speculative decoding have demonstrated considerable speedup across a wide array of large language model (LLM) tasks. Speculative decoding inherently relies on sacrificing extra memory allocations to generate several candidate tokens, of which acceptance rate drives the speedup. However, deploying speculative decoding on memory-constrained devices, such as mobile GPUs, remains as a significant challenge in real-world scenarios. In this work, we present a device-aware inference engine named SpecMemo that can smartly control memory allocations at finer levels to enable multi-turn chatbots with speculative decoding on such limited memory devices. Our methodology stems from theoretically modeling memory footprint of speculative decoding to determine a lower bound on the required memory budget while retaining speedup. SpecMemo empirically acquires a careful balance between minimizing redundant memory allocations for rejected candidate tokens and maintaining competitive performance gains from speculation. Notably, with SpecMemo's memory management, we maintain 96% of overall throughput from speculative decoding on MT-Bench, with reduced generation-memory by 65% on single Nvidia Titan RTX. Given multiple constrained GPUs, we build on top of previous speculative decoding architectures to facilitate big-model inference by distributing Llama-2-70B-Chat model, on which we provide novel batched speculative decoding to increase usability of multiple small server GPUs. This novel framework demonstrates 2x speedup over distributed and batched vanilla decoding with the base model on eight AMD MI250 GPUs. Moreover, inference throughput increases remarkably 8x with batch size 10. Our work contributes to democratized LLM applications in resource-constrained environments, providing a pathway for faster and cheaper deployment of real-world LLM applications with robust performance.
• 摘要：推测解码方面的最新进展表明，各种大型语言模型（LLM）任务的加速相当大。投机性解码本质上依赖于牺牲额外的记忆分配来生成几个候选令牌，其中接受率驱动了加速。但是，在现实世界中，在内存受限的设备（例如移动GPU）上部署投机解码仍然是一个重大挑战。在这项工作中，我们提出了一个名为SpecMemo的设备感知的推理引擎，该引擎可以在较优质的级别上智能控制内存分配，以在如此有限的内存设备上进行投机解码，从而启用多转弯聊天机器人。我们的方法源于从理论上对投机解码的内存足迹进行建模，以确定所需内存预算的下限，同时保留加速。 Specmemo从经验上获得了仔细的平衡，在最大程度地减少被拒绝的候选代币的冗余内存分配和从投机中维持竞争性绩效提高。值得注意的是，通过SpecMemo的内存管理，我们维持了MT基台上投机解码的总吞吐量的96％，而单个NVIDIA Titan RTX的发电生成记忆减少了65％。鉴于多个受限的GPU，我们以先前投机解码体系结构为基础，通过分发Llama-2-70B-Chat模型来促进大型模型，在该模型上，我们在其上提供了新颖的批处理投机解码，以提高多个小型服务器GPU的可用性。这个新颖的框架表明，在八个AMD MI250 GPU上的基本模型上分布式和批处理的香草进行了2倍的加速。此外，推理吞吐量以批次10的范围显着增加8倍。我们的工作有助于在资源受限环境中民主化的LLM应用程序，为更快，更便宜的现实世界LLM应用程序提供了良好性能的途径。

Title: Object-centric Self-improving Preference Optimization for Text-to-Image Generation

• Authors: Yoonjin Oh, Yongjin Kim, Hyomin Kim, Donghwan Chi, Sungwoong Kim
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02015
• Pdf URL: https://arxiv.org/pdf/2506.02015
• Copy Paste: [[2506.02015]] Object-centric Self-improving Preference Optimization for Text-to-Image Generation(https://arxiv.org/abs/2506.02015)
• Keywords: generation
• Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have significantly improved both image understanding and generation capabilities. Despite these improvements, MLLMs still struggle with fine-grained visual comprehension, particularly in text-to-image generation tasks. While preference optimization methods have been explored to address these limitations in image understanding tasks, their application to image generation remains largely underexplored. To address this gap, we propose an Object-centric Self-improving Preference Optimization (OSPO) framework designed for text-to-image generation by MLLMs. OSPO leverages the intrinsic reasoning abilities of MLLMs without requiring any external datasets or models. OSPO emphasizes the importance of high-quality preference pair data, which is critical for effective preference optimization. To achieve this, it introduces a self-improving mechanism that autonomously constructs object-level contrastive preference pairs through object-centric prompt perturbation, densification and VQA scoring. This process eliminates ambiguous or disproportionate variations commonly found in naively generated preference pairs, thereby enhancing the effectiveness of preference optimization. We validate OSPO on three representative compositional text-to-image benchmarks, demonstrating substantial performance gains over baseline models.
• 摘要：多模式大语言模型（MLLM）的最新进展已显着提高了图像理解和发电能力。尽管有这些改进，但MLLM仍在精细的视觉理解中挣扎，尤其是在文本到图像生成任务中。虽然已经探索了偏好优化方法来解决图像理解任务中的这些限制，但它们在图像生成中的应用仍然很大程度上没有被逐渐倍增。为了解决这一差距，我们提出了一个以对象为中心的自我提出优先优化（OSPO）框架，该框架为MLLM设计为文本到图像生成。 OSPO利用MLLM的内在推理能力，而无需任何外部数据集或模型。 OSPO强调了高质量偏好对数据的重要性，这对于有效的偏好优化至关重要。为了实现这一目标，它引入了一种自主改进机制，该机制通过以对象为中心的扰动，致密化和VQA评分来自主构建对象级对比度偏好对。该过程消除了在天真生成的偏好对中常见的歧义或不成比例的变化，从而增强了偏好优化的有效性。我们验证了三个代表性构图的文本对图像基准的OSPO，这表明基线模型的性能很大。

Title: EWGN: Elastic Weight Generation and Context Switching in Deep Learning

• Authors: Shriraj P. Sawant, Krishna P. Miyapuram
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02065
• Pdf URL: https://arxiv.org/pdf/2506.02065
• Copy Paste: [[2506.02065]] EWGN: Elastic Weight Generation and Context Switching in Deep Learning(https://arxiv.org/abs/2506.02065)
• Keywords: generation, generative
• Abstract: The ability to learn and retain a wide variety of tasks is a hallmark of human intelligence that has inspired research in artificial general intelligence. Continual learning approaches provide a significant step towards achieving this goal. It has been known that task variability and context switching are challenging for learning in neural networks. Catastrophic forgetting refers to the poor performance on retention of a previously learned task when a new task is being learned. Switching between different task contexts can be a useful approach to mitigate the same by preventing the interference between the varying task weights of the network. This paper introduces Elastic Weight Generative Networks (EWGN) as an idea for context switching between two different tasks. The proposed EWGN architecture uses an additional network that generates the weights of the primary network dynamically while consolidating the weights learned. The weight generation is input-dependent and thus enables context switching. Using standard computer vision datasets, namely MNIST and fashion-MNIST, we analyse the retention of previously learned task representations in Fully Connected Networks, Convolutional Neural Networks, and EWGN architectures with Stochastic Gradient Descent and Elastic Weight Consolidation learning algorithms. Understanding dynamic weight generation and context-switching ability can be useful in enabling continual learning for improved performance.
• 摘要：学习和保留各种任务的能力是人工智能研究的人类智能的标志。持续的学习方法为实现这一目标提供了重要的一步。众所周知，任务可变性和上下文切换在神经网络中学习具有挑战性。灾难性忘记是指在学习新任务时保留以前学习的任务方面的绩效不佳。在不同的任务上下文之间切换可以是一种有用的方法，可以通过防止网络的不同任务权重之间的干扰来减轻相同的方法。本文介绍了弹性重量生成网络（EWGN），作为在两个不同任务之间进行上下文切换的想法。所提出的EWGN体系结构使用了一个附加的网络，该网络在整合所学习的权重的同时，动态地生成了主要网​​络的权重。体重生成是取决于输入的，因此可以启用上下文切换。我们使用标准的计算机视觉数据集，即MNIST和时尚 - 纳斯特，我们分析了以前连接的网络，卷积神经网络以及具有随机梯度下降和弹性重量整合学习算法的EWGN体系结构中先前学习的任务表示的保留。了解动态的体重产生和上下文切换能力可能有助于促进持续学习以提高性能。

Title: An Introduction to Flow Matching and Diffusion Models

• Authors: Peter Holderrieth, Ezra Erives
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.02070
• Pdf URL: https://arxiv.org/pdf/2506.02070
• Copy Paste: [[2506.02070]] An Introduction to Flow Matching and Diffusion Models(https://arxiv.org/abs/2506.02070)
• Keywords: generative
• Abstract: Diffusion and flow-based models have become the state of the art for generative AI across a wide range of data modalities, including images, videos, shapes, molecules, music, and more! These notes are originally from this https URL, as taught at MIT over the 2025 IAP (winter) term, and are intended to accompany other course content, including lectures and labs. Overall, they function as a self-contained introduction to both flow matching and diffusion models, starting with ordinary and stochastic differential equations, and culminating in flow matching, score matching, classifier-free guidance, and the inner workings of modern, state-of-the-art models for image and video. These notes, and the accompanying course, are ideal for students and practitioners alike who want to develop a principled understanding of the theory and practice of generative AI.
• 摘要：扩散和基于流的模型已成为广泛的数据模式的生成AI的最新技术，包括图像，视频，形状，分子，音乐等！这些注释最初来自该HTTPS URL，如2025年IAP（冬季）期间在MIT上教授的那样，旨在伴随其他课程内容，包括讲座和实验室。总体而言，它们是对流程匹配和扩散模型的独立介绍，从普通和随机微分方程开始，并在流量匹配，得分匹配，无分类器指导以及现代，最先进的图像和视频的内部连接中达到顶点。这些笔记以及随附的课程非常适合想要对生成AI的理论和实践有原则的理解的学生和从业人员的理想选择。

Title: Temporal Causal-based Simulation for Realistic Time-series Generation

• Authors: Nikolaos Gkorgkolis, Nikolaos Kougioulis, MingXue Wang, Bora Caglayan, Andrea Tonon, Dario Simionato, Ioannis Tsamardinos
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.02084
• Pdf URL: https://arxiv.org/pdf/2506.02084
• Copy Paste: [[2506.02084]] Temporal Causal-based Simulation for Realistic Time-series Generation(https://arxiv.org/abs/2506.02084)
• Keywords: generation
• Abstract: Causal Discovery plays a pivotal role in revealing relationships among observed variables, particularly in the temporal setup. While the majority of CD methods rely on synthetic data for evaluation, and recently for training, these fall short in accurately mirroring real-world scenarios; an effect even more evident in temporal data. Generation techniques depending on simplified assumptions on causal structure, effects and time, limit the quality and diversity of the simulated data. In this work, we introduce Temporal Causal-based Simulation (TCS), a robust framework for generating realistic time-series data and their associated temporal causal graphs. The approach is structured in three phases: estimating the true lagged causal structure of the data, approximating the functional dependencies between variables and learning the noise distribution of the corresponding causal model, each part of which can be explicitly tailored based on data assumptions and characteristics. Through an extensive evaluation process, we highlight that single detection methods for generated data discrimination prove inadequate, accentuating it as a multifaceted challenge. For this, we detail a Min-max optimization phase that draws on AutoML techniques. Our contributions include a flexible, model-agnostic pipeline for generating realistic temporal causal data, a thorough evaluation setup which enhances the validity of the generated datasets and insights into the challenges posed by realistic data generation. Through experiments involving not only real but also semi-synthetic and purely synthetic datasets, we demonstrate that while sampling realistic causal data remains a complex task, our method enriches the domain of generating sensible causal-based temporal data.
• 摘要：因果发现在揭示观察到的变量之间的关系中起着关键作用，尤其是在时间设置中。尽管大多数CD方法都依赖于合成数据进行评估，而最近进行培训，但这些方法在准确地反映现实世界的情况下掉落了。效果在时间数据中更为明显。生成技术取决于对因果结构，效果和时间的简化假设，请限制模拟数据的质量和多样性。在这项工作中，我们引入了基于时间因果的模拟（TCS），这是一个可实现的框架，用于生成现实的时间序列数据及其相关的时间因果图。该方法分为三个阶段：估计数据的真实滞后因果结构，近似变量之间的功能依赖性和学习相应因果模型的噪声分布，每个部分都可以根据数据假设和特征明确地定制。通过广泛的评估过程，我们强调了生成数据歧视的单个检测方法证明不足，将其作为多方面的挑战。为此，我们详细介绍了借鉴汽车技术的最低最大优化阶段。我们的贡献包括一种灵活的模型无形管道，用于生成现实的时间因果数据，这是一种彻底的评估设置，可提高生成的数据集的有效性以及对现实数据生成带来的挑战的见解。通过不仅涉及真实的实验，而且还涉及半合成和纯合成数据集，我们证明，在对现实的因果数据进行采样时，我们的方法富含生成基于因果关系的时间数据的领域。

Title: SALAD: Systematic Assessment of Machine Unlearing on LLM-Aided Hardware Design

• Authors: Zeng Wang, Minghao Shao, Rupesh Karn, Jitendra Bhandari, Likhitha Mankali, Ramesh Karri, Ozgur Sinanoglu, Muhammad Shafique, Johann Knechtel
• Subjects: cs.LG, cs.AI, cs.CR
• Abstract URL: https://arxiv.org/abs/2506.02089
• Pdf URL: https://arxiv.org/pdf/2506.02089
• Copy Paste: [[2506.02089]] SALAD: Systematic Assessment of Machine Unlearing on LLM-Aided Hardware Design(https://arxiv.org/abs/2506.02089)
• Keywords: generation
• Abstract: Large Language Models (LLMs) offer transformative capabilities for hardware design automation, particularly in Verilog code generation. However, they also pose significant data security challenges, including Verilog evaluation data contamination, intellectual property (IP) design leakage, and the risk of malicious Verilog generation. We introduce SALAD, a comprehensive assessment that leverages machine unlearning to mitigate these threats. Our approach enables the selective removal of contaminated benchmarks, sensitive IP and design artifacts, or malicious code patterns from pre-trained LLMs, all without requiring full retraining. Through detailed case studies, we demonstrate how machine unlearning techniques effectively reduce data security risks in LLM-aided hardware design.
• 摘要：大型语言模型（LLMS）为硬件设计自动化提供了变换功能，尤其是在Verilog代码生成中。但是，它们还构成了重大的数据安全挑战，包括Verilog评估数据污染，知识产权（IP）设计泄漏以及恶意Verilog产生的风险。我们介绍了沙拉，这是一项全面的评估，该评估利用机器来减轻这些威胁。我们的方法可以选择性删除受污染的基准，敏感的IP和设计文物，或者是从预训练的LLMS中选择的恶意代码模式，而无需完全重新培训。通过详细的案例研究，我们演示了如何有效地降低LLM辅助硬件设计中的数据安全风险。

Title: Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences

• Authors: Hyojin Bahng, Caroline Chan, Fredo Durand, Phillip Isola
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.02095
• Pdf URL: https://arxiv.org/pdf/2506.02095
• Copy Paste: [[2506.02095]] Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences(https://arxiv.org/abs/2506.02095)
• Keywords: generation
• Abstract: Learning alignment between language and vision is a fundamental challenge, especially as multimodal data becomes increasingly detailed and complex. Existing methods often rely on collecting human or AI preferences, which can be costly and time-intensive. We propose an alternative approach that leverages cycle consistency as a supervisory signal. Given an image and generated text, we map the text back to image space using a text-to-image model and compute the similarity between the original image and its reconstruction. Analogously, for text-to-image generation, we measure the textual similarity between an input caption and its reconstruction through the cycle. We use the cycle consistency score to rank candidates and construct a preference dataset of 866K comparison pairs. The reward model trained on our dataset outperforms state-of-the-art alignment metrics on detailed captioning, with superior inference-time scalability when used as a verifier for Best-of-N sampling. Furthermore, performing DPO and Diffusion DPO using our dataset enhances performance across a wide range of vision-language tasks and text-to-image generation. Our dataset, model, and code are at this https URL
• 摘要：语言和视觉之间的学习对齐是一个根本的挑战，尤其是当多模式数据变得越来越详细且复杂时。现有方法通常依赖于收集人类或人工智能偏好，这可能是昂贵且耗时的。我们提出了一种替代方法，该方法将循环一致性作为监督信号。给定图像和生成的文本，我们使用文本对图像模型将文本映射回图像空间，并计算原始图像及其重建之间的相似性。类似地，对于文本到图像生成，我们在整个周期中测量了输入字幕及其重建之间的文本相似性。我们使用周期一致性评分来对候选人进行排名，并构建一个866K比较对的偏好数据集。在我们的数据集上训练的奖励模型在详细字幕上均优于最先进的对准指标，当用作验证器以进行最佳n采样时，具有出色的推理时间可扩展性。此外，使用我们的数据集执行DPO和扩散DPO可以在各种视觉语言任务和文本到图像生成中增强性能。我们的数据集，模型和代码在此HTTPS URL处

Title: Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment

• Authors: Johannes Schusterbauer, Ming Gui, Frank Fundel, Björn Ommer
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.02221
• Pdf URL: https://arxiv.org/pdf/2506.02221
• Copy Paste: [[2506.02221]] Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment(https://arxiv.org/abs/2506.02221)
• Keywords: generative
• Abstract: Diffusion models have revolutionized generative tasks through high-fidelity outputs, yet flow matching (FM) offers faster inference and empirical performance gains. However, current foundation FM models are computationally prohibitive for finetuning, while diffusion models like Stable Diffusion benefit from efficient architectures and ecosystem support. This work addresses the critical challenge of efficiently transferring knowledge from pre-trained diffusion models to flow matching. We propose Diff2Flow, a novel framework that systematically bridges diffusion and FM paradigms by rescaling timesteps, aligning interpolants, and deriving FM-compatible velocity fields from diffusion predictions. This alignment enables direct and efficient FM finetuning of diffusion priors with no extra computation overhead. Our experiments demonstrate that Diff2Flow outperforms naïve FM and diffusion finetuning particularly under parameter-efficient constraints, while achieving superior or competitive performance across diverse downstream tasks compared to state-of-the-art methods. We will release our code at this https URL.
• 摘要：扩散模型通过高保真输出彻底改变了生成任务，但是流量匹配（FM）提供了更快的推理和经验性能提高。但是，当前的基础FM模型对于填充而言是计算上的过敏，而诸如有效的体系结构和生态系统支持的稳定扩散益处之类的扩散模型。这项工作解决了有效地将知识从预训练的扩散模型转移到流匹配的关键挑战。我们提出了Diff2Flow，这是一个新型框架，该框架是通过重新缩放时间段，对齐插值并从扩散预测中得出兼容FM兼容的速度场来系统地桥接扩散和FM范式。这种对齐使扩散先验的直接有效的FM登录没有额外的计算开销。我们的实验表明，DIFF2Flow在参数效率高效的约束下尤其是在参数效率的约束下优于幼稚的FM和扩散的填充，而与最先进的方法相比，在各种下游任务中实现了卓越或竞争性的性能。我们将在此HTTPS URL上发布代码。

Title: From Street Views to Urban Science: Discovering Road Safety Factors with Multimodal Large Language Models

• Authors: Yihong Tang, Ao Qu, Xujing Yu, Weipeng Deng, Jun Ma, Jinhua Zhao, Lijun Sun
• Subjects: cs.LG, cs.CY
• Abstract URL: https://arxiv.org/abs/2506.02242
• Pdf URL: https://arxiv.org/pdf/2506.02242
• Copy Paste: [[2506.02242]] From Street Views to Urban Science: Discovering Road Safety Factors with Multimodal Large Language Models(https://arxiv.org/abs/2506.02242)
• Keywords: generation
• Abstract: Urban and transportation research has long sought to uncover statistically meaningful relationships between key variables and societal outcomes such as road safety, to generate actionable insights that guide the planning, development, and renewal of urban and transportation systems. However, traditional workflows face several key challenges: (1) reliance on human experts to propose hypotheses, which is time-consuming and prone to confirmation bias; (2) limited interpretability, particularly in deep learning approaches; and (3) underutilization of unstructured data that can encode critical urban context. Given these limitations, we propose a Multimodal Large Language Model (MLLM)-based approach for interpretable hypothesis inference, enabling the automated generation, evaluation, and refinement of hypotheses concerning urban context and road safety outcomes. Our method leverages MLLMs to craft safety-relevant questions for street view images (SVIs), extract interpretable embeddings from their responses, and apply them in regression-based statistical models. UrbanX supports iterative hypothesis testing and refinement, guided by statistical evidence such as coefficient significance, thereby enabling rigorous scientific discovery of previously overlooked correlations between urban design and safety. Experimental evaluations on Manhattan street segments demonstrate that our approach outperforms pretrained deep learning models while offering full interpretability. Beyond road safety, UrbanX can serve as a general-purpose framework for urban scientific discovery, extracting structured insights from unstructured urban data across diverse socioeconomic and environmental outcomes. This approach enhances model trustworthiness for policy applications and establishes a scalable, statistically grounded pathway for interpretable knowledge discovery in urban and transportation studies.
• 摘要：城市和运输研究长期以来一直试图发现关键变量与社会成果（例如道路安全）之间的统计有意义的关系，以产生可行的见解，以指导城市和运输系统的规划，开发和更新。但是，传统的工作流面临着几个关键挑战：（1）依靠人类专家提出假设，这很耗时，容易确认偏见； （2）有限的解释性，特别是在深度学习方法中； （3）未实现无法编码关键城市环境的非结构化数据。鉴于这些局限性，我们提出了一种多模式大语模型（MLLM）基于可解释的假设推断的方法，从而使有关城市环境和道路安全结果的假设的自动生成，评估和完善。我们的方法利用MLLM来制作与安全性的问题有关街道视图图像（SVI），从其响应中提取可解释的嵌入，并将其应用于基于回归的统计模型中。 Urbanx支持迭代假设检验和改进，并在诸如系数意义之类的统计证据的指导下，从而对城市设计与安全之间的先前相关性进行了严格的科学发现。对曼哈顿街细分市场的实验评估表明，我们的方法的表现优于预算的深度学习模型，同时提供了完全的解释性。除了道路安全外，Urbanx还可以作为城市科学发现的通用框架，从各种社会经济和环境成果中从非结构化的城市数据中提取结构化见解。这种方法增强了对政策应用的模型可信度，并为城市和运输研究中的可解释知识发现建立了可扩展的统计基础途径。

Title: Motion aware video generative model

• Authors: Bowen Xue, Giuseppe Claudio Guarnera, Shuang Zhao, Zahra Montazeri
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.02244
• Pdf URL: https://arxiv.org/pdf/2506.02244
• Copy Paste: [[2506.02244]] Motion aware video generative model(https://arxiv.org/abs/2506.02244)
• Keywords: generation, generative
• Abstract: Recent advances in diffusion-based video generation have yielded unprecedented quality in visual content and semantic coherence. However, current approaches predominantly rely on statistical learning from vast datasets without explicitly modeling the underlying physics of motion, resulting in subtle yet perceptible non-physical artifacts that diminish the realism of generated videos. This paper introduces a physics-informed frequency domain approach to enhance the physical plausibility of generated videos. We first conduct a systematic analysis of the frequency-domain characteristics of diverse physical motions (translation, rotation, scaling), revealing that each motion type exhibits distinctive and identifiable spectral signatures. Building on this theoretical foundation, we propose two complementary components: (1) a physical motion loss function that quantifies and optimizes the conformity of generated videos to ideal frequency-domain motion patterns, and (2) a frequency domain enhancement module that progressively learns to adjust video features to conform to physical motion constraints while preserving original network functionality through a zero-initialization strategy. Experiments across multiple video diffusion architectures demonstrate that our approach significantly enhances motion quality and physical plausibility without compromising visual quality or semantic alignment. Our frequency-domain physical motion framework generalizes effectively across different video generation architectures, offering a principled approach to incorporating physical constraints into deep learning-based video synthesis pipelines. This work seeks to establish connections between data-driven models and physics-based motion models.
• 摘要：基于扩散的视频生成的最新进展已在视觉内容和语义连贯性方面产生了前所未有的质量。但是，当前的方法主要依赖于从广泛的数据集中进行的统计学习，而无需明确建模基本的运动物理学，从而导致微妙但可感知的非物理伪像，从而减少了生成的视频的现实主义。本文介绍了一种物理信息域的方法，以增强生成视频的物理合理性。我们首先对各种物理运动的频域特征进行系统分析（翻译，旋转，缩放），表明每种运动类型都表现出独特而可识别的光谱特征。 Building on this theoretical foundation, we propose two complementary components: (1) a physical motion loss function that quantifies and optimizes the conformity of generated videos to ideal frequency-domain motion patterns, and (2) a frequency domain enhancement module that progressively learns to adjust video features to conform to physical motion constraints while preserving original network functionality through a zero-initialization strategy.跨多个视频扩散体系结构进行的实验表明，我们的方法显着提高了运动质量和物理合理性，而不会损害视觉质量或语义对齐。我们的频域物理运动框架在不同的视频生成体系结构上有效地概括了，提供了一种原则性的方法，可以将物理约束纳入基于深度学习的视频综合管道中。这项工作旨在在数据驱动模型和基于物理的运动模型之间建立联系。

Title: Latent Stochastic Interpolants

• Authors: Saurabh Singh, Dmitry Lagun
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.02276
• Pdf URL: https://arxiv.org/pdf/2506.02276
• Copy Paste: [[2506.02276]] Latent Stochastic Interpolants(https://arxiv.org/abs/2506.02276)
• Keywords: generation, generative
• Abstract: Stochastic Interpolants (SI) are a powerful framework for generative modeling, capable of flexibly transforming between two probability distributions. However, their use in jointly optimized latent variable models remains unexplored as they require direct access to the samples from the two distributions. This work presents Latent Stochastic Interpolants (LSI) enabling joint learning in a latent space with end-to-end optimized encoder, decoder and latent SI models. We achieve this by developing a principled Evidence Lower Bound (ELBO) objective derived directly in continuous time. The joint optimization allows LSI to learn effective latent representations along with a generative process that transforms an arbitrary prior distribution into the encoder-defined aggregated posterior. LSI sidesteps the simple priors of the normal diffusion models and mitigates the computational demands of applying SI directly in high-dimensional observation spaces, while preserving the generative flexibility of the SI framework. We demonstrate the efficacy of LSI through comprehensive experiments on the standard large scale ImageNet generation benchmark.
• 摘要：随机插值（SI）是生成建模的强大框架，能够在两个概率分布之间灵活地转换。但是，它们在共同优化的潜在变量模型中的使用仍未开发，因为它们需要直接访问两个分布中的样本。这项工作提出了潜在的随机插入术（LSI），可在潜在空间中具有端到端优化的编码器，解码器和潜在SI模型。我们通过开发有原则的证据下限（ELBO）目标来实现这一目标。关节优化使LSI可以学习有效的潜在表示以及将任意先验分布转换为编码器定义的聚合后部的生成过程。 LSI避开了正常扩散模型的简单先验，并减轻了直接在高维观测空间中应用SI的计算需求，同时保留了SI框架的生成灵活性。我们通过对标准大型成像网生成基准的全面实验来证明LSI的功效。

Title: Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Models

• Authors: Yuchen Liang, Renxiang Huang, Lifeng Lai, Ness Shroff, Yingbin Liang
• Subjects: cs.LG, eess.SP, math.ST
• Abstract URL: https://arxiv.org/abs/2506.02318
• Pdf URL: https://arxiv.org/pdf/2506.02318
• Copy Paste: [[2506.02318]] Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Models(https://arxiv.org/abs/2506.02318)
• Keywords: generation
• Abstract: Discrete state space diffusion models have shown significant advantages in applications involving discrete data, such as text and image generation. It has also been observed that their performance is highly sensitive to the choice of rate matrices, particularly between uniform and absorbing rate matrices. While empirical results suggest that absorbing rate matrices often yield better generation quality compared to uniform rate matrices, existing theoretical works have largely focused on the uniform rate matrices case. Notably, convergence guarantees and error analyses for absorbing diffusion models are still missing. In this work, we provide the first finite-time error bounds and convergence rate analysis for discrete diffusion models using absorbing rate matrices. We begin by deriving an upper bound on the KL divergence of the forward process, introducing a surrogate initialization distribution to address the challenge posed by the absorbing stationary distribution, which is a singleton and causes the KL divergence to be ill-defined. We then establish the first convergence guarantees for both the $\tau$-leaping and uniformization samplers under absorbing rate matrices, demonstrating improved rates over their counterparts using uniform rate matrices. Furthermore, under suitable assumptions, we provide convergence guarantees without early stopping. Our analysis introduces several new technical tools to address challenges unique to absorbing rate matrices. These include a Jensen-type argument for bounding forward process convergence, novel techniques for bounding absorbing score functions, and a non-divergent upper bound on the score near initialization that removes the need of early-stopping.
• 摘要：离散的状态空间扩散模型在涉及离散数据（例如文本和图像生成）的应用中显示出显着优势。还观察到，它们的性能对速率矩阵的选择高度敏感，尤其是在均匀和吸收速率矩阵之间。尽管经验结果表明，与均匀速率矩阵相比，吸收率矩阵通常会产生更好的发电质量，但现有的理论工作主要集中在均匀的速率矩阵案例上。值得注意的是，吸收扩散模型的收敛保证和错误分析仍然缺失。在这项工作中，我们使用吸收速率矩阵为离散扩散模型提供了第一个有限的时间误差界和收敛速率分析。首先，我们将上界的上限推导出远期过程的KL差异，引入了替代初始化分布，以应对吸收的固定分布所带来的挑战，这是单胎，并导致kl差异不明显。然后，我们建立了在吸收率矩阵下的$ \ tau $ leaping和均匀化采样器的第一个收敛保证，这表明使用统一的速率矩阵比对应物的利率提高了。此外，在适当的假设下，我们提供融合保证，而无需尽早停止。我们的分析介绍了几种新的技术工具，以应对吸收利率矩阵独有的挑战。这些包括用于界定正向过程收敛的詹森类型论点，用于界定吸收分数功能的新技术以及在初始化附近的分数上的非发散上限，从而消除了早期停止的需求。

Title: Medical World Model: Generative Simulation of Tumor Evolution for Treatment Planning

• Authors: Yijun Yang, Zhao-Yang Wang, Qiuping Liu, Shuwen Sun, Kang Wang, Rama Chellappa, Zongwei Zhou, Alan Yuille, Lei Zhu, Yu-Dong Zhang, Jieneng Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02327
• Pdf URL: https://arxiv.org/pdf/2506.02327
• Copy Paste: [[2506.02327]] Medical World Model: Generative Simulation of Tumor Evolution for Treatment Planning(https://arxiv.org/abs/2506.02327)
• Keywords: generative
• Abstract: Providing effective treatment and making informed clinical decisions are essential goals of modern medicine and clinical care. We are interested in simulating disease dynamics for clinical decision-making, leveraging recent advances in large generative models. To this end, we introduce the Medical World Model (MeWM), the first world model in medicine that visually predicts future disease states based on clinical decisions. MeWM comprises (i) vision-language models to serve as policy models, and (ii) tumor generative models as dynamics models. The policy model generates action plans, such as clinical treatments, while the dynamics model simulates tumor progression or regression under given treatment conditions. Building on this, we propose the inverse dynamics model that applies survival analysis to the simulated post-treatment tumor, enabling the evaluation of treatment efficacy and the selection of the optimal clinical action plan. As a result, the proposed MeWM simulates disease dynamics by synthesizing post-treatment tumors, with state-of-the-art specificity in Turing tests evaluated by radiologists. Simultaneously, its inverse dynamics model outperforms medical-specialized GPTs in optimizing individualized treatment protocols across all metrics. Notably, MeWM improves clinical decision-making for interventional physicians, boosting F1-score in selecting the optimal TACE protocol by 13%, paving the way for future integration of medical world models as the second readers.
• 摘要：提供有效的治疗并做出明智的临床决策是现代医学和临床护理的重要目标。我们有兴趣模拟疾病动态以进行临床决策，并利用大型生成模型的最新进展。为此，我们介绍了医学世界模型（MEWM），这是医学领域的第一个世界模型，它在视觉上根据临床决策预测未来的疾病状态。 MEWM包括（i）视觉模型，可作为策略模型，（ii）肿瘤生成模型作为动力学模型。该政策模型生成了行动计划，例如临床治疗，而动力学模型在给定的治疗条件下模拟了肿瘤的进展或回归。在此基础上，我们提出了将生存分析应用于模拟后处理后肿瘤的逆动力学模型，从而可以评估治疗疗效和最佳临床动作计划的选择。结果，拟议的MEWM通过合成治疗后肿瘤来模拟疾病动态，并在放射学家评估的图灵测试中具有最新的特异性。同时，其逆动力学模型在优化所有指标的个性化治疗方案方面优于医学专业的GPT。值得注意的是，MEWM改善了介入医师的临床决策，从而使F1评分在选择最佳TACE方案方面提高了13％，为将医学世界模型作为第二读者的未来整合铺平了道路。

Title: Approximate Borderline Sampling using Granular-Ball for Classification Tasks

• Authors: Qin Xie, Qinghua Zhang, Shuyin Xia
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.02366
• Pdf URL: https://arxiv.org/pdf/2506.02366
• Copy Paste: [[2506.02366]] Approximate Borderline Sampling using Granular-Ball for Classification Tasks(https://arxiv.org/abs/2506.02366)
• Keywords: generation
• Abstract: Data sampling enhances classifier efficiency and robustness through data compression and quality improvement. Recently, the sampling method based on granular-ball (GB) has shown promising performance in generality and noisy classification tasks. However, some limitations remain, including the absence of borderline sampling strategies and issues with class boundary blurring or shrinking due to overlap between GBs. In this paper, an approximate borderline sampling method using GBs is proposed for classification tasks. First, a restricted diffusion-based GB generation (RD-GBG) method is proposed, which prevents GB overlaps by constrained expansion, preserving precise geometric representation of GBs via redefined ones. Second, based on the concept of heterogeneous nearest neighbor, a GB-based approximate borderline sampling (GBABS) method is proposed, which is the first general sampling method capable of both borderline sampling and improving the quality of class noise datasets. Additionally, since RD-GBG incorporates noise detection and GBABS focuses on borderline samples, GBABS performs outstandingly on class noise datasets without the need for an optimal purity threshold. Experimental results demonstrate that the proposed methods outperform the GB-based sampling method and several representative sampling methods. Our source code is publicly available at this https URL.
• 摘要：数据采样通过数据压缩和质量改进提高了分类器效率和鲁棒性。最近，基于粒状球（GB）的采样方法在一般性和嘈杂的分类任务中表现出了有希望的表现。但是，仍然存在一些局限性，包括缺乏边界抽样策略以及由于GB之间的重叠而导致的类边界模糊或缩小的问题。在本文中，提出了使用GBS进行分类任务的近似边界抽样方法。首先，提出了一种基于限制扩散的GB生成（RD-GBG）方法，该方法通过约束扩展来防止GB重叠，从而通过重新定义的方法保留GB的精确几何表示。其次，根据异质最近的邻居的概念，提出了基于GB的近似边界采样（GBABS）方法，这是第一种能够既有边界抽样又提高类噪声数据集质量的通用采样方法。此外，由于RD-GBG结合了噪声检测，而GBAB集中在边界样本上，因此GBAB在类噪声数据集上表现出色，而无需最佳的纯度阈值。实验结果表明，所提出的方法的表现优于基于GB的采样方法和几种代表性抽样方法。我们的源代码可在此HTTPS URL上公开获得。

Title: SFBD Flow: A Continuous-Optimization Framework for Training Diffusion Models with Noisy Samples

• Authors: Haoye Lu, Darren Lo, Yaoliang Yu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.02371
• Pdf URL: https://arxiv.org/pdf/2506.02371
• Copy Paste: [[2506.02371]] SFBD Flow: A Continuous-Optimization Framework for Training Diffusion Models with Noisy Samples(https://arxiv.org/abs/2506.02371)
• Keywords: generative
• Abstract: Diffusion models achieve strong generative performance but often rely on large datasets that may include sensitive content. This challenge is compounded by the models' tendency to memorize training data, raising privacy concerns. SFBD (Lu et al., 2025) addresses this by training on corrupted data and using limited clean samples to capture local structure and improve convergence. However, its iterative denoising and fine-tuning loop requires manual coordination, making it burdensome to implement. We reinterpret SFBD as an alternating projection algorithm and introduce a continuous variant, SFBD flow, that removes the need for alternating steps. We further show its connection to consistency constraint-based methods, and demonstrate that its practical instantiation, Online SFBD, consistently outperforms strong baselines across benchmarks.
• 摘要：扩散模型具有强大的生成性能，但通常依赖于可能包括敏感内容的大型数据集。模型倾向于记住培训数据并引起隐私问题的趋势，这一挑战更加复杂。 SFBD（Lu等，2025）通过对损坏的数据进行培训，并使用有限的干净样品来捕获本地结构并改善收敛性来解决此问题。但是，它的迭代性降解和微调循环需要手动协调，使实施变得繁重。我们将SFBD重新解释为一种交替的投影算法，并引入了连续的变体SFBD流，该流程消除了对交替步骤的需求。我们进一步显示了它与基于一致性约束方法的联系，并证明其实例化在线SFBD始终超过基准测试的强大基线。

Title: GAdaBoost: An Efficient and Robust AdaBoost Algorithm Based on Granular-Ball Structure

• Authors: Qin Xie, Qinghua Zhang, Shuyin Xia, Xinran Zhou, Guoyin Wang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.02390
• Pdf URL: https://arxiv.org/pdf/2506.02390
• Copy Paste: [[2506.02390]] GAdaBoost: An Efficient and Robust AdaBoost Algorithm Based on Granular-Ball Structure(https://arxiv.org/abs/2506.02390)
• Keywords: generation
• Abstract: Adaptive Boosting (AdaBoost) faces significant challenges posed by label noise, especially in multiclass classification tasks. Existing methods either lack mechanisms to handle label noise effectively or suffer from high computational costs due to redundant data usage. Inspired by granular computing, this paper proposes granular adaptive boosting (GAdaBoost), a novel two-stage framework comprising a data granulation stage and an adaptive boosting stage, to enhance efficiency and robustness under noisy conditions. To validate its feasibility, an extension of SAMME, termed this http URL, is proposed. Specifically, first, a granular-ball generation method is designed to compress data while preserving diversity and mitigating label noise. Second, the granular ball-based SAMME algorithm focuses on granular balls rather than individual samples, improving efficiency and reducing sensitivity to noise. Experimental results on some noisy datasets show that the proposed approach achieves superior robustness and efficiency compared with existing methods, demonstrating that this work effectively extends AdaBoost and SAMME.
• 摘要：自适应提升（ADABOOST）面临着标签噪声提出的重大挑战，尤其是在多类分类任务中。现有方法要么缺乏有效处理标签噪声的机制，要么由于冗余数据使用而遭受高计算成本。受颗粒计算的启发，本文提出了颗粒状自适应增强（Gadaboost），这是一个新型的两阶段框架，包括数据颗粒阶段和适应性增强阶段，以提高噪音条件下的效率和鲁棒性。为了验证其可行性，提出了该HTTP URL的Samme的扩展。具体而言，首先，一种颗粒球生成的方法旨在压缩数据，同时保留多样性和减轻标签噪声。其次，基于颗粒球的Samme算法的重点是颗粒球，而不是单个样本，从而提高了效率并降低对噪声的敏感性。一些嘈杂数据集的实验结果表明，与现有方法相比，所提出的方法可实现出色的鲁棒性和效率，这表明这项工作有效地扩展了adaboost和samme。

Title: The Devil is in the Darkness: Diffusion-Based Nighttime Dehazing Anchored in Brightness Perception

• Authors: Xiaofeng Cong, Yu-Xin Zhang, Haoran Wei, Yeying Jin, Junming Hou, Jie Gui, Jing Zhang, Dacheng Tao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02395
• Pdf URL: https://arxiv.org/pdf/2506.02395
• Copy Paste: [[2506.02395]] The Devil is in the Darkness: Diffusion-Based Nighttime Dehazing Anchored in Brightness Perception(https://arxiv.org/abs/2506.02395)
• Keywords: restoration, generative
• Abstract: While nighttime image dehazing has been extensively studied, converting nighttime hazy images to daytime-equivalent brightness remains largely unaddressed. Existing methods face two critical limitations: (1) datasets overlook the brightness relationship between day and night, resulting in the brightness mapping being inconsistent with the real world during image synthesis; and (2) models do not explicitly incorporate daytime brightness knowledge, limiting their ability to reconstruct realistic lighting. To address these challenges, we introduce the Diffusion-Based Nighttime Dehazing (DiffND) framework, which excels in both data synthesis and lighting reconstruction. Our approach starts with a data synthesis pipeline that simulates severe distortions while enforcing brightness consistency between synthetic and real-world scenes, providing a strong foundation for learning night-to-day brightness mapping. Next, we propose a restoration model that integrates a pre-trained diffusion model guided by a brightness perception network. This design harnesses the diffusion model's generative ability while adapting it to nighttime dehazing through brightness-aware optimization. Experiments validate our dataset's utility and the model's superior performance in joint haze removal and brightness mapping.
• 摘要：虽然夜间图像进行了广泛的研究，但将夜间朦胧的图像转换为白天等效的亮度仍然很大程度上尚未得到解决。现有方法面临两个关键局限性：（1）数据集忽略了白天和黑夜之间的亮度关系，从而导致图像合成过程中的亮度映射与现实世界不一致； （2）模型不会明确纳入白天的亮度知识，从而限制了它们重建逼真的照明的能力。为了应对这些挑战，我们介绍了基于扩散的夜间飞行（DIFFND）框架，该框架在数据综合和照明重建方面均出色。我们的方法始于数据合成管道，该数据综合管道模拟了严重的扭曲，同时在合成场景和现实世界之间实施亮度一致性，为学习夜间的亮度映射提供了坚实的基础。接下来，我们提出了一个恢复模型，该模型集成了以亮度感知网络为指导的预训练扩散模型。该设计可以利用扩散模型的生成能力，同时通过亮度感知到的优化来调整夜间的夜间飞行。实验验证了我们的数据集的实用程序以及该模型在关节去除和亮度映射方面的出色性能。

Title: Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models

• Authors: Zhiya Tan, Xin Zhang, Joey Tianyi Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02405
• Pdf URL: https://arxiv.org/pdf/2506.02405
• Copy Paste: [[2506.02405]] Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models(https://arxiv.org/abs/2506.02405)
• Keywords: generative
• Abstract: As generative techniques become increasingly accessible, authentic visuals are frequently subjected to iterative alterations by various individuals employing a variety of tools. Currently, to avoid misinformation and ensure accountability, a lot of research on detection and attribution is emerging. Although these methods demonstrate promise in single-stage manipulation scenarios, they fall short when addressing complex real-world iterative manipulation. In this paper, we are the first, to the best of our knowledge, to systematically model this real-world challenge and introduce a novel method to solve it. We define a task called "Modelship Attribution", which aims to trace the evolution of manipulated images by identifying the generative models involved and reconstructing the sequence of edits they performed. To realistically simulate this scenario, we utilize three generative models, StyleMapGAN, DiffSwap, and FacePartsSwap, that sequentially modify distinct regions of the same image. This process leads to the creation of the first modelship dataset, comprising 83,700 images (16,740 images*5). Given that later edits often overwrite the fingerprints of earlier models, the focus shifts from extracting blended fingerprints to characterizing each model's distinctive editing patterns. To tackle this challenge, we introduce the modelship attribution transformer (MAT), a purpose-built framework designed to effectively recognize and attribute the contributions of various models within complex, multi-stage manipulation workflows. Through extensive experiments and comparative analysis with other related methods, our results, including comprehensive ablation studies, demonstrate that the proposed approach is a highly effective solution for modelship attribution.
• 摘要：随着生成技术变得越来越易于​​获取，采用各种工具的各个人经常进行真实的视觉效果。目前，为了避免错误信息并确保问责制，正在出现大量有关检测和归因的研究。尽管这些方法在单阶段的操作场景中表现出了希望，但在解决复杂的现实世界迭代操作时，它们缺乏。在本文中，据我们所知，我们是第一个系统地模拟这一现实世界挑战并引入一种新方法来解决它的方法。我们定义了一个称为“模型归因”的任务，该任务旨在通过识别所涉及的生成模型并重建其执行的编辑序列来追踪操纵图像的演变。为了实际模拟这种情况，我们使用了三种生成模型，即Stylemapgan，diffswap和facepartsswap，它们顺序修改了同一图像的不同区域。此过程导致创建第一个模型数据集，其中包括83,700张图像（16,740张图像*5）。鉴于后来的编辑经常覆盖早期模型的指纹，因此焦点从提取混合指纹转变为表征每个模型的独特编辑模式。为了应对这一挑战，我们介绍了模型归因变压器（MAT），这是一个专门构建的框架，旨在有效地识别和归因于复杂的多阶段操作工作流程中各种模型的贡献。通过广泛的实验和其他相关方法的比较分析，我们的结果（包括全面的消融研究）表明，所提出的方法是模型归因的高效解决方案。

Title: Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Models

• Authors: Nurislam Tursynbek, Hastings Greer, Basar Demir, Marc Niethammer
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02419
• Pdf URL: https://arxiv.org/pdf/2506.02419
• Copy Paste: [[2506.02419]] Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Models(https://arxiv.org/abs/2506.02419)
• Keywords: generation
• Abstract: Diffusion models, while trained for image generation, have emerged as powerful foundational feature extractors for downstream tasks. We find that off-the-shelf diffusion models, trained exclusively to generate natural RGB images, can identify semantically meaningful correspondences in medical images. Building on this observation, we propose to leverage diffusion model features as a similarity measure to guide deformable image registration networks. We show that common intensity-based similarity losses often fail in challenging scenarios, such as when certain anatomies are visible in one image but absent in another, leading to anatomically inaccurate alignments. In contrast, our method identifies true semantic correspondences, aligning meaningful structures while disregarding those not present across images. We demonstrate superior performance of our approach on two tasks: multimodal 2D registration (DXA to X-Ray) and monomodal 3D registration (brain-extracted to non-brain-extracted MRI). Code: this https URL
• 摘要：扩散模型虽然受过图像生成的训练，但已成为下游任务的强大基础提取器。我们发现，专门训练以生成天然RGB图像的训练的现成扩散模型可以在医学图像中识别语义上有意义的对应关系。在此观察结果的基础上，我们建议利用扩散模型特征作为相似性度量，以指导可变形的图像注册网络。我们表明，基于强度的相似性损失通常在具有挑战性的情况下失败，例如，当某些解剖学在一个图像中可见但在另一个图像中不存在时，导致解剖不准确的比对。相比之下，我们的方法确定了真实的语义对应关系，使有意义的结构对准有意义的结构，同时忽略了那些不存在于图像中的结构。我们在两项任务上证明了方法的卓越性能：多模式2D登记（DXA至X射线）和单差3D注册（对非脑部提取的MRI提取脑部）。代码：此HTTPS URL

Title: Empowering Functional Neuroimaging: A Pre-trained Generative Framework for Unified Representation of Neural Signals

• Authors: Weiheng Yao, Xuhang Chen, Shuqiang Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02433
• Pdf URL: https://arxiv.org/pdf/2506.02433
• Copy Paste: [[2506.02433]] Empowering Functional Neuroimaging: A Pre-trained Generative Framework for Unified Representation of Neural Signals(https://arxiv.org/abs/2506.02433)
• Keywords: generative
• Abstract: Multimodal functional neuroimaging enables systematic analysis of brain mechanisms and provides discriminative representations for brain-computer interface (BCI) decoding. However, its acquisition is constrained by high costs and feasibility limitations. Moreover, underrepresentation of specific groups undermines fairness of BCI decoding model. To address these challenges, we propose a unified representation framework for multimodal functional neuroimaging via generative artificial intelligence (AI). By mapping multimodal functional neuroimaging into a unified representation space, the proposed framework is capable of generating data for acquisition-constrained modalities and underrepresented groups. Experiments show that the framework can generate data consistent with real brain activity patterns, provide insights into brain mechanisms, and improve performance on downstream tasks. More importantly, it can enhance model fairness by augmenting data for underrepresented groups. Overall, the framework offers a new paradigm for decreasing the cost of acquiring multimodal functional neuroimages and enhancing the fairness of BCI decoding models.
• 摘要：多模式功能神经成像能够系统地分析大脑机制，并为脑机构界面（BCI）解码提供了歧视性表示。但是，其收购受到高成本和可行性限制的限制。此外，特定组的代表性不足会破坏BCI解码模型的公平性。为了应对这些挑战，我们提出了一个通过生成人工智能（AI）的多模式功能神经影像的统一表示框架。通过将多模式功能神经成像映射到统一的表示空间中，该框架能够生成数据以获取被限制的被限制的模式和代表性不足的组。实验表明，该框架可以生成与真实的大脑活动模式一致的数据，提供对大脑机制的见解，并提高下游任务的性能。更重要的是，它可以通过增强代表性不足的群体的数据来增强模型公平性。总体而言，该框架提供了一个新的范式，可降低获取多模式功能神经图像并增强BCI解码模型的公平性的成本。

Title: SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios

• Authors: Lingwei Dang, Ruizhi Shao, Hongwen Zhang, Wei Min, Yebin Liu, Qingyao Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02444
• Pdf URL: https://arxiv.org/pdf/2506.02444
• Copy Paste: [[2506.02444]] SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios(https://arxiv.org/abs/2506.02444)
• Keywords: generation
• Abstract: Hand-Object Interaction (HOI) generation has significant application potential. However, current 3D HOI motion generation approaches heavily rely on predefined 3D object models and lab-captured motion data, limiting generalization capabilities. Meanwhile, HOI video generation methods prioritize pixel-level visual fidelity, often sacrificing physical plausibility. Recognizing that visual appearance and motion patterns share fundamental physical laws in the real world, we propose a novel framework that combines visual priors and dynamic constraints within a synchronized diffusion process to generate the HOI video and motion simultaneously. To integrate the heterogeneous semantics, appearance, and motion features, our method implements tri-modal adaptive modulation for feature aligning, coupled with 3D full-attention for modeling inter- and intra-modal dependencies. Furthermore, we introduce a vision-aware 3D interaction diffusion model that generates explicit 3D interaction sequences directly from the synchronized diffusion outputs, then feeds them back to establish a closed-loop feedback cycle. This architecture eliminates dependencies on predefined object models or explicit pose guidance while significantly enhancing video-motion consistency. Experimental results demonstrate our method's superiority over state-of-the-art approaches in generating high-fidelity, dynamically plausible HOI sequences, with notable generalization capabilities in unseen real-world scenarios. Project page at \href{this https URL}{this https URL}.
• 摘要：手动相互作用（HOI）发电具有巨大的应用潜力。但是，当前的3D HOI运动生成方法在很大程度上依赖于预定义的3D对象模型和实验室捕获的运动数据，从而限制了概括能力。同时，HOI视频生成方法优先考虑像素级的视觉保真度，通常会牺牲身体上的合理性。认识到视觉外观和运动模式在现实世界中共享基本的物理定律，我们提出了一个新颖的框架，将视觉先验和动态约束结合在同步扩散过程中，以同时生成HOI视频和运动。为了整合异质语义，外观和运动特征，我们的方法实现了三模式自适应调制，以进行特征对齐，并与3D全注意相结合，用于建模模型间和模式内依赖性。此外，我们引入了一个视觉感知的3D相互作用扩散模型，该模型直接从同步扩散输出中生成显式3D相互作用序列，然后将其馈回以建立闭环反馈周期。该体系结构消除了对预定义的对象模型或明确姿势指导的依赖性，同时显着增强了视频动作一致性。实验结果证明了我们的方法优于生成高保真性，动态合理的HOI序列，在看不见的现实世界情景中具有显着的概括能力。 \ href {此https url} {此https url}的项目页面。

Title: ANT: Adaptive Neural Temporal-Aware Text-to-Motion Model

• Authors: Wenshuo Chen, Kuimou Yu, Haozhe Jia, Kaishen Yuan, Bowen Tian, Songning Lai, Hongru Xiao, Erhang Zhang, Lei Wang, Yutao Yue
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02452
• Pdf URL: https://arxiv.org/pdf/2506.02452
• Copy Paste: [[2506.02452]] ANT: Adaptive Neural Temporal-Aware Text-to-Motion Model(https://arxiv.org/abs/2506.02452)
• Keywords: generation
• Abstract: While diffusion models advance text-to-motion generation, their static semantic conditioning ignores temporal-frequency demands: early denoising requires structural semantics for motion foundations while later stages need localized details for text alignment. This mismatch mirrors biological morphogenesis where developmental phases demand distinct genetic programs. Inspired by epigenetic regulation governing morphological specialization, we propose **(ANT)**, an **A**daptive **N**eural **T**emporal-Aware architecture. ANT orchestrates semantic granularity through: **(i) Semantic Temporally Adaptive (STA) Module:** Automatically partitions denoising into low-frequency structural planning and high-frequency refinement via spectral analysis. **(ii) Dynamic Classifier-Free Guidance scheduling (DCFG):** Adaptively adjusts conditional to unconditional ratio enhancing efficiency while maintaining fidelity. **(iii) Temporal-semantic reweighting:** Quantitatively aligns text influence with phase requirements. Extensive experiments show that ANT can be applied to various baselines, significantly improving model performance, and achieving state-of-the-art semantic alignment on StableMoFusion.
• 摘要：虽然扩散模型提高了文本到动作的生成，但它们的静态语义调节忽略了暂时的频率需求：早期的DeNoising需要运动基础的结构语义，而后期阶段则需要本地化的详细信息才能进行文本对齐。这种不匹配反映了生物形态发生的，发展阶段需要独特的遗传程序。受形态专业化的表观遗传调节的启发，我们提出了**（ant）**，** a ** a ** daptive ** n ** eural ** t ** emporal-saware-saware-aware-aware shadectuct。蚂蚁通过：**（i）语义在时间上自适应（STA）模块来策划语义粒度：**自动通过光谱分析将变形的划分为低频结构计划和高频完善。 **（ii）无动态分类器指导计划（DCFG）：**自适应地调整有条件至无条件比率提高效率，同时保持忠诚度。 **（iii）时间语义重新加权：**定量地将文本影响与阶段需求保持一致。广泛的实验表明，ANT可以应用于各种基线，可显着提高模型性能，并在稳定膜上实现最新的语义一致性。

Title: ReSpace: Text-Driven 3D Scene Synthesis and Editing with Preference Alignment

• Authors: Martin JJ. Bucher, Iro Armeni
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02459
• Pdf URL: https://arxiv.org/pdf/2506.02459
• Copy Paste: [[2506.02459]] ReSpace: Text-Driven 3D Scene Synthesis and Editing with Preference Alignment(https://arxiv.org/abs/2506.02459)
• Keywords: generative
• Abstract: Scene synthesis and editing has emerged as a promising direction in computer graphics. Current trained approaches for 3D indoor scenes either oversimplify object semantics through one-hot class encodings (e.g., 'chair' or 'table'), require masked diffusion for editing, ignore room boundaries, or rely on floor plan renderings that fail to capture complex layouts. In contrast, LLM-based methods enable richer semantics via natural language (e.g., 'modern studio with light wood furniture') but do not support editing, remain limited to rectangular layouts or rely on weak spatial reasoning from implicit world models. We introduce ReSpace, a generative framework for text-driven 3D indoor scene synthesis and editing using autoregressive language models. Our approach features a compact structured scene representation with explicit room boundaries that frames scene editing as a next-token prediction task. We leverage a dual-stage training approach combining supervised fine-tuning and preference alignment, enabling a specially trained language model for object addition that accounts for user instructions, spatial geometry, object semantics, and scene-level composition. For scene editing, we employ a zero-shot LLM to handle object removal and prompts for addition. We further introduce a novel voxelization-based evaluation that captures fine-grained geometry beyond 3D bounding boxes. Experimental results surpass state-of-the-art on object addition while maintaining competitive results on full scene synthesis.
• 摘要：场景综合和编辑已成为计算机图形方向的有希望的方向。当前的3D室内场景的训练方法要么通过一式式类别的编码（例如“椅子”或“桌子”）过度简化对象语义，需要掩盖扩散以进行编辑，忽略房间边界或依靠无法捕获复杂布局的平面图。相比之下，基于LLM的方法可以通过自然语言（例如，“带有轻型木制家具的现代工作室”），但不支持编辑，不仅限于矩形布局或依赖于隐式世界模型的弱空间推理。我们介绍了Respace，这是一种用于文本驱动的3D室内场景综合的生成框架，并使用自回归语言模型进行编辑。我们的方法具有紧凑的结构化场景表示形式，具有明确的房间边界，将场景编辑作为下一步的预测任务。我们利用一种双阶段训练方法结合了监督的微调和偏好对齐方式，为对象添加提供了专门训练的语言模型，以说明用户说明，空间几何，对象语义和场景级别的组成。对于场景编辑，我们采用零射门LLM来处理对象删除并提示添加。我们进一步介绍了一种新型的基于体素化的评估，该评估捕获了3D边界框以外的细粒几何形状。实验结果超过了对象添加的最新结果，同时保持完整场景合成的竞争结果。

Title: Generative Perception of Shape and Material from Differential Motion

• Authors: Xinran Nicole Han, Ko Nishino, Todd Zickler
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02473
• Pdf URL: https://arxiv.org/pdf/2506.02473
• Copy Paste: [[2506.02473]] Generative Perception of Shape and Material from Differential Motion(https://arxiv.org/abs/2506.02473)
• Keywords: generative
• Abstract: Perceiving the shape and material of an object from a single image is inherently ambiguous, especially when lighting is unknown and unconstrained. Despite this, humans can often disentangle shape and material, and when they are uncertain, they often move their head slightly or rotate the object to help resolve the ambiguities. Inspired by this behavior, we introduce a novel conditional denoising-diffusion model that generates samples of shape-and-material maps from a short video of an object undergoing differential motions. Our parameter-efficient architecture allows training directly in pixel-space, and it generates many disentangled attributes of an object simultaneously. Trained on a modest number of synthetic object-motion videos with supervision on shape and material, the model exhibits compelling emergent behavior: For static observations, it produces diverse, multimodal predictions of plausible shape-and-material maps that capture the inherent ambiguities; and when objects move, the distributions quickly converge to more accurate explanations. The model also produces high-quality shape-and-material estimates for less ambiguous, real-world objects. By moving beyond single-view to continuous motion observations, our work suggests a generative perception approach for improving visual reasoning in physically-embodied systems.
• 摘要：从单个图像中感知物体的形状和材料本质上是模棱两可的，尤其是当照明未知且不受限时。尽管如此，人类通常可以消除形状和材料，当它们不确定时，它们通常会稍微移动头部或旋转物体以帮助解决歧义。受这种行为的启发，我们引入了一种新型的条件denoisis-扩散模型，该模型从一个简短的视频中生成了形状和物质图的样本，该视频的视频是进行差分运动的对象。我们的参数有效体系结构允许直接在像素空间中训练，并且同时生成了对象的许多分离属性。该模型经过适度数量的合成对象运动视频，并具有对形状和材料的监督，该模型表现出令人信服的紧急行为：对于静态观察，它产生了多种模式的形状和物质图，以捕获固有的歧义；当对象移动时，分布迅速收敛于更准确的解释。该模型还产生高质量的形状和物质估计值，以减少模棱两可的现实世界对象。通过超越单视图到连续运动观察，我们的工作提出了一种生成的感知方法，用于改善物理体积系统中的视觉推理。

Title: Towards Better De-raining Generalization via Rainy Characteristics Memorization and Replay

• Authors: Kunyu Wang, Xueyang Fu, Chengzhi Cao, Chengjie Ge, Wei Zhai, Zheng-Jun Zha
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02477
• Pdf URL: https://arxiv.org/pdf/2506.02477
• Copy Paste: [[2506.02477]] Towards Better De-raining Generalization via Rainy Characteristics Memorization and Replay(https://arxiv.org/abs/2506.02477)
• Keywords: generative
• Abstract: Current image de-raining methods primarily learn from a limited dataset, leading to inadequate performance in varied real-world rainy conditions. To tackle this, we introduce a new framework that enables networks to progressively expand their de-raining knowledge base by tapping into a growing pool of datasets, significantly boosting their adaptability. Drawing inspiration from the human brain's ability to continuously absorb and generalize from ongoing experiences, our approach borrow the mechanism of the complementary learning system. Specifically, we first deploy Generative Adversarial Networks (GANs) to capture and retain the unique features of new data, mirroring the hippocampus's role in learning and memory. Then, the de-raining network is trained with both existing and GAN-synthesized data, mimicking the process of hippocampal replay and interleaved learning. Furthermore, we employ knowledge distillation with the replayed data to replicate the synergy between the neocortex's activity patterns triggered by hippocampal replays and the pre-existing neocortical knowledge. This comprehensive framework empowers the de-raining network to amass knowledge from various datasets, continually enhancing its performance on previously unseen rainy scenes. Our testing on three benchmark de-raining networks confirms the framework's effectiveness. It not only facilitates continuous knowledge accumulation across six datasets but also surpasses state-of-the-art methods in generalizing to new real-world scenarios.
• 摘要：当前的图像去伤害方法主要从有限的数据集中学习，从而导致现实世界中多种雨天的性能不足。为了解决这个问题，我们引入了一个新的框架，该框架使网络能够通过利用越来越多的数据集来逐步扩大其脱颖而出的知识库，从而大大提高了其适应性。我们的方法从人类大脑不断吸收和概括的经验中汲取灵感，我们的方法借用了互补学习系统的机制。具体来说，我们首先部署生成对抗网络（GAN）来捕获并保留新数据的独特功能，从而反映海马在学习和记忆中的作用。然后，对割伤网络进行了现有和GAN合成的数据训练，从而模仿了海马重播和交错学习的过程。此外，我们使用重播数据采用知识蒸馏来复制新皮层的活动模式与海马重播触发的活动模式与现有的新皮质知识触发的。这个综合的框架赋予了De Rain Network的能力，可以从各种数据集中积累知识，从而在以前看不见的下雨场景上不断提高其性能。我们对三个基准DERAINT网络的测试证实了该框架的有效性。它不仅促进了六个数据集的持续知识积累，而且还超过了对新的现实世界情景的最新方法。

Title: Flexiffusion: Training-Free Segment-Wise Neural Architecture Search for Efficient Diffusion Models

• Authors: Hongtao Huang, Xiaojun Chang, Lina Yao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.02488
• Pdf URL: https://arxiv.org/pdf/2506.02488
• Copy Paste: [[2506.02488]] Flexiffusion: Training-Free Segment-Wise Neural Architecture Search for Efficient Diffusion Models(https://arxiv.org/abs/2506.02488)
• Keywords: generation, generative
• Abstract: Diffusion models (DMs) are powerful generative models capable of producing high-fidelity images but are constrained by high computational costs due to iterative multi-step inference. While Neural Architecture Search (NAS) can optimize DMs, existing methods are hindered by retraining requirements, exponential search complexity from step-wise optimization, and slow evaluation relying on massive image generation. To address these challenges, we propose Flexiffusion, a training-free NAS framework that jointly optimizes generation schedules and model architectures without modifying pre-trained parameters. Our key insight is to decompose the generation process into flexible segments of equal length, where each segment dynamically combines three step types: full (complete computation), partial (cache-reused computation), and null (skipped computation). This segment-wise search space reduces the candidate pool exponentially compared to step-wise NAS while preserving architectural diversity. Further, we introduce relative FID (rFID), a lightweight evaluation metric for NAS that measures divergence from a teacher model's outputs instead of ground truth, slashing evaluation time by over $90\%$. In practice, Flexiffusion achieves at least $2\times$ acceleration across LDMs, Stable Diffusion, and DDPMs on ImageNet and MS-COCO, with FID degradation under $5\%$, outperforming prior NAS and caching methods. Notably, it attains $5.1\times$ speedup on Stable Diffusion with near-identical CLIP scores. Our work pioneers a resource-efficient paradigm for searching high-speed DMs without sacrificing quality.
• 摘要：扩散模型（DMS）是能够产生高保真图像的强大生成模型，但由于迭代性多步推断而受到高计算成本的约束。尽管神经体系结构搜索（NAS）可以优化DMS，但现有方法受到重新培训要求，逐步优化的指数搜索复杂性以及依赖大量图像生成的缓慢评估而受到阻碍。为了应对这些挑战，我们提出了Flexiffusion，这是一个无训练的NAS框架，可以共同优化生成时间表和模型体系结构，而无需修改预训练的参数。我们的关键见解是将生成过程分解为相等长度的灵活段，其中每个段都动态结合了三个步骤类型：完整（完整的计算），部分（CACHE REUSE REUSE COMPUTITION）和NAULL（跳过计算）。与逐步的NA相比，该细分市场的搜索空间在保留建筑多样性的同时，将候选人池成倍减少。此外，我们介绍了相对FID（RFID），这是NAS轻量级评估指标，可衡量与教师模型的输出而不是地面真相的分歧，将评估时间削减超过$ 90 \％\％$。实际上，Flexiffusion在ImageNet和MS-Coco上跨LDM的加速度至少达到$ 2 \ times $加速度，并且在$ 5 \％下的FID降解均优于先前的NAS和CACHING方法。值得注意的是，它的稳定扩散率$ 5.1 \ times $ speedup，并获得了近相同的夹得分。我们的工作开创了一个资源有效的范式，用于搜索高速DM，而无需牺牲质量。

Title: LumosFlow: Motion-Guided Long Video Generation

• Authors: Jiahao Chen, Hangjie Yuan, Yichen Qian, Jingyun Liang, Jiazheng Xing, Pengwei Liu, Weihua Chen, Fan Wang, Bing Su
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02497
• Pdf URL: https://arxiv.org/pdf/2506.02497
• Copy Paste: [[2506.02497]] LumosFlow: Motion-Guided Long Video Generation(https://arxiv.org/abs/2506.02497)
• Keywords: generation
• Abstract: Long video generation has gained increasing attention due to its widespread applications in fields such as entertainment and simulation. Despite advances, synthesizing temporally coherent and visually compelling long sequences remains a formidable challenge. Conventional approaches often synthesize long videos by sequentially generating and concatenating short clips, or generating key frames and then interpolate the intermediate frames in a hierarchical manner. However, both of them still remain significant challenges, leading to issues such as temporal repetition or unnatural transitions. In this paper, we revisit the hierarchical long video generation pipeline and introduce LumosFlow, a framework introduce motion guidance explicitly. Specifically, we first employ the Large Motion Text-to-Video Diffusion Model (LMTV-DM) to generate key frames with larger motion intervals, thereby ensuring content diversity in the generated long videos. Given the complexity of interpolating contextual transitions between key frames, we further decompose the intermediate frame interpolation into motion generation and post-hoc refinement. For each pair of key frames, the Latent Optical Flow Diffusion Model (LOF-DM) synthesizes complex and large-motion optical flows, while MotionControlNet subsequently refines the warped results to enhance quality and guide intermediate frame generation. Compared with traditional video frame interpolation, we achieve 15x interpolation, ensuring reasonable and continuous motion between adjacent frames. Experiments show that our method can generate long videos with consistent motion and appearance. Code and models will be made publicly available upon acceptance. Our project page: this https URL
• 摘要：由于其在娱乐和模拟等领域的广泛应用，长期的视频发电引起了人们的关注。尽管有进步，但在时间上连贯和视觉上引人注目的长序列仍然是一个巨大的挑战。传统的方法通常通过顺序生成和串联短剪辑或生成关键帧，然后以层次结构方式插入中间帧，从而综合长视频。但是，他们俩仍然仍然面临重大挑战，导致诸如时间重复或不自然的过渡等问题。在本文中，我们重新访问了分层长的视频生成管道并引入LumoSflow，一个框架明确介绍了运动指导。具体而言，我们首先采用大型动作文本到视频扩散模型（LMTV-DM）来生成具有较大运动间隔的关键帧，从而确保生成的长视频中的内容多样性。鉴于关键帧之间插值的上下文过渡的复杂性，我们将中间框架插值进一步分解为运动产生和事后改进。对于每对关键帧，潜在的光流扩散模型（LOF-DM）综合了复合和大动态光流，而MotionControlnet随后完善了扭曲的结果，以提高质量并引导中间框架的生成。与传统的视频框架插值相比，我们实现了15倍的插值，确保相邻帧之间合理且连续的运动。实验表明，我们的方法可以以一致的运动和外观生成长视频。接受后，代码和模型将在接受后公开可用。我们的项目页面：此HTTPS URL

Title: RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers

• Authors: Yan Gong, Yiren Song, Yicheng Li, Chenglin Li, Yin Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02528
• Pdf URL: https://arxiv.org/pdf/2506.02528
• Copy Paste: [[2506.02528]] RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers(https://arxiv.org/abs/2506.02528)
• Keywords: generation
• Abstract: Inspired by the in-context learning mechanism of large language models (LLMs), a new paradigm of generalizable visual prompt-based image editing is emerging. Existing single-reference methods typically focus on style or appearance adjustments and struggle with non-rigid transformations. To address these limitations, we propose leveraging source-target image pairs to extract and transfer content-aware editing intent to novel query images. To this end, we introduce RelationAdapter, a lightweight module that enables Diffusion Transformer (DiT) based models to effectively capture and apply visual transformations from minimal examples. We also introduce Relation252K, a comprehensive dataset comprising 218 diverse editing tasks, to evaluate model generalization and adaptability in visual prompt-driven scenarios. Experiments on Relation252K show that RelationAdapter significantly improves the model's ability to understand and transfer editing intent, leading to notable gains in generation quality and overall editing performance.
• 摘要：受到大语言模型（LLMS）的文化学习机制的启发，正在出现了可推广的基于视觉及时的图像编辑的新范式。现有的单参考方法通常集中于样式或外观调整以及与非刚性转换的斗争。为了解决这些限制，我们建议利用源目标图对提取和将内容感知的编辑意图提取到新颖的查询图像。为此，我们引入了RelationAdapter，这是一个轻巧的模块，可实现基于扩散变压器（DIT）模型，以有效地捕获和应用最小示例的视觉变换。我们还介绍了Relation252K，这是一个包含218个不同编辑任务的综合数据集，以评估视觉迅速驱动的方案中的模型概括和适应性。 Relation252K上的实验表明，RelationAdapter显着提高了模型理解和传输编辑意图的能力，从而导致发电质量和整体编辑性能的显着提高。

Title: Rethinking Post-Unlearning Behavior of Large Vision-Language Models

• Authors: Minsung Kim, Nakyeong Yang, Kyomin Jung
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02541
• Pdf URL: https://arxiv.org/pdf/2506.02541
• Copy Paste: [[2506.02541]] Rethinking Post-Unlearning Behavior of Large Vision-Language Models(https://arxiv.org/abs/2506.02541)
• Keywords: generative
• Abstract: Machine unlearning is used to mitigate the privacy risks of Large Vision-Language Models (LVLMs) arising from training on large-scale web data. However, existing unlearning methods often fail to carefully select substitute outputs for forget targets, resulting in Unlearning Aftermaths-undesirable behaviors such as degenerate, hallucinated, or excessively refused responses. We highlight that, especially for generative LVLMs, it is crucial to consider the quality and informativeness of post-unlearning responses rather than relying solely on naive suppression. To address this, we introduce a new unlearning task for LVLMs that requires models to provide privacy-preserving yet informative and visually grounded responses. We also propose PUBG, a novel unlearning method that explicitly guides post-unlearning behavior toward a desirable output distribution. Experiments show that, while existing methods suffer from Unlearning Aftermaths despite successfully preventing privacy violations, PUBG effectively mitigates these issues, generating visually grounded and informative responses without privacy leakage for forgotten targets.
• 摘要：Machine Unerning用于减轻大型网络数据培训引起的大视觉模型（LVLM）的隐私风险。但是，现有的未学习方法通​​常无法仔细地为忘记目标选择替代输出，从而导致不可避免的可调性行为，例如退化，幻觉或过度拒绝的响应。我们强调的是，特别是对于生成的LVLM，考虑到未检验后反应的质量和信息性至关重要，而不是仅仅依靠天真的抑制。为了解决这个问题，我们为LVLM介绍了一项新的未学习任务，该任务需要模型提供隐私权且内容丰富且视觉上扎根的响应。我们还提出了PUBG，这是一种新颖的学习方法，该方法明确指导了未学习后的行为，以实现理想的输出分布。实验表明，尽管现有方法尽管成功防止了侵犯隐私行为，但现有方法却遭受了艰难的后果，但PUBG有效地减轻了这些问题，产生了视觉扎根和信息丰富的响应，而不会为被遗忘的目标泄漏隐私泄漏。

Title: Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective

• Authors: Shenghua He, Tian Xia, Xuan Zhou, Hui Wei
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.02553
• Pdf URL: https://arxiv.org/pdf/2506.02553
• Copy Paste: [[2506.02553]] Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective(https://arxiv.org/abs/2506.02553)
• Keywords: generation
• Abstract: We study a common challenge in reinforcement learning for large language models (LLMs): the Zero-Reward Assumption, where non-terminal actions (i.e., intermediate token generations) receive zero task-specific immediate reward, while only the final token receives a reward for the entire response. This assumption arises frequently in practice, as precise token-level rewards are often difficult or infeasible to obtain in LLM applications. In this work, we provide a unifying theoretical perspective. We introduce the Trajectory Policy Gradient Theorem, which shows that the policy gradient based on true, unknown token-level rewards can be unbiasedly estimated using only a response-level reward model, regardless of whether the Zero-Reward Assumption holds or not, for algorithms in the REINFORCE and Actor-Critic families. This result reveals that widely used methods such as PPO, GRPO, ReMax, and RLOO inherently possess the capacity to model token-level reward signals, offering a theoretical justification for response-level reward approaches. Our findings pave the way for more practical, efficient LLM fine-tuning, allowing developers to treat training algorithms as black boxes and focus on improving the response-level reward model with auxiliary sub-models. We also offer a detailed analysis of popular RL and non-RL methods, comparing their theoretical foundations and practical advantages across common LLM tasks. Finally, we propose a new algorithm: Token-Reinforced Policy Optimization (TRePO), a theoretically grounded method that is simpler than PPO, matches GRPO in memory efficiency, and holds promise for broad applicability.
• 摘要：我们研究了大语模型（LLMS）的强化学习方面的共同挑战：零奖励假设，其中非末端动作（即中级代币世代）获得零任务特定的即时奖励，而最终代币只能获得整个响应的奖励。在实践中，经常出现此假设，因为在LLM应用程序中通常很难或不可行。在这项工作中，我们提供了一个统一的理论观点。我们介绍了轨迹策略梯度定理，该定理表明，基于真实的，未知令牌级别的奖励的策略梯度只能使用响应级别的奖励模型来公正地估算，无论零奖励假设是否成立，对于增强和参与者中的算法是否具有零奖励假设。该结果表明，PPO，GRPO，Remax和Rloo等广泛使用的方法固有地具有对令牌级奖励信号进行建模的能力，为响应级别的奖励方法提供了理论上的理由。我们的发现为更实用，有效的LLM微调铺平了道路，使开发人员能够将培训算法视为黑匣子，并专注于使用辅助子模型改善响应级别奖励模型。我们还对流行的RL和非RL方法进行了详细的分析，比较了他们在常见的LLM任务中的理论基础和实际优势。最后，我们提出了一种新的算法：令牌 - 增强策略优化（TREPO），这是一种比PPO更简单的理论扎根方法，与GRPO匹配内存效率，并保持了广泛适用性的希望。

Title: DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing

• Authors: Zixiang Li, Haoyu Wang, Wei Wang, Chuangchuang Tan, Yunchao Wei, Yao Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02560
• Pdf URL: https://arxiv.org/pdf/2506.02560
• Copy Paste: [[2506.02560]] DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing(https://arxiv.org/abs/2506.02560)
• Keywords: generation
• Abstract: Diffusion models have achieved remarkable success in image generation and editing tasks. Inversion within these models aims to recover the latent noise representation for a real or generated image, enabling reconstruction, editing, and other downstream tasks. However, to date, most inversion approaches suffer from an intrinsic trade-off between reconstruction accuracy and editing flexibility. This limitation arises from the difficulty of maintaining both semantic alignment and structural consistency during the inversion process. In this work, we introduce Dual-Conditional Inversion (DCI), a novel framework that jointly conditions on the source prompt and reference image to guide the inversion process. Specifically, DCI formulates the inversion process as a dual-condition fixed-point optimization problem, minimizing both the latent noise gap and the reconstruction error under the joint guidance. This design anchors the inversion trajectory in both semantic and visual space, leading to more accurate and editable latent representations. Our novel setup brings new understanding to the inversion process. Extensive experiments demonstrate that DCI achieves state-of-the-art performance across multiple editing tasks, significantly improving both reconstruction quality and editing precision. Furthermore, we also demonstrate that our method achieves strong results in reconstruction tasks, implying a degree of robustness and generalizability approaching the ultimate goal of the inversion process.
• 摘要：扩散模型在图像生成和编辑任务中取得了巨大的成功。这些模型中的倒置旨在为真实或生成的图像恢复潜在的噪声表示形式，从而启用重建，编辑和其他下游任务。但是，迄今为止，大多数反转方法都在重建准确性和编辑灵活性之间存在固有的权衡。这种局限性源于在反转过程中保持语义一致性和结构一致性的困难。在这项工作中，我们引入了双条件倒置（DCI），这是一个新颖的框架，可以在源提示和参考图像上共同调节以指导反转过程。具体而言，DCI将反转过程作为双条件定点优化问题制定，从而最大程度地减少了关节指导下的潜在噪声差距和重建误差。该设计锚定了语义和视觉空间中的反转轨迹，从而导致更准确和可编辑的潜在表示。我们的小说设置为反转过程带来了新的理解。广泛的实验表明，DCI在多个编辑任务中实现了最先进的性能，从而显着提高了重建质量和编辑精度。此外，我们还证明了我们的方法在重建任务中取得了强大的成果，这意味着一定程度的鲁棒性和可推广性接近反演过程的最终目标。

Title: Hyperspectral Image Generation with Unmixing Guided Diffusion Model

• Authors: Shiyu Shen, Bin Pan, Ziye Zhang, Zhenwei Shi
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.02601
• Pdf URL: https://arxiv.org/pdf/2506.02601
• Copy Paste: [[2506.02601]] Hyperspectral Image Generation with Unmixing Guided Diffusion Model(https://arxiv.org/abs/2506.02601)
• Keywords: generation, generative
• Abstract: Recently, hyperspectral image generation has received increasing attention, but existing generative models rely on conditional generation schemes, which limits the diversity of generated images. Diffusion models are popular for their ability to generate high-quality samples, but adapting these models from RGB to hyperspectral data presents the challenge of high dimensionality and physical constraints. To address these challenges, we propose a novel diffusion model guided by hyperspectral unmixing. Our model comprises two key modules: an unmixing autoencoder module and an abundance diffusion module. The unmixing autoencoder module leverages unmixing guidance to shift the generative task from the image space to the low-dimensional abundance space, significantly reducing computational complexity while preserving high fidelity. The abundance diffusion module generates samples that satisfy the constraints of non-negativity and unity, ensuring the physical consistency of the reconstructed HSIs. Additionally, we introduce two evaluation metrics tailored to hyperspectral data. Empirical results, evaluated using both traditional metrics and our proposed metrics, indicate that our model is capable of generating high-quality and diverse hyperspectral images, offering an advancement in hyperspectral data generation.
• 摘要：最近，高光谱图像产生受到了越来越多的关注，但是现有的生成模型依赖于有条件的生成方案，这限制了生成图像的多样性。扩散模型因其产生高质量样本的能力而受欢迎，但是将这些模型从RGB调整为高光谱数据表现出了高维度和物理约束的挑战。为了应对这些挑战，我们提出了一个新型扩散模型，该模型由高光谱脉冲引导。我们的模型包括两个关键模块：一个Unmixing自动编码器模块和一个丰度扩散模块。 Unmixing自动编码器模块利用Unmixing Guidance将生成任务从图像空间转移到低维丰度空间，从而大大降低了计算复杂性，同时保留了高保真度。丰度扩散模块生成满足非负和统一约束的样品，从而确保重建的HSIS的物理一致性。此外，我们介绍了针对高光谱数据量身定制的两个评估指标。使用传统指标和我们提出的指标评估的经验结果表明，我们的模型能够产生高质量和多样化的高光谱图像，从而提高了高光谱数据的发展。

Title: Application of convolutional neural networks in image super-resolution

• Authors: Tian Chunwei, Song Mingjian, Zuo Wangmeng, Du Bo, Zhang Yanning, Zhang Shichao
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.02604
• Pdf URL: https://arxiv.org/pdf/2506.02604
• Copy Paste: [[2506.02604]] Application of convolutional neural networks in image super-resolution(https://arxiv.org/abs/2506.02604)
• Keywords: super-resolution
• Abstract: Due to strong learning abilities of convolutional neural networks (CNNs), they have become mainstream methods for image super-resolution. However, there are big differences of different deep learning methods with different types. There is little literature to summarize relations and differences of different methods in image super-resolution. Thus, summarizing these literatures are important, according to loading capacity and execution speed of devices. This paper first introduces principles of CNNs in image super-resolution, then introduces CNNs based bicubic interpolation, nearest neighbor interpolation, bilinear interpolation, transposed convolution, sub-pixel layer, meta up-sampling for image super-resolution to analyze differences and relations of different CNNs based interpolations and modules, and compare performance of these methods by experiments. Finally, this paper gives potential research points and drawbacks and summarizes the whole paper, which can facilitate developments of CNNs in image super-resolution.
• 摘要：由于卷积神经网络（CNN）的强大学习能力，它们已成为图像超分辨率的主流方法。但是，不同类型的不同深度学习方法存在很大的差异。几乎没有文献来总结图像超分辨率中不同方法的关系和差异。因此，根据设备的加载能力和执行速度，总结这些文献很重要。本文首先在图像超分辨率中介绍了CNN的原理，然后引入基于CNN的双色插值，最近的邻居插值，双线性插值，转置卷积，子像素层，元像素层，元更新以分析基于CNNS的互动和模型的差异和模式，以分析图像超级分辨率，以分析这些方法和模式。最后，本文提供了潜在的研究点和缺点，并总结了整个论文，这可以促进图像超分辨率中CNN的发展。

Title: One-Step Diffusion-based Real-World Image Super-Resolution with Visual Perception Distillation

• Authors: Xue Wu, Jingwei Xin, Zhijun Tu, Jie Hu, Jie Li, Nannan Wang, Xinbo Gao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02605
• Pdf URL: https://arxiv.org/pdf/2506.02605
• Copy Paste: [[2506.02605]] One-Step Diffusion-based Real-World Image Super-Resolution with Visual Perception Distillation(https://arxiv.org/abs/2506.02605)
• Keywords: super-resolution, generation
• Abstract: Diffusion-based models have been widely used in various visual generation tasks, showing promising results in image super-resolution (SR), while typically being limited by dozens or even hundreds of sampling steps. Although existing methods aim to accelerate the inference speed of multi-step diffusion-based SR methods through knowledge distillation, their generated images exhibit insufficient semantic alignment with real images, resulting in suboptimal perceptual quality reconstruction, specifically reflected in the CLIPIQA score. These methods still have many challenges in perceptual quality and semantic fidelity. Based on the challenges, we propose VPD-SR, a novel visual perception diffusion distillation framework specifically designed for SR, aiming to construct an effective and efficient one-step SR model. Specifically, VPD-SR consists of two components: Explicit Semantic-aware Supervision (ESS) and High-Frequency Perception (HFP) loss. Firstly, the ESS leverages the powerful visual perceptual understanding capabilities of the CLIP model to extract explicit semantic supervision, thereby enhancing semantic consistency. Then, Considering that high-frequency information contributes to the visual perception quality of images, in addition to the vanilla distillation loss, the HFP loss guides the student model to restore the missing high-frequency details in degraded images that are critical for enhancing perceptual quality. Lastly, we expand VPD-SR in adversarial training manner to further enhance the authenticity of the generated content. Extensive experiments conducted on synthetic and real-world datasets demonstrate that the proposed VPD-SR achieves superior performance compared to both previous state-of-the-art methods and the teacher model with just one-step sampling.
• 摘要：基于扩散的模型已被广泛用于各种视觉生成任务，显示出令人鼓舞的图像超分辨率（SR）的结果，而通常受到数十个甚至数百个采样步骤的限制。尽管现有的方法旨在通过知识蒸馏加速基于多步扩散的SR方法的推理速度，但其生成的图像与真实图像的语义对齐不足，从而导致次优质量重建，并在ClipiQA评分中进行了专门反映。这些方法在感知质量和语义忠诚方面仍然面临许多挑战。基于挑战，我们提出了VPD-SR，这是一种专门为SR设计的新型视觉感知扩散蒸馏框架，旨在构建有效有效的一步SR模型。具体而言，VPD-SR由两个组成部分组成：显式语义感知监督（ESS）和高频感知（HFP）损失。首先，ESS利用了剪辑模型的强大视觉感知理解能力提取明确的语义监督，从而增强了语义一致性。然后，考虑到高频信息有助于图像的视觉感知质量，除了香草蒸馏损失之外，HFP损失指导学生模型在降低的图像中恢复缺失的高频细节，这对于增强感知质量至关重要。最后，我们以对抗性训练方式扩展VPD-SR，以进一步增强生成内容的真实性。对合成和现实世界数据集进行的广泛实验表明，所提出的VPD-SR与以前的最新方法和教师模型相比，仅使用一步采样而获得了优越的性能。

Title: Synthetic Iris Image Databases and Identity Leakage: Risks and Mitigation Strategies

• Authors: Ada Sawilska, Mateusz Trokielewicz
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02626
• Pdf URL: https://arxiv.org/pdf/2506.02626
• Copy Paste: [[2506.02626]] Synthetic Iris Image Databases and Identity Leakage: Risks and Mitigation Strategies(https://arxiv.org/abs/2506.02626)
• Keywords: generation, generative
• Abstract: This paper presents a comprehensive overview of iris image synthesis methods, which can alleviate the issues associated with gathering large, diverse datasets of biometric data from living individuals, which are considered pivotal for biometric methods development. These methods for synthesizing iris data range from traditional, hand crafted image processing-based techniques, through various iterations of GAN-based image generators, variational autoencoders (VAEs), as well as diffusion models. The potential and fidelity in iris image generation of each method is discussed and examples of inferred predictions are provided. Furthermore, the risks of individual biometric features leakage from the training sets are considered, together with possible strategies for preventing them, which have to be implemented should these generative methods be considered a valid replacement of real-world biometric datasets.
• 摘要：本文介绍了虹膜图像合成方法的全面概述，这些方法可以减轻与从活着的个体收集大量的生物识别数据数据集相关的问题，这被认为是生物识别方法开发的关键。这些用于合成虹膜数据的方法范围从传统的，手工制作的基于图像处理的技术到基于GAN的图像发生器，变量自动编码器（VAE）以及扩散模型的各种迭代。讨论了每种方法的虹膜图像产生的潜力和忠诚度，并提供了推断预测的示例。此外，考虑到训练集中泄漏的单个生物特征特征的风险，以及防止它们的可能策略，如果这些生成方法被认为是对现实世界中生物识别数据集的有效替换，则必须实现。

Title: ControlMambaIR: Conditional Controls with State-Space Model for Image Restoration

• Authors: Cheng Yang, Lijing Liang, Zhixun Su
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02633
• Pdf URL: https://arxiv.org/pdf/2506.02633
• Copy Paste: [[2506.02633]] ControlMambaIR: Conditional Controls with State-Space Model for Image Restoration(https://arxiv.org/abs/2506.02633)
• Keywords: restoration, generation
• Abstract: This paper proposes ControlMambaIR, a novel image restoration method designed to address perceptual challenges in image deraining, deblurring, and denoising tasks. By integrating the Mamba network architecture with the diffusion model, the condition network achieves refined conditional control, thereby enhancing the control and optimization of the image generation process. To evaluate the robustness and generalization capability of our method across various image degradation conditions, extensive experiments were conducted on several benchmark datasets, including Rain100H, Rain100L, GoPro, and SSID. The results demonstrate that our proposed approach consistently surpasses existing methods in perceptual quality metrics, such as LPIPS and FID, while maintaining comparable performance in image distortion metrics, including PSNR and SSIM, highlighting its effectiveness and adaptability. Notably, ablation experiments reveal that directly noise prediction in the diffusion process achieves better performance, effectively balancing noise suppression and detail preservation. Furthermore, the findings indicate that the Mamba architecture is particularly well-suited as a conditional control network for diffusion models, outperforming both CNN- and Attention-based approaches in this context. Overall, these results highlight the flexibility and effectiveness of ControlMambaIR in addressing a range of image restoration perceptual challenges.
• 摘要：本文提出了ControlMambair，这是一种新型的图像恢复方法，旨在解决图像der，脱张和变性任务中的感知挑战。通过将MAMBA网络体系结构与扩散模型集成，条件网络实现了精致的条件控制，从而增强了图像生成过程的控制和优化。为了评估我们方法在各种图像退化条件下的鲁棒性和概括能力，在几个基准数据集上进行了广泛的实验，包括Rain 100H，Rain100L，GoPro和SSID。结果表明，我们提出的方法始终超过感知质量指标（例如LPIPS和FID）中的现有方法，同时保持图像失真指标（包括PSNR和SSIM）的可比性，突出了其有效性和适应性。值得注意的是，消融实验表明，扩散过程中的直接噪声预测可实现更好的性能，有效地平衡了抑制噪声和细节保存。此外，研究结果表明，Mamba体系结构特别适合作为扩散模型的条件控制网络，在这种情况下表现优于CNN和基于注意力的方法。总体而言，这些结果突出了ControlMambair在应对一系列图像恢复感知挑战方面的灵活性和有效性。

Title: Solving Inverse Problems with FLAIR

• Authors: Julius Erbach, Dominik Narnhofer, Andreas Dombos, Bernt Schiele, Jan Eric Lenssen, Konrad Schindler
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.02680
• Pdf URL: https://arxiv.org/pdf/2506.02680
• Copy Paste: [[2506.02680]] Solving Inverse Problems with FLAIR(https://arxiv.org/abs/2506.02680)
• Keywords: generation, generative
• Abstract: Flow-based latent generative models such as Stable Diffusion 3 are able to generate images with remarkable quality, even enabling photorealistic text-to-image generation. Their impressive performance suggests that these models should also constitute powerful priors for inverse imaging problems, but that approach has not yet led to comparable fidelity. There are several key obstacles: (i) the encoding into a lower-dimensional latent space makes the underlying (forward) mapping non-linear; (ii) the data likelihood term is usually intractable; and (iii) learned generative models struggle to recover rare, atypical data modes during inference. We present FLAIR, a novel training free variational framework that leverages flow-based generative models as a prior for inverse problems. To that end, we introduce a variational objective for flow matching that is agnostic to the type of degradation, and combine it with deterministic trajectory adjustments to recover atypical modes. To enforce exact consistency with the observed data, we decouple the optimization of the data fidelity and regularization terms. Moreover, we introduce a time-dependent calibration scheme in which the strength of the regularization is modulated according to off-line accuracy estimates. Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
• 摘要：基于流动的潜在生成模型（例如稳定扩散3）能够以质量出色，甚至可以产生逼真的文本对图像生成。他们令人印象深刻的性能表明，这些模型也应构成反向成像问题的强大先验，但是这种方法尚未导致可比的保真度。有几个关键的障碍：（i）编码为较低维的潜在空间使基础（前向）映射非线性； （ii）数据可能性期限通常是棘手的； （iii）学到的生成模型在推断期间努力恢复稀有的，非典型的数据模式。我们提出了一种新型培训的自由变分框架，该框架将基于流量的生成模型作为逆问题的先验。为此，我们引入了一个流动匹配的变分目标，该目标对降解的类型不可知，并将其与确定性轨迹调整结合起来，以恢复非典型模式。为了与观察到的数据确切一致性，我们将数据保真度和正则化项的优化分发。此外，我们引入了一个时间依赖性的校准方案，在该方案中，根据离线精度估计值调节正则化的强度。标准成像基准的结果表明，就重建质量和样品多样性而言，天赋始终优于现有的扩散和基于流动的方法。

Title: Towards Geometry Problem Solving in the Large Model Era: A Survey

• Authors: Yurui Zhao, Xiang Wang, Jiahong Liu, Irwin King, Zhitao Huang
• Subjects: cs.CV, math.GT
• Abstract URL: https://arxiv.org/abs/2506.02690
• Pdf URL: https://arxiv.org/pdf/2506.02690
• Copy Paste: [[2506.02690]] Towards Geometry Problem Solving in the Large Model Era: A Survey(https://arxiv.org/abs/2506.02690)
• Keywords: generation
• Abstract: Geometry problem solving (GPS) represents a critical frontier in artificial intelligence, with profound applications in education, computer-aided design, and computational graphics. Despite its significance, automating GPS remains challenging due to the dual demands of spatial understanding and rigorous logical reasoning. Recent advances in large models have enabled notable breakthroughs, particularly for SAT-level problems, yet the field remains fragmented across methodologies, benchmarks, and evaluation frameworks. This survey systematically synthesizes GPS advancements through three core dimensions: (1) benchmark construction, (2) textual and diagrammatic parsing, and (3) reasoning paradigms. We further propose a unified analytical paradigm, assess current limitations, and identify emerging opportunities to guide future research toward human-level geometric reasoning, including automated benchmark generation and interpretable neuro-symbolic integration.
• 摘要：几何问题解决（GPS）代表了人工智能中的关键边界，在教育，计算机辅助设计和计算图形中采用了深刻的应用。尽管具有重要意义，但由于空间理解和严格的逻辑推理的双重要求，自动化的GP仍然具有挑战性。大型模型的最新进展已引发了显着的突破，尤其是对于SAT级问题，但是该领域在方法论，基准和评估框架之间仍然存在分散。这项调查通过三个核心维度系统地综合了GPS的进步：（1）基准构造，（2）文本和图表解析，以及（3）推理范式。我们进一步提出了一个统一的分析范式，评估当前的局限性，并确定新兴的机会，以指导未来的研究对人类水平的几何推理，包括自动基准产生和可解释的神经符号符号整合。

Title: LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generation

• Authors: Yuxuan Wu, Le Wang, Sanping Zhou, Mengnan Liu, Gang Hua, Haoxiang Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02697
• Pdf URL: https://arxiv.org/pdf/2506.02697
• Copy Paste: [[2506.02697]] LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generation(https://arxiv.org/abs/2506.02697)
• Keywords: generation
• Abstract: Controllable layout generation aims to create plausible visual arrangements of element bounding boxes within a graphic design according to certain optional constraints, such as the type or position of a specific component. While recent diffusion or flow-matching models have achieved considerable advances in multifarious conditional generation tasks, there remains considerable room for generating optimal arrangements under given conditions. In this work, we propose to carry out layout generation through retrieving by conditions and reference-guided generation. Specifically, we retrieve appropriate layout templates according to given conditions as references. The references are then utilized to guide the denoising or flow-based transport process. By retrieving layouts compatible with the given conditions, we can uncover the potential information not explicitly provided in the given condition. Such an approach offers more effective guidance to the model during the generation process, in contrast to previous models that feed the condition to the model and let the model infer the unprovided layout attributes directly. Meanwhile, we design a condition-modulated attention that selectively absorbs retrieval knowledge, adapting to the difference between retrieved templates and given conditions. Extensive experiment results show that our method successfully produces high-quality layouts that meet the given conditions and outperforms existing state-of-the-art models. Code will be released upon acceptance.
• 摘要：可控的布局生成旨在根据某些可选约束（例如特定组件的类型或位置）在图形设计中创建元素边界框的合理视觉布置。尽管最近的扩散或流动匹配模型在多种有条件生成任务方面取得了很大的进步，但在给定条件下，仍有相当大的空间来产生最佳布置。在这项工作中，我们建议通过通过条件和参考引导生成检索来进行布局生成。具体而言，我们根据给定条件作为参考文献检索适当的布局模板。然后，将参考文献用于指导基于流动或基于流动的运输过程。通过检索与给定条件兼容的布局，我们可以发现在给定条件下未明确提供的潜在信息。这种方法在生成过程中为模型提供了更有效的指导，与以前的模型相反，该模型将条件馈送到模型并让模型直接推断未经证实的布局属性。同时，我们设计了一种条件调节的注意力，可选择性地吸收检索知识，适应检索模板和给定条件之间的差异。广泛的实验结果表明，我们的方法成功地产生了符合给定条件的高质量布局，并且胜过现有的最新模型。代码将在接受后发布。

Title: Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences

• Authors: Yunhong Lu, Qichao Wang, Hengyuan Cao, Xiaoyin Xu, Min Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02698
• Pdf URL: https://arxiv.org/pdf/2506.02698
• Copy Paste: [[2506.02698]] Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences(https://arxiv.org/abs/2506.02698)
• Keywords: generation
• Abstract: Direct Preference Optimization (DPO) aligns text-to-image (T2I) generation models with human preferences using pairwise preference data. Although substantial resources are expended in collecting and labeling datasets, a critical aspect is often neglected: \textit{preferences vary across individuals and should be represented with more granularity.} To address this, we propose SmPO-Diffusion, a novel method for modeling preference distributions to improve the DPO objective, along with a numerical upper bound estimation for the diffusion optimization objective. First, we introduce a smoothed preference distribution to replace the original binary distribution. We employ a reward model to simulate human preferences and apply preference likelihood averaging to improve the DPO loss, such that the loss function approaches zero when preferences are similar. Furthermore, we utilize an inversion technique to simulate the trajectory preference distribution of the diffusion model, enabling more accurate alignment with the optimization objective. Our approach effectively mitigates issues of excessive optimization and objective misalignment present in existing methods through straightforward modifications. Our SmPO-Diffusion achieves state-of-the-art performance in preference evaluation, outperforming baselines across metrics with lower training costs. The project page is this https URL.
• 摘要：直接优先优化（DPO）使用成对偏好数据将文本对图像（T2I）生成模型与人类偏好相结合。尽管在收集和标记数据集时花费了大量资源，但通常会忽略一个关键方面：\ textIt {偏好{偏好在各个个人中都有不同的代表，并且应更详细地表示。}为了解决这个问题，我们提出了SMPO扩散，一种新的方法，是一种建模偏好目标的新方法，以改善DPO目标，并提高了数值的上限估计，以实现优化型号的优化差异。首先，我们引入了平滑的偏好分布，以替换原始的二进制分布。我们采用奖励模型来模拟人类的偏好并应用偏好可能性平均以改善DPO损失，从而在偏好相似时损失函数接近零。此外，我们利用一种反转技术来模拟扩散模型的轨迹偏好分布，从而使能够与优化目标更准确地对齐。我们的方法有效地减轻了通过直接修改现有方法中过度优化和客观未对准的问题。我们的SMPO扩散在偏好评估方面取得了最先进的表现，在培训成本较低的指标之间表现优于基线。项目页面是此HTTPS URL。

Title: ToothForge: Automatic Dental Shape Generation using Synchronized Spectral Embeddings

• Authors: Tibor Kubík, François Guibault, Michal Španěl, Hervé Lombaert
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02702
• Pdf URL: https://arxiv.org/pdf/2506.02702
• Copy Paste: [[2506.02702]] ToothForge: Automatic Dental Shape Generation using Synchronized Spectral Embeddings(https://arxiv.org/abs/2506.02702)
• Keywords: generation
• Abstract: We introduce ToothForge, a spectral approach for automatically generating novel 3D teeth, effectively addressing the sparsity of dental shape datasets. By operating in the spectral domain, our method enables compact machine learning modeling, allowing the generation of high-resolution tooth meshes in milliseconds. However, generating shape spectra comes with the instability of the decomposed harmonics. To address this, we propose modeling the latent manifold on synchronized frequential embeddings. Spectra of all data samples are aligned to a common basis prior to the training procedure, effectively eliminating biases introduced by the decomposition instability. Furthermore, synchronized modeling removes the limiting factor imposed by previous methods, which require all shapes to share a common fixed connectivity. Using a private dataset of real dental crowns, we observe a greater reconstruction quality of the synthetized shapes, exceeding those of models trained on unaligned embeddings. We also explore additional applications of spectral analysis in digital dentistry, such as shape compression and interpolation. ToothForge facilitates a range of approaches at the intersection of spectral analysis and machine learning, with fewer restrictions on mesh structure. This makes it applicable for shape analysis not only in dentistry, but also in broader medical applications, where guaranteeing consistent connectivity across shapes from various clinics is unrealistic. The code is available at this https URL.
• 摘要：我们引入了Toothforge，这是一种自动生成新型3D牙齿的光谱方法，有效地解决了牙齿形状数据集的稀疏性。通过在光谱域中操作，我们的方法可以实现紧凑的机器学习建模，从而可以以毫秒为单位的高分辨率齿晶体。但是，产生的形状光谱是分解谐波的不稳定性。为了解决这个问题，我们建议在同步频繁嵌入上建模潜在的歧管。在训练程序之前，将所有数据样本的光谱对齐与共同基础，有效地消除了分解不稳定性引入的偏差。此外，同步建模消除了先前方法所施加的限制因子，这需要所有形状共享共同的固定连接。使用真实牙冠的私人数据集，我们观察到合成形状的重建质量更高，超过了在未对准的嵌入式上训练的模型的形状。我们还探讨了光谱分析在数字牙科中的其他应用，例如形状压缩和插值。 Toothforge在光谱分析和机器学习的交集中有助于范围的方法，对网格结构的限制较少。这使得它不仅适用于牙科分析，而且适用于更广泛的医学应用，在这些应用程序中，可以保证各种诊所的形状持续连通性是不现实的。该代码可在此HTTPS URL上找到。

Title: LinkTo-Anime: A 2D Animation Optical Flow Dataset from 3D Model Rendering

• Authors: Xiaoyi Feng, Kaifeng Zou, Caichun Cen, Tao Huang, Hui Guo, Zizhou Huang, Yingli Zhao, Mingqing Zhang, Diwei Wang, Yuntao Zou, Dagang Li
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.02733
• Pdf URL: https://arxiv.org/pdf/2506.02733
• Copy Paste: [[2506.02733]] LinkTo-Anime: A 2D Animation Optical Flow Dataset from 3D Model Rendering(https://arxiv.org/abs/2506.02733)
• Keywords: generation
• Abstract: Existing optical flow datasets focus primarily on real-world simulation or synthetic human motion, but few are tailored to Celluloid(cel) anime character motion: a domain with unique visual and motion characteristics. To bridge this gap and facilitate research in optical flow estimation and downstream tasks such as anime video generation and line drawing colorization, we introduce LinkTo-Anime, the first high-quality dataset specifically designed for cel anime character motion generated with 3D model rendering. LinkTo-Anime provides rich annotations including forward and backward optical flow, occlusion masks, and Mixamo Skeleton. The dataset comprises 395 video sequences, totally 24,230 training frames, 720 validation frames, and 4,320 test frames. Furthermore, a comprehensive benchmark is constructed with various optical flow estimation methods to analyze the shortcomings and limitations across multiple datasets.
• 摘要：现有的光流数据集主要集中在现实世界模拟或合成人类运动上，但很少针对赛璐oid（CEL）动漫角色运动量身定制：具有独特的视觉和运动特征的域。为了弥合这一差距，并促进了光流估计和下游任务（例如动漫视频生成和线绘制着色）的研究，我们介绍了Linkto Anime，这是第一个专门为CEL动漫角色运动设计的高质量数据集，该数据集由3D模型渲染产生。 Linkto-anime提供了丰富的注释，包括前进和后向光流，遮挡口罩和Mixamo骨架。该数据集包含395个视频序列，完全24,230架培训帧，720个验证帧和4,320个测试框架。此外，采用各种光流估计方法构建了全面的基准测试，以分析多个数据集的缺点和局限性。

Title: FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Prompts

• Authors: Tongyuan Bai, Wangyuanfan Bai, Dong Chen, Tieru Wu, Manyi Li, Rui Ma
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02781
• Pdf URL: https://arxiv.org/pdf/2506.02781
• Copy Paste: [[2506.02781]] FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Prompts(https://arxiv.org/abs/2506.02781)
• Keywords: generation
• Abstract: Controllability plays a crucial role in the practical applications of 3D indoor scene synthesis. Existing works either allow rough language-based control, that is convenient but lacks fine-grained scene customization, or employ graph based control, which offers better controllability but demands considerable knowledge for the cumbersome graph design process. To address these challenges, we present FreeScene, a user-friendly framework that enables both convenient and effective control for indoor scene this http URL, FreeScene supports free-form user inputs including text description and/or reference images, allowing users to express versatile design intentions. The user inputs are adequately analyzed and integrated into a graph representation by a VLM-based Graph Designer. We then propose MG-DiT, a Mixed Graph Diffusion Transformer, which performs graph-aware denoising to enhance scene generation. Our MG-DiT not only excels at preserving graph structure but also offers broad applicability to various tasks, including, but not limited to, text-to-scene, graph-to-scene, and rearrangement, all within a single model. Extensive experiments demonstrate that FreeScene provides an efficient and user-friendly solution that unifies text-based and graph based scene synthesis, outperforming state-of-the-art methods in terms of both generation quality and controllability in a range of applications.
• 摘要：可控性在3D室内场景合成的实际应用中起着至关重要的作用。现有的作品要么允许基于语言的粗略控制，这很方便，但缺乏细粒度的自定义，或者采用基于图形的控制，该控制能力更好，但对繁琐的图形设计过程需要大量知识。为了应对这些挑战，我们提出了FreeScene，这是一个用户友好的框架，可以为室内场景提供方便且有效的控制此HTTP URL，Freescene支持自由形式的用户输入，包括文本说明和/或参考图像，允许用户表达多功能设计意图。用户输入通过基于VLM的图形设计器对用户输入进行了充分的分析并集成到图表表示中。然后，我们提出了MG-DIT，这是一种混合图扩散变压器，它可以执行图形感知的DeNoising，以增强场景的生成。我们的MG-DIT不仅擅长保存图形结构，而且还为各种任务提供了广泛的适用性，包括但不限于文本对场景，绘制到场景和重新排列，都在单个模型中。广泛的实验表明，Freescene提供了一种有效且用户友好的解决方案，该解决方案统一了基于文本和图形的场景综合，在一系列应用程序中的发电质量和可控性方面都优于最先进的方法。

Title: CART-based Synthetic Tabular Data Generation for Imbalanced Regression

• Authors: António Pedro Pinheiro, Rita P. Ribeiro
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.02811
• Pdf URL: https://arxiv.org/pdf/2506.02811
• Copy Paste: [[2506.02811]] CART-based Synthetic Tabular Data Generation for Imbalanced Regression(https://arxiv.org/abs/2506.02811)
• Keywords: generation, generative
• Abstract: Handling imbalanced target distributions in regression tasks remains a significant challenge in tabular data settings where underrepresented regions can hinder model performance. Among data-level solutions, some proposals, such as random sampling and SMOTE-based approaches, propose adapting classification techniques to regression tasks. However, these methods typically rely on crisp, artificial thresholds over the target variable, a limitation inherited from classification settings that can introduce arbitrariness, often leading to non-intuitive and potentially misleading problem formulations. While recent generative models, such as GANs and VAEs, provide flexible sample synthesis, they come with high computational costs and limited interpretability. In this study, we propose adapting an existing CART-based synthetic data generation method, tailoring it for imbalanced regression. The new method integrates relevance and density-based mechanisms to guide sampling in sparse regions of the target space and employs a threshold-free, feature-driven generation process. Our experimental study focuses on the prediction of extreme target values across benchmark datasets. The results indicate that the proposed method is competitive with other resampling and generative strategies in terms of performance, while offering faster execution and greater transparency. These results highlight the method's potential as a transparent, scalable data-level strategy for improving regression models in imbalanced domains.
• 摘要：在回归任务中处理不平衡的目标分布仍然是表格数据设置的重大挑战，在该设置中，代表性不足的区域可能会阻碍模型性能。在数据级解决方案中，一些建议，例如随机抽样和基于SMOTE的方法，提出了对回归任务的适应分类技术。但是，这些方法通常依赖于清晰的目标变量上的人造阈值，这是可以从分类设置中继承的限制，这些设置可以引入任意性，通常会导致非直觉且潜在的误导性问题。虽然最近的生成模型（例如gan和vaes）提供了灵活的样品合成，但它们具有高计算成本和有限的解释性。在这项研究中，我们提出了适应现有的基于卡车的合成数据生成方法，并将其定制为不平衡的回归。新方法集成了相关性和基于密度的机制，以指导目标空间稀疏区域的采样，并采用无阈值，特征驱动的生成过程。我们的实验研究重点是对基准数据集对极端目标值的预测。结果表明，所提出的方法在绩效方面与其他重采样和生成策略具有竞争力，同时提供更快的执行和更高的透明度。这些结果突出了该方法是一种透明，可扩展的数据级策略，用于改善不平衡域中的回归模型。

Title: PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors

• Authors: Yujin Chen, Yinyu Nie, Benjamin Ummenhofer, Reiner Birkl, Michael Paulitsch, Matthias Nießner
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02846
• Pdf URL: https://arxiv.org/pdf/2506.02846
• Copy Paste: [[2506.02846]] PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors(https://arxiv.org/abs/2506.02846)
• Keywords: super-resolution
• Abstract: We present PBR-SR, a novel method for physically based rendering (PBR) texture super resolution (SR). It outputs high-resolution, high-quality PBR textures from low-resolution (LR) PBR input in a zero-shot manner. PBR-SR leverages an off-the-shelf super-resolution model trained on natural images, and iteratively minimizes the deviations between super-resolution priors and differentiable renderings. These enhancements are then back-projected into the PBR map space in a differentiable manner to produce refined, high-resolution textures. To mitigate view inconsistencies and lighting sensitivity, which is common in view-based super-resolution, our method applies 2D prior constraints across multi-view renderings, iteratively refining the shared, upscaled textures. In parallel, we incorporate identity constraints directly in the PBR texture domain to ensure the upscaled textures remain faithful to the LR input. PBR-SR operates without any additional training or data requirements, relying entirely on pretrained image priors. We demonstrate that our approach produces high-fidelity PBR textures for both artist-designed and AI-generated meshes, outperforming both direct SR models application and prior texture optimization methods. Our results show high-quality outputs in both PBR and rendering evaluations, supporting advanced applications such as relighting.
• 摘要：我们提出了PBR-SR，这是一种基于物理的渲染（PBR）纹理超级分辨率（SR）的新方法。它以零拍的方式从低分辨率（LR）PBR输入中输出高分辨率的高质量PBR纹理。 PBR-SR利用了训练有自然图像的现成的超分辨率模型，并迭代地最大程度地减少了超分辨率先验和可区分效果图之间的偏差。然后以可区分的方式将这些增强功能重新投影到PBR地图空间中，以产生精致的高分辨率纹理。为了减轻视图不一致和照明灵敏度，这在基于视图的超分辨率中很常见，我们的方法在多视图渲染中应用了2D先前的约束，迭代地完善了共享的，高尺度的纹理。同时，我们将身份约束直接合并到PBR纹理域中，以确保升级的纹理仍然忠实于LR输入。 PBR-SR完全依赖于验证的图像先验，无需任何其他培训或数据要求。我们证明我们的方法为艺术家设计和AI生成的网格产生高保真性PBR纹理，表现优于直接SR模型应用程序和先前的纹理优化方法。我们的结果表明，PBR和渲染评估中都有高质量的产出，支持了重新启动等高级应用程序。

Title: Enhancing Abnormality Identification: Robust Out-of-Distribution Strategies for Deepfake Detection

• Authors: Luca Maiano, Fabrizio Casadei, Irene Amerini
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02857
• Pdf URL: https://arxiv.org/pdf/2506.02857
• Copy Paste: [[2506.02857]] Enhancing Abnormality Identification: Robust Out-of-Distribution Strategies for Deepfake Detection(https://arxiv.org/abs/2506.02857)
• Keywords: generative
• Abstract: Detecting deepfakes has become a critical challenge in Computer Vision and Artificial Intelligence. Despite significant progress in detection techniques, generalizing them to open-set scenarios continues to be a persistent difficulty. Neural networks are often trained on the closed-world assumption, but with new generative models constantly evolving, it is inevitable to encounter data generated by models that are not part of the training distribution. To address these challenges, in this paper, we propose two novel Out-Of-Distribution (OOD) detection approaches. The first approach is trained to reconstruct the input image, while the second incorporates an attention mechanism for detecting OODs. Our experiments validate the effectiveness of the proposed approaches compared to existing state-of-the-art techniques. Our method achieves promising results in deepfake detection and ranks among the top-performing configurations on the benchmark, demonstrating their potential for robust, adaptable solutions in dynamic, real-world applications.
• 摘要：检测深击已成为计算机视觉和人工智能中的关键挑战。尽管检测技术取得了重大进展，但将它们推广到开放式场景仍然是一个持续的困难。神经网络经常受到封闭世界假设的训练，但是随着新的生成模型不断发展，不可避免地会遇到不属于训练分布的模型生成的数据。为了应对这些挑战，在本文中，我们提出了两种新颖的分布（OOD）检测方法。第一种方法是训练以重建输入图像的，而第二种方法结合了检测OOD的注意机制。与现有的最新技术相比，我们的实验验证了拟议方法的有效性。我们的方法在深层检测中获得了有希望的结果，并在基准测试中表现出色的配置中排名，这证明了它们在动态，现实世界中的强大，适应性解决方案的潜力。

Title: NTIRE 2025 XGC Quality Assessment Challenge: Methods and Results

• Authors: Xiaohong Liu, Xiongkuo Min, Qiang Hu, Xiaoyun Zhang, Jie Guo, Guangtao Zhai, Shushi Wang, Yingjie Zhou, Lu Liu, Jingxin Li, Liu Yang, Farong Wen, Li Xu, Yanwei Jiang, Xilei Zhu, Chunyi Li, Zicheng Zhang, Huiyu Duan, Xiele Wu, Yixuan Gao, Yuqin Cao, Jun Jia, Wei Sun, Jiezhang Cao, Radu Timofte, Baojun Li, Jiamian Huang, Dan Luo, Tao Liu, Weixia Zhang, Bingkun Zheng, Junlin Chen, Ruikai Zhou, Meiya Chen, Yu Wang, Hao Jiang, Xiantao Li, Yuxiang Jiang, Jun Tang, Yimeng Zhao, Bo Hu, Zelu Qi, Chaoyang Zhang, Fei Zhao, Ping Shi, Lingzhi Fu, Heng Cong, Shuai He, Rongyu Zhang, Jiarong He, Zongyao Hu, Wei Luo, Zihao Yu, Fengbin Guan, Yiting Lu, Xin Li, Zhibo Chen, Mengjing Su, Yi Wang, Tuo Chen, Chunxiao Li, Shuaiyu Zhao, Jiaxin Wen, Chuyi Lin, Sitong Liu, Ningxin Chu, Jing Wan, Yu Zhou, Baoying Chen, Jishen Zeng, Jiarui Liu, Xianjin Liu, Xin Chen, Lanzhi Zhou, Hangyu Li, You Han, Bibo Xiang, Zhenjie Liu, Jianzhang Lu, Jialin Gui, Renjie Lu, Shangfei Wang, Donghao Zhou, Jingyu Lin, Quanjian Song, Jiancheng Huang, Yufeng Yang, Changwei Wang, Shupeng Zhong, Yang Yang, Lihuo He, Jia Liu, Yuting Xing, Tida Fang, Yuchun Jin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02875
• Pdf URL: https://arxiv.org/pdf/2506.02875
• Copy Paste: [[2506.02875]] NTIRE 2025 XGC Quality Assessment Challenge: Methods and Results(https://arxiv.org/abs/2506.02875)
• Keywords: restoration, quality assessment
• Abstract: This paper reports on the NTIRE 2025 XGC Quality Assessment Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2025. This challenge is to address a major challenge in the field of video and talking head processing. The challenge is divided into three tracks, including user generated video, AI generated video and talking head. The user-generated video track uses the FineVD-GC, which contains 6,284 user generated videos. The user-generated video track has a total of 125 registered participants. A total of 242 submissions are received in the development phase, and 136 submissions are received in the test phase. Finally, 5 participating teams submitted their models and fact sheets. The AI generated video track uses the Q-Eval-Video, which contains 34,029 AI-Generated Videos (AIGVs) generated by 11 popular Text-to-Video (T2V) models. A total of 133 participants have registered in this track. A total of 396 submissions are received in the development phase, and 226 submissions are received in the test phase. Finally, 6 participating teams submitted their models and fact sheets. The talking head track uses the THQA-NTIRE, which contains 12,247 2D and 3D talking heads. A total of 89 participants have registered in this track. A total of 225 submissions are received in the development phase, and 118 submissions are received in the test phase. Finally, 8 participating teams submitted their models and fact sheets. Each participating team in every track has proposed a method that outperforms the baseline, which has contributed to the development of fields in three tracks.
• 摘要：本文报告了NTIRE 2025 XGC质量评估挑战，该挑战将与CVPR 2025的图像恢复和增强研讨会（NTIRE）的新趋势结合在一起。这一挑战是应对视频领域的重大挑战和交谈。挑战分为三个曲目，包括用户生成的视频，AI生成的视频和谈话头。用户生成的视频轨道使用FineVD-GC，其中包含6,284个用户生成的视频。用户生成的视频轨道共有125名注册参与者。在开发阶段总共收到了242份提交，在测试阶段收到了136份提交。最后，5个参与的团队提交了他们的模型和事实表。 AI生成的视频轨道使用Q-eval-Video，其中包含由11种流行的文本对视频（T2V）模型生成的34,029个AI生成的视频（AIGV）。该曲目中共有133名参与者注册。在开发阶段，总共收到了396份提交，在测试阶段收到了226份提交。最后，有6个参与的团队提交了他们的模型和事实表。会说话的头部轨道使用THQA-NTIRE，其中包含12,247 2D和3D会话头。在此曲目中共有89名参与者注册。在开发阶段总共收到了225份提交，并在测试阶段收到118项提交。最后，有8个参与的团队提交了他们的模型和事实表。每个曲目中的每个参与团队都提出了一种胜过基线的方法，这有助于三个轨道的田野发展。

Title: MIND: Material Interface Generation from UDFs for Non-Manifold Surface Reconstruction

• Authors: Xuhui Chen, Fei Hou, Wencheng Wang, Hong Qin, Ying He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02938
• Pdf URL: https://arxiv.org/pdf/2506.02938
• Copy Paste: [[2506.02938]] MIND: Material Interface Generation from UDFs for Non-Manifold Surface Reconstruction(https://arxiv.org/abs/2506.02938)
• Keywords: generation
• Abstract: Unsigned distance fields (UDFs) are widely used in 3D deep learning due to their ability to represent shapes with arbitrary topology. While prior work has largely focused on learning UDFs from point clouds or multi-view images, extracting meshes from UDFs remains challenging, as the learned fields rarely attain exact zero distances. A common workaround is to reconstruct signed distance fields (SDFs) locally from UDFs to enable surface extraction via Marching Cubes. However, this often introduces topological artifacts such as holes or spurious components. Moreover, local SDFs are inherently incapable of representing non-manifold geometry, leading to complete failure in such cases. To address this gap, we propose MIND (Material Interface from Non-manifold Distance fields), a novel algorithm for generating material interfaces directly from UDFs, enabling non-manifold mesh extraction from a global perspective. The core of our method lies in deriving a meaningful spatial partitioning from the UDF, where the target surface emerges as the interface between distinct regions. We begin by computing a two-signed local field to distinguish the two sides of manifold patches, and then extend this to a multi-labeled global field capable of separating all sides of a non-manifold structure. By combining this multi-labeled field with the input UDF, we construct material interfaces that support non-manifold mesh extraction via a multi-labeled Marching Cubes algorithm. Extensive experiments on UDFs generated from diverse data sources, including point cloud reconstruction, multi-view reconstruction, and medial axis transforms, demonstrate that our approach robustly handles complex non-manifold surfaces and significantly outperforms existing methods.
• 摘要：未签名的距离字段（UDF）在3D深度学习中被广泛使用，因为它们具有任意拓扑的形状。虽然先前的工作主要集中在从点云或多视图图像中学习UDF，但从UDF中提取网格仍然具有挑战性，因为学习的字段很少获得确切的零距离。一个常见的解决方法是从UDF重建签名的距离字段（SDF），以通过行进立方体进行表面提取。但是，这通常会引入拓扑伪像，例如孔或虚假成分。此外，局部SDF本质上无能力代表非字母的几何形状，导致在这种情况下完全失败。为了解决这一差距，我们提出了思维（来自非模型距离字段的材料接口），这是一种直接从UDF生成材料接口的新型算法，从全球角度启用了非Manifold网格提取。我们方法的核心在于从UDF得出有意义的空间分区，在该空间分区中，目标表面作为不同区域之间的界面出现。我们首先要计算一个两符号的本地场来区分歧管斑块的两个侧面，然后将其扩展到一个多标记的全局场，能够分离非字母结构的所有侧面。通过将这个多标签的字段与输入UDF相结合，我们构建了通过多标记的行进立方体算法来支持非Manifold网格提取的材料接口。对来自不同数据源产生的UDF的广泛实验，包括点云重建，多视图重建和内侧轴的变化，表明我们的方法可以鲁棒处理复杂的非字母表面，并且显着胜过现有方法。

Title: Abstract Counterfactuals for Language Model Agents

• Authors: Edoardo Pona, Milad Kazemi, Yali Du, David Watson, Nicola Paoletti
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.02946
• Pdf URL: https://arxiv.org/pdf/2506.02946
• Copy Paste: [[2506.02946]] Abstract Counterfactuals for Language Model Agents(https://arxiv.org/abs/2506.02946)
• Keywords: generation
• Abstract: Counterfactual inference is a powerful tool for analysing and evaluating autonomous agents, but its application to language model (LM) agents remains challenging. Existing work on counterfactuals in LMs has primarily focused on token-level counterfactuals, which are often inadequate for LM agents due to their open-ended action spaces. Unlike traditional agents with fixed, clearly defined action spaces, the actions of LM agents are often implicit in the strings they output, making their action spaces difficult to define and interpret. Furthermore, the meanings of individual tokens can shift depending on the context, adding complexity to token-level reasoning and sometimes leading to biased or meaningless counterfactuals. We introduce \emph{Abstract Counterfactuals}, a framework that emphasises high-level characteristics of actions and interactions within an environment, enabling counterfactual reasoning tailored to user-relevant features. Our experiments demonstrate that the approach produces consistent and meaningful counterfactuals while minimising the undesired side effects of token-level methods. We conduct experiments on text-based games and counterfactual text generation, while considering both token-level and latent-space interventions.
• 摘要：反事实推断是分析和评估自主代理的强大工具，但其在语言模型（LM）代理中的应用仍然具有挑战性。现有在LMS中反事实的工作主要集中在令牌级的反事实上，由于其开放式动作空间，LM代理通常不足。与具有固定，明确定义的动作空间的传统代理不同，LM代理的作用通常是在输出的字符串中隐含的，这使得其动作空间难以定义和解释。此外，单个代币的含义可能会根据上下文而变化，从而增加了令牌级别的推理，有时会导致偏见或毫无意义的反事实。我们介绍\ emph {抽象反事实}，该框架强调环境中的动作和交互的高级特征，从而实现了针对用户相关的功能量身定制的反事实推理。我们的实验表明，该方法会产生一致且有意义的反事实，同时最大程度地减少了令牌级方法的不需要的副作用。我们对基于文本的游戏和反事实文本生成进行实验，同时考虑令牌级别和潜在空间干预措施。

Title: Interaction Field Matching: Overcoming Limitations of Electrostatic Models

• Authors: Stepan I. Manukhov, Alexander Kolesov, Vladimir V. Palyulin, Alexander Korotin
• Subjects: cs.LG, cs.AI, cs.CV
• Abstract URL: https://arxiv.org/abs/2506.02950
• Pdf URL: https://arxiv.org/pdf/2506.02950
• Copy Paste: [[2506.02950]] Interaction Field Matching: Overcoming Limitations of Electrostatic Models(https://arxiv.org/abs/2506.02950)
• Keywords: generation
• Abstract: Electrostatic field matching (EFM) has recently appeared as a novel physics-inspired paradigm for data generation and transfer using the idea of an electric capacitor. However, it requires modeling electrostatic fields using neural networks, which is non-trivial because of the necessity to take into account the complex field outside the capacitor plates. In this paper, we propose Interaction Field Matching (IFM), a generalization of EFM which allows using general interaction fields beyond the electrostatic one. Furthermore, inspired by strong interactions between quarks and antiquarks in physics, we design a particular interaction field realization which solves the problems which arise when modeling electrostatic fields in EFM. We show the performance on a series of toy and image data transfer problems.
• 摘要：静电场匹配（EFM）最近是一种新型物理启发的范式，用于使用电容器的想法进行数据生成和传输。但是，它需要使用神经网络对静电场进行建模，这是非平凡的，因为有必要考虑到电容器板外的复杂场。在本文中，我们提出了相互作用场匹配（IFM），EFM的概括允许使用静电量之外的一般相互作用场。此外，我们灵感来自物理学中夸克与古怪之间的强烈相互作用，我们设计了一种特定的相互作用场实现，该场介绍了在对EFM中的静电场进行建模时出现的问题。我们显示了一系列玩具和图像数据传输问题的性能。

Title: HaploOmni: Unified Single Transformer for Multimodal Video Understanding and Generation

• Authors: Yicheng Xiao, Lin Song, Rui Yang, Cheng Cheng, Zunnan Xu, Zhaoyang Zhang, Yixiao Ge, Xiu Li, Ying Shan
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.02975
• Pdf URL: https://arxiv.org/pdf/2506.02975
• Copy Paste: [[2506.02975]] HaploOmni: Unified Single Transformer for Multimodal Video Understanding and Generation(https://arxiv.org/abs/2506.02975)
• Keywords: generation
• Abstract: With the advancement of language models, unified multimodal understanding and generation have made significant strides, with model architectures evolving from separated components to unified single-model frameworks. This paper explores an efficient training paradigm to build a single transformer for unified multimodal understanding and generation. Specifically, we propose a multimodal warmup strategy utilizing prior knowledge to extend capabilities. To address cross-modal compatibility challenges, we introduce feature pre-scaling and multimodal AdaLN techniques. Integrating the proposed technologies, we present the HaploOmni, a new single multimodal transformer. With limited training costs, HaploOmni achieves competitive performance across multiple image and video understanding and generation benchmarks over advanced unified models. All codes will be made public at this https URL.
• 摘要：随着语言模型的发展，统一的多模式理解和生成取得了长足的进步，模型体系结构从分离的组件演变为统一的单模框架。本文探讨了有效的训练范式，以建立一个单一的变压器，以统一多模式理解和产生。具体而言，我们提出了一种利用先验知识来扩展能力的多模式热身策略。为了应对跨模式兼容性挑战，我们引入了特征前缩放和多模式Adaln技术。整合了提出的技术，我们提出了一种新的单模式变压器Haploomni。凭借有限的培训成本，Haploomni在高级统一模型上实现了多个图像，视频理解和生成基准的竞争性能。所有代码将在此HTTPS URL上公开。

Title: Deep Learning for Retinal Degeneration Assessment: A Comprehensive Analysis of the MARIO AMD Progression Challenge

• Authors: Rachid Zeghlache, Ikram Brahim, Pierre-Henri Conze, Mathieu Lamard, Mohammed El Amine Lazouni, Zineb Aziza Elaouaber, Leila Ryma Lazouni, Christopher Nielsen, Ahmad O. Ahsan, Matthias Wilms, Nils D. Forkert, Lovre Antonio Budimir, Ivana Matovinović, Donik Vršnak, Sven Lončarić, Philippe Zhang, Weili Jiang, Yihao Li, Yiding Hao, Markus Frohmann, Patrick Binder, Marcel Huber, Taha Emre, Teresa Finisterra Araújo, Marzieh Oghbaie, Hrvoje Bogunović, Amerens A. Bekkers, Nina M. van Liebergen, Hugo J. Kuijf, Abdul Qayyum, Moona Mazher, Steven A. Niederer, Alberto J. Beltrán-Carrero, Juan J. Gómez-Valverde, Javier Torresano-Rodríquez, Álvaro Caballero-Sastre, María J. Ledesma Carbayo, Yosuke Yamagishi, Yi Ding, Robin Peretzke, Alexandra Ertl, Maximilian Fischer, Jessica Kächele, Sofiane Zehar, Karim Boukli Hacene, Thomas Monfort, Béatrice Cochener, Mostafa El Habib Daho, Anas-Alexis Benyoussef, Gwenolé Quellec
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.02976
• Pdf URL: https://arxiv.org/pdf/2506.02976
• Copy Paste: [[2506.02976]] Deep Learning for Retinal Degeneration Assessment: A Comprehensive Analysis of the MARIO AMD Progression Challenge(https://arxiv.org/abs/2506.02976)
• Keywords: generation
• Abstract: The MARIO challenge, held at MICCAI 2024, focused on advancing the automated detection and monitoring of age-related macular degeneration (AMD) through the analysis of optical coherence tomography (OCT) images. Designed to evaluate algorithmic performance in detecting neovascular activity changes within AMD, the challenge incorporated unique multi-modal datasets. The primary dataset, sourced from Brest, France, was used by participating teams to train and test their models. The final ranking was determined based on performance on this dataset. An auxiliary dataset from Algeria was used post-challenge to evaluate population and device shifts from submitted solutions. Two tasks were involved in the MARIO challenge. The first one was the classification of evolution between two consecutive 2D OCT B-scans. The second one was the prediction of future AMD evolution over three months for patients undergoing anti-vascular endothelial growth factor (VEGF) therapy. Thirty-five teams participated, with the top 12 finalists presenting their methods. This paper outlines the challenge's structure, tasks, data characteristics, and winning methodologies, setting a benchmark for AMD monitoring using OCT, infrared imaging, and clinical data (such as the number of visits, age, gender, etc.). The results of this challenge indicate that artificial intelligence (AI) performs as well as a physician in measuring AMD progression (Task 1) but is not yet able of predicting future evolution (Task 2).
• 摘要：在2024年Miccai举行的马里奥挑战赛致力于通过分析光学相干断层扫描（OCT）图像来推进与年龄相关的黄斑变性（AMD）的自动检测和监测。旨在评估算法性能在检测AMD内的新血管活动变化时，该挑战包含了独特的多模式数据集。参与团队使用来自法国布雷斯特的主要数据集来训练和测试他们的模型。最终排名是根据此数据集的性能确定的。利用阿尔及利亚的辅助数据集使用后挑战后来评估人口和设备从已提交的解决方案转移。马里奥挑战赛涉及两项任务。第一个是连续两个2d OCT B扫描之间的演变分类。第二个是对接受抗血管内皮生长因子（VEGF）治疗的患者进行三个月的未来AMD进化的预测。有35支球队参加了比赛，前12名决赛入围者介绍了他们的方法。本文概述了挑战的结构，任务，数据特征和获奖方法，为使用OCT，红外成像和临床数据（例如访问，年龄，性别等）设定了AMD监视的基准。这一挑战的结果表明，人工智能（AI）以及测量AMD进展的医生（任务1），但尚不能够预测未来的进化（任务2）。

Title: Astrophotography turbulence mitigation via generative models

• Authors: Joonyeoup Kim, Yu Yuan, Xingguang Zhang, Xijun Wang, Stanley Chan
• Subjects: cs.CV, eess.IV
• Abstract URL: https://arxiv.org/abs/2506.02981
• Pdf URL: https://arxiv.org/pdf/2506.02981
• Copy Paste: [[2506.02981]] Astrophotography turbulence mitigation via generative models(https://arxiv.org/abs/2506.02981)
• Keywords: restoration, generative
• Abstract: Photography is the cornerstone of modern astronomical and space research. However, most astronomical images captured by ground-based telescopes suffer from atmospheric turbulence, resulting in degraded imaging quality. While multi-frame strategies like lucky imaging can mitigate some effects, they involve intensive data acquisition and complex manual processing. In this paper, we propose AstroDiff, a generative restoration method that leverages both the high-quality generative priors and restoration capabilities of diffusion models to mitigate atmospheric turbulence. Extensive experiments demonstrate that AstroDiff outperforms existing state-of-the-art learning-based methods in astronomical image turbulence mitigation, providing higher perceptual quality and better structural fidelity under severe turbulence conditions. Our code and additional results are available at this https URL
• 摘要：摄影是现代天文和太空研究的基石。但是，大多数由地面望远镜捕获的天文图像都具有大气湍流，导致成像质量降解。尽管Lucky Imaging等多框架策略可以减轻某些效果，但它们涉及密集的数据获取和复杂的手动处理。在本文中，我们提出了Astrodiff，这是一种生成的恢复方法，它利用了扩散模型的高质量生成先验和恢复能力来减轻大气湍流。广泛的实验表明，Astrodiff在天文图像湍流缓解中的现有基于最新的学习方法优于现有的最新学习方法，从而在严重的湍流条件下提供了更高的感知质量和更好的结构保真度。我们的代码和其他结果可在此HTTPS URL上获得

Title: DFBench: Benchmarking Deepfake Image Detection Capability of Large Multimodal Models

• Authors: Jiarui Wang, Huiyu Duan, Juntong Wang, Ziheng Jia, Woo Yi Yang, Xiaorong Zhu, Yu Zhao, Jiaying Qian, Yuke Xing, Guangtao Zhai, Xiongkuo Min
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03007
• Pdf URL: https://arxiv.org/pdf/2506.03007
• Copy Paste: [[2506.03007]] DFBench: Benchmarking Deepfake Image Detection Capability of Large Multimodal Models(https://arxiv.org/abs/2506.03007)
• Keywords: generation, generative
• Abstract: With the rapid advancement of generative models, the realism of AI-generated images has significantly improved, posing critical challenges for verifying digital content authenticity. Current deepfake detection methods often depend on datasets with limited generation models and content diversity that fail to keep pace with the evolving complexity and increasing realism of the AI-generated content. Large multimodal models (LMMs), widely adopted in various vision tasks, have demonstrated strong zero-shot capabilities, yet their potential in deepfake detection remains largely unexplored. To bridge this gap, we present \textbf{DFBench}, a large-scale DeepFake Benchmark featuring (i) broad diversity, including 540,000 images across real, AI-edited, and AI-generated content, (ii) latest model, the fake images are generated by 12 state-of-the-art generation models, and (iii) bidirectional benchmarking and evaluating for both the detection accuracy of deepfake detectors and the evasion capability of generative models. Based on DFBench, we propose \textbf{MoA-DF}, Mixture of Agents for DeepFake detection, leveraging a combined probability strategy from multiple LMMs. MoA-DF achieves state-of-the-art performance, further proving the effectiveness of leveraging LMMs for deepfake detection. Database and codes are publicly available at this https URL.
• 摘要：随着生成模型的快速发展，AI生成的图像的现实主义已大大改善，对验证数字内容真实性提出了关键的挑战。当前的DeepFake检测方法通常取决于具有有限生成模型和内容多样性的数据集，这些数据集无法与不断发展的复杂性和不断发展的AI生成内容的现实主义保持同步。在各种视觉任务中广泛采用的大型多模型模型（LMM）表现出强大的零射击功能，但是它们在深层检测中的潜力仍未得到探索。 To bridge this gap, we present \textbf{DFBench}, a large-scale DeepFake Benchmark featuring (i) broad diversity, including 540,000 images across real, AI-edited, and AI-generated content, (ii) latest model, the fake images are generated by 12 state-of-the-art generation models, and (iii) bidirectional benchmarking and evaluating for both the detection accuracy of deepfake探测器和生成模型的逃避能力。基于dfbench，我们提出了\ textbf {MoA-df}，是用于深击检测的试剂的混合物，利用了来自多个LMM的组合概率策略。 MOA-DF实现了最先进的性能，进一步证明了利用LMM进行深泡剂检测的有效性。数据库和代码可在此HTTPS URL上公开可用。

Title: Sample complexity of Schrödinger potential estimation

• Authors: Nikita Puchkin, Iurii Pustovalov, Yuri Sapronov, Denis Suchkov, Alexey Naumov, Denis Belomestny
• Subjects: cs.LG, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.03043
• Pdf URL: https://arxiv.org/pdf/2506.03043
• Copy Paste: [[2506.03043]] Sample complexity of Schrödinger potential estimation(https://arxiv.org/abs/2506.03043)
• Keywords: generative
• Abstract: We address the problem of Schrödinger potential estimation, which plays a crucial role in modern generative modelling approaches based on Schrödinger bridges and stochastic optimal control for SDEs. Given a simple prior diffusion process, these methods search for a path between two given distributions $\rho_0$ and $\rho_T^*$ requiring minimal efforts. The optimal drift in this case can be expressed through a Schrödinger potential. In the present paper, we study generalization ability of an empirical Kullback-Leibler (KL) risk minimizer over a class of admissible log-potentials aimed at fitting the marginal distribution at time $T$. Under reasonable assumptions on the target distribution $\rho_T^*$ and the prior process, we derive a non-asymptotic high-probability upper bound on the KL-divergence between $\rho_T^*$ and the terminal density corresponding to the estimated log-potential. In particular, we show that the excess KL-risk may decrease as fast as $O(\log^2 n / n)$ when the sample size $n$ tends to infinity even if both $\rho_0$ and $\rho_T^*$ have unbounded supports.
• 摘要：我们解决了Schrödinger潜在估计的问题，该估计在基于SchrödingerRidges和SDES随机最佳控制的现代生成建模方法中起着至关重要的作用。给定一个简单的先前扩散过程，这些方法搜索两个给定分布之间的路径$ \ rho_0 $和$ \ rho_t^*$，需要最少的努力。在这种情况下，最佳漂移可以通过Schrödinger电位表示。在本文中，我们研究了一类可允许的日志潜能的经验kullback-leibler（KL）风险最小化的概括能力，旨在拟合时间$ t $的边际分布。在目标分布$ \ rho_t^*$的合理假设和先前的过程中，我们在$ \ rho_t^*$之间的KL-Divergence上得出了一个非反应性高概率上限和与估计的日志点相对应的端子密度。特别是，我们表明，即使样本大小$ n $倾向于无限，多余的KL风险可能会降低到$ O（\ log^2 n / n）$，即使$ \ rho_0 $和$ \ rho_t^*$都有无界的支持。

Title: Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers

• Authors: Pengtao Chen, Xianfang Zeng, Maosen Zhao, Peng Ye, Mingzhu Shen, Wei Cheng, Gang Yu, Tao Chen
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.03065
• Pdf URL: https://arxiv.org/pdf/2506.03065
• Copy Paste: [[2506.03065]] Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers(https://arxiv.org/abs/2506.03065)
• Keywords: generation
• Abstract: While Diffusion Transformers (DiTs) have achieved breakthroughs in video generation, this long sequence generation task remains constrained by the quadratic complexity of attention mechanisms, resulting in significant inference latency. Through detailed analysis of attention maps in Video Diffusion Transformer (vDiT), we identify three recurring sparsity patterns: diagonal, multi-diagonal, and vertical-stripe structures. And even 3-6\% attention heads can be skipped. Crucially, these patterns exhibit strong layer-depth and head-position correlations but show limited dependence on the input content. Leveraging these findings, we propose Sparse-vDiT, a sparsity acceleration framework for vDiT comprising: 1) Pattern-optimized sparse kernels that replace dense attention with computationally efficient implementations for each identified sparsity pattern. 2) An offline sparse diffusion search algorithm that selects the optimal sparse computation strategy per layer and head via hardware-aware cost modeling. After determining the optimal configuration, we fuse heads within the same layer that share the same attention strategy, enhancing inference efficiency. Integrated into state-of-the-art vDiT models (CogVideoX1.5, HunyuanVideo, and Wan2.1), Sparse-vDiT achieves 2.09$\times$, 2.38$\times$, and 1.67$\times$ theoretical FLOP reduction, and actual inference speedups of 1.76$\times$, 1.85$\times$, and 1.58$\times$, respectively, while maintaining high visual fidelity, with PSNR values reaching 24.13, 27.09, and 22.59. Our work demonstrates that latent structural sparsity in vDiTs can be systematically exploited for long video synthesis.
• 摘要：尽管扩散变压器（DIT）在视频生成中取得了突破，但这项长序列生成任务仍然受到注意机制的二次复杂性的限制，从而导致了明显的推论潜伏期。通过对视频扩散变压器（VDIT）中的注意图的详细分析，我们确定了三种反复出现的稀疏模式：对角线，多对角线和垂直条纹结构。甚至3-6 \％的注意力头也可以跳过。至关重要的是，这些模式表现出强大的层深度和头位相关性，但对输入含量的依赖性有限。利用这些发现，我们提出了稀疏VDIT，这是一个用于VDIT的稀疏性加速框架：1）模式优化的稀疏核，可以用每种已识别的稀疏模式的计算有效实现来代替密集的注意力。 2）一个离线稀疏扩散搜索算法，该算法每层选择最佳的稀疏计算策略，然后通过硬件感知的成本建模来牵头。确定最佳配置后，我们将头部融合在具有相同注意力策略的同一层中，从而提高了推理效率。 Integrated into state-of-the-art vDiT models (CogVideoX1.5, HunyuanVideo, and Wan2.1), Sparse-vDiT achieves 2.09$\times$, 2.38$\times$, and 1.67$\times$ theoretical FLOP reduction, and actual inference speedups of 1.76$\times$, 1.85$\times$, and 1.58$\times$,在保持高视觉保真度的同时，PSNR值达到24.13、27.09和22.59。我们的工作表明，可以系统利用VDIT中的潜在结构稀疏性来进行长时间的视频综合。

Title: EDITOR: Effective and Interpretable Prompt Inversion for Text-to-Image Diffusion Models

• Authors: Mingzhe Li, Gehao Zhang, Zhenting Wang, Shiqing Ma, Siqi Pan, Richard Cartwright, Juan Zhai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03067
• Pdf URL: https://arxiv.org/pdf/2506.03067
• Copy Paste: [[2506.03067]] EDITOR: Effective and Interpretable Prompt Inversion for Text-to-Image Diffusion Models(https://arxiv.org/abs/2506.03067)
• Keywords: generation
• Abstract: Text-to-image generation models~(e.g., Stable Diffusion) have achieved significant advancements, enabling the creation of high-quality and realistic images based on textual descriptions. Prompt inversion, the task of identifying the textual prompt used to generate a specific artifact, holds significant potential for applications including data attribution, model provenance, and watermarking validation. Recent studies introduced a delayed projection scheme to optimize for prompts representative of the vocabulary space, though challenges in semantic fluency and efficiency remain. Advanced image captioning models or visual large language models can generate highly interpretable prompts, but they often lack in image similarity. In this paper, we propose a prompt inversion technique called \sys for text-to-image diffusion models, which includes initializing embeddings using a pre-trained image captioning model, refining them through reverse-engineering in the latent space, and converting them to texts using an embedding-to-text model. Our experiments on the widely-used datasets, such as MS COCO, LAION, and Flickr, show that our method outperforms existing methods in terms of image similarity, textual alignment, prompt interpretability and generalizability. We further illustrate the application of our generated prompts in tasks such as cross-concept image synthesis, concept manipulation, evolutionary multi-concept generation and unsupervised segmentation.
• 摘要：文本到图像生成模型〜（例如，稳定的扩散）已取得了重大进步，从而可以基于文本描述创建高质量和现实的图像。提示反演，确定用于生成特定工件的文本提示的任务，在应用程序归因，模型出处和水印验证（包括数据归因，模型出处和水印验证）方面具有巨大潜力。最近的研究介绍了一种延迟的投影方案，以优化代表词汇空间的提示，尽管语义流利性和效率仍然存在挑战。高级图像字幕模型或视觉大语模型可以产生高度易于解释的提示，但它们通常缺乏图像相似性。在本文中，我们提出了一种提示反转技术，称为\ sys，用于文本到图像扩散模型，其中包括使用预训练的图像字幕模型初始化嵌入模型，通过潜在空间中的反向工程进行精炼，并使用嵌入式对象模型将其转换为文本。我们在广泛使用的数据集上进行的实验，例如Coco，Laion和Flickr，表明我们的方法在图像相似性，文本对齐，及时解释性和可推广性方面优于现有方法。我们进一步说明了生成的提示在诸如跨概念图像综合，概念操纵，进化多概念概念生成和无监督分段等任务中的应用。

Title: ORV: 4D Occupancy-centric Robot Video Generation

• Authors: Xiuyu Yang, Bohan Li, Shaocong Xu, Nan Wang, Chongjie Ye, Zhaoxi Chen, Minghan Qin, Yikang Ding, Xin Jin, Hang Zhao, Hao Zhao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03079
• Pdf URL: https://arxiv.org/pdf/2506.03079
• Copy Paste: [[2506.03079]] ORV: 4D Occupancy-centric Robot Video Generation(https://arxiv.org/abs/2506.03079)
• Keywords: generation, generative
• Abstract: Acquiring real-world robotic simulation data through teleoperation is notoriously time-consuming and labor-intensive. Recently, action-driven generative models have gained widespread adoption in robot learning and simulation, as they eliminate safety concerns and reduce maintenance efforts. However, the action sequences used in these methods often result in limited control precision and poor generalization due to their globally coarse alignment. To address these limitations, we propose ORV, an Occupancy-centric Robot Video generation framework, which utilizes 4D semantic occupancy sequences as a fine-grained representation to provide more accurate semantic and geometric guidance for video generation. By leveraging occupancy-based representations, ORV enables seamless translation of simulation data into photorealistic robot videos, while ensuring high temporal consistency and precise controllability. Furthermore, our framework supports the simultaneous generation of multi-view videos of robot gripping operations - an important capability for downstream robotic learning tasks. Extensive experimental results demonstrate that ORV consistently outperforms existing baseline methods across various datasets and sub-tasks. Demo, Code and Model: this https URL
• 摘要：众所周知，通过远程操作获取现实世界的机器人模拟数据是耗时且劳动力密集的。最近，动作驱动的生成模型在机器人学习和模拟中广泛采用，因为它们消除了安全问题并减少维护工作。但是，这些方法中使用的动作序列通常会导致控制精度有限，由于其全球粗对齐，因此概括不良。为了解决这些局限性，我们提出了一个以占用率的机器人视频生成框架ORV，该框架利用4D语义占用序列作为细粒度表示，以提供更准确的语义和几何形状指南，以进行视频生成。通过利用基于占用的表示形式，ORV可以将模拟数据无缝翻译为光真逼真的机器人视频，同时确保了高度的时间一致性和精确的可控性。此外，我们的框架还支持机器人抓地操作的多视图视频，这是下游机器人学习任务的重要功能。广泛的实验结果表明，ORV始终优于各个数据集和子任务的现有基线方法。演示，代码和模型：此HTTPS URL

Title: SG2VID: Scene Graphs Enable Fine-Grained Control for Video Synthesis

• Authors: Ssharvien Kumar Sivakumar, Yannik Frisch, Ghazal Ghazaei, Anirban Mukhopadhyay
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03082
• Pdf URL: https://arxiv.org/pdf/2506.03082
• Copy Paste: [[2506.03082]] SG2VID: Scene Graphs Enable Fine-Grained Control for Video Synthesis(https://arxiv.org/abs/2506.03082)
• Keywords: generative
• Abstract: Surgical simulation plays a pivotal role in training novice surgeons, accelerating their learning curve and reducing intra-operative errors. However, conventional simulation tools fall short in providing the necessary photorealism and the variability of human anatomy. In response, current methods are shifting towards generative model-based simulators. Yet, these approaches primarily focus on using increasingly complex conditioning for precise synthesis while neglecting the fine-grained human control aspect. To address this gap, we introduce SG2VID, the first diffusion-based video model that leverages Scene Graphs for both precise video synthesis and fine-grained human control. We demonstrate SG2VID's capabilities across three public datasets featuring cataract and cholecystectomy surgery. While SG2VID outperforms previous methods both qualitatively and quantitatively, it also enables precise synthesis, providing accurate control over tool and anatomy's size and movement, entrance of new tools, as well as the overall scene layout. We qualitatively motivate how SG2VID can be used for generative augmentation and present an experiment demonstrating its ability to improve a downstream phase detection task when the training set is extended with our synthetic videos. Finally, to showcase SG2VID's ability to retain human control, we interact with the Scene Graphs to generate new video samples depicting major yet rare intra-operative irregularities.
• 摘要：手术模拟在训练新手外科医师，加速他们的学习曲线并减少术中错误中起着关键作用。但是，传统的仿真工具在提供必要的光真相和人体解剖学的变化方面缺乏。作为响应，当前方法正在转向基于生成模型的模拟器。然而，这些方法主要集中于使用日益复杂的条件进行精确合成，同时忽略了细粒度​​的人类控制方面。为了解决这一差距，我们介绍了SG2VID，这是第一个基于扩散的视频模型，该模型利用场景图来确切视频综合和细粒度的人类控制。我们在三个包含白内障和胆囊切除手术的公共数据集中证明了SG2VID的功能。尽管SG2VID在定性和定量上都胜过以前的方法，但它也可以精确合成，从而准确控制工具和解剖学的大小和运动，新工具的入口以及整体场景布局。我们在定性上激励如何将SG2VID用于生成增强，并提出了一个实验，证明了其在使用合成视频扩展训练集时改善下游相位检测任务的能力。最后，为了展示SG2VID保留人类控制的能力，我们与场景图进行了互动，以生成描绘主要但罕见的术中不规则性的新视频样本。

Title: InterMamba: Efficient Human-Human Interaction Generation with Adaptive Spatio-Temporal Mamba

• Authors: Zizhao Wu, Yingying Sun, Yiming Chen, Xiaoling Gu, Ruyu Liu, Jiazhou Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03084
• Pdf URL: https://arxiv.org/pdf/2506.03084
• Copy Paste: [[2506.03084]] InterMamba: Efficient Human-Human Interaction Generation with Adaptive Spatio-Temporal Mamba(https://arxiv.org/abs/2506.03084)
• Keywords: generation
• Abstract: Human-human interaction generation has garnered significant attention in motion synthesis due to its vital role in understanding humans as social beings. However, existing methods typically rely on transformer-based architectures, which often face challenges related to scalability and efficiency. To address these issues, we propose a novel, efficient human-human interaction generation method based on the Mamba framework, designed to meet the demands of effectively capturing long-sequence dependencies while providing real-time feedback. Specifically, we introduce an adaptive spatio-temporal Mamba framework that utilizes two parallel SSM branches with an adaptive mechanism to integrate the spatial and temporal features of motion sequences. To further enhance the model's ability to capture dependencies within individual motion sequences and the interactions between different individual sequences, we develop two key modules: the self-adaptive spatio-temporal Mamba module and the cross-adaptive spatio-temporal Mamba module, enabling efficient feature learning. Extensive experiments demonstrate that our method achieves state-of-the-art results on two interaction datasets with remarkable quality and efficiency. Compared to the baseline method InterGen, our approach not only improves accuracy but also requires a minimal parameter size of just 66M ,only 36% of InterGen's, while achieving an average inference speed of 0.57 seconds, which is 46% of InterGen's execution time.
• 摘要：人类人类的互动产生因其在将人类作为社会生物中的至关重要的作用而引起了运动合成的大大关注。但是，现有方法通常依赖于基于变压器的体系结构，这些结构通常面临与可扩展性和效率相关的挑战。为了解决这些问题，我们提出了一种基于MAMBA框架的新颖，有效的人类互动生成方法，旨在满足有效捕获长期依赖依赖的需求，同时提供实时反馈。具体而言，我们引入了一个自适应时空的amamba框架，该框架利用两个平行的SSM分支，具有自适应机制来整合运动序列的时空特征和时间特征。为了进一步增强模型在单个运动序列中捕获依赖性的能力以及不同单个序列之间的相互作用，我们开发了两个关键模块：自适应时空跨山姆巴模块和交叉自适应时空的山脉模块，从而实现了有效的特征学习。广泛的实验表明，我们的方法在两个相互作用数据集上获得了最先进的结果，其质量和效率很高。与基线方法Intergen相比，我们的方法不仅提高了准确性，而且还需要最小的参数大小仅为66m，仅占Intergen的36％，同时达到平均推理速度为0.57秒，这是Intergen执行时间的46％。

Title: Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds

• Authors: Yang Guo, Yutian Tao, Yifei Ming, Robert D. Nowak, Yingyu Liang
• Subjects: cs.LG, cs.AI, cs.CL, cs.IR, math.ST
• Abstract URL: https://arxiv.org/abs/2506.03100
• Pdf URL: https://arxiv.org/pdf/2506.03100
• Copy Paste: [[2506.03100]] Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds(https://arxiv.org/abs/2506.03100)
• Keywords: generation
• Abstract: Retrieval-augmented generation (RAG) has seen many empirical successes in recent years by aiding the LLM with external knowledge. However, its theoretical aspect has remained mostly unexplored. In this paper, we propose the first finite-sample generalization bound for RAG in in-context linear regression and derive an exact bias-variance tradeoff. Our framework views the retrieved texts as query-dependent noisy in-context examples and recovers the classical in-context learning (ICL) and standard RAG as the limit cases. Our analysis suggests that an intrinsic ceiling on generalization error exists on RAG as opposed to the ICL. Furthermore, our framework is able to model retrieval both from the training data and from external corpora by introducing uniform and non-uniform RAG noise. In line with our theory, we show the sample efficiency of ICL and RAG empirically with experiments on common QA benchmarks, such as Natural Questions and TriviaQA.
• 摘要：近年来，通过以外部知识为帮助的LLM通过帮助LLM获得了许多经验成功。但是，其理论方面仍未开发。在本文中，我们提出了第一个有限样本的概括，该概括是在内部文化线性回归中绑定的抹布，并得出了确切的偏见差异权衡。我们的框架将检索到的文本视为依赖查询的嘈杂词中的示例，并恢复经典的内在学习（ICL）和标准抹布作为极限情况。我们的分析表明，与ICL相反，在碎布上存在固有的上限。此外，我们的框架能够通过引入统一和不均匀的抹布噪声来从培训数据和外部语料库中进行建模。与我们的理论一致，我们通过对常见的QA基准（例如自然问题和Triviaqa）进行实验，从经验上显示了ICL和RAG的样本效率。

Title: ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions

• Authors: Di Chang, Mingdeng Cao, Yichun Shi, Bo Liu, Shengqu Cai, Shijie Zhou, Weilin Huang, Gordon Wetzstein, Mohammad Soleymani, Peng Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03107
• Pdf URL: https://arxiv.org/pdf/2506.03107
• Copy Paste: [[2506.03107]] ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions(https://arxiv.org/abs/2506.03107)
• Keywords: generation
• Abstract: Editing images with instructions to reflect non-rigid motions, camera viewpoint shifts, object deformations, human articulations, and complex interactions, poses a challenging yet underexplored problem in computer vision. Existing approaches and datasets predominantly focus on static scenes or rigid transformations, limiting their capacity to handle expressive edits involving dynamic motion. To address this gap, we introduce ByteMorph, a comprehensive framework for instruction-based image editing with an emphasis on non-rigid motions. ByteMorph comprises a large-scale dataset, ByteMorph-6M, and a strong baseline model built upon the Diffusion Transformer (DiT), named ByteMorpher. ByteMorph-6M includes over 6 million high-resolution image editing pairs for training, along with a carefully curated evaluation benchmark ByteMorph-Bench. Both capture a wide variety of non-rigid motion types across diverse environments, human figures, and object categories. The dataset is constructed using motion-guided data generation, layered compositing techniques, and automated captioning to ensure diversity, realism, and semantic coherence. We further conduct a comprehensive evaluation of recent instruction-based image editing methods from both academic and commercial domains.
• 摘要：用指令编辑图像，以反映非刚性动作，摄像机的观点变化，对象变形，人类表达和复杂的相互作用，在计算机视觉中提出了一个具有挑战性但充满疏忽的问题。现有的方法和数据集主要集中在静态场景或刚性转换上，从而限制了其处理涉及动态运动的表达编辑的能力。为了解决这一差距，我们介绍了Bytemorph，这是一个基于教学的图像编辑的综合框架，重点是非刚性动作。 Bytemorph包含一个大型数据集，Bytemorph-6m和一个强大的基线模型，该模型构建了扩散变压器（DIT），名为Bytemorpher。 Bytemorph-6M包括超过600万个高分辨率图像编辑对训练，以及经过精心策划的评估基准Bytemorph-Bench。两者都捕获了各种环境，人物和对象类别的各种非刚性运动类型。该数据集是使用运动引导的数据生成，分层合成技术和自动字幕构建的，以确保多样性，现实主义和语义连贯性。我们进一步对学术和商业领域的最新基于教学的图像编辑方法进行了全面评估。

Title: Controllable Human-centric Keyframe Interpolation with Generative Prior

• Authors: Zujin Guo, Size Wu, Zhongang Cai, Wei Li, Chen Change Loy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03119
• Pdf URL: https://arxiv.org/pdf/2506.03119
• Copy Paste: [[2506.03119]] Controllable Human-centric Keyframe Interpolation with Generative Prior(https://arxiv.org/abs/2506.03119)
• Keywords: generative
• Abstract: Existing interpolation methods use pre-trained video diffusion priors to generate intermediate frames between sparsely sampled keyframes. In the absence of 3D geometric guidance, these methods struggle to produce plausible results for complex, articulated human motions and offer limited control over the synthesized dynamics. In this paper, we introduce PoseFuse3D Keyframe Interpolator (PoseFuse3D-KI), a novel framework that integrates 3D human guidance signals into the diffusion process for Controllable Human-centric Keyframe Interpolation (CHKI). To provide rich spatial and structural cues for interpolation, our PoseFuse3D, a 3D-informed control model, features a novel SMPL-X encoder that transforms 3D geometry and shape into the 2D latent conditioning space, alongside a fusion network that integrates these 3D cues with 2D pose embeddings. For evaluation, we build CHKI-Video, a new dataset annotated with both 2D poses and 3D SMPL-X parameters. We show that PoseFuse3D-KI consistently outperforms state-of-the-art baselines on CHKI-Video, achieving a 9% improvement in PSNR and a 38% reduction in LPIPS. Comprehensive ablations demonstrate that our PoseFuse3D model improves interpolation fidelity.
• 摘要：现有的插值方法使用预先训练的视频扩散先验来生成稀疏采样的密钥帧之间的中间帧。在没有3D几何指导的情况下，这些方法难以为复杂，铰接的人类运动产生合理的结果，并对合成动力学提供有限的控制。在本文中，我们介绍了Posefuse3D KeyFrame Interpolator（Posefuse3D-KI），这是一个新颖的框架，将3D人类指导信号集成到可控制的以人为中心的人为中心的钥匙扣插值（CHKI）的扩散过程中。为了提供丰富的空间和结构提示进行插值，我们的PoseFuse3D（一个3D启用的控制模型）具有一种新型的SMPL-X编码器，该编码器将3D几何形状和形状转换为2D潜在条件空间，并与将这些3D提示与2D姿势嵌入的融合网络一起进行。为了进行评估，我们构建了CHKI-VIDEO，这是一个带有2D姿势和3D SMPL-X参数的新数据集。我们表明，Posefuse3D-KI始终优于CHKI-VIDEO上最先进的基线，在PSNR中提高了9％，LPIPS的提高了38％。全面的消融表明，我们的posefuse3d模型改善了插值保真度。

Title: DCM: Dual-Expert Consistency Model for Efficient and High-Quality Video Generation

• Authors: Zhengyao Lv, Chenyang Si, Tianlin Pan, Zhaoxi Chen, Kwan-Yee K. Wong, Yu Qiao, Ziwei Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03123
• Pdf URL: https://arxiv.org/pdf/2506.03123
• Copy Paste: [[2506.03123]] DCM: Dual-Expert Consistency Model for Efficient and High-Quality Video Generation(https://arxiv.org/abs/2506.03123)
• Keywords: generation
• Abstract: Diffusion Models have achieved remarkable results in video synthesis but require iterative denoising steps, leading to substantial computational overhead. Consistency Models have made significant progress in accelerating diffusion models. However, directly applying them to video diffusion models often results in severe degradation of temporal consistency and appearance details. In this paper, by analyzing the training dynamics of Consistency Models, we identify a key conflicting learning dynamics during the distillation process: there is a significant discrepancy in the optimization gradients and loss contributions across different timesteps. This discrepancy prevents the distilled student model from achieving an optimal state, leading to compromised temporal consistency and degraded appearance details. To address this issue, we propose a parameter-efficient \textbf{Dual-Expert Consistency Model~(DCM)}, where a semantic expert focuses on learning semantic layout and motion, while a detail expert specializes in fine detail refinement. Furthermore, we introduce Temporal Coherence Loss to improve motion consistency for the semantic expert and apply GAN and Feature Matching Loss to enhance the synthesis quality of the detail this http URL approach achieves state-of-the-art visual quality with significantly reduced sampling steps, demonstrating the effectiveness of expert specialization in video diffusion model distillation. Our code and models are available at \href{this https URL}{this https URL}.
• 摘要：扩散模型在视频综合中取得了显着的结果，但需要迭代的剥离步骤，从而导致大量的计算开销。一致性模型在加速扩散模型方面取得了重大进展。但是，将它们直接应用于视频扩散模型通常会导致时间一致性和外观细节的严重降解。在本文中，通过分析一致性模型的训练动力学，我们在蒸馏过程中确定了一个关键的矛盾学习动态：在不同时间段的优化梯度和损失贡献中存在很大差异。这种差异阻止了蒸馏的学生模型达到最佳状态，从而导致时间一致性损害和降级外观细节。为了解决这个问题，我们提出了一个参数效率\ textbf {dual-expert一致性模型〜（dcm）}，其中语义专家专注于学习语义布局和运动，而细节专家则专门研究细节细节。此外，我们引入了时间连贯性损失，以提高语义专家的运动一致性，并应用GAN和功能匹配损失，以提高细节的合成质量。该HTTP URL方法可实现最先进的视觉质量，并显着降低采样步骤，以大大降低采样步骤，以证明视频扩散模型模型蒸馏的专家专业效果。我们的代码和模型可在\ href {this HTTPS url} {此https url}上获得。

Title: AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video Generation

• Authors: Lu Qiu, Yizhuo Li, Yuying Ge, Yixiao Ge, Ying Shan, Xihui Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03126
• Pdf URL: https://arxiv.org/pdf/2506.03126
• Copy Paste: [[2506.03126]] AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video Generation(https://arxiv.org/abs/2506.03126)
• Keywords: generation
• Abstract: Recent advances in AI-generated content (AIGC) have significantly accelerated animation production. To produce engaging animations, it is essential to generate coherent multi-shot video clips with narrative scripts and character references. However, existing public datasets primarily focus on real-world scenarios with global descriptions, and lack reference images for consistent character guidance. To bridge this gap, we present AnimeShooter, a reference-guided multi-shot animation dataset. AnimeShooter features comprehensive hierarchical annotations and strong visual consistency across shots through an automated pipeline. Story-level annotations provide an overview of the narrative, including the storyline, key scenes, and main character profiles with reference images, while shot-level annotations decompose the story into consecutive shots, each annotated with scene, characters, and both narrative and descriptive visual captions. Additionally, a dedicated subset, AnimeShooter-audio, offers synchronized audio tracks for each shot, along with audio descriptions and sound sources. To demonstrate the effectiveness of AnimeShooter and establish a baseline for the reference-guided multi-shot video generation task, we introduce AnimeShooterGen, which leverages Multimodal Large Language Models (MLLMs) and video diffusion models. The reference image and previously generated shots are first processed by MLLM to produce representations aware of both reference and context, which are then used as the condition for the diffusion model to decode the subsequent shot. Experimental results show that the model trained on AnimeShooter achieves superior cross-shot visual consistency and adherence to reference visual guidance, which highlight the value of our dataset for coherent animated video generation.
• 摘要：AI生成的内容（AIGC）的最新进展已大大加速了动画制作。为了产生引人入胜的动画，必须生成具有叙事脚本和角色参考的连贯的多拍视频剪辑。但是，现有的公共数据集主要关注具有全球描述的现实情况，而缺乏参考图像以进行一致的角色指导。为了弥合这一差距，我们提出了AnimesHooter，这是一个参考引导的多弹性动画数据集。 Animeshooter通过自动管道具有全面的层次结构注释和强烈的视觉一致性。故事级注释提供了叙述的概述，包括故事情节，关键场景和带有参考图像的主角概况，而射击级注释将故事分解为连续的镜头，每个故事都用场景，角色以及叙事和描述性的视觉贴图进行注释。此外，一个专用的子集Animeshooter-Audio为每次镜头提供同步的音轨，以及音频说明和声音源。为了展示动漫射击器的有效性并为参考引导的多拍视频生成任务建立了基线，我们介绍了动漫剂，该动脉凝胶利用了多模式大语言模型（MLLMS）和视频扩散模型。首先由MLLM处理参考图像和先前生成的镜头，以产生意识到参考和上下文的表示，然后将其用作扩散模型解码后续射击的条件。实验结果表明，经过动脉刺管训练的模型实现了较高的横镜视觉一致性和依从性，以参考视觉指导，这突出了我们数据集对相干动画视频生成的价值。

Title: Native-Resolution Image Synthesis

• Authors: Zidong Wang, Lei Bai, Xiangyu Yue, Wanli Ouyang, Yiyuan Zhang
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.03131
• Pdf URL: https://arxiv.org/pdf/2506.03131
• Copy Paste: [[2506.03131]] Native-Resolution Image Synthesis(https://arxiv.org/abs/2506.03131)
• Keywords: generative
• Abstract: We introduce native-resolution image synthesis, a novel generative modeling paradigm that enables the synthesis of images at arbitrary resolutions and aspect ratios. This approach overcomes the limitations of conventional fixed-resolution, square-image methods by natively handling variable-length visual tokens, a core challenge for traditional techniques. To this end, we introduce the Native-resolution diffusion Transformer (NiT), an architecture designed to explicitly model varying resolutions and aspect ratios within its denoising process. Free from the constraints of fixed formats, NiT learns intrinsic visual distributions from images spanning a broad range of resolutions and aspect ratios. Notably, a single NiT model simultaneously achieves the state-of-the-art performance on both ImageNet-256x256 and 512x512 benchmarks. Surprisingly, akin to the robust zero-shot capabilities seen in advanced large language models, NiT, trained solely on ImageNet, demonstrates excellent zero-shot generalization performance. It successfully generates high-fidelity images at previously unseen high resolutions (e.g., 1536 x 1536) and diverse aspect ratios (e.g., 16:9, 3:1, 4:3), as shown in Figure 1. These findings indicate the significant potential of native-resolution modeling as a bridge between visual generative modeling and advanced LLM methodologies.
• 摘要：我们介绍了天然分辨率图像综合，这是一种新型的生成建模范式，可在任意分辨率和长宽比下综合图像。这种方法通过本质处理可变的视觉令牌来克服常规固定分辨率，平方图像方法的局限性，这是传统技术的核心挑战。为此，我们介绍了本地分辨率扩散变压器（NIT），该体系结构旨在在其降解过程中明确模型改变分辨率和长宽比。 NIT不受固定格式的约束，从跨越各种分辨率和宽高比的图像中学习了内在的视觉分布。值得注意的是，单个NIT模型同时达到了Imagenet-256x256和512x512基准的最新性能。令人惊讶的是，类似于在高级大语言模型中看到的强大的零击功能，NIT仅在Imagenet上训练，表现出了出色的零弹性概括性能。如图1所示，它成功地生成了先前看不见的高分辨率（例如1536 x 1536）和不同的纵横比（例如16：9，3：1，4：3）的高保真图像，如图1所示。这些发现表明，天然分辨率建模作为视觉生成模型和高级LLM方法之间的桥梁模型的重要潜力。

Title: SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation

• Authors: Siqi Chen, Xinyu Dong, Haolei Xu, Xingyu Wu, Fei Tang, Hang Zhang, Yuchen Yan, Linjuan Wu, Wenqi Zhang, Guiyang Hou, Yongliang Shen, Weiming Lu, Yueting Zhuang
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.03139
• Pdf URL: https://arxiv.org/pdf/2506.03139
• Copy Paste: [[2506.03139]] SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation(https://arxiv.org/abs/2506.03139)
• Keywords: generation
• Abstract: Large Language Models (LLMs) and Multimodal LLMs have shown promising capabilities for SVG processing, yet existing benchmarks suffer from limited real-world coverage, lack of complexity stratification, and fragmented evaluation paradigms. We introduce SVGenius, a comprehensive benchmark comprising 2,377 queries across three progressive dimensions: understanding, editing, and generation. Built on real-world data from 24 application domains with systematic complexity stratification, SVGenius evaluates models through 8 task categories and 18 metrics. We assess 22 mainstream models spanning different scales, architectures, training paradigms, and accessibility levels. Our analysis reveals that while proprietary models significantly outperform open-source counterparts, all models exhibit systematic performance degradation with increasing complexity, indicating fundamental limitations in current approaches; however, reasoning-enhanced training proves more effective than pure scaling for overcoming these limitations, though style transfer remains the most challenging capability across all model types. SVGenius establishes the first systematic evaluation framework for SVG processing, providing crucial insights for developing more capable vector graphics models and advancing automated graphic design applications. Appendix and supplementary materials (including all data and code) are available at this https URL.
• 摘要：大型语言模型（LLM）和多模式LLM已显示出有希望的SVG处理能力，但是现有的基准测试的现实世界覆盖范围有限，缺乏复杂性分层和分散的评估范围。我们介绍了Svgenius，这是一个综合基准，包括三个渐进维度的2377个查询：理解，编辑和一代。 Svgenius建立在具有系统复杂性分层的24个应用程序域中的实际数据上，通过8个任务类别和18个指标评估模型。我们评估了22种涵盖不同尺度，架构，训练范式和可访问性水平的主流模型。我们的分析表明，虽然专有模型的表现明显优于开源对应物，但所有模型均表现出系统的性能降低，复杂性越来越大，表明当前方法的基本限制。但是，尽管样式转移仍然是所有模型类型中最具挑战性的能力，但推理增强培训比纯缩放比纯缩放更有效。 Svgenius建立了用于SVG处理的第一个系统评估框架，为开发功能更强大的矢量图形模型并推进自动图形设计应用程序提供了关键见解。附录和补充材料（包括所有数据和代码）可在此HTTPS URL上找到。

Title: CamCloneMaster: Enabling Reference-based Camera Control for Video Generation

• Authors: Yawen Luo, Jianhong Bai, Xiaoyu Shi, Menghan Xia, Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, Tianfan Xue
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03140
• Pdf URL: https://arxiv.org/pdf/2506.03140
• Copy Paste: [[2506.03140]] CamCloneMaster: Enabling Reference-based Camera Control for Video Generation(https://arxiv.org/abs/2506.03140)
• Keywords: generation
• Abstract: Camera control is crucial for generating expressive and cinematic videos. Existing methods rely on explicit sequences of camera parameters as control conditions, which can be cumbersome for users to construct, particularly for intricate camera movements. To provide a more intuitive camera control method, we propose CamCloneMaster, a framework that enables users to replicate camera movements from reference videos without requiring camera parameters or test-time fine-tuning. CamCloneMaster seamlessly supports reference-based camera control for both Image-to-Video and Video-to-Video tasks within a unified framework. Furthermore, we present the Camera Clone Dataset, a large-scale synthetic dataset designed for camera clone learning, encompassing diverse scenes, subjects, and camera movements. Extensive experiments and user studies demonstrate that CamCloneMaster outperforms existing methods in terms of both camera controllability and visual quality.
• 摘要：相机控制对于生成表现力和电影视频至关重要。现有方法依赖于摄像机参数的明确序列作为控制条件，这对于用户构建可能很麻烦，尤其是对于复杂的相机运动。为了提供一种更直观的相机控制方法，我们提出了Camclonemaster，该框架使用户能够从参考视频中复制相机运动而无需相机参数或测试时间进行微调。 Camclonemaster无缝支持基于参考的相机控制，用于统一框架内的图像到视频和视频对视频任务。此外，我们介绍了相机克隆数据集，这是一个大型合成数据集，专为相机克隆学习而设计，涵盖了各种场景，主题和相机运动。广泛的实验和用户研究表明，Camclonemaster在相机可控性和视觉质量方面都优于现有方法。

Title: Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval

• Authors: Jiwen Yu, Jianhong Bai, Yiran Qin, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, Xihui Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.03141
• Pdf URL: https://arxiv.org/pdf/2506.03141
• Copy Paste: [[2506.03141]] Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval(https://arxiv.org/abs/2506.03141)
• Keywords: generation
• Abstract: Recent advances in interactive video generation have shown promising results, yet existing approaches struggle with scene-consistent memory capabilities in long video generation due to limited use of historical context. In this work, we propose Context-as-Memory, which utilizes historical context as memory for video generation. It includes two simple yet effective designs: (1) storing context in frame format without additional post-processing; (2) conditioning by concatenating context and frames to be predicted along the frame dimension at the input, requiring no external control modules. Furthermore, considering the enormous computational overhead of incorporating all historical context, we propose the Memory Retrieval module to select truly relevant context frames by determining FOV (Field of View) overlap between camera poses, which significantly reduces the number of candidate frames without substantial information loss. Experiments demonstrate that Context-as-Memory achieves superior memory capabilities in interactive long video generation compared to SOTAs, even generalizing effectively to open-domain scenarios not seen during training. The link of our project page is this https URL.
• 摘要：交互式视频生成的最新进展已显示出令人鼓舞的结果，但是由于历史上下文的使用有限，现有的方法与长期视频生成中的场景一致记忆能力挣扎。在这项工作中，我们提出了上下文，即记忆，它利用历史上下文作为视频生成的记忆。它包括两个简单但有效的设计：（1）以框架格式存储上下文，而无需其他后处理； （2）通过串联上下文和沿输入框架尺寸预测的框架来进行调节，不需要外部控制模块。此外，考虑到结合了所有历史上下文的巨大计算开销，我们提出了记忆检索模块，以通过确定相机姿势之间的FOV（视野）重叠来选择真正相关的上下文帧，从而大大减少了候选框架的数量而没有实质性信息损失。实验表明，与SOTA相比，与记忆的上下文相比，可以在互动长期视频生成中实现出色的记忆能力，甚至有效地推广到训练过程中未见的开放域情景。我们项目页面的链接是此HTTPS URL。

Title: Not All Tokens Are Meant to Be Forgotten

• Authors: Xiangyu Zhou, Yao Qiang, Saleh Zare Zade, Douglas Zytko, Prashant Khanduri, Dongxiao Zhu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.03142
• Pdf URL: https://arxiv.org/pdf/2506.03142
• Copy Paste: [[2506.03142]] Not All Tokens Are Meant to Be Forgotten(https://arxiv.org/abs/2506.03142)
• Keywords: generation
• Abstract: Large Language Models (LLMs), pre-trained on massive text corpora, exhibit remarkable human-level language understanding, reasoning, and decision-making abilities. However, they tend to memorize unwanted information, such as private or copyrighted content, raising significant privacy and legal concerns. Unlearning has emerged as a promising solution, but existing methods face a significant challenge of over-forgetting. This issue arises because they indiscriminately suppress the generation of all the tokens in forget samples, leading to a substantial loss of model utility. To overcome this challenge, we introduce the Targeted Information Forgetting (TIF) framework, which consists of (1) a flexible targeted information identifier designed to differentiate between unwanted words (UW) and general words (GW) in the forget samples, and (2) a novel Targeted Preference Optimization approach that leverages Logit Preference Loss to unlearn unwanted information associated with UW and Preservation Loss to retain general information in GW, effectively improving the unlearning process while mitigating utility degradation. Extensive experiments on the TOFU and MUSE benchmarks demonstrate that the proposed TIF framework enhances unlearning effectiveness while preserving model utility and achieving state-of-the-art results.
• 摘要：大型语言模型（LLMS）在大规模文本语料库中进行了预先培训，具有出色的人类语言理解，推理和决策能力。但是，他们倾向于记住不需要的信息，例如私人或受版权保护的内容，从而引发了严重的隐私和法律问题。不学习是一种有希望的解决方案，但是现有的方法面临着过度遗忘的重大挑战。之所以出现这个问题，是因为它们不加打破了忘记样本中所有令牌的产生，从而导致了模型效用的大量损失。为了克服这一挑战，我们介绍了有针对性的信息遗忘（TIF）框架，该框架由（1）旨在区分忘记样本中不需要单词（UW）和一般单词（UW）和（GW）的灵活目标信息标识符以及（2）新颖的有针对性的目标偏好方法，该方法利用了与不需要的信息相关的，效率为无效的信息，以改进了与UW的损失相关的信息，并将其与UW相关联，UW和UW的效率为UW，UW效率为UW，UW效率为UW，UW效率为UW，并将其置于损失。在减轻效用降解的同时进行学习过程。在豆腐和穆塞尔基准上进行的广泛实验表明，所提出的TIF框架提高了学习效率，同时保留了模型实用程序并实现了最先进的结果。

Title: UniWorld: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation

• Authors: Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, Yatian Pang, Li Yuan
• Subjects: cs.CV, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.03147
• Pdf URL: https://arxiv.org/pdf/2506.03147
• Copy Paste: [[2506.03147]] UniWorld: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation(https://arxiv.org/abs/2506.03147)
• Keywords: generation, generative
• Abstract: Although existing unified models deliver strong performance on vision-language understanding and text-to-image generation, their models are limited in exploring image perception and manipulation tasks, which are urgently desired by users for wide applications. Recently, OpenAI released their powerful GPT-4o-Image model for comprehensive image perception and manipulation, achieving expressive capability and attracting community interests. By observing the performance of GPT-4o-Image in our carefully constructed experiments, we infer that GPT-4o-Image leverages features extracted by semantic encoders instead of VAE, while VAEs are considered essential components in many image manipulation models. Motivated by such inspiring observations, we present a unified generative framework named UniWorld based on semantic features provided by powerful visual-language models and contrastive semantic encoders. As a result, we build a strong unified model using only 1% amount of BAGEL's data, which consistently outperforms BAGEL on image editing benchmarks. UniWorld also maintains competitive image understanding and generation capabilities, achieving strong performance across multiple image perception tasks. We fully open-source our models, including model weights, training and evaluation scripts, and datasets.
• 摘要：尽管现有的统一模型在视觉理解和文本到图像生成方面表现出色，但他们的模型在探索图像感知和操纵任务方面受到限制，这些任务迫切需要广泛的应用程序。最近，OpenAI发布了其强大的GPT-4O图像模型，以实现全面的图像感知和操纵，实现表达能力并吸引社区利益。通过观察我们精心构造的实验中GPT-4O图像的性能，我们推断出语义编码器而不是VAE提取的GPT-4O图像杠杆功能，而VAE在许多图像操作模型中被认为是必不可少的组件。由这种鼓舞人心的观察激励，我们基于强大的视觉语言模型和对比性语义编码器提供的语义特征，提出了一个名为uniworld的统一生成框架。结果，我们仅使用1％的百吉饼数据构建了一个强大的统一模型，该模型始终超过图像编辑基准测试的百吉饼。 Uniworld还保持了竞争性的图像理解和发电能力，从而在多个图像感知任务中实现了强劲的性能。我们完全开放我们的模型，包括模型权重，培训和评估脚本以及数据集。

Title: IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation

• Authors: Yuanze Lin, Yi-Wen Chen, Yi-Hsuan Tsai, Ronald Clark, Ming-Hsuan Yang
• Subjects: cs.CV, cs.AI, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2506.03150
• Pdf URL: https://arxiv.org/pdf/2506.03150
• Copy Paste: [[2506.03150]] IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation(https://arxiv.org/abs/2506.03150)
• Keywords: generation
• Abstract: Although diffusion-based models can generate high-quality and high-resolution video sequences from textual or image inputs, they lack explicit integration of geometric cues when controlling scene lighting and visual appearance across frames. To address this limitation, we propose IllumiCraft, an end-to-end diffusion framework accepting three complementary inputs: (1) high-dynamic-range (HDR) video maps for detailed lighting control; (2) synthetically relit frames with randomized illumination changes (optionally paired with a static background reference image) to provide appearance cues; and (3) 3D point tracks that capture precise 3D geometry information. By integrating the lighting, appearance, and geometry cues within a unified diffusion architecture, IllumiCraft generates temporally coherent videos aligned with user-defined prompts. It supports background-conditioned and text-conditioned video relighting and provides better fidelity than existing controllable video generation methods. Project Page: this https URL
• 摘要：尽管基于扩散的模型可以从文本或图像输入中生成高质量和高分辨率的视频序列，但在控制场景照明和跨帧的视觉外观时，它们缺乏明确的几何线索集成。为了解决此限制，我们提出了Illumicraft，这是一个端到端扩散框架，接受三个互补输入：（1）用于详细照明控制的高动态范围（HDR）视频图； （2）随机照明更改（可选与静态背景参考图像配对）的合成框架可提供外观提示； （3）3D点轨道捕获精确的3D几何信息。通过将照明，外观和几何提示集成在统一扩散体系结构中，Illumicraft生成了与用户定义的提示对齐的时间连贯的视频。它支持背景条件和文本条件的视频重新考虑，并提供比现有可控视频生成方法更好的保真度。项目页面：此HTTPS URL