2025-04-23

Title: LLM-Enabled Style and Content Regularization for Personalized Text-to-Image Generation

• Authors: Anran Yu, Wei Feng, Yaochen Zhang, Xiang Li, Lei Meng, Lei Wu, Xiangxu Meng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15309
• Pdf URL: https://arxiv.org/pdf/2504.15309
• Copy Paste: [[2504.15309]] LLM-Enabled Style and Content Regularization for Personalized Text-to-Image Generation(https://arxiv.org/abs/2504.15309)
• Keywords: generation
• Abstract: The personalized text-to-image generation has rapidly advanced with the emergence of Stable Diffusion. Existing methods, which typically fine-tune models using embedded identifiers, often struggle with insufficient stylization and inaccurate image content due to reduced textual controllability. In this paper, we propose style refinement and content preservation strategies. The style refinement strategy leverages the semantic information of visual reasoning prompts and reference images to optimize style embeddings, allowing a more precise and consistent representation of style information. The content preservation strategy addresses the content bias problem by preserving the model's generalization capabilities, ensuring enhanced textual controllability without compromising stylization. Experimental results verify that our approach achieves superior performance in generating consistent and personalized text-to-image outputs.
• 摘要：随着稳定扩散的出现，个性化的文本到图像生成迅速发展。现有的方法通常使用嵌入式标识符微调模型，通常由于文本可控性降低而困难的风格不足和不准确的图像内容。在本文中，我们提出了风格的完善和内容保存策略。样式改进策略利用视觉推理提示和参考图像的语义信息来优化样式嵌入，从而可以更精确，一致地表示样式信息。内容保存策略通过保留模型的概括功能来解决内容偏差问题，从而确保增强的文本可控性而不会损害风格。实验结果验证了我们的方法在产生一致和个性化的文本对象输出方面取得了卓越的性能。

Title: Diffusion-Driven Inertial Generated Data for Smartphone Location Classification

• Authors: Noa Cohen, Rotem Dror, Itzik Klein
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.15315
• Pdf URL: https://arxiv.org/pdf/2504.15315
• Copy Paste: [[2504.15315]] Diffusion-Driven Inertial Generated Data for Smartphone Location Classification(https://arxiv.org/abs/2504.15315)
• Keywords: generation, generative
• Abstract: Despite the crucial role of inertial measurements in motion tracking and navigation systems, the time-consuming and resource-intensive nature of collecting extensive inertial data has hindered the development of robust machine learning models in this field. In recent years, diffusion models have emerged as a revolutionary class of generative models, reshaping the landscape of artificial data generation. These models surpass generative adversarial networks and other state-of-the-art approaches to complex tasks. In this work, we propose diffusion-driven specific force-generated data for smartphone location recognition. We provide a comprehensive evaluation methodology by comparing synthetic and real recorded specific force data across multiple metrics. Our results demonstrate that our diffusion-based generative model successfully captures the distinctive characteristics of specific force signals across different smartphone placement conditions. Thus, by creating diverse, realistic synthetic data, we can reduce the burden of extensive data collection while providing high-quality training data for machine learning models.
• 摘要：尽管惯性测量在运动跟踪和导航系统中起着至关重要的作用，但收集广泛的惯性数据的耗时和资源密集的性质阻碍了该领域强大的机器学习模型的发展。近年来，扩散模型已成为革命性的生成模型类别，重塑了人工数据生成的景观。这些模型超越了生成的对抗网络和其他最先进的方法来解决复杂的任务。在这项工作中，我们建议以扩散驱动的特定力生成的数据来识别智能手机位置识别。我们通过比较跨多个指标的合成和记录的特定力数据来提供全面的评估方法。我们的结果表明，我们的基于扩散的生成模型成功地捕获了不同智能手机放置条件上特定力信号的独特特征。因此，通过创建多样化的现实合成数据，我们可以减轻广泛数据收集的负担，同时为机器学习模型提供高质量的培训数据。

Title: LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception

• Authors: Yuan-Hong Liao, Sven Elflein, Liu He, Laura Leal-Taixé, Yejin Choi, Sanja Fidler, David Acuna
• Subjects: cs.CV, cs.CL, cs.LG
• Abstract URL: https://arxiv.org/abs/2504.15362
• Pdf URL: https://arxiv.org/pdf/2504.15362
• Copy Paste: [[2504.15362]] LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception(https://arxiv.org/abs/2504.15362)
• Keywords: generation
• Abstract: Recent reasoning models through test-time scaling have demonstrated that long chain-of-thoughts can unlock substantial performance boosts in hard reasoning tasks such as math and code. However, the benefit of such long thoughts for system-2 reasoning is relatively less explored in other domains such as perceptual tasks where shallower, system-1 reasoning seems sufficient. In this paper, we introduce LongPerceptualThoughts, a new synthetic dataset with 30K long-thought traces for perceptual tasks. The key challenges in synthesizing elaborate reasoning thoughts for perceptual tasks are that off-the-shelf models are not yet equipped with such thinking behavior and that it is not straightforward to build a reliable process verifier for perceptual tasks. Thus, we propose a novel three-stage data synthesis framework that first synthesizes verifiable multiple-choice questions from dense image descriptions, then extracts simple CoTs from VLMs for those verifiable problems, and finally expands those simple thoughts to elaborate long thoughts via frontier reasoning models. In controlled experiments with a strong instruction-tuned 7B model, we demonstrate notable improvements over existing visual reasoning data-generation methods. Our model, trained on the generated dataset, achieves an average +3.4 points improvement over 5 vision-centric benchmarks, including +11.8 points on V$^*$ Bench. Notably, despite being tuned for vision tasks, it also improves performance on the text reasoning benchmark, MMLU-Pro, by +2 points.
• 摘要：通过测试时间缩放的最新推理模型表明，长期的思考可以在数学和代码等硬性推理任务中解锁大量绩效。但是，在System-2推理中，这么长的想法的好处在其他领域（例如较浅，System-1推理似乎足够的感知任务）中所探索的较少。在本文中，我们介绍了LongcecepcepatualThoughts，这是一个新的合成数据集，具有30k的长期思考痕迹，可感知任务。综合知名任务的详尽的推理思想所面临的主要挑战是，现成的模型尚未配备这种思维行为，并且为感知任务构建可靠的过程验证程序并不直接。因此，我们提出了一个新颖的三阶段数据综合框架，该框架首先从密集的图像描述中综合了可验证的多项选择问题，然后从VLMS中提取简单的COTS来解决这些可验证的问题，最后扩展了这些简单的想法，以通过边境推理来阐述长期的思想。在具有强大的指令调整的7B模型的受控实验中，我们证明了对现有视觉推理数据生成方法的显着改进。在生成的数据集上训练的模型平均取得了+3.4点的提高，比5个以视觉为中心的基准测试，其中包括V $^*$ bench上的+11.8分。值得注意的是，尽管对视觉任务进行了调整，但它还提高了文本推理基准MMLU-PRO的性能，提高了+2分。

Title: Solving New Tasks by Adapting Internet Video Knowledge

• Authors: Calvin Luo, Zilai Zeng, Yilun Du, Chen Sun
• Subjects: cs.LG, cs.AI, cs.RO
• Abstract URL: https://arxiv.org/abs/2504.15369
• Pdf URL: https://arxiv.org/pdf/2504.15369
• Copy Paste: [[2504.15369]] Solving New Tasks by Adapting Internet Video Knowledge(https://arxiv.org/abs/2504.15369)
• Keywords: generative
• Abstract: Video generative models demonstrate great promise in robotics by serving as visual planners or as policy supervisors. When pretrained on internet-scale data, such video models intimately understand alignment with natural language, and can thus facilitate generalization to novel downstream behavior through text-conditioning. However, they may not be sensitive to the specificities of the particular environment the agent inhabits. On the other hand, training video models on in-domain examples of robotic behavior naturally encodes environment-specific intricacies, but the scale of available demonstrations may not be sufficient to support generalization to unseen tasks via natural language specification. In this work, we investigate different adaptation techniques that integrate in-domain information with large-scale pretrained video models, and explore the extent to which they enable novel text-conditioned generalization for robotic tasks, while also considering their independent data and resource considerations. We successfully demonstrate across robotic environments that adapting powerful video models with small scales of example data can successfully facilitate generalization to novel behaviors. In particular, we present a novel adaptation strategy, termed Inverse Probabilistic Adaptation, that not only consistently achieves strong generalization performance across robotic tasks and settings, but also exhibits robustness to the quality of adaptation data, successfully solving novel tasks even when only suboptimal in-domain demonstrations are available.
• 摘要：视频生成模型通过担任视觉计划者或政策主管来展示机器人技术的巨大希望。当在互联网规模的数据上预估计时，此类视频模型与自然语言有着密切理解的一致性，因此可以通过文本条件促进对新型下游行为的概括。但是，它们可能对代理人所居住的特定环境的特殊性不敏感。另一方面，关于机器人行为的内域示例的培训视频模型自然编码环境特定的独立性，但是可用演示的规模可能不足以支持通过自然语言规范看不见的任务的概括。在这项工作中，我们研究了不同的适应技术，这些技术将内域信息与大规模预处理的视频模型整合在一起，并探讨了它们为机器人任务实现新颖的文本条件的概括的程度，同时还考虑了其​​独立的数据和资源考虑因素。我们成功地展示了跨机器人环境，这些机器人环境将功能强大的视频模型与较小的示例相适应数据可以成功地促进对新型行为的概括。特别是，我们提出了一种新型的适应策略，称为逆概率适应，不仅可以始终在机器人任务和设置中实现强大的泛化性能，而且还表现出适应性数据质量的鲁棒性，即使仅在次优的内途中证明也可以成功地解决新任务。

Title: Towards Understanding Camera Motions in Any Video

• Authors: Zhiqiu Lin, Siyuan Cen, Daniel Jiang, Jay Karhade, Hewei Wang, Chancharik Mitra, Tiffany Ling, Yuhan Huang, Sifan Liu, Mingyu Chen, Rushikesh Zawar, Xue Bai, Yilun Du, Chuang Gan, Deva Ramanan
• Subjects: cs.CV, cs.AI, cs.CL, cs.LG, cs.MM
• Abstract URL: https://arxiv.org/abs/2504.15376
• Pdf URL: https://arxiv.org/pdf/2504.15376
• Copy Paste: [[2504.15376]] Towards Understanding Camera Motions in Any Video(https://arxiv.org/abs/2504.15376)
• Keywords: generative
• Abstract: We introduce CameraBench, a large-scale dataset and benchmark designed to assess and improve camera motion understanding. CameraBench consists of ~3,000 diverse internet videos, annotated by experts through a rigorous multi-stage quality control process. One of our contributions is a taxonomy of camera motion primitives, designed in collaboration with cinematographers. We find, for example, that some motions like "follow" (or tracking) require understanding scene content like moving subjects. We conduct a large-scale human study to quantify human annotation performance, revealing that domain expertise and tutorial-based training can significantly enhance accuracy. For example, a novice may confuse zoom-in (a change of intrinsics) with translating forward (a change of extrinsics), but can be trained to differentiate the two. Using CameraBench, we evaluate Structure-from-Motion (SfM) and Video-Language Models (VLMs), finding that SfM models struggle to capture semantic primitives that depend on scene content, while VLMs struggle to capture geometric primitives that require precise estimation of trajectories. We then fine-tune a generative VLM on CameraBench to achieve the best of both worlds and showcase its applications, including motion-augmented captioning, video question answering, and video-text retrieval. We hope our taxonomy, benchmark, and tutorials will drive future efforts towards the ultimate goal of understanding camera motions in any video.
• 摘要：我们介绍了Camerabench，这是一个大规模数据集和基准测试，旨在评估和改善相机运动的理解。 Camerabench由约3,000个不同的互联网视频组成，通过严格的多级质量控制过程，由专家注释。我们的贡献之一是与摄影师合​​作设计的相机运动原始图的分类法。例如，我们发现某些动议（例如“关注”（或跟踪））需要了解诸如移动主题之类的场景内容。我们进行了一项大规模的人类研究，以量化人类注释绩效，揭示了领域的专业知识和基于教程的培训可以显着提高准确性。例如，新手可能会使缩放（内在的变化）与向前的翻译（外部更改）混淆，但可以训练以区分两者。我们使用摄影仪，我们评估结构 - 触发（SFM）和视频语言模型（VLMS），发现SFM模型难以捕获依赖场景内容的语义原始素，而VLMS则难以捕获需要精确估算轨迹的几何原始图。然后，我们在Camerabench上微调了生成的VLM，以实现两全其美的最佳状态并展示其应用程序，包括运动提示字幕，视频询问和视频文本检索。我们希望我们的分类学，基准和教程将推动未来的努力，以了解任何视频中相机动作的最终目标。

Title: MirrorVerse: Pushing Diffusion Models to Realistically Reflect the World

• Authors: Ankit Dhiman, Manan Shah, R Venkatesh Babu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15397
• Pdf URL: https://arxiv.org/pdf/2504.15397
• Copy Paste: [[2504.15397]] MirrorVerse: Pushing Diffusion Models to Realistically Reflect the World(https://arxiv.org/abs/2504.15397)
• Keywords: generation, generative
• Abstract: Diffusion models have become central to various image editing tasks, yet they often fail to fully adhere to physical laws, particularly with effects like shadows, reflections, and occlusions. In this work, we address the challenge of generating photorealistic mirror reflections using diffusion-based generative models. Despite extensive training data, existing diffusion models frequently overlook the nuanced details crucial to authentic mirror reflections. Recent approaches have attempted to resolve this by creating synhetic datasets and framing reflection generation as an inpainting task; however, they struggle to generalize across different object orientations and positions relative to the mirror. Our method overcomes these limitations by introducing key augmentations into the synthetic data pipeline: (1) random object positioning, (2) randomized rotations, and (3) grounding of objects, significantly enhancing generalization across poses and placements. To further address spatial relationships and occlusions in scenes with multiple objects, we implement a strategy to pair objects during dataset generation, resulting in a dataset robust enough to handle these complex scenarios. Achieving generalization to real-world scenes remains a challenge, so we introduce a three-stage training curriculum to develop the MirrorFusion 2.0 model to improve real-world performance. We provide extensive qualitative and quantitative evaluations to support our approach. The project page is available at: this https URL.
• 摘要：扩散模型已成为各种图像编辑任务的核心，但是它们通常无法完全遵守物理定律，尤其是诸如阴影，反射和遮挡之类的效果。在这项工作中，我们解决了使用基于扩散的生成模型生成光真镜反射的挑战。尽管大量培训数据，但现有的扩散模型经常忽略对真实镜像至关重要的细微细节。最近的方法试图通过创建同步数据集并将反射生成作为介入任务来解决这一问题。但是，他们很难跨越相对于镜子的不同对象取向和位置概括。我们的方法通过将密钥增强引入合成数据管道来克服这些局限性：（1）随机对象定位，（2）随机旋转和（3）对象的接地，从而显着增强跨姿势和位置的概括。为了进一步解决具有多个对象的场景中的空间关系和遮挡，我们在数据集生成过程中实施了配对对象的策略，从而使数据集稳健地稳健以处理这些复杂的方案。实现对实际场景的概括仍然是一个挑战，因此我们引入了三阶段的培训课程，以开发MirrorFusion 2.0模型以提高现实世界的性能。我们提供广泛的定性和定量评估以支持我们的方法。项目页面可用：此HTTPS URL。

Title: Manifold Induced Biases for Zero-shot and Few-shot Detection of Generated Images

• Authors: Jonathan Brokman, Amit Giloni, Omer Hofman, Roman Vainshtein, Hisashi Kojima, Guy Gilboa
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15470
• Pdf URL: https://arxiv.org/pdf/2504.15470
• Copy Paste: [[2504.15470]] Manifold Induced Biases for Zero-shot and Few-shot Detection of Generated Images(https://arxiv.org/abs/2504.15470)
• Keywords: generative
• Abstract: Distinguishing between real and AI-generated images, commonly referred to as 'image detection', presents a timely and significant challenge. Despite extensive research in the (semi-)supervised regime, zero-shot and few-shot solutions have only recently emerged as promising alternatives. Their main advantage is in alleviating the ongoing data maintenance, which quickly becomes outdated due to advances in generative technologies. We identify two main gaps: (1) a lack of theoretical grounding for the methods, and (2) significant room for performance improvements in zero-shot and few-shot regimes. Our approach is founded on understanding and quantifying the biases inherent in generated content, where we use these quantities as criteria for characterizing generated images. Specifically, we explore the biases of the implicit probability manifold, captured by a pre-trained diffusion model. Through score-function analysis, we approximate the curvature, gradient, and bias towards points on the probability manifold, establishing criteria for detection in the zero-shot regime. We further extend our contribution to the few-shot setting by employing a mixture-of-experts methodology. Empirical results across 20 generative models demonstrate that our method outperforms current approaches in both zero-shot and few-shot settings. This work advances the theoretical understanding and practical usage of generated content biases through the lens of manifold analysis.
• 摘要：区分实际和AI生成的图像，通常称为“图像检测”，提出了及时而重大的挑战。尽管对（半）监督制度进行了广泛的研究，但零射门和少量解决方案最近才成为有前途的替代方案。他们的主要优势是减轻正在进行的数据维护，由于生成技术的进步，该数据迅速变得过时。我们确定了两个主要差距：（1）方法缺乏理论基础，以及（2）零射击和少量射击方案的绩效改善的重要空间。我们的方法是建立在理解和量化生成内容中固有的偏见的基础上的，我们将这些数量用作表征生成图像的标准。具体而言，我们探讨了由预训练的扩散模型捕获的隐式概率歧管的偏差。通过分数功能分析，我们近似于概率歧管上的曲率，梯度和偏置，并在零拍摄方面建立了检测标准。我们通过采用专家的混合方法来进一步将贡献扩展到少量拍摄设置。 20种生成模型的经验结果表明，我们的方法在零拍和很少的设置中都优于当前方法。这项工作通过流动分析的角度提高了理论上的理解和实际用法。

Title: Emergence and Evolution of Interpretable Concepts in Diffusion Models

• Authors: Berk Tinaz, Zalan Fabian, Mahdi Soltanolkotabi
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2504.15473
• Pdf URL: https://arxiv.org/pdf/2504.15473
• Copy Paste: [[2504.15473]] Emergence and Evolution of Interpretable Concepts in Diffusion Models(https://arxiv.org/abs/2504.15473)
• Keywords: generation, generative
• Abstract: Diffusion models have become the go-to method for text-to-image generation, producing high-quality images from noise through a process called reverse diffusion. Understanding the dynamics of the reverse diffusion process is crucial in steering the generation and achieving high sample quality. However, the inner workings of diffusion models is still largely a mystery due to their black-box nature and complex, multi-step generation process. Mechanistic Interpretability (MI) techniques, such as Sparse Autoencoders (SAEs), aim at uncovering the operating principles of models through granular analysis of their internal representations. These MI techniques have been successful in understanding and steering the behavior of large language models at scale. However, the great potential of SAEs has not yet been applied toward gaining insight into the intricate generative process of diffusion models. In this work, we leverage the SAE framework to probe the inner workings of a popular text-to-image diffusion model, and uncover a variety of human-interpretable concepts in its activations. Interestingly, we find that even before the first reverse diffusion step is completed, the final composition of the scene can be predicted surprisingly well by looking at the spatial distribution of activated concepts. Moreover, going beyond correlational analysis, we show that the discovered concepts have a causal effect on the model output and can be leveraged to steer the generative process. We design intervention techniques aimed at manipulating image composition and style, and demonstrate that (1) in early stages of diffusion image composition can be effectively controlled, (2) in the middle stages of diffusion image composition is finalized, however stylistic interventions are effective, and (3) in the final stages of diffusion only minor textural details are subject to change.
• 摘要：扩散模型已成为文本到图像生成的首选方法，通过称为反向扩散的过程从噪声中产生高质量的图像。了解反向扩散过程的动力学对于转向生成并达到高样本质量至关重要。但是，由于其黑盒性质和复杂的多步生成过程，扩散模型的内部运作仍然很大程度上是一个谜。机械性解释性（MI）技术，例如稀疏的自动编码器（SAE），旨在通过其内部表示形式分析模型的操作原理。这些MI技术已经成功地理解和指导大型语言模型的行为。但是，SAE的巨大潜力尚未用于洞悉扩散模型的复杂生成过程。在这项工作中，我们利用SAE框架来探测流行的文本对图像扩散模型的内部运作，并在其激活中发现各种人解剖概念。有趣的是，我们发现，即使在第一个反向扩散步骤完成之前，可以通过查看激活概念的空间分布来很好地预测场景的最终组成。此外，除了相关分析之外，我们表明发现的概念对模型输出有因果影响，并且可以利用以引导生成过程。我们设计了旨在操纵图像组成和样式的干预技术，并证明（1）在扩散图像组成的早期阶段可以有效地控制，（2）在扩散图像组成的中间阶段，在扩散图像组合的中间阶段是有效的，但是（3）在扩散的最终阶段仅是较小的纹理细节。

Title: Unifying Image Counterfactuals and Feature Attributions with Latent-Space Adversarial Attacks

• Authors: Jeremy Goldwasser, Giles Hooker
• Subjects: cs.LG, cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15479
• Pdf URL: https://arxiv.org/pdf/2504.15479
• Copy Paste: [[2504.15479]] Unifying Image Counterfactuals and Feature Attributions with Latent-Space Adversarial Attacks(https://arxiv.org/abs/2504.15479)
• Keywords: generative
• Abstract: Counterfactuals are a popular framework for interpreting machine learning predictions. These what if explanations are notoriously challenging to create for computer vision models: standard gradient-based methods are prone to produce adversarial examples, in which imperceptible modifications to image pixels provoke large changes in predictions. We introduce a new, easy-to-implement framework for counterfactual images that can flexibly adapt to contemporary advances in generative modeling. Our method, Counterfactual Attacks, resembles an adversarial attack on the representation of the image along a low-dimensional manifold. In addition, given an auxiliary dataset of image descriptors, we show how to accompany counterfactuals with feature attribution that quantify the changes between the original and counterfactual images. These importance scores can be aggregated into global counterfactual explanations that highlight the overall features driving model predictions. While this unification is possible for any counterfactual method, it has particular computational efficiency for ours. We demonstrate the efficacy of our approach with the MNIST and CelebA datasets.
• 摘要：反事实是解释机器学习预测的流行框架。如果众所周知，这些解释是为计算机视觉模型创建的挑战，那么这些解释很容易产生对抗性示例，在该示例中，不可察觉的修改图像像素会引起大量预测变化。我们为反事实图像引入了一个新的，易于实现的框架，可以灵活地适应当代生成建模的进步。我们的方法是反事实攻击，类似于对图像沿低维歧管的表示的对抗性攻击。此外，给定图像描述符的辅助数据集，我们展示了如何伴随反事实归因，以量化原始图像和反事实图像之间的变化。这些重要性得分可以汇总到全球反事实解释中，这些解释突出了驱动模型预测的整体特征。尽管对于任何反事实方法都是可能的统一，但它具有特殊的计算效率。我们证明了MNIST和CELEBA数据集的方法的功效。

Title: Application of Deep Generative Models for Anomaly Detection in Complex Financial Transactions

• Authors: Tengda Tang, Jianhua Yao, Yixian Wang, Qiuwu Sha, Hanrui Feng, Zhen Xu
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.15491
• Pdf URL: https://arxiv.org/pdf/2504.15491
• Copy Paste: [[2504.15491]] Application of Deep Generative Models for Anomaly Detection in Complex Financial Transactions(https://arxiv.org/abs/2504.15491)
• Keywords: generative
• Abstract: This study proposes an algorithm for detecting suspicious behaviors in large payment flows based on deep generative models. By combining Generative Adversarial Networks (GAN) and Variational Autoencoders (VAE), the algorithm is designed to detect abnormal behaviors in financial transactions. First, the GAN is used to generate simulated data that approximates normal payment flows. The discriminator identifies anomalous patterns in transactions, enabling the detection of potential fraud and money laundering behaviors. Second, a VAE is introduced to model the latent distribution of payment flows, ensuring that the generated data more closely resembles real transaction features, thus improving the model's detection accuracy. The method optimizes the generative capabilities of both GAN and VAE, ensuring that the model can effectively capture suspicious behaviors even in sparse data conditions. Experimental results show that the proposed method significantly outperforms traditional machine learning algorithms and other deep learning models across various evaluation metrics, especially in detecting rare fraudulent behaviors. Furthermore, this study provides a detailed comparison of performance in recognizing different transaction patterns (such as normal, money laundering, and fraud) in large payment flows, validating the advantages of generative models in handling complex financial data.
• 摘要：这项研究提出了一种算法，用于根据深层生成模型检测大量付款流中的可疑行为。通过结合生成对抗网络（GAN）和变异自动编码器（VAE），该算法旨在检测金融交易中的异常行为。首先，GAN用于生成近似正常付款流量的模拟数据。歧视者确定了交易中的异常模式，从而可以发现潜在的欺诈和洗钱行为。其次，引入了VAE来对付款流的潜在分布进行建模，从而确保生成的数据更类似于实际交易功能，从而提高了模型的检测准确性。该方法优化了GAN和VAE的生成能力，从而确保该模型即使在稀疏数据条件下也可以有效地捕获可疑行为。实验结果表明，所提出的方法在各种评估指标上，尤其是在检测罕见的欺诈行为时，该方法显着胜过传统的机器学习算法和其他深度学习模型。此外，这项研究在识别大量支付流量中识别不同交易模式（例如正常，洗钱和欺诈）方面的性能进行了详细的比较，从而验证了生成模型在处理复杂财务数据中的优势。

Title: InstaRevive: One-Step Image Enhancement via Dynamic Score Matching

• Authors: Yixuan Zhu, Haolin Wang, Ao Li, Wenliang Zhao, Yansong Tang, Jingxuan Niu, Lei Chen, Jie Zhou, Jiwen Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15513
• Pdf URL: https://arxiv.org/pdf/2504.15513
• Copy Paste: [[2504.15513]] InstaRevive: One-Step Image Enhancement via Dynamic Score Matching(https://arxiv.org/abs/2504.15513)
• Keywords: generative
• Abstract: Image enhancement finds wide-ranging applications in real-world scenarios due to complex environments and the inherent limitations of imaging devices. Recent diffusion-based methods yield promising outcomes but necessitate prolonged and computationally intensive iterative sampling. In response, we propose InstaRevive, a straightforward yet powerful image enhancement framework that employs score-based diffusion distillation to harness potent generative capability and minimize the sampling steps. To fully exploit the potential of the pre-trained diffusion model, we devise a practical and effective diffusion distillation pipeline using dynamic control to address inaccuracies in updating direction during score matching. Our control strategy enables a dynamic diffusing scope, facilitating precise learning of denoising trajectories within the diffusion model and ensuring accurate distribution matching gradients during training. Additionally, to enrich guidance for the generative power, we incorporate textual prompts via image captioning as auxiliary conditions, fostering further exploration of the diffusion model. Extensive experiments substantiate the efficacy of our framework across a diverse array of challenging tasks and datasets, unveiling the compelling efficacy and efficiency of InstaRevive in delivering high-quality and visually appealing results. Code is available at this https URL.
• 摘要：由于复杂的环境和成像设备的固有局限性，图像增强功能在实际场景中找到了广泛的应用程序。最近的基于扩散的方法产生了有希望的结果，但需要延长计算密集型的迭代采样。作为响应，我们提出了InstareVive，这是一种直接而强大的图像增强框架，它采用基于得分的扩散蒸馏来利用有效的生成能力并最大程度地减少采样步骤。为了充分利用预训练的扩散模型的潜力，我们使用动态控制来设计一种实用有效的扩散蒸馏管，以解决分数匹配期间更新方向的不准确性。我们的控制策略使动态扩散范围有助于在扩散模型中精确学习denoising轨迹，并确保训练期间的准确分布匹配梯度。此外，为了丰富生成能力的指导，我们通过图像字幕作为辅助条件结合了文本提示，从而进一步探索了扩散模型。广泛的实验证实了我们框架在各种具有挑战性的任务和数据集中的功效，从而揭示了InspareVive在提供高质量和视觉吸引力的结果方面的引人注目的功效和效率。代码可在此HTTPS URL上找到。

Title: MetaMolGen: A Neural Graph Motif Generation Model for De Novo Molecular Design

• Authors: Zimo Yan, Jie Zhang, Zheng Xie, Chang Liu, Yizhen Liu, Yiping Song
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.15587
• Pdf URL: https://arxiv.org/pdf/2504.15587
• Copy Paste: [[2504.15587]] MetaMolGen: A Neural Graph Motif Generation Model for De Novo Molecular Design(https://arxiv.org/abs/2504.15587)
• Keywords: generation, generative
• Abstract: Molecular generation plays an important role in drug discovery and materials science, especially in data-scarce scenarios where traditional generative models often struggle to achieve satisfactory conditional generalization. To address this challenge, we propose MetaMolGen, a first-order meta-learning-based molecular generator designed for few-shot and property-conditioned molecular generation. MetaMolGen standardizes the distribution of graph motifs by mapping them to a normalized latent space, and employs a lightweight autoregressive sequence model to generate SMILES sequences that faithfully reflect the underlying molecular structure. In addition, it supports conditional generation of molecules with target properties through a learnable property projector integrated into the generative this http URL results demonstrate that MetaMolGen consistently generates valid and diverse SMILES sequences under low-data regimes, outperforming conventional baselines. This highlights its advantage in fast adaptation and efficient conditional generation for practical molecular design.
• 摘要：Molecular generation plays an important role in drug discovery and materials science, especially in data-scarce scenarios where traditional generative models often struggle to achieve satisfactory conditional generalization.为了应对这一挑战，我们提出了Metamolgen，这是一种基于一阶元学习的分子发生器，旨在几乎没有拍摄和财产条件的分子生成。 metamolgen通过将图基序映射到归一化潜在空间，并采用轻质自动回归序列模型来生成忠实地反映基本分子结构的微笑序列，从而标准化图基序的分布。此外，它通过集成到生成的HTTP URL结果中的可学习的投影仪来支持具有目标性质的有条件生成，该HTTP URL结果表明，Metamolgen始终在低数据表方面生成有效和多样的微笑序列，超过了传统的基础线。这突出了它在快速适应和有效的有条件产生方面的优势。

Title: RadioDiff-$k^2$: Helmholtz Equation Informed Generative Diffusion Model for Multi-Path Aware Radio Map Construction

• Authors: Xiucheng Wang, Qiming Zhang, Nan Cheng, Ruijin Sun, Zan Li, Shuguang Cui, Xuemin Shen
• Subjects: cs.LG, eess.SY
• Abstract URL: https://arxiv.org/abs/2504.15623
• Pdf URL: https://arxiv.org/pdf/2504.15623
• Copy Paste: [[2504.15623]] RadioDiff-$k^2$: Helmholtz Equation Informed Generative Diffusion Model for Multi-Path Aware Radio Map Construction(https://arxiv.org/abs/2504.15623)
• Keywords: generation, generative
• Abstract: In this paper, we propose a novel physics-informed generative learning approach, termed RadioDiff-$\bm{k^2}$, for accurate and efficient multipath-aware radio map (RM) construction. As wireless communication evolves towards environment-aware paradigms, driven by the increasing demand for intelligent and proactive optimization in sixth-generation (6G) networks, accurate construction of RMs becomes crucial yet highly challenging. Conventional electromagnetic (EM)-based methods, such as full-wave solvers and ray-tracing approaches, exhibit substantial computational overhead and limited adaptability to dynamic scenarios. Although, existing neural network (NN) approaches have efficient inferencing speed, they lack sufficient consideration of the underlying physics of EM wave propagation, limiting their effectiveness in accurately modeling critical EM singularities induced by complex multipath environments. To address these fundamental limitations, we propose a novel physics-inspired RM construction method guided explicitly by the Helmholtz equation, which inherently governs EM wave propagation. Specifically, we theoretically establish a direct correspondence between EM singularities, which correspond to the critical spatial features influencing wireless propagation, and regions defined by negative wave numbers in the Helmholtz equation. Based on this insight, we design an innovative dual generative diffusion model (DM) framework comprising one DM dedicated to accurately inferring EM singularities and another DM responsible for reconstructing the complete RM using these singularities along with environmental contextual information. Our physics-informed approach uniquely combines the efficiency advantages of data-driven methods with rigorous physics-based EM modeling, significantly enhancing RM accuracy, particularly in complex propagation environments dominated by multipath effects.
• 摘要：In this paper, we propose a novel physics-informed generative learning approach, termed RadioDiff-$\bm{k^2}$, for accurate and efficient multipath-aware radio map (RM) construction.随着无线沟通朝着环境感知的范式发展，这是由于对智能和主动优化在第六代（6G）网络中的需求不断增长所致，因此，准确的RMS构造变得至关重要却高度挑战。常规电磁（EM）的方法（例如全波求解器和射线追踪方法）表现出很大的计算开销，并且对动态场景的适应性有限。尽管现有的神经网络（NN）方法具有有效的推理速度，但它们缺乏足够的考虑到EM波传播的潜在物理，从而限制了它们在准确地建模由复杂多径环境引起的临界EM奇异性中的有效性。为了解决这些基本局限性，我们提出了一种由Helmholtz方程明确指导的新型物理启发的RM构造方法，该方法固有地控制EM波传播。具体而言，我们从理论上建立了EM奇点之间的直接对应关系，这与影响无线传播的关键空间特征相对应，而由Helmholtz方程中的负波数定义的区域。基于这种见解，我们设计了一个创新的双重生成扩散模型（DM）框架，该框架包含一个DM，该框架专门用于准确地推断出EM奇异性，另一个负责使用这些奇异性以及环境上下文信息来重建完整RM的DM。我们的物理知识方法唯一地将数据驱动方法的效率优势与严格的基于物理的EM建模相结合，从而显着提高了RM的精度，尤其是在以多径效应为主的复杂传播环境中。

Title: DiTPainter: Efficient Video Inpainting with Diffusion Transformers

• Authors: Xian Wu, Chang Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15661
• Pdf URL: https://arxiv.org/pdf/2504.15661
• Copy Paste: [[2504.15661]] DiTPainter: Efficient Video Inpainting with Diffusion Transformers(https://arxiv.org/abs/2504.15661)
• Keywords: generation
• Abstract: Many existing video inpainting algorithms utilize optical flows to construct the corresponding maps and then propagate pixels from adjacent frames to missing areas by mapping. Despite the effectiveness of the propagation mechanism, they might encounter blurry and inconsistencies when dealing with inaccurate optical flows or large masks. Recently, Diffusion Transformer (DiT) has emerged as a revolutionary technique for video generation tasks. However, pretrained DiT models for video generation all contain a large amount of parameters, which makes it very time consuming to apply to video inpainting tasks. In this paper, we present DiTPainter, an end-to-end video inpainting model based on Diffusion Transformer (DiT). DiTPainter uses an efficient transformer network designed for video inpainting, which is trained from scratch instead of initializing from any large pretrained models. DiTPainter can address videos with arbitrary lengths and can be applied to video decaptioning and video completion tasks with an acceptable time cost. Experiments show that DiTPainter outperforms existing video inpainting algorithms with higher quality and better spatial-temporal consistency.
• 摘要：Many existing video inpainting algorithms utilize optical flows to construct the corresponding maps and then propagate pixels from adjacent frames to missing areas by mapping.尽管传播机制具有有效性，但在处理不准确的光流或大面具时，它们可能会遇到模糊和不一致。最近，扩散变压器（DIT）已成为视频生成任务的革命性技术。但是，预测的视频生成DIT模型都包含大量参数，这使得应用于视频介绍任务非常耗时。在本文中，我们提出了Ditpainter，这是一种基于扩散变压器（DIT）的端到端视频介绍模型。 Ditpainter使用了设计用于视频介绍的高效变压器网络，该网络是从头开始训练的，而不是从任何大型审慎模型中初始化。 DitPainter可以用任意长度解决视频，并可以将视频脱发和视频完成任务应用于可接受的时间成本。实验表明，Ditpainter的表现优于现有的视频，具有更高质量和更好的时空一致性的视频介绍算法。

Title: Riemannian Neural Geodesic Interpolant

• Authors: Jiawen Wu, Bingguang Chen, Yuyi Zhou, Qi Meng, Rongchan Zhu, Zhi-Ming Ma
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2504.15736
• Pdf URL: https://arxiv.org/pdf/2504.15736
• Copy Paste: [[2504.15736]] Riemannian Neural Geodesic Interpolant(https://arxiv.org/abs/2504.15736)
• Keywords: generation, generative
• Abstract: Stochastic interpolants are efficient generative models that bridge two arbitrary probability density functions in finite time, enabling flexible generation from the source to the target distribution or vice versa. These models are primarily developed in Euclidean space, and are therefore limited in their application to many distribution learning problems defined on Riemannian manifolds in real-world scenarios. In this work, we introduce the Riemannian Neural Geodesic Interpolant (RNGI) model, which interpolates between two probability densities on a Riemannian manifold along the stochastic geodesics, and then samples from one endpoint as the final state using the continuous flow originating from the other endpoint. We prove that the temporal marginal density of RNGI solves a transport equation on the Riemannian manifold. After training the model's the neural velocity and score fields, we propose the Embedding Stochastic Differential Equation (E-SDE) algorithm for stochastic sampling of RNGI. E-SDE significantly improves the sampling quality by reducing the accumulated error caused by the excessive intrinsic discretization of Riemannian Brownian motion in the classical Geodesic Random Walk (GRW) algorithm. We also provide theoretical bounds on the generative bias measured in terms of KL-divergence. Finally, we demonstrate the effectiveness of the proposed RNGI and E-SDE through experiments conducted on both collected and synthetic distributions on S2 and SO(3).
• 摘要：Stochastic interpolants are efficient generative models that bridge two arbitrary probability density functions in finite time, enabling flexible generation from the source to the target distribution or vice versa.这些模型主要是在欧几里得空间中开发的，因此在实际情况下在Riemannian歧管上定义的许多分布学习问题的应用中，它们的应用限制。在这项工作中，我们介绍了Riemannian神经地球插入术（RNGI）模型，该模型在沿随机地理学的Riemannian歧管上的两个概率密度之间进行了插值，然后使用一个端点作为最终状态的样品作为最终状态，使用来自另一个端点的连续流。我们证明，RNGI的时间边缘密度解决了Riemannian歧管上的传输方程。在训练模型的神经速度和评分场之后，我们提出了用于RNGI随机采样的嵌入随机微分方程（E-SDE）算法。 E-SDE通过降低经典的测量随机步行（GRW）算法中的Riemannian Brownian运动的过度固有离散化引起的累积误差来显着提高采样质量。我们还可以根据KL差异测量的生成偏差提供理论界限。最后，我们通过对S2等上的收集和合成分布进行的实验来证明所提出的RNGI和E-SDE的有效性（3）。

Title: Grounded in Context: Retrieval-Based Method for Hallucination Detection

• Authors: Assaf Gerner, Netta Madvil, Nadav Barak, Alex Zaikman, Jonatan Liberman, Liron Hamra, Rotem Brazilay, Shay Tsadok, Yaron Friedman, Neal Harow, Noam Bresler, Shir Chorev, Philip Tannor
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2504.15771
• Pdf URL: https://arxiv.org/pdf/2504.15771
• Copy Paste: [[2504.15771]] Grounded in Context: Retrieval-Based Method for Hallucination Detection(https://arxiv.org/abs/2504.15771)
• Keywords: generation
• Abstract: Despite advancements in grounded content generation, production Large Language Models (LLMs) based applications still suffer from hallucinated answers. We present "Grounded in Context" - Deepchecks' hallucination detection framework, designed for production-scale long-context data and tailored to diverse use cases, including summarization, data extraction, and RAG. Inspired by RAG architecture, our method integrates retrieval and Natural Language Inference (NLI) models to predict factual consistency between premises and hypotheses using an encoder-based model with only a 512-token context window. Our framework identifies unsupported claims with an F1 score of 0.83 in RAGTruth's response-level classification task, matching methods that trained on the dataset, and outperforming all comparable frameworks using similar-sized models.
• 摘要：Despite advancements in grounded content generation, production Large Language Models (LLMs) based applications still suffer from hallucinated answers.我们介绍了“以上下文为基础” - Deepnecks的幻觉检测框架，专为生产规模的长篇小写数据而设计，并针对各种用例量身定制，包括汇总，数据提取和抹布。受拉格体系结构的启发，我们的方法集成了检索和自然语言推断（NLI）模型，以使用基于编码器的模型仅使用512 token上下文窗口来预测前提和假设之间的事实一致性。我们的框架在Ragtruth的响应级分类任务，在数据集中训练的匹配方法中识别了不支持的主张，F1得分为0.83，并使用相似尺寸的模型优于所有可比较的框架。

Title: Clifford Group Equivariant Diffusion Models for 3D Molecular Generation

• Authors: Cong Liu, Sharvaree Vadgama, David Ruhe, Erik Bekkers, Patrick Forrè
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.15773
• Pdf URL: https://arxiv.org/pdf/2504.15773
• Copy Paste: [[2504.15773]] Clifford Group Equivariant Diffusion Models for 3D Molecular Generation(https://arxiv.org/abs/2504.15773)
• Keywords: generation, generative
• Abstract: This paper explores leveraging the Clifford algebra's expressive power for $\E(n)$-equivariant diffusion models. We utilize the geometric products between Clifford multivectors and the rich geometric information encoded in Clifford subspaces in \emph{Clifford Diffusion Models} (CDMs). We extend the diffusion process beyond just Clifford one-vectors to incorporate all higher-grade multivector subspaces. The data is embedded in grade-$k$ subspaces, allowing us to apply latent diffusion across complete multivectors. This enables CDMs to capture the joint distribution across different subspaces of the algebra, incorporating richer geometric information through higher-order features. We provide empirical results for unconditional molecular generation on the QM9 dataset, showing that CDMs provide a promising avenue for generative modeling.
• 摘要：本文探讨了利用Clifford代数的表达能力（$ \ e（n）$ - 等效扩散模型）。我们在\ emph {Clifford扩散模型}（CDMS）的Clifford子空间中编码的Clifford多生和丰富的几何信息之间使用了几何产物。我们将扩散过程不仅仅是Clifford One-One-On-Quern扩展，以结合所有高级的多派子空间。数据嵌入到级$ k $子空间中，使我们能够在完整的多元电机之间应用潜在扩散。这使CDM能够捕获跨代数的不同子空间的关节分布，从而通过高阶特征结合了更丰富的几何信息。我们为QM9数据集上的无条件分子生成提供了经验结果，这表明CDM为生成建模提供了有希望的途径。

Title: Satellite to GroundScape -- Large-scale Consistent Ground View Generation from Satellite Views

• Authors: Ningli Xu, Rongjun Qin
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15786
• Pdf URL: https://arxiv.org/pdf/2504.15786
• Copy Paste: [[2504.15786]] Satellite to GroundScape -- Large-scale Consistent Ground View Generation from Satellite Views(https://arxiv.org/abs/2504.15786)
• Keywords: generation
• Abstract: Generating consistent ground-view images from satellite imagery is challenging, primarily due to the large discrepancies in viewing angles and resolution between satellite and ground-level domains. Previous efforts mainly concentrated on single-view generation, often resulting in inconsistencies across neighboring ground views. In this work, we propose a novel cross-view synthesis approach designed to overcome these challenges by ensuring consistency across ground-view images generated from satellite views. Our method, based on a fixed latent diffusion model, introduces two conditioning modules: satellite-guided denoising, which extracts high-level scene layout to guide the denoising process, and satellite-temporal denoising, which captures camera motion to maintain consistency across multiple generated views. We further contribute a large-scale satellite-ground dataset containing over 100,000 perspective pairs to facilitate extensive ground scene or video generation. Experimental results demonstrate that our approach outperforms existing methods on perceptual and temporal metrics, achieving high photorealism and consistency in multi-view outputs.
• 摘要：从卫星图像中产生一致的地面视图图像是具有挑战性的，这主要是由于观察角度和卫星和地面域之间的分辨率较大的差异。先前的努力主要集中在单视图上，通常会导致邻近地面视图的不一致。在这项工作中，我们提出了一种新颖的跨视图合成方法，旨在通过确保从卫星视图产生的地面视图图像的一致性来克服这些挑战。我们的方法基于固定的潜在扩散模型，引入了两个条件模块：卫星引导的Denoisising，它提取高级场景布局以指导Denoising Process和satellite-tomporal denoisising，从而捕获相机运动以保持跨多个生成的视图的一致性。我们进一步贡献了一个大规模的卫星地面数据集，其中包含100,000多个透视对，以促进广泛的地面场景或视频生成。实验结果表明，我们的方法优于感知和时间指标的现有方法，从而达到了多视图输出的高光真实性和一致性。

Title: DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers

• Authors: Xuyang Zhong, Haochen Luo, Chen Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2504.15827
• Pdf URL: https://arxiv.org/pdf/2504.15827
• Copy Paste: [[2504.15827]] DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers(https://arxiv.org/abs/2504.15827)
• Keywords: generation
• Abstract: Existing machine unlearning (MU) approaches exhibit significant sensitivity to hyperparameters, requiring meticulous tuning that limits practical deployment. In this work, we first empirically demonstrate the instability and suboptimal performance of existing popular MU methods when deployed in different scenarios. To address this issue, we propose Dual Optimizer (DualOptim), which incorporates adaptive learning rate and decoupled momentum factors. Empirical and theoretical evidence demonstrates that DualOptim contributes to effective and stable unlearning. Through extensive experiments, we show that DualOptim can significantly boost MU efficacy and stability across diverse tasks, including image classification, image generation, and large language models, making it a versatile approach to empower existing MU algorithms.
• 摘要：现有的机器未学习（MU）方法对超参数表现出明显的敏感性，需要细致的调整，以限制实际部署。在这项工作中，我们首先在经验上证明了在不同情况下部署时现有流行的MU方法的不稳定性和次优性能。为了解决这个问题，我们提出了双重优化器（DualOptim），该优化器结合了自适应学习率和脱钩动量因素。经验和理论证据表明，DualopTim有助于有效稳定的学习。通过广泛的实验，我们表明DualopTim可以显着提高各种任务的MU功效和稳定性，包括图像分类，图像产生和大型语言模型，使其成为增强现有MU算法能力的多功能方法。

Title: Text-based Animatable 3D Avatars with Morphable Model Alignment

• Authors: Yiqian Wu, Malte Prinzler, Xiaogang Jin, Siyu Tang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15835
• Pdf URL: https://arxiv.org/pdf/2504.15835
• Copy Paste: [[2504.15835]] Text-based Animatable 3D Avatars with Morphable Model Alignment(https://arxiv.org/abs/2504.15835)
• Keywords: generation
• Abstract: The generation of high-quality, animatable 3D head avatars from text has enormous potential in content creation applications such as games, movies, and embodied virtual assistants. Current text-to-3D generation methods typically combine parametric head models with 2D diffusion models using score distillation sampling to produce 3D-consistent results. However, they struggle to synthesize realistic details and suffer from misalignments between the appearance and the driving parametric model, resulting in unnatural animation results. We discovered that these limitations stem from ambiguities in the 2D diffusion predictions during 3D avatar distillation, specifically: i) the avatar's appearance and geometry is underconstrained by the text input, and ii) the semantic alignment between the predictions and the parametric head model is insufficient because the diffusion model alone cannot incorporate information from the parametric model. In this work, we propose a novel framework, AnimPortrait3D, for text-based realistic animatable 3DGS avatar generation with morphable model alignment, and introduce two key strategies to address these challenges. First, we tackle appearance and geometry ambiguities by utilizing prior information from a pretrained text-to-3D model to initialize a 3D avatar with robust appearance, geometry, and rigging relationships to the morphable model. Second, we refine the initial 3D avatar for dynamic expressions using a ControlNet that is conditioned on semantic and normal maps of the morphable model to ensure accurate alignment. As a result, our method outperforms existing approaches in terms of synthesis quality, alignment, and animation fidelity. Our experiments show that the proposed method advances the state of the art in text-based, animatable 3D head avatar generation.
• 摘要：文本中的高质量，动画3D头像的产生在内容创建应用程序（例如游戏，电影和体现的虚拟助手）方面具有巨大的潜力。当前的文本到3D生成方法通常将参数头模型与2D扩散模型相结合，并使用得分蒸馏采样来产生3D一致的结果。但是，他们很难综合现实的细节，并遭受外观和驾驶参数模型之间的不对对准，从而产生了不自然的动画结果。我们发现这些局限性源于3D阿凡达蒸馏期间2D扩散预测的歧义，特别是：i）阿凡达的外观和几何形状不受文本输入的限制，ii）ii）ii）预测和参数模型之间的语义一致性不足，因为单独使用参数模型不包含参数模型。在这项工作中，我们提出了一个新颖的框架AnimporTrait3D，用于基于文本的现实动画3DGS AVATAR生成，具有可变形的模型对齐，并引入了两种关键策略来应对这些挑战。首先，我们通过利用预验证的文本到3D模型的先前信息来解决外观和几何歧义，以初始化具有强大外观，几何形状和与可变形模型的3D头像。其次，我们使用ControlNET改进了动态表达式的初始3D AVATAR，该控制网基于可变形模型的语义和正常地图，以确保准确的比对。结果，我们的方法在合成质量，对齐和动画保真度方面优于现有方法。我们的实验表明，所提出的方法在基于文本的，动画的3D Head Avatar生成中推进了最新技术。

Title: StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation

• Authors: Yinmin Zhong, Zili Zhang, Xiaoniu Song, Hanpeng Hu, Chao Jin, Bingyang Wu, Nuo Chen, Yukun Chen, Yu Zhou, Changyi Wan, Hongyu Zhou, Yimin Jiang, Yibo Zhu, Daxin Jiang
• Subjects: cs.LG, cs.DC
• Abstract URL: https://arxiv.org/abs/2504.15930
• Pdf URL: https://arxiv.org/pdf/2504.15930
• Copy Paste: [[2504.15930]] StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation(https://arxiv.org/abs/2504.15930)
• Keywords: generation
• Abstract: Reinforcement learning (RL) has become the core post-training technique for large language models (LLMs). RL for LLMs involves two stages: generation and training. The LLM first generates samples online, which are then used to derive rewards for training. The conventional view holds that the colocated architecture, where the two stages share resources via temporal multiplexing, outperforms the disaggregated architecture, in which dedicated resources are assigned to each stage. However, in real-world deployments, we observe that the colocated architecture suffers from resource coupling, where the two stages are constrained to use the same resources. This coupling compromises the scalability and cost-efficiency of colocated RL in large-scale training. In contrast, the disaggregated architecture allows for flexible resource allocation, supports heterogeneous training setups, and facilitates cross-datacenter deployment. StreamRL is designed with disaggregation from first principles and fully unlocks its potential by addressing two types of performance bottlenecks in existing disaggregated RL frameworks: pipeline bubbles, caused by stage dependencies, and skewness bubbles, resulting from long-tail output length distributions. To address pipeline bubbles, StreamRL breaks the traditional stage boundary in synchronous RL algorithms through stream generation and achieves full overlapping in asynchronous RL. To address skewness bubbles, StreamRL employs an output-length ranker model to identify long-tail samples and reduces generation time via skewness-aware dispatching and scheduling. Experiments show that StreamRL improves throughput by up to 2.66x compared to existing state-of-the-art systems, and improves cost-effectiveness by up to 1.33x in a heterogeneous, cross-datacenter setting.
• 摘要：强化学习（RL）已成为大型语言模型（LLMS）的核心培训技术。 LLM的RL涉及两个阶段：发电和培训。 LLM首先在线生成样本，然后将其用于获得培训的奖励。传统的观点认为，两个阶段通过时间多路复用共享资源的共裂结构优于分类的体系结构，在该体系结构中，将专用资源分配给每个阶段。但是，在现实世界的部署中，我们观察到，共裂的体系结构遭受资源耦合的困扰，其中两个阶段被约束以使用相同的资源。这种耦合损害了大规模培训中共凝结的RL的可扩展性和成本效益。相比之下，分类的体系结构允许灵活的资源分配，支持异质培训设置，并促进交叉部署。 StreamRL的设计与第一原则分类，并通过在现有的分类RL框架中解决两种类型的性能瓶颈完全解锁其潜力：由舞台依赖性引​​起的管道气泡和偏度气泡，这是由长尾输出长度分布引起的。为了解决管道气泡，StreamRL通过流生成中的同步RL算法打破了传统的阶段边界，并在异步RL中实现了完全重叠的。为了解决偏斜气泡，StreamRL采用输出长度排名模型来识别长尾样本并通过偏斜感知的分配和调度来减少生成时间。实验表明，与现有的最新系统相比，StreamRL的吞吐量最高可提高2.66倍，并且在异质，交叉杂货店设置中，成本效益提高了1.33倍。

Title: Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learning

• Authors: Wang Lin, Liyu Jia, Wentao Hu, Kaihang Pan, Zhongqi Yue, Wei Zhao, Jingyuan Chen, Fei Wu, Hanwang Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15932
• Pdf URL: https://arxiv.org/pdf/2504.15932
• Copy Paste: [[2504.15932]] Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learning(https://arxiv.org/abs/2504.15932)
• Keywords: generation
• Abstract: Despite recent progress in video generation, producing videos that adhere to physical laws remains a significant challenge. Traditional diffusion-based methods struggle to extrapolate to unseen physical conditions (eg, velocity) due to their reliance on data-driven approximations. To address this, we propose to integrate symbolic reasoning and reinforcement learning to enforce physical consistency in video generation. We first introduce the Diffusion Timestep Tokenizer (DDT), which learns discrete, recursive visual tokens by recovering visual attributes lost during the diffusion process. The recursive visual tokens enable symbolic reasoning by a large language model. Based on it, we propose the Phys-AR framework, which consists of two stages: The first stage uses supervised fine-tuning to transfer symbolic knowledge, while the second stage applies reinforcement learning to optimize the model's reasoning abilities through reward functions based on physical conditions. Our approach allows the model to dynamically adjust and improve the physical properties of generated videos, ensuring adherence to physical laws. Experimental results demonstrate that PhysAR can generate videos that are physically consistent.
• 摘要：尽管视频生成最近取得了进展，但制作遵守物理定律的视频仍然是一个重大挑战。传统的基于扩散的方法由于依赖数据驱动的近似值而难以推断不见的身体状况（例如，速度）。为了解决这个问题，我们建议整合符号推理和增强学习，以在视频生成中实现身体一致性。我们首先介绍了扩散时间段令牌（DDT），该传递通过恢复在扩散过程中丢失的视觉属性来学习离散的递归视觉令牌。递归视觉令牌可以通过大型语言模型实现符号推理。基于它，我们提出了由两个阶段组成的Phys-AR框架：第一阶段使用监督的微调来传输符号知识，而第二阶段则采用强化学习来通过基于物理条件来优化模型的推理能力。我们的方法使模型可以动态调整和改善生成视频的物理特性，从而确保遵守物理定律。实验结果表明，物理可以生成身体一致的视频。

Title: FreeGraftor: Training-Free Cross-Image Feature Grafting for Subject-Driven Text-to-Image Generation

• Authors: Zebin Yao, Lei Ren, Huixing Jiang, Chen Wei, Xiaojie Wang, Ruifan Li, Fangxiang Feng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.15958
• Pdf URL: https://arxiv.org/pdf/2504.15958
• Copy Paste: [[2504.15958]] FreeGraftor: Training-Free Cross-Image Feature Grafting for Subject-Driven Text-to-Image Generation(https://arxiv.org/abs/2504.15958)
• Keywords: generation
• Abstract: Subject-driven image generation aims to synthesize novel scenes that faithfully preserve subject identity from reference images while adhering to textual guidance, yet existing methods struggle with a critical trade-off between fidelity and efficiency. Tuning-based approaches rely on time-consuming and resource-intensive subject-specific optimization, while zero-shot methods fail to maintain adequate subject consistency. In this work, we propose FreeGraftor, a training-free framework that addresses these limitations through cross-image feature grafting. Specifically, FreeGraftor employs semantic matching and position-constrained attention fusion to transfer visual details from reference subjects to the generated image. Additionally, our framework incorporates a novel noise initialization strategy to preserve geometry priors of reference subjects for robust feature matching. Extensive qualitative and quantitative experiments demonstrate that our method enables precise subject identity transfer while maintaining text-aligned scene synthesis. Without requiring model fine-tuning or additional training, FreeGraftor significantly outperforms existing zero-shot and training-free approaches in both subject fidelity and text alignment. Furthermore, our framework can seamlessly extend to multi-subject generation, making it practical for real-world deployment. Our code is available at this https URL.
• 摘要：主题驱动的图像生成旨在综合新的场景，这些新场景在遵守文本指导的同时忠实地将主题身份从参考图像中保存下来，但现有的方法在忠诚度和效率之间的重要权衡。基于调整的方法依赖于时间耗时和资源密集型主题特定优化，而零击方法无法保持足够的主题一致性。在这项工作中，我们提出了FreeGraftor，这是一个无训练的框架，通过交叉图像功能移植来解决这些限制。具体而言，FreeGroftor采用语义匹配和位置受限的注意力融合，从参考主题传递到生成的图像中的视觉细节。此外，我们的框架还结合了一种新型的噪声初始化策略，以保留参考主体的几何学先验，以匹配可靠的特征。广泛的定性和定量实验表明，我们的方法可以在维持文本对准场景综合的同时进行精确的主题身份转移。在不需要模型进行微调或额外培训的情况下，FreeGroftor在主题保真度和文本一致性方面的现有零击和无训练方法的表现都显着超过了。此外，我们的框架可以无缝扩展到多主体生成，使其可用于现实部署。我们的代码可在此HTTPS URL上找到。

Title: MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessment

• Authors: Yachun Mi, Yu Li, Weicheng Meng, Chaofeng Chen, Chen Hui, Shaohui Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.16003
• Pdf URL: https://arxiv.org/pdf/2504.16003
• Copy Paste: [[2504.16003]] MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessment(https://arxiv.org/abs/2504.16003)
• Keywords: quality assessment
• Abstract: The rapid growth of long-duration, high-definition videos has made efficient video quality assessment (VQA) a critical challenge. Existing research typically tackles this problem through two main strategies: reducing model parameters and resampling inputs. However, light-weight Convolution Neural Networks (CNN) and Transformers often struggle to balance efficiency with high performance due to the requirement of long-range modeling capabilities. Recently, the state-space model, particularly Mamba, has emerged as a promising alternative, offering linear complexity with respect to sequence length. Meanwhile, efficient VQA heavily depends on resampling long sequences to minimize computational costs, yet current resampling methods are often weak in preserving essential semantic information. In this work, we present MVQA, a Mamba-based model designed for efficient VQA along with a novel Unified Semantic and Distortion Sampling (USDS) approach. USDS combines semantic patch sampling from low-resolution videos and distortion patch sampling from original-resolution videos. The former captures semantically dense regions, while the latter retains critical distortion details. To prevent computation increase from dual inputs, we propose a fusion mechanism using pre-defined masks, enabling a unified sampling strategy that captures both semantic and quality information without additional computational burden. Experiments show that the proposed MVQA, equipped with USDS, achieve comparable performance to state-of-the-art methods while being $2\times$ as fast and requiring only $1/5$ GPU memory.
• 摘要：长期，高清视频的快速增长使有效的视频质量评估（VQA）成为了关键的挑战。现有研究通常通过两种主要策略解决此问题：减少模型参数和重新采样输入。但是，由于需要远程建模能力，轻巧的卷积神经网络（CNN）和变形金刚通常很难平衡效率与高性能。最近，状态空间模型，尤其是Mamba，已成为一种有希望的替代方案，相对于序列长度提供了线性复杂性。同时，有效的VQA在很大程度上取决于重采样长序列以最大程度地减少计算成本，但是当前的重采样方法在保留基本语义信息时通常却很弱。在这项工作中，我们提出了MVQA，这是一种基于MAMBA的模型，旨在有效的VQA以及一种新颖的统一语义和失真采样（USDS）方法。 USDS结合了低分辨率视频中的语义补丁采样和原始分辨率视频中的失真补丁采样。前者捕获了语义密集的区域，而后者保留了关键的失真细节。为了防止双重输入的计算增加，我们提出了使用预定义的面具提出的融合机制，从而实现了统一的采样策略，该策略可以捕获语义和质量信息，而无需其他计算负担。实验表明，配备了USD的拟议MVQA可以实现与最先进方法的可比性能，而$ 2 \ times $的$ 2 \ times $，仅需$ 1/5 $ GPU内存。

Title: Efficient Temporal Consistency in Diffusion-Based Video Editing with Adaptor Modules: A Theoretical Framework

• Authors: Xinyuan Song, Yangfan He, Sida Li, Jianhui Wang, Hongyang He, Xinhang Yuan, Ruoyu Wang, Jiaqi Chen, Keqin Li, Kuan Lu, Menghao Huo, Binxu Li, Pei Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.16016
• Pdf URL: https://arxiv.org/pdf/2504.16016
• Copy Paste: [[2504.16016]] Efficient Temporal Consistency in Diffusion-Based Video Editing with Adaptor Modules: A Theoretical Framework(https://arxiv.org/abs/2504.16016)
• Keywords: generation
• Abstract: Adapter-based methods are commonly used to enhance model performance with minimal additional complexity, especially in video editing tasks that require frame-to-frame consistency. By inserting small, learnable modules into pretrained diffusion models, these adapters can maintain temporal coherence without extensive retraining. Approaches that incorporate prompt learning with both shared and frame-specific tokens are particularly effective in preserving continuity across frames at low training cost. In this work, we want to provide a general theoretical framework for adapters that maintain frame consistency in DDIM-based models under a temporal consistency loss. First, we prove that the temporal consistency objective is differentiable under bounded feature norms, and we establish a Lipschitz bound on its gradient. Second, we show that gradient descent on this objective decreases the loss monotonically and converges to a local minimum if the learning rate is within an appropriate range. Finally, we analyze the stability of modules in the DDIM inversion procedure, showing that the associated error remains controlled. These theoretical findings will reinforce the reliability of diffusion-based video editing methods that rely on adapter strategies and provide theoretical insights in video generation tasks.
• 摘要：基于适配器的方法通常用于增强模型性能，尤其是在需要框架到框架一致性的视频编辑任务中。通过将较小的可学习模块插入预验证的扩散模型中，这些适配器可以保持时间连贯性而无需大量重新训练。将迅速学习与共享和特定框架的代币一起融合在一起的方法在以低训练成本以跨框架保持连续性特别有效。在这项工作中，我们希望为适配器提供一个一般的理论框架，这些框架在时间一致性损失下保持基于DDIM的模型中的框架一致性。首先，我们证明，在有限的特征规范下，时间一致性目标是可以区分的，并且我们建立了lipschitz的梯度。其次，我们表明，该目标上的梯度下降会单调降低损失，如果学习率在适当的范围内，则会收敛到局部最小值。最后，我们分析了DDIM反转过程中模块的稳定性，表明关联的误差仍然受到控制。这些理论发现将增强基于扩散的视频编辑方法的可靠性，这些视频编辑方法依靠适配器策略并提供了在视频生成任务中的理论见解。

Title: LLMs meet Federated Learning for Scalable and Secure IoT Management

• Authors: Yazan Otoum, Arghavan Asad, Amiya Nayak
• Subjects: cs.LG, cs.AI, cs.ET
• Abstract URL: https://arxiv.org/abs/2504.16032
• Pdf URL: https://arxiv.org/pdf/2504.16032
• Copy Paste: [[2504.16032]] LLMs meet Federated Learning for Scalable and Secure IoT Management(https://arxiv.org/abs/2504.16032)
• Keywords: generative
• Abstract: The rapid expansion of IoT ecosystems introduces severe challenges in scalability, security, and real-time decision-making. Traditional centralized architectures struggle with latency, privacy concerns, and excessive resource consumption, making them unsuitable for modern large-scale IoT deployments. This paper presents a novel Federated Learning-driven Large Language Model (FL-LLM) framework, designed to enhance IoT system intelligence while ensuring data privacy and computational efficiency. The framework integrates Generative IoT (GIoT) models with a Gradient Sensing Federated Strategy (GSFS), dynamically optimizing model updates based on real-time network conditions. By leveraging a hybrid edge-cloud processing architecture, our approach balances intelligence, scalability, and security in distributed IoT environments. Evaluations on the IoT-23 dataset demonstrate that our framework improves model accuracy, reduces response latency, and enhances energy efficiency, outperforming traditional FL techniques (i.e., FedAvg, FedOpt). These findings highlight the potential of integrating LLM-powered federated learning into large-scale IoT ecosystems, paving the way for more secure, scalable, and adaptive IoT management solutions.
• 摘要：物联网生态系统的快速扩展引入了可伸缩性，安全性和实时决策的严重挑战。传统的集中式体系结构在潜伏期，隐私问题和过度资源消耗中挣扎，这使得它们不适合现代大规模的物联网部署。本文介绍了一种新颖的联合学习驱动的大语言模型（FL-LLM）框架，旨在增强物联网系统智能，同时确保数据隐私和计算效率。该框架将生成物IoT（Giot）模型与梯度传感联合策略（GSF）集成在一起，并根据实时网络条件动态优化模型更新。通过利用混合边缘云处理体系结构，我们的方法平衡了分布式物联网环境中的智能，可扩展性和安全性。对IOT-23数据集的评估表明，我们的框架提高了模型的准确性，降低了响应潜伏期并提高了能源效率，超过了传统的FL技术（即FedAvg，FedOpt）。这些发现突出了将LLM驱动的联合学习整合到大规模IoT生态系统中的潜力，为更安全，可扩展和自适应的物联网管理解决方案铺平了道路。

Title: Boosting Generative Image Modeling via Joint Image-Feature Synthesis

• Authors: Theodoros Kouzelis, Efstathios Karypidis, Ioannis Kakogeorgiou, Spyros Gidaris, Nikos Komodakis
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.16064
• Pdf URL: https://arxiv.org/pdf/2504.16064
• Copy Paste: [[2504.16064]] Boosting Generative Image Modeling via Joint Image-Feature Synthesis(https://arxiv.org/abs/2504.16064)
• Keywords: generation, generative
• Abstract: Latent diffusion models (LDMs) dominate high-quality image generation, yet integrating representation learning with generative modeling remains a challenge. We introduce a novel generative image modeling framework that seamlessly bridges this gap by leveraging a diffusion model to jointly model low-level image latents (from a variational autoencoder) and high-level semantic features (from a pretrained self-supervised encoder like DINO). Our latent-semantic diffusion approach learns to generate coherent image-feature pairs from pure noise, significantly enhancing both generative quality and training efficiency, all while requiring only minimal modifications to standard Diffusion Transformer architectures. By eliminating the need for complex distillation objectives, our unified design simplifies training and unlocks a powerful new inference strategy: Representation Guidance, which leverages learned semantics to steer and refine image generation. Evaluated in both conditional and unconditional settings, our method delivers substantial improvements in image quality and training convergence speed, establishing a new direction for representation-aware generative modeling.
• 摘要：潜在扩散模型（LDMS）主导了高质量的图像生成，但是将表示学习与生成建模集成在一起仍然是一个挑战。我们介绍了一个新颖的生成图像建模框架，该框架通过利用扩散模型来无缝地弥合这一差距，以共同模拟低级图像潜在（来自差异自动编码器）和高级语义特征（来自预处理的自我避免的自我求解的编码器，如Dino）。我们的潜在语义扩散方法学会了从纯噪声中生成相干图像功能对，从而显着提高了生成质量和训练效率，同时仅需要对标准扩散变压器架构的最小修改。通过消除对复杂蒸馏目标的需求，我们的统一设计简化了培训，并解锁了强大的新推理策略：代表指导，该指导利用了学习语义来引导和完善图像的生成。在有条件的和无条件的设置中，我们的方法在图像质量和训练速度方面都有实质性改进，为表示意识到的生成建模建立了新的方向。

Title: From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuning

• Authors: Le Zhuo, Liangbing Zhao, Sayak Paul, Yue Liao, Renrui Zhang, Yi Xin, Peng Gao, Mohamed Elhoseiny, Hongsheng Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2504.16080
• Pdf URL: https://arxiv.org/pdf/2504.16080
• Copy Paste: [[2504.16080]] From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuning(https://arxiv.org/abs/2504.16080)
• Keywords: generation
• Abstract: Recent text-to-image diffusion models achieve impressive visual quality through extensive scaling of training data and model parameters, yet they often struggle with complex scenes and fine-grained details. Inspired by the self-reflection capabilities emergent in large language models, we propose ReflectionFlow, an inference-time framework enabling diffusion models to iteratively reflect upon and refine their outputs. ReflectionFlow introduces three complementary inference-time scaling axes: (1) noise-level scaling to optimize latent initialization; (2) prompt-level scaling for precise semantic guidance; and most notably, (3) reflection-level scaling, which explicitly provides actionable reflections to iteratively assess and correct previous generations. To facilitate reflection-level scaling, we construct GenRef, a large-scale dataset comprising 1 million triplets, each containing a reflection, a flawed image, and an enhanced image. Leveraging this dataset, we efficiently perform reflection tuning on state-of-the-art diffusion transformer, FLUX.1-dev, by jointly modeling multimodal inputs within a unified framework. Experimental results show that ReflectionFlow significantly outperforms naive noise-level scaling methods, offering a scalable and compute-efficient solution toward higher-quality image synthesis on challenging tasks.
• 摘要：最近的文本到图像扩散模型通过广泛的训练数据和模型参数实现了令人印象深刻的视觉质量，但是它们经常在复杂的场景和细粒度的细节上挣扎。受到大语言模型中出现的自我反射功能的启发，我们提出了反射流，这是一个推理时间框架，使扩散模型可以迭代地反思和完善其输出。 ReflectionFlow引入了三个互补推理时间缩放轴：（1）噪声级缩放以优化潜在的初始化； （2）迅速级别的缩放，以进行精确的语义指导；最值得注意的是，（3）反射级缩放比例，明确为迭代评估和纠正前几代提供了可行的反射。为了促进反思级别的缩放，我们构建了一个大规模数据集，其中包含100万个三重态，每个数据集包含反射，有缺陷的图像和增强的图像。利用此数据集，我们通过在统一框架中共同建模多模式输入来有效地对最新扩散变压器Flux.1-DEV进行反射调谐。实验结果表明，反射流显着超过了幼稚的噪声级缩放方法，提供了可扩展的计算效率解决方案，以实现在具有挑战性的任务上的高质量图像合成。

Title: Survey of Video Diffusion Models: Foundations, Implementations, and Applications

• Authors: Yimu Wang, Xuye Liu, Wei Pang, Li Ma, Shuai Yuan, Paul Debevec, Ning Yu
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2504.16081
• Pdf URL: https://arxiv.org/pdf/2504.16081
• Copy Paste: [[2504.16081]] Survey of Video Diffusion Models: Foundations, Implementations, and Applications(https://arxiv.org/abs/2504.16081)
• Keywords: super-resolution, generation, generative
• Abstract: Recent advances in diffusion models have revolutionized video generation, offering superior temporal consistency and visual quality compared to traditional generative adversarial networks-based approaches. While this emerging field shows tremendous promise in applications, it faces significant challenges in motion consistency, computational efficiency, and ethical considerations. This survey provides a comprehensive review of diffusion-based video generation, examining its evolution, technical foundations, and practical applications. We present a systematic taxonomy of current methodologies, analyze architectural innovations and optimization strategies, and investigate applications across low-level vision tasks such as denoising and super-resolution. Additionally, we explore the synergies between diffusionbased video generation and related domains, including video representation learning, question answering, and retrieval. Compared to the existing surveys (Lei et al., 2024a;b; Melnik et al., 2024; Cao et al., 2023; Xing et al., 2024c) which focus on specific aspects of video generation, such as human video synthesis (Lei et al., 2024a) or long-form content generation (Lei et al., 2024b), our work provides a broader, more updated, and more fine-grained perspective on diffusion-based approaches with a special section for evaluation metrics, industry solutions, and training engineering techniques in video generation. This survey serves as a foundational resource for researchers and practitioners working at the intersection of diffusion models and video generation, providing insights into both the theoretical frameworks and practical implementations that drive this rapidly evolving field. A structured list of related works involved in this survey is also available on this https URL.
• 摘要：扩散模型的最新进展彻底改变了视频的产生，与传统的基于基于逆向网络的方法相比，具有卓越的时间一致性和视觉质量。尽管这个新兴领域在应用中显示出巨大的希望，但它在运动一致性，计算效率和道德考虑方面面临着重大挑战。这项调查对基于扩散的视频生成进行了全面综述，研究了其演变，技术基础和实际应用。我们介绍了当前方法论的系统分类学，分析建筑创新和优化策略，并调查跨低级视觉任务（例如denoising and Super-Losolution and Super-Losolution）的应用。此外，我们探讨了基于扩散的视频生成和相关域之间的协同作用，包括视频表示学习，问答答案和检索。与现有的调查相比（Lei等，2024a; b; Melnik等，2024; Cao等，2023; Xing等，2024c），该方面的重点是视频发电的特定方面，例如人类视频合成等特定方面，例如Lei等人（Lei等，2024a）或长期的内容产生（Lei等人，更多的是lei et and forei nefore forked and fordie forking and forking and forking and forking and in。在视频生成中，具有用于评估指标，行业解决方案和培训工程技术的特殊部分的基于扩散的方法的观点。这项调查是在扩散模型和视频生成交集中工作的研究人员和从业人员的基础资源，从而提供了对理论框架和实践实现的见解，这些实现又可以推动这一迅速发展的领域。此调查中涉及的相关作品的结构化列表也可以在此HTTPS URL上获得。