2025-06-25

Title: HI-SQL: Optimizing Text-to-SQL Systems through Dynamic Hint Integration

• Authors: Ganesh Parab, Zishan Ahmad, Dagnachew Birru
• Subjects: cs.LG, cs.DB
• Abstract URL: https://arxiv.org/abs/2506.18916
• Pdf URL: https://arxiv.org/pdf/2506.18916
• Copy Paste: [[2506.18916]] HI-SQL: Optimizing Text-to-SQL Systems through Dynamic Hint Integration(https://arxiv.org/abs/2506.18916)
• Keywords: generation
• Abstract: Text-to-SQL generation bridges the gap between natural language and databases, enabling users to query data without requiring SQL expertise. While large language models (LLMs) have significantly advanced the field, challenges remain in handling complex queries that involve multi-table joins, nested conditions, and intricate operations. Existing methods often rely on multi-step pipelines that incur high computational costs, increase latency, and are prone to error propagation. To address these limitations, we propose HI-SQL, a pipeline that incorporates a novel hint generation mechanism utilizing historical query logs to guide SQL generation. By analyzing prior queries, our method generates contextual hints that focus on handling the complexities of multi-table and nested operations. These hints are seamlessly integrated into the SQL generation process, eliminating the need for costly multi-step approaches and reducing reliance on human-crafted prompts. Experimental evaluations on multiple benchmark datasets demonstrate that our approach significantly improves query accuracy of LLM-generated queries while ensuring efficiency in terms of LLM calls and latency, offering a robust and practical solution for enhancing Text-to-SQL systems.
• 摘要：文本到SQL生成桥接自然语言和数据库之间的差距，使用户能够查询数据而无需SQL专业知识。尽管大型语言模型（LLMS）显着提高了该领域，但仍在处理涉及多桌连接，嵌套条件和复杂操作的复杂查询中仍然存在挑战。现有的方法通常依赖于产生高计算成本，增加延迟并容易出现错误传播的多步管道。为了解决这些局限性，我们提出了HI-SQL，这是一种利用历史查询日志来指导SQL生成的新型提示生成机制的管道。通过分析先前的查询，我们的方法生成了上下文提示，这些提示着重于处理多桌和嵌套操作的复杂性。这些提示无缝地集成到SQL生成过程中，消除了对昂贵的多步骤方法的需求，并减少了对人类制作的提示的依赖。对多个基准数据集的实验评估表明，我们的方法显着提高了LLM生成的查询的查询准确性，同时确保LLM呼叫和延迟效率，为增强文本到SQL系统提供了强大而实用的解决方案。

Title: LLMs on a Budget? Say HOLA

• Authors: Zohaib Hasan Siddiqui, Jiechao Gao, Ebad Shabbir, Mohammad Anas Azeez, Rafiq Ali, Gautam Siddharth Kashyap, Usman Naseem
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.18952
• Pdf URL: https://arxiv.org/pdf/2506.18952
• Copy Paste: [[2506.18952]] LLMs on a Budget? Say HOLA(https://arxiv.org/abs/2506.18952)
• Keywords: generation
• Abstract: Running Large Language Models (LLMs) on edge devices is constrained by high compute and memory demands posing a barrier for real-time applications in sectors like healthcare, education, and embedded systems. Current solutions such as quantization, pruning, and retrieval-augmented generation (RAG) offer only partial optimizations and often compromise on speed or accuracy. We introduce HOLA, an end-to-end optimization framework for efficient LLM deployment. Internally, it leverages Hierarchical Speculative Decoding (HSD) for faster inference without quality loss. Externally, AdaComp-RAG adjusts retrieval complexity based on context needs. Together with LoBi, which blends structured pruning (LoRA) and quantization, HOLA delivers significant gains: 17.6% EMA on GSM8K, 10.5% MCA on ARC, and reduced latency and memory on edge devices like Jetson Nano--proving both scalable and production-ready.
• 摘要：边缘设备上运行大型语言模型（LLM）受到高计算的限制，并且内存需求为医疗保健，教育和嵌入式系统等领域的实时应用构成障碍。当前的解决方案（例如量化，修剪和检索生成生成（RAG））仅提供部分优化，并且通常会妥协速度或准确性。我们介绍了Hola，这是一个用于有效LLM部署的端到端优化框架。在内部，它利用分层投机解码（HSD）进行更快的推断而不会损失质量。在外部，Adacomp-rag根据上下文需求调整检索复杂性。 Hola与结构化修剪（LORA）和量化的LOBI一起提供了显着增长：GSM8K上的EMA 17.6％，ARC上的MCA为10.5％，并减少了Jetson Nano（例如Jetson Nano）的延迟和记忆 - 可伸缩和生产准备就绪。

Title: GLIMPSE: Gradient-Layer Importance Mapping for Prompted Visual Saliency Explanation for Generative LVLMs

• Authors: Guanxi Shen
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.18985
• Pdf URL: https://arxiv.org/pdf/2506.18985
• Copy Paste: [[2506.18985]] GLIMPSE: Gradient-Layer Importance Mapping for Prompted Visual Saliency Explanation for Generative LVLMs(https://arxiv.org/abs/2506.18985)
• Keywords: generative
• Abstract: Recent advances in large vision language models (LVLMs) have unlocked unprecedented capabilities in generating coherent responses from visual inputs. However, interpreting where LVLMs direct their visual attention while generating free-form textual responses remains a significant challenge, yet is essential for understanding model behavior, diagnosing hallucination, exposing bias and ensuring transparency. We introduce GLIMPSE (Gradient-Layer Importance Mapping for Prompted Visual Saliency Explanation), a lightweight, model-agnostic framework for visualizing the salient image regions that LVLMs rely upon during open-ended visual question answering (VQA), while concurrently revealing the multimodal textual saliency. GLIMPSE fuses gradient-weighted attention, adaptive layer propagation, and weighted token aggregation to produce holistic response-level attribution heat maps for interpreting cross-modal reasoning, outperforming prior interpretability methods in human-alignment. We demonstrate an analytic explainable AI (XAI) approach using GLIMPSE to uncover fine-grained insights into LVLM cross-modal attribution, trace token-level reasoning dynamics, and analyze systematic human-attention misalignment, hallucination, and bias.
• 摘要：大型视觉语言模型（LVLM）的最新进展已解锁了从视觉输入产生相干响应的前所未有的功能。但是，解释LVLM在产生自由形式的文本响应的同时引导其视觉注意力仍然是一个重大挑战，但对于理解模型行为，诊断幻觉，暴露偏见和确保透明度至关重要。我们介绍了瞥见（梯度层的重要性映射，以提示视觉显着性说明），这是一个轻巧的模型无关的框架，用于可视化LVLMS在开放式视觉询问（VQA）中依赖的显着图像区域（VQA），同时透露了多模式的文本不合理。瞥见梯度加权的注意力，自适应层的传播和加权令牌聚集，以产生整体响应级别的归因热图，用于解释跨模式推理，在人类一致性中表现出优于先前的可解释性方法。我们展示了一种可解释的AI（XAI）方法，该方法使用瞥见来发现对LVLM跨模式归因，痕迹令牌级别的推理动力学的细粒度见解，并分析系统的人类注意力对齐，幻觉，幻觉和偏见。

Title: Diffusion Transformer-to-Mamba Distillation for High-Resolution Image Generation

• Authors: Yuan Yao, Yicong Hong, Difan Liu, Long Mai, Feng Liu, Jiebo Luo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.18999
• Pdf URL: https://arxiv.org/pdf/2506.18999
• Copy Paste: [[2506.18999]] Diffusion Transformer-to-Mamba Distillation for High-Resolution Image Generation(https://arxiv.org/abs/2506.18999)
• Keywords: generation
• Abstract: The quadratic computational complexity of self-attention in diffusion transformers (DiT) introduces substantial computational costs in high-resolution image generation. While the linear-complexity Mamba model emerges as a potential alternative, direct Mamba training remains empirically challenging. To address this issue, this paper introduces diffusion transformer-to-mamba distillation (T2MD), forming an efficient training pipeline that facilitates the transition from the self-attention-based transformer to the linear complexity state-space model Mamba. We establish a diffusion self-attention and Mamba hybrid model that simultaneously achieves efficiency and global dependencies. With the proposed layer-level teacher forcing and feature-based knowledge distillation, T2MD alleviates the training difficulty and high cost of a state space model from scratch. Starting from the distilled 512$\times$512 resolution base model, we push the generation towards 2048$\times$2048 images via lightweight adaptation and high-resolution fine-tuning. Experiments demonstrate that our training path leads to low overhead but high-quality text-to-image generation. Importantly, our results also justify the feasibility of using sequential and causal Mamba models for generating non-causal visual output, suggesting the potential for future exploration.
• 摘要：扩散变压器（DIT）中自我注意的二次计算复杂性引入了高分辨率图像生成中的实质性计算成本。尽管线性复杂性Mamba模型是一种潜在的替代方案，但直接的Mamba培训仍然在经验上具有挑战性。为了解决这个问题，本文介绍了扩散变压器到mamba蒸馏（T2MD），形成了有效的训练管道，该管道有助于从基于自我注意力的变压器到线性复杂性状态空间模型Mamba的过渡。我们建立了一个扩散的自我注意力和MAMBA混合模型，同时实现了效率和全球依赖性。借助提议的层级教师强迫和基于特征的知识蒸馏，T2MD从头开始缓解了国家空间模型的训练难度和高成本。从蒸馏的512 $ \ times $ 512分辨率基础型号开始，我们通过轻巧的适应和高分辨率微调将一代推向2048 $ \ times $ 2048的图像。实验表明，我们的训练路径会导致较低的开销但高质量的文本对象产生。重要的是，我们的结果还证明了使用顺序和因果MAMBA模型生成非毒物视觉输出的可行性，这表明了将来探索的潜力。

Title: Automating Traffic Monitoring with SHM Sensor Networks via Vision-Supervised Deep Learning

• Authors: Hanshuo Wu, Xudong Jian, Christos Lataniotis, Cyprien Hoelzl, Eleni Chatzi, Yves Reuland
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19023
• Pdf URL: https://arxiv.org/pdf/2506.19023
• Copy Paste: [[2506.19023]] Automating Traffic Monitoring with SHM Sensor Networks via Vision-Supervised Deep Learning(https://arxiv.org/abs/2506.19023)
• Keywords: generation
• Abstract: Bridges, as critical components of civil infrastructure, are increasingly affected by deterioration, making reliable traffic monitoring essential for assessing their remaining service life. Among operational loads, traffic load plays a pivotal role, and recent advances in deep learning - particularly in computer vision (CV) - have enabled progress toward continuous, automated monitoring. However, CV-based approaches suffer from limitations, including privacy concerns and sensitivity to lighting conditions, while traditional non-vision-based methods often lack flexibility in deployment and validation. To bridge this gap, we propose a fully automated deep-learning pipeline for continuous traffic monitoring using structural health monitoring (SHM) sensor networks. Our approach integrates CV-assisted high-resolution dataset generation with supervised training and inference, leveraging graph neural networks (GNNs) to capture the spatial structure and interdependence of sensor data. By transferring knowledge from CV outputs to SHM sensors, the proposed framework enables sensor networks to achieve comparable accuracy of vision-based systems, with minimal human intervention. Applied to accelerometer and strain gauge data in a real-world case study, the model achieves state-of-the-art performance, with classification accuracies of 99% for light vehicles and 94% for heavy vehicles.
• 摘要：作为民用基础设施的关键组成部分，桥梁越来越受到恶化的影响，这使得可靠的交通监控对于评估其剩余使用寿命至关重要。在操作负载中，交通负荷起着关键作用，并且在深度学习（尤其是计算机视觉（CV）中）的最新进展使得能够朝着连续的自动监视方向发展。但是，基于简历的方法受到限制，包括隐私问题和对照明条件的敏感性，而传统的基于非视觉的方法通常缺乏部署和验证的灵活性。为了弥合这一差距，我们建议使用结构性健康监测（SHM）传感器网络进行全自动的深度学习管道，以连续进行交通监控。我们的方法将CV辅助的高分辨率数据集生成与监督培训和推理相结合，利用图形神经网络（GNNS）捕获传感器数据的空间结构和相互依赖性。通过将知识从简历输出转移到SHM传感器，提出的框架使传感器网络能够实现基于视力的系统的可比精度，并最少的人为干预。该模型应用于实际案例研究中的加速度计和应变计数据，可实现最先进的性能，轻型车辆的分类精度为99％，重型车辆的分类精度为94％。

Title: FairCauseSyn: Towards Causally Fair LLM-Augmented Synthetic Data Generation

• Authors: Nitish Nagesh, Ziyu Wang, Amir M. Rahmani
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.19082
• Pdf URL: https://arxiv.org/pdf/2506.19082
• Copy Paste: [[2506.19082]] FairCauseSyn: Towards Causally Fair LLM-Augmented Synthetic Data Generation(https://arxiv.org/abs/2506.19082)
• Keywords: generation, generative
• Abstract: Synthetic data generation creates data based on real-world data using generative models. In health applications, generating high-quality data while maintaining fairness for sensitive attributes is essential for equitable outcomes. Existing GAN-based and LLM-based methods focus on counterfactual fairness and are primarily applied in finance and legal domains. Causal fairness provides a more comprehensive evaluation framework by preserving causal structure, but current synthetic data generation methods do not address it in health settings. To fill this gap, we develop the first LLM-augmented synthetic data generation method to enhance causal fairness using real-world tabular health data. Our generated data deviates by less than 10% from real data on causal fairness metrics. When trained on causally fair predictors, synthetic data reduces bias on the sensitive attribute by 70% compared to real data. This work improves access to fair synthetic data, supporting equitable health research and healthcare delivery.
• 摘要：合成数据生成使用生成模型基于实际数据创建数据。在健康应用中，生成高质量数据的同时保持敏感属性的公平性对于公平结果至关重要。现有的基于GAN的基于LLM的方法侧重于反事实公平，主要用于金融和法律领域。因果公平通过保留因果结构提供了更全面的评估框架，但是当前的合成数据生成方法并未在健康环境中解决。为了填补这一空白，我们开发了第一个LLM-EAGMENT合成数据生成方法，以使用现实世界中的表格健康数据来增强因果公平性。我们生成的数据偏离了有关因果公平指标的实际数据不到10％。与实际数据相比，当对因果公平的预测因素进行培训时，合成数据将敏感属性的偏差降低了70％。这项工作改善了获得公平合成数据的访问，支持公平的健康研究和医疗保健提供。

Title: Benchmarking Music Generation Models and Metrics via Human Preference Studies

• Authors: Florian Grötschla, Ahmet Solak, Luca A. Lanzendörfer, Roger Wattenhofer
• Subjects: cs.LG, cs.SD
• Abstract URL: https://arxiv.org/abs/2506.19085
• Pdf URL: https://arxiv.org/pdf/2506.19085
• Copy Paste: [[2506.19085]] Benchmarking Music Generation Models and Metrics via Human Preference Studies(https://arxiv.org/abs/2506.19085)
• Keywords: generation
• Abstract: Recent advancements have brought generated music closer to human-created compositions, yet evaluating these models remains challenging. While human preference is the gold standard for assessing quality, translating these subjective judgments into objective metrics, particularly for text-audio alignment and music quality, has proven difficult. In this work, we generate 6k songs using 12 state-of-the-art models and conduct a survey of 15k pairwise audio comparisons with 2.5k human participants to evaluate the correlation between human preferences and widely used metrics. To the best of our knowledge, this work is the first to rank current state-of-the-art music generation models and metrics based on human preference. To further the field of subjective metric evaluation, we provide open access to our dataset of generated music and human evaluations.
• 摘要：最近的进步使产生的音乐更接近人类创建的作品，但是评估这些模型仍然具有挑战性。尽管人类的偏好是评估质量的黄金标准，但将这些主观判断转化为客观指标，尤其是用于文本原告的一致性和音乐质量，但事实证明很困难。在这项工作中，我们使用12种最先进的模型生成了6K歌曲，并对2.5k人参与者进行了15K成对音频比较的调查，以评估人类偏好与广泛使用的指标之间的相关性。据我们所知，这项工作是第一个基于人类偏好对当前最新音乐生成模型和指标进行排名的工作。为了进一步的主观度量评估领域，我们为生成的音乐和人类评估数据集提供了公开访问。

Title: Inverse-and-Edit: Effective and Fast Image Editing by Cycle Consistency Models

• Authors: Ilia Beletskii, Andrey Kuznetsov, Aibek Alanov
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19103
• Pdf URL: https://arxiv.org/pdf/2506.19103
• Copy Paste: [[2506.19103]] Inverse-and-Edit: Effective and Fast Image Editing by Cycle Consistency Models(https://arxiv.org/abs/2506.19103)
• Keywords: generation
• Abstract: Recent advances in image editing with diffusion models have achieved impressive results, offering fine-grained control over the generation process. However, these methods are computationally intensive because of their iterative nature. While distilled diffusion models enable faster inference, their editing capabilities remain limited, primarily because of poor inversion quality. High-fidelity inversion and reconstruction are essential for precise image editing, as they preserve the structural and semantic integrity of the source image. In this work, we propose a novel framework that enhances image inversion using consistency models, enabling high-quality editing in just four steps. Our method introduces a cycle-consistency optimization strategy that significantly improves reconstruction accuracy and enables a controllable trade-off between editability and content preservation. We achieve state-of-the-art performance across various image editing tasks and datasets, demonstrating that our method matches or surpasses full-step diffusion models while being substantially more efficient. The code of our method is available on GitHub at this https URL.
• 摘要：通过扩散模型的图像编辑的最新进展取得了令人印象深刻的结果，从而对生成过程提供了细粒度的控制。但是，由于其迭代性质，这些方法在计算上是密集的。尽管蒸馏扩散模型可以更快地推断，但它们的编辑功能仍然有限，这主要是由于反转质量差。高保真倒置和重建对于精确图像编辑至关重要，因为它们保留了源图像的结构和语义完整性。在这项工作中，我们提出了一个新颖的框架，该框架可以使用一致性模型来增强图像反演，从而仅需四个步骤即可进行高质量的编辑。我们的方法介绍了一种周期抗性优化策略，该策略可显着提高重建精度，并在编辑性和内容保存之间实现可控制的权衡。我们在各种图像编辑任务和数据集中实现了最新的性能，表明我们的方法匹配或超过了全步扩散模型，同时又更加有效。我们方法的代码可在此HTTPS URL的GitHub上获得。

Title: PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Scenes

• Authors: Christina Ourania Tze, Daniel Dauner, Yiyi Liao, Dzmitry Tsishkou, Andreas Geiger
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19117
• Pdf URL: https://arxiv.org/pdf/2506.19117
• Copy Paste: [[2506.19117]] PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Scenes(https://arxiv.org/abs/2506.19117)
• Keywords: generation
• Abstract: Large-scale 3D semantic scene generation has predominantly relied on voxel-based representations, which are memory-intensive, bound by fixed resolutions, and challenging to edit. In contrast, primitives represent semantic entities using compact, coarse 3D structures that are easy to manipulate and compose, making them an ideal representation for this task. In this paper, we introduce PrITTI, a latent diffusion-based framework that leverages primitives as the main foundational elements for generating compositional, controllable, and editable 3D semantic scene layouts. Our method adopts a hybrid representation, modeling ground surfaces in a rasterized format while encoding objects as vectorized 3D primitives. This decomposition is also reflected in a structured latent representation that enables flexible scene manipulation of ground and object components. To overcome the orientation ambiguities in conventional encoding methods, we introduce a stable Cholesky-based parameterization that jointly encodes object size and orientation. Experiments on the KITTI-360 dataset show that PrITTI outperforms a voxel-based baseline in generation quality, while reducing memory requirements by up to $3\times$. In addition, PrITTI enables direct instance-level manipulation of objects in the scene and supports a range of downstream applications, including scene inpainting, outpainting, and photo-realistic street-view synthesis.
• 摘要：大规模的3D语义场景产生主要依赖于基于体素的表示，这些表示是记忆密集型的，受固定分辨率和编辑挑战性的限制。相比之下，原语代表了使用紧凑的粗3D结构的语义实体，这些结构易于操纵和组成，使其成为此任务的理想表示。在本文中，我们介绍了Pritti，Pritti是一种基于潜在扩散的框架，它利用原语作为生成构图，可控制和可编辑的3D语义场景布局的主要基础元素。我们的方法采用混合表示形式，以栅格化格式对地面表面进行建模，同时将对象编码为矢量化的3D原语。该分解也反映在结构化的潜在表示中，该表示可以灵活地对地面和对象组件进行操纵。为了克服常规编码方法中的方向歧义，我们引入了一个稳定的基于Cholesky的参数化，共同编码对象大小和方向。 Kitti-360数据集的实验表明，Pritti在生成质量方面的表现优于基于体素的基线，同时将内存需求降低了$ 3 \ times $。此外，Pritti可以直接对场景中的对象进行直接的实例级操作，并支持一系列下游应用程序，包括场景介绍，支出和光真实的街道视图合成。

Title: Riemannian generative decoder

• Authors: Andreas Bjerregaard, Søren Hauberg, Anders Krogh
• Subjects: cs.LG, q-bio.QM, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.19133
• Pdf URL: https://arxiv.org/pdf/2506.19133
• Copy Paste: [[2506.19133]] Riemannian generative decoder(https://arxiv.org/abs/2506.19133)
• Keywords: generative
• Abstract: Riemannian representation learning typically relies on approximating densities on chosen manifolds. This involves optimizing difficult objectives, potentially harming models. To completely circumvent this issue, we introduce the Riemannian generative decoder which finds manifold-valued maximum likelihood latents with a Riemannian optimizer while training a decoder network. By discarding the encoder, we vastly simplify the manifold constraint compared to current approaches which often only handle few specific manifolds. We validate our approach on three case studies -- a synthetic branching diffusion process, human migrations inferred from mitochondrial DNA, and cells undergoing a cell division cycle -- each showing that learned representations respect the prescribed geometry and capture intrinsic non-Euclidean structure. Our method requires only a decoder, is compatible with existing architectures, and yields interpretable latent spaces aligned with data geometry.
• 摘要：里曼表示学习通常依赖于所选歧管上的近似密度。这涉及优化困难的目标，潜在损害模型。为了完全避免此问题，我们介绍了Riemannian生成解码器，该解码器在训​​练解码器网络时，通过riemannian优化器找到了具有歧管值的最大似然潜在潜在。通过丢弃编码器，我们与通常仅处理特定歧管的当前方法相比，大大简化了歧管约束。我们在三个案例研究中验证了我们的方法 - 一种合成的分支扩散过程，从线粒体DNA推断的人类迁移以及经历细胞分裂周期的细胞 - 每个细胞都表明，学到的表示表示尊重规定的几何形状并捕获了内在的非欧几里德结构。我们的方法仅需要一个解码器，与现有架构兼容，并且产生可解释的潜在空间与数据几何形状一致。

Title: Local Learning Rules for Out-of-Equilibrium Physical Generative Models

• Authors: Cyrill Bösch, Geoffrey Roeder, Marc Serra-Garcia, Ryan P. Adams
• Subjects: cs.LG, cond-mat.mes-hall, cs.ET, cs.NE
• Abstract URL: https://arxiv.org/abs/2506.19136
• Pdf URL: https://arxiv.org/pdf/2506.19136
• Copy Paste: [[2506.19136]] Local Learning Rules for Out-of-Equilibrium Physical Generative Models(https://arxiv.org/abs/2506.19136)
• Keywords: generative
• Abstract: We show that the out-of-equilibrium driving protocol of score-based generative models (SGMs) can be learned via a local learning rule. The gradient with respect to the parameters of the driving protocol are computed directly from force measurements or from observed system dynamics. As a demonstration, we implement an SGM in a network of driven, nonlinear, overdamped oscillators coupled to a thermal bath. We first apply it to the problem of sampling from a mixture of two Gaussians in 2D. Finally, we train a network of 10x10 oscillators to sample images of 0s and 1s from the MNIST dataset.
• 摘要：我们表明，可以通过本地学习规则来学习基于得分的生成模型（SGM）的不平衡驾驶方案。相对于驾驶协议参数的梯度直接根据力测量或观察到的系统动力学计算。作为演示，我们在驱动的，非线性，过度阻尼振荡器的网络中实现了SGM，并与热浴耦合。我们首先将其应用于2D中两个高斯人的混合物采样问题。最后，我们训练一个由10x10振荡器组成的网络，从MNIST数据集采样0s和1s的图像。

Title: MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models

• Authors: Yinan Xia, Yilei Jiang, Yingshui Tan, Xiaoyong Zhu, Xiangyu Yue, Bo Zheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19257
• Pdf URL: https://arxiv.org/pdf/2506.19257
• Copy Paste: [[2506.19257]] MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models(https://arxiv.org/abs/2506.19257)
• Keywords: generation
• Abstract: Vision-Language Models (VLMs) have achieved remarkable progress in multimodal reasoning tasks through enhanced chain-of-thought capabilities. However, this advancement also introduces novel safety risks, as these models become increasingly vulnerable to harmful multimodal prompts that can trigger unethical or unsafe behaviors. Existing safety alignment approaches, primarily designed for unimodal language models, fall short in addressing the complex and nuanced threats posed by multimodal inputs. Moreover, current safety datasets lack the fine-grained, policy-grounded reasoning required to robustly align reasoning-capable VLMs. In this work, we introduce {MSR-Align}, a high-quality Multimodal Safety Reasoning dataset tailored to bridge this gap. MSR-Align supports fine-grained, deliberative reasoning over standardized safety policies across both vision and text modalities. Our data generation pipeline emphasizes multimodal diversity, policy-grounded reasoning, and rigorous quality filtering using strong multimodal judges. Extensive experiments demonstrate that fine-tuning VLMs on MSR-Align substantially improves robustness against both textual and vision-language jailbreak attacks, while preserving or enhancing general reasoning performance. MSR-Align provides a scalable and effective foundation for advancing the safety alignment of reasoning-capable VLMs. Our dataset is made publicly available at this https URL.
• 摘要：视觉语言模型（VLM）通过增强的思想链能力在多模式推理任务中取得了显着进步。但是，由于这些模型变得越来越容易受到有害的多模式提示，这可能会触发不道德或不安全的行为，因此这种进步也引入了新的安全风险。现有的安全对准方法主要是为单峰语言模型而设计的，在解决多模式输入带来的复杂和细微差别的威胁方面缺乏。此外，当前的安全数据集缺乏牢固地使具有推理能力的VLM的细粒度，政策基础的推理。在这项工作中，我们介绍了{MSR-Align}，这是一种量身定制的高质量的多模式安全推理数据集，用于弥合此间隙。 MSR-Align支持视力和文本模式的标准化安全政策的细粒度，审议的推理。我们的数据生成管道强调了使用强大的多模式法官的多模式多样性，政策基础的推理和严格的质量过滤。广泛的实验表明，对MSR Align的微调VLM显着提高了针对文本和视力语言越狱攻击的鲁棒性，同时保持或提高了一般推理性能。 MSR-Align为推进具有推理能力的VLM的安全对准提供了可扩展有效的基础。我们的数据集可在此HTTPS URL上公开提供。

Title: Automated Image Recognition Framework

• Authors: Quang-Binh Nguyen, Trong-Vu Hoang, Ngoc-Do Tran, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19261
• Pdf URL: https://arxiv.org/pdf/2506.19261
• Copy Paste: [[2506.19261]] Automated Image Recognition Framework(https://arxiv.org/abs/2506.19261)
• Keywords: generative
• Abstract: While the efficacy of deep learning models heavily relies on data, gathering and annotating data for specific tasks, particularly when addressing novel or sensitive subjects lacking relevant datasets, poses significant time and resource challenges. In response to this, we propose a novel Automated Image Recognition (AIR) framework that harnesses the power of generative AI. AIR empowers end-users to synthesize high-quality, pre-annotated datasets, eliminating the necessity for manual labeling. It also automatically trains deep learning models on the generated datasets with robust image recognition performance. Our framework includes two main data synthesis processes, AIR-Gen and AIR-Aug. The AIR-Gen enables end-users to seamlessly generate datasets tailored to their specifications. To improve image quality, we introduce a novel automated prompt engineering module that leverages the capabilities of large language models. We also introduce a distribution adjustment algorithm to eliminate duplicates and outliers, enhancing the robustness and reliability of generated datasets. On the other hand, the AIR-Aug enhances a given dataset, thereby improving the performance of deep classifier models. AIR-Aug is particularly beneficial when users have limited data for specific tasks. Through comprehensive experiments, we demonstrated the efficacy of our generated data in training deep learning models and showcased the system's potential to provide image recognition models for a wide range of objects. We also conducted a user study that achieved an impressive score of 4.4 out of 5.0, underscoring the AI community's positive perception of AIR.
• 摘要：尽管深度学习模型的功效在很大程度上依赖于数据，但为特定任务收集和注释数据，尤其是在解决缺乏相关数据集的新颖或敏感受试者时，会带来重大的时间和资源挑战。为此，我们提出了一个新型的自动图像识别（AIR）框架，该框架利用了生成AI的力量。空气使最终用户能够合成高质量的预注册数据集，从而消除了手动标记的必要性。它还以强大的图像识别性能自动在生成的数据集上训练深度学习模型。我们的框架包括两个主要数据合成过程，即Air-Gen和Air-Aug。气象使最终用户能够无缝生成根据其规格量身定制的数据集。为了提高图像质量，我们引入了一个新型的自动化及时工程模块，该模块利用大型语言模型的功能。我们还引入了分发调整算法，以消除重复和异常值，从而增强生成的数据集的鲁棒性和可靠性。另一方面，Air-Aug增强了给定的数据集，从而提高了深层分类器模型的性能。当用户对特定任务的数据有限时，AIR-AUG尤其有益。通过全面的实验，我们证明了生成的数据在训练深度学习模型中的功效，并展示了该系统为各种对象提供图像识别模型的潜力。我们还进行了一项用户研究，在5.0中取得了令人印象深刻的分数4.4，强调了AI社区对空气的积极看法。

Title: Da Yu: Towards USV-Based Image Captioning for Waterway Surveillance and Scene Understanding

• Authors: Runwei Guan, Ningwei Ouyang, Tianhao Xu, Shaofeng Liang, Wei Dai, Yafeng Sun, Shang Gao, Songning Lai, Shanliang Yao, Xuming Hu, Ryan Wen Liu, Yutao Yue, Hui Xiong
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2506.19288
• Pdf URL: https://arxiv.org/pdf/2506.19288
• Copy Paste: [[2506.19288]] Da Yu: Towards USV-Based Image Captioning for Waterway Surveillance and Scene Understanding(https://arxiv.org/abs/2506.19288)
• Keywords: generation
• Abstract: Automated waterway environment perception is crucial for enabling unmanned surface vessels (USVs) to understand their surroundings and make informed decisions. Most existing waterway perception models primarily focus on instance-level object perception paradigms (e.g., detection, segmentation). However, due to the complexity of waterway environments, current perception datasets and models fail to achieve global semantic understanding of waterways, limiting large-scale monitoring and structured log generation. With the advancement of vision-language models (VLMs), we leverage image captioning to introduce WaterCaption, the first captioning dataset specifically designed for waterway environments. WaterCaption focuses on fine-grained, multi-region long-text descriptions, providing a new research direction for visual geo-understanding and spatial scene cognition. Exactly, it includes 20.2k image-text pair data with 1.8 million vocabulary size. Additionally, we propose Da Yu, an edge-deployable multi-modal large language model for USVs, where we propose a novel vision-to-language projector called Nano Transformer Adaptor (NTA). NTA effectively balances computational efficiency with the capacity for both global and fine-grained local modeling of visual features, thereby significantly enhancing the model's ability to generate long-form textual outputs. Da Yu achieves an optimal balance between performance and efficiency, surpassing state-of-the-art models on WaterCaption and several other captioning benchmarks.
• 摘要：自动化的水道环境感知对于使无人的地表船（USV）了解其周围环境并做出明智的决定至关重要。大多数现有的水道知觉模型主要集中在实例级对象感知范式上（例如检测，细分）。但是，由于水道环境的复杂性，当前的感知数据集和模型无法实现对水道的全球语义理解，从而限制了大规模监控和结构化对数的生成。随着视觉模型（VLMS）的发展，我们利用图像字幕来引入水上卡，这是专为水道环境设计的第一个字幕数据集。水上习惯着重于细粒度的多区域长篇文本描述，为视觉地理学理解和空间场景认知提供了新的研究方向。确切地说，它包括20.2k图像文本对数据，具有180万个词汇大小。此外，我们提出了Da Yu，这是USV的一个可边除外的多模式大型语言模型，我们在其中提出了一个新型的视觉到语言投影仪，称为Nano Transformer Adapter（NTA）。 NTA有效地平衡了计算效率与视觉特征的全球和细粒度局部建模的能力，从而显着增强了模型生成长形式文本输出的能力。 Da Yu在绩效和效率之间取得了最佳的平衡，超过了水上的最新模型和其他几个字幕基准。

Title: Efficient Extreme Operating Condition Search for Online Relay Setting Calculation in Renewable Power Systems Based on Parallel Graph Neural Network

• Authors: Yan Li, Zengli Yang, Youhuai Wang, Jing Wang, Xiaoyu Han, Jingyu Wang, Dongyuan Shi
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19289
• Pdf URL: https://arxiv.org/pdf/2506.19289
• Copy Paste: [[2506.19289]] Efficient Extreme Operating Condition Search for Online Relay Setting Calculation in Renewable Power Systems Based on Parallel Graph Neural Network(https://arxiv.org/abs/2506.19289)
• Keywords: generation
• Abstract: The Extreme Operating Conditions Search (EOCS) problem is one of the key problems in relay setting calculation, which is used to ensure that the setting values of protection relays can adapt to the changing operating conditions of power systems over a period of time after deployment. The high penetration of renewable energy and the wide application of inverter-based resources make the operating conditions of renewable power systems more volatile, which urges the adoption of the online relay setting calculation strategy. However, the computation speed of existing EOCS methods based on local enumeration, heuristic algorithms, and mathematical programming cannot meet the efficiency requirement of online relay setting calculation. To reduce the time overhead, this paper, for the first time, proposes an efficient deep learning-based EOCS method suitable for online relay setting calculation. First, the power system information is formulated as four layers, i.e., a component parameter layer, a topological connection layer, an electrical distance layer, and a graph distance layer, which are fed into a parallel graph neural network (PGNN) model for feature extraction. Then, the four feature layers corresponding to each node are spliced and stretched, and then fed into the decision network to predict the extreme operating condition of the system. Finally, the proposed PGNN method is validated on the modified IEEE 39-bus and 118-bus test systems, where some of the synchronous generators are replaced by renewable generation units. The nonlinear fault characteristics of renewables are fully considered when computing fault currents. The experiment results show that the proposed PGNN method achieves higher accuracy than the existing methods in solving the EOCS problem. Meanwhile, it also provides greater improvements in online computation time.
• 摘要：极端操作条件搜索（EOC）问题是继电器设置计算中的关键问题之一，该问题用于确保保护继电器的设置值可以在部署后一段时间内适应电源系统的不断变化的操作条件。可再生能源的高渗透和基于逆变器的资源的广泛应用使可再生能源系统的运行条件更加动荡，这敦促采用在线继电器设置计算策略。但是，基于本地枚举，启发式算法和数学编程的现有EOC方法的计算速度无法满足在线继电器设置计算的效率要求。为了减少开销的时间，本文首次提出了一种有效的基于深度学习的EOC方法，适合在线继电器设置计算。首先，电源系统信息被配制为四层，即组件参数层，拓扑连接层，电气距离层和图形距离层，它们被馈入以进行特征提取的平行图神经网络（PGNN）模型。然后，将与每个节点相对应的四个特征层被拼接和​​拉伸，然后馈入决策网络以预测系统的极端工作条件。最后，在修改后的IEEE 39-BUS和118总线测试系统上验证了所提出的PGNN方法，其中某些同步发电机被可再生的生成单元代替。计算断层电流时，可再生能源的非线性故障特性将被完全考虑。实验结果表明，所提出的PGNN方法比解决EOC问题的现有方法具有更高的准确性。同时，它还在在线计算时间方面提供了更大的改进。

Title: Progressive Modality Cooperation for Multi-Modality Domain Adaptation

• Authors: Weichen Zhang, Dong Xu, Jing Zhang, Wanli Ouyang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19316
• Pdf URL: https://arxiv.org/pdf/2506.19316
• Copy Paste: [[2506.19316]] Progressive Modality Cooperation for Multi-Modality Domain Adaptation(https://arxiv.org/abs/2506.19316)
• Keywords: generation
• Abstract: In this work, we propose a new generic multi-modality domain adaptation framework called Progressive Modality Cooperation (PMC) to transfer the knowledge learned from the source domain to the target domain by exploiting multiple modality clues (\eg, RGB and depth) under the multi-modality domain adaptation (MMDA) and the more general multi-modality domain adaptation using privileged information (MMDA-PI) settings. Under the MMDA setting, the samples in both domains have all the modalities. In two newly proposed modules of our PMC, the multiple modalities are cooperated for selecting the reliable pseudo-labeled target samples, which captures the modality-specific information and modality-integrated information, respectively. Under the MMDA-PI setting, some modalities are missing in the target domain. Hence, to better exploit the multi-modality data in the source domain, we further propose the PMC with privileged information (PMC-PI) method by proposing a new multi-modality data generation (MMG) network. MMG generates the missing modalities in the target domain based on the source domain data by considering both domain distribution mismatch and semantics preservation, which are respectively achieved by using adversarial learning and conditioning on weighted pseudo semantics. Extensive experiments on three image datasets and eight video datasets for various multi-modality cross-domain visual recognition tasks under both MMDA and MMDA-PI settings clearly demonstrate the effectiveness of our proposed PMC framework.
• 摘要：在这项工作中，我们提出了一个新的通用多模式域适应框架，称为渐进式模态合作（PMC），通过利用多种模态线索（\ eg，rgb和depth）将知识从源域中移植到目标域，并使用多模式域适应（MMDA）和更具多模型的多型域名divive（MMDA）和更高的通用多人数（MMDA）来转移知识。 设置。在MMDA设置下，两个域中的样品都具有所有模式。在我们PMC的两个新提出的模块中，多种模式是为了选择可靠的伪标记的目标样本，分别捕获了特定于模态的信息和模态综合信息。在MMDA-PI设置下，目标域中缺少一些方式。因此，为了更好地利用源域中的多模式数据，我们通过提出新的多模式数据生成（MMG）网络，进一步提出了使用特权信息（PMC-PI）方法的PMC。 MMG通过考虑域分布不匹配和语义保存来基于源域数据在目标域中生成缺失的模式，这些域分别是通过使用对抗性学习和对加权伪语义上的调节来实现的。在MMDA和MMDA-PI设置下的三个图像数据集和八个视频数据集上进行了大量实验，清楚地证明了我们提出的PMC框架的有效性。

Title: Training-Free Motion Customization for Distilled Video Generators with Adaptive Test-Time Distillation

• Authors: Jintao Rong, Xin Xie, Xinyi Yu, Linlin Ou, Xinyu Zhang, Chunhua Shen, Dong Gong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19348
• Pdf URL: https://arxiv.org/pdf/2506.19348
• Copy Paste: [[2506.19348]] Training-Free Motion Customization for Distilled Video Generators with Adaptive Test-Time Distillation(https://arxiv.org/abs/2506.19348)
• Keywords: generation, generative
• Abstract: Distilled video generation models offer fast and efficient synthesis but struggle with motion customization when guided by reference videos, especially under training-free settings. Existing training-free methods, originally designed for standard diffusion models, fail to generalize due to the accelerated generative process and large denoising steps in distilled models. To address this, we propose MotionEcho, a novel training-free test-time distillation framework that enables motion customization by leveraging diffusion teacher forcing. Our approach uses high-quality, slow teacher models to guide the inference of fast student models through endpoint prediction and interpolation. To maintain efficiency, we dynamically allocate computation across timesteps according to guidance needs. Extensive experiments across various distilled video generation models and benchmark datasets demonstrate that our method significantly improves motion fidelity and generation quality while preserving high efficiency. Project page: this https URL
• 摘要：蒸馏视频生成模型提供了快速有效的综合，但在参考视频引导时，尤其是在无培训的设置下，与运动定制斗争。最初是为标准扩散模型设计的现有无培训方法，由于加速生成过程和蒸馏模型中的大型降解步骤，无法推广。为了解决这个问题，我们提出了MotioneCho，这是一种新型的无训练测试时间蒸馏框架，通过利用扩散教师的强迫来实现运动定制。我们的方法使用高质量的慢老师模型来通过终点预测和插值来指导快速学生模型的推断。为了维持效率，我们根据指导需求将计算在时间段中动态分配。各种蒸馏视频生成模型和基准数据集的广泛实验表明，我们的方法可显着提高运动保真度和发电质量，同时保持高效率。项目页面：此HTTPS URL

Title: Deep Electromagnetic Structure Design Under Limited Evaluation Budgets

• Authors: Shijian Zheng, Fangxiao Jin, Shuhai Zhang, Quan Xue, Mingkui Tan
• Subjects: cs.LG, eess.SP, physics.comp-ph
• Abstract URL: https://arxiv.org/abs/2506.19384
• Pdf URL: https://arxiv.org/pdf/2506.19384
• Copy Paste: [[2506.19384]] Deep Electromagnetic Structure Design Under Limited Evaluation Budgets(https://arxiv.org/abs/2506.19384)
• Keywords: generative
• Abstract: Electromagnetic structure (EMS) design plays a critical role in developing advanced antennas and materials, but remains challenging due to high-dimensional design spaces and expensive evaluations. While existing methods commonly employ high-quality predictors or generators to alleviate evaluations, they are often data-intensive and struggle with real-world scale and budget constraints. To address this, we propose a novel method called Progressive Quadtree-based Search (PQS). Rather than exhaustively exploring the high-dimensional space, PQS converts the conventional image-like layout into a quadtree-based hierarchical representation, enabling a progressive search from global patterns to local details. Furthermore, to lessen reliance on highly accurate predictors, we introduce a consistency-driven sample selection mechanism. This mechanism quantifies the reliability of predictions, balancing exploitation and exploration when selecting candidate designs. We evaluate PQS on two real-world engineering tasks, i.e., Dual-layer Frequency Selective Surface and High-gain Antenna. Experimental results show that our method can achieve satisfactory designs under limited computational budgets, outperforming baseline methods. In particular, compared to generative approaches, it cuts evaluation costs by 75-85%, effectively saving 20.27-38.80 days of product designing cycle.
• 摘要：电磁结构（EMS）设计在开发高级天线和材料中起着至关重要的作用，但由于高维设计空间和昂贵的评估，因此仍然具有挑战性。尽管现有方法通常采用高质量的预测因子或发电机来减轻评估，但它们通常是数据密集型的，并且在现实世界中的规模和预算限制方面进行了斗争。为了解决这个问题，我们提出了一种称为“渐进Quadtree基于Quadtree”（PQS）的新颖方法。 PQ并没有详尽地探索高维空间，而是将类似图像的布局转换为基于Quadtree的层次结构表示形式，从而使渐进式搜索从全局模式到本地详细信息。此外，为了减少对高度准确的预测因子的依赖，我们引入了一种一致性驱动的样本选择机制。选择候选设计时，这种机制量化了预测，平衡剥削和探索的可靠性。我们评估了两个现实世界工程任务的PQ，即双层频率选择性表面和高增益天线。实验结果表明，在有限的计算预算下，我们的方法可以实现令人满意的设计，表现优于基线方法。特别是，与生成方法相比，它将评估成本降低了75-85％，有效节省了20.27-38.80天的产品设计周期。

Title: Stylized Structural Patterns for Improved Neural Network Pre-training

• Authors: Farnood Salehi, Vandit Sharma, Amirhossein Askari Farsangi, Tunç Ozan Aydın
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19465
• Pdf URL: https://arxiv.org/pdf/2506.19465
• Copy Paste: [[2506.19465]] Stylized Structural Patterns for Improved Neural Network Pre-training(https://arxiv.org/abs/2506.19465)
• Keywords: generation
• Abstract: Modern deep learning models in computer vision require large datasets of real images, which are difficult to curate and pose privacy and legal concerns, limiting their commercial use. Recent works suggest synthetic data as an alternative, yet models trained with it often underperform. This paper proposes a two-step approach to bridge this gap. First, we propose an improved neural fractal formulation through which we introduce a new class of synthetic data. Second, we propose reverse stylization, a technique that transfers visual features from a small, license-free set of real images onto synthetic datasets, enhancing their effectiveness. We analyze the domain gap between our synthetic datasets and real images using Kernel Inception Distance (KID) and show that our method achieves a significantly lower distributional gap compared to existing synthetic datasets. Furthermore, our experiments across different tasks demonstrate the practical impact of this reduced gap. We show that pretraining the EDM2 diffusion model on our synthetic dataset leads to an 11% reduction in FID during image generation, compared to models trained on existing synthetic datasets, and a 20% decrease in autoencoder reconstruction error, indicating improved performance in data representation. Furthermore, a ViT-S model trained for classification on this synthetic data achieves over a 10% improvement in ImageNet-100 accuracy. Our work opens up exciting possibilities for training practical models when sufficiently large real training sets are not available.
• 摘要：计算机视觉中的现代深度学习模型需要大量的真实图像数据集，这些图像很难策划，构成隐私和法律问题，从而限制了它们的商业用途。最近的作品将合成数据作为替代方案，但经常表现不佳的模型。本文提出了两步方法来弥合这一差距。首先，我们提出了一种改进的神经分形式，通过它引入了新的合成数据。其次，我们提出了反向风格化，该技术将视觉特征从一组无许可的真实图像集传输到合成数据集中，从而提高了它们的有效性。我们使用内核成立距离（KID）分析了合成数据集和真实图像之间的域间隙，并表明我们的方法与现有的合成数据集相比达到了明显较低的分布差距。此外，我们跨不同任务的实验证明了这一减少差距的实际影响。我们表明，与对现有合成数据集训练的模型相比，在合成数据集上预处理EDM2扩散模型会导致图像生成期间的FID降低11％，并且自动编码器重建误差降低了20％，这表明数据表示的性能提高了。此外，通过该合成数据进行分类的VIT-S模型在Imagenet-100精度上提高了10％。当没有足够大的真实训练集时，我们的工作为训练实用模型提供了令人兴奋的可能性。

Title: SceneCrafter: Controllable Multi-View Driving Scene Editing

• Authors: Zehao Zhu, Yuliang Zou, Chiyu Max Jiang, Bo Sun, Vincent Casser, Xiukun Huang, Jiahao Wang, Zhenpei Yang, Ruiqi Gao, Leonidas Guibas, Mingxing Tan, Dragomir Anguelov
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19488
• Pdf URL: https://arxiv.org/pdf/2506.19488
• Copy Paste: [[2506.19488]] SceneCrafter: Controllable Multi-View Driving Scene Editing(https://arxiv.org/abs/2506.19488)
• Keywords: generation, generative
• Abstract: Simulation is crucial for developing and evaluating autonomous vehicle (AV) systems. Recent literature builds on a new generation of generative models to synthesize highly realistic images for full-stack simulation. However, purely synthetically generated scenes are not grounded in reality and have difficulty in inspiring confidence in the relevance of its outcomes. Editing models, on the other hand, leverage source scenes from real driving logs, and enable the simulation of different traffic layouts, behaviors, and operating conditions such as weather and time of day. While image editing is an established topic in computer vision, it presents fresh sets of challenges in driving simulation: (1) the need for cross-camera 3D consistency, (2) learning ``empty street" priors from driving data with foreground occlusions, and (3) obtaining paired image tuples of varied editing conditions while preserving consistent layout and geometry. To address these challenges, we propose SceneCrafter, a versatile editor for realistic 3D-consistent manipulation of driving scenes captured from multiple cameras. We build on recent advancements in multi-view diffusion models, using a fully controllable framework that scales seamlessly to multi-modality conditions like weather, time of day, agent boxes and high-definition maps. To generate paired data for supervising the editing model, we propose a novel framework on top of Prompt-to-Prompt to generate geometrically consistent synthetic paired data with global edits. We also introduce an alpha-blending framework to synthesize data with local edits, leveraging a model trained on empty street priors through novel masked training and multi-view repaint paradigm. SceneCrafter demonstrates powerful editing capabilities and achieves state-of-the-art realism, controllability, 3D consistency, and scene editing quality compared to existing baselines.
• 摘要：模拟对于开发和评估自动驾驶汽车（AV）系统至关重要。最近的文献建立在新一代的生成模型上，以合成高度逼真的图像以进行全堆栈模拟。但是，纯粹是合成生成的场景并不是现实的基础，并且很难激发其对结果的相关性的信心。另一方面，编辑模型从实际驾驶日志中利用源场景，并可以模拟不同的交通布局，行为和操作条件，例如天气和一天中的时间。虽然图像编辑是计算机视觉中的一个既定主题，但它在驱动模拟中提出了一系列挑战：（1）需要跨摄像机3D一致性，（2）学习````空心街道''先知，从驱动数据的驱动数据中，以及（3）获得各种编辑条件的成对映像，同时保存一致的布局和挑战，这些挑战是这些挑战。对于从多个摄像机捕获的驱动场景的现实，我们基于多视图扩散模型的最新进步，使用完全可控的框架，将其无缝地缩放到多模式的条件下，例如天气，时间的时间，代理盒子，代理盒子和高度定位图，以构建成对的框架。一致的合成数据与全球编辑。我们还引入了一个alpha融合的框架，以通过本地编辑进行综合，通过新颖的屏蔽训练和多视图重新绘制范式在空旷的街道先验上进行训练的模型。

Title: COLUR: Confidence-Oriented Learning, Unlearning and Relearning with Noisy-Label Data for Model Restoration and Refinement

• Authors: Zhihao Sui, Liang Hu, Jian Cao, Usman Naseem, Zhongyuan Lai, Qi Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19496
• Pdf URL: https://arxiv.org/pdf/2506.19496
• Copy Paste: [[2506.19496]] COLUR: Confidence-Oriented Learning, Unlearning and Relearning with Noisy-Label Data for Model Restoration and Refinement(https://arxiv.org/abs/2506.19496)
• Keywords: restoration
• Abstract: Large deep learning models have achieved significant success in various tasks. However, the performance of a model can significantly degrade if it is needed to train on datasets with noisy labels with misleading or ambiguous information. To date, there are limited investigations on how to restore performance when model degradation has been incurred by noisy label data. Inspired by the ``forgetting mechanism'' in neuroscience, which enables accelerating the relearning of correct knowledge by unlearning the wrong knowledge, we propose a robust model restoration and refinement (MRR) framework COLUR, namely Confidence-Oriented Learning, Unlearning and Relearning. Specifically, we implement COLUR with an efficient co-training architecture to unlearn the influence of label noise, and then refine model confidence on each label for relearning. Extensive experiments are conducted on four real datasets and all evaluation results show that COLUR consistently outperforms other SOTA methods after MRR.
• 摘要：大型深度学习模型在各种任务中取得了重大成功。但是，如果需要在具有误导性或模棱两可信息的嘈杂标签上训练数据集，则模型的性能会大大降低。迄今为止，当嘈杂的标签数据引起模型降解时，对如何恢复性能的研究有限。受神经科学中的``遗忘机制''的启发，这使我们提出了一个强大的模型恢复和改进（MRR）框架Colur，从而使正确知识的重新学习，即以信心为导向的学习，学习和重新学习。具体而言，我们实施具有有效的共同训练结构的Colur，以取消标签噪声的影响，然后在每个标签上提高模型置信度以进行重新学习。在四个实际数据集上进行了广泛的实验，所有评估结果表明，Colur在MRR之后始终优于其他SOTA方法。

Title: HOIverse: A Synthetic Scene Graph Dataset With Human Object Interactions

• Authors: Mrunmai Vivek Phatak, Julian Lorenz, Nico Hörmann, Jörg Hähner, Rainer Lienhart
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19639
• Pdf URL: https://arxiv.org/pdf/2506.19639
• Copy Paste: [[2506.19639]] HOIverse: A Synthetic Scene Graph Dataset With Human Object Interactions(https://arxiv.org/abs/2506.19639)
• Keywords: generation
• Abstract: When humans and robotic agents coexist in an environment, scene understanding becomes crucial for the agents to carry out various downstream tasks like navigation and planning. Hence, an agent must be capable of localizing and identifying actions performed by the human. Current research lacks reliable datasets for performing scene understanding within indoor environments where humans are also a part of the scene. Scene Graphs enable us to generate a structured representation of a scene or an image to perform visual scene understanding. To tackle this, we present HOIverse a synthetic dataset at the intersection of scene graph and human-object interaction, consisting of accurate and dense relationship ground truths between humans and surrounding objects along with corresponding RGB images, segmentation masks, depth images and human keypoints. We compute parametric relations between various pairs of objects and human-object pairs, resulting in an accurate and unambiguous relation definitions. In addition, we benchmark our dataset on state-of-the-art scene graph generation models to predict parametric relations and human-object interactions. Through this dataset, we aim to accelerate research in the field of scene understanding involving people.
• 摘要：当人类和机器人代理在环境中共存时，场景的理解对于代理人执行导航和计划等各种下游任务至关重要。因此，代理必须能够定位和识别人类执行的动作。当前的研究缺乏可靠的数据集来在人类也是现场的一部分的室内环境中进行场景理解。场景图使我们能够生成场景或图像的结构化表示形式，以执行视觉场景的理解。为了解决这个问题，我们在场景图和人类对象相互作用的交点上介绍了Hoiverse一个合成数据集，包括人类与周围物体之间的准确而密集的关系地面真理以及相应的RGB图像，分割掩码，深度图像和人类关键点。我们计算各对对象和人对象对之间的参数关系，从而产生准确且明确的关系定义。此外，我们将数据集基于最先进的场景图生成模型，以预测参数关系和人类对象相互作用。通过此数据集，我们旨在加快涉及人的场景理解领域的研究。

Title: Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model

• Authors: Shuncheng He, Hongchang Zhang, Jianzhun Shao, Yuhang Jiang, Xiangyang Ji
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19643
• Pdf URL: https://arxiv.org/pdf/2506.19643
• Copy Paste: [[2506.19643]] Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model(https://arxiv.org/abs/2506.19643)
• Keywords: generation
• Abstract: Offline reinforcement learning (RL) recently gains growing interests from RL researchers. However, the performance of offline RL suffers from the out-of-distribution problem, which can be corrected by feedback in online RL. Previous offline RL research focuses on restricting the offline algorithm in in-distribution even in-sample action sampling. In contrast, fewer work pays attention to the influence of the batch data. In this paper, we first build a bridge over the batch data and the performance of offline RL algorithms theoretically, from the perspective of model-based offline RL optimization. We draw a conclusion that, with mild assumptions, the distance between the state-action pair distribution generated by the behavioural policy and the distribution generated by the optimal policy, accounts for the performance gap between the policy learned by model-based offline RL and the optimal policy. Secondly, we reveal that in task-agnostic settings, a series of policies trained by unsupervised RL can minimize the worst-case regret in the performance gap. Inspired by the theoretical conclusions, UDG (Unsupervised Data Generation) is devised to generate data and select proper data for offline training under tasks-agnostic settings. Empirical results demonstrate that UDG can outperform supervised data generation on solving unknown tasks.
• 摘要：离线增强学习（RL）最近从RL研究人员那里获得了不断增长的兴趣。但是，离线RL的性能遭受了分布外问题的影响，可以通过在线RL中的反馈来纠正。以前的离线RL研究重点是限制离线算法在分布中，即使样本中的动作抽样也是如此。相比之下，更少的工作引起人们对批处理数据的影响的关注。在本文中，我们首先从批处理数据和离线RL算法的性能从理论上讲，从基于模型的离线RL优化的角度来看。我们得出一个结论，即有了温和的假设，即行为策略产生的国家行动对分布之间的距离与最佳策略产生的分布之间的距离说明了基于模型的离线RL与最佳策略所学的策略之间的绩效差距。其次，我们透露，在任务不合时宜的环境中，一系列由无监督的RL训练的政策可以最大程度地减少性能差距中最糟糕的遗憾。受理论结论的启发，UDG（无监督的数据生成）是为了生成数据并在任务 - 不可能设置下的离线培训的适当数据而设计的。经验结果表明，在解决未知任务方面，UDG可以胜过监督的数据生成。

Title: SAM2-SGP: Enhancing SAM2 for Medical Image Segmentation via Support-Set Guided Prompting

• Authors: Yang Xing, Jiong Wu, Yuheng Bu, Kuang Gong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19658
• Pdf URL: https://arxiv.org/pdf/2506.19658
• Copy Paste: [[2506.19658]] SAM2-SGP: Enhancing SAM2 for Medical Image Segmentation via Support-Set Guided Prompting(https://arxiv.org/abs/2506.19658)
• Keywords: generation
• Abstract: Although new vision foundation models such as Segment Anything Model 2 (SAM2) have significantly enhanced zero-shot image segmentation capabilities, reliance on human-provided prompts poses significant challenges in adapting SAM2 to medical image segmentation tasks. Moreover, SAM2's performance in medical image segmentation was limited by the domain shift issue, since it was originally trained on natural images and videos. To address these challenges, we proposed SAM2 with support-set guided prompting (SAM2-SGP), a framework that eliminated the need for manual prompts. The proposed model leveraged the memory mechanism of SAM2 to generate pseudo-masks using image-mask pairs from a support set via a Pseudo-mask Generation (PMG) module. We further introduced a novel Pseudo-mask Attention (PMA) module, which used these pseudo-masks to automatically generate bounding boxes and enhance localized feature extraction by guiding attention to relevant areas. Furthermore, a low-rank adaptation (LoRA) strategy was adopted to mitigate the domain shift issue. The proposed framework was evaluated on both 2D and 3D datasets across multiple medical imaging modalities, including fundus photography, X-ray, computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and ultrasound. The results demonstrated a significant performance improvement over state-of-the-art models, such as nnUNet and SwinUNet, as well as foundation models, such as SAM2 and MedSAM2, underscoring the effectiveness of the proposed approach. Our code is publicly available at this https URL.
• 摘要：尽管新视觉基础模型（例如任何模型2（SAM2））具有显着增强的零照片图像分割功能，但对人提供的提示的依赖在使SAM2适应医疗图像分割任务方面构成了重大挑战。此外，SAM2在医疗图像细分中的性能受到域名偏移问题的限制，因为它最初是对自然图像和视频进行培训的。为了应对这些挑战，我们提出了SAM2，并提出了支持设置的指导提示（SAM2-SGP），该框架消除了对手动提示的需求。提出的模型利用SAM2的存储机理使用来自支持集的图像掩码对通过伪掩码生成（PMG）模块生成伪面罩。我们进一步引入了一种新颖的伪遮罩注意力（PMA）模块，该模块使用这些伪遮罩自动生成边界框，并通过指导注意相关区域来增强局部特征提取。此外，采用了低级适应（LORA）策略来减轻域转移问题。在多种医学成像方式上对2D和3D数据集进行了评估，包括底底摄影，X射线，计算机断层扫描（CT），磁共振成像（MRI），正电子发射断层扫描（PET）和超声。结果表明，诸如NNUNET和SWINUNET等最新模型以及SAM2和MEDSAM2等基础模型的性能有了显着改善，强调了所提出的方法的有效性。我们的代码在此HTTPS URL上公开可用。

Title: Recurrent Visual Feature Extraction and Stereo Attentions for CT Report Generation

• Authors: Yuanhe Tian, Lei Mao, Yan Song
• Subjects: cs.CV, cs.CL
• Abstract URL: https://arxiv.org/abs/2506.19665
• Pdf URL: https://arxiv.org/pdf/2506.19665
• Copy Paste: [[2506.19665]] Recurrent Visual Feature Extraction and Stereo Attentions for CT Report Generation(https://arxiv.org/abs/2506.19665)
• Keywords: generation
• Abstract: Generating reports for computed tomography (CT) images is a challenging task, while similar to existing studies for medical image report generation, yet has its unique characteristics, such as spatial encoding of multiple images, alignment between image volume and texts, etc. Existing solutions typically use general 2D or 3D image processing techniques to extract features from a CT volume, where they firstly compress the volume and then divide the compressed CT slices into patches for visual encoding. These approaches do not explicitly account for the transformations among CT slices, nor do they effectively integrate multi-level image features, particularly those containing specific organ lesions, to instruct CT report generation (CTRG). In considering the strong correlation among consecutive slices in CT scans, in this paper, we propose a large language model (LLM) based CTRG method with recurrent visual feature extraction and stereo attentions for hierarchical feature modeling. Specifically, we use a vision Transformer to recurrently process each slice in a CT volume, and employ a set of attentions over the encoded slices from different perspectives to selectively obtain important visual information and align them with textual features, so as to better instruct an LLM for CTRG. Experiment results and further analysis on the benchmark M3D-Cap dataset show that our method outperforms strong baseline models and achieves state-of-the-art results, demonstrating its validity and effectiveness.
• 摘要：Generating reports for computed tomography (CT) images is a challenging task, while similar to existing studies for medical image report generation, yet has its unique characteristics, such as spatial encoding of multiple images, alignment between image volume and texts, etc. Existing solutions typically use general 2D or 3D image processing techniques to extract features from a CT volume, where they firstly compress the volume and then divide the compressed CT slices into patches for visual encoding.这些方法不能明确说明CT切片之间的转换，也不能有效地整合多级图像特征，尤其是那些包含特定器官病变的图像特征，以指导CT报告生成（CTRG）。在考虑CT扫描中连续切片之间的强相关性时，在本文中，我们提出了一种基于大型语言模型（LLM）的CTRG方法，具有复发性视觉特征提取和立体声专注于层次功能建模。具体而言，我们使用视觉变压器在CT卷中反复处理每个切片，并从不同的角度对编码切片进行一系列关注，以选择性地获得重要的视觉信息，并与文本功能对齐，以便更好地指导CTRG的LLM。实验结果和对基准M3D-CAP数据集的进一步分析表明，我们的方法的表现优于强大的基线模型，并实现了最先进的结果，证明了其有效性和有效性。

Title: Learning-aided Bigraph Matching Approach to Multi-Crew Restoration of Damaged Power Networks Coupled with Road Transportation Networks

• Authors: Nathan Maurer, Harshal Kaushik, Roshni Anna Jacob, Jie Zhang, Souma Chowdhury
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19703
• Pdf URL: https://arxiv.org/pdf/2506.19703
• Copy Paste: [[2506.19703]] Learning-aided Bigraph Matching Approach to Multi-Crew Restoration of Damaged Power Networks Coupled with Road Transportation Networks(https://arxiv.org/abs/2506.19703)
• Keywords: restoration
• Abstract: The resilience of critical infrastructure networks (CINs) after disruptions, such as those caused by natural hazards, depends on both the speed of restoration and the extent to which operational functionality can be regained. Allocating resources for restoration is a combinatorial optimal planning problem that involves determining which crews will repair specific network nodes and in what order. This paper presents a novel graph-based formulation that merges two interconnected graphs, representing crew and transportation nodes and power grid nodes, into a single heterogeneous graph. To enable efficient planning, graph reinforcement learning (GRL) is integrated with bigraph matching. GRL is utilized to design the incentive function for assigning crews to repair tasks based on the graph-abstracted state of the environment, ensuring generalization across damage scenarios. Two learning techniques are employed: a graph neural network trained using Proximal Policy Optimization and another trained via Neuroevolution. The learned incentive functions inform a bipartite graph that links crews to repair tasks, enabling weighted maximum matching for crew-to-task allocations. An efficient simulation environment that pre-computes optimal node-to-node path plans is used to train the proposed restoration planning methods. An IEEE 8500-bus power distribution test network coupled with a 21 square km transportation network is used as the case study, with scenarios varying in terms of numbers of damaged nodes, depots, and crews. Results demonstrate the approach's generalizability and scalability across scenarios, with learned policies providing 3-fold better performance than random policies, while also outperforming optimization-based solutions in both computation time (by several orders of magnitude) and power restored.
• 摘要：干扰后关键基础设施网络（CIN）的弹性（例如由自然危害引起的）取决于恢复速度和可以恢复运营功能的程度。为恢复资源分配资源是一个组合最佳计划问题，涉及确定哪些机组人员将以什么顺序修复特定的网络节点。本文提出了一种基于图的新型公式，该公式将两个相互联系的图合并为代表机组人员和传输节点和功率网格节点，以单个异构图。为了实现有效的计划，图形增强学习（GRL）与Bigraph匹配集成在一起。 GRL用于设计激励功能，用于根据图形的环境状态分配机组人员来修复任务，从而确保跨伤害方案的概括。采用了两种学习技术：使用近端策略优化训练的图形神经网络，另一种通过神经进化进行了培训。学到的激励功能为二分图提供了链接工作人员来维修任务，从而为机组人员到任务分配提供了加权最大匹配。预先计算最佳节点到节点路径计划的有效仿真环境用于训练拟议的恢复计划方法。 IEEE 8500-BUS配电测试网络，加上21平方公里的运输网络用作案例研究，场景在损坏的节点，仓库和机组人员的数量方面有所不同。结果证明了该方法在各场景中的可推广性和可伸缩性，学到的策略的性能比随机策略要好3倍，同时在计算时间（通过几个数量级）和恢复功率的计算时间内也优于基于优化的解决方案。

Title: Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low CFG Scales

• Authors: Seyedmorteza Sadat, Tobias Vontobel, Farnood Salehi, Romann M. Weber
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19713
• Pdf URL: https://arxiv.org/pdf/2506.19713
• Copy Paste: [[2506.19713]] Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low CFG Scales(https://arxiv.org/abs/2506.19713)
• Keywords: generation
• Abstract: Classifier-free guidance (CFG) has become an essential component of modern conditional diffusion models. Although highly effective in practice, the underlying mechanisms by which CFG enhances quality, detail, and prompt alignment are not fully understood. We present a novel perspective on CFG by analyzing its effects in the frequency domain, showing that low and high frequencies have distinct impacts on generation quality. Specifically, low-frequency guidance governs global structure and condition alignment, while high-frequency guidance mainly enhances visual fidelity. However, applying a uniform scale across all frequencies -- as is done in standard CFG -- leads to oversaturation and reduced diversity at high scales and degraded visual quality at low scales. Based on these insights, we propose frequency-decoupled guidance (FDG), an effective approach that decomposes CFG into low- and high-frequency components and applies separate guidance strengths to each component. FDG improves image quality at low guidance scales and avoids the drawbacks of high CFG scales by design. Through extensive experiments across multiple datasets and models, we demonstrate that FDG consistently enhances sample fidelity while preserving diversity, leading to improved FID and recall compared to CFG, establishing our method as a plug-and-play alternative to standard classifier-free guidance.
• 摘要：无分类器指导（CFG）已成为现代条件扩散模型的重要组成部分。尽管在实践中非常有效，但CFG提高质量，细节和及时对齐方式的基本机制尚未完全理解。我们通过分析其在频域中的效果，表明低频和高频对发电质量有明显的影响，从而介绍了CFG的新观点。具体而言，低频指导控制着全球结构和条件对准，而高频指导主要增强了视觉保真度。但是，在所有频率上应用均匀的量表（如标准CFG中所做的那样）会导致过度饱和度和降低高度的多样性，并在低尺度下降低视觉质量。基于这些见解，我们提出了频率耦合指导（FDG），这是一种有效的方法，将CFG分解为低频和高频组件，并对每个组件应用单独的指导强度。 FDG在低引导量表上提高了图像质量，并避免了设计高CFG量表的缺点。通过对多个数据集和模型进行的广泛实验，我们证明FDG始终增强样本保真度，同时保持多样性，从而改善了与CFG相比的FID和召回，并确定了我们的方法作为无需标准分类器指南的插件替代品。

Title: Who Does What in Deep Learning? Multidimensional Game-Theoretic Attribution of Function of Neural Units

• Authors: Shrey Dixit, Kayson Fakhar, Fatemeh Hadaeghi, Patrick Mineault, Konrad P. Kording, Claus C. Hilgetag
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.19732
• Pdf URL: https://arxiv.org/pdf/2506.19732
• Copy Paste: [[2506.19732]] Who Does What in Deep Learning? Multidimensional Game-Theoretic Attribution of Function of Neural Units(https://arxiv.org/abs/2506.19732)
• Keywords: generation, generative
• Abstract: Neural networks now generate text, images, and speech with billions of parameters, producing a need to know how each neural unit contributes to these high-dimensional outputs. Existing explainable-AI methods, such as SHAP, attribute importance to inputs, but cannot quantify the contributions of neural units across thousands of output pixels, tokens, or logits. Here we close that gap with Multiperturbation Shapley-value Analysis (MSA), a model-agnostic game-theoretic framework. By systematically lesioning combinations of units, MSA yields Shapley Modes, unit-wise contribution maps that share the exact dimensionality of the model's output. We apply MSA across scales, from multi-layer perceptrons to the 56-billion-parameter Mixtral-8x7B and Generative Adversarial Networks (GAN). The approach demonstrates how regularisation concentrates computation in a few hubs, exposes language-specific experts inside the LLM, and reveals an inverted pixel-generation hierarchy in GANs. Together, these results showcase MSA as a powerful approach for interpreting, editing, and compressing deep neural networks.
• 摘要：神经网络现在以数十亿个参数生成文本，图像和语音，从而需要知道每个神经单元如何贡献这些高维输出。现有的可解释的AI方法，例如Shap，将重要性归因于输入，但无法量化数以千计的输出像素，令牌或logits的神经单位的贡献。在这里，我们使用多重扰动Shapley-Value Analysis（MSA）缩小差距，这是一个模型 - 敏捷的游戏理论框架。通过系统地损害单元的组合，MSA产生shapley模式，单位贡献图具有共享模型输出的确切维度的单位贡献图。我们在范围内应用MSA，从多层感知器到560亿参数Mixtral-8x7b和生成对抗网络（GAN）。该方法证明了正则化如何在一些枢纽中浓缩计算，揭露了LLM中语言特定的专家，并揭示了GAN中的倒立像素代代层次结构。这些结果共同展示了MSA作为解释，编辑和压缩深神经网络的有力方法。

Title: Noise Consistency Training: A Native Approach for One-Step Generator in Learning Additional Controls

• Authors: Yihong Luo, Shuchen Xue, Tianyang Hu, Jing Tang
• Subjects: cs.LG, cs.CV, stat.ML
• Abstract URL: https://arxiv.org/abs/2506.19741
• Pdf URL: https://arxiv.org/pdf/2506.19741
• Copy Paste: [[2506.19741]] Noise Consistency Training: A Native Approach for One-Step Generator in Learning Additional Controls(https://arxiv.org/abs/2506.19741)
• Keywords: generation
• Abstract: The pursuit of efficient and controllable high-quality content generation remains a central challenge in artificial intelligence-generated content (AIGC). While one-step generators, enabled by diffusion distillation techniques, offer excellent generation quality and computational efficiency, adapting them to new control conditions--such as structural constraints, semantic guidelines, or external inputs--poses a significant challenge. Conventional approaches often necessitate computationally expensive modifications to the base model and subsequent diffusion distillation. This paper introduces Noise Consistency Training (NCT), a novel and lightweight approach to directly integrate new control signals into pre-trained one-step generators without requiring access to original training images or retraining the base diffusion model. NCT operates by introducing an adapter module and employs a noise consistency loss in the noise space of the generator. This loss aligns the adapted model's generation behavior across noises that are conditionally dependent to varying degrees, implicitly guiding it to adhere to the new control. Theoretically, this training objective can be understood as minimizing the distributional distance between the adapted generator and the conditional distribution induced by the new conditions. NCT is modular, data-efficient, and easily deployable, relying only on the pre-trained one-step generator and a control signal model. Extensive experiments demonstrate that NCT achieves state-of-the-art controllable generation in a single forward pass, surpassing existing multi-step and distillation-based methods in both generation quality and computational efficiency. Code is available at this https URL
• 摘要：在人工智能生成的内容（AIGC）中，追求高效且可控制的高质量内容生成仍然是一个核心挑战。通过扩散蒸馏技术启用的一步生成器，提供了出色的发电质量和计算效率，使其适应了新的控制条件（例如结构性约束，语义指南或外部输入），这是一个重大挑战。常规方法通常需要对基本模型进行计算昂贵的修改，并需要进行随后的扩散蒸馏。本文介绍了噪声一致性训练（NCT），这是一种新颖且轻巧的方法，可将新的控制信号直接整合到预训练的一步发电机中，而无需访问原始训练图像或重新训练基础扩散模型。 NCT通过引入适配器模块并在发电机的噪声空间中采用噪声一致性损失来运行。这种损失使改编的模型在有条件地取决于不同程度的噪声上的噪声使它的生成行为保持一致，从而暗中引导其遵守新的控制。从理论上讲，可以将这个训练目标理解为最小化适应性发电机和新条件引起的条件分布之间的分布距离。 NCT仅取决于预先训练的一步生成器和控制信号模型，是模块化的，数据效率且易于部署的。广泛的实验表明，NCT在单个前向通行证中实现了最新的可控生成，超过了在发电质量和计算效率方面的现有多步骤和基于蒸馏的方法。代码可在此HTTPS URL上找到

Title: CoCo4D: Comprehensive and Complex 4D Scene Generation

• Authors: Junwei Zhou, Xueting Li, Lu Qi, Ming-Hsuan Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19798
• Pdf URL: https://arxiv.org/pdf/2506.19798
• Copy Paste: [[2506.19798]] CoCo4D: Comprehensive and Complex 4D Scene Generation(https://arxiv.org/abs/2506.19798)
• Keywords: generation
• Abstract: Existing 4D synthesis methods primarily focus on object-level generation or dynamic scene synthesis with limited novel views, restricting their ability to generate multi-view consistent and immersive dynamic 4D scenes. To address these constraints, we propose a framework (dubbed as CoCo4D) for generating detailed dynamic 4D scenes from text prompts, with the option to include images. Our method leverages the crucial observation that articulated motion typically characterizes foreground objects, whereas background alterations are less pronounced. Consequently, CoCo4D divides 4D scene synthesis into two responsibilities: modeling the dynamic foreground and creating the evolving background, both directed by a reference motion sequence. Given a text prompt and an optional reference image, CoCo4D first generates an initial motion sequence utilizing video diffusion models. This motion sequence then guides the synthesis of both the dynamic foreground object and the background using a novel progressive outpainting scheme. To ensure seamless integration of the moving foreground object within the dynamic background, CoCo4D optimizes a parametric trajectory for the foreground, resulting in realistic and coherent blending. Extensive experiments show that CoCo4D achieves comparable or superior performance in 4D scene generation compared to existing methods, demonstrating its effectiveness and efficiency. More results are presented on our website this https URL.
• 摘要：现有的4D合成方法主要集中于对象级生成或动态场景综合，具有有限的新颖视图，从而限制了它们生成一致和沉浸式动态4D场景的多视图的能力。为了解决这些约束，我们提出了一个框架（称为可可4D），用于从文本提示中生成详细的动态4D场景，并选择包括图像。我们的方法利用了至关重要的观察，即表达运动通常表征前景对象，而背景改变则不太明显。因此，可可4D将4D场景的合成分为两个职责：对动态前景进行建模并创建不断发展的背景，均由参考运动序列导演。给定文本提示和可选参考图像，Coco4D首先生成了使用视频扩散模型的初始运动序列。然后，该运动序列使用一种新型的渐进支出方案来指导动态前景对象和背景的合成。为了确保动态背景中移动前景对象的无缝集成，Coco4D优化了前景的参数轨迹，从而导致了逼真而连贯的混合。广泛的实验表明，与现有方法相比，可可4D在4D场景产生中实现了可比性或出色的性能，这表明其有效性和效率。此HTTPS URL在我们的网站上显示了更多结果。

Title: Bind-Your-Avatar: Multi-Talking-Character Video Generation with Dynamic 3D-mask-based Embedding Router

• Authors: Yubo Huang, Weiqiang Wang, Sirui Zhao, Tong Xu, Lin Liu, Enhong Chen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19833
• Pdf URL: https://arxiv.org/pdf/2506.19833
• Copy Paste: [[2506.19833]] Bind-Your-Avatar: Multi-Talking-Character Video Generation with Dynamic 3D-mask-based Embedding Router(https://arxiv.org/abs/2506.19833)
• Keywords: generation
• Abstract: Recent years have witnessed remarkable advances in audio-driven talking head generation. However, existing approaches predominantly focus on single-character scenarios. While some methods can create separate conversation videos between two individuals, the critical challenge of generating unified conversation videos with multiple physically co-present characters sharing the same spatial environment remains largely unaddressed. This setting presents two key challenges: audio-to-character correspondence control and the lack of suitable datasets featuring multi-character talking videos within the same scene. To address these challenges, we introduce Bind-Your-Avatar, an MM-DiT-based model specifically designed for multi-talking-character video generation in the same scene. Specifically, we propose (1) A novel framework incorporating a fine-grained Embedding Router that binds `who' and `speak what' together to address the audio-to-character correspondence control. (2) Two methods for implementing a 3D-mask embedding router that enables frame-wise, fine-grained control of individual characters, with distinct loss functions based on observed geometric priors and a mask refinement strategy to enhance the accuracy and temporal smoothness of the predicted masks. (3) The first dataset, to the best of our knowledge, specifically constructed for multi-talking-character video generation, and accompanied by an open-source data processing pipeline, and (4) A benchmark for the dual-talking-characters video generation, with extensive experiments demonstrating superior performance over multiple state-of-the-art methods.
• 摘要：近年来，在音频驱动的谈话校长一代中取得了显着进步。但是，现有方法主要集中在单个字符方案上。虽然某些方法可以在两个人之间创建单独的对话视频，但产生统一的对话视频的关键挑战具有多个物理共同的角色共享相同的空间环境，这在很大程度上仍未得到满足。此设置提出了两个关键挑战：音频到字符对应控制和缺乏在同一场景中具有多字符说话视频的合适数据集。为了应对这些挑战，我们介绍了bind-your-avatar，这是一种基于MM-DIT的模型，专门为在同一场景中的多词字符视频生成而设计。具体而言，我们建议（1）一个新型框架，其中包含了一个绑定了“谁”并“说话”的细粒嵌入路由器，以解决音频到字符的对应关系控制。 （2）实现3D面罩嵌入路由器的两种方法，该路由器能够对单个角色进行框架，细粒度的控制，具有基于观察到的几何学先验的明显损失功能，以及一个掩盖改进策略，以增强预测蒙版的准确性和时间平稳性。 （3）据我们所知，第一个数据集是专门为多聊天字符视频的生成而构建的，并伴随着开源数据处理管道，（4）（4）双向词性视频生成的基准测试，具有广泛的实验，表明超过了多个州立大学的出色表现。

Title: SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution

• Authors: Liangbin Xie, Yu Li, Shian Du, Menghan Xia, Xintao Wang, Fanghua Yu, Ziyan Chen, Pengfei Wan, Jiantao Zhou, Chao Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19838
• Pdf URL: https://arxiv.org/pdf/2506.19838
• Copy Paste: [[2506.19838]] SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution(https://arxiv.org/abs/2506.19838)
• Keywords: super-resolution, generation
• Abstract: Latent diffusion models have emerged as a leading paradigm for efficient video generation. However, as user expectations shift toward higher-resolution outputs, relying solely on latent computation becomes inadequate. A promising approach involves decoupling the process into two stages: semantic content generation and detail synthesis. The former employs a computationally intensive base model at lower resolutions, while the latter leverages a lightweight cascaded video super-resolution (VSR) model to achieve high-resolution output. In this work, we focus on studying key design principles for latter cascaded VSR models, which are underexplored currently. First, we propose two degradation strategies to generate training pairs that better mimic the output characteristics of the base model, ensuring alignment between the VSR model and its upstream generator. Second, we provide critical insights into VSR model behavior through systematic analysis of (1) timestep sampling strategies, (2) noise augmentation effects on low-resolution (LR) inputs. These findings directly inform our architectural and training innovations. Finally, we introduce interleaving temporal unit and sparse local attention to achieve efficient training and inference, drastically reducing computational overhead. Extensive experiments demonstrate the superiority of our framework over existing methods, with ablation studies confirming the efficacy of each design choice. Our work establishes a simple yet effective baseline for cascaded video super-resolution generation, offering practical insights to guide future advancements in efficient cascaded synthesis systems.
• 摘要：潜在扩散模型已成为有效的视频生成的领先范式。但是，随着用户期望转向高分辨率输出，仅依靠潜在计算就变得不足。有希望的方法涉及将过程分为两个阶段：语义内容产生和细节综合。前者在较低的分辨率下采用计算密集型基础模型，而后者则利用轻质级联的视频超分辨率（VSR）模型来实现高分辨率输出。在这项工作中，我们专注于研究后者级联VSR模型的关键设计原理，目前却没有被淘汰。首先，我们提出了两种退化策略，以生成训练对，以更好地模仿基本模型的输出特征，从而确保VSR模型与其上游生成器之间的对齐。其次，我们通过对（1）时间段采样策略，（2）对低分辨率（LR）输入的降噪影响的系统分析来提供对VSR模型行为的关键见解。这些发现直接为我们的建筑和培训创新提供了信息。最后，我们引入了交织的时间单元和稀疏的局部关注，以实现有效的训练和推理，从而大大降低了计算开销。广泛的实验证明了我们的框架优于现有方法，通过消融研究证实了每个设计选择的功效。我们的工作为级联的视频超分辨率一代建立了一个简单而有效的基线，提供了实用的见解，以指导有效级联合成系统的未来进步。

Title: Improving Progressive Generation with Decomposable Flow Matching

• Authors: Moayed Haji-Ali, Willi Menapace, Ivan Skorokhodov, Arpit Sahni, Sergey Tulyakov, Vicente Ordonez, Aliaksandr Siarohin
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2506.19839
• Pdf URL: https://arxiv.org/pdf/2506.19839
• Copy Paste: [[2506.19839]] Improving Progressive Generation with Decomposable Flow Matching(https://arxiv.org/abs/2506.19839)
• Keywords: generation
• Abstract: Generating high-dimensional visual modalities is a computationally intensive task. A common solution is progressive generation, where the outputs are synthesized in a coarse-to-fine spectral autoregressive manner. While diffusion models benefit from the coarse-to-fine nature of denoising, explicit multi-stage architectures are rarely adopted. These architectures have increased the complexity of the overall approach, introducing the need for a custom diffusion formulation, decomposition-dependent stage transitions, add-hoc samplers, or a model cascade. Our contribution, Decomposable Flow Matching (DFM), is a simple and effective framework for the progressive generation of visual media. DFM applies Flow Matching independently at each level of a user-defined multi-scale representation (such as Laplacian pyramid). As shown by our experiments, our approach improves visual quality for both images and videos, featuring superior results compared to prior multistage frameworks. On Imagenet-1k 512px, DFM achieves 35.2% improvements in FDD scores over the base architecture and 26.4% over the best-performing baseline, under the same training compute. When applied to finetuning of large models, such as FLUX, DFM shows faster convergence speed to the training distribution. Crucially, all these advantages are achieved with a single model, architectural simplicity, and minimal modifications to existing training pipelines.
• 摘要：产生高维视觉方式是一项计算密集的任务。一个常见的解决方案是渐进式生成，其中输出以粗到1的光谱自回归方式合成。尽管扩散模型受益于脱诺的粗到精细性质，但很少采用明确的多阶段体系结构。这些架构增加了整体方法的复杂性，引入了自定义扩散公式，依赖分解的阶段过渡，添加-HOC采样器或模型级联的需求。我们的贡献是可分解的流量匹配（DFM），是逐步生成视觉媒体的简单有效框架。 DFM在用户定义的多尺度表示（例如Laplacian Pyramid）的每个级别上独立应用流量匹配。如我们的实验所示，与先前的多阶段框架相比，我们的方法提高了图像和视频的视觉质量，其结果具有优越的结果。在Imagenet-1K 512PX上，DFM在同一训练计算下，FDD得分的提高35.2％，比基线最佳基线提高了26.4％。当应用于大型模型（例如Flux）时，DFM显示出更快的收敛速度与训练分布。至关重要的是，所有这些优点都是通过单个模型，建筑简单性以及对现有培训管道的最小修改实现的。

Title: GenHSI: Controllable Generation of Human-Scene Interaction Videos

• Authors: Zekun Li, Rui Zhou, Rahul Sajnani, Xiaoyan Cong, Daniel Ritchie, Srinath Sridhar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19840
• Pdf URL: https://arxiv.org/pdf/2506.19840
• Copy Paste: [[2506.19840]] GenHSI: Controllable Generation of Human-Scene Interaction Videos(https://arxiv.org/abs/2506.19840)
• Keywords: generation
• Abstract: Large-scale pre-trained video diffusion models have exhibited remarkable capabilities in diverse video generation. However, existing solutions face several challenges in using these models to generate long movie-like videos with rich human-object interactions that include unrealistic human-scene interaction, lack of subject identity preservation, and require expensive training. We propose GenHSI, a training-free method for controllable generation of long human-scene interaction videos (HSI). Taking inspiration from movie animation, our key insight is to overcome the limitations of previous work by subdividing the long video generation task into three stages: (1) script writing, (2) pre-visualization, and (3) animation. Given an image of a scene, a user description, and multiple images of a person, we use these three stages to generate long-videos that preserve human-identity and provide rich human-scene interactions. Script writing converts complex human tasks into simple atomic tasks that are used in the pre-visualization stage to generate 3D keyframes (storyboards). These 3D keyframes are rendered and animated by off-the-shelf video diffusion models for consistent long video generation with rich contacts in a 3D-aware manner. A key advantage of our work is that we alleviate the need for scanned, accurate scenes and create 3D keyframes from single-view images. We are the first to generate a long video sequence with a consistent camera pose that contains arbitrary numbers of character actions without training. Experiments demonstrate that our method can generate long videos that effectively preserve scene content and character identity with plausible human-scene interaction from a single image scene. Visit our project homepage this https URL for more information.
• 摘要：大规模的预训练视频扩散模型在不同的视频生成中表现出显着的功能。但是，现有的解决方案在使用这些模型来生成具有丰富人类对象相互作用的长电影式视频方面面临一些挑战，包括不切实际的人类场景互动，缺乏主题身份的保存以及需要昂贵的培训。我们提出了Genhsi，这是一种无训练的方法，可控制长期的人类习惯互动视频（HSI）。从电影动画中汲取灵感，我们的关键见解是通过将长时间的视频生成任务细分为三个阶段来克服以前的作品的局限性：（1）脚本写作，（2）预言和（3）动画。鉴于场景的图像，用户描述和一个人的多个图像，我们使用这三个阶段来生成长期视频，以保留人类身份并提供丰富的人类习惯相互作用。脚本写作将复杂的人类任务转换为简单的原子任务，这些任务用于生成3D密钥帧（故事板）的访问阶段。这些3D密钥帧由现成的视频扩散模型渲染和动画，以持续的长期视频生成，并以3D感知的方式进行丰富的触点。我们工作的关键优势是，我们减轻了对扫描，准确的场景的需求，并从单视图图像中创建3D密钥帧。我们是第一个以一致的相机姿势生成长时间视频序列的人，其中包含无需训练的角色动作数量的任意数量。实验表明，我们的方法可以生成长视频，从而有效地保留场景内容和角色身份，并通过单个图像场景中的合理的人类场景相互作用。请访问我们的项目主页此HTTPS URL以获取更多信息。

Title: Active View Selector: Fast and Accurate Active View Selection with Cross Reference Image Quality Assessment

• Authors: Zirui Wang, Yash Bhalgat, Ruining Li, Victor Adrian Prisacariu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19844
• Pdf URL: https://arxiv.org/pdf/2506.19844
• Copy Paste: [[2506.19844]] Active View Selector: Fast and Accurate Active View Selection with Cross Reference Image Quality Assessment(https://arxiv.org/abs/2506.19844)
• Keywords: quality assessment
• Abstract: We tackle active view selection in novel view synthesis and 3D reconstruction. Existing methods like FisheRF and ActiveNeRF select the next best view by minimizing uncertainty or maximizing information gain in 3D, but they require specialized designs for different 3D representations and involve complex modelling in 3D space. Instead, we reframe this as a 2D image quality assessment (IQA) task, selecting views where current renderings have the lowest quality. Since ground-truth images for candidate views are unavailable, full-reference metrics like PSNR and SSIM are inapplicable, while no-reference metrics, such as MUSIQ and MANIQA, lack the essential multi-view context. Inspired by a recent cross-referencing quality framework CrossScore, we train a model to predict SSIM within a multi-view setup and use it to guide view selection. Our cross-reference IQA framework achieves substantial quantitative and qualitative improvements across standard benchmarks, while being agnostic to 3D representations, and runs 14-33 times faster than previous methods.
• 摘要：我们在新型视图合成和3D重建中处理主动视图选择。 Fisherf和Activenerf等现有方法通过最大程度地减少3D的不确定性或最大化信息增益来选择下一个最佳视图，但是它们需要针对不同3D表示的专门设计，并涉及3D空间中的复杂建模。取而代之的是，我们将其重新构架为2D图像质量评估（IQA）任务，从而选择当前渲染质量最低的视图。由于无法获得候选视图的地面图像，因此PSNR和SSIM等全参考指标是不可应用的，而No-Reference指标（例如Musiq和Maniqa）都缺乏基本的多视图上下文。受到最近的交叉引用质量框架横向索的启发，我们训练模型以预测多视图设置中的SSIM并使用它来指导视图选择。我们的交叉参考IQA框架在标准基准的范围内实现了实质性的定量和定性改进，同时对3D表示不可知，并且运行速度比以前的方法快14-33倍。

Title: A Comparative Study of NAFNet Baselines for Image Restoration

• Authors: Vladislav Esaulov, M. Moein Esfahani
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19845
• Pdf URL: https://arxiv.org/pdf/2506.19845
• Copy Paste: [[2506.19845]] A Comparative Study of NAFNet Baselines for Image Restoration(https://arxiv.org/abs/2506.19845)
• Keywords: restoration
• Abstract: We study NAFNet (Nonlinear Activation Free Network), a simple and efficient deep learning baseline for image restoration. By using CIFAR10 images corrupted with noise and blur, we conduct an ablation study of NAFNet's core components. Our baseline model implements SimpleGate activation, Simplified Channel Activation (SCA), and LayerNormalization. We compare this baseline to different variants that replace or remove components. Quantitative results (PSNR, SSIM) and examples illustrate how each modification affects restoration performance. Our findings support the NAFNet design: the SimpleGate and simplified attention mechanisms yield better results than conventional activations and attention, while LayerNorm proves to be important for stable training. We conclude with recommendations for model design, discuss potential improvements, and future work.
• 摘要：我们研究NAFNET（非线性激活网络），这是一个简单有效的深度学习基线，用于图像恢复。通过使用噪声和模糊破坏的CIFAR10图像，我们对NAFNET的核心组件进行了消融研究。我们的基线模型实现了简单的激活，简化的通道激活（SCA）和分层构型。我们将此基线与替换或删除组件的不同变体进行比较。定量结果（PSNR，SSIM）和示例说明了每种修饰如何影响恢复性能。我们的发现支持NAFNET设计：与常规激活和注意力相比，简单盖特和简化的注意力机制可以产生更好的结果，而Layernorm对稳定训练非常重要。我们最终以模型设计的建议，讨论潜在的改进以及未来的工作。

Title: AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Models

• Authors: Zehuan Huang, Haoran Feng, Yangtian Sun, Yuanchen Guo, Yanpei Cao, Lu Sheng
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2506.19851
• Pdf URL: https://arxiv.org/pdf/2506.19851
• Copy Paste: [[2506.19851]] AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Models(https://arxiv.org/abs/2506.19851)
• Keywords: generation
• Abstract: We present AnimaX, a feed-forward 3D animation framework that bridges the motion priors of video diffusion models with the controllable structure of skeleton-based animation. Traditional motion synthesis methods are either restricted to fixed skeletal topologies or require costly optimization in high-dimensional deformation spaces. In contrast, AnimaX effectively transfers video-based motion knowledge to the 3D domain, supporting diverse articulated meshes with arbitrary skeletons. Our method represents 3D motion as multi-view, multi-frame 2D pose maps, and enables joint video-pose diffusion conditioned on template renderings and a textual motion prompt. We introduce shared positional encodings and modality-aware embeddings to ensure spatial-temporal alignment between video and pose sequences, effectively transferring video priors to motion generation task. The resulting multi-view pose sequences are triangulated into 3D joint positions and converted into mesh animation via inverse kinematics. Trained on a newly curated dataset of 160,000 rigged sequences, AnimaX achieves state-of-the-art results on VBench in generalization, motion fidelity, and efficiency, offering a scalable solution for category-agnostic 3D animation. Project page: \href{this https URL}{this https URL}.
• 摘要：我们提出了Animax，这是一种馈送3D动画框架，它将视频扩散模型的运动先验与基于骨架的动画的可控结构相桥。传统运动合成方法要么仅限于固定骨骼拓扑，要么需要在高维变形空间中进行昂贵的优化。相比之下，Animax有效地将基于视频的运动知识转移到了3D域，从而支持与任意骨架的各种铰接式网格。我们的方法将3D运动表示为多视图，多帧2D姿势图，并启用以模板渲染和文本运动提示为条件的联合视频置式扩散。我们介绍共享的位置编码和模态感知的嵌入，以确保视频和姿势序列之间的时空对齐，从而有效地将视频先验转移到运动生成任务。所得的多视图姿势序列将三角构为3D关节位置，并通过反向运动学转换为网状动画。 Animax在新策划的160,000个操纵序列的新策划数据集上进行了培训，可在概括，运动保真度和效率的VBench上获得最新的结果，为类别 - 敏捷的3D动画提供了可扩展的解决方案。项目页面：\ href {此https url} {this https url}。

Title: Radial Attention: $O(n\log n)$ Sparse Attention with Energy Decay for Long Video Generation

• Authors: Xingyang Li, Muyang Li, Tianle Cai, Haocheng Xi, Shuo Yang, Yujun Lin, Lvmin Zhang, Songlin Yang, Jinbo Hu, Kelly Peng, Maneesh Agrawala, Ion Stoica, Kurt Keutzer, Song Han
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2506.19852
• Pdf URL: https://arxiv.org/pdf/2506.19852
• Copy Paste: [[2506.19852]] Radial Attention: $O(n\log n)$ Sparse Attention with Energy Decay for Long Video Generation(https://arxiv.org/abs/2506.19852)
• Keywords: generation
• Abstract: Recent advances in diffusion models have enabled high-quality video generation, but the additional temporal dimension significantly increases computational costs, making training and inference on long videos prohibitively expensive. In this paper, we identify a phenomenon we term Spatiotemporal Energy Decay in video diffusion models: post-softmax attention scores diminish as spatial and temporal distance between tokens increase, akin to the physical decay of signal or waves over space and time in nature. Motivated by this, we propose Radial Attention, a scalable sparse attention mechanism with $O(n \log n)$ complexity that translates energy decay into exponentially decaying compute density, which is significantly more efficient than standard $O(n^2)$ dense attention and more expressive than linear attention. Specifically, Radial Attention employs a simple, static attention mask where each token attends to spatially nearby tokens, with the attention window size shrinking with temporal distance. Moreover, it allows pre-trained video diffusion models to extend their generation length with efficient LoRA-based fine-tuning. Extensive experiments show that Radial Attention maintains video quality across Wan2.1-14B, HunyuanVideo, and Mochi 1, achieving up to a 1.9$\times$ speedup over the original dense attention. With minimal tuning, it enables video generation up to 4$\times$ longer while reducing training costs by up to 4.4$\times$ compared to direct fine-tuning and accelerating inference by up to 3.7$\times$ compared to dense attention inference.
• 摘要：扩散模型的最新进展已使高质量的视频生成，但是额外的时间维度显着提高了计算成本，从而使长期视频的培训和推断非常昂贵。在本文中，我们确定了一种现象，我们在视频扩散模型中称呼时空能量衰减：随着代币之间的空间和时间距离增加，类似于自然界和时间上的信号衰减或自然界时间的物理衰减，柔软后的注意力评分会降低。在此激励的情况下，我们提出了径向关注，这是一种具有$ O（n \ log n）$复杂性的可扩展稀疏注意机制，它将能量衰减转化为指数衰减的计算密度，比标准$ O（N^2）$密集的注意力和表达性更高的效率要高得多。具体而言，径向注意力采用了一个简单的静态注意面膜，每个令牌都可以在空间附近的代币聚会，而注意力窗口的大小随时间距离而缩小。此外，它允许预先训练的视频扩散模型通过有效的基于洛拉的微调来扩​​展其发电长度。广泛的实验表明，Radial注意力在WAN2.1-14B，Hunyuanvideo和Mochi 1中保持视频质量，在原始密集的关注下达到了高达1.9 $ \ times $加速。随着最小的调整，与直接调整和加速推断相比，与密集的关注推断相比，与直接调整和加速推断相比，视频生成最多可长达4 $ \ tims $ $ herm $ $ $ \ times $。