2025-03-24

Title: Adams Bashforth Moulton Solver for Inversion and Editing in Rectified Flow

• Authors: Yongjia Ma, Donglin Di, Xuan Liu, Xiaokai Chen, Lei Fan, Wei Chen, Tonghua Su
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.16522
• Pdf URL: https://arxiv.org/pdf/2503.16522
• Copy Paste: [[2503.16522]] Adams Bashforth Moulton Solver for Inversion and Editing in Rectified Flow(https://arxiv.org/abs/2503.16522)
• Keywords: generation
• Abstract: Rectified flow models have achieved remarkable performance in image and video generation tasks. However, existing numerical solvers face a trade-off between fast sampling and high-accuracy solutions, limiting their effectiveness in downstream applications such as reconstruction and editing. To address this challenge, we propose leveraging the Adams-Bashforth-Moulton (ABM) predictor-corrector method to enhance the accuracy of ODE solving in rectified flow models. Specifically, we introduce ABM-Solver, which integrates a multi step predictor corrector approach to reduce local truncation errors and employs Adaptive Step Size Adjustment to improve sampling speed. Furthermore, to effectively preserve non edited regions while facilitating semantic modifications, we introduce a Mask Guided Feature Injection module. We estimate self-similarity to generate a spatial mask that differentiates preserved regions from those available for editing. Extensive experiments on multiple high-resolution image datasets validate that ABM-Solver significantly improves inversion precision and editing quality, outperforming existing solvers without requiring additional training or optimization.
• 摘要：整流的流模型在图像和视频生成任务中取得了出色的性能。但是，现有的数值求解器面临快速采样和高准确解决方案之间的权衡，从而限制了它们在重建和编辑等下游应用中的有效性。为了应对这一挑战，我们提出了利用Adams-Bashforth-Moulton（ABM）预测器 - 矫正器方法，以增强整流流模型中ODE求解的准确性。具体来说，我们介绍了ABM溶剂，该溶剂剂集成了多步预测器校正方法，以减少局部截断误差并采用自适应步骤尺寸调整以提高采样速度。此外，为了在促进语义修改的同时有效地保留非编辑区域，我们引入了掩盖引导的特征注入模块。我们估计自相似性，以产生一个空间面具，该面具将保存的区域与可用于编辑的区域区分开来。在多个高分辨率图像数据集上进行的广泛实验验证了ABM溶剂可显着提高反转精度和编辑质量，超过现有的求解器，而无需其他培训或优化。

Title: A Comprehensive Survey on Architectural Advances in Deep CNNs: Challenges, Applications, and Emerging Research Directions

• Authors: Saddam Hussain Khan, Rashid Iqbal (Artificial Intelligence Lab, Department of Computer Systems Engineering, University of Engineering and Applied Sciences (UEAS), Swat, Pakistan)
• Subjects: cs.CV, cs.AI, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.16546
• Pdf URL: https://arxiv.org/pdf/2503.16546
• Copy Paste: [[2503.16546]] A Comprehensive Survey on Architectural Advances in Deep CNNs: Challenges, Applications, and Emerging Research Directions(https://arxiv.org/abs/2503.16546)
• Keywords: generative
• Abstract: Deep Convolutional Neural Networks (CNNs) have significantly advanced deep learning, driving breakthroughs in computer vision, natural language processing, medical diagnosis, object detection, and speech recognition. Architectural innovations including 1D, 2D, and 3D convolutional models, dilated and grouped convolutions, depthwise separable convolutions, and attention mechanisms address domain-specific challenges and enhance feature representation and computational efficiency. Structural refinements such as spatial-channel exploitation, multi-path design, and feature-map enhancement contribute to robust hierarchical feature extraction and improved generalization, particularly through transfer learning. Efficient preprocessing strategies, including Fourier transforms, structured transforms, low-precision computation, and weight compression, optimize inference speed and facilitate deployment in resource-constrained environments. This survey presents a unified taxonomy that classifies CNN architectures based on spatial exploitation, multi-path structures, depth, width, dimensionality expansion, channel boosting, and attention mechanisms. It systematically reviews CNN applications in face recognition, pose estimation, action recognition, text classification, statistical language modeling, disease diagnosis, radiological analysis, cryptocurrency sentiment prediction, 1D data processing, video analysis, and speech recognition. In addition to consolidating architectural advancements, the review highlights emerging learning paradigms such as few-shot, zero-shot, weakly supervised, federated learning frameworks and future research directions include hybrid CNN-transformer models, vision-language integration, generative learning, etc. This review provides a comprehensive perspective on CNN's evolution from 2015 to 2025, outlining key innovations, challenges, and opportunities.
• 摘要：深度卷积神经网络（CNN）具有明显的高级深度学习，推动了计算机视觉，自然语言处理，医学诊断，对象检测和语音识别的突破。建筑创新，包括1D，2D和3D卷积模型，扩张和分组的卷积，深度可分离的卷积以及注意机制解决了特定领域的挑战，并提高了特征表示和计算效率。结构性改进，例如空间通道开发，多路径设计和特征映射增强功能，有助于稳健的层次特征提取和改进的概括，尤其是通过转移学习。有效的预处理策略，包括傅立叶变换，结构化变换，低精度计算和权重压缩，优化推理速度并促进在资源约束环境中的部署。这项调查提出了一种统一的分类法，该分类法根据空间剥削，多路径结构，深度，宽度，维度扩展，渠道增强和注意机制对CNN体系结构进行了分类。它系统地回顾了CNN在面部识别，姿势估计，行动识别，文本分类，统计语言建模，疾病诊断，放射学分析，加密货币情绪预测，1D数据处理，视频分析和语音识别方面的应用。除了巩固建筑的进步之外，审查还重点介绍了新兴的学习范例，例如射击，零射击，弱监督，联合学习框架和未来的研究方向，包括混合CNN转换器模型，视觉语言整合，生成性学习等。本综述，综述提供了对CNN的综合范围的范围，可在2015年及205号的范围内进行挑战，并提供了2025的范围。机会。

Title: Chem42: a Family of chemical Language Models for Target-aware Ligand Generation

• Authors: Aahan Singh, Engin Tekin, Maryam Nadeem, Nancy A. ElNaker, Mohammad Amaan Sayeed, Natalia Vassilieva, Boulbaba Ben Amor
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.16563
• Pdf URL: https://arxiv.org/pdf/2503.16563
• Copy Paste: [[2503.16563]] Chem42: a Family of chemical Language Models for Target-aware Ligand Generation(https://arxiv.org/abs/2503.16563)
• Keywords: generation, generative
• Abstract: Revolutionizing drug discovery demands more than just understanding molecular interactions - it requires generative models that can design novel ligands tailored to specific biological targets. While chemical Language Models (cLMs) have made strides in learning molecular properties, most fail to incorporate target-specific insights, restricting their ability to drive de-novo ligand generation. Chem42, a cutting-edge family of generative chemical Language Models, is designed to bridge this gap. By integrating atomic-level interactions with multimodal inputs from Prot42, a complementary protein Language Model, Chem42 achieves a sophisticated cross-modal representation of molecular structures, interactions, and binding patterns. This innovative framework enables the creation of structurally valid, synthetically accessible ligands with enhanced target specificity. Evaluations across diverse protein targets confirm that Chem42 surpasses existing approaches in chemical validity, target-aware design, and predicted binding affinity. By reducing the search space of viable drug candidates, Chem42 could accelerate the drug discovery pipeline, offering a powerful generative AI tool for precision medicine. Our Chem42 models set a new benchmark in molecule property prediction, conditional molecule generation, and target-aware ligand design. The models are publicly available at this http URL.
• 摘要：革命性的药物发现不仅需要了解分子相互作用，还需要生成模型，这些模型可以设计针对特定生物学靶标量身定制的新型配体。尽管化学语言模型（CLM）在学习分子特性方面取得了进步，但大多数人无法纳入目标特异性见解，从而限制了它们驱动De-Novo配体产生的能力。 Chem42是生成化学语言模型的尖端家族，旨在弥合这一差距。通过将原子水平的相互作用与Prot42的多模式输入（一种互补的蛋白质语言模型）整合在一起，Chem42实现了分子结构，相互作用和结合模式的复杂的交叉模式表示。这个创新的框架可以创建具有增强目标特异性的结构有效，可访问的配体。各种蛋白质靶标的评估证实，Chem42超过了化学有效性，目标感知设计和预测结合亲和力的现有方法。通过减少可行候选药物的搜索空间，Chem42可以加速药物发现管道，提供有力的生成AI工具，用于精密医学。我们的Chem42模型在分子属性预测，有条件分子的产生和靶向配体设计方面设定了新的基准。这些型号在此HTTP URL上公开可用。

Title: World Knowledge from AI Image Generation for Robot Control

• Authors: Jonas Krumme, Christoph Zetzsche
• Subjects: cs.CV, cs.RO
• Abstract URL: https://arxiv.org/abs/2503.16579
• Pdf URL: https://arxiv.org/pdf/2503.16579
• Copy Paste: [[2503.16579]] World Knowledge from AI Image Generation for Robot Control(https://arxiv.org/abs/2503.16579)
• Keywords: generation, generative
• Abstract: When interacting with the world robots face a number of difficult questions, having to make decisions when given under-specified tasks where they need to make choices, often without clearly defined right and wrong answers. Humans, on the other hand, can often rely on their knowledge and experience to fill in the gaps. For example, the simple task of organizing newly bought produce into the fridge involves deciding where to put each thing individually, how to arrange them together meaningfully, e.g. putting related things together, all while there is no clear right and wrong way to accomplish this task. We could encode all this information on how to do such things explicitly into the robots' knowledge base, but this can quickly become overwhelming, considering the number of potential tasks and circumstances the robot could encounter. However, images of the real world often implicitly encode answers to such questions and can show which configurations of objects are meaningful or are usually used by humans. An image of a full fridge can give a lot of information about how things are usually arranged in relation to each other and the full fridge at large. Modern generative systems are capable of generating plausible images of the real world and can be conditioned on the environment in which the robot operates. Here we investigate the idea of using the implicit knowledge about the world of modern generative AI systems given by their ability to generate convincing images of the real world to solve under-specified tasks.
• 摘要：当与世界机器人互动时，要面对许多困难的问题，必须在给出未指定的任务时做出决定时做出选择的决定，通常没有明确定义的正确和错误的答案。另一方面，人类通常可以依靠他们的知识和经验来填补空白。例如，组织新购买的农产品在冰箱中的简单任务涉及决定单独放置每件事的位置，如何有意义地将它们放在一起，例如将相关的内容放在一起，虽然没有明确的正确和错误的方法来完成这项任务。我们可以将有关如何将这些事情明确地进行到机器人的知识库中进行编码，但是考虑到机器人可能遇到的潜在任务和情况的数量，这很快就会变得不知所措。但是，现实世界的图像通常会隐式编码此类问题的答案，并可以显示对象的配置有意义或通常被人类使用。完整的冰箱图像可以提供大量有关通常如何相对于彼此和整个冰箱的内容的信息。现代生成系统能够生成现实世界中合理的图像，并且可以在机器人运行的环境下进行条件。在这里，我们调查了使用有关现代生成AI系统世界的隐性知识的想法，它们具有产生令人信服的现实世界图像解决未指定任务的能力。

Title: Explainable AI-Guided Efficient Approximate DNN Generation for Multi-Pod Systolic Arrays

• Authors: Ayesha Siddique, Khurram Khalil, Khaza Anuarul Hoque
• Subjects: cs.LG, cs.AI, cs.AR
• Abstract URL: https://arxiv.org/abs/2503.16583
• Pdf URL: https://arxiv.org/pdf/2503.16583
• Copy Paste: [[2503.16583]] Explainable AI-Guided Efficient Approximate DNN Generation for Multi-Pod Systolic Arrays(https://arxiv.org/abs/2503.16583)
• Keywords: generation
• Abstract: Approximate deep neural networks (AxDNNs) are promising for enhancing energy efficiency in real-world devices. One of the key contributors behind this enhanced energy efficiency in AxDNNs is the use of approximate multipliers. Unfortunately, the simulation of approximate multipliers does not usually scale well on CPUs and GPUs. As a consequence, this slows down the overall simulation of AxDNNs aimed at identifying the appropriate approximate multipliers to achieve high energy efficiency with a minimum accuracy loss. To address this problem, we present a novel XAI-Gen methodology, which leverages the analytical model of the emerging hardware accelerator (e.g., Google TPU v4) and explainable artificial intelligence (XAI) to precisely identify the non-critical layers for approximation and quickly discover the appropriate approximate multipliers for AxDNN layers. Our results show that XAI-Gen achieves up to 7x lower energy consumption with only 1-2% accuracy loss. We also showcase the effectiveness of the XAI-Gen approach through a neural architecture search (XAI-NAS) case study. Interestingly, XAI-NAS achieves 40\% higher energy efficiency with up to 5x less execution time when compared to the state-of-the-art NAS methods for generating AxDNNs.
• 摘要：近似深的神经网络（AXDNN）有望提高现实世界设备的能源效率。 AXDNNS增强能源效率的关键因素之一是使用近似乘数。不幸的是，近似乘数的模拟通常在CPU和GPU上尺寸不佳。结果，这减慢了AXDNN的整体模拟，旨在识别适当的近似乘数，以实现高能量效率，并以最小的精度损失。为了解决这个问题，我们提出了一种新颖的Xai-gen方法，该方法利用了新兴硬件加速器的分析模型（例如Google TPU V4）和可解释的人工智能（XAI）精确地识别非临界层以识别近似值，并迅速发现AXDNN层的适当近似近似值。我们的结果表明，Xai-gen的能源消耗量最大高达7倍，精度损失仅为1-2％。我们还通过神经建筑搜索（XAI-NAS）案例研究展示了Xai-Gen方法的有效性。有趣的是，与生成AXDNN的最先进的NAS方法相比，Xai-NAS的能源效率提高了40 \％，执行时间更少5倍。

Title: A Recipe for Generating 3D Worlds From a Single Image

• Authors: Katja Schwarz, Denys Rozumnyi, Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.16611
• Pdf URL: https://arxiv.org/pdf/2503.16611
• Copy Paste: [[2503.16611]] A Recipe for Generating 3D Worlds From a Single Image(https://arxiv.org/abs/2503.16611)
• Keywords: generative
• Abstract: We introduce a recipe for generating immersive 3D worlds from a single image by framing the task as an in-context learning problem for 2D inpainting models. This approach requires minimal training and uses existing generative models. Our process involves two steps: generating coherent panoramas using a pre-trained diffusion model and lifting these into 3D with a metric depth estimator. We then fill unobserved regions by conditioning the inpainting model on rendered point clouds, requiring minimal fine-tuning. Tested on both synthetic and real images, our method produces high-quality 3D environments suitable for VR display. By explicitly modeling the 3D structure of the generated environment from the start, our approach consistently outperforms state-of-the-art, video synthesis-based methods along multiple quantitative image quality metrics. Project Page: this https URL
• 摘要：我们介绍了一种食谱，从单个图像中从单个图像中产生沉浸式3D世界，通过将任务作为2D授课模型的秘密学习问题。这种方法需要最少的培训，并使用现有的生成模型。我们的过程涉及两个步骤：使用预训练的扩散模型生成相干全景图，并使用度量深度估计器将其提升为3D。然后，我们通过在渲染点云上调节介绍模型来填充未观察到的区域，需要微调的微调。在合成图像和真实图像上测试，我们的方法可产生适合VR显示的高质量3D环境。通过从一开始就明确对生成环境的3D结构进行建模，我们的方法始终超过了最先进的基于视频综合的方法，沿多个定量图像质量指标。项目页面：此HTTPS URL

Title: iFlame: Interleaving Full and Linear Attention for Efficient Mesh Generation

• Authors: Hanxiao Wang, Biao Zhang, Weize Quan, Dong-Ming Yan, Peter Wonka
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.16653
• Pdf URL: https://arxiv.org/pdf/2503.16653
• Copy Paste: [[2503.16653]] iFlame: Interleaving Full and Linear Attention for Efficient Mesh Generation(https://arxiv.org/abs/2503.16653)
• Keywords: generation, generative
• Abstract: This paper propose iFlame, a novel transformer-based network architecture for mesh generation. While attention-based models have demonstrated remarkable performance in mesh generation, their quadratic computational complexity limits scalability, particularly for high-resolution 3D data. Conversely, linear attention mechanisms offer lower computational costs but often struggle to capture long-range dependencies, resulting in suboptimal outcomes. To address this trade-off, we propose an interleaving autoregressive mesh generation framework that combines the efficiency of linear attention with the expressive power of full attention mechanisms. To further enhance efficiency and leverage the inherent structure of mesh representations, we integrate this interleaving approach into an hourglass architecture, which significantly boosts efficiency. Our approach reduces training time while achieving performance comparable to pure attention-based models. To improve inference efficiency, we implemented a caching algorithm that almost doubles the speed and reduces the KV cache size by seven-eighths compared to the original Transformer. We evaluate our framework on ShapeNet and Objaverse, demonstrating its ability to generate high-quality 3D meshes efficiently. Our results indicate that the proposed interleaving framework effectively balances computational efficiency and generative performance, making it a practical solution for mesh generation. The training takes only 2 days with 4 GPUs on 39k data with a maximum of 4k faces on Objaverse.
• 摘要：本文提出了Iflame，这是一种基于变压器的新型网络架构，用于网格生成。尽管基于注意力的模型在网格生成中表现出了显着的性能，但它们的二次计算复杂性限制了可扩展性，尤其是对于高分辨率3D数据。相反，线性注意机制提供了较低的计算成本，但通常很难捕获长期依赖性，从而导致了次优的结果。为了解决这一权衡，我们提出了一个交织的自回归网格生成框架，将线性注意力的效率与全部注意机制的表现力相结合。为了进一步提高效率并利用网格表示的固有结构，我们将这种交织方法整合到沙漏体系结构中，从而大大提高了效率。我们的方法减少了训练时间，同时实现了与纯粹的基于注意力的模型相当的性能。为了提高推论效率，我们实施了一种缓存算法，该算法几乎使速度翻了一番，并且与原始变压器相比，KV高速缓存的大小降低了八分之七。我们在Shapenet和Objaverse上评估了我们的框架，证明了其有效生成高质量3D网格的能力。我们的结果表明，提议的交织框架有效地平衡了计算效率和生成性能，从而使其成为网格生成的实用解决方案。在39K数据上，训练只需4 GPU，最多4K面孔的训练。

Title: A preliminary data fusion study to assess the feasibility of Foundation Process-Property Models in Laser Powder Bed Fusion

• Authors: Oriol Vendrell-Gallart, Nima Negarandeh, Zahra Zanjani Foumani, Mahsa Amiri, Lorenzo Valdevit, Ramin Bostanabad
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.16667
• Pdf URL: https://arxiv.org/pdf/2503.16667
• Copy Paste: [[2503.16667]] A preliminary data fusion study to assess the feasibility of Foundation Process-Property Models in Laser Powder Bed Fusion(https://arxiv.org/abs/2503.16667)
• Keywords: generation
• Abstract: Foundation models are at the forefront of an increasing number of critical applications. In regards to technologies such as additive manufacturing (AM), these models have the potential to dramatically accelerate process optimization and, in turn, design of next generation materials. A major challenge that impedes the construction of foundation process-property models is data scarcity. To understand the impact of this challenge, and since foundation models rely on data fusion, in this work we conduct controlled experiments where we focus on the transferability of information across different material systems and properties. More specifically, we generate experimental datasets from 17-4 PH and 316L stainless steels (SSs) in Laser Powder Bed Fusion (LPBF) where we measure the effect of five process parameters on porosity and hardness. We then leverage Gaussian processes (GPs) for process-property modeling in various configurations to test if knowledge about one material system or property can be leveraged to build more accurate machine learning models for other material systems or properties. Through extensive cross-validation studies and probing the GPs' interpretable hyperparameters, we study the intricate relation among data size and dimensionality, complexity of the process-property relations, noise, and characteristics of machine learning models. Our findings highlight the need for structured learning approaches that incorporate domain knowledge in building foundation process-property models rather than relying on uninformed data fusion in data-limited applications.
• 摘要：基础模型位于越来越多的关键应用程序的最前沿。关于诸如增材制造（AM）之类的技术，这些模型具有显着加速过程优化的潜力，进而可能是下一代材料的设计。阻碍基础流程 - 专业模型构建的主要挑战是数据稀缺。为了了解这一挑战的影响，并且由于基础模型依赖于数据融合，在这项工作中，我们进行了受控的实验，我们将重点放在信息跨不同材料系统和属性之间的传递性。更具体地说，我们在激光粉末床融合（LPBF）中生成了从17-4 pH和316L不锈钢（SSS）生成实验数据集，其中我们测量了五个过程参数对孔隙率和硬度的影响。然后，我们利用高斯流程（GPS）进行各种配置中的过程 - 培训建模，以测试是否可以利用有关一种材料系统或属性的知识来为其他材料系统或属性构建更准确的机器学习模型。通过广泛的交叉验证研究并探测GPS的可解释的超参数，我们研究了数据大小和维度之间的复杂关系，过程 - 培训关系的复杂性，机器学习模型的特征和特征。我们的发现强调了对结构化学习方法的需求，这些方法将域知识纳入构建基础流程 - 托管模型中，而不是依靠数据限制应用程序中的无信息数据融合。

Title: EDiT: Efficient Diffusion Transformers with Linear Compressed Attention

• Authors: Philipp Becker, Abhinav Mehrotra, Ruchika Chavhan, Malcolm Chadwick, Luca Morreale, Mehdi Noroozi, Alberto Gil Ramos, Sourav Bhattacharya
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.16726
• Pdf URL: https://arxiv.org/pdf/2503.16726
• Copy Paste: [[2503.16726]] EDiT: Efficient Diffusion Transformers with Linear Compressed Attention(https://arxiv.org/abs/2503.16726)
• Keywords: generation
• Abstract: Diffusion Transformers (DiTs) have emerged as a leading architecture for text-to-image synthesis, producing high-quality and photorealistic images. However, the quadratic scaling properties of the attention in DiTs hinder image generation with higher resolution or on devices with limited resources. This work introduces an efficient diffusion transformer (EDiT) to alleviate these efficiency bottlenecks in conventional DiTs and Multimodal DiTs (MM-DiTs). First, we present a novel linear compressed attention method that uses a multi-layer convolutional network to modulate queries with local information while keys and values are spatially aggregated. Second, we formulate a hybrid attention scheme for multi-modal inputs that combines linear attention for image-to-image interactions and standard scaled dot-product attention for interactions involving prompts. Merging these two approaches leads to an expressive, linear-time Multimodal Efficient Diffusion Transformer (MM-EDiT). We demonstrate the effectiveness of the EDiT and MM-EDiT architectures by integrating them into PixArt-Sigma(conventional DiT) and Stable Diffusion 3.5-Medium (MM-DiT), achieving up to 2.2x speedup with comparable image quality after distillation.
• 摘要：扩散变压器（DIT）已成为文本对图像合成的领先体系结构，产生了高质量和逼真的图像。但是，DIT中注意力的二次扩展属性阻碍了具有更高分辨率的图像生成或资源有限的设备上。这项工作引入了有效的扩散变压器（编辑），以减轻常规dit和多模式dit（mm-dits）中的这些效率瓶颈。首先，我们提出了一种新型的线性压缩注意方法，该方法使用多层卷积网络使用局部信息调节查询，而键和值则在空间上汇总。其次，我们为多模式输入制定了混合注意力方案，该方案结合了图像到图像相互作用的线性注意力和涉及提示的相互作用的标准缩放点产生关注。合并这两种方法会导致表达性的线性时间多模式的扩散变压器（MM-edit）。我们通过将它们集成到Pixart-Sigma（常规DIT）和稳定的扩散3.5-米形（MM-DIT）中来证明编辑和MM编辑架构的有效性，在蒸馏后具有可比的图像质量，可达到2.2倍的速度。

Title: Ordered Topological Deep Learning: a Network Modeling Case Study

• Authors: Guillermo Bernárdez, Miquel Ferriol-Galmés, Carlos Güemes-Palau, Mathilde Papillon, Pere Barlet-Ros, Albert Cabellos-Aparicio, Nina Miolane
• Subjects: cs.LG, cs.NI
• Abstract URL: https://arxiv.org/abs/2503.16746
• Pdf URL: https://arxiv.org/pdf/2503.16746
• Copy Paste: [[2503.16746]] Ordered Topological Deep Learning: a Network Modeling Case Study(https://arxiv.org/abs/2503.16746)
• Keywords: generation
• Abstract: Computer networks are the foundation of modern digital infrastructure, facilitating global communication and data exchange. As demand for reliable high-bandwidth connectivity grows, advanced network modeling techniques become increasingly essential to optimize performance and predict network behavior. Traditional modeling methods, such as packet-level simulators and queueing theory, have notable limitations --either being computationally expensive or relying on restrictive assumptions that reduce accuracy. In this context, the deep learning-based RouteNet family of models has recently redefined network modeling by showing an unprecedented cost-performance trade-off. In this work, we revisit RouteNet's sophisticated design and uncover its hidden connection to Topological Deep Learning (TDL), an emerging field that models higher-order interactions beyond standard graph-based methods. We demonstrate that, although originally formulated as a heterogeneous Graph Neural Network, RouteNet serves as the first instantiation of a new form of TDL. More specifically, this paper presents OrdGCCN, a novel TDL framework that introduces the notion of ordered neighbors in arbitrary discrete topological spaces, and shows that RouteNet's architecture can be naturally described as an ordered topological neural network. To the best of our knowledge, this marks the first successful real-world application of state-of-the-art TDL principles --which we confirm through extensive testbed experiments--, laying the foundation for the next generation of ordered TDL-driven applications.
• 摘要：计算机网络是现代数字基础架构的基础，促进了全球通信和数据交换。随着对可靠的高宽带连接性的需求的增长，高级网络建模技术对于优化性能和预测网络行为变得越来越重要。传统的建模方法，例如数据包级模拟器和排队理论，具有明显的局限性 - 在计算上是昂贵的或依靠降低准确性的限制假设。在这种情况下，基于深度学习的Routenet模型家族最近通过显示前所未有的成本绩效权衡来重新定义网络建模。在这项工作中，我们重新访问了Routenet的复杂设计，并发现了其与拓扑深度学习（TDL）的隐藏连接，这是一个新兴领域，对基于标准的基于图形的方法的高阶交互进行了建模。我们证明，尽管最初以异质图神经网络的形式表达，但Routenet是新形式的TDL的首次实例化。更具体地说，本文介绍了OrdgCCN，这是一个新颖的TDL框架，它在任意离散的拓扑空间中介绍了有序的邻居的概念，并表明Routenet的体系结构可以自然地描述为有序的拓扑神经网络。据我们所知，这标志着最先进的TDL原则的第一个成功的现实应用程序 - 我们通过广泛的测试床实验确认这一点，为下一代有序的TDL驱动应用程序奠定了基础。

Title: Safe and Reliable Diffusion Models via Subspace Projection

• Authors: Huiqiang Chen, Tianqing Zhu, Linlin Wang, Xin Yu, Longxiang Gao, Wanlei Zhou
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.16835
• Pdf URL: https://arxiv.org/pdf/2503.16835
• Copy Paste: [[2503.16835]] Safe and Reliable Diffusion Models via Subspace Projection(https://arxiv.org/abs/2503.16835)
• Keywords: generation
• Abstract: Large-scale text-to-image (T2I) diffusion models have revolutionized image generation, enabling the synthesis of highly detailed visuals from textual descriptions. However, these models may inadvertently generate inappropriate content, such as copyrighted works or offensive images. While existing methods attempt to eliminate specific unwanted concepts, they often fail to ensure complete removal, allowing the concept to reappear in subtle forms. For instance, a model may successfully avoid generating images in Van Gogh's style when explicitly prompted with 'Van Gogh', yet still reproduce his signature artwork when given the prompt 'Starry Night'. In this paper, we propose SAFER, a novel and efficient approach for thoroughly removing target concepts from diffusion models. At a high level, SAFER is inspired by the observed low-dimensional structure of the text embedding space. The method first identifies a concept-specific subspace $S_c$ associated with the target concept c. It then projects the prompt embeddings onto the complementary subspace of $S_c$, effectively erasing the concept from the generated images. Since concepts can be abstract and difficult to fully capture using natural language alone, we employ textual inversion to learn an optimized embedding of the target concept from a reference image. This enables more precise subspace estimation and enhances removal performance. Furthermore, we introduce a subspace expansion strategy to ensure comprehensive and robust concept erasure. Extensive experiments demonstrate that SAFER consistently and effectively erases unwanted concepts from diffusion models while preserving generation quality.
• 摘要：大规模的文本对图像（T2I）扩散模型已彻底改变了图像的产生，从而使文本描述中高度详细的视觉效果合成。但是，这些模型可能会无意间产生不适当的内容，例如受版权保护的作品或令人反感的图像。尽管现有的方法试图消除特定的不需要的概念，但它们通常无法确保完全删除，从而使概念重新出现以微妙的形式出现。例如，当明确提示“ van gogh”时，模型可能会成功避免以梵高风格生成图像，但当及时赋予“星空之夜”时，仍然可以重现他的标志性艺术品。在本文中，我们提出了更安全的，这是一种新颖有效的方法，用于从扩散模型中彻底消除目标概念。在高水平上，更安全的灵感来自观察到的文本嵌入空间的低维结构。该方法首先标识了与目标概念相关的特定于概念的子空间$ s_c $。然后，它将提示嵌入在$ s_c $的互补子空间上投射，从而有效地从生成的图像中删除了该概念。由于概念可能是抽象的，并且很难单独使用自然语言完全捕获，因此我们使用文本反演从参考图像中学习对目标概念的优化嵌入。这可以实现更精确的子空间估计并增强删除性能。此外，我们引入了一个子空间扩展策略，以确保全面，健壮的概念擦除。广泛的实验表明，更安全，有效地从扩散模型中删除不需要的概念，同时保留发电质量。

Title: Generative Compositor for Few-Shot Visual Information Extraction

• Authors: Zhibo Yang, Wei Hua, Sibo Song, Cong Yao, Yingying Zhu, Wenqing Cheng, Xiang Bai
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.16854
• Pdf URL: https://arxiv.org/pdf/2503.16854
• Copy Paste: [[2503.16854]] Generative Compositor for Few-Shot Visual Information Extraction(https://arxiv.org/abs/2503.16854)
• Keywords: generative
• Abstract: Visual Information Extraction (VIE), aiming at extracting structured information from visually rich document images, plays a pivotal role in document processing. Considering various layouts, semantic scopes, and languages, VIE encompasses an extensive range of types, potentially numbering in the thousands. However, many of these types suffer from a lack of training data, which poses significant challenges. In this paper, we propose a novel generative model, named Generative Compositor, to address the challenge of few-shot VIE. The Generative Compositor is a hybrid pointer-generator network that emulates the operations of a compositor by retrieving words from the source text and assembling them based on the provided prompts. Furthermore, three pre-training strategies are employed to enhance the model's perception of spatial context information. Besides, a prompt-aware resampler is specially designed to enable efficient matching by leveraging the entity-semantic prior contained in prompts. The introduction of the prompt-based retrieval mechanism and the pre-training strategies enable the model to acquire more effective spatial and semantic clues with limited training samples. Experiments demonstrate that the proposed method achieves highly competitive results in the full-sample training, while notably outperforms the baseline in the 1-shot, 5-shot, and 10-shot settings.
• 摘要：旨在从视觉上丰富的文档图像中提取结构化信息的视觉信息提取（VIE）在文档处理中起关键作用。考虑到各种布局，语义范围和语言，VIE涵盖了广泛的类型，有可能在数千个中进行编号。但是，其中许多类型都缺乏培训数据，这构成了重大挑战。在本文中，我们提出了一个新颖的生成模型，称为“生成合成器”，以应对少量竞争的挑战。 Generbrid Pointer-Iner-Generator网络是生成合并器，它通过从源文本中检索单词并根据提供的提示来模仿合成器的操作。此外，采用了三种预训练策略来增强模型对空间上下文信息的看法。此外，通过利用提示中包含的实体语义提前，及时意识到的重新采样器是专门设计的，可实现有效的匹配。引入基于及时的检索机制和预训练策略，使该模型能够通过有限的培训样本获得更有效的空间和语义线索。实验表明，所提出的方法在全样本训练中实现了高度竞争的结果，而在1次，5次和10次设置中的基线尤其优于基线。

Title: Nonparametric Factor Analysis and Beyond

• Authors: Yujia Zheng, Yang Liu, Jiaxiong Yao, Yingyao Hu, Kun Zhang
• Subjects: cs.LG, math.ST, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.16865
• Pdf URL: https://arxiv.org/pdf/2503.16865
• Copy Paste: [[2503.16865]] Nonparametric Factor Analysis and Beyond(https://arxiv.org/abs/2503.16865)
• Keywords: generative
• Abstract: Nearly all identifiability results in unsupervised representation learning inspired by, e.g., independent component analysis, factor analysis, and causal representation learning, rely on assumptions of additive independent noise or noiseless regimes. In contrast, we study the more general case where noise can take arbitrary forms, depend on latent variables, and be non-invertibly entangled within a nonlinear function. We propose a general framework for identifying latent variables in the nonparametric noisy settings. We first show that, under suitable conditions, the generative model is identifiable up to certain submanifold indeterminacies even in the presence of non-negligible noise. Furthermore, under the structural or distributional variability conditions, we prove that latent variables of the general nonlinear models are identifiable up to trivial indeterminacies. Based on the proposed theoretical framework, we have also developed corresponding estimation methods and validated them in various synthetic and real-world settings. Interestingly, our estimate of the true GDP growth from alternative measurements suggests more insightful information on the economies than official reports. We expect our framework to provide new insight into how both researchers and practitioners deal with latent variables in real-world scenarios.
• 摘要：几乎所有可识别性都会导致无监督的表示学习启发，例如独立组件分析，因子分析和因果表示学习，依赖于加性独立噪声或无噪声制度的假设。相比之下，我们研究了更普遍的情况，即噪声可以采用任意形式，取决于潜在变量，并且在非线性函数中不可拒绝。我们提出了一个通用框架，用于识别非参数噪声设置中的潜在变量。我们首先表明，在适当的条件下，即使在存在不可忽略的噪声的情况下，生成模型也可以识别为某些亚曼叶不确定性。此外，在结构或分布变异条件下，我们证明一般非线性模型的潜在变量可识别为琐碎的不确定。基于提出的理论框架，我们还开发了相应的估计方法，并在各种合成和现实世界中验证了它们。有趣的是，我们对替代测量的真正GDP增长的估计表明，关于经济的信息比官方报告更有见识。我们希望我们的框架可以为研究人员和从业人员如何处理现实世界中的潜在变量提供新的见解。

Title: ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question Generation and Answering

• Authors: Kaisi Guan, Zhengfeng Lai, Yuchong Sun, Peng Zhang, Wei Liu, Kieran Liu, Meng Cao, Ruihua Song
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.16867
• Pdf URL: https://arxiv.org/pdf/2503.16867
• Copy Paste: [[2503.16867]] ETVA: Evaluation of Text-to-Video Alignment via Fine-grained Question Generation and Answering(https://arxiv.org/abs/2503.16867)
• Keywords: generation
• Abstract: Precisely evaluating semantic alignment between text prompts and generated videos remains a challenge in Text-to-Video (T2V) Generation. Existing text-to-video alignment metrics like CLIPScore only generate coarse-grained scores without fine-grained alignment details, failing to align with human preference. To address this limitation, we propose ETVA, a novel Evaluation method of Text-to-Video Alignment via fine-grained question generation and answering. First, a multi-agent system parses prompts into semantic scene graphs to generate atomic questions. Then we design a knowledge-augmented multi-stage reasoning framework for question answering, where an auxiliary LLM first retrieves relevant common-sense knowledge (e.g., physical laws), and then video LLM answers the generated questions through a multi-stage reasoning mechanism. Extensive experiments demonstrate that ETVA achieves a Spearman's correlation coefficient of 58.47, showing a much higher correlation with human judgment than existing metrics which attain only 31.0. We also construct a comprehensive benchmark specifically designed for text-to-video alignment evaluation, featuring 2k diverse prompts and 12k atomic questions spanning 10 categories. Through a systematic evaluation of 15 existing text-to-video models, we identify their key capabilities and limitations, paving the way for next-generation T2V generation.
• 摘要：精确评估文本提示和生成视频之间的语义一致性仍然是文本到视频（T2V）一代的挑战。现有的文本到视频对齐指标（例如夹克）仅产生粗粒的分数，而无需细粒度的细节细节，无法与人类偏好保持一致。为了解决这一限制，我们提出了ETVA，这是一种通过细粒度的问题产生和回答的新颖的文本到视频对齐方式的评估方法。首先，一个多代理系统解析提示进入语义场景图以生成原子问题。然后，我们为问题回答设计了一个知识增强的多阶段推理框架，其中辅助LLM首先检索相关的常识知识（例如物理定律），然后Video LLM通过多阶段的推理机制回答了生成的问题。广泛的实验表明，ETVA达到了Spearman的相关系数为58.47，与仅达到31.0的现有指标相比，与人类判断的相关性更高。我们还构建了一个专门为文本到视频对齐评估而设计的综合基准，其中包含2K多样化的提示和12K原子问题，涉及10个类别。通过对15个现有文本对视频模型的系统评估，我们确定了它们的关键功能和局限性，为下一代T2V生成铺平了道路。

Title: Malliavin-Bismut Score-based Diffusion Models

• Authors: Ehsan Mirafzali, Utkarsh Gupta, Patrick Wyrod, Frank Proske, Daniele Venturi, Razvan Marinescu
• Subjects: cs.LG, math.PR
• Abstract URL: https://arxiv.org/abs/2503.16917
• Pdf URL: https://arxiv.org/pdf/2503.16917
• Copy Paste: [[2503.16917]] Malliavin-Bismut Score-based Diffusion Models(https://arxiv.org/abs/2503.16917)
• Keywords: generative
• Abstract: We introduce a new framework that employs Malliavin calculus to derive explicit expressions for the score function -- i.e., the gradient of the log-density -- associated with solutions to stochastic differential equations (SDEs). Our approach integrates classical integration-by-parts techniques with modern tools, such as Bismut's formula and Malliavin calculus, to address linear and nonlinear SDEs. In doing so, we establish a rigorous connection between the Malliavin derivative, its adjoint (the Malliavin divergence or the Skorokhod integral), Bismut's formula, and diffusion generative models, thus providing a systematic method for computing $\nabla \log p_t(x)$. For the linear case, we present a detailed study proving that our formula is equivalent to the actual score function derived from the solution of the Fokker--Planck equation for linear SDEs. Additionally, we derive a closed-form expression for $\nabla \log p_t(x)$ for nonlinear SDEs with state-independent diffusion coefficients. These advancements provide fresh theoretical insights into the smoothness and structure of probability densities and practical implications for score-based generative modelling, including the design and analysis of new diffusion models. Moreover, our findings promote the adoption of the robust Malliavin calculus framework in machine learning research. These results directly apply to various pure and applied mathematics fields, such as generative modelling, the study of SDEs driven by fractional Brownian motion, and the Fokker--Planck equations associated with nonlinear SDEs.
• 摘要：我们引入了一个新的框架，该框架采用Malliavin演算来得出与随机微分方程（SDES）解决方案相关的分数函数的明确表达式（即对数密度的梯度）。我们的方法将逐一的经典集成与现代工具（例如Bismut的公式和Malliavin Colculus）相结合，以解决线性和非线性SDE。为此，我们在Malliavin衍生物（Malliavin Divergence或Skorokhod积分），Bismut的公式和扩散生成模型之间建立了严格的联系，从而提供了一种用于计算$ \ nabla \ nabla \ log log log p_t p_t（x）$的系统方法。对于线性案例，我们提供了一项详细的研究，证明我们的公式等同于源自fokker的解决方案的实际得分函数 - 线性SDE的Planck方程。此外，我们为具有与状态无关的扩散系数的非线性SDE的$ \ nabla \ log p_t（x）$得出了封闭形式的表达式。这些进步提供了对概率密度的平稳性和结构的全新理论见解，以及对基于得分的生成建模的实践含义，包括新扩散模型的设计和分析。此外，我们的发现促进了机器学习研究中强大的Malliavin微积分框架的采用。这些结果直接适用于各种纯和应用数学领域，例如生成建模，分数布朗运动驱动的SDE和与非线性SDE相关的planck方程。

Title: When Preferences Diverge: Aligning Diffusion Models with Minority-Aware Adaptive DPO

• Authors: Lingfan Zhang, Chen Liu, Chengming Xu, Kai Hu, Donghao Luo, Chengjie Wang, Yanwei Fu, Yuan Yao
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.16921
• Pdf URL: https://arxiv.org/pdf/2503.16921
• Copy Paste: [[2503.16921]] When Preferences Diverge: Aligning Diffusion Models with Minority-Aware Adaptive DPO(https://arxiv.org/abs/2503.16921)
• Keywords: generation
• Abstract: In recent years, the field of image generation has witnessed significant advancements, particularly in fine-tuning methods that align models with universal human preferences. This paper explores the critical role of preference data in the training process of diffusion models, particularly in the context of Diffusion-DPO and its subsequent adaptations. We investigate the complexities surrounding universal human preferences in image generation, highlighting the subjective nature of these preferences and the challenges posed by minority samples in preference datasets. Through pilot experiments, we demonstrate the existence of minority samples and their detrimental effects on model performance. We propose Adaptive-DPO -- a novel approach that incorporates a minority-instance-aware metric into the DPO objective. This metric, which includes intra-annotator confidence and inter-annotator stability, distinguishes between majority and minority samples. We introduce an Adaptive-DPO loss function which improves the DPO loss in two ways: enhancing the model's learning of majority labels while mitigating the negative impact of minority samples. Our experiments demonstrate that this method effectively handles both synthetic minority data and real-world preference data, paving the way for more effective training methodologies in image generation tasks.
• 摘要：近年来，图像生成领域已经取得了重大进步，尤其是在与人类普遍偏好相结合的微调方法中。本文探讨了偏好数据在扩散模型的训练过程中的关键作用，尤其是在扩散-DPO及其随后的适应的背景下。我们研究了图像产生中普遍人类偏好的复杂性，突出了这些偏好的主观性质以及偏好数据集中少数族裔样本所带来的挑战。通过试点实验，我们证明了少数样本的存在及其对模型性能的有害影响。我们提出了Adaptive-DPO-一种新颖的方法，将少数族裔意识到的指标纳入DPO目标。该指标包括通道内的置信度和通道间稳定性，区分了多数样本和少数样本。我们引入了一种自适应-DPO损失函数，该功能通过两种方式改善了DPO损失：增强模型对多数标签的学习，同时减轻少数样本的负面影响。我们的实验表明，该方法有效地处理合成少数族裔数据和现实世界偏好数据，为图像生成任务中更有效的培训方法铺平了道路。

Title: MerGen: Micro-electrode recording synthesis using a generative data-driven approach

• Authors: Thibault Martin, Paul Sauleau, Claire Haegelen, Pierre Jannin, John S. H. Baxter
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.16928
• Pdf URL: https://arxiv.org/pdf/2503.16928
• Copy Paste: [[2503.16928]] MerGen: Micro-electrode recording synthesis using a generative data-driven approach(https://arxiv.org/abs/2503.16928)
• Keywords: generation, generative
• Abstract: The analysis of electrophysiological data is crucial for certain surgical procedures such as deep brain stimulation, which has been adopted for the treatment of a variety of neurological disorders. During the procedure, auditory analysis of these signals helps the clinical team to infer the neuroanatomical location of the stimulation electrode and thus optimize clinical outcomes. This task is complex, and requires an expert who in turn requires significant training. In this paper, we propose a generative neural network, called MerGen, capable of simulating de novo electrophysiological recordings, with a view to providing a realistic learning tool for clinicians trainees for identifying these signals. We demonstrate that the generated signals are perceptually indistinguishable from real signals by experts in the field, and that it is even possible to condition the generation efficiently to provide a didactic simulator adapted to a particular surgical scenario. The efficacy of this conditioning is demonstrated, comparing it to intra-observer and inter-observer variability amongst experts. We also demonstrate the use of this network for data augmentation for automatic signal classification which can play a role in decision-making support in the operating theatre.
• 摘要：电生理数据的分析对于某些外科手术（例如深脑刺激）至关重要，这些手术已被用于治疗多种神经系统疾病。在过程中，对这些信号的听觉分析有助于临床团队推断刺激电极的神经解剖位置，从而优化临床结果。这项任务很复杂，需要专家又需要进行大量培训。在本文中，我们提出了一个称为Mergen的生成神经网络，能够模拟从头电生理记录，以期为临床医生受训者提供一个现实的学习工具，以识别这些信号。我们证明，生成的信号在感知上与该领域的专家无法区分，甚至有可能有效地调节发电，以提供适合特定外科手术场景的教学模拟器。证明了这种调理的功效，将其与专家之间的观察者内部和观察者间变异性进行了比较。我们还证明了该网络用于数据扩展进行自动信号分类，这可以在手术室的决策支持中发挥作用。

Title: TEMPO: Temporal Preference Optimization of Video LLMs via Difficulty Scheduling and Pre-SFT Alignment

• Authors: Shicheng Li, Lei Li, Kun Ouyang, Shuhuai Ren, Yuanxin Liu, Yuanxing Zhang, Fuzheng Zhang, Lingpeng Kong, Qi Liu, Xu Sun
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.16929
• Pdf URL: https://arxiv.org/pdf/2503.16929
• Copy Paste: [[2503.16929]] TEMPO: Temporal Preference Optimization of Video LLMs via Difficulty Scheduling and Pre-SFT Alignment(https://arxiv.org/abs/2503.16929)
• Keywords: generation
• Abstract: Video Large Language Models (Video LLMs) have achieved significant success by leveraging a two-stage paradigm: pretraining on large-scale video-text data for vision-language alignment, followed by supervised fine-tuning (SFT) for task-specific capabilities. However, existing approaches struggle with temporal reasoning due to weak temporal correspondence in the data and reliance on the next-token prediction paradigm during training. To address these limitations, we propose TEMPO (TEMporal Preference Optimization), a systematic framework that enhances Video LLMs' temporal reasoning capabilities through Direct Preference Optimization (DPO). To facilitate this, we introduce an automated preference data generation pipeline that systematically constructs preference pairs by selecting videos that are rich in temporal information, designing video-specific perturbation strategies, and finally evaluating model responses on clean and perturbed video inputs. Our temporal alignment features two key innovations: curriculum learning which that progressively increases perturbation difficulty to improve model robustness and adaptability; and ``Pre-SFT Alignment'', applying preference optimization before instruction tuning to prioritize fine-grained temporal comprehension. Extensive experiments demonstrate that our approach consistently improves Video LLM performance across multiple benchmarks with a relatively small set of self-generated DPO data. We further analyze the transferability of DPO data across architectures and the role of difficulty scheduling in optimization. Our findings highlight our TEMPO as a scalable and efficient complement to SFT-based methods, paving the way for developing reliable Video LLMs.
• 摘要：视频大型语言模型（视频LLM）通过利用两阶段范式来取得了重大成功：在大规模视频文本数据上进行审计，以进行视觉语言对齐，然后进行监督的微调（SFT），以获得特定于任务的功能。但是，由于数据中的时间对应关系弱，并且在培训期间对下一步的预测范式的依赖，现有方法与时间推理困难。为了解决这些限制，我们提出了节奏（时间偏好优化），该框架是一个系统的框架，可通过直接偏好优化（DPO）增强Video LLMS的时间推理功能。为了促进这一点，我们引入了一个自动偏好数据生成管道，该管道通过选择丰富的时间信息，设计特定于视频的扰动策略，并最终评估对清洁和扰动视频输入的模型响应来系统地构建偏好对。我们的时间一致性具有两个关键的创新：课程学习，从而逐渐增加了扰动难以提高模型的鲁棒性和适应性；和````sft prep ailignment'''，在说明调整之前应用偏好优化，以优先考虑细粒度的时间理解。广泛的实验表明，我们的方法始终通过相对较小的自我生成的DPO数据来提高多个基准的视频LLM性能。我们进一步分析了跨体系结构的DPO数据的可传递性以及调度难度在优化中的作用。我们的发现突出了我们的节奏是对基于SFT的方法的可扩展有效补充，为开发可靠的视频LLM铺平了道路。

Title: Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks

• Authors: Haijin Zeng, Xiangming Wang, Yongyong Chen, Jingyong Su, Jie Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.16930
• Pdf URL: https://arxiv.org/pdf/2503.16930
• Copy Paste: [[2503.16930]] Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks(https://arxiv.org/abs/2503.16930)
• Keywords: restoration
• Abstract: Dynamic image degradations, including noise, blur and lighting inconsistencies, pose significant challenges in image restoration, often due to sensor limitations or adverse environmental conditions. Existing Deep Unfolding Networks (DUNs) offer stable restoration performance but require manual selection of degradation matrices for each degradation type, limiting their adaptability across diverse scenarios. To address this issue, we propose the Vision-Language-guided Unfolding Network (VLU-Net), a unified DUN framework for handling multiple degradation types simultaneously. VLU-Net leverages a Vision-Language Model (VLM) refined on degraded image-text pairs to align image features with degradation descriptions, selecting the appropriate transform for target degradation. By integrating an automatic VLM-based gradient estimation strategy into the Proximal Gradient Descent (PGD) algorithm, VLU-Net effectively tackles complex multi-degradation restoration tasks while maintaining interpretability. Furthermore, we design a hierarchical feature unfolding structure to enhance VLU-Net framework, efficiently synthesizing degradation patterns across various levels. VLU-Net is the first all-in-one DUN framework and outperforms current leading one-by-one and all-in-one end-to-end methods by 3.74 dB on the SOTS dehazing dataset and 1.70 dB on the Rain100L deraining dataset.
• 摘要：动态图像降解，包括噪声，模糊和照明不一致，在图像恢复方面构成了重大挑战，通常是由于传感器局限性或不利的环境条件所致。现有的深层展开网络（DUNS）提供稳定的恢复性能，但需要为每种降解类型手动选择降解矩阵，从而限制了它们在各种情况下的适应性。为了解决这个问题，我们提出了视觉引导的展开网络（VLU-NET），这是一个统一的DUN DUN框架，用于同时处理多种退化类型。 VLU-NET利用在退化的图像文本对上进行了视觉模型（VLM），以使图像特征与降解描述相结合，从而为目标降解选择适当的转换。通过将基于VLM的自动梯度估计策略集成到近端梯度下降（PGD）算法中，VLU-NET有效地解决了复杂的多降解恢复任务，同时保持可解释性。此外，我们设计了一个层次特征展开结构，以增强VLU-NET框架，有效地综合了各个级别的降解模式。 VLU-NET是第一个多合一的DUN框架，在SOTS Dehazing DataTet上，均优于当前领先的一对端到端方法，而在SOTS上，降低了3.74 dB，而Rain 100L Deraining数据集则均优于1.70 dB。

Title: Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model

• Authors: Yingying Fan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou, Yingying Li, Haocheng Feng, Yu Wu, Jingdong Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.16942
• Pdf URL: https://arxiv.org/pdf/2503.16942
• Copy Paste: [[2503.16942]] Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model(https://arxiv.org/abs/2503.16942)
• Keywords: generation
• Abstract: Current digital human studies focusing on lip-syncing and body movement are no longer sufficient to meet the growing industrial demand, while human video generation techniques that support interacting with real-world environments (e.g., objects) have not been well investigated. Despite human hand synthesis already being an intricate problem, generating objects in contact with hands and their interactions presents an even more challenging task, especially when the objects exhibit obvious variations in size and shape. To cope with these issues, we present a novel video Reenactment framework focusing on Human-Object Interaction (HOI) via an adaptive Layout-instructed Diffusion model (Re-HOLD). Our key insight is to employ specialized layout representation for hands and objects, respectively. Such representations enable effective disentanglement of hand modeling and object adaptation to diverse motion sequences. To further improve the generation quality of HOI, we have designed an interactive textural enhancement module for both hands and objects by introducing two independent memory banks. We also propose a layout-adjusting strategy for the cross-object reenactment scenario to adaptively adjust unreasonable layouts caused by diverse object sizes during inference. Comprehensive qualitative and quantitative evaluations demonstrate that our proposed framework significantly outperforms existing methods. Project page: this https URL.
• 摘要：当前关注唇部同步和身体运动的数字人类研究不足以满足不断增长的工业需求，而支持与现实世界环境（例如，对象）相互作用的人类视频生成技术并没有得到很好的研究。尽管人类的综合已经是一个复杂的问题，但生成与手接触的对象及其相互作用提出了更具挑战性的任务，尤其是当对象在大小和形状上表现出明显的变化时。为了应对这些问题，我们提出了一个新颖的视频重演框架，该框架通过自适应布局实验性扩散模型（重新构建），重点介绍了人类对象相互作用（HOI）。我们的主要见解是分别为手和物体采用专门的布局表示。这样的表示可以有效地解开手动建模和对象适应各种运动序列。为了进一步提高HOI的发电质量，我们通过引入两个独立的记忆库为手和对象设计了一个交互式纹理增强模块。我们还为跨对象重新制作场景提出了一种布局调整策略，以适应推断期间由不同的物体大小引起的不合理布局。全面的定性和定量评估表明，我们提出的框架显着优于现有方法。项目页面：此HTTPS URL。

Title: HyperLoRA: Parameter-Efficient Adaptive Generation for Portrait Synthesis

• Authors: Mengtian Li, Jinshu Chen, Wanquan Feng, Bingchuan Li, Fei Dai, Songtao Zhao, Qian He
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.16944
• Pdf URL: https://arxiv.org/pdf/2503.16944
• Copy Paste: [[2503.16944]] HyperLoRA: Parameter-Efficient Adaptive Generation for Portrait Synthesis(https://arxiv.org/abs/2503.16944)
• Keywords: generation
• Abstract: Personalized portrait synthesis, essential in domains like social entertainment, has recently made significant progress. Person-wise fine-tuning based methods, such as LoRA and DreamBooth, can produce photorealistic outputs but need training on individual samples, consuming time and resources and posing an unstable risk. Adapter based techniques such as IP-Adapter freeze the foundational model parameters and employ a plug-in architecture to enable zero-shot inference, but they often exhibit a lack of naturalness and authenticity, which are not to be overlooked in portrait synthesis tasks. In this paper, we introduce a parameter-efficient adaptive generation method, namely HyperLoRA, that uses an adaptive plug-in network to generate LoRA weights, merging the superior performance of LoRA with the zero-shot capability of adapter scheme. Through our carefully designed network structure and training strategy, we achieve zero-shot personalized portrait generation (supporting both single and multiple image inputs) with high photorealism, fidelity, and editability.
• 摘要：个性化的肖像综合，在社交娱乐等领域中必不可少，最近取得了重大进展。基于人的微调方法，例如Lora和Dreambooth，可以产生逼真的产出，但需要对单个样本，消耗时间和资源的培训，并带来不稳定的风险。基于适配器的技术（例如IP-ADAPTER）冻结了基础模型参数并采用插件体系结构来启用零射击推断，但它们通常表现出缺乏自然性和真实性，而在肖像合成任务中并不忽略这一点。在本文中，我们引入了一种使用自适应插件网络生成洛拉权重的参数效率自适应生成方法，即Hyperlora，将Lora的出色性能与适配器方案的零发能力合并。通过我们精心设计的网络结构和培训策略，我们实现了具有高光真实，忠诚度和编辑性的零拍摄的个性化肖像生成（支持单图像和多个图像输入）。

Title: MagicColor: Multi-Instance Sketch Colorization

• Authors: Yinhan Zhang, Yue Ma, Bingyuan Wang, Qifeng Chen, Zeyu Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.16948
• Pdf URL: https://arxiv.org/pdf/2503.16948
• Copy Paste: [[2503.16948]] MagicColor: Multi-Instance Sketch Colorization(https://arxiv.org/abs/2503.16948)
• Keywords: generative
• Abstract: We present \textit{MagicColor}, a diffusion-based framework for multi-instance sketch colorization. The production of multi-instance 2D line art colorization adheres to an industry-standard workflow, which consists of three crucial stages: the design of line art characters, the coloring of individual objects, and the refinement process. The artists are required to repeat the process of coloring each instance one by one, which is inaccurate and inefficient. Meanwhile, current generative methods fail to solve this task due to the challenge of multi-instance pair data collection. To tackle these challenges, we incorporate three technical designs to ensure precise character detail transcription and achieve multi-instance sketch colorization in a single forward. Specifically, we first propose the self-play training strategy to solve the lack of training data. Then we introduce an instance guider to feed the color of the instance. To achieve accurate color matching, we present fine-grained color matching with edge loss to enhance visual quality. Equipped with the proposed modules, MagicColor enables automatically transforming sketches into vividly-colored images with accurate consistency and multi-instance control. Experiments on our collected datasets show that our model outperforms existing methods regarding chromatic precision. Specifically, our model critically automates the colorization process with zero manual adjustments, so novice users can produce stylistically consistent artwork by providing reference instances and the original line art. Our code and additional details are available at this https URL
• 摘要：我们提出\ textit {MagicColor}，这是一个基于扩散的框架，用于多个实体草图着色。多企业2D系列艺术色彩的生产遵循行业标准的工作流程，该工作流程包括三个关键阶段：线条艺术角色的设计，各个对象的着色和改进过程。需要艺术家重复对每个实例逐一着色的过程，这是不准确且效率低下的。同时，由于多实施对数据收集的挑战，当前的生成方法无法解决此任务。为了应对这些挑战，我们结合了三种技术设计，以确保精确的字符细节转录并在单个前锋中实现多实体草图着色。具体而言，我们首先提出了自我播放培训策略，以解决缺乏培训数据。然后，我们介绍了一个实例指南来馈送实例的颜色。为了实现准确的颜色匹配，我们提出了细粒度匹配，并具有边缘损失，以提高视觉质量。 MagicColor配备了提出的模块，可以自动将草图转换为具有准确的一致性和多构度控制的生动彩色图像。我们收集的数据集中的实验表明，我们的模型优于有关色精度的现有方法。具体而言，我们的模型通过零手动调整非常重要地自动化着色过程，因此新手用户可以通过提供参考实例和原始线条艺术来制作风格上一致的艺术品。我们的代码和其他详细信息可在此HTTPS URL上找到

Title: Enabling Versatile Controls for Video Diffusion Models

• Authors: Xu Zhang, Hao Zhou, Haoming Qin, Xiaobin Lu, Jiaxing Yan, Guanzhong Wang, Zeyu Chen, Yi Liu
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.16983
• Pdf URL: https://arxiv.org/pdf/2503.16983
• Copy Paste: [[2503.16983]] Enabling Versatile Controls for Video Diffusion Models(https://arxiv.org/abs/2503.16983)
• Keywords: generation
• Abstract: Despite substantial progress in text-to-video generation, achieving precise and flexible control over fine-grained spatiotemporal attributes remains a significant unresolved challenge in video generation research. To address these limitations, we introduce VCtrl (also termed PP-VCtrl), a novel framework designed to enable fine-grained control over pre-trained video diffusion models in a unified manner. VCtrl integrates diverse user-specified control signals-such as Canny edges, segmentation masks, and human keypoints-into pretrained video diffusion models via a generalizable conditional module capable of uniformly encoding multiple types of auxiliary signals without modifying the underlying generator. Additionally, we design a unified control signal encoding pipeline and a sparse residual connection mechanism to efficiently incorporate control representations. Comprehensive experiments and human evaluations demonstrate that VCtrl effectively enhances controllability and generation quality. The source code and pre-trained models are publicly available and implemented using the PaddlePaddle framework at this http URL.
• 摘要：尽管文本到视频的生成方面取得了长足的进步，但在视频生成研究中，对精确而灵活的控制进行了精确而灵活的控制仍然是一项重大尚未解决的挑战。为了解决这些局限性，我们引入了VCTRL（也称为PP-VCTRL），这是一个新颖的框架，旨在以统一的方式对预训练的预训练视频扩散模型进行细粒度控制。 VCTRL通过一个可均匀编码多种类型的辅助信号，而无需修改底层生成器而均匀地编码多种类型的辅助信号，从而集成了各种用户指定的控制信号，例如诸如chry边缘，分割掩码和人类关键的视频扩散模型。此外，我们设计了一个统一的控制信号编码管道和稀疏的残留连接机制，以有效地包含控制表示。全面的实验和人类评估表明，VCTRL有效提高了可控性和发电质量。源代码和预培训模型可公开使用，并使用此HTTP URL上的PaddlePaddle框架实现。

Title: AnimatePainter: A Self-Supervised Rendering Framework for Reconstructing Painting Process

• Authors: Junjie Hu, Shuyong Gao, Qianyu Guo, Yan Wang, Qishan Wang, Yuang Feng, Wenqiang Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17029
• Pdf URL: https://arxiv.org/pdf/2503.17029
• Copy Paste: [[2503.17029]] AnimatePainter: A Self-Supervised Rendering Framework for Reconstructing Painting Process(https://arxiv.org/abs/2503.17029)
• Keywords: generation
• Abstract: Humans can intuitively decompose an image into a sequence of strokes to create a painting, yet existing methods for generating drawing processes are limited to specific data types and often rely on expensive human-annotated datasets. We propose a novel self-supervised framework for generating drawing processes from any type of image, treating the task as a video generation problem. Our approach reverses the drawing process by progressively removing strokes from a reference image, simulating a human-like creation sequence. Crucially, our method does not require costly datasets of real human drawing processes; instead, we leverage depth estimation and stroke rendering to construct a self-supervised dataset. We model human drawings as "refinement" and "layering" processes and introduce depth fusion layers to enable video generation models to learn and replicate human drawing behavior. Extensive experiments validate the effectiveness of our approach, demonstrating its ability to generate realistic drawings without the need for real drawing process data.
• 摘要：人类可以直观地将图像分解为一系列笔触以创建绘画，但是现有用于生成图纸过程的方法仅限于特定的数据类型，并且通常依赖于昂贵的人类通知数据集。我们提出了一个新颖的自我监督框架，用于从任何类型的图像中生成绘图过程，将任务视为视频生成问题。我们的方法通过从参考图像中逐步删除笔触，模拟类似人类的创造序列来逆转绘图过程。至关重要的是，我们的方法不需要昂贵的人类绘画过程的昂贵数据集；取而代之的是，我们利用深度估计和中风渲染来构建一个自我监督的数据集。我们将人类图纸建模为“改进”和“分层”过程，并引入深度融合层，以使视频生成模型学习和复制人类绘画行为。广泛的实验验证了我们方法的有效性，证明了它在不需要实际绘图过程数据的情况下生成逼真的图纸的能力。

Title: Unitless Unrestricted Markov-Consistent SCM Generation: Better Benchmark Datasets for Causal Discovery

• Authors: Rebecca J. Herman, Jonas Wahl, Urmi Ninad, Jakob Runge
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.17037
• Pdf URL: https://arxiv.org/pdf/2503.17037
• Copy Paste: [[2503.17037]] Unitless Unrestricted Markov-Consistent SCM Generation: Better Benchmark Datasets for Causal Discovery(https://arxiv.org/abs/2503.17037)
• Keywords: generation
• Abstract: Causal discovery aims to extract qualitative causal knowledge in the form of causal graphs from data. Because causal ground truth is rarely known in the real world, simulated data plays a vital role in evaluating the performance of the various causal discovery algorithms proposed in the literature. But recent work highlighted certain artifacts of commonly used data generation techniques for a standard class of structural causal models (SCM) that may be nonphysical, including var- and R2-sortability, where the variables' variance and coefficients of determination (R2) after regressing on all other variables, respectively, increase along the causal order. Some causal methods exploit such artifacts, leading to unrealistic expectations for their performance on real-world data. Some modifications have been proposed to remove these artifacts; notably, the internally-standardized structural causal model (iSCM) avoids varsortability and largely alleviates R2-sortability on sparse causal graphs, but exhibits a reversed R2-sortability pattern for denser graphs not featured in their work. We analyze which sortability patterns we expect to see in real data, and propose a method for drawing coefficients that we argue more effectively samples the space of SCMs. Finally, we propose a novel extension of our SCM generation method to the time series setting.
• 摘要：因果发现旨在从数据中以因果图的形式提取定性因果知识。由于因果关系真理在现实世界中很少知道，因此模拟数据在评估文献中提出的各种因果发现算法的性能中起着至关重要的作用。但是最近的工作突出了某些常用数据生成技术的某些伪像，用于标准的结构性因果模型（SCM），可能是非物理的，包括VAR-和R2可蚀性，其中变量的方差和确定的方差（R2）分别沿着所有变量物进行了重新分配后，沿着Causal阶数增加。一些因果方法利用了此类人工制品，从而导致对它们在现实数据上的表现的不切实际期望。已经提出了一些修改来去除这些文物。值得注意的是，内部标准化的结构因果模型（ISCM）避免了可变性，并且在很大程度上可以减轻稀疏因果图上的R2可态性，但在其工作中表现出了未在其工作中未显示的密集图的反向R2可蚀性模式。我们分析了我们期望在实际数据中看到的可分解性模式，并提出了一种绘制系数的方法，我们认为我们更有效地采样了SCM的空间。最后，我们将SCM生成方法的新型扩展到时间序列设置。

Title: Zero-Shot Styled Text Image Generation, but Make It Autoregressive

• Authors: Vittorio Pippi, Fabio Quattrini, Silvia Cascianelli, Alessio Tonioni, Rita Cucchiara
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17074
• Pdf URL: https://arxiv.org/pdf/2503.17074
• Copy Paste: [[2503.17074]] Zero-Shot Styled Text Image Generation, but Make It Autoregressive(https://arxiv.org/abs/2503.17074)
• Keywords: generation
• Abstract: Styled Handwritten Text Generation (HTG) has recently received attention from the computer vision and document analysis communities, which have developed several solutions, either GAN- or diffusion-based, that achieved promising results. Nonetheless, these strategies fail to generalize to novel styles and have technical constraints, particularly in terms of maximum output length and training efficiency. To overcome these limitations, in this work, we propose a novel framework for text image generation, dubbed Emuru. Our approach leverages a powerful text image representation model (a variational autoencoder) combined with an autoregressive Transformer. Our approach enables the generation of styled text images conditioned on textual content and style examples, such as specific fonts or handwriting styles. We train our model solely on a diverse, synthetic dataset of English text rendered in over 100,000 typewritten and calligraphy fonts, which gives it the capability to reproduce unseen styles (both fonts and users' handwriting) in zero-shot. To the best of our knowledge, Emuru is the first autoregressive model for HTG, and the first designed specifically for generalization to novel styles. Moreover, our model generates images without background artifacts, which are easier to use for downstream applications. Extensive evaluation on both typewritten and handwritten, any-length text image generation scenarios demonstrates the effectiveness of our approach.
• 摘要：风格的手写文本生成（HTG）最近从计算机视觉和文档分析社区中受到了关注，这些社区开发了几种基于gan-或扩散的解决方案，这些解决方案取得了令人鼓舞的结果。但是，这些策略未能推广到新型样式并具有技术限制，尤其是在最大产出长度和训练效率方面。为了克服这些局限性，在这项工作中，我们提出了一个新颖的文本图像生成框架，称为Emuru。我们的方法利用强大的文本图像表示模型（一种变异自动编码器）与自回归变压器相结合。我们的方法使生成样式的文本图像以文本内容和样式示例（例如特定字体或手写样式）为条件。我们仅根据超过100,000个打字字体和书法字体呈现出多样化的英语文本数据集训练模型，这使其能够以零发出的方式复制看不见的样式（包括字体和用户手写）。据我们所知，Emuru是HTG的第一个自回归模型，也是专门为新型风格概括而设计的第一个模型。此外，我们的模型生成没有背景工件的图像，该图像易于用于下游应用程序。对打字和手写的任何长篇文本图像生成方案的广泛评估都证明了我们方法的有效性。

Title: Halton Scheduler For Masked Generative Image Transformer

• Authors: Victor Besnier, Mickael Chen, David Hurych, Eduardo Valle, Matthieu Cord
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17076
• Pdf URL: https://arxiv.org/pdf/2503.17076
• Copy Paste: [[2503.17076]] Halton Scheduler For Masked Generative Image Transformer(https://arxiv.org/abs/2503.17076)
• Keywords: generation, generative
• Abstract: Masked Generative Image Transformers (MaskGIT) have emerged as a scalable and efficient image generation framework, able to deliver high-quality visuals with low inference costs. However, MaskGIT's token unmasking scheduler, an essential component of the framework, has not received the attention it deserves. We analyze the sampling objective in MaskGIT, based on the mutual information between tokens, and elucidate its shortcomings. We then propose a new sampling strategy based on our Halton scheduler instead of the original Confidence scheduler. More precisely, our method selects the token's position according to a quasi-random, low-discrepancy Halton sequence. Intuitively, that method spreads the tokens spatially, progressively covering the image uniformly at each step. Our analysis shows that it allows reducing non-recoverable sampling errors, leading to simpler hyper-parameters tuning and better quality images. Our scheduler does not require retraining or noise injection and may serve as a simple drop-in replacement for the original sampling strategy. Evaluation of both class-to-image synthesis on ImageNet and text-to-image generation on the COCO dataset demonstrates that the Halton scheduler outperforms the Confidence scheduler quantitatively by reducing the FID and qualitatively by generating more diverse and more detailed images. Our code is at this https URL.
• 摘要：蒙面的生成图像变压器（MaskGit）已成为可扩展有效的图像生成框架，能够以低推理成本提供高质量的视觉效果。但是，MaskGit的令牌揭露计划程序（框架的重要组成部分）尚未得到应有的关注。我们根据令牌之间的相互信息分析了MaskGit中的采样目标，并阐明了其缺点。然后，我们提出了一种基于Halton调度程序而不是原始信心调度程序的新抽样策略。更确切地说，我们的方法根据准随机，低均值验证的霍尔顿序列选择令牌的位置。直观地，该方法在每个步骤上逐渐覆盖图像，在空间上逐渐覆盖图像。我们的分析表明，它允许减少不可恢复的采样误差，从而导致更简单的超参数调整和质量更好的图像。我们的调度程序不需要重新训练或噪声注入，并且可以作为原始采样策略的简单替换。对COCO数据集对ImageNet和文本形象生成的班级形象合成的评估表明，Halton调度程序通过通过生成更多样化和更详细的图像来定量减少FID和定性来量化置信度调度程序。我们的代码在此HTTPS URL上。

Title: Seeing What Matters: Empowering CLIP with Patch Generation-to-Selection

• Authors: Gensheng Pei, Tao Chen, Yujia Wang, Xinhao Cai, Xiangbo Shu, Tianfei Zhou, Yazhou Yao
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17080
• Pdf URL: https://arxiv.org/pdf/2503.17080
• Copy Paste: [[2503.17080]] Seeing What Matters: Empowering CLIP with Patch Generation-to-Selection(https://arxiv.org/abs/2503.17080)
• Keywords: generation
• Abstract: The CLIP model has demonstrated significant advancements in aligning visual and language modalities through large-scale pre-training on image-text pairs, enabling strong zero-shot classification and retrieval capabilities on various domains. However, CLIP's training remains computationally intensive, with high demands on both data processing and memory. To address these challenges, recent masking strategies have emerged, focusing on the selective removal of image patches to improve training efficiency. Although effective, these methods often compromise key semantic information, resulting in suboptimal alignment between visual features and text descriptions. In this work, we present a concise yet effective approach called Patch Generation-to-Selection to enhance CLIP's training efficiency while preserving critical semantic content. Our method introduces a gradual masking process in which a small set of candidate patches is first pre-selected as potential mask regions. Then, we apply Sobel edge detection across the entire image to generate an edge mask that prioritizes the retention of the primary object areas. Finally, similarity scores between the candidate mask patches and their neighboring patches are computed, with optimal transport normalization refining the selection process to ensure a balanced similarity matrix. Our approach, CLIP-PGS, sets new state-of-the-art results in zero-shot classification and retrieval tasks, achieving superior performance in robustness evaluation and language compositionality benchmarks.
• 摘要：剪辑模型通过在图像文本对上进行大规模预训练，在对齐视觉和语言方式方面取得了重大进步，从而在各个域中实现了强大的零局部分类和检索能力。但是，夹子的培训在计算密集程度上仍然存在，对数据处理和内存的需求很高。为了应对这些挑战，最近出现了最近的掩盖策略，重点是选择性去除图像贴片以提高训练效率。尽管有效，但这些方法通常会损害关键语义信息，从而导致视觉特征和文本描述之间的次优对准。在这项工作中，我们提出了一种简洁而有效的方法，称为斑块生成到选择，以提高Clip的训练效率，同时保留关键的语义内容。我们的方法引入了逐渐掩盖的过程，其中首先将一组候选贴片作为潜在的掩模区域预先选择。然后，我们在整个图像上应用SOBEL边缘检测，以生成优先级保留主要对象区域的边缘面膜。最后，计算候选掩码贴片及其相邻贴片之间的相似性得分，最佳传输归一化来完善选择过程，以确保平衡的相似性矩阵。我们的方法剪辑PGS设置了新的最新最新方法，从而实现了零拍的分类和检索任务，从而在鲁棒性评估和语言组成性基准中实现了卓越的性能。

Title: ColabSfM: Collaborative Structure-from-Motion by Point Cloud Registration

• Authors: Johan Edstedt, André Mateus, Alberto Jaenal
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17093
• Pdf URL: https://arxiv.org/pdf/2503.17093
• Copy Paste: [[2503.17093]] ColabSfM: Collaborative Structure-from-Motion by Point Cloud Registration(https://arxiv.org/abs/2503.17093)
• Keywords: generation
• Abstract: Structure-from-Motion (SfM) is the task of estimating 3D structure and camera poses from images. We define Collaborative SfM (ColabSfM) as sharing distributed SfM reconstructions. Sharing maps requires estimating a joint reference frame, which is typically referred to as registration. However, there is a lack of scalable methods and training datasets for registering SfM reconstructions. In this paper, we tackle this challenge by proposing the scalable task of point cloud registration for SfM reconstructions. We find that current registration methods cannot register SfM point clouds when trained on existing datasets. To this end, we propose a SfM registration dataset generation pipeline, leveraging partial reconstructions from synthetically generated camera trajectories for each scene. Finally, we propose a simple but impactful neural refiner on top of the SotA registration method RoITr that yields significant improvements, which we call RefineRoITr. Our extensive experimental evaluation shows that our proposed pipeline and model enables ColabSfM. Code is available at this https URL
• 摘要：结构 - 轻度（SFM）是估计图像中3D结构和相机姿势的任务。我们将协作SFM（COLABSFM）定义为共享分布式SFM重建。共享地图需要估计联合参考框架，该框架通常称为注册。但是，缺乏用于注册SFM重建的可扩展方法和培训数据集。在本文中，我们通过提出针对SFM重建的点云注册的可扩展任务来应对这一挑战。我们发现，在现有数据集中训练时，当前的注册方法无法注册SFM点云。为此，我们提出了一个SFM注册数据集生成管道，为每个场景的合成生成的相机轨迹利用部分重建。最后，我们在SOTA登记方法的基础上提出了一个简单但有影响力的神经炼油厂，该方法可以进行重大改进，我们称之为RefineroItr。我们广泛的实验评估表明，我们提出的管道和模型可实现ColabSFM。代码可在此HTTPS URL上找到

Title: R2LDM: An Efficient 4D Radar Super-Resolution Framework Leveraging Diffusion Model

• Authors: Boyuan Zheng, Shouyi Lu, Renbo Huang, Minqing Huang, Fan Lu, Wei Tian, Guirong Zhuo, Lu Xiong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17097
• Pdf URL: https://arxiv.org/pdf/2503.17097
• Copy Paste: [[2503.17097]] R2LDM: An Efficient 4D Radar Super-Resolution Framework Leveraging Diffusion Model(https://arxiv.org/abs/2503.17097)
• Keywords: super-resolution
• Abstract: We introduce R2LDM, an innovative approach for generating dense and accurate 4D radar point clouds, guided by corresponding LiDAR point clouds. Instead of utilizing range images or bird's eye view (BEV) images, we represent both LiDAR and 4D radar point clouds using voxel features, which more effectively capture 3D shape information. Subsequently, we propose the Latent Voxel Diffusion Model (LVDM), which performs the diffusion process in the latent space. Additionally, a novel Latent Point Cloud Reconstruction (LPCR) module is utilized to reconstruct point clouds from high-dimensional latent voxel features. As a result, R2LDM effectively generates LiDAR-like point clouds from paired raw radar data. We evaluate our approach on two different datasets, and the experimental results demonstrate that our model achieves 6- to 10-fold densification of radar point clouds, outperforming state-of-the-art baselines in 4D radar point cloud super-resolution. Furthermore, the enhanced radar point clouds generated by our method significantly improve downstream tasks, achieving up to 31.7% improvement in point cloud registration recall rate and 24.9% improvement in object detection accuracy.
• 摘要：我们介绍了R2LDM，这是一种创新的方法，用于生成密度和准确的4D雷达点云，以相应的LiDar Point云为导向。我们没有使用范围图像或鸟类视图（BEV）图像，而是使用Voxel特征代表LiDAR和4D雷达点云，这些特征更有效地捕获了3D形状信息。随后，我们提出了潜在体素扩散模型（LVDM），该模型在潜在空间中执行扩散过程。此外，新型潜在点云重建（LPCR）模块可用于从高维潜在体素特征重建点云。结果，R2LDM有效地从配对的原始雷达数据中生成了LIDAR样点云。我们在两个不同的数据集上评估了我们的方法，实验结果表明，我们的模型达到了6至10倍的雷达点云的致密化，在4D雷达点云超级分辨率中表现优于最先进的基线。此外，我们方法生成的增强雷达点云显着改善了下游任务，在点云注册召回率的提高了31.7％，对象检测准确性提高了24.9％。

Title: Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrieval

• Authors: Yuanmin Tang, Jing Yu, Keke Gai, Jiamin Zhuang, Gang Xiong, Gaopeng Gou, Qi Wu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17109
• Pdf URL: https://arxiv.org/pdf/2503.17109
• Copy Paste: [[2503.17109]] Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrieval(https://arxiv.org/abs/2503.17109)
• Keywords: generation
• Abstract: Zero-Shot Composed Image Retrieval (ZS-CIR) involves diverse tasks with a broad range of visual content manipulation intent across domain, scene, object, and attribute. The key challenge for ZS-CIR tasks is to modify a reference image according to manipulation text to accurately retrieve a target image, especially when the reference image is missing essential target content. In this paper, we propose a novel prediction-based mapping network, named PrediCIR, to adaptively predict the missing target visual content in reference images in the latent space before mapping for accurate ZS-CIR. Specifically, a world view generation module first constructs a source view by omitting certain visual content of a target view, coupled with an action that includes the manipulation intent derived from existing image-caption pairs. Then, a target content prediction module trains a world model as a predictor to adaptively predict the missing visual information guided by user intention in manipulating text at the latent space. The two modules map an image with the predicted relevant information to a pseudo-word token without extra supervision. Our model shows strong generalization ability on six ZS-CIR tasks. It obtains consistent and significant performance boosts ranging from 1.73% to 4.45% over the best methods and achieves new state-of-the-art results on ZS-CIR. Our code is available at this https URL.
• 摘要：零射击组成的图像检索（ZS-CIR）涉及各种跨域，场景，对象和属性各种视觉内容操作的多种任务。 ZS-CIR任务的关键挑战是根据操纵文本修改参考图像，以准确检索目标图像，尤其是当参考图像缺少基本目标内容时。在本文中，我们提出了一个新颖的基于预测的映射网络，名为Predicir，以自适应预测潜在空间中参考图像中缺少的目标视觉内容，然后才能映射准确的ZS-CIR。具体而言，世界视图生成模块首先通过省略目标视图的某些视觉内容，再加上包括从现有图像符号对获得的操作意图来构建源视图。然后，目标内容预测模块训练世界模型，作为预测指标，以适应性地预测用户意图在潜在空间操纵文本时引导的视觉信息。这两个模块将图像映射到带有预测的相关信息的图像中，无需额外的监督即可到达伪字代币。我们的模型在六个ZS-CIR任务上显示出强大的概括能力。它获得了一致且显着的性能提高，比最佳方法的范围从1.73％到4.45％不等，并在ZS-CIR上获得了新的最先进结果。我们的代码可在此HTTPS URL上找到。

Title: Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction

• Authors: Léo Meynent, Ivan Melev, Konstantin Schürholt, Göran Kauermann, Damian Borth
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.17138
• Pdf URL: https://arxiv.org/pdf/2503.17138
• Copy Paste: [[2503.17138]] Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction(https://arxiv.org/abs/2503.17138)
• Keywords: generation
• Abstract: The weights of neural networks (NNs) have recently gained prominence as a new data modality in machine learning, with applications ranging from accuracy and hyperparameter prediction to representation learning or weight generation. One approach to leverage NN weights involves training autoencoders (AEs), using contrastive and reconstruction losses. This allows such models to be applied to a wide variety of downstream tasks, and they demonstrate strong predictive performance and low reconstruction error. However, despite the low reconstruction error, these AEs reconstruct NN models with deteriorated performance compared to the original ones, limiting their usability with regard to model weight generation. In this paper, we identify a limitation of weight-space AEs, specifically highlighting that a structural loss, that uses the Euclidean distance between original and reconstructed weights, fails to capture some features critical for reconstructing high-performing models. We analyze the addition of a behavioral loss for training AEs in weight space, where we compare the output of the reconstructed model with that of the original one, given some common input. We show a strong synergy between structural and behavioral signals, leading to increased performance in all downstream tasks evaluated, in particular NN weights reconstruction and generation.
• 摘要：神经网络（NNS）的权重最近作为机器学习中的新数据模式而引起了人们的关注，其应用程序从准确性和超参数预测到表示学习或体重产生。一种利用NN权重的方法涉及使用对比和重建损失的训练自动编码器（AES）。这使此类模型可以应用于各种下游任务，它们显示出强大的预测性能和低重建误差。但是，尽管重建误差较低，但与原始AES相比，这些AES重建了性能恶化的NN模型，从而限制了它们在模型重量产生方面的可用性。在本文中，我们确定了重量空间AES的局限性，特别强调了使用原始重量和重建权重之间使用欧几里得距离的结构损失未能捕获一些对于重建高性能模型至关重要的某些功能。我们分析了在体重空间中训练AE的行为损失的增加，在给定一些共同的输入给定的情况下，我们将重建模型的输出与原始模型的输出进行了比较。我们在结构和行为信号之间表现出强烈的协同作用，从而导致所有评估的下游任务的性能提高，尤其是NN权重的重建和产生。

Title: D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokens

• Authors: Panpan Wang, Liqiang Niu, Fandong Meng, Jinan Xu, Yufeng Chen, Jie Zhou
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17155
• Pdf URL: https://arxiv.org/pdf/2503.17155
• Copy Paste: [[2503.17155]] D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokens(https://arxiv.org/abs/2503.17155)
• Keywords: generation, generative
• Abstract: In the domain of image generation, latent-based generative models occupy a dominant status; however, these models rely heavily on image tokenizer. To meet modeling requirements, autoregressive models possessing the characteristics of scalability and flexibility embrace a discrete-valued tokenizer, but face the challenge of poor image generation quality. In contrast, diffusion models take advantage of the continuous-valued tokenizer to achieve better generation quality but are subject to low efficiency and complexity. The existing hybrid models are mainly to compensate for information loss and simplify the diffusion learning process. The potential of merging discrete-valued and continuous-valued tokens in the field of image generation has not yet been explored. In this paper, we propose D2C, a novel two-stage method to enhance model generation capacity. In the first stage, the discrete-valued tokens representing coarse-grained image features are sampled by employing a small discrete-valued generator. Then in the second stage, the continuous-valued tokens representing fine-grained image features are learned conditioned on the discrete token sequence. In addition, we design two kinds of fusion modules for seamless interaction. On the ImageNet-256 benchmark, extensive experiment results validate that our model achieves superior performance compared with several continuous-valued and discrete-valued generative models on the class-conditional image generation tasks.
• 摘要：在图像产生的领域，基于潜在的生成模型占据了主导地位。但是，这些模型在很大程度上依赖图像令牌。为了满足建模要求，具有可伸缩性和灵活性特征的自回旋模型包含了离散值的令牌仪，但面临着图像产生质量差的挑战。相比之下，扩散模型利用连续价值的令牌仪获得更好的生成质量，但效率低下。现有的混合模型主要是为了补偿信息损失并简化扩散学习过程。尚未探索在图像生成领域合并离散值和连续值的代币的潜力。在本文中，我们提出了D2C，这是一种新型的两阶段方法来增强模型产生能力。在第一阶段，通过使用小的离散值发电机来对代表粗粒图像特征的离散值代币进行采样。然后在第二阶段，以离散令牌序列为条件学习了代表细粒图像特征的连续值值。此外，我们为无缝相互作用设计了两种融合模块。在ImagEnet-256基准测试中，广泛的实验结果验证了我们的模型在类条件图像生成任务上的几个连续值和离散值的生成模型相比，我们的模型具有出色的性能。

Title: TreeSynth: Synthesizing Diverse Data from Scratch via Tree-Guided Subspace Partitioning

• Authors: Sheng Wang, Pengan Chen, Jingqi Zhou, Qintong Li, Jingwei Dong, Jiahui Gao, Boyang Xue, Jiyue Jiang, Lingpeng Kong, Chuan Wu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.17195
• Pdf URL: https://arxiv.org/pdf/2503.17195
• Copy Paste: [[2503.17195]] TreeSynth: Synthesizing Diverse Data from Scratch via Tree-Guided Subspace Partitioning(https://arxiv.org/abs/2503.17195)
• Keywords: generation
• Abstract: Model customization requires high-quality and diverse datasets, but acquiring such data remains challenging and costly. Although large language models (LLMs) can synthesize training data, current approaches are constrained by limited seed data, model bias and insufficient control over the generation process, resulting in limited diversity and biased distribution with the increase of data scales. To tackle this challenge, we present TreeSynth, a tree-guided subspace-based data synthesis framework that recursively partitions the entire data space into hierar-chical subspaces, enabling comprehensive and diverse scaling of data synthesis. Briefly, given a task-specific description, we construct a data space partitioning tree by iteratively executing criteria determination and subspace coverage steps. This hierarchically divides the whole space (i.e., root node) into mutually exclusive and complementary atomic subspaces (i.e., leaf nodes). By collecting synthesized data according to the attributes of each leaf node, we obtain a diverse dataset that fully covers the data space. Empirically, our extensive experiments demonstrate that TreeSynth surpasses both human-designed datasets and the state-of-the-art data synthesis baselines, achieving maximum improvements of 45.2% in data diversity and 17.6% in downstream task performance across various models and tasks. Hopefully, TreeSynth provides a scalable solution to synthesize diverse and comprehensive datasets from scratch without human intervention.
• 摘要：模型自定义需要高质量和不同的数据集，但是获取此类数据仍然具有挑战性且昂贵。尽管大型语言模型（LLMS）可以综合培训数据，但当前方法受到有限的种子数据，模型偏差和对生成过程的控制不足的限制，从而导致多样性有限，并且随着数据量表的增加而产生了有限的分布。为了应对这一挑战，我们提出了Treeynth，这是一个基于树的基于子空间的数据综合框架，将整个数据空间递归分为Hierar-Chical-chical子空间，从而实现了数据合成的全面和多样化的比例。简而言之，给定特定于任务的描述，我们通过迭代执行标准确定和子空间覆盖步骤来构建数据空间分区树。该分层将整个空间（即根节点）分为相互排斥和互补的原子空间（即叶子节点）。通过根据每个叶子节点的属性收集综合数据，我们获得了一个完全覆盖数据空间的不同数据集。从经验上讲，我们的广泛实验表明，树木超过人类设计的数据集和最先进的数据合成基线，在各种模型和任务中，数据多样性的最大提高了45.2％，下游任务绩效的最大提高为17.6％。希望Treemynth提供了可扩展的解决方案，可以从头开始合成各种和全面的数据集，而无需人工干预。

Title: A Deep Learning Framework for Visual Attention Prediction and Analysis of News Interfaces

• Authors: Matthew Kenely, Dylan Seychell, Carl James Debono, Chris Porter
• Subjects: cs.CV, cs.HC
• Abstract URL: https://arxiv.org/abs/2503.17212
• Pdf URL: https://arxiv.org/pdf/2503.17212
• Copy Paste: [[2503.17212]] A Deep Learning Framework for Visual Attention Prediction and Analysis of News Interfaces(https://arxiv.org/abs/2503.17212)
• Keywords: generation
• Abstract: News outlets' competition for attention in news interfaces has highlighted the need for demographically-aware saliency prediction models. Despite recent advancements in saliency detection applied to user interfaces (UI), existing datasets are limited in size and demographic representation. We present a deep learning framework that enhances the SaRa (Saliency Ranking) model with DeepGaze IIE, improving Salient Object Ranking (SOR) performance by 10.7%. Our framework optimizes three key components: saliency map generation, grid segment scoring, and map normalization. Through a two-fold experiment using eye-tracking (30 participants) and mouse-tracking (375 participants aged 13--70), we analyze attention patterns across demographic groups. Statistical analysis reveals significant age-based variations (p < 0.05, {\epsilon^2} = 0.042), with older users (36--70) engaging more with textual content and younger users (13--35) interacting more with images. Mouse-tracking data closely approximates eye-tracking behavior (sAUC = 0.86) and identifies UI elements that immediately stand out, validating its use in large-scale studies. We conclude that saliency studies should prioritize gathering data from a larger, demographically representative sample and report exact demographic distributions.
• 摘要：新闻媒体在新闻界面中引起关注的竞争强调了对人口统计学意识的显着性预测模型的需求。尽管最近应用于用户界面（UI）的显着性检测方面取得了进步，但现有数据集的大小和人口统计表示范围有限。我们提出了一个深度学习框架，该框架通过Deepgaze IIE增强了Sara（显着性排名）模型，从而将显着对象排名（SOR）的性能提高了10.7％。我们的框架优化了三个关键组成部分：显着图的生成，网格段评分和地图归一化。通过使用眼球跟踪（30名参与者）和小鼠跟踪（375名13---70岁的参与者）的两倍实验，我们分析了跨人口组的注意力模式。统计分析表明，基于年龄的显着变化（p <0.05，{\ epsilon^2} = 0.042），较老的用户（36---70）与文本内容和年轻用户（13---35）互动更多地与图像相互作用。小鼠跟踪数据紧密近似眼睛跟踪行为（SAUC = 0.86），并确定了立即脱颖而出的UI元素，从而验证了其在大规模研究中的使用。我们得出的结论是，显着性研究应优先考虑从更大的人口统计学样本中收集数据，并报告确切的人口统计分布。

Title: ML-Based Bidding Price Prediction for Pay-As-Bid Ancillary Services Markets: A Use Case in the German Control Reserve Market

• Authors: Vincent Bezold, Lukas Baur, Alexander Sauer
• Subjects: cs.LG, cs.CE, stat.AP, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.17214
• Pdf URL: https://arxiv.org/pdf/2503.17214
• Copy Paste: [[2503.17214]] ML-Based Bidding Price Prediction for Pay-As-Bid Ancillary Services Markets: A Use Case in the German Control Reserve Market(https://arxiv.org/abs/2503.17214)
• Keywords: generation
• Abstract: The increasing integration of renewable energy sources has led to greater volatility and unpredictability in electricity generation, posing challenges to grid stability. Ancillary service markets, such as the German control reserve market, allow industrial consumers and producers to offer flexibility in their power consumption or generation, contributing to grid stability while earning additional income. However, many participants use simple bidding strategies that may not maximize their revenues. This paper presents a methodology for forecasting bidding prices in pay-as-bid ancillary service markets, focusing on the German control reserve market. We evaluate various machine learning models, including Support Vector Regression, Decision Trees, and k-Nearest Neighbors, and compare their performance against benchmark models. To address the asymmetry in the revenue function of pay-as-bid markets, we introduce an offset adjustment technique that enhances the practical applicability of the forecasting models. Our analysis demonstrates that the proposed approach improves potential revenues by 27.43 % to 37.31 % compared to baseline models. When analyzing the relationship between the model forecasting errors and the revenue, a negative correlation is measured for three markets; according to the results, a reduction of 1 EUR/MW model price forecasting error (MAE) statistically leads to a yearly revenue increase between 483 EUR/MW and 3,631 EUR/MW. The proposed methodology enables industrial participants to optimize their bidding strategies, leading to increased earnings and contributing to the efficiency and stability of the electrical grid.
• 摘要：可再生能源的综合整合不断增加，导致发电的波动性和不可预测性不可预测性，对电网稳定构成了挑战。辅助服务市场，例如德国控制储备市场，使工业消费者和生产商可以在其功耗或发电方面具有灵活性，从而有助于网格稳定性，同时赚取额外的收入。但是，许多参与者使用简单的招标策略，这些策略可能无法最大化收入。本文介绍了一种预测付费辅助服务市场的竞标价格的方法，重点是德国控制储备市场。我们评估了各种机器学习模型，包括支持向量回归，决策树和K-Nearest邻居，并将其性能与基准模型进行比较。为了解决付费市场收入功能的不对称性，我们引入了一种偏移调整技术，可增强预测模型的实际适用性。我们的分析表明，与基线模型相比，提出的方法将潜在收入提高27.43％，至37.31％。在分析模型预测错误与收入之间的关系时，为三个市场衡量了负相关性。根据结果​​，减少1欧元/MW型号的价格预测错误（MAE）统计学上会导致每年的收入在483欧元/兆瓦至3,631欧元/兆瓦之间增加。提出的方法使工业参与者能够优化其招标策略，从而增加收入并促进电网的效率和稳定性。

Title: UniCon: Unidirectional Information Flow for Effective Control of Large-Scale Diffusion Models

• Authors: Fanghua Yu, Jinjin Gu, Jinfan Hu, Zheyuan Li, Chao Dong
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17221
• Pdf URL: https://arxiv.org/pdf/2503.17221
• Copy Paste: [[2503.17221]] UniCon: Unidirectional Information Flow for Effective Control of Large-Scale Diffusion Models(https://arxiv.org/abs/2503.17221)
• Keywords: generation
• Abstract: We introduce UniCon, a novel architecture designed to enhance control and efficiency in training adapters for large-scale diffusion models. Unlike existing methods that rely on bidirectional interaction between the diffusion model and control adapter, UniCon implements a unidirectional flow from the diffusion network to the adapter, allowing the adapter alone to generate the final output. UniCon reduces computational demands by eliminating the need for the diffusion model to compute and store gradients during adapter training. Our results indicate that UniCon reduces GPU memory usage by one-third and increases training speed by 2.3 times, while maintaining the same adapter parameter size. Additionally, without requiring extra computational resources, UniCon enables the training of adapters with double the parameter volume of existing ControlNets. In a series of image conditional generation tasks, UniCon has demonstrated precise responsiveness to control inputs and exceptional generation capabilities.
• 摘要：我们介绍了Unicon，这是一种新型的体系结构，旨在增强大规模扩散模型的训练适配器的控制和效率。与依赖于扩散模型和控制适配器之间双向相互作用的现有方法不同，Unicon实现了从扩散网络到适配器的单向流，从而使适配器单独生成了最终输出。 Unicon通过消除在适配器训练期间计算和存储梯度的扩散模型的需求来减少计算需求。我们的结果表明，UNICON将GPU存储器的使用量减少了三分之一，并将训练速度提高了2.3倍，同时保持相同的适配器参数大小。此外，在不需要额外的计算资源的情况下，Unicon可以以现有控制网的参数量的两倍来培训适配器。在一系列图像条件生成任务中，Unicon证明了对控制输入和出色生成能力的精确响应能力。

Title: Neuro-Symbolic Scene Graph Conditioning for Synthetic Image Dataset Generation

• Authors: Giacomo Savazzi, Eugenio Lomurno, Cristian Sbrolli, Agnese Chiatti, Matteo Matteucci
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.17224
• Pdf URL: https://arxiv.org/pdf/2503.17224
• Copy Paste: [[2503.17224]] Neuro-Symbolic Scene Graph Conditioning for Synthetic Image Dataset Generation(https://arxiv.org/abs/2503.17224)
• Keywords: generation, generative
• Abstract: As machine learning models increase in scale and complexity, obtaining sufficient training data has become a critical bottleneck due to acquisition costs, privacy constraints, and data scarcity in specialised domains. While synthetic data generation has emerged as a promising alternative, a notable performance gap remains compared to models trained on real data, particularly as task complexity grows. Concurrently, Neuro-Symbolic methods, which combine neural networks' learning strengths with symbolic reasoning's structured representations, have demonstrated significant potential across various cognitive tasks. This paper explores the utility of Neuro-Symbolic conditioning for synthetic image dataset generation, focusing specifically on improving the performance of Scene Graph Generation models. The research investigates whether structured symbolic representations in the form of scene graphs can enhance synthetic data quality through explicit encoding of relational constraints. The results demonstrate that Neuro-Symbolic conditioning yields significant improvements of up to +2.59% in standard Recall metrics and +2.83% in No Graph Constraint Recall metrics when used for dataset augmentation. These findings establish that merging Neuro-Symbolic and generative approaches produces synthetic data with complementary structural information that enhances model performance when combined with real data, providing a novel approach to overcome data scarcity limitations even for complex visual reasoning tasks.
• 摘要：随着机器学习模型的规模和复杂性的增加，由于获得的培训成本，隐私限制和专用域中的数据稀缺，获得足够的培训数据已成为关键的瓶颈。尽管合成数据的生成已成为有希望的替代方案，但与对实际数据训练的模型相比，尤其是随着任务复杂性的增长，其性能差距仍然存在。同时，将神经网络的学习强度与象征性推理的结构化表示结合的神经符号方法表现出了各种认知任务的巨大潜力。本文探讨了合成图像数据集生成的神经符号调节的实用性，专门针对改善场景图生成模型的性能。该研究调查了场景图的形式结构化符号表示是否可以通过明确编码关系约束来增强合成数据质量。结果表明，在标准召回指标中，神经符号调节可在 +2.59％的 +2.59％和 +2.83％的情况下，无图形约束召回指标在数据集扩展时产生显着改善。这些发现表明，合并神经符号和生成的方法可以通过互补的结构信息产生综合数据，从而在与真实数据结合使用时增强了模型性能，从而提供了一种新颖的方法来克服数据稀缺性限制，即使是复杂的视觉推理任务也是如此。

Title: Leveraging Text-to-Image Generation for Handling Spurious Correlation

• Authors: Aryan Yazdan Parast, Basim Azam, Naveed Akhtar
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17226
• Pdf URL: https://arxiv.org/pdf/2503.17226
• Copy Paste: [[2503.17226]] Leveraging Text-to-Image Generation for Handling Spurious Correlation(https://arxiv.org/abs/2503.17226)
• Keywords: generation
• Abstract: Deep neural networks trained with Empirical Risk Minimization (ERM) perform well when both training and test data come from the same domain, but they often fail to generalize to out-of-distribution samples. In image classification, these models may rely on spurious correlations that often exist between labels and irrelevant features of images, making predictions unreliable when those features do not exist. We propose a technique to generate training samples with text-to-image (T2I) diffusion models for addressing the spurious correlation problem. First, we compute the best describing token for the visual features pertaining to the causal components of samples by a textual inversion mechanism. Then, leveraging a language segmentation method and a diffusion model, we generate new samples by combining the causal component with the elements from other classes. We also meticulously prune the generated samples based on the prediction probabilities and attribution scores of the ERM model to ensure their correct composition for our objective. Finally, we retrain the ERM model on our augmented dataset. This process reduces the model's reliance on spurious correlations by learning from carefully crafted samples for in which this correlation does not exist. Our experiments show that across different benchmarks, our technique achieves better worst-group accuracy than the existing state-of-the-art methods.
• 摘要：当训练和测试数据来自同一领域时，接受经验风险最小化（ERM）培训的深神经网络（ERM）的表现良好，但它们通常无法推广到分布式样本。在图像分类中，这些模型可能依赖于图像的标签和无关特征之间经常存在的虚假相关性，从而使预测在不存在时不可靠。我们提出了一种技术，以生成具有文本对图像（T2I）扩散模型的培训样本，以解决虚假相关问题。首先，我们通过文本反演机制计算了与样本的因果成分有关的视觉特征的最佳描述令牌。然后，利用语言分割方法和扩散模型，我们通过将因果成分与其他类别的元素相结合来生成新样本。我们还根据ERM模型的预测概率和归因分数对生成的样品进行了精心修剪，以确保其目标的正确组成。最后，我们在增强数据集中重新验证ERM模型。这个过程通过从精心制作的样本中学习这种相关性不存在的样本来减少模型对虚假相关性的依赖。我们的实验表明，在不同的基准测试中，我们的技术比现有的最新方法实现了最差的组精度。

Title: Unsupervised Joint Learning of Optical Flow and Intensity with Event Cameras

• Authors: Shuang Guo, Friedhelm Hamann, Guillermo Gallego
• Subjects: cs.CV, cs.LG, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.17262
• Pdf URL: https://arxiv.org/pdf/2503.17262
• Copy Paste: [[2503.17262]] Unsupervised Joint Learning of Optical Flow and Intensity with Event Cameras(https://arxiv.org/abs/2503.17262)
• Keywords: generation
• Abstract: Event cameras rely on motion to obtain information about scene appearance. In other words, for event cameras, motion and appearance are seen both or neither, which are encoded in the output event stream. Previous works consider recovering these two visual quantities as separate tasks, which does not fit with the nature of event cameras and neglects the inherent relations between both tasks. In this paper, we propose an unsupervised learning framework that jointly estimates optical flow (motion) and image intensity (appearance), with a single network. Starting from the event generation model, we newly derive the event-based photometric error as a function of optical flow and image intensity, which is further combined with the contrast maximization framework, yielding a comprehensive loss function that provides proper constraints for both flow and intensity estimation. Exhaustive experiments show that our model achieves state-of-the-art performance for both optical flow (achieves 20% and 25% improvement in EPE and AE respectively in the unsupervised learning category) and intensity estimation (produces competitive results with other baselines, particularly in high dynamic range scenarios). Last but not least, our model achieves shorter inference time than all the other optical flow models and many of the image reconstruction models, while they output only one quantity. Project page: this https URL
• 摘要：事件摄像机依靠运动来获取有关场景外观的信息。换句话说，对于事件摄像机，运动和外观都可以看到或两者都在输出事件流中编码。以前的作品考虑将这两个视觉量恢复为单独的任务，这与事件摄像机的性质不符，并忽略了这两个任务之间的固有关系。在本文中，我们提出了一个无监督的学习框架，该框架与单个网络共同估计光流（运动）和图像强度（外观）。从事件生成模型开始，我们新得出基于事件的光度误差作为光流和图像强度的函数，该误差与对比度最大化框架相结合，产生了全面的损耗函数，为流量和强度估计提供了适当的约束。详尽的实验表明，我们的模型可实现光流的最新性能（在无监督学习类别中分别提高了EPE和AE的20％和25％）和强度估计（在高动态范围方案中与其他基础线产生竞争成果）。最后但并非最不重要的一点是，我们的模型比所有其他光流模型和许多图像重建模型的推理时间更短，而它们仅输出一个数量。项目页面：此HTTPS URL

Title: Physical Plausibility-aware Trajectory Prediction via Locomotion Embodiment

• Authors: Hiromu Taketsugu, Takeru Oba, Takahiro Maeda, Shohei Nobuhara, Norimichi Ukita
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17267
• Pdf URL: https://arxiv.org/pdf/2503.17267
• Copy Paste: [[2503.17267]] Physical Plausibility-aware Trajectory Prediction via Locomotion Embodiment(https://arxiv.org/abs/2503.17267)
• Keywords: generation
• Abstract: Humans can predict future human trajectories even from momentary observations by using human pose-related cues. However, previous Human Trajectory Prediction (HTP) methods leverage the pose cues implicitly, resulting in implausible predictions. To address this, we propose Locomotion Embodiment, a framework that explicitly evaluates the physical plausibility of the predicted trajectory by locomotion generation under the laws of physics. While the plausibility of locomotion is learned with an indifferentiable physics simulator, it is replaced by our differentiable Locomotion Value function to train an HTP network in a data-driven manner. In particular, our proposed Embodied Locomotion loss is beneficial for efficiently training a stochastic HTP network using multiple heads. Furthermore, the Locomotion Value filter is proposed to filter out implausible trajectories at inference. Experiments demonstrate that our method enhances even the state-of-the-art HTP methods across diverse datasets and problem settings. Our code is available at: this https URL.
• 摘要：人类即使使用与人类姿势相关的线索也可以从瞬间观察中预测未来的人类轨迹。但是，以前的人类轨迹预测（HTP）方法隐含了姿势线索，从而导致了难以置信的预测。为了解决这个问题，我们提出了运动实施例，该框架明确评估了根据物理定律通过运动产生预测轨迹的物理合理性。虽然具有无动于衷的物理模拟器学习了运动的合理性，但以数据驱动的方式训练HTP网络，它被我们可区分的运动价值函数所取代。特别是，我们提出的体现运动损失对使用多个头部有效训练随机HTP网络是有益的。此外，提出了运动值滤波器在推理时过滤出令人难以置信的轨迹。实验表明，我们的方法甚至增强了各种数据集和问题设置的最新HTP方法。我们的代码可用：此HTTPS URL。

Title: Offline Model-Based Optimization: Comprehensive Review

• Authors: Minsu Kim, Jiayao Gu, Ye Yuan, Taeyoung Yun, Zixuan Liu, Yoshua Bengio, Can Chen
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.17286
• Pdf URL: https://arxiv.org/pdf/2503.17286
• Copy Paste: [[2503.17286]] Offline Model-Based Optimization: Comprehensive Review(https://arxiv.org/abs/2503.17286)
• Keywords: generative
• Abstract: Offline optimization is a fundamental challenge in science and engineering, where the goal is to optimize black-box functions using only offline datasets. This setting is particularly relevant when querying the objective function is prohibitively expensive or infeasible, with applications spanning protein engineering, material discovery, neural architecture search, and beyond. The main difficulty lies in accurately estimating the objective landscape beyond the available data, where extrapolations are fraught with significant epistemic uncertainty. This uncertainty can lead to objective hacking(reward hacking), exploiting model inaccuracies in unseen regions, or other spurious optimizations that yield misleadingly high performance estimates outside the training distribution. Recent advances in model-based optimization(MBO) have harnessed the generalization capabilities of deep neural networks to develop offline-specific surrogate and generative models. Trained with carefully designed strategies, these models are more robust against out-of-distribution issues, facilitating the discovery of improved designs. Despite its growing impact in accelerating scientific discovery, the field lacks a comprehensive review. To bridge this gap, we present the first thorough review of offline MBO. We begin by formalizing the problem for both single-objective and multi-objective settings and by reviewing recent benchmarks and evaluation metrics. We then categorize existing approaches into two key areas: surrogate modeling, which emphasizes accurate function approximation in out-of-distribution regions, and generative modeling, which explores high-dimensional design spaces to identify high-performing designs. Finally, we examine the key challenges and propose promising directions for advancement in this rapidly evolving field including safe control of superintelligent systems.
• 摘要：离线优化是科学和工程学中的一个基本挑战，其目标是仅使用离线数据集优化黑框功能。当查询目标函数的昂贵或不可行时，这种设置尤其重要，其应用程序涵盖了蛋白质工程，材料发现，神经体系结构搜索以及其他。主要困难在于准确估算可用数据之外的客观格局，在这种数据中，外推充满了认识论的明显不确定性。这种不确定性可能会导致客观的黑客入侵（奖励黑客），在看不见的地区利用模型不准确或其他虚假优化，这些优化产生了训练分布以外的误导性高性能估计。基于模型的优化（MBO）的最新进展已利用深度神经网络的概括能力来开发离线特定的替代模型和生成模型。经过精心设计的策略培训，这些模型在分布外的问题上更为强大，从而促进了改进的设计。尽管该领域在加速科学发现方面的影响越来越大，但仍缺乏全面的审查。为了弥合这一差距，我们介绍了离线MBO的首次彻底综述。我们首先将单目标和多目标设置的问题形式化，并审查最近的基准和评估指标。然后，我们将现有方法分为两个关键领域：替代建模，该模型强调了分布外区域的准确函数近似以及生成建模，该建模探索了高维设计空间以识别高性能设计。最后，我们研究了关键挑战，并提出了在这个快速发展的领域中进步的有希望的方向，包括对超级智能系统的安全控制。

Title: Preference-Guided Diffusion for Multi-Objective Offline Optimization

• Authors: Yashas Annadani, Syrine Belakaria, Stefano Ermon, Stefan Bauer, Barbara E Engelhardt
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.17299
• Pdf URL: https://arxiv.org/pdf/2503.17299
• Copy Paste: [[2503.17299]] Preference-Guided Diffusion for Multi-Objective Offline Optimization(https://arxiv.org/abs/2503.17299)
• Keywords: generative
• Abstract: Offline multi-objective optimization aims to identify Pareto-optimal solutions given a dataset of designs and their objective values. In this work, we propose a preference-guided diffusion model that generates Pareto-optimal designs by leveraging a classifier-based guidance mechanism. Our guidance classifier is a preference model trained to predict the probability that one design dominates another, directing the diffusion model toward optimal regions of the design space. Crucially, this preference model generalizes beyond the training distribution, enabling the discovery of Pareto-optimal solutions outside the observed dataset. We introduce a novel diversity-aware preference guidance, augmenting Pareto dominance preference with diversity criteria. This ensures that generated solutions are optimal and well-distributed across the objective space, a capability absent in prior generative methods for offline multi-objective optimization. We evaluate our approach on various continuous offline multi-objective optimization tasks and find that it consistently outperforms other inverse/generative approaches while remaining competitive with forward/surrogate-based optimization methods. Our results highlight the effectiveness of classifier-guided diffusion models in generating diverse and high-quality solutions that approximate the Pareto front well.
• 摘要：离线多目标优化旨在识别帕累托最佳解决方案，并给定设计数据集及其目标值。在这项工作中，我们提出了一个优先引导的扩散模型，该模型通过利用基于分类器的指导机制来生成帕累托最佳设计。我们的指导分类器是一种偏好模型，该模型训练有素，可以预测一个设计主导另一个设计的概率，将扩散模型指向设计空间的最佳区域。至关重要的是，该偏好模型概括了训练分布之外，从而在观察到的数据集之外发现了帕累托最佳解决方案。我们引入了一种新颖的多样性偏好指导，以多样性标准增强了帕累托优势偏好。这样可以确保生成的解决方案在整个目标空间中都是最佳且分布良好的，这是离线多目标优化的先前生成方法中缺乏的能力。我们在各种连续的离线多目标优化任务上评估了我们的方法，并发现它始终优于其他逆/生成方法，同时与基于远期/替代的优化方法保持竞争力。我们的结果突出了分类器引导的扩散模型在生成近似帕累托阵线的多样化和高质量解决方案方面的有效性。

Title: Dereflection Any Image with Diffusion Priors and Diversified Data

• Authors: Jichen Hu, Chen Yang, Zanwei Zhou, Jiemin Fang, Xiaokang Yang, Qi Tian, Wei Shen
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17347
• Pdf URL: https://arxiv.org/pdf/2503.17347
• Copy Paste: [[2503.17347]] Dereflection Any Image with Diffusion Priors and Diversified Data(https://arxiv.org/abs/2503.17347)
• Keywords: restoration
• Abstract: Reflection removal of a single image remains a highly challenging task due to the complex entanglement between target scenes and unwanted reflections. Despite significant progress, existing methods are hindered by the scarcity of high-quality, diverse data and insufficient restoration priors, resulting in limited generalization across various real-world scenarios. In this paper, we propose Dereflection Any Image, a comprehensive solution with an efficient data preparation pipeline and a generalizable model for robust reflection removal. First, we introduce a dataset named Diverse Reflection Removal (DRR) created by randomly rotating reflective mediums in target scenes, enabling variation of reflection angles and intensities, and setting a new benchmark in scale, quality, and diversity. Second, we propose a diffusion-based framework with one-step diffusion for deterministic outputs and fast inference. To ensure stable learning, we design a three-stage progressive training strategy, including reflection-invariant finetuning to encourage consistent outputs across varying reflection patterns that characterize our dataset. Extensive experiments show that our method achieves SOTA performance on both common benchmarks and challenging in-the-wild images, showing superior generalization across diverse real-world scenes.
• 摘要：由于目标场景和不需要的反射之间的复杂纠缠，因此删除单个图像仍然是一项高度挑战的任务。尽管取得了重大进展，但由于高质量，多样化的数据和不足的恢复先验的稀缺而阻碍了现有的方法，从而导致各种现实世界中的概括有限。在本文中，我们提出了任何图像的反思，一种具有有效数据准备管道的综合解决方案，以及可推广的模型，用于鲁棒反射去除。首先，我们介绍了一个名为“不同反射删除”（DRR）的数据集，该数据集是通过在目标场景中随机旋转反射介质创建的，实现了反射角度和强度的变化，并在规模，质量和多样性上设定了新的基准标准。其次，我们提出了一个基于扩散的框架，具有一步扩散的确定性输出和快速推断。为了确保稳定的学习，我们设计了一个三阶段的渐进式培训策略，包括反思不变的填充，以鼓励跨不同反射模式的一致输出来表征我们的数据集。广泛的实验表明，我们的方法在共同的基准和挑战的内部图像上都能达到SOTA性能，从而在各种现实世界中表现出卓越的概括。

Title: NdLinear Is All You Need for Representation Learning

• Authors: Alex Reneau, Jerry Yao-Chieh Hu, Zhongfang Zhuang, Ting-Chun Liu
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.17353
• Pdf URL: https://arxiv.org/pdf/2503.17353
• Copy Paste: [[2503.17353]] NdLinear Is All You Need for Representation Learning(https://arxiv.org/abs/2503.17353)
• Keywords: generation
• Abstract: Many high-impact machine learning tasks involve multi-dimensional data (e.g., images, volumetric medical scans, multivariate time-series). Yet, most neural architectures flatten inputs, discarding critical cross-dimension information. We introduce NdLinear, a novel linear transformation that preserves these structures without extra overhead. By operating separately along each dimension, NdLinear captures dependencies that standard fully connected layers overlook. Extensive experiments across convolutional, recurrent, and transformer-based networks show significant improvements in representational power and parameter efficiency. Crucially, NdLinear serves as a foundational building block for large-scale foundation models by operating on any unimodal or multimodal data in its native form. This removes the need for flattening or modality-specific preprocessing. Ndlinear rethinks core architectural priorities beyond attention, enabling more expressive, context-aware models at scale. We propose NdLinear as a drop-in replacement for standard linear layers -- marking an important step toward next-generation neural architectures.
• 摘要：许多高影响力的机器学习任务涉及多维数据（例如，图像，体积医学扫描，多元时间序列）。然而，大多数神经体系结构都会变平输入，丢弃关键的跨维信息。我们介绍了Ndlinear，这是一种新型的线性变换，可保留这些结构而没有额外的开销。通过沿每个维度单独操作，NDLinear捕获了标准完全连接层忽略的依赖项。跨卷积，经常性和基于变压器网络的广泛实验显示出表示能力和参数效率的显着提高。至关重要的是，Ndlinear通过以其天然形式的任何单峰或多模式数据运行，是大规模基础模型的基础构建块。这消除了对扁平或特定方式预处理的需求。 Ndlinear不受关注的核心建筑优先级重新考虑，从而使更具表现力的上下文感知模型进行了扩展。我们建议NDLinear作为标准线性层的倒入替代品 - 标志着迈向下一代神经体系结构的重要一步。

Title: Position: Interactive Generative Video as Next-Generation Game Engine

• Authors: Jiwen Yu, Yiran Qin, Haoxuan Che, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, Xihui Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.17359
• Pdf URL: https://arxiv.org/pdf/2503.17359
• Copy Paste: [[2503.17359]] Position: Interactive Generative Video as Next-Generation Game Engine(https://arxiv.org/abs/2503.17359)
• Keywords: generation, generative
• Abstract: Modern game development faces significant challenges in creativity and cost due to predetermined content in traditional game engines. Recent breakthroughs in video generation models, capable of synthesizing realistic and interactive virtual environments, present an opportunity to revolutionize game creation. In this position paper, we propose Interactive Generative Video (IGV) as the foundation for Generative Game Engines (GGE), enabling unlimited novel content generation in next-generation gaming. GGE leverages IGV's unique strengths in unlimited high-quality content synthesis, physics-aware world modeling, user-controlled interactivity, long-term memory capabilities, and causal reasoning. We present a comprehensive framework detailing GGE's core modules and a hierarchical maturity roadmap (L0-L4) to guide its evolution. Our work charts a new course for game development in the AI era, envisioning a future where AI-powered generative systems fundamentally reshape how games are created and experienced.
• 摘要：由于传统游戏引擎的预定内容，现代游戏开发在创造力和成本方面面临重大挑战。视频生成模型的最新突破，能够综合现实和互动的虚拟环境，为革新游戏创作提供了机会。在该职位论文中，我们提出了交互式生成视频（IGV）作为生成游戏引擎（GGE）的基础，从而在下一代游戏中实现了无限的新颖内容。 GGE在无限的高质量内容综合，物理感知世界建模，用户控制的互动，长期记忆能力和因果推理中利用IGV的独特优势。我们提出了一个综合框架，详细介绍了GGE的核心模块和一个分层的成熟路线图（L0-L4），以指导其发展。我们的工作为AI时代的游戏开发绘制了新的课程，并设想了一个未来的AI驱动生成系统从根本上重塑游戏的创建和体验。

Title: Gumbel-Softmax Flow Matching with Straight-Through Guidance for Controllable Biological Sequence Generation

• Authors: Sophia Tang, Yinuo Zhang, Alexander Tong, Pranam Chatterjee
• Subjects: cs.LG, q-bio.BM
• Abstract URL: https://arxiv.org/abs/2503.17361
• Pdf URL: https://arxiv.org/pdf/2503.17361
• Copy Paste: [[2503.17361]] Gumbel-Softmax Flow Matching with Straight-Through Guidance for Controllable Biological Sequence Generation(https://arxiv.org/abs/2503.17361)
• Keywords: generation, generative
• Abstract: Flow matching in the continuous simplex has emerged as a promising strategy for DNA sequence design, but struggles to scale to higher simplex dimensions required for peptide and protein generation. We introduce Gumbel-Softmax Flow and Score Matching, a generative framework on the simplex based on a novel Gumbel-Softmax interpolant with a time-dependent temperature. Using this interpolant, we introduce Gumbel-Softmax Flow Matching by deriving a parameterized velocity field that transports from smooth categorical distributions to distributions concentrated at a single vertex of the simplex. We alternatively present Gumbel-Softmax Score Matching which learns to regress the gradient of the probability density. Our framework enables high-quality, diverse generation and scales efficiently to higher-dimensional simplices. To enable training-free guidance, we propose Straight-Through Guided Flows (STGFlow), a classifier-based guidance method that leverages straight-through estimators to steer the unconditional velocity field toward optimal vertices of the simplex. STGFlow enables efficient inference-time guidance using classifiers pre-trained on clean sequences, and can be used with any discrete flow method. Together, these components form a robust framework for controllable de novo sequence generation. We demonstrate state-of-the-art performance in conditional DNA promoter design, sequence-only protein generation, and target-binding peptide design for rare disease treatment.
• 摘要：连续单纯形中的流量匹配已成为DNA序列设计的一种有希望的策略，但是努力缩放到肽和蛋白质产生所需的更高单纯尺寸。我们介绍了Gumbel-Softmax流量和得分匹配，这是基于新颖的Gumbel-Softmax interpolant的单纯框架，并具有时间相关的温度。使用此插值，我们通过得出一个参数化速度字段来引入Gumbel-Softmax流量匹配，该速度字段从光滑的分类分布传输到集中在单纯形的单个顶点的分布中。或者，我们呈现Gumbel-Softmax分数匹配，该分数学会回归概率密度的梯度。我们的框架使高质量，多样化的生成并有效地缩放到更高维的简单。为了实现无培训指导，我们提出了直通引导流（STGFLOD），这是一种基于分类器的指导方法，该方法利用直通估计器来指导无条件的速度字段朝着简单的最佳顶点。 STGFLOW可以使用在干净序列上预先训练的分类器进行有效的推理时间指导，并且可以与任何离散流量方法一起使用。这些组件一起构成了可控的从头序列生成的强大框架。我们在条件DNA启动子设计，仅序列的蛋白质产生和靶标的肽设计中展示了最先进的性能。