2025-03-31

Title: Leveraging Large Language Models for Automated Causal Loop Diagram Generation: Enhancing System Dynamics Modeling through Curated Prompting Techniques

• Authors: Ning-Yuan Georgia Liu, David R. Keith
• Subjects: cs.LG, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.21798
• Pdf URL: https://arxiv.org/pdf/2503.21798
• Copy Paste: [[2503.21798]] Leveraging Large Language Models for Automated Causal Loop Diagram Generation: Enhancing System Dynamics Modeling through Curated Prompting Techniques(https://arxiv.org/abs/2503.21798)
• Keywords: generation
• Abstract: Transforming a dynamic hypothesis into a causal loop diagram (CLD) is crucial for System Dynamics Modelling. Extracting key variables and causal relationships from text to build a CLD is often challenging and time-consuming for novice modelers, limiting SD tool adoption. This paper introduces and tests a method for automating the translation of dynamic hypotheses into CLDs using large language models (LLMs) with curated prompting techniques. We first describe how LLMs work and how they can make the inferences needed to build CLDs using a standard digraph structure. Next, we develop a set of simple dynamic hypotheses and corresponding CLDs from leading SD textbooks. We then compare the four different combinations of prompting techniques, evaluating their performance against CLDs labeled by expert modelers. Results show that for simple model structures and using curated prompting techniques, LLMs can generate CLDs of a similar quality to expert-built ones, accelerating CLD creation.
• 摘要：将动态假设转换为因果环图（CLD）对于系统动力学建模至关重要。从文本中提取关键变量和因果关系来构建CLD通常对新手建模者来说通常是具有挑战性的，并且限制了SD工具的采用。本文介绍并测试了一种使用大型语言模型（LLMS）自动化动态假设将动态假设自动化为CLD的方法。我们首先描述了LLM的工作原理以及它们如何使用标准的Digraph结构进行构建CLD所需的推论。接下来，我们从领先的SD教科书中开发了一组简单的动态假设和相应的CLD。然后，我们比较了提示技术的四种不同组合，从而评估了它们与专家建模者标记的CLD的性能。结果表明，对于简单的模型结构并使用策划的提示技术，LLM可以生成与专家构建的质量相似的CLD，从而加速CLD创建。

Title: LERO: LLM-driven Evolutionary framework with Hybrid Rewards and Enhanced Observation for Multi-Agent Reinforcement Learning

• Authors: Yuan Wei, Xiaohan Shan, Jianmin Li
• Subjects: cs.LG, cs.AI, cs.MA
• Abstract URL: https://arxiv.org/abs/2503.21807
• Pdf URL: https://arxiv.org/pdf/2503.21807
• Copy Paste: [[2503.21807]] LERO: LLM-driven Evolutionary framework with Hybrid Rewards and Enhanced Observation for Multi-Agent Reinforcement Learning(https://arxiv.org/abs/2503.21807)
• Keywords: generation
• Abstract: Multi-agent reinforcement learning (MARL) faces two critical bottlenecks distinct from single-agent RL: credit assignment in cooperative tasks and partial observability of environmental states. We propose LERO, a framework integrating Large language models (LLMs) with evolutionary optimization to address these MARL-specific challenges. The solution centers on two LLM-generated components: a hybrid reward function that dynamically allocates individual credit through reward decomposition, and an observation enhancement function that augments partial observations with inferred environmental context. An evolutionary algorithm optimizes these components through iterative MARL training cycles, where top-performing candidates guide subsequent LLM generations. Evaluations in Multi-Agent Particle Environments (MPE) demonstrate LERO's superiority over baseline methods, with improved task performance and training efficiency.
• 摘要：多机构增强学习（MARL）面临着两种与单位Agent RL不同的关键瓶颈：合作任务中的信用分配和环境国家的部分可观察性。我们提出了Lero，这是一个将大型语言模型（LLM）与进化优化集成的框架，以应对这些特定于MARL的挑战。该解决方案集中在两个LLM生成的组件上：一种混合奖励函数，通过奖励分解动态分配个人信用，以及一种观察增强功能，可以通过推断的环境环境增强部分观察。进化算法通过迭代MARL训练周期优化了这些组件，在该周期中，表现最佳的候选人引导后来的LLM世代。多代理粒子环境（MPE）的评估证明了Lero优于基线方法，并提高了任务性能和训练效率。

Title: IPGO: Indirect Prompt Gradient Optimization on Text-to-Image Generative Models with High Data Efficiency

• Authors: Jianping Ye, Michel Wedel, Kunpeng Zhang
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.21812
• Pdf URL: https://arxiv.org/pdf/2503.21812
• Copy Paste: [[2503.21812]] IPGO: Indirect Prompt Gradient Optimization on Text-to-Image Generative Models with High Data Efficiency(https://arxiv.org/abs/2503.21812)
• Keywords: generation, generative
• Abstract: Text-to-Image Diffusion models excel at generating images from text prompts but often lack optimal alignment with content semantics, aesthetics, and human preferences. To address these issues, in this study we introduce a novel framework, Indirect Prompt Gradient Optimization (IPGO), for prompt-level fine-tuning. IPGO enhances prompt embeddings by injecting continuously differentiable tokens at the beginning and end of the prompt embeddings, while exploiting low-rank benefits and flexibility from rotations. It allows for gradient-based optimization of injected tokens while enforcing value, orthonormality, and conformity constraints, facilitating continuous updates and empowering computational efficiency. To evaluate the performance of IPGO, we conduct prompt-wise and prompt-batch training with three reward models targeting image aesthetics, image-text alignment, and human preferences under three datasets of different complexity. The results show that IPGO consistently matches or outperforms cutting-edge benchmarks, including stable diffusion v1.5 with raw prompts, training-based approaches (DRaFT and DDPO), and training-free methods (DPO-Diffusion, Promptist, and ChatGPT-4o). Furthermore, we demonstrate IPGO's effectiveness in enhancing image generation quality while requiring minimal training data and limited computational resources.
• 摘要：文本到图像扩散模型在从文本提示中生成图像方面表现出色，但通常缺乏与内容语义，美学和人类偏好的最佳对齐。为了解决这些问题，在本研究中，我们介绍了一个新颖的框架，间接及时梯度优化（IPGO），以及时进行微调。 IPGO通过在及时嵌入的开始和结尾处注入连续可区分的令牌来增强提示嵌入，同时利用低级福利和旋转的灵活性。它允许基于梯度的注射令牌优化，同时实施价值，正常值和合格约束，从而促进连续更新并赋予计算效率。为了评估IPGO的性能，我们在三个不同复杂性的三个数据集中进行了三个奖励模型，以三种奖励模型进行迅速和及时的培训。结果表明，IPGO始终匹配或优于尖端基准测试，包括稳定的扩散v1.5，具有原始提示，基于培训的方法（草稿和DDPO）以及无培训方法（DPO扩散，促使，促使主义者和CANTGPT-4O）。此外，我们证明了IPGO在提高图像生成质量方面的有效性，同时需要最少的培训数据和有限的计算资源。

Title: Low-Rank Adaptation of Pre-Trained Stable Diffusion for Rigid-Body Target ISAR Imaging

• Authors: Boan Zhang, Hang Dong, Jiongge Zhang, Long Tian, Rongrong Wang, Zhenhua Wu, Xiyang Liu, Hongwei Liu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.21823
• Pdf URL: https://arxiv.org/pdf/2503.21823
• Copy Paste: [[2503.21823]] Low-Rank Adaptation of Pre-Trained Stable Diffusion for Rigid-Body Target ISAR Imaging(https://arxiv.org/abs/2503.21823)
• Keywords: super-resolution, generative
• Abstract: Traditional range-instantaneous Doppler (RID) methods for rigid-body target imaging often suffer from low resolution due to the limitations of time-frequency analysis (TFA). To address this challenge, our primary focus is on obtaining high resolution time-frequency representations (TFRs) from their low resolution counterparts. Recognizing that the curve features of TFRs are a specific type of texture feature, we argue that pre trained generative models such as Stable Diffusion (SD) are well suited for enhancing TFRs, thanks to their powerful capability in capturing texture representations. Building on this insight, we propose a novel inverse synthetic aperture radar (ISAR) imaging method for rigid-body targets, leveraging the low-rank adaptation (LoRA) of a pre-trained SD model. Our approach adopts the basic structure and pre-trained parameters of SD Turbo while incorporating additional linear operations for LoRA and adversarial training to achieve super-resolution and noise suppression. Then we integrate LoRA-SD into the RID-based ISAR imaging, enabling sharply focused and denoised imaging with super-resolution capabilities. We evaluate our method using both simulated and real radar data. The experimental results demonstrate the superiority of our approach in frequency es timation and ISAR imaging compared to traditional methods. Notably, the generalization capability is verified by training on simulated radar data and testing on measured radar data.
• 摘要：由于时间频率分析的局限性（TFA），传统的范围内跨多普勒（RID）用于刚体目标成像的方法通常会遭受低分辨率。为了应对这一挑战，我们的主要重点是从低分辨率对应物中获得高分辨率的时频表示（TFR）。认识到TFR的曲线功能是一种特定类型的纹理功能，我们认为，诸如稳定扩散（SD）之类的经过训练的生成模型非常适合增强TFR，这要归功于它们在捕获纹理表示方面的强大能力。在此洞察力的基础上，我们提出了一种用于刚体靶标的新型逆合成孔径（ISAR）成像方法，利用了预训练的SD模型的低级别适应性（LORA）。我们的方法采用了SD涡轮增压的基本结构和预训练的参数，同时结合了用于洛拉和对抗训练的其他线性操作，以实现超分辨率和抑制噪声。然后，我们将Lora-SD集成到基于RID的ISAR成像中，从而可以通过超分辨率功能引起人们的焦点和去核成像。我们使用模拟和实际雷达数据评估我们的方法。实验结果表明，与传统方法相比，我们在频率ES时和ISAR成像中的方法的优势。值得注意的是，通过对模拟雷达数据进行培训并测试测得的雷达数据，可以验证概括能力。

Title: Protecting Your Video Content: Disrupting Automated Video-based LLM Annotations

• Authors: Haitong Liu, Kuofeng Gao, Yang Bai, Jinmin Li, Jinxiao Shan, Tao Dai, Shu-Tao Xia
• Subjects: cs.CV, cs.CR
• Abstract URL: https://arxiv.org/abs/2503.21824
• Pdf URL: https://arxiv.org/pdf/2503.21824
• Copy Paste: [[2503.21824]] Protecting Your Video Content: Disrupting Automated Video-based LLM Annotations(https://arxiv.org/abs/2503.21824)
• Keywords: generation
• Abstract: Recently, video-based large language models (video-based LLMs) have achieved impressive performance across various video comprehension tasks. However, this rapid advancement raises significant privacy and security concerns, particularly regarding the unauthorized use of personal video data in automated annotation by video-based LLMs. These unauthorized annotated video-text pairs can then be used to improve the performance of downstream tasks, such as text-to-video generation. To safeguard personal videos from unauthorized use, we propose two series of protective video watermarks with imperceptible adversarial perturbations, named Ramblings and Mutes. Concretely, Ramblings aim to mislead video-based LLMs into generating inaccurate captions for the videos, thereby degrading the quality of video annotations through inconsistencies between video content and captions. Mutes, on the other hand, are designed to prompt video-based LLMs to produce exceptionally brief captions, lacking descriptive detail. Extensive experiments demonstrate that our video watermarking methods effectively protect video data by significantly reducing video annotation performance across various video-based LLMs, showcasing both stealthiness and robustness in protecting personal video content. Our code is available at this https URL.
• 摘要：最近，基于视频的大型语言模型（基于视频的LLM）在各种视频理解任务中取得了令人印象深刻的表现。但是，这种快速的进步引起了严重的隐私和安全问题，特别是关于在基于视频的LLM的自动注释中未经授权使用的个人视频数据的使用。然后可以使用这些未经授权的注释的视频文本对来改善下游任务的性能，例如文本到视频生成。为了保护个人视频免受未经授权的使用，我们提出了两种具有不可察觉的对抗性扰动的保护性视频水印，命名为漫游和静音。具体而言，漫游者旨在误导基于视频的LLM，以生成视频的不准确字幕，从而通过视频内容和字幕之间的不一致来降低视频注释的质量。另一方面，静音旨在提示基于视频的LLM，以产生异常简短的字幕，缺乏描述性细节。广泛的实验表明，我们的视频水印方法通过显着降低各种基于视频的LLM的视频注释性能有效地保护视频数据，从而展示了保护个人视频内容的隐秘性和鲁棒性。我们的代码可在此HTTPS URL上找到。

Title: Shape Generation via Weight Space Learning

• Authors: Maximilian Plattner, Arturs Berzins, Johannes Brandstetter
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.21830
• Pdf URL: https://arxiv.org/pdf/2503.21830
• Copy Paste: [[2503.21830]] Shape Generation via Weight Space Learning(https://arxiv.org/abs/2503.21830)
• Keywords: generation, generative
• Abstract: Foundation models for 3D shape generation have recently shown a remarkable capacity to encode rich geometric priors across both global and local dimensions. However, leveraging these priors for downstream tasks can be challenging as real-world data are often scarce or noisy, and traditional fine-tuning can lead to catastrophic forgetting. In this work, we treat the weight space of a large 3D shape-generative model as a data modality that can be explored directly. We hypothesize that submanifolds within this high-dimensional weight space can modulate topological properties or fine-grained part features separately, demonstrating early-stage evidence via two experiments. First, we observe a sharp phase transition in global connectivity when interpolating in conditioning space, suggesting that small changes in weight space can drastically alter topology. Second, we show that low-dimensional reparameterizations yield controlled local geometry changes even with very limited data. These results highlight the potential of weight space learning to unlock new approaches for 3D shape generation and specialized fine-tuning.
• 摘要：3D形成生成的基础模型最近显示出在全球和局部维度上编码丰富的几何先验的显着能力。但是，由于现实世界中的数据通常很少或嘈杂，因此利用这些先验的下游任务可能会具有挑战性，并且传统的微调可能导致灾难性的遗忘。在这项工作中，我们将大型3D形状生成模型的重量空间视为可以直接探索的数据模式。我们假设在此高维重量空间内的子延伸物可以分别调节拓扑特性或细粒部分特征，从而通过两个实验证明了早期证据。首先，当在调节空间中插值时，我们观察到全球连通性的急剧过渡，这表明体重空间的较小变化可以大大改变拓扑。其次，我们表明，即使数据非常有限，低维度重量化也会产生控制的局部几何形状的变化。这些结果突出了体重空间学习的潜力，以解锁3D形状生成和专业微调的新方法。

Title: M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization?

• Authors: Haolong Yan, Kaijun Tan, Yeqing Shen, Xin Huang, Zheng Ge, Xiangyu Zhang, Si Li, Daxin Jiang
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.21839
• Pdf URL: https://arxiv.org/pdf/2503.21839
• Copy Paste: [[2503.21839]] M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization?(https://arxiv.org/abs/2503.21839)
• Keywords: generation
• Abstract: We investigate a critical yet under-explored question in Large Vision-Language Models (LVLMs): Do LVLMs genuinely comprehend interleaved image-text in the document? Existing document understanding benchmarks often assess LVLMs using question-answer formats, which are information-sparse and difficult to guarantee the coverage of long-range dependencies. To address this issue, we introduce a novel and challenging Multimodal Document Summarization Benchmark (M-DocSum-Bench), which comprises 500 high-quality arXiv papers, along with interleaved multimodal summaries aligned with human preferences. M-DocSum-Bench is a reference-based generation task and necessitates the generation of interleaved image-text summaries using provided reference images, thereby simultaneously evaluating capabilities in understanding, reasoning, localization, and summarization within complex multimodal document scenarios. To facilitate this benchmark, we develop an automated framework to construct summaries and propose a fine-grained evaluation method called M-DocEval. Moreover, we further develop a robust summarization baseline, i.e., M-DocSum-7B, by progressive two-stage training with diverse instruction and preference data. The extensive results on our M-DocSum-Bench reveal that the leading LVLMs struggle to maintain coherence and accurately integrate information within long and interleaved contexts, often exhibiting confusion between similar images and a lack of robustness. Notably, M-DocSum-7B achieves state-of-the-art performance compared to larger and closed-source models (including GPT-4o, Gemini Pro, Claude-3.5-Sonnet and Qwen2.5-VL-72B, etc.), demonstrating the potential of LVLMs for improved interleaved image-text understanding. The code, data, and models are available at this https URL.
• 摘要：我们在大型视力语言模型（LVLMS）中研究了一个关键但爆炸不足的问题：LVLM是否真的理解了文档中的交织图像文本？现有的文档理解基准通常会使用问题解答格式评估LVLM，这些格式是信息的，并且难以保证对远程依赖的覆盖范围。为了解决这个问题，我们介绍了一个新颖且具有挑战性的多式模式文档摘要基准（M-Docsum-Bench），该基准包括500个高质量的Arxiv论文，以及与人类偏爱相符的交织的多模式摘要。 M-Docsum-Bench是一项基于参考的生成任务，需要使用提供的参考图像来生成交织的图像文本摘要，从而同时评估复杂多模式文档方案中理解，推理，本地化和汇总的能力。为了促进此基准，我们开发了一个自动化框架来构建摘要并提出一种称为M-Doceval的细粒评估方法。此外，我们进一步开发了一个强大的摘要基线，即M-Docsum-7b，通过具有多种指导和偏好数据的进行性两阶段培训。在我们的M-Docsum台上的广泛结果表明，领先的LVLM努力保持连贯性并准确地将信息整合在长而相互交织的环境中，通常会在相似的图像和缺乏健壮性之间表现出混乱。值得注意的是，M-Docsum-7b与较大和闭合源模型相比（包括GPT-4O，Gemini Pro，Claude-3.5-Sonnet和Qwen2.5-VL-72B等），实现了最先进的性能，证明了LVLMS对改进的Interleaved Interleaved Interleaved Interleaved Intealed Image-Text的了解。代码，数据和模型可在此HTTPS URL上找到。

Title: StarFlow: Generating Structured Workflow Outputs From Sketch Images

• Authors: Patrice Bechard, Chao Wang, Amirhossein Abaskohi, Juan Rodriguez, Christopher Pal, David Vazquez, Spandana Gella, Sai Rajeswar, Perouz Taslakian
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.21889
• Pdf URL: https://arxiv.org/pdf/2503.21889
• Copy Paste: [[2503.21889]] StarFlow: Generating Structured Workflow Outputs From Sketch Images(https://arxiv.org/abs/2503.21889)
• Keywords: generation, generative
• Abstract: Workflows are a fundamental component of automation in enterprise platforms, enabling the orchestration of tasks, data processing, and system integrations. Despite being widely used, building workflows can be complex, often requiring manual configuration through low-code platforms or visual programming tools. To simplify this process, we explore the use of generative foundation models, particularly vision-language models (VLMs), to automatically generate structured workflows from visual inputs. Translating hand-drawn sketches or computer-generated diagrams into executable workflows is challenging due to the ambiguity of free-form drawings, variations in diagram styles, and the difficulty of inferring execution logic from visual elements. To address this, we introduce StarFlow, a framework for generating structured workflow outputs from sketches using vision-language models. We curate a diverse dataset of workflow diagrams -- including synthetic, manually annotated, and real-world samples -- to enable robust training and evaluation. We finetune and benchmark multiple vision-language models, conducting a series of ablation studies to analyze the strengths and limitations of our approach. Our results show that finetuning significantly enhances structured workflow generation, outperforming large vision-language models on this task.
• 摘要：工作流程是企业平台中自动化的基本组成部分，可以使任务，数据处理和系统集成的编排。尽管被广泛使用，但建筑工作流程可能很复杂，通常需要通过低编码平台或视觉编程工具进行手动配置。为了简化此过程，我们探讨了生成基础模型，尤其是视觉语言模型（VLM）的使用，以自动从视觉输入中生成结构化的工作流。由于自由图纸的模棱两可，图样式的变化以及从视觉元素推断执行逻辑的难度，将手绘草图或计算机生成的图转换为可执行的工作流程。为了解决这个问题，我们介绍了Starflow，这是一个框架，用于使用视觉模型从草图中生成结构化工作流量。我们策划了各种工作流程图的数据集，包括合成，手动注释和现实世界样本，以实现强大的培训和评估。我们对多种视觉语言模型进行了验证和基准测试，进行了一系列消融研究，以分析我们方法的优势和局限性。我们的结果表明，填充显着增强了结构化的工作流程的产生，在此任务上表现优于大型视觉模型。

Title: KernelFusion: Assumption-Free Blind Super-Resolution via Patch Diffusion

• Authors: Oliver Heinimann, Assaf Shocher, Tal Zimbalist, Michal Irani
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.21907
• Pdf URL: https://arxiv.org/pdf/2503.21907
• Copy Paste: [[2503.21907]] KernelFusion: Assumption-Free Blind Super-Resolution via Patch Diffusion(https://arxiv.org/abs/2503.21907)
• Keywords: super-resolution
• Abstract: Traditional super-resolution (SR) methods assume an ``ideal'' downscaling SR-kernel (e.g., bicubic downscaling) between the high-resolution (HR) image and the low-resolution (LR) image. Such methods fail once the LR images are generated differently. Current blind-SR methods aim to remove this assumption, but are still fundamentally restricted to rather simplistic downscaling SR-kernels (e.g., anisotropic Gaussian kernels), and fail on more complex (out of distribution) downscaling degradations. However, using the correct SR-kernel is often more important than using a sophisticated SR algorithm. In ``KernelFusion'' we introduce a zero-shot diffusion-based method that makes no assumptions about the kernel. Our method recovers the unique image-specific SR-kernel directly from the LR input image, while simultaneously recovering its corresponding HR image. KernelFusion exploits the principle that the correct SR-kernel is the one that maximizes patch similarity across different scales of the LR image. We first train an image-specific patch-based diffusion model on the single LR input image, capturing its unique internal patch statistics. We then reconstruct a larger HR image with the same learned patch distribution, while simultaneously recovering the correct downscaling SR-kernel that maintains this cross-scale relation between the HR and LR images. Empirical results show that KernelFusion vastly outperforms all SR baselines on complex downscaling degradations, where existing SotA Blind-SR methods fail miserably. By breaking free from predefined kernel assumptions, KernelFusion pushes Blind-SR into a new assumption-free paradigm, handling downscaling kernels previously thought impossible.
• 摘要：传统的超分辨率（SR）方法假设高分辨率（HR）图像和低分辨率（LR）图像之间的``理想'理想'理想'缩放尺寸SR-KERNEL（例如，双孔降尺度）。一旦LR图像的生成不同，这种方法就会失败。当前的盲型SR方法旨在消除此假设，但从根本上仍仅限于相当简单的降低缩小的sr-kernels（例如各向异性高斯核），并且会因更复杂的（不分布）降低降低降低而失败。但是，使用正确的SR-Kernel通常比使用复杂的SR算法更重要。在``kernelfusion''中引入了一种基于零射传扩散的方法，该方法对内核没有任何假设。我们的方法直接从LR输入图像中恢复了唯一的图像特异性SR-KERNEL，同时恢复了其相应的HR图像。 Kernelfusion利用了正确的SR-KERNEL是在LR图像的不同尺度上最大化贴片相似性的原理。我们首先在单个LR输入图像上训练基于图像的基于图像的扩散模型，从而捕获其独特的内部补丁统计。然后，我们以相同的学习贴片分布重建了较大的人力资源图像，同时恢复了正确的缩小缩放SR-KERNEL，该SR-KERNEL保持了HR和LR图像之间的这种跨尺度关系。经验结果表明，对于复杂的缩减降解，即可大大胜过所有SR基准，在这些降级降解上，现有的SOTA盲sR方法惨败。通过摆脱预定义的内核假设，Kernelfusion将Blind-SR推向了一个新的无假设范式，处理以前认为不可能的缩减内核。

Title: Flexible Moment-Invariant Bases from Irreducible Tensors

• Authors: Roxana Bujack, Emily Shinkle, Alice Allen, Tomas Suk, Nicholas Lubbers
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.21939
• Pdf URL: https://arxiv.org/pdf/2503.21939
• Copy Paste: [[2503.21939]] Flexible Moment-Invariant Bases from Irreducible Tensors(https://arxiv.org/abs/2503.21939)
• Keywords: generation
• Abstract: Moment invariants are a powerful tool for the generation of rotation-invariant descriptors needed for many applications in pattern detection, classification, and machine learning. A set of invariants is optimal if it is complete, independent, and robust against degeneracy in the input. In this paper, we show that the current state of the art for the generation of these bases of moment invariants, despite being robust against moment tensors being identically zero, is vulnerable to a degeneracy that is common in real-world applications, namely spherical functions. We show how to overcome this vulnerability by combining two popular moment invariant approaches: one based on spherical harmonics and one based on Cartesian tensor algebra.
• 摘要：力矩不变是在模式检测，分类和机器学习中生成许多应用所需的旋转不变描述符的强大工具。如果一组不变性是完整的，独立的，并且对输入中的堕落性是最佳的。在本文中，我们表明，尽管有强大的力矩张力量相同零，但在产生这些瞬间的这些基础的现行状态很容易受到在现实世界应用中常见的变性，即球形函数。我们通过结合两种流行的时刻不变方法来展示如何克服这一脆弱性：一种基于球形谐波，一种基于笛卡尔张量代数。

Title: Parametric Shadow Control for Portrait Generationin Text-to-Image Diffusion Models

• Authors: Haoming Cai, Tsung-Wei Huang, Shiv Gehlot, Brandon Y. Feng, Sachin Shah, Guan-Ming Su, Christopher Metzler
• Subjects: cs.CV, cs.AI, eess.IV
• Abstract URL: https://arxiv.org/abs/2503.21943
• Pdf URL: https://arxiv.org/pdf/2503.21943
• Copy Paste: [[2503.21943]] Parametric Shadow Control for Portrait Generationin Text-to-Image Diffusion Models(https://arxiv.org/abs/2503.21943)
• Keywords: generation
• Abstract: Text-to-image diffusion models excel at generating diverse portraits, but lack intuitive shadow control. Existing editing approaches, as post-processing, struggle to offer effective manipulation across diverse styles. Additionally, these methods either rely on expensive real-world light-stage data collection or require extensive computational resources for training. To address these limitations, we introduce Shadow Director, a method that extracts and manipulates hidden shadow attributes within well-trained diffusion models. Our approach uses a small estimation network that requires only a few thousand synthetic images and hours of training-no costly real-world light-stage data needed. Shadow Director enables parametric and intuitive control over shadow shape, placement, and intensity during portrait generation while preserving artistic integrity and identity across diverse styles. Despite training only on synthetic data built on real-world identities, it generalizes effectively to generated portraits with diverse styles, making it a more accessible and resource-friendly solution.
• 摘要：文本对图像扩散模型在产生多样化的肖像方面表现出色，但缺乏直观的阴影控制。现有的编辑方法，作为后处理，努力提供跨不同样式的有效操纵。此外，这些方法要么依赖于昂贵的现实世界中的轻型数据收集，要么需要大量的计算资源进行培训。为了解决这些局限性，我们介绍了Shadow Director，该方法在训练有素的扩散模型中提取并操纵隐藏的阴影属性。我们的方法使用一个小的估计网络，该网络仅需要几千个合成图像和训练时间，而不需要真实的现实世界中的光阶段数据。影子导演可以在肖像生成期间对阴影形状，放置和强度进行参数和直观的控制，同时保留各种风格的艺术完整性和身份。尽管仅对基于现实世界身份的合成数据进行培训，但它有效地将其概括为具有多种样式的肖像，使其成为更容易访问和资源友好的解决方案。

Title: NeRF-based Point Cloud Reconstruction using a Stationary Camera for Agricultural Applications

• Authors: Kibon Ku, Talukder Z Jubery, Elijah Rodriguez, Aditya Balu, Soumik Sarkar, Adarsh Krishnamurthy, Baskar Ganapathysubramanian
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.21958
• Pdf URL: https://arxiv.org/pdf/2503.21958
• Copy Paste: [[2503.21958]] NeRF-based Point Cloud Reconstruction using a Stationary Camera for Agricultural Applications(https://arxiv.org/abs/2503.21958)
• Keywords: generation
• Abstract: This paper presents a NeRF-based framework for point cloud (PCD) reconstruction, specifically designed for indoor high-throughput plant phenotyping facilities. Traditional NeRF-based reconstruction methods require cameras to move around stationary objects, but this approach is impractical for high-throughput environments where objects are rapidly imaged while moving on conveyors or rotating pedestals. To address this limitation, we develop a variant of NeRF-based PCD reconstruction that uses a single stationary camera to capture images as the object rotates on a pedestal. Our workflow comprises COLMAP-based pose estimation, a straightforward pose transformation to simulate camera movement, and subsequent standard NeRF training. A defined Region of Interest (ROI) excludes irrelevant scene data, enabling the generation of high-resolution point clouds (10M points). Experimental results demonstrate excellent reconstruction fidelity, with precision-recall analyses yielding an F-score close to 100.00 across all evaluated plant objects. Although pose estimation remains computationally intensive with a stationary camera setup, overall training and reconstruction times are competitive, validating the method's feasibility for practical high-throughput indoor phenotyping applications. Our findings indicate that high-quality NeRF-based 3D reconstructions are achievable using a stationary camera, eliminating the need for complex camera motion or costly imaging equipment. This approach is especially beneficial when employing expensive and delicate instruments, such as hyperspectral cameras, for 3D plant phenotyping. Future work will focus on optimizing pose estimation techniques and further streamlining the methodology to facilitate seamless integration into automated, high-throughput 3D phenotyping pipelines.
• 摘要：本文介绍了基于NERF的点云（PCD）重建框架，该框架专为室内高通量工厂表型设施而设计。传统的基于NERF的重建方法需要摄像机在固定物体周围移动，但是对于高通量环境，这种方法是不切实际的，在高通量环境中，在移动输送机或旋转基座时，对象迅速成像。为了解决此限制，我们开发了基于NERF的PCD重建的变体，该变体使用单个固定相机捕获图像，因为该物体在基座上旋转。我们的工作流程包括基于COLMAP的姿势估计，直接的姿势转换以模拟相机运动以及随后的标准NERF培训。定义的感兴趣区域（ROI）不包括无关紧要的场景数据，从而能够生成高分辨率点云（10m点）。实验结果证明了出色的重建保真度，精确恢复分析在所有评估的植物对象中产生接近100.00的F分数。尽管姿势估计通过固定相机设置在计算中保持密集，但是整体培训和重建时间具有竞争力，从而验证了该方法对实用高通量室内表型应用程序的可行性。我们的发现表明，使用固定摄像头可以实现高质量的基于NERF的3D重建，从而消除了对复杂的相机运动或昂贵的成像设备的需求。当使用昂贵且精致的仪器（例如高光谱摄像机）进行3D植物表型时，这种方法尤其有益。未来的工作将着重于优化姿势估计技术，并进一步简化方法，以促进无缝集成到自动化的高通量3D表型管道中。

Title: Q-MambaIR: Accurate Quantized Mamba for Efficient Image Restoration

• Authors: Yujie Chen, Haotong Qin, Zhang Zhang, Michelo Magno, Luca Benini, Yawei Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.21970
• Pdf URL: https://arxiv.org/pdf/2503.21970
• Copy Paste: [[2503.21970]] Q-MambaIR: Accurate Quantized Mamba for Efficient Image Restoration(https://arxiv.org/abs/2503.21970)
• Keywords: restoration
• Abstract: State-Space Models (SSMs) have attracted considerable attention in Image Restoration (IR) due to their ability to scale linearly sequence length while effectively capturing long-distance dependencies. However, deploying SSMs to edge devices is challenging due to the constraints in memory, computing capacity, and power consumption, underscoring the need for efficient compression strategies. While low-bit quantization is an efficient model compression strategy for reducing size and accelerating IR tasks, SSM suffers substantial performance drops at ultra-low bit-widths (2-4 bits), primarily due to outliers that exacerbate quantization error. To address this challenge, we propose Q-MambaIR, an accurate, efficient, and flexible Quantized Mamba for IR tasks. Specifically, we introduce a Statistical Dynamic-balancing Learnable Scalar (DLS) to dynamically adjust the quantization mapping range, thereby mitigating the peak truncation loss caused by extreme values. Furthermore, we design a Range-floating Flexible Allocator (RFA) with an adaptive threshold to flexibly round values. This approach preserves high-frequency details and maintains the SSM's feature extraction capability. Notably, RFA also enables pre-deployment weight quantization, striking a balance between computational efficiency and model accuracy. Extensive experiments on IR tasks demonstrate that Q-MambaIR consistently outperforms existing quantized SSMs, achieving much higher state-of-the-art (SOTA) accuracy results with only a negligible increase in training computation and storage saving.
• 摘要：国家空间模型（SSM）由于能力缩放线性序列长度而有效地捕获长距离依赖性，因此在图像恢复（IR）中引起了很大的关注。但是，由于内存，计算能力和功耗的限制，将SSM部署到Edge设备是具有挑战性的，强调了对有效压缩策略的需求。尽管低位量化是一种有效的模型压缩策略，可降低大小和加速IR任务，但SSM在超低位宽度（2-4位）下遭受了大量的性能下降，这主要是由于加剧量化量化错误的异常值所致。为了应对这一挑战，我们提出了Q-Mambair，这是用于IR任务的准确，高效且灵活的MAMBA。具体而言，我们引入了一个统计动态平衡的可学习标量（DLS），以动态调整量化映射范围，从而减轻由极端值引起​​的峰值截断损失。此外，我们设计了一个具有自适应阈值以灵活的圆形值的浮动柔性分配器（RFA）。这种方法可以保留高频细节并保持SSM的特征提取能力。值得注意的是，RFA还可以实现前部的重量量化，从而在计算效率和模型准确性之间达到平衡。关于IR任务的广泛实验表明，Q-Mambair始终胜过现有的量化SSM，实现了更高的最先进（SOTA）精度结果，而训练计算和存储节省仅可忽略不计。

Title: RocketPPA: Ultra-Fast LLM-Based PPA Estimator at Code-Level Abstraction

• Authors: Armin Abdollahi, Mehdi Kamal, Massoud Pedram
• Subjects: cs.LG, cs.SE
• Abstract URL: https://arxiv.org/abs/2503.21971
• Pdf URL: https://arxiv.org/pdf/2503.21971
• Copy Paste: [[2503.21971]] RocketPPA: Ultra-Fast LLM-Based PPA Estimator at Code-Level Abstraction(https://arxiv.org/abs/2503.21971)
• Keywords: generation
• Abstract: Large language models have recently transformed hardware design, yet bridging the gap between code synthesis and PPA (power, performance, and area) estimation remains a challenge. In this work, we introduce a novel framework that leverages a 21k dataset of thoroughly cleaned and synthesizable Verilog modules, each annotated with detailed power, delay, and area metrics. By employing chain-of-thought techniques, we automatically debug and curate this dataset to ensure high fidelity in downstream applications. We then fine-tune CodeLlama using LoRA-based parameter-efficient methods, framing the task as a regression problem to accurately predict PPA metrics from Verilog code. Furthermore, we augment our approach with a mixture-of-experts architecture-integrating both LoRA and an additional MLP expert layer-to further refine predictions. Experimental results demonstrate significant improvements: power estimation accuracy is enhanced by 5.9% at a 20% error threshold and by 7.2% at a 10% threshold, delay estimation improves by 5.1% and 3.9%, and area estimation sees gains of 4% and 7.9% for the 20% and 10% thresholds, respectively. Notably, the incorporation of the mixture-of-experts module contributes an additional 3--4% improvement across these tasks. Our results establish a new benchmark for PPA-aware Verilog generation, highlighting the effectiveness of our integrated dataset and modeling strategies for next-generation EDA workflows.
• 摘要：大型语言模型最近改变了硬件设计，但是弥合了代码合成与PPA（功率，性能和区域）估计之间的差距仍然是一个挑战。在这项工作中，我们介绍了一个新颖的框架，该框架利用21k数据集的彻底清洁和合成的Verilog模块，每个模块都以详细的功率，延迟和面积指标注释。通过采用思想链技术，我们会自动调试并策划该数据集，以确保在下游应用程序中高保真。然后，我们使用基于LORA的参数效率方法对Codellama进行微调，将任务作为回归问题构建，以准确预测Verilog代码中的PPA指标。此外，我们通过融合洛拉（Lora）和额外的MLP专家层来进一步完善预测，以增强体系结构的混合物来增强我们的方法。实验结果表明了显着的改进：在误差阈值20％的情况下，功率估计精度提高了5.9％，在10％阈值时提高了7.2％，延迟估计提高了5.1％和3.9％，面积估计分别为20％和10％的阈值的增长率为4％和7.9％。值得注意的是，融合到专家的混合物模块在这些任务中贡献了3--4％的改善。我们的结果为PPA感知的Verilog生成建立了一个新的基准测试，突出了我们集成数据集的有效性和对下一代EDA工作流的建模策略的有效性。

Title: Harmonizing Visual Representations for Unified Multimodal Understanding and Generation

• Authors: Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Zhonghua Wu, Qingyi Tao, Wentao Liu, Wei Li, Chen Change Loy
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.21979
• Pdf URL: https://arxiv.org/pdf/2503.21979
• Copy Paste: [[2503.21979]] Harmonizing Visual Representations for Unified Multimodal Understanding and Generation(https://arxiv.org/abs/2503.21979)
• Keywords: generation
• Abstract: Unifying visual understanding and generation within a single multimodal framework remains a significant challenge, as the two inherently heterogeneous tasks require representations at different levels of granularity. Current approaches that utilize vector quantization (VQ) or variational autoencoders (VAE) for unified visual representation prioritize intrinsic imagery features over semantics, compromising understanding performance. In this work, we take inspiration from masked image modelling (MIM) that learns rich semantics via a mask-and-reconstruct pre-training and its successful extension to masked autoregressive (MAR) image generation. A preliminary study on the MAR encoder's representation reveals exceptional linear probing accuracy and precise feature response to visual concepts, which indicates MAR's potential for visual understanding tasks beyond its original generation role. Based on these insights, we present \emph{Harmon}, a unified autoregressive framework that harmonizes understanding and generation tasks with a shared MAR encoder. Through a three-stage training procedure that progressively optimizes understanding and generation capabilities, Harmon achieves state-of-the-art image generation results on the GenEval, MJHQ30K and WISE benchmarks while matching the performance of methods with dedicated semantic encoders (e.g., Janus) on image understanding benchmarks. Our code and models will be available at this https URL.
• 摘要：在单个多模式框架内统一视觉理解和产生仍然是一个重大挑战，因为这两个固有的异质任务需要在不同级别的粒度上表示。利用矢量量化（VQ）或变化自动编码器（VAE）的当前方法用于统一的视觉表示优先级以于语义上的固有图像特征优先，从而损害了理解性能。在这项工作中，我们从蒙面图像建模（MIM）中汲取灵感，通过掩护和重新构建的预训练，并成功扩展到掩盖自动回归（MAR）图像生成。对MAR编码器表示的初步研究揭示了出色的线性探测精度和对视觉概念的精确特征响应，这表明MAR超出了其原始生成角色的视觉理解任务的潜力。基于这些见解，我们提出了\ emph {Harmon}，这是一个统一的自动回归框架，可与共享的MAR编码器协调理解和生成任务。通过一个逐步优化理解和产生能力的三阶段训练程序，Harmon在Geneval，MJHQ30K和Wise基准测试中实现了最新的图像生成结果，同时在图像理解基准的情况下与奉献的语义编码器（例如Janus）相匹配的方法的性能（例如Janus）。我们的代码和模型将在此HTTPS URL上可用。

Title: Improving Equivariant Networks with Probabilistic Symmetry Breaking

• Authors: Hannah Lawrence, Vasco Portilheiro, Yan Zhang, Sékou-Oumar Kaba
• Subjects: cs.LG, stat.ML
• Abstract URL: https://arxiv.org/abs/2503.21985
• Pdf URL: https://arxiv.org/pdf/2503.21985
• Copy Paste: [[2503.21985]] Improving Equivariant Networks with Probabilistic Symmetry Breaking(https://arxiv.org/abs/2503.21985)
• Keywords: generative
• Abstract: Equivariance encodes known symmetries into neural networks, often enhancing generalization. However, equivariant networks cannot break symmetries: the output of an equivariant network must, by definition, have at least the same self-symmetries as the input. This poses an important problem, both (1) for prediction tasks on domains where self-symmetries are common, and (2) for generative models, which must break symmetries in order to reconstruct from highly symmetric latent spaces. This fundamental limitation can be addressed by considering equivariant conditional distributions, instead of equivariant functions. We present novel theoretical results that establish necessary and sufficient conditions for representing such distributions. Concretely, this representation provides a practical framework for breaking symmetries in any equivariant network via randomized canonicalization. Our method, SymPE (Symmetry-breaking Positional Encodings), admits a simple interpretation in terms of positional encodings. This approach expands the representational power of equivariant networks while retaining the inductive bias of symmetry, which we justify through generalization bounds. Experimental results demonstrate that SymPE significantly improves performance of group-equivariant and graph neural networks across diffusion models for graphs, graph autoencoders, and lattice spin system modeling.
• 摘要：模棱两可将已知对称性编码到神经网络中，通常会增强概括。但是，均值网络不能打破对称性：按照定义，等效网络的输出必须至少具有与输入相同的自我对称性。这构成了一个重要的问题，这两种问题（1）对于自我对称是常见的域上的预测任务，以及（2）对于生成模型，必须打破对称性才能从高度对称的潜在空间重建。可以通过考虑模棱两可的条件分布而不是均衡功能来解决这种基本限制。我们提出了新的理论结果，这些结果建立了代表此类分布的必要条件。具体而言，该表示形式提供了一个实用的框架，可以通过随机规范化在任何e夫网络中打破对称性。我们的方法，Sympe（破坏对称性的位置编码），在位置编码方面接受了一种简单的解释。这种方法扩大了模棱两可的网络的代表力，同时保留了对称性的电感偏见，我们通过概括范围证明这是合理的。实验结果表明，Sympe显着提高了图形，图形自动编码器和晶格旋转系统建模的扩散模型的组等级和图神经网络的性能。

Title: BOOTPLACE: Bootstrapped Object Placement with Detection Transformers

• Authors: Hang Zhou, Xinxin Zuo, Rui Ma, Li Cheng
• Subjects: cs.CV, cs.AI, cs.GR
• Abstract URL: https://arxiv.org/abs/2503.21991
• Pdf URL: https://arxiv.org/pdf/2503.21991
• Copy Paste: [[2503.21991]] BOOTPLACE: Bootstrapped Object Placement with Detection Transformers(https://arxiv.org/abs/2503.21991)
• Keywords: generative
• Abstract: In this paper, we tackle the copy-paste image-to-image composition problem with a focus on object placement learning. Prior methods have leveraged generative models to reduce the reliance for dense supervision. However, this often limits their capacity to model complex data distributions. Alternatively, transformer networks with a sparse contrastive loss have been explored, but their over-relaxed regularization often leads to imprecise object placement. We introduce BOOTPLACE, a novel paradigm that formulates object placement as a placement-by-detection problem. Our approach begins by identifying suitable regions of interest for object placement. This is achieved by training a specialized detection transformer on object-subtracted backgrounds, enhanced with multi-object supervisions. It then semantically associates each target compositing object with detected regions based on their complementary characteristics. Through a boostrapped training approach applied to randomly object-subtracted images, our model enforces meaningful placements through extensive paired data augmentation. Experimental results on established benchmarks demonstrate BOOTPLACE's superior performance in object repositioning, markedly surpassing state-of-the-art baselines on Cityscapes and OPA datasets with notable improvements in IOU scores. Additional ablation studies further showcase the compositionality and generalizability of our approach, supported by user study evaluations.
• 摘要：在本文中，我们将重点放在对象放置学习的重点上，解决拷贝性模拟图像到图像的组成问题。先前的方法已利用生成模型来减少对密集监督的依赖。但是，这通常会限制他们对复杂数据分布进行建模的能力。另外，已经探索了具有稀疏对比度损失的变压器网络，但是它们过度删除的正则化通常会导致对象放置不精确。我们介绍了Bootplace，这是一种新颖的范式，将对象放置作为逐个定位问题。我们的方法首先要确定适合对象放置的关注区域。这是通过在对象提取的背景上训练专门的检测变压器来实现的，并通过多对象的监督增强。然后，它根据其互补特性将每个目标组合对象与检测区域相关联。通过应用于随机对象提取的图像的boost训练方法，我们的模型通过大量的配对数据增强来实施有意义的位置。对既定基准的实验结果表明，Bootplace在对象重新定位方面的出色性能，显着超过了城市景观和OPA数据集的最先进的基线，并且具有显着的改进。其他消融研究进一步展示了我们方法的组成性和概括性，并得到用户研究评估的支持。

Title: AGILE: A Diffusion-Based Attention-Guided Image and Label Translation for Efficient Cross-Domain Plant Trait Identification

• Authors: Earl Ranario, Lars Lundqvist, Heesup Yun, Brian N. Bailey, J. Mason Earles
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22019
• Pdf URL: https://arxiv.org/pdf/2503.22019
• Copy Paste: [[2503.22019]] AGILE: A Diffusion-Based Attention-Guided Image and Label Translation for Efficient Cross-Domain Plant Trait Identification(https://arxiv.org/abs/2503.22019)
• Keywords: generation, generative
• Abstract: Semantically consistent cross-domain image translation facilitates the generation of training data by transferring labels across different domains, making it particularly useful for plant trait identification in agriculture. However, existing generative models struggle to maintain object-level accuracy when translating images between domains, especially when domain gaps are significant. In this work, we introduce AGILE (Attention-Guided Image and Label Translation for Efficient Cross-Domain Plant Trait Identification), a diffusion-based framework that leverages optimized text embeddings and attention guidance to semantically constrain image translation. AGILE utilizes pretrained diffusion models and publicly available agricultural datasets to improve the fidelity of translated images while preserving critical object semantics. Our approach optimizes text embeddings to strengthen the correspondence between source and target images and guides attention maps during the denoising process to control object placement. We evaluate AGILE on cross-domain plant datasets and demonstrate its effectiveness in generating semantically accurate translated images. Quantitative experiments show that AGILE enhances object detection performance in the target domain while maintaining realism and consistency. Compared to prior image translation methods, AGILE achieves superior semantic alignment, particularly in challenging cases where objects vary significantly or domain gaps are substantial.
• 摘要：语义上一致的跨域图像翻译通过将标签转移到不同领域，从而有助于生成训练数据，从而使其对于农业中的植物特征识别特别有用。但是，现有的生成模型在转换域之间的图像时努力保持对象级的准确性，尤其是当域间隙显着时。在这项工作中，我们引入了敏捷（注意引导的图像和标签翻译，以进行有效的跨域植物特征识别），这是一个基于扩散的框架，利用优化的文本嵌入和注意力指导来限制图像翻译。 Agile利用预处理的扩散模型和公开可用的农业数据集来提高翻译图像的保真度，同时保留关键对象语义。我们的方法优化了文本嵌入，以增强源图像和目标图像之间的对应关系，并在降级过程中指导注意力图以控制对象放置。我们在跨域植物数据集上评估了敏捷，并证明了其在生成语义精确翻译图像中的有效性。定量实验表明，敏捷可增强目标域中的对象检测性能，同时保持现实主义和一致性。与先前的图像翻译方法相比，敏捷实现了上级的语义比对，尤其是在物体差异很大或域间隙差异很大的挑战性情况下。

Title: Arch-LLM: Taming LLMs for Neural Architecture Generation via Unsupervised Discrete Representation Learning

• Authors: Deshani Geethika Poddenige, Sachith Seneviratne, Damith Senanayake, Mahesan Niranjan, PN Suganthan, Saman Halgamuge
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.22063
• Pdf URL: https://arxiv.org/pdf/2503.22063
• Copy Paste: [[2503.22063]] Arch-LLM: Taming LLMs for Neural Architecture Generation via Unsupervised Discrete Representation Learning(https://arxiv.org/abs/2503.22063)
• Keywords: generation, generative
• Abstract: Unsupervised representation learning has been widely explored across various modalities, including neural architectures, where it plays a key role in downstream applications like Neural Architecture Search (NAS). These methods typically learn an unsupervised representation space before generating/ sampling architectures for the downstream search. A common approach involves the use of Variational Autoencoders (VAEs) to map discrete architectures onto a continuous representation space, however, sampling from these spaces often leads to a high percentage of invalid or duplicate neural architectures. This could be due to the unnatural mapping of inherently discrete architectural space onto a continuous space, which emphasizes the need for a robust discrete representation of these architectures. To address this, we introduce a Vector Quantized Variational Autoencoder (VQ-VAE) to learn a discrete latent space more naturally aligned with the discrete neural architectures. In contrast to VAEs, VQ-VAEs (i) map each architecture into a discrete code sequence and (ii) allow the prior to be learned by any generative model rather than assuming a normal distribution. We then represent these architecture latent codes as numerical sequences and train a text-to-text model leveraging a Large Language Model to learn and generate sequences representing architectures. We experiment our method with Inception/ ResNet-like cell-based search spaces, namely NAS-Bench-101 and NAS-Bench-201. Compared to VAE-based methods, our approach improves the generation of valid and unique architectures by over 80% on NASBench-101 and over 8% on NASBench-201. Finally, we demonstrate the applicability of our method in NAS employing a sequence-modeling-based NAS algorithm.
• 摘要：无监督的表示学习已在包括神经体系结构在内的各种方式中广泛探索，在这些方式中，它在神经体系结构搜索（NAS）等下游应用中起着关键作用。这些方法通常在生成/采样体系结构以进行下游搜索之前学习无监督的表示空间。一种常见的方法涉及使用变异自动编码器（VAE）将离散体系结构映射到连续的表示空间上，但是，从这些空间进行采样通常会导致高度无效或重复的神经体系结构。这可能是由于将固有离散的架构空间不自然的映射到连续空间上，这强调了对这些架构的强大离散表示的需求。为了解决这个问题，我们介绍了量化的量化变量自动编码器（VQ-VAE），以学习一个离散的神经体系结构更自然地对齐的离散潜在空间。与VAE相反，VQ-VAE（i）将每个体系结构映射到离散的代码序列中，并且（ii）允许任何生成模型都学到之前，而不是假设正态分布。然后，我们将这些架构潜在代码表示为数值序列，并训练文本到文本模型，该模型利用大型语言模型来学习和生成代表体系结构的序列。我们使用Inception/ Resnet样细胞的搜索空间（即NAS Bench-101和NAS Bench-2010）实验我们的方法。与基于VAE的方法相比，我们的方法在NASBENCH-101上将有效和独特体系结构的生成增长了80％，而NASBENCH-2010则超过8％。最后，我们使用基于序列模型的NAS算法的NAS中证明了我们的方法的适用性。

Title: A Semantic-Enhanced Heterogeneous Graph Learning Method for Flexible Objects Recognition

• Authors: Kunshan Yang, Wenwei Luo, Yuguo Hu, Jiafu Yan, Mengmeng Jing, Lin Zuo
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22079
• Pdf URL: https://arxiv.org/pdf/2503.22079
• Copy Paste: [[2503.22079]] A Semantic-Enhanced Heterogeneous Graph Learning Method for Flexible Objects Recognition(https://arxiv.org/abs/2503.22079)
• Keywords: generation
• Abstract: Flexible objects recognition remains a significant challenge due to its inherently diverse shapes and sizes, translucent attributes, and subtle inter-class differences. Graph-based models, such as graph convolution networks and graph vision models, are promising in flexible objects recognition due to their ability of capturing variable relations within the flexible objects. These methods, however, often focus on global visual relationships or fail to align semantic and visual information. To alleviate these limitations, we propose a semantic-enhanced heterogeneous graph learning method. First, an adaptive scanning module is employed to extract discriminative semantic context, facilitating the matching of flexible objects with varying shapes and sizes while aligning semantic and visual nodes to enhance cross-modal feature correlation. Second, a heterogeneous graph generation module aggregates global visual and local semantic node features, improving the recognition of flexible objects. Additionally, We introduce the FSCW, a large-scale flexible dataset curated from existing sources. We validate our method through extensive experiments on flexible datasets (FDA and FSCW), and challenge benchmarks (CIFAR-100 and ImageNet-Hard), demonstrating competitive performance.
• 摘要：灵活的对象识别仍然是一个重大挑战，因为它固有的各种形状和大小，半透明属性和细微的阶层差异。基于图形的模型（例如图形卷积网络和图形视觉模型）在灵活对象识别中有希望，因为它们在灵活对象中捕获可变关系的能力。但是，这些方法通常集中在全球视觉关系上，或者无法对齐语义和视觉信息。为了减轻这些局限性，我们提出了一种语义增强的异质图学习方法。首先，使用自适应扫描模块来提取歧视性语义上下文，从而促进具有不同形状和尺寸的柔性对象的匹配，同时使语义和视觉节点对齐以增强交叉模式特征相关性。其次，异质图生成模块汇总了全局视觉和局部语义节点特征，从而提高了灵活对象的识别。此外，我们介绍了FSCW，这是一个从现有来源策划的大规模灵活数据集。我们通过在灵活数据集（FDA和FSCW）上进行广泛的实验来验证我们的方法，并挑战基准测试（CIFAR-100和Imagenet-Hard），表明了竞争性能。

Title: Detecting Localized Deepfake Manipulations Using Action Unit-Guided Video Representations

• Authors: Tharun Anand, Siva Sankar, Pravin Nair
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22121
• Pdf URL: https://arxiv.org/pdf/2503.22121
• Copy Paste: [[2503.22121]] Detecting Localized Deepfake Manipulations Using Action Unit-Guided Video Representations(https://arxiv.org/abs/2503.22121)
• Keywords: generation, generative
• Abstract: With rapid advancements in generative modeling, deepfake techniques are increasingly narrowing the gap between real and synthetic videos, raising serious privacy and security concerns. Beyond traditional face swapping and reenactment, an emerging trend in recent state-of-the-art deepfake generation methods involves localized edits such as subtle manipulations of specific facial features like raising eyebrows, altering eye shapes, or modifying mouth expressions. These fine-grained manipulations pose a significant challenge for existing detection models, which struggle to capture such localized variations. To the best of our knowledge, this work presents the first detection approach explicitly designed to generalize to localized edits in deepfake videos by leveraging spatiotemporal representations guided by facial action units. Our method leverages a cross-attention-based fusion of representations learned from pretext tasks like random masking and action unit detection, to create an embedding that effectively encodes subtle, localized changes. Comprehensive evaluations across multiple deepfake generation methods demonstrate that our approach, despite being trained solely on the traditional FF+ dataset, sets a new benchmark in detecting recent deepfake-generated videos with fine-grained local edits, achieving a $20\%$ improvement in accuracy over current state-of-the-art detection methods. Additionally, our method delivers competitive performance on standard datasets, highlighting its robustness and generalization across diverse types of local and global forgeries.
• 摘要：随着生成建模方面的快速发展，Deepfake技术越来越缩小真实视频和合成视频之间的差距，从而提出了严重的隐私和安全问题。除了传统的面部交换和重演外，最近最新的深层生成方法的新兴趋势还涉及局部编辑，例如对特定面部特征的微妙操纵，例如抬起眉毛，改变眼睛形状或修饰口腔表情。这些细粒度的操纵对现有检测模型构成了重大挑战，这些模型难以捕获这种局部变化。据我们所知，这项工作介绍了通过利用面部动作单元指导的时空表示，明确设计了旨在推广到Deepfake视频中的本地化编辑的第一种检测方法。我们的方法利用了从借口任务（如随机掩蔽和动作单元检测）中学到的表示形式的跨注意融合，以创建有效编码微妙的局部变化的嵌入式。对多种深层生成方法进行的全面评估表明，尽管我们仅接受了传统的FF+数据集培训，但在检测最新的DeepFake生成视频的新基准方面，具有精细的本地编辑，可实现20美元的准确性，比目前的现状检测方法获得了20美元的准确性。此外，我们的方法在标准数据集上提供了竞争性能，突出了其在不同类型的本地和全球伪造之间的稳健性和概括性。

Title: Semantic segmentation for building houses from wooden cubes

• Authors: Ivan Beleacov
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22125
• Pdf URL: https://arxiv.org/pdf/2503.22125
• Copy Paste: [[2503.22125]] Semantic segmentation for building houses from wooden cubes(https://arxiv.org/abs/2503.22125)
• Keywords: generation
• Abstract: Automated construction is one of the most promising areas that can improve efficiency, reduce costs and minimize errors in the process of building construction. In this paper, a comparative analysis of three neural network models for semantic segmentation, U-Net(light), LinkNet and PSPNet, is performed. Two specialized datasets with images of houses built from wooden cubes were created for the experiments. The first dataset contains 4 classes (background, foundation, walls, roof ) and is designed for basic model evaluation, while the second dataset includes 44 classes where each cube is labeled as a separate object. The models were trained with the same hyperparameters and their accuracy was evaluated using MeanIoU and F1 Score metrics. According to the results obtained, U-Net(light) showed the best performance with 78% MeanIoU and 87% F1 Score on the first dataset and 17% and 25% respectively on the second dataset. The poor results on the second dataset are due to the limited amount of data, the complexity of the partitioning and the imbalance of classes, making it difficult to accurately select individual cubes. In addition, overtraining was observed in all experiments, manifested by high accuracy on the training dataset and its significant decrease on the validation dataset. The present work is the basis for the development of algorithms for automatic generation of staged building plans, which can be further scaled to design complete buildings. Future research is planned to extend the datasets and apply methods to combat overfitting (L1/L2 regularization, Early Stopping). The next stage of work will be the development of algorithms for automatic generation of a step-by-step plan for building houses from cubes using manipulators. Index Terms-Deep Learning, Computer vision, CNN, Semantic segmentation, Construction materials.
• 摘要：自动化结构是可以提高效率，降低成本并最大程度地减少建筑物建设过程中的错误的最有希望的领域之一。在本文中，对三个用于语义分割的神经网络模型进行了比较分析，u-net（Light），linknet和pspNet。为实验创建了两个专门的数据集，其中包含用木箱建造的房屋图像。第一个数据集包含4个类（背景，基础，墙壁，屋顶），设计用于基本模型评估，而第二个数据集则包含44个类，其中每个立方体都标记为单独的对象。使用相同的超参数训练了模型，并使用Meaniou和F1评分指标评估其精度。根据获得的结果，U-NET（LIGHT）在第一个数据集中表现出最佳性能，而第一个数据集的含量为78％，F1分别为17％和25％。第二个数据集的结果较差是由于数据量有限，分区的复杂性和类不平衡，因此难以准确选择单个立方体。此外，在所有实验中都观察到过度训练，这表现为训练数据集的高精度及其在验证数据集上的显着降低。目前的工作是开发自动生成分阶段建筑计划算法的基础，可以将其进一步扩展到设计完整的建筑物。未来的研究计划扩展数据集并应用方法来对抗过度拟合（L1/L2正则化，早期停止）。下一阶段的工作将是开发算法，以自动生成使用操纵器来建造立方体建造房屋的逐步计划。索引术语深度学习，计算机视觉，CNN，语义细分，建筑材料。

Title: Tokenization of Gaze Data

• Authors: Tim Rolff, Jurik Karimian, Niklas Hypki, Susanne Schmidt, Markus Lappe, Frank Steinicke
• Subjects: cs.LG, cs.CL, cs.CV, cs.HC
• Abstract URL: https://arxiv.org/abs/2503.22145
• Pdf URL: https://arxiv.org/pdf/2503.22145
• Copy Paste: [[2503.22145]] Tokenization of Gaze Data(https://arxiv.org/abs/2503.22145)
• Keywords: generation, generative
• Abstract: A considerable part of the performance of today's large language models (LLM's) and multimodal large language models (MLLM's) depends on their tokenization strategies. While tokenizers are extensively researched for textual and visual input, there is no research on tokenization strategies for gaze data due to its nature. However, a corresponding tokenization strategy would allow using the vision capabilities of pre-trained MLLM's for gaze data, for example, through fine-tuning. In this paper, we aim to close this research gap by analyzing five different tokenizers for gaze data on three different datasets for the forecasting and generation of gaze data through LLMs (cf.~\cref{fig:teaser}). We evaluate the tokenizers regarding their reconstruction and compression abilities. Further, we train an LLM for each tokenization strategy, measuring its generative and predictive performance. Overall, we found that a quantile tokenizer outperforms all others in predicting the gaze positions and k-means is best when predicting gaze velocities.
• 摘要：当今大型语言模型（LLM）和多模式大语模型（MLLM）表现的相当大的一部分取决于其令牌化策略。尽管对文本和视觉输入进行了广泛的研究，但由于其性质，目前尚无关于凝视数据的令牌化策略的研究。但是，相应的令牌化策略将允许使用预先训练的MLLM的视觉功能进行凝视数据，例如，通过微调。在本文中，我们的目标是通过分析五个不同的引物数据来弥合这一研究差距，以在三个不同的数据集上进行凝视数据，以通过LLMS通过LLMS进行预测和生成（参见〜\ cref {fig：fig：teaser}）。我们评估了令牌有关其重建和压缩能力的评估。此外，我们为每个令牌化策略训练LLM，以衡量其生成和预测性能。总体而言，我们发现，在预测凝视速度时，最好的分位数令牌在预测凝视位置和K-均值方面表现最好。

Title: Spatial Transport Optimization by Repositioning Attention Map for Training-Free Text-to-Image Synthesis

• Authors: Woojung Han, Yeonkyung Lee, Chanyoung Kim, Kwanghyun Park, Seong Jae Hwang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22168
• Pdf URL: https://arxiv.org/pdf/2503.22168
• Copy Paste: [[2503.22168]] Spatial Transport Optimization by Repositioning Attention Map for Training-Free Text-to-Image Synthesis(https://arxiv.org/abs/2503.22168)
• Keywords: generation
• Abstract: Diffusion-based text-to-image (T2I) models have recently excelled in high-quality image generation, particularly in a training-free manner, enabling cost-effective adaptability and generalization across diverse tasks. However, while the existing methods have been continuously focusing on several challenges, such as "missing objects" and "mismatched attributes," another critical issue of "mislocated objects" remains where generated spatial positions fail to align with text prompts. Surprisingly, ensuring such seemingly basic functionality remains challenging in popular T2I models due to the inherent difficulty of imposing explicit spatial guidance via text forms. To address this, we propose STORM (Spatial Transport Optimization by Repositioning Attention Map), a novel training-free approach for spatially coherent T2I synthesis. STORM employs Spatial Transport Optimization (STO), rooted in optimal transport theory, to dynamically adjust object attention maps for precise spatial adherence, supported by a Spatial Transport (ST) Cost function that enhances spatial understanding. Our analysis shows that integrating spatial awareness is most effective in the early denoising stages, while later phases refine details. Extensive experiments demonstrate that STORM surpasses existing methods, effectively mitigating mislocated objects while improving missing and mismatched attributes, setting a new benchmark for spatial alignment in T2I synthesis.
• 摘要：基于扩散的文本图像（T2I）模型最近在高质量的图像生成方面表现出色，尤其是在无训练的方式上，实现了具有成本效益的适应性和跨不同任务的概括。但是，尽管现有方法一直关注几个挑战，例如“丢失对象”和“错配属性”，但“错误分配对象”的另一个关键问题仍然是生成的空间位置无法与文本提示保持一致的地方。令人惊讶的是，由于通过文本形式强加明确的空间指导的固有困难，确保这种看似基本的功能在流行的T2I模型中仍然具有挑战性。为了解决这个问题，我们提出了风暴（通过重新定位注意图的空间传输优化），这是一种新型的无训练方法，用于空间相干T2I合成。 Storm采用植根于最佳传输理论的空间传输优化（STO），以动态调整对象注意图，以确切的空间依从性，并由空间传输（ST）成本函数支持，从而增强了空间理解。我们的分析表明，整合空间意识在早期的deno阶段中最有效，而后期阶段则完善了细节。广泛的实验表明，风暴超过了现有方法，有效地减轻了错误的对象，同时改善了缺失和不匹配的属性，为T2i合成中的空间对齐设定了新的基准。

Title: An Empirical Study of Validating Synthetic Data for Text-Based Person Retrieval

• Authors: Min Cao, ZiYin Zeng, YuXin Lu, Mang Ye, Dong Yi, Jinqiao Wang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22171
• Pdf URL: https://arxiv.org/pdf/2503.22171
• Copy Paste: [[2503.22171]] An Empirical Study of Validating Synthetic Data for Text-Based Person Retrieval(https://arxiv.org/abs/2503.22171)
• Keywords: generation, generative
• Abstract: Data plays a pivotal role in Text-Based Person Retrieval (TBPR) research. Mainstream research paradigm necessitates real-world person images with manual textual annotations for training models, posing privacy-sensitive and labor-intensive issues. Several pioneering efforts explore synthetic data for TBPR but still rely on real data, keeping the aforementioned issues and also resulting in diversity-deficient issue in synthetic datasets, thus impacting TBPR performance. Moreover, these works tend to explore synthetic data for TBPR through limited perspectives, leading to exploration-restricted issue. In this paper, we conduct an empirical study to explore the potential of synthetic data for TBPR, highlighting three key aspects. (1) We propose an inter-class image generation pipeline, in which an automatic prompt construction strategy is introduced to guide generative Artificial Intelligence (AI) models in generating various inter-class images without reliance on original data. (2) We develop an intra-class image augmentation pipeline, in which the generative AI models are applied to further edit the images for obtaining various intra-class images. (3) Building upon the proposed pipelines and an automatic text generation pipeline, we explore the effectiveness of synthetic data in diverse scenarios through extensive experiments. Additionally, we experimentally investigate various noise-robust learning strategies to mitigate the inherent noise in synthetic data. We will release the code, along with the synthetic large-scale dataset generated by our pipelines, which are expected to advance practical TBPR research.
• 摘要：数据在基于文本的人检索（TBPR）研究中起关键作用。主流研究范式需要对现实世界中的人物图像，其中具有手动文本注释的培训模型，从而提出了对隐私敏感和劳动密集型的问题。几项开创性的工作探索了TBPR的合成数据，但仍然依靠实际数据，保留上述问题，并在合成数据集中导致多样性问题，从而影响TBPR性能。此外，这些作品倾向于通过有限的观点探索TBPR的合成数据，从而导致勘探限制性问题。在本文中，我们进行了一项经验研究，以探索TBPR合成数据的潜力，突出了三个关键方面。 （1）我们提出了一条类间图像生成管道，其中引入了自动及时构建策略，以指导生成人工智能（AI）模型，以生成各种类间图像而不依赖原始数据。 （2）我们开发了一类内部图像增强管道，其中应用生成的AI模型进一步编辑图像，以获取各种阶层内图像。 （3）在提议的管道和自动文本生成管道上，我们通过广泛的实验探索了合成数据的有效性。此外，我们通过实验研究各种噪声学习策略，以减轻合成数据中的固有噪声。我们将发布该代码，以及我们的管道生成的合成大规模数据集，这些数据集有望推进实用的TBPR研究。

Title: Concept-Aware LoRA for Domain-Aligned Segmentation Dataset Generation

• Authors: Minho Park, Sunghyun Park, Jungsoo Lee, Hyojin Park, Kyuwoong Hwang, Fatih Porikli, Jaegul Choo, Sungha Choi
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22172
• Pdf URL: https://arxiv.org/pdf/2503.22172
• Copy Paste: [[2503.22172]] Concept-Aware LoRA for Domain-Aligned Segmentation Dataset Generation(https://arxiv.org/abs/2503.22172)
• Keywords: generation
• Abstract: This paper addresses the challenge of data scarcity in semantic segmentation by generating datasets through text-to-image (T2I) generation models, reducing image acquisition and labeling costs. Segmentation dataset generation faces two key challenges: 1) aligning generated samples with the target domain and 2) producing informative samples beyond the training data. Fine-tuning T2I models can help generate samples aligned with the target domain. However, it often overfits and memorizes training data, limiting their ability to generate diverse and well-aligned samples. To overcome these issues, we propose Concept-Aware LoRA (CA-LoRA), a novel fine-tuning approach that selectively identifies and updates only the weights associated with necessary concepts (e.g., style or viewpoint) for domain alignment while preserving the pretrained knowledge of the T2I model to produce informative samples. We demonstrate its effectiveness in generating datasets for urban-scene segmentation, outperforming baseline and state-of-the-art methods in in-domain (few-shot and fully-supervised) settings, as well as in domain generalization tasks, especially under challenging conditions such as adverse weather and varying illumination, further highlighting its superiority.
• 摘要：本文通过文本对图像（T2I）生成模型生成数据集，从而解决了语义细分中数据稀缺的挑战，从而降低了图像采集和标记成本。分割数据集生成面临两个关键挑战：1）将生成的样品与目标域对齐，以及2）在培训数据之外生成内容丰富的样本。微调T2I模型可以帮助生成与目标域对齐的样品。但是，它经常过度贴合并记住培训数据，从而限制了它们生成多样化和良好样本的能力。为了克服这些问题，我们提出了一种概念感知的洛拉（CA-lora），这是一种新颖的微调方法，选择性地识别和更新了与必要概念相关的权重（例如样式或样式或观点），以确保域的一致性，同时保留对T2i模型的知识知识来产生信息的样本。我们证明了它在为城市场景分割生成数据集方面的有效性，超过基线和最先进的方法（几乎没有弹药和完全监督）的设置，以及在域名的概括任务中，尤其是在不利的天气和诸如不利天气和变化的微妙条件之下，进一步亮相了其优势。

Title: High-Fidelity Diffusion Face Swapping with ID-Constrained Facial Conditioning

• Authors: Dailan He, Xiahong Wang, Shulun Wang, Guanglu Song, Bingqi Ma, Hao Shao, Yu Liu, Hongsheng Li
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22179
• Pdf URL: https://arxiv.org/pdf/2503.22179
• Copy Paste: [[2503.22179]] High-Fidelity Diffusion Face Swapping with ID-Constrained Facial Conditioning(https://arxiv.org/abs/2503.22179)
• Keywords: generative
• Abstract: Face swapping aims to seamlessly transfer a source facial identity onto a target while preserving target attributes such as pose and expression. Diffusion models, known for their superior generative capabilities, have recently shown promise in advancing face-swapping quality. This paper addresses two key challenges in diffusion-based face swapping: the prioritized preservation of identity over target attributes and the inherent conflict between identity and attribute conditioning. To tackle these issues, we introduce an identity-constrained attribute-tuning framework for face swapping that first ensures identity preservation and then fine-tunes for attribute alignment, achieved through a decoupled condition injection. We further enhance fidelity by incorporating identity and adversarial losses in a post-training refinement stage. Our proposed identity-constrained diffusion-based face-swapping model outperforms existing methods in both qualitative and quantitative evaluations, demonstrating superior identity similarity and attribute consistency, achieving a new state-of-the-art performance in high-fidelity face swapping.
• 摘要：面部交换的目的是无缝将源面部身份传递到目标上，同时保留诸如姿势和表达之类的目标属性。以其出色的生成能力而闻名的扩散模型最近在提高面部交换质量方面表现出了希望。本文解决了基于扩散的面部交换中的两个关键挑战：优先保存身份而不是目标属性以及身份和属性条件之间的固有冲突。为了解决这些问题，我们为面部交换引入了一个身份约束的属性调节框架，该框架首先确保了身份保存，然后通过脱钩的条件注入来实现属性对齐。我们通过将身份和对抗性损失纳入训练后的精致阶段，进一步提高了忠诚度。我们提出的身份受限的基于基于扩散的面部交换模型在定性和定量评估中都优于现有方法，表明了卓越的身份相似性和属性一致性，在高保真性面部交换中实现了新的最新性能。

Title: Knowledge Rectification for Camouflaged Object Detection: Unlocking Insights from Low-Quality Data

• Authors: Juwei Guan, Xiaolin Fang, Donghyun Kim, Haotian Gong, Tongxin Zhu, Zhen Ling, Ming Yang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22180
• Pdf URL: https://arxiv.org/pdf/2503.22180
• Copy Paste: [[2503.22180]] Knowledge Rectification for Camouflaged Object Detection: Unlocking Insights from Low-Quality Data(https://arxiv.org/abs/2503.22180)
• Keywords: super-resolution
• Abstract: Low-quality data often suffer from insufficient image details, introducing an extra implicit aspect of camouflage that complicates camouflaged object detection (COD). Existing COD methods focus primarily on high-quality data, overlooking the challenges posed by low-quality data, which leads to significant performance degradation. Therefore, we propose KRNet, the first framework explicitly designed for COD on low-quality data. KRNet presents a Leader-Follower framework where the Leader extracts dual gold-standard distributions: conditional and hybrid, from high-quality data to drive the Follower in rectifying knowledge learned from low-quality data. The framework further benefits from a cross-consistency strategy that improves the rectification of these distributions and a time-dependent conditional encoder that enriches the distribution diversity. Extensive experiments on benchmark datasets demonstrate that KRNet outperforms state-of-the-art COD methods and super-resolution-assisted COD approaches, proving its effectiveness in tackling the challenges of low-quality data in COD.
• 摘要：低质量的数据通常遭受图像细节不足，引入了伪装的额外隐式方面，使伪装对象检测（COD）复杂化。现有的COD方法主要关注高质量数据，忽视低质量数据带来的挑战，从而导致绩效降低。因此，我们建议KRNET，这是针对低质量数据的COD设计的第一个框架。 KRNET提出了一个领导者的框架，其中领导者提取双金标准分布：条件和混合动力，从高质量的数据来推动追随者从低质量数据中学到的知识。该框架进一步从跨矛盾策略中受益，从而改善了这些分布的整流以及具有时间依赖的条件编码器，从而丰富了分布多样性。基准数据集的广泛实验表明，KRNET优于最先进的COD方法和超分辨率辅助的COD方法，证明了其在应对COD中低质量数据的挑战方面的有效性。

Title: ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation

• Authors: Yunhong Min, Daehyeon Choi, Kyeongmin Yeo, Jihyun Lee, Minhyuk Sung
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.22194
• Pdf URL: https://arxiv.org/pdf/2503.22194
• Copy Paste: [[2503.22194]] ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation(https://arxiv.org/abs/2503.22194)
• Keywords: generation, generative
• Abstract: We introduce ORIGEN, the first zero-shot method for 3D orientation grounding in text-to-image generation across multiple objects and diverse categories. While previous work on spatial grounding in image generation has mainly focused on 2D positioning, it lacks control over 3D orientation. To address this, we propose a reward-guided sampling approach using a pretrained discriminative model for 3D orientation estimation and a one-step text-to-image generative flow model. While gradient-ascent-based optimization is a natural choice for reward-based guidance, it struggles to maintain image realism. Instead, we adopt a sampling-based approach using Langevin dynamics, which extends gradient ascent by simply injecting random noise--requiring just a single additional line of code. Additionally, we introduce adaptive time rescaling based on the reward function to accelerate convergence. Our experiments show that ORIGEN outperforms both training-based and test-time guidance methods across quantitative metrics and user studies.
• 摘要：我们介绍了Origen，这是第一种零射击方法，用于在多个对象和不同类别中进行文本对象生成3D方向接地。尽管图像生成中的空间接地的先前工作主要集中在2D定位上，但它缺乏对3D方向的控制。为了解决这个问题，我们建议使用预告片的判别模型进行3D方向估计和一步文本对图像生成流模型的奖励指导采样方法。尽管基于梯度的优化是基于奖励的指导的自然选择，但它努力维持图像现实主义。取而代之的是，我们使用Langevin Dynamics采用了一种基于采样的方法，该方法通过简单地注入随机噪声来扩展梯度的上升 - 仅获得一条附加的代码线。此外，我们根据奖励功能引入自适应时间重新缩放以加速收敛。我们的实验表明，Origen在定量指标和用户研究中均优于基于培训和测试时间指导方法。

Title: Extremely Simple Out-of-distribution Detection for Audio-visual Generalized Zero-shot Learning

• Authors: Yang Liu, Xun Zhang, Jiale Du, Xinbo Gao, Jungong Han
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22197
• Pdf URL: https://arxiv.org/pdf/2503.22197
• Copy Paste: [[2503.22197]] Extremely Simple Out-of-distribution Detection for Audio-visual Generalized Zero-shot Learning(https://arxiv.org/abs/2503.22197)
• Keywords: generative
• Abstract: Zero-shot Learning(ZSL) attains knowledge transfer from seen classes to unseen classes by exploring auxiliary category information, which is a promising yet difficult research topic. In this field, Audio-Visual Generalized Zero-Shot Learning~(AV-GZSL) has aroused researchers' great interest in which intricate relations within triple modalities~(audio, video, and natural language) render this task quite challenging but highly research-worthy. However, both existing embedding-based and generative-based AV-GZSL methods tend to suffer from domain shift problem a lot and we propose an extremely simple Out-of-distribution~(OOD) detection based AV-GZSL method~(EZ-AVOOD) to further mitigate bias problem by differentiating seen and unseen samples at the initial beginning. EZ-AVOOD accomplishes effective seen-unseen separation by exploiting the intrinsic discriminative information held in class-specific logits and class-agnostic feature subspace without training an extra OOD detector network. Followed by seen-unseen binary classification, we employ two expert models to classify seen samples and unseen samples separately. Compared to existing state-of-the-art methods, our model achieves superior ZSL and GZSL performances on three audio-visual datasets and becomes the new SOTA, which comprehensively demonstrates the effectiveness of the proposed EZ-AVOOD.
• 摘要：零射门学习（ZSL）通过探索辅助类别信息从可见的课程转移到看不见的课程，这是一个有希望而又困难的研究主题。在这一领域中，视听概括的零局部学习〜（AV-GZSL）引起了研究人员的极大兴趣，在这种研究中，三重模式中的复杂关系〜（音频，视频和自然语言）使这项任务变得非常具有挑战性，但非常值得研究。但是，现有的基于嵌入的基于嵌入的AV-GZSL方法往往会经常遭受域移位问题的困扰，我们提出了一种非常简单的基于检测的av-gzsl方法〜（ez-avood），以通过在初次开始时区分和未见样本来进一步减轻偏见问题，以进一步减轻偏见问题。 EZ-AVOOD通过利用在类别特定的逻辑和类别不可能的特征子空间中利用固有的歧视性信息来实现有效的看不见的分离，而无需培训额外的OOD检测器网络。然后是可见的二进制分类，我们采用两个专家模型分别对所见样本进行分类和看不见的样本。与现有的最新方法相比，我们的模型在三个音频视频数据集上实现了出色的ZSL和GZSL性能，并成为了新的SOTA，这全面证明了拟议的EZ-Avood的有效性。

Title: Intrinsic Image Decomposition for Robust Self-supervised Monocular Depth Estimation on Reflective Surfaces

• Authors: Wonhyeok Choi, Kyumin Hwang, Minwoo Choi, Kiljoon Han, Wonjoon Choi, Mingyu Shin, Sunghoon Im
• Subjects: cs.CV, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.22209
• Pdf URL: https://arxiv.org/pdf/2503.22209
• Copy Paste: [[2503.22209]] Intrinsic Image Decomposition for Robust Self-supervised Monocular Depth Estimation on Reflective Surfaces(https://arxiv.org/abs/2503.22209)
• Keywords: generation
• Abstract: Self-supervised monocular depth estimation (SSMDE) has gained attention in the field of deep learning as it estimates depth without requiring ground truth depth maps. This approach typically uses a photometric consistency loss between a synthesized image, generated from the estimated depth, and the original image, thereby reducing the need for extensive dataset acquisition. However, the conventional photometric consistency loss relies on the Lambertian assumption, which often leads to significant errors when dealing with reflective surfaces that deviate from this model. To address this limitation, we propose a novel framework that incorporates intrinsic image decomposition into SSMDE. Our method synergistically trains for both monocular depth estimation and intrinsic image decomposition. The accurate depth estimation facilitates multi-image consistency for intrinsic image decomposition by aligning different view coordinate systems, while the decomposition process identifies reflective areas and excludes corrupted gradients from the depth training process. Furthermore, our framework introduces a pseudo-depth generation and knowledge distillation technique to further enhance the performance of the student model across both reflective and non-reflective surfaces. Comprehensive evaluations on multiple datasets show that our approach significantly outperforms existing SSMDE baselines in depth prediction, especially on reflective surfaces.
• 摘要：自我监督的单眼深度估计（SSMDE）在深度学习领域引起了人们的关注，因为它可以估算深度，而无需地面真相深度图。这种方法通常使用从估计深度生成的合成图像和原始图像之间的光度一致性损失，从而减少了广泛的数据集采集的需求。但是，常规的光度一致性损失取决于兰伯特的假设，在处理偏离该模型的反射表面时，通常会导致重大误差。为了解决这一限制，我们提出了一个新型框架，将固有的图像分解纳入SSMDE。我们的方法协同训练单眼深度估计和内在图像分解。准确的深度估计通过对齐不同的视图坐标系统来促进固有图像分解的多图像一致性，而分解过程则识别反射区域，并将损坏的梯度排除在深度训练过程中。此外，我们的框架引入了伪深入的一代和知识蒸馏技术，以进一步提高反射性和非反射表面的学生模型的性能。对多个数据集的全面评估表明，我们的方法在深度预测中大大优于现有的SSMDE基准，尤其是在反射表面上。

Title: Fuzzy Cluster-Aware Contrastive Clustering for Time Series

• Authors: Congyu Wang, Mingjing Du, Xiang Jiang, Yongquan Dong
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.22211
• Pdf URL: https://arxiv.org/pdf/2503.22211
• Copy Paste: [[2503.22211]] Fuzzy Cluster-Aware Contrastive Clustering for Time Series(https://arxiv.org/abs/2503.22211)
• Keywords: generation
• Abstract: The rapid growth of unlabeled time series data, driven by the Internet of Things (IoT), poses significant challenges in uncovering underlying patterns. Traditional unsupervised clustering methods often fail to capture the complex nature of time series data. Recent deep learning-based clustering approaches, while effective, struggle with insufficient representation learning and the integration of clustering objectives. To address these issues, we propose a fuzzy cluster-aware contrastive clustering framework (FCACC) that jointly optimizes representation learning and clustering. Our approach introduces a novel three-view data augmentation strategy to enhance feature extraction by leveraging various characteristics of time series data. Additionally, we propose a cluster-aware hard negative sample generation mechanism that dynamically constructs high-quality negative samples using clustering structure information, thereby improving the model's discriminative ability. By leveraging fuzzy clustering, FCACC dynamically generates cluster structures to guide the contrastive learning process, resulting in more accurate clustering. Extensive experiments on 40 benchmark datasets show that FCACC outperforms the selected baseline methods (eight in total), providing an effective solution for unsupervised time series learning.
• 摘要：由物联网（IoT）驱动的未标记时间序列数据的快速增长在发现潜在的模式方面构成了重大挑战。传统的无监督聚类方法通常无法捕获时间序列数据的复杂性质。最近的基于深度学习的聚类方法虽然有效，但仍在代表学习不足和集群目标的整合而苦苦挣扎。为了解决这些问题，我们提出了一个模糊的群集吸引的对比聚类框架（FCACC），该框架共同优化表示表示和聚类。我们的方法引入了一种新颖的三视数据增强策略，以通过利用时间序列数据的各种特征来增强特征提取。此外，我们提出了一种群集感知的硬性样本生成机制，该机制使用聚类结构信息动态构建高质量的负样品，从而提高了模型的判别能力。通过利用模糊聚类，FCACC动态生成集群结构来指导对比度学习过程，从而导致更准确的聚类。 40个基准数据集的大量实验表明，FCACC的表现优于所选的基线方法（总共八种），为无监督时间序列学习提供了有效的解决方案。

Title: Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback

• Authors: Wei Shen, Guanlin Liu, Zheng Wu, Ruofei Zhu, Qingping Yang, Chao Xin, Yu Yue, Lin Yan
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.22230
• Pdf URL: https://arxiv.org/pdf/2503.22230
• Copy Paste: [[2503.22230]] Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback(https://arxiv.org/abs/2503.22230)
• Keywords: generative
• Abstract: Reinforcement Learning from Human Feedback (RLHF) is crucial for aligning large language models with human preferences. While recent research has focused on algorithmic improvements, the importance of prompt-data construction has been overlooked. This paper addresses this gap by exploring data-driven bottlenecks in RLHF performance scaling, particularly reward hacking and decreasing response diversity. We introduce a hybrid reward system combining reasoning task verifiers (RTV) and a generative reward model (GenRM) to mitigate reward hacking. We also propose a novel prompt-selection method, Pre-PPO, to maintain response diversity and enhance learning effectiveness. Additionally, we find that prioritizing mathematical and coding tasks early in RLHF training significantly improves performance. Experiments across two model sizes validate our methods' effectiveness and scalability. Results show that RTV is most resistant to reward hacking, followed by GenRM with ground truth, and then GenRM with SFT Best-of-N responses. Our strategies enable rapid capture of subtle task-specific distinctions, leading to substantial improvements in overall RLHF performance. This work highlights the importance of careful data construction and provides practical methods to overcome performance barriers in RLHF.
• 摘要：从人类反馈（RLHF）中学习的强化学习对于使大语模型与人类偏好保持一致至关重要。尽管最近的研究集中在算法改进上，但迅速建筑的重要性被忽略了。本文通过在RLHF性能缩放中探索数据驱动的瓶颈来解决这一差距，尤其是奖励黑客入侵和降低响应多样性。我们介绍了一个混合奖励系统，结合了推理任务验证符（RTV）和生成奖励模型（GENRM），以减轻奖励黑客攻击。我们还提出了一种新颖的及时选择方法，即PRE-PPO，以保持响应多样性并提高学习效率。此外，我们发现在RLHF培训的早期，优先考虑数学和编码任务可以显着提高性能。跨两个模型尺寸的实验验证了我们的方法的有效性和可扩展性。结果表明，RTV对奖励黑客攻击最具抵抗力，其次是具有地面真理的GenRM，然后是具有SFT最佳响应的GenRM。我们的策略能够快速捕获特定于任务的区别，从而实现了整体RLHF性能的实质性改善。这项工作突出了仔细的数据构建的重要性，并提供了克服RLHF中性能障碍的实用方法。

Title: CoGen: 3D Consistent Video Generation via Adaptive Conditioning for Autonomous Driving

• Authors: Yishen Ji, Ziyue Zhu, Zhenxin Zhu, Kaixin Xiong, Ming Lu, Zhiqi Li, Lijun Zhou, Haiyang Sun, Bing Wang, Tong Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22231
• Pdf URL: https://arxiv.org/pdf/2503.22231
• Copy Paste: [[2503.22231]] CoGen: 3D Consistent Video Generation via Adaptive Conditioning for Autonomous Driving(https://arxiv.org/abs/2503.22231)
• Keywords: generation
• Abstract: Recent progress in driving video generation has shown significant potential for enhancing self-driving systems by providing scalable and controllable training data. Although pretrained state-of-the-art generation models, guided by 2D layout conditions (e.g., HD maps and bounding boxes), can produce photorealistic driving videos, achieving controllable multi-view videos with high 3D consistency remains a major challenge. To tackle this, we introduce a novel spatial adaptive generation framework, CoGen, which leverages advances in 3D generation to improve performance in two key aspects: (i) To ensure 3D consistency, we first generate high-quality, controllable 3D conditions that capture the geometry of driving scenes. By replacing coarse 2D conditions with these fine-grained 3D representations, our approach significantly enhances the spatial consistency of the generated videos. (ii) Additionally, we introduce a consistency adapter module to strengthen the robustness of the model to multi-condition control. The results demonstrate that this method excels in preserving geometric fidelity and visual realism, offering a reliable video generation solution for autonomous driving.
• 摘要：通过提供可扩展可控制的培训数据，驾驶视频生成的最新进展显示出了增强自动驾驶系统的巨大潜力。尽管在2D布局条件（例如HD地图和边界框）的指导下进行了验证的最先进的生成模型，但可以产生逼真的驾驶视频，并实现具有高3D一致性的可控多视频视频仍然是一个主要挑战。为了解决这个问题，我们介绍了一种新型的空间自适应生成框架Cogen，该框架利用3D代的进步来提高两个关键方面的性能：（i）确保3D一致性，我们首先生成了捕获驾驶场景几何形状的高质量，可控制的3D条件。通过用这些细粒3D表示替换粗糙的2D条件，我们的方法显着提高了生成的视频的空间一致性。 （ii）此外，我们引入了一个一致性适配器模块，以增强模型对多条件控制的鲁棒性。结果表明，该方法在保留几何忠诚度和视觉现实主义方面表现出色，为自动驾驶提供了可靠的视频生成解决方案。

Title: Process Reward Modeling with Entropy-Driven Uncertainty

• Authors: Lang Cao, Renhong Chen, Yingtian Zou, Chao Peng, Wu Ning, Huacong Xu, Qian Chen, Yuxian Wang, Peishuo Su, Mofan Peng, Zijie Chen, Yitong Li
• Subjects: cs.LG, cs.AI, cs.CL
• Abstract URL: https://arxiv.org/abs/2503.22233
• Pdf URL: https://arxiv.org/pdf/2503.22233
• Copy Paste: [[2503.22233]] Process Reward Modeling with Entropy-Driven Uncertainty(https://arxiv.org/abs/2503.22233)
• Keywords: generation
• Abstract: This paper presents the Entropy-Driven Unified Process Reward Model (EDU-PRM), a novel framework that approximates state-of-the-art performance in process supervision while drastically reducing training costs. EDU-PRM introduces an entropy-guided dynamic step partitioning mechanism, using logit distribution entropy to pinpoint high-uncertainty regions during token generation dynamically. This self-assessment capability enables precise step-level feedback without manual fine-grained annotation, addressing a critical challenge in process supervision. Experiments on the Qwen2.5-72B model with only 7,500 EDU-PRM-generated training queries demonstrate accuracy closely approximating the full Qwen2.5-72B-PRM (71.1% vs. 71.6%), achieving a 98% reduction in query cost compared to prior methods. This work establishes EDU-PRM as an efficient approach for scalable process reward model training.
• 摘要：本文介绍了熵驱动的统一过程奖励模型（EDU-PRM），该模型是一个新颖的框架，近似于过程监督的最新性能，同时大大降低了培训成本。 EDU-PRM使用logit分布熵引入了熵引导的动态步骤分区机制，以动态的代币生成过程中查明高固定度区域。这种自我评估能力可以无需手动细粒注释即可精确的阶梯反馈，从而解决了过程监督的关键挑战。在QWEN2.5-72B模型上进行的实验只有7,500个EDU-PRM生成的训练查询表明准确性近似于完整的QWEN2.5-72B-PRM（71.1％vs. 71.6％），与先验方法相比，查询成本减少了98％。这项工作确立了EDU-PRM作为可扩展过程奖励模型培训的有效方法。

Title: Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion

• Authors: Songsong Yu, Yuxin Chen, Zhongang Qi, Zeke Xie, Yifan Wang, Lijun Wang, Ying Shan, Huchuan Lu
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22262
• Pdf URL: https://arxiv.org/pdf/2503.22262
• Copy Paste: [[2503.22262]] Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion(https://arxiv.org/abs/2503.22262)
• Keywords: generation
• Abstract: With the rapid proliferation of 3D devices and the shortage of 3D content, stereo conversion is attracting increasing attention. Recent works introduce pretrained Diffusion Models (DMs) into this task. However, due to the scarcity of large-scale training data and comprehensive benchmarks, the optimal methodologies for employing DMs in stereo conversion and the accurate evaluation of stereo effects remain largely unexplored. In this work, we introduce the Mono2Stereo dataset, providing high-quality training data and benchmark to support in-depth exploration of stereo conversion. With this dataset, we conduct an empirical study that yields two primary findings. 1) The differences between the left and right views are subtle, yet existing metrics consider overall pixels, failing to concentrate on regions critical to stereo effects. 2) Mainstream methods adopt either one-stage left-to-right generation or warp-and-inpaint pipeline, facing challenges of degraded stereo effect and image distortion respectively. Based on these findings, we introduce a new evaluation metric, Stereo Intersection-over-Union, which prioritizes disparity and achieves a high correlation with human judgments on stereo effect. Moreover, we propose a strong baseline model, harmonizing the stereo effect and image quality simultaneously, and notably surpassing current mainstream methods. Our code and data will be open-sourced to promote further research in stereo conversion. Our models are available at this http URL.
• 摘要：随着3D设备的快速扩散和3D含量的短缺，立体声转换引起了越来越多的关注。最近的作品将经过预定的扩散模型（DMS）引入此任务。但是，由于大型培训数据和全面的基准缺乏，因此在立体声转换中采用DMS的最佳方法以及对立体声效应的准确评估仍然很大程度上尚未得到探索。在这项工作中，我们介绍了Mono2stereo数据集，提供了高质量的培训数据和基准，以支持立体声转换的深入探索。使用该数据集，我们进行了一项实证研究，该研究产生了两个主要发现。 1）左和右视图之间的差异是微妙的，但是现有的指标考虑了整体像素，未能集中于对立体效应至关重要的区域。 2）主流方法采用从左到右的一阶段或翘曲管道，分别面临着立体声效应和图像失真的挑战。基于这些发现，我们引入了新的评估度量标准，即立体声交集，该指标优先考虑差异并与人类在立体声效应方面的判断高度相关。此外，我们提出了一个强大的基线模型，同时协调了立体声效果和图像质量，并且尤其超过了当前主流方法。我们的代码和数据将被开源，以促进立体声转换的进一步研究。我们的模型可在此HTTP URL上找到。

Title: DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generation

• Authors: Haomin Zhang, Chang Liu, Junjie Zheng, Zihao Chen, Chaofan Ding, Xinhan Di
• Subjects: cs.CV, cs.SD, eess.AS
• Abstract URL: https://arxiv.org/abs/2503.22265
• Pdf URL: https://arxiv.org/pdf/2503.22265
• Copy Paste: [[2503.22265]] DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generation(https://arxiv.org/abs/2503.22265)
• Keywords: generation
• Abstract: Currently, high-quality, synchronized audio is synthesized using various multi-modal joint learning frameworks, leveraging video and optional text inputs. In the video-to-audio benchmarks, video-to-audio quality, semantic alignment, and audio-visual synchronization are effectively achieved. However, in real-world scenarios, speech and audio often coexist in videos simultaneously, and the end-to-end generation of synchronous speech and audio given video and text conditions are not well studied. Therefore, we propose an end-to-end multi-modal generation framework that simultaneously produces speech and audio based on video and text conditions. Furthermore, the advantages of video-to-audio (V2A) models for generating speech from videos remain unclear. The proposed framework, DeepAudio, consists of a video-to-audio (V2A) module, a text-to-speech (TTS) module, and a dynamic mixture of modality fusion (MoF) module. In the evaluation, the proposed end-to-end framework achieves state-of-the-art performance on the video-audio benchmark, video-speech benchmark, and text-speech benchmark. In detail, our framework achieves comparable results in the comparison with state-of-the-art models for the video-audio and text-speech benchmarks, and surpassing state-of-the-art models in the video-speech benchmark, with WER 16.57% to 3.15% (+80.99%), SPK-SIM 78.30% to 89.38% (+14.15%), EMO-SIM 66.24% to 75.56% (+14.07%), MCD 8.59 to 7.98 (+7.10%), MCD SL 11.05 to 9.40 (+14.93%) across a variety of dubbing settings.
• 摘要：当前，使用各种多模式关节学习框架，利用视频和可选的文本输入来合成高质量的同步音频。在视频与原告基准中，有效地实现了视频与原声质量，语义对齐和视听同步。但是，在实际情况下，语音和音频通常同时在视频中共存，并且对视频和文本条件的端到端同步语音和音频的端到端都没有得到很好的研究。因此，我们提出了一个端到端的多模式生成框架，该框架同时根据视频和文本条件产生语音和音频。此外，从视频中产生语音的视频与原模型的优势尚不清楚。所提出的框架，Deepaudio，由视频对ADIO（V2A）模块，文本到语音（TTS）模块以及模态融合（MOF）模块的动态混合物组成。在评估中，拟议的端到端框架在视频ADIO基准，视频语音基准和文本语音基准上实现了最先进的性能。详细介绍，我们的框架与视频审计和文本语音基准的最先进模型进行了比较，在视频语音基准中超过了最先进的模型，其中16.57％至3.15％至3.15％（+80.99％）（+80.99％），SPK-SIM 78.30％至89.38％（+14.38％（+14.38％），在各种配音设置中，75.56％（+14.07％），MCD 8.59至7.98（+7.10％），MCD SL 11.05至9.40（+14.93％）。

Title: Meta-LoRA: Meta-Learning LoRA Components for Domain-Aware ID Personalization

• Authors: Barış Batuhan Topal, Umut Özyurt, Zafer Doğan Budak, Ramazan Gokberk Cinbis
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22352
• Pdf URL: https://arxiv.org/pdf/2503.22352
• Copy Paste: [[2503.22352]] Meta-LoRA: Meta-Learning LoRA Components for Domain-Aware ID Personalization(https://arxiv.org/abs/2503.22352)
• Keywords: generative
• Abstract: Recent advancements in text-to-image generative models, particularly latent diffusion models (LDMs), have demonstrated remarkable capabilities in synthesizing high-quality images from textual prompts. However, achieving identity personalization-ensuring that a model consistently generates subject-specific outputs from limited reference images-remains a fundamental challenge. To address this, we introduce Meta-Low-Rank Adaptation (Meta-LoRA), a novel framework that leverages meta-learning to encode domain-specific priors into LoRA-based identity personalization. Our method introduces a structured three-layer LoRA architecture that separates identity-agnostic knowledge from identity-specific adaptation. In the first stage, the LoRA Meta-Down layers are meta-trained across multiple subjects, learning a shared manifold that captures general identity-related features. In the second stage, only the LoRA-Mid and LoRA-Up layers are optimized to specialize on a given subject, significantly reducing adaptation time while improving identity fidelity. To evaluate our approach, we introduce Meta-PHD, a new benchmark dataset for identity personalization, and compare Meta-LoRA against state-of-the-art methods. Our results demonstrate that Meta-LoRA achieves superior identity retention, computational efficiency, and adaptability across diverse identity conditions. The code, model weights, and dataset will be released publicly upon acceptance.
• 摘要：文本到图像生成模型的最新进展，尤其是潜在扩散模型（LDMS），在从文本提示中综合高质量图像方面表现出了显着的功能。但是，实现个性化的个性化 - 模型始终从有限的参考图像捕获中生成特定于主题的输出是一个基本挑战。为了解决这个问题，我们介绍了一个新型框架，该框架利用元学习将特定于域的先验编码为基于洛拉的个性个性化的个性化。我们的方法引入了结构化的三层洛拉体系结构，该结构将身份敏捷知识与特定于身份的适应性分开。在第一阶段，洛拉元偏移的层是在多个主题中进行元训练的，学习了捕获一般身份相关特征的共享歧管。在第二阶段，仅优化了洛拉 - 中间和洛拉层层以专门研究给定受试者，从而大大减少了适应时间，同时提高了身份保真度。为了评估我们的方法，我们介绍了一种新的基准数据集Meta-PHD，以进行身份​​个性化，并将Meta-Lora与最新方法进行比较。我们的结果表明，Meta-Lora可以在各种身份条件下实现卓越的身份保留，计算效率和适应性。接受代码，模型权重和数据集将在接受后公开发布。

Title: EchoFlow: A Foundation Model for Cardiac Ultrasound Image and Video Generation

• Authors: Hadrien Reynaud, Alberto Gomez, Paul Leeson, Qingjie Meng, Bernhard Kainz
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22357
• Pdf URL: https://arxiv.org/pdf/2503.22357
• Copy Paste: [[2503.22357]] EchoFlow: A Foundation Model for Cardiac Ultrasound Image and Video Generation(https://arxiv.org/abs/2503.22357)
• Keywords: generation
• Abstract: Advances in deep learning have significantly enhanced medical image analysis, yet the availability of large-scale medical datasets remains constrained by patient privacy concerns. We present EchoFlow, a novel framework designed to generate high-quality, privacy-preserving synthetic echocardiogram images and videos. EchoFlow comprises four key components: an adversarial variational autoencoder for defining an efficient latent representation of cardiac ultrasound images, a latent image flow matching model for generating accurate latent echocardiogram images, a latent re-identification model to ensure privacy by filtering images anatomically, and a latent video flow matching model for animating latent images into realistic echocardiogram videos conditioned on ejection fraction. We rigorously evaluate our synthetic datasets on the clinically relevant task of ejection fraction regression and demonstrate, for the first time, that downstream models trained exclusively on EchoFlow-generated synthetic datasets achieve performance parity with models trained on real datasets. We release our models and synthetic datasets, enabling broader, privacy-compliant research in medical ultrasound imaging at this https URL.
• 摘要：深度学习的进步已经显着增强了医学图像分析，但是大规模医疗数据集的可用性仍受到患者隐私问题的限制。我们提出了Echoflow，这是一个新颖的框架，旨在生成高质量，保护隐私的合成超声心动图图像和视频。 ECHOFLOW包括四个关键组成部分：一种可对的差异自动编码器，用于定义心脏超声图像的有效潜在代表，这是一种潜在图像流量匹配模型，用于生成准确的潜在超声心动图图像，潜在的超声心动图，潜在的重新识别模型，以确保通过视频图像进行启发图像，以确保匹配的视频图像，并具有潜在的图像，并具有潜在的图像，并具有潜在的图像，并具有潜在的图像，并具有潜在的图像。射血分数。我们严格评估合成数据集关于临床相关的射血分数回归任务，并首次证明了仅在Echoflow-eneragen-echoflow生成的合成数据集方面训练的下游模型实现了在真实数据集中受过训练的模型。我们释放我们的模型和合成数据集，在此HTTPS URL的医学超声成像中更广泛，符合隐私的研究。

Title: ViSketch-GPT: Collaborative Multi-Scale Feature Extraction for Sketch Recognition and Generation

• Authors: Giulio Federico, Giuseppe Amato, Fabio Carrara, Claudio Gennaro, Marco Di Benedetto
• Subjects: cs.CV, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.22374
• Pdf URL: https://arxiv.org/pdf/2503.22374
• Copy Paste: [[2503.22374]] ViSketch-GPT: Collaborative Multi-Scale Feature Extraction for Sketch Recognition and Generation(https://arxiv.org/abs/2503.22374)
• Keywords: generation
• Abstract: Understanding the nature of human sketches is challenging because of the wide variation in how they are created. Recognizing complex structural patterns improves both the accuracy in recognizing sketches and the fidelity of the generated sketches. In this work, we introduce ViSketch-GPT, a novel algorithm designed to address these challenges through a multi-scale context extraction approach. The model captures intricate details at multiple scales and combines them using an ensemble-like mechanism, where the extracted features work collaboratively to enhance the recognition and generation of key details crucial for classification and generation tasks. The effectiveness of ViSketch-GPT is validated through extensive experiments on the QuickDraw dataset. Our model establishes a new benchmark, significantly outperforming existing methods in both classification and generation tasks, with substantial improvements in accuracy and the fidelity of generated sketches. The proposed algorithm offers a robust framework for understanding complex structures by extracting features that collaborate to recognize intricate details, enhancing the understanding of structures like sketches and making it a versatile tool for various applications in computer vision and machine learning.
• 摘要：理解人类草图的本质是具有挑战性的，因为它们的创造方式存在很大变化。识别复杂的结构模式可以提高识别草图和生成草图的保真度的准确性。在这项工作中，我们介绍了Visketch-GPT，这是一种新型算法，旨在通过多尺度上下文提取方法来应对这些挑战。该模型在多个尺度上捕获了复杂的细节，并使用类似于合奏的机制将它们结合在一起，其中提取的功能协同起作用，以增强对分类和生成任务至关重要的关键细节的识别和生成。 Visketch-GPT的有效性通过在QuickDraw数据集上进行的大量实验验证。我们的模型建立了一个新的基准，在分类和生成任务中的现有方法大大优于现有方法，其准确性和生成草图的忠诚度得到了很大的提高。提出的算法通过提取协作以识别复杂的细节，增强对草图之类的结构的理解，并使其成为计算机视觉和机器学习中各种应用程序的多功能工具，从而提供了一个强大的框架来理解复杂的结构。

Title: GAITGen: Disentangled Motion-Pathology Impaired Gait Generative Model -- Bringing Motion Generation to the Clinical Domain

• Authors: Vida Adeli, Soroush Mehraban, Majid Mirmehdi, Alan Whone, Benjamin Filtjens, Amirhossein Dadashzadeh, Alfonso Fasano, Andrea Iaboni Babak Taati
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22397
• Pdf URL: https://arxiv.org/pdf/2503.22397
• Copy Paste: [[2503.22397]] GAITGen: Disentangled Motion-Pathology Impaired Gait Generative Model -- Bringing Motion Generation to the Clinical Domain(https://arxiv.org/abs/2503.22397)
• Keywords: generation, generative
• Abstract: Gait analysis is crucial for the diagnosis and monitoring of movement disorders like Parkinson's Disease. While computer vision models have shown potential for objectively evaluating parkinsonian gait, their effectiveness is limited by scarce clinical datasets and the challenge of collecting large and well-labelled data, impacting model accuracy and risk of bias. To address these gaps, we propose GAITGen, a novel framework that generates realistic gait sequences conditioned on specified pathology severity levels. GAITGen employs a Conditional Residual Vector Quantized Variational Autoencoder to learn disentangled representations of motion dynamics and pathology-specific factors, coupled with Mask and Residual Transformers for conditioned sequence generation. GAITGen generates realistic, diverse gait sequences across severity levels, enriching datasets and enabling large-scale model training in parkinsonian gait analysis. Experiments on our new PD-GaM (real) dataset demonstrate that GAITGen outperforms adapted state-of-the-art models in both reconstruction fidelity and generation quality, accurately capturing critical pathology-specific gait features. A clinical user study confirms the realism and clinical relevance of our generated sequences. Moreover, incorporating GAITGen-generated data into downstream tasks improves parkinsonian gait severity estimation, highlighting its potential for advancing clinical gait analysis.
• 摘要：步态分析对于帕金森氏病等运动障碍的诊断和监测至关重要。尽管计算机视觉模型显示出可能客观地评估帕金森步态的潜力，但其有效性受到稀缺的临床数据集的限制以及收集大型且标记良好的数据的挑战，从而影响了模型的准确性和偏见的风险。为了解决这些差距，我们提出了Gaitgen，这是一个新型框架，该框架生成了以指定病理严重程度为条件的逼真的步态序列。 Gaitgen采用有条件的残留矢量量化变异自动编码器来学习运动动力学和病理特异性因素的分离表示，并与掩模和残留变压器结合条件序列的产生。 Gaitgen在严重程度上产生了现实的步态序列，丰富了数据集并实现了帕金森步态分析中的大规模模型培训。我们新的PD-GAM（真实）数据集的实验表明，盖特根（Gaitgen）在重建忠诚度和发电质量中的最先进模型优于适用的最新模型，可以准确捕获关键的特定于病理特异性步态特征。一项临床用户研究证实了我们生成的序列的现实性和临床相关性。此外，将Gaitgen生成的数据纳入下游任务可改善帕金森步态的严重性估计，强调其推进临床步态分析的潜力。

Title: Generative Reliability-Based Design Optimization Using In-Context Learning Capabilities of Large Language Models

• Authors: Zhonglin Jiang, Qian Tang, Zequn Wang
• Subjects: cs.LG, stat.ME
• Abstract URL: https://arxiv.org/abs/2503.22401
• Pdf URL: https://arxiv.org/pdf/2503.22401
• Copy Paste: [[2503.22401]] Generative Reliability-Based Design Optimization Using In-Context Learning Capabilities of Large Language Models(https://arxiv.org/abs/2503.22401)
• Keywords: generation, generative
• Abstract: Large Language Models (LLMs) have demonstrated remarkable in-context learning capabilities, enabling flexible utilization of limited historical information to play pivotal roles in reasoning, problem-solving, and complex pattern recognition tasks. Inspired by the successful applications of LLMs in multiple domains, this paper proposes a generative design method by leveraging the in-context learning capabilities of LLMs with the iterative search mechanisms of metaheuristic algorithms for solving reliability-based design optimization problems. In detail, reliability analysis is performed by engaging the LLMs and Kriging surrogate modeling to overcome the computational burden. By dynamically providing critical information of design points to the LLMs with prompt engineering, the method enables rapid generation of high-quality design alternatives that satisfy reliability constraints while achieving performance optimization. With the Deepseek-V3 model, three case studies are used to demonstrated the performance of the proposed approach. Experimental results indicate that the proposed LLM-RBDO method successfully identifies feasible solutions that meet reliability constraints while achieving a comparable convergence rate compared to traditional genetic algorithms.
• 摘要：大型语言模型（LLM）表现出了非凡的内在学习能力，从而可以灵活利用有限的历史信息来在推理，解决问题和复杂的模式识别任务中扮演关键角色。受LLM在多个领域的成功应用的启发，本文提出了一种生成设计方法，该方法通过利用LLM的文化学习能力以及元关节化算法的迭代搜索机制来解决基于可靠性的设计优化问题。详细说明，可靠性分析是通过与LLM和Kriging替代建模来克服计算负担来执行的。通过通过及时的工程为LLM的设计点动态提供关键信息，该方法可以快速生成高质量的设计替代方案，以满足可靠性约束，同时实现性能优化。使用DeepSeek-V3模型，使用三个案例研究证明了所提出的方法的性能。实验结果表明，所提出的LLM-RBDO方法成功地识别了可行的解决方案，这些解决方案与传统的遗传算法相比，同时达到可靠性约束，同时达到可比的收敛速率。

Title: Robustness quantification and how it allows for reliable classification, even in the presence of distribution shift and for small training sets

• Authors: Adrián Detavernier, Jasper De Bock
• Subjects: cs.LG, math.PR
• Abstract URL: https://arxiv.org/abs/2503.22418
• Pdf URL: https://arxiv.org/pdf/2503.22418
• Copy Paste: [[2503.22418]] Robustness quantification and how it allows for reliable classification, even in the presence of distribution shift and for small training sets(https://arxiv.org/abs/2503.22418)
• Keywords: generative
• Abstract: Based on existing ideas in the field of imprecise probabilities, we present a new approach for assessing the reliability of the individual predictions of a generative probabilistic classifier. We call this approach robustness quantification, compare it to uncertainty quantification, and demonstrate that it continues to work well even for classifiers that are learned from small training sets that are sampled from a shifted distribution.
• 摘要：根据不精确概率领域的现有思想，我们提出了一种评估生成概率分类器个人预测的可靠性的新方法。我们称这种方法稳健性量化，将其与不确定性量化进行比较，并证明它仍然可以很好地工作，即使是从分布中从变化的分布中采样的小型训练集中学到的分类器。

Title: Comparing Methods for Bias Mitigation in Graph Neural Networks

• Authors: Barbara Hoffmann, Ruben Mayer
• Subjects: cs.LG
• Abstract URL: https://arxiv.org/abs/2503.22569
• Pdf URL: https://arxiv.org/pdf/2503.22569
• Copy Paste: [[2503.22569]] Comparing Methods for Bias Mitigation in Graph Neural Networks(https://arxiv.org/abs/2503.22569)
• Keywords: generative
• Abstract: This paper examines the critical role of Graph Neural Networks (GNNs) in data preparation for generative artificial intelligence (GenAI) systems, with a particular focus on addressing and mitigating biases. We present a comparative analysis of three distinct methods for bias mitigation: data sparsification, feature modification, and synthetic data augmentation. Through experimental analysis using the german credit dataset, we evaluate these approaches using multiple fairness metrics, including statistical parity, equality of opportunity, and false positive rates. Our research demonstrates that while all methods improve fairness metrics compared to the original dataset, stratified sampling and synthetic data augmentation using GraphSAGE prove particularly effective in balancing demographic representation while maintaining model performance. The results provide practical insights for developing more equitable AI systems while maintaining model performance.
• 摘要：本文研究了图神经网络（GNN）在生成人工智能（Genai）系统的数据准备中的关键作用，并特别着眼于解决和缓解偏见。我们对减轻偏置的三种不同方法进行了比较分析：数据稀疏，特征修改和合成数据增强。通过使用德国信用数据集的实验分析，我们使用多种公平指标（包括统计平等，机会平等和误报利率）评估了这些方法。我们的研究表明，尽管与原始数据集相比，所有方法都改善了公平指标，但使用图形进行了分层的采样和合成数据增强，这被证明在维持模型性能的同时，在平衡人口统计学表示方面特别有效。结果为开发更公平的AI系统提供了实用的见解，同时保持模型性能。

Title: Generative Latent Neural PDE Solver using Flow Matching

• Authors: Zijie Li, Anthony Zhou, Amir Barati Farimani
• Subjects: cs.LG, cs.AI
• Abstract URL: https://arxiv.org/abs/2503.22600
• Pdf URL: https://arxiv.org/pdf/2503.22600
• Copy Paste: [[2503.22600]] Generative Latent Neural PDE Solver using Flow Matching(https://arxiv.org/abs/2503.22600)
• Keywords: generative
• Abstract: Autoregressive next-step prediction models have become the de-facto standard for building data-driven neural solvers to forecast time-dependent partial differential equations (PDEs). Denoise training that is closely related to diffusion probabilistic model has been shown to enhance the temporal stability of neural solvers, while its stochastic inference mechanism enables ensemble predictions and uncertainty quantification. In principle, such training involves sampling a series of discretized diffusion timesteps during both training and inference, inevitably increasing computational overhead. In addition, most diffusion models apply isotropic Gaussian noise on structured, uniform grids, limiting their adaptability to irregular domains. We propose a latent diffusion model for PDE simulation that embeds the PDE state in a lower-dimensional latent space, which significantly reduces computational costs. Our framework uses an autoencoder to map different types of meshes onto a unified structured latent grid, capturing complex geometries. By analyzing common diffusion paths, we propose to use a coarsely sampled noise schedule from flow matching for both training and testing. Numerical experiments show that the proposed model outperforms several deterministic baselines in both accuracy and long-term stability, highlighting the potential of diffusion-based approaches for robust data-driven PDE learning.
• 摘要：自回归的下一步预测模型已成为构建数据驱动的神经求解器以预测时间依赖时间偏微分方程（PDE）的事实上的标准。与扩散概率模型密切相关的DENOISE训练已被证明可以增强神经求解器的时间稳定性，而其随机推理机制可以实现集合预测和不确定性定量。原则上，这种训练涉及在训练和推理过程中对一系列离散的扩散时间步骤进行取样，不可避免地增加了计算开销。此外，大多数扩散模型在结构化的均匀网格上应用了各向同性高斯噪声，从而限制了它们对不规则结构域的适应性。我们为PDE模拟提出了一个潜在扩散模型，该模拟将PDE状态嵌入较低维的潜在空间，从而大大降低了计算成本。我们的框架使用自动编码器将不同类型的网格映射到统一的结构潜在网格上，从而捕获复杂的几何形状。通过分析常见的扩散路径，我们建议使用训练和测试的流量匹配中的粗噪声时间表。数值实验表明，所提出的模型在准确性和长期稳定性方面的表现都优于确定性基准，从而突出了基于扩散的方法对强大数据驱动的PDE学习的潜力。

Title: Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusion Model

• Authors: Jangho Park, Taesung Kwon, Jong Chul Ye
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22622
• Pdf URL: https://arxiv.org/pdf/2503.22622
• Copy Paste: [[2503.22622]] Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusion Model(https://arxiv.org/abs/2503.22622)
• Keywords: generation
• Abstract: Recently, multi-view or 4D video generation has emerged as a significant research topic. Nonetheless, recent approaches to 4D generation still struggle with fundamental limitations, as they primarily rely on harnessing multiple video diffusion models with additional training or compute-intensive training of a full 4D diffusion model with limited real-world 4D data and large computational costs. To address these challenges, here we propose the first training-free 4D video generation method that leverages the off-the-shelf video diffusion models to generate multi-view videos from a single input video. Our approach consists of two key steps: (1) By designating the edge frames in the spatio-temporal sampling grid as key frames, we first synthesize them using a video diffusion model, leveraging a depth-based warping technique for guidance. This approach ensures structural consistency across the generated frames, preserving spatial and temporal coherence. (2) We then interpolate the remaining frames using a video diffusion model, constructing a fully populated and temporally coherent sampling grid while preserving spatial and temporal consistency. Through this approach, we extend a single video into a multi-view video along novel camera trajectories while maintaining spatio-temporal consistency. Our method is training-free and fully utilizes an off-the-shelf video diffusion model, offering a practical and effective solution for multi-view video generation.
• 摘要：最近，多视图或4D视频生成已成为一个重要的研究主题。尽管如此，最近的4D代方法仍在基本限制中挣扎，因为它们主要依靠利用多个视频扩散模型以及对完整4D扩散模型的额外培训或计算密集型培训，具有有限的现实世界中4D数据和大量计算成本。为了应对这些挑战，我们在这里提出了第一种无培训的4D视频生成方法，该方法利用现成的视频扩散模型来生成单个输入视频的多视频视频。我们的方法包括两个关键步骤：（1）通过将时空采样网格指定为关键帧，我们首先使用视频扩散模型合成它们，利用基于深度的翘曲技术来进行指导。这种方法确保了生成的框架之间的结构一致性，从而保留了空间和时间连贯性。 （2）然后，我们使用视频扩散模型插入其余框架，构建一个填充且具有时间连贯的采样网格，同时保持空间和时间一致性。通过这种方法，我们将单个视频扩展到一个多视频视频沿着新颖的相机轨迹，同时保持时空的一致性。我们的方法是无训练的，并且充分利用了现成的视频扩散模型，为多视频视频生成提供了实用有效的解决方案。

Title: DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundness

• Authors: Ruining Li, Chuanxia Zheng, Christian Rupprecht, Andrea Vedaldi
• Subjects: cs.CV, cs.AI, cs.LG
• Abstract URL: https://arxiv.org/abs/2503.22677
• Pdf URL: https://arxiv.org/pdf/2503.22677
• Copy Paste: [[2503.22677]] DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundness(https://arxiv.org/abs/2503.22677)
• Keywords: generative
• Abstract: Most 3D object generators focus on aesthetic quality, often neglecting physical constraints necessary in applications. One such constraint is that the 3D object should be self-supporting, i.e., remains balanced under gravity. Prior approaches to generating stable 3D objects used differentiable physics simulators to optimize geometry at test-time, which is slow, unstable, and prone to local optima. Inspired by the literature on aligning generative models to external feedback, we propose Direct Simulation Optimization (DSO), a framework to use the feedback from a (non-differentiable) simulator to increase the likelihood that the 3D generator outputs stable 3D objects directly. We construct a dataset of 3D objects labeled with a stability score obtained from the physics simulator. We can then fine-tune the 3D generator using the stability score as the alignment metric, via direct preference optimization (DPO) or direct reward optimization (DRO), a novel objective, which we introduce, to align diffusion models without requiring pairwise preferences. Our experiments show that the fine-tuned feed-forward generator, using either DPO or DRO objective, is much faster and more likely to produce stable objects than test-time optimization. Notably, the DSO framework works even without any ground-truth 3D objects for training, allowing the 3D generator to self-improve by automatically collecting simulation feedback on its own outputs.
• 摘要：大多数3D对象发电机都专注于美学质量，通常会忽略应用程序中必要的物理约束。这样的限制是，3D对象应该是自支撑的，即在重力下保持平衡。生成稳定3D对象的先前方法使用可区分的物理模拟器在测试时优化几何形状，后者缓慢，不稳定且容易出现局部Optima。受到有关将生成模型与外部反馈对齐的文献的启发，我们提出了直接模拟优化（DSO），该框架是使用（非差异）模拟器的反馈来增加3D发电机直接输出稳定3D对象的可能性。我们构建一个标记为从物理模拟器获得的稳定性得分标记的3D对象的数据集。然后，我们可以通过直接偏好优化（DPO）或直接奖励优化（DRO）使用稳定性得分来微调3D发电机，这是一个新颖的目标，我们介绍了，以使得无需成对偏好而对齐扩散模型。我们的实验表明，使用DPO或DRO目标的微型馈电馈发电机比测试时间优化更快，更可能产生稳定的对象。值得注意的是，DSO框架即使没有任何地面3D对象进行训练，也可以通过自动收集自己的输出来自动收集模拟反馈来自我破坏。

Title: Q-Insight: Understanding Image Quality via Visual Reinforcement Learning

• Authors: Weiqi Li, Xuanyu Zhang, Shijie Zhao, Yabin Zhang, Junlin Li, Li Zhang, Jian Zhang
• Subjects: cs.CV
• Abstract URL: https://arxiv.org/abs/2503.22679
• Pdf URL: https://arxiv.org/pdf/2503.22679
• Copy Paste: [[2503.22679]] Q-Insight: Understanding Image Quality via Visual Reinforcement Learning(https://arxiv.org/abs/2503.22679)
• Keywords: generation, quality assessment
• Abstract: Image quality assessment (IQA) focuses on the perceptual visual quality of images, playing a crucial role in downstream tasks such as image reconstruction, compression, and generation. The rapid advancement of multi-modal large language models (MLLMs) has significantly broadened the scope of IQA, moving toward comprehensive image quality understanding that incorporates content analysis, degradation perception, and comparison reasoning beyond mere numerical scoring. Previous MLLM-based methods typically either generate numerical scores lacking interpretability or heavily rely on supervised fine-tuning (SFT) using large-scale annotated datasets to provide descriptive assessments, limiting their flexibility and applicability. In this paper, we propose Q-Insight, a reinforcement learning-based model built upon group relative policy optimization (GRPO), which demonstrates strong visual reasoning capability for image quality understanding while requiring only a limited amount of rating scores and degradation labels. By jointly optimizing score regression and degradation perception tasks with carefully designed reward functions, our approach effectively exploits their mutual benefits for enhanced performance. Extensive experiments demonstrate that Q-Insight substantially outperforms existing state-of-the-art methods in both score regression and degradation perception tasks, while exhibiting impressive zero-shot generalization to comparison reasoning tasks. Code will be available at this https URL.
• 摘要：图像质量评估（IQA）着重于图像的感知视觉质量，在下游任务中起着至关重要的作用，例如图像重建，压缩和发电。多模式大语言模型（MLLM）的快速发展已显着扩大了IQA的范围，朝着全面的图像质量理解迈进，结合了内容分析，退化感知以及超出数值得分以外的比较推理。以前的基于MLLM的方法通常要么生成缺乏可解释性的数值分数，要么使用大规模注释的数据集在严重依靠监督的微调（SFT）来提供描述性评估，从而限制了其灵活性和适用性。在本文中，我们提出了Q-Instright，这是基于组相对政策优化（GRPO）的基于强化学习的模型，该模型证明了图像质量理解的强大视觉推理能力，同时仅需要有限的评级分数和退化标签。通过精心设计的奖励功能，通过共同优化分数回归和退化感知任务，我们的方法有效利用了它们的相互利益以增强性能。广泛的实验表明，Q-Inmight在得分回归和降解感知任务中的现有最新方法基本上优于现有的最新方法，同时表现出令人印象深刻的零局部概括与比较推理任务。代码将在此HTTPS URL上可用。